DEP 0018 Dictionary-based EMAIL_PROVIDERS (ticket-35514)

[medmunds]

📄 Read the formatted draft DEP 0018, latest revision.

See ticket-35514#comment:24

Django ticket-35514 will implement a dictionary-based EMAIL_PROVIDERS setting, similar to CACHES, DATABASES, STORAGES and TASKS. The ticket was approved following discussion on the django-developers list and at DjangoCon Europe 2024 sprints.

The purpose of this DEP is to facilitate discussion and decisions on the proposed API and related deprecations.

[nessita]

Submitting my initial set of comments to see if there are any red flags.

[nessita]

Amazing work, super well laid out, clear and precise. Thank you @medmunds! I'm overall onboard with the proposal as a whole pending some minor fixes. I tried to provide guidance on every 🤔 callout.

Let me know how to progress this!

[nessita]

@medmunds I'm happy to be added as Shepherd, and I think we could use number 18 for this and putting it up for review once the small bits are taken care of. What do you think?

[medmunds]

@nessita thanks as always for the thorough review and thoughtful suggestions 🙌

I'll update to incorporate all the feedback. (I suspect this may unearth some new questions about fail_silently, but we're converging on a solution.)

I hadn't really thought about handling this as a real DEP; I was just borrowing the format and the repo for discussion. (And I was going to close it unmerged once the details were worked out, since the ticket is already approved.) But now that you suggest it, a DEP makes sense. Thanks for shepherding!

Once there's a new draft, I think it's important to get Jacob R's input. (Recognizing the enormous amount of time he's already put into this, I wanted to wait to @ him until there were fewer open questions.)

I'll also try to reach out directly to some third-party packages that would be affected (django-mailer, django-celery-email, django-ses, etc.), to make sure they're aware of the proposal and have a chance to weigh in.

And it would be helpful to get feedback from people who are actively using the SMTP EmailBackend in production, particularly on some of the deprecations. Maybe a forum post to re-introduce the feature? (My own production experience is with http-API-based backends like django-anymail and django-ses, plus a private fork of django-mailer for queuing and error recovery. So I may have some blind spots around things like get_connection() with username/password overrides.)

[nessita]

@medmunds On a related tanget, I just came across this PR that allows cache backends to "know" their alias. I haven't reviewed that but figured you may be interested in taking a quick look, in the interest of consistency? django/django#20469

[medmunds]

@medmunds On a related tanget, I just came across this PR that allows cache backends to "know" their alias. I haven't reviewed that but figured you may be interested in taking a quick look, in the interest of consistency? django/django#20469

Ouroboros! I've been citing the new-features discussion that led to that PR as justification here for providing alias on EmailBackends. I'll take a look at the PR.

[medmunds]

I've updated the DEP to incorporate @nessita's feedback. There are still a few open questions (including a few new ones), but I think it's ready for @jrief to take a look if he's available.

Based on discussion in this pr, the DEP now proposes moving fail_silently out of individual EmailBackends, instead handling it in EmailMessage.send(). This both resolves current inconsistencies in fail_silently behavior and avoids an API-shape mismatch that would complicate mail.providers[alias]. See New handling of fail_silently (and EmailBackend fail_silently deprecated) in the current draft.

The fail_silently change can (and likely should) be split out from implementing EMAIL_PROVIDERS. I'll start a forum discussion on it shortly.

[LilyFirefly]

I've given this a full read through and I didn't spot any red flags.

[ahmedasar00]

Hi Mike,

I'm Ahmed I worked on the fail_silently documentation ticket (#36907)
that was recently closed as wontfix due to EMAIL_PROVIDERS.

I've been following the DEP discussion and I'm interested in contributing
to this feature. I understand it's a large undertaking with a tight timeline.

Is there any part of the work where you could use help? Even smaller tasks
like writing tests, updating documentation, or reviewing specific sections
would be valuable learning for me.
Thanks Mike

[medmunds]

Is there any part of the work where you could use help? Even smaller tasks
like writing tests, updating documentation, or reviewing specific sections
would be valuable learning for me.

@ahmedasar00 I appreciate the offer. Because the elements of this feature are so tightly intertwined, I don't think it would easily lend itself to dividing up the coding or even docs.

One area Django always needs assistance is with code reviews, and that can be a great way to learn more about both the codebase and the contribution process. There are some tips in Django's contributor docs under Reviewing patches, and I highly recommend Sarah Boyce's Django needs you! (to do code review) keynote video from DjangoCon Europe last year. (While you're waiting for this PR to show up, there are around 100 other tickets with PRs waiting for review.)

[medmunds]

To improve DX around email configuration, when EMAIL_PROVIDERS is in use I'd like to have the SMTP EmailBackend:

• Require an explicit "host" in its OPTIONS. (There would be no default SMTP host, similar to how there is no default "file_path" directory for the filebased EmailBackend. Explicit is better than implicit.)
• Default the "port" dynamically, based on whether "use_ssl" or "use_tls" are enabled. (To 25, 465, or 587, so the user doesn't need to think about which port goes with which connection type. OPTIONS would only need "port" for non-standard usage.)

Both of these should help avoid unexpected behavior. They would only apply in updated configurations where the user has added EMAIL_PROVIDERS to their settings; the old defaults would still apply in compatibility configurations.

(In adding EMAIL_PROVIDERS, we have the opportunity to revisit some of the email config defaults without having to go through a separate deprecation period—but only in its initial release.)

(Related: I'm getting to the point in the coding where I discover a bunch of details that aren't quite right in the DEP. I'll push some corrections later this week once I've opened the draft PR.)

[medmunds]

We have a new draft PR: django/django#21052

I just pushed a handful of DEP changes motivated by experience with the actual code. More details in the commit messages, but briefly:

• Unknown OPTIONS are now reported by BaseEmailBackend (and all backends should accept and forward **kwargs)
• mail.providers[using] returns the default provider if using is None, which simplifies implementation in all the APIs that support a using arg
• SMTP backend requires host and dynamically defaults port (per my previous comment)
• Locmem backend adds sent_using property to message copies in the testing outbox (had been in the "future" bin, but was really helpful for Django's own test cases)
• Some undocumented EmailMessage behaviors are removed without deprecation

[tim-schilling]

I'm on board with this as is! Thank you @medmunds and @nessita for your work on this. I had a few nitpick comments, some formatting questions, and then provided my opinion around the 🤔 points.

I don't have any substantive requests for change, though if we engage other 3rd party package maintainers, they may have thoughts on the upgrade process.

[tim-schilling]

Suggested change

	Something like (exact wording TBD), "Django 7.0 will not have a default email
	Something like, "Django 7.0 will not have a default email

nit: This makes it seem like there's a TODO in the DEP. I don't think we need the exact wording to match and "Something like" implies the wording will change.

[medmunds]

We're actively discussing this warning in the implementation PR. There's a concern it's unnecessarily noisy to warn Django 7.0 won't have a default email provider for projects that aren't even using email.

I'll update the DEP to say that projects that are using email with default settings need a deprecation warning, explaining they'll need to add EMAIL_PROVIDERS to avoid errors in Django 7.0. (But I'll leave both the timing and wording details to the implementation PR.)

[tim-schilling]

Thank you @medmunds and @nessita for pushing this forwards!