Function arg list mismatch between provider code and main code when using AWS SES

[zachliu]

Apache Airflow version

2.2.4 (latest released)

What happened

After setting up alert emails using AWS SES as instructed here, I received the following error message:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 2103, in email_alert
    send_email(self.task.email, subject, html_content)
  File "/usr/local/lib/python3.8/site-packages/airflow/utils/email.py", line 57, in send_email
    return backend(
TypeError: send_email() got multiple values for argument 'from_email'

What you expected to happen

the send_email method in

airflow/airflow/providers/amazon/aws/utils/emailer.py

Lines 25 to 37 in d94fa37

	def send_email(
	from_email: str,
	to: Union[List[str], str],
	subject: str,
	html_content: str,
	files: Optional[List] = None,
	cc: Optional[Union[List[str], str]] = None,
	bcc: Optional[Union[List[str], str]] = None,
	mime_subtype: str = 'mixed',
	mime_charset: str = 'utf-8',
	conn_id: str = 'aws_default',
	**kwargs,
	) -> None:

doesn't match the invocation of the backend method in

airflow/airflow/utils/email.py

Lines 57 to 70 in ee9049c

	return backend(
	to_comma_separated,
	subject,
	html_content,
	files=files,
	dryrun=dryrun,
	cc=cc,
	bcc=bcc,
	mime_subtype=mime_subtype,
	mime_charset=mime_charset,
	conn_id=backend_conn_id,
	from_email=from_email,
	**kwargs,
	)

in the former (provider-amazon), the from_email is the first positional argument. However in the latter (main airflow), from_email is misplaced as a keyword argument.

How to reproduce

1. Use Airflow 2.2.4 with providers-amazon/3.0.0
2. Observe the log from a failed DAG

Operating System

Amazon Linux 2

Versions of Apache Airflow Providers

apache-airflow-providers-amazon 3.0.0

Deployment

Other Docker-based deployment

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[uranusjr]

Looks like this is introduced in 1543dc2. I think we should fix this (by moving the from_email argument to the correct place), and also use this chance to 1. convert send_email caller to always use keyword arguments.

[potiuk]

I beleive that the incompatibilities have been fixed in 3.0.0 (the issue was raised here #21671 and I thought I fixed it in #21681).

What is exactly your configuration @zachliu? Do you have maybe a connection setup with "from_email" extra? That could explain the duplicate from_email entry.

[potiuk]

Ah no... I see the problem :(

[potiuk]

Ah yeah. My fix has not yet been released when 2.2.4 has been released simply. It will be fixed in the release I am about to release today. Maybe (I will see how many changes we have) since this is quite an important problem, we might even update constraints and re-generate images for 2.2.4 to include the new amazon provider with the fix.

[potiuk]

Closing as duplicate of #21681

[zachliu]

@potiuk great, thanks! eagerly waiting for providers-amazon-3.0.1 😁

[potiuk]

Yeah. I merge a few last changes and I will be ready to prepare the relase. Loking forward to helping testing it @zachliu 😄

[repl-mike-roest]

@potiuk any update on a release to get this fix out?

[potiuk]

It's being voted and tested now #22264.

Since you are interested int the release @repl-mike-roest could you please test the release candidate and report back? Since the software you get is free, and you apparenly are very interested in getting it out, testing it and confirming tha the fix is working is the best way to pay back to the communiy. Can I count on your help with that?

[potiuk]

See #22264 - it contains information about the release candidate and link to instructions on how to test it

[repl-mike-roest]

Awesome yep @potiuk let me see what I can do to help out on things