AIP-72: Handling failed TI state for AirflowTaskTerminated

[amoghrajesh]

related: #44414

This is an addition to #44954 and is dependent on it.
Only last commit is relevant cb493ee

We do not have to handle the case of external task state updates specifically in task_runner because public UI APIs will directly do that job - core APIs. So there is no scenario when one of these exceptions are thrown due to this:

airflow/task_sdk/src/airflow/sdk/execution_time/supervisor.py

Lines 608 to 615 in 83da311

	except ServerResponseError as e:
	if e.response.status_code in {HTTPStatus.NOT_FOUND, HTTPStatus.CONFLICT}:
	log.error(
	"Server indicated the task shouldn't be running anymore",
	detail=e.detail,
	status_code=e.response.status_code,
	)
	self.kill(signal.SIGTERM, force=True)

We check with the database for the state with the heartbeat and if ever there is a different state, other than running:

airflow/airflow/api_fastapi/execution_api/routes/task_instances.py

Lines 210 to 218 in 83da311

	if previous_state != State.RUNNING:
	raise HTTPException(
	status_code=status.HTTP_409_CONFLICT,
	detail={
	"reason": "not_running",
	"message": "TI is no longer in the running state and task should terminate",
	"current_state": previous_state,
	},
	)

, we fail with a 409.

But, if this state is ever hit, we should just mark the TI as failed. There are UTs https://github.com/apache/airflow/compare/main...astronomer:airflow:AIP72-taskterminated-or-afexception?expand=1#diff-413c3c59636a3c7b41b8bb822827d18a959778d0b6331532e0db175c829dbfd2R342-R397 which handle this portion to check if it fails or not.

---

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.