Refactor `S3KeysUnchangedSensor` system test

[eladkal]

S3KeysUnchangedSensor doesn't work well with DebugExecutor which causes issues for system tests.

Spawn of Originally posted by @eladkal in #26606 (comment)

[suselunarwolf]

HI can i work on this ..is issue is still open

[o-nikolas]

Hey @suselunarwolf
Sorry we hadn't gotten back sooner! I've assigned this task to you :)

[eladkal]

@vandonr-amz Now that #28528 is merged. Is this issue still valid?

airflow/tests/system/providers/amazon/aws/example_s3.py

Lines 205 to 209 in 9ab1a6a

	# This task skips the `sensor_keys_unchanged` task because the S3KeysUnchangedSensor
	# runs in poke mode only, which is not supported by the DebugExecutor, causing system tests to fail.
	branching = BranchPythonOperator(
	task_id="branch_to_delete_objects", python_callable=lambda: "delete_objects"
	)

[vandonr-amz]

Yes, #28528 was about not rescheduling immediately and respecting the set interval, but sensors still run in reschedule mode, so this one still doesn't work.

[o-nikolas]

I looked more deeply into this issue. There actually isn’t a small bug fix here. It’s a fundamental conflict between how the sensor in question works (which is stateful, it needs to keep track of the current keys to compare against future keys in each poke, and in reschedule mode there is no way to do this (cleanly)) and how the debug executor works (which is single threaded and must use reschedule mode for sensors). I think the best course of action is to invest in teaching the Airflow system test mechanism to use other executors than the DebugExecutor. This will solve this problem and allow us to get test coverage for other new executors (ECS executor for an example). But that will be a much bigger task.

CC: @shubham22