[FLINK-38974][runtime] Handle jobs in running applications during HA recovery

[eemario]

What is the purpose of the change

This PR is part of the application high-availability (HA) improvements, focusing on job recovery behavior during JobManager failover for running applications.

Brief change log

• For running jobs that are recovered after a JM failover, execution is deferred until an explicit application recovery signal is received. If no such signal arrives before the application reaches a terminal state (e.g., finished, failed, or cancelled), the job is marked as failed, basic metadata is preserved for visibility, and proper cleanup is performed.
• For already-terminated jobs, the cleanup of job results (i.e., marking them as "clean") is delayed until after the application itself terminates. If the application later recovers and restarts, any "dirty" job results from the previous run are cleaned up during the recovery process.

Verifying this change

This change added tests and can be verified as follows:

• Added tests that validate the job submission behavior of the PackagedProgramApplication
• Added tests that validate the HA recovery behavior of the Dispatcher
• Manually verified the change by running a standalone cluster with zookeeper, confirming the job recovery during JM failover

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): (no)
• The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
• The serializers: (no)
• The runtime per-record code paths (performance sensitive): (no)
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
• The S3 file system connector: (no)

Documentation

• Does this pull request introduce a new feature? (no)
• If yes, how is the feature documented? (not applicable)

[flinkbot]

CI report:

• 63d337e Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build