pysmurf-monitor main process robustness

[jlashner]

This PR adds some protection to the pysmurf-monitor main process, to protect against sqlalchemy operational errors crashing the main loop.

Description

• On OperationalError, this will now just set the session.degraded = True and sleep 5 sec until next iteration.
  • It will not clear cached files that need to be added to the database
• This PR also adds basic type hinting to enable type checking

Motivation and Context

Resolves #712.

How Has This Been Tested?

This has not been tested yet. We really need to figure out how to test on a system that's not at site.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.

[BrianJKoopman]

This looks good to me. If we had a system to test on, do you have a method for testing this?

[jlashner]

I do not really have a good way to test it beyond the existing CI tests...