parallel computing for storing output

[AminTorabi-NOAA]

Update on write_flowveldepth_netcdf function for parallel computing. In this setup, each iteration of the loop prepares a job for parallel processing, and the Parallel block at the end executes all these jobs. The helper function will receive each subset of the data and be responsible for processing it and writing it to the appropriate file format.

Additions

• helper function added and some part of the write_flowveldepth_netcdf function moved there for more readability

Removals

Changes

Testing

Screenshots

Notes

Todos

Checklist

• PR has an informative and human-readable title
• Changes are limited to a single goal (no scope creep)
• Code can be automatically merged (no conflicts)
• Code follows project standards (link if applicable)
• Passes all existing automated tests
• Any change in functionality is tested
• New functions are documented (with a description, list of inputs, and expected output)
• Placeholder code is flagged / future todos are captured in comments
• Visually tested in supported browsers and devices (see checklist below 👇)
• Project documentation has been updated (including the "Unreleased" section of the CHANGELOG)
• Reviewers requested with the Reviewers tool ➡️

Testing checklist

Target Environment support

• Windows
• Linux
• Browser

Accessibility

• Keyboard friendly
• Screen reader friendly

Other

• Is useable without CSS
• Is useable without JS
• Flexible from small to large screens
• No linting errors or warnings
• JavaScript tests are passing

[shorvath-noaa]

It looks like this use_parallel has not been added to output.py when the function gets called? If you want to do it this way, you also need to determine when/how use_parallel is determined based on user input. I think you might be able to just drop this though and use only cpu_pool in the if/else statements. What do you think?

[AminTorabi-NOAA]

Yes sure. So I can just say if cpu_pul > 1 do parallel otherwise don't. The only caveat is user have to set cpu_pool = 1 in output.py not here (the value 6 here doesn't matter)

[shorvath-noaa]

cpu_pool is already an input into nwm_output_generator in output.py, so it just needs to be added as in input into nhd_io.write_flowveldepth_netcdf()