v2.4.0

Summary

This update represents the efforts to make t-route compatible with the NextGen framework. Most of the updates have focused on making sure routing features from NWMv3.0 work in the NextGen framework. The major features are listed here:

• Integration of t-route and the coastal total water level models (SCHISM and DFLOW) via the NextGen Framework
• Data Assimilation methodology (e.g. nudging) integrated into t-route at all locations where USGS gauge data is currently available
• Reservoir routing methodologies integrated with t-route at all locations available in NWM v.3.0
• Diffusive Wave routing capability integrated into t-route for approximately 10,000 river miles where bathymetric data is currently available
• Capability for the user to specify the most appropriate routing technique from Muskingum-Cunge and Diffusive Wave options where appropriate integrated into t-route

Initial BMI modules were created and tested, but t-route has not been connected to the model engine via BMI yet. The existing BMI modules/functions in t-route will need major updates as both t-route and model engine have progressed beyond the initial creation of these modules.

What's Changed

• added synthetic_wb_segments parameter under supernetwork_parameters i… by @shorvath-noaa in #575
• Map back water elevation from refactored to unrefactored hydrofabric by interpolating water depth not elevation by @kumdonoaa in #576
• troute network abstractions for NHD and Hy Features based networks by @hellkite500 in #577
• Da abstract classes by @shorvath-noaa in #579
• NHD network class by @shorvath-noaa in #584
• refactor main.py for making hydbrid routing and reservoir DA fully compatible with NHDNetwork class by @kumdonoaa in #586
• Ngen hydrofabric v1 updates by @hellkite500 in #585
• unit test for NHDNetwork class w/wo reservoirs by @kumdonoaa in #588
• Preprocess Hyfeature channel connectivity, forcing, t0, hybrid routing network. by @kumdonoaa in #594
• Reformat qlat inputs by @shorvath-noaa in #595
• t-route on larger hyfeature domain with hybrid routing by @kumdonoaa in #602
• Fix troute-nwm package setup.cfg by @hellkite500 in #603
• Abstract network update by @shorvath-noaa in #600
• Restructure DA classes by @shorvath-noaa in #605
• Build system updates for Intel compilers by @mattw-nws in #606
• BMI t-route wrapper by @shorvath-noaa in #607
• default creation of empty waterbody dataframes by @shorvath-noaa in #610
• BMI for Standalone Reservoirs by @shorvath-noaa in #611
• Updated get_values() functionality by @shorvath-noaa in #612
• Fix typos found by running ispell on comments and strings by @PhilMiller in #615
• update use of numpy str dtype by @hellkite500 in #623
• Fix bugs in v3_doc.yaml by @aaraney in #613
• Update running t-route through BMI functions with latest hydrofabric by @shorvath-noaa in #622
• actually fix numpy str type deprecation by @hellkite500 in #634
• troute.config: Implementation of t-route configuration file schema using pydantic by @aaraney in #628
• Time log based on new update by @AminTorabi-NOAA in #635
• Da forcing part1 by @kumdonoaa in #638
• Add support for NetCDF/Time-partitioned CSV QLAT inputs by @program-- in #636
• Integrate DA Forcing Module by @shorvath-noaa in #642
• nudged value added in return by @AminTorabi-NOAA in #644
• Integrate troute.config by @shorvath-noaa in #645
• Fix preprocess param issue by @shorvath-noaa in #647
• fix read_geopkg() to skip reading lakes and network tables if not needed by @shorvath-noaa in #648
• update conditional checking for lite_restart by @shorvath-noaa in #649
• Fix bmi issues by @shorvath-noaa in #650
• Build package updates by @hellkite500 in #643
• adding missing editable/no-editable functionality to network-config by @TrupeshKumarPatel in #657
• add outline of configuration file that works with pydantic module by @shorvath-noaa in #659
• Started directory for canonical Lower Colorado test for v4/ngen by @JurgenZach-NOAA in #653
• renamed channel forcing files by @AminTorabi-NOAA in #662
• generalize bandaid() function to work on subdomains of CONUS by @shorvath-noaa in #663
• yaml file for V4 HYFeature and NHD by @AminTorabi-NOAA in #664
• BMI restart and lastobs files by @shorvath-noaa in #654
• Stream output function & V4 yaml for NHD by @AminTorabi-NOAA in #661
• update usgs timeslice files for Lower Colorado river TX by @kumdonoaa in #672
• drop waterbodies with missing parameters by @shorvath-noaa in #676
• Bmi compliant arrays by @JurgenZach-NOAA in #666
• Minor fixes nov2 2023 by @shorvath-noaa in #678
• -V3 test fix by @JoshCu in #668
• Updated geopackage and forcing files to NGEN v2.0 in Lower Colorado t… by @JurgenZach-NOAA in #679
• Repair output functions by @shorvath-noaa in #681
• Added subset forcings, added missing directories and set flags in yam… by @JurgenZach-NOAA in #682
• fix - lock cython version, add missing pip deps by @JoshCu in #667
• Check before performing CRS reprojection by @shorvath-noaa in #686
• updating the column names as timestamp by @AminTorabi-NOAA in #687
• Adding lite-restart and lastObs to BMI conversions by @JurgenZach-NOAA in #685
• use only python version of RFC DA by @kumdonoaa in #688
• feature - add github actions ubuntu latest build by @JoshCu in #670
• add _rfc_timeseries_df slot by @shorvath-noaa in #691
• Update logic to call assemble_rfc_dataframes() function by @shorvath-noaa in #692
• feature - adds dockerfile, helper script, build action by @JoshCu in #669
• add rfc_lake_gage_crosswalk file to network directory by @shorvath-noaa in #694
• Update GitHub action to run V4 by @shorvath-noaa in #695
• Read json/geojson files by @shorvath-noaa in #699
• Bug fix in QGIS based routine: tnx nexus is now in the geopackage and… by @JurgenZach-NOAA in #700
• fix checking for lite_restart and config test for lite_restart_directory by @shorvath-noaa in #703
• Implemented fully BMI compliant transport code into troute by @JurgenZach-NOAA in #702
• Update on readme.md file (Documentation) by @AminTorabi-NOAA in #704
• writing nc file with different frequency (Update for #697) by @AminTorabi-NOAA in #708
• add LastObs reading and writing for command-line run of main_V04 with HYfeature hydrofabric by @kumdonoaa in #701
• Correct subroutine calling in diffusive.f90 and update makefile by @kumdonoaa in #709
• Updated run_with_BMI with BMI compliant arrays by @JurgenZach-NOAA in #711
• A new diffusive_utils file without features designed only for refactored hydrofabric by @kumdonoaa in #710
• update to read the parquet file by @AminTorabi-NOAA in #712
• Nex to bin speedup by @JoshCu in #714
• simple fix for flowpaths with unavailable 3Dxsec data by @kumdonoaa in #715
• parallel computing for storing output by @AminTorabi-NOAA in #716
• function that build diffusive domain by @AminTorabi-NOAA in #717
• updates to run NHD by @AminTorabi-NOAA in #718
• Always drop waterbody col from network df by @aaraney in #725
• awk call is now gnu and bsd compliant. fixes #720 by @aaraney in #722
• Update on NHD to convert byte string to normal string by @AminTorabi-NOAA in #728
• Update coastal_df by @shorvath-noaa in #723
• Make empty nudge array returned from diffusive the proper size by @shorvath-noaa in #729
• Diffusive tailwater latitude/longitude information by @shorvath-noaa in #731
• Coastal dataframe assembly from BMI type arrays with example by @JurgenZach-NOAA in #730
• Handling Warnings by @AminTorabi-NOAA in #733
• simple fix for empty lastobs_df when streamnudging unknowingly turned on while no usgs data is available by @kumdonoaa in #732
• Correction to reading schism data from file by @shorvath-noaa in #734
• Added accommodations for empty dataframes in diffusive by @JurgenZach-NOAA in #735
• add water depth output variable to diffusive fortran kernel by @kumdonoaa in #736
• Changes for coastal, mainly bed depth coordinate direction by @JurgenZach-NOAA in #739
• fix: support python 3.8 by @aaraney in #741
• Check sanity of tests by @JurgenZach-NOAA in #744
• Fix NHD issues related to gage byte strings by @shorvath-noaa in #742
• Speed up output by @shorvath-noaa in #747
• Expand diffusive domain to include tributaries by @shorvath-noaa in #740
• Quick fix to remove leading white space from datetime by @shorvath-noaa in #748
• fix setting 'num_files' to 1 if it is 0 by @shorvath-noaa in #749
• Fix config issues relating to 'columns' default values and retrieving rfc da parameters by @shorvath-noaa in #751
• Repair creation of diffusive domain by @shorvath-noaa in #754
• Upper limit on celerity value in diffusive Fortran kernel by @kumdonoaa in #756
• Missing topo fix by @shorvath-noaa in #755
• Diffusive fortran kernel correctly executes diffusive wave on stream segments defined by Python preprocessing by @kumdonoaa in #758
• Fix all_links object when running diffusive on just the mainstem by @shorvath-noaa in #760
• Make smoothing conveyance and dK/dA curves of xsec lookup table of diffusive wave more robust by @kumdonoaa in #759
• update to save other formats by @AminTorabi-NOAA in #762
• Accommodate new subset SCHISM data provided by Coastal by @JurgenZach-NOAA in #763
• Updates to handle problematic waterbody data by @shorvath-noaa in #764
• Default _duplicate_ids_df to empty data frame by @shorvath-noaa in #765
• update on reading DFlow by @AminTorabi-NOAA in #767
• Update for usace issue by @AminTorabi-NOAA in #768
• canadian_timeslices added by @AminTorabi-NOAA in #771
• Checked for existence of gage directories by @JurgenZach-NOAA in #770
• Importing HyFeat-2-Coastal grid crosswalk from yaml by @JurgenZach-NOAA in #766
• Great lakes routing, network connectivity by @shorvath-noaa in #772
• Add tests for troute.config.FilePath and troute.config.DirectoryPath by @aaraney in #778
• pip>23.1 compliance by @aaraney in #780
• Add build dep wheel to requirements.txt by @aaraney in #782
• Improve Cython dependency version specificity by @aaraney in #784
• style: improve InvalidIndexError error message by @aaraney in #785
• great lake by @AminTorabi-NOAA in #776
• fix for diffusive to run with synthetic and topo on NHD by @kumdonoaa in #779
• Remove check for existing parquet files by @shorvath-noaa in #788
• DataAssimilation logger by @AminTorabi-NOAA in #786
• Simplify config module by @shorvath-noaa in #790
• feat(docs): add CLI tool for generating troute config documentation from source by @aaraney in #792
• Redone old PR 750 - empty lake DF by @JurgenZach-NOAA in #781
• Fix #793: chanobs_filepath does not need to exist at startup by @aaraney in #794
• remove raising exception of forcing parquet file already exists by @shorvath-noaa in #800
• Update readme.md by @JurgenZach-NOAA in #801
• Don't validate ReservoirRfcParameters paths if reservoir forecasts are disabled by @aaraney in #802
• update csv to parquet to use multiprocessing by @JoshCu in #795
• Create a standalone hydraulic lookup table for channel cross sections by @kumdonoaa in #797
• Merge PR 775 and 789, and make them conditional on 786 by @JurgenZach-NOAA in #796
• save in one file by @AminTorabi-NOAA in #791
• Delete unnecessary directories by @AminTorabi-NOAA in #803
• chore(build): fix editable builds by @aaraney in #804
• Fixed broken diffusive routing in Lower Colorado example by @JurgenZach-NOAA in #805
• Deleted file with obsolete routines by @JurgenZach-NOAA in #807
• Added functionality to output to parquet by @karnesh in #761
• mask output by @AminTorabi-NOAA in #806
• Included pip & venv based instructions for WSL2 (#812) by @JurgenZach-NOAA in #813
• Great Lakes Data Assimilation by @shorvath-noaa in #808
• speeding up Parquet_output function by vectorizing by @AminTorabi-NOAA in #816
• updating time on csv output by @AminTorabi-NOAA in #818
• update default values for parquet output by @shorvath-noaa in #821
• fix: parquet prefix_ids defaults to wb by @aaraney in #825
• Optimizing read_geopkg function by @AminTorabi-NOAA in #819
• troute.network: handle fiona dependency for geopandas>=1.0.0 by @program-- in #827
• update for csv_output by @AminTorabi-NOAA in #828
• issue 826 by @AminTorabi-NOAA in #832
• fix to issue 829_ by @AminTorabi-NOAA in #831
• if the folder doesn't exist by @AminTorabi-NOAA in #835
• Update readme.md: removed obsolete no-e option by @JurgenZach-NOAA in #843
• Speeding up writing output by @AminTorabi-NOAA in #841
• Update readme.md by @JurgenZach-NOAA in #844
• Add config notes by @shorvath-noaa in #810

New Contributors

• @mattw-nws made their first contribution in #606
• @PhilMiller made their first contribution in #615
• @aaraney made their first contribution in #613
• @AminTorabi-NOAA made their first contribution in #635
• @program-- made their first contribution in #636
• @TrupeshKumarPatel made their first contribution in #657
• @JoshCu made their first contribution in #668
• @karnesh made their first contribution in #761

Full Changelog: v2.3.0...v2.4.0