Cascade DELETEs in the database by gtfierro · Pull Request #363 · NatLabRockies/BuildingMOTIF

gtfierro · 2025-01-15T16:24:43Z

I'm pretty sure the issue is that if I create library A with templates, and then I create library B which depends on A through B's templates, and the nI delete library A, I need to decide what happens to A's templates and B's templates' dependencies on A's templates. Right now (outside of this PR) I think we leave dangling references, which causes exceptions in SQLalchemy when you interact with that table after messing with the dependency structure.

…lates/etc when deleting libraries

TShapinsky · 2025-02-17T21:03:30Z

@gtfierro I just rebased which mostly required changes around of exceptions, cause your adding exceptions PR went through to develop. Let me know if anything looks messed up.

TShapinsky · 2025-02-17T23:35:38Z

Documenting what appears to be the desired functionality so that it can be tested.

On Model Delete

Delete associated shape collection

On Library Delete

Delete shape collection
Delete all templates owned by library
Delete all dependency relationships associated with deleted templates.

Standing issues:

What happens when you delete a library which has templates depended on by another libraries templates.

gtfierro · 2025-02-18T06:50:17Z

This looks great, @TShapinsky ! Thanks for doing the merge. I just wrote a couple unit tests to verify the behavior under different conditions.

I decided to see what would happen if we had a default CASCADE between dependencies.
Currently, the test_cascade_delete_dependency_multi_library test fails because it specifies that if I delete the library my template is dependent on, then that should also delete the whole library (this is the behavior I am proposing). What do you think of this behavior?

TShapinsky · 2025-02-18T16:43:36Z

This looks great, @TShapinsky ! Thanks for doing the merge. I just wrote a couple unit tests to verify the behavior under different conditions.

I decided to see what would happen if we had a default CASCADE between dependencies. Currently, the test_cascade_delete_dependency_multi_library test fails because it specifies that if I delete the library my template is dependent on, then that should also delete the whole library (this is the behavior I am proposing). What do you think of this behavior?

So if a library is deleted it should cascade delete all libraries which have dependent links to that library.

I think that makes sense. We should make sure that the deletes don't accidentally cascade the other direction. I haven't looked at your new tests yet so you may have already addressed that. Unless we want to go through the process of rebuilding dependencies for all libraries anytime one is imported I think this is the right call.

One other thought. If this isn't already the case, we should make sure that you can hotswap a library without destroying the entire dependency tree. For Example: if you overwrite a library it should rebuild the dependencies but not cascade delete everything. This could probably be a different issue.

gtfierro · 2025-02-25T05:50:36Z

@TShapinsky I've implemented this behavior -- it had to be handled using an event instead of within the ORM entirely because they don't like to do cascading deletes on many-to-many relationships. Maybe there are different types of behavior we want to enable:

leave dependent templates there, but now their dependencies don't point anywhere
delete dependent templates, but not the library
delete the dependent templates and the library and all the other templates in the library and all of their dependents, etc (this is what is currently implemented, though I need to test if chains of dependencies work as epxected)

I do like your idea about being able to swap in a new library of the same name.

Looks like I broke more tests when I run the whole suite...I'll return to this when I have some more time

buildingmotif/database/tables.py

TShapinsky · 2025-02-25T17:26:58Z

Based on the tests, it looks like either the library deletion is too aggressive, or we were relying on deletions not propagating in our testing.

The integration tests are failing with the libraries not being found.

TShapinsky · 2025-02-25T21:08:23Z

Ok my current theory is that in the failing tests brick is loaded twice. I think the second time it is loaded it first deletes the library which causes all dependent libraries to be deleted. Causing the failure we are seeing.

Edit: But we could change the library load to overwrite=False to fix for now

TShapinsky · 2025-03-03T19:25:16Z

Handling Overwrites

Overwrites pose a particular issue since the intent of the user is not to nuke their entire environment, but to swap out a single library.

Needs

Maintain dependency associations between templates
Do not require special operations from the user on any library other than the one which is being overwritten.

Potential issues

What if the new version of the template does not have all the templates which are being depended on the to-be-overwritten library?

Update method

Instead of deleting the library, update the existing object to match the new version. In this scenario templates could also potentially be updated.

Pros

We can keep the existing deletion cascade.

Cons

The downside of this method is it can get complicated if there is not a 1-1 relationship between the old and new versions of the library.

Lazy template linking

Instead of populating template dependencies at load time we could do so lazily. When a template's dependencies are queried it could look for an existing deps association, if not it would try to make one. In the case where a template is deleted, that would cascade only to the association leaving other libraries and templates untouched. In this way deps associations only really act as a cached connection.

Pros

Conceptually simple. Makes deletion and updates more simple.

Cons

Would require a rework of templates. Could make loading of templates slower at runtime.

Reworked Template primary keys

Instead of linking templates with deps associations. We can just cache the library id of the dependency and then just use the name. Right now templates are unique by library id and template name so this should work.

Pros

Could simplify usage

Cons

Might require a bunch of rewriting.

Statically compiled templates

Instead of "compiling" templates at run time we could bring all of the dependencies into the template when it is created. For example if template A requires template B and C, when template A is created templates B and C are found and copied into A so that A no longer depends on the existence of B and C.

Pros

No runtime dependencies. Faster template loading.

Cons

Templates would not respond to updated if a depended template is updated. Could make import process slower.

TShapinsky · 2025-03-06T21:05:04Z

@gtfierro If you get a second check that this is what we were talking about. There's a good amount of refactoring still to go, but I think the basic idea is there.

buildingmotif/dataclasses/template.py

TShapinsky · 2025-03-20T20:13:49Z

@gtfierro The main thing that is failing right now are dependencies populated by the shape collection when generating templates. The issue is that right now they are not getting the right library assigned.

buildingmotif/dataclasses/template.py

…le-deletes

TShapinsky · 2025-03-24T19:40:08Z

I'm currently implementing the error_on_missing_dependency option for template functions which iterate through the list of dependencies.

I'm choosing to not include inline_dependencies as I think that should always fail if it is missing a dependency.
transitive_parameters will also always error on a failed resolution as it is a property of the class as cannot have arguments.

…ted from a ttl file

TShapinsky · 2025-03-25T21:33:55Z

@gtfierro Everything is passing now

TShapinsky · 2025-03-26T15:16:31Z

One caveat. This definitely breaks migrations. I'm not super concerned with that, at this point in time I don't think there are any persistent databases that need to be migrated.

gtfierro · 2025-04-01T16:50:24Z

One caveat. This definitely breaks migrations. I'm not super concerned with that, at this point in time I don't think there are any persistent databases that need to be migrated.

Yes, i'm also ok with breaking those migrations. Perhaps we can do this as a new point release so that it is clear it isn't backwards compatible? If someone does have a persistent version then we can develop a migration script that meets their needs.

gtfierro · 2025-04-03T15:32:16Z

@TShapinsky this is technically "my" PR in GitHub's eyes so I don't think I can review it. I've requested a review from you but I've looked through and I don't have any comments for it. I approve! I'm ok with you self-reviewing and merging

…le-deletes

TShapinsky · 2025-06-05T15:10:26Z

@gtfierro The failing library tests are the same as in develop. I'm going to merge this PR

* Fix CLI (#290) * Fix docker (#288) * Squish (#289) * Exclude python 3.12 from support (#293) * exclude python 3.12 from support * change pygit2 version * move cd.yaml to poetry * prevent werkzeug from breaching 3.0.0 * update checkout and setup-python. Add 3.11 to test matrix * Refactor Table Connection (#291) * unify getting objects by id and consolidate to delete language * fix tests * Documenting shape-to-template process and adding some QoL improvements (#278) * documenting shape-to-template process and adding some QoL improvements * fix test * skip 223P tests until gabe can fix them after next standard prerelease is done * remember to add shapes from the graph even when loading from directory * add exception to catch lack of sh:path * add test cases for recent code changes * dedicated test suite for library shape test * dedicated test suite for library shape test * Add Application Explorer (#292) * Add Application Explorer * Address comment * Fix test * improve performance by not requiring OPTIONAL clause in SPARQL queries --------- Co-authored-by: Gabe Fierro <gtfierro@mines.edu> * Generate SPARQL Queries from SHACL shapes (#273) * add method to generate sparql query from shacl shape * add some tests * fix typo in query generation * add sh:node, sh:or * fixup tests * handle or inside qualifiedvalue shape, complex property paths * add test file * update docs with image and guide on query generation * augment docstring with more features * add test case * add section on supported SHACL features * add algorithm documentation * support other target definitions * add more tests * Fixing template inlining (#298) * address ambiguous template naming in inline_dependencies * fill non-graphs, propagate the optional arg requirement * update tests * adjusting tests for inlining * Allow Library to find template definitions from dependent libraries (#307) * allow Library to find template definitions from dependent libraries * fixing tests to catch warnings, test dependency tracking * use non-deprecated warnings; make sure warnings are emitted when necessary * clarify when dependencies are infered for SHACL shapes -> templates * update dependencies * adding documentation on templates * placeholder for further template docs * clarify load order * update jsonschema, skip mypy checking on imports * fix API response and test to handle Brick * cleaning some code up * Add ability to use TopQuadrant SHACL implementation as engine (#308) * add shacl_validate/infer functions and use these as the entrypoint. Augment tests to check both shacl engines * fix interactions with shacl inference * tightening up the implementation and use of the shacl_* methods * support specifying shacl engine in the API * update tests; test both pyshacl and topquadrant * add brick-tq-shacl dep * add TODOs * Formatting * no more 3.8! * ignoring some imported packages without type annotations * more type annotations * add types, ignore type errors for imports * update mypy, fix some issues and ignore some others * fix union type annotation * update docker containers * 3.8.1 python for higher * add back python 3.8 * change 3.8 version * add test for finding reasons with a given severity * update brick-tq-shacl, fix type signature * remove debug serializations * bump shacl version * fixing skolemization for validation * move shacl engine config inside buildingmotif object * install rtd deps * jupyter-book needs to be installed pre-build * cleanup and update to default config comments * change from ubuntu:latest to python:3.9 for bacnet testing (#317) * Fix tutorials (#310) * updating validation tutorial * fixing some tutorials * updates * more components * fix imports and formatting and type annotations * fix formatting * fix merge * fix bacnet docker * fixing versions in docker bacnet setup * alphabetize prefixes * fixing model_correction file * Add new transitive_parameters method to accelerate library loading (#314) * Add new transitive_parameters method to accelerate library loading The prior implementation of check_template_dependency_relationship used inline_dependencies to check the parameters of template dependencies. inline_dependencies can be slow-ish because it copies the bodies of templates (these are graphs, so they inherit the performance characteristics of the underlying store). We can compute the parameters through the dependency tree automatically without copying the graphs, which is much faster. * use cached_property * remove cached_property * fix docker * fix bacnet docker container * bump workflow action versions * BMS naming convention parsing (#286) * adding label parsing initial implementation with tests * fix bug in many parsing * test first_true * add ingress and example * adding/testing maybe parser * add errors * propagate errors * update notebook * fix test import * capture errors in the token stream * fix notebook kernel * some changes * Make Parsers classes * Add Application Explorer (#292) * Add Application Explorer * Address comment * Fix test * improve performance by not requiring OPTIONAL clause in SPARQL queries --------- Co-authored-by: Gabe Fierro <gtfierro@mines.edu> * Generate SPARQL Queries from SHACL shapes (#273) * add method to generate sparql query from shacl shape * add some tests * fix typo in query generation * add sh:node, sh:or * fixup tests * handle or inside qualifiedvalue shape, complex property paths * add test file * update docs with image and guide on query generation * augment docstring with more features * add test case * add section on supported SHACL features * add algorithm documentation * support other target definitions * add more tests * Fixing template inlining (#298) * address ambiguous template naming in inline_dependencies * fill non-graphs, propagate the optional arg requirement * update tests * adjusting tests for inlining * Allow Library to find template definitions from dependent libraries (#307) * allow Library to find template definitions from dependent libraries * fixing tests to catch warnings, test dependency tracking * use non-deprecated warnings; make sure warnings are emitted when necessary * clarify when dependencies are infered for SHACL shapes -> templates * update dependencies * adding documentation on templates * placeholder for further template docs * clarify load order * update jsonschema, skip mypy checking on imports * fix API response and test to handle Brick * cleaning some code up * remove unused import, fix type signature * fix poetry lock * moving tokens/parser to different files * adding combinators module * add some negative cases * add missing combinators file * fix notebook --------- Co-authored-by: Hannah Eslinger <heslinge@nrel.gov> * New model builder (#301) * address ambiguous template naming in inline_dependencies * fill non-graphs, propagate the optional arg requirement * update tests * adjusting tests for inlining * update 223p templates * update 223 * add model builder impl * add QUDT * update templates * allow adding triples directly to the builder graph * allow Library to find template definitions from dependent libraries * fixing tests to catch warnings, test dependency tracking * use non-deprecated warnings; make sure warnings are emitted when necessary * clarify when dependencies are infered for SHACL shapes -> templates * update dependencies * adding documentation on templates * placeholder for further template docs * clarify load order * update jsonschema, skip mypy checking on imports * fix API response and test to handle Brick * updating 223p and templates * filtering validationcontext by severity * add shacl_validate/infer functions and use these as the entrypoint. Augment tests to check both shacl engines * fix interactions with shacl inference * tightening up the implementation and use of the shacl_* methods * support specifying shacl engine in the API * update tests; test both pyshacl and topquadrant * add brick-tq-shacl dep * add TODOs * Formatting * no more 3.8! * ignoring some imported packages without type annotations * more type annotations * add types, ignore type errors for imports * update mypy, fix some issues and ignore some others * fix union type annotation * update docker containers * 3.8.1 python for higher * add back python 3.8 * change 3.8 version * add test for finding reasons with a given severity * update brick-tq-shacl, fix type signature * remove debug serializations * bump shacl version * fixing skolemization for validation * update 223p, fix merge error * fix notebook * fix the merge * remove qudt from commit * remove more bad merge * fix more bad merge * remove more bad merge * add s223 namespace, parameter prop on template builder * add model builder draft * add call syntax to simplify evaluating templates * do not waste cycles re-binding namespaces on copied graphs * update model builder notebook * add java for topquadrant support * use topquadrant in 223 notebook * updating dockerfile; this started failing, possibly because of updated ubuntu * properly initialize graph superclass * Add graph hashing method to utils (#296) * Add graph hashing method to utils * change to approximate hash and use existing model for test * make test use a model graph to show positive functionality with DB stored graphs * use new rdflib triple canonicalizer to calculate graph hashes * update docstring * rename from approximate_graph_hash to graph_hash * Create generic serializer deserializer for parsers (#322) * Strip param inlining (#325) * add _strip_param function and tests to support inlining * fix test * Changes to ingresses to support external work (#312) * Allow TemplateHandler to handle edge cases in template eval Adds two flags: - require_optional - fill_unused These allow adjustment of how the template ingress handler deal with records that don't necessarily have all the parameters for the chosen template (fill_unused), and how the handler deal with optional parameters in the chosen template (require_optional) * more configuration on template ingress * put limit on xlsx * cleaning up parameter,s adding docstrings * Update template.py * Update xlsx.py * use pathlike * Fix autogeneration of templates (#329) * factor out the guarantee method and add unit tests * add function to utils * Update 223P to public advisory review version (#309) * updating 223p again * update 223p templates * update 223 * add QUDT * update templates * allow Library to find template definitions from dependent libraries * fixing tests to catch warnings, test dependency tracking * use non-deprecated warnings; make sure warnings are emitted when necessary * clarify when dependencies are infered for SHACL shapes -> templates * update dependencies * adding documentation on templates * placeholder for further template docs * clarify load order * update jsonschema, skip mypy checking on imports * fix API response and test to handle Brick * updating 223p and templates * filtering validationcontext by severity * add shacl_validate/infer functions and use these as the entrypoint. Augment tests to check both shacl engines * fix interactions with shacl inference * tightening up the implementation and use of the shacl_* methods * support specifying shacl engine in the API * update tests; test both pyshacl and topquadrant * add brick-tq-shacl dep * add TODOs * Formatting * no more 3.8! * ignoring some imported packages without type annotations * more type annotations * add types, ignore type errors for imports * update mypy, fix some issues and ignore some others * fix union type annotation * update docker containers * 3.8.1 python for higher * add back python 3.8 * change 3.8 version * add test for finding reasons with a given severity * update brick-tq-shacl, fix type signature * remove debug serializations * bump shacl version * fixing skolemization for validation * update 223p, fix merge error * fix notebook * fix more templates; test 223p again * move shacl engine config inside buildingmotif object * parameterize testing of templates for 223p * change some brick templates to make tests less noisy * fixing up templates * add brick integration tests * upadte QUDT * 223p fixup * add shacl engine and qudt to tests * use public review version of 223p * add qudt 2.1.37 release * better exception message * fixing system, sensor templates * fixing up 223 templates * fix plugging connection points in 223p * add the shape graph to the data graph for validation * clean up a little; skip some templates for pyshacl + 223p * fixing use of tmpdir -> tmp_path * try persistence test without path * use path again; try print statements * Add new transitive_parameters method to accelerate library loading The prior implementation of check_template_dependency_relationship used inline_dependencies to check the parameters of template dependencies. inline_dependencies can be slow-ish because it copies the bodies of templates (these are graphs, so they inherit the performance characteristics of the underlying store). We can compute the parameters through the dependency tree automatically without copying the graphs, which is much faster. * use cached_property * tryin-memory? * skip the test -- is this the only one that iss broken? * remove this file * add back path, remove other conftest * add back path, remove other conftest * remove test from in-progress code * remove cached_property * add back conf tests with optimized loading * fix docker * more test hacking * remove duplicate parameterization * remove bad merge * fix bacnet docker container * bump workflow action versions * use more recent Brick, try to write to local db file for persistence test * fix brick references, fix persistence test * support configuring the shacl engine when starting the buildingmotif API * make sure SHACL engine is passed around * handle dependencies for Brick validation in integration tests * bump deps * remove debug * add imports for brick * update brick 1.3 -> 1.4 * reduce tests temporarily * fixing flags to customize how much work gets done when a library is loaded * build templates off of imports closure * more recent 223p build * update again * newer 223p * temporarily pause bacnet integration * add custom option; this needs to be done at the "top level" of the pytest config * add bacnet back * fix ci configuration * fix dependencies on template * fixing some libraries * add _strip_param function and tests to support inlining * fix last class * fix test * add _strip_param function and tests to support inlining * fix test * revert db persistence test * change how we indicate buildingMOTIF's singleton metaclass * update chiller pointlist library * add new test subdirectory to fucus on library/template tests * remove debugs * monkeypatching to handle the singleton * more adjustments on the monkeypatching * more monkeypatching * add comments for explaining the library conftest * add library tests for github, add back notebook fixture * add IDs to notebook tests * factor out the guarantee method and add unit tests * add function to utils * update internal Brick * update notebooks to match buildingmotif API * fixing a couple notebooks * fix kernel * skip some 223p templates that cannot stand on their own * more templates to skip for 223p * increase notebook run time * remove unneeded flag * small changes to template compilation * fix chiller templates * add libraries to dockerfile so that pytest can enumerate the tests * add docstring for shacl_engine * cleaner bypass of the singleton for library tests --------- Co-authored-by: Matt Steen <MatthewSteen@users.noreply.github.com> * Create parser UI (#319) * Add parsers endpoint * Update node version * Create Parser UIs * Fix linting * Fix linting? * Add in Blockly * WIP * Fix * use inspect to help generate argument serialization * Clean up * Fix spilt --------- Co-authored-by: TShapinsky <tobiasshapinsky@gmail.com> * rename 5.16 * cleanup 5.16 prefixes * add 5.16 missing fault conditions 1 and 9-15 * cleanup 5.16 fault conditions 2-8 * typos * add 5.17 fault conditions * add 5.18 fault conditions * remove unused components prefix * add 5.22 fault conditions * update Operating State points * changing the model of outside air fraction * remove internal variables (constants) * Updating setuptool to use the newest version, bumping other deps too (#340) * Updating setuptool to use the newest version, bumping other deps too * update pyshacl, other deps * Update pyproject.toml * update poetry.lock * Update NREL 223P Templates (#343) * add new pipe and belimo energy valve templates, update property definitions for water and air mediums, remove QUDT files not needed for model validation * fix dependency * skip pipe template due to connections * add %OAmin * remove %OA * update test_model.py * fix typo in FCU prefix * Skip ipynb checkpoint directory (#330) * skip .ipynb_checkpoints when scanning for files * do the same filter in utils * add ipynb checkpoint test * force-add the ipynb checkpoitn file * Service clean up (#335) Co-authored-by: Gabe Fierro <gtfierro@mines.edu> * Add test and fix for #326 (#327) * add test and fix for #237 * add negative test * fix tests for windows (#318) * fix tests for windows * flush buffer in tests * Add explanation of point label parsing (#344) * add explanation of point label parsing * add example from notebook * add note on error handling * add links to a few classes * Update docs/explanations/point-label-parsing.md Co-authored-by: Matt Steen <MatthewSteen@users.noreply.github.com> * Update docs/explanations/point-label-parsing.md Co-authored-by: Matt Steen <MatthewSteen@users.noreply.github.com> * Update docs/explanations/point-label-parsing.md Co-authored-by: Matt Steen <MatthewSteen@users.noreply.github.com> * Update docs/explanations/point-label-parsing.md Co-authored-by: Matt Steen <MatthewSteen@users.noreply.github.com> * Update docs/explanations/point-label-parsing.md Co-authored-by: Matt Steen <MatthewSteen@users.noreply.github.com> --------- Co-authored-by: Matt Steen <MatthewSteen@users.noreply.github.com> * Fixing runtime issues in Jupyter notebook (#348) * move the target query out of the hot loop; we probably do not need to compute the cbd because we are already passing in the ontology graph * reduce the number of shapes we check in the notebook * fix composition of shacl shapes * break out integration tests to only run once * remove commented line * break out libraries into its own test * remove libraries from integration * try to run library tests on approval/merge * use new model * try more triggers * make only library tests run when event type is not push * remove style dependency for library tests --------- Co-authored-by: TShapinsky <tobiasshapinsky@gmail.com> * Update Brick to 1.4.1 (#346) * update Brick to 1.4.1 * Update Brick 1.3 -> 1.4 references * move the target query out of the hot loop; we probably do not need to compute the cbd because we are already passing in the ontology graph * reduce the number of shapes we check in the notebook * fix composition of shacl shapes * break out integration tests to only run once * remove commented line * break out libraries into its own test * remove libraries from integration * updating unit tests for brick 1.4 * adjust test to use brick 1.4 * using Brick 1.4 for more tests * fix brick uri * fix 4.6 shape for library tests * Add parser vis (#334) * update pyproject.toml for release * fix CD action to publish to pypi once per release (#356) * remove needs: deploy-docs from CD (#357) * Improve validation report output (#320) * improve report output * working on orshapes * add reason to orshape * add orshape diff to report * hashable * fixing result_uri? * remove print * use actual validation report * fix imports * formatting * fix tests to match new shacl interpretation output * fix tests to match new shacl interpretation output * add test case for property shapes * use pyshacl parsing of paths * formatting and imports * remove Literal import * clean up commented code * add test case for Or * format * qname for paths in validation reasons * handle prefixes whe nconverting to sparql paths * format code * fixing up shape_to_query and adding tests * dedup count code * move count formatting function to graphdiff and fix test * Add custom error classes (#360) * add custom library/template not found exceptions * modelnotfound * add errors! * fix API error detection * fix exception * use the correct error * reduce redundancy in error messages, alphabetize classes * reduce redundancy in error messages * reformatting * fix use of exception * Update buildingmotif/api/views/model.py --------- Co-authored-by: Matt Steen <MatthewSteen@users.noreply.github.com> * Add new model creation methods (#365) * add custom library/template not found exceptions * modelnotfound * add errors! * add endpoint to get template body * change pyshacl engine when validating * fix API error detection * fix exception * use the correct error * format * add new model creation methods * test: Add tests for Model.from_file and Model.from_graph methods * fix: Import OWL namespace to resolve undefined name error in tests * make sure description can be pulled from the file, add tests for new methods * fix tests, and do not emit triple for empty description * fix imports * reformat * normalize name to string * fix the test * fix data type on test * adding more tests for file extension, adding comments to clarify behavior * fix error * Add infer_templates method on shape collection (#361) * add custom library/template not found exceptions * modelnotfound * add errors! * add infer_templates method * add more docs * ad inline method for qualified value shape * fix API error detection * fix exception * use the correct error * Update buildingmotif/utils.py Co-authored-by: Matt Steen <MatthewSteen@users.noreply.github.com> * fix import of Library dataclass --------- Co-authored-by: Matt Steen <MatthewSteen@users.noreply.github.com> * Add automatic development releases to pypi (#368) * initial attempt to push development builds * add this branch to list allowed to cd * try python 3.11 * fix my mistake * add dependency install * run build push after tests successfully complete. Try querying test pypi to confirm that incrementing works. * cleanup and documentation * skip existing for test pypi * retarget to develop now that it appears to be working properly * move development deployment into cd.yaml * Web API changes to support demo features (#362) * add custom library/template not found exceptions * modelnotfound * add errors! * add endpoint to get template body * change pyshacl engine when validating * fix API error detection * fix exception * use the correct error * format * Update buildingmotif/api/views/template.py Co-authored-by: Matt Steen <MatthewSteen@users.noreply.github.com> * allow overriding of shacl_engine * reformat * use shacl engine override in view * fix lint * use None isntead of "default" for shacl_engine * fix formatting * fix errant import --------- Co-authored-by: Matt Steen <MatthewSteen@users.noreply.github.com> * Add CompiledModel class which can generate DataFrames and Tables (#359) * add CompiledModel class * format code * fix variable names, python deps * format * add compiled model in its own class, fix some deps * avoid subclassing Model * python 3.10 thru 3.13 * no python 3.13 yet * udpate poetry from merge * bump pygit2 * bumping versions, fixing poetry lock * bump flake * bump flake8 * poetry lock * changes for shacl path * remove snakeviz * fixing test against shapes * saving time on 223P validation * fixing notebooks * Improving tests * updating tests and fixing some subtle bugs * use renamed method * updating QUDT * update Brick * update imports for brick library tests * Cascade DELETEs in the database (#363) * add errors! * made these changes when building the demo; this removes dangling templates/etc when deleting libraries * Fix styling issues * Add tests for cascading delete behavior on models and libraries * formatting * adding comments to tests, add another test case * formatting * delete dependant templates and libraries * patch test by using overwrite=False to prevent library deletion cascade * Implement lazy resolution * begin refactor with new dependency design * add error_on_missing_dependency * Fix template library resolution for cases where library is being created from a ttl file * fix dict to Dict * default resolution is to current library --------- Co-authored-by: TShapinsky <tobiasshapinsky@gmail.com> * Optimizations to Improve testing speed (#373) * Use graph subtraction instead of canonicalization to improve performance * Make notebook tets use topquadrant * run unit and integration tests in parallel * organize tests in lists instead of sets for reproducability across threads * Make library tests multi threaded * make library tests run sparingly again * 0.4.0 docs (#378) * update Model Creation * fix Model Validation * fix Model Correction by updating manifest Brick version * fix Model Correction KeyError * update Brick-subset.ttl version to temporarily fix errors * update README * update config * add Outside_Damper to Brick-subset for Model Creation * update dev doc * delete extra line --------- Co-authored-by: Hannah Eslinger <heslinge@nrel.gov> Co-authored-by: Tobias Shapinsky <tobiasshapinsky@gmail.com> Co-authored-by: Gabe Fierro <gtfierro@mines.edu> Co-authored-by: daniel <78173152+dllliu@users.noreply.github.com>

TShapinsky self-assigned this Feb 17, 2025

gtfierro added 2 commits February 17, 2025 13:59

add errors!

f2e7d3a

made these changes when building the demo; this removes dangling temp…

a6c50cf

…lates/etc when deleting libraries

TShapinsky force-pushed the gtf-handle-deletes branch from f72bc74 to a6c50cf Compare February 17, 2025 21:02

Fix styling issues

7d02d76

gtfierro added 3 commits February 17, 2025 23:37

Add tests for cascading delete behavior on models and libraries

84aa275

formatting

e1415a7

adding comments to tests, add another test case

f8882a4

formatting

2137943

gtfierro added 2 commits February 24, 2025 22:46

delete dependant templates and libraries

c341dcd

Merge remote-tracking branch 'origin/develop' into gtf-handle-deletes

89a1e58

gtfierro marked this pull request as ready for review February 25, 2025 05:47

TShapinsky reviewed Feb 25, 2025

View reviewed changes

buildingmotif/database/tables.py Outdated Show resolved Hide resolved

patch test by using overwrite=False to prevent library deletion cascade

6d73865

Implement lazy resolution

cfb08c2

TShapinsky force-pushed the gtf-handle-deletes branch from 88fa868 to cfb08c2 Compare March 7, 2025 16:17

TShapinsky reviewed Mar 7, 2025

View reviewed changes

buildingmotif/dataclasses/template.py Outdated Show resolved Hide resolved

begin refactor with new dependency design

fc1bb6b

TShapinsky reviewed Mar 20, 2025

View reviewed changes

buildingmotif/dataclasses/template.py Show resolved Hide resolved

Merge branch 'develop' of github.com:NREL/buildingmotif into gtf-hand…

d411028

…le-deletes

TShapinsky added 4 commits March 24, 2025 14:19

add error_on_missing_dependency

a0827fe

Fix template library resolution for cases where library is being crea…

9354ae3

…ted from a ttl file

fix dict to Dict

2e13692

default resolution is to current library

3217fc3

gtfierro requested a review from TShapinsky April 3, 2025 15:31

Merge branch 'develop' of github.com:NREL/buildingmotif into gtf-hand…

ab0c732

…le-deletes

TShapinsky approved these changes Jun 3, 2025

View reviewed changes

TShapinsky merged commit 2f80d77 into develop Jun 5, 2025
11 of 12 checks passed

Conversation

gtfierro commented Jan 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TShapinsky commented Feb 17, 2025

Uh oh!

TShapinsky commented Feb 17, 2025

On Model Delete

On Library Delete

Uh oh!

gtfierro commented Feb 18, 2025

Uh oh!

TShapinsky commented Feb 18, 2025

Uh oh!

gtfierro commented Feb 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

TShapinsky commented Feb 25, 2025

Uh oh!

TShapinsky commented Feb 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TShapinsky commented Mar 3, 2025

Handling Overwrites

Needs

Potential issues

Update method

Pros

Cons

Lazy template linking

Pros

Cons

Reworked Template primary keys

Pros

Cons

Statically compiled templates

Pros

Cons

Uh oh!

TShapinsky commented Mar 6, 2025

Uh oh!

Uh oh!

TShapinsky commented Mar 20, 2025

Uh oh!

Uh oh!

TShapinsky commented Mar 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TShapinsky commented Mar 25, 2025

Uh oh!

TShapinsky commented Mar 26, 2025

Uh oh!

gtfierro commented Apr 1, 2025

Uh oh!

gtfierro commented Apr 3, 2025

Uh oh!

TShapinsky commented Jun 5, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

gtfierro commented Jan 15, 2025 •

edited

Loading

gtfierro commented Feb 25, 2025 •

edited

Loading

TShapinsky commented Feb 25, 2025 •

edited

Loading

TShapinsky commented Mar 24, 2025 •

edited

Loading