Skip to content

Fix date range without times #161

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
May 16, 2025
Merged

Fix date range without times #161

merged 15 commits into from
May 16, 2025

Conversation

amc-corey-cox
Copy link
Contributor

As shown in #160 , we don't properly assign the date range when a field doesn't include time. This should fix that and add a test to ensure it is properly assigned.

@amc-corey-cox
Copy link
Contributor Author

Fixes #160

@amc-corey-cox amc-corey-cox linked an issue May 8, 2025 that may be closed by this pull request
@amc-corey-cox amc-corey-cox requested a review from Copilot May 13, 2025 16:08
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes the issue of improperly assigning date ranges without times and improves schema slot range normalization. Key changes include updating tests for date and datetime inference, adding duplicate slot warnings and normalization in the RDF import engine, and adjusting dependency and GitHub Action configurations.

  • Updated test expectations in the RDFS importer and CSV data generalizer tests.
  • Enhanced type annotations and slot range normalization in the RDF import engine.
  • Upgraded configuration in pyproject.toml and GitHub workflows.

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/test_importers/test_rdfs_importer.py Updated expected slot count and added assertions for new slot names.
tests/test_generalizers/test_csv_data_generalizer.py Added new test cases for date and datetime inference.
schema_automator/importers/rdfs_import_engine.py Refactored type annotations, added warnings for duplicate slots, and introduced slot range normalization.
schema_automator/generalizers/csv_data_generalizer.py Extended range inference with a check for dates without times.
pyproject.toml Changed development dependency grouping.
.github/workflows/check-pull-request.yaml Upgraded the caching action version from v2 to v3.

amc-corey-cox and others added 3 commits May 13, 2025 11:22
Adding Co-pilots nit-pick suggestions.. they seem reasonable

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@amc-corey-cox amc-corey-cox linked an issue May 13, 2025 that may be closed by this pull request
@amc-corey-cox
Copy link
Contributor Author

Okay, I finally have the workflows all working and everything passing. I'm going to set this for merge and review. I had to disable windows-latest from the current workflow because it is having a jasonobj issue likely due to a stale dependency stack. I'll address the stale dependencies in #164 and #163 .

Fixes #162

@dalito
Copy link
Member

dalito commented May 14, 2025

The windows issue is related to linkml-runtime. The problem is in .venv\Lib\site-packages\linkml_runtime\utils\context_utils.py. I have a fix but need to find out how to add a proper test.

I believe it is the same problem as here https://github.com/linkml/linkml/pull/2675/files/eed9908418173ab084ac6963c68ea4e4c65d85ad#r2089780280

@dalito
Copy link
Member

dalito commented May 15, 2025

Here is a fix for windows: linkml/linkml-runtime#391

With the proposed change this PR passes all tests on Windows (I tested on a local machine). This was still with the old linkml and linkml-runtime from poetry.lock.

After updating all dependencies which brings linkml to 1.9.1 and linkml-runtime to 1.9.2, I had to fix the import of re in schema_automator\annotators\schema_annotator.py. Then I see only one other error:

FAILED tests/test_importers/test_jsonschema_importer.py::TestJsonSchemaImporter::test_import_hca_project -
FileNotFoundError: [Errno 2] No such file or directory: C:\\Users\\david\\MyProg\\gh-dalito\\schema-automator\\tests\\outputs\\hca\\type\\protocol\\sequencing\\module\\process\\sequencing\\S10x.yaml'

@ptgolden
Copy link
Contributor

@dalito I fixed that one in 617433b. We'll move to that PR once this one is merged.

Copy link
Contributor

@ptgolden ptgolden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made a couple suggestions about making the check for dates a little clearer.

@amc-corey-cox
Copy link
Contributor Author

@ptgolden Does that look like it deals with the date parsing a little more sensibly?

Copy link
Contributor

@ptgolden ptgolden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good! I think it's good to go.

Nitpickily, I would say that is_date_or_datetime is named like a function that returns a boolean, rather than False | "date" | "datetime".

(Also, is it necessary to include the fuzzy arg if it's not used?)

@amc-corey-cox amc-corey-cox merged commit da64950 into main May 16, 2025
3 checks passed
@amc-corey-cox amc-corey-cox deleted the fix_dates_160 branch May 16, 2025 21:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fix 3.9 and 3.10 workflow failures. Schema-Automator incorrectly assigns dates to date-time type
3 participants