Skip to content

Add method to populate the cache of the schema manually #360

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 16, 2025

Conversation

dgarros
Copy link
Contributor

@dgarros dgarros commented Apr 16, 2025

Fixes #342

This PR mainly add a method to manually populate the cache for the schema manager, the primary use case for this method is to simplify the initialization of the client during unit tests

I also deprecated the timeout parameter while fetching the schema, it was not consistently used within the code and overall I think using the default timeout should be enough for the schema.
I'll open another issue shortly to track its removal

@dgarros dgarros marked this pull request as ready for review April 16, 2025 06:39
@dgarros dgarros requested a review from a team April 16, 2025 06:40
Copy link

codecov bot commented Apr 16, 2025

Codecov Report

Attention: Patch coverage is 76.27119% with 14 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
infrahub_sdk/schema/__init__.py 66.66% 7 Missing and 7 partials ⚠️
@@             Coverage Diff             @@
##           develop     #360      +/-   ##
===========================================
+ Coverage    69.50%   73.79%   +4.28%     
===========================================
  Files           92       92              
  Lines         8202     8499     +297     
  Branches      1582     1664      +82     
===========================================
+ Hits          5701     6272     +571     
+ Misses        2082     1789     -293     
- Partials       419      438      +19     
Flag Coverage Δ
integration-tests 25.17% <22.03%> (?)
python-3.10 46.92% <45.76%> (-0.28%) ⬇️
python-3.11 46.92% <45.76%> (-0.25%) ⬇️
python-3.12 46.92% <45.76%> (-0.25%) ⬇️
python-3.13 46.94% <45.76%> (-0.25%) ⬇️
python-3.9 45.32% <45.76%> (-0.23%) ⬇️
python-filler-3.12 24.36% <33.89%> (-0.69%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
infrahub_sdk/checks.py 70.19% <ø> (ø)
infrahub_sdk/schema/main.py 89.41% <100.00%> (+5.75%) ⬆️
infrahub_sdk/schema/__init__.py 68.65% <66.66%> (-1.32%) ⬇️

... and 14 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Contributor

@ogenstad ogenstad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. For the comments regarding client.timeout I think it's fine to deprecate those in an upcoming PR to as long as we do it a while before 2.0.

I think it could be useful to have some method to get the schema from the cache so that the users are returned a BranchSchema object that they'd then can store locally as ysml or yaml.


Returns:
dict[str, MainSchemaTypes]: Dictionary of all schema organized by kind
"""
branch_schema = self._fetch(branch=branch, namespaces=namespaces, timeout=timeout)
if timeout:
self._deprecated_schema_timeout()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also deprecate timeout in the client.create() methods it's only used to pass the argument to the schema function.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's already covered because if someone use timeout while creating a node, it will generate a deprecation warning.
Having said that I've updated the changelog to reflect that properly

def _deprecated_schema_timeout() -> None:
warnings.warn(
"The 'timeout' parameter is deprecated while fetching the schema and will be removed version 2.0.0 of the Infrahub Python SDK. "
"Use client.default_timeout instead.",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will you also create an issue in the 2.0 milestone to remove this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do

branch = branch or self.client.default_branch

if isinstance(schema, dict):
schema = BranchSchema.from_api_response(data=schema)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In a way I'd prefer if .set_cache took a BranchSchema object directly. That way there's no confusion regarding what the dict[str, Any] should look like. It would mostly be relevant if there are any errors in the schema, then those errors would be raised in the correct location. On the other hand it might be slightly easier for some users if they don't have to convert it into a dict.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BranchSchema is primarily an internal object so in most cases I would expect users to use the dict most of the time.
To clarify, I've added support for SchemaRootAPI as well which is the format returned by the /api/schema endpoint

Copy link

cloudflare-workers-and-pages bot commented Apr 16, 2025

Deploying infrahub-sdk-python with  Cloudflare Pages  Cloudflare Pages

Latest commit: 11fa9f8
Status: ✅  Deploy successful!
Preview URL: https://e55ba8c3.infrahub-sdk-python.pages.dev
Branch Preview URL: https://dga-20250416-set-schema-cach.infrahub-sdk-python.pages.dev

View logs

@@ -102,6 +110,23 @@ def validate_data_against_schema(self, schema: MainSchemaTypesAPI, data: dict) -
message=f"{key} is not a valid value for {identifier}",
)

def set_cache(self, schema: dict[str, Any] | SchemaRootAPI | BranchSchema, branch: str | None = None) -> None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think set_cache should be a private method as it's meant for internal use.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's not meant for internal use, it's meant for user to use during unit tests

@dgarros dgarros merged commit a96e547 into develop Apr 16, 2025
18 checks passed
@dgarros dgarros deleted the dga-20250416-set-schema-cache branch April 16, 2025 09:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants