Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

POC: Add storage to cache policy #15368

Closed
wants to merge 3 commits into from
Closed

Conversation

desertaxle
Copy link
Member

@desertaxle desertaxle commented Sep 12, 2024

Potential DX for specifying where to store cache entries.

Example:

from prefect import task
from prefect.cache_policies import Inputs
from prefect.transactions import IsolationLevel


@task(
    cache_policy=Inputs(
        storage="~/.prefect/cache",
        locks="~/.prefect/locks",
        isolation_level=IsolationLevel.SERIALIZABLE,
    )
)
def my_task(x):
    return x


if __name__ == "__main__":
    print(my_task(10))

This task will store result metadata as a cache/transaction record at ~/.prefect/cache and the raw result to ~/.prefect/storage.

Checklist

  • This pull request references any related issue by including "closes <link to issue>"
    • If no issue exists and your change is not a small fix, please create an issue first.
  • If this pull request adds new functionality, it includes unit tests that cover the changes
  • If this pull request removes docs files, it includes redirect settings in mint.json.
  • If this pull request adds functions or classes, it includes helpful docstrings.

pick 2277e7f Fixes failing tests
Copy link

codspeed-hq bot commented Sep 12, 2024

CodSpeed Performance Report

Merging #15368 will not alter performance

Comparing cache-policy-storage (f2c768c) with main (164d52e)

Summary

✅ 3 untouched benchmarks

@desertaxle
Copy link
Member Author

Closing in favor of #15382

@desertaxle desertaxle closed this Sep 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant