Skip to content

Add Scoped per-evaluation external properties to config evaluators#1585

Open
gootik wants to merge 1 commit into
apple:mainfrom
useswype:gootik/evaluation-cache
Open

Add Scoped per-evaluation external properties to config evaluators#1585
gootik wants to merge 1 commit into
apple:mainfrom
useswype:gootik/evaluation-cache

Conversation

@gootik

@gootik gootik commented May 14, 2026

Copy link
Copy Markdown

We would like to use Pkl in our request path, by evaluating the
configuration per request based on the provided context. This usecase
allows us to have the niceties of Pkl and allow for a runtime
per-request evaluation.

For example the following would be very useful:

userId: String = read?("prop:userId") ?? "NO_USER_ID"
env: String = read?("prop:env") ?? "development"

featuresFlags: Listing<Flag> = new Listing {
  new Flag {
    name = "SPECIAL_FEATURE"
    enabled = if (userId == "1234") true else false
  }
  new Flag {
    name = "NEW_FEATURE"
    enabled = if (env == "staging") true else false
  }
}

In RPC services, evaluator setup is stable. The request inputs are the
only things changing. As an example

// Load on startup
var source = ModuleSource.path("config/config.pkl")

...

// Keep calling this per request
var config =
    evaluator.evaluate(
        source,
        Map.of(
            "userId", request.userId(),
            "env", environment
       )
    )

Before this change, the safe option was to create a new evaluator
whenever those properties changed. However, this is not ideal for latency
sensitive environments. This flow uses more CPU and memory for
evaluation of config per request.

Reusing an evaluator was also risky because module/resource evaluation
caches can retain values derived from read("prop:..."), so one request
could see stale inputs from another request.

This PR adds per-evaluation external property overloads. Scoped
evaluations overlay the evaluator’s configured external properties for
one call and use isolated module/resource evaluation caches for that call.

I'm not entirely sure if this is the right approach, so opening the PR
for feedback. Also, note that this PR was achieved with help of Codex,
I'm not sure what is the policy of this repo regarding AI usage for
contributions.

Please let me know if this is not okay.

@bioball

bioball commented May 14, 2026

Copy link
Copy Markdown
Member

In general, I agree that this should be doable. read() and import() are cached per evaluation session, and it should be possible eval with different external properties from the same Evaluator.

Some concerns:

  • I don't think ThreadLocal is correct; we don't want to rely on the same eval all happening in the same thread (a possible optimization in the future is to mux different branches into different threads within the same eval session)
  • The method signature is not future proof. Would be better if it accepted a wrapper class, which allows for adding more things (e.g. env vars) to be configured per-evaluation. Something like evaluateOutputText(ModuleSource moduleSource, Context context), where Context is a record of external properties, env vars, etc, and where there is a builder for it.

@gootik

gootik commented May 14, 2026

Copy link
Copy Markdown
Author

@bioball that's great feedback thank you. I'll work on these items

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants