Make token estimate ratio configurable (#50) - #100
Merged
leynos merged 32 commits intoMay 31, 2026
Conversation
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
WalkthroughThread configurable chars_per_token into adapter config and heuristics; split the adapter into episodic.llm.openai_api submodules with a facade; add preflight and post-response token-budget enforcement, structured diagnostic logs, tests, snapshots, fixtures, and docs. ChangesConfigurable chars-per-token estimation
Sequence Diagram(s)sequenceDiagram
participant Client
participant Adapter as OpenAICompatibleLLMAdapter
participant Utils as openai_api.utils
participant Retry as _send_with_retries
participant Provider as LLM Provider (HTTP)
Client->>Adapter: generate(LLMRequest)
Adapter->>Utils: _estimate_token_count(chars_per_token, prompt)
Utils-->>Adapter: estimated_input_tokens / validation
Adapter->>Retry: _send_with_retries(request_payload)
Retry->>Provider: HTTP POST /chat/completions or /responses
Provider-->>Retry: HTTP response (status + body)
Retry-->>Adapter: provider JSON payload
Adapter->>Utils: validate response usage and shape
Utils-->>Adapter: usage validation result (or raise)
Adapter-->>Client: LLMResponse (or raise)
Possibly related issues
Poem
🚥 Pre-merge checks | ✅ 18 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (18 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. 📋 Issue PlannerBuilt with CodeRabbit's Coding Plans for faster development and fewer bugs. View plan used: ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Reviewer's GuideAdds a configurable chars-per-token ratio to the OpenAI-compatible LLM adapter and threads it through config, validation, preflight token estimation, fixtures, tests, and documentation so operators can tune budget preflight behavior per adapter. Sequence diagram for generate preflight budget using chars_per_tokensequenceDiagram
actor User
participant OpenAICompatibleLLMAdapter
participant LLMRequest
participant LLMTokenBudget
participant _validate_preflight_budget
participant _estimate_token_count
User->>OpenAICompatibleLLMAdapter: generate(request)
OpenAICompatibleLLMAdapter->>LLMRequest: token_budget
LLMRequest-->>OpenAICompatibleLLMAdapter: token_budget
OpenAICompatibleLLMAdapter->>_validate_preflight_budget: request, token_budget, _chars_per_token
_validate_preflight_budget->>_estimate_token_count: chars_per_token, system_prompt, prompt
_estimate_token_count-->>_validate_preflight_budget: estimated_input_tokens
_validate_preflight_budget-->>OpenAICompatibleLLMAdapter: return
OpenAICompatibleLLMAdapter-->>User: LLMResponse
File-Level Changes
Assessment against linked issues
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![codescene-delta-analysis[bot]](https://avatars.githubusercontent.com/in/53921?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
lodyai
Bot
force-pushed
the
issue-50-configurable-estimate-token-count-char-ratio
branch
from
May 15, 2026 11:49
2d784bc to
adc7690
Compare
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
leynos
marked this pull request as ready for review
May 15, 2026 14:16
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: adc7690ff7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@episodic/llm/openai_adapter.py`:
- Line 240: The validation for config.chars_per_token currently only checks >0
and therefore permits NaN/Infinity; update the predicate in the validation tuple
that mentions config.chars_per_token to also call math.isfinite (e.g., change
the tuple from checking config.chars_per_token <= 0 to checking not
math.isfinite(config.chars_per_token) or config.chars_per_token <= 0 and update
the error message to "chars_per_token must be a finite number greater than
zero."); also add parametrised unit tests for the preflight validation (the same
test that covers 0 and -1.0) to include float('nan') and float('inf') to prevent
regressions.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 713d3cee-8bd6-4a8b-bb3d-6b524dc802d2
📒 Files selected for processing (5)
docs/users-guide.mdepisodic/llm/openai_adapter.pytests/fixtures/llm.pytests/test_llm_openai_adapter_budgets.pytests/test_llm_openai_adapter_config.py
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Owner
|
@coderabbitai Have the following failed checks now been resolved? If further work is required, please provide an AI agent prompt for the remaining work to be done to address these failures. Do not treat warnings as optional or aspirational. Where a change is out of scope for this PR, propose a GitHub issue unless one exists already. ❌ Failed checks (4 errors, 6 warnings, 1 inconclusive)
|
Contributor
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: Length of output: 31003 🏁 Script executed: Length of output: 616 🏁 Script executed: Length of output: 46
The one remaining gap — structured log emission at |
Owner
|
@coderabbitai review |
Contributor
✅ Actions performedReview triggered.
|
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/developers-guide.md`:
- Around line 762-767: The paragraph describing
OpenAICompatibleLLMConfig(chars_per_token=...) is too long for the 80-column
rule; rewrap the Markdown paragraph and any bullet lines so no line exceeds 80
columns while preserving the original wording and inline code (chars_per_token,
ceil(len(prompt_text) / chars_per_token), etc.), breaking sentences into shorter
lines and keeping the same meaning and formatting.
- Line 762: The docs use the hyphenated form "pre-flight" which is inconsistent
with codebase terminology; update the occurrence in the sentence containing
OpenAICompatibleLLMConfig(chars_per_token=...) to use "preflight" (one word)
instead of "pre-flight" so it matches PR summary and established technical usage
in the repo.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 55f90282-ba12-4c62-9421-dfd02ada1d35
📒 Files selected for processing (1)
docs/developers-guide.md
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (1)
docs/developers-guide.md (1)
762-767:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winRewrap the paragraph to comply with the 80-column limit.
Lines 763 and 765 exceed 80 columns. Markdown paragraphs and bullet points must be wrapped at 80 columns.
Triage:
[type:syntax/md]Proposed fix
-- `OpenAICompatibleLLMConfig(chars_per_token=...)` controls the preflight - estimate. The value defaults to `4.0`, must be finite and greater than zero, - and is applied as `ceil(len(prompt_text) / chars_per_token)` across the - request prompt and optional system prompt. Tune it by comparing sampled prompt - character counts with provider-reported input-token usage for the target - model and prompt shape. +- `OpenAICompatibleLLMConfig(chars_per_token=...)` controls the preflight + estimate. The value defaults to `4.0`, must be finite and greater than + zero, and is applied as `ceil(len(prompt_text) / chars_per_token)` across + the request prompt and optional system prompt. Tune it by comparing sampled + prompt character counts with provider-reported input-token usage for the + target model and prompt shape.🤖 Prompt for AI agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/developers-guide.md` at lines 762-767, rewrap the bullet point describing OpenAICompatibleLLMConfig(chars_per_token=...) so that no line exceeds 80 columns while preserving inline code formatting (backticks around chars_per_token, 4.0, ceil(len(prompt_text) / chars_per_token)) and the original wording.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/developers-guide.md` around lines 762 - 767, Rewrap the markdown paragraph that documents `OpenAICompatibleLLMConfig(chars_per_token=...)` so no line exceeds 80 columns while preserving inline code formatting; specifically break the sentence about the default value (`4.0`), validity (finite and > 0), and the application `ceil(len(prompt_text) / chars_per_token)` across the request prompt and optional system prompt into shorter lines (and keep the guidance about tuning by comparing sampled prompt character counts with provider-reported input-token usage intact) so each line is ≤80 characters.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@episodic/llm/openai_adapter.py`:
- Line 14: Remove the file-level "# pylint: disable=too-many-lines" from
episodic/llm/openai_adapter.py and refactor the module into smaller files so no
file exceeds 400 logical lines: extract the OpenAIAdapter class (or similarly
named adapter) into its own file, move request-building logic (e.g.
_build_request or build_request) into a separate api/request module, move
response-parsing logic (e.g. _parse_response or parse_response) into a separate
api/response module, and move utility helpers (e.g. retry/formatting/validation
functions) into a utils module; update imports so openai_adapter.py becomes a
thin facade that composes OpenAIClient/OpenAIAdapter with the new modules and
remove the blanket lint suppression.
- Around line 55-60: Refactor the _operation_label function to use Python
structural pattern matching: replace the isinstance-based branching with a
match/case on the input value so it explicitly handles None (return "default"),
LLMProviderOperation (return value.value), and str (return the string);
reference the LLMProviderOperation type in the match patterns and preserve the
current return semantics for all three cases and any fallback.
---
Duplicate comments:
In `@docs/developers-guide.md`:
- Around line 762-767: Rewrap the markdown paragraph that documents
`OpenAICompatibleLLMConfig(chars_per_token=...)` so no line exceeds 80 columns
while preserving inline code formatting; specifically break the sentence about
the default value (`4.0`), validity (finite and > 0), and the application
`ceil(len(prompt_text) / chars_per_token)` across the request prompt and
optional system prompt into shorter lines (and keep the guidance about tuning by
comparing sampled prompt character counts with provider-reported input-token
usage intact) so each line is ≤80 characters.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: e57106ee-7f57-4617-b6be-80aa3a9dd5ad
📒 Files selected for processing (5)
docs/developers-guide.mdepisodic/llm/openai_adapter.pytests/fixtures/llm.pytests/test_llm_openai_adapter_budgets.pytests/test_llm_openai_adapter_config.py
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Owner
|
@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix. Include the file and symbol names indicated in the issue at the head of your response. episodic/llm/openai_api/utils.py Comment on file """Validation, estimation, and logging helpers for the OpenAI adapter."""❌ New issue: Overall Code Complexity |
Owner
|
@coderabbitai review |
Contributor
✅ Actions performedReview triggered.
|
This comment was marked as resolved.
This comment was marked as resolved.
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@episodic/llm/openai_api/adapter.py`:
- Line 3: Remove the line "from __future__ import annotations" from the top of
episodic/llm/openai_api/adapter.py; since the project targets Python >=3.14,
postponed evaluation is default, so delete this import statement (search for the
exact "from __future__ import annotations" line in adapter.py and remove it).
In `@episodic/llm/openai_api/response.py`:
- Line 3: Remove the redundant import `from __future__ import annotations` from
the module episodic.llm.openai_api.response by deleting that import line; since
the project targets Python >=3.14, deferred annotations are implicit, so simply
remove the `from __future__ import annotations` statement to eliminate the
unnecessary import.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 00b4b442-db76-407a-9213-d5e86de36316
📒 Files selected for processing (7)
docs/developers-guide.mdepisodic/llm/openai_adapter.pyepisodic/llm/openai_api/__init__.pyepisodic/llm/openai_api/adapter.pyepisodic/llm/openai_api/request.pyepisodic/llm/openai_api/response.pyepisodic/llm/openai_api/utils.py
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Switch `_log_error_event` to consult a `contextvars.ContextVar` for log output redirection, and replace the `monkeypatch.setattr` fixture with a ContextVar-based `openai_log_spy`. This removes the shared mutable state that required `@pytest.mark.xdist_group` serialisation across budget and config test modules. Remove the now-unnecessary `xdist_group` markers from both `tests/test_llm_openai_adapter_budgets.py` and `tests/test_llm_openai_adapter_config.py`.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Owner
|
@coderabbitai review |
Contributor
✅ Actions performedReview triggered.
|
Contributor
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
tests/steps/test_llm_adapter_steps.py (1)
332-343:⚠️ Potential issue | 🟠 Major | ⚡ Quick winMake the custom-ratio BDD scenario discriminative.
Line 339 sets a permissive budget, so the step passes whether
chars_per_tokenis applied or ignored. Assert boundary behaviour that can only pass when the configured ratio is actually used.🧪 Patch sketch to make the scenario prove ratio wiring
+import math @@ def adapter_generates_custom_ratio( @@ - """Generate text with chars_per_token=2.0 over real HTTP.""" + """Generate text with a ratio-sensitive preflight budget over real HTTP.""" + combined_chars = ( + len(typ.cast("RenderedPrompt", context.guardrail_prompt).text) + + len(typ.cast("RenderedPrompt", context.rendered_prompt).text) + ) + ratio = 8.0 + max_input_tokens = math.ceil(combined_chars / ratio) _run_generate( _function_scoped_runner, context, _GenerateOptions( max_attempts=1, - chars_per_token=2.0, + chars_per_token=ratio, token_budget=LLMTokenBudget( - max_input_tokens=2000, + max_input_tokens=max_input_tokens, max_output_tokens=200, - max_total_tokens=2200, + max_total_tokens=max_input_tokens + 200, ), ), ) @@ def assert_generated_first_attempt(context: LLMAdapterContext) -> None: @@ - assert context.generated_text == "BDD generated episode draft.", ( - "adapter should return generated text when chars_per_token=2.0 is configured" - ) + assert context.generated_text == "BDD generated episode draft.", ( + "adapter should honour configured chars_per_token at a tight preflight boundary" + )Also applies to: 348-356
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/steps/test_llm_adapter_steps.py` around lines 332 - 343, The BDD scenario is too permissive: update the _GenerateOptions/LLMTokenBudget used in the chars_per_token test so the token budget is tight enough to fail if chars_per_token is ignored; specifically reduce max_output_tokens (or max_total_tokens) in the LLMTokenBudget passed to _run_generate in the chars_per_token=2.0 case so only a response that respects the 2.0 chars-per-token ratio can fit, and make the analogous change for the second scenario around the other call (lines 348-356) so both tests assert boundary behaviour that proves the ratio is actually applied (refer to _GenerateOptions, chars_per_token, and LLMTokenBudget to locate and adjust).
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@tests/steps/test_llm_adapter_steps.py`:
- Around line 332-343: The BDD scenario is too permissive: update the
_GenerateOptions/LLMTokenBudget used in the chars_per_token test so the token
budget is tight enough to fail if chars_per_token is ignored; specifically
reduce max_output_tokens (or max_total_tokens) in the LLMTokenBudget passed to
_run_generate in the chars_per_token=2.0 case so only a response that respects
the 2.0 chars-per-token ratio can fit, and make the analogous change for the
second scenario around the other call (lines 348-356) so both tests assert
boundary behaviour that proves the ratio is actually applied (refer to
_GenerateOptions, chars_per_token, and LLMTokenBudget to locate and adjust).
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 095cda74-459b-4986-a9ab-f524b05854df
📒 Files selected for processing (5)
episodic/llm/openai_api/utils.pytests/fixtures/llm.pytests/steps/test_llm_adapter_steps.pytests/test_llm_openai_adapter_budgets.pytests/test_llm_openai_adapter_config.py
💤 Files with no reviewable changes (2)
- tests/test_llm_openai_adapter_config.py
- tests/test_llm_openai_adapter_budgets.py
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
…50) Reduce max_input_tokens and max_total_tokens in both BDD scenarios so the preflight budget checks are meaningfully exercised rather than passing with 4x+ headroom. The tighter budgets prove that chars_per_token is actually applied during adapter.generate(). Note: chars_per_token=2.0 produces a higher (stricter) token estimate than the default 4.0, so a budget that passes at 2.0 inherently passes at 4.0. The tightening makes both budgets proportional to their respective estimates rather than attempting an impossible cross-ratio discrimination.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Pull OpenAI adapter classes out of the public `episodic.llm` re-exports so the domain port namespace exposes only port contracts. Update four call sites to import from `episodic.llm.openai_adapter` directly. `episodic.llm.__init__` now exports only the 11 port-contract symbols defined in `episodic.llm.ports`.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Emit an `openai_adapter.retries_exhausted` log event when Tenacity retries are exhausted, including the `max_attempts` and the last error type, so operators can observe retry exhaustion from structured logs.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Owner
|
@coderabbitai review |
Contributor
✅ Actions performedReview triggered.
|
Contributor
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
echo "=== episodic/llm/__init__.py ==="
cat episodic/llm/__init__.py
echo ""
echo "=== episodic/llm/openai_api/adapter.py (_send_with_retries section) ==="
grep -n "_send_with_retries\|RetryError\|_log_error_event\|retries_exhausted\|LLMTransientProviderError" episodic/llm/openai_api/adapter.py
echo ""
echo "=== Open GitHub issues ==="
gh issue list --repo leynos/episodic --state open --json number,title,url | head -30Length of output: 4929
|
Owner
|
@coderabbitai review |
Contributor
✅ Actions performedReview triggered.
|
Owner
|
@coderabbitai review |
Contributor
✅ Actions performedReview triggered.
|
Add explicit logging side-effect notes to OpenAI adapter validation helper Docstrings, so their command-style responsibilities are visible at the call site. Document the user-facing migration path for OpenAI adapter imports and clarify why the internal OpenAI API package stays split by request, response, validation, and adapter responsibilities.
Owner
|
@coderabbitai review |
Contributor
✅ Actions performedReview triggered.
|
Owner
|
@coderabbitai please create a new GitHub issue for the following: ❌ Failed checks (2 warnings)
|
Contributor
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
gh issue create \
--repo leynos/episodic \
--title "Observability: add chars_per_token to usage_budget_exceeded logs, budget-rejection metrics, and HTTP tracing spans" \
--body "## Background
PR `#100` (https://github.com/leynos/episodic/pull/100) introduced structured logging at configuration-validation and budget-rejection decision boundaries. A subsequent observability review identified three remaining gaps.
## Required work
### 1. Add \`chars_per_token\` to \`openai_adapter.usage_budget_exceeded\` log events
The \`_validate_usage_budget\` helper in \`episodic/llm/openai_api/utils.py\` currently emits \`openai_adapter.usage_budget_exceeded\` events without including the \`chars_per_token\` value that was used during preflight estimation. Add \`chars_per_token\` as a field to those log events so operators can correlate the configured ratio with observed budget overruns.
### 2. Add latency and budget-rejection metrics
Add counters and/or histograms at the following points in \`episodic/llm/openai_api/adapter.py\` and \`episodic/llm/openai_api/utils.py\`:
- Preflight budget rejections (\`openai_adapter.preflight_budget_exceeded\`)
- Post-response usage budget rejections (\`openai_adapter.usage_budget_exceeded\`)
- HTTP request latency per attempt in \`_send_once\`
- Retry exhaustion events (\`openai_adapter.retries_exhausted\`)
### 3. Add distributed tracing spans at HTTP boundaries
Instrument \`_send_once\` and \`_send_with_retries\` in \`episodic/llm/openai_api/adapter.py\` with distributed tracing spans (e.g., OpenTelemetry) so operators can observe per-attempt latency and error types in a trace.
## Context
- Observability gap identified during review of PR `#100` (https://github.com/leynos/episodic/pull/100).
- Related issue: `#110` (structured logging at validation/budget failure boundaries).
- Requested by `@leynos`." \
--assignee leynosLength of output: 1989
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This branch makes the OpenAI-compatible adapter's preflight token estimate configurable so operators can tune the characters-per-token ratio for models whose tokenizer shape differs from the default heuristic.
Closes #50.
Review walkthrough
chars_per_tokenonOpenAICompatibleLLMConfig, the positive-value validation, and the threaded preflight estimate.Validation
make check-fmt: passed.make lint: passed.make typecheck: passed.make markdownlint: passed.make nixie: passed.make test PYTEST_XDIST_WORKERS=1: OpenAI adapter coverage passed; the full suite hit unrelatedpytest-timeoutsetup errors in rotating py-pglite-backed async fixture tests.make test PYTEST_XDIST_WORKERS=0: OpenAI adapter coverage passed; the full suite hit one unrelatedpytest-timeoutsetup error in a py-pglite-backed async fixture test.tests/canonical_storage/test_tei_headers.py::test_tei_header_large_raw_xml_round_trip_uses_compressed_storage,tests/canonical_storage/test_workflow_checkpoints.py::test_checkpoint_store_persists_across_unit_of_work,tests/test_ingestion_integration.py::test_ingest_multi_source_preserves_all_sources, andtests/canonical_storage/test_episodes.py::test_episode_get_remains_compatible_with_legacy_uncompressed_rows.Notes
The full-suite timeouts occurred in unrelated async fixture setup while the event loop was waiting in
selectors.EpollSelector.select. The affected tests changed between runs and passed when isolated, while the OpenAI adapter tests consistently passed.Summary by Sourcery
Make the OpenAI-compatible adapter’s preflight token estimation use a configurable characters-per-token ratio and validate it via adapter config.
New Features:
Enhancements:
Documentation:
Tests: