feat(blocks): dynamic COST_USD billing + close 8 cost-leak surfaces - #12909
Merged
Conversation
github-project-automation
Bot
moved this to 🆕 Needs initial review
in AutoGPT development kanban
github-actions
Bot
added
platform/backend
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughAdds USD-based provider cost extraction and merging into NodeExecutionStats for multiple blocks (Claude Code, Exa, BaaS, Pinecone insert, Codex, Perplexity, ElevenLabs narration), updates BLOCK_COSTS to register several blocks under COST_USD, and adds/updates tests and many small formatting refinements across the codebase. Changes
Sequence Diagram(s)(omitted) Estimated Code Review Effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested Labels
Suggested Reviewers
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
🔍 PR Overlap DetectionThis check compares your PR against all other open PRs targeting the same branch to detect potential merge conflicts early. 🔴 Merge Conflicts DetectedThe following PRs have been tested and will have merge conflicts if merged after this PR. Consider coordinating with the authors.
🟢 Low Risk — File Overlap OnlyThese PRs touch the same files but different sections (click to expand)
Summary: 4 conflict(s), 0 medium risk, 3 low risk (out of 7 PRs with file overlap) Auto-generated on push. Ignores: |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
autogpt_platform/backend/backend/blocks/claude_code.py (1)
439-446: Optional: guardfloat()against a malformed CLI value.If a future CLI build ever emits
total_cost_usdas a non-numeric (e.g."n/a"or an object),float(total_cost_usd)raisesValueError/TypeError, which propagates out of the JSON-parsetryand is re-raised asClaudeCodeExecutionError— losing the already-producedresponse. Cheap to harden:🛡️ Proposed defensive conversion
- total_cost_usd = output_data.get("total_cost_usd") - if total_cost_usd is not None: - self.merge_stats( - NodeExecutionStats( - provider_cost=float(total_cost_usd), - provider_cost_type="cost_usd", - ) - ) + total_cost_usd = output_data.get("total_cost_usd") + if isinstance(total_cost_usd, (int, float)): + self.merge_stats( + NodeExecutionStats( + provider_cost=float(total_cost_usd), + provider_cost_type="cost_usd", + ) + )Note: this would add an
isinstancecheck, which the coding guidelines discourage for type dispatch. An alternative is a narrowtry/except (TypeError, ValueError)around the conversion.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/blocks/claude_code.py` around lines 439 - 446, The code currently calls float(total_cost_usd) when total_cost_usd is present, which can raise ValueError/TypeError on malformed values; update the block around total_cost_usd in claude_code.py to defensively convert: attempt to cast inside a narrow try/except (catch TypeError and ValueError) and only call self.merge_stats(NodeExecutionStats(... provider_cost=...)) when conversion succeeds (or skip/omit provider_cost if it fails), referencing total_cost_usd, merge_stats, NodeExecutionStats, and provider_cost in your changes.autogpt_platform/backend/backend/blocks/claude_code_cost_test.py (1)
85-85: Nit: hoistimport jsonto module top.♻️ Suggested change
@@ top of file @@ +import json from unittest.mock import MagicMock, patch @@ in tests @@ - import json as _json - - raw_output = _json.dumps(...) - output_data = _json.loads(raw_output) + raw_output = json.dumps(...) + output_data = json.loads(raw_output)As per coding guidelines: "Use top-level imports only — avoid local/inner imports except for lazy imports of heavy optional dependencies like
openpyxl".Also applies to: 123-123
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/blocks/claude_code_cost_test.py` at line 85, Hoist the local import "import json as _json" out of the function body to the module top-level (add a single "import json" at the top of claude_code_cost_test.py), remove the inner alias import and update any uses of "_json" in this module to "json" (also adjust the second occurrence noted around the other local import) so the module uses a single top-level json import per the import guidelines.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@autogpt_platform/backend/backend/blocks/claude_code_cost_test.py`:
- Around line 74-136: The tests are tautological because they replicate the
parsing logic instead of exercising ClaudeCodeBlock's real parser; extract the
parsing into a helper on ClaudeCodeBlock (e.g. _record_cli_cost(self,
output_data: dict) or similar) and update claude_code.py to call that helper
from execute_claude_code; then change these tests to call the new helper
directly (asserting merge_stats behavior via patching) so they validate the
block's parsing path (alternatively, implement the heavier approach by invoking
execute_claude_code with a mocked sandbox/commands.run that returns JSON stdout
and assert merge_stats was/wasn't called).
---
Nitpick comments:
In `@autogpt_platform/backend/backend/blocks/claude_code_cost_test.py`:
- Line 85: Hoist the local import "import json as _json" out of the function
body to the module top-level (add a single "import json" at the top of
claude_code_cost_test.py), remove the inner alias import and update any uses of
"_json" in this module to "json" (also adjust the second occurrence noted around
the other local import) so the module uses a single top-level json import per
the import guidelines.
In `@autogpt_platform/backend/backend/blocks/claude_code.py`:
- Around line 439-446: The code currently calls float(total_cost_usd) when
total_cost_usd is present, which can raise ValueError/TypeError on malformed
values; update the block around total_cost_usd in claude_code.py to defensively
convert: attempt to cast inside a narrow try/except (catch TypeError and
ValueError) and only call self.merge_stats(NodeExecutionStats(...
provider_cost=...)) when conversion succeeds (or skip/omit provider_cost if it
fails), referencing total_cost_usd, merge_stats, NodeExecutionStats, and
provider_cost in your changes.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: bea94ea1-2c4d-4529-a94e-9a7704ce3657
📒 Files selected for processing (3)
autogpt_platform/backend/backend/blocks/claude_code.pyautogpt_platform/backend/backend/blocks/claude_code_cost_test.pyautogpt_platform/backend/backend/data/block_cost_config.py
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (12)
- GitHub Check: check API types
- GitHub Check: Seer Code Review
- GitHub Check: test (3.11)
- GitHub Check: type-check (3.13)
- GitHub Check: test (3.12)
- GitHub Check: test (3.13)
- GitHub Check: type-check (3.12)
- GitHub Check: type-check (3.11)
- GitHub Check: end-to-end tests
- GitHub Check: Analyze (typescript)
- GitHub Check: Analyze (python)
- GitHub Check: Check PR Status
🧰 Additional context used
📓 Path-based instructions (6)
autogpt_platform/backend/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
autogpt_platform/backend/**/*.py: Use Python 3.11 (required; managed by Poetry via pyproject.toml) for backend development
Always run 'poetry run format' (Black + isort) before linting in backend development
Always run 'poetry run lint' (ruff) after formatting in backend development
autogpt_platform/backend/**/*.py: Usepoetry run ...command for executing Python package dependencies
Use top-level imports only — avoid local/inner imports except for lazy imports of heavy optional dependencies likeopenpyxl
Use absolute imports withfrom backend.module import ...for cross-package imports; single-dot relative imports are acceptable for sibling modules within the same package; avoid double-dot relative imports
Do not use duck typing — avoidhasattr/getattr/isinstancefor type dispatch; use typed interfaces/unions/protocols instead
Use Pydantic models over dataclass/namedtuple/dict for structured data
Do not use linter suppressors — no# type: ignore,# noqa,# pyright: ignore; fix the type/code instead
Prefer list comprehensions over manual loop-and-append patterns
Use early return with guard clauses first to avoid deep nesting
Use%sfor deferred interpolation indebuglog statements for efficiency; use f-strings elsewhere for readability (e.g.,logger.debug("Processing %s items", count)vslogger.info(f"Processing {count} items"))
Sanitize error paths by usingos.path.basename()in error messages to avoid leaking directory structure
Be aware of TOCTOU (Time-Of-Check-Time-Of-Use) issues — avoid check-then-act patterns for file access and credit charging
Usetransaction=Truefor Redis pipelines to ensure atomicity on multi-step operations
Usemax(0, value)guards for computed values that should never be negative
Keep files under ~300 lines; if a file grows beyond this, split by responsibility (extract helpers, models, or a sub-module into a new file)
Keep functions under ~40 lines; extract named helpers when a function grows longer
...
Files:
autogpt_platform/backend/backend/blocks/claude_code.pyautogpt_platform/backend/backend/data/block_cost_config.pyautogpt_platform/backend/backend/blocks/claude_code_cost_test.py
autogpt_platform/backend/backend/blocks/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
autogpt_platform/backend/backend/blocks/**/*.py: Inherit from 'Block' base class with input/output schemas when adding new blocks in backend
Implement 'run' method with proper error handling in backend blocks
Generate block UUID using 'uuid.uuid4()' when creating new blocks in backend
Write tests alongside block implementation when adding new blocks in backend
autogpt_platform/backend/backend/blocks/**/*.py: For blocks handling files, usestore_media_file()withreturn_format="for_local_processing"when processing with local tools (ffmpeg, MoviePy, PIL)
For blocks handling files, usestore_media_file()withreturn_format="for_external_api"when sending content to external APIs (Replicate, OpenAI)
For blocks returning files, usestore_media_file()withreturn_format="for_block_output"to enable auto-adaptation to execution context (workspace:// in CoPilot, data URI in graphs)
When creating new blocks, inherit fromBlockbase class, define input/output schemas usingBlockSchema, implement asyncrunmethod, and generate unique block ID usinguuid.uuid4()
Files:
autogpt_platform/backend/backend/blocks/claude_code.pyautogpt_platform/backend/backend/blocks/claude_code_cost_test.py
autogpt_platform/{backend,autogpt_libs}/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
Format Python code with
poetry run format
Files:
autogpt_platform/backend/backend/blocks/claude_code.pyautogpt_platform/backend/backend/data/block_cost_config.pyautogpt_platform/backend/backend/blocks/claude_code_cost_test.py
autogpt_platform/backend/backend/data/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
All data access in backend requires user ID checks; verify this for any 'data/*.py' changes
Files:
autogpt_platform/backend/backend/data/block_cost_config.py
autogpt_platform/**/data/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
For changes touching
data/*.py, validate user ID checks or explain why not needed
Files:
autogpt_platform/backend/backend/data/block_cost_config.py
autogpt_platform/backend/**/*_test.py
📄 CodeRabbit inference engine (autogpt_platform/backend/AGENTS.md)
autogpt_platform/backend/**/*_test.py: Use pytest with snapshot testing for API responses
Colocate test files with source files using*_test.pynaming convention
Mock at boundaries — mock where the symbol is used, not where it's defined; after refactoring, update mock targets to match new module paths
UseAsyncMockfromunittest.mockfor async functions in tests
When writing tests, use Test-Driven Development (TDD): write failing tests marked with@pytest.mark.xfailbefore implementation, then remove the marker once the implementation is complete
When creating snapshots in tests, usepoetry run pytest path/to/test.py --snapshot-update; always review snapshot changes withgit diffbefore committing
Files:
autogpt_platform/backend/backend/blocks/claude_code_cost_test.py
🧠 Learnings (25)
📓 Common learnings
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:1064-1111
Timestamp: 2026-04-23T13:53:29.246Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, the E2B sandbox blocks (`ExecuteCodeBlock`, `InstantiateCodeSandboxBlock`, `ExecuteCodeStepBlock`) are intentionally billed per-node-execution walltime (`BlockCostType.SECOND`, `cost_divisor=10`, 1 credit per 10s) using `stats.walltime` from the `async_time_measured` decorator in `manager.py::_on_node_execution`. Idle time between steps is deliberately absorbed by the platform — this makes per-step charges predictable and user-visible. Do NOT flag the absence of sandbox-lifetime billing as a bug. Future migration to raw sandbox uptime billing would require plumbing `provider_cost_type="sandbox_seconds"` from the E2B SDK into `NodeExecutionStats` and switching to `COST_USD`/`SECOND` against that field; until then, walltime-per-execution is the correct and intentional model. Established in PR `#12894`.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12876
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:0-0
Timestamp: 2026-04-22T04:01:32.723Z
Learning: In `autogpt_platform/backend/backend/copilot/tools/helpers.py`, the `run_block` copilot path charges ONLY the user credit wallet (via `spend_credits` / `_charge_block_credits`). The microdollar rate-limit counter (`record_cost_usage`) is NOT incremented for `run_block` block executions — the `_record_block_microdollar_cost` helper was explicitly reverted (commit 16ae0f7b5, PR `#12876`) to avoid double-accounting. Do NOT flag missing microdollar recording in `execute_block` as a bug; the credit wallet is the sole billing mechanism for copilot `run_block` calls.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:271-277
Timestamp: 2026-04-23T13:53:40.315Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, `compute_token_credits()` intentionally returns `MODEL_COST[model]` (the flat tier) on pre-flight (when `stats is None`) for the `TOKENS` billing path. Returning 0 pre-flight would allow a zero-balance user to bypass the credit gate and trigger an LLM call, with the insufficient-balance error only surfacing post-flight (a billing leak). The overcharge concern (actual token cost < MODEL_COST estimate) is handled by `_charge_reconciled_usage_sync` in `autogpt_platform/backend/backend/executor/billing.py`, which issues a negative-delta refund via `spend_credits(cost=negative)` when real usage falls below the pre-flight estimate. Do NOT flag the MODEL_COST pre-flight floor in this function as an overcharge bug; the refund path covers it.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12893
File: autogpt_platform/backend/backend/blocks/ayrshare/post_to_tiktok.py:24-24
Timestamp: 2026-04-23T12:55:26.122Z
Learning: In `Significant-Gravitas/AutoGPT`, the `cost(*costs)` decorator in `autogpt_platform/backend/backend/sdk/cost_integration.py` evaluates `input_data` **at input-evaluation time, before `run()` executes**. Mutating fields of `input_data` inside a block's `run()` method (e.g., `input_data.is_video = has_video`) has **no effect on billing** because the cost filter has already been applied. For blocks that derive a computed boolean from both an explicit field and URL-sniffing (like `has_video` in `PostToTikTokBlock`), the explicit field (e.g., `is_video`) is the sole billing signal; callers must set it correctly. This applies to all blocks under `autogpt_platform/backend/backend/blocks/`.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:1112-1119
Timestamp: 2026-04-23T13:55:24.409Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, `AIVideoGeneratorBlock` is intentionally billed by walltime seconds (`BlockCostType.SECOND`, `cost_amount=3`, no `cost_divisor`) using `fal_credentials`. The FAL SDK does not currently expose per-request `provider_cost` or output duration in the response path used; walltime is the best available billing signal. Do NOT flag the absence of `COST_USD` or output-duration billing as a bug. Migration to `BlockCostType.COST_USD` against FAL-reported spend would require an SDK upgrade or a new stats-scraping path and is deferred until the FAL SDK exposes that signal. Established in PR `#12894`.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12385
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:149-185
Timestamp: 2026-03-15T15:30:09.706Z
Learning: In autogpt_platform/backend/backend/copilot/tools/helpers.py, within execute_block, when InsufficientBalanceError occurs after post-execution credit charging (concurrent balance drain after pre-check passed), this is treated as a non-fatal billing leak: log at ERROR level with structured JSON fields `{"billing_leak": True, "user_id": ..., "cost": ...}` for monitoring/alerting, then return BlockOutputResponse normally. Discarding the output would worsen UX since the block already executed with potential side effects. Reuse the credit_model obtained during the pre-execution balance check (guarded by `if cost > 0 and credit_model:`) for the post-execution charge; do not perform a second get_user_credit_model call.
📚 Learning: 2026-04-23T13:55:24.409Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:1112-1119
Timestamp: 2026-04-23T13:55:24.409Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, `AIVideoGeneratorBlock` is intentionally billed by walltime seconds (`BlockCostType.SECOND`, `cost_amount=3`, no `cost_divisor`) using `fal_credentials`. The FAL SDK does not currently expose per-request `provider_cost` or output duration in the response path used; walltime is the best available billing signal. Do NOT flag the absence of `COST_USD` or output-duration billing as a bug. Migration to `BlockCostType.COST_USD` against FAL-reported spend would require an SDK upgrade or a new stats-scraping path and is deferred until the FAL SDK exposes that signal. Established in PR `#12894`.
Applied to files:
autogpt_platform/backend/backend/blocks/claude_code.pyautogpt_platform/backend/backend/data/block_cost_config.pyautogpt_platform/backend/backend/blocks/claude_code_cost_test.py
📚 Learning: 2026-04-23T12:55:26.122Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12893
File: autogpt_platform/backend/backend/blocks/ayrshare/post_to_tiktok.py:24-24
Timestamp: 2026-04-23T12:55:26.122Z
Learning: Cost billing via the cost(*costs) decorator is applied at input-evaluation time (before a block’s run() executes). Therefore, mutating input_data inside run() will not change billing. When a block’s billing depends on a field plus URL/sniff-derived signals, treat the explicitly declared billing field (e.g., is_video) as the only billing source—set it correctly before run() (or in the code path that occurs before the decorator evaluates input_data). This should be checked for all blocks under autogpt_platform/backend/backend/blocks/ so billing signals are not mistakenly assumed to update during run().
Applied to files:
autogpt_platform/backend/backend/blocks/claude_code.pyautogpt_platform/backend/backend/blocks/claude_code_cost_test.py
📚 Learning: 2026-04-23T13:53:29.246Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:1064-1111
Timestamp: 2026-04-23T13:53:29.246Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, the E2B sandbox blocks (`ExecuteCodeBlock`, `InstantiateCodeSandboxBlock`, `ExecuteCodeStepBlock`) are intentionally billed per-node-execution walltime (`BlockCostType.SECOND`, `cost_divisor=10`, 1 credit per 10s) using `stats.walltime` from the `async_time_measured` decorator in `manager.py::_on_node_execution`. Idle time between steps is deliberately absorbed by the platform — this makes per-step charges predictable and user-visible. Do NOT flag the absence of sandbox-lifetime billing as a bug. Future migration to raw sandbox uptime billing would require plumbing `provider_cost_type="sandbox_seconds"` from the E2B SDK into `NodeExecutionStats` and switching to `COST_USD`/`SECOND` against that field; until then, walltime-per-execution is the correct and intentional model. Established in PR `#12894`.
Applied to files:
autogpt_platform/backend/backend/blocks/claude_code.pyautogpt_platform/backend/backend/data/block_cost_config.pyautogpt_platform/backend/backend/blocks/claude_code_cost_test.py
📚 Learning: 2026-04-22T04:01:32.723Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12876
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:0-0
Timestamp: 2026-04-22T04:01:32.723Z
Learning: In `autogpt_platform/backend/backend/copilot/tools/helpers.py`, the `run_block` copilot path charges ONLY the user credit wallet (via `spend_credits` / `_charge_block_credits`). The microdollar rate-limit counter (`record_cost_usage`) is NOT incremented for `run_block` block executions — the `_record_block_microdollar_cost` helper was explicitly reverted (commit 16ae0f7b5, PR `#12876`) to avoid double-accounting. Do NOT flag missing microdollar recording in `execute_block` as a bug; the credit wallet is the sole billing mechanism for copilot `run_block` calls.
Applied to files:
autogpt_platform/backend/backend/blocks/claude_code.pyautogpt_platform/backend/backend/data/block_cost_config.pyautogpt_platform/backend/backend/blocks/claude_code_cost_test.py
📚 Learning: 2026-03-15T15:30:09.706Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12385
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:149-185
Timestamp: 2026-03-15T15:30:09.706Z
Learning: In autogpt_platform/backend/backend/copilot/tools/helpers.py, within execute_block, when InsufficientBalanceError occurs after post-execution credit charging (concurrent balance drain after pre-check passed), this is treated as a non-fatal billing leak: log at ERROR level with structured JSON fields `{"billing_leak": True, "user_id": ..., "cost": ...}` for monitoring/alerting, then return BlockOutputResponse normally. Discarding the output would worsen UX since the block already executed with potential side effects. Reuse the credit_model obtained during the pre-execution balance check (guarded by `if cost > 0 and credit_model:`) for the post-execution charge; do not perform a second get_user_credit_model call.
Applied to files:
autogpt_platform/backend/backend/blocks/claude_code.pyautogpt_platform/backend/backend/data/block_cost_config.pyautogpt_platform/backend/backend/blocks/claude_code_cost_test.py
📚 Learning: 2026-04-03T13:50:10.521Z
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12206
File: autogpt_platform/backend/backend/api/external/v2/integrations/helpers.py:25-46
Timestamp: 2026-04-03T13:50:10.521Z
Learning: In `autogpt_platform/backend/backend/api/external/v2/integrations/helpers.py`, `CredentialInfo.from_internal` is intentionally a read-only external API view that exposes only: id, type, provider, title, scopes, expires_at. It omits internal metadata and secret fields by design. Do not flag omitted fields in CredentialInfo as missing information — the limited field set is the correct external API contract.
Applied to files:
autogpt_platform/backend/backend/blocks/claude_code.py
📚 Learning: 2026-02-05T04:11:00.596Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 11796
File: autogpt_platform/backend/backend/blocks/video/concat.py:3-4
Timestamp: 2026-02-05T04:11:00.596Z
Learning: In autogpt_platform/backend/backend/blocks/**/*.py, when creating a new block, generate a UUID once with uuid.uuid4() and hard-code the resulting string as the block's id parameter. Do not call uuid.uuid4() at runtime; IDs must be constant across all imports and runs to ensure stability.
Applied to files:
autogpt_platform/backend/backend/blocks/claude_code.pyautogpt_platform/backend/backend/blocks/claude_code_cost_test.py
📚 Learning: 2026-03-16T16:32:21.686Z
Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/pods.py:62-74
Timestamp: 2026-03-16T16:32:21.686Z
Learning: In autogpt_platform/backend/backend/blocks/, the Block base class execute() already wraps run() in a try/except to convert uncaught exceptions into BlockExecutionError/BlockUnknownError. Do not add per-block try/except in individual block run() methods, as this is not the established pattern (e.g., Gmail, Slack, Todoist blocks omit it). Only use explicit try/except within blocks that need to distinguish between success and error yield paths inside a generator (e.g., attachment blocks). This guidance applies to all Python files under autogpt_platform/backend/backend/blocks/ and similar block implementations; avoid duplicating error handling in run() unless a block requires generator-based branching.
Applied to files:
autogpt_platform/backend/backend/blocks/claude_code.pyautogpt_platform/backend/backend/blocks/claude_code_cost_test.py
📚 Learning: 2026-02-26T17:02:22.448Z
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12211
File: .pre-commit-config.yaml:160-179
Timestamp: 2026-02-26T17:02:22.448Z
Learning: Keep the pre-commit hook pattern broad for autogpt_platform/backend to ensure OpenAPI schema changes are captured. Do not narrow to backend/api/ alone, since the generated schema depends on Pydantic models across multiple directories (backend/data/, backend/blocks/, backend/copilot/, backend/integrations/, backend/util/). Narrowing could miss schema changes and cause frontend type desynchronization.
Applied to files:
autogpt_platform/backend/backend/blocks/claude_code.pyautogpt_platform/backend/backend/data/block_cost_config.pyautogpt_platform/backend/backend/blocks/claude_code_cost_test.py
📚 Learning: 2026-03-05T15:42:08.207Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12297
File: .claude/skills/backend-check/SKILL.md:14-16
Timestamp: 2026-03-05T15:42:08.207Z
Learning: In Python files under autogpt_platform/backend (recursively), rely on poetry run format to perform formatting (Black + isort) and linting (ruff). Do not run poetry run lint as a separate step after poetry run format, since format already includes linting checks.
Applied to files:
autogpt_platform/backend/backend/blocks/claude_code.pyautogpt_platform/backend/backend/data/block_cost_config.pyautogpt_platform/backend/backend/blocks/claude_code_cost_test.py
📚 Learning: 2026-03-16T16:30:11.452Z
Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/threads.py:80-102
Timestamp: 2026-03-16T16:30:11.452Z
Learning: In autogpt_platform/backend/backend/blocks/ (and related blocks under autogpt_platform/backend/backend/blocks/), do not add try/except blocks around a block's run() method for standard error propagation. The block executor framework (backend/executor/manager.py) catches uncaught exceptions from run() and emits them on the 'error' output. Only add explicit try/except blocks when you need to control partial outputs in failure cases (e.g., certain outputs must not be yielded on error, as in attachment blocks). This is the standard pattern across the codebase; apply it broadly to blocks' run() implementations.
Applied to files:
autogpt_platform/backend/backend/blocks/claude_code.pyautogpt_platform/backend/backend/blocks/claude_code_cost_test.py
📚 Learning: 2026-03-16T16:30:23.196Z
Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/pods.py:62-74
Timestamp: 2026-03-16T16:30:23.196Z
Learning: In any Python file under autogpt_platform/backend/backend/blocks, do not add a try/except around run() solely for standard error handling. The block framework’s _execute() in _base.py already catches unhandled exceptions and re-raises as BlockExecutionError or BlockUnknownError. If you yield ("error", message), _execute() raises BlockExecutionError immediately, so the error port will not propagate downstream. Reserve explicit try/except for scenarios where you must control partial output (e.g., attachment blocks that must skip yielding content_base64 on failure).
Applied to files:
autogpt_platform/backend/backend/blocks/claude_code.pyautogpt_platform/backend/backend/blocks/claude_code_cost_test.py
📚 Learning: 2026-03-16T16:30:11.452Z
Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/threads.py:80-102
Timestamp: 2026-03-16T16:30:11.452Z
Learning: Do not wrap synchronous AgentMail SDK calls with asyncio.to_thread() in blocks under autogpt_platform/backend/backend/blocks (and across the codebase). The block executor runs node execution in dedicated threads via asyncio.run_coroutine_threadsafe (see manager.py around lines ~745-752 and ~1079). The existing pattern avoids using asyncio.to_thread for SDK calls inside async run() methods, so maintain that approach and do not add to_thread usage in these code paths.
Applied to files:
autogpt_platform/backend/backend/blocks/claude_code.pyautogpt_platform/backend/backend/blocks/claude_code_cost_test.py
📚 Learning: 2026-03-16T16:35:40.236Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/api/features/workflow_import.py:54-63
Timestamp: 2026-03-16T16:35:40.236Z
Learning: Avoid using the word 'competitor' in public-facing identifiers and text. Use neutral naming for API paths, model names, function names, and UI text. Examples: rename 'CompetitorFormat' to 'SourcePlatform', 'convert_competitor_workflow' to 'convert_workflow', '/competitor-workflow' to '/workflow'. Apply this guideline to files under autogpt_platform/backend and autogpt_platform/frontend.
Applied to files:
autogpt_platform/backend/backend/blocks/claude_code.pyautogpt_platform/backend/backend/data/block_cost_config.pyautogpt_platform/backend/backend/blocks/claude_code_cost_test.py
📚 Learning: 2026-03-31T15:37:38.626Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12623
File: autogpt_platform/backend/backend/copilot/tools/agent_generator/fixer.py:37-47
Timestamp: 2026-03-31T15:37:38.626Z
Learning: When validating/constructing Anthropic API model IDs in Significant-Gravitas/AutoGPT, allow the hyphen-separated Claude Opus 4.6 model ID `claude-opus-4-6` (it corresponds to `LlmModel.CLAUDE_4_6_OPUS` in `autogpt_platform/backend/backend/blocks/llm.py`). Do NOT require the dot-separated form in Anthropic contexts. Only OpenRouter routing variants should use the dot separator (e.g., `anthropic/claude-opus-4.6`); `claude-opus-4-6` should be treated as correct when passed to Anthropic, and flagged only if it’s used in the OpenRouter path where the dot form is expected.
Applied to files:
autogpt_platform/backend/backend/blocks/claude_code.pyautogpt_platform/backend/backend/data/block_cost_config.pyautogpt_platform/backend/backend/blocks/claude_code_cost_test.py
📚 Learning: 2026-04-15T02:43:36.890Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12780
File: autogpt_platform/backend/backend/copilot/tools/workspace_files.py:0-0
Timestamp: 2026-04-15T02:43:36.890Z
Learning: When reviewing Python exception handlers, do not flag `isinstance(e, X)` checks as dead/unreachable if the caught exception `X` is a subclass of the exception type being handled. For example, if `X` (e.g., `VirusScanError`) inherits from `ValueError` (directly or via an intermediate class) and it can be raised within an `except ValueError:` block, then `isinstance(e, X)` inside that handler is reachable and should not be treated as dead code.
Applied to files:
autogpt_platform/backend/backend/blocks/claude_code.pyautogpt_platform/backend/backend/data/block_cost_config.pyautogpt_platform/backend/backend/blocks/claude_code_cost_test.py
📚 Learning: 2026-04-22T11:46:04.431Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12881
File: autogpt_platform/backend/backend/copilot/config.py:0-0
Timestamp: 2026-04-22T11:46:04.431Z
Learning: Do not flag the Claude Sonnet 4.6 model ID as incorrect when it uses the project’s established hyphenated convention: `anthropic/claude-sonnet-4-6`. This hyphen form is the intentional, production convention and should be treated as valid (including in files like llm.py, blocks tests, reasoning.py, `_is_anthropic_model` tests, and config defaults). Note that OpenRouter also accepts the dot variant `anthropic/claude-sonnet-4.6`, so either form may be tolerated, but `anthropic/claude-sonnet-4-6` should be considered the standard to match project usage.
Applied to files:
autogpt_platform/backend/backend/blocks/claude_code.pyautogpt_platform/backend/backend/data/block_cost_config.pyautogpt_platform/backend/backend/blocks/claude_code_cost_test.py
📚 Learning: 2026-04-22T11:46:12.892Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12881
File: autogpt_platform/backend/backend/copilot/baseline/service.py:322-332
Timestamp: 2026-04-22T11:46:12.892Z
Learning: In this codebase (Significant-Gravitas/AutoGPT), OpenRouter-routed Anthropic model IDs should use the hyphen-separated convention (e.g., `anthropic/claude-sonnet-4-6`, `anthropic/claude-opus-4-6`). Although OpenRouter may accept both hyphen and dot variants, treat the hyphen-separated form as the intended, correct codebase-wide convention and do not flag it as an error. Only flag the dot-separated variant (e.g., `anthropic/claude-sonnet-4.6`) as incorrect when reviewing/validating model ID strings for OpenRouter-routed Anthropic models.
Applied to files:
autogpt_platform/backend/backend/blocks/claude_code.pyautogpt_platform/backend/backend/data/block_cost_config.pyautogpt_platform/backend/backend/blocks/claude_code_cost_test.py
📚 Learning: 2026-04-23T12:55:26.122Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12893
File: autogpt_platform/backend/backend/blocks/ayrshare/post_to_tiktok.py:24-24
Timestamp: 2026-04-23T12:55:26.122Z
Learning: In `Significant-Gravitas/AutoGPT`, the `cost(*costs)` decorator in `autogpt_platform/backend/backend/sdk/cost_integration.py` evaluates `input_data` **at input-evaluation time, before `run()` executes**. Mutating fields of `input_data` inside a block's `run()` method (e.g., `input_data.is_video = has_video`) has **no effect on billing** because the cost filter has already been applied. For blocks that derive a computed boolean from both an explicit field and URL-sniffing (like `has_video` in `PostToTikTokBlock`), the explicit field (e.g., `is_video`) is the sole billing signal; callers must set it correctly. This applies to all blocks under `autogpt_platform/backend/backend/blocks/`.
Applied to files:
autogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-23T13:53:40.315Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:271-277
Timestamp: 2026-04-23T13:53:40.315Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, `compute_token_credits()` intentionally returns `MODEL_COST[model]` (the flat tier) on pre-flight (when `stats is None`) for the `TOKENS` billing path. Returning 0 pre-flight would allow a zero-balance user to bypass the credit gate and trigger an LLM call, with the insufficient-balance error only surfacing post-flight (a billing leak). The overcharge concern (actual token cost < MODEL_COST estimate) is handled by `_charge_reconciled_usage_sync` in `autogpt_platform/backend/backend/executor/billing.py`, which issues a negative-delta refund via `spend_credits(cost=negative)` when real usage falls below the pre-flight estimate. Do NOT flag the MODEL_COST pre-flight floor in this function as an overcharge bug; the refund path covers it.
Applied to files:
autogpt_platform/backend/backend/data/block_cost_config.pyautogpt_platform/backend/backend/blocks/claude_code_cost_test.py
📚 Learning: 2026-03-27T08:39:45.696Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12592
File: autogpt_platform/frontend/AGENTS.md:1-3
Timestamp: 2026-03-27T08:39:45.696Z
Learning: In Significant-Gravitas/AutoGPT, Claude is the primary coding agent. AGENTS.md files intentionally retain Claude-specific wording (e.g., "CLAUDE.md - Frontend", "This file provides guidance to Claude Code") even though AGENTS.md is the canonical cross-agent instruction source. Do not flag Claude-specific titles or phrasing in AGENTS.md files as issues.
Applied to files:
autogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-23T00:07:27.117Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 0
File: :0-0
Timestamp: 2026-04-23T00:07:27.117Z
Learning: In `autogpt_platform/backend/backend/copilot/sdk/openrouter_cost.py`, the `openrouter-cost-reconcile` Langfuse event carries `cost_source` ("openrouter" for authoritative OpenRouter-resolved cost, "fallback" for rate-card fallback) and `resolved_generation_id_count` alongside the reconciled cost and token/model/provider metadata. This lets operators distinguish authoritative reconciliations from fallbacks in Langfuse. Established in PR `#12889` commit 5ce3d0388.
Applied to files:
autogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-21T04:35:34.710Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12865
File: autogpt_platform/backend/backend/data/credit.py:1584-1584
Timestamp: 2026-04-21T04:35:34.710Z
Learning: When reviewing this codebase, don’t flag snake_case attribute names (e.g., `subscription_tier`, `stripe_customer_id`, `top_up_config`) on the app-layer Pydantic `User` model as “wrong” field names. These are correct for the app-layer model and are expected to be mapped from the Prisma-layer camelCase fields (e.g., `subscriptionTier`, `stripeCustomerId`) inside methods like `User.from_db()`. Only Prisma-returned/raw objects would use camelCase, but functions like `get_user_by_id(user_id: str)` are expected to return the Pydantic app-layer model.
Applied to files:
autogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-02-04T16:49:42.490Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2026-02-04T16:49:42.490Z
Learning: Applies to autogpt_platform/backend/backend/blocks/**/*.py : Write tests alongside block implementation when adding new blocks in backend
Applied to files:
autogpt_platform/backend/backend/blocks/claude_code_cost_test.py
🔇 Additional comments (2)
autogpt_platform/backend/backend/data/block_cost_config.py (1)
1232-1250: LGTM — COST_USD migration is internally consistent.
- Matches the
_coerce_usdcontract (provider_cost_type="cost_usd"set by the block atclaude_code.py:444).block_usage_costCOST_USDbranch doesmax(0, ceil(usd * 150)), so sub-cent runs (e.g.$0.001 → 1 cr) still bill, matching the comment inTOKEN_COSTabout the 1.5× margin convention.cost_filterkeye2b_credentialscorrectly mirrors theInput.e2b_credentialsfield name; pre-flight returns 0, with the zero-balance guard preventing free execution on drained wallets (per established pattern).autogpt_platform/backend/backend/blocks/claude_code.py (1)
436-446: Parsing + merge_stats contract looks correct.
provider_cost+provider_cost_type="cost_usd"aligns withresolve_tracking(block-declared path takes precedence) and_coerce_usd(tag-gated to avoid billing non-USD amounts). Gracefully skipped onjson.JSONDecodeErrorand when the key is absent, so legacy/raw outputs emit no cost telemetry — matches the PR objective.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## dev #12909 +/- ##
==========================================
+ Coverage 68.12% 68.15% +0.03%
==========================================
Files 1934 1938 +4
Lines 149285 149586 +301
Branches 15558 15573 +15
==========================================
+ Hits 101698 101955 +257
- Misses 44564 44605 +41
- Partials 3023 3026 +3
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
Actionable comments posted: 4
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
autogpt_platform/backend/backend/blocks/exa/websets_enrichment.py (1)
227-231:⚠️ Potential issue | 🟠 MajorIn-loop
merge_exa_coston enrichment status poll — same double-count risk aswebsets_polling.py.
merge_exa_cost(self, current_enrich)at Line 231 runs on every poll iteration until the enrichment reaches a terminal status. If the Exa SDK ever surfacescost_dollarson enrichment status fetches (today it likely doesn't, making this a no-op), each poll would re-merge cumulative cost and overbill.Move the merge into the terminal branch below (alongside the existing webset merge at Line 241), or rely solely on the final
sdk_enrichment/current_enrichobject for cost attribution.♻️ Suggested fix
while time.time() - poll_start < input_data.polling_timeout: current_enrich = await aexa.websets.enrichments.get( webset_id=input_data.webset_id, id=enrichment_id ) - merge_exa_cost(self, current_enrich) current_status = ( current_enrich.status.value if hasattr(current_enrich.status, "value") else str(current_enrich.status) ) if current_status in ["completed", "failed", "cancelled"]: + merge_exa_cost(self, current_enrich) # Estimate items from webset searches webset = await aexa.websets.get(id=input_data.webset_id) merge_exa_cost(self, webset)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/blocks/exa/websets_enrichment.py` around lines 227 - 231, The call to merge_exa_cost inside the polling loop causes potential duplicate cost attribution; move the merge_exa_cost(self, current_enrich) call out of the while loop and into the terminal-status branch where you already merge the webset (i.e., after the enrichment reaches a final state using sdk_enrichment/current_enrich), or only call merge_exa_cost once using the final sdk_enrichment/current_enrich object; locate the loop that polls aexa.websets.enrichments.get (uses input_data.polling_timeout and enrichment_id) and remove the in-loop merge, invoking merge_exa_cost in the same place you handle terminal enrichment results (matching the existing webset merge logic).
🧹 Nitpick comments (4)
autogpt_platform/backend/backend/blocks/exa/websets_polling.py (1)
167-170: Remove in-loopmerge_exa_costcalls; status responses don't expose cost data.Lines 170, 355, and 515 call
merge_exa_coston status-poll responses (aexa.websets.get,aexa.websets.searches.get,aexa.websets.enrichments.get). These status endpoints do not surfacecost_dollars(as noted in websets_import_export.py:607 and websets_items.py:411), making these calls safe no-ops. However, they're unnecessary and inconsistent with the pattern elsewhere in the file (lines 130, 216, 412, 565, 591 merge only on terminal/final states). Drop the in-loop merges and callmerge_exa_costonly after the loop exits on the final status object, matching the existing defensive pattern.♻️ Example fix for `ExaWaitForSearchBlock`
while time.time() - start_time < input_data.timeout: # Get current search status using SDK search = await aexa.websets.searches.get( webset_id=input_data.webset_id, id=input_data.search_id ) - merge_exa_cost(self, search) # Extract status status = ( search.status.value if hasattr(search.status, "value") else str(search.status) ) # Check if search is complete if status in ["completed", "failed", "canceled"]: + merge_exa_cost(self, search) elapsed = time.time() - start_timeAlso applies to: 350–355, 510–515
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/blocks/exa/websets_polling.py` around lines 167 - 170, Remove the in-loop merge_exa_cost calls when polling status responses (the await calls to aexa.websets.get, aexa.websets.searches.get, and aexa.websets.enrichments.get) because these endpoints don't provide cost_dollars; instead, only call merge_exa_cost once after the polling loop exits using the final status object. Locate the polling loops (e.g., in ExaWaitForSearchBlock and the equivalent webset/enrichment polling blocks), delete the merge_exa_cost(self, webset/search/enrichment) invocation inside the while timeout loop, and add a single merge_exa_cost(self, final_status) immediately after the loop where you handle the terminal/final state.autogpt_platform/backend/backend/data/block_cost_config.py (1)
1276-1283:PineconeInsertBlockITEMS entry has nocost_filter— intentional but worth noting.Without a
cost_filter, this entry always matches regardless of credential configuration. That's consistent with the inline comment ("user brings their own Pinecone API key") since Pinecone is the only provider the block targets, but it departs from the pattern used by every other block in this file where acredentialsfilter pins the cost to a specific credential id. If a future change ever adds a second storage backend toPineconeInsertBlock, this entry would silently bill that path too.Also, since ITEMS pre-flight returns 0 (unknown count), balance-guard gating on zero-balance wallets relies on the post-flight charge path working end-to-end — same model as COST_USD, so this should be fine, but worth a mental check.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/data/block_cost_config.py` around lines 1276 - 1283, PineconeInsertBlock's BlockCost entry for BlockCostType.ITEMS is missing a cost_filter so it will match all credential configurations; update the PineconeInsertBlock BlockCost for ITEMS to include a cost_filter that pins the cost to Pinecone credentials only (e.g., match credential id or provider == "pinecone"), so the ITEMS charge only applies when the Pinecone credential is present; reference the PineconeInsertBlock BlockCost and BlockCostType.ITEMS when adding the cost_filter and preserve the existing comment about user-provided Pinecone API keys.autogpt_platform/backend/backend/blocks/codex.py (1)
197-201: Hardcoded per-token rates will silently mis-bill when more Codex models are added.The
$1.25/$10per-1M rates are correct forCodexModel.GPT5_1_CODEXtoday (the only enum value), but the USD calculation uses the same rates for any futureCodexModelvalue. When a cheaper variant (e.g., a Codex-mini) is added to the enum andBLOCK_COSTS[CodeGenerationBlock], users would be billed at Codex-pro rates.Consider keying rates by model from the start, even though only one exists now:
♻️ Suggested refactor — model-keyed rate table
+# USD per 1M tokens, per Codex model. Keep in sync with BLOCK_COSTS filter. +_CODEX_RATES: dict[CodexModel, tuple[float, float]] = { + CodexModel.GPT5_1_CODEX: (1.25, 10.0), +} ... - # GPT-5.1-Codex: $1.25/1M input + $10/1M output. Compute USD and - # feed COST_USD resolver so billing scales with real token usage. - usd = (input_tokens * 1.25 + output_tokens * 10.0) / 1_000_000 + input_rate, output_rate = _CODEX_RATES[model] + usd = (input_tokens * input_rate + output_tokens * output_rate) / 1_000_000 self.execution_stats.provider_cost = usd self.execution_stats.provider_cost_type = "cost_usd"🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/blocks/codex.py` around lines 197 - 201, The USD cost computation is using hardcoded per-token rates (1.25/10) which will mis-bill if new CodexModel values are added; replace the literal rates with a model-keyed rate table (e.g., a dict mapping CodexModel values like CodexModel.GPT5_1_CODEX to (input_rate_per_1M, output_rate_per_1M)), look up the current model (the attribute used in this block/class) to pick rates with a sensible default fallback, compute usd = (input_tokens * input_rate + output_tokens * output_rate) / 1_000_000, and then assign self.execution_stats.provider_cost and self.execution_stats.provider_cost_type exactly as before; update any references to BLOCK_COSTS[CodeGenerationBlock] or related cost configuration to use or document the new model-keyed table so future variants (e.g., Codex-mini) are billed correctly.autogpt_platform/backend/backend/blocks/email_block.py (1)
138-239: Yield reformatting is a no-op and unrelated to the PR's COST_USD migration.These
yield ("error", ("<message>"))changes are semantically identical toyield "error", "<message>"(parentheses around a single string don't create a tuple). The edits are cosmetic and don't touch billing, so they're harmless, but they do enlarge the PR diff beyond the cost-migration scope stated in the description — worth splitting into a separate formatting-only PR next time to keep billing reviews focused.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/blocks/email_block.py` around lines 138 - 239, The PR includes cosmetic changes converting yields like yield "error", "<msg>" into yield ("error", ("<msg>")), which is a no-op and unrelated to the COST_USD migration; revert those formatting-only changes in email_block.py (look for occurrences around ALLOWED_SMTP_PORTS, resolve_and_check_blocked, send_email, and the exception handlers referencing input_data.config.smtp_server/ smtp_port) back to the original simple form yield "error", "<message>" and remove any extra parentheses so the diff only contains migration-related changes.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@autogpt_platform/backend/backend/blocks/baas/bots.py`:
- Around line 190-207: Replace the three-way fallback lookup with a direct read
of the authoritative field by using bot_meta.get("duration_seconds") for
duration_seconds, and lift the magic rate 0.000192 into a clearly named
module-level constant (e.g., RECORDING_USD_PER_SECOND) so the provider_cost
calculation uses float(duration_seconds) * RECORDING_USD_PER_SECOND; update the
code that calls merge_stats with NodeExecutionStats(provider_cost=...,
provider_cost_type="cost_usd") to use the new constant and remove the unused
fallbacks.
In `@autogpt_platform/backend/backend/blocks/exa/websets_enrichment.py`:
- Around line 430-434: The call to merge_exa_cost(self, data) after the PATCH
request is dead because the /websets/{id}/enrichments/{id} PATCH response never
contains costDollars; remove the merge_exa_cost(self, data) invocation from the
try block where response = await Requests().patch(url, headers=headers,
json=payload) and data = response.json(). Also check for and remove any
now-unused import or reference to merge_exa_cost in this module to keep the code
clean.
In `@autogpt_platform/backend/backend/blocks/perplexity.py`:
- Around line 260-263: The code unconditionally sets
self.execution_stats.provider_cost_type = "cost_usd" even when
extract_openrouter_cost(response) returns None, causing silent under-billing;
change the logic in the PerplexityBlock handling so that after calling
extract_openrouter_cost(response) you check the returned value and only set
self.execution_stats.provider_cost_type = "cost_usd" when
self.execution_stats.provider_cost is not None (a concrete numeric value);
otherwise leave provider_cost_type unset (or None) so the block follows the
unmapped-cost path and exposes missing cost telemetry.
In `@autogpt_platform/backend/backend/blocks/video/narration.py`:
- Around line 227-236: The current hardcoded per-char cost (script_usd =
len(input_data.script) * 0.000167) can over-charge model variants; update the
cost calculation in the narration flow that calls merge_stats/NodeExecutionStats
to be model-aware by branching on input_data.model_id (e.g., map Flash/Turbo
v2.5 -> 0.5 credits/char rate, Multilingual/Turbo v2 -> 1.0 credits/char rate
and convert credits->USD) and prefer the SDK raw-response path (use
with_raw_response instead of text_to_speech.convert()) to read the accurate
x-character-count header for the true billed character count instead of
len(input_data.script), then compute provider_cost using that header value and
include it in NodeExecutionStats provider_cost/provider_cost_type before calling
merge_stats.
---
Outside diff comments:
In `@autogpt_platform/backend/backend/blocks/exa/websets_enrichment.py`:
- Around line 227-231: The call to merge_exa_cost inside the polling loop causes
potential duplicate cost attribution; move the merge_exa_cost(self,
current_enrich) call out of the while loop and into the terminal-status branch
where you already merge the webset (i.e., after the enrichment reaches a final
state using sdk_enrichment/current_enrich), or only call merge_exa_cost once
using the final sdk_enrichment/current_enrich object; locate the loop that polls
aexa.websets.enrichments.get (uses input_data.polling_timeout and enrichment_id)
and remove the in-loop merge, invoking merge_exa_cost in the same place you
handle terminal enrichment results (matching the existing webset merge logic).
---
Nitpick comments:
In `@autogpt_platform/backend/backend/blocks/codex.py`:
- Around line 197-201: The USD cost computation is using hardcoded per-token
rates (1.25/10) which will mis-bill if new CodexModel values are added; replace
the literal rates with a model-keyed rate table (e.g., a dict mapping CodexModel
values like CodexModel.GPT5_1_CODEX to (input_rate_per_1M, output_rate_per_1M)),
look up the current model (the attribute used in this block/class) to pick rates
with a sensible default fallback, compute usd = (input_tokens * input_rate +
output_tokens * output_rate) / 1_000_000, and then assign
self.execution_stats.provider_cost and self.execution_stats.provider_cost_type
exactly as before; update any references to BLOCK_COSTS[CodeGenerationBlock] or
related cost configuration to use or document the new model-keyed table so
future variants (e.g., Codex-mini) are billed correctly.
In `@autogpt_platform/backend/backend/blocks/email_block.py`:
- Around line 138-239: The PR includes cosmetic changes converting yields like
yield "error", "<msg>" into yield ("error", ("<msg>")), which is a no-op and
unrelated to the COST_USD migration; revert those formatting-only changes in
email_block.py (look for occurrences around ALLOWED_SMTP_PORTS,
resolve_and_check_blocked, send_email, and the exception handlers referencing
input_data.config.smtp_server/ smtp_port) back to the original simple form yield
"error", "<message>" and remove any extra parentheses so the diff only contains
migration-related changes.
In `@autogpt_platform/backend/backend/blocks/exa/websets_polling.py`:
- Around line 167-170: Remove the in-loop merge_exa_cost calls when polling
status responses (the await calls to aexa.websets.get,
aexa.websets.searches.get, and aexa.websets.enrichments.get) because these
endpoints don't provide cost_dollars; instead, only call merge_exa_cost once
after the polling loop exits using the final status object. Locate the polling
loops (e.g., in ExaWaitForSearchBlock and the equivalent webset/enrichment
polling blocks), delete the merge_exa_cost(self, webset/search/enrichment)
invocation inside the while timeout loop, and add a single merge_exa_cost(self,
final_status) immediately after the loop where you handle the terminal/final
state.
In `@autogpt_platform/backend/backend/data/block_cost_config.py`:
- Around line 1276-1283: PineconeInsertBlock's BlockCost entry for
BlockCostType.ITEMS is missing a cost_filter so it will match all credential
configurations; update the PineconeInsertBlock BlockCost for ITEMS to include a
cost_filter that pins the cost to Pinecone credentials only (e.g., match
credential id or provider == "pinecone"), so the ITEMS charge only applies when
the Pinecone credential is present; reference the PineconeInsertBlock BlockCost
and BlockCostType.ITEMS when adding the cost_filter and preserve the existing
comment about user-provided Pinecone API keys.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 9e51518e-ef0f-4753-9ac4-75318f51a6c5
📒 Files selected for processing (76)
autogpt_platform/backend/backend/blocks/ai_music_generator.pyautogpt_platform/backend/backend/blocks/airtable/_api.pyautogpt_platform/backend/backend/blocks/airtable/_api_test.pyautogpt_platform/backend/backend/blocks/autopilot_permissions_test.pyautogpt_platform/backend/backend/blocks/ayrshare/post_to_bluesky.pyautogpt_platform/backend/backend/blocks/ayrshare/post_to_facebook.pyautogpt_platform/backend/backend/blocks/ayrshare/post_to_gmb.pyautogpt_platform/backend/backend/blocks/ayrshare/post_to_instagram.pyautogpt_platform/backend/backend/blocks/ayrshare/post_to_linkedin.pyautogpt_platform/backend/backend/blocks/ayrshare/post_to_pinterest.pyautogpt_platform/backend/backend/blocks/ayrshare/post_to_reddit.pyautogpt_platform/backend/backend/blocks/ayrshare/post_to_snapchat.pyautogpt_platform/backend/backend/blocks/ayrshare/post_to_telegram.pyautogpt_platform/backend/backend/blocks/ayrshare/post_to_threads.pyautogpt_platform/backend/backend/blocks/ayrshare/post_to_tiktok.pyautogpt_platform/backend/backend/blocks/ayrshare/post_to_x.pyautogpt_platform/backend/backend/blocks/ayrshare/post_to_youtube.pyautogpt_platform/backend/backend/blocks/baas/bots.pyautogpt_platform/backend/backend/blocks/basic.pyautogpt_platform/backend/backend/blocks/codex.pyautogpt_platform/backend/backend/blocks/email_block.pyautogpt_platform/backend/backend/blocks/enrichlayer/linkedin.pyautogpt_platform/backend/backend/blocks/exa/answers.pyautogpt_platform/backend/backend/blocks/exa/code_context.pyautogpt_platform/backend/backend/blocks/exa/contents.pyautogpt_platform/backend/backend/blocks/exa/cost_tracking_test.pyautogpt_platform/backend/backend/blocks/exa/helpers.pyautogpt_platform/backend/backend/blocks/exa/research.pyautogpt_platform/backend/backend/blocks/exa/search.pyautogpt_platform/backend/backend/blocks/exa/similar.pyautogpt_platform/backend/backend/blocks/exa/websets.pyautogpt_platform/backend/backend/blocks/exa/websets_enrichment.pyautogpt_platform/backend/backend/blocks/exa/websets_import_export.pyautogpt_platform/backend/backend/blocks/exa/websets_items.pyautogpt_platform/backend/backend/blocks/exa/websets_monitor.pyautogpt_platform/backend/backend/blocks/exa/websets_polling.pyautogpt_platform/backend/backend/blocks/exa/websets_search.pyautogpt_platform/backend/backend/blocks/github/issues.pyautogpt_platform/backend/backend/blocks/github/pull_requests.pyautogpt_platform/backend/backend/blocks/google/docs.pyautogpt_platform/backend/backend/blocks/google/gmail.pyautogpt_platform/backend/backend/blocks/google/sheets.pyautogpt_platform/backend/backend/blocks/ideogram.pyautogpt_platform/backend/backend/blocks/io.pyautogpt_platform/backend/backend/blocks/linear/_api.pyautogpt_platform/backend/backend/blocks/mcp/block.pyautogpt_platform/backend/backend/blocks/mcp/test_mcp.pyautogpt_platform/backend/backend/blocks/perplexity.pyautogpt_platform/backend/backend/blocks/persistence.pyautogpt_platform/backend/backend/blocks/pinecone.pyautogpt_platform/backend/backend/blocks/reddit.pyautogpt_platform/backend/backend/blocks/replicate/flux_advanced.pyautogpt_platform/backend/backend/blocks/slant3d/base.pyautogpt_platform/backend/backend/blocks/smartlead/campaign.pyautogpt_platform/backend/backend/blocks/sql_query_block_test.pyautogpt_platform/backend/backend/blocks/stagehand/blocks.pyautogpt_platform/backend/backend/blocks/telegram/blocks.pyautogpt_platform/backend/backend/blocks/telegram/triggers.pyautogpt_platform/backend/backend/blocks/test/test_block.pyautogpt_platform/backend/backend/blocks/test/test_llm.pyautogpt_platform/backend/backend/blocks/test/test_orchestrator.pyautogpt_platform/backend/backend/blocks/test/test_orchestrator_dict.pyautogpt_platform/backend/backend/blocks/test/test_orchestrator_dynamic_fields.pyautogpt_platform/backend/backend/blocks/test/test_orchestrator_responses_api.pyautogpt_platform/backend/backend/blocks/test/test_orchestrator_tool_dedup.pyautogpt_platform/backend/backend/blocks/text.pyautogpt_platform/backend/backend/blocks/time_blocks.pyautogpt_platform/backend/backend/blocks/todoist/projects.pyautogpt_platform/backend/backend/blocks/todoist/sections.pyautogpt_platform/backend/backend/blocks/todoist/tasks.pyautogpt_platform/backend/backend/blocks/twitter/_types.pyautogpt_platform/backend/backend/blocks/twitter/lists/manage_lists.pyautogpt_platform/backend/backend/blocks/twitter/tweets/tweet_lookup.pyautogpt_platform/backend/backend/blocks/video/narration.pyautogpt_platform/backend/backend/blocks/youtube.pyautogpt_platform/backend/backend/data/block_cost_config.py
💤 Files with no reviewable changes (3)
- autogpt_platform/backend/backend/blocks/linear/_api.py
- autogpt_platform/backend/backend/blocks/twitter/_types.py
- autogpt_platform/backend/backend/blocks/twitter/tweets/tweet_lookup.py
✅ Files skipped from review due to trivial changes (48)
- autogpt_platform/backend/backend/blocks/ayrshare/post_to_telegram.py
- autogpt_platform/backend/backend/blocks/enrichlayer/linkedin.py
- autogpt_platform/backend/backend/blocks/replicate/flux_advanced.py
- autogpt_platform/backend/backend/blocks/ayrshare/post_to_facebook.py
- autogpt_platform/backend/backend/blocks/github/issues.py
- autogpt_platform/backend/backend/blocks/basic.py
- autogpt_platform/backend/backend/blocks/ai_music_generator.py
- autogpt_platform/backend/backend/blocks/autopilot_permissions_test.py
- autogpt_platform/backend/backend/blocks/stagehand/blocks.py
- autogpt_platform/backend/backend/blocks/youtube.py
- autogpt_platform/backend/backend/blocks/ayrshare/post_to_snapchat.py
- autogpt_platform/backend/backend/blocks/telegram/triggers.py
- autogpt_platform/backend/backend/blocks/text.py
- autogpt_platform/backend/backend/blocks/ayrshare/post_to_reddit.py
- autogpt_platform/backend/backend/blocks/slant3d/base.py
- autogpt_platform/backend/backend/blocks/telegram/blocks.py
- autogpt_platform/backend/backend/blocks/ideogram.py
- autogpt_platform/backend/backend/blocks/test/test_orchestrator_dynamic_fields.py
- autogpt_platform/backend/backend/blocks/todoist/tasks.py
- autogpt_platform/backend/backend/blocks/ayrshare/post_to_bluesky.py
- autogpt_platform/backend/backend/blocks/sql_query_block_test.py
- autogpt_platform/backend/backend/blocks/io.py
- autogpt_platform/backend/backend/blocks/test/test_orchestrator_tool_dedup.py
- autogpt_platform/backend/backend/blocks/mcp/test_mcp.py
- autogpt_platform/backend/backend/blocks/persistence.py
- autogpt_platform/backend/backend/blocks/github/pull_requests.py
- autogpt_platform/backend/backend/blocks/ayrshare/post_to_linkedin.py
- autogpt_platform/backend/backend/blocks/todoist/sections.py
- autogpt_platform/backend/backend/blocks/ayrshare/post_to_gmb.py
- autogpt_platform/backend/backend/blocks/ayrshare/post_to_threads.py
- autogpt_platform/backend/backend/blocks/ayrshare/post_to_pinterest.py
- autogpt_platform/backend/backend/blocks/test/test_orchestrator_dict.py
- autogpt_platform/backend/backend/blocks/reddit.py
- autogpt_platform/backend/backend/blocks/ayrshare/post_to_x.py
- autogpt_platform/backend/backend/blocks/ayrshare/post_to_youtube.py
- autogpt_platform/backend/backend/blocks/test/test_block.py
- autogpt_platform/backend/backend/blocks/exa/cost_tracking_test.py
- autogpt_platform/backend/backend/blocks/ayrshare/post_to_instagram.py
- autogpt_platform/backend/backend/blocks/google/docs.py
- autogpt_platform/backend/backend/blocks/airtable/_api.py
- autogpt_platform/backend/backend/blocks/ayrshare/post_to_tiktok.py
- autogpt_platform/backend/backend/blocks/airtable/_api_test.py
- autogpt_platform/backend/backend/blocks/test/test_orchestrator_responses_api.py
- autogpt_platform/backend/backend/blocks/test/test_llm.py
- autogpt_platform/backend/backend/blocks/test/test_orchestrator.py
- autogpt_platform/backend/backend/blocks/time_blocks.py
- autogpt_platform/backend/backend/blocks/google/gmail.py
- autogpt_platform/backend/backend/blocks/google/sheets.py
majdyz
changed the title
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
autogpt_platform/backend/backend/blocks/video/narration.py (1)
227-236: Consider lifting rate constants to module level.For consistency with the
_MEETING_BAAS_USD_PER_SECONDpattern introduced elsewhere in this PR, the magic numbers (0.000167, the Flash/Turbo-v2.5 half-rate set, and the 0.5/1.0 credit-per-char factors) would read more clearly as named module-level constants. This also makes future rate adjustments a one-liner.♻️ Proposed refactor
+# ElevenLabs Starter: $5 per 30K credits. +_ELEVENLABS_USD_PER_CREDIT = 5 / 30_000 # ≈ $0.000167 +# Models that consume 0.5 credits per character (half the base rate). +_ELEVENLABS_HALF_RATE_MODELS = frozenset({"eleven_flash_v2_5", "eleven_turbo_v2_5"}) + + class VideoNarrationBlock(Block): @@ - # ElevenLabs billing: Starter $5 / 30K credits = $0.000167/credit. - # Flash/Turbo v2.5 consume 0.5 credits/char; Multilingual v2 and - # Turbo v2 consume 1.0 credit/char — scale the rate accordingly - # so we don't 2x over-bill cheap-tier users. - credits_per_char = ( - 0.5 - if input_data.model_id in {"eleven_flash_v2_5", "eleven_turbo_v2_5"} - else 1.0 - ) - script_usd = len(input_data.script) * 0.000167 * credits_per_char + credits_per_char = ( + 0.5 if input_data.model_id in _ELEVENLABS_HALF_RATE_MODELS else 1.0 + ) + script_usd = ( + len(input_data.script) * _ELEVENLABS_USD_PER_CREDIT * credits_per_char + )🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/blocks/video/narration.py` around lines 227 - 236, Lift the magic numbers in the ElevenLabs pricing calculation into module-level constants: replace the literal 0.000167, the set {"eleven_flash_v2_5", "eleven_turbo_v2_5"}, and the 0.5/1.0 factors used in credits_per_char with descriptive constants (e.g., ELEVEN_USD_PER_CREDIT, ELEVEN_HALF_RATE_MODELS, ELEVEN_HALF_CREDITS_PER_CHAR, ELEVEN_FULL_CREDITS_PER_CHAR) and update the calculation in the block that computes credits_per_char and script_usd (which uses input_data.model_id and len(input_data.script)) to reference those constants for clarity and easier future adjustments.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@autogpt_platform/backend/backend/blocks/video/narration.py`:
- Line 236: The current cost calculation uses len(input_data.script); update
_generate_narration_audio to call
client.text_to_speech.with_raw_response.convert(...) so you can read the
ElevenLabs response headers and use the x-character-count header to compute
script_usd (instead of len(...)); extract the header value, parse it to an
integer, multiply by 0.000167 * credits_per_char, and fall back to
len(input_data.script) only if the header is missing or not parseable to
preserve behavior.
---
Nitpick comments:
In `@autogpt_platform/backend/backend/blocks/video/narration.py`:
- Around line 227-236: Lift the magic numbers in the ElevenLabs pricing
calculation into module-level constants: replace the literal 0.000167, the set
{"eleven_flash_v2_5", "eleven_turbo_v2_5"}, and the 0.5/1.0 factors used in
credits_per_char with descriptive constants (e.g., ELEVEN_USD_PER_CREDIT,
ELEVEN_HALF_RATE_MODELS, ELEVEN_HALF_CREDITS_PER_CHAR,
ELEVEN_FULL_CREDITS_PER_CHAR) and update the calculation in the block that
computes credits_per_char and script_usd (which uses input_data.model_id and
len(input_data.script)) to reference those constants for clarity and easier
future adjustments.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 3b31d169-2b8e-4dc5-aaaf-de4fd675840a
📒 Files selected for processing (40)
autogpt_platform/backend/backend/api/features/admin/store_admin_routes_test.pyautogpt_platform/backend/backend/api/features/integrations/router.pyautogpt_platform/backend/backend/api/features/postmark/postmark.pyautogpt_platform/backend/backend/api/features/store/content_handlers.pyautogpt_platform/backend/backend/api/features/store/media_test.pyautogpt_platform/backend/backend/api/features/v1_test.pyautogpt_platform/backend/backend/api/utils/api_key_auth_test.pyautogpt_platform/backend/backend/api/ws_api_test.pyautogpt_platform/backend/backend/blocks/baas/bots.pyautogpt_platform/backend/backend/blocks/exa/websets.pyautogpt_platform/backend/backend/blocks/exa/websets_enrichment.pyautogpt_platform/backend/backend/blocks/perplexity.pyautogpt_platform/backend/backend/blocks/test/test_orchestrator.pyautogpt_platform/backend/backend/blocks/test/test_orchestrator_dynamic_fields.pyautogpt_platform/backend/backend/blocks/test/test_orchestrator_responses_api.pyautogpt_platform/backend/backend/blocks/video/narration.pyautogpt_platform/backend/backend/check_db.pyautogpt_platform/backend/backend/cli/oauth_tool.pyautogpt_platform/backend/backend/copilot/executor/processor_test.pyautogpt_platform/backend/backend/copilot/pending_messages.pyautogpt_platform/backend/backend/copilot/sdk/sdk_compat_test.pyautogpt_platform/backend/backend/copilot/sdk/tool_adapter_test.pyautogpt_platform/backend/backend/data/credit_concurrency_test.pyautogpt_platform/backend/backend/data/execution.pyautogpt_platform/backend/backend/data/redis_helpers_test.pyautogpt_platform/backend/backend/data/workspace.pyautogpt_platform/backend/backend/executor/activity_status_generator_test.pyautogpt_platform/backend/backend/executor/automod/manager.pyautogpt_platform/backend/backend/executor/manager_insufficient_funds_test.pyautogpt_platform/backend/backend/executor/manager_low_balance_test.pyautogpt_platform/backend/backend/executor/manager_test.pyautogpt_platform/backend/backend/integrations/managed_providers/agentmail.pyautogpt_platform/backend/backend/notifications/notifications.pyautogpt_platform/backend/backend/notifications/test_notifications.pyautogpt_platform/backend/backend/sdk/__init__.pyautogpt_platform/backend/backend/sdk/registry.pyautogpt_platform/backend/backend/util/dynamic_fields.pyautogpt_platform/backend/backend/util/file_test.pyautogpt_platform/backend/backend/util/service_test.pyautogpt_platform/backend/backend/util/test_json.py
💤 Files with no reviewable changes (3)
- autogpt_platform/backend/backend/util/service_test.py
- autogpt_platform/backend/backend/executor/automod/manager.py
- autogpt_platform/backend/backend/sdk/registry.py
✅ Files skipped from review due to trivial changes (31)
- autogpt_platform/backend/backend/sdk/init.py
- autogpt_platform/backend/backend/util/dynamic_fields.py
- autogpt_platform/backend/backend/executor/manager_test.py
- autogpt_platform/backend/backend/copilot/pending_messages.py
- autogpt_platform/backend/backend/integrations/managed_providers/agentmail.py
- autogpt_platform/backend/backend/api/features/postmark/postmark.py
- autogpt_platform/backend/backend/api/features/integrations/router.py
- autogpt_platform/backend/backend/blocks/test/test_orchestrator_dynamic_fields.py
- autogpt_platform/backend/backend/copilot/sdk/sdk_compat_test.py
- autogpt_platform/backend/backend/data/credit_concurrency_test.py
- autogpt_platform/backend/backend/notifications/notifications.py
- autogpt_platform/backend/backend/copilot/sdk/tool_adapter_test.py
- autogpt_platform/backend/backend/blocks/test/test_orchestrator_responses_api.py
- autogpt_platform/backend/backend/check_db.py
- autogpt_platform/backend/backend/api/utils/api_key_auth_test.py
- autogpt_platform/backend/backend/api/features/store/content_handlers.py
- autogpt_platform/backend/backend/executor/manager_low_balance_test.py
- autogpt_platform/backend/backend/api/ws_api_test.py
- autogpt_platform/backend/backend/copilot/executor/processor_test.py
- autogpt_platform/backend/backend/data/execution.py
- autogpt_platform/backend/backend/api/features/admin/store_admin_routes_test.py
- autogpt_platform/backend/backend/notifications/test_notifications.py
- autogpt_platform/backend/backend/api/features/v1_test.py
- autogpt_platform/backend/backend/cli/oauth_tool.py
- autogpt_platform/backend/backend/data/redis_helpers_test.py
- autogpt_platform/backend/backend/api/features/store/media_test.py
- autogpt_platform/backend/backend/data/workspace.py
- autogpt_platform/backend/backend/executor/activity_status_generator_test.py
- autogpt_platform/backend/backend/util/file_test.py
- autogpt_platform/backend/backend/executor/manager_insufficient_funds_test.py
- autogpt_platform/backend/backend/util/test_json.py
🚧 Files skipped from review as they are similar to previous changes (3)
- autogpt_platform/backend/backend/blocks/perplexity.py
- autogpt_platform/backend/backend/blocks/exa/websets.py
- autogpt_platform/backend/backend/blocks/test/test_orchestrator.py
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (11)
- GitHub Check: check API types
- GitHub Check: Seer Code Review
- GitHub Check: Check PR Status
- GitHub Check: end-to-end tests
- GitHub Check: test (3.13)
- GitHub Check: type-check (3.13)
- GitHub Check: test (3.12)
- GitHub Check: test (3.11)
- GitHub Check: check-overlaps
- GitHub Check: Analyze (typescript)
- GitHub Check: Analyze (python)
🧰 Additional context used
📓 Path-based instructions (3)
autogpt_platform/backend/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
autogpt_platform/backend/**/*.py: Use Python 3.11 (required; managed by Poetry via pyproject.toml) for backend development
Always run 'poetry run format' (Black + isort) before linting in backend development
Always run 'poetry run lint' (ruff) after formatting in backend development
autogpt_platform/backend/**/*.py: Usepoetry run ...command for executing Python package dependencies
Use top-level imports only — avoid local/inner imports except for lazy imports of heavy optional dependencies likeopenpyxl
Use absolute imports withfrom backend.module import ...for cross-package imports; single-dot relative imports are acceptable for sibling modules within the same package; avoid double-dot relative imports
Do not use duck typing — avoidhasattr/getattr/isinstancefor type dispatch; use typed interfaces/unions/protocols instead
Use Pydantic models over dataclass/namedtuple/dict for structured data
Do not use linter suppressors — no# type: ignore,# noqa,# pyright: ignore; fix the type/code instead
Prefer list comprehensions over manual loop-and-append patterns
Use early return with guard clauses first to avoid deep nesting
Use%sfor deferred interpolation indebuglog statements for efficiency; use f-strings elsewhere for readability (e.g.,logger.debug("Processing %s items", count)vslogger.info(f"Processing {count} items"))
Sanitize error paths by usingos.path.basename()in error messages to avoid leaking directory structure
Be aware of TOCTOU (Time-Of-Check-Time-Of-Use) issues — avoid check-then-act patterns for file access and credit charging
Usetransaction=Truefor Redis pipelines to ensure atomicity on multi-step operations
Usemax(0, value)guards for computed values that should never be negative
Keep files under ~300 lines; if a file grows beyond this, split by responsibility (extract helpers, models, or a sub-module into a new file)
Keep functions under ~40 lines; extract named helpers when a function grows longer
...
Files:
autogpt_platform/backend/backend/blocks/video/narration.pyautogpt_platform/backend/backend/blocks/exa/websets_enrichment.pyautogpt_platform/backend/backend/blocks/baas/bots.py
autogpt_platform/backend/backend/blocks/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
autogpt_platform/backend/backend/blocks/**/*.py: Inherit from 'Block' base class with input/output schemas when adding new blocks in backend
Implement 'run' method with proper error handling in backend blocks
Generate block UUID using 'uuid.uuid4()' when creating new blocks in backend
Write tests alongside block implementation when adding new blocks in backend
autogpt_platform/backend/backend/blocks/**/*.py: For blocks handling files, usestore_media_file()withreturn_format="for_local_processing"when processing with local tools (ffmpeg, MoviePy, PIL)
For blocks handling files, usestore_media_file()withreturn_format="for_external_api"when sending content to external APIs (Replicate, OpenAI)
For blocks returning files, usestore_media_file()withreturn_format="for_block_output"to enable auto-adaptation to execution context (workspace:// in CoPilot, data URI in graphs)
When creating new blocks, inherit fromBlockbase class, define input/output schemas usingBlockSchema, implement asyncrunmethod, and generate unique block ID usinguuid.uuid4()
Files:
autogpt_platform/backend/backend/blocks/video/narration.pyautogpt_platform/backend/backend/blocks/exa/websets_enrichment.pyautogpt_platform/backend/backend/blocks/baas/bots.py
autogpt_platform/{backend,autogpt_libs}/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
Format Python code with
poetry run format
Files:
autogpt_platform/backend/backend/blocks/video/narration.pyautogpt_platform/backend/backend/blocks/exa/websets_enrichment.pyautogpt_platform/backend/backend/blocks/baas/bots.py
🧠 Learnings (20)
📓 Common learnings
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:1064-1111
Timestamp: 2026-04-23T13:53:29.246Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, the E2B sandbox blocks (`ExecuteCodeBlock`, `InstantiateCodeSandboxBlock`, `ExecuteCodeStepBlock`) are intentionally billed per-node-execution walltime (`BlockCostType.SECOND`, `cost_divisor=10`, 1 credit per 10s) using `stats.walltime` from the `async_time_measured` decorator in `manager.py::_on_node_execution`. Idle time between steps is deliberately absorbed by the platform — this makes per-step charges predictable and user-visible. Do NOT flag the absence of sandbox-lifetime billing as a bug. Future migration to raw sandbox uptime billing would require plumbing `provider_cost_type="sandbox_seconds"` from the E2B SDK into `NodeExecutionStats` and switching to `COST_USD`/`SECOND` against that field; until then, walltime-per-execution is the correct and intentional model. Established in PR `#12894`.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:1112-1119
Timestamp: 2026-04-23T13:55:24.409Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, `AIVideoGeneratorBlock` is intentionally billed by walltime seconds (`BlockCostType.SECOND`, `cost_amount=3`, no `cost_divisor`) using `fal_credentials`. The FAL SDK does not currently expose per-request `provider_cost` or output duration in the response path used; walltime is the best available billing signal. Do NOT flag the absence of `COST_USD` or output-duration billing as a bug. Migration to `BlockCostType.COST_USD` against FAL-reported spend would require an SDK upgrade or a new stats-scraping path and is deferred until the FAL SDK exposes that signal. Established in PR `#12894`.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12893
File: autogpt_platform/backend/backend/blocks/ayrshare/post_to_tiktok.py:24-24
Timestamp: 2026-04-23T12:55:26.122Z
Learning: In `Significant-Gravitas/AutoGPT`, the `cost(*costs)` decorator in `autogpt_platform/backend/backend/sdk/cost_integration.py` evaluates `input_data` **at input-evaluation time, before `run()` executes**. Mutating fields of `input_data` inside a block's `run()` method (e.g., `input_data.is_video = has_video`) has **no effect on billing** because the cost filter has already been applied. For blocks that derive a computed boolean from both an explicit field and URL-sniffing (like `has_video` in `PostToTikTokBlock`), the explicit field (e.g., `is_video`) is the sole billing signal; callers must set it correctly. This applies to all blocks under `autogpt_platform/backend/backend/blocks/`.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12876
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:0-0
Timestamp: 2026-04-22T04:01:32.723Z
Learning: In `autogpt_platform/backend/backend/copilot/tools/helpers.py`, the `run_block` copilot path charges ONLY the user credit wallet (via `spend_credits` / `_charge_block_credits`). The microdollar rate-limit counter (`record_cost_usage`) is NOT incremented for `run_block` block executions — the `_record_block_microdollar_cost` helper was explicitly reverted (commit 16ae0f7b5, PR `#12876`) to avoid double-accounting. Do NOT flag missing microdollar recording in `execute_block` as a bug; the credit wallet is the sole billing mechanism for copilot `run_block` calls.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:271-277
Timestamp: 2026-04-23T13:53:40.315Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, `compute_token_credits()` intentionally returns `MODEL_COST[model]` (the flat tier) on pre-flight (when `stats is None`) for the `TOKENS` billing path. Returning 0 pre-flight would allow a zero-balance user to bypass the credit gate and trigger an LLM call, with the insufficient-balance error only surfacing post-flight (a billing leak). The overcharge concern (actual token cost < MODEL_COST estimate) is handled by `_charge_reconciled_usage_sync` in `autogpt_platform/backend/backend/executor/billing.py`, which issues a negative-delta refund via `spend_credits(cost=negative)` when real usage falls below the pre-flight estimate. Do NOT flag the MODEL_COST pre-flight floor in this function as an overcharge bug; the refund path covers it.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12385
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:149-185
Timestamp: 2026-03-15T15:30:09.706Z
Learning: In autogpt_platform/backend/backend/copilot/tools/helpers.py, within execute_block, when InsufficientBalanceError occurs after post-execution credit charging (concurrent balance drain after pre-check passed), this is treated as a non-fatal billing leak: log at ERROR level with structured JSON fields `{"billing_leak": True, "user_id": ..., "cost": ...}` for monitoring/alerting, then return BlockOutputResponse normally. Discarding the output would worsen UX since the block already executed with potential side effects. Reuse the credit_model obtained during the pre-execution balance check (guarded by `if cost > 0 and credit_model:`) for the post-execution charge; do not perform a second get_user_credit_model call.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 0
File: :0-0
Timestamp: 2026-04-23T00:07:27.117Z
Learning: In `autogpt_platform/backend/backend/copilot/sdk/openrouter_cost.py`, the `openrouter-cost-reconcile` Langfuse event carries `cost_source` ("openrouter" for authoritative OpenRouter-resolved cost, "fallback" for rate-card fallback) and `resolved_generation_id_count` alongside the reconciled cost and token/model/provider metadata. This lets operators distinguish authoritative reconciliations from fallbacks in Langfuse. Established in PR `#12889` commit 5ce3d0388.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 0
File: :0-0
Timestamp: 2026-04-21T11:41:05.877Z
Learning: In `autogpt_platform/backend/backend/copilot/baseline/service.py` (PR `#12870`, commits 080d42b9d and 3d7b38162), the `_close_reasoning_block_if_open(state)` helper centralises all four reasoning-block-close call sites (text branch, tool_calls branch, stream-end, exception path). The outer `finally` block of `_baseline_llm_caller` calls this helper plus stripper flush + `StreamTextEnd` to guarantee matched end events are emitted before `StreamFinishStep` on both normal and exception paths. Do NOT flag duplicated close logic or missing reasoning-end-on-exception as issues in this function.
📚 Learning: 2026-04-23T13:55:24.409Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:1112-1119
Timestamp: 2026-04-23T13:55:24.409Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, `AIVideoGeneratorBlock` is intentionally billed by walltime seconds (`BlockCostType.SECOND`, `cost_amount=3`, no `cost_divisor`) using `fal_credentials`. The FAL SDK does not currently expose per-request `provider_cost` or output duration in the response path used; walltime is the best available billing signal. Do NOT flag the absence of `COST_USD` or output-duration billing as a bug. Migration to `BlockCostType.COST_USD` against FAL-reported spend would require an SDK upgrade or a new stats-scraping path and is deferred until the FAL SDK exposes that signal. Established in PR `#12894`.
Applied to files:
autogpt_platform/backend/backend/blocks/video/narration.pyautogpt_platform/backend/backend/blocks/exa/websets_enrichment.pyautogpt_platform/backend/backend/blocks/baas/bots.py
📚 Learning: 2026-04-23T12:55:26.122Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12893
File: autogpt_platform/backend/backend/blocks/ayrshare/post_to_tiktok.py:24-24
Timestamp: 2026-04-23T12:55:26.122Z
Learning: Cost billing via the cost(*costs) decorator is applied at input-evaluation time (before a block’s run() executes). Therefore, mutating input_data inside run() will not change billing. When a block’s billing depends on a field plus URL/sniff-derived signals, treat the explicitly declared billing field (e.g., is_video) as the only billing source—set it correctly before run() (or in the code path that occurs before the decorator evaluates input_data). This should be checked for all blocks under autogpt_platform/backend/backend/blocks/ so billing signals are not mistakenly assumed to update during run().
Applied to files:
autogpt_platform/backend/backend/blocks/video/narration.pyautogpt_platform/backend/backend/blocks/exa/websets_enrichment.pyautogpt_platform/backend/backend/blocks/baas/bots.py
📚 Learning: 2026-04-23T13:53:29.246Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:1064-1111
Timestamp: 2026-04-23T13:53:29.246Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, the E2B sandbox blocks (`ExecuteCodeBlock`, `InstantiateCodeSandboxBlock`, `ExecuteCodeStepBlock`) are intentionally billed per-node-execution walltime (`BlockCostType.SECOND`, `cost_divisor=10`, 1 credit per 10s) using `stats.walltime` from the `async_time_measured` decorator in `manager.py::_on_node_execution`. Idle time between steps is deliberately absorbed by the platform — this makes per-step charges predictable and user-visible. Do NOT flag the absence of sandbox-lifetime billing as a bug. Future migration to raw sandbox uptime billing would require plumbing `provider_cost_type="sandbox_seconds"` from the E2B SDK into `NodeExecutionStats` and switching to `COST_USD`/`SECOND` against that field; until then, walltime-per-execution is the correct and intentional model. Established in PR `#12894`.
Applied to files:
autogpt_platform/backend/backend/blocks/video/narration.pyautogpt_platform/backend/backend/blocks/exa/websets_enrichment.pyautogpt_platform/backend/backend/blocks/baas/bots.py
📚 Learning: 2026-04-22T04:01:32.723Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12876
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:0-0
Timestamp: 2026-04-22T04:01:32.723Z
Learning: In `autogpt_platform/backend/backend/copilot/tools/helpers.py`, the `run_block` copilot path charges ONLY the user credit wallet (via `spend_credits` / `_charge_block_credits`). The microdollar rate-limit counter (`record_cost_usage`) is NOT incremented for `run_block` block executions — the `_record_block_microdollar_cost` helper was explicitly reverted (commit 16ae0f7b5, PR `#12876`) to avoid double-accounting. Do NOT flag missing microdollar recording in `execute_block` as a bug; the credit wallet is the sole billing mechanism for copilot `run_block` calls.
Applied to files:
autogpt_platform/backend/backend/blocks/video/narration.pyautogpt_platform/backend/backend/blocks/baas/bots.py
📚 Learning: 2026-04-23T13:53:40.315Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:271-277
Timestamp: 2026-04-23T13:53:40.315Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, `compute_token_credits()` intentionally returns `MODEL_COST[model]` (the flat tier) on pre-flight (when `stats is None`) for the `TOKENS` billing path. Returning 0 pre-flight would allow a zero-balance user to bypass the credit gate and trigger an LLM call, with the insufficient-balance error only surfacing post-flight (a billing leak). The overcharge concern (actual token cost < MODEL_COST estimate) is handled by `_charge_reconciled_usage_sync` in `autogpt_platform/backend/backend/executor/billing.py`, which issues a negative-delta refund via `spend_credits(cost=negative)` when real usage falls below the pre-flight estimate. Do NOT flag the MODEL_COST pre-flight floor in this function as an overcharge bug; the refund path covers it.
Applied to files:
autogpt_platform/backend/backend/blocks/video/narration.pyautogpt_platform/backend/backend/blocks/baas/bots.py
📚 Learning: 2026-03-15T15:30:09.706Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12385
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:149-185
Timestamp: 2026-03-15T15:30:09.706Z
Learning: In autogpt_platform/backend/backend/copilot/tools/helpers.py, within execute_block, when InsufficientBalanceError occurs after post-execution credit charging (concurrent balance drain after pre-check passed), this is treated as a non-fatal billing leak: log at ERROR level with structured JSON fields `{"billing_leak": True, "user_id": ..., "cost": ...}` for monitoring/alerting, then return BlockOutputResponse normally. Discarding the output would worsen UX since the block already executed with potential side effects. Reuse the credit_model obtained during the pre-execution balance check (guarded by `if cost > 0 and credit_model:`) for the post-execution charge; do not perform a second get_user_credit_model call.
Applied to files:
autogpt_platform/backend/backend/blocks/video/narration.pyautogpt_platform/backend/backend/blocks/baas/bots.py
📚 Learning: 2026-02-05T04:11:00.596Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 11796
File: autogpt_platform/backend/backend/blocks/video/concat.py:3-4
Timestamp: 2026-02-05T04:11:00.596Z
Learning: In autogpt_platform/backend/backend/blocks/**/*.py, when creating a new block, generate a UUID once with uuid.uuid4() and hard-code the resulting string as the block's id parameter. Do not call uuid.uuid4() at runtime; IDs must be constant across all imports and runs to ensure stability.
Applied to files:
autogpt_platform/backend/backend/blocks/video/narration.pyautogpt_platform/backend/backend/blocks/exa/websets_enrichment.pyautogpt_platform/backend/backend/blocks/baas/bots.py
📚 Learning: 2026-03-16T16:32:21.686Z
Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/pods.py:62-74
Timestamp: 2026-03-16T16:32:21.686Z
Learning: In autogpt_platform/backend/backend/blocks/, the Block base class execute() already wraps run() in a try/except to convert uncaught exceptions into BlockExecutionError/BlockUnknownError. Do not add per-block try/except in individual block run() methods, as this is not the established pattern (e.g., Gmail, Slack, Todoist blocks omit it). Only use explicit try/except within blocks that need to distinguish between success and error yield paths inside a generator (e.g., attachment blocks). This guidance applies to all Python files under autogpt_platform/backend/backend/blocks/ and similar block implementations; avoid duplicating error handling in run() unless a block requires generator-based branching.
Applied to files:
autogpt_platform/backend/backend/blocks/video/narration.pyautogpt_platform/backend/backend/blocks/exa/websets_enrichment.pyautogpt_platform/backend/backend/blocks/baas/bots.py
📚 Learning: 2026-02-26T17:02:22.448Z
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12211
File: .pre-commit-config.yaml:160-179
Timestamp: 2026-02-26T17:02:22.448Z
Learning: Keep the pre-commit hook pattern broad for autogpt_platform/backend to ensure OpenAPI schema changes are captured. Do not narrow to backend/api/ alone, since the generated schema depends on Pydantic models across multiple directories (backend/data/, backend/blocks/, backend/copilot/, backend/integrations/, backend/util/). Narrowing could miss schema changes and cause frontend type desynchronization.
Applied to files:
autogpt_platform/backend/backend/blocks/video/narration.pyautogpt_platform/backend/backend/blocks/exa/websets_enrichment.pyautogpt_platform/backend/backend/blocks/baas/bots.py
📚 Learning: 2026-03-05T15:42:08.207Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12297
File: .claude/skills/backend-check/SKILL.md:14-16
Timestamp: 2026-03-05T15:42:08.207Z
Learning: In Python files under autogpt_platform/backend (recursively), rely on poetry run format to perform formatting (Black + isort) and linting (ruff). Do not run poetry run lint as a separate step after poetry run format, since format already includes linting checks.
Applied to files:
autogpt_platform/backend/backend/blocks/video/narration.pyautogpt_platform/backend/backend/blocks/exa/websets_enrichment.pyautogpt_platform/backend/backend/blocks/baas/bots.py
📚 Learning: 2026-03-16T16:30:11.452Z
Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/threads.py:80-102
Timestamp: 2026-03-16T16:30:11.452Z
Learning: In autogpt_platform/backend/backend/blocks/ (and related blocks under autogpt_platform/backend/backend/blocks/), do not add try/except blocks around a block's run() method for standard error propagation. The block executor framework (backend/executor/manager.py) catches uncaught exceptions from run() and emits them on the 'error' output. Only add explicit try/except blocks when you need to control partial outputs in failure cases (e.g., certain outputs must not be yielded on error, as in attachment blocks). This is the standard pattern across the codebase; apply it broadly to blocks' run() implementations.
Applied to files:
autogpt_platform/backend/backend/blocks/video/narration.pyautogpt_platform/backend/backend/blocks/exa/websets_enrichment.pyautogpt_platform/backend/backend/blocks/baas/bots.py
📚 Learning: 2026-03-16T16:30:23.196Z
Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/pods.py:62-74
Timestamp: 2026-03-16T16:30:23.196Z
Learning: In any Python file under autogpt_platform/backend/backend/blocks, do not add a try/except around run() solely for standard error handling. The block framework’s _execute() in _base.py already catches unhandled exceptions and re-raises as BlockExecutionError or BlockUnknownError. If you yield ("error", message), _execute() raises BlockExecutionError immediately, so the error port will not propagate downstream. Reserve explicit try/except for scenarios where you must control partial output (e.g., attachment blocks that must skip yielding content_base64 on failure).
Applied to files:
autogpt_platform/backend/backend/blocks/video/narration.pyautogpt_platform/backend/backend/blocks/exa/websets_enrichment.pyautogpt_platform/backend/backend/blocks/baas/bots.py
📚 Learning: 2026-03-16T16:30:11.452Z
Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/threads.py:80-102
Timestamp: 2026-03-16T16:30:11.452Z
Learning: Do not wrap synchronous AgentMail SDK calls with asyncio.to_thread() in blocks under autogpt_platform/backend/backend/blocks (and across the codebase). The block executor runs node execution in dedicated threads via asyncio.run_coroutine_threadsafe (see manager.py around lines ~745-752 and ~1079). The existing pattern avoids using asyncio.to_thread for SDK calls inside async run() methods, so maintain that approach and do not add to_thread usage in these code paths.
Applied to files:
autogpt_platform/backend/backend/blocks/video/narration.pyautogpt_platform/backend/backend/blocks/exa/websets_enrichment.pyautogpt_platform/backend/backend/blocks/baas/bots.py
📚 Learning: 2026-03-16T16:35:40.236Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/api/features/workflow_import.py:54-63
Timestamp: 2026-03-16T16:35:40.236Z
Learning: Avoid using the word 'competitor' in public-facing identifiers and text. Use neutral naming for API paths, model names, function names, and UI text. Examples: rename 'CompetitorFormat' to 'SourcePlatform', 'convert_competitor_workflow' to 'convert_workflow', '/competitor-workflow' to '/workflow'. Apply this guideline to files under autogpt_platform/backend and autogpt_platform/frontend.
Applied to files:
autogpt_platform/backend/backend/blocks/video/narration.pyautogpt_platform/backend/backend/blocks/exa/websets_enrichment.pyautogpt_platform/backend/backend/blocks/baas/bots.py
📚 Learning: 2026-03-31T15:37:38.626Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12623
File: autogpt_platform/backend/backend/copilot/tools/agent_generator/fixer.py:37-47
Timestamp: 2026-03-31T15:37:38.626Z
Learning: When validating/constructing Anthropic API model IDs in Significant-Gravitas/AutoGPT, allow the hyphen-separated Claude Opus 4.6 model ID `claude-opus-4-6` (it corresponds to `LlmModel.CLAUDE_4_6_OPUS` in `autogpt_platform/backend/backend/blocks/llm.py`). Do NOT require the dot-separated form in Anthropic contexts. Only OpenRouter routing variants should use the dot separator (e.g., `anthropic/claude-opus-4.6`); `claude-opus-4-6` should be treated as correct when passed to Anthropic, and flagged only if it’s used in the OpenRouter path where the dot form is expected.
Applied to files:
autogpt_platform/backend/backend/blocks/video/narration.pyautogpt_platform/backend/backend/blocks/exa/websets_enrichment.pyautogpt_platform/backend/backend/blocks/baas/bots.py
📚 Learning: 2026-04-15T02:43:36.890Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12780
File: autogpt_platform/backend/backend/copilot/tools/workspace_files.py:0-0
Timestamp: 2026-04-15T02:43:36.890Z
Learning: When reviewing Python exception handlers, do not flag `isinstance(e, X)` checks as dead/unreachable if the caught exception `X` is a subclass of the exception type being handled. For example, if `X` (e.g., `VirusScanError`) inherits from `ValueError` (directly or via an intermediate class) and it can be raised within an `except ValueError:` block, then `isinstance(e, X)` inside that handler is reachable and should not be treated as dead code.
Applied to files:
autogpt_platform/backend/backend/blocks/video/narration.pyautogpt_platform/backend/backend/blocks/exa/websets_enrichment.pyautogpt_platform/backend/backend/blocks/baas/bots.py
📚 Learning: 2026-04-22T11:46:04.431Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12881
File: autogpt_platform/backend/backend/copilot/config.py:0-0
Timestamp: 2026-04-22T11:46:04.431Z
Learning: Do not flag the Claude Sonnet 4.6 model ID as incorrect when it uses the project’s established hyphenated convention: `anthropic/claude-sonnet-4-6`. This hyphen form is the intentional, production convention and should be treated as valid (including in files like llm.py, blocks tests, reasoning.py, `_is_anthropic_model` tests, and config defaults). Note that OpenRouter also accepts the dot variant `anthropic/claude-sonnet-4.6`, so either form may be tolerated, but `anthropic/claude-sonnet-4-6` should be considered the standard to match project usage.
Applied to files:
autogpt_platform/backend/backend/blocks/video/narration.pyautogpt_platform/backend/backend/blocks/exa/websets_enrichment.pyautogpt_platform/backend/backend/blocks/baas/bots.py
📚 Learning: 2026-04-22T11:46:12.892Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12881
File: autogpt_platform/backend/backend/copilot/baseline/service.py:322-332
Timestamp: 2026-04-22T11:46:12.892Z
Learning: In this codebase (Significant-Gravitas/AutoGPT), OpenRouter-routed Anthropic model IDs should use the hyphen-separated convention (e.g., `anthropic/claude-sonnet-4-6`, `anthropic/claude-opus-4-6`). Although OpenRouter may accept both hyphen and dot variants, treat the hyphen-separated form as the intended, correct codebase-wide convention and do not flag it as an error. Only flag the dot-separated variant (e.g., `anthropic/claude-sonnet-4.6`) as incorrect when reviewing/validating model ID strings for OpenRouter-routed Anthropic models.
Applied to files:
autogpt_platform/backend/backend/blocks/video/narration.pyautogpt_platform/backend/backend/blocks/exa/websets_enrichment.pyautogpt_platform/backend/backend/blocks/baas/bots.py
📚 Learning: 2026-04-02T14:27:41.807Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12651
File: autogpt_platform/frontend/src/app/api/openapi.json:8653-8696
Timestamp: 2026-04-02T14:27:41.807Z
Learning: Repo: Significant-Gravitas/AutoGPT — Platform costs
The PlatformCostLog.duration is stored in DB but intentionally omitted from the CostLogRow API response to keep the raw logs compact. Do not flag this omission; suggest documenting the intent in the route description if needed.
Applied to files:
autogpt_platform/backend/backend/blocks/baas/bots.py
🔇 Additional comments (3)
autogpt_platform/backend/backend/blocks/baas/bots.py (1)
193-203: LGTM — authoritative duration + named rate constant.Switch to
bot_meta.get("duration_seconds")matches the Meeting BaaS v2 schema, and lifting the rate to_MEETING_BAAS_USD_PER_SECOND = 0.69 / 3600removes the rounded magic number (the previous0.000192under-billed by ~0.09% versus the true $0.69/hr). Thefloat(... or 0)+> 0guard correctly handlesNone/missing fields so zero-duration responses simply skip billing and fall back to the Join block's flat 30 cr.One edge case to be aware of (not blocking): if
get_meeting_datais ever called before the recording completes (i.e.duration_secondsisnull), the fetch itself emits noprovider_costand is effectively free. That appears intentional per the module-level comment, but worth noting in case a user polls this block in a loop.autogpt_platform/backend/backend/blocks/video/narration.py (1)
227-242: Model-aware USD estimation correctly addresses prior over-billing.The Flash/Turbo v2.5 → 0.5 credits/char vs Multilingual v2 / Turbo v2 → 1.0 credit/char split matches ElevenLabs' billed-credit rates, so cheap-tier users are no longer 2× over-charged.
merge_statsis only reached after_generate_narration_audioreturns, so API failures correctly skip the charge.autogpt_platform/backend/backend/blocks/exa/websets_enrichment.py (1)
227-231: Remove this comment — no over-billing risk exists.The polling loop's
merge_exa_cost(self, current_enrich)is a no-op.enrichments.get()is a GET request (free per Exa API docs), and the response does not includecostDollars— evidenced by theWebsetEnrichmentModel.from_sdk()method explicitly mapping all enrichment fields without extracting any cost information. Whenextract_exa_cost_usd()finds no cost field,merge_exa_cost()returns early and performs no merge.> Likely an incorrect or invalid review comment.
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (4)
autogpt_platform/backend/backend/blocks/baas/bots_cost_test.py (2)
77-86: LGTM — parametrization covers the key emission branches.Good coverage of the emission gate: positive durations emit exactly one
NodeExecutionStatswithprovider_cost_type="cost_usd"and the scaled USD amount, while0and missingduration_secondscorrectly skip emission. MockingMeetingBaasAPIat the use site and capturingmerge_statsviaside_effect=captured.appendis clean.Minor optional tweak: the
transcriptoutput is also yielded byBaasBotFetchMeetingDataBlock.run(seebots.py:181-207) but isn't asserted here — consider asserting all three output names for parity with the block contract, and addingids=to the parametrize for nicer test IDs.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/blocks/baas/bots_cost_test.py` around lines 77 - 86, Add an assertion that the test verifies all three outputs yielded by BaasBotFetchMeetingDataBlock.run by including "transcript" alongside "mp4_url" and "metadata" when building the names list (refer to outputs and BaasBotFetchMeetingDataBlock.run), and optionally add ids= to the `@pytest.mark.parametrize` decorator so each case has a readable test id; keep the existing captured-stat checks intact.
23-25: Tautological assertion — tie to the published rate instead.Line 25 asserts the constant equals
0.69 / 3600, which is the exact literal used to define_MEETING_BAAS_USD_PER_SECONDinbots.py. The test will still pass if someone accidentally changes both the constant and the test to, e.g.,0.79 / 3600. The docstring already cites the documented per-second figure (~$0.000192); asserting against that value (or against a separately-computed0.69 / 3600with a tight tolerance plus the hourly rate) gives the test meaningful regression value.♻️ Suggested tightening
def test_usd_per_second_derives_from_published_rate(): """$0.69/hour published rate → ~$0.000192/second.""" - assert _MEETING_BAAS_USD_PER_SECOND == pytest.approx(0.69 / 3600) + # Pin to the published per-hour and per-second figures so accidental + # drift in bots.py is caught here. + assert _MEETING_BAAS_USD_PER_SECOND * 3600 == pytest.approx(0.69) + assert _MEETING_BAAS_USD_PER_SECOND == pytest.approx(1.92e-4, rel=1e-3)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/blocks/baas/bots_cost_test.py` around lines 23 - 25, The test test_usd_per_second_derives_from_published_rate currently tautologically compares _MEETING_BAAS_USD_PER_SECOND to the exact expression used to define it; update the test to assert the constant is derived from the documented hourly rate instead: compute expected = <published hourly rate> / 3600 (preferably by importing the published hourly rate constant from bots.py, e.g., _MEETING_BAAS_PUBLISHED_RATE if it exists) and assert _MEETING_BAAS_USD_PER_SECOND == pytest.approx(expected, rel=1e-3) (or, if no published-rate constant exists, assert against the literal 0.69/3600 with pytest.approx and a tight tolerance) so the test fails if the per-second value drifts independently from the published hourly rate.autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.py (2)
149-149: Stray section header with no test underneath.The banner
# -------- ClaudeCode COST_USD registration sanity (already tested in claude_code_cost_test.py) --------is followed by nothing. Either drop the banner entirely or add a one-line# Covered in claude_code_cost_test.py::...pointer so it doesn't read like a forgotten test.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.py` at line 149, Remove or clarify the stray section header comment "# -------- ClaudeCode COST_USD registration sanity (already tested in claude_code_cost_test.py) --------" in cost_leak_fixes_test.py: either delete that banner line entirely, or replace it with a one-line explanatory pointer such as "# Covered in claude_code_cost_test.py::test_claude_code_cost_usd" (or the actual test name) so the file no longer contains a dangling header suggesting a missing test.
35-35: Move block imports to module top.These imports (
PineconeInsertBlock,CodeGenerationBlock,PerplexityBlock,VideoNarrationBlock,PineconeInitBlock/QueryBlock) are not heavy optional dependencies and don't need deferral. Hoisting them to the top of the file keeps the test module consistent with the rest of the codebase and surfaces import errors at collection time rather than inside individual tests.As per coding guidelines: "Use top-level imports only — avoid local/inner imports except for lazy imports of heavy optional dependencies like
openpyxl".Also applies to: 140-140, 156-156, 170-170, 182-186
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.py` at line 35, The tests currently perform local imports for PineconeInsertBlock, CodeGenerationBlock, PerplexityBlock, VideoNarrationBlock, PineconeInitBlock and PineconeQueryBlock inside individual test functions; move these import statements to the top of the module as normal top-level imports (hoist the from backend.blocks... import PineconeInsertBlock, CodeGenerationBlock, PerplexityBlock, VideoNarrationBlock, PineconeInitBlock, PineconeQueryBlock) so import failures surface at collection time and avoid unnecessary inner imports.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.py`:
- Around line 87-109: The test is tautological because it computes the same rate
logic locally instead of exercising VideoNarrationBlock's billing; update
test_narration_per_char_rate_scales_with_model to call the real billing path in
backend/blocks/video/narration.py (e.g., instantiate/invoke VideoNarrationBlock
or its cost helper) with a mocked ElevenLabs client and assert the resulting
merge_stats(...).provider_cost or the block's computed script_usd matches the
expected value; remove the duplicated half set and import/use the production
constant or lookup from VideoNarrationBlock (or its module) so the test fails if
the block's branching changes.
- Around line 115-133: The test currently duplicates the guard logic instead of
exercising the real code; update it to call the actual implementation by either
(A) importing and calling extract_openrouter_cost with a mocked OpenRouter
response (patch extract_openrouter_cost to return None and then a positive
value) and assert the NodeExecutionStats.provider_cost_type after those calls,
or (B) instantiate and execute PerplexityBlock (or the method that sets
self.execution_stats.provider_cost_type) while patching extract_openrouter_cost
to return None / 0.0421 and then assert that
PerplexityBlock.execution_stats.provider_cost_type is None in the first case and
"cost_usd" in the second; use NodeExecutionStats, PerplexityBlock, and
extract_openrouter_cost as the reference symbols to locate the code to test.
---
Nitpick comments:
In `@autogpt_platform/backend/backend/blocks/baas/bots_cost_test.py`:
- Around line 77-86: Add an assertion that the test verifies all three outputs
yielded by BaasBotFetchMeetingDataBlock.run by including "transcript" alongside
"mp4_url" and "metadata" when building the names list (refer to outputs and
BaasBotFetchMeetingDataBlock.run), and optionally add ids= to the
`@pytest.mark.parametrize` decorator so each case has a readable test id; keep the
existing captured-stat checks intact.
- Around line 23-25: The test test_usd_per_second_derives_from_published_rate
currently tautologically compares _MEETING_BAAS_USD_PER_SECOND to the exact
expression used to define it; update the test to assert the constant is derived
from the documented hourly rate instead: compute expected = <published hourly
rate> / 3600 (preferably by importing the published hourly rate constant from
bots.py, e.g., _MEETING_BAAS_PUBLISHED_RATE if it exists) and assert
_MEETING_BAAS_USD_PER_SECOND == pytest.approx(expected, rel=1e-3) (or, if no
published-rate constant exists, assert against the literal 0.69/3600 with
pytest.approx and a tight tolerance) so the test fails if the per-second value
drifts independently from the published hourly rate.
In `@autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.py`:
- Line 149: Remove or clarify the stray section header comment "# --------
ClaudeCode COST_USD registration sanity (already tested in
claude_code_cost_test.py) --------" in cost_leak_fixes_test.py: either delete
that banner line entirely, or replace it with a one-line explanatory pointer
such as "# Covered in claude_code_cost_test.py::test_claude_code_cost_usd" (or
the actual test name) so the file no longer contains a dangling header
suggesting a missing test.
- Line 35: The tests currently perform local imports for PineconeInsertBlock,
CodeGenerationBlock, PerplexityBlock, VideoNarrationBlock, PineconeInitBlock and
PineconeQueryBlock inside individual test functions; move these import
statements to the top of the module as normal top-level imports (hoist the from
backend.blocks... import PineconeInsertBlock, CodeGenerationBlock,
PerplexityBlock, VideoNarrationBlock, PineconeInitBlock, PineconeQueryBlock) so
import failures surface at collection time and avoid unnecessary inner imports.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: c91793bd-6530-49e7-aa96-e1008c4f16e5
📒 Files selected for processing (2)
autogpt_platform/backend/backend/blocks/baas/bots_cost_test.pyautogpt_platform/backend/backend/blocks/cost_leak_fixes_test.py
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (11)
- GitHub Check: check API types
- GitHub Check: Seer Code Review
- GitHub Check: test (3.13)
- GitHub Check: test (3.11)
- GitHub Check: test (3.12)
- GitHub Check: type-check (3.11)
- GitHub Check: type-check (3.13)
- GitHub Check: end-to-end tests
- GitHub Check: Analyze (python)
- GitHub Check: Check PR Status
- GitHub Check: Analyze (typescript)
🧰 Additional context used
📓 Path-based instructions (4)
autogpt_platform/backend/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
autogpt_platform/backend/**/*.py: Use Python 3.11 (required; managed by Poetry via pyproject.toml) for backend development
Always run 'poetry run format' (Black + isort) before linting in backend development
Always run 'poetry run lint' (ruff) after formatting in backend development
autogpt_platform/backend/**/*.py: Usepoetry run ...command for executing Python package dependencies
Use top-level imports only — avoid local/inner imports except for lazy imports of heavy optional dependencies likeopenpyxl
Use absolute imports withfrom backend.module import ...for cross-package imports; single-dot relative imports are acceptable for sibling modules within the same package; avoid double-dot relative imports
Do not use duck typing — avoidhasattr/getattr/isinstancefor type dispatch; use typed interfaces/unions/protocols instead
Use Pydantic models over dataclass/namedtuple/dict for structured data
Do not use linter suppressors — no# type: ignore,# noqa,# pyright: ignore; fix the type/code instead
Prefer list comprehensions over manual loop-and-append patterns
Use early return with guard clauses first to avoid deep nesting
Use%sfor deferred interpolation indebuglog statements for efficiency; use f-strings elsewhere for readability (e.g.,logger.debug("Processing %s items", count)vslogger.info(f"Processing {count} items"))
Sanitize error paths by usingos.path.basename()in error messages to avoid leaking directory structure
Be aware of TOCTOU (Time-Of-Check-Time-Of-Use) issues — avoid check-then-act patterns for file access and credit charging
Usetransaction=Truefor Redis pipelines to ensure atomicity on multi-step operations
Usemax(0, value)guards for computed values that should never be negative
Keep files under ~300 lines; if a file grows beyond this, split by responsibility (extract helpers, models, or a sub-module into a new file)
Keep functions under ~40 lines; extract named helpers when a function grows longer
...
Files:
autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.pyautogpt_platform/backend/backend/blocks/baas/bots_cost_test.py
autogpt_platform/backend/backend/blocks/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
autogpt_platform/backend/backend/blocks/**/*.py: Inherit from 'Block' base class with input/output schemas when adding new blocks in backend
Implement 'run' method with proper error handling in backend blocks
Generate block UUID using 'uuid.uuid4()' when creating new blocks in backend
Write tests alongside block implementation when adding new blocks in backend
autogpt_platform/backend/backend/blocks/**/*.py: For blocks handling files, usestore_media_file()withreturn_format="for_local_processing"when processing with local tools (ffmpeg, MoviePy, PIL)
For blocks handling files, usestore_media_file()withreturn_format="for_external_api"when sending content to external APIs (Replicate, OpenAI)
For blocks returning files, usestore_media_file()withreturn_format="for_block_output"to enable auto-adaptation to execution context (workspace:// in CoPilot, data URI in graphs)
When creating new blocks, inherit fromBlockbase class, define input/output schemas usingBlockSchema, implement asyncrunmethod, and generate unique block ID usinguuid.uuid4()
Files:
autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.pyautogpt_platform/backend/backend/blocks/baas/bots_cost_test.py
autogpt_platform/{backend,autogpt_libs}/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
Format Python code with
poetry run format
Files:
autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.pyautogpt_platform/backend/backend/blocks/baas/bots_cost_test.py
autogpt_platform/backend/**/*_test.py
📄 CodeRabbit inference engine (autogpt_platform/backend/AGENTS.md)
autogpt_platform/backend/**/*_test.py: Use pytest with snapshot testing for API responses
Colocate test files with source files using*_test.pynaming convention
Mock at boundaries — mock where the symbol is used, not where it's defined; after refactoring, update mock targets to match new module paths
UseAsyncMockfromunittest.mockfor async functions in tests
When writing tests, use Test-Driven Development (TDD): write failing tests marked with@pytest.mark.xfailbefore implementation, then remove the marker once the implementation is complete
When creating snapshots in tests, usepoetry run pytest path/to/test.py --snapshot-update; always review snapshot changes withgit diffbefore committing
Files:
autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.pyautogpt_platform/backend/backend/blocks/baas/bots_cost_test.py
🧠 Learnings (20)
📓 Common learnings
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:1064-1111
Timestamp: 2026-04-23T13:53:29.246Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, the E2B sandbox blocks (`ExecuteCodeBlock`, `InstantiateCodeSandboxBlock`, `ExecuteCodeStepBlock`) are intentionally billed per-node-execution walltime (`BlockCostType.SECOND`, `cost_divisor=10`, 1 credit per 10s) using `stats.walltime` from the `async_time_measured` decorator in `manager.py::_on_node_execution`. Idle time between steps is deliberately absorbed by the platform — this makes per-step charges predictable and user-visible. Do NOT flag the absence of sandbox-lifetime billing as a bug. Future migration to raw sandbox uptime billing would require plumbing `provider_cost_type="sandbox_seconds"` from the E2B SDK into `NodeExecutionStats` and switching to `COST_USD`/`SECOND` against that field; until then, walltime-per-execution is the correct and intentional model. Established in PR `#12894`.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:1112-1119
Timestamp: 2026-04-23T13:55:24.409Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, `AIVideoGeneratorBlock` is intentionally billed by walltime seconds (`BlockCostType.SECOND`, `cost_amount=3`, no `cost_divisor`) using `fal_credentials`. The FAL SDK does not currently expose per-request `provider_cost` or output duration in the response path used; walltime is the best available billing signal. Do NOT flag the absence of `COST_USD` or output-duration billing as a bug. Migration to `BlockCostType.COST_USD` against FAL-reported spend would require an SDK upgrade or a new stats-scraping path and is deferred until the FAL SDK exposes that signal. Established in PR `#12894`.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12893
File: autogpt_platform/backend/backend/blocks/ayrshare/post_to_tiktok.py:24-24
Timestamp: 2026-04-23T12:55:26.122Z
Learning: In `Significant-Gravitas/AutoGPT`, the `cost(*costs)` decorator in `autogpt_platform/backend/backend/sdk/cost_integration.py` evaluates `input_data` **at input-evaluation time, before `run()` executes**. Mutating fields of `input_data` inside a block's `run()` method (e.g., `input_data.is_video = has_video`) has **no effect on billing** because the cost filter has already been applied. For blocks that derive a computed boolean from both an explicit field and URL-sniffing (like `has_video` in `PostToTikTokBlock`), the explicit field (e.g., `is_video`) is the sole billing signal; callers must set it correctly. This applies to all blocks under `autogpt_platform/backend/backend/blocks/`.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12876
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:0-0
Timestamp: 2026-04-22T04:01:32.723Z
Learning: In `autogpt_platform/backend/backend/copilot/tools/helpers.py`, the `run_block` copilot path charges ONLY the user credit wallet (via `spend_credits` / `_charge_block_credits`). The microdollar rate-limit counter (`record_cost_usage`) is NOT incremented for `run_block` block executions — the `_record_block_microdollar_cost` helper was explicitly reverted (commit 16ae0f7b5, PR `#12876`) to avoid double-accounting. Do NOT flag missing microdollar recording in `execute_block` as a bug; the credit wallet is the sole billing mechanism for copilot `run_block` calls.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:271-277
Timestamp: 2026-04-23T13:53:40.315Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, `compute_token_credits()` intentionally returns `MODEL_COST[model]` (the flat tier) on pre-flight (when `stats is None`) for the `TOKENS` billing path. Returning 0 pre-flight would allow a zero-balance user to bypass the credit gate and trigger an LLM call, with the insufficient-balance error only surfacing post-flight (a billing leak). The overcharge concern (actual token cost < MODEL_COST estimate) is handled by `_charge_reconciled_usage_sync` in `autogpt_platform/backend/backend/executor/billing.py`, which issues a negative-delta refund via `spend_credits(cost=negative)` when real usage falls below the pre-flight estimate. Do NOT flag the MODEL_COST pre-flight floor in this function as an overcharge bug; the refund path covers it.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12385
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:149-185
Timestamp: 2026-03-15T15:30:09.706Z
Learning: In autogpt_platform/backend/backend/copilot/tools/helpers.py, within execute_block, when InsufficientBalanceError occurs after post-execution credit charging (concurrent balance drain after pre-check passed), this is treated as a non-fatal billing leak: log at ERROR level with structured JSON fields `{"billing_leak": True, "user_id": ..., "cost": ...}` for monitoring/alerting, then return BlockOutputResponse normally. Discarding the output would worsen UX since the block already executed with potential side effects. Reuse the credit_model obtained during the pre-execution balance check (guarded by `if cost > 0 and credit_model:`) for the post-execution charge; do not perform a second get_user_credit_model call.
📚 Learning: 2026-04-23T12:55:26.122Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12893
File: autogpt_platform/backend/backend/blocks/ayrshare/post_to_tiktok.py:24-24
Timestamp: 2026-04-23T12:55:26.122Z
Learning: Cost billing via the cost(*costs) decorator is applied at input-evaluation time (before a block’s run() executes). Therefore, mutating input_data inside run() will not change billing. When a block’s billing depends on a field plus URL/sniff-derived signals, treat the explicitly declared billing field (e.g., is_video) as the only billing source—set it correctly before run() (or in the code path that occurs before the decorator evaluates input_data). This should be checked for all blocks under autogpt_platform/backend/backend/blocks/ so billing signals are not mistakenly assumed to update during run().
Applied to files:
autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.pyautogpt_platform/backend/backend/blocks/baas/bots_cost_test.py
📚 Learning: 2026-04-23T13:55:24.409Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:1112-1119
Timestamp: 2026-04-23T13:55:24.409Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, `AIVideoGeneratorBlock` is intentionally billed by walltime seconds (`BlockCostType.SECOND`, `cost_amount=3`, no `cost_divisor`) using `fal_credentials`. The FAL SDK does not currently expose per-request `provider_cost` or output duration in the response path used; walltime is the best available billing signal. Do NOT flag the absence of `COST_USD` or output-duration billing as a bug. Migration to `BlockCostType.COST_USD` against FAL-reported spend would require an SDK upgrade or a new stats-scraping path and is deferred until the FAL SDK exposes that signal. Established in PR `#12894`.
Applied to files:
autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.pyautogpt_platform/backend/backend/blocks/baas/bots_cost_test.py
📚 Learning: 2026-04-23T13:53:29.246Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:1064-1111
Timestamp: 2026-04-23T13:53:29.246Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, the E2B sandbox blocks (`ExecuteCodeBlock`, `InstantiateCodeSandboxBlock`, `ExecuteCodeStepBlock`) are intentionally billed per-node-execution walltime (`BlockCostType.SECOND`, `cost_divisor=10`, 1 credit per 10s) using `stats.walltime` from the `async_time_measured` decorator in `manager.py::_on_node_execution`. Idle time between steps is deliberately absorbed by the platform — this makes per-step charges predictable and user-visible. Do NOT flag the absence of sandbox-lifetime billing as a bug. Future migration to raw sandbox uptime billing would require plumbing `provider_cost_type="sandbox_seconds"` from the E2B SDK into `NodeExecutionStats` and switching to `COST_USD`/`SECOND` against that field; until then, walltime-per-execution is the correct and intentional model. Established in PR `#12894`.
Applied to files:
autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.pyautogpt_platform/backend/backend/blocks/baas/bots_cost_test.py
📚 Learning: 2026-03-15T15:30:09.706Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12385
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:149-185
Timestamp: 2026-03-15T15:30:09.706Z
Learning: In autogpt_platform/backend/backend/copilot/tools/helpers.py, within execute_block, when InsufficientBalanceError occurs after post-execution credit charging (concurrent balance drain after pre-check passed), this is treated as a non-fatal billing leak: log at ERROR level with structured JSON fields `{"billing_leak": True, "user_id": ..., "cost": ...}` for monitoring/alerting, then return BlockOutputResponse normally. Discarding the output would worsen UX since the block already executed with potential side effects. Reuse the credit_model obtained during the pre-execution balance check (guarded by `if cost > 0 and credit_model:`) for the post-execution charge; do not perform a second get_user_credit_model call.
Applied to files:
autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.py
📚 Learning: 2026-04-23T13:53:40.315Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:271-277
Timestamp: 2026-04-23T13:53:40.315Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, `compute_token_credits()` intentionally returns `MODEL_COST[model]` (the flat tier) on pre-flight (when `stats is None`) for the `TOKENS` billing path. Returning 0 pre-flight would allow a zero-balance user to bypass the credit gate and trigger an LLM call, with the insufficient-balance error only surfacing post-flight (a billing leak). The overcharge concern (actual token cost < MODEL_COST estimate) is handled by `_charge_reconciled_usage_sync` in `autogpt_platform/backend/backend/executor/billing.py`, which issues a negative-delta refund via `spend_credits(cost=negative)` when real usage falls below the pre-flight estimate. Do NOT flag the MODEL_COST pre-flight floor in this function as an overcharge bug; the refund path covers it.
Applied to files:
autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.py
📚 Learning: 2026-02-04T16:49:42.490Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2026-02-04T16:49:42.490Z
Learning: Applies to autogpt_platform/backend/backend/blocks/**/*.py : Write tests alongside block implementation when adding new blocks in backend
Applied to files:
autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.pyautogpt_platform/backend/backend/blocks/baas/bots_cost_test.py
📚 Learning: 2026-04-22T04:01:32.723Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12876
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:0-0
Timestamp: 2026-04-22T04:01:32.723Z
Learning: In `autogpt_platform/backend/backend/copilot/tools/helpers.py`, the `run_block` copilot path charges ONLY the user credit wallet (via `spend_credits` / `_charge_block_credits`). The microdollar rate-limit counter (`record_cost_usage`) is NOT incremented for `run_block` block executions — the `_record_block_microdollar_cost` helper was explicitly reverted (commit 16ae0f7b5, PR `#12876`) to avoid double-accounting. Do NOT flag missing microdollar recording in `execute_block` as a bug; the credit wallet is the sole billing mechanism for copilot `run_block` calls.
Applied to files:
autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.py
📚 Learning: 2026-02-05T04:11:00.596Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 11796
File: autogpt_platform/backend/backend/blocks/video/concat.py:3-4
Timestamp: 2026-02-05T04:11:00.596Z
Learning: In autogpt_platform/backend/backend/blocks/**/*.py, when creating a new block, generate a UUID once with uuid.uuid4() and hard-code the resulting string as the block's id parameter. Do not call uuid.uuid4() at runtime; IDs must be constant across all imports and runs to ensure stability.
Applied to files:
autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.pyautogpt_platform/backend/backend/blocks/baas/bots_cost_test.py
📚 Learning: 2026-03-16T16:32:21.686Z
Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/pods.py:62-74
Timestamp: 2026-03-16T16:32:21.686Z
Learning: In autogpt_platform/backend/backend/blocks/, the Block base class execute() already wraps run() in a try/except to convert uncaught exceptions into BlockExecutionError/BlockUnknownError. Do not add per-block try/except in individual block run() methods, as this is not the established pattern (e.g., Gmail, Slack, Todoist blocks omit it). Only use explicit try/except within blocks that need to distinguish between success and error yield paths inside a generator (e.g., attachment blocks). This guidance applies to all Python files under autogpt_platform/backend/backend/blocks/ and similar block implementations; avoid duplicating error handling in run() unless a block requires generator-based branching.
Applied to files:
autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.pyautogpt_platform/backend/backend/blocks/baas/bots_cost_test.py
📚 Learning: 2026-02-26T17:02:22.448Z
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12211
File: .pre-commit-config.yaml:160-179
Timestamp: 2026-02-26T17:02:22.448Z
Learning: Keep the pre-commit hook pattern broad for autogpt_platform/backend to ensure OpenAPI schema changes are captured. Do not narrow to backend/api/ alone, since the generated schema depends on Pydantic models across multiple directories (backend/data/, backend/blocks/, backend/copilot/, backend/integrations/, backend/util/). Narrowing could miss schema changes and cause frontend type desynchronization.
Applied to files:
autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.pyautogpt_platform/backend/backend/blocks/baas/bots_cost_test.py
📚 Learning: 2026-03-05T15:42:08.207Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12297
File: .claude/skills/backend-check/SKILL.md:14-16
Timestamp: 2026-03-05T15:42:08.207Z
Learning: In Python files under autogpt_platform/backend (recursively), rely on poetry run format to perform formatting (Black + isort) and linting (ruff). Do not run poetry run lint as a separate step after poetry run format, since format already includes linting checks.
Applied to files:
autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.pyautogpt_platform/backend/backend/blocks/baas/bots_cost_test.py
📚 Learning: 2026-03-16T16:30:11.452Z
Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/threads.py:80-102
Timestamp: 2026-03-16T16:30:11.452Z
Learning: In autogpt_platform/backend/backend/blocks/ (and related blocks under autogpt_platform/backend/backend/blocks/), do not add try/except blocks around a block's run() method for standard error propagation. The block executor framework (backend/executor/manager.py) catches uncaught exceptions from run() and emits them on the 'error' output. Only add explicit try/except blocks when you need to control partial outputs in failure cases (e.g., certain outputs must not be yielded on error, as in attachment blocks). This is the standard pattern across the codebase; apply it broadly to blocks' run() implementations.
Applied to files:
autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.pyautogpt_platform/backend/backend/blocks/baas/bots_cost_test.py
📚 Learning: 2026-03-16T16:30:23.196Z
Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/pods.py:62-74
Timestamp: 2026-03-16T16:30:23.196Z
Learning: In any Python file under autogpt_platform/backend/backend/blocks, do not add a try/except around run() solely for standard error handling. The block framework’s _execute() in _base.py already catches unhandled exceptions and re-raises as BlockExecutionError or BlockUnknownError. If you yield ("error", message), _execute() raises BlockExecutionError immediately, so the error port will not propagate downstream. Reserve explicit try/except for scenarios where you must control partial output (e.g., attachment blocks that must skip yielding content_base64 on failure).
Applied to files:
autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.pyautogpt_platform/backend/backend/blocks/baas/bots_cost_test.py
📚 Learning: 2026-03-16T16:30:11.452Z
Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/threads.py:80-102
Timestamp: 2026-03-16T16:30:11.452Z
Learning: Do not wrap synchronous AgentMail SDK calls with asyncio.to_thread() in blocks under autogpt_platform/backend/backend/blocks (and across the codebase). The block executor runs node execution in dedicated threads via asyncio.run_coroutine_threadsafe (see manager.py around lines ~745-752 and ~1079). The existing pattern avoids using asyncio.to_thread for SDK calls inside async run() methods, so maintain that approach and do not add to_thread usage in these code paths.
Applied to files:
autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.pyautogpt_platform/backend/backend/blocks/baas/bots_cost_test.py
📚 Learning: 2026-03-16T16:35:40.236Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/api/features/workflow_import.py:54-63
Timestamp: 2026-03-16T16:35:40.236Z
Learning: Avoid using the word 'competitor' in public-facing identifiers and text. Use neutral naming for API paths, model names, function names, and UI text. Examples: rename 'CompetitorFormat' to 'SourcePlatform', 'convert_competitor_workflow' to 'convert_workflow', '/competitor-workflow' to '/workflow'. Apply this guideline to files under autogpt_platform/backend and autogpt_platform/frontend.
Applied to files:
autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.pyautogpt_platform/backend/backend/blocks/baas/bots_cost_test.py
📚 Learning: 2026-03-31T15:37:38.626Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12623
File: autogpt_platform/backend/backend/copilot/tools/agent_generator/fixer.py:37-47
Timestamp: 2026-03-31T15:37:38.626Z
Learning: When validating/constructing Anthropic API model IDs in Significant-Gravitas/AutoGPT, allow the hyphen-separated Claude Opus 4.6 model ID `claude-opus-4-6` (it corresponds to `LlmModel.CLAUDE_4_6_OPUS` in `autogpt_platform/backend/backend/blocks/llm.py`). Do NOT require the dot-separated form in Anthropic contexts. Only OpenRouter routing variants should use the dot separator (e.g., `anthropic/claude-opus-4.6`); `claude-opus-4-6` should be treated as correct when passed to Anthropic, and flagged only if it’s used in the OpenRouter path where the dot form is expected.
Applied to files:
autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.pyautogpt_platform/backend/backend/blocks/baas/bots_cost_test.py
📚 Learning: 2026-04-15T02:43:36.890Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12780
File: autogpt_platform/backend/backend/copilot/tools/workspace_files.py:0-0
Timestamp: 2026-04-15T02:43:36.890Z
Learning: When reviewing Python exception handlers, do not flag `isinstance(e, X)` checks as dead/unreachable if the caught exception `X` is a subclass of the exception type being handled. For example, if `X` (e.g., `VirusScanError`) inherits from `ValueError` (directly or via an intermediate class) and it can be raised within an `except ValueError:` block, then `isinstance(e, X)` inside that handler is reachable and should not be treated as dead code.
Applied to files:
autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.pyautogpt_platform/backend/backend/blocks/baas/bots_cost_test.py
📚 Learning: 2026-04-22T11:46:04.431Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12881
File: autogpt_platform/backend/backend/copilot/config.py:0-0
Timestamp: 2026-04-22T11:46:04.431Z
Learning: Do not flag the Claude Sonnet 4.6 model ID as incorrect when it uses the project’s established hyphenated convention: `anthropic/claude-sonnet-4-6`. This hyphen form is the intentional, production convention and should be treated as valid (including in files like llm.py, blocks tests, reasoning.py, `_is_anthropic_model` tests, and config defaults). Note that OpenRouter also accepts the dot variant `anthropic/claude-sonnet-4.6`, so either form may be tolerated, but `anthropic/claude-sonnet-4-6` should be considered the standard to match project usage.
Applied to files:
autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.pyautogpt_platform/backend/backend/blocks/baas/bots_cost_test.py
📚 Learning: 2026-04-22T11:46:12.892Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12881
File: autogpt_platform/backend/backend/copilot/baseline/service.py:322-332
Timestamp: 2026-04-22T11:46:12.892Z
Learning: In this codebase (Significant-Gravitas/AutoGPT), OpenRouter-routed Anthropic model IDs should use the hyphen-separated convention (e.g., `anthropic/claude-sonnet-4-6`, `anthropic/claude-opus-4-6`). Although OpenRouter may accept both hyphen and dot variants, treat the hyphen-separated form as the intended, correct codebase-wide convention and do not flag it as an error. Only flag the dot-separated variant (e.g., `anthropic/claude-sonnet-4.6`) as incorrect when reviewing/validating model ID strings for OpenRouter-routed Anthropic models.
Applied to files:
autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.pyautogpt_platform/backend/backend/blocks/baas/bots_cost_test.py
🔇 Additional comments (1)
autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.py (1)
14-14: No action required. The importfrom backend.blocks._base import BlockCostTypeat line 14 is correct;BlockCostTypeis defined inbackend.blocks._baseand this import path is consistent with usage across the codebase.
## ClaudeCodeBlock (headline) Migrate from flat RUN 100 cr to COST_USD 150 cr/$. Claude Code CLI returns `total_cost_usd` on every response; block emits it via new `ClaudeCodeBlock._record_cli_cost` helper. ## Dynamic-pricing audit fixes (cost-leak surfaces) - **Exa websets** (~40 blocks): registered as COST_USD 100 cr/$ but never emitted provider_cost → wallet-free execution. Added `extract_exa_cost_usd` + `merge_exa_cost` helpers in exa/helpers.py, threaded through every Exa SDK call (14 files, 59 sites). Future-proof — will light up as soon as exa_py surfaces cost_dollars on webset response types. - **AIConditionBlock**: LLM block with token instrumentation but no BLOCK_COSTS entry. Registered under LLM_COST. - **Pinecone × 3**: Init/Query bill flat 1 cr RUN (platform overhead; user pays Pinecone directly). Insert bills ITEMS scaling with len(vectors) emitted via merge_stats. - **Perplexity Sonar (all 3 tiers)**: RUN 1/5/10 cr → COST_USD 150 cr/$. Block already extracted OpenRouter x-total-cost; extracted `_record_openrouter_cost` helper with None/0 guard (keeps the billing gap observable instead of silently billing 0). - **CodeGenerationBlock (Codex GPT-5.1-Codex)**: RUN 5 cr → COST_USD 150 cr/$. Block computes USD from response.usage input/output tokens using GPT-5.1-Codex rates. - **VideoNarrationBlock (ElevenLabs)**: RUN 5 cr → COST_USD 150 cr/$. Extracted `_record_script_cost` helper with model-aware per-char rate (Flash/Turbo v2.5 at 0.5 cr/char, Multilingual/Turbo v2 at 1.0 cr/char). - **BaaS FetchMeetingData**: flat RUN → COST_USD 150 cr/$. Block extracts duration_seconds from bot metadata and emits cost_usd scaled by published $0.69/hr rate. ## OpenRouter → COST_USD Migrate OpenRouter LLM models from TOKENS to COST_USD 150 cr/$ so billing uses OpenRouter's authoritative x-total-cost (no more per-model rate maintenance drift). LLM block tags provider_cost_type='cost_usd' when merging. ## TOKEN_COST refresh Re-sourced every entry against current provider pricing at uniform 1.5× margin. 35 entries updated; highlights: - Anthropic Opus 4.5/4.6: 2250/11250 → 750/3750 (provider dropped $15/$75 → $5/$25) - GPT-5 family: severely stale, holding GPT-4 rates (GPT-5 was over-billing input 4×) - Gemini 2.5 Pro: corrected to $1.25/$10 (was using 2.0 Pro rate) - Perplexity Sonar family: corrected upward (were under-priced 3-5×) - Mistral Medium 3.1: 405/1215 → 60/300 (was 5-6× over) ## Tests New / updated: `claude_code_cost_test.py`, `cost_leak_fixes_test.py`, `baas/bots_cost_test.py`, updated `helpers_test.py` for new COST_USD registrations. Full cost pipeline: 119/119 pass.
majdyz
force-pushed
the
feat/claude-code-dynamic-pricing
branch
from
April 24, 2026 13:44
b08a214 to
85bf625
Compare
…-coverage target - Extracted CodeGenerationBlock._compute_token_usd so the Codex USD computation is testable without running the full SDK path. - Added exa/helpers_cost_test.py covering extract_exa_cost_usd across all response shapes (dataclass, camelCase dict, snake_case dict, numeric string, nested total, invalid/missing — 12 variants) plus merge_exa_cost emission/no-op paths. - Rewrote the earlier tautological Codex test to call the real helper.
Contributor
Author
/pr-test --fix — PR #12909 cost-leak closures E2E verificationDate: 2026-04-24 Setup
Deep E2E — real DB + real
|
| # | Scenario | Expected | Actual | Result |
|---|---|---|---|---|
| S1 | OpenRouter LLM migrated TOKENS → COST_USD 150 ($0.02 × 150 = 3) | pre=0, delta=3, net=3 | pre=0, delta=3, net=3 | PASS |
| S2 | ClaudeCodeBlock new COST_USD 150 ($0.50 × 150 = 75) | pre=0, delta=75, net=75 | pre=0, delta=75, net=75 | PASS |
| S3 | Perplexity Sonar Deep Research migrated RUN 10 → COST_USD 150 ($0.25 × 150 ceil = 38) | pre=0, delta=38, net=38 | pre=0, delta=38, net=38 | PASS |
| S4a | Narration Flash v2.5 half-rate (5K chars × 0.5 × $0.000167 = $0.4175) | provider_cost=$0.4175 | $0.4175 | PASS |
| S4b | Narration Multilingual v2 full-rate (5K chars × 1.0 × $0.000167 = $0.835) | provider_cost=$0.835 | $0.8350 | PASS |
| S5a | Codex _compute_token_usd (100K input + 10K output) |
$0.225 | $0.2250 | PASS |
| S5b | Codex COST_USD E2E ($0.225 × 150 ceil = 34) | pre=0, delta=34, net=34 | pre=0, delta=34, net=34 | PASS |
| S6 | Pinecone Insert ITEMS (50 vectors × 1 cr = 50) | pre=0, delta=50, net=50 | pre=0, delta=50, net=50 | PASS |
Total: 8/8 PASS
API cross-check
Fetched /api/blocks against the running backend and confirmed the new cost-type registrations surface correctly:
ClaudeCodeBlock→cost_usd / 150✅PerplexityBlock(all 3 tiers: SONAR / SONAR_PRO / SONAR_DEEP_RESEARCH) →cost_usd / 150✅VideoNarrationBlock→cost_usd / 150✅AIConditionBlock→tokens / 21(LLM_COST default, was wallet-free before) ✅PineconeInsertBlock→items / 1✅
What this verifies
- OpenRouter billing switches correctly from per-model TOKEN_COST tiers to the authoritative
x-total-cost× 150 cr/$ path. - ClaudeCodeBlock, Perplexity, Codex all migrated RUN → COST_USD with the 150 cr/$ margin consistent across the board.
- Narration model-aware helper correctly scales Flash/Turbo v2.5 at half the per-char rate of Multilingual v2 — no more 2× over-bill on cheap-tier users.
- Pinecone Insert ITEMS scaling emits
len(vectors)and charges 1 cr/item. - AIConditionBlock is no longer wallet-free (was missing from BLOCK_COSTS entirely pre-PR).
- All COST_USD calculations use
ceil()— sub-cent spend ($0.001 × 150 = $0.15) still bills 1 cr instead of 0.
Environment
- Native stack on
/Users/majdyz/Code/AutoGPT7@6f3874a4d - Infra via docker compose (supabase, redis, rabbitmq, clamav)
- Test user:
test@test.com/ id89d070d0-…
github-project-automation
Bot
moved this from 🆕 Needs initial review
to ✅ Done
in AutoGPT development kanban
3 tasks
majdyz
added a commit
that referenced
this pull request
Apr 24, 2026
…te generic ReplicateModelBlock to COST_USD (#12912) ## Why PR #12909's pricing refresh was sourced from aggregators (pricepertoken, blog mirrors) instead of provider pricing pages. Follow-up audit against **official provider docs** caught **22 stale entries** — 9 LLM token rates + 12 non-LLM block rates + 1 block that needed a code refactor to bill dynamically. Also flagged by Sentry: Mistral models were sitting on the wrong provider's rate table. Cross-verified JS-rendered pages (docs.x.ai, DeepSeek, Kimi) via agent-browser. ## Corrections applied ### LLM TOKEN_COST (9 entries) | Model | Old | New | Reason | |---|---|---|---| | `GPT5` | 94/750 | **188/1500** | Was OpenAI Batch API rate; Standard is $1.25/$10 | | `DEEPSEEK_CHAT` | 42/63 | **21/42** | Unified to deepseek-v4-flash at $0.14/$0.28 (Sept 2025) | | `DEEPSEEK_R1_0528` | 82/329 | **21/42** | Same v4-flash routing | | `MISTRAL_LARGE_3` | 300/900 | **300/900** (restored after brief 75/225 detour) | Routes via OpenRouter ($2/$6), not Mistral direct | | `MISTRAL_NEMO` | 3/6 → 23/23 | **5/5** | Routes via OpenRouter ($0.035/$0.035); Mistral-direct $0.15 doesn't apply | | `KIMI_K2_0905` | 82/330 | **90/375** | Matches K2 family $0.60/$2.50 | | `KIMI_K2_5` | 90/450 | **66/300** | OpenRouter pass-through $0.44/$2 | | `KIMI_K2_6` | 143/600 | **112/698** | OpenRouter pass-through $0.7448/$4.655 | | `META_LLAMA_4_MAVERICK` | 30/90 | **75/116** | Groq $0.50/$0.77 (deprecated 2026-02-20) | ### Non-LLM BLOCK_COSTS — rate corrections (11 entries) Under-billing fixes: - `AIVideoGeneratorBlock` (FAL) SECOND 3 → **15 cr/s** - `CreateTalkingAvatarVideoBlock` (D-ID) RUN 15 → **100 cr** - Nano Banana Pro/2 across 3 blocks: RUN 14 → **21 cr** - `UnrealTextToSpeechBlock` RUN 5 → **COST_USD 150 cr/$** (block now emits `chars × $0.000016`) Over-billing fixes: - `IdeogramModelBlock` default 16 → **12**, V_3 18 → **14** - `AIImageEditorBlock` FLUX_KONTEXT_MAX 20 → **12** - `ValidateEmailsBlock` 250 → **150 cr/$** - `SearchTheWebBlock` 100 → **150 cr/$** - `GetLinkedinProfilePictureBlock` 3 → **1 cr** ### Non-LLM BLOCK_COSTS — block refactored for dynamic billing (1 entry) - **`ReplicateModelBlock`** (the generic "run any Replicate model" wrapper) migrated from flat RUN 10 cr → **COST_USD 150 cr/$**. Block now uses `client.predictions.async_create + async_wait` instead of `async_run(wait=False)` so it can read `prediction.metrics.predict_time` and bill `predict_time × $0.0014/s` (Nvidia L40S mid-tier, where most popular public models run). Additionally (addressing CodeRabbit's critical review on this refactor): `async_wait()` returns normally regardless of terminal status — it doesn't raise on `failed`/`canceled` like the old `async_run` did. The block now explicitly checks `prediction.status` after `async_wait()` and raises `RuntimeError` on `failed` (with `prediction.error` as context) or `canceled` **before** `merge_stats`, so failed runs are never billed for partial compute time. **Why this matters:** flat 10 cr was 10–500× under-billing long video/LLM runs (users could wire in a $50/hr A100 Llama inference and pay us $0.10). It was also 20× over-billing trivial SDXL runs. Now scales with real compute time AND no longer bills failed predictions. ### Documentation-only - **Grok legacy models** (grok-3, grok-4-0709, grok-4-fast, grok-code-fast-1): dropped from docs.x.ai's public pricing page but still callable via the API. Added inline comment noting this; rates kept at their verified launch pricing. - **Mistral routing**: added comment explaining why TOKEN_COST for MISTRAL_* is the OpenRouter safety floor (not Mistral-direct) since `ModelMetadata.provider = "open_router"` for all Mistral entries. ## How - For each entry, opened the **official provider pricing page** directly and computed `our_cr = round(1.5 × provider_usd × 100)`. - For JS-rendered pages (docs.x.ai, api-docs.deepseek.com), used agent-browser headless to render + extract rates from the DOM. - Migrated 2 blocks (`UnrealTextToSpeechBlock`, `ReplicateModelBlock`) from flat RUN to COST_USD — the Replicate migration touched the block's SDK interaction. - Updated 2 FAL-video unit tests that asserted the old `3 cr/s` rate. - Updated 3 stale test assertions: 2 for Unreal TTS (still on `characters` cost_type) + 1 for ZeroBounce (old 250 cr). ## Known remaining risk (explicitly out of scope) - **`ReplicateFluxAdvancedModelBlock`** not migrated — bounded to Flux models ($0.04–$0.08), flat 10 cr stays within 1.25–2.5× margin. Separate PR if desired. - **AgentMail** on free tier (1 RUN). When paid pricing publishes, revisit. - **Live Replicate API verification**: mitigated via 9 unit tests covering the refactored path (`async_create` version-vs-model branching, metrics-based billing emission, failed/canceled raises, zero/missing-metrics no-emission, `async_wait` ordering), and SDK signature confirmed via `inspect.signature` — but no real API call executed. A smoke test on a cheap model before merge is still recommended. ## Test plan - [x] `poetry run pytest backend/data/block_cost_config_test.py backend/executor/block_usage_cost_test.py backend/blocks/claude_code_cost_test.py backend/blocks/cost_leak_fixes_test.py backend/blocks/block_cost_tracking_test.py backend/copilot/tools/helpers_test.py backend/blocks/replicate/replicate_block_cost_test.py -q` — all passing (80+ tests). - [x] Sources: openai.com/api/pricing, claude.com/pricing, api-docs.deepseek.com, mistral.ai/pricing, platform.kimi.ai/docs/pricing, docs.x.ai, groq.com/pricing, replicate.com, fal.ai, d-id.com, ideogram.ai, zerobounce.net, jina.ai, unrealspeech.com, enrichlayer.com. - [ ] Live Replicate API call to verify `predictions.async_create + async_wait + metrics.predict_time` path.
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
ClaudeCodeBlockwas a flatRUN, 100 cr/runentry when real cost is $0.02–$1.50/run. Plugging that leak surfaced the question "are other blocks doing the same?" — an audit found 7 more cost-leak surfaces. This PR closes all of them atomically so the cost pipeline is uniform post-#12894.What
1. ClaudeCodeBlock → COST_USD 150 cr/$ (the headline)
Claude Code CLI's
--output-format jsonalready returnstotal_cost_usdon every call, rolling up Anthropic LLM + internal tool-call spend. Block now emits it viamerge_stats:Registered as
COST_USD, 150 cr/$— matches the 1.5× margin baked into everyTOKEN_COSTentry.2. Exa websets — ~40 blocks instrumented
Registered as
COST_USD 100 cr/$but never emittedprovider_cost→ ran wallet-free. Addedextract_exa_cost_usd+merge_exa_costhelpers inexa/helpers.pyand threadedmerge_exa_cost(self, response)through every Exa SDK call across 14 files (59 call sites). Future-proof: lights up as soon asexa_pysurfacescost_dollarson webset response types.3. AIConditionBlock — registered under LLM_COST
Full LLM block with token-count instrumentation already in place, but no
BLOCK_COSTSentry at all → wallet-free. One-line fix: added to the LLM_COST group next to AIConversationBlock.4. Pinecone × 3 — added BLOCK_COSTS
PineconeInitBlock+PineconeQueryBlock: 1 cr/run RUN (platform overhead; user pays Pinecone directly).PineconeInsertBlock: ITEMS scaling withlen(vectors)emitted viamerge_stats.5. Perplexity Sonar (all 3 tiers) → COST_USD 150 cr/$
Block already extracted OpenRouter's
x-total-costheader intoexecution_stats.provider_cost; just tagged itcost_usdand flipped the registry. Deep Research was under-billing up to 30× ($0.20–$2.00 real vs flat 10 cr).6. CodeGenerationBlock (Codex / GPT-5.1-Codex) → COST_USD 150 cr/$
Block computes USD from
response.usage.input_tokens / output_tokensusing GPT-5.1-Codex rates ($1.25/M in + $10/M out) and emitscost_usd. Was flat 5 cr for arbitrary-length generations.7. VideoNarrationBlock (ElevenLabs) → COST_USD 150 cr/$
Block computes USD from
len(script) × $0.000167(Starter tier per-char price) and emitscost_usd. Was under-billing ~25–30× on long scripts (5K-char narration: flat 5 cr vs ~$0.83 real = 125 cr).8. Meeting BaaS FetchMeetingData → COST_USD 150 cr/$
Join block keeps its flat 30 cr commit. FetchMeetingData now extracts
duration_secondsfrom the response metadata, computes USD viaduration × $0.000192/sec, and emitscost_usd. Long meetings (hours) no longer fit inside the 30 cr deposit.Why 150 cr/$
Matches the 1.5× margin already baked into
TOKEN_COSTfor every direct LLM block:Applying the same ratio to$0.01 → 100 cr/$ pass-through × 1.5× = 150).
total_cost_usd≡cost_amount=150(1 cr ≈Test plan
claude_code_cost_test.py(9 tests) + existingexa/cost_tracking_test.py(16 tests) + full cost pipeline. 119/119 pass.poetry run ruff format+poetry run ruff check backend/— clean.Follow-ups (not in this PR)
exa_pySDK update to surfacecost_dollarson Webset response types (upstream) — unlocks real billing for the 40 webset blocks.prediction.metrics["predict_time"] × per-model $/sec.