Skip to content

Conversation

JakeCowton
Copy link

@JakeCowton JakeCowton commented Sep 4, 2025

Currently the SQL generated for recommended questions does not check if the SQL is valid.

This PR runs a configurable number of retries to fix any invalid SQL, bringing it inline with how SQL for regular asks are generated.

Summary by CodeRabbit

  • New Features
    • Added automatic SQL correction retries in the Ask flow to recover from invalid SQL generations and improve answer success rates.
    • Introduced a configurable retry limit for SQL correction, allowing fine-tuning of reliability and performance.
    • Enhanced robustness of question recommendations without changing existing user workflows or UI.

Copy link
Contributor

coderabbitai bot commented Sep 4, 2025

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check ✅ Passed The title concisely and accurately summarizes the primary change—adding validation for SQL generated for recommended questions—using a conventional-commit prefix and clear wording that matches the PR objectives and code changes (SQL correction pipeline and retry configuration).
✨ Finishing touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d02d54f and b6a0686.

📒 Files selected for processing (1)
  • wren-ai-service/src/globals.py (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • wren-ai-service/src/globals.py

Tip

👮 Agentic pre-merge checks are now available in preview!

Pro plan users can now enable pre-merge checks in their settings to enforce checklists before merging PRs.

  • Built-in checks – Quickly apply ready-made checks to enforce title conventions, require pull request descriptions that follow templates, validate linked issues for compliance, and more.
  • Custom agentic checks – Define your own rules using CodeRabbit’s advanced agentic capabilities to enforce organization-specific policies and workflows. For example, you can instruct CodeRabbit’s agent to verify that API documentation is updated whenever API schema files are modified in a PR. Note: Upto 5 custom checks are currently allowed during the preview period. Pricing for this feature will be announced in a few weeks.

Please see the documentation for more information.

Example:

reviews:
  pre_merge_checks:
    custom_checks:
      - name: "Undocumented Breaking Changes"
        mode: "warning"
        instructions: |
          Pass/fail criteria: All breaking changes to public APIs, CLI flags, environment variables, configuration keys, database schemas, or HTTP/GraphQL endpoints must be documented in the "Breaking Change" section of the PR description and in CHANGELOG.md. Exclude purely internal or private changes (e.g., code not exported from package entry points or explicitly marked as internal).

Please share your feedback with us on this Discord post.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (3)
wren-ai-service/src/web/v1/services/question_recommendation.py (3)

34-35: Validate and clamp retry count

Prevent negative or non-int values from misconfigurations.

Apply:

-        max_sql_correction_retries: int = 3,
+        max_sql_correction_retries: int = 3,

And assign (see Line 43):

-        self._max_sql_correction_retries = max_sql_correction_retries
+        # Clamp and coerce
+        self._max_sql_correction_retries = max(0, int(max_sql_correction_retries))

43-43: Use clamped value for internal state — see suggested diff above

Stores the validated count; no other changes needed.


132-167: Refactor SQL correction loop in question_recommendation.py for readability and parity

  • Replace the while … else with a for attempt in range(1, self._max_sql_correction_retries + 1): loop and emit a logger.debug("Request %s: SQL correction attempt %d/%d", request_id, attempt, self._max_sql_correction_retries) before each retry.
  • Include the same dry-plan flags used in the ask flow—add use_dry_plan=ask_request.use_dry_plan and allow_dry_plan_fallback=ask_request.allow_dry_plan_fallback to the sql_correction.run(…) invocation.
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 8e32746 and d02d54f.

📒 Files selected for processing (2)
  • wren-ai-service/src/globals.py (1 hunks)
  • wren-ai-service/src/web/v1/services/question_recommendation.py (3 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
wren-ai-service/src/web/v1/services/question_recommendation.py (2)
wren-ai-service/src/pipelines/generation/sql_correction.py (2)
  • post_process (97-111)
  • run (146-174)
wren-ai-service/src/pipelines/generation/utils/sql.py (1)
  • run (29-69)
🔇 Additional comments (4)
wren-ai-service/src/globals.py (3)

223-223: Addition of sql_correction to QuestionRecommendation pipelines — LGTM

This unblocks the correction loop in QuestionRecommendation.


226-226: Settings.max_sql_correction_retries defined and documented
Settings.max_sql_correction_retries is declared with a default of 3 in src/config.py and is included in all example configs and docs.


216-218: Remove invalid fallback suggestion
The proposed fallback to pipe_components["sql_generation"] is incorrect—no sql_generation component is defined in any of the configuration files (they only declare question_recommendation and question_recommendation_sql_generation) so that lookup would itself raise a KeyError.

Likely an incorrect or invalid review comment.

wren-ai-service/src/web/v1/services/question_recommendation.py (1)

169-170: Valid-SQL assignment path — LGTM

Happy path uses the initial valid result; consistent with the new correction path.

@cyyeh
Copy link
Member

cyyeh commented Sep 5, 2025

@JakeCowton thanks for contribution, I'll take a look in a few days

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants