Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(agents-api): fix unwrap in prompt step #1093

Merged
merged 3 commits into from
Jan 25, 2025
Merged

Conversation

Ahmad-mtos
Copy link
Contributor

@Ahmad-mtos Ahmad-mtos commented Jan 25, 2025

PR Type

Bug fix, Tests


Description

  • Fixed logic in _handle_PromptStep to handle unwrap and auto_run_tools conditions.

  • Updated test case to reflect changes in _handle_PromptStep.

  • Improved code readability by refactoring conditional logic.


Changes walkthrough 📝

Relevant files
Bug fix
__init__.py
Refactor `_handle_PromptStep` logic for clarity                   

agents-api/agents_api/workflows/task_execution/init.py

  • Refactored conditional logic in _handle_PromptStep.
  • Moved choice assignment after conditional block.
  • Improved readability of unwrap and auto_run_tools checks.
  • +7/-3     
    Tests
    test_task_execution_workflow.py
    Update test case for `_handle_PromptStep` changes               

    agents-api/tests/test_task_execution_workflow.py

  • Updated test case to use a string message instead of a dictionary.
  • Adjusted test to align with _handle_PromptStep changes.
  • +1/-1     

    Need help?
  • Type /help how to ... in the comments thread for any questions about Qodo Merge usage.
  • Check out the documentation for more information.

  • Important

    Fix condition in _handle_PromptStep to correctly handle early return cases and update corresponding test.

    • Behavior:
      • Fix condition in _handle_PromptStep in __init__.py to return early with PartialTransition if step.unwrap is true, step.auto_run_tools is false, or finish_reason is not tool_calls.
    • Tests:
      • Update test case in test_task_execution_workflow.py to use a string for message instead of a dictionary, aligning with the new logic in _handle_PromptStep.

    This description was created by Ellipsis for c0e6c66. It will automatically update as commits are pushed.

    Copy link
    Contributor

    PR Reviewer Guide 🔍

    Here are some key observations to aid the review process:

    ⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
    🧪 PR contains tests
    🔒 No security concerns identified
    ⚡ Recommended focus areas for review

    Test Data Mismatch

    The test case was updated to use a string message "Hello there" but _handle_PromptStep expects a dictionary with "choices" key. This will cause the test to fail when accessing message["choices"].

    message = "Hello there"
    outcome = StepOutcome(output=message)
    Error Handling

    No error handling for cases where message dictionary is missing expected keys or has invalid structure. Could raise KeyError when accessing message["choices"] or choice["message"].

    message = self.outcome.output
    
    if (
        step.unwrap
        or not step.auto_run_tools
        or message["choices"][0]["finish_reason"] != "tool_calls"
    ):
        workflow.logger.debug(f"Prompt step: Received response: {message}")
        return PartialTransition(output=message)
    
    choice = message["choices"][0]
    tool_calls_input = choice["message"]["tool_calls"]

    Copy link
    Contributor

    PR Code Suggestions ✨

    Explore these optional code suggestions:

    CategorySuggestion                                                                                                                                    Score
    Possible issue
    Match test data with expected format

    The test message was changed from a dictionary to a string, but the code expects a
    dictionary with 'choices' key. Update the test message to match the expected format.

    agents-api/tests/test_task_execution_workflow.py [510-511]

    -message = "Hello there"
    +message = {"choices": [{"finish_reason": "stop"}]}
     outcome = StepOutcome(output=message)
    • Apply this suggestion
    Suggestion importance[1-10]: 9

    Why: Critical bug fix that prevents test failures. The code in _handle_PromptStep expects a dictionary with 'choices' key, but the test provides a string, which would cause runtime errors.

    9

    Copy link
    Contributor

    @ellipsis-dev ellipsis-dev bot left a comment

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    👍 Looks good to me! Reviewed everything up to c0e6c66 in 35 seconds

    More details
    • Looked at 37 lines of code in 2 files
    • Skipped 0 files when reviewing.
    • Skipped posting 1 drafted comments based on config settings.
    1. agents-api/tests/test_task_execution_workflow.py:510
    • Draft comment:
      The message should be a dictionary with a 'choices' key, not a string. This is inconsistent with the expected structure in _handle_PromptStep.
    • Reason this comment was not posted:
      Decided after close inspection that this draft comment was likely wrong and/or not actionable:
      The test case is specifically testing the unwrap=True scenario of a prompt step. Looking at the test case, it's verifying that when unwrap is True, the raw message is passed through without any special handling. The test passes with the string message, suggesting that this is actually the intended behavior. The comment seems to be making assumptions about required message structure that don't apply in this unwrap=True case.
      I could be wrong about the intended behavior of unwrap=True. Maybe there are requirements about message structure that are documented elsewhere or enforced in the actual implementation.
      The test clearly shows that with unwrap=True, the message is passed through directly in the assertion on line 518. If there were requirements about message structure, the test would fail. The fact that this test passes suggests this is working as intended.
      The comment should be deleted. The test is specifically for the unwrap=True case where the raw message is passed through, and the string message is valid in this context.

    Workflow ID: wflow_2UobB6IFpM0EQY9e


    You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

    Copy link
    Contributor

    qodo-merge-pro-for-open-source bot commented Jan 25, 2025

    CI Feedback 🧐

    (Feedback updated until commit c0e6c66)

    A test triggered by this PR failed. Here is an AI-generated analysis of the failure:

    Action: Typecheck

    Failed stage: Typecheck [❌]

    Failed test name: agents_api.common.protocol.tasks

    Failure summary:

    The pytype static type checker failed due to two errors in the file
    agents_api/common/protocol/tasks.py:

  • Line 209: Type error where list_execution_transitions object is being called but is not callable
  • Line 210: Attribute error trying to access .id on a None value in self.execution_input.execution.id

  • Relevant error logs:
    1:  ##[group]Operating System
    2:  Ubuntu
    ...
    
    1222:  [13/299] check agents_api.autogen.Entries
    1223:  [14/299] check agents_api.autogen.Tasks
    1224:  [15/299] check agents_api.autogen.Agents
    1225:  [16/299] check agents_api.common.utils.datetime
    1226:  [17/299] check agents_api.common.protocol.developers
    1227:  [18/299] check agents_api.common.utils.db_exceptions
    1228:  [19/299] check agents_api.common.protocol.agents
    1229:  [20/299] check agents_api.queries.utils
    1230:  ERROR:pytype.matcher Invalid type: <class 'pytype.abstract.function.ParamSpecMatch'>
    ...
    
    1332:  [122/299] check agents_api.queries.docs.__init__
    1333:  [123/299] check agents_api.activities.utils
    1334:  [124/299] check agents_api.queries.developers.__init__
    1335:  [125/299] check agents_api.queries.agents.__init__
    1336:  [126/299] check agents_api.common.protocol.remote
    1337:  [127/299] check agents_api.queries.__init__
    1338:  [128/299] check agents_api.clients.integrations
    1339:  [129/299] check agents_api.common.protocol.tasks
    1340:  FAILED: /home/runner/work/julep/julep/agents-api/.pytype/pyi/agents_api/common/protocol/tasks.pyi 
    1341:  /home/runner/work/julep/julep/agents-api/.venv/bin/python -m pytype.main --disable pyi-error --imports_info /home/runner/work/julep/julep/agents-api/.pytype/imports/agents_api.common.protocol.tasks.imports --module-name agents_api.common.protocol.tasks --platform linux -V 3.12 -o /home/runner/work/julep/julep/agents-api/.pytype/pyi/agents_api/common/protocol/tasks.pyi --analyze-annotated --nofail --none-is-not-bool --quick --strict-none-binding /home/runner/work/julep/julep/agents-api/agents_api/common/protocol/tasks.py
    1342:  /home/runner/work/julep/julep/agents-api/agents_api/common/protocol/tasks.py:209:29: error: in get_inputs: 'agents_api.queries.executions.list_execution_transitions' object is not callable [not-callable]
    ...
    
    1345:  execution_id=self.execution_input.execution.id,
    1346:  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    1347:  limit=1000,
    1348:  ~~~~~~~~~~~~~~~~~~~~~~~
    1349:  direction="asc",
    1350:  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    1351:  )
    1352:  ~~~~~~~~~
    1353:  /home/runner/work/julep/julep/agents-api/agents_api/common/protocol/tasks.py:210:26: error: in get_inputs: No attribute 'id' on None [attribute-error]
    1354:  In Optional[agents_api.autogen.Executions.Execution]
    1355:  execution_id=self.execution_input.execution.id,
    1356:  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    1357:  For more details, see https://google.github.io/pytype/errors.html
    ...
    
    1457:  [229/299] check agents_api.queries.chat.__init__
    1458:  [230/299] check tests.sample_tasks.test_find_selector
    1459:  [231/299] check tests.__init__
    1460:  [232/299] check tests.test_nlp_utilities
    1461:  [233/299] check agents_api.metrics.__init__
    1462:  [234/299] check agents_api.common.exceptions.sessions
    1463:  [235/299] check agents_api.clients.__init__
    1464:  [236/299] check agents_api.rec_sum.entities
    1465:  ninja: build stopped: cannot make progress due to previous errors.
    1466:  Computing dependencies
    1467:  Generated API key since not set in the environment: 72670854398539784241069788237269
    1468:  Sentry DSN not found. Sentry will not be enabled.
    1469:  Analyzing 289 sources with 0 local dependencies
    1470:  Leaving directory '.pytype'
    1471:  ##[error]Process completed with exit code 1.
    

    @whiterabbit1983 whiterabbit1983 merged commit f49ea57 into dev Jan 25, 2025
    11 of 15 checks passed
    @whiterabbit1983 whiterabbit1983 deleted the x/prompt-step-hotfix branch January 25, 2025 08:33
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    2 participants