Skip to content

Add integration test for upserting graph in StateManager#353

Merged
NiveditJain merged 4 commits intoexospherehost:mainfrom
NiveditJain:adding-actual-integration-tests
Sep 4, 2025
Merged

Add integration test for upserting graph in StateManager#353
NiveditJain merged 4 commits intoexospherehost:mainfrom
NiveditJain:adding-actual-integration-tests

Conversation

@NiveditJain
Copy link
Member

  • Introduced a new test case in test_upsert_graph.py to validate the upsert_graph functionality of the StateManager.
  • Implemented a PrintNode class to handle input messages and verify the execution of graph nodes.
  • Utilized asyncio for non-blocking runtime management and ensured proper cleanup of tasks after execution.

- Introduced a new test case in test_upsert_graph.py to validate the upsert_graph functionality of the StateManager.
- Implemented a PrintNode class to handle input messages and verify the execution of graph nodes.
- Utilized asyncio for non-blocking runtime management and ensured proper cleanup of tasks after execution.
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Sep 4, 2025

Caution

Review failed

The pull request is closed.

📝 Walkthrough

Summary by CodeRabbit

  • Tests
    • Added integration tests validating graph upsert behavior, ensuring responses are returned and marked valid.
    • Introduced end-to-end fan-out test that starts a runtime, upserts a simple graph, triggers a run, and verifies completion via the API.
    • Confirms run metrics (success/total counts, zero errors/retries) and overall success status.
    • Improves coverage for runtime startup, graph creation, execution, and result retrieval scenarios.

Summary by CodeRabbit

  • Tests
    • Added end-to-end integration tests that validate creating/upserting graphs and executing them via a running runtime: one test for a single-node upsert (test_upsert_graph.py) and one test that upserts a graph, triggers a run, and validates fan-out behavior and run results (test_fan_out.py).
    • Tests exercise StateManager.upsert_graph/get_graph and the runtime REST API, improving coverage for graph lifecycle and run reporting.

Walkthrough

Adds two asynchronous integration tests: one upserts a single-node graph ("test_graph") using a PrintNode and asserts validation; the other upserts a graph with a node that emits 10 outputs, triggers a run, and validates run results via the runtime API.

Changes

Cohort / File(s) Summary
New integration tests
integration-tests/test_upsert_graph.py, integration-tests/test_fan_out.py
Adds test_upsert_graph (defines PrintNode, starts a Runtime, calls StateManager.upsert_graph with a single GraphNodeModel, asserts non-None result and validation status "VALID"). Adds test_fan_out (defines Node1 with Inputs/Outputs, starts a Runtime, upserts test_graph, triggers a run, polls runtime API to retrieve run info, and asserts run metrics: success_count=10, total_count=10, pending/errored/retried=0, status="SUCCESS"). Both use running_server fixture, run the Runtime in a daemon thread, and rely on async pytest support.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor Test
  participant Runtime
  participant StateManager
  participant Node

  rect rgba(200,230,255,0.35)
  Test->>Runtime: start runtime (daemon thread)
  Test-->>Test: await short sleep (init)
  end

  rect rgba(220,255,220,0.35)
  Test->>StateManager: upsert_graph("test_graph", [GraphNodeModel(Node, inputs)])
  StateManager->>Runtime: register/translate graph
  Runtime->>Node: register/instantiate node
  Node->>Node: execute(inputs) (async)
  StateManager-->>Test: return upsert result (validation status)
  end

  rect rgba(255,245,215,0.35)
  Test->>Runtime: trigger run (test_fan_out)
  Runtime->>Node: fan-out execution (produce 10 outputs)
  Runtime-->>Test: run_id
  Test->>Runtime: poll runs API -> retrieve run details
  Runtime-->>Test: run status and metrics
  end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Poem

In burrows soft my whiskers twirl,
I spin a node and watch it whirl.
I print and fan the outputs bright,
Tests pass under the moonlit night.
A rabbit claps — the graph is right 🐇✨


📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between f425aca and a6e9d30.

📒 Files selected for processing (2)
  • integration-tests/test_fan_out.py (1 hunks)
  • integration-tests/test_upsert_graph.py (1 hunks)
✨ Finishing Touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

  • Add @coderabbitai ignore or @coderabbit ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Status, Documentation and Community

  • Visit our Status Page to check the current availability of CodeRabbit.
  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary of Changes

Hello @NiveditJain, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new integration test to ensure the correct functionality of the StateManager's upsert_graph method. The test validates the ability to dynamically add or update graph configurations within the system, utilizing asynchronous operations for efficient execution and proper resource management.

Highlights

  • New Integration Test: A new test case test_upsert_graph has been added to validate the upsert_graph functionality of the StateManager.
  • PrintNode Implementation: A PrintNode class was implemented to simulate graph node execution and handle input messages within the test.
  • Asynchronous Runtime Management: The test leverages asyncio for non-blocking runtime management and includes robust cleanup mechanisms for asynchronous tasks.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in issue comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds an integration test for StateManager.upsert_graph. The test is a good foundation. My feedback provides a single, comprehensive suggestion to refactor the test to be more robust and complete. The proposed changes include using public APIs, making the test less reliant on fixed waits, and adding verification for the end-to-end execution of the graph, which significantly increases the test's value.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 2dcd442 and 14cceb0.

📒 Files selected for processing (1)
  • integration-tests/test_upsert_graph.py (1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
integration-tests/test_upsert_graph.py (5)
python-sdk/exospherehost/node/BaseNode.py (1)
  • BaseNode (6-100)
python-sdk/exospherehost/runtime.py (2)
  • Runtime (46-461)
  • _start (431-445)
python-sdk/exospherehost/statemanager.py (1)
  • upsert_graph (128-188)
python-sdk/exospherehost/models.py (1)
  • GraphNodeModel (16-75)
integration-tests/conftest.py (2)
  • running_server (99-107)
  • base_url (93-95)
🔇 Additional comments (2)
integration-tests/test_upsert_graph.py (2)

57-64: Remove task-cancel cleanup when not starting the runtime

If you adopt the registration-only approach above, this block is dead code.

-    finally:
-        # Ensure proper cleanup of the runtime task
-        if runtime_task and not runtime_task.done():
-            runtime_task.cancel()
-            try:
-                await runtime_task
-            except asyncio.CancelledError:
-                pass  # Expected when cancelling
+    finally:
+        pass

Likely an incorrect or invalid review comment.


4-4: Imports valid: exospherehost/__init__.py exports required symbols
BaseNode, Runtime, StateManager, and GraphNodeModel are all imported and included in __all__, so the top-level imports in the test are correct.

- Replaced asyncio task with a threading approach for starting the runtime, ensuring proper initialization without blocking.
- Simplified the upsert_graph test case by removing unnecessary cleanup logic related to asyncio tasks.
- Maintained the functionality of the test while improving the structure and readability.
@coderabbitai coderabbitai bot added the enhancement New feature or request label Sep 4, 2025
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

♻️ Duplicate comments (5)
integration-tests/test_upsert_graph.py (5)

10-15: Drop type ignore; assert inputs are set

Assert self.inputs instead of using a type ignore.

         async def execute(self):
-            print(self.inputs.message) # type: ignore
+            assert self.inputs is not None
+            print(self.inputs.message)

18-19: Use consistent naming: state_manager_url

Align variable naming with API args.

-    state_machine_url = running_server.base_url
+    state_manager_url = running_server.base_url
@@
-        state_manager_uri=state_machine_url,
+        state_manager_uri=state_manager_url,
@@
-        state_manager_uri=state_machine_url,
+        state_manager_uri=state_manager_url,

Also applies to: 25-25, 35-35


1-4: Import uuid for unique graph names

Prevents CI collisions across runs.

 import pytest
 import asyncio
+import uuid
 from pydantic import BaseModel

7-15: This test doesn’t verify node execution

If execution coverage is desired, add a separate test that calls StateManager.trigger(...) and asserts a node-completed signal/output.

Would you like a follow-up test that triggers the graph and asserts PrintNode ran using stdout capture or runtime status?


28-31: Replace thread+sleep with deterministic registration to avoid flakiness and leaks

Start-up threading and fixed sleep are brittle; directly register nodes for validation.

-    thread = threading.Thread(target=runtime.start, daemon=True)
-    thread.start()
-
-    await asyncio.sleep(2)
+    # Deterministically register node schemas; no worker loop needed for upsert
+    await runtime._register()
📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 14cceb0 and f425aca.

📒 Files selected for processing (1)
  • integration-tests/test_upsert_graph.py (1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
integration-tests/test_upsert_graph.py (5)
python-sdk/exospherehost/node/BaseNode.py (1)
  • BaseNode (6-100)
python-sdk/exospherehost/runtime.py (1)
  • Runtime (46-461)
python-sdk/exospherehost/statemanager.py (2)
  • StateManager (9-188)
  • upsert_graph (128-188)
python-sdk/exospherehost/models.py (1)
  • GraphNodeModel (16-75)
integration-tests/conftest.py (2)
  • running_server (99-107)
  • base_url (93-95)

- Introduced a new test case in test_fan_out.py to validate the fan-out behavior of the StateManager.
- Implemented a Node1 class to handle input messages and return a count of executions.
- Utilized threading for runtime management and ensured proper initialization without blocking.
- Enhanced assertions to verify the success of the graph execution and the correctness of run details.
@NiveditJain NiveditJain merged commit 1d9c4e9 into exospherehost:main Sep 4, 2025
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant