Add subprocess.Popen example

[Chesars]

Summary

• Add tabbed interface showing both subprocess.run and subprocess.Popen approaches
• Add "Understanding subprocess.Popen" info section with Python docs reference
• Link to OpenHands Software Agent SDK as production example

[klieret]

Thanks!

but what advantage does that have over run? we don't make use of the asynchornicity, right? because we do call join at the end (also seems quite a bit more complicated than the .run)

[Chesars]

You're right that if we just call wait() at the end, there's no advantage over run().

The advantages of Popen could be useful for:

1. Agent with parallel execution

import subprocess
from subprocess import PIPE

def execute_action(command: str) -> str:
    """Execute action, supports parallel commands with 'parallel:' prefix"""

    # Parallel execution: parallel:npm install && eslint .
    if command.startswith("parallel:"):
        commands = command[9:].split(" && ")
        processes = []

        # Start all processes
        for cmd in commands:
            proc = subprocess.Popen(
                cmd.strip(),
                shell=True,
                stdout=PIPE,
                stderr=subprocess.STDOUT,
                text=True
            )
            processes.append((cmd.strip(), proc))

        # Wait for all and collect output
        results = []
        for cmd, proc in processes:
            output, _ = proc.communicate()
            results.append(f"[{cmd}]\n{output}")

        return "\n---\n".join(results)

    # Normal blocking command
    result = subprocess.run(
        command,
        shell=True,
        text=True,
        stdout=PIPE,
        stderr=subprocess.STDOUT,
        timeout=30,
    )
    return result.stdout

The agent could run npm install and eslint simultaneously with:

parallel:npm install && eslint .

Output:

  [npm install]
  added 150 packages in 3s
  [eslint .]
  ✓ 

2. Agent with persistent shell session

import subprocess
from subprocess import PIPE

class PersistentShell:
    def __init__(self):
        self.proc = subprocess.Popen(
            ["bash"],
            stdin=PIPE,
            stdout=PIPE,
            stderr=subprocess.STDOUT,
            text=True,
            bufsize=1
        )

    def execute(self, command: str) -> str:
        end_marker = f"__END_{id(command)}__"
        self.proc.stdin.write(f"{command}\n")
        self.proc.stdin.write(f"echo '{end_marker}'\n")
        self.proc.stdin.flush()

        output = []
        for line in self.proc.stdout:
            if end_marker in line:
                break
            output.append(line)
        return "".join(output)

shell = PersistentShell()

def execute_action(command: str) -> str:
    return shell.execute(command)

# cd and env vars persist between commands

3. Agent with streaming output (see shell output)

import subprocess
from subprocess import PIPE

def execute_action(command: str) -> str:
    proc = subprocess.Popen(
        command,
        shell=True,
        stdout=PIPE,
        stderr=subprocess.STDOUT,
        text=True
    )

    output_lines = []
    for line in proc.stdout:
        output_lines.append(line)
        print(f"[live] {line}", end="")  # Agent sees output as it happens

        # React to specific patterns
        if "error" in line.lower():
            proc.kill()
            output_lines.append("\n[Agent: Detected error, stopping process]")
            break

        if len(output_lines) > 500:
            proc.kill()
            output_lines.append("\n[Agent: Output too long, stopping]")
            break

    proc.wait()
    return "".join(output_lines)

The agent sees npm install output line by line as packages install:

Installing dependencies...
added 50 packages
added 100 packages
npm warn deprecated package@1.0.0
added 150 packages in 3s

For the current minimal agent, subprocess.run() is indeed simpler and sufficient.

Popen would make sense if we wanted to add features mentioned above.