Skip to content

fix(sandbox): scrub MYSQL_PWD and REDISCLI_AUTH from the inherited skill env - #4018

Merged
WillemJiang merged 2 commits into
bytedance:mainfrom
AnnaSuSu:fix/sandbox-env-scrub-db-password-vars
Jul 9, 2026
Merged

WillemJiang merged 2 commits into
bytedance:mainfrom
AnnaSuSu:fix/sandbox-env-scrub-db-password-vars

Conversation

@AnnaSuSu

@AnnaSuSu AnnaSuSu commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Why

sandbox/env_policy.py (added in #3871 for #3861) scrubs secret-looking variables out of the environment a skill subprocess inherits, so that platform credentials in the Gateway's os.environ cannot simply be read by skill scripts. Its exact-name denylist already blocks the MySQL and Redis connection strings — MYSQL_URL, REDIS_URL — on the grounds that they embed a password.

It does not block the password variables those same clients read directly:

  • mysql / libmysqlclient takes MYSQL_PWD as the password with no flag and no config file.
  • redis-cli takes REDISCLI_AUTH the same way.

Neither name contains KEY / SECRET / TOKEN / PASSWORD / PASSWD / CREDENTIAL / DSN, so no wildcard in _SECRET_NAME_PATTERNS catches them, and neither is in _BLOCKED_EXACT_NAMES. Both are therefore inherited by every skill subprocess:

>>> os.environ["MYSQL_PWD"] = "prod-db-password"
>>> "MYSQL_PWD" in build_sandbox_env()
True

A skill script — or the agent's own bash tool, whose environment is built by the same build_sandbox_env() — can read the operator's database password with echo $MYSQL_PWD. That is the leak surface this module was written to close, for a service whose URL form the module already withholds.

This is a gap in a defence-in-depth layer, not a default-deployment vulnerability: it only bites when the Gateway process environment actually carries one of these variables. Raising it here rather than through a private advisory follows how this project has handled comparable sandbox environment-leak hardening in the open (#2534, #3922).

What changed

Three names are added to _BLOCKED_EXACT_NAMES: MYSQL_PWD, REDISCLI_AUTH, and REDIS_AUTH (the common non-canonical variant). Nothing else changes — benign variables are untouched, and injected request-scoped secrets still win over the filter.

They need exact entries rather than a pattern, in keeping with the existing design: a *PWD* wildcard would also strip PWD and OLDPWD, and *AUTH* is similarly too broad. (*PASSWORD* / *PASSWD* already cover PGPASSWORD, MYSQL_PASSWORD, REDIS_PASSWORD, ...)

From a caller's perspective: a skill that legitimately needs one of these values must declare it via required-secrets, exactly as it already must for DATABASE_URL — the caller then supplies it through context.secrets and the injected value wins.

Surface area

  • Sandbox — sandboxed execution (backend/packages/harness/deerflow/sandbox/env_policy.py)

Bug fix verification

  • Test path: backend/tests/test_skill_request_scoped_secrets.py::TestEnvPolicy::test_db_password_vars_do_not_reach_the_subprocess_env
  • Did it go red on main and green on this branch? yes — red on main with AssertionError: assert 'MYSQL_PWD' not in {...}; green on this branch.
  • The existing test_secret_like_names_are_blocked parametrization gains the three names (red on main for each). OLDPWD is added to test_benign_names_are_allowed to pin why a *PWD* wildcard is not an acceptable fix.

Validation

  • cd backend && make lint — All checks passed!, 797 files already formatted.
  • cd backend && make test — 6749 passed, 26 skipped, 8 failed. The 8 failures are pre-existing and unrelated: test_browserless_client.py, test_crawl4ai_tools.py and test_fastcrw_tools.py web_fetch cases fail identically on a clean main checkout in my environment, because it has no outbound DNS and the SSRF guard therefore rejects https://example.com with Error: Refusing to fetch a private, loopback, or metadata address. Baseline on main: 6744 passed / same 8 failed. The delta of +5 passing is exactly the 5 test cases added here.
  • Adjacent suites (test_sandbox_tools_security.py, test_channel_user_id_env.py, test_aio_sandbox.py): 162 passed.

AI assistance

Tool(s) used: Claude Code

How you used it: Claude Code assisted with locating the root cause and writing the fix and the regression tests. The contributor reviewed every line and takes responsibility for it.

  • I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.

…ill env

env_policy blocks the MySQL and Redis connection strings (MYSQL_URL, REDIS_URL)
because they embed a password, but not the password variables the same clients
read directly: mysql/libmysqlclient takes MYSQL_PWD and redis-cli takes
REDISCLI_AUTH with no further configuration. Both names carry no
KEY/SECRET/TOKEN/PASSWORD/PASSWD substring, so they were inherited by every
skill subprocess -- the leak surface this module exists to close (bytedance#3861).

They need exact entries rather than a pattern: *PWD* would also strip PWD and
OLDPWD. REDIS_AUTH is included as the common non-canonical variant.
@github-actions github-actions Bot added area:sandbox Sandboxed execution and docker/ risk:high High risk: backend API, agents, sandbox, auth, deps, CI size/S PR changes 20-100 lines labels Jul 9, 2026

@willem-bd willem-bd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A clean, well-scoped security hardening. The gap is real — MYSQL_PWD and REDISCLI_AUTH slip every wildcard in _SECRET_NAME_PATTERNS, so they were being inherited by skill subprocesses and the agent's own bash tool (same build_sandbox_env()). The exact-entry rationale is correct (*PWD* would strip PWD/OLDPWD), the required-secrets escape hatch is intact, and the tests are strong (parametrized red→green plus a dedicated build_sandbox_env() path test pinning PWD survival).

Approve — the three inline notes are minor polish / a follow-up thought, none blocking. PR body is excellent on threat model and verification.

# (the caller then supplies it through context.secrets, and injection wins).
#
# The same reasoning covers the password variables those clients read directly:
# ``MYSQL_PWD`` and ``REDISCLI_AUTH`` are the documented no-flag credential

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REDIS_AUTH provenance — worth distinguishing it from the other two.

MYSQL_PWD and REDISCLI_AUTH are the documented no-flag credential sources for mysql and redis-cli (the PR body is precise about this). REDIS_AUTH, however, is not a canonical env var for any standard Redis client — redis-cli reads REDISCLI_AUTH. It is used by various client libs and Helm/deployment setups, so adding it defensively is reasonable, but the comment here currently lumps all three under "the documented no-flag credential sources for mysql and redis-cli," which overstates REDIS_AUTH. A short inline note distinguishing the two canonical vars from the defensive third would keep the comment honest (and help a future reader deciding whether to prune it).

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Fixed in ead0a99 — the comment now says plainly that MYSQL_PWD and REDISCLI_AUTH are the documented no-flag credential sources for mysql and redis-cli, while REDIS_AUTH is not canonical for any standard Redis client and is blocked defensively because client libraries and deployment charts commonly set it. That also gives a future reader what they need to decide whether to prune it.


assert is_blocked_env_name(name) is False

def test_db_password_vars_do_not_reach_the_subprocess_env(self, monkeypatch):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: assert injection still wins for these specific names.

The general "injected secret wins over scrub" contract is tested in test_build_sandbox_env_scrubs_inherited_and_layers_injected, so this is low priority — but a single assertion that build_sandbox_env(injected={"MYSQL_PWD": "v"})["MYSQL_PWD"] == "v" would make the escape-hatch guarantee explicit for the exact names this PR introduces, closing the loop between "we scrub it" and "a skill can still get it via required-secrets."

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in ead0a99 as test_injection_still_wins_for_the_newly_blocked_names: with the host env setting MYSQL_PWD, build_sandbox_env(injected={"MYSQL_PWD": ...}) must return the request-scoped value. That closes the loop between "we scrub it" and "a skill can still get it via required-secrets", and it also pins the per-user-key-overrides-shared-key case from #3861.

"GITHUB_PAT",
"MYSQL_PWD",
"REDISCLI_AUTH",
"REDIS_AUTH",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up thought (out of scope here): PGPASSFILE.

PostgreSQL's PGPASSFILE (path to a .pgpass file) is the same shape of gap — it carries no pattern substring (KEY/SECRET/TOKEN/PASSWORD/PASSWD/CREDENTIAL/DSN) and slips the filter. It's a path, not a credential, so exploiting it needs a file read inside the sandbox (which the sandbox restricts) — correctly out of scope for this PR. Worth a tracked follow-up so it's not forgotten, rather than blocking this merge.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, and agreed it is out of scope here. One wrinkle worth carrying into the follow-up: PGPASSFILE is a path, and libpq falls back to ~/.pgpass when it is unset — so scrubbing the variable alone would not close the hole. The follow-up is probably "scrub the var and decide what the sandbox should expose of the host home directory", rather than one more denylist entry. Happy to open it once this lands.

- Distinguish REDIS_AUTH (defensive, non-canonical) from MYSQL_PWD and
  REDISCLI_AUTH (documented no-flag credential sources) in the comment.
- Pin that request-scoped injection still overrides the host value for the
  newly blocked names, keeping required-secrets a working escape hatch.
@WillemJiang WillemJiang added this to the 2.1.0 milestone Jul 9, 2026
@WillemJiang
WillemJiang merged commit c11713c into bytedance:main Jul 9, 2026
19 of 20 checks passed
jtaynl added a commit to jtaynl/deer-flow that referenced this pull request Jul 10, 2026
…y rebuilt

488ec17 -> 9097642. Migration-free (head stays 0003), no compose/Dockerfile/
frontend changes, zero conflicts, all carried patches intact.

Notable: bytedance#3996 adds memory consolidation, which is LOSSY (source fact content is
permanently replaced, only consolidatedFrom IDs survive). It defaults to false
and we do not set the key, so we inherit false — verified against the new image,
and the code gates it at both prompt-build and apply time. Keep it off.

bytedance#4019 auto-promote is gated on tool_search.enabled (false here). bytedance#4012 fixes the
redis stream bridge (we run in-memory). bytedance#4018 scrubs MYSQL_PWD/REDISCLI_AUTH
from the skill env.

Verified 3a/3b/3c incl. a live Aio bash-smoke, since bytedance#4009/bytedance#4017 touch the
tool-output path the LGI pipeline depends on.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
WillemJiang pushed a commit that referenced this pull request Jul 10, 2026
…rom skill env (#4026)

* fix(sandbox): scrub abbreviated *_PASS vars and Postgres PGPASSFILE from skill env

env_policy blocks the full PASSWORD/PASSWD spellings but not the ubiquitous
abbreviated _PASS form (DB_PASS, SMTP_PASS, MYSQL_PASS, ...), whose value is the
plaintext password itself, so every skill subprocess inherited it. Postgres's
file-based credential sources PGPASSFILE (.pgpass locator) and PGSERVICEFILE
(pg_service.conf, may carry a password) were likewise missed -- the psql analog
of the MYSQL_PWD/REDISCLI_AUTH no-flag sources #4018 added.

Collapse *PASSWORD*/*PASSWD* into a single *PASS* pattern (which subsumes both,
covers the abbreviated family, and catches PGPASSFILE) and add PGSERVICEFILE to
the exact-name denylist. *PASS* deliberately does not touch PWD/OLDPWD, which
carry no PASS substring. Capability-handle vars (KUBECONFIG, DOCKER_CONFIG,
GIT_ASKPASS, ...) are a broader, non-value-bearing class and are left out of
this fix.

test_secret_like_names_are_blocked already pins the contract; the new names are
added to its parametrize list (red before, green after) and the benign-names
test guards against over-scrubbing PWD/OLDPWD.

* test(sandbox): pin the *_ASKPASS over-scrub that *PASS* introduces

*PASS* also matches the credential-helper vars GIT_ASKPASS, SSH_ASKPASS and
SUDO_ASKPASS, which main inherits. Each names a program whose purpose is to hand
the caller a credential, so inheriting the pointer is the same leak class as
inheriting the value -- a skill can simply invoke it. Scrubbing them is intended;
pin all three in test_secret_like_names_are_blocked (red on main) so the
behaviour is a decision rather than a side effect of the pattern's shape.

test_benign_names_are_allowed could not have caught this: none of its 14 names
carries a PASS substring, so it passes identically before and after the
broadening and asserts nothing about it. Record that in its docstring -- the
absence of a benign PASS-bearing name is a decision, and PWD/OLDPWD are the
boundary the list does pin.

Also note the incidental COMPASS_*/BYPASS_* over-scrub in the module comment
(over-scrubbing is this module's fail-safe direction; required-secrets is the
escape hatch), and list PGPASSFILE alongside PGSERVICEFILE in AGENTS.md.
wangzhengxi pushed a commit to wangzhengxi/deer-flow that referenced this pull request Jul 14, 2026
…rom skill env (bytedance#4026)

* fix(sandbox): scrub abbreviated *_PASS vars and Postgres PGPASSFILE from skill env

env_policy blocks the full PASSWORD/PASSWD spellings but not the ubiquitous
abbreviated _PASS form (DB_PASS, SMTP_PASS, MYSQL_PASS, ...), whose value is the
plaintext password itself, so every skill subprocess inherited it. Postgres's
file-based credential sources PGPASSFILE (.pgpass locator) and PGSERVICEFILE
(pg_service.conf, may carry a password) were likewise missed -- the psql analog
of the MYSQL_PWD/REDISCLI_AUTH no-flag sources bytedance#4018 added.

Collapse *PASSWORD*/*PASSWD* into a single *PASS* pattern (which subsumes both,
covers the abbreviated family, and catches PGPASSFILE) and add PGSERVICEFILE to
the exact-name denylist. *PASS* deliberately does not touch PWD/OLDPWD, which
carry no PASS substring. Capability-handle vars (KUBECONFIG, DOCKER_CONFIG,
GIT_ASKPASS, ...) are a broader, non-value-bearing class and are left out of
this fix.

test_secret_like_names_are_blocked already pins the contract; the new names are
added to its parametrize list (red before, green after) and the benign-names
test guards against over-scrubbing PWD/OLDPWD.

* test(sandbox): pin the *_ASKPASS over-scrub that *PASS* introduces

*PASS* also matches the credential-helper vars GIT_ASKPASS, SSH_ASKPASS and
SUDO_ASKPASS, which main inherits. Each names a program whose purpose is to hand
the caller a credential, so inheriting the pointer is the same leak class as
inheriting the value -- a skill can simply invoke it. Scrubbing them is intended;
pin all three in test_secret_like_names_are_blocked (red on main) so the
behaviour is a decision rather than a side effect of the pattern's shape.

test_benign_names_are_allowed could not have caught this: none of its 14 names
carries a PASS substring, so it passes identically before and after the
broadening and asserts nothing about it. Record that in its docstring -- the
absence of a benign PASS-bearing name is a decision, and PWD/OLDPWD are the
boundary the list does pin.

Also note the incidental COMPASS_*/BYPASS_* over-scrub in the module comment
(over-scrubbing is this module's fail-safe direction; required-secrets is the
escape hatch), and list PGPASSFILE alongside PGSERVICEFILE in AGENTS.md.
@AnnaSuSu
AnnaSuSu deleted the fix/sandbox-env-scrub-db-password-vars branch July 20, 2026 02:01
marvin9551 pushed a commit to marvin9551/deer-flow that referenced this pull request Aug 21, 2026
…ill env (bytedance#4018)

* fix(sandbox): scrub MYSQL_PWD and REDISCLI_AUTH from the inherited skill env

env_policy blocks the MySQL and Redis connection strings (MYSQL_URL, REDIS_URL)
because they embed a password, but not the password variables the same clients
read directly: mysql/libmysqlclient takes MYSQL_PWD and redis-cli takes
REDISCLI_AUTH with no further configuration. Both names carry no
KEY/SECRET/TOKEN/PASSWORD/PASSWD substring, so they were inherited by every
skill subprocess -- the leak surface this module exists to close (bytedance#3861).

They need exact entries rather than a pattern: *PWD* would also strip PWD and
OLDPWD. REDIS_AUTH is included as the common non-canonical variant.

* Apply suggestions from code review

- Distinguish REDIS_AUTH (defensive, non-canonical) from MYSQL_PWD and
  REDISCLI_AUTH (documented no-flag credential sources) in the comment.
- Pin that request-scoped injection still overrides the host value for the
  newly blocked names, keeping required-secrets a working escape hatch.
marvin9551 pushed a commit to marvin9551/deer-flow that referenced this pull request Aug 21, 2026
…rom skill env (bytedance#4026)

* fix(sandbox): scrub abbreviated *_PASS vars and Postgres PGPASSFILE from skill env

env_policy blocks the full PASSWORD/PASSWD spellings but not the ubiquitous
abbreviated _PASS form (DB_PASS, SMTP_PASS, MYSQL_PASS, ...), whose value is the
plaintext password itself, so every skill subprocess inherited it. Postgres's
file-based credential sources PGPASSFILE (.pgpass locator) and PGSERVICEFILE
(pg_service.conf, may carry a password) were likewise missed -- the psql analog
of the MYSQL_PWD/REDISCLI_AUTH no-flag sources bytedance#4018 added.

Collapse *PASSWORD*/*PASSWD* into a single *PASS* pattern (which subsumes both,
covers the abbreviated family, and catches PGPASSFILE) and add PGSERVICEFILE to
the exact-name denylist. *PASS* deliberately does not touch PWD/OLDPWD, which
carry no PASS substring. Capability-handle vars (KUBECONFIG, DOCKER_CONFIG,
GIT_ASKPASS, ...) are a broader, non-value-bearing class and are left out of
this fix.

test_secret_like_names_are_blocked already pins the contract; the new names are
added to its parametrize list (red before, green after) and the benign-names
test guards against over-scrubbing PWD/OLDPWD.

* test(sandbox): pin the *_ASKPASS over-scrub that *PASS* introduces

*PASS* also matches the credential-helper vars GIT_ASKPASS, SSH_ASKPASS and
SUDO_ASKPASS, which main inherits. Each names a program whose purpose is to hand
the caller a credential, so inheriting the pointer is the same leak class as
inheriting the value -- a skill can simply invoke it. Scrubbing them is intended;
pin all three in test_secret_like_names_are_blocked (red on main) so the
behaviour is a decision rather than a side effect of the pattern's shape.

test_benign_names_are_allowed could not have caught this: none of its 14 names
carries a PASS substring, so it passes identically before and after the
broadening and asserts nothing about it. Record that in its docstring -- the
absence of a benign PASS-bearing name is a decision, and PWD/OLDPWD are the
boundary the list does pin.

Also note the incidental COMPASS_*/BYPASS_* over-scrub in the module comment
(over-scrubbing is this module's fail-safe direction; required-secrets is the
escape hatch), and list PGPASSFILE alongside PGSERVICEFILE in AGENTS.md.
jihtsan pushed a commit to jihtsan/dnx-deer-flow that referenced this pull request Aug 29, 2026
…ill env (bytedance#4018)

* fix(sandbox): scrub MYSQL_PWD and REDISCLI_AUTH from the inherited skill env

env_policy blocks the MySQL and Redis connection strings (MYSQL_URL, REDIS_URL)
because they embed a password, but not the password variables the same clients
read directly: mysql/libmysqlclient takes MYSQL_PWD and redis-cli takes
REDISCLI_AUTH with no further configuration. Both names carry no
KEY/SECRET/TOKEN/PASSWORD/PASSWD substring, so they were inherited by every
skill subprocess -- the leak surface this module exists to close (bytedance#3861).

They need exact entries rather than a pattern: *PWD* would also strip PWD and
OLDPWD. REDIS_AUTH is included as the common non-canonical variant.

* Apply suggestions from code review

- Distinguish REDIS_AUTH (defensive, non-canonical) from MYSQL_PWD and
  REDISCLI_AUTH (documented no-flag credential sources) in the comment.
- Pin that request-scoped injection still overrides the host value for the
  newly blocked names, keeping required-secrets a working escape hatch.
jihtsan pushed a commit to jihtsan/dnx-deer-flow that referenced this pull request Aug 29, 2026
…rom skill env (bytedance#4026)

* fix(sandbox): scrub abbreviated *_PASS vars and Postgres PGPASSFILE from skill env

env_policy blocks the full PASSWORD/PASSWD spellings but not the ubiquitous
abbreviated _PASS form (DB_PASS, SMTP_PASS, MYSQL_PASS, ...), whose value is the
plaintext password itself, so every skill subprocess inherited it. Postgres's
file-based credential sources PGPASSFILE (.pgpass locator) and PGSERVICEFILE
(pg_service.conf, may carry a password) were likewise missed -- the psql analog
of the MYSQL_PWD/REDISCLI_AUTH no-flag sources bytedance#4018 added.

Collapse *PASSWORD*/*PASSWD* into a single *PASS* pattern (which subsumes both,
covers the abbreviated family, and catches PGPASSFILE) and add PGSERVICEFILE to
the exact-name denylist. *PASS* deliberately does not touch PWD/OLDPWD, which
carry no PASS substring. Capability-handle vars (KUBECONFIG, DOCKER_CONFIG,
GIT_ASKPASS, ...) are a broader, non-value-bearing class and are left out of
this fix.

test_secret_like_names_are_blocked already pins the contract; the new names are
added to its parametrize list (red before, green after) and the benign-names
test guards against over-scrubbing PWD/OLDPWD.

* test(sandbox): pin the *_ASKPASS over-scrub that *PASS* introduces

*PASS* also matches the credential-helper vars GIT_ASKPASS, SSH_ASKPASS and
SUDO_ASKPASS, which main inherits. Each names a program whose purpose is to hand
the caller a credential, so inheriting the pointer is the same leak class as
inheriting the value -- a skill can simply invoke it. Scrubbing them is intended;
pin all three in test_secret_like_names_are_blocked (red on main) so the
behaviour is a decision rather than a side effect of the pattern's shape.

test_benign_names_are_allowed could not have caught this: none of its 14 names
carries a PASS substring, so it passes identically before and after the
broadening and asserts nothing about it. Record that in its docstring -- the
absence of a benign PASS-bearing name is a decision, and PWD/OLDPWD are the
boundary the list does pin.

Also note the incidental COMPASS_*/BYPASS_* over-scrub in the module comment
(over-scrubbing is this module's fail-safe direction; required-secrets is the
escape hatch), and list PGPASSFILE alongside PGSERVICEFILE in AGENTS.md.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:sandbox Sandboxed execution and docker/ risk:high High risk: backend API, agents, sandbox, auth, deps, CI size/S PR changes 20-100 lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants