Add literal twins of Bash loop values (#242)

* Add literal twins of Bash loop values

Add BashParser.TryProjectLiteralTwins. For each occurrence whose
changeable argument words have a proved finite set of values, the
projection writes one literal twin for each combination of values. A
twin is the full source with each such word written as one literal
word. The parser parses it again, so the twin occurrence has the facts
of the typed literal: command words, path facts, and expansion facts.

An occurrence gets no twins when it is incomplete, when a verb word is
not static, when a word has no complete finite value set, when an
unquoted value can split or glob, when a redirect depends on a value,
or when the combinations pass the limit (32 for one occurrence, 128 in
total). An unquoted word uses its authored values only under
FreshNonInteractiveNoStartup, and only when each value is nonempty and
has no space, tab, newline, glob character, or backslash.

The rules use general shell facts only. A twin grants no authority.
Parse output does not change. Prepare 0.4.0-beta.22.

* Close review round 1 gaps in literal twins

Give no twins for text that the lexer does not model as Bash does:

- a source with a backslash-newline or a carriage return;
- an occurrence with `=~` or `:~` in a word;
- an occurrence in a source with `~` or `HOME` when no live launch
  HOME proves the home directory;
- an unquoted value with a parenthesis, which an extglob pattern needs.

Count every twin parse, also a failed one, against one budget for a
call: 128 parses and 32,768 source characters. An occurrence whose
parses do not fit gets no twins and costs no parse. Run the cheap
checks for all occurrences first.

Add a Linux test that runs each source and each twin in GNU Bash and
compares the argv. Document the consumer warnings. Move the release
to 0.4.0-beta.23, after the beta.22 lexer fix.

* Parse literal twins as single commands and guard list tildes

Check `=~` and `:~` in the full source, as the backslash-newline and
carriage-return guards do. Bash expands a tilde in a loop list item
such as `for f in if=~/x`, and the parser keeps it as text.

Parse each twin text as one command, not as the full source. The twin
text is the authored text from the first to the last word of the
command. The parser parses it in the exact directory of the command,
with the launch facts that are live there. A twin parse then costs
about its own length. Compute the length of every twin text before any
build, and check the budget first: 128 twin parses and 2,048 + 8 x
(source length) characters. The projection now costs about one Parse.
On an 80 KB source with 8,000 commands it takes about as long as Parse
(before: 23 s and 80 GB).

A command gets no twins when its directory is not exact, when it has an
assignment prefix, or when it has a heredoc. BashLiteralTwin gets a
WorkingDirectory property.

* Give no twins under an unknown state or an unproved home directory

Under BashInitialStateMode.Unknown, the parser can report an exact
value for a word that Bash expands: `-o"$n"` gives `-o$n`, but Bash
passes `-oa`. The projection now gives no twins in that mode. Parse is
not changed here.

A cd without an operand goes to $HOME. Without a live launch HOME, the
parser takes that directory from HomeDirectory, but Bash uses its real
HOME. The projection now parses the source once more with another
HomeDirectory. A command whose exact directory changes gets no twins.
The check does not read the cd forms, so it also covers `cd --`,
`builtin cd`, `command cd`, and `if cd; then`.

Add the inputs as tests and to the real-Bash comparison, which now
also compares the working directory. Fix the stale parser comment.

* Prepare 0.4.0-beta.23 on top of the beta.22 lexer fix

Add the 0.4.0-beta.23 release notes above 0.4.0-beta.22, and set the
README version line to 0.4.0-beta.23. Move the beta.22 plan section to
Completed.

Since 0.4.0-beta.22, Parse gives Unknown or the Bash value for a loop
list item with `=~` or `:~`, never the text. Update the real-Bash test
to that rule. Keep the line-text guards as defense in depth and say so
in the code, the SPEC, and the consumer guide. Add a carriage return in
single quotes as a guard test, because a carriage return outside quotes
is now unparseable.

Author: Aaronontheweb

Directory.Build.props

@@ -8,7 +8,7 @@
     <Nullable>enable</Nullable>
     <TreatWarningsAsErrors>true</TreatWarningsAsErrors>
     <VersionPrefix>0.4.0</VersionPrefix>
-    <VersionSuffix>beta.22</VersionSuffix>
+    <VersionSuffix>beta.23</VersionSuffix>
   </PropertyGroup>
   <PropertyGroup>
     <!-- Target framework matrix -->

IMPLEMENTATION_PLAN.md

@@ -10,15 +10,25 @@ priorities.
 
 ---
 
-## NOW (0.4.0-beta.22 Bash line continuations, security)
+## NOW (0.4.0-beta.23 literal twins of loop values)
+
+- [x] Add `BashParser.TryProjectLiteralTwins`. Write each changeable argument word as one proved literal value.
+- [x] Give no twins for an incomplete occurrence, a verb word that is not static, a word without a complete finite value set, an unquoted value that can split or glob, a redirect that depends on a value, and combinations over the limit. Kill a hand mutant of each gate with a test.
+- [x] Review round 1: guard a backslash-newline, a carriage return, `=~` and `:~`, an unproved home directory, and `extglob` patterns. Count every twin parse against one budget of parses and source characters. Compare twins with GNU Bash argv on Linux.
+- [x] Review round 2: check `=~` and `:~` in the full source. Parse each twin as one command in its exact directory, and check the budget before a twin is built.
+- [x] Verification round: give no twins under the Unknown initial state, and for a directory that comes from an unproved `HOME`.
+- [x] Rebase on 0.4.0-beta.22 (#243). Keep the line-text guards. Run the twin tests, the real-Bash comparison, and the twin probe again.
+- [ ] Merge the PR after Linux and Windows CI. Tag `0.4.0-beta.23` and verify NuGet publication.
+
+## Completed (0.4.0-beta.22 Bash line continuations, security)
 
 - [x] Remove a Bash line continuation before the lexer reads the next character of an expansion or an operator (#243). `echo "$\⏎(touch /tmp/x)"` gives the `touch` occurrence. Fail closed on a split `$((` marker.
 - [x] Fail closed on a CR in Bash code and on `\` + CR (owner decision). Comments end only at LF.
 - [x] Take a Bash `--name=value` value from the decoded word (owner decision).
 - [x] Review fixes: `#` after a word part is word text; reserved words, `((`, and `;;` read across a continuation; a `~` after `=` or `:` of an assignment-shaped word expands from HOME.
 - [x] Pin each case against real Bash. Prove the general rule: a continuation at any point gives the same facts or fails closed. Run manual mutants and an output comparison against beta.21.
-- [ ] Merge the PR after Linux and Windows CI. Tag `0.4.0-beta.22` and verify NuGet publication.
-- [ ] Move the literal twins of #242 to 0.4.0-beta.23.
+- [x] Merge the PR after Linux and Windows CI. Tag `0.4.0-beta.22`.
+- [x] Move the literal twins of #242 to 0.4.0-beta.23.
 
 ## Completed (0.4.0-beta.21 restore the beta.19 verb-slot rule)
 

README.md

@@ -13,10 +13,10 @@ Hand-rolled, AOT-trim friendly, zero native dependencies. Multi-targets
 
 ```bash
 # Current package
-dotnet add package ShellSyntaxTree --version 0.4.0-beta.22
+dotnet add package ShellSyntaxTree --version 0.4.0-beta.23
 ```
 
-Version `0.4.0-beta.22` includes the Bash line-continuation, carriage-return, and word-`#` security fixes, exact Bash inline option values and assignment-word tildes, the restored beta.19 command-word verb-slot rule, Bash ANSI-C quote decoding, per-argument pathname-expansion and field-splitting facts, bounded Bash arithmetic expansion, bounded loop `break` and `continue`, unresolved brace words, effective values and binding command words under the fresh-process mode, bounded Bash `export` and `set --`, unknown variable reads under the fresh-process mode, heredocs in command substitutions, Bash background lists, Bash control-flow statements, wider Bash assignment facts, pathname-expansion facts, launcher-proved environment facts, command-word facts, bounded Bash and PowerShell assignment facts,
+Version `0.4.0-beta.23` includes literal twins of Bash loop values, the Bash line-continuation, carriage-return, and word-`#` security fixes, exact Bash inline option values and assignment-word tildes, the restored beta.19 command-word verb-slot rule, Bash ANSI-C quote decoding, per-argument pathname-expansion and field-splitting facts, bounded Bash arithmetic expansion, bounded loop `break` and `continue`, unresolved brace words, effective values and binding command words under the fresh-process mode, bounded Bash `export` and `set --`, unknown variable reads under the fresh-process mode, heredocs in command substitutions, Bash background lists, Bash control-flow statements, wider Bash assignment facts, pathname-expansion facts, launcher-proved environment facts, command-word facts, bounded Bash and PowerShell assignment facts,
 bounded PowerShell collection facts, shell-neutral filesystem tree-access
 effects, and fail-closed approval diagnostics alongside the typed syntax tree,
 command-occurrence API, bounded authored-value analysis, audited filesystem and

RELEASE_NOTES.md

@@ -1,3 +1,29 @@
+#### 0.4.0-beta.23 2026-10-06 ####
+
+This prerelease adds literal twins of Bash commands whose argument words have a proved finite set of values.
+
+## Added
+
+- `BashParser.TryProjectLiteralTwins` and the records `BashLiteralTwinProjection`, `BashLiteralTwinCommand`, `BashLiteralTwin`, and `BashLiteralTwinWord`.
+- A twin writes each changeable argument word of one command (an expansion or a tilde word) as one literal value. The parser parses the text of that one command in the exact directory of the command, with the launch facts that are live there. The twin occurrence has the normal facts of the typed literal: command words, path facts, and expansion facts. `BashLiteralTwin.WorkingDirectory` names the directory.
+- The projection gives one twin for each combination of values, up to 32 for one command.
+- Example: `for n in 8250 8244; do gh api repos/o/r/issues/$n; done` gives the twins `gh api repos/o/r/issues/8250` and `gh api repos/o/r/issues/8244`. `for f in a.txt b.txt; do cat "$f"; done` gives twins with the command words `cat`.
+
+## Security and compatibility
+
+- A command gets no twins when it is incomplete, when its directory is not exact, when it has an assignment prefix, when a verb word is not static, when a word has no complete finite value set, when an unquoted value can split or glob, when a redirect depends on a value, when it has a heredoc, or when its twins do not fit in the budget. The caller keeps its current behavior for such a command.
+- An unquoted word uses its authored values only under `FreshNonInteractiveNoStartup`, and only when each value is nonempty and has no space, tab, newline, `*`, `?`, `[`, parenthesis, or backslash. The parenthesis rule covers a Bash build with `extglob` on by default.
+- A source with a backslash-newline, a carriage return, `=~`, or `:~` anywhere gets no twins. The lexer does not model this text as Bash does. The `=~` and `:~` rule covers a loop list item such as `for f in if=~/x`, and it also stops `[[ $x =~ re ]]`.
+- When the source has `~` or `HOME`, a command gets twins only with a live launch `HOME`. Without it, the home directory is the caller assumption `HomeDirectory`.
+- A command whose exact directory comes from `HOME` (after `cd`, `cd --`, `builtin cd`, or `if cd; then`) gets twins only with a live launch `HOME`. The projection finds such a command with one more parse that uses another `HomeDirectory`.
+- The projection gives no twins under `BashInitialStateMode.Unknown`. In that mode the parser can report `-o"$n"` as the exact value `-o$n`.
+- Budget: one call parses at most 128 twins and at most 2,048 + 8 x (source length) characters of twin text, plus at most one more parse of the source for the `HOME` check. The projection checks the budget before it builds a twin, so its cost stays a small constant factor of one `Parse`. On an 80 KB source with 8,000 commands, the projection takes about as long as `Parse`.
+- Values of different words combine independently. The twin set can hold combinations that never run, so a check of every twin is stricter, not weaker.
+- A Linux test runs each source and each twin in GNU Bash and compares the argv and the working directory.
+- With the 0.4.0-beta.22 lexer, a backslash-newline, a carriage return, and an assignment-shaped `=~` or `:~` give correct `Parse` facts or fail closed. The projection keeps its guards for this text as defense in depth. Twins of a source without such text do not change.
+- A twin grants no authority. Existing facts do not change: `Parse` gives the same output as 0.4.0-beta.22.
+- Public API: additive.
+
 #### 0.4.0-beta.22 2026-10-06 ####
 
 This prerelease fixes security bugs. A Bash line continuation, a carriage return, or a `#` after a quoted part could hide a command from `Parse` (#243). In the examples, `⏎` is a real newline and CR is a carriage return.

SPEC.md

@@ -111,6 +111,9 @@ public sealed class BashParser : IShellParser
     public bool TryProjectFiniteScopes(
         string command,
         out BashFiniteScopeProjection? projection);
+    public bool TryProjectLiteralTwins(
+        string command,
+        out BashLiteralTwinProjection? projection);
 }
 
 /// <summary>PowerShell implementation of IShellParser (v0.2.0). The
@@ -276,6 +279,12 @@ public enum ShellTreeTraversalMode { ... }
 public sealed record BashFiniteScopeProjection { ... }
 public sealed record BashScopedCommand { ... }
 
+// v0.4.0-beta.23 literal twin evidence — see §3.
+public sealed record BashLiteralTwinProjection { ... }
+public sealed record BashLiteralTwinCommand { ... }
+public sealed record BashLiteralTwin { ... }
+public sealed record BashLiteralTwinWord { ... }
+
 // v0.4.0-beta.7 command-word evidence — see §3.
 public abstract record ShellCommandWords { ... }
 ```
@@ -367,6 +376,37 @@ public sealed record BashScopedCommand
 }
 ```
 
+`TryProjectLiteralTwins` (v0.4.0-beta.23) returns these parser-owned records:
+
+```csharp
+public sealed record BashLiteralTwinProjection
+{
+    public ParsedCommand Parsed { get; internal init; }
+    public IReadOnlyList<BashLiteralTwinCommand> Commands { get; internal init; }
+}
+
+public sealed record BashLiteralTwinCommand
+{
+    public CommandOccurrence SourceOccurrence { get; internal init; }
+    public int SourceOccurrenceIndex { get; internal init; }
+    public IReadOnlyList<BashLiteralTwin> Twins { get; internal init; }
+}
+
+public sealed record BashLiteralTwin
+{
+    public string Source { get; internal init; }
+    public string WorkingDirectory { get; internal init; }
+    public CommandOccurrence Occurrence { get; internal init; }
+    public IReadOnlyList<BashLiteralTwinWord> Words { get; internal init; }
+}
+
+public sealed record BashLiteralTwinWord
+{
+    public int ClauseElementIndex { get; internal init; }
+    public string Value { get; internal init; }
+}
+```
+
 `WindowsPowerShell51` uses separately oracle-pinned grammar and metadata. Its
 command-owned execution-region catalog is limited to Windows PowerShell
 `5.1.19041.6456` `ForEach-Object` Begin/Process/RemainingScripts/End and
@@ -1488,6 +1528,137 @@ For example, `cd /work/sub && true; touch marker.txt` from `/work` yields
 files. By contrast, `cd "$target" && touch marker.txt` yields no projection.
 The caller owns filesystem checks, policy, grant matching, and process launch.
 
+#### Literal twin projection (v0.4.0-beta.23)
+
+`BashParser.TryProjectLiteralTwins` gives a consumer the facts of each literal
+command that a loop or a bounded variable can run. A changeable argument word
+is a word that the shell can change before the program runs: an expansion
+(`$n`, `"$f"`, `repos/x/$n`) or a tilde word. For each occurrence whose
+changeable words all have a proved finite set of values, the projection
+writes one literal twin for each combination of values. A twin text is the
+authored text of that one command, from its first word to its last word, with
+each changeable word written as one literal word. The parser parses the twin
+text in the exact directory of the occurrence, with the launch facts that are
+live there. The twin occurrence then has its own normal facts: `CommandWords`,
+verb tokens, path facts, `AuthoredFileSystemValue`, and the expansion facts of
+a typed literal. Its structure (role, ancestry) is that of one top-level
+command, so read the structure from `SourceOccurrence`.
+
+The projection uses this schematic flow:
+
+```text
+give nothing when the initial state mode is Unknown
+give nothing when the source has a backslash-newline, a carriage return,
+    `=~`, or `:~`
+for each occurrence of the full parse:
+    give nothing unless the occurrence is complete
+    give nothing unless its directory is exact
+    without a live launch HOME, give nothing when its directory changes
+        in one more parse with another HomeDirectory
+    give nothing when it has an assignment prefix (`X=1 cmd`)
+    give nothing when the source has `~` or `HOME` and no live launch HOME
+    give nothing unless each word has a span in the submitted source
+    give nothing unless each redirect is fixed
+    for each word that is not static:
+        give nothing unless it is one argument with an analyzed value
+        values = effective Exact or FiniteSet value
+        if the word can split or glob:
+            values = authored Exact or FiniteSet value when the effective value is Unknown
+            give nothing unless the mode is FreshNonInteractiveNoStartup
+            give nothing unless each value is nonempty and has no
+                space, tab, newline, *, ?, [, (, ), or backslash
+    give nothing when the combinations pass 32
+    compute the length of every twin text before any build
+for each occurrence that passed, in order:
+    give nothing when its twins do not fit in the rest of the budget:
+        128 twin parses, and 2,048 + 8 x (source length) twin characters
+    charge its twin count and twin characters
+    for each combination:
+        write each value as one literal word (plain text, else single quotes)
+        parse the twin text in the directory of the occurrence
+        give nothing unless the twin is one complete command in that
+            directory, and each written word is one literal word with that
+            exact value
+```
+
+A word is static when it has no expansion, glob character, brace list, or
+leading tilde. A twin keeps every static word as authored. A verb word must be
+static. A fixed redirect is an exact file target, a descriptor operation, or a
+here-string with exact data. A heredoc body is outside the words of the
+command, so the twin text cannot hold it, and a command with a heredoc gets no
+twins.
+
+The effective value of an unquoted expansion stays `Unknown`, because the
+shell can split or glob it. Under `FreshNonInteractiveNoStartup`, the caller
+removes an inherited `IFS`, and the parser rejects source that changes `IFS`
+or a glob option. Bash then splits only on space, tab, and newline. It globs
+on `*`, `?`, and `[`, and, when a Bash build turns `extglob` on by default,
+also on a pattern such as `@(a)` or `+(a)`. A nonempty value without these
+characters, without a parenthesis, and without a backslash stays one
+unchanged word. Only then does the projection use the authored value of an
+unquoted word. An empty value can remove the word, so it gives no twin. The
+authored value needs `PublishAuthoredSourceFacts`.
+
+Some guards are wider than Bash, because the twin facts are only as good as
+the `Parse` value facts. A value can come from any part of the source, such as
+a loop list, so these guards read the full source. The first two guards cover
+text that `Parse` modeled wrongly before v0.4.0-beta.22. Since then `Parse`
+gives correct facts for it or fails closed, and the guards stay as defense in
+depth:
+
+- Bash removes a backslash-newline before it reads a word, also inside an
+  expansion. Bash reads a carriage return as a word character. A source with
+  either text gets no twins.
+- Bash expands a tilde after `=` and after `:` in an argument or a loop list
+  item that looks like an assignment (`dd if=~/x`, `for f in if=~/x`). A source
+  with `=~` or `:~` anywhere gets no twins. This also stops `[[ $x =~ re ]]`.
+- Without a live launch `HOME`, the home directory is the caller assumption
+  `HomeDirectory`. When the source has `~` or `HOME`, an occurrence gets twins
+  only with a live launch `HOME`.
+- A `cd` without an operand (also `cd --`, `builtin cd`, `if cd; then`) goes
+  to `$HOME`. Without a live launch `HOME`, the parser takes that directory
+  from `HomeDirectory`. The projection does not read the `cd` forms. It parses
+  the source once more with another `HomeDirectory`, and an occurrence whose
+  exact directory changes gets no twins. When the launch facts export `HOME`
+  but `HOME` is not live at the occurrence, the occurrence gets no twins.
+- Under `BashInitialStateMode.Unknown`, the parser can report an exact value
+  for a word that Bash expands (`-o"$n"` gives `-o$n`). The projection gives
+  no twins in that mode.
+
+The budget is checked before a twin text is built. A twin text is one simple
+command with no loop, so its parse costs about as much as its length. The
+character budget grows with the source, so the twin work of one call stays a
+small constant factor of one `Parse`. An occurrence that does not fit gets no
+twins and costs no build and no parse. A charged occurrence keeps its charge
+when a twin fails.
+
+The values of different words combine independently. Two words that read one
+loop variable can give a twin with two different values. That combination
+does not run, but each of its values does. The twin set is a superset of the
+runs, so a consumer that checks every twin checks every run.
+
+An occurrence that is not in `Commands` has no twins. The method returns
+`false` and a null projection when no occurrence has twins. Bash runs only
+the authored source, never `twin.Source`. A literal value can move a word
+between the verb and argument roles, as the typed literal does: the twins of
+`for v in push fetch; do git $v origin; done` have the command words
+`git push origin` and `git fetch origin`.
+
+Positive example: from `/work`,
+`for n in 8250 8244; do gh api repos/o/r/issues/$n >/dev/null; done` gives two
+twins, `gh api repos/o/r/issues/8250 >/dev/null` and
+`gh api repos/o/r/issues/8244 >/dev/null`. Each twin occurrence has the
+command words `gh api` and the path fact of its literal word.
+
+Negative examples: `for n in $(gh issue list); do gh api repos/o/r/issues/$n; done`
+gives no twins, because the list has no proved values. `for f in 'a b' c; do cat $f; done`
+gives no twins, because `a b` splits. `for d in a b; do ~/bin/tool "$d"; done`
+gives no twins, because the program word is not static. `for f in if=~/x; do dd "$f"; done`
+gives no twins, because Bash expands the tilde in the list item.
+
+A twin is syntax evidence. It does not grant authority. The caller owns path
+policy, grant matching, and process launch, and it must check every twin.
+
 Effects join per authored occurrence. Two `Unchanged` visits remain
 `Unchanged`; two success-only changes join their bounded targets. Unknown,
 mixed unchanged/change visits, missing visits, or lost correlation join to