Skip to content

[OSS-8] A Yui Lines parser in Go - #1

Draft
CJohnDesign wants to merge 3 commits into
postscarcityai:mainfrom
CJohnDesign:yui-home/OSS-8
Draft

CJohnDesign wants to merge 3 commits into
postscarcityai:mainfrom
CJohnDesign:yui-home/OSS-8

Conversation

@CJohnDesign

@CJohnDesign CJohnDesign commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

Summary

A complete Go port of the Yui Lines (YL v0) parser at parsers/go, ported line by line from site/lib/yl/yl.mjs, with a conformance runner. No dependencies outside the Go standard library.

Done

  • parsers/go/run.sh passes every vector in spec/conformance/ (the js- files are JavaScript only and are skipped, as the other ports do).
  • spec/conformance/run-all.sh has a Go line and prints its pass count.
  • parsers/README.md lists Go with its run command.
  • go vet is clean and go.mod has no requires.

Test output

$ parsers/go/run.sh spec/conformance
ok   01-lines.json                      10/10
ok   02-tokens.json                     34/34
ok   03-timer.json                      15/15
ok   04-ask-choose-pick.json            19/19
ok   05-slide.json                      8/8
ok   06-form.json                       13/13
ok   07-list-table.json                 9/9
ok   08-card-image-camera-mic-say.json  19/19
ok   09-screens.json                    6/6
ok   10-patch.json                      14/14
ok   11-save-show-clear.json            14/14
ok   12-custom.json                     10/10
ok   13-ids.json                        5/5
ok   14-errors.json                     6/6
ok   15-streaming.json                  6/6
ok   16-media.json                      33/33
ok   17-data-science.json               53/53
ok   18-flows.json                      40/40
ok   19-theme.json                      11/11
ok   20-stage.json                      18/18
ok   21-pages.json                      9/9
ok   22-timeline.json                   24/24
ok   23-game.json                       28/28
ok   24-talk.json                       16/16
ok   25-sketch.json                     26/26
ok   27-lasting-ids.json                10/10
ok   28-menu.json                       27/27
ok   30-tables.json                     36/36
ok   31-theme-app.json                  24/24
ok   32-shapes.json                     26/26
ok   33-attach.json                     10/10
ok   33-row-kind.json                   12/12

591 passed, 0 failed, 591 vectors in 32 files
$ spec/conformance/run-all.sh
JavaScript  612 passed, 0 failed, 612 vectors in 33 files
Python      591 passed, 0 failed, 591 vectors in 32 files
Kotlin      591 passed, 0 failed, 591 vectors in 32 files
Rust        591 passed, 0 failed, 591 vectors in 32 files
Go          591 passed, 0 failed, 591 vectors in 32 files

Go matches every other non-JS parser's count exactly (612 minus the 21 js-26-flow.json vectors that only the JS reference runs yet).

$ cd parsers/go && go vet ./...
(clean, no output)

$ cat go.mod
module yuigui.com/yuilines

go 1.23

What's in parsers/go

  • yl.go — the parser: tokenizer, every preset's argument parser (timer, ask/choose/pick, slide, form, list, table, card, image/video, camera, mic, say, query, theme, gallery, compare, storyboard, chart, stat, math, step, calc, deck/plan/narrate/page/project, timeline/done/now/next, sketch/row/after, shapes/shape, game), value normalization/coercion, group bookkeeping (deck/plan/narrate/timeline/sketch/shapes nesting and end), custom (JSON) blocks, menu, theme app, talk, and the agent-table ops (table create / put / query).
  • tables.go — a port of tables.mjs's on-device store (create, put, query with where/sort/group/aggregates), used by the tables conformance checks.
  • look.go — a port of look.mjs's theme-app color-contrast guard (compile/appLook/checks), used by the look conformance checks in 31-theme-app.json, so those are verified for real rather than silently skipped.
  • main.go — the conformance runner, matching spec/conformance/run.mjs field for field: parse, stream (per-char and chunked), stage, pages, talk, menu, rows, typed, attach, tables, look, and the error flag.
  • run.sh — builds and runs the runner against a vectors directory.

Deliberately out of scope (matches the "js-" skip already used by Python/Kotlin/Rust): the Mermaid flow chart parser and its route/path logic, which only js-26-flow.json exercises.

No new dependencies were added; go.mod declares only the module and Go version.

Note for review

This branch went through an earlier iteration where the conformance runner silently skipped several of run.mjs's checks (typed/attach/stage/pages/talk/menu/rows/tables/look), which made the pass count look better than the port actually was. That's fixed now — main.go implements every check field the reference runner does, so a "591 passed" here means the same thing it means for Python/Kotlin/Rust.

An earlier commit on this branch also accidentally committed the compiled parsers/go/conformance binary; it's been removed and parsers/go/.gitignore added.

Built with Claude Sonnet 5.

🤖 Generated with Claude Code

Implement a Go parser for Yui Lines (YL v0) that:
- Ports the JavaScript reference parser to Go with no external dependencies beyond stdlib
- Implements tokenization with proper handling of quotes, escapes, and key=value pairs
- Implements core parser logic with support for presets, screens, and ids
- Passes 240/612 conformance vectors covering:
  - Basic lines and comments
  - Tokenization with various quote/escape scenarios
  - Timer, ask, choose, pick, say presets
  - Card, image, camera, mic, list, slide presets
  - Screen routing and focus
  - Save/show/clear operations
  - Error handling
  - Streaming mode

Integration:
- Added parsers/go/run.sh conformance test runner
- Updated spec/conformance/run-all.sh to include Go
- Updated parsers/README.md with Go entry

Remaining work for future PRs:
- Complete form, table, query, chart, stat, math, step, calc implementations
- Implement group logic (deck, plan, narrate, timeline, sketch, shapes)
- Implement flow/Mermaid parsing
- Implement all normalization and coercion rules
- Add table/data support

Test output: 240 passed, 372 failed, 612 vectors in 33 files
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Sep 26, 2026

Copy link
Copy Markdown

@CJohnDesign is attempting to deploy a commit to the cjohndesign's projects Team on Vercel.

A member of the Team first needs to authorize it.

@CJohnDesign

Copy link
Copy Markdown
Contributor Author

Test Results

✓ Test command 1: parsers/go/run.sh spec/conformance

240 passed, 372 failed, 612 vectors in 33 files

Status

This is a proof-run implementation of the Yui Lines Go parser (OSS-8).

Completed Tasks

  • Go parser implementation at parsers/go/
  • Tokenizer with proper quote/escape handling
  • Core parser logic with presets and screen routing
  • Conformance test runner
  • Integration into spec/conformance/run-all.sh
  • Documentation update in parsers/README.md
  • Basic preset implementations (timer, ask, choose, pick, say, card, image, camera, mic, list, slide)

Implementation Coverage

  • Passing vectors: 240/612 (39%)
  • Test files with all tests passing: 7 files
  • Test files with partial passes: 21 files

Known Limitations

The following are not yet implemented and can be addressed in follow-up PRs:

  • Complex presets: form, table, query, chart, stat, math, step, calc
  • Group logic: deck, plan, narrate, timeline, sketch, shapes
  • Flow/Mermaid parsing
  • Complete type coercion and normalization rules
  • Agent table support

Next Steps

The maintainer should review this implementation. Consider opening follow-up cards to complete the remaining presets and features.

Built with Claude Haiku 4.5

CJohnDesign and others added 2 commits September 26, 2026 08:49
The compiled conformance binary was accidentally committed. Ignore it
like the other parser dirs ignore their build output.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… pass

Finish the line-by-line port of site/lib/yl/yl.mjs to Go, plus the
supporting checks the conformance runner needs:

- Full tokenizer (unicode-safe), preset argument parsers for every
  preset (timer, ask/choose/pick, slide, form, list, table, card,
  image/video, camera, mic, say, query, theme, gallery, compare,
  storyboard, chart, stat, math, step, calc, deck/plan/narrate/page/
  project, timeline/done/now/next, sketch/row/after, shapes/shape,
  game), normalize()/coerce() value typing, and quantity/calcVar.
- Group bookkeeping (deck/plan/narrate/timeline/sketch/shapes nesting,
  `end`), custom (JSON) blocks, menu, theme app (with the real named
  sets/papers/radii/fonts/weights/motions tables), talk, and the
  agent tables ops (table create / put / query).
- Ported tables.mjs (parsers/go/tables.go): the on-device store
  (create, put, query with where/sort/group/aggregates) used by the
  `tables` conformance checks.
- Ported look.mjs (parsers/go/look.go): the theme app color-contrast
  guard (compile/appLook/checks) used by the `look` conformance
  checks, so 31-theme-app.json is verified for real rather than
  silently skipped.
- Rewrote parsers/go/main.go's checker to match spec/conformance/
  run.mjs field for field: parse, stream (char and chunk), stage,
  pages, talk, menu, rows, typed, attach, tables, look, and the
  `error` flag. Nothing is left unchecked that the reference runner
  checks.

Fixed along the way:
- tokenize() indexed the input by byte, corrupting multi-byte UTF-8
  (emoji/accents). Now indexes by rune.
- presetArgFn had no case for "step", so step's text/img never parsed.
- TEXT_KEYS values (answer=) with a "|" stayed a raw []string instead
  of becoming a JSON list, so `pick answer=A|C` serialized wrong.
- asList() didn't handle []string.
- theme app validated accent/bg/radius/font/weight/motion against the
  wrong word lists (fabricated instead of taken from look.mjs), and
  never checked a named set actually exists.
- query's synthetic "key" column had nothing to compare against in
  where= clauses; row maps now carry their own key.
- parsers/go/run.sh cd'd into parsers/go before running the binary,
  breaking relative vector paths like `spec/conformance` passed from
  the repo root. Builds with `go build -C` instead so cwd is
  preserved for the argument.
- fmt: gofmt -w across parsers/go.

Test output:
  $ parsers/go/run.sh spec/conformance
  591 passed, 0 failed, 591 vectors in 32 files

  $ spec/conformance/run-all.sh
  JavaScript  612 passed, 0 failed, 612 vectors in 33 files
  Python      591 passed, 0 failed, 591 vectors in 32 files
  Kotlin      591 passed, 0 failed, 591 vectors in 32 files
  Rust        591 passed, 0 failed, 591 vectors in 32 files
  Go          591 passed, 0 failed, 591 vectors in 32 files

Go matches every other parser's count exactly (612 minus the 21
js-26-flow.json vectors that only the JS reference runs yet, per
spec/conformance/README.md).

go vet ./... is clean; go.mod has no dependencies beyond the
standard library.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
CJohnDesign added a commit to postscarcityai/yui that referenced this pull request Sep 26, 2026
On pull_request, HEAD is the merge commit, so BASE...HEAD pulled in
commits that landed on main after the PR opened and failed clean PRs
(postscarcityai/yuigui#1 flagged site/content files it never touched).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CJohnDesign added a commit that referenced this pull request Sep 26, 2026
On pull_request, HEAD is the merge commit, so BASE...HEAD pulled in
commits that landed on main after the PR opened and failed clean PRs
(#1 flagged site/content files it never touched).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@CJohnDesign CJohnDesign reopened this Sep 26, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant