AST - Abstract Syntax Tree Tools 🌳

Type-safe AST parsing for JavaScript with extensible architecture

ANTLR-powered • Type-safe • Extensible • Monorepo

Architecture • Packages • Development

---

🚀 Overview

A production-grade Abstract Syntax Tree (AST) parsing toolkit built with TypeScript and ANTLR v4. Currently supports JavaScript parsing with an architecture designed for multi-language extensibility.

The Problem:

Traditional AST tools:
- Tightly coupled to single languages ❌
- Difficult to extend or customize ❌
- Limited type safety ❌
- Complex integration ❌

The Solution:

AST Toolkit:
- Clean, extensible architecture ✅
- ANTLR-powered parsing ✅
- Full TypeScript type safety ✅
- Monorepo structure for modularity ✅

Result: Production-ready AST parsing with a foundation for supporting any programming language.

---

⚡ Key Features

Parser Architecture

Feature	Description	Benefit
ANTLR v4	Industry-standard parser generator	Proven, battle-tested
Type-safe	Full TypeScript with strict mode	Catch errors at compile-time
Extensible	Core + language modules	Easy to add new languages
Visitor Pattern	ANTLR parse tree → Custom AST	Clean transformation

Development Experience

• Monorepo structure - Organized with pnpm workspaces + Turborepo
• Modern tooling - ESLint flat config, Prettier, Husky, commitlint
• Testing - Vitest for fast, reliable tests
• Build - tsup for lightning-fast builds with type declarations
• Versioning - Changesets for version management

---

📦 Packages

Core Package

@sylphlab/ast-core

• Generic AST node interfaces and types
• Foundation for all language parsers
• Zero dependencies
• Fully typed

Language Parsers

@sylphlab/ast-javascript

• JavaScript/ECMAScript parser
• ANTLR-generated lexer and parser
• Custom visitor for AST transformation
• Depends on: @sylphlab/ast-core, antlr4ts

Architecture

┌─────────────────────────────────────────┐
│  @sylphlab/ast-javascript               │
│  (JavaScript-specific parser)           │
└────────────────┬────────────────────────┘
                 │ depends on
                 ▼
┌─────────────────────────────────────────┐
│  @sylphlab/ast-core                     │
│  (Generic AST interfaces & types)       │
└─────────────────────────────────────────┘

---

🛠️ Technology Stack

Component	Technology	Purpose
Language	TypeScript 5.8	Type safety & modern JS
Parser	ANTLR v4	Grammar-based parsing
Monorepo	pnpm workspaces	Dependency management
Build	Turborepo	Task orchestration
Bundler	tsup	Fast ESM/CJS builds
Testing	Vitest	Unit testing
Linting	ESLint 9 (flat config)	Code quality
Formatting	Prettier	Code style
Git Hooks	Husky + lint-staged	Pre-commit checks
Versioning	Changesets	Version management

---

🚀 Quick Start

Installation

# Clone repository
git clone https://github.com/SylphxAI/ast.git
cd ast

# Install dependencies
pnpm install

# Build all packages
pnpm build

# Run tests
pnpm test

Using the Parser

import { parse } from '@sylphlab/ast-javascript';

const code = `
  const greeting = "Hello, World!";
  console.log(greeting);
`;

const ast = parse(code);
console.log(ast);

---

🏗️ Architecture

Design Principles

1. Separation of Concerns

   • Core types separated from language implementations
   • Each language parser in its own package

2. Extensibility

   • Easy to add new language parsers
   • Shared core interfaces for consistency

3. Type Safety

   • Strict TypeScript throughout
   • Full type inference

4. ANTLR Integration

   • Grammar files (.g4) define language syntax
   • Generated lexer/parser from grammar
   • Custom visitor transforms parse tree to AST

Parsing Flow

Source Code (String)
       │
       ▼
  ANTLR Lexer
  (Tokenization)
       │
       ▼
  ANTLR Parser
  (Parse Tree)
       │
       ▼
  Custom Visitor
  (Transformation)
       │
       ▼
  Custom AST
  (@sylphlab/ast-core types)

---

💻 Development

Workspace Structure

ast/
├── packages/
│   ├── core/              # @sylphlab/ast-core
│   │   ├── src/
│   │   ├── package.json
│   │   └── tsconfig.json
│   └── javascript/        # @sylphlab/ast-javascript
│       ├── src/
│       ├── grammar/       # .g4 grammar files
│       ├── package.json
│       └── tsconfig.json
├── package.json           # Root workspace config
├── pnpm-workspace.yaml    # pnpm workspace definition
├── turbo.json             # Turborepo configuration
└── README.md

Common Commands

# Development
pnpm dev                   # Watch mode for all packages

# Building
pnpm build                 # Build all packages
turbo run build            # Build with Turborepo

# Testing
pnpm test                  # Run all tests
pnpm test:watch            # Watch mode for tests

# Code Quality
pnpm lint                  # Lint all packages
pnpm lint:fix              # Auto-fix linting issues
pnpm format                # Format code with Prettier
pnpm check-format          # Check formatting
pnpm typecheck             # TypeScript type checking

# Full Validation
pnpm validate              # Run all checks (format, lint, typecheck, test)

# ANTLR (JavaScript package)
cd packages/javascript
pnpm antlr                 # Generate parser from grammar

Adding a New Language Parser

1. Create package directory

   mkdir packages/new-language
   cd packages/new-language

2. Create package.json

   {
     "name": "@sylphlab/ast-new-language",
     "version": "0.0.0",
     "dependencies": {
       "@sylphlab/ast-core": "workspace:*",
       "antlr4ts": "^0.5.0-alpha.4"
     }
   }

3. Add grammar file

   • Find or create ANTLR .g4 grammar for the language
   • Place in grammar/ directory

4. Implement visitor

   • Create custom visitor to transform ANTLR parse tree
   • Map to @sylphlab/ast-core types

5. Export parser

   export function parse(code: string): AstNode {
     // Implementation
   }

---

🎯 Current Status

✅ Completed

• Monorepo structure with pnpm + Turborepo
• Core AST type definitions (@sylphlab/ast-core)
• JavaScript parser package (@sylphlab/ast-javascript)
• ANTLR v4 integration
• ECMAScript grammar integration
• Basic visitor structure
• Build system configuration
• Testing infrastructure
• Code quality tooling (ESLint, Prettier, Husky)

🚧 In Progress

• Complete AST visitor implementation
• Comprehensive test coverage
• Position tracking improvements
• Grammar validation and refinement

🔮 Planned

• Support for additional JavaScript features
• TypeScript parser
• Python parser
• Go parser
• AST manipulation utilities
• Pretty printer (AST → source code)
• Documentation site

---

🧪 Testing

Running Tests

# Run all tests
pnpm test

# Watch mode
pnpm test:watch

# Test specific package
cd packages/javascript
pnpm test

Test Structure

import { describe, it, expect } from 'vitest';
import { parse } from '@sylphlab/ast-javascript';

describe('JavaScript Parser', () => {
  it('should parse variable declarations', () => {
    const code = 'const x = 42;';
    const ast = parse(code);
    expect(ast).toBeDefined();
  });
});

---

🔧 Configuration

TypeScript

Strict mode enabled with:

• noImplicitAny: true
• strictNullChecks: true
• strictFunctionTypes: true

ESLint

Using ESLint 9 flat config format:

// eslint.config.js
export default [
  {
    ignores: ['**/dist/**', '**/generated/**'],
    // ... rules
  }
];

ANTLR

Grammar generation:

antlr4ts -visitor -listener -o src/generated grammar/*.g4

---

📚 Resources

ANTLR Documentation

• ANTLR Official Site
• antlr4ts Documentation
• ANTLR Grammar Repository

AST Resources

• ESTree Specification (JavaScript AST spec)
• TypeScript Compiler API

---

🤝 Contributing

Contributions are welcome! Please follow these guidelines:

1. Open an issue - Discuss changes before implementing
2. Fork the repository
3. Create a feature branch - git checkout -b feature/my-feature
4. Follow code standards - Run pnpm validate
5. Write tests - Maintain high coverage
6. Commit with conventional commits - feat:, fix:, docs:, etc.
7. Submit a pull request

Commit Message Format

feat(javascript): add support for async/await
fix(core): correct position tracking for nested nodes
docs(readme): update installation instructions

---

📄 License

MIT © Sylphx

---

🙏 Credits

Built with:

• ANTLR - Parser generator
• antlr4ts - ANTLR runtime for TypeScript
• TypeScript - Language
• pnpm - Package manager
• Turborepo - Monorepo tool
• Vitest - Testing framework

---

Type-safe AST parsing for the modern web
_{Built with TypeScript and ANTLR}

sylphx.com • @SylphxAI • hi@sylphx.com