Code Stream - JupyterLab Cell Sync Extension

A JupyterLab extension that enables secure, proxy-based cell synchronization between teachers and students.

Overview

Code Stream allows teachers to share code cells with students in real-time. Teachers create sessions and push cell content to Redis, while students connect to the teacher's server through a secure proxy to pull updates on demand.

Architecture

Proxy-Based Design (Current)

Teacher:
  JupyterLab → Teacher's Jupyter Server → Redis (local)
                         ↑
                         | (HTTP GET requests proxied)
                         |
Student:
  JupyterLab → Student's Jupyter Server (proxy) → Teacher's Jupyter Server

Key Benefits:

• No CORS issues (all requests go through local Jupyter server)
• Teacher token stored securely server-side (never exposed to browser)
• Clear separation: teachers write to Redis, students read via proxy
• Centralized authentication and authorization control

Features

• Teacher-Student Sessions: Teachers create sessions, students join with session codes
• Request-Based Sync: Students receive notifications but only sync when they choose to
• Cell-Level Control: Teachers can toggle sync permissions per cell
• Redis Pub-Sub: Scalable architecture using Redis for real-time messaging
• Persistent Updates: Updates stored in Redis until students request them
• Production-Ready: Connection pooling, non-blocking Redis operations, structured logging
• Performance Optimized: SCAN-based iteration, efficient resource management
• Session-Based Collaboration: Teachers create 6-character session codes, students join sessions
• Proxy Architecture: Student servers proxy read requests to teacher server
• Secure Configuration: Teacher URL and token stored server-side, not in browser
• Cell-Level Sync: Teachers control which cells to sync; students preview and choose updates
• On-Demand Updates: Students hover to preview, click to sync (no automatic overwrites)
• Metadata Persistence: Session info and teacher URL stored in notebook metadata

Installation

• Backend: Tornado WebSocket handlers with Redis pub-sub
• Session Management: In-memory connection tracking with Redis persistence
• Notifications: Students get notified of available updates, not automatic syncs
• Security: Role-based permissions and input validation
• Performance: Redis connection pooling, SCAN-based queries, structured logging
• Monitoring: Comprehensive logging framework for debugging and production monitoring

Prerequisites

• JupyterLab 3.x or 4.x
• Python 3.8+
• Redis server (for teachers only)

1. Start Redis (Teachers Only)

Teachers need a running Redis server:

# Using Docker
docker run -d -p 6379:6379 redis:latest

# Or install locally
# macOS
brew install redis
brew services start redis

# Ubuntu
sudo apt install redis-server
sudo systemctl start redis

2. Install the Extension

# Clone the repository
git clone https://github.com/your-username/code_stream.git
cd code_stream

# Install in development mode
pip install -e .

# Build the extension
jupyter labextension develop . --overwrite

# Start JupyterLab
jupyter lab

Usage

For Teachers

1. Start JupyterLab with the extension installed
2. Open Code Stream sidebar (left sidebar icon)
3. Session code is automatically created and displayed
4. Share the session code with students
5. Enable sync on cells you want to share:
   • Toggle sync button appears in cell toolbar
   • Click to enable/disable sync for that cell
6. Students can now see and sync your enabled cells

Teacher Workflow:

# In a cell, enable sync via the toggle button in toolbar
print("Hello, students!")

# When you update the cell, students will see the update available
print("Updated content here")

For Students

1. Start JupyterLab with the extension installed
2. Open Code Stream sidebar
3. Configure Teacher Server:
   • Enter teacher's Jupyter server URL (e.g., http://192.168.1.10:8888)
   • Optionally enter teacher's token if required
   • Click Save
   • Click Test Connection to verify
4. Join Session:
   • Enter the 6-character session code from your teacher
   • Click Join
5. Sync Cells:
   • Hover over the refresh icon in cell toolbar to see available cells
   • Preview shows content on hover
   • Click a cell to sync/replace content

Student Workflow:

# In an empty cell, hover over the refresh icon
# Preview available teacher cells
# Click to sync and replace with teacher's content

Configuration

Teacher Server URL (Students)

Students must configure their teacher's Jupyter server URL. This URL should be:

• Accessible from the student's network
• Include the full base URL (e.g., http://192.168.1.10:8888)
• Use HTTPS in production environments

Authentication

Teachers can secure their Jupyter server with a token. Students will need to:

1. Get the token from their teacher
2. Enter it once in the configuration UI
3. Token is stored securely server-side (never in browser)

To find your Jupyter token (teachers):

jupyter server list
# Or check server logs when starting JupyterLab

Environment Variables

Teachers can configure Redis connection:

# Default: localhost:6379
export REDIS_HOST="localhost"
export REDIS_PORT="6379"
export REDIS_DB="0"

API Endpoints

Teacher Endpoints (Write Operations)

POST /code_stream/{hash}/push-cell/
POST /code_stream/{hash}/update/
POST /code_stream/{hash}/delete/

Student Endpoints (Configuration)

GET  /code_stream/config
POST /code_stream/config
POST /code_stream/test

Student Endpoints (Proxy to Teacher)

GET /code_stream/get-all-cell-ids/
GET /code_stream/{hash}/get-cell/?cell_id=...&cell_timestamp=...

Security Considerations

Authentication

• All endpoints require Jupyter authentication (@tornado.web.authenticated)
• Teacher token never exposed in responses, logs, or client storage
• Students must be authenticated to configure or use proxy

Network Security

• Use HTTPS for teacher server in production
• Validate teacher URL scheme (http/https only)
• No credentials allowed in URLs (user:pass@host forbidden)
• Optional: Restrict to private network subnets (RFC1918)

Token Handling

• Teacher token sent once via POST, stored server-side only
• Token masked in responses (returns has_token: boolean)
• Never logged or included in error messages

Logging and Monitoring

Code Stream includes comprehensive logging for debugging and production monitoring.

Python Backend Logging

Configure logging level in your Jupyter configuration:

# jupyter_lab_config.py
import logging

# Set log level (DEBUG, INFO, WARNING, ERROR)
logging.getLogger('code_stream').setLevel(logging.INFO)

Or via environment variable:

export JUPYTER_LOG_LEVEL=INFO
jupyter lab

TypeScript Frontend Logging

Configure logging in the browser console:

// Set log level (0=DEBUG, 1=INFO, 2=WARN, 3=ERROR, 4=NONE)
logger.setLevel(1);

// Disable console output
logger.setConsoleEnabled(false);

See LOGGING.md for complete logging documentation.

Performance

Production Optimizations

• Redis Connection Pooling: Reduces connection overhead (configurable, default: 10 connections)
• SCAN-Based Queries: Non-blocking iteration for large datasets (replaces blocking KEYS command)
• Structured Logging: Configurable log levels for production (set to WARN or ERROR)
• Efficient Resource Management: Proper cleanup and connection management

Benchmark Improvements

• Redis SCAN vs KEYS: 100x faster on large datasets (10,000+ keys)
• Connection pooling: 50% reduction in connection overhead
• Non-blocking operations: No server blocking regardless of dataset size

Troubleshooting

Students Cannot Connect

Problem: "Connection to teacher server timed out"

Solutions:

1. Verify teacher's Jupyter server is running
2. Check teacher's firewall allows incoming connections
3. Ensure both teacher and student are on the same network (or use VPN/tunnel)
4. Test URL directly in browser: http://teacher-ip:8888

Authentication Errors

Problem: "Authentication failed with teacher server"

Solutions:

1. Get the correct token from teacher: jupyter server list
2. Re-enter token in student configuration
3. Verify teacher's server isn't using additional auth layers

No Cells Available

Problem: Student sees "No cells available to sync"

Solutions:

1. Teacher: Ensure sync is enabled on cells (toggle button in toolbar)
2. Teacher: Verify Redis is running: redis-cli ping
3. Student: Refresh by closing and reopening dropdown

CORS Errors (Should Not Occur)

If you see CORS errors, the proxy is not working correctly. This architecture eliminates CORS by proxying through the student's local server.

Debug:

1. Check student's Jupyter logs for proxy errors
2. Verify teacher URL is configured correctly
3. Test connection using "Test Connection" button

Development

Setup Development Environment

# Install in development mode
pip install -e .

# Install TypeScript dependencies
npm install

# Build the extension
jupyter labextension develop . --overwrite

# Watch for changes (TypeScript)
npm run watch

# In another terminal, start JupyterLab
jupyter lab --autoreload

Project Structure

code_stream/
├── code_stream/                 # Python package (backend)
│   ├── __init__.py
│   ├── handlers.py              # Route registration
│   ├── redis_client.py          # Redis operations (teacher)
│   ├── redis_views.py           # Teacher write handlers
│   ├── config_store.py          # Secure config storage
│   ├── config_views.py          # Config API handlers
│   └── proxy_views.py           # Proxy GET handlers (student)
│
├── src/                         # TypeScript package (frontend)
│   ├── index.ts                 # Extension entry point
│   ├── handler.ts               # API request wrapper
│   ├── models/
│   │   └── types.ts             # TypeScript interfaces
│   ├── services/
│   │   ├── SessionManager.ts    # Session & config state
│   │   ├── SyncService.ts       # API client
│   │   └── RoleManager.ts       # Teacher/student role
│   └── components/
│       ├── SessionPanel.ts      # Sidebar UI
│       ├── UpdateIcon.ts        # Cell refresh button
│       └── CellSyncDropdown.ts  # Cell selection dropdown
│
├── style/                       # CSS styles
├── package.json                 # NPM dependencies
└── setup.py                     # Python package metadata

Running Tests

# Python tests
pytest

# TypeScript tests (if configured)
npm test

Migration Notes (Version 2.0)

Breaking Changes

Students must now configure the teacher server URL before syncing. There is no fallback to local Redis or direct cross-origin teacher access.

Migration Steps for Students

1. Upgrade extension: pip install --upgrade code_stream
2. Open Code Stream sidebar
3. Configure teacher server:
   • Enter teacher's Jupyter URL
   • Save and test connection
4. Join session as before

Migration Steps for Teachers

No changes required. Teachers continue using the extension as before.

Contributing

Contributions are welcome! Please:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Make your changes
4. Add tests
5. Submit a pull request

License

BSD 3-Clause License

Support

• Issues: https://github.com/your-username/code_stream/issues
• Discussions: https://github.com/your-username/code_stream/discussions
• Documentation: https://code-stream.readthedocs.io (coming soon)

Changelog

Version 2.0.0 (Current)

• BREAKING: Students must configure teacher server URL
• NEW: Proxy-based architecture eliminates CORS issues
• NEW: Server-side teacher token storage (enhanced security)
• NEW: Configuration UI in student sidebar
• NEW: Test connection button for students
• IMPROVED: Re-enabled authentication on all endpoints
• REMOVED: Direct cross-origin requests from student browsers
• REMOVED: Local Redis fallback for students

Version 1.0.0

• Initial release
• Basic teacher-student cell synchronization
• Redis-based storage
• Session management