Lecture Assistant - Real-time Transcription & AI Support

A web application designed to provide real-time transcription of lectures using OpenAI's Realtime API, store them in Firebase, and offer AI-powered explanations and summaries to students. Includes an instructor dashboard for managing lectures and initiating recordings.

Features

• Instructor Dashboard:

  • Secure Instructor Login/Signup with email validation and password protection.
  • Generate unique 6-character codes for new lectures with collision detection.
  • Manage lecture metadata (Course Code, Date, Time, Instructor Name).
  • View list of previously created lectures with filtering and sorting options.
  • Select and activate a lecture for recording with real-time status indicators.
  • Start/Stop real-time audio recording streamed via WebSockets with automatic reconnection.
  • Live transcription preview during recording with timestamps.
  • Visual audio level indicator with adjustable sensitivity.
  • Export lecture transcriptions to PDF with LaTeX formatting.
  • Ability to edit or annotate transcriptions after recording.
  

Student Lecture Services

• Student Lecture View:
  • Access lectures using the unique 6-character code with validation.
  • View real-time transcriptions streamed from the server with auto-scrolling.
  • Click on transcription bubbles to request AI assistance with context-aware responses:
    • Define Terms - Get concise definitions of technical concepts
    • Explain in Detail - Receive comprehensive explanations with examples
    • Provide Real-World Examples - See practical applications of concepts
    • Simplify (ELI5) - Get simplified explanations for complex topics
  • Request summaries of recent lecture content (Last 1, 5, 30 minutes).
  • Save favorite explanations for later review.
  • Search functionality within lecture transcripts.
  • Student engagement tracking with facial detection.

• Real-time Transcription:

  • Uses WebSockets to stream audio from the instructor's browser to the backend.
  • Forwards audio stream to OpenAI's Realtime Transcription API with error handling.
  • Transcriptions are saved to Firebase Realtime Database with timestamps.
  • Support for multiple concurrent lecture sessions.
  • Automatic handling of disconnections and reconnections.

• AI Explanations & Summaries:

  • Leverages OpenAI's Chat Completions API for intelligent responses.
  • Context-aware explanations based on selected transcriptions.
  • Ability to summarize lecture segments with customizable timeframes.
  • Support for generating lecture notes in multiple formats.
  • Optimized prompts for educational content.

• Engagement Analytics:
  • Face detection to measure student attention levels.
  • Aggregated engagement metrics for instructors.
  • Privacy-focused approach that doesn't store personal data.

• Security Features:
  • Secure authentication for instructors.
  • Rate limiting to prevent abuse.
  • Firebase security rules to protect data.
  • Input validation and sanitization throughout.

Tech Stack

• Backend:

  • Node.js v18+
  • Express.js v4.21.2 for API routes and serving static files
  • Socket.IO v4.8.1 for enhanced WebSocket communication

• Real-time Communication:

  • Socket.IO for reliable bidirectional event-based communication
  • Native browser WebSocket API for fallback

• Database:

  • Firebase Realtime Database v13.2.0 for lecture storage and real-time updates
  • Structured data model with security rules

• Authentication:

  • Express-session v1.18.1 for session management
  • Crypto module for password hashing and validation
  • Email verification with Nodemailer v6.10.1

• Transcription & AI:

  • OpenAI API v4.89.0 for transcription and AI capabilities
  • Streaming response handling for real-time updates

• PDF Generation:

  • LaTeX for professional-quality PDF exports
  • PDFKit v0.16.0 for dynamic PDF generation
  • Marked v15.0.7 for Markdown parsing

• Frontend:

  • HTML5, CSS3, Vanilla JavaScript (ES6+)
  • Modular JavaScript with ES Modules pattern
  • Responsive design for multi-device support
  • Face-api.js for facial detection capabilities

• Other Tools:

  • UUID v11.1.0 for unique identifiers
  • ShortID v2.2.17 for generating lecture codes
  • Multer v1.4.5 for handling file uploads
  • CORS v2.8.5 for cross-origin resource sharing
  • Dotenv v16.4.7 for environment variable management

Project Structure

.
├── client/
│   └── public/                  # Frontend static files
│       ├── images/              # Static image assets
│       │   ├── default-instructor.webp
│       │   └── scroll-feature-*.png
│       ├── models/              # Face detection models
│       │   ├── tiny_face_detector_model-shard1
│       │   └── tiny_face_detector_model-weights_manifest.json
│       ├── scripts/             # Frontend JavaScript
│       │   ├── advanced-animations.js  # Enhanced UI animations
│       │   ├── app.js           # Main application logic
│       │   ├── audioRecorder.js # Audio capture functionality
│       │   ├── config.js        # Frontend configuration
│       │   ├── engagement.js    # Student engagement tracking
│       │   ├── faceDetection.js # Face detection implementation
│       │   ├── firebase.js      # Firebase client integration
│       │   ├── instructor.js    # Instructor dashboard logic
│       │   └── landing.js       # Landing page functionality
│       ├── styles/              # CSS stylesheets
│       │   ├── advanced-effects.css  # Advanced visual effects
│       │   ├── animations.css   # Animation definitions
│       │   ├── main.css         # Core styling
│       │   └── main.css.additions  # Supplementary styles
│       ├── 404.html             # Custom 404 error page
│       ├── index.html           # Main landing page
│       ├── instructor_login.html  # Instructor login page
│       ├── instructor_signup.html # Instructor signup page
│       ├── instructor.html      # Instructor dashboard
│       ├── lecture.html         # Student lecture view
│       ├── student_dashboard.html # Student dashboard
│       ├── student_login.html   # Student login page
│       └── student_signup.html  # Student signup page
├── server/
│   ├── server.js               # Main Express server & WebSocket handling
│   ├── tmp/                    # Temporary file storage
│   │   └── uploads/            # Uploaded files (audio, PDFs)
│   └── utils/                  # Server utility functions
│       ├── auth.js             # Authentication utilities
│       └── verification.js     # Input validation
├── firebase-credentials.json   # Firebase service account credentials (PRIVATE)
├── package.json                # Project dependencies
├── README.md                   # Project documentation
└── .env                        # Environment variables (PRIVATE)

Setup and Installation

Prerequisites:

• Node.js (version 18.x or later recommended)
• Git for version control
• A Firebase Project with Realtime Database enabled
• An OpenAI API Key with access to the Realtime Transcription API and Chat Completions models
• LaTeX - BasicTeX distribution for PDF lecture notes generation

Steps:

1. Clone the Repository:

   git clone https://github.com/Capedbitmap/JS-Lecture-Transciption.git
   cd JS-Lecture-Transciption

2. Install Backend Dependencies:

   # From the project root directory
   npm install

3. LaTeX Installation:

   • For macOS:

     • Install BasicTeX (lightweight LaTeX distribution):

       # Using Homebrew (recommended)
       brew install --cask basictex
       
       # OR download and install the BasicTeX .pkg manually from
       # https://www.tug.org/mactex/morepackages.html

     • After installation, the LaTeX binaries should be available at /Library/TeX/texbin/.
     • Verify installation:

       /Library/TeX/texbin/pdflatex --version

   • For Linux:

     • Install TeX Live base package:

       # Ubuntu/Debian
       sudo apt-get install texlive-base
       
       # CentOS/RHEL/Fedora
       sudo dnf install texlive-scheme-basic

   • For Windows:

     • Install MiKTeX (basic version): Download from https://miktex.org/download and run the installer Choose the "Basic MiKTeX Installer" option
     • Ensure the LaTeX binaries are in your PATH

4. Firebase Setup:

   • Create a new Firebase project at Firebase Console
   • Enable the Realtime Database feature in your project
   • Set up Authentication with Email/Password provider
   • Go to your Firebase project settings > Service accounts
   • Generate a new private key and download the JSON file
   • Rename the downloaded file to firebase-credentials.json
   • Place this file inside the root directory of your project
   • Note your Database URL (e.g., https://your-project-id-default-rtdb.firebaseio.com)
   • Go to the "Rules" tab and paste the security rules provided in the Firebase Rules section below
   • Publish the rules

5. OpenAI API Key:

   • Create or log in to your account at OpenAI Platform
   • Navigate to API keys and generate a new key
   • Make sure your account has access to the required models:
     • Speech-to-Text API for transcription
     • GPT-4o-mini or other suitable models for AI responses
   • Ensure you have sufficient credits for your usage needs

6. Environment Variables:

   • In the root directory of the project, create a file named .env
   • Fill in the required values:

   # .env file
   
   # OpenAI API Key - Required for transcription and AI explanations
   OPENAI_API_KEY=sk-YourOpenAiApiKeyHere
   
   # Firebase Configuration - Required for database access
   FIREBASE_CREDENTIALS_PATH=./firebase-credentials.json
   FIREBASE_DATABASE_URL=https://your-project-id-default-rtdb.firebaseio.com
   
   # Session Secret (change this to a long, random string for production)
   SECRET_KEY=replace_this_with_a_strong_random_string_at_least_32_characters
   
   # Node Environment (development or production)
   NODE_ENV=development
   
   # Server port
   PORT=8080
   
   # Email Configuration (for verification emails)
   EMAIL_SERVICE=gmail  # or another service like 'outlook', 'yahoo', etc.
   EMAIL_USER=your-email@example.com
   EMAIL_PASS=your-app-specific-password
   
   # LaTeX Binary Path (adjust based on your OS)
   LATEX_PATH=/Library/TeX/texbin/pdflatex  # for macOS
   # LATEX_PATH=pdflatex  # for Linux/Windows if in PATH

7. Create Required Directories:

   # Create the temporary uploads directory if it doesn't exist
   mkdir -p server/tmp/uploads

8. Add to .gitignore: Create or update your .gitignore file to include:

   # Dependencies
   node_modules/
   
   # Environment variables and secrets
   .env
   .env.local
   .env.*.local
   
   # Firebase credentials 
   firebase-credentials.json
   *-credentials.json
   
   # Logs
   *.log
   npm-debug.log*
   
   # Runtime data and temporary files
   server/tmp/
   
   # OS files
   .DS_Store
   Thumbs.db
   

9. Frontend Configuration (API URL):

   • The frontend needs to know the URL of the running backend server. This is configured in client/public/scripts/config.js.
   • Find the line: export const API_URL = 'http://localhost:8080';
   • During development, this points to your local server.
   • IMPORTANT: When deploying the application to a live environment, you must update this URL to the public address of your deployed backend server (e.g., https://your-api.yourdomain.com).

10. Setup npm Scripts: Update your package.json scripts section for convenience:

    "scripts": {
      "start": "node server/server.js",
      "dev": "nodemon server/server.js",
      "test": "echo \"Error: no test specified\" && exit 1"
    }

Running the Application

1. Start the Server:

   • For development with automatic restarts:

     npm run dev

   • For production:

     npm start

   • You should see output indicating the server is running, such as:

     Lecture Assistant Server running on port 8080
     Connected to Firebase Realtime Database
     

2. Access the Application:

   • Open your web browser and navigate to:

     • Home/Student Page: http://localhost:8080
     • Instructor Login: http://localhost:8080/instructor_login.html
     • Instructor Signup: http://localhost:8080/instructor_signup.html

   • If you configured a custom port in your .env file, replace 8080 with your port number.

Authentication Endpoints

• POST /api/auth/instructor/signup - Register a new instructor
• POST /api/auth/instructor/login - Authenticate an instructor
• GET /api/auth/instructor/logout - End an instructor session
• GET /api/auth/session - Check current session status

Lecture Management

• POST /api/lectures/create - Create a new lecture
• GET /api/lectures/list - Get all lectures for the authenticated instructor
• POST /api/lectures/activate - Set a lecture as active
• GET /api/lectures/:code - Get details for a specific lecture
• PUT /api/lectures/:code - Update lecture metadata
• DELETE /api/lectures/:code - Delete a lecture

Transcription

• GET /api/transcriptions/:lectureCode - Get all transcriptions for a lecture
• GET /api/transcriptions/:lectureCode/recent - Get recent transcriptions
• PUT /api/transcriptions/:lectureCode/:id - Edit a transcription
• POST /api/transcriptions/:lectureCode/export - Generate a PDF export

AI Assistance

• POST /api/ai/explain - Get AI explanation for a transcription
• POST /api/ai/summarize - Generate a summary of recent content
• POST /api/ai/define - Define terms from a transcription
• POST /api/ai/simplify - Simplify a complex transcription
• POST /api/ai/examples - Get real-world examples related to a transcription