Memo-GPT

A complete Software As a Service application that helps users take notes during meetings, providing summaries of recorded or uploaded audio, chat about the meeting or audio with a GPT-based AI and query the AI for specific answers within the audio without the necessity of listening to the entire audio.

This SAS application effectively eliminates the need for manual note-taking during meetings, lectures, and conversations, as it autonomously does it for you with intelligence.

---

VIDEO DEMO

This video demonstration emphasizes the essential features of the app using the Gradio interface. If you prefer to skip the recording process and go straight to the AI demo, feel free to jump to 3:10.

memogpt_main_demo.2.mp4

---

Memo-GPT is an advanced AI-powered application designed to enhance your meeting experiences. It offers audio recording, audio file uploading, transcript generation, and summarization. Additionally, it enables querying/chatting and provides responses based on the meeting content, complete with timestamps and speaker diarization. This tool aims to increase efficiency and elevate the outcomes of your meetings.

Features

• Audio recording

  • Start and stop recording

• Audio file uploading

• Speech transcription using AssemblyAI API

• Text summarization using the Meta's LLama-2 API

• Conversational question answering based on the transcript using Meta's LLama-2 70b API

• User account management and authentication

• Email verification on signup

• Premium subscriptions

  • Limits on transcription length for free accounts
  • Unlimited transcription for paid accounts

• Payment processing with Paystack

• FRONTEND with Gradio

Technologies

• Python
• FastAPI
• MySQL
• SQLAlchemy
• JWT Authentication
• OAuth2 Password Flow
• AssemblyAI API
• LLama API
• Paystack API
• Gradio
• ffmpeg

Folder Structure

├── database
│   ├── db.py
│   └── model
│       ├── transcript.py
│       └── users_model.py
├── schema
│   ├── transcript.py 
│   └── users_schema.py
├── services
│   ├── assemblyai_services.py
│   ├── audio_services.py
│   ├── history_services.py  
│   ├── llama_services.py
│   ├── premium_services.py
│   ├── storage_services.py
│   ├── tokenizer_services.py
│   └── user_services.py
├── tmp
├── .env
├── .gitignore
├── audio.py
├── email_setup.py
├── email_verification.py
├── hashing.py
├── history.py 
├── llama.py
├── main.py
├── main_gradio.py
├── oauth.py
├── paystack.py
├── prompts.py
├── README.md
├── token_key.py
├── user.py
└── user_login.py

Database Models

The main database models are:

• User - Stores user account info
• Audio - Stores audio files and transcripts
• Summary - Transcript summaries generated by LLama
• History - Conversation history with the LLama bot

Relationships are defined between user, audio, summaries and history.

API Endpoints

The API endpoints are organized by HTTP method below:

GET Endpoints

• GET /verify/{token} - Verify email

• GET /audio - Stream audio

• GET /webhook_verification - Verify webhook

• GET /audio/data - Get audio data

• GET /all_chats - Get all chats

POST Endpoints

• POST /signup - Create user

• POST /login - Login user

• POST /audio/update_data - Update audio

• POST /pay - Process payment

• POST /response - Get bot response

• POST /audio/start - Start recording

• POST /audio/play - Play audio

• POST /audio/upload - Upload audio

PUT Endpoints

• PUT /delete_account - Delete account

DELETE Endpoints

• DELETE /delete_chat/{id} - Delete chat

Gradio UI

This project incorporates a Gradio user interface for demonstration and testing purposes. The objective is to facilitate comprehension of the project for non-technical individuals, without the necessity of interacting with the API endpoints.

Premium Accounts

To access transcripts over 5 minutes users need a premium account. This is handled via:

• Premium pricing plans defined in users_schema.py
• paystack.py handles payments with Paystack API
• premium_services.py checks if user account is premium
• Duration stored in User model to track when premium expires

Installation

1. Clone the repository

git clone https://github.com/user/meeting-assistant

2. Install dependencies

pip install -r requirements.txt

3. Set environment variables

Copy the .env.example file to .env and fill in values for:

• DB_URL
• AssemblyAI_API_KEY
• LLama_API_KEY
• Paystack_SECRET_KEY
• SMTP_Credentials

4. Initialize the database

alembic upgrade head

5. Run Gradio frontend

python main_gradio.py

6. Run the server API

uvicorn main:app

The Gradio UI will be available at http://127.0.0.1:7860

The API will be available at http://localhost:8000

Contributing

Contributions are welcome! Please open an issue or PR if you would like to help add features or fix bugs.