Welcome to MedBot, an AI-powered medical diagnosis chatbot that uses YOLO-based disease detection and Gemini API for medical insights. This project is designed to provide users with AI-assisted medical diagnosis and advice through an interactive and user-friendly web interface.
-
CNN-Based Disease Detection:
- Uses YOLO models for real-time detection of medical conditions (e.g., eye diseases, lung cancer, fractures).
- Supports multiple disease models and accepts X-rays, CT scans, MRI scans, or eye images.
-
Multi-Model AI Chatbot:
- Integrates multiple CNN models to create a multi-specialist AI chatbot.
- Automatically detects which model to use based on the uploaded image.
-
Interactive Web UI:
- Built with Streamlit for a smooth and accessible user experience.
- Allows users to upload medical images, receive detection results, and interact with the chatbot.
-
Gemini-Powered Medical Insights:
- Utilizes the Gemini API to provide contextual medical insights.
- Explains possible causes, basic medical advice, and when to seek professional consultation.
-
Multimodal AI Chatbot:
- Accepts both text and image inputs.
- Answers general medical queries even without image uploads.
-
User-Friendly Interaction:
- Provides multilingual support and text-to-speech (TTS) options for better engagement.
- Python 3.8+
- Streamlit
- Ultralytics (for YOLO models)
- Gemini API key
-
Clone the repository:
git clone https://github.com/vendotha/Medical_Diagnosis_Chatbot.git cd Medical_Diagnosis_Chatbot -
Install the required dependencies:
pip install -r requirements.txt
-
Set up your Gemini API key:
- Replace
"your_gemini_api_key"inutils/chatbot.pywith your actual API key.
- Replace
-
Run the Streamlit app:
streamlit run app.py
-
Open your browser and navigate to
http://localhost:8501.
🌐 Live Demo Try MedBot now: https://medibot-r454.onrender.com
Check out the video demo of MedBot below:
Medical_Diagnosis_Chatbot/
│
├── app.py # Main Streamlit application
├── style.css # Custom CSS for styling
├── logo.png # Logo image for the header
├── models/ # Directory for YOLO models
│ ├── eye_disease.pt # Pre-trained YOLO model for eye diseases
│ ├── lung_cancer.pt # Pre-trained YOLO model for lung cancer
│ └── fracture.pt # Pre-trained YOLO model for fractures
├── utils/ # Utility functions
│ ├── disease_detection.py # YOLO model inference and plotting
│ └── chatbot.py # Gemini API integration
├── requirements.txt # Python dependencies
└── README.md # Project documentation
-
Image Upload:
- Users upload a medical image (X-ray, CT scan, or eye image).
- YOLO detects diseases and provides bounding boxes, labels, and confidence scores.
-
Gemini Analysis:
- The annotated image and YOLO output are sent to the Gemini API for analysis.
- Gemini provides a diagnosis, possible causes, medical advice, and recommendations.
-
Chatbot Interaction:
- Users can ask questions about the detected condition or general medical queries.
- The chatbot provides confident and professional responses.
- Streamlit: For building the interactive web UI.
- YOLO (Ultralytics): For real-time disease detection.
- Gemini API: For generating medical insights and answering queries.
- OpenCV: For image processing and annotation.
- Pillow: For image handling.
This project is licensed under the MIT License. See the LICENSE file for details.
Contributions are welcome! If you'd like to contribute, please follow these steps:
- Fork the repository.
- Create a new branch (
git checkout -b feature/YourFeatureName). - Commit your changes (
git commit -m 'Add some feature'). - Push to the branch (
git push origin feature/YourFeatureName). - Open a pull request.
For any questions or feedback, feel free to reach out:
- Email: vendotha@gmail.com
- GitHub: vendotha
- LinkedIn: Buvananand Vendotha
- Thanks to Google for the Gemini API.
- Thanks to Ultralytics for the YOLO models.
- Thanks to the Streamlit team for the amazing framework.
Check out the live demo of MedBot here (coming soon).
Made with ❤️ by Bhuvan Vendotha.