DriveSense: AI Assistant for Car Navigation with Obstacle Detection and Retrieval-Augmented Generation
Advanced Smart Car Assistant System
This project is an advanced smart car assistant system that combines conversational AI with video processing and machine learning capabilities. Here's a brief overview of how the system is designed and integrated:
- Langchain and Llama-3.1 are used to handle natural language processing tasks. The assistant can interpret and respond to user commands, utilizing large language models to provide relevant information and perform actions.
- Hugging Face Models are incorporated for enhanced language understanding and generation.
- YOLO (You Only Look Once) is employed for real-time object detection in video streams. The assistant can process video feeds to detect specific objects, such as potholes, and annotate the video with bounding boxes and labels.
- The system uses OpenCV to handle video capture, processing, and output. Detected objects are highlighted in the video, which is then saved for further analysis.
- Ultralytics YOLO is used for training custom object detection models. The system includes functionality to train a YOLO model on a specified dataset and save the trained model for future use.
- A dataset is extracted from a ZIP file and used to train the model, which is then utilized for object detection tasks in video processing.
- A FastAPI server provides an interface for interacting with the smart car assistant. Users can send commands to the server, such as requesting video processing or model training, and receive responses accordingly.
This system combines state-of-the-art AI and machine learning technologies with practical video processing capabilities to create a versatile and intelligent smart car assistant.
Clone the project
git clone https://github.com/TensorTrove/innovent-projectGo to the project directory
cd innovent-projectInstall dependencies
pip install -r requirements.txt
Now run the jutyper Notebooks as usual
