The technologies used in this application are :
- NLP Conversational Model
- Google Dialogflow
- Large Language Model
- Google Vertex AI (Palm2)
- Computer Vision Model
- Roboflow
- Ultralytics YoloV8
- Web Application
- Gradio
The goal of the chatbot is to help people increase their nutritional awareness by creating a chatbot experience.
The NLP Conversational Model has been developed using the following technologies:
- Google Dialogflow
- Python
- Node.Js
The persona of this chatbot is anyone that is concerned about his health and wants to have visibility on his eating.
Some further future improvements to be implemented:
- Add identification in the frontend and improve context awareness by having access to previous conversations of the user once this one is identified.
- Have a non-relational database linked to each account to keep track of each account's data.
- Create a Web Application that, upon identification, visualize your data.
- Continuosly store and process the 80% FAQ in order to improve the usefulness and robustness of the conversational model to user's most common prompts.
The NLP Conversational Model has been developed using Google Dialogflow, the intents have been architected using the following schema:
Some of the answers provided by the chatbot will be returned using Vertex AI and the LLM model offered by Google. In order to do so the interaction between the Conversational Model and Vertex AI has been coded using Python.
Entities utilized In order for the intents to work properly, the following entities have been created:
- @entity_foodtype: Set of all the food types that the computer vision model can recognize
- @entity_gender: Gender types, pre-built entity
- @entity_meal: Type of meal, breakfast, lunch or dinner
The goal of the Computer Vision Model is to recognize the food in the picture
The Computer Vision Model has been developed using the following technologies:
- Roboflow
- Ultralytics YOLOV8
- Python
Some further future improvements to be implemented:
- Create a more accurate model by increasing the number and diversity of the train images (i.e., foods from different cultures).
- Design a portion estimator that won't need to ask the user for the total number of grams of the dish.
The goal of the Web Application is to render a chat bot with which the user can interact by text and images
The Web Application has been developed using the following technologies:
- Python
- Gradio
Some further future improvements to be implemented:
- Fix the picture upload functionality in order to be processed by the Computer Vision model instead of submitting the image local path.
