Navlead
3D Navigation Assistance Chatbot
NavLead is an AI-driven chatbot that gives navigation assistance in a 3D enviroment.
The chatbot was trained on real-world navigation assistance dialogs to give natural responses.
The 3D enviroments were reconstructed form real-world houses.
No downloads are neeeded; both the chatbot and 3D enviroments are on this site.
Demo instructions:
Choose a target household item
Navigate the house (using WASD keys)
Ask the chatbot for assistance
How does it work?
Each question is processed roughly as follows:
- The 3D enviroment is displayed on your browser. Then you ask a question to the chatbot.
- This pages sends the question along with the user's location in the 3D enviroment to an API server requesting the answer.
- The API server validates and authentificates the request.
- The server runs the NavLeadNet deep learning model. It outputs a textual answer.
- Finally the answer is sent all the way back to the chatbot.
3D simulated enviroment
The enviroment used in the demo is a reconstruction of Frank Lloyd Wright's final design: the circular sun house.
It was constructed using Blender and exported to Unity to run on your browser.
The chatbot's UI was then added in Unity as well.
NavLeadNet
NavLeadNet is the navigation guidance deep learning model used here.
This model is a sequence-to-sequence architecture that takes the traveler's question and location (as input sequence) and, then, returns a textual answer (as output sequence).
It was trained using the CVDN (Cooperative Vision-and-Dialog Navigation) dataset. CVDN is a dataset of human-human dialogs situated in simulated, photorealistic home environments. The 3D enviroment in the demo was reconstructed based on one of those enviroments.