During my master's semester project at the Spinal Cord Injury & Artificial Intelligence (SCAI) Lab at ETH Zürich, I worked on Vision-Language Navigation (VLN) in indoor environments using only frontal camera input. The goal was to follow natural language instructions and reach the specified target in the R2R dataset, addressing the challenge of monocular camera navigation in continuous environments — compared to prior methods relying on panoramic input and pre-defined graph environments.
Our research explored the use of 3D Feature Fields to reconstruct panoramic views and integrated a waypoint predictor with a hallucination module, enabling action predictions beyond the visible frontal view. The project was conducted in collaboration with PhD student Rashid Alyassi and leveraged state-of-the-art techniques from VLN research to improve navigation in monocular settings.