ROURKELA: National Institute of Technology Rourkela (NIT Rourkela) innovators have developed a robotic system designed to interact with people in the most human-like manner. By using Artificial Intelligence (AI) and Large Language Models (LLMs), the innovation focuses on creating seamless human-machine interaction by integrating verbal and non-verbal communication.
The developed system has the ability to understand everyday language, follow verbal instructions, respond to questions, and engage in live conversations beyond pre-coded responses.
Designed to serve as a friendly companion, the developed system is capable of interacting in homes, classrooms, offices, hospitals, and community spaces. NIT Rourkela has secured a patent for the developed innovation and system (Patent No. 574589, Application No. 202531022107).
The findings of this research have been published in the prestigious Computers and Electrical Engineering journal by Elsevier, in a paper co-authored by Dr. Anup Nandy, Associate Professor, Department of Computer Science and Engineering, along with Dr. Asim Kumar Naskar, Associate Professor, Electrical Engineering, and Research scholar, Mr. Sougatamoy Biswas, and M. Tech. graduate, Mr. Rahul Saw at NIT Rourkela.
The integration of AI in the developed system enables the robot to recognise users’ emotions by interpreting facial expressions such as happy, neutral, or upset. This feature enables the robot to respond in an empathetic and comforting manner, enhancing the user experience and quality of interaction.
Additionally, the system can detect simple gestures such as waving and raising a hand, and react appropriately. This ability makes the robot approachable for every age category of users, including children and elderly individuals who may rely more on natural gestures than spoken commands.
On the verbal communication front, the system can greet users, provide instructions, and reply using natural-sounding speech. The robot uses the Raspberry Pi system to process user inputs such as spoken commands or text-based queries. The LLM analyses the input, determines the context, and produces an appropriate, human-like response.
This output is then delivered through the robot’s speech system using Google Text-to-Speech. Speaking about the developed system, Dr. Anup Nandy, Associate Professor, Department of Computer Science and Engineering, NIT Rourkela, said, “The indigenously developed robotic system significantly aligns with the Government of India’s Make in India and Atmanirbhar Bharat initiatives.
In contrast to existing systems available globally, NIT Rourkela’s social robot introduces a uniquely integrated framework that combines gesture, emotion, speech, and LLM-based conversational abilities within a cost-effective platform suitable for Indian contexts.”
The robot is equipped with a wheel-based platform and navigation system for easy movement. The built-in distance-sensing module allows it to avoid obstacles and ensure safe movement in a crowded setup.
The developed system has broad potential to use across various social settings.
- In a domestic setup, the system can assist elderly individuals by recognising gestures, facial expressions, provide companionship, and respond to simple commands.
- In an educational space, it can support learning modules by engaging with students through gestures and provide conversation-based explanations for complex topics.
- In hospitals, workplaces, and community facilities, it can provide friendly assistance, help with navigation, and create a welcoming atmosphere. The robot’s ability to recognise speech and respond clearly to verbal and non-verbal cues enables it to interact naturally in a crowded setup.
The developed robot is expected to cost between Rs. 80,000 and Rs. 90,000 based on manufacturing scale and component optimisation. As a next step, the research team plans to improve the robot’s interaction capabilities, conduct real-world pilot testing in schools, hospitals, and community environments, and gather user feedback to refine its performance.
Additionally, the researchers are also looking for potential collaboration opportunities with organisations working in robotics, AI, and assistive technologies for technology commercialization.