Chercheur Doctorant H/F

Le poste proposé

0 has led robot manufacturers to develop their own programming software to equip their robots with intelligent capabilities. However, these programming environments have certain limitations: they are often proprietary and do not provide access to low-level programming layers. To address these constraints, robotic middleware emerged in the early 2000s. The concept is simple: robotic middleware is an additional software layer that can be installed on an existing operating system (such as Linux or Windows), which natively contains generic tools and libraries to facilitate the programming of robot intelligence capabilities (Sahni, Cao and Jiang, 2019). Despite these middlewares, programming robotic missions remains largely inaccessible to non-experts. This is the context in which the FUSION project is positioned, with its main objective being to democratize the use of robotics, particularly through the introduction of XR for the design of robotic missions via digital twins.

The development of the Tactile Internet (IEEE 1918.1) aims to create networks enabling real-time remote access, perception, manipulation, or control of real or virtual objects or processes by humans or machines. This is crucial for applications requiring high precision and low latency. In highly constrained environments where physical presence is impractical and where robots are sent instead, teleoperation becomes necessary. Digital Twins (DT)—virtual replicas of physical systems—combined with Extended Reality (XR) can be an effective solution to enhance and assist human-robot interactions (Feddoul et al., 2023; Havard et al., 2023).

Teleoperation assisted through XR is providing innovative methods for humans to interact with robots. Research demonstrates the efficiency of XR interfaces in teleoperating industrial robot manipulators, despite initial challenges such as limited control exposure (Rosen and Jha, 2023). Moreover, combining Extended Reality with Digital Twin (DTXR) techniques in multi-agents systems enable intuitive human-in-the-loop control to improve efficiency in industrial environments. XR addresses issues like distance and robots’ limitations perception, improving the immersive experience for operators engaged in remote manipulation tasks. This technology holds potential applications across social and high-risk environments, offering robust solutions for complex operational challenges.

DTXR provides real-time tracking of remote environments, reducing bandwidth requirements and enhancing operator awareness (Kremer, Nourani-Vatani and Park, 2023). In industrial applications, DTXR facilitate human-in-the-loop control of robots, streamlining coordination and improving operational efficiency (Li et al., 2022; Xu et al., 2022).

DTXR not only replicates physical systems, but also enables simulations that run parallel to real-time operations. This capability allows operators to test scenarios, refine algorithms, and predict outcomes without interrupting ongoing processes. By integrating real-time data from sensors and actuators, digital twins ensure that simulations accurately reflect current operational conditions, enhancing decision-making and performance optimization.

However, robots also have autonomous capabilities which allows them to freely execute their missions and thus alleviates teleoperator’s cognitive workload while doing it. But, in highly constraint environment and fine manipulation, teleoperator must be in control even partially of the robot. This approach, called shared-control, leverages the strengths of both human intuition and machine precision, enhancing task efficiency and flexibility. In DTXR environment, this synergy amplified, as operators can monitor and modify robot actions in real-time using XR interfaces, ensuring efficiency and adaptability in dynamic conditions (Li et al., 2023; Luo et al., 2024). 

Key challenges in DTXR-assisted teleoperation include issues related to perception of robotic autonomy during fine manipulation tasks and the need to ensure workspace visibility and limitations in operational contexts (Pryor et al., 2023). Addressing these challenges is crucial for optimizing operator load, enhancing task performance, and improving overall user experience in DTXR-assisted teleoperation systems.

PhD description

This thesis aims to explore and optimize XR interfaces for robotic shared control teleoperation within digital twin environments. By enhancing user-robot interactions and effectively visualizing robot capabilities and limitations while letting teleoperator to partially control the robot, this research will contribute improving the efficiency, safety, and ergonomics of teleoperation systems while reducing cognitive load on human operators. The integration of DT with XR not only replicates physical environments but also enhances decision-making through real-time simulations that run parallel to actual operations, ensuring robust performance optimization and scenario testing in dynamic industrial context.

How can we develop and optimize XR interfaces to enhance the shared-control teleoperation of industrial robots within digital twin environments, ensuring effective interaction and visualization of robots’ capabilities and limitations?

Methodology and research axes:

·      Develop XR-based Human-Robot Interfaces (HRI): Create immersive interfaces enabling users to interact with and comprehend the robot’s state, current and future actions, and limitations within digital twin environments.

·      Optimize Visual Feedback in DTXR: Design visual feedback systems within DTXR to deliver real-time updates on robot capabilities and current tasks effectively.

·      Enhance Interaction Mechanisms with shared control: Implement intuitive interaction techniques within DTXR, enabling operators to guide robots and react based on visual cues. This part will need using state of the art of implementation of:

1. Robots model with Mechanical Limitations: Develop physical models of robots within DT including mechanical constraints, such as joint limits, maximum force, and speed capabilities, reflecting real-time data from the physical robots to perform realistic simulations and predictions.
2. Path Planning and Prediction algorithms: Implement advanced path planning algorithms within DT to optimize robot trajectories, considering environmental constraints and task requirements. Integrate predictive models to anticipate potential issues and optimize robot actions, improving efficiency and safety in teleoperation tasks.

·      Evaluate Cognitive Load Impact: Assess the influence of DTXR interfaces on operator cognitive load, task performance, and overall user satisfaction.

Découvrez le métier