Project Overview

Autonomous robots have long been recognised for their potential to benefit various industries, and they have achieved significant success worldwide, particularly in controlled environments. Examples include autonomous warehouse robots and urban logistic robots. However, reliable high-level autonomy solutions (level 4 and 5) for field robots, which have a broader application spectrum and larger market, are still lacking because robust understanding of their diverse and complex 3D environments, safe navigation under dynamic and stochastic conditions, remain critically challenging.  

To address these challenges, the forefront of current robotics research largely revolves around integrating machine learning techniques. One promising solution is end-to-end deep reinforcement learning (DRL), which learns a policy directly mapping robotic sensory information to robotic actions. However, achieving satisfactory performance with DRL methods typically requires a large dataset, which is expensive to obtain due to the high cost of operating robots, or even infeasible for field robots as they may wear out before the training converges. Additionally, a DRL agent typically lacks safety guarantees due to its required exploration mechanism and has poor generalization capabilities in diverse and unseen environments. These current limitations hinder their real-world applications. The robotics community has proposed various techniques to overcome some of these limitations, including sim2real, offline deep reinforcement learning, safe reinforcement learning, decision transformers, and others. However, no solutions have been developed to overcome all the aforementioned limitations. 

On the other hand, the most recent era in artificial intelligence has witnessed the emergence of large multimodal models (LMMs), which possess the profound ability to offer a holistic understanding of the environment and its future states by processing multimodal sensory data. For instance, ImageBind, developed by Meta, can integrate data from six modalities – images, video, depth, thermal imaging, and inertial measurement units (IMUs) – to achieve (super)human-level performance in understanding sensory data. This advancement renders the traditional method of training multiple-task perception stacks, which aims to understand an autonomous robot’s world, unnecessary because LMMs are more versatile, general, and can provide a more reliable, effective, and deeper understanding of their environments. Additionally, the latest progress in AI computing makes it possible to integrate LMMs into the computing unit onboard the robot. However, LMMs are not error-free, so the next layers of mapping the output of LMMs to robot actions should be able to ensure that the robot can perform the allocated tasks normally, even in the presence of perception errors, and that these errors are not catastrophic to both the robot and its environment.

Aims

This project aims to integrate LMMs’ understanding of field mobile robots’ exteroceptive sensory data, and the outputs of robots’ interoceptive sensors to form a world model of the robot. Based on this new world model, we develop a DRL agent being able to map the state of the world model to robot actions directly to perform allocated tasks safely. The trained DRL agent aims to achieve zero shot transfer to unseen data.

Objectives

• To develop an effective  way to combine the outputs of both LMMs and robots’ interoceptive sensors to form the state representation for modelling  the robot’s world environments.  
• To design and train a sample efficient offline DRL agent that maps the world model state to actions for a specific autonomous mobile robot platform. 
• To develop a safe exploration mechanism to transform an offline DRL agent to online DRL agent for its fine tuning and deployment on a real autonomous robot platform. 
• To analyse the  agent’s performance of various parameters in the world modelling, DRL agent, and exploration mechanism, and identify the optimal parameter setting.  
• To test the effectiveness of the optimised autonomous system paradigm on a real mobile robot in multiple field environments.

Significance 

• The research represents a pioneering effort globally in harnessing the advantages of LMMs for robotic world representation. While LMMs have demonstrated remarkable success in various domains such as natural language processing and computer vision, their application in the field of robotics remains relatively uncharted territory. 
• The outcomes have the potential to establish a new paradigm of designing an effective, reliable, and safe autonomous systems, and thereby various industries reliant on autonomous systems, including agriculture, mining, search and rescue, etc.  
• The research can contribute to the growing body of knowledge concerning efficient and safe reinforcement learning techniques, and their application in autonomous robotic systems.

The PhD student will be supervised by two supervisors with complementary expertise in robotics and machine learning. They have strong track records of research excellence with numerous publications in top machine learning, computer vision, and robotics journals and conferences. Their extensive knowledge and guidance will lead the student to the successful completion of the project.  

Facilities for the project is ready, including 
• 10 autonomous mobile robots 
• 1 industrial grade mobile robot for field test 
• 1 powerful computer with high end graphic card for software development 
• 2 high performance servers with GPUs are available for training DRL agents 

Significance to the EECMS and Curtin 
• The ground breaking nature of the proposed research is poised to capture widespread attention from both academic and industrial audiences. Through strategic dissemination and outreach efforts, we can establish a prominent global presence in the field of autonomous robotics on behalf of Curtin. 
• EECMS plans to put robotics as one of its strategic directions and this scholarship will greatly support this plan.

• Future Students

• Faculty of Science & Engineering
  • Science courses
  • Engineering courses

• Higher Degree by Research

• Australian Citizen
• Australian Permanent Resident
• New Zealand Citizen
• Permanent Humanitarian Visa
• International Student

• Merit Based

The annual scholarship package, covering both stipend and tuition fees, amounts to approximately $70,000 per year.

In 2024, the RTP stipend scholarship offers $35,000 per annum for a duration of up to three years. Exceptional progress and adherence to timelines may qualify students for a six-month completion scholarship.

Selection for these scholarships involves a competitive process, with shortlisted applicants notified of outcomes by November 2024.

1

All applicable HDR courses.

We are seeking a self-motivated PhD candidate who: 
• Is eligible to enrol in PhD programs at Curtin University. 
• Holds at least a bachelor’s degree in a relevant field such as Robotics, Computer Science, Engineering (Electrical, Mechanical, Mechatronics), Statistics, or Optimization. 
• Demonstrates proficiency in conducting research in both robotics and machine learning. 
• Possesses skills in Python programming and application of Machine Learning tools, along with experience with ROS 2 ecosystem. 
• Exhibits excellent communication skills and works effectively in a team environment.

 

Please send your CV, academic transcripts and brief rationale why you want to join this research project via the HDR Expression of Interest form to the project lead researcher, listed below. 

You must be enrolled in a Higher Degree by Research Course at Curtin University by March 2025.

Project Lead: Dr Hui Xie