Multi-robot systems working with humans

Service robots such as automatic lawnmowers are becoming increasingly commonplace. More futuristic robots, capable of cooperating with each other and with people, will take a while longer, according to Dimos Dimarogonas. He is researching multi-robot systems capable of processing information from their surroundings, collaborating and making flexible decisions. The robots are being tested in various contexts, including office environments and an Italian vineyard.

Dimos Dimarogonas

Professor of Automatic Control

Wallenberg Academy Fellow, prolongation grant 2020

KTH Royal Institute of Technology

Research field:
Decentralized multi-robot systems, in which multiple robots can interact and collaborate with each other and with people

“Robot development involves many exciting challenges. I’m researching decentralized multi-robot systems for use in manufacturing, among other things. We’re developing applications for better logistics and delivery operations in a factory environment. A lot has happened over the past few years in our work, particularly in the field of multi-robot systems with people in the loop, and in what is called collaborative manipulation.”

In recent years Dimos Dimarogonas, who comes from Greece, has established a research team with a broad range of skills in automatic control at KTH Royal Institute of Technology. Team members possess expertise in mathematics, computer science, electronics and mechanics. A Wallenberg Academy Fellow, he has been given the opportunity to set challenges for himself as a researcher and team leader.

“And it has led to fine results. For example, one of my PhD students, Christos Verginis, recently received the 2021 European Systems & Control PhD Thesis Award for his doctoral thesis on interactive mobile robots. This was the first time someone from a Swedish university received the award – it was big news and uplifting for the entire team.”

“Most of all, the grant gives me ample academic freedom, which I appreciate. It has enabled me to adapt my research to the needs and key future trends we have identified.”

Heterogeneous robot platforms

Dimarogonas’ model studies combine different robot platforms: airborne robots with a simple manipulator, a kind of gripping arm, ground robots capable only of movement, advanced static robot manipulators capable of lifting heavy objects, and robots fitted with mobile manipulators of various kinds.

“The closest we get to humanoid robots are mobile robots fitted with a gripper. Maybe it’s easier for common man to relate to humanoid robots, but I still think we can achieve more applications using monolithic robots with simpler designs.”

The model systems are being tested in the Smart Mobility Lab at KTH. Is it, for instance, better to have a person who moves an object with a robot’s help than to have two robots?

The abilities of the different robot platforms can be seen as building blocks in a long-term autonomous scenario. Dimarogonas explains:

“Scenarios include performing day-to-day factory processes, or service robots for recurring tasks in an office environment. The way I see it, we will eventually have autonomous systems with a plan for performing tasks throughout the day. I want to see how we can make larger-scale multi-robotic tasks based on their abilities. It would be a great improvement for manufacturing and service applications.”

Safety and interaction

Over the next few years Dimarogonas wants to further explore multi-robot systems involving humans.

“We want people to be present, as workers or machine operators in agriculture or industry. And we’re interested in questions of safety but also co-existence.”

They are testing robot systems in Italian vineyards in collaboration with European partners in the EU CANOPIES project. 

“Agriculture and cultivation are suitable environments to work with – we can test advanced control algorithms without worrying about hundred percent success. It’s not the end of the world if we miss a few grapes. After all, there are other applications where a ninety-nine or one hundred percent success rate is vital.”

They are also studying aspects of the interaction itself, such as how a human can plug in new requirements in the robotic system in real time.

“To do this, we’ve developed user interfaces based on formal verification – a mathematical tool that can be used to ask control systems to perform a sequence of tasks rather than an individual task. We’re now moving toward more quantifiable specifications in time and space, such as saying to the robot ‘move five meters from this spot within ten seconds’. We’re also trying to develop collaborative transport and manipulation, where the robot lifts a very heavy object, for example, and the worker gives ‘hands-on’ instructions to the robot about where to take it.”

Mathematics and mobile sensing

All the work being done by Dimarogonas’ team relates to the three common denominators in his research: multi-robot systems, formal verification and collaborative control approaches.

“At present our attention is focused on quantification methods and models. This is very much about dynamic systems and applied mathematics, but we’re still building on the computer science know-how we’ve acquired in recent years. The applications we’re working on have moved on from standard navigation to mobile sensing and manipulation, so we also need to study sensory and communication-related limitations.”

Dimarogonas is also heading a new collaborative project under WASP – the Wallenberg AI, Autonomous Systems and Software Program. 

“The project concerns multi-robot systems in weightless environments like space and under water. It’s truly fascinating.”

Text Susanne Rosén
Translation Maxwell Arding
Photo Magnus Bergström