Some well-used robot fingers lie on a table in Kragic Jensfelt’s laboratory. They have been used many times over in experiments in which robots practice seeing and gripping objects, and interacting with their surroundings.

“Much of my previous research has been about automating difficult tasks – enabling robots to recognize objects of various kinds, pick them up, and do something with them. But in recent years I have become increasingly interested in understanding human behaviors, and how we can use different robotic abilities in interaction with humans.”

Kragic Jensfelt explains that this is possible now that computer scientists know how automated systems can use copious quantities of data to learn more about the world around them – a process known as “machine learning”.

These data may come from video sequences, for instance. A person is filmed performing a task. Advanced algorithms and software can then be used to interpret what the person is doing, and incorporate that information in a robot system.

“My next step is to use this technology in situations where people and robots cooperate in some way – where robots do what robots are good at, and people do what they do best.”

“This is an extremely large and broad project. What’s so fantastic about the Wallenberg Scholar program is that it gives us opportunities to adopt an interdisciplinary approach, and discover new things together.”

Deviant behaviors

As a Wallenberg Scholar, Kragic Jensfelt wants to develop robots that can function in environments with people without being seen as frightening. She is particularly interested in understanding deviant behaviors.

“For instance – right now we’re having a conversation. We look at one another and register each other’s hand movements. We exchange information on multiple planes. Yet while you are talking, you are doing something else – you are writing.”

She swivels round in her office chair and draws on the whiteboard behind her.

“The machine learning component of this is about discovering language, movements, and how many situations there are in a conversation, for example, and also what is most important. We don’t have the vocabulary yet – there is no book describing the grammar for these situations.”

The research is interdisciplinary, and Kragic Jensfelt is collaborating with both neuroresearchers and behavioral scientists. Among other things, she has recruited a PhD student from a gaming company who already has a PhD in mathematics.

“Modeling these problems involves a lot of mathematics. And it’s interesting to involve someone from the gaming industry, because they learn a lot about human behaviors in their business.”

Explosive trend

Technologies such as machine learning, and its variant “deep learning”, are also called data-driven artificial intelligence – AI. These are expected to change the approaches to work in many sectors. They can be used in the insurance industry, for example, to determine premiums, or in health care to make better diagnoses.

“It’s often necessary for the systems to learn how people ‘on the other side’ think and act. In other words, human processes are recreated and analyzed using mathematical algorithms. This also enables us to create systems that are more objective, and that can use more information than a person can. But we also have to consider data protection and privacy when the systems collect data. This is a factor that is much discussed.”

Data-driven AI is also a central aspect of a national research program known as WASP – Wallenberg AI, Autonomous Systems and Software Program, involving participants from the academic world and industry. Kragic Jensfelt is responsible for the machine learning and deep learning initiative.

“We can achieve a great deal by collaborating. Many sectors have a shortage of AI experts. WASP is training holders of PhDs, who can then be employed in the private sector, and in the academic world. My role as an academic includes explaining the research objectively – educating society. Demonstrating the potential, but also describing the challenges that the technology poses.”

Drinking coffee

The complex work is done in stages. In the lab Kragic Jensfelt and her team are working on a sequence of events in which a robot learns to serve coffee in a natural way. Time series and movements are converted into a computer model.

“It’s not so much about the social aspects to start with. The question is more what we do when we serve coffee. We start with a coffee cup and a pot of coffee. In the end, coffee has been poured out into the cup. An awful lot happens in the interim, in an infinite number of ways, and with numerous parameters”.

The goal is a system in which the robot looks at the person, and can foresee what will happen next.

“We want the robot to be able to change the way it performs tasks based on what is true right now. Systemically speaking, this need not be all that different from when an insurance company receives information about your current status. So we see that the methods we are using are essentially general, whatever their application, since it’s a question of adapting to altered circumstances.”

Text Susanne Rosén
Translation Maxwell Arding
Photo Magnus Bergström