COLLABORATE El Meu Clínic . This link opens in a new tab.
Search
COLLABORATE El Meu Clínic . This link opens in a new tab.
Navigation thread
Clínic Barcelona

BEAMING: Enabling real-time interaction between humans and between human and rat

The TV series Star Trek originally popularised the idea of instantaneous transportation of people to distant places. This involved decomposition of human bodies and objects into their atomic constituents, and then reconstruction at the remote location. Clearly such technology is perhaps centuries away even if it could ever be realised. Today there is an alternative technology that aims to realise the same idea but through quite different means. We refer to the idea of ‘beaming’ as digitally transporting a representation of yourself to a distant place, where you can interact with the people there, as if you were there.

Prof. Mel Slater is the ICREA Professor scientifically leading the European Project BEAMING from the Faculty of Psycology of the University of Barcelona, working together with international collaborators from around Europe including the team led by Dr. M. Victoria Sánchez-Vives, ICREA professor at Institut d’Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS). This technology has been used to interact physically between humans far away from each other. The BEAMING consortium, including partners from LUNAM Université (Nantes, France), Guger Technologies (Austria), University College London (UK) and Technical Univerity of Munich (Germany), publishes today in PLOS ONE a new study where the interactive experience involved a human and a rat, each at their own scale.

This is achieved through a combination of virtual reality and teleoperator systems. The visitor to the remote place (the destination) is represented there ideally by a physical robot. Such rapid transportation to distant locations, where you have the strong feeling to be there, and where the local people in the destination experience you as there, has many economic and practical advantages. It is a step beyond approaches such as video conferencing which do not give participants the physical sensation of being in the same shared space, and certainly not the physical capability to actually carry out actions in that space.

The European BEAMING project (www.beaming-eu.org) has achieved early examples of this goal, in relation to humans beaming to distant places and interacting with people there. For example, see how a scientist in Barcelona was digitally beamed to London to be interviewed by a BBC journalist there. It is just one of the multiple possible applications of such technology. In the framework of this project a system for remote medical assistance for patients at home that can be “transported” to the hospital is also being developed.

In this new paper to be published today in PLoS ONE researchers from the University of Barcelona and IDIBAPS further extend the beaming idea by showing how it is possible to beam to even what might be considered as ‘alien worlds’. We show how a person can be ‘beamed’ into a rat open arena, where the person interacts with the rat as if it were another person, and the rat interacts with a rat-sized robotic representation of the distant person. This not only shows the range of this technology, but also provides a new tool for scientists, explorers or others to visit distant and alien places without themselves being placed in any kind of danger, and importantly, to be able to see animal behaviour in a totally new way - as if it were the behaviour of humans.

A combination of several different technologies were used:

  • Virtual reality places people in an alternate computer-generated world where they can look and move around, carry out tasks - all at life-size and perceived in stereo 3D. People generally have the illusion of presence - that is of being in the computer-generated space as if it were a real space. Thus the human participants in the system were in a virtual reality lab at the event Lab which is at the Mundet campus of the University of Barcelona. The rat was located about 12 km away in an animal care facility in Bellvitge.
  • Tracking technology was used to track the movements of the rat in its arena, and the tracking data was transmitted over the internet to the computers running the virtual reality simulation in Mundet. This tracking information was used to control a virtual human character (an avatar) that represented the rat. Hence whenever the rat moved its avatar moved too, in a representation of the rat arena but scaled up to human size. Hence the human participant shared the virtual arena (which looked like a room with some pictures on the walls) with a humanoid avatar.
  • Teleoperation technology in conjunction with tracking was also used. The movements of the human in the virtual reality were also tracked, and this tracking data was sent to computers in Bellvitge which controlled a small robot that was located in the rat arena. So whenever the human moved in the virtual space so the robot moved in the rat space.
So putting all this together - the rat interacted with a rat sized robot that represented the remotely located human, and the human interacted with a human sized avatar that represented the remotely located rat. In order to make the rat interested in the robot, a small tray attached to the robot body had some jelly on it, which the rats had previously eaten in some training sessions.

The humans in a small study interacted with the rat. They had to learn to entice the rat-avatar to go near some specific pictures in the virtual reality. The humans didn’t know about the jelly on the robot, but they typically learned that the rats would follow them, so they had to draw the rats out of their normal behaviours (staying in the corners and staying close to the edges) to get them to move around so that both could stand by the same picture.

Overall the study showed that that the system technically performed well and that there could be an interesting interaction between the animal and the human and remotely located beings at different scales.