1.1.1 Back to the Roots

The basic vision of robotics and AI can be traced back to twelfth-century Europe.Footnote ¹ Literature from this period mentions a mystical creature called the golem, which had a human-like shape but was significantly stronger than a normal human. The golem was described as a harmless creature used by its creator as a servant. In the legend of the golem of Prague, first written down at the beginning of the nineteenth century, Rabbi Löw created the golem to relieve him of heavy physical work and to serve humans in general.Footnote ² The real-world realization of this idea had a long way to go. Some of the earliest scientific writings relating to machine intelligence date back to the fifteenth century, the period of the Renaissance. Leonardo da Vinci (1452‒1519), the universal savant of his time,Footnote ³ decisively influenced both art and science with a variety of inventions, including, for example, a mechanical jumper, hydraulic pumps, musical instruments, and many more. However, the two inventions that stand out from a robotics point of view were Leonardo’s autonomous flying machine and his mechanical knight, also known as Leonardo’s robot.Footnote ⁴ The latter is a mechanism integrated into a knight’s armor, which could be operated via rope pulls and deflection pulleys, enabling it to perform various human-like movements ‒ clearly first steps in robotics. Wilhelm Schickard (1592‒1635)Footnote ⁵ developed and built the first known working mechanical calculator. It was a gear-based multiplication machine that was also used for some of Kepler’s lunar orbit calculations.

Sir Isaac Newton (1642‒1726), one of the world’s greatest physicists, is best known for laying the foundations of classical physics by formulating the three laws of motion.Footnote ⁶ He was also an outstanding mathematician, astronomer and theologian. In the field of mathematics, he developed a widely used technique for solving optimization problems (nowadays called Newton’s method) and founded the field of infinitesimal calculus. Gottfried Wilhelm Leibniz (1646‒1716) worked in parallel with Newton on this topic but conceived the ideas of differential and integral calculus independently of Newton.Footnote ⁷ Leibniz, who is known for various other contributions to science, is often referred to as one of the first computer scientists due to his research on the binary number system. Slightly later, Pierre Jaquet-Droz (1721‒1790) built amazing mechanical inventions such as The Writer, The Musician and The Draughtsman.Footnote ⁸ The Draughtsman, for example, is a mechanical doll that draws with a quill pen and real ink on paper. The input device was a cam disk that essentially functions as a programmable memory defining the picture to be drawn. With three different cam disks, the The Draughtsman was able to draw four different artworks. In addition to these fascinating machines, Jaquet-Droz and his business partner Jean-Frédéric Leschot later started to build prosthetic limbs for amputees.

Another memorable figure in the history of machine intelligence is Augusta Ada Byron King (1815‒1852).Footnote ⁹ The Countess of Lovelace is known to be one of the first to recognize the full potential of a computing machine. She wrote the first computer program in history, which was designed to be used for the theoretical analytical engine proposed by Charles Baggage. The programming language Ada was named after her. These fundamental technological advances in the areas of mechanics, electronics, communications and computation paved the way for the introduction of the first usable computing machines and control systems, which began around 1868. The first automatic motion machines were systematically analyzed, documented, reconstructed, and taught via collections of mechanisms.

A mechanism can be defined as an automaton that transforms continuous, typically linear, movements into complex spatial motions. Ludwig Burmester (1840‒1927) was a mathematician, engineer and inventor, and the first person to develop a theory for the analysis and synthesis of motion machines.Footnote ¹⁰ Later in this period, Czech writer and dramatist Karel Čapek (1890‒1938) first used the word “robot” in his science-fiction work. The word “robot” is derived from robota, which originally meant serfdom, but is now used in Czech for “hard work.” Through his 1920 play R.U.R. (Rossums Universal Robots), Čapek spread his definition of robot to a wider audience.Footnote ¹¹ In this play, the robots were manufactured to industry standards from synthetic organic materials and used as workers in industry to relieve people from heavy and hard work.

We now come to the pre-eminent philosopher and mathematician Norbert Wiener (1894‒1964). From his original research field of stochastic and mathematical noise processes, he and his colleagues Arturo Rosenblueth, Julian Bigelow and others founded the discipline of cybernetics in the 1940s.Footnote ¹² Cybernetics combines the analysis of self-regulatory processes with information theory to produce new concepts, which can be said to be the precursors of modern control engineering, thus building significant aspects of the theoretical foundations of robotics and AI. Wiener developed a new and deeper understanding of the notion of feedback, which has significantly influenced a broad spectrum of natural science disciplines. Alan Turing (1912‒1954) worked in parallel with Wiener in the field of theoretical computer science and artificial intelligence.Footnote ¹³ Most people interested in artificial intelligence today are familiar with his name through the Turing test. This test was devised to determine whether a computer or, more generally a machine, could think like a human. His groundbreaking mathematical model of an automatic calculating machine that can solve complex calculations is today known as a Turing machine. The Turing machine models the process of calculating in such a way that its mode of operation can be easily analyzed mathematically, making the terms “algorithm” and “computability” mathematically manageable for the first time.

A similarly renowned researcher and colleague of Turing was John von Neumann (1903‒1957).Footnote ¹⁴ He developed the von Neumann computer architecture, which still forms the basis of the operation of most computers today. As well as collaborating with Turing on AI research, he also worked on other mathematical topics like linear programming and sorting programs. Von Neumann’s concept of self-reproducing machines, developed in 1940, testifies to his outstanding capabilities.Footnote ¹⁵ The aim of this concept was to describe an abstract machine, which, when in operation, replicates itself. To achieve this goal von Neumann also developed the concept of cellular automata. According to von Neumann, a cellular automaton is a collection of states in a two-dimensional grid of cells, which forms a certain pattern. A cell represents one of twenty-nine possible states, which can change over time. The change of state of a cell is determined by the states of the neighboring cells from the previous time step as input. The theory of cellular automata defined the elementary building blocks responsible for the concept of self-replicating machines. With these building blocks, von Neumann created the universal constructor, which is a particular pattern of different cell states. This pattern contains three different sub-units: an information carrier for storing its own construction plan, a construction arm, which builds itself up in the free grid according to the construction plan, and a copying machine for copying the construction plan. This made it possible for von Neumann to develop a self-replicating machine within the concept of cellular automata.

A famous mathematician and inventor who also worked in the field of digital computing is Claude Elwood Shannon (1916‒2001). His groundbreaking ideas on logical circuit design for digital computers and information theory had an enormous impact on the research community of his time, and continue to do so today. In 1948, with his book A Mathematical Theory of Communication,Footnote ¹⁶ he laid important foundations for today’s high-speed telecommunications and data processing by mathematically tackling the problem of data transmission via a lossy communication channel. He developed a coding algorithm that made it possible to restore the originally transmitted information from previously coded lossy data. In a further publication,Footnote ¹⁷ he developed a complete theory of channel capacity, which defined the maximum data rate that can be transmitted lossless over a specific communication channel type. In 1949, he published the formal basics of cryptography, thus establishing it as a scientific discipline.Footnote ¹⁸

At the beginning of 1941, the engineer and computer scientist Konrad Zuse (1910‒1995) made headlines with the world’s first functional programmable digital computer, the Z3, built in cooperation with Helmut Schreyer.Footnote ¹⁹ Zuse also demonstrated that machines can assemble themselves on a variable scale, long before the idea of robotic assembly systems had been conceived.Footnote ²⁰ Based on John von Neumann’s ideas and proofs that it is theoretically possible to build a machine that can reproduce itself, Zuse published his implementation ideas for such a machine in the journal Unternehmensforschung under the title “Gedanken zur Automation und zum Problem der technischen Keimzelle” (“Thoughts on Automation and the Problem of the Technical Germ Cell”).Footnote ²¹ In the 1970s he designed the assembly robot SRS72 in his own construction workshop as a functional demonstration of this idea. The SRS72 machine could automatically assemble prefabricated manually supplied parts by positioning two work pieces and connecting them with screws. This prototype machine was the starting point for a complete self-reproducing system. According to Zuse, an entire automated workshop is required to perform all the complex manufacturing and assembly steps necessary to obtain a self-producing system.Footnote ²²

Independently of Zuse, the physicist Richard Phillips Feynman (1918‒1988) also studied von Neumann’s ideas. His own research area was quantum field theory, and he was awarded the Nobel Prize in 1965 for his work on quantum electro dynamics. Today, however, he is also regarded as a visionary of self-reproducing machine technology. His famous lecture, “There’s Plenty of Room at the Bottom,” on the future opportunities for designing miniaturized machines that could build smaller reproductions of themselves was delivered in 1959 at the annual meeting of the American Institute of Physics at the California Institute of Technology and published the following year in the journal Engineering and Science.Footnote ²³ Feynman’s speech is frequently referenced in today’s technical literature in the fields of micro- and nanotechnology, which speaks for the high regard in which his early vision is held in expert circles.

Very few people had the knowledge and skills to program complex early computing machines like the Z3 computer. Unlike today’s programming languages that use digital sequence code, these machines were programmed with the help of strip-shaped data carriers made of paper, plastic or a metal-plastic laminate, which store the information or the code lines in the punched hole patterns. One person who mastered and shaped this type of programming was American computer scientist Grace Hopper (1906‒1992).Footnote ²⁴ She did not work with the Z3, but on the Mark I and II computers she designed the first compiler called A-0. A compiler is a program that translates human readable programming code into machine-readable code. She also invented the first machine-independent programming language, which led to high-level languages as we know them today.

Returning to robotics in literature, a short story that still exerts a powerful influence on real-world implementation of modern robotics and AI systems as we know them today is Isaac Asimov’s (1920‒1992) science-fiction story “Runaround,” published in 1942, which contained his famous “Three Laws of Robotics”:Footnote ²⁵

Asimov’s early ideas, including his vision of human‒robot coexistence, paved the way for the concept of safety in robotics. Asimov’s Three Laws, formulated as basic guidance for limiting the behavior of autonomous robots in human environments, are enshrined, for example, in the Principles of Robotics of the UK’s Engineering and Physical Sciences Research Council (EPSRC)/Art and Humanities Research Council (AHRC), published in 2011.Footnote ²⁶ These principles lay down five ethical doctrines for developers, designers and end users of robots, together with seven high-level statements for real-world applications.

Shortly before the vast technological advancements in the second half of the twentieth century began, the first rudimentary telerobotic system was developed in 1945 by Raymond Goertz at the Argonne National Laboratory.Footnote ²⁷ It was designed to control, from a shelter, a robot that could safely handle radioactive material. From the 1950s on, the first complex electronics were developed, further optimized and miniaturized, and modern concepts of mechanics were created. The first mechatronic machines, such as fully automated electric washing machinesFootnote ²⁸ or the first industrial robots,Footnote ²⁹ were invented, and the concept of AI was further developed. Through the mathematical work of Jacques S Denavit (1930‒2013), Richard Hartenberg (1907‒1997) and Rudolf August Beyer (1892‒1960), one of the most important methods of calculating the direct kinematics of robots was developed around the year 1955.Footnote ³⁰ This matrix calculus, known today as the Denavit‒Hartenberg Convention, calculates how the joints of a robot have to be adjusted in order for it to be able to approach a specific point in space. In the same year, John McCarthy (1927‒2011), an American cognitive computer scientist and inventor of the famous programming language Lisp, introduced the term “artificial intelligence.”Footnote ³¹ He also organized the famous Dartmouth Conference in the summer of 1956, which is considered the birth of AI as a research field.

Marvin Lee Minsky (1927‒2016) was an American mathematician and cognitive scientist as well as a colleague of McCarthy in the same AI working group at Massachusetts Institute of Technology (MIT).Footnote ³² He is known for the invention of head-mounted graphical displays and for his work in artificial neural networks. Together with Seymour Papert, he wrote the book Perceptrons, which is still required reading for the analysis of artificial neural networks. He introduced several famous AI models and developed SNARC, the first neural network simulator. The late 1950s can also be seen as an important opening stage in the modern theory of optimization and optimal control. The field of optimal control deals with the process of calculating appropriate control laws for a given system in order to meet certain desired optimality criteria. In this context, at the end of the 1950s the mathematicians Lev Semyonovich Pontryagin (1908‒1988) and Richard E Bellman (1920‒1984) published a series of new fundamental optimization methods, such as Pontryagin’s maximum principle,Footnote ³³ Bang-Bang control,Footnote ³⁴ the Hamilton‒Jacobi‒Bellman equation or the Bellman equation for dynamic programming,Footnote ³⁵ which changed the entire field of mathematical optimization and control. These advances continue to this day to have a major influence on various practical areas from engineering to economics.

In 1957 the first autonomous underwater vehicle, the Self-Propelled Underwater Research Vehicle (SPURV), was invented at the Applied Physics Laboratory at the University of Washington by Stan Murphy, Bob Van Wagennen, Wayne Nodland, and Terry Ewart;Footnote ³⁶ this system was used to measure the physical properties of the sea.

A few years later, in 1960, electrical engineer and mathematician Rudolf Emil Kalman (1930‒2016) developed the Kalman filter in cooperation with Richard S Bucy and Ruslan L Stratonovich.Footnote ³⁷ This mathematical algorithm is capable of predicting system behavior based on a dynamic model and suppressing additive noise at the same time. In the context of this algorithm Kalman introduced two new system analysis concepts: system observability and controllability.Footnote ³⁸ The concept of observability analyzes how well the internal states of a system can be calculated by measuring its output. Controllability measures how an input signal changes the internal states of a system. These system analysis methods are crucial for the design of a Kalman filter, but also provide very important system information for the design of stable control loops in robots, process machines or driver assistance systems in cars. The Kalman filter itself is still one of the most important signal-processing tools in modern robotics, but is also used in various other disciplines such as AI, navigation, communications and macroeconomics.

The basic theories of robotics continued to expand, with developments in hardware and control, such as electric motor and sensor systems. In 1961 Joseph Engelberger (1925‒2015), an American entrepreneur, physicist and engineer known as the father of industrial robots, developed, together with his company, the first industrial robot, Unimate.Footnote ³⁹ A few years later, in 1964, a machine-learning algorithm called support-vector machine (SVM) was invented by mathematicians Vladimir Naumovich Vapnik and Alexey Yakovlevich Chervonenkis (1938‒2014).Footnote ⁴⁰ The original SVM algorithm is a linear classifier for pattern recognition. In 1992 the original method was extended to a nonlinear classifier by applying the so-called kernel trick;Footnote ⁴¹ the algorithm’s final stage of development, still used today, was reached in 1995.Footnote ⁴²

Back in 1966, the computer program ELIZA was developed and introduced at MIT’s Artificial Intelligence Laboratory under the direction of Joseph Weizenbaum.Footnote ⁴³ ELIZA is a program for natural language processing that uses pattern matching and substitution methodologies to demonstrate communication between humans and machines by simulating a coherent conversation. Three years later American engineer Victor Scheinman (1942‒2016) designed the first successful electrically operated, computer-controlled manipulator.Footnote ⁴⁴ This robotic arm had six degrees of freedom, and was light, multi-programmable and versatile in its motion capabilities. Later on, the robot was amended for industrial uses such as spot welding for the automotive industries. In the field of machine learning, David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams introduced the modern version of the backpropagation algorithm in 1968.Footnote ⁴⁵ This method is used in artificial neural networks to train networks and is a standard tool in this field today.

1.1.2 The Modern Era of Robotics and AI

The modern era of robotics and AI is characterized by ever greater miniaturization of electronics and mechatronics and an enormous increase in computing power, developments that have led to more practical robotic systems. The first humanoid robot to mimic human motion, the WaBot 1, was introduced by a Japanese research team from Waseda University in 1973.Footnote ⁴⁶ WaBot 1 had very basic capabilities to walk, grab objects and transport them from one place to another. In 1978 Unimation released a new and more versatile version of the Unimate, called the Programmable Universal Machine for Assembly (PUMA).Footnote ⁴⁷ PUMA has become very popular in industry and academia and over time has become an archetype for anthropomorphic robots. It remains widely used today as a reference example and benchmark system in academic robotics books and publications worldwide.Footnote ⁴⁸

In the 1980s the modern field of reinforcement learning was founded by combining different approaches from various disciplines. The starting point was the idea of trial-and-error learning, which was derived from psychological studies on animal learning dating from the early eighteenth century.Footnote ⁴⁹ Reinforcement is the expression of a certain behavior pattern in connection with an interaction of an animal with its environment. The animal receives different stimuli in temporal correlation with its behavior, causing certain behavior patterns to persist even after the stimuli have subsided. From the technical point of view, this process can be described as an optimization problem with some stochastic features in terms of incomplete knowledge of the whole system. A further development of the optimal control framework already mentioned can be used to describe and solve such a system. One of the first to implement this idea was Witten, with his adaptive optimal control approach.Footnote ⁵⁰

Another important aspect of the rise of the modern theory of reinforcement learning is temporal-difference (TD) learning, the origins of which lie in animal learning psychology. It can be seen as either a subclass or an extension of the general reinforcement learning idea. In contrast to the standard reinforcement approach, in TD learning the learner’s behavior or strategy is adjusted not only after receiving a reward, but after each action before receiving it, based on an estimate of an expected reward with the help of a state value function. The algorithm is thus controlled by the difference between successive estimates. In 1959 Arthur Samuel implemented this approach for the first time in his checkers-playing program.Footnote ⁵¹

In 1983, a further development of this reinforcement learning algorithm, the so-called actor‒critic architecture, was applied to the control problem of pole balancing.Footnote ⁵² The year 1989 can be described as the year of full integration of optimal control methods with online learning. The time difference and optimal control methods were fully merged in this year with Chris Watkin’s development of the Q-Learning algorithm.Footnote ⁵³

In addition to reinforcement learning, the 1980s saw seminal work in robot manipulator control. Early in the decade John J Craig and Marc Raibert published a new hybrid control technique for manipulators. Their system made it possible to simultaneously satisfy the position and force constraints of trajectories, enabling compliant motions of robot manipulators.Footnote ⁵⁴ In the mid-1980s, Neville Hogan developed impedance control for physical interaction,Footnote ⁵⁵ which was an important step toward enabling the safe human‒robot interactions of today. In 1986,Footnote ⁵⁶ Oussama Khatib published his work on real-time obstacle avoidance for manipulators and mobile robots, which was the beginning of time-varying artificial potential fields for collision avoidance. This concept made real-time robot operations in dynamic and complex environments possible. A year later Khatib developed a new operational space framework for unified motion and force control.Footnote ⁵⁷ This new mathematical formulation of robotic manipulators made the modeling and control of these nonlinear dynamic systems much easier to understand.

With the introduction of its P1 system, Honda entered humanoid research and development in the early 1990s.Footnote ⁵⁸ P1 was 191.5 cm tall, weighed 175 kg and was able to walk at a speed of up to 2 km/h with his battery lasting for around 15 minutes. Further developments in the field of telerobotics led to the success of the Rotex mission in 1993, in which researchers around Gerd Hirzinger developed the first Earth-controlled space robot.Footnote ⁵⁹

In 1995 Ernst Dickmanns and his team pioneered autonomous driving, conducting a journey from Munich in Germany to Odense in Denmark and back (approximately 1,758 km) as part of the PROMETHEUS project. They used a Mercedes-Benz S-class vehicle converted for autonomous driving. About 95% of this distance could be covered completely autonomously, a milestone in autonomous driving.Footnote ⁶⁰

In the following years, IBM developed the Deep Blue system.Footnote ⁶¹ Deep Blue was an intelligent computer program designed for playing chess. It is known for being the first computer system that, with the physical support of a human to execute the actual moves, won a game of chess against reigning world champion Garry Kasparov under regular time rules.

Following on from the pioneering work of RC Smith and P Cheeseman in 1986Footnote ⁶² and the research group of Hugh F Durrant-Whyte in the early 1990s,Footnote ⁶³ the next steps toward autonomous propulsion systems were taken at the beginning of the twenty-first century with the foundations of modern simultaneous localization and mapping (SLAM) algorithms for vehicle or robot navigation. As part of this development, in 1998 Wolfram Burgard and colleagues published a new software architecture for an autonomous tour-guide robot used in the Deutsche Museum in Bonn.Footnote ⁶⁴ These innovative algorithms for autonomous navigation provided the capability for the robot to guide museum visitors quickly and safely through a large crowd. In 2005 Thrun and his Stanford University racing team won the DARPA Grand Challenge with their Stanley autonomous driving system, showing the capabilities of SLAM. Their self-driving car completed a 212-kilometer off-road circuit in 6 hours and 54 minutes.Footnote ⁶⁵ Nowadays, SLAM algorithms are implemented in some consumer robot vacuum cleaners like the Roomba system from iRobot.Footnote ⁶⁶

In the year 2000 a significant technological step forward in humanoid robots came with Honda’s introduction of its latest humanoid system, Asimo.Footnote ⁶⁷ Asimo had basic abilities to walk and socially interact with people. In the same year Intuitive Surgical released the Da Vinci robot-assisted surgical system for usage in teleoperative minimally invasive surgery, based on development work at Stanford Research Institute.Footnote ⁶⁸ To this day, this system and its successors are used in hospitals around the world in a range of surgical procedures ranging from hysterectomies in gynecology to general surgery.Footnote ⁶⁹ In 2002, the German Aerospace Center (DLR) introduced the lightweight robot III (LWR III), which marked a technological leap forward in the field of lightweight robotics.Footnote ⁷⁰ Its new design paradigms enabled direct measurements and active damping of joint vibrations, together with almost immediate detection of collisions with the environment.Footnote ⁷¹ The robot was also able to carry and manipulate loads up to its own weight.

Around the same time, the Mars Exploration Rover (MER) mission was launched, showing new possibilities in telerobotics and space robotics.Footnote ⁷² The year 2010 was the year that drones became commercially available with the launch by French company Parrot of its Parrot AR Drone, the first ready-to-fly drone available on the open market.Footnote ⁷³

After years of basic research in the field of safe physical human‒robot interaction, ranging from standardized dummy crash tests to injury analysis of human‒robot impacts by soft-tissue experiments, in 2011 Sami Haddadin published a comprehensive study of how robots could for the first time meet Asimov’s First Law in everyday situations.Footnote ⁷⁴ The study developed the injury analysis, design paradigms and collision-handling algorithms to ensure that robots could interact safely with humans. It laid the foundations for the essential international safety standardization and regulation of physical human‒robot interaction, paving the way for robotics in everyday life.

In the same year, a new AI system was introduced by IBM.Footnote ⁷⁵ Watson was the first computer system that could answer questions on the American quiz show Jeopardy! In 2013, IBM made the Watson API available for software application providers. The system is frequently used today as an assistive system in medical data analysis, for example in cancer research.Footnote ⁷⁶

1.1.3 A Big Step Forward

The year 2012 saw the revival of deep neural networks (DNNs), also referred to as deep learning, which are further developments from the standard neural network approaches.Footnote ⁷⁷ The idea of DNN was first introduced in 1965 by Oleksiy Ivakhnenko and Valentin Lapa.Footnote ⁷⁸ However, it took decades and substantial progress in computing technology before this idea could be used in well-functioning applications. In 2012 this stage was reached by Geoffrey Hinton and his team when their algorithm won the image or object recognition competition ImageNet.Footnote ⁷⁹ Other researchers such as Yoshua Bengio and Yann LeCun also contributed significant papers to progress in deep learning.Footnote ⁸⁰

Boston Dynamics, founded by ex-MIT professor Marc Raibert, first made the news in 2012 with its four-legged robot BigDog.Footnote ⁸¹ BigDog was a dynamically stable four-legged military robot that could withstand strong physical hits and remain stable. In 2013 Boston Dynamics unveiled their two-legged humanoid robot, Atlas.Footnote ⁸² Its humanoid shape was designed to allow it to work with tools and interact with the environment. The system has since been further developed and equipped with increasingly complex acrobatic skills.

In the same year a team from Johns Hopkins University and DLR conducted a telepresence experiment in which a Da Vinci master console in Baltimore, USA controlled a DLR lightweight robot in Oberpfaffenhofen, Germany, over 4,000 miles away.Footnote ⁸³ This marked a milestone in telerobotics by combining telepresence via standard internet with the slave robot system’s local AI capabilities.

In 2014, a major step forward in certification and standardization of personal care robot safety requirements was taken with the publication of the ISO 13482 standard, a catalogue of requirements, protective measures and guidelines for the safe design and use of personal care robots, including mobile servant robots, physical assistant robots and person-carrier robots, generally earthbound robots for nonmedical use.Footnote ⁸⁴

The next step in software-based AI was demonstrated a year later, in 2015, by DeepMind’s AlphaGo system.Footnote ⁸⁵ AlphaGo’s learning algorithms included a self-improvement capability through which it could master highly complex board games, such as Go, chess and shogi, by playing the games with itself.

By 2016, virtual assistants had finally arrived in everyday life.Footnote ⁸⁶ In 2011, Apple started to deliver smartphones with a beta version of their virtual assistant Siri. Further systems have been launched, including Cortana from Mirosoft, Alexa from Amazon and finally Google Assistant from Google. Virtual assistants in general are designed to perform tasks given by a user, usually by voice command, and reflect current state-of-the-art speech-based human‒machine communication technologies.

The next level of underwater robotics and telerobotics was introduced by Khatib and his research team at Stanford University in 2016. The teleoperated underwater humanoid robot system OceanOne demonstrated its bimanual manipulation capabilities in an underwater research mission to study the wreck of La Lune, King Louis XIV’s flagship, off the Mediterranean coast of France in 2016.Footnote ⁸⁷ In 2017 Franka Emika’s human-centered industrial robot system Panda was introduced.Footnote ⁸⁸ This next-generation industrial robot is the first sensitive, networked, cost-effective and adaptive tactile robot. It is operated via simple apps on personal devices like tablets or smartphones. This first mass-produced robot is self-assembled, showing the potential for versatile manufacturing and marking the first step into the future of self-replicating machines.Footnote ⁸⁹

One year later Skydio launched its Skydio R1 drone, a further step in the direction of intelligent flying robots. This system has stable flying capability in windy environments, can follow its user reliably and while following avoids obstacles in its way.Footnote ⁹⁰

A new concept in neural networks was also published in 2018.Footnote ⁹¹ First-order principles networks (FOPnet) use basic physical assumptions to build a physically informed neural network. With the application of this new concept, it has already been shown that both the body structure and dynamics of a humanoid can be learned on the basis of basic kinematic laws as well as the balance of force and moments acting on this kind of multi-body system. This can be regarded as the first step toward machines able to learn self-awareness.

The lighthouse initiative Geriatronics from the School of Robotics and Machine Intelligence at the Technical University of Munich was launched in 2018 with the aim of developing robot assistants for independent living for the elderly.Footnote ⁹² This initiative is sustainably supported by the Bavarian State Ministry of Economic Affairs, Energy and Technology and LongLeif GaPa Gemeinnützige GmbH.

In early 2019, Haddadin, Johannsmeier, and Ledezma published a paper in which they discussed a concept they called Tactile Internet as the next-generation Internet of Things.Footnote ⁹³ They propose that 5G communication infrastructures combined with rich tactile feedback and advanced robotics provide the potential for a meaningful and immersive connection to human operators via advanced “smart wearables” and Mixed Reality devices, effectively making real avatars a reality.

1.2.1 Trustworthy Artificial Intelligence

Artificial intelligence (AI) is a superordinate term for the discipline that creates intelligent algorithms and systems, which can be software-based or actual physical systems, or combinations of the two. An AI system uses sensors to perceive its surroundings, may use actors to interact with it, and collects and analyzes large amounts of partly unstructured data, processing and interpreting it to uncover latent knowledge and skills. Using this knowledge, it supports decision-making to reach the desired objectives of humans, for example, by acting as a software-based advisor or by adjusting its embodiment with actuators. AI systems are capable of learning from their previous actions and the corresponding responses, making them self-optimizing. AI has wide fields of application and great potential to help with the challenges of, for example, improving medical diagnostics and therapy, finding ethically acceptable ways to cope with demographic change and reducing the effects of environmental problems such as climate change or pollution. Other useful applications are promoting sustainability in everyday life, for example by optimizing transport and logistics, promoting sustainable agriculture, or reducing strenuous physical labor in the workplace.

In order for AI to find its way into people’s everyday lives as a useful helper, it is important that this technology is trustworthy. AI is often used where humans reach their limits, such as when analyzing and interpreting large amounts of unstructured data. Trust in this context means that the human can rely on the correctness and unbiasedness of the resulting information, and is therefore able to make informed decisions. Among the many examples of the importance of trust in the evaluation of data by AI are the security of private data, human rights, respect for the rule of law and the preservation of democratic freedoms. If AI does not consider these aspects, its output may lead, among other things, to diversity and inclusion issues. In a nutshell, trustworthy AI has to be human centered and have human values and well-being at its core. It has to comply with human rights, the rule of law and democratic freedoms. From a technological point of view, its robustness and reliability need to be guaranteed, which has significant effects on transparency and explainability.

1.2.2 Safety in Physical Human‒Robot Interaction

Safety in robotics and AI has been and still is a widely researched topic. For a very long time, it was assumed that safety between humans and robots could only be ensured by installing protective safety systems on or near the robot, such as a safety fence for workspace segregation. However, such protective enclosures are very obstructive in general physical and intuitive human‒robot interactions and real collaboration. The practical goal is to enable the safe coexistence of humans and robots in the same workspace, where interactions may occur intentionally and safely. A variety of potential risks can arise that depend on the dynamically changing system state and its environment.

The first approach to safe robotics was to quantify mechanical hazards inducing potential injuries during human‒robot interactions. Dummy crash-test and soft-tissue collision experiments were performed. Impact scenarios can be simulated and analyzed using information from impact experiments already carried out in areas such as injury biomechanics or forensics, combined with suitable mathematical models. Characteristic force profiles can then be defined for specific parts of the human body representing targeted physical collisions between a human and a robot. These force profiles in turn serve as the basis for defining safety limits for robot velocities so that safe human‒robot interaction is guaranteed.Footnote ⁹⁴

Based on injury analyses from various impact scenarios with robots, international safety standards for human‒robot interaction were devised, such as the ISO 13482 standard. This is the first non-industrial standard to specify safety requirements for personal care robots such as mobile servant or physical assistant robots. It defines the guidelines for safe design and general safety measures for the operation of earthbound nonmedical robots in non-industrial applications. However, there are still many research questions to be solved before complete standardization of robot safety is achieved.Footnote ⁹⁵

1.2.3 Robot Mechatronics As AI Embodiment

The physical parts of a robotic system are an example of an AI embodiment. The physical body, which is the mechatronic design of such systems, must be specifically designed for safe physical human‒robot interaction, which requires human-centered development for optimal security and performance in human-centered environments. Research in this field has led to new and innovative design paradigms based on active and/or passive compliance in combination with lightweight design principles.

Lightweight concepts involve the whole system and the moving parts are designed to be as light as possible to reduce possible collision metrics. Generally, there are two major approaches, the mechatronic approach and the tendon-based approach.Footnote ⁹⁶ In both, the robot structure consists of light and strong materials such as light metal alloys or composites. In order to optimize power consumption and to meet safety standards, both motors and moving parts are designed to have low inertia.

The mechatronic approach is based on a highly modular structure. To achieve this, the majority of the robot’s electronics are integrated into its joints. This modularity enables the development of highly complex, self-contained robotic systems that can be controlled efficiently. An important feature of the motors used in this approach is that they can generate high torque, enabling the system to act and react fast and dynamically. One characteristic that stands out in the mechatronic approach is the use of a redundant sensor. Normally only motor-position sensors are used, but with this concept, additional sensors for measuring torque, force or current are integrated into the system. These additional sensors can be used to increase the measuring accuracy and/or to provide certain safety features.

In contrast to the mechatronic approach, tendon-based robots use remotely located motors to reduce weight. The motors are connected to the parts to be moved via a cable. One disadvantage of this approach is that the motors required to move such a system are quite large: the weight of the moving parts is reduced but the total weight of the system remains relatively high. Further information on robot design concepts and other important classes of robot structures can be found in the literature.Footnote ⁹⁷

1.2.4 Multimodal Perception and Cognition

Perception technologies are the artificial sense organs of machines and are indispensable for interacting with the world. The human example shows that to cope well with dynamically changing environments in daily life it is also important to use more than one sense at a time. Multimodal perception combines, for example, tactile with visual perception. Three common types of perception in close physical human‒robot interaction and general robotics are explained in the following sections: force/torque sensing, tactile perception and visual perception.Footnote ⁹⁸

Taken together, standard proprioceptive position sensing and force/torque measurement provide a sense of touch to sensitively grasp and hold very fragile objects. The most commonly used sensing techniques are strain gauges within a measuring bridge or implicit deflection-based measurement. This perceptual technique enables force-regulated manipulations and sensitive haptic interactions with humans.

The tactile perception approach was inspired by the properties of human skin. Here, the entire robot is enveloped in a tactile skin consisting of many small-networked sensor elements. In contrast to the previous type of sensing, contacts occurring in close proximity to each other can be specifically measured by the sensor skin during the completion of a task. The skin can give the robots significant sensory capabilities, but also increases complexity and computational cost. Distributed data processing could help here. If each sensor element was equipped with its own microcontroller, which prepared the sensor data in such a way that the central computer only has to process simple high-level signals, the high computing effort for the main controller could be reduced. Such systems still require a lot of research work in order to be fully mature and robust.

Visual perception is a quite common non-contact sensor technology, often used for the autonomous execution of robotic tasks without interaction with humans or for preparatory activities, such as identifying humans or objects in the environment, in connection with a human‒robot interaction. One technique in this field, marker-based visual sensing, is used as a high-resolution tracking system, for example to navigate drones safely through a room. These systems usually consist of infrared cameras, which measure the positions of the highly reflective markers in a room even during very fast movements. The use of such a system is not always practicable or universally applicable, since markers must always be positioned and calibrated beforehand. In addition, this principle is often sensitive to interference, for example from sunlight, or has problems with sensor shading. Another type of visual perception is the use of inexpensive 3D RGB depth cameras in combination with AI algorithms for the visual tracking of objects or people or for general navigation in space during everyday operations. However, from a robustness and performance point of view, visual perception with 3D RGB depth cameras still needs several years of research before it can be used reliably in all everyday conditions.

1.2.5 Navigation and Cognition

Research into autonomous navigation has been a high priority for several decades.Footnote ⁹⁹ Particularly in the field of mobility and transport or logistics, it promises to finally give robotic systems such as autonomous vehicles the ability to relieve people of the mostly strenuous and tiring work at the wheel of vehicles. In order to achieve autonomous navigation capability in space, an intelligent robotic system needs robust algorithms for self-localization, route planning and mapping as well as map interpretation.Self-localization is the ability of a robot to determine its own position in the reference system. There are several techniques to do this as Global Positioning System (GPS)-based techniques are quite accurate for outdoor self-localization but not suitable for indoor applications. For indoor navigation, visual perception-based techniques combined with inertial sensors are more promising. Once the robot has its position, it must plan the route to the target position. The first step is to calculate the distance between the robot’s position and its destination. The next step is map generation, which in general terms means the analysis of the environment between the robot’s own position and the destination. The subsequent interpretation of this generated map is crucial in order to execute the overall task of movement. Here, the algorithm performs a semantic recognition of the environment, for example recognizing obstacles on the map as non-movable areas between the robot’s own position and the target.

A more specific application area is indoor navigation and cartography without a comprehensive decentralized tracking system. A quite simple and robust method of solving the navigation problem is the use of line markings on the ground that are recognized and tracked by the robotic system’s sensors and controls. This is a rather static method, since the predefined paths ‒ the environment map ‒ are fixed on an abstract level. Dynamic changes, which can occur frequently when interacting with humans, are difficult to update online with this approach.

The SLAM algorithmFootnote ¹⁰⁰ is more suitable for use in environments with fast dynamically changing conditions. This algorithm can simultaneously determine the robot’s own position and create an online map of the previously unknown environment using sensing systems such as 3D RGB depth cameras or LIDAR (laser detection and ranging) systems.Footnote ¹⁰¹ The robot performs relative measurements of its own motion and of features in its environment to obtain the necessary information for navigation. Both measurements are often noisy due to disturbances, so the SLAM algorithm now tries to reconstruct a map of the environment from these noisy measurements and to calculate the distance the robot has covered during the measurement.Footnote ¹⁰² The biggest issue with using SLAM is that the complexity of constantly changing dynamic environments leads to a high computing effort, thus the real-time capability of the overall system cannot always be guaranteed.

1.2.6 Modern Control Approaches in Robotics

The goal of modern control in robotics is to develop approaches that enable the robot to act optimally on its own but also to handle potentially physical interactions with humans gently and in a human-centered way. A very common approach to control physical interaction is impedance control or compliance control.Footnote ¹⁰³ This approach is based on controlling the connection between force and position on interaction ports, such that the robot has the ability to interact compliantly with the environment. For this purpose, the contact behavior between the robotic system and the object it is to interact with is modeled by a mass-spring-damper system, whereby the controller can adjust the stiffness and damping of this system. Classical impedance control quickly reaches its limits in dynamic, rapidly changing processes, which include human‒robot interactions. The impedance control parameters must be known in advance and are usually set by experiments and calibration. In order to avoid this limitation, adaptive impedance control (AIC) was developed, whereby these parameters can also be changed online.Footnote ¹⁰⁴ New approaches combine AIC with approaches from machine learning to teach the robot certain impedance behaviors as well as how to deal with disturbances in the system. One example is the combined use of AIC and artificial neural networks to map complex disturbances that cannot be modeled analytically.

1.2.7 Machine-Learning Algorithms

When one thinks of machine learning, certain keywords like deep learning, neural networks or pattern recognition immediately come to mind. This section, which provides a brief overview of the topic of machine learning, aims to shed light on these and other terms.

Machine learning originated in computer science with the aim of developing algorithms to efficiently process complex signals and data.Footnote ¹⁰⁵ The main problem in signal processing remains the handling of uncertainties caused, for example, by measurement noise or low data density. Another problem is the analysis and interpretation of extremely high amounts of data, which mostly represent very complex and highly dynamic systems. One of the central foundations on which machine learning to deal with these kinds of problems is based is stochastic theory. With stochastic theory as the baseline, general machine learning can be split into (semi-)supervised learning, unsupervised learning and reinforcement learning. Before applying machine-learning algorithms, the raw data must often be pre-processed, for example by feature extraction algorithms such as filter algorithms, dimensionality reduction algorithms or other approaches to build up a “feature space.”

The first type of learning concept to be discussed here is supervised or semi-supervised learning, which attempts to train a model with labeled training data (input‒output is known). Semi-supervised learning is the harder variant of this training phase. It has only incomplete training data for the training phase, which means that the sample inputs lack some desired outputs. After sufficiently long training, the quality and generalization abilities of the model can be tested using a data set that contains new and slightly different data. This type of machine learning is mostly used for classification tasks like pattern recognition. Unsupervised learning uses only input data for the training without any knowledge of the desired outputs. One goal here is to discover new information such as similar structures in the data set, known as clustering. The last type are reinforcement learning algorithms, which are based on the principle of goal-directed trial-and-error learning, where an improvement is rewarded or a deterioration is penalized.Footnote ¹⁰⁶ The difference between this and other approaches is that reinforcement learning uses direct interaction with the environment for the learning process. These algorithms are not based on experience-based supervision or an overall model of the environment. Typical applications are self-optimizing systems such as in game theory or control theory. Next we look at some of the models which use these training concepts.

Commonly used machine-learning models are artificial neural networks,Footnote ¹⁰⁷ support-vector machines,Footnote ¹⁰⁸ Bayesian networks,Footnote ¹⁰⁹ and genetic algorithms.Footnote ¹¹⁰ The most popular model approach in the field of machine learning is neural networks, often used in supervised learning. The idea behind this approach is to simulate aspect of the behavior of neurons in the human brain using the so-called perceptron algorithm.Footnote ¹¹¹ A perceptron or neural network consists of several artificial digital neurons that are networked along different layers: the input layer, hidden layer and output layer. This approach is also known as a black-box algorithm because interpretable information about the dynamics between input and output layer is not available. An artificial digital neuron is represented by a nonlinear function, the activation function and a weight function (transfer function) with variable weight parameters. The special feature of the nonlinear function is that it has a threshold. If this threshold value is exceeded by the input value of the function, the function outputs a one, and otherwise a zero. This behavior can be used to train a specific input-output mapping between the input and output layer of this type of network. If a specific network structure is then designed for a desired application, the network can be trained to a desired behavior, using the backpropagation algorithm and training data, by setting the parameters of the network accordingly. In this context, a deep neural network is a more complex variant of a normal neural network, where, for example, a higher number of hidden layers are used.Footnote ¹¹² The hidden layers can generally be seen as a not directly reachable layer with encoded information after the training phase. The dynamics and properties of these layers are not yet fully understood.

Another machine-learning model is the support-vector machine, which is often used as a classifier or regressor for pattern recognition tasks. This mathematical algorithm tries to calculate so-called hyperplanes (decision boundaries) to separate and therefore classify two or more objects in the feature space, using labeled training data. Important training data is the data that is close to the transition from one object to the neighboring object and only this data is needed to span the hyperplane mathematically. These data points are called support vectors and give this model approach its name.

Bayesian networks are used for decision-making. They are basically directed acyclic graphs, but each node represents a conditional probability distribution of a random variable and each edge, the associated conditional relationships or dependencies between the random variables. If one now considers a random variable that is not conditionally independent, that is, it has relations to other random variables represented by the connected edges, one can easily recognize the functionality of a Bayesian net. This node gets input values for its probability function via the edges directed to it, then the probability of the random variable belonging to the probability function is obtained as an output. If you calculate this for the whole network, you get a compact representation of the common probability distribution of all variables involved. From this a conclusion or inference about complex problems such as unobserved variables can be obtained. Not every Bayesian network is fully specified because some conditional probability distributions may be unknown. These missing pieces can be obtained by learning the probability distribution parameters from data, for example by using maximum likelihood estimation (MLE). Sometimes the relations between the random variables are unknown. In this case, structure learning is applied to estimate the structure of the network and the parameters of the local probability distributions from data. Various optimization-based search approaches such as the Markov chain Monte Carlo algorithm can be used.

The last machine-learning model to be presented here is the genetic algorithm, which belongs to the evolutionary class of algorithm. This algorithm works with metaheuristics and is based on the idea of natural selection. In general, the algorithm starts with a population of possible solutions, where each solution has certain parameters that can be used to mutate or vary it. At the beginning, individuals are randomly selected from the starting population, from which the strongest individuals are then selected using an object function. Now the parameters of these individuals are changed according to a measure given by the number of remaining individuals in this generation. From this, the new generation is created, from which the fittest ones are selected again. This continues until a previously defined number of generations or a specific fitness level is reached.

1.2.8 Learning in Intelligent and Networked Machines

Now that we have discussed some approaches from the field of machine learning, we next examine how some of them are used in robotics. One field of application previously considered is in combination with adaptive impedance control. In addition to the control of robots, machine learning is also used to avoid complex manual programming of robotic task execution. One approach is apprenticeship learning, where the human acts as teacher for the robot system by demonstrating the task to it.Footnote ¹¹³ The robot then tries to imitate what is shown in order to learn the skills needed to complete the task. After a short training phase, the system should improve itself independently, completing the task optimally after some time. Today, reinforcement learning is often used for this autonomous self-improvement.Footnote ¹¹⁴ An explicit application of these learning algorithms is the robotic gripping and manipulation of objects. Here, automatic development of complete scene understanding using object-centric description is necessary to find generalizable solutions for more complex manipulation tasks.Footnote ¹¹⁵ The learned processes are not complete imitations, but only the interaction points and movements with the object are modeled, which makes generalization for applications to other systems possible. Another important technological advance making complex manipulation tasks in robotics autonomously solvable was the further development of image-processing algorithms in combination with powerful object localization in a dynamic environment.Footnote ¹¹⁶

However, to achieve the next step in robotic manipulation or in the field of learning machines generally, approaches are needed that are even more general and scalable. A promising approach is the concept of collective learning. This concept is based on the prediction of a dramatic increase in robots in society over the coming decadesFootnote ¹¹⁷ and on the idea of ever greater interconnectedness. Today, almost everyone walks around with a smartphone that can be interpreted as part of a huge networked cluster of small supercomputers. This trend will not stop at robotics either, producing networked robots operating via the internet or bringing up entire robot clusters with possibly highly complex hierarchical network structures. New communication architectures, planning and control methods will become necessary for the optimal use of these highly networked robot clusters. A new capability of such robot clusters would be, for example, to exchange learned information with each other while they perform complex manipulation or interaction tasks. In this way, the robots would learn from each other as in a collective, by exchanging already acquired knowledge about different but similar tasks. This transfer of knowledge, a crucial aspect of the collective learning concept, will help the networked robots to master new problems in everyday life more easily or to learn much faster.

1.3.1 Flying Robots

Ever cheaper and more powerful computer hardware in ever smaller forms, together with advances in sensors and real-time signal-processing algorithms, has brought enormous progress in the field of flying robots. Not only do these small unmanned aircraft vehicles (UAVs) have the ability to stay in the air longer than previous systems, but their autonomy capabilities have also increased drastically. What does autonomy mean in the field of flying robots? In general, autonomy in robotics means the ability of robots to work in unknown, unsafe and unpredictable environments without the intervention of a human operator. Many aspects of navigation already mentioned in the section on key technologies play a role here. These include estimating the robot’s position, mapping the environment, creating trajectories and deciding or interpreting the created maps. Especially in the field of flying robots, computational algorithms for aerodynamic modeling and wind estimation are important. Novel sensor systems are crucial to ensure that the flying robot can use these algorithms in real time. The focus here is on the fusion of exteroceptive sensors such as cameras and laser rangefinders with proprioceptive sensors such as an inertial measurement unit, to form a multimodal sensor system. Modern UAVs today have the capability to use six stereo cameras simultaneously in real time combined with various other sensors to perform occupancy grid mapping, motion planning, visual odometry, state estimation and person tracking using deep learning algorithms. These high-tech systems come with actuators, sensors and computing systems that are integrated in a lightweight structure to a weight of about one kilogram and manage a flight time of about 16 minutes. The purchase price of these systems is around €2,500. Less intelligent flying robots, those with limited or non-existent obstacle avoidance, cost about €200‒1,000, weigh several hundred grams and have an average flight time of 10–30 minutes.

Looking at the missing pieces of these systems from a scientific point of view, generalizable approaches to aerodynamic modeling are still lacking. Developing generally valid models would reduce the development time and the costs of these systems. Another problem is to find an elegant and at best purely model-based approach to distinguish aerodynamic forces from collision and interaction forces. A secure physical human‒flying-robot interaction interface still requires a lot of research before it could enter the market in a product. Flight time would also have to be extended to reach a level suitable for everyday use. This could be achieved, for example, by further reducing the total weight with new materials or structural approaches. This development would also increase the safety of human‒robot interaction, since less energy would be transferred to the human body in the event of a collision. It is clear from all these factors that there is still a long way to go before small and affordable fully autonomous flying robots become ubiquitous.

1.3.2 Mobile Ground Robots

In the history of mobile robotics, the Shakey system can be seen as the first mobile robotic system to be used in practice. This system laid the foundations for technologies such as hierarchical control architecture more than 40 years ago. Since then much research has been done in the field of mobile robot platforms and many different approaches for these systems have been developed for uses ranging from industrial applications to applications in disaster zones or in general environments dangerous to humans. In order for mobile robots to move from the laboratory environment to applications for everyday life, research and development must focus on the safe human‒robot interaction capabilities of these systems. One robot developed specifically for safe human‒robot interaction is called Rollin’Justin. This system is very powerful but its development did not focus on cost-effective production and it therefore cannot be easily commercialized in the near future. One key element to enabling safe human‒robot interaction is the use of impedance control in mobile platforms. Until now, this approach has been rare and can only be found in research work, if at all. If the research focus were to be increasingly directed toward safe human‒robot interaction with the goal of bringing mobile robot technology to an affordable product, these systems could become much more common in everyday life and contribute to shaping our society. A comparison of the available mobile platforms is shown in Figure 1.1.

Figure 1.1 Overview of available mobile robotic systems

1.3.3 Tactile Robots

For more than 50 years, position-controlled rigid robots have been supporting assembly and welding in industry. Since these robots were developed to perform heavy work requiring high force, their systems are inappropriate for safe close interaction with humans and they are therefore usually separated from humans by a safety fence. In recent decades, the paradigm for the use of robots has changed. Sensitive manipulation and close physical human‒robot interaction have become the order of the day. To achieve this, highly integrated lightweight designs with low inertia and high active compliance have been developed and implemented. The result is systems such as the Barrett WAM armFootnote ¹¹⁸ and the DLR lightweight robot series,Footnote ¹¹⁹ whose arm technology later led to the LWR iiwa robot from the company KUKA. One of the most modern, human-centered lightweight robot systems developed to date is Franka Emika’s Panda system.Footnote ¹²⁰ A high-precision force and impedance control system allows the system to perform sensitive and accurate manipulation and enables a high degree of compliance, which, in conjunction with safety aspects already considered in the design phase of this robot, guarantee safe human‒robot collaboration. One of the most important pragmatic aspects of human‒robot collaboration besides safety is the operating, programming and interaction interface between human and robot. Many collaborative robots use a tablet computer and complex software as operating, programming and interaction interface. The Panda system offers an elegantly designed interface in which the human can interact with the robot in a natural way via haptic interactions such as tapping on the robot gripper to stop the robot or to give a process confirmation. In addition, in the teaching mode, it is possible to teach the compliant robot various work processes by taking it by the hand and guiding it extremely smoothly through the process. Once the process has been shown, it can be played repeatedly by simply pressing a button. This kind of programming is extended by apps representing two levels of interaction with the robot: the expert-level robot apps programmer and the user who does not need any special robotics knowledge. The expert provides the basic robot capabilities, which are assembled and operated by the user for complex processes and solutions. These basic robot apps will be shared over a cloud-based robotic app store and made available to a broad range of users. With the growth of this robotics skills database, many new applications will emerge, bringing robotics more and more into our daily lives.

1.4.1 From Cleaning Robots to Service Humanoids

Drones in the park, vacuum-cleaning robots at home or lawn-mowing robots in the backyard, all these robotic systems are nowadays nothing special to look at. However, finding extraordinary intelligent service robots, which can act in a similar social manner to humans, for example while supporting elderly people in their everyday life, still presents a gap in the technology. Furthermore, technologies available today are not able to adapt to short-term changes, are not user friendly in terms of “programmability” and do not learn from experience. In addition, unlike the case of industrial robots, security aspects have not been considered in these systems.

Early approaches in this direction can already be seen, for example in the user interface developed by Franka Emika for their robotic arm system. Nevertheless, what is still missing in these systems is the possibility of improving learned abilities autonomously. Intelligent service robots have to be able to adapt to new conditions. They have to meet the “lifelong learning” paradigm in order to be also accepted by older people, who may be more skeptical about new technologies. In addition, specific design and technology decisions regarding the acceptance and usability of these robots need to be made in the development phase of these systems if they are to be usable in the private sector.

A promising subfield of service robots are humanoids. As we have seen, service robots should be human centered from the beginning of their development, especially from the point of view of safety. For this reason, systems like the NASA Robonauten, DLR’s Justin or Boston Dynamics’ Atlas System are not considered here. Figure 1.2 gives a current overview of existing service-oriented humanoid systems or those under development.

Figure 1.2 Overview of existing and upcoming service-oriented humanoid systems

One of the first complex service humanoids available was the PR2 system from Willow Garage.Footnote ¹²¹ It consists of a mobile motion platform, two grab arms and numerous sensors to navigate in space by using position control. In addition to “pick-and-place” tasks, the user can teach this humanoid simple motion sequences. PR2 has relatively simple interaction channels such as motion control via a gamepad or tablet. Other service robots such as the Care-O-Bot 4 from Fraunhofer IPA,Footnote ¹²² the Tiago system from PAL RoboticsFootnote ¹²³ and the HSR robot from ToyotaFootnote ¹²⁴ have similar capabilities to the PR2, but some systems also have additional human interaction channels such as voice command input. The Care-O-Bot 4 can even gesticulate and interact with people via facial expressions or by touch from its built-in display. Furthermore, all of the humanoids mentioned here can be teleoperated to a certain extent. Two systems that stand out here are the Twendy-One robot from Waseda UniversityFootnote ¹²⁵ and the RIBA II robot from Riken.Footnote ¹²⁶ Both systems have special features making human‒robot interactions possible. Twendy-One has the ability to actively help a person to stand up from seated. It also has a tactile skin, which enables complex tactile manipulations. The RIBA II system is designed to be able to lift and relocate bedridden people, reducing the burden on medical staff.

In general, service robots in nursing have the potential to partially solve the lack of applicants and to enable older people to live independently as long as possible. The value of direct human‒robot interactions, apart from these approaches to physical interaction with the patient, has so far gone largely unnoticed. The systems presented here are not yet equipped with the necessary capabilities to perform smaller pick-up and delivery services or even sensitive manipulation tasks such as tying shoe laces. In general, there is great potential for helping humans in daily tasks and for human‒robot communication through haptic gestures.

The company Franka Emika is currently working on a humanoid service robot called GARMI, which will provide a sensitive human‒robot interaction. GARMI will be equipped with two multi-sensorial robotic arms, which will have soft-robotic features and the solutions required for direct human interaction and safe human‒robot interaction. In addition, the small robot will have a multisensory “head” and an agile platform, allowing it to move from a standing position in the desired direction. It should be able to perform both simple tasks and pick-up services, but also to be remotely controlled by relatives and professional helpers.

1.4.2 Production and Logistics

Low-cost and flexible national production of the next generation of industrial robots will eliminate the need to exploit developing countries. Robotics will finally live up to its original credo of freeing humanity from slavery. These new industrial robots will be highly networked and mobile with extensive sensory capabilities enabling them to autonomously perform a wide range of complex manipulation tasks and safely collaborate with humans. Innovative design concepts with extreme lightweight construction combined with new control approaches will lead to very low energy consumption by these systems. Mutual exchange of information and knowledge between robots in a collective set-up can lead to a rapid increase in learning speed. New complex tasks can thus be learned not over weeks, but over hours or even minutes.

In the coming decades, the development of autonomous vehicles will also create major changes. Autonomous vehicles are already widely used today, but mostly in closed warehouses or in confined areas that have been completely mapped in advance. These application areas are also predominantly shielded from dynamic sources of interference such as humans. One example is American online retailer Amazon’s warehouse system. In a completely systematic environment, hundreds of robots arrange themselves autonomously to select goods or goods shelves and drive them to the parcel assembly. Simply put, these robots are nothing more than powerful cleaning robots that can carry up to 300 kg. A lot of research will still be required to move this technology on from the retail environment to transporting people autonomously in our world. However, this next generation of autonomous ground and air vehicles will not only be able to navigate safely in the real world, but will also provide much more energy-efficient and environmentally friendly drives. The interconnectedness of these systems now makes it possible to automate complete logistics chains, and passengers can now be transported on demand, optimally in terms of both time and energy. Through the temporary networking and coordination of heterogeneous vehicle fleets, the fundamental principles of public transport are being redefined.

1.4.3 Robotic Disaster Relief

The application of robots in unsafe environments will be of great importance in our future world. It will allow us to use technology instead of risking human lives to save buried or trapped people or to perform highly risky maintenance tasks. The key technology for these applications is called telerobotics. A technology originally developed for space applications in the space agencies of the USA, Germany and Japan, telerobotics has been designed to enable a transparent (bilateral) remote control of robots in human-unfriendly environments. The first use of such a technology was in 1993, when the Rotex mission used Shared Autonomy/Supervised Autonomy on the first Earth-controlled space robot.Footnote ¹²⁷ Recently, a Da Vinci master console (in Baltimore, USA) controlled a DLR lightweight robot (in Oberpfaffenhofen, Germany), over 4,000 miles away. The robot was able to recognize its environment independently and perform selectable semi-autonomous functions on site with perceptual support. It could initiate the most likely actions desired by the user, such as gripping an object or inserting it, semi-automatically.Footnote ¹²⁸ The aim of this research was to investigate functional tasks that lie between pure teleoperation and full autonomy. In order to enable a more natural teleoperation that can also handle long delays, model-based teleoperation approaches use environmental models generated from knowledge gained a priori and updated step by step during manipulation.Footnote ¹²⁹ Thus, the teleoperation remains applicable even in the presence of delays of up to 4 seconds, as an approach with model-based teleoperation and haptic feedback has shown.Footnote ¹³⁰ Franka Emika goes one step further with the market launch of the first cloud and distributed telerobotic-capable commercial robot system Panda. The possibilities of this system were demonstrated in late 2018, when 37 Panda systems were connected in real time, with twelve operating in Düsseldorf (Germany) and twenty-five in Munich (Germany). As a result, thirty-six robots could be successfully teleoperated with one robot as an input device, with a maximum distance of approximately 600 km between them.

The future benefits of this technology will be available in various applications enabled by its high level of robustness, such as operating in space, defusing bombs, firefighting or rescue and containment in the event of a nuclear catastrophe.

1.4.4 Multimodal Communication for AI-Enabled Telemedicine

Telemedicine is a technology that has emerged from telerobotics in combination with real-time 3D visualization of the human body and multimodal communication technologies.

Multimodal communication represents the future of communications. Instead of communicating purely via voice, text or video, additional information channels are used to increase the transparency and interactivity between communicators. One channel, for example, would be the telepresence channel. This channel can be attached to a haptic input device with force feedback on one side and a robotic output device on the other. The robotic system moves according to the user’s input, but also returns information on haptic interaction to the input device. The user does not have direct access to the robot’s motion control system via the input device, but instead gives more abstract high-level commands, which are then translated into the desired motion. A framework for predictive and semi-autonomous interaction control in combination with a robot-side action recommendation system makes suggestions to the user for further action based on the local information. This telepresence channel is also available to be used for telemedicine. If an authorized physician uses this interface, a module will be unlocked which enables the use of diagnostic devices at the patient’s site, intelligent processing and visualization, and secure handling of sensitive medical data.

Crucial to the process of justifiable diagnosis at a distance is real-time 3D visualization of the human body, one element of which is the acquisition of the kinematics of the body. Today, motion-capturing systems equipped with infrared cameras for 3D detection of retroreflective markers positioned on anatomical landmarks are used for this purpose. By synchronizing the real-time data of human movements with musculoskeletal biomechanical modelsFootnote ¹³¹ and dynamic models of the internal organs, as well as 3D visualization models of the patient, it is now possible to provide the physician with the patient’s digital twin. The medical data obtained during examination by diagnostic devices, such as ultrasound, are then displayed and synchronized with the digital twin.

The next paragraph describes a typical telemedicine scenario (see Figure 1.3). Here, the humanoid GARMI is used as a teleoperated robot on the patient’s premises.

Figure 1.3 Telemedicine case scenario

Telemedicine emergency: shortly after his daily nap, Heinz suddenly feels unwell. He calls out to GARMI: “I don’t feel well. Please call a doctor.” GARMI comes immediately and establishes contact with the emergency doctor. At the doctor’s office, Heinz’s emergency call appears on the user avatar remote station display. The doctor can react immediately to the emergency as he is connected to GARMI. After a brief analytical dialog, the doctor lets GARMI perform an ultrasound and ECG examination. The ultrasound images and the ECG are transmitted to the doctor in real time. From the analysis of the transmitted data, which is supported by machine-learning algorithms, the doctor is able to quickly identify an emergency and immediately call the emergency service.

1.4.5 The Future of Medicine with Molecular Robots

The next step in medicine will be in the direction of personalized diagnostics and therapy locally at the site of the disease. The vision is to develop an intelligent medical machine that can perform measurements in the human body on the cellular level and, if necessary, treat directly. Such treatment could be performed in the future by molecular robots.

Molecular robots are small autonomous synthetic systems that can be used for numerous medical purposes. Different molecule chains can map both structural and functional properties of the molecular robot. Internal sensors will make it possible to explore the human body and explore areas of medical interest. Through controlled movement, they can penetrate the body, move to the treatment site (such as a tumor) and perform medical treatment only where it is needed. In addition, these robots will be able to take tissue samples and control the delivery of drugs based on sophisticated micro sensors. The movement and control mechanisms used here can be chemical, electromagnetic, bio-hybrid cell-driven or completely new mechanisms that are yet to be researched. Robotic theory should be translated to molecular and cellular-level systems, the dynamics of which are explained via first-order principle-based machine-learning algorithms. In addition, the practical closed-loop control and analysis of these systems via macro-robotic human‒machine interaction technologies should be explored, enabling a multitude of applications ranging from basic understanding of cellular dynamics and control to various medical applications such as targeted drug transportation.

Cellular manipulation is one field of research that will serve as an indispensable basis for molecular robotics. The mechanisms to be researched may be used to communicate with cells in a natural way and, if necessary, to control them. For example, it will be possible to have cells targeting certain positions, proliferating, producing certain proteins or, if the cell is harmful to the body, to have it removed through the body’s own degradation system. This research field combines concepts from biology research (cell biology, genetics, biochemistry, biophysics, etc.) with approaches from modern engineering sciences (systems theory, control engineering, computer science, information theory, robotics, AI, etc.) to create a standardized analysis environment for cell research. Over the next few years, this field will provide completely new insights into how cells function or communicate and can be expected to deliver new technologies.