Robots with sticky feet can climb up, down, and all around

Jet engines can have up to 25,000 individual parts, making regular maintenance a tedious task that can take over a month per engine. Many components are located deep inside the engine and cannot be inspected without taking the machine apart, adding time and costs to maintenance. This problem is not only confined to jet engines, either; many complicated, expensive machines like construction equipment, generators, and scientific instruments require large investments of time and money to inspect and maintain.

Researchers at Harvard University’s Wyss Institute for Biologically Inspired Engineering and John A. Paulson School of Engineering and Applied Sciences (SEAS) have created a micro-robot whose electroadhesive foot pads, origami ankle joints, and specially engineered walking gait allow it to climb on vertical and upside-down conductive surfaces, like the inside walls of a commercial jet engine. The work is reported in Science Robotics.

“Now that these robots can explore in three dimensions instead of just moving back and forth on a flat surface, there’s a whole new world that they can move around in and engage with,” said first author Sébastien de Rivaz, a former Research Fellow at the Wyss Institute and SEAS who now works at Apple. “They could one day enable non-invasive inspection of hard-to-reach areas of large machines, saving companies time and money and making those machines safer.”

The new robot, called HAMR-E (Harvard Ambulatory Micro-Robot with Electroadhesion), was developed in response to a challenge issued to the Harvard Microrobotics Lab by Rolls-Royce, which asked if it would be possible to design and build an army of micro-robots capable of climbing inside parts of its jet engines that are inaccessible to human workers. Existing climbing robots can tackle vertical surfaces, but experience problems when trying to climb upside-down, as they require a large amount of adhesive force to prevent them from falling.

The team based HAMR-E on one of its existing micro-robots, HAMR, whose four legs enable it to walk on flat surfaces and swim through water. While the basic design of HAMR-E is similar to HAMR, the scientists had to solve a series of challenges to get HAMR-E to successfully stick to and traverse the vertical, inverted, and curved surfaces that it would encounter in a jet engine.

First, they needed to create adhesive foot pads that would keep the robot attached to the surface even when upside-down, but also release to allow the robot to “walk” by lifting and placing its feet. The pads consist of a polyimide-insulated copper electrode, which enables the generation of electrostatic forces between the pads and the underlying conductive surface. The foot pads can be easily released and re-engaged by switching the electric field on and off, which operates at a voltage similar to that required to move the robot’s legs, thus requiring very little additional power. The electroadhesive foot pads can generate shear forces of 5.56 grams and normal forces of 6.20 grams — more than enough to keep the 1.48-gram robot from sliding down or falling off its climbing surface. In addition to providing high adhesive forces, the pads were designed to be able to flex, thus allowing the robot to climb on curved or uneven surfaces.

The scientists also created new ankle joints for HAMR-E that can rotate in three dimensions to compensate for rotations of its legs as it walks, allowing it to maintain its orientation on its climbing surface. The joints were manufactured out of layered fiberglass and polyimide, and folded into an origami-like structure that allows the ankles of all the legs to rotate freely, and to passively align with the terrain as HAMR-E climbs.

Finally, the researchers created a special walking pattern for HAMR-E, as it needs to have three foot pads touching a vertical or inverted surface at all times to prevent it from falling or sliding off. One foot releases from the surface, swings forward, and reattaches while the remaining three feet stay attached to the surface. At the same time, a small amount of torque is applied by the foot diagonally across from the lifted foot to keep the robot from moving away from the climbing surface during the leg-swinging phase. This process is repeated for the three other legs to create a full walking cycle, and is synchronized with the pattern of electric field switching on each foot.

When HAMR-E was tested on vertical and inverted surfaces, it was able to achieve more than one hundred steps in a row without detaching. It walked at speeds comparable to other small climbing robots on inverted surfaces and slightly slower than other climbing robots on vertical surfaces, but was significantly faster than other robots on horizontal surfaces, making it a good candidate for exploring environments that have a variety of surfaces in different arrangements in space. It is also able to perform 180-degree turns on horizontal surfaces.

HAMR-E also successfully maneuvered around a curved, inverted section of a jet engine while staying attached, and its passive ankle joints and adhesive foot pads were able to accommodate the rough and uneven features of the engine surface simply by increasing the electroadhesion voltage.

The team is continuing to refine HAMR-E, and plans to incorporate sensors into its legs that can detect and compensate for detached foot pads, which will help prevent it from falling off of vertical or inverted surfaces. HAMR-E’s payload capacity is also greater than its own weight, opening the possibility of carrying a power supply and other electronics and sensors to inspect various environments. The team is also exploring options for using HAMR-E on non-conductive surfaces.

“This iteration of HAMR-E is the first and most convincing step towards showing that this approach to a centimeter-scale climbing robot is possible, and that such robots could in the future be used to explore any sort of infrastructure, including buildings, pipes, engines, generators, and more,” said corresponding author Robert Wood, Ph.D., who is a Founding Core Faculty member of the Wyss Institute as well as the Charles River Professor of Engineering and Applied Sciences at SEAS.

“While academic scientists are very good at coming up with fundamental questions to explore in the lab, sometimes collaborations with industrial scientists who understand real-world problems are required to develop innovative technologies that can be translated into useful products. We are excited to help catalyze these collaborations here at the Wyss Institute, and to see the breakthrough advances that emerge,” said Wyss Founding Director Donald Ingber, M.D., Ph.D., who is also the Judah Folkman Professor of Vascular Biology at Harvard Medical School and the Vascular Biology Program at Boston Children’s Hospital, and Professor of Bioengineering at SEAS.

New models sense human trust in smart machines

New “classification models” sense how well humans trust intelligent machines they collaborate with, a step toward improving the quality of interactions and teamwork.

The long-term goal of the overall field of research is to design intelligent machines capable of changing their behavior to enhance human trust in them. The new models were developed in research led by assistant professor Neera Jain and associate professor Tahira Reid, in Purdue University’s School of Mechanical Engineering.

“Intelligent machines, and more broadly, intelligent systems are becoming increasingly common in the everyday lives of humans,” Jain said. “As humans are increasingly required to interact with intelligent systems, trust becomes an important factor for synergistic interactions.”

For example, aircraft pilots and industrial workers routinely interact with automated systems. Humans will sometimes override these intelligent machines unnecessarily if they think the system is faltering.

“It is well established that human trust is central to successful interactions between humans and machines,” Reid said.

The researchers have developed two types of “classifier-based empirical trust sensor models,” a step toward improving trust between humans and intelligent machines.

A YouTube video is available at https://www.youtube.com/watch?v=Mucl6pAgEQg.

The work aligns with Purdue’s Giant Leaps celebration, acknowledging the university’s global advancements made in AI, algorithms and automation as part of Purdue’s 150th anniversary. This is one of the four themes of the yearlong celebration’s Ideas Festival, designed to showcase Purdue as an intellectual center solving real-world issues.

The models use two techniques that provide data to gauge trust: electroencephalography and galvanic skin response. The first records brainwave patterns, and the second monitors changes in the electrical characteristics of the skin, providing psychophysiological “feature sets” correlated with trust.

Forty-five human subjects donned wireless EEG headsets and wore a device on one hand to measure galvanic skin response.

One of the new models, a “general trust sensor model,” uses the same set of psychophysiological features for all 45 participants. The other model is customized for each human subject, resulting in improved mean accuracy but at the expense of an increase in training time. The two models had a mean accuracy of 71.22 percent, and 78.55 percent, respectively.

It is the first time EEG measurements have been used to gauge trust in real time, or without delay.

“We are using these data in a very new way,” Jain said. “We are looking at it in sort of a continuous stream as opposed to looking at brain waves after a specific trigger or event.”

Findings are detailed in a research paper appearing in a special issue of the Association for Computing Machinery’s Transactions on Interactive Intelligent Systems. The journal’s special issue is titled “Trust and Influence in Intelligent Human-Machine Interaction.” The paper was authored by mechanical engineering graduate student Kumar Akash; former graduate student Wan-Lin Hu, who is now a postdoctoral research associate at Stanford University; Jain and Reid.

“We are interested in using feedback-control principles to design machines that are capable of responding to changes in human trust level in real time to build and manage trust in the human-machine relationship,” Jain said. “In order to do this, we require a sensor for estimating human trust level, again in real-time. The results presented in this paper show that psychophysiological measurements could be used to do this.”

The issue of human trust in machines is important for the efficient operation of “human-agent collectives.”

“The future will be built around human-agent collectives that will require efficient and successful coordination and collaboration between humans and machines,” Jain said. “Say there is a swarm of robots assisting a rescue team during a natural disaster. In our work we are dealing with just one human and one machine, but ultimately we hope to scale up to teams of humans and machines.”

Algorithms have been introduced to automate various processes.

“But we still have humans there who monitor what’s going on,” Jain said. “There is usually an override feature, where if they think something isn’t right they can take back control.”

Sometimes this action isn’t warranted.

“You have situations in which humans may not understand what is happening so they don’t trust the system to do the right thing,” Reid said. “So they take back control even when they really shouldn’t.”

In some cases, for example in the case of pilots overriding the autopilot, taking back control might actually hinder safe operation of the aircraft, causing accidents.

“A first step toward designing intelligent machines that are capable of building and maintaining trust with humans is the design of a sensor that will enable machines to estimate human trust level in real time,” Jain said.

To validate their method, 581 online participants were asked to operate a driving simulation in which a computer identified road obstacles. In some scenarios, the computer correctly identified obstacles 100 percent of the time, whereas in other scenarios the computer incorrectly identified the obstacles 50 percent of the time.

“So, in some cases it would tell you there is an obstacle, so you hit the brakes and avoid an accident, but in other cases it would incorrectly tell you an obstacle exists when there was none, so you hit the breaks for no reason,” Reid said.

The testing allowed the researchers to identify psychophysiological features that are correlated to human trust in intelligent systems, and to build a trust sensor model accordingly. “We hypothesized that the trust level would be high in reliable trials and be low in faulty trials, and we validated this hypothesis using responses collected from 581 online participants,” she said.

The results validated that the method effectively induced trust and distrust in the intelligent machine.

“In order to estimate trust in real time, we require the ability to continuously extract and evaluate key psychophysiological measurements,” Jain said. “This work represents the first use of real-time psychophysiological measurements for the development of a human trust sensor.”

The EEG headset records signals over nine channels, each channel picking up different parts of the brain.

“Everyone’s brainwaves are different, so you need to make sure you are building a classifier that works for all humans.”

For autonomous systems, human trust can be classified into three categories: dispositional, situational, and learned.

Dispositional trust refers to the component of trust that is dependent on demographics such as gender and culture, which carry potential biases.

“We know there are probably nuanced differences that should be taken into consideration,” Reid said. “Women trust differently than men, for example, and trust also may be affected by differences in age and nationality.”

Situational trust may be affected by a task’s level of risk or difficulty, while learned is based on the human’s past experience with autonomous systems.

The models they developed are called classification algorithms.

“The idea is to be able to use these models to classify when someone is likely feeling trusting versus likely feeling distrusting,” she said.

Jain and Reid have also investigated dispositional trust to account for gender and cultural differences, as well as dynamic models able to predict how trust will change in the future based on the data.

Mountain splendor? Scientists know where your eyes will look

Using precise brain measurements, Yale researchers predicted how people’s eyes move when viewing natural scenes, an advance in understanding the human visual system that can improve a host of artificial intelligence efforts, such as the development of driverless cars, said the researchers.

“We are visual beings and knowing how the brain rapidly computes where to look is fundamentally important,” said Yale’s Marvin Chun, Richard M. Colgate Professor of Psychology, professor of neuroscience and co-author of new research published Dec. 4 in the journal Nature Communications.

Eye movements have been extensively studied, and researchers can tell with some certainty where a gaze will be directed at different elements in the environment. What hasn’t been understood is how the brain orchestrates this ability, which is so fundamental to survival.

In a previous example of “mind reading,” Chun’s group successfully reconstructed facial images viewed while people were scanned in an MRI machine, based on their brain imaging data alone.

In the new paper, Chun and lead author Thomas P. O’Connell took a similar approach and showed that by analyzing the brain responses to complex, natural scenes, they could predict where people would direct their attention and gaze. This was made possible by analyzing the brain data with deep convolutional neural networks — models that are extensively used in artificial intelligence (AI).

“The work represents a perfect marriage of neuroscience and data science,” Chun said.

The findings have a myriad of potential applications — such as testing competing artificial intelligence systems that categorize images and guide driverless cars.

“People can see better than AI systems can,” Chun said. “Understanding how the brain performs its complex calculations is an ultimate goal of neuroscience and benefits AI efforts.”

 

Story Source:

Materials provided by Yale University. Original written by Bill Hathaway. Note: Content may be edited for style and length.

Computers successfully trained to identify animals in photos

A computer model developed at the University of Wyoming by UW researchers and others has demonstrated remarkable accuracy and efficiency in identifying images of wild animals from camera-trap photographs in North America.

The artificial-intelligence breakthrough, detailed in a paper published in the scientific journal Methods in Ecology and Evolution, is described as a significant advancement in the study and conservation of wildlife. The computer model is now available in a software package for Program R, a widely used programming language and free software environment for statistical computing.

“The ability to rapidly identify millions of images from camera traps can fundamentally change the way ecologists design and implement wildlife studies,” says the paper, whose lead authors are recent UW Department of Zoology and Physiology Ph.D. graduate Michael Tabak and Ryan Miller, both of the U.S. Department of Agriculture’s Center for Epidemiology and Animal Health in Fort Collins, Colo.

The study builds on UW research published earlier this year in the Proceedings of the National Academy of Sciences (PNAS) in which a computer model analyzed 3.2 million images captured by camera traps in Africa by a citizen science project called Snapshot Serengeti. The artificial-intelligence technique called deep learning categorized animal images at a 96.6 percent accuracy rate, the same as teams of human volunteers achieved, at a much more rapid pace than did the people.

In the latest study, the researchers trained a deep neural network on Mount Moran, UW’s high-performance computer cluster, to classify wildlife species using 3.37 million camera-trap images of 27 species of animals obtained from five states across the United States. The model then was tested on nearly 375,000 animal images at a rate of about 2,000 images per minute on a laptop computer, achieving 97.6 percent accuracy — likely the highest accuracy to date in using machine learning for wildlife image classification.

The computer model also was tested on an independent subset of 5,900 images of moose, cattle, elk and wild pigs from Canada, producing an accuracy rate of 81.8 percent. And it was 94 percent successful in removing “empty” images (without any animals) from a set of photographs from Tanzania.

The researchers have made their model freely available in a software package in Program R. The package, “Machine Learning for Wildlife Image Classification in R (MLWIC),” allows other users to classify their images containing the 27 species in the dataset, but it also allows users to train their own machine learning models using images from new datasets.

The lead author of the PNAS article, recent UW computer science Ph.D. graduate Mohammad Sadegh (Arash) Norouzzadeh, is one of multiple contributors to the new paper in Methods in Ecology and Evolution. Other participating researchers from UW are Department of Computer Science Associate Professor Jeff Clune and postdoctoral researcher Elizabeth Mandeville of the Wyoming Cooperative Fish and Wildlife Research Unit.

Other organizations represented in the research group are the USDA’s National Wildlife Research Center, Arizona State University, California’s Tejon Ranch Conservancy, the University of Georgia, the University of Florida, Colorado Parks and Wildlife, the University of Saskatchewan and the University of Montana.

Story Source:

Materials provided by University of Wyoming. Note: Content may be edited for style and length.

Smarter AI: Machine learning without negative data

A research team from the RIKEN Center for Advanced Intelligence Project (AIP) has successfully developed a new method for machine learning that allows an AI to make classifications without what is known as “negative data,” a finding which could lead to wider application to a variety of classification tasks.

Classifying things is critical for our daily lives. For example, we have to detect spam mail, fake political news, as well as more mundane things such as objects or faces. When using AI, such tasks are based on “classification technology” in machine learning — having the computer learn using the boundary separating positive and negative data. For example, “positive” data would be photos including a happy face, and “negative” data photos that include a sad face. Once a classification boundary is learned, the computer can determine whether a certain data is positive or negative. The difficulty with this technology is that it requires both positive and negative data for the learning process, and negative data are not available in many cases (for instance, it is hard to find photos with the label, “this photo includes a sad face,” since most people smile in front of a camera.)

In terms of real-life programs, when a retailer is trying to predict who will make a purchase, it can easily find data on customers who purchased from them (positive data), but it is basically impossible to obtain data on customers who did not purchase from them (negative data), since they do not have access to their competitors’ data. Another example is a common task for app developers: they need to predict which users will continue using the app (positive) or stop (negative). However, when a user unsubscribes, the developers lose the user’s data because they have to completely delete data regarding that user in accordance with the privacy policy to protect personal information.

According to lead author Takashi Ishida from RIKEN AIP, “Previous classification methods could not cope with the situation where negative data were not available, but we have made it possible for computers to learn with only positive data, as long as we have a confidence score for our positive data, constructed from information such as buying intention or the active rate of app users. Using our new method, we can let computers learn a classifier only from positive data equipped with confidence.”

Ishida proposed, together with researcher Niu Gang from his group and team leader Masashi Sugiyama, that they let computers learn well by adding the confidence score, which mathematically corresponds to the probability whether the data belongs to a positive class or not. They succeeded in developing a method that can let computers learn a classification boundary only from positive data and information on its confidence (positive reliability) against classification problems of machine learning that divide data positively and negatively.

To see how well the system functioned, they used it on a set of photos that contains various labels of fashion items. For example, they chose “T-shirt,” as the positive class and one other item, e.g., “sandal,” as the negative class. Then they attached a confidence score to the “T-shirt” photos. They found that without accessing the negative data (e.g., “sandal” photos), in some cases, their method was just as good as a method that involves using positive and negative data.

According to Ishida, “This discovery could expand the range of applications where classification technology can be used. Even in fields where machine learning has been actively used, our classification technology could be used in new situations where only positive data can be gathered due to data regulation or business constraints. In the near future, we hope to put our technology to use in various research fields, such as natural language processing, computer vision, robotics, and bioinformatics.”

Story Source:

Materials provided by RIKEN. Note: Content may be edited for style and length.

Android child’s face strikingly expressive

Japan’s affection for robots is no secret. But is the feeling mutual in the country’s amazing androids? We may now be a step closer to giving androids greater facial expressions to communicate with.

While robots have featured in advances in healthcare, industrial, and other settings in Japan, capturing humanistic expression in a robotic face remains an elusive challenge. Although their system properties have been generally addressed, androids’ facial expressions have not been examined in detail. This is owing to factors such as the huge range and asymmetry of natural human facial movements, the restrictions of materials used in android skin, and of course the intricate engineering and mathematics driving robots’ movements.

A trio of researchers at Osaka University has now found a method for identifying and quantitatively evaluating facial movements on their android robot child head. Named Affetto, the android’s first-generation model was reported in a 2011 publication. The researchers have now found a system to make the second-generation Affetto more expressive. Their findings offer a path for androids to express greater ranges of emotion, and ultimately have deeper interaction with humans.

The researchers reported their findings in the journal Frontiers in Robotics and AI.

“Surface deformations are a key issue in controlling android faces,” study co-author Minoru Asada explains. “Movements of their soft facial skin create instability, and this is a big hardware problem we grapple with. We sought a better way to measure and control it.”

The researchers investigated 116 different facial points on Affetto to measure its three-dimensional movement. Facial points were underpinned by so-called deformation units. Each unit comprises a set of mechanisms that create a distinctive facial contortion, such as lowering or raising of part of a lip or eyelid. Measurements from these were then subjected to a mathematical model to quantify their surface motion patterns.

While the researchers encountered challenges in balancing the applied force and in adjusting the synthetic skin, they were able to employ their system to adjust the deformation units for precise control of Affetto’s facial surface motions.

“Android robot faces have persisted in being a black box problem: they have been implemented but have only been judged in vague and general terms,” study first author Hisashi Ishihara says. “Our precise findings will let us effectively control android facial movements to introduce more nuanced expressions, such as smiling and frowning.”

Story Source:

Materials provided by Osaka University. Note: Content may be edited for style and length.

New method peeks inside the ‘black box’ of artificial intelligence

Artificial intelligence — specifically, machine learning — is a part of daily life for computer and smartphone users. From autocorrecting typos to recommending new music, machine learning algorithms can help make life easier. They can also make mistakes.

It can be challenging for computer scientists to figure out what went wrong in such cases. This is because many machine learning algorithms learn from information and make their predictions inside a virtual “black box,” leaving few clues for researchers to follow.

A group of computer scientists at the University of Maryland has developed a promising new approach for interpreting machine learning algorithms. Unlike previous efforts, which typically sought to “break” the algorithms by removing key words from inputs to yield the wrong answer, the UMD group instead reduced the inputs to the bare minimum required to yield the correct answer. On average, the researchers got the correct answer with an input of less than three words.

In some cases, the researchers’ model algorithms provided the correct answer based on a single word. Frequently, the input word or phrase appeared to have little obvious connection to the answer, revealing important insights into how some algorithms react to specific language. Because many algorithms are programmed to give an answer no matter what — even when prompted by a nonsensical input — the results could help computer scientists build more effective algorithms that can recognize their own limitations.

The researchers will present their work on November 4, 2018 at the 2018 Conference on Empirical Methods in Natural Language Processing.

“Black-box models do seem to work better than simpler models, such as decision trees, but even the people who wrote the initial code can’t tell exactly what is happening,” said Jordan Boyd-Graber, the senior author of the study and an associate professor of computer science at UMD. “When these models return incorrect or nonsensical answers, it’s tough to figure out why. So instead, we tried to find the minimal input that would yield the correct result. The average input was about three words, but we could get it down to a single word in some cases.”

In one example, the researchers entered a photo of a sunflower and the text-based question, “What color is the flower?” as inputs into a model algorithm. These inputs yielded the correct answer of “yellow.” After rephrasing the question into several different shorter combinations of words, the researchers found that they could get the same answer with “flower?” as the only text input for the algorithm.

In another, more complex example, the researchers used the prompt, “In 1899, John Jacob Astor IV invested $100,000 for Tesla to further develop and produce a new lighting system. Instead, Tesla used the money to fund his Colorado Springs experiments.”

They then asked the algorithm, “What did Tesla spend Astor’s money on?” and received the correct answer, “Colorado Springs experiments.” Reducing this input to the single word “did” yielded the same correct answer.

The work reveals important insights about the rules that machine learning algorithms apply to problem solving. Many real-world issues with algorithms result when an input that makes sense to humans results in a nonsensical answer. By showing that the opposite is also possible — that nonsensical inputs can also yield correct, sensible answers — Boyd-Graber and his colleagues demonstrate the need for algorithms that can recognize when they answer a nonsensical question with a high degree of confidence.

“The bottom line is that all this fancy machine learning stuff can actually be pretty stupid,” said Boyd-Graber, who also has co-appointments at the University of Maryland Institute for Advanced Computer Studies (UMIACS) as well as UMD’s College of Information Studies and Language Science Center. “When computer scientists train these models, we typically only show them real questions or real sentences. We don’t show them nonsensical phrases or single words. The models don’t know that they should be confused by these examples.”

Most algorithms will force themselves to provide an answer, even with insufficient or conflicting data, according to Boyd-Graber. This could be at the heart of some of the incorrect or nonsensical outputs generated by machine learning algorithms — in model algorithms used for research, as well as real-world algorithms that help us by flagging spam email or offering alternate driving directions. Understanding more about these errors could help computer scientists find solutions and build more reliable algorithms.

“We show that models can be trained to know that they should be confused,” Boyd-Graber said. “Then they can just come right out and say, ‘You’ve shown me something I can’t understand.'”

In addition to Boyd-Graber, UMD-affiliated researchers involved with this work include undergraduate researcher Eric Wallace; graduate students Shi Feng and Pedro Rodriguez; and former graduate student Mohit Iyyer (M.S. ’14, Ph.D. ’17, computer science).

The research presentation, “Pathologies of Neural Models Make Interpretation Difficult,” Shi Feng, Eric Wallace, Alvin Grissom II, Pedro Rodriguez, Mohit Iyyer, and Jordan Boyd-Graber, will be presented at the 2018 Conference on Empirical Methods in Natural Language Processing on November 4, 2018.

This work was supported by the Defense Advanced Research Projects Agency (Award No. HR0011-15-C-011) and the National Science Foundation (Award No. IIS1652666). The content of this article does not necessarily reflect the views of these organizations.

Machines that learn language more like kids do

Children learn language by observing their environment, listening to the people around them, and connecting the dots between what they see and hear. Among other things, this helps children establish their language’s word order, such as where subjects and verbs fall in a sentence.

In computing, learning language is the task of syntactic and semantic parsers. These systems are trained on sentences annotated by humans that describe the structure and meaning behind words. Parsers are becoming increasingly important for web searches, natural-language database querying, and voice-recognition systems such as Alexa and Siri. Soon, they may also be used for home robotics.

But gathering the annotation data can be time-consuming and difficult for less common languages. Additionally, humans don’t always agree on the annotations, and the annotations themselves may not accurately reflect how people naturally speak.

In a paper being presented at this week’s Empirical Methods in Natural Language Processing conference, MIT researchers describe a parser that learns through observation to more closely mimic a child’s language-acquisition process, which could greatly extend the parser’s capabilities. To learn the structure of language, the parser observes captioned videos, with no other information, and associates the words with recorded objects and actions. Given a new sentence, the parser can then use what it’s learned about the structure of the language to accurately predict a sentence’s meaning, without the video.

This “weakly supervised” approach — meaning it requires limited training data — mimics how children can observe the world around them and learn language, without anyone providing direct context. The approach could expand the types of data and reduce the effort needed for training parsers, according to the researchers. A few directly annotated sentences, for instance, could be combined with many captioned videos, which are easier to come by, to improve performance.

In the future, the parser could be used to improve natural interaction between humans and personal robots. A robot equipped with the parser, for instance, could constantly observe its environment to reinforce its understanding of spoken commands, including when the spoken sentences aren’t fully grammatical or clear. “People talk to each other in partial sentences, run-on thoughts, and jumbled language. You want a robot in your home that will adapt to their particular way of speaking … and still figure out what they mean,” says co-author Andrei Barbu, a researcher in the Computer Science and Artificial Intelligence Laboratory (CSAIL) and the Center for Brains, Minds, and Machines (CBMM) within MIT’s McGovern Institute.

The parser could also help researchers better understand how young children learn language. “A child has access to redundant, complementary information from different modalities, including hearing parents and siblings talk about the world, as well as tactile information and visual information, [which help him or her] to understand the world,” says co-author Boris Katz, a principal research scientist and head of the InfoLab Group at CSAIL. “It’s an amazing puzzle, to process all this simultaneous sensory input. This work is part of bigger piece to understand how this kind of learning happens in the world.”

Co-authors on the paper are: first author Candace Ross, a graduate student in the Department of Electrical Engineering and Computer Science and CSAIL, and a researcher in CBMM; Yevgeni Berzak PhD ’17, a postdoc in the Computational Psycholinguistics Group in the Department of Brain and Cognitive Sciences; and CSAIL graduate student Battushig Myanganbayar.

Visual learner

For their work, the researchers combined a semantic parser with a computer-vision component trained in object, human, and activity recognition in video. Semantic parsers are generally trained on sentences annotated with code that ascribes meaning to each word and the relationships between the words. Some have been trained on still images or computer simulations.

The new parser is the first to be trained using video, Ross says. In part, videos are more useful in reducing ambiguity. If the parser is unsure about, say, an action or object in a sentence, it can reference the video to clear things up. “There are temporal components — objects interacting with each other and with people — and high-level properties you wouldn’t see in a still image or just in language,” Ross says.

The researchers compiled a dataset of about 400 videos depicting people carrying out a number of actions, including picking up an object or putting it down, and walking toward an object. Participants on the crowdsourcing platform Mechanical Turk then provided 1,200 captions for those videos. They set aside 840 video-caption examples for training and tuning, and used 360 for testing. One advantage of using vision-based parsing is “you don’t need nearly as much data — although if you had [the data], you could scale up to huge datasets,” Barbu says.

In training, the researchers gave the parser the objective of determining whether a sentence accurately describes a given video. They fed the parser a video and matching caption. The parser extracts possible meanings of the caption as logical mathematical expressions. The sentence, “The woman is picking up an apple,” for instance, may be expressed as: ?xy. woman x, pick_up x y, apple y.

Those expressions and the video are inputted to the computer-vision algorithm, called “Sentence Tracker,” developed by Barbu and other researchers. The algorithm looks at each video frame to track how objects and people transform over time, to determine if actions are playing out as described. In this way, it determines if the meaning is possibly true of the video.

Connecting the dots

The expression with the most closely matching representations for objects, humans, and actions becomes the most likely meaning of the caption. The expression, initially, may refer to many different objects and actions in the video, but the set of possible meanings serves as a training signal that helps the parser continuously winnow down possibilities. “By assuming that all of the sentences must follow the same rules, that they all come from the same language, and seeing many captioned videos, you can narrow down the meanings further,” Barbu says.

In short, the parser learns through passive observation: To determine if a caption is true of a video, the parser by necessity must identify the highest probability meaning of the caption. “The only way to figure out if the sentence is true of a video [is] to go through this intermediate step of, ‘What does the sentence mean?’ Otherwise, you have no idea how to connect the two,” Barbu explains. “We don’t give the system the meaning for the sentence. We say, ‘There’s a sentence and a video. The sentence has to be true of the video. Figure out some intermediate representation that makes it true of the video.'”

The training produces a syntactic and semantic grammar for the words it’s learned. Given a new sentence, the parser no longer requires videos, but leverages its grammar and lexicon to determine sentence structure and meaning.

Ultimately, this process is learning “as if you’re a kid,” Barbu says. “You see world around you and hear people speaking to learn meaning. One day, I can give you a sentence and ask what it means and, even without a visual, you know the meaning.”

In future work, the researchers are interested in modeling interactions, not just passive observations. “Children interact with the environment as they’re learning. Our idea is to have a model that would also use perception to learn,” Ross says.

This work was supported, in part, by the CBMM, the National Science Foundation, a Ford Foundation Graduate Research Fellowship, the Toyota Research Institute, and the MIT-IBM Brain-Inspired Multimedia Comprehension project.

Humans help robots learn tasks

In the basement of the Gates Computer Science Building at Stanford University, a screen attached to a red robotic arm lights up. A pair of cartoon eyes blinks. “Meet Bender,” says Ajay Mandlekar, PhD student in electrical engineering.

Bender is one of the robot arms that a team of Stanford researchers is using to test two frameworks that, together, could make it faster and easier to teach robots basic skills. The RoboTurk framework allows people to direct the robot arms in real time with a smartphone and a browser by showing the robot how to carry out tasks like picking up objects. SURREAL speeds the learning process by running multiple experiences at once, essentially allowing the robots to learn from many experiences simultaneously.

“With RoboTurk and SURREAL, we can push the boundary of what robots can do by combining lots of data collected by humans and coupling that with large-scale reinforcement learning,” said Mandlekar, a member of the team that developed the frameworks.

The group will be presenting RoboTurk and SURREAL Oct. 29 at the conference on robot learning in Zurich, Switzerland.

Humans teaching robots

Yuke Zhu, a PhD student in computer science and a member of the team, showed how the system works by opening the app on his iPhone and waving it through the air. He guided the robot arm — like a mechanical crane in an arcade game — to hover over his prize: a wooden block painted to look like a steak. This is a simple pick-and-place task that involves identifying objects, picking them up and putting them into the bin with the correct label.

To humans, the task seems ridiculously easy. But for the robots of today, it’s quite difficult. Robots typically learn by interacting with and exploring their environment — which usually results in lots of random arm waving — or from large datasets. Neither of these is as efficient as getting some human help. In the same way that parents teach their children to brush their teeth by guiding their hands, people can demonstrate to robots how to do specific tasks.

However, those lessons aren’t always perfect. When Zhu pressed hard on his phone screen and the robot released its grip, the wooden steak hit the edge of the bin and clattered onto the table. “Humans are by no means optimal at this,” Mandlekar said, “but this experience is still integral for the robots.”

Faster learning in parallel

These trials — even the failures — provide invaluable information. The demonstrations collected through RoboTurk will give the robots background knowledge to kickstart their learning. SURREAL can run thousands of simulated experiences by people worldwide at once to speed the learning process.

“With SURREAL, we want to accelerate this process of interacting with the environment,” said Linxi Fan, a PhD student in computer science and a member of the team. These frameworks drastically increase the amount of data for the robots to learn from.

“The twin frameworks combined can provide a mechanism for AI-assisted human performance of tasks where we can bring humans away from dangerous environments while still retaining a similar level of task execution proficiency,” said postdoctoral fellow Animesh Garg, a member of the team that developed the frameworks.

The team envisions that robots will be an integral part of everyday life in the future: helping with household chores, performing repetitive assembly tasks in manufacturing or completing dangerous tasks that may pose a threat to humans.

“You shouldn’t have to tell the robot to twist its arm 20 degrees and inch forward 10 centimeters,” said Zhu. “You want to be able to tell the robot to go to the kitchen and get an apple.”

Current members of the RoboTurk and SURREAL team include Ajay Mandlekar, Yuke Zhu, Linxi Fan, Animesh Garg and faculty members Fei-Fei Li and Silvio Savarese.

Story Source:

Materials provided by Stanford University. Note: Content may be edited for style and length.

Shielded quantum bits

The researchers have found ways to shield electric and magnetic noise for a short time. This will make it possible to use spins as memory for quantum computers, as the coherence time is extended and many thousand computer operations can be performed during this interval. The study was published in the current issue of the journal Physical Review Letters.

The technological vision of building a quantum computer does not only depend on computer and information science. New insights in theoretical physics, too, are decisive for progress in the practical implementation. Every computer or communication device contains information embedded in physical systems. “In the case of a quantum computer, we use spin qubits, for example, to realize information processing,” explains Professor Guido Burkard, who carries out his research in cooperation with colleagues from Princeton University. The theoretical findings that led to the current publication were largely made by the lead author of the study, doctoral researcher Maximilian Russ from the University of Konstanz.

In the quest for the quantum computer, spin qubits and their magnetic properties are the centre of attention. To use spins as memory in quantum technology, they must be lined up, because otherwise they cannot be controlled specifically. “Usually magnets are controlled by magnetic fields — like a compass needle in the Earth’s magnetic field’, explains Guido Burkard. “In our case the particles are extremely small and the magnets very weak, which makes it really difficult to control them.” The physicists meet this challenge with electric fields and a procedure in which several electrons, in this case four, form a quantum bit. Another problem they have to face is the electron spins, which are rather sensitive and fragile. Even in solid bodies of silicon they react to external interferences with electric or magnetic noise. The current study focuses on theoretical models and calculations of how the quantum bits can be shielded from this noise — an important contribution to basic research for a quantum computer: If this noise can be shielded for even the briefest of times, thousands of computer operations can be carried out in these fractions of a second — at least theoretically.

The next step for the physicists from Konstanz will now be to work with their experimental colleagues towards testing their theory in experiments. For the first time, four instead of three electrons will be used in these experiments, which could, e.g., be implemented by the research partners in Princeton. While the Konstanz-based physicists provide the theoretical basis, the collaboration partners in the US perform the experimental part. This research is not the only reason why Konstanz is now on the map for qubit research. This autumn, for example, Konstanz attracted the internationally leading scientific community in this field for the “4th School and Conference on Based Quantum Information Processing.”

Story Source:

Materials provided by University of Konstanz. Note: Content may be edited for style and length.

Robots as tools and partners in rehabilitation

In future decades the need for effective strategies for medical rehabilitation will increase significantly, because patients’ rate of survival after diseases with severe functional deficits, such as a stroke, will increase. Socially assistive robots (SARs) are already being used in rehabilitation for this reason. In the journal Science Robotics, a research team led by neuroscientist Dr. Philipp Kellmeyer of the Freiburg University Medical Center and Prof. Dr. Oliver Müller from the Department of Philosophy of the University of Freiburg, analyzes the improvements necessary to make SARs valuable and trustworthy assistants for medical therapies.

The researchers conclude that the development of SARs not only requires technical improvements, but primarily social, trust-building measures. Rehabilitation patients in particular are dependent on a reliable relationship with their therapists. So there must be trust in the safety of the robotic system, especially regarding the predictability of the machines’ behavior. Given the ever-growing intelligence of the robots and with it their independence, this is highly important.

In addition, robots and patients can only interact well, the scientists explain, when they have shared goals that they pursue through the therapy. To achieve this, aspects of philosophical and developmental psychology must also be taken into account in the development of SARs: the ability of robots to recognize the aims and motives of a patient is a critical requirement if cooperation is to be successful. So there must also be trust for the participants to adapt to one another. The frustration felt by patients, for instance as a result of physical or linguistic limitations, would be avoided if the robots were adapted to the specific needs and vulnerabilities of the patient in question.

Philipp Kellmeyer and Oliver Müller are members of the Cluster of Excellence BrainLinks-BrainTools of the University of Freiburg. The study also involved Prof. Dr. Shelly Levy-Tzedek and Ronit Feingold-Polak from the Ben Gurion University of the Negev, Israel. In the 2018/19 academic year, the Freiburg researchers together with the legal academic Prof. Dr. Silja Vöneky and the IT specialist Prof. Dr. Wolfram Burgard, both from the University of Freiburg, are developing a Research Focus into normative aspects of interaction between people and autonomous intelligent systems at the Freiburg Institute for Advanced Studies (FRIAS).

Story Source:

Materials provided by University of Freiburg. Note: Content may be edited for style and length.

More workers working might not get more work done, ants (and robots) show

For ants and robots operating in confined spaces like tunnels, having more workers does not necessarily mean getting more work done. Just as too many cooks in a kitchen get in each other’s way, having too many robots in tunnels creates clogs that can bring the work to a grinding halt.

A study published August 17 in the journal Science shows that in fire ant colonies, a small number of workers does most of digging, leaving the other ants to look somewhat less than industrious. For digging nest tunnels, this less busy approach gets the job done without ant traffic jams — ensuring smooth excavation flow. Researchers found that applying the ant optimization strategy to autonomous robots avoids mechanized clogs and gets the work done with the least amount of energy.

Optimizing the activity of autonomous underground robots could be useful for tasks such as disaster recovery, mining or even digging underground shelters for future planetary explorers. The research was supported by the National Science Foundation’s Physics of Living Systems program, the Army Research Office and the Dunn Family Professorship.

“We noticed that if you have 150 ants in a container, only 10 or 15 of them will actually be digging in the tunnels at any given time,” said Daniel Goldman, a professor in the School of Physics at the Georgia Institute of Technology. “We wanted to know why, and to understand how basic laws of physics might be at work. We found a functional, community benefit to this seeming inequality in the work environment. Without it, digging just doesn’t get done.”

By monitoring the activities of 30 ants that had been painted to identify each individual, Goldman and colleagues, including former postdoctoral fellow Daria Monaenkova and Ph.D. student Bahnisikha Dutta, discovered that just 30 percent of the ants were doing 70 percent of the work — an inequality that seems to keep the work humming right along. However, that is apparently not because the busiest ants are the most qualified. When the researchers removed the five hardest working ants from the nest container, they saw no productivity decline as the remaining 25 continued to dig.

Having a nest is essential to fire ants, and if a colony is displaced — by a flood, for instance — the first thing the ants will do upon reaching dry land is start digging. Their tunnels are narrow, barely wide enough for two ants to pass, a design feature hypothesized to give locomotion advantages in the developing vertical tunnels. Still, the ants know how to avoid creating clogs by retreating from tunnels already occupied by other workers — and sometimes by not doing anything much at all.

To avoid clogs and maximize digging in the absence of a leader, robots built by Goldman’s master’s degree student Vadim Linevich were programmed to capture aspects of the dawdling and retreating ants. The researchers found that as many as three robots could work effectively in a narrow horizontal tunnel digging 3D printed magnetic plastic balls that simulated sticky soil. If a fourth robot entered the tunnel, however, that produced a clog that stopped the work entirely.

“When we put four robots into a confined environment and tried to get them to dig, they immediately jammed up,” said Goldman, who is the Dunn Family Professor in the School of Physics. “While observing the ants, we were surprised to see that individuals would sometimes go to the tunnel and if they encountered even a small amount of clog, they’d just turn around and retreat. When we put those rules into combinations with the robots, that created a good strategy for digging rapidly with low amounts of energy use per robot.”

Experimentally, the research team tested three potential behaviors for the robots, which they termed “eager,” “reversal” or “lazy.” Using the eager strategy, all four robots plunged into the work — and quickly jammed up. In the reversal behavior, robots gave up and turned around when they encountered delays reaching the work site. In the lazy strategy, dawdling was encouraged.

“Eager is the best strategy if you only have three robots, but if you add a fourth, that behavior tanks because they get in each other’s way,” said Goldman. “Reversal produces relatively sane and sensible digging. It is not the fastest strategy, but there are no jams. If you look at energy consumed, lazy is the best course.” Analysis techniques based on glassy and supercooled fluids, led by former Ph.D. student Jeffrey Aguilar, gave insight into how the different strategies mitigated and prevented clog-forming clusters.

To understand what was going on and experiment with the parameters, Goldman and colleagues — including Will Savoie, a Georgia Tech Ph.D. student, Research Assistant Hui-Shun Kuan and Professor Meredith Betterton from the School of Physics at University of Colorado at Boulder — used computer modeling known as cellular automata that has similarities to the way in which traffic engineers model the movement of cars and trucks on a highway.

“On highways, too few cars don’t provide much flow, while too many cars create a jam,” Goldman said. “There is an intermediate level where things are best, and that is called the fundamental diagram. From our modeling, we learned that the ants are working right at the peak of the diagram. The right mix of unequal work distributions and reversal behaviors has the benefit of keeping them moving at maximum efficiency without jamming.”

The researchers used robots designed and built for the research, but they were no match for the capabilities of the ants. The ants are flexible and robust, able to squeeze past each other in confines that would cause the inflexible robots to jam. In some cases, the robots in Goldman’s lab even damaged each other while jostling into position for digging.

The research findings could be useful for space exploration where tunnels might be needed to quickly shield humans from approaching dust storms or other threats. “If you were a robot swarm on Mars and needed to dig deeply in a hurry to get away from dust storms, this strategy might help provide shelter without having perfect information about what everybody was doing,” Goldman explained.

Beyond the potential robotics applications, the work provides insights into the complex social skills of ants and adds to the understanding of active matter.

“Ants that live in complex subterranean environments have to develop sophisticated social rules to avoid the bad things that can happen when you have a lot of individuals in a crowded environment,” Goldman said. “We are also contributing to understanding the physics of task-oriented active matter, putting more experimental knowledge into phenomenon such as swarms.”

Robots have power to significantly influence children’s opinions

Young children are significantly more likely than adults to have their opinions and decisions influenced by robots, according to new research.

The study, conducted at the University of Plymouth, compared how adults and children respond to an identical task when in the presence of both their peers and humanoid robots.

It showed that while adults regularly have their opinions influenced by peers, something also demonstrated in previous studies, they are largely able to resist being persuaded by robots.

However, children aged between seven and nine were more likely to give the same responses as the robots, even if they were obviously incorrect.

The study used the Asch paradigm, first developed in the 1950s, which asks people to look at a screen showing four lines and say which two match in length. When alone, people almost never make a mistake but when doing the experiment with others, they tend to follow what others are saying.

When children were alone in the room in this research, they scored 87% on the test, but when the robots join in their score drops to 75%. And of the wrong answers, 74% matched those of the robot.

Writing in Science Robotics, scientists say the study provides an interesting insight into how robots could be used positively within society. However, they also say it does raise some concerns around the potential for robots to have a negative influence on vulnerable young children.

The research was led by former Plymouth researcher Anna Vollmer, now a Postdoctoral Researcher at the University of Bielefeld, and Professor in Robotics Tony Belpaeme, from the University of Plymouth and Ghent University.

Professor Belpaeme said: “People often follow the opinions of others and we’ve known for a long time that it is hard to resist taking over views and opinions of people around us. We know this as conformity. But as robots will soon be found in the home and the workplace, we were wondering if people would conform to robots.

“What our results show is that adults do not conform to what the robots are saying. But when we did the experiment with children, they did. It shows children can perhaps have more of an affinity with robots than adults, which does pose the question: what if robots were to suggest, for example, what products to buy or what to think?”

Researchers in Plymouth have worked extensively to explore the positive impact robots can have in health and education settings.

They led the four-year ALIZ-E programme, which showed that social robots can help diabetic children accept the nature of their condition, and are leading L2TOR, which aims to design a robot that can be used to support preschool children learning a second language.

In their conclusion to the current study, the researchers add: “A future in which autonomous social robots are used as aids for education professionals or child therapists is not distant. In these applications, the robot is in a position in which the information provided can significantly affect the individuals they interact with. A discussion is required about whether protective measures, such as a regulatory framework, should be in place that minimise the risk to children during social child-robot interaction and what form they might take so as not to adversely affect the promising development of the field.”

Story Source:

Materials provided by University of Plymouth. Note: Content may be edited for style and length.

Robots will never replace teachers but can boost children’s education

Robots can play an important role in the education of young people but will never fully replace teachers, a new study suggests.

Writing in Science Robotics, scientists say social robots are proving effective in the teaching of certain narrow subjects, such as vocabulary or prime numbers.

But current technical limitations — particularly around speech recognition and the ability for social interaction — mean their role will largely be confined to that of teaching assistants or tutors, at least for the foreseeable future.

The study was led by Professor in Robotics Tony Belpaeme, from the University of Plymouth and Ghent University, who has worked in the field of social robotics for around two decades.

He said: “In recent years scientists have started to build robots for the classroom — not the robot kits used to learn about technology and mathematics, but social robots that can actually teach. This is because pressures on teaching budgets, and calls for more personalised teaching, have led to a search for technological solutions.

“In the broadest sense, social robots have the potential to become part of the educational infrastructure just like paper, white boards, and computer tablets. But a social robot has the potential to support and challenge students in ways unavailable in current resource-limited educational environments. Robots can free up precious time for teachers, allowing the teacher to focus on what people still do best — provide a comprehensive, empathic, and rewarding educational experience.”

The current study, compiled in conjunction with academics at Yale University and the University of Tsukuba, involved a review of more than 100 published articles, which have shown robots to be effective at increasing outcomes, largely because of their physical presence.

However it also explored in detail some of the technical constraints highlighting that speech recognition, for example, is still insufficiently robust to allow the robot to understand spoken utterances from young children.

It also says that introducing social robots into the school curriculum would pose significant logistical challenges and might in fact carry risks, with some children being seen to rely too heavily on the help offered by robots rather than simply using them when they are in difficulty.

In their conclusion, the authors add: “Next to the practical considerations of introducing robots in education, there are also ethical issues. How far do we want the education of our children to be delegated to machines? Overall, learners are positive about their experiences, but parents and teaching staff adopt a more cautious attitude.

“Notwithstanding that, robots show great promise when teaching restricted topics with the effects almost matching those of human tutoring. So although the use of robots in educational settings is limited by technical and logistical challenges for now, it is highly likely that classrooms of the future will feature robots that assist a human teacher.”

Story Source:

Materials provided by University of Plymouth. Note: Content may be edited for style and length.

Toward a universal quantum computer

Researchers have demonstrated holonomic quantum gates under zero-magnetic field at room temperature, which will enable the realization of fast and fault-tolerant universal quantum computers.

A quantum computer is a powerful machine with the potential to solve complex problems much faster than today’s conventional computer can. Researchers are currently working on the next step in quantum computing: building a universal quantum computer.

The paper, published in the journal Nature Communications, reports experimental demonstration of non-adiabatic and non-abelian holonomic quantum gates over a geometric spin qubit on an electron or nitrogen nucleus, which paves the way to realizing a universal quantum computer.

The geometric phase is currently a key issue in quantum physics. A holonomic quantum gate manipulating purely the geometric phase in the degenerate ground state system is believed to be an ideal way to build a fault-tolerant universal quantum computer. The geometric phase gate or holonomic quantum gate has been experimentally demonstrated in several quantum systems including nitrogen-vacancy (NV) centers in diamond. However, previous experiments required microwaves or light waves to manipulate the non-degenerate subspace, leading to the degradation of gate fidelity due to unwanted interference of the dynamic phase.

“To avoid unwanted interference, we used a degenerate subspace of the triplet spin qutrit to form an ideal logical qubit, which we call a geometric spin qubit, in an NV center. This method facilitated fast and precise geometric gates at a temperature below 10 K, and the gate fidelity was limited by radiative relaxation,” says the corresponding author Hideo Kosaka, Professor, Yokohama National University. “Based on this method, in combination with polarized microwaves, we succeeded in manipulation of the geometric phase in an NV center in diamond under a zero-magnetic field at room temperature.”

The group also demonstrated a two-qubit holonomic gate to show universality by manipulating the electron-nucleus entanglement. The scheme renders a purely holonomic gate without requiring an energy gap, which would have induced dynamic phase interference to degrade the gate fidelity, and thus enables precise and fast control over long-lived quantum memories, for realizing quantum repeaters interfacing between universal quantum computers and secure communication networks.

Story Source:

Materials provided by Yokohama National University. Note: Content may be edited for style and length.

Machine learning technique reconstructs images passing through a multimode fiber

Through innovative use of a neural network that mimics image processing by the human brain, a research team reports accurate reconstruction of images transmitted over optical fibers for distances of up to a kilometer.

In The Optical Society’s journal for high-impact research, Optica, the researchers report teaching a type of machine learning algorithm known as a deep neural network to recognize images of numbers from the pattern of speckles they create when transmitted to the far end of a fiber. The work could improve endoscopic imaging for medical diagnosis, boost the amount of information carried over fiber-optic telecommunication networks, or increase the optical power delivered by fibers.

“We use modern deep neural network architectures to retrieve the input images from the scrambled output of the fiber,” said Demetri Psaltis, Swiss Federal Institute of Technology, Lausanne, who led the research in collaboration with colleague Christophe Moser. “We demonstrate that this is possible for fibers up to 1 kilometer long” he added, calling the work “an important milestone.”

Deciphering the blur

Optical fibers transmit information with light. Multimode fibers have much greater information-carrying capacity than single-mode fibers. Their many channels — known as spatial modes because they have different spatial shapes — can transmit different streams of information simultaneously.

While multimode fibers are well suited for carrying light-based signals, transmitting images is problematic. Light from the image travels through all of the channels and what comes out the other end is a pattern of speckles that the human eye cannot decode.

To tackle this problem, Psaltis and his team turned to a deep neural network, a type of machine learning algorithm that functions much the way the brain does. Deep neural networks can give computers the ability to identify objects in photographs and help improve speech recognition systems. Input is processed through several layers of artificial neurons, each of which performs a small calculation and passes the result on to the next layer. The machine learns to identify the input by recognizing the patterns of output associated with it.

“If we think about the origin of neural networks, which is our very own brain, the process is simple,” explains Eirini Kakkava, a doctoral student working on the project. “When a person stares at an object, neurons in the brain are activated, indicating recognition of a familiar object. Our brain can do this because it gets trained throughout our life with images or signals of the same category of objects, which changes the strength of the connections between the neurons.” To train an artificial neural network, researchers follow essentially the same process, teaching the network to recognize certain images (in this case, handwritten digits) until it is able to recognize images in the same category as the training images that it has not seen before.

Learning by the numbers

To train their system, the researchers turned to a database containing 20,000 samples of handwritten numbers, 0 through 9. They selected 16,000 to be used as training data, and kept aside 2,000 to validate the training and another 2,000 for testing the validated system. They used a laser to illuminate each digit and sent the light beam through an optical fiber, which had approximately 4,500 channels, to a camera on the far end. A computer measured how the intensity of the output light varied across the captured image, and they collected a series of examples for each digit.

Although the speckle patterns collected for each digit looked the same to the human eye, the neural network was able to discern differences and recognize patterns of intensity associated with each digit. Testing with the set-aside images showed that the algorithm achieved 97.6 percent accuracy for images transmitted through a 0.1 meter long fiber and 90 percent accuracy with a 1 kilometer length of fiber.

A simpler method

Navid Borhani, a research-team member, says this machine learning approach is much simpler than other methods to reconstruct images passed through optical fibers, which require making a holographic measurement of the output. The neural network was also able to cope with distortions caused by environmental disturbances to the fiber such as temperature fluctuations or movements caused by air currents that can add noise to the image — a situation that gets worse with fiber length.

“The remarkable ability of deep neural networks to retrieve information transmitted through multimode fibers is expected to benefit medical procedures like endoscopy and communications applications,” Psaltis said. Telecommunication signals often have to travel through many kilometers of fiber and can suffer distortions, which this method could correct. Doctors could use ultrathin fiber probes to collect images of the tracts and arteries inside the human body without needing complex holographic recorders or worrying about movement. “Slight movements because of breathing or circulation can distort the images transmitted through a multimode fiber,” Psaltis said. The deep neural networks are a promising solution for dealing with that noise.

Psaltis and his team plan to try the technique with biological samples, to see if that works as well as reading handwritten numbers. They hope to conduct a series of studies using different categories of images to explore the possibilities and limits of their technique.

Story Source:

Materials provided by The Optical Society. Note: Content may be edited for style and length.

How a computer learns to dribble: Practice, practice, practice

Basketball players need lots of practice before they master the dribble, and it turns out that’s true for computer-animated players as well. By using deep reinforcement learning, players in video basketball games can glean insights from motion capture data to sharpen their dribbling skills.

Researchers at Carnegie Mellon University and DeepMotion Inc., a California company that develops smart avatars, have for the first time developed a physics-based, real-time method for controlling animated characters that can learn dribbling skills from experience. In this case, the system learns from motion capture of the movements performed by people dribbling basketballs.

This trial-and-error learning process is time consuming, requiring millions of trials, but the results are arm movements that are closely coordinated with physically plausible ball movement. Players learn to dribble between their legs, dribble behind their backs and do crossover moves, as well as how to transition from one skill to another.

“Once the skills are learned, new motions can be simulated much faster than real-time,” said Jessica Hodgins, Carnegie Mellon professor of computer science and robotics.

Hodgins and Libin Liu, chief scientist at DeepMotion, will present the method at SIGGRAPH 2018, the Conference on Computer Graphics and Interactive Techniques, Aug. 12-18, in Vancouver.

“This research opens the door to simulating sports with skilled virtual avatars,” said Liu, the report’s first author. “The technology can be applied beyond sport simulation to create more interactive characters for gaming, animation, motion analysis, and in the future, robotics.”

Motion capture data already add realism to state-of-the-art video games. But these games also include disconcerting artifacts, Liu noted, such as balls that follow impossible trajectories or that seem to stick to a player’s hand.

A physics-based method has the potential to create more realistic games, but getting the subtle details right is difficult. That’s especially so for dribbling a basketball because player contact with the ball is brief and finger position is critical. Some details, such as the way a ball may continue spinning briefly when it makes light contact with the player’s hands, are tough to reproduce. And once the ball is released, the player has to anticipate when and where the ball will return.

Liu and Hodgins opted to use deep reinforcement learning to enable the model to pick up these important details. Artificial intelligence programs have used this form of deep learning to figure out a variety of video games and the AlphaGo program famously employed it to master the board game Go.

The motion capture data used as input was of people doing things such as rotating the ball around the waist, dribbling while running and dribbling in place both with the right hand and while switching hands. This capture data did not include the ball movement, which Liu explained is difficult to record accurately. Instead, they used trajectory optimization to calculate the ball’s most likely paths for a given hand motion.

The program learned the skills in two stages — first it mastered locomotion and then learned how to control the arms and hands and, through them, the motion of the ball. This decoupled approach is sufficient for actions such as dribbling or perhaps juggling, where the interaction between the character and the object doesn’t have an effect on the character’s balance. Further work is required to address sports, such as soccer, where balance is tightly coupled with game maneuvers, Liu said.

Story Source:

Materials provided by Carnegie Mellon University. Note: Content may be edited for style and length.

Soft multi-functional robots get really small … and spider-shaped

Roboticists are envisioning a future in which soft, animal-inspired robots could be safely deployed in difficult-to-access natural and human-made environments, such as in delicate surgical procedures in the human body, or in spaces too small and unpredictable to be conquered with rigid robots or too dangerous for humans to work with rigid robots in. Centimeter-sized soft robots have been created, but thus far it has not been possible to fabricate multifunctional flexible robots that can move and operate at smaller size scales.

A team of researchers at Harvard’s Wyss Institute for Biologically Inspired Engineering, Harvard John A. Paulson School of Engineering and Applied Sciences (SEAS), and Boston University now has overcome this challenge by developing an integrated fabrication process that enables the design of soft robots on the millimeter scale with micrometer-scale features. To demonstrate the capabilities of their new technology, they created a robotic soft spider — inspired by the millimeter-sized colorful Australian peacock spider — from a single elastic material with body-shaping, motion, and color features. The study is published in Advanced Materials.

“The smallest soft robotic systems still tend to be very simple, with usually only one degree of freedom, which means that they can only actuate one particular change in shape or type of movement,” said Sheila Russo, Ph.D., co-author of the study. Russo helped initiate the project as a Postdoctoral Fellow in Robert Wood’s group at the Wyss Institute and SEAS and now is Assistant Professor at Boston University. “By developing a new hybrid technology that merges three different fabrication techniques, we created a soft robotic spider made only of silicone rubber with 18 degrees of freedom, encompassing changes in structure, motion, and color, and with tiny features in the micrometer range.”

Wood, Ph.D., is a Core Faculty member and co-leader of the Bioinspired Soft Robotics platform at the Wyss Institute and the Charles River Professor of Engineering and Applied Sciences at SEAS. “In the realm of soft robotic devices, this new fabrication approach can pave the way towards achieving similar levels of complexity and functionality on this small scale as those exhibited by their rigid counterparts. In the future, it can also help us emulate and understand structure-function relationships in small animals much better than rigid robots can,” he said.

In their Microfluidic Origami for Reconfigurable Pneumatic/Hydrolic (MORPH) devices, the team first used a soft lithography technique to generate 12 layers of an elastic silicone that together constitute the soft spider’s material basis. Each layer is precisely cut out of a mold with a laser micromachining technique, and then bonded to the one below to create the rough 3D structure of the soft spider.

Key to transforming this intermediate structure into the final design is a pre-conceived network of hollow microfluidic channels that is integrated into individual layers. With a third technique known as injection-induced self-folding, pressurized one set of these integrated microfluidic channels with a curable resin from the outside. This induces individual layers, and with them also their neighboring layers, to locally bend into their final configuration, which is fixed in space when the resin hardens. This way, for example, the soft spider’s swollen abdomen and downward-curved legs become permanent features.

“We can precisely control this origami-like folding process by varying the thickness and relative consistency of the silicone material adjacent to the channels across different layers or by laser-cutting at different distances from the channels. During pressurization, the channels then function as actuators that induce a permanent structural change,” said first and corresponding author Tommaso Ranzani, Ph.D., who started the study as a Postdoctoral Fellow in Wood’s group and now also is Assistant Professor at Boston University.

The remaining set of integrated microfluidic channels were used as additional actuators to colorize the eyes and simulate the abdominal color patterns of the peacock species by flowing colored fluids; and to induce walking-like movements in the leg structures. “This first MORPH system was fabricated in a single, monolithic process that can be performed in few days and easily iterated in design optimization efforts,” said Ranzani.

“The MORPH approach could open up the field of soft robotics to researchers who are more focused on medical applications where the smaller sizes and flexibility of these robots could enable an entirely new approach to endoscopy and microsurgery,” said Wyss Institute Founding Director Donald Ingber, M.D., Ph.D., who is also the Judah Folkman Professor of Vascular Biology at HMS and the Vascular Biology Program at Boston Children’s Hospital, as well as Professor of Bioengineering at SEAS.

A kernel of promise in popcorn-powered robots

Cornell University researchers have discovered how to power simple robots with a novel substance that, when heated, can expand more than 10 times in size, change its viscosity by a factor of 10 and transition from regular to highly irregular granules with surprising force.

You can also eat it with a little butter and salt.

“Popcorn-Driven Robotic Actuators,” a recent paper co-authored by Steven Ceron, mechanical engineering doctoral student, and Kirstin H. Petersen, assistant professor of electrical and computer engineering, examines how popcorn’s unique qualities can power inexpensive robotic devices that grip, expand or change rigidity.

“The goal of our lab is to try to make very minimalistic robots which, when deployed in high numbers, can still accomplish great things,” said Petersen, who runs Cornell’s Collective Embodied Intelligence Lab. “Simple robots are cheap and less prone to failures and wear, so we can have many operating autonomously over a long time. So we are always looking for new and innovative ideas that will permit us to have more functionalities for less, and popcorn is one of those.”

The study is the first to consider powering robots with popcorn, which is inexpensive, readily available, biodegradable and of course, edible. Since kernels can expand rapidly, exerting force and motion when heated, they could potentially power miniature jumping robots. Edible devices could be ingested for medical procedures. The mix of hard, unpopped granules and lighter popped corn could replace fluids in soft robots without the need for air pumps or compressors.

“Pumps and compressors tend to be more expensive, and they add a lot of weight and expense to your robot,” said Ceron, the paper’s lead author. “With popcorn, in some of the demonstrations that we showed, you just need to apply voltage to get the kernels to pop, so it would take all the bulky and expensive parts out of the robots.”

Since kernels can’t shrink once they’ve popped, a popcorn-powered mechanism can generally be used only once, though multiple uses are conceivable because popped kernels can dissolve in water, Ceron said.

The paper was presented at the IEEE International Conference on Robotics and Automation. Petersen said she hopes it inspires researchers to explore the possibilities of other nontraditional materials.

“Robotics is really good at embracing new ideas, and we can be super creative about what we use to generate multifunctional properties,” she said. “In the end we come up with very simple solutions to fairly complex problems. We don’t always have to look for high-tech solutions. Sometimes the answer is right in front of us.”

The work was supported by the Cornell Engineering Learning Initiative, the Cornell Electrical and Computer Engineering Early Career Award and the Cornell Sloan Fellowship.

Story Source:

Materials provided by Cornell University. Original written by Melanie Lefkowitz. Note: Content may be edited for style and length.

‘Blurred face’ news anonymity gets an artificial intelligence spin

A research team from SFU’s School of Interactive Arts and Technology (SIAT) has come up with a way to replace the use of ‘blurring’ faces in news reports when anonymity is needed. The team’s method uses artificial intelligence (AI) techniques that aim to improve visuals while amplifying emotions tied to the story.

SIAT professors Steve DiPaola and Kate Hennessy, together with Taylor Owen from UBC’s journalism school, received a Google/Knight Foundation grant to carry out the research. They presented the work to international journalists at a Journalism 360 demo event honoring grantees in New York on July 24, and the next day at a full conference held across the street from the New York Times headquarters.

“Our goal is to create a working technique that would be much better at conveying emotional and knowledge information than current anonymization techniques,” says DiPaola, a pioneer in AI/VR facial recognition processes.

Based on its research, the team has created an updated pixelating technique using an AI “painting” approach to anonymization.

“When artists paint a portrait, they try to convey the subject’s outer and inner resemblance,” says DiPaola, who heads SFU’s Interactive Visualization Lab. “With our AI, which learns from more than 1,000 years of artistic technique, we have taught the system to lower the outer resemblance and keep as high as possible the subject’s inner resemblance — in other words, what they are conveying and how they are feeling.

“Our system uses five levels of AI processing to simulate a smart painter, using art abstraction to repaint the video as if an artist was painting every frame. The result is more engaging, especially since not everyone listens to stories — so the art component becomes more relevant.”

The system doesn’t change the pixels of the video frames as Adobe-like systems can, but instead produces a painting-style result of every frame, DiPaola notes. “It’s actually an open and dynamic process that allows levels of control throughout. We eventually want to the subject or producer to be able to customize the final result based on their needs.”

DiPaola says the tool’s effectiveness at anonymizing while retaining a strong degree of emotional connection or resonance should result in better final product for anonymized video, especially in 360 or VR. The team plans to put the technology to work in a variety of projects.

While at the conference DiPaola’s team garnished interest from several major media companies looking to explore new media approaches, including the New York Times, Washington Post and the BBC.

Story Source:

Materials provided by Simon Fraser University. Note: Content may be edited for style and length.

AI senses people’s pose through walls

X-ray vision has long seemed like a far-fetched sci-fi fantasy, but over the last decade a team led by Professor Dina Katabi from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) has continually gotten us closer to seeing through walls.

Their latest project, “RF-Pose,” uses artificial intelligence (AI) to teach wireless devices to sense people’s postures and movement, even from the other side of a wall.

The researchers use a neural network to analyze radio signals that bounce off people’s bodies, and can then create a dynamic stick figure that walks, stops, sits and moves its limbs as the person performs those actions.

The team says that the system could be used to monitor diseases like Parkinson’s and multiple sclerosis (MS), providing a better understanding of disease progression and allowing doctors to adjust medications accordingly. It could also help elderly people live more independently, while providing the added security of monitoring for falls, injuries and changes in activity patterns.

(All data the team collected has subjects’ consent and is anonymized and encrypted to protect user privacy. For future real-world applications, the team plans to implement a “consent mechanism” in which the person who installs the device is cued to do a specific set of movements in order for it to begin to monitor the environment.)

The team is currently working with doctors to explore multiple applications in healthcare.

“We’ve seen that monitoring patients’ walking speed and ability to do basic activities on their own gives healthcare providers a window into their lives that they didn’t have before, which could be meaningful for a whole range of diseases,” says Katabi, who co-wrote a new paper about the project. “A key advantage of our approach is that patients do not have to wear sensors or remember to charge their devices.”

Besides health-care, the team says that RF-Pose could also be used for new classes of video games where players move around the house, or even in search-and-rescue missions to help locate survivors.

“Just like how cellphones and Wi-Fi routers have become essential parts of today’s households, I believe that wireless technologies like these will help power the homes of the future,” says Katabi, who co-wrote the new paper with PhD student and lead author Mingmin Zhao, MIT professor Antonio Torralba, postdoc Mohammad Abu Alsheikh, graduate student Tianhong Li and PhD students Yonglong Tian and Hang Zhao. They will present it later this month at the Conference on Computer Vision and Pattern Recognition (CVPR) in Salt Lake City, Utah.

One challenge the researchers had to address is that most neural networks are trained using data labeled by hand. A neural network trained to identify cats, for example, requires that people look at a big dataset of images and label each one as either “cat” or “not cat.” Radio signals, meanwhile, can’t be easily labeled by humans.

To address this, the researchers collected examples using both their wireless device and a camera. They gathered thousands of images of people doing activities like walking, talking, sitting, opening doors and waiting for elevators.

They then used these images from the camera to extract the stick figures, which they showed to the neural network along with the corresponding radio signal. This combination of examples enabled the system to learn the association between the radio signal and the stick figures of the people in the scene.

Post-training, RF-Pose was able to estimate a person’s posture and movements without cameras, using only the wireless reflections that bounce off people’s bodies.

Since cameras can’t see through walls, the network was never explicitly trained on data from the other side of a wall — which is what made it particularly surprising to the MIT team that the network could generalize its knowledge to be able to handle through-wall movement.

“If you think of the computer vision system as the teacher, this is a truly fascinating example of the student outperforming the teacher,” says Torralba.

Besides sensing movement, the authors also showed that they could use wireless signals to accurately identify somebody 83 percent of the time out of a line-up of 100 individuals. This ability could be particularly useful for the application of search-and-rescue operations, when it may be helpful to know the identity of specific people.

For this paper, the model outputs a 2-D stick figure, but the team is also working to create 3-D representations that would be able to reflect even smaller micromovements. For example, it might be able to see if an older person’s hands are shaking regularly enough that they may want to get a check-up.

“By using this combination of visual data and AI to see through walls, we can enable better scene understanding and smarter environments to live safer, more productive lives,” says Zhao.

Use artificial intelligence to identify, count, describe wild animals

A new paper in the Proceedings of the National Academy of Sciences (PNAS) reports how a cutting-edge artificial intelligence technique called deep learning can automatically identify, count and describe animals in their natural habitats.

Photographs that are automatically collected by motion-sensor cameras can then be automatically described by deep neural networks. The result is a system that can automate animal identification for up to 99.3 percent of images while still performing at the same 96.6 percent accuracy rate of crowdsourced teams of human volunteers.

“This technology lets us accurately, unobtrusively and inexpensively collect wildlife data, which could help catalyze the transformation of many fields of ecology, wildlife biology, zoology, conservation biology and animal behavior into ‘big data’ sciences. This will dramatically improve our ability to both study and conserve wildlife and precious ecosystems,” says Jeff Clune, the senior author of the paper. He is the Harris Associate Professor at the University of Wyoming and a senior research manager at Uber’s Artificial Intelligence Labs.

The paper was written by Clune; his Ph.D. student Mohammad Sadegh Norouzzadeh; his former Ph.D. student Anh Nguyen (now at Auburn University); Margaret Kosmala (Harvard University); Ali Swanson (University of Oxford); and Meredith Palmer and Craig Packer (both from the University of Minnesota).

Deep neural networks are a form of computational intelligence loosely inspired by how animal brains see and understand the world. They require vast amounts of training data to work well, and the data must be accurately labeled (e.g., each image being correctly tagged with which species of animal is present, how many there are, etc.).

This study obtained the necessary data from Snapshot Serengeti, a citizen science project on the http://www.zooniverse.org platform. Snapshot Serengeti has deployed a large number of “camera traps” (motion-sensor cameras) in Tanzania that collect millions of images of animals in their natural habitat, such as lions, leopards, cheetahs and elephants. The information in these photographs is only useful once it has been converted into text and numbers. For years, the best method for extracting such information was to ask crowdsourced teams of human volunteers to label each image manually. The study published today harnessed 3.2 million labeled images produced in this manner by more than 50,000 human volunteers over several years.

“When I told Jeff Clune we had 3.2 million labeled images, he stopped in his tracks,” says Packer, who heads the Snapshot Serengeti project. “We wanted to test whether we could use machine learning to automate the work of human volunteers. Our citizen scientists have done phenomenal work, but we needed to speed up the process to handle ever greater amounts of data. The deep learning algorithm is amazing and far surpassed my expectations. This is a game changer for wildlife ecology.”

Swanson, who founded Snapshot Serengeti, adds: “There are hundreds of camera-trap projects in the world, and very few of them are able to recruit large armies of human volunteers to extract their data. That means that much of the knowledge in these important data sets remains untapped. Although projects are increasingly turning to citizen science for image classification, we’re starting to see it take longer and longer to label each batch of images as the demand for volunteers grows. We believe deep learning will be key in alleviating the bottleneck for camera-trap projects: the effort of converting images into usable data.”

“Not only does the artificial intelligence system tell you which of 48 different species of animal is present, but it also tells you how many there are and what they are doing. It will tell you if they are eating, sleeping, if babies are present, etc.,” adds Kosmala, another Snapshot Serengeti leader. “We estimate that the deep learning technology pipeline we describe would save more than eight years of human labeling effort for each additional 3 million images. That is a lot of valuable volunteer time that can be redeployed to help other projects.”

First-author Sadegh Norouzzadeh points out that “Deep learning is still improving rapidly, and we expect that its performance will only get better in the coming years. Here, we wanted to demonstrate the value of the technology to the wildlife ecology community, but we expect that as more people research how to improve deep learning for this application and publish their datasets, the sky’s the limit. It is exciting to think of all the different ways this technology can help with our important scientific and conservation missions.”

Story Source:

Materials provided by University of Wyoming. Note: Content may be edited for style and length.

Future robots need no motors

To develop micro- and biomimetic-robots, artificial muscles and medical devices, actuating materials that can reversibly change their volume under various stimuli are researched in the past thirty years to replace traditional bulky and heavy actuators including motors and pneumatic actuators.

A mechanical engineering team led by Professor Alfonso Ngan Hing-wan, Chair Professor in Materials Science and Engineering, and Kingboard Professor in Materials Engineering, Faculty of Engineering, the University of Hong Kong (HKU) published an article in Science Robotics on 30 May 2018 (EST) that introduces a novel actuating material — nickel hydroxide-oxyhydroxide — that can be powered by visible (Vis) light, electricity, and other stimuli. The material actuation can be instantaneously triggered by Vis light to produce a fast deformation and exert a force equivalent to 3000 times of its own weight. The material cost of a typical actuator is as low as HKD 4 per cm2 and can be easily fabricated within three hours.

Among various stimuli, light-induced actuating materials are highly desirable because they enable wireless operation of robots. However, very few light driven materials are available in the past, and their material and production costs are high, which hinder their development in actual applications such as artificial muscles for robotics and human assist device, and minimally invasive surgical and diagnostic tools.

Developing actuating materials was identified as the top of the 10 challenges in “The grand challenges of Science Robotics.” Research in actuating materials can radically change the concept of robots which are now mainly motor-driven. Therefore, materials that can be actuated by wireless stimuli including a change in temperature, humidity, magnetic fields and light is one of the main research focus in recent years. In particular, a material that can be actuated by Vis light and produces strong, quick and stable actuation has never been achieved. The novel actuating material system — nickel hydroxide-oxyhydroxide that can be actuated by Vis light at relatively low intensity to produce high stress and speed comparable to mammalian skeletal muscles has been developed in this research initiated by engineers in HKU.

In addition to its Vis light actuation properties, this novel material system can also be actuated by electricity, enabling it to be integrated into the present well-developed robotics technology. It is also responsive to heat and humidity changes so that they might potentially be applied in autonomous machines that harness the tiny energy change in the environment. Because the major component is nickel, the material cost is low.

The fabrication only involves electrodeposition which is a simple process, and the time required for the fabrication is around three hours, therefore the material can be easily scaled up and manufactured in industry.

The newly invented nickel hydroxide-oxyhydroxide responses to light almost instantaneously and produces a force corresponding to about 3000 times of its own weight.

When integrated into a well-designed structure, a “mini arm” made by two hinges of actuating materials can easily lift an object 50 times of its weight. Similarly, by utilizing a light blocker, a mini walking-bot in which only the “front leg” bent and straighten alternatively and therefore moves under illumination was made so that it can walk towards the light source. These demonstrate that future applications in micro-robotics including rescue robots are possible.

The evidences above revealed that this nickel hydroxide-oxyhydroxide actuating material can have different applications in the future, including rescue robots or other mini-robots. The intrinsic actuating properties of the materials obtained from our research show that by scaling up the fabrication, artificial muscles comparable to that of mammalian skeletal muscles can be achieved, and applying it in robotics, human assist device and medical devices are possible.

From a scientific point of view, this nickel hydroxide-oxyhydroxide actuating material is the world’s first material system that can be actuated directly by Vis light and electricity without any additional fabrication procedures. This also opens up a new research field on light-induced actuating behaviour for this material type (hydroxide-oxyhydroxides) because it has never been reported before.

The research team members are all from the Department of Mechanical Engineering at HKU Faculty of Engineering, led by Professor Alfonso Ngan’s group in collaboration with Dr Li Wen-di’s group on light actuation experiment and Dr Feng Shien-ping’s group on electrodeposition experiment. The research has been published in the journal Science Robotics on 30 May 2018 with a title of “Light-stimulated actuators based on nickel hydroxide-oxyhydroxide.” The first author of this paper is Dr Kwan Kin-wa who is currently a post-doctoral fellow in Prof. Ngan’s group.

The corresponding author is Prof. Ngan. The complete author list is as below: K-W. Kwan, S-J. Li, N-Y. Hau, W-D. Li, S-P. Feng, A.H.W. Ngan. This research is funded by the Research Grants Council, Hong Kong.

An artificial nerve system gives prosthetic devices and robots a sense of touch

Stanford and Seoul National University researchers have developed an artificial sensory nerve system that can activate the twitch reflex in a cockroach and identify letters in the Braille alphabet.

The work, reported May 31 in Science, is a step toward creating artificial skin for prosthetic limbs, to restore sensation to amputees and, perhaps, one day give robots some type of reflex capability.

“We take skin for granted but it’s a complex sensing, signaling and decision-making system,” said Zhenan Bao, a professor of chemical engineering and one of the senior authors. “This artificial sensory nerve system is a step toward making skin-like sensory neural networks for all sorts of applications.”

Building blocks

This milestone is part of Bao’s quest to mimic how skin can stretch, repair itself and, most remarkably, act like a smart sensory network that knows not only how to transmit pleasant sensations to the brain, but also when to order the muscles to react reflexively to make prompt decisions.

The new Science paper describes how the researchers constructed an artificial sensory nerve circuit that could be embedded in a future skin-like covering for neuro-prosthetic devices and soft robotics. This rudimentary artificial nerve circuit integrates three previously described components.

The first is a touch sensor that can detect even minuscule forces. This sensor sends signals through the second component — a flexible electronic neuron. The touch sensor and electronic neuron are improved versions of inventions previously reported by the Bao lab.

Sensory signals from these components stimulate the third component, an artificial synaptic transistor modeled after human synapses. The synaptic transistor is the brainchild of Tae-Woo Lee of Seoul National University, who spent his sabbatical year in Bao’s Stanford lab to initiate the collaborative work.

“Biological synapses can relay signals, and also store information to make simple decisions,” said Lee, who was a second senior author on the paper. “The synaptic transistor performs these functions in the artificial nerve circuit.”

Lee used a knee reflex as an example of how more-advanced artificial nerve circuits might one day be part of an artificial skin that would give prosthetic devices or robots both senses and reflexes.

In humans, when a sudden tap causes the knee muscles to stretch, certain sensors in those muscles send an impulse through a neuron. The neuron in turn sends a series of signals to the relevant synapses. The synaptic network recognizes the pattern of the sudden stretch and emits two signals simultaneously, one causing the knee muscles to contract reflexively and a second, less urgent signal to register the sensation in the brain.

Making it work

The new work has a long way to go before it reaches that level of complexity. But in the Science paper, the group describes how the electronic neuron delivered signals to the synaptic transistor, which was engineered in such a way that it learned to recognize and react to sensory inputs based on the intensity and frequency of low-power signals, just like a biological synapse.

The group members tested the ability of the system to both generate reflexes and sense touch.

In one test they hooked up their artificial nerve to a cockroach leg and applied tiny increments of pressure to their touch sensor. The electronic neuron converted the sensor signal into digital signals and relayed them through the synaptic transistor, causing the leg to twitch more or less vigorously as the pressure on the touch sensor increased or decreased.

They also showed that the artificial nerve could detect various touch sensations. In one experiment the artificial nerve was able to differentiate Braille letters. In another, they rolled a cylinder over the sensor in different directions and accurately detected the direction of the motion.

Bao’s graduate students Yeongin Kim and Alex Chortos, plus Wentao Xu, a researcher from Lee’s own lab, were also central to integrating the components into the functional artificial sensory nervous system.

The researchers say artificial nerve technology remains in its infancy. For instance, creating artificial skin coverings for prosthetic devices will require new devices to detect heat and other sensations, the ability to embed them into flexible circuits and then a way to interface all of this to the brain.

The group also hopes to create low-power, artificial sensor nets to cover robots, the idea being to make them more agile by providing some of the same feedback that humans derive from their skin.

Story Source:

Materials provided by Stanford University. Original written by Tom Abate. Note: Content may be edited for style and length.

AI researchers design ‘privacy filter’ for your photos

Each time you upload a photo or video to a social media platform, its facial recognition systems learn a little more about you. These algorithms ingest data about who you are, your location and people you know — and they’re constantly improving.

As concerns over privacy and data security on social networks grow, U of T Engineering researchers led by Professor Parham Aarabi and graduate student Avishek Bose have created an algorithm to dynamically disrupt facial recognition systems.

“Personal privacy is a real issue as facial recognition becomes better and better,” says Aarabi. “This is one way in which beneficial anti-facial-recognition systems can combat that ability.”

Their solution leverages a deep learning technique called adversarial training, which pits two artificial intelligence algorithms against each other. Aarabi and Bose designed a set of two neural networks: the first working to identify faces, and the second working to disrupt the facial recognition task of the first. The two are constantly battling and learning from each other, setting up an ongoing AI arms race.

The result is an Instagram-like filter that can be applied to photos to protect privacy. Their algorithm alters very specific pixels in the image, making changes that are almost imperceptible to the human eye.

“The disruptive AI can ‘attack’ what the neural net for the face detection is looking for,” says Bose. “If the detection AI is looking for the corner of the eyes, for example, it adjusts the corner of the eyes so they’re less noticeable. It creates very subtle disturbances in the photo, but to the detector they’re significant enough to fool the system.”

Aarabi and Bose tested their system on the 300-W face dataset, an industry standard pool of more than 600 faces that includes a wide range of ethnicities, lighting conditions and environments. They showed that their system could reduce the proportion of faces that were originally detectable from nearly 100 per cent down to 0.5 per cent.

“The key here was to train the two neural networks against each other — with one creating an increasingly robust facial detection system, and the other creating an ever stronger tool to disable facial detection,” says Bose, the lead author on the project. The team’s study will be published and presented at the 2018 IEEE International Workshop on Multimedia Signal Processing later this summer.

In addition to disabling facial recognition, the new technology also disrupts image-based search, feature identification, emotion and ethnicity estimation, and all other face-based attributes that could be extracted automatically.

Next, the team hopes to make the privacy filter publicly available, either via an app or a website.

“Ten years ago these algorithms would have to be human defined, but now neural nets learn by themselves — you don’t need to supply them anything except training data,” says Aarabi. “In the end they can do some really amazing things. It’s a fascinating time in the field, there’s enormous potential.”

Aerial robot that can morph in flight

Marking a world first, researchers from the Étienne Jules Marey Institute of Movement Sciences (CNRS / Aix-Marseille Université) have drawn inspiration from birds to design an aerial robot capable of altering its profile during flight. To reduce its wingspan and navigate through tight spaces, it can reorient its arms, which are equipped with propellers that let it fly like a helicopter. The scientists’ work is the subject of an article published in Soft Robotics (May 30, 2018). It paves the way for a new generation of large robots that can move through narrow passages, making them ideal for exploration as well as search and rescue missions.

Birds and winged insects have the remarkable ability to maneuver quickly during flight to clear obstacles. Such extreme agility is necessary to navigate through cramped spaces and crowded environments, like forests. There are already miniature flying machines that can roll, pitch, or otherwise alter their flight attitude to pass through small apertures. But birds illustrate another strategy that is just as effective for flying through bottlenecks. They can quickly fold their wings during high-speed flight, reducing their imposing span, to easily negotiate the challenging paths before them.[1]

Deployment of aerial robots in constricted and cluttered areas for search and rescue, exploratory, or mapping operations will become more and more commonplace. They will need to be able to circumnavigate many obstacles and travel through fairly tight passages to complete their missions. Accordingly, researchers from the Étienne Jules Marey Institute of Movement Sciences (CNRS / Aix-Marseille Université) have designed a flying robot that can reduce its wingspan in flight to move through a small opening, without intensive steering that would consume too much energy and require a robotic platform featuring a low-inertia (light and small robot).[2]

Dubbed Quad-Morphing, the new robot has two rotating arms each equipped with two propellers for helicopter-like flight. A system of elastic and rigid wires allows the robot to change the orientation of its arms in flight so that they are either perpendicular or parallel to its central axis. It adopts the parallel position, halving its wingspan, to traverse a narrow stretch and then switches back to perpendicular position to stabilize its flight, all while flying at a speed of 9 km/h, which is pretty fast for an aerial robot.

At present, it is the precision of the Quad-Morphing autopilot mechanism that determines the robot’s agility. The autopilot activates arm reorientation when the robot nears a tight passage, as determined by a 3D localization system used at the institute.[3] The researchers have also equipped the robot with a miniature camera that can take 120 pictures per second. In the future, this will allow Quad-Morphing to independently assess the size of the gap before it and fold its wings accordingly if necessary. Flight testing with the new camera will begin this month.

Notes:

[1] Such impressive behavior has been observed among budgerigars and goshawks flying at speeds above 14 km/h.

[2] Flying robots have typical transversal speed of 4-5 km/h in indoor conditions.

[3] The studies were conducted at the AVM flying machine arena, built with the financial support of the French Equipex Robotex program. The arena has 17 cameras for recording movement.

Story Source:

Materials provided by CNRS. Note: Content may be edited for style and length.

Cometh the cyborg: Improved integration of living muscles into robots

The new field of biohybrid robotics involves the use of living tissue within robots, rather than just metal and plastic. Muscle is one potential key component of such robots, providing the driving force for movement and function. However, in efforts to integrate living muscle into these machines, there have been problems with the force these muscles can exert and the amount of time before they start to shrink and lose their function.

Now, in a study reported in the journal Science Robotics, researchers at The University of Tokyo Institute of Industrial Science have overcome these problems by developing a new method that progresses from individual muscle precursor cells, to muscle-cell-filled sheets, and then to fully functioning skeletal muscle tissues. They incorporated these muscles into a biohybrid robot as antagonistic pairs mimicking those in the body to achieve remarkable robot movement and continued muscle function for over a week.

The team first constructed a robot skeleton on which to install the pair of functioning muscles. This included a rotatable joint, anchors where the muscles could attach, and electrodes to provide the stimulus to induce muscle contraction. For the living muscle part of the robot, rather than extract and use a muscle that had fully formed in the body, the team built one from scratch. For this, they used hydrogel sheets containing muscle precursor cells called myoblasts, holes to attach these sheets to the robot skeleton anchors, and stripes to encourage the muscle fibers to form in an aligned manner.

“Once we had built the muscles, we successfully used them as antagonistic pairs in the robot, with one contracting and the other expanding, just like in the body,” study corresponding author Shoji Takeuchi says. “The fact that they were exerting opposing forces on each other stopped them shrinking and deteriorating, like in previous studies.”

The team also tested the robots in different applications, including having one pick up and place a ring, and having two robots work in unison to pick up a square frame. The results showed that the robots could perform these tasks well, with activation of the muscles leading to flexing of a finger-like protuberance at the end of the robot by around 90°.

“Our findings show that, using this antagonistic arrangement of muscles, these robots can mimic the actions of a human finger,” lead author Yuya Morimoto says. “If we can combine more of these muscles into a single device, we should be able to reproduce the complex muscular interplay that allow hands, arms, and other parts of the body to function.”

Story Source:

Materials provided by Institute of Industrial Science, The University of Tokyo. Note: Content may be edited for style and length.

Activity simulator could eventually teach robots tasks like making coffee or setting the table

For many people, household chores are a dreaded, inescapable part of life that we often put off or do with little care — but what if a robot maid could help lighten the load?

Recently, computer scientists have been working on teaching machines to do a wider range of tasks around the house. In a new paper spearheaded by MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and the University of Toronto, researchers demonstrate “VirtualHome,” a system that can simulate detailed household tasks and then have artificial “agents” execute them, opening up the possibility of one day teaching robots to do such tasks.

The team trained the system using nearly 3,000 programs of various activities, which are further broken down into subtasks for the computer to understand. A simple task like “making coffee,” for example, would also include the step “grabbing a cup.” The researchers demonstrated VirtualHome in a 3-D world inspired by the Sims video game.

The team’s AI agent can execute 1,000 of these interactions in the Sims-style world, with eight different scenes including a living room, kitchen, dining room, bedroom, and home office.

“Describing actions as computer programs has the advantage of providing clear and unambiguous descriptions of all the steps needed to complete a task,” says PhD student Xavier Puig, who was lead author on the paper. “These programs can instruct a robot or a virtual character, and can also be used as a representation for complex tasks with simpler actions.”

The project was co-developed by CSAIL and the University of Toronto alongside researchers from McGill University and the University of Ljubljana. It will be presented at the Computer Vision and Pattern Recognition (CVPR) conference, which takes place this month in Salt Lake City.

How it works

Unlike humans, robots need more explicit instructions to complete easy tasks — they can’t just infer and reason with ease.

For example, one might tell a human to “switch on the TV and watch it from the sofa.” Here, actions like “grab the remote control” and “sit/lie on sofa” have been omitted, since they’re part of the commonsense knowledge that humans have.

To better demonstrate these kinds of tasks to robots, the descriptions for actions needed to be much more detailed. To do so, the team first collected verbal descriptions of household activities, and then translated them into simple code. A program like this might include steps like: walk to the television, switch on the television, walk to the sofa, sit on the sofa, and watch television.

Once the programs were created, the team fed them to the VirtualHome 3-D simulator to be turned into videos. Then, a virtual agent would execute the tasks defined by the programs, whether it was watching television, placing a pot on the stove, or turning a toaster on and off.

The end result is not just a system for training robots to do chores, but also a large database of household tasks described using natural language. Companies like Amazon that are working to develop Alexa-like robotic systems at home could eventually use data like this to train their models to do more complex tasks.

The team’s model successfully demonstrated that, their agents could learn to reconstruct a program, and therefore perform a task, given either a description: “pour milk into glass,” or a video demonstration of the activity.

“This line of work could facilitate true robotic personal assistants in the future,” says Qiao Wang, a research assistant in arts, media, and engineering at Arizona State University. “Instead of each task programmed by the manufacturer, the robot can learn tasks just by listening to or watching the specific person it accompanies. This allows the robot to do tasks in a personalized way, or even some day invoke an emotional connection as a result of this personalized learning process.”

In the future, the team hopes to train the robots using actual videos instead of Sims-style simulation videos, which would enable a robot to learn simply by watching a YouTube video. The team is also working on implementing a reward-learning system in which the agent gets positive feedback when it does tasks correctly.

“You can imagine a setting where robots are assisting with chores at home and can eventually anticipate personalized wants and needs, or impending action,” says Puig. “This could be especially helpful as an assistive technology for the elderly, or those who may have limited mobility.”

Face recognition experts perform better with AI as partner

Experts at recognizing faces often play a crucial role in criminal cases. A photo from a security camera can mean prison or freedom for a defendant — and testimony from highly trained forensic face examiners informs the jury whether that image actually depicts the accused. Just how good are facial recognition experts? Would artificial intelligence help?

A study appearing today in the Proceedings of the National Academy of Sciences has brought answers. In work that combines forensic science with psychology and computer vision research, a team of scientists from the National Institute of Standards and Technology (NIST) and three universities has tested the accuracy of professional face identifiers, providing at least one revelation that surprised even the researchers: Trained human beings perform best with a computer as a partner, not another person.

“This is the first study to measure face identification accuracy for professional forensic facial examiners, working under circumstances that apply in real-world casework,” said NIST electronic engineer P. Jonathon Phillips. “Our deeper goal was to find better ways to increase the accuracy of forensic facial comparisons.”

The team’s effort began in response to a 2009 report by the National Research Council, “Strengthening Forensic Science in the United States: A Path Forward,” which underscored the need to measure the accuracy of forensic examiner decisions.

The NIST study is the most comprehensive examination to date of face identification performance across a large, varied group of people. The study also examines the best technology as well, comparing the accuracy of state-of-the-art face recognition algorithms to human experts.

Their result from this classic confrontation of human versus machine? Neither gets the best results alone. Maximum accuracy was achieved with a collaboration between the two.

“Societies rely on the expertise and training of professional forensic facial examiners, because their judgments are thought to be best,” said co-author Alice O’Toole, a professor of cognitive science at the University of Texas at Dallas. “However, we learned that to get the most highly accurate face identification, we should combine the strengths of humans and machines.”

The results arrive at a timely moment in the development of facial recognition technology, which has been advancing for decades, but has only very recently attained competence approaching that of top-performing humans.

“If we had done this study three years ago, the best computer algorithm’s performance would have been comparable to an average untrained student,” Phillips said. “Nowadays, state-of-the-art algorithms perform as well as a highly trained professional.”

The study itself involved a total of 184 participants, a large number for an experiment of this type. Eighty-seven were trained professional facial examiners, while 13 were “super recognizers,” a term implying exceptional natural ability. The remaining 84 — the control groups — included 53 fingerprint examiners and 31 undergraduate students, none of whom had training in facial comparisons.

For the test, the participants received 20 pairs of face images and rated the likelihood of each pair being the same person on a seven-point scale. The research team intentionally selected extremely challenging pairs, using images taken with limited control of illumination, expression and appearance. They then tested four of the latest computerized facial recognition algorithms, all developed between 2015 and 2017, using the same image pairs.

Three of the algorithms were developed by Rama Chellappa, a professor of electrical and computer engineering at the University of Maryland, and his team, who contributed to the study. The algorithms were trained to work in general face recognition situations and were applied without modification to the image sets.

One of the findings was unsurprising but significant to the justice system: The trained professionals did significantly better than the untrained control groups. This result established the superior ability of the trained examiners, thus providing for the first time a scientific basis for their testimony in court.

The algorithms also acquitted themselves well, as might be expected from the steady improvement in algorithm performance over the past few years.

What raised the team’s collective eyebrows regarded the performance of multiple examiners. The team discovered that combining the opinions of multiple forensic face examiners did not bring the most accurate results.

“Our data show that the best results come from a single facial examiner working with a single top-performing algorithm,” Phillips said. “While combining two human examiners does improve accuracy, it’s not as good as combining one examiner and the best algorithm.”

Combining examiners and AI is not currently used in real-world forensic casework. While this study did not explicitly test this fusion of examiners and AI in such an operational forensic environment, results provide an roadmap for improving the accuracy of face identification in future systems.

While the three-year project has revealed that humans and algorithms use different approaches to compare faces, it poses a tantalizing question to other scientists: Just what is the underlying distinction between the human and the algorithmic approach?

“If combining decisions from two sources increases accuracy, then this method demonstrates the existence of different strategies,” Phillips said. “But it does not explain how the strategies are different.”

The research team also included psychologist David White from Australia’s University of New South Wales.

An elastic fiber filled with electrodes set to revolutionize smart clothes

It’s a whole new way of thinking about sensors. The tiny fibers developed at EPFL are made of elastomer and can incorporate materials like electrodes and nanocomposite polymers. The fibers can detect even the slightest pressure and strain and can withstand deformation of close to 500% before recovering their initial shape. All that makes them perfect for applications in smart clothing and prostheses, and for creating artificial nerves for robots.

The fibers were developed at EPFL’s Laboratory of Photonic Materials and Fiber Devices (FIMAP), headed by Fabien Sorin at the School of Engineering. The scientists came up with a fast and easy method for embedding different kinds of microstructures in super-elastic fibers. For instance, by adding electrodes at strategic locations, they turned the fibers into ultra-sensitive sensors. What’s more, their method can be used to produce hundreds of meters of fiber in a short amount of time. Their research has just been published in Advanced Materials.

Heat, then stretch

To make their fibers, the scientists used a thermal drawing process, which is the standard process for optical-fiber manufacturing. They started by creating a macroscopic preform with the various fiber components arranged in a carefully designed 3D pattern. They then heated the preform and stretched it out, like melted plastic, to make fibers of a few hundreds microns in diameter. And while this process stretched out the pattern of components lengthwise, it also contracted it crosswise, meaning the components’ relative positions stayed the same. The end result was a set of fibers with an extremely complicated microarchitecture and advanced properties.

Until now, thermal drawing could be used to make only rigid fibers. But Sorin and his team used it to make elastic fibers. With the help of a new criterion for selecting materials, they were able to identify some thermoplastic elastomers that have a high viscosity when heated. After the fibers are drawn, they can be stretched and deformed but they always return to their original shape.

Rigid materials like nanocomposite polymers, metals and thermoplastics can be introduced into the fibers, as well as liquid metals that can be easily deformed. “For instance, we can add three strings of electrodes at the top of the fibers and one at the bottom. Different electrodes will come into contact depending on how the pressure is applied to the fibers. This will cause the electrodes to transmit a signal, which can then be read to determine exactly what type of stress the fiber is exposed to — such as compression or shear stress, for example,” says Sorin.

Artificial nerves for robots

Working in association with Professor Dr. Oliver Brock (Robotics and Biology Laboratory, Technical University of Berlin), the scientists integrated their fibers into robotic fingers as artificial nerves. Whenever the fingers touch something, electrodes in the fibers transmit information about the robot’s tactile interaction with its environment. The research team also tested adding their fibers to large-mesh clothing to detect compression and stretching. “Our technology could be used to develop a touch keyboard that’s integrated directly into clothing, for instance” says Sorin.

The researchers see many other potential applications. Especially since the thermal drawing process can be easily tweaked for large-scale production. This is a real plus for the manufacturing sector. The textile sector has already expressed interest in the new technology, and patents have been filed.

Story Source:

Materials provided by Ecole Polytechnique Fédérale de Lausanne. Note: Content may be edited for style and length.