SPORTS  |  SOCCER

One step at a time

The finalized exoskeleton designed by Dr. Miguel Nicolelis and his Walk Again Project.
The finalized exoskeleton designed by Dr. Miguel Nicolelis and his Walk Again Project.

For many, walking is like breathing—and in a soccer-centric country like Brazil, dribbling a ball does not come far behind. Walking is done subconsciously everyday, putting one foot in front of the other in hopes of reaching new destinations and new experiences. But what if the experience was walking?

Dr. Miguel Nicolelis, co-director of the Duke Center for Neuroengineering, asked a similar question nearly 30 years ago and found the answer in an exoskeleton on the pitch of the World Cup opening ceremony.

The answer he found on the pitch was not hailed by the masses, rather it was criticized by a disappointed media looking for more than what they saw. But the purpose of Nicoelis' demonstration was not to cater to the media, rather it was meant to plant a seed in the mind of viewers of what could be accomplished. And in this, he succeeded.

Half a world away: from Sao Paulo to Durham

To find the seed of the exoskeleton, one must go back to 1988, the year Nicolelis left São Paulo. The then-27-year-old had just earned his Ph.D. from the University of São Paulo and begun his Ph.D. fellowship at Hahnemann University in Philadelphia. It was there that he first began toying with the revolutionary idea of a brain-machine interface—which would, in essence, allow a human to control a robotic piece of equipment based on signals sent by the brain.

But this was far easier said than done. The technology available was simply not prepared to handle such tall tasks.

Fast forward five years, and technology had begun to catch up, and he was able to monitor the neurons in the area of a rat’s brain that was directly linked with the intake of sensory information to dictate physical movement. It was not close to being ready for human trials, but it was a start. A year later, in 1994, Nicolelis began his work at Duke, where the brain-machine interface idea took off.

“The idea of brain-machine interface that I had came from the lab at Duke and it has evolved in rodents, primates and even humans,” Nicolelis said. “This whole thing has developed at Duke at the Center for Neuroengineering. I’m really proud of all these years, because in the beginning, very few people believed that this was worth it, even at Duke. There was a lot of resistance in the beginning.”

In 2002, Nicolelis jointly published a paper with John Chapin in Scientific American called “Controlling Robots with the Mind” that detailed the experiments the duo had been working on, starting with a rodent, moving on to a small owl monkey, Belle, and concluding with a macaque monkey, Aurora. The final trials consisted of having Aurora play a simple video game with a joystick. Throughout the tests, they would turn off the joystick in an attempt to teach Aurora to play the game simply by thinking about what she wanted to do. According to the paper, after a few weeks of weaning, Aurora “didn’t even bother to move her hand.” The cursor on the screen was controlled entirely by her thoughts.

There was only one more step to take, and in 2004, Nicolelis and his lab took it. Nicolelis replaced Belle and Aurora with a human and replaced the microwires with a cap that monitored the brain waves, marking the first human trial for brain-machine interface in the lab.

“It has come a long way,” he said.

Back to Sao Paulo

The human trials were still a far cry from the exoskeleton that would be displayed at the World Cup. The link between these two steps in the process came not from Nicolelis' research done in America, but rather his Brazilian roots.

“I initiated the contact with the Brazilian government,” Nicolelis said. “When they mentioned, way back in 2009, that they wanted to do something in the opening ceremony to highlight a new kind of country, a country that wanted to invest in science, education, technology and innovation, I went and talked to people at the Ministry of Science of Technology and Ministry of Education about doing this idea, of doing a demo at the Opening Ceremony, and they loved it.”

Nicolelis met with Dilma Rousseff, who would succeed Luiz Inácio Lula da Silva as President of Brazil in 2010. Rousseff supported the decision to have the Brazilian government sponsor Nicolelis’ team and their research by way of the government organization Financiadora de Estudos e Projetos, or FINEP, which is dedicated to the funding of scientific research in Brazil.

But a project as hefty as this needed more than Nicolelis's dreams and Brazil’s money. It needed the top scientists in their select fields to come together and create an exoskeleton that would not fail on the biggest of stages. Luckily for Nicolelis, he has made quite a few friends over the years.

“I recruited friends [from] all over the world in each one of the domains we needed to work on,” Nicolelis said. “Robotics, computer science [and] rehabilitation medicine. We put together a team, an international team.”

This international team became the Walk Again consortium, a group of 150 scientists and engineers from around the world that joined forces in pursuit of Nicolelis’ goal. Together, the group descended upon Nicolelis’ Sao Paulo lab to begin work on the project.

The team did not consist solely of scientists with tenure, dozens of publications and world-famous last names. Nicolelis also brought along fellow Blue Devil Anthony Lin, Trinity '13.

Lin, a three-time NCAA Championship qualifying fencer in his time at Duke, had worked in Nicolelis’ lab all four years of his undergraduate career. In his junior year, he was granted a research fellowship by Duke to do independent research in Nicolelis' lab which he used to develop a brain-machine interface working with primates and small robots.

Prior to his graduation, he was awarded the NCAA Postgraduate Scholarship along with Olympic diver Abby Johnston, providing him with $7,500 to put toward medical school. But instead of using the money right away and heading to back to school, Lin decided to forgo medical school for a year after Nicolelis noticed the work he had been doing in his lab and made him an offer he couldn't refuse.

"My work eventually attracted the attention of Dr. Nicolelis," Lin said. "After graduation, he invited me to join his team in Brazil to work with the Walk Again Project."

Dr. Miguel Nicolelis' lab has put the work of numerous graduate and undergraduate students towards creating the exoskeleton.

Lin is not the only Duke student to have left his mark on the Walk Again Project. Although Lin was unique in being able to spend a year working with the group in Brazil, Nicolelis is quick to point out that the lab work of many students made the dream a reality.

“The graduate students from the medical school and the [biomedical engineering] department that participate in all the stages of this work, they’re all Duke students,” he said. “And we had had a lot of undergraduates work in the lab over the years. Close to 60 or 70 undergrads over these 20 years. There’s a huge Duke lineage around brain-machine interface.”

Learning to walk again

Once in Brazil, the team selected eight patients aged 20-35, all paralyzed at least from the waist down. None of the patients were aware of the ultimate goal of the project, only that Sao Paulo’s poster-boy scientist had selected them to be part of his experiment. And as soon as the patients were in place, the real work began.

“We had three levels of training,” Nicolelis said. “It started in a visual reality environment, where we simulate a soccer stadium and they have these visual reality goggles, so they saw themselves as being a soccer player on the pitch. Then they get this tactile feedback on the sleeves of a shirt that every one of them had to wear when they were in the visual reality environment.”

The virtual reality the patients were in allowed them to feel as though they were walking on the pitch of the World Cup.

With each step the patients’ avatar would take in the virtual reality, a vibration would occur on the skin of their arm via the shirt, giving them the sensation of a reaction to the movement. Within minutes, Nicolelis reported that the patients’ brains generated the illusion that the feeling in their arm was actually coming from their legs—a phantom sensation.

The paralyzed individuals felt as though their legs were moving, even though, physically, they could not feel it. The first step was complete.

“That was when we knew we were on the right direction,” Nicolelis said.

The second step was more physical, as the patients were placed in a machine brought into the lab that would hold them up and move their legs so as to simulate the sensation of walking. The same brain patterns were monitored, only this time, they were not in the quiet confines of a virtual reality. The team would simulate the distractions of the World Cup by presenting the patients with loud noises and flashing lights. Their focus was just as imperative to the process as the functionality of the suit. If the brain signals were not focused enough, the suit would be left motionless on the pitch and the project would be a dud.

Once they had completed this part of the training, the project moved on to the third and final stage—wearing the exoskeleton. Nicolelis and his team had the patients perform a series of motor tasks in the suit while receiving feedback from the legs.

Nicolelis and his team have been working on the exoskeleton for years prior to the World Cup.

Although the scientists had been preparing years for this, there was no down time. Every day was an opportunity to collect data, work on the bugs in the suit and train the patients. And the team did not waste a single opportunity.

“We were there a minimum of 12 hours, usually 14 to 16 a day every day,” he said. “No weekends, no holidays, no nothing. It was the most demanding seven months that I remember in my scientific career.”

The Kick

And like that, seven months of preparation and five years of planning came to a head. The World Cup was upon the team and the training was completed. The only issue was that the demonstration was not for a team of eight. It was for one. And that one was Juliano Pinto.

Pinto’s selection to be the one to kick off the World Cup in the exoskeleton was not a decision made lightly. It was the culmination of months of research and data gathering. At the conclusion of the training, Pinto’s brain signals registered as the highest and most concentrated of the group, and the 29-year old Brazilian—who had not walked since a 2006 car accident—was granted the opportunity. But his chance in the exoskeleton did not come without its own set of surprises.

Of the group of eight, Pinto had the highest spinal cord lesion and suffered from a high-collapsed lateral region that paralyzed him from the mid-chest down. Nicolelis and his team were shocked that the person with the highest level of paralysis was their strongest performer. But results were results. And if there was anything Nicolelis had gotten used to throughout his trials, it was nature surprising him.

But the results should not have been that surprising. Pinto’s high performance stemmed from a sports background, something Nicolelis said helped him in the long run.

“He’s a pretty cool guy. He’s a para-athlete. He’s actually trying to go to the Paralympics, so he has a very good mindset for outdoor competitions,” Nicolelis said. “He understands what it is to perform in a stadium full of people. And it came down to the fact that he was the best guy in terms of modulating his EEG. his brain signals.”

Pinto was not notified that he had been selected until four days prior to the opening ceremony. From that point on, it was a whirlwind of events—rehearsals, tests, retests and sleepless nights comprised the four days leading up to the opening ceremony and opening match between Brazil and Croatia.

On the day of the event, Nicolelis’ team was brought to the pitch, carefully escorting Pinto and the machine along with them. A red carpet was laid out, and with FIFA officials flanking him on both sides, the ball was placed before him. A referee—or a man dressed as a referee—raised his arm and blew his whistle to give Pinto the green light.

The moment Pinto’s foot touched the ball, kicking it off the end of the carpet, a collective deep breath of relief was exhaled by Nicolelis and his team as Pinto thrust his arms in the air in victory. The demo was a success, and more than that, a dream was realized.


“I was born in Brazil 53 years ago. Everybody in Brazil was born to play football. That is the dream of every kid and every kid dreams about being on that World Cup stage,” Nicolelis said. “Certainly I couldn’t make it as a football player, but at the end of the demo, I was thinking about that. Because I was there, on the pitch at the opening game of the World Cup. In Brazil. The land of football. I couldn't escape my attention that somehow, I had made it to the World Cup. From a very unusual door but nevertheless I was there.”

Unusual or not, the moment belonged to Pinto, Nicolelis, Lin and the numerous other scientists who had put in 16-hour days for this special, once-in-a-lifetime moment.

Unfortunately, the kick was not deemed as monumental by the viewing world.

Six seconds

“Disappointing.”

Articles from news sites such as Slate and Deadspin have since referred to the kick as such. But the backlash is founded on limited information. Prior to the World Cup, Nicolelis penned a piece for Scientific American in 2012 in which he outlined his plans to have the exoskeleton come onto the field and kick the ball "Brazillian style", building the excitement for this unseen marvel.

"[Nicolelis' exoskeleton] is the kind of technological achievement that gets me excited about writing," Will Oremus, author of the Slate blog article said. "You tend, when someone's working on something important and experimental, to percieve the thing to be straight out of science fiction. Often, the reality of progress on scientific projects is more mundane."

Those disappointed in the presentation of the Nicolelis-led team’s accomplishment echoed Oremus' statements, claiming the spectacle Nicolelis promised was not what was delivered. What they did not know is that the presentation's length and scope was out of his control. The Brazillian style kick, the saunter, the bravado—all this and more is what Nicolelis, his team and the Brazilian government had planned on delivering on one of the world's biggest stages.

Then FIFA stepped in.

“We only came to talk to FIFA a year and a half ago,” Nicolelis said. “FIFA is very difficult to deal with.... It was only this past March that we started getting the direction of what they will allow us to do. It was at that point that we discovered they would only give like 29 seconds at the ceremony."

But the team had planned for so much more.

"The Brazilian government wanted much more time," Nicolelis said. "And [FIFA] told us that they would only have time for the ceremonial, what they called the opening kickoff of the World Cup. Since we saw that there was no alternative, we decided to go for that.”

Along with the orders from the organization, FIFA also forced Nicolelis’ camp to sign a nondisclosure agreement, meaning they could not speak of the terms FIFA had forced on them with anybody outside the program until the kick was completed at the opening ceremony.

FIFA declined to comment on the demonstration.

For all of the unknowing fans and media, what had been built up to be a giant spectacle seemed rushed. Even those watching at home caught a mere glimpse of the process due to a miscommunication on the part of the cameramen, who only managed to capture six seconds worth of the 29-second demonstration.

But six seconds was all it took. For six seconds, millions of people worldwide witnessed a paralyzed man do the impossible. For six seconds, science ruled one of the biggest sporting events in the world. And for Nicolelis, his team and Pinto, six seconds was enough.

"As far as the demonstration goes, I believe it was a success," Lin said. "It is unfortunate that the media coverage was so small, but regardless, the event opened up the world's imagination of what was possible at the intersection of neuroscience and medicine. For the first time, “the World Cup” meant much more than soccer."

Although much has been made of Nicolelis’ research and the science behind it—and rightfully so—what happened on the pitch following the kick was not carefully planned. It couldn’t be. For Pinto, it was the moment of a lifetime. No amount of FIFA restrictions or cameraman mishaps could take away the moment every Brazilian dreams of as a child. Brazil boasts a 23-man roster headlined by names like Neymar, Hulk and Fred and for 29 seconds June 12, Pinto joined these national sports deities on the pitch.

“The greatest thing Juliano [Pinto] told me—I mean he was crying like a baby, we were all crying with him—he said to me, ‘You know, after several years I could feel again what it was to kick a ball.’ I think that’s what I’ll take for the rest of my life. That feeling. Hearing him telling me that not only he made it to the World Cup, but he felt the kick,” Nicolelis said. “This is the bottom line: there was a paraplegic man standing on the edge of the World Cup pitch kicking and feeling the kick. Everybody broke when he said that. I saw people that I’ve known for 20 years—very harsh, tough cookies—crying like babies.”

The next step

The project got its fair share of love too, namely from a man far more powerful in relation to Nicolelis than the disappointed media. The director of the National Institute of Health, Dr. Francis Collins, made the trip down to Brazil in order to see Nicolelis' groundbreaking work firsthand, as the NIH funds his lab at Duke. Collins was thoroughly impressed, going so far as to blog about the prototype the day of the demo.

Nicolelis has the attention—and funding—of Francis Collins and the National Institute of Health.

And with the NIH's continued backing, Nicolelis—the 2010 NIH Pioneer Award recipient— has also secured funding from the Brazilian government for the next two-and-a-half years.

"The repercussion was tremendous here," he said. "We’re talking to the Brazilian government to see where we can go from here on a larger scale. But also we have all the work that we do at Duke that we are going to continue."

Although there is surely much more work to be done in the future, for Nicolelis, the 20-year Duke scientist and 53-year old Brazilian soccer fan, the moment will never fade.

“I have spent half my life in the United States and half in Brazil,” Nicolelis said. “Somehow the two halves came together on the pitch of the World Cup. That’s not too bad. As we say [in America], not too shabby. Not too shabby.”

Editor's note: This article was updated July 11, 2014.

Discussion

Share and discuss “One step at a time” on social media.