Tag Archives: Cleveland Clinic

Better motor control for prosthetic hands (the illusion of feeling) and a discussion of superprostheses and reality

I have two bits about prosthetics, one which focuses on how most of us think of them and another about science fiction fantasies.

Better motor control

This new technology comes via a collaboration between the University of Alberta, the University of New Brunswick (UNB) and Ohio’s Cleveland Clinic, from a March 18, 2018 article by Nicole Ireland for the Canadian Broadcasting Corporation’s (CBC) news online,

Rob Anderson was fighting wildfires in Alberta when the helicopter he was in crashed into the side of a mountain. He survived, but lost his left arm and left leg.

More than 10 years after that accident, Anderson, now 39, says prosthetic limb technology has come a long way, and he feels fortunate to be using “top of the line stuff” to help him function as normally as possible. In fact, he continues to work for the Alberta government’s wildfire fighting service.

His powered prosthetic hand can do basic functions like opening and closing, but he doesn’t feel connected to it — and has limited ability to perform more intricate movements with it, such as shaking hands or holding a glass.

Anderson, who lives in Grande Prairie, Alta., compares its function to “doing things with a long pair of pliers.”

“There’s a disconnect between what you’re physically touching and what your body is doing,” he told CBC News.

Anderson is one of four Canadian participants in a study that suggests there’s a way to change that. …

Six people, all of whom had arm amputations from below the elbow or higher, took part in the research. It found that strategically placed vibrating “robots” made them “feel” the movements of their prosthetic hands, allowing them to grasp and grip objects with much more control and accuracy.

All of the participants had all previously undergone a specialized surgical procedure called “targeted re-innervation.” The nerves that had connected to their hands before they were amputated were rewired to link instead to muscles (including the biceps and triceps) in their remaining upper arms and in their chests.

For the study, researchers placed the robotic devices on the skin over those re-innervated muscles and vibrated them as the participants opened, closed, grasped or pinched with their prosthetic hands.

While the vibration was turned on, the participants “felt” their artificial hands moving and could adjust their grip based on the sensation. …

I have an April 24, 2017 posting about a tetraplegic patient who had a number of electrodes implanted in his arms and hands linked to a brain-machine interface and which allowed him to move his hands and arms; the implants were later removed. It is a different problem with a correspondingly different technological solution but there does seem to be increased interest in implanting sensors and electrodes into the human body to increase mobility and/or sensation.

Anderson describes how it ‘feels,

“It was kind of surreal,” Anderson said. “I could visually see the hand go out, I would touch something, I would squeeze it and my phantom hand felt like it was being closed and squeezing on something and it was sending the message back to my brain.

“It was a very strange sensation to actually be able to feel that feedback because I hadn’t in 10 years.”

The feeling of movement in the prosthetic hand is an illusion, the researchers say, since the vibration is actually happening to a muscle elsewhere in the body. But the sensation appeared to have a real effect on the participants.

“They were able to control their grasp function and how much they were opening the hand, to the same degree that someone with an intact hand would,” said study co-author Dr. Jacqueline Hebert, an associate professor in the Faculty of Rehabilitation Medicine at the University of Alberta.

Although the researchers are encouraged by the study findings, they acknowledge that there was a small number of participants, who all had access to the specialized re-innervation surgery to redirect the nerves from their amputated hands to other parts of their body.

The next step, they say, is to see if they can also simulate the feeling of movement in a broader range of people who have had other types of amputations, including legs, and have not had the re-innervation surgery.

Here’s a March 15, 2018  CBC New Brunswick radio interview about the work,

This is a bit longer than most of the embedded audio pieces that I have here but it’s worth it. Sadly, I can’t identify the interviewer who did a very good job with Jon Sensinger, associate director of UNB’s Institute of Biomedical Engineering. One more thing, I noticed that the interviewer made no mention of the University of Alberta in her introduction or in the subsequent interview. I gather regionalism reigns supreme everywhere in Canada. Or, maybe she and Sensinger just forgot. It happens when you’re excited. Also, there were US institutions in Ohio and Virginia that participated in this work.

Here’s a link to and a citation for the team’s paper,

Illusory movement perception improves motor control for prosthetic hands by Paul D. Marasco, Jacqueline S. Hebert, Jon W. Sensinger, Courtney E. Shell, Jonathon S. Schofield, Zachary C. Thumser, Raviraj Nataraj, Dylan T. Beckler, Michael R. Dawson, Dan H. Blustein, Satinder Gill, Brett D. Mensh, Rafael Granja-Vazquez, Madeline D. Newcomb, Jason P. Carey, and Beth M. Orzell. Science Translational Medicine 14 Mar 2018: Vol. 10, Issue 432, eaao6990 DOI: 10.1126/scitranslmed.aao6990

This paper is open access.

Superprostheses and our science fiction future

A March 20, 2018 news item on phys.org features an essay on about superprostheses and/or assistive devices,

Assistive devices may soon allow people to perform virtually superhuman feats. According to Robert Riener, however, there are more pressing goals than developing superhumans.

What had until recently been described as a futuristic vision has become a reality: the first self-declared “cyborgs” have had chips implanted in their bodies so that they can open doors and make cashless payments. The latest robotic hand prostheses succeed in performing all kinds of grips and tasks requiring dexterity. Parathletes fitted with running and spring prostheses compete – and win – against the best, non-impaired athletes. Then there are robotic pets and talking humanoid robots adding a bit of excitement to nursing homes.

Some media are even predicting that these high-tech creations will bring about forms of physiological augmentation overshadowing humans’ physical capabilities in ways never seen before. For instance, hearing aids are eventually expected to offer the ultimate in hearing; retinal implants will enable vision with a sharpness rivalling that of any eagle; motorised exoskeletons will transform soldiers into tireless fighting machines.

Visions of the future: the video game Deus Ex: Human Revolution highlights the emergence of physiological augmentation. (Visualisations: Square Enix) Courtesy: ETH Zurich

Professor Robert Riener uses the image above to illustrate the notion of superprosthese in his March 20, 2018 essay on the ETH Zurich website,

All of these prophecies notwithstanding, our robotic transformation into superheroes will not be happening in the immediate future and can still be filed under Hollywood hero myths. Compared to the technology available today, our bodies are a true marvel whose complexity and performance allows us to perform an extremely wide spectrum of tasks. Hundreds of efficient muscles, thousands of independently operating motor units along with millions of sensory receptors and billions of nerve cells allow us to perform delicate and detailed tasks with tweezers or lift heavy loads. Added to this, our musculoskeletal system is highly adaptable, can partly repair itself and requires only minimal amounts of energy in the form of relatively small amounts of food consumed.

Machines will not be able to match this any time soon. Today’s assistive devices are still laboratory experiments or niche products designed for very specific tasks. Markus Rehm, an athlete with a disability, does not use his innovative spring prosthesis to go for walks or drive a car. Nor can today’s conventional arm prostheses help a person tie their shoes or button up their shirt. Lifting devices used for nursing care are not suitable for helping with personal hygiene tasks or in psychotherapy. And robotic pets quickly lose their charm the moment their batteries die.

Solving real problems

There is no denying that advances continue to be made. Since the scientific and industrial revolutions, we have become dependent on relentless progress and growth, and we can no longer separate today’s world from this development. There are, however, more pressing issues to be solved than creating superhumans.

On the one hand, engineers need to dedicate their efforts to solving the real problems of patients, the elderly and people with disabilities. Better technical solutions are needed to help them lead normal lives and assist them in their work. We need motorised prostheses that also work in the rain and wheelchairs that can manoeuvre even with snow on the ground. Talking robotic nurses also need to be understood by hard-of-hearing pensioners as well as offer simple and dependable interactivity. Their batteries need to last at least one full day to be recharged overnight.

In addition, financial resources need to be available so that all people have access to the latest technologies, such as a high-quality household prosthesis for the family man, an extra prosthesis for the avid athlete or a prosthesis for the pensioner. [emphasis mine]

Breaking down barriers

What is just as important as the ongoing development of prostheses and assistive devices is the ability to minimise or eliminate physical barriers. Where there are no stairs, there is no need for elaborate special solutions like stair lifts or stairclimbing wheelchairs – or, presumably, fully motorised exoskeletons.

Efforts also need to be made to transform the way society thinks about people with disabilities. More acknowledgement of the day-to-day challenges facing patients with disabilities is needed, which requires that people be confronted with the topic of disability when they are still children. Such projects must be promoted at home and in schools so that living with impairments can also attain a state of normality and all people can partake in society. It is therefore also necessary to break down mental barriers.

The road to a virtually superhuman existence is still far and long. Anyone reading this text will not live to see it. In the meantime, the task at hand is to tackle the mundane challenges in order to simplify people’s daily lives in ways that do not require technology, that allow people to be active participants and improve their quality of life – instead of wasting our time getting caught up in cyborg euphoria and digital mania.

I’m struck by Riener’s reference to financial resources and access. Sensinger mentions financial resources in his CBC radio interview although his concern is with convincing funders that prostheses that mimic ‘feeling’ are needed.

I’m also struck by Riener’s discussion about nontechnological solutions for including people with all kinds of abilities and disabilities.

There was no grand plan for combining these two news bits; I just thought they were interesting together.

‘Smart’ fabric that’s bony

Researchers at Australia’s University of New South of Wales (UNSW) have devised a means of ‘weaving’ a material that mimics *bone tissue, periosteum according to a Jan. 11, 2017 news item on ScienceDaily,

For the first time, UNSW [University of New South Wales] biomedical engineers have woven a ‘smart’ fabric that mimics the sophisticated and complex properties of one nature’s ingenious materials, the bone tissue periosteum.

Having achieved proof of concept, the researchers are now ready to produce fabric prototypes for a range of advanced functional materials that could transform the medical, safety and transport sectors. Patents for the innovation are pending in Australia, the United States and Europe.

Potential future applications range from protective suits that stiffen under high impact for skiers, racing-car drivers and astronauts, through to ‘intelligent’ compression bandages for deep-vein thrombosis that respond to the wearer’s movement and safer steel-belt radial tyres.

A Jan. 11, 2017 UNSW press release on EurekAlert, which originated the news item, expands on the theme,

Many animal and plant tissues exhibit ‘smart’ and adaptive properties. One such material is the periosteum, a soft tissue sleeve that envelops most bony surfaces in the body. The complex arrangement of collagen, elastin and other structural proteins gives periosteum amazing resilience and provides bones with added strength under high impact loads.

Until now, a lack of scalable ‘bottom-up’ approaches by researchers has stymied their ability to use smart tissues to create advanced functional materials.

UNSW’s Paul Trainor Chair of Biomedical Engineering, Professor Melissa Knothe Tate, said her team had for the first time mapped the complex tissue architectures of the periosteum, visualised them in 3D on a computer, scaled up the key components and produced prototypes using weaving loom technology.

“The result is a series of textile swatch prototypes that mimic periosteum’s smart stress-strain properties. We have also demonstrated the feasibility of using this technique to test other fibres to produce a whole range of new textiles,” Professor Knothe Tate said.

In order to understand the functional capacity of the periosteum, the team used an incredibly high fidelity imaging system to investigate and map its architecture.

“We then tested the feasibility of rendering periosteum’s natural tissue weaves using computer-aided design software,” Professor Knothe Tate said.

The computer modelling allowed the researchers to scale up nature’s architectural patterns to weave periosteum-inspired, multidimensional fabrics using a state-of-the-art computer-controlled jacquard loom. The loom is known as the original rudimentary computer, first unveiled in 1801.

“The challenge with using collagen and elastin is their fibres, that are too small to fit into the loom. So we used elastic material that mimics elastin and silk that mimics collagen,” Professor Knothe Tate said.

In a first test of the scaled-up tissue weaving concept, a series of textile swatch prototypes were woven, using specific combinations of collagen and elastin in a twill pattern designed to mirror periosteum’s weave. Mechanical testing of the swatches showed they exhibited similar properties found in periosteum’s natural collagen and elastin weave.

First author and biomedical engineering PhD candidate, Joanna Ng, said the technique had significant implications for the development of next-generation advanced materials and mechanically functional textiles.

While the materials produced by the jacquard loom have potential manufacturing applications – one tyremaker believes a titanium weave could spawn a new generation of thinner, stronger and safer steel-belt radials – the UNSW team is ultimately focused on the machine’s human potential.

“Our longer term goal is to weave biological tissues – essentially human body parts – in the lab to replace and repair our failing joints that reflect the biology, architecture and mechanical properties of the periosteum,” Ms Ng said.

An NHMRC development grant received in November [2016] will allow the team to take its research to the next phase. The researchers will work with the Cleveland Clinic and the University of Sydney’s Professor Tony Weiss to develop and commercialise prototype bone implants for pre-clinical research, using the ‘smart’ technology, within three years.

In searching for more information about this work, I found a Winter 2015 article (PDF; pp. 8-11) by Amy Coopes and Steve Offner for UNSW Magazine about Knothe Tate and her work (Note: In Australia, winter would be what we in the Northern Hemisphere consider summer),

Tucked away in a small room in UNSW’s Graduate School of Biomedical Engineering sits a 19th century–era weaver’s wooden loom. Operated by punch cards and hooks, the machine was the first rudimentary computer when it was unveiled in 1801. While on the surface it looks like a standard Jacquard loom, it has been enhanced with motherboards integrated into each of the loom’s five hook modules and connected to a computer. This state-of-the-art technology means complex algorithms control each of the 5,000 feed-in fibres with incredible precision.

That capacity means the loom can weave with an extraordinary variety of substances, from glass and titanium to rayon and silk, a development that has attracted industry attention around the world.

The interest lies in the natural advantage woven materials have over other manufactured substances. Instead of manipulating material to create new shades or hues as in traditional weaving, the fabrics’ mechanical properties can be modulated, to be stiff at one end, for example, and more flexible at the other.

“Instead of a pattern of colours we get a pattern of mechanical properties,” says Melissa Knothe Tate, UNSW’s Paul Trainor Chair of Biomedical Engineering. “Think of a rope; it’s uniquely good in tension and in bending. Weaving is naturally strong in that way.”


The interface of mechanics and physiology is the focus of Knothe Tate’s work. In March [2015], she travelled to the United States to present another aspect of her work at a meeting of the international Orthopedic Research Society in Las Vegas. That project – which has been dubbed “Google Maps for the body” – explores the interaction between cells and their environment in osteoporosis and other degenerative musculoskeletal conditions such as osteoarthritis.

Using previously top-secret semiconductor technology developed by optics giant Zeiss, and the same approach used by Google Maps to locate users with pinpoint accuracy, Knothe Tate and her team have created “zoomable” anatomical maps from the scale of a human joint down to a single cell.

She has also spearheaded a groundbreaking partnership that includes the Cleveland Clinic, and Brown and Stanford universities to help crunch terabytes of data gathered from human hip studies – all processed with the Google technology. Analysis that once took 25 years can now be done in a matter of weeks, bringing researchers ever closer to a set of laws that govern biological behaviour. [p. 9]

I gather she was recruited from the US to work at the University of New South Wales and this article was to highlight why they recruited her and to promote the university’s biomedical engineering department, which she chairs.

Getting back to 2017, here’s a link to and citation for the paper,

Scale-up of nature’s tissue weaving algorithms to engineer advanced functional materials by Joanna L. Ng, Lillian E. Knothe, Renee M. Whan, Ulf Knothe & Melissa L. Knothe Tate. Scientific Reports 7, Article number: 40396 (2017) doi:10.1038/srep40396 Published online: 11 January 2017

This paper is open access.

One final comment, that’s a lot of people (three out of five) with the last name Knothe in the author’s list for the paper.

*’the bone tissue’ changed to ‘bone tissue’ on July 17,2017.

Virtual Reality (VR) becomes educational (at Case Western Reserve University and Miami Children’s Hospital)

I have two virtual reality news bits the most recent concerning Case Western Reserve University (CWRU; located in Cleveland, Ohio) and Microsoft’s HoloLens in an April 29, 2015 CWRU press release (also on EurekAlert), Note: Some of this academic press release reads like marketing collateral,

Case Western Reserve University Radiology Professor Mark Griswold knew his world had changed the moment he first used a prototype of Microsoft’s HoloLens headset. Two months later, one of the university’s medical students illustrated exactly why.

“There’s the aortic valve,” Satyam Ghodasara exclaimed as he used Microsoft’s device to examine a holographic heart. “Now I understand.”

Today, Griswold told tens of thousands of people how HoloLens can transform learning across countless subjects, including those as complex as the human body. Speaking to an in-person and online audience at Microsoft’s annual Build conference, he highlighted disciplines as disparate as art history and engineering–but started with a holographic heart. In traditional anatomy, after all, students like Ghodasara cut into cadavers to understand the body’s intricacies.

With HoloLens, Griswold explained, “you see it truly in 3D. You can take parts in and out. You can turn it around. You can see the blood pumping–the entire system.”

In other words, technology not only can match existing educational methods–it can actually improve upon them. Which, in many ways, is why Cleveland Clinic CEO Toby Cosgrove contacted then-Microsoft executive Craig Mundie in 2013, after the hospital and university first agreed to partner on a new education building.

“We launched this collaboration to prepare students for a health care future that is still being imagined,” Cleveland Clinic CEO Delos “Toby” Cosgrove said of what has become a 485,000-square-foot Health Education Campus project. “By combining a state-of-the-art structure, pioneering technology, and cutting-edge teaching techniques, we will provide them the innovative education required to lead in this new era.”

As Cosgrove, Case Western Reserve President Barbara R. Snyder and other academic leaders engaged more extensively with Microsoft, the more potential everyone saw.

“For more than a century, our medical school has been renowned for inventing and reinventing approaches to teaching and learning that take root nationwide,” President Snyder said. “When we match that expertise with the interdisciplinary knowledge of our faculty, we create a rich environment to explore the educational potential of Microsoft’s extraordinary technology.”

After a small group including Griswold, engineering professor Marc Buchner and Cleveland Clinic education technology leader Neil Mehta first experienced HoloLens in December, the faculty returned to Cleveland to create a core team dedicated to exploring the technology’s academic potential. In February, 10 members of the team–including Ghodasara–returned to Microsoft for a HoloLens programming deep dive.

Ghodasara already had taken the traditional anatomy class at Case Western Reserve, but it wasn’t until he used the HoloLens headset that he first visualized the aortic valve in its entirety–unobstructed by other elements of the cardiac system and undamaged by earlier dissection efforts. Members of the Microsoft team were in the room when Ghodasara had his “aha” moment; a few weeks later, the heart demonstration became part of the Build conference agenda.

Case Western Reserve is the only university represented during the three-day event, a distinction Griswold attributes in part to the core team’s breadth of expertise and collegial approach.

“Without all of those people coming together,” Griswold said, “this would not have happened.”

When Griswold took the stage as part of Microsoft’s opening keynote at the Build conference, Ghodasara, Buchner and Chief Information Officer Sue Workman also were in the audience. Back in Cleveland, three of Professor Buchner’s undergraduates–John Billingsley, Henry Eastman and Tim Sesler–demonstrated some of the potential of the HoloLens technology live in the Tinkham Veale University Center.

Buchner, whose specialties include simulation and game design, believes Microsoft’s innovation “has the capability to transform engineering education.”

Because the technology is relatively easy to use, students will be able to build, operate and analyze all manner of devices and systems. “[It will] encourage experimentation,” Buchner said, “leading to deeper understanding and improved product design.”

In truth, HoloLens ultimately could have applications for dozens of Case Western Reserve’s academic programs. NASA’s Jet Propulsion Laboratory already has worked with Microsoft to develop software that will allow Earth-based scientists to work on Mars with a specially designed rover vehicle. A similar collaboration could enable students here to take part in archeological digs around the world. Or astronomy students could stand in the midst of colliding galaxies, securing front-row view of the unfolding chaos. Art history professors could present masterpieces in their original settings–a centuries-old castle, or even the Sistine Chapel.

“The whole campus has the potential to use this,” Griswold said. “Our ability to use this for education is almost limitless.”

For now, however, the top priority is creating a full digital anatomy curriculum, a process launched with the advent of the Health Education Campus, and now experiencing even greater momentum. Among the key collaborators are a team of medical students and anatomy and radiology faculty who are already investigating the use of these kinds of technology. This team, led by Amy Wilson­Delfosse, the medical school’s associate dean for curriculum, and Suzanne Wish-Baratz, an assistant professor who is one of the primary leaders of anatomy education, fully expects to have a digital curriculum ready for the new Health Education Campus.

Also essential, Griswold said, has been the advice and assistance of Microsoft’s HoloLens team and executives.

“It’s been a joy to work with them. They have been so friendly, so collaborative, so willing to work with us on this,” Griswold said. “We’re going to do incredible things together.”

Ohio is not the only state where virtual reality is being incorporated into medical education.

Florida

From an April 30, 2015 Next Galaxy Corp. news release,

Incorporating eye gaze control, gestures, and voice commands while “walking around” inside an emergency medical experience, Next Galaxy’s Virtual Reality Model educates participants far beyond today’s methodology of passively watching video and taking written tests.

Next Galaxy Corp (OTC: NXGA) recently announced the signing of an agreement with Miami Children’s Hospital to use the Company’s VR Model to develop immersive Virtual Reality medical instructional content for patient and medical professional education. Per the multi-year agreement, Next Galaxy and Miami Children’s Hospital are jointly developing VR Instructionals on cardiopulmonary resuscitation (CPR) and other lifesaving procedures, which will be released as an application for smartphones.

Assessments are incorporated directly into the medical VR models, creating situations where participants are required to make the appropriate decisions about proper techniques. The Virtual CPR instructional will measure metrics and provide real-time feedback ensuring participants accurately perform CPR techniques. Further, the instructional will explain any mistake and prompt users to try again when errors are made. Supportive messages are delivered upon success.

The medical VR models will be viewable through smartphones and desktops as 3D, and via VR devices such as Google Cardboard, VRONE and Oculus Rift.

About Next Galaxy Corporation

Next Galaxy Corporation is a leading developer of innovative content solutions and fully Immersive Consumer Virtual Reality technology. The Company’s flagship consumer product in development is CEEK, a next-generation fully immersive entertainment and educational social virtual reality platform featuring a combination of live action and 3D experiences. Next Galaxy’s CEEK simulates the communal experience of attending events, such as concerts, sporting events, movies or conferences through Virtual Reality. Next Galaxy is developing entertainment and educational experiences for VR Cinema, VR Concerts, VR Sports, VR Business, VR Tourism and more. In short, Next Galaxy is building the meeting places of the future. For further information, visit www.nextgalaxycorp.com

This seems to be the second time this information has been distributed (March 11, 2015 news release on PRNewswire), a widely adopted practice. Consequently and thankfully, there’s a March 11, 2015 article by Celia Ampel for the South Florida Business Journal which provides more details about the technology and explaining how a smartphone fits into virtual reality,

The best way to learn CPR is an immersive experience, Miami Children’s Hospital leaders believe — not a video.

“If I’m watching a video, I can pause and count, but there’s no way to tell if I counted to six or seven,” Next Galaxy President Mary Spio said. “Because [the virtual reality application] is voice-activated, they’re going to be able to count out loud and self-assess whether they’re doing it correctly.”

Next Galaxy (Pink Sheets: NXGA)’s virtual reality technology uses a smartphone app. Users can put their smartphone into a virtual reality headset for an immersive experience, or see 3D content through the phone.

The application will be available to the public in the next few months, Spio said.

This deal and another with Miami-Dade Country Public Schools are transforming Next Galaxy Corp according to Ampel’s article,

The five-person company will be hiring about 20 full-time employees in the next six months, focusing on developers with 3D modeling and gaming experience, she said.

Quadrupling the size of your company in six months can be quite a challenge. I wish them good luck with their expansion and their virtual reality course materials.

As to what all this mixed-reality/virtual reality might look like, there’s this image from Case Western Reserve University,

Courtesy: Case Western Reserve University

Courtesy: Case Western Reserve University