Wearable Cognitive Assistance (Gabriel)
Home
Today,
over 20 million Americans are affected by some form of cognitive
decline that significantly affects their ability to function as
independent members of society. This includes people with
neurodegenerative conditions such as Alzheimer’s disease (~4.5M) and
mild cognitive impairment (>6M), survivors of stroke (~2.5M) and
people with traumatic brain injury (~5.3M). These numbers are
expected to grow significantly due to an aging population and an
increase in long-term post-traumatic stress disorders arising from
occupational and social causes. Cognitive impairment can manifest
itself in many ways, including the inability to recognize people,
locations and objects, loss of short- and long-term memory, and changes
in behavior and appearance such as decreased attention to personal
hygene. Among the many challenges faced by older Americans, cognitive
decline often has the largest negative impact on them and their family
members. The potential cost savings from even modest steps
towards addressing this challenge are enormous: it is estimated that
just a one-month delay in nursing home admissions nationwide could save
$1.12B annually.
Can information technology reduce the
burden of cognitive impairment? Inspired by early examples of
"smart spaces" such as the Aware Home at Georgia Institute of
Technology, the Gator Tech Smart House at the University of Florida, and
Elite Care's Oatfield Estates in Milwaukie, OR, we seek to greatly
increase the scope and range of systems that offer cognitive
assistance. Our goal is to free such systems from the confines of
a purpose-built smart space, while simultaneously enriching the user
experience and assistive value to the user. In other words, we
aim to create cognitive assistive
systems that can function ``in the wild'' with sufficient
functionality, performance and usability to be valuable at any time and
place. Aided by such systems, commonplace activities such as
going to a shopping mall, attending a baseball game, or transacting
business downtown should become attainable goals for people in need of
modest cognitive assistance. Here is a hypothetical
scenario that is suggestive of the future we hope to create:
Ron
is a young veteran who was wounded in Afghanistan and is slowly
recovering from traumatic brain injury. He faces an uncertain
future, with a lack of close family nearby and with limited financial
resources for professional caregivers. He has suffered a sharp
decline in his mental acuity and is often unable to remember the names
of friends and relatives. He also frequently forgets to do simple
daily tasks. Even modest improvements in his cognitive ability
would greatly improve his quality of life, while also reducing the
attention demanded from caregivers. This would allow him to live
independently in dignity and comfort, as a productive member of his
community.
Fortunately,
a new experimental technology may provide Ron with cognitive
assistance. At the heart of this technology is a lightweight
wearable computer built into the frame of Ron's eyeglasses. Integrated
with the eyeglass frame are a camera for scene capture and
bone-conduction earphones for audio feedback. These hardware
components offer the essentials of a system to aid cognition when they
are combined with software for scene interpretation, face recognition,
context awareness and voice synthesis. When Ron looks at a person
for a few seconds, that person's name is whispered in his ear along
with additional cues to guide Ron's greeting and interactions; when he
looks at his thirsty houseplant, ``water me'' is whispered; when he
look at his long-suffering dog, ``take me out'' is whispered.
Ron's magic glasses travel with him, transforming his surroundings into a helpful smart environment.
The hardware for "magic glasses" is almost here. Google's Project Glass is a good example. The First Person Vision (FPV) system from the Quality of Life Technology Center at Carnegie Mellon is another example. Here is a cool YouTube video
of the very first wearable cognitive assistance application that we
have built. Check out the papers on the "Publications" link
off the home page.