Finding A Suitable Solution

Senthil Kumaran

2010-09-01 05:20

This text was first published in Avinash Sonnad Blog

Finding a suitable solution

Written by Senthil Kumaran

Presented at the conference by Senthil and Avinash, Spastics Society of Karnataka.

Avinash was a student with Spastics Society of Karnataka and currently a student with Christ University. He has multiple disabilities and suffers from Cerebral palsy Cerebral palsy (CP) is a group of movement disorders that appear in early childhood. Signs and symptoms vary among people and over time, but include poor coordination, stiff muscles, weak muscles, and tremors. There may be problems with sensation, vision, hearing, and speech. Often, babies with cerebral palsy do not roll over, sit, crawl or walk as early as other children. Other symptoms may include seizures and problems with thinking or reasoning. While symptoms may get more noticeable over the first years of life, underlying problems do not worsen over time. . Senthil is a Software Developer working in Bangalore. He knows Avinash from the time he was in Spastics Society of Karnataka and has been working him in identifying a suitable technology for overcome his challenges in communication.

Avinash and I started looking out for a suitable Assistive technology Assistive technology (AT) is a term for assistive, adaptive, and rehabilitative devices for people with disabilities and the elderly. Disabled people often have difficulty performing activities of daily living (ADLs) independently, or even with assistance. ADLs are self-care activities that include toileting, mobility (ambulation), eating, bathing, dressing, grooming, and personal device care. Assistive technology can ameliorate the effects of disabilities that limit the ability to perform ADLs. Assistive technology promotes greater independence by enabling people to perform tasks they were formerly unable to accomplish, or had great difficulty accomplishing, by providing enhancements to, or changing methods of interacting with, the technology needed to accomplish such tasks. For example, wheelchairs provide independent mobility for those who cannot walk, while assistive eating devices can enable people who cannot feed themselves to do so. Due to assistive technology, disabled people have an opportunity of a more positive and easygoing lifestyle, with an increase in "social participation", "security and control", and a greater chance to "reduce institutional costs without significantly increasing household expenses." In schools, assistive technology can be critical in allowing students with disabilities to access the general education curriculum. Students who experience challenges writing or keyboarding, for example, can use voice recognition software instead. Assistive technologies assist people who are recovering from strokes and people who have sustained injuries that affect their daily tasks. for a long time and we have discovered a number of things with our trial and error methods. It was quite clear to me that Assistive Technologies will be useful for people like Avinash.

Just after meeting Avinash, I realized, a software called Dasher (software) Dasher is an input method and computer accessibility tool which enables users to compose text without using a keyboard, by entering text on a screen with a pointing device such as a mouse, touch screen, or mice operated by the foot or head. Such instruments could serve as prosthetic devices for disabled people who cannot use standard keyboards, or where the use of one is impractical. could be useful to him. So, I went to his house and I remember we started with Dasher. We did not know how to use it. I read and studied the documentation and it was of no avail. I also realized the limitations of Avinash then. I saw that he was able to move only his thumb and the index finger and he had a lot of involuntary movements. I tried different kinds of mouse which he can hold on with his two fingers and my search for an Assistive technology device started along the lines of finding a suitable mouse device for Avinash. It was four years ago, that we also tried Speech recognition Speech recognition is an interdisciplinary subfield of computer science and computational linguistics that develops methodologies and technologies that enable the recognition and translation of spoken language into text by computers. It is also known as automatic speech recognition (ASR), computer speech recognition or speech-to-text (STT). It incorporates knowledge and research in the computer science, linguistics and computer engineering fields. The reverse process is speech synthesis. to see if it would be helpful. Very few people were using Voice Recognition then and I had heard that it requires considerable training to use the Voice Recognition. So I started with the Voice Recognition training and I soon realized that the software was demanding a certain accent and it was not able to recognize Avinash's style of speaking. It led me to give up the thought on Voice Recognition itself.

Our First Accessibility Device

Avinash is an avid reader. I was surprised by the way he used to read his books. He used to lie down on his side on bed and his mother used to flip pages for him. Reducing his dependency on his Mother to flip pages for him might be the first step forward. I knew that Adobe Acrobat Adobe Acrobat is a family of application software and web services developed by Adobe Inc. to view, create, manipulate, print and manage Portable Document Format (PDF) files. reader had the auto-scroll option that would help in reading the book.

In his personal laptop with books loaded as PDF Portable Document Format (PDF), standardized as ISO 32000, is a file format developed by Adobe in 1992 to present documents, including text formatting and images, in a manner independent of application software, hardware, and operating systems. Based on the PostScript language, each PDF file encapsulates a complete description of a fixed-layout flat document, including the text, fonts, vector graphics, raster images and other information needed to display it. PDF has its roots in "The Camelot Project" initiated by Adobe co-founder John Warnock in 1991. documents in the auto-scroll mode, the book will automatically scroll at regular pace set by us, Avinash would be able to read the entire book without his mom's help. Viola! This was our first accessibility device.

With this feature, he read 5 books completely. He read, " Alice in Wonderland Alice's Adventures in Wonderland (also known as Alice in Wonderland) is an 1865 English children's novel by Lewis Carroll, a mathematics don at the University of Oxford. It details the story of a girl named Alice who falls through a rabbit hole into a fantasy world of anthropomorphic creatures. It is seen as an example of the literary nonsense genre. The artist Sir John Tenniel provided 42 wood-engraved illustrations for the book. ", a set of 14 short-stories of Sherlock Holmes Sherlock Holmes () is a fictional detective created by British author Arthur Conan Doyle. Referring to himself as a "consulting detective" in his stories, Holmes is known for his proficiency with observation, deduction, forensic science and logical reasoning that borders on the fantastic, which he employs when investigating cases for a wide variety of clients, including Scotland Yard. and H.G.Well's Herbert George Wells (21 September 1866 – 13 August 1946) was an English writer, prolific in many genres. He wrote more than fifty novels and dozens of short stories. His non-fiction output included works of social commentary, politics, history, popular science, satire, biography, and autobiography. Wells's science fiction novels are so well regarded that he has been called the "father of science fiction". " First men on Moon The First Men in the Moon by the English author H. G. Wells is a scientific romance, originally serialised in The Strand Magazine and The Cosmopolitan from November 1900 to June 1901 and published in hardcover in 1901. Wells called it one of his "fantastic stories". The novel recounts a journey to the Moon by the two English protagonists: a businessman narrator, Mr. Bedford; and an eccentric scientist, Mr. Cavor. Bedford and Cavor discover that the interior of the Moon is inhabited by a sophisticated extraterrestrial civilisation of insect-like creatures they call "Selenites". ".

The Adobe Acrobat software also has a reader option where the software can read the words aloud. However, it was not desirable as it was very mechanical and it was not enjoyable for Avinash.

With the auto-scrolling feature, there still was a problem. It was not possible for Avinash to take a break while reading as it would require manual intervention to stop the computer from scrolling. So, Avinash had to be constantly on his toes, so as to figuratively speak, to keep pace with the automatic scrolling of the book.

We definitely needed a better solution with more control.

Second Accessibility Device - Mobile phone

One of the mobile phones in the market had a stick like pointer in the middle and it was very suitable for Avinash. If someone placed that mobile in his hands, he was able to control it with the stick interface. So, I got the idea of connecting the mobile via Bluetooth Bluetooth is a short-range wireless technology standard that is used for exchanging data between fixed and mobile devices over short distances and building personal area networks (PANs). In the most widely used mode, transmission power is limited to 2.5 milliwatts, giving it a very short range of up to 10 metres (33 ft). It employs UHF radio waves in the ISM bands, from 2.402 GHz to 2.48 GHz. It is mainly used as an alternative to wired connections to exchange files between nearby portable devices and connect cell phones and music players with wireless headphones, wireless speakers, HIFI systems, car audio and wireless transmission between TVs and soundbars. to the laptop cursor, so that the scrolling of the book can be controlled. But the mobile which I got was slippery and also it required its cover to be removed in order to expose the middle stick interface properly.

Tearing down a mobile just to use the pointer was something I daringly tried, but proved, not effective.

We did try with controlling the cursor, but it was simply inefficient given the limited control which Avinash could exercise on his mobile phone.

Third Accessibility Device - A very small infra-red mouse

Given that mobile phone was not suitable, I started looking out for a small mouse which could fit into Avinash's palm. I got a Infra-red Infrared (IR; sometimes called infrared light) is electromagnetic radiation (EMR) with wavelengths longer than that of visible light but shorter than microwaves. The infrared spectral band begins with the waves that are just longer than those of red light (the longest waves in the visible spectrum), so IR is invisible to the human eye. IR is generally (according to ISO, CIE) understood to include wavelengths from around 780 nm (380 THz) to 1 mm (300 GHz). IR is commonly divided between longer-wavelength thermal IR, emitted from terrestrial sources, and shorter-wavelength IR or near-IR, part of the solar spectrum. Longer IR wavelengths (30–100 μm) are sometimes included as part of the terahertz radiation band. Almost all black-body radiation from objects near room temperature is in the IR band. As a form of EMR, IR carries energy and momentum, exerts radiation pressure, and has properties corresponding to both those of a wave and of a particle, the photon. wireless mouse from Staples store at Marathali, Bangalore. This was incidentally the first purchase, specifically made for 'trying things out'.

I tried if we could control our original solution of Dasher with this small-mouse in the way such that it could be used like a click device. I studied Dasher again and saw that the whole operation can be controlled using a single switch, but I did not find a way to interface that single switch to our mouse.

So, I wrote to the dasher mailing list to seek help from experts. Dr. Julius who is an expert in assistive technology suggested that I try out camera mouse, which can recognize Avinash's face and thus he should be able to to control the mouse movements with his head. This was an innovative suggestion, which we had not tried in our earlier attempts.

Fourth Accessibility Device - A camera mouse

The camera mouse solution was an interesting one. I setup the camera mouse that it could recognize some fixed point in Avinash's face and as he moved his head the position of the mouse pointer could be controlled.

And to our surprise, we found that "It worked!". He practised a lot with the camera mouse solution, working in tandem with Dasher. These were the first few words written by Avinash using the Camera Mouse on Dasher.

"Education is the only possible way to enlighten the people's mind to make this world a beaieul place to live in. "

It is a from Dr. Kalam's Avul Pakir Jainulabdeen Abdul Kalam ( ; 15 October 1931 – 27 July 2015) was an Indian aerospace scientist and statesman who served as the president of India from 2002 to 2007. book, "Inspiring thoughts". Avinash was able to write this down with great difficulty. There is a mistake in the sentence, and I left it consciously, because it always believe, it is okay to make mistakes.

The camera mouse was not the solution yet. Due to involuntary movements, the mouse pointer deviated frequently from the intended position. Julius suggested to us that by gently nudging it back to the specific point this could be controlled and he advised us to practise more. However, someone had to assist Avinash in adjusting the camera-mouse settings properly and then load the required software. Avinash could exhibit only a certain level of control from this point onwards. It was a good improvement from where we started with, but it still lacked something which we desired, namely the ease of use.

Fifth Accessibility Technology - Voice Recognition

Meanwhile in the Dasher mailing list, someone had mentioned that he was using Voice Recognition in composing the mail and he uses Voice Recognition and Dasher simultaneously. I approached him and he suggested that Voice Recognition technology has improved a lot in the recent years and suggested that I try with the latest version of Microsoft Speech API The Speech Application Programming Interface or SAPI is an API developed by Microsoft to allow the use of speech recognition and speech synthesis within Windows applications. To date, a number of versions of the API have been released, which have shipped either as part of a Speech SDK or as part of the Windows OS itself. Applications that use SAPI include Microsoft Office, Microsoft Agent and Microsoft Speech Server. .

This required us to upgrade the speech recognition software in the operating system. Once we did it, we tried the Voice Recognition training program again. To our surprise, it worked very well for Avinash's voice and his accent was not a problem like before. We were just enthralled. He quickly finished the training and saw if he can use the voice recognition to control the computer by voice. However, to our disappointment, it did not recognise the correct words when Avinash was using the software. It was due to the way the software is designed. It had a huge sample space to search and it did not identify what Avinash was trying to say.

Then I set about to find a software which provides a limited voice recognition capability, something like it could do only 10 tasks for the commands we give. Given the limited and well defined set of tasks, the software may work without any problems for Avinash.

Sixth Accessibility Technology - e-Speaking Voice Recognition software

Now, I did find a software that was meeting our exact needs. It was e-Speaking Voice Recognition software. It used the System's voice recognition engine and provided a limited set of commands to control the computer. It was readily available for a nominal price. I purchased it and found that it was exactly what we wanted at the moment.

Thus, Avinash could use the software effectively using speech. He could control the scrolling of the adobe acrobat reader to read books, browse the folder to go and get a new book, Connect to Internet and read news etc.

This was wonderful, it enhanced his ability to work independently on his computer. With more practise he was only getting better and this proved to be a convenient solution for Avinash. Just switch-on the computer with with these software in the auto-start mode, if the microphone is attached to the computer, then he could control it from that point onwards. No manual intervention further required.

Seventh Accessibility Technology - Writing via Dasher using Speech

A complete solution required combining the above individual elements. Avinash had tried and succeeded using Dasher via head-mouse and then he could now control his computer using e-Speaking voice recognition software. How about the idea of combining both? Namely controlling the cursor of computer via speech. We tried and it worked again. It was immensely helpful and satisfying. Avinash was able to write on his computer using Dasher! This required more practise in understanding the way Dasher works. Over time, he gained the ability to control his computer and dasher together to write sentences effectively.

Avinash still uses On-screen keyboard A virtual keyboard is a software component that allows the input of characters without the need for physical keys. Interaction with a virtual keyboard happens mostly via a touchscreen interface, but can also take place in a different form when in virtual or augmented reality. to click on letters and composing words. He takes a long time to compose in this way. However, I believe with his speed can be increased significantly using Dasher, which would be as close to the average speed of one among us.

Finally something useful

This was a very good result. We both overjoyed with the outcome. Avinash's mom was free from the task of flipping the pages for him. Avinash was able to immerse himself in some creative pursuit for hours together on computer and Internet and thus be engaged with some activity or the other. Both Avinash's father and his brother, Sanjeev, are both happy with this new found capability and the way he keeps himself engaged in his studies.

It was very nice to find a solution which was useful and effective.

For me, Senthil, I found that, I took on a very hard problem in relatable space, dedicated myself to find a suitable solution. It was satisfying. When someone suggest about "scaling" the solution, I say, solutions to disabilities are person specific. Needs of each and every person is different, a solution needs to be specific to every person.

I hope this article provided a glimpse into the process of finding an effective solution for Avinash. He uses Dasher effectively for a variety of purposes, even for taking tests in college now.

This  was written  by senthil  for the book released on the beginning of Assistive Technology Conference.

i thank Senthil for all that he has done for me.

- Avinash

Here is the video of accessiblity tool in action