Please note as of Wednesday, August 15th, 2018 this wiki has been set to read only. If you are a TI Employee and require Edit ability please contact x0211426 from the company directory.
Project Smart Pill Box
- 1 Thanks for voting for Project Smart Pill Box!!!
Thanks for voting for Project Smart Pill Box!!!
The American Society of Health System Pharmacists found that about one third of older adults are prescribed to take eight or more medications each day. With such a large amount of prescriptions, the consequences of forgetting to take medications at a particular time or, worse, accidentally taking the same medication twice are significantly heightened. The British Pharmacological Society noted that greater than 80 percent of elderly hospitalizations due to harmful drug reactions are caused by dosage errors. While prescription bottles have strict instructions covering their surfaces, all of these reminders are futile in patients struggling with memory pathologies, such as Alzheimer’s.
While it may be sound like an adequate solution to simply provide an elderly Alzheimer’s patient with a pill box and alarm clock, that’s honestly not enough. Even if the patient hears the alarm, understands it’s time to take his/her 3:00pm meds, and opens to the correct day, there is still a large possibility that the patient will become distracted before actually ingesting any pills. On the other hand, the patient could successfully take the 3:00pm pills but forget that he/she took them, so when 3:15pm comes around, the result would be an accidental overdose or a very confused patient.
To combat some of these unfortunate situations, Project Smart Pill Box aims to add a visual verification system that will keep reminding patients to take their medications until they provide visual proof of taking the pills out of the box and lifting them up to their mouths to ingest.
It can be assumed that the patients opting to use the Smart Pill Box desire to maintain a healthy medication schedule; in other words, the patients would not be trying to trick the software. Also, the pill box is assumed to be filled correctly each week by either the patient or a caretaker. Ultimately, this stage of Project Smart Pill Box aims to eliminate accidental overdoses caused by patients who forget that they already took their medication and to prevent exacerbating diseases by failing to take a prescribed medication at the correct time. Please see “Future Work” for our plans to improve Project Smart Pill Box.
The BeagleBoard is an ideal platform for Project Smart Pill Box due to its small form factor, low cost, and processing power. Since our project requires a large amount of video processing, we needed a device capable of completing all of these tasks in real-time. Additionally, we want the Smart Pill Box to be portable, so user’s can travel comfortably with it. Finally, we want our product to be accessible to all of those who can benefit from it.
Project Smart Pill Box incorporates a BeagleBoard with a screen and a webcam. The software was developed using the Open Source Computer Vision (OpenCV) library. Using this library, our program detects the face of a user and then attempts to track a hand as it moves towards the user’s mouth. This action will constitute the taking of the pill. The software will also notify a user if he/she has taken his/her pills already for the day. The clock is kept accurate by automatically updating over the internet immediately after the Smart Pill Box is turned on. If desired, a log file could keep track of the user’s pill-taking.
Project Smart Pill Box proved to be a challenge for us due to our inexperience with Linux, OpenCV, and video processing in general. We are much more comfortable in Windows, so we decided to do most development there instead of on Linux. Since OpenCV is just a set of libraries, the code could be written in Windows and later compiled for Linux.
During the course of development, we recognized that hand-tracking was too difficult given the lack of formal training in video processing. Therefore, some modifications were necessary in order to make the project more feasible (see "Modifications" below).
The largest issue faced by Project Smart Pill Box arose when we tried to put everything together on the BeagleBoard. At the beginning of the project, some preliminary tests were done on the BeagleBoard to ensure things would go smoothly at the end. These tests included the following: installing Ubuntu Linux on the BeagleBoard (link), verifying a video stream from a Logitech Orbit Webcam (link) on the BeagleBoard, and running some of the basic sample code for OpenCV on the BeagleBoard. Unfortunately, the webcam did not work with OpenCV in Linux when we were combining all of the parts of our project. At the beginning, we did not test OpenCV with the webcam in Linux; we just assumed it would work, since it is on the list of verified webcams on Linux for OpenCV. We searched for hours on Google, looking for possible solutions to our problem, but none of the information we found helped. Ideally, we would have bought another webcam from the approved list for OpenCV, but we lacked the funds to do so. Since the software worked in Windows, we were left with a BeagleBoard without a working webcam that was ready to run the platform-independent software for Project Smart Pill Box.
Since we found hand-tracking to be too intricate, we focused on the more manageable task of tracking a colored dot. For our purposes, a bright green dot was placed on the back of the user’s hand in order to accomplish a primitive form of hand-tracking. This plan proved to be very successful.
There are many issues that could be resolved and features that could be added if more time were available. As far as our current issues are concerned, we would find a working webcam and get our base software running properly on the BeagleBoard. We can split potential new features into two categories, hardware and software.
On the software side, we could implement a much better hand-tracking algorithm that would not require the colored square. Another useful feature would be streaming the pill-taking-action to someone else, so that the user’s activity could be monitored by a family member or caretaker. We could also add lots of configuration for alarms so that our product could be customized to each patient’s needs.
On the hardware side, we could add an actual pill box with sensors to detect if a specific day’s pills were taken out of the box. Each week, the pill box could be filled either by the user or by someone looking after the user. This physical pill box would act as a security measure to minimize false positives where, for instance, the user happen to scratch his/her nose in front of the webcam instead of taking the pills. Additionally, in order to create an actual product, the BeagleBoard along with the necessary components (screen, webcam, wifi dongle) would need to be packaged in a relatively small case.
Below is a zip file of our code. Additionally, we created a startup script by editing the file /etc/rc.local for the BeagleBoard that enables the internet and updates the time. The code is shown below for reference. Finally, we set the BeagleBoard to automatically login, and we added the Smart Pill Box software to the startup programs, so that it is run immediately when the BeagleBoard is switched on.
/etc/rc.local #!/bin/sh -e # # rc.local # # This script is executed at the end of each multiuser runlevel. # Make sure that the script will “exit 0” on success or any other # value on error. # # In order to enable or disable this script just change the execution # bits. # # This script enables ethernet and updates the time. dhclient eth0 ntpdate 0.pool.ntp.org hwclock --systohc exit 0
Updated code with speed optimizations for the BeagleBoard can be found here: File:Smart Pill Box.zip
BeagleBoard Challenge: http://processors.wiki.ti.com/index.php/Beagle_Board_Open_Source_Design_Challenge
BeagleBoard Ubuntu: http://elinux.org/BeagleBoardUbuntu
Intro to OpenCV on Linux: http://www.linuxconfig.org/introduction-to-computer-vision-with-opencv-on-linux
Face Detection: http://opencv.willowgarage.com/wiki/FaceDetection
Supported Cameras in OpenCV on Linux: http://opencv.willowgarage.com/wiki/Welcome/OS
OpenCV and Visual Studio: http://opencv.willowgarage.com/wiki/VisualC%2B%2B
Jackie Leverett is a 3rd year Biomedical Engineering (Track 1 Imaging and Instrumentation) student at The University of Texas at Austin.
Zach Wasson is a 4th year Electrical Engineering student at The University of Texas at Austin.
If you have any questions or comments, please contact us at <email@example.com>.