Computer vision for self-diagnosis telemedicine app
to support physical therapy
Project summary
Our solution for remote physio sessions – no traveling, no friction, only obvious health benefits for patients. The application will measure joint angles and flag technique errors to send detailed reports for physicians.
Reviews highlight telerehabilitation increases appointment attendance (plus 8%) and adherence (plus 9%).
Services:
Project overview
Our client – a renowned healthcare provider that specializes in technology for complex health management. They bridge the disconnected healthcare process to create an ecosystem for holistic health journeys.
- Their mission: to make high-quality care both safe and affordable
- Their products: a portfolio of platforms for clinicians and patients
Within former successful cooperation, our company has covered the integration of personal medical devices. During more recent collaboration, our engineers have applied their expertise in camera-based pose detection and upgraded another solution.
We extended an existing telemedicine application by leveraging computer vision for seamless video analysis.
“They are great professionals who are always ready to go the extra mile to deliver the best services.”
Main goals
Our team has entered into cooperation to upgrade a comprehensive telemedicine application helping clinicians to improve patient outcomes by ensuring personalized treatment and easy-to-follow exercise tutorials.
Our engineers were responsible for developing the PoC for mobile human body pose estimation in real-time. The solution should cover different exercises, for example, cervical flexion utilizing face landmark detection (nose bridge, nose tip, chin, tragus, ear, etc.) and angle of flexion, knee extension and flexion, body bending, and others.
Abto Software focused on:
- The research and identification of the best-suiting approaches for human body detection
- The adjustment and implementation of the selected techniques for accurate movement estimation, which facilitate digital care and monitoring
Accurate assessments with real-time posture insights
How the application works
The application is designed to improve physical care:
- The app first guides the patient step-by-step through a series of exercises and tracks every movement
- After analyzing the performance – the angles of joints, the speed of movement – the app then sends the collected health indicators to the treating clinician, so he can adjust the prescribed treatment plan and schedule an appointment if needed
The application can be successfully utilized to improve:
– Digital rehabilitation
– Digital therapeutics
– Spinal rehab
– Sports medicine & orthopedics
Our contribution
Our team has covered:
- Business logic
- Demo-version design
- Computer vision technique implementation, which encompassed:
– In-depth research
– Application prototyping - UI/UX design
At the initial stages, we trained the built CV model to recognize human motion on ready-made video material. Having achieved our goals, we implemented CV algorithms that process human movement in real-time, allowing end-users to receive immediate feedback from the telemedicine platform, with no additional hardware or sensors.
Main challenges
Determining correct measurement points
At the discovery phase, we had to determine the unerring measurement points for accurate limb assessment, in particular for precise neck tracking:
- For the final implementation, we chose to monitor the ear-nose line segment
- The cervical flexion angle is calculated as the relative change of that line segment in relation to the initial position
Viewpoint variation
The object’s viewpoints differ, which means the object’s shape changes, which alters the object’s features – that causes the model primarily trained from one specific perspective to fail on other, variant viewpoints.
Pose variation
The objects of interest are not steady bodies, which means they can be adulterated in many different ways – for example, in a human pose detection solution, the person can change the posture, he or she may be sitting, standing, walking, running, etc.
To handle this problem, our engineers made sure the used training sets included all possible variations of pose – while learning, the model will give more weightage to the various scenarios.
Modernize physiotherapy & rehab without needing extra hardware
Tools and technologies
- Python
- Swift
- Flutter
- OpenCV
- iOS
- Google ML Kit
- Apple Vision Face
Timeline:
- April 2021 – October 2021
Team:
- 1 project manager
- 1 CV engineer
- 1 iOS developer
- 1 Flutter developer
- 1 UI/UX designer
Value delivered to business
Abto Software has entered into cooperation with the healthcare-focused vendor to help medical professionals notably improve patient outcomes by ensuring personalized treatment and easy-to-follow exercise tutorials. Our team provided human body movement detection to facilitate remote monitoring and value-added, patient-first care by implementing computer vision.
By introducing the solution, our client can achieve:
- CV-based application, enabling therapists to treat more patients, scales up business growth
- CV-based telehealth, being an on-demand solution, gives a competitive edge
- The app enables seamless appointment scheduling and direct information transmission to clinicians, accelerating the patient’s recovery
The solution can benefit:
1. Healthcare businesses
2. Medical professionals, in particular physical therapists
3. Physiotherapy patients, providing personalized digital care with an appropriate guidance
And make healthcare services more accessible for patients:
- That undergo physical therapy
for rehabilitation - That have chronic conditions