Computer vision based self-diagnosis telemedicine app
for physiotherapy monitoring
Our client provides all-encompassing healthcare products, precisely tailored to facilitate healthcare delivery. Their portfolio comprises software for medical facilities as well as patients, from custom management solutions to easy-to-navigate mobile applications.
The client was looking for expertise in AI for a computer vision based self-diagnosis telemedicine application – within our former cooperation, our team successfully covered personal medical device integration for them. This time, our engineers have applied domain-specific knowledge and experience to upgrade another solution by implementing computer vision for seamless image and video assessment.
Our team has entered into cooperation to upgrade a comprehensive telemedicine application helping clinicians to improve patient outcomes by ensuring personalized treatment and easy-to-follow exercise tutorials.
Our engineers were responsible for developing the PoC for mobile human body pose estimation in real-time. The solution should cover different exercises, for example, cervical flexion utilizing face landmark detection (nose bridge, nose tip, chin, tragus, ear, etc.) and angle of flexion, knee extension and flexion, body bending, and others.
Abto Software focused on:
- The research and identification of the best-suiting approaches for human body detection
- The adjustment and implementation of the selected techniques for accurate movement estimation, which facilitate digital care and monitoring
How the application works
The application is designed to improve physical care:
- The app first guides the patient step-by-step through a series of exercises and tracks every movement
- After analyzing the performance – the angles of joints, the speed of movement – the app then sends the collected health indicators to the treating clinician, so he can adjust the prescribed treatment plan and schedule an appointment if needed
The application can be successfully utilized to improve:
– Digital rehabilitation
– Digital therapeutics
– Spinal rehab
– Sports medicine & orthopedics
Our team has covered:
- Business logic
- Demo-version design
- Computer vision technique implementation, which encompassed:
– In-depth research
– Application prototyping
- UI/UX design
At the initial stages, we trained the built CV model to recognize human motion on ready-made video material. Having achieved our goals, we implemented CV algorithms that process human movement in real-time, allowing end-users to receive immediate feedback from the telemedicine platform, with no additional hardware or sensors.
Determining correct measurement points
At the discovery phase, we had to determine the unerring measurement points for accurate limb assessment, in particular for precise neck tracking:
- For the final implementation, we chose to monitor the ear-nose line segment
- The cervical flexion angle is calculated as the relative change of that line segment in relation to the initial position
The object’s viewpoints differ, which means the object’s shape changes, which alters the object’s features – that causes the model primarily trained from one specific perspective to fail on other, variant viewpoints.
The objects of interest are not steady bodies, which means they can be adulterated in many different ways – for example, in a human pose detection solution, the person can change the posture, he or she may be sitting, standing, walking, running, etc.
To handle this problem, our engineers made sure the used training sets included all possible variations of pose – while learning, the model will give more weightage to the various scenarios.
Tools and technologies
- Google ML Kit
- Apple Vision Face
- April 2021 – October 2021
- 1 project manager
- 1 CV engineer
- 1 iOS developer
- 1 Flutter developer
- 1 UI/UX designer
Value delivered to business
Abto Software has entered into cooperation with the healthcare-focused vendor to help medical professionals notably improve patient outcomes by ensuring personalized treatment and easy-to-follow exercise tutorials. Our team provided human body movement detection to facilitate remote monitoring and value-added, patient-first care by implementing computer vision.
By introducing the solution, our client can achieve:
- CV-based application, enabling therapists to treat more patients, scales up business growth
- CV-based telehealth, being an on-demand solution, gives a competitive edge
- The app enables seamless appointment scheduling and direct information transmission to clinicians, accelerating the patient’s recovery
The solution can benefit:
1. Healthcare businesses
2. Medical professionals, in particular physical therapists
3. Physiotherapy patients, providing personalized digital care with an appropriate guidance
And make healthcare services more accessible for patients:
- That undergo physical therapy
- That have chronic conditions
To find out more about Abto Software expertise, request a quote or get a demo of your custom solution.