Computer vision based self-diagnosis telemedicine app. Case study cover. Case study by Abto Software

Computer vision based self-diagnosis telemedicine app

Markerless human pose detection
for physiotherapy monitoring
Industry:

Services:

Software development
Technical consulting
Solution design
UI/UX design
1

Project overview

Our client provides all-encompassing healthcare products, precisely tailored to facilitate healthcare delivery. Their portfolio comprises software for medical facilities as well as patients, from custom management solutions to easy-to-navigate mobile applications.

The client was looking for expertise in AI for a computer vision based self-diagnosis telemedicine application – within our former cooperation, our team successfully covered personal medical device integration for them. This time, our engineers have applied domain-specific knowledge and experience to upgrade another solution by implementing computer vision for seamless image and video assessment.

2

Main goals

Our team has entered into cooperation to upgrade a comprehensive telemedicine application helping clinicians to improve patient outcomes by ensuring personalized treatment and easy-to-follow exercise tutorials. 

 

Our engineers were responsible for developing the PoC for mobile human body pose estimation in real-time. The solution should cover different exercises, for example, cervical flexion utilizing face landmark detection (nose bridge, nose tip, chin, tragus, ear, etc.) and angle of flexion, knee extension and flexion, body bending, and others.

 

Abto Software focused on:

  • The research and identification of the best-suiting approaches for human body detection
  • The adjustment and implementation of the selected techniques for accurate movement estimation, which facilitate digital care and monitoring
abto software
3

How the application works

The application is designed to improve physical care:

  • The app first guides the patient step-by-step through a series of exercises and tracks every movement
  • After analyzing the performance – the angles of joints, the speed of movement – the app then sends the collected health indicators to the treating clinician, so he can adjust the prescribed treatment plan and schedule an appointment if needed

The application can be successfully utilized to improve:

Digital rehabilitation

Digital therapeutics

Spinal rehab

Sports medicine & orthopedics 

4

Our contribution

Our team has covered:

  • Business logic
  • Demo-version design
  • Computer vision technique implementation, which encompassed:
    In-depth research
    Application prototyping
  • UI/UX design

At the initial stages, we trained the built CV model to recognize human motion on ready-made video material. Having achieved our goals, we implemented CV algorithms that process human movement in real-time, allowing end-users to receive immediate feedback from the telemedicine platform, with no additional hardware or sensors. 

5

Main challenges

Determining correct measurement points

At the discovery phase, we had to determine the unerring measurement points for accurate limb assessment, in particular for precise neck tracking:

  • For the final implementation, we chose to monitor the ear-nose line segment 
  • The cervical flexion angle is calculated as the relative change of that line segment in relation to the initial position 

Viewpoint variation

The object’s viewpoints differ, which means the object’s shape changes, which alters the object’s features – that causes the model primarily trained from one specific perspective to fail on other, variant viewpoints.

Pose variation

The objects of interest are not steady bodies, which means they can be adulterated in many different ways – for example, in a human pose detection solution, the person can change the posture, he or she may be sitting, standing, walking, running, etc.

To handle this problem, our engineers made sure the used training sets included all possible variations of pose – while learning, the model will give more weightage to the various scenarios. 

Tools and technologies

  • Python
  • Swift
  • Flutter
  • OpenCV

  • iOS
  • Google ML Kit
  • Apple Vision Face

Timeline:

  • April 2021 – October 2021

Team:

  • 1 project manager
  • 1 CV engineer
  • 1 iOS developer
  • 1 Flutter developer
  • 1 UI/UX designer
6

Value delivered to business

Abto Software has entered into cooperation with the healthcare-focused vendor to help medical professionals notably improve patient outcomes by ensuring personalized treatment and easy-to-follow exercise tutorials. Our team provided human body movement detection to facilitate remote monitoring and value-added, patient-first care by implementing computer vision.

 

By introducing the solution, our client can achieve: 

  • CV-based application, enabling therapists to treat more patients, scales up business growth
  • CV-based telehealth, being an on-demand solution, gives a competitive edge
  • The app enables seamless appointment scheduling and direct information transmission to clinicians, accelerating the patient’s recovery

The solution can benefit:

 

1. Healthcare businesses

2. Medical professionals, in particular physical therapists

3. Physiotherapy patients, providing personalized digital care with an appropriate guidance

And make healthcare services more accessible for patients:

  • That undergo physical therapy
    for rehabilitation
  • That have chronic conditions

Categories:

Contact Us

To find out more about Abto Software expertise, request a quote or get a demo of your custom solution.

  • Clicking this button, I agree to the processing of my personal data.

Contact us

Tell your idea, request a quote or ask us a question