Computer vision for self-diagnosis telemedicine app

Our client – a renowned healthcare provider that specializes in technology for complex health management. They bridge the disconnected healthcare process to create an ecosystem for holistic health journeys.

• Their mission: to make high-quality care both safe and affordable
• Their products: a portfolio of platforms for clinicians and patients

Within former successful cooperation, our company has covered the integration of personal medical devices. During more recent collaboration, our engineers have applied their expertise in camera-based pose detection and upgraded another solution.

We extended an existing telemedicine application by leveraging computer vision for seamless video analysis.

Our team has entered into cooperation to upgrade a comprehensive telemedicine application helping clinicians to improve patient outcomes by ensuring personalized treatment and easy-to-follow exercise tutorials.

Our engineers were responsible for developing the PoC for mobile human body pose estimation in real-time. The solution should cover different exercises, for example, cervical flexion utilizing face landmark detection (nose bridge, nose tip, chin, tragus, ear, etc.) and angle of flexion, knee extension and flexion, body bending, and others.

Abto Software focused on:

• The research and identification of the best-suiting approaches for human body detection
• The adjustment and implementation of the selected techniques for accurate movement estimation, which facilitate digital care and monitoring

The application is designed to improve physical care:

• The app first guides the patient step-by-step through a series of exercises and tracks every movement
• After analyzing the performance – the angles of joints, the speed of movement – the app then sends the collected health indicators to the treating clinician, so he can adjust the prescribed treatment plan and schedule an appointment if needed

The application can be successfully utilized to improve:

• Digital rehabilitation
• Digital therapeutics
• Spinal rehab
• Sports medicine & orthopedics

Our team has covered:

• Business logic
• Demo-version design
• Computer vision technique implementation, which encompassed:
  • In-depth research
  • Application prototyping
• UI/UX design

At the initial stages, we trained the built CV model to recognize human motion on ready-made video material. Having achieved our goals, we implemented CV algorithms that process human movement in real-time, allowing end-users to receive immediate feedback from the telemedicine platform, with no additional hardware or sensors.

Determining correct measurement points

At the discovery phase, we had to determine the unerring measurement points for accurate limb assessment, in particular for precise neck tracking:

• For the final implementation, we chose to monitor the ear-nose line segment
• The cervical flexion angle is calculated as the relative change of that line segment in relation to the initial position

Viewpoint variation

The object’s viewpoints differ, which means the object’s shape changes, which alters the object’s features – that causes the model primarily trained from one specific perspective to fail on other, variant viewpoints.

Pose variation

The objects of interest are not steady bodies, which means they can be adulterated in many different ways – for example, in a human pose detection solution, the person can change the posture, he or she may be sitting, standing, walking, running, etc.

To handle this problem, our engineers made sure the used training sets included all possible variations of pose – while learning, the model will give more weightage to the various scenarios.

Abto Software has entered into cooperation with the healthcare-focused vendor to help medical professionals notably improve patient outcomes by ensuring personalized treatment and easy-to-follow exercise tutorials. Our team provided human body movement detection to facilitate remote monitoring and value-added, patient-first care by implementing computer vision.

By introducing the solution, our client can achieve:

• CV-based application, enabling therapists to treat more patients, scales up business growth
• CV-based telehealth, being an on-demand solution, gives a competitive edge
• The app enables seamless appointment scheduling and direct information transmission to clinicians, accelerating the patient’s recovery

The solution can benefit:

1. Healthcare businesses
2. Medical professionals, in particular physical therapists
3. Physiotherapy patients, providing personalized digital care with an appropriate guidance