AI supported jump recognition and analysis. Case study cover. Abto Software

AI supported jump recognition and analysis

Sensorless human activity recognition to improve public health


Software development
Cross-platform development
Technical consulting
Solution design

Project overview

Our client was looking for partners with required technical expertise to deliver an enterprise school platform. The project was centered around integrating jump recognition and analysis by leveraging artificial intelligence.

Abto Software has formerly successfully delivered education software for the Ireland-based organization. Having entered into cooperation this time, our engineers have implemented artificial intelligence to deliver sensorless human activity recognition to extend an enterprise-level school platform to improve physical health of children at schools.


Main goals

Our team was aiming to adapt complex algorithms that could:

  • Accurately recognize physical activity in real-time
  • Quickly evaluate & measure performed movements
  • Automatically compare the movements with the reference pattern

Our engineers were working on using live streams from both:

  • Web cameras from the user’s PC or laptop
  • Front cameras from the user’s smartphone or tablet

How the solution works

The platform was created to accelerate physical health state assessment as well as motivation across schools. Our concept is designed for accurate jump recognition and analysis, but can be extended to assist in monitoring other exercises – stretching, squatting, and more. 


The concept is simple:


1. The student is shown a easy-to-follow reference pattern of the physical exercise 

2. The student then performs the exercise (at this project stage, it is jumps up)

3. The system quickly compares the execution with the reference template and evaluates
the score

4. The student or the responsible teacher can review the performance for further:
Physical health state assessment
Future training plan adjustment


Our contribution

Abto Software has entered the cooperation to implement artificial intelligence, in particular pose estimation. While working on the described concept, we covered:

  • Business consulting
  • Solution design and research
  • PoC development
  • Integration planning

To ensure quality control, we suggest to implement in-built guides to help the users:

  • Position themselves in a proper distance to the used camera
  • Detect obstacles that might block accurate landmark detection

In the first prototype, the designed CV model was working with ready-made, smartphone captured video files. After the solution’s integration, the trained CV algorithms are working with live video streams derived from different devices – PC’s, laptops, smartphones, tablets.


The delivered proof-of-concept consists of a linear pipeline (in Python) as a command line interface application. For integration, we suggest to conduct landmark and jump detection (in JavaScript) on the client’s side, ensuring smooth real-time performance.

abto software
abto software

Main challenges

The project’s main challenge was accurate AI based pose estimation, to handle which we:

  • Created a reference pattern to compare and evaluate the performance of the executed exercises: those can have different dynamics, that’s why we worked on minimizing its impact 
  • Taught the CV model:
    To recognize and assess both the start and end points of the executed jump
    To localize the movements in time
  • Stabilized the movement assessment
  • Developed techniques to compare human proportions considering height and weight to standardize the parameters

Tools and technologies

  • Python
  • MediaPose
  • SciPy
  • OpenCV

  • Pandas
  • Numpy
  • PyTorch


  • August 2022 – September 2022


  • 1 project manager
  • 1 software architect
  • 1 CV/ML engineer

Value delivered to business

Our team has covered business consulting, solution design and research, PoC development, and smooth integration planning to implement jump recognition and analysis for a school platform to improve public health.


The solution might benefit:

  • National authorities that aim to improve public health
  • Educational institutions that aim to improve student wellbeing and motivation
  • Physical therapists, personal trainers, sports teams, and private fitness centers by helping to monitor the progress of clients and athletes, offer personalized workout plans, adjust techniques, and enhance overall performance
  • Individual users working on improving their health


Contact Us

To find out more about Abto Software expertise, request a quote or get a demo of your custom solution.

  • Clicking this button, I agree to the processing of my personal data.

Contact us

Tell your idea, request a quote or ask us a question