Computer Vision development services for automatic puzzle solver
Abto Software consistently researches artificial intelligence, ML, DL, computer vision, along with data science to ensure our partners can maintain competitive advantage in today’s constantly changing business landscape.
We deliver computer vision services transforming business operations since 2007.
Among various computer vision concepts explored, here’s one worth mentioning – an automatic puzzle solver. By utilizing image acquisition, as well as other computer vision algorithms applied, the automatic jigsaw solver is capable of aligning the scattered puzzle pieces without using any references.
In the following overview, we’ll dive deeper into the concept and those potential applications across industries we see talking about computer vision software development.
CV algorithm for automatic puzzle solving: an overview
The concept, enabled through computer vision, can assemble a jigsaw puzzle from the captured puzzle pieces. For this particular proof-of-concept, we didn’t use the already complete puzzle picture as a final reference.
The below described method of automatic puzzle assembling is comprised of the following modules:
- Puzzle cropping
- Lock detection
- Puzzle alignment
- Puzzle rotation
- Puzzle assembling
- Color description & visualization
Let’s get down to every module to explain the role of each.
Module I. Cropping pieces
After converting the image, the automatic puzzle solver:
- First removes the image background color to create an inverted puzzle mask
- Then saves a vector of 54 separate contours as split puzzle pieces
Module II. Detecting locks
To align the chaotically rotated pieces, the AI puzzle solver must locate the interlocking puzzle parts:
- The program first calculates the convex closure and convexity defects to locate the needed puzzle parts
- At the next step, the program then determines both the inner and outer locks of those puzzle parts, and adjusts the points to pinpoint and align them correctly
Module III. Aligning positions
In order to find the center of mass and determine the matrix for every puzzle piece to align them properly:
- To begin, we bring the pieces to a rectangle shape by filling the piece’s inner locks in white, and the outer locks in black
- When done, we calculate their width and height, as well as the rotation angle
Module IV. Rotating pieces
To align the rotated puzzle pieces, we need to account for all potential orientations in the assembled mosaics. The algorithm is handling this challenge by creating and saving 4х7 copies of each puzzle piece by rotating those pieces by four quadrantal angles and additionally by ±3 degrees for every rotation.
Module V. Aligning pieces
And finally, the algorithm is assembling the pieces by following these steps:
- The selection of positioning to place the processed puzzle pieces with the most adjacent sides known (the initial is the upper-left corner)
- The calculation of the geometric fit between positions and remaining puzzle pieces
- The ranking of the puzzle pieces as to their fit to display the best 8 options for the end user to choose
Module VI. Color description & visualization
What’s more, the number of mistakes can be additionally minimized by adding another criterion – the color.
To match the pieces in accordance to color, the program first extracts the borders from processed puzzle pieces. To extract the color, the algorithm then averages the values of several edge pixels along borders and saves those values in vectors that serve as descriptions for the puzzle pieces.
To go even further than this, we can also create a representation of the achieved results by defining a so-called transformation function.
This function is meant to describe the transition of the puzzle pieces from the initial layout to the final position. To achieve this result, we need to save every transformation being applied to the puzzle pieces in a matrix mi.
In this particular scenario, the pieces have undergone seven transformations, which brings us to this formula:
m = m7 * m6 * m5 * m4 * m3 * m2 * m1
The computations are producing overall 54 transformation matrices and allow us to visually represent the path of each puzzle piece from its initial position to positions they take in the final picture.
CV model for automatic puzzle solving: potential applications
Healthcare
The jigsaw puzzle solver was one fun proof-of-concept, but let’s get down to more domain-specific expertise.
One application within today’s healthcare landscape – physical therapy motion capture and rehabilitation.
Human body pose detection is the computerized recognition of the skeletal structure from images and videos, which involves the identification of joints – the head, shoulders, elbows, hips, knees, and other critical points. Human body pose estimation is implemented for reconstructing 2D and 3D models from received visual input.
Motion analysis might eliminate excessive equipment, personal assistance, and usually subjective assessment. Increased accessibility, enhanced engagement, objective assessment, and other future-looking opportunities – movement assessment might revolutionize the traditional healthcare practice.
Retail
Our auto puzzle solver hasn’t much to do with the retail industry, but our technical knowledge and experience in applying computer vision might benefit business leaders across sectors.
One application worth mentioning – our technology for virtual fitting rooms.
3D online body measurement is the computerized analysis of images and videos to measure bodies remotely. By identifying critical landmarks – neck, chest, shoulders, waist, hips, thighs – the technology can estimate accurate online body measurements in both 2D and 3D formats.
Other scenarios where applying computer vision might benefit retail leaders:
Manufacturing
The expertise we proved through building the image puzzle solver, can be also used to design:
- A quality control system – a solution to recognize slight misalignments, abnormal patterns, and defects in processes by analyzing the fit and finish of components
- A robotic assembly assistant – a solution to guide smart robots assembling machinery and electronics by recognizing and positioning parts accurately
Logistics
The expertise we confirmed when building the picture puzzle solver also resonates with another recent project – a platform for high-density housing environments we upgraded to enable package and delivery management.
We automated day-to-day operations, from administration to delivery, thus optimizing allocated resources. Significantly automating everyday scheduling, logistics planning, driver tracking, package storage and retrieval, we provided for better user experience and greater market competitiveness.
Education
Our team has the required expertise to deliver smart simulators for training across fields:
- Physical therapy and rehabilitation, surgical training, and medical procedure simulations
- Store layout and merchandising, customer service, and retail inventory management
- Quality control and inspection, equipment operation, and assembly line training
- Building and architectural design, safety training, heavy machinery operation simulations, and more
Security & surveillance
Another project to mention talking about our expertise – our solution for accurate bike and helmet detection.
The technology is designed to detect, track and analyze traffic and its individual participants at a specific area. The algorithm can be successfully implemented to transform public safety, territory monitoring, city traffic, transportation and construction planning, and even advertising campaigns.
How we can help
Abto Software is leveraging specialized expertise to extract meaningful information from images and videos. Our engineers can cover every stage from consulting to implementation, continual learning, and maintenance, no matter the requirements, timeline, budget, and industry.
Object recognition and tracking, image and video processing, metadata extraction, anomaly detection – leveraging proven, domain-specific expertise, we apply computer vision to empower business leaders.
Our services:
- .NET development
- ASP.NET development
- Web app development
- Mobile app development
- Cloud services
- Full-cycle, custom software development
Our expertise:
Our R&D engineers developed a CV program for the automatic assembly of a jigsaw puzzle together with the students from our latest CV workshop.