Computer vision in body measurement app for fitting

Shopping online is easy, it takes less time and you can find everything you need in one go. However, the experience is often disappointing, since there are always concerns about fit, textures, or the maker’s ability to render the true color of the item. According to recent studies, between 20-40% of all apparel purchased online is returned, 70 % due to issues with fit alone, costing fashion e-tailers millions. Returns stemming from poor fit are also a major pain point for online retailers.

25 percent of shoppers prefer to buy clothing online

Clothing Fit Solutions

In the last five years, fashion and clothing industry has become the fastest growing e-commerce category and the second largest after consumer electronics. However, fit-related returns cost the retailers millions yearly, not only in terms of lost revenue but also due to the shipping costs. 

This problem has driven many companies to seek the solution in the tech space. The existing solutions for clothing fit on the market vary from 3D body measurement scanners to fit and style AI recommendations engines. 

Fit solution companies routinely emphasize the need for tools that are low-friction, easy-to-use and engaging. But many of them still rely heavily on user-provided data to surface size recommendations and suffer from low adoption. 

Fit guidance solutions can also be costly with too many frictions to implement and operate. To produce accurate results, fit recommendation algorithms also require a heavy and consistent input of measurement data. 

Fast fashion is a reality today with thousands of new products being launched each week; we believe it will be hard to gather enough data quickly enough to come up with accurate fit recommendations. Finally, to produce accurate 3-D renderings, you need a lot of data, which is costly for the retailers. 

Here we suggest a new innovative way for body measurement to determine the clothing size. The solution you can use on your mobile phone at home, and get a quick answer to whether the clothes will fit you or not in a few minutes.

How Our Body Measurement Technology Works

Basically, our clothing fitting technology applies a set of algorithms to the video to capture dimensional data from a mobile device’s camera, measuring the said user. For now, the application can perform body measurement of men only. That is due to the lack of training sets for developing an algorithm that would process the images of women. The user should wear form-fitting clothes, their color should be distinct from the background. Our R&D team developed a mobile body measurement for Android devices, as well as we have private API. You can try it out, just send us the request by filling out the form below.  The mobile app works as follows:

• Place the phone on the floor in a vertical position (screen facing a person standing at full height), the front-facing camera is used to capture frames.
• Then take a photo of background without anything, the app provides audio clues on what position a person should take (you have three seconds to change the position, you have to take four positions in total).
• The body measurement app captures each frame three times.
• Then you will need to provide your height in centimeters.
• The clothing fit solution returns 6 body measurements (neck, chest, waist, and thighs circumference, hands, and legs length).

Body Measurement Computer Vision Solution Development

Phase I of Body Measurement Technology Development – Gathering Data

15 photos with the resolution 640 x 480 (frames):

• Background x 3
• A person in different positions regarding the camera (image):
  • Front x 3
  • Left x 3
  • Right x 3
  • Back x 3

Capturing each frame three times is necessary to detect and erase the noise from the pictures.

Phase II of Body Measurement Solution Development – Creating a Mask

This phase encompasses 11 steps:

1. Converting all frames to HSV color space;
2. Assigning Hue value to zero;
3. Finding the edges for frames;
4. Dilating the edges for background;
5. Creating blobMask – absolute value of subtracting background from the image;
6. Creating edgeMask by subtracting background edges from the edges of the image;
7. Combining blobMask and edgeMask;
8. Morphological transformations of the combined mask (7);
9. Histogram Filtering: this step is crucial for obtaining correct measurements;
10. Calculating all the blobs (a group of connected pixels in an image that share some common property, e.g. pixel value) of the image and computing their area. Deleting the blobs with the area less than 1% of the image;
11. Binarizing the mask, filling the contours to avoid “holes” in the person’s silhouette.

Phase III of Fitting Solution Development – Pixel Measurements

The mask is divided into eight ROIs (regions of interest) with the next meaning:

1. head
2. neck
3. hands (shoulder width for two side views)
4. chest
5. waist
6. thighs
7. legs
8. height

The ROIs are defined relatively to the bounding rectangle of the mask. The relations were calculated experimentally using the combined mask of a set of training images (Figure 2).

Specific pixel measurement is carried out for each ROI (Figure 3):

1. head width – mode of horizontal mask width;
2. neck width – minimum value of horizontal mask width;
3. hands length (shoulder width for two side views) – the length between the extreme left and the extreme right pixel of the ROI;
4. chest width – the mode of horizontal mask width;
5. waist width – the mode of horizontal mask width;
6. thighs width – maximum value of horizontal mask width;
7. legs length (hip width for two side views) is measured by defining the legs intersection spot (groin) and calculating the length of the perpendicular from this spot to the ROI downside;
8. height – the length between uppermost and extreme bottom pixel of the ROI.

Phase IV of Body Measurement Technology Development- Averaging the Parameters and Results

1. Normalization
The user inputs his height in centimeters and all the measurements are normalized in accordance with this value – pixel measurements are translated into centimeters.

2. Least Squares Method & Downhill Solver
For each person, a dozen of measurements are acquired regarding each parameter. To obtain one averaged value for each parameter, the method of the least squares (linear model) in combination with Downhill Solver is applied. These methods allow reducing the calculation of body parameters to the optimization process. The results are calculated as follows:

Ap=b,

where p – parameters, measured in pixels; b – the resulted averaged value for each parameter.

Matrix A is obtained during the training phase (therefore it has a constant value for all input frames):

e(A) = N(Ap-b)

A*=argmin e(A),

where e(A) – the error of the model in the process of training;

p – measurements, calculated by the app;

b – actual measurements of training models; N – Euclidean norm. Matrix A is trained to handle the ellipticity of some measurements; that is, measurements in projection are translated to the three-dimensional measurements. Rough explanation: If a – averaged waist measurement from the front and back view; b – averaged waist measurement from side views, then the actual waist circumference is equal to:

(a+b)coefficient

3. Output
The user receives 6 parameters, measured in centimeters.

Accuracy and Results

Having completed the investigation of the body measurement problem, we have delivered a proof of concept. Our body measurement uses a front camera on the mobile phone to perform body measurement. The solution provides results on average with 97-98% accuracy for measuring men. Benefits of implementing body measurement solution for online shopping:

• Increased customer satisfaction and a higher chance of the customer becoming a repeat customer.
• Our clothing fit technology can help retailers increase sales. By removing the uncertainty around the size you can increase average order values.
• Decreased return rates. Since 70% of all clothing returns lie in the wrong size of the garment, such an app can significantly reduce the return rates.

Summary

Article Name

New Computer Vision Technology for Body Measurement

Description

A question of what will fit you finally gets an answer with our new mobile app that applies innovative CV algorithms for body measurement.

Publisher Name

Abto Software

Publisher Logo