Get the Right Number: Image Processing for Object Counting

Get the Right Number: Image Processing for Object Counting

Automated counting applications for production lines are designed and developed to track, identify, separate and count products, and all sorts of objects in a bounded image area, and provide fast and highly accurate results. Many of today’s systems are built following traditional approaches to image processing which lowers their efficiency and accuracy. The automated counting system developed by A-Grade IT dedicated developers team for the Bakery factory is designed to locate, identify, and count objects in the input image we get from video processing. One of the major advantages of this system is that it’s capable of separating touching products prior to counting, thus the results are highly accurate at 99.5%.

Today’s post from one of our image processing experts describes the problem of traditional approaches and introduces a new one that eliminated the issues that would arise otherwise, and touches upon other areas it could be applied to identify and count products.

Generally speaking, image processing is considered as any form of signal processing with an image input, such as a photo or video frames. The counting problem in image/video processing is the estimation of the number of objects in a still image or video frame. It arises in many real-world applications including cell counting in microscopic images, monitoring crowds in surveillance systems, or in our case, it’s a problem of counting the baked products on the production lines.

Computer Vision Solutions

Give meaning to images, analyze video, and recognize objects with the highest accuracy.

The initial version of the production counting system was designed and developed in MATLAB and had certain limits and restrictions.

  1. Rather a low counting accuracy that was dropping even more, if the bakery products were touching and in uneven rows on the production line.
  2. Poorly optimized, the system required too many resources.
  3. Limited to processing only 4 video channels at once.

So, how can we solve it to get an accurate result?

object counting using image processing
Fig. 1. Montage image built from the first 25 video frames

I. Traditional image processing approach

Here we cover object counting following a traditional approach to image processing, and what challenges we may face, if we were to do it.

The first thing we need to do is to separate the foreground and background. The easiest way would be to use color segmentation, however then we need to resolve the problem of uneven illumination. To avoid it, it is better to apply adaptive thresholding with the Otsu segmentation algorithm for simple cases, using the RGB channel. As a result, we get a binary image (where every pixel equals 1 to refer to an object, and 0 otherwise). This binary image will have a lot of artificial objects with a little square (about 1-10 pixels) which is why we have to remove them using median filtering or methods of the mathematic morphology.  In our case, for counting objects on the production line, the morphological opening operation will be a better choice.

At this stage, it is crucial to separate all objects into individual 4-connected areas. We define a pixel set as a 4-connected area where every pixel has at least one northern, southern, eastern, or western neighbour.

Now that we have cleared the binary image that corresponds to the given frame, we move to the issue of counting the objects themselves. If we were to have one video frame, we could use a wave algorithm to count the products. However, we need to count them during a certain time; hence every object will be present at several sequent frames. Therefore, we have to count every object that crosses a certain imaginary line that is perpendicular to the moving direction.

object counting using image processing
Fig. 2. Image processing:
a) source colour image; b) red channel of the source image;
c) Otsu threshold applied; d) cleared binary image

 

The main challenge for the traditional approach is to identify every object and track it to determine whether it has crossed given a line or not. In other words, instead of counting the objects, we need to track the object trajectories. And to do that you can apply one of the standard tracking techniques, Kalman filter etc., though it will require large computational resources and results are not that accurate. For example, the change in illumination causes errors in tracking or object recapturing. Those are clear disadvantages of the standard approach.

Sometimes they suggest using rectangular areas of interest instead of lines. The rectangle height is bigger than the object height, allowing us to count objects that are inside of the area completely. Still, this method works for counting similar objects that lay in a single row only. Therefore, we have to look for another way to count the objects on the production line.

object counting using image processing
Fig. 3. A line of counting (a line of interest)

II. Our approach to image processing for object counting

Let’s consider the video as a sequence of frames that are arranged in a stack of paper sheets. As it was mentioned before, we have to count every object that crosses the given line.  Therefore, only pixels that are on this line for each frame should be taken into account. That is, we need to consider the frames cross-sectioning along the line. This method is known as a slit-scan camera algorithm and is widely used in the sport for registering a winner crossing the finish line, as well as for artistic purposes.

How does it work?

1) During a certain period of time, we just copy pixels along the given line from every frame and append them to the resulting image. Thus, we should place the camera in a way that would record objects moving strictly horizontally or vertically. In this case, it is enough to copy a row (or column, respectively) from every frame.

object counting using image processing
Fig. 4. Slit-scan image formation

 

From the object counting perspective, the resulting image is equivalent to the source video and can be considered as its hash, and it is something quite peculiar.

object counting using image processing
Fig. 5. Resulting image

2) The binary image is built from the resulting image in a way that was described above. Now it is easy to estimate background as a mean or median for each resulting image column.  Every column is the image of the same point, but at different points in time. To clear binary image and separate every object we can use the usual way.

object counting using image processing
Fig. 6. Image processing: a) resulting colour image; b) red channel of the resulting image; c) Otsu threshold applied;
d) cleared binary image

 

3) Now we have to find 4-connected areas amount using a wave algorithm.

object counting using image processing
Fig. 7. Labelled 4-connected areas

And it wraps it up! We have a solution for objects counting on production lines.

In practice, though, we face some additional problems such as stitching of the resulting images. It is necessary to take into account the objects that are between resulting images. This task is solved efficiently using mathematical morphology, too.

object counting using image processing
Fig. 8. Image stitching example:
a, b) two sequential resulting images, parts to be stitched are marked by colour; c) result of the stitching

The benefits of image processing techniques for object detection

The utilization of image counting software can provide significant advantages across industries and processes. If compared to traditional object detection, the solution might facilitate production tracking and accuracy.

Key benefits:

  • Increased accuracy and efficiency:
    • Modern-day image processing counting objects techniques (for example, slit-scan algorithms) can ensure 99.5% precision, even when products touch or appear in uneven product rows
    • Advanced image processing counting objects methods can process large datasets
  • Reduced time and cost – by eliminating human error and rework, a company might save on overall operational resources
  • More versatility – with image processing and object detection, the practical use cases are manifold, from pharmaceuticals to manufacturing
  • Greater scalability – with object counting using image processing, industry leaders can scale with ease, whether thousands of items or just a few

Real-world applications of using object detection methods in image processing

The impact of image counting software extends beyond production lines across industries and operations:

  • Food & beverage domain – with image counting software, you can ensure accurate packaging counting, minimize waste, and facilitate quality control
  • Retail & inventory segment – using image counting systems, you can automate stock-taking and avoid potential shortages and overstocking
  • Traffic monitoring – identify and count vehicles to optimize traffic flow
  • Security & surveillance monitoring – detect and track people for better crowd control
Computer vision and advanced image processing
Abto Software’s proven expertise
Learn more

Conclusion

The main goal of this article is to show a new promising approach to implementing an automated counting system for production lines. What are the main advantages of the suggested approach?

  1. Low computational cost
  2. High accuracy: the counting accuracy reaches 99.5%, as it is capable of separating touching products in uneven rows
  3. Flexible and universal solution: our automated counting system can be applied to counting other manufactured goods on the production lines, including Pharmaceutical products, Food & beverages, Can counting, Part and component counting, etc.

Overall, the system developed with this approach proved to be simple, flexible, and highly accurate and could be applied to solve issues in other areas, such as road traffic identification and monitoring. And here is a little sneak peek into our future post that touches upon video processing, and how can we apply it to road traffic analysis.

counting cars using image processing
Fig.9. Example of video processing for road traffic: a) a video frame with the red line as a line of interest;b) resulting image; c) background image estimation; d) detected foreground objects, namely cars, other objects were disregarded.

If you found this interesting, contact us by filling the form below.

Contact us

Tell your idea, request a quote or ask us a question