Visual sort assist system
Services:
Project overview
Our client, a well-known North America-based company, is an integrated package, freight, and logistics provider. The enterprise is a leading company in the logistics industry, with over 10K employees, more than 150 facilities, and over 1M packages delivered daily.
Our client was struggling with optimizing everyday operations, in particular – parcel sorting and distribution. Being handled traditionally manually, this process deteriorated overall business productivity, blocked efficient resource allocation, and, even more important, contributed to work-related accidents and injuries.
Abto Software has entered the cooperation to build a system for automatic parcel recognition and monitoring to empower the client to organize everyday operations more efficiently.
By utilizing computer-vision-based recognition and monitoring of multiple packages moving along conveyors, we aimed at increasing business productivity, decreasing time and cost, and facilitating workplace safety.
Main goals
Our client’s key objective was accelerating the rate of sorting by moving smaller batches of mail and parcels from the unloading area to appropriate delivery routes within warehouses with minimal manual intervention. This way, the enterprise would process higher volumes without employing and training additional workers, simultaneously streamlining business productivity.
We delivered a system for automatic package recognition and monitoring to provide:
- Improved productivity (splitter, pre-load, bulk sort)
- Enhanced time and cost (labeling, scanning, label positioning)
- Reduced training
- Reduced accidents & injuries
– No labeling in the unloading area
– Less congestion and hazards in the unloading area
How the solution works
To process high volumes of packages and mail, the enterprise must allocate significant resources to training. And, regardless, with today’s staff turnover, the company might experience reputational and financial damage by spending extra funds without achieving actual optimization.
Before cooperation, package sorting was managed in the following manner:
- The parcels were delivered to the operation facility
- In dependence of the process organization of the specific facility, the packages and mail were sorted either using wrist scanners or manually, by memory, which meant complete dependency on the employee’s skills
The client has already tested some ready-to-use applications to optimize package recognition and monitoring. Having discovered every application to have specific limitations – integration, and compatibility limitations, singulation requirements, and others – the company has changed its strategy and decided to design its own assisting system.
Our team suggested utilizing specialized hardware together with computer vision to implement a more efficient workflow:
a. After unloading, a scanner installed above detects the barcodes of the incoming parcels
b. After scanning, the system generates labels later used to guide the workers
c. As the incoming parcels are moving along conveyors, those details are projected onto them to highlight their type and appropriate delivery route
d. The workers distribute parcels using the projected instructions without spending additional resources on decision-making
Main challenges
Environmental conditions
To achieve the desired precision, we had to consider multiple variables:
- Lightning conditions (lightning variability severely affects the capability to accurately recognize labels, distinguish parcels, and correctly project information onto packages)
- Parcel characteristics (various shapes, sizes, colors, packaging materials, obscured barcodes, and other influential factors)
- Conveyor movement and speed
- Parcel positioning and orientation
Hardware equipment
To handle the inconsistent environmental conditions mentioned above, we had to conduct careful research before choosing barcode scanners, depth cameras, and projectors, which involved multiple challenges:
- Hardware compatibility
- Hardware reliability and durability
- Shipping and handling complexities
- Installation challenges
- Hardware scalability and future-proofing
- Integration with existing infrastructure
Device synchronization
Successfully implementing complex algorithms with synchronized hardware components (barcode scanners, depth cameras, and projectors) has presented multiple challenges:
- Hardware calibration
- Aligned timing and synchronization
- Spatial coordination
- Data processing
Device installation
When implementing computer vision, we faced another challenge – hardware placement over conveyors:
- Optimal positioning to achieve desired accuracy
- Comprehensive coverage to capture necessary areas
- Avoiding interference and obstructions
- Considering varying environmental conditions
- Maintenance access
- Worker safety and ergonomics
Reproducing realistic warehouse conditions
Conducting an investigation, we realized the implementation of the CV-based system must go beyond simulation. To obtain applied insight, we built custom conveyors and continued thorough testing outside conventional office settings.
Image processing
The implementation of a CV-based system might encounter significant challenges associated with time lags. Addressing these involves optimizing transfer channels, ensuring robust software and hardware integration, and, additionally, continuously monitoring the solution to maintain desired precision.
Technology stack:
- C++
- Python
- OpenCV
- Open3D
Timeline:
- Spring 2021 – Fall 2023
Team:
- 1 project manager
- 1 tech lead
- 4 software developers
- 1 QA engineer
Value delivered to business
By implementing computer vision, we designed a system for automatic parcel recognition and monitoring. Enabling the logistics enterprise to process higher volumes without employing and training additional workers, the solution can eliminate unjustified investment and facilitate business productivity.
Our concept is designed to provide:
- Increased productivity (splitter, pre-load, bulk sort)
- Strategically efficient resource allocation
- Less training
- Less accidents & injuries