Smart Logistics Mixed Reality Rapid Prototype

Recently, the logistics team, together with a partner, took part in a rapid prototyping workshop at the Microsoft Innovation Hub to validate the capture of barcodes from containers to guide frontline workers. The containers are tagged with a barcode (not a 2D QR code) and an RF ID. Containers are stapled to pallets delivered by trucks at the receiving dock for inbound triage. In larger DCs, fully automated intelligent inbound triage covers this process. For smaller DCs, however, the cost of implementing a similar fully automated system is too high. In this case, frontline workers do the receiving and triage for fine distribution based on the 'old' readable tag.

Frontline worker scenarios

Trucks are delivery pallets with containers of various sizes stowed at the distribution centre's receiving point. A frontline worker identifies the readable tag to decide where to deliver the totes for later transport. In some cases, the 'old' readable tag does not match the container barcode ID number or is simply missing. In this case, the frontline worker must read the barcode ID to obtain the detailed logistics information for collection and distribution.

Business objectives

• Eliminate the redundancy of the 'old' readable tag to reduce printing and assembly costs and reduce errors.
• Provide frontline workers with an intelligent guidance system to increase their productivity.
• Leverage existing barcode ID numbers as a single point of truth.

Our challenges

• RF ID does not work in all cases. For example, if the RF-ID of a tote doesn't work, the scanner will scan the RF-ID of the next tote nearby and the frontline worker will have no glue if the scanned tote is the one selected.
• The device must be mounted in such a way that the front-line worker can use his or her hands to pick one or more totes at the same time.
• The barcode ID number tag must be recognised in all environmental conditions. For example, the tag may be wet or dirty if it contains chilled food.
• The barcode ID number recognition needs to scan the container barn when the frontline worker walks in front of the barn to get distribution insights about the containers.
• HoloLens does support scanning QR codes (2D codes) out of the box without much latency. But it does not support barcode scanning at all.

Art-of-the-possible discovery

During the discussion, we reviewed six use cases to address the challenges and business objectives.

1. ID scan with HoloLens - Preferred option as it is hands free and provides guidance to the frontline working in a real world context.
2. QR code scan with HoloLens - Best option from a code recognition point of view, but still requires printing out an 'old' readable tag including the QR codes. No business justification.
3. Scan with smartphone - Requires a two-step approach to scan the container with one hand and hold the smartphone to pick up the containers with two hands.
4. Scan with smartphone and HoloLens - Combination of use cases 1 and 3 with increased cost.
5. Scan with Azure Kinect and HoloLens - Extend use case 1 to reduce latency to scan all containers for barcode ID number and share in context with HoloLens. If the latency of use case 1 and the experience of frontline workers does not reduce efficiency, this is the way to go.
6. Scanning with Azure Kinect (body tracking and audio) - Extending Use Case 5 without HoloLens works for a distribution centre with clear receiving lines.

Validated preferred use case 1

Scanning the barcode didn’t give us a good confident level, as in some cases the barcode image was not recognizable via image recognition (Azure Cognitive Services). But identifying the Migros Logo on the tag to identify the ID number below the barcodes give us a good confident level using image recognition. The team tested this with different pallets and stocked containers.

The test outcome shows the scanned and recognized ID numbers within the context of the stabled containers. It works!

An MVP should now prove the end-to-end solution by

• Bringing the image recognition model to the device.
• Pre-caching the distribution data on the device based on incoming scheduled pallets.

Increasing the speed of recognition and proving it in a real-world environment to validate the user experience.

Conclusion

This was a great team work experience together with a mixed Migros stakeholder team and a partner with deep understanding of mixed reality. Would like to thanks the and afca. ag team for this rapid prototype workshop at the Microsoft Innovation Hub.

Special thanks to the Migros Logistics team Paul Affentranger and Eliane Zihlmann from afca. ag for the great engagement.

References

• afca. ag - Pick-by-View