Digital Twin of Quick Service

The Digitalstates AI/ML, IoT, and Design Innovation teams have been tinkering with Nvidia Omniverse to generate syntactic data and create digital twins, with the goal of determining how best this can help our customers embrace the future and become more efficient and data-driven.

Digitalstates serves a wide range of industries and offers innovative solutions based on cloud, AI/ML, IoT, and modern data architectures. Digitalstates serves the following industries, among others:

• Healthcare
• Manufacturing
• Textile
• Oil and Gas
• Retail
• Quick Service Restaurants
• Education
• Commercial Real Estate

Fast-food restaurants were among the first to embrace innovation, leveraging AI and other technologies to enhance their premier channel. One of the world’s largest fast-food restaurant chains, specializing in fried chicken, desired to do just that by embracing new technology and adding value to its customers and business. The customer wanted to utilize video analytics to monitor and gain insights into traffic flow in their drive-throughs. The customer also wanted to roll out the technology in 22 markets in the Middle East and Asia. This introduced many challenges for building a customized but scalable and adaptable solution.

Digital Twin, Simulations, and Omniverse

Digitalstates and the customer performed design and brainstorming workshops which resulted in a far greater understanding of customer requirements and solutions capabilities. The vision got clearer and it was evident that it is no more a simple video analytics problem but we were required to tackle challenges including :

• Solving camera angle problems for different types of drive-throughs
• Number Plate recognition for diversified types
• Camera viability issues due to time of day, weather, and other environmental factors
• issues due to various car speed and driver behaviors

apart from computer vision and video analytics challenges, there were business challenges that were required to be addressed:

• Planning for new branches
• Reducing the wait time
• Personalizing the experience for its customers
• Realtime remote viability into operations
• Analytics and Insights
• Understanding dynamic behavior that is defined for each component, including process delays, congestion, reliability, schedules, and more.

After performing some initial proof of concept using video analytics it was proposed to build a Digital Twin solution and Nvidia Omniverse became the platform of choice as it solved multiple problems including synthetic data generation.

In recent years, the concept of digital twins has gained popularity across various industries, including manufacturing, architecture, and construction. A digital twin is a virtual replica of a physical object, system, or process that enables real-time monitoring, analysis, and simulation. With the advent of new technologies such as NVIDIA Omniverse, it is now possible to create digital twins of even more complex systems, such as drive-throughs, to optimize operations and enhance customer experience.

The Process of Digitization and Simulation

Let’s try to understand how we addressed the challenge. We started by initially building a deep learning-based video analytics solution. A common process to build video analytics includes following steps :

The first step for any computer vision or deep learning problem is to collect a huge amount of real data which in this case was required to be actual video footage from specific angles from drive-through points. This was an extremally big challenge and we hit roadblocks when deploying the POC from one branch to another.

This is where synthetic data comes in. Synthetic data is extremely helpful as it provides photo-realistic production-ready data sets which can be utilized to train the deep learning models. NVIDIA Omniverse Replicator is a tool that can be used to generate synthetic data for machine learning applications.

To generate synthetic data with Omniverse Replicator, we typically perform the following tasks:

1. First, create a 3D environment that resembles the real-world scenario. Creating 3D models of objects, buildings, and landscapes. Use Blender, Unity, Unreal or Autodesk 3ds Max to create models or use pre-built 3D assets from Omniverse library or providers like Sketchfab and ActorCore.

2. Second, let’s create and setup the scene in Nvidia Omniverse by importing the assets created in the previous step. Omniverse USD Composer (Formerly Create) is used to perform this step.

3. The next step is to setup the environment including the light settings, to simulate the scenario you want to generate data for. Once done we utilized the Nvidia Omniverse Replicator with a little bit of Python code to vary the assets, materials, lighting, and camera positions in your scene .

4. Once this is completed, we need to generate the variations with and without annotations and export them for tainting. This is done by using the export options in Omniverse Replicator.

5. The generated dataset is reviewed, labeled and loaded into our deep learning model for training and validation.

6. Finally, the updated model is loaded into inference servers for production.

The amount of data that Omniverse Replicator generates will depend on the settings that you choose. You can choose to generate a small dataset for testing purposes, or you can generate a large dataset for training a machine learning model. Here are some additional considerations for generating synthetic data using Omniverse Replicator:

1. Use a variety of objects and scenes. This will help to ensure that your machine learning model is trained on a diverse set of data.
2. Use realistic lighting and materials. This will help to make your synthetic data look more realistic.
3. Use a high-quality camera. This will help to ensure that your synthetic data is high-resolution.
4. Use a large dataset. The more data you have, the better your machine learning model will be trained.

The following video provides a brief overview of the transition from 3D Studio Max to Nvidia Omniverse.

Conclusion

In conclusion, Nvidia Omniverse is a powerful platform that enables the creation and management of Digital Twins. The platform provides a comprehensive set of tools and features for creating highly realistic virtual replicas of physical products and systems. The integration of physics-based simulation and advanced visualization capabilities makes Nvidia Omniverse an ideal platform for testing, optimization, and marketing purposes. The collaboration tools provided by the platform also make it easy for teams to work together on Digital Twin projects. As technology continues to advance, we can expect to see more companies adopting Digital Twin technology, and platforms like Nvidia Omniverse will play a critical role in enabling this transition.