Developing Digital Twin for Quick Service

Developing Digital Twin of Quick Service Drive Through Omniverse

Digitalstates has created digital twins for various industries, which includes the following:

  • Healthcare
  • Manufacturing
  • Textile
  • Oil and Gas
  • Retail
  • Quick Service Restaurants
  • Education
  • Commercial Real Estate.

Here are some of the key reasons why customers adopt the concept of the digital twin:

  • Design and Development: Digital twins allow organizations to simulate, model, and optimize their products, systems, or processes before physical implementation. This helps in identifying design flaws, improving efficiency, and reducing costs.
  • Performance Monitoring and Maintenance: Digital twins provide real-time monitoring of physical assets or systems. By collecting data from sensors and other sources, organizations can gain insights into the performance, condition, and health of their assets. This enables predictive maintenance, early fault detection, and optimization of maintenance schedules, leading to reduced downtime and improved reliability.
  • Simulation and Testing: Digital twins facilitate virtual testing and simulation scenarios that may be difficult or costly to carry out in the physical world. This allows organizations to evaluate different operational scenarios, test changes, and assess the impact of various factors without disrupting real-world operations.
  • Remote Monitoring and Control: Digital twins enable remote monitoring and control of assets, processes, or systems. This is particularly useful in industries such as energy, manufacturing, and infrastructure, where assets may be in remote or hazardous environments. Operators can access and control the digital twin from a central location, enhancing safety, efficiency, and productivity.
  • Decision Support and Optimization: By integrating data analytics and machine learning techniques, digital twins can provide valuable insights and decision support tools. They can help optimize processes, predict outcomes, and suggest improvements based on historical data and real-time information.
  • Training and Education: Digital twins offer a realistic and immersive environment for training and education purposes. They allow users to simulate complex scenarios, learn from virtual experiences, and acquire skills without the risks associated with physical assets or systems.
  • Lifecycle Management: Digital twins can span the entire lifecycle of a product or system, from design and development to operation and maintenance. They enable organizations to track and manage assets throughout their lifecycle, facilitating better planning, resource allocation, and optimization.

    Overall, digital twins provide a virtual representation of physical entities, enabling organizations to make informed decisions, improve performance, and drive innovation across various domains. Fast-food restaurants were among the first to embrace innovation, leveraging AI and other technologies to enhance their premier channels. One of the world’s largest fast-food restaurant chains, specializing in fried chicken, desired to do just that by embracing new technology and adding value to its customers and business. The customer wanted to utilize video analytics to monitor and gain insights into traffic flow in their drive-throughs. Additionally, the customer aimed to roll out the technology in 22 markets across the Middle East and Asia. This introduced several challenges in building a customized, scalable, and adaptable solution.

Digital Twin, Simulations, and Omniverse

Digitalstates and the customer performed design and brainstorming workshops, resulting in a far greater understanding of customer requirements and solution capabilities. The vision became clearer, and it was evident that it was no longer just a simple video analytics problem; we were required to tackle challenges, including:

  • Solving camera angle problems for different types of drive-throughs
  • Number plate recognition for diverse types
  • Camera viability issues due to time of day, weather, and other environmental factors
  • Issues caused by various car speeds and driver behaviors.

Apart from the computer vision and video analytics challenges, there were business challenges that needed to be addressed:

  • Planning for new branches
  • Reducing wait times
  • Personalizing the experience for customers
  • Real-time remote visibility into operations
  • Analytics and insights
  • Understanding dynamic behavior defined for each component, including process delays, congestion, reliability, schedules, and more.

After performing some initial proof of concept using video analytics, it was proposed to build a Digital Twin solution, and NVIDIA Omniverse became the platform of choice as it addressed multiple problems, including synthetic data generation.

In recent years, the concept of digital twins has gained popularity across various industries, including manufacturing, architecture, and construction. A digital twin is a virtual replica of a physical object, system, or process that enables real-time monitoring, analysis, and simulation. With the advent of new technologies such as NVIDIA Omniverse, it is now possible to create digital twins of even more complex systems, such as drive-throughs, to optimize operations and enhance the customer experience.

The Process of Digitization and Simulation

Let’s try to understand how we addressed the challenge. We began by initially building a deep learning-based video analytics solution. The common process for building video analytics involves the following steps:

I. The first step for any computer vision or deep learning problem is to collect a large amount of real data, which in this case required actual video footage from specific angles at drive-through points. This posed an extremely significant challenge, and we encountered roadblocks when deploying the POC from one branch to another.

II. This is where synthetic data comes in. Synthetic data is incredibly helpful as it provides photo-realistic, production-ready datasets that can be used to train deep learning models. NVIDIA Omniverse Replicator is a tool that can be utilized to generate synthetic data for machine learning applications.

To generate synthetic data with Omniverse Replicator, we typically perform the following tasks:

I. We create a 3D environment that resembles a real-world scenario by creating 3D models of objects, buildings, and landscapes. We can use tools like Blender, Unity, Unreal, or Autodesk 3ds Max to create models or utilize pre-built 3D assets from the Omniverse library or providers such as Sketchfab and ActorCore.

II. Set up the scene in NVIDIA Omniverse by importing the assets created in the previous step. We use Omniverse USD Composer (formerly Create) for this step.

III. Set up the environment, including the light settings, to simulate the scenario for which we want to generate data. Once this is done, we utilize NVIDIA Omniverse Replicator along with a little bit of Python code to vary the assets, materials, lighting, and camera positions in the scene.

IV. Generate variations of the scene with and without annotations and export them for training. This can be done using the export options in Omniverse Replicator.

V. Review the generated dataset, labeled it, and loaded into our deep learning model for training and validation.

VI. Updated model is loaded into inference servers for production.

The amount of data generated by Omniverse Replicator will depend on the settings you choose. You can opt to generate a small dataset for testing purposes or a large dataset for training a machine learning model. Here are some additional considerations for generating synthetic data using Omniverse Replicator:

  • Utilize a variety of objects and scenes to ensure your machine-learning model is trained on a diverse set of data.
  • Employ realistic lighting and materials to enhance the realism of your synthetic data.
  • Use a high-quality camera to ensure your synthetic data is high-resolution.
  • Aim for a large dataset, as the more data you have, the better your machine learning model will be trained.

The following video provides a brief overview of the transition from 3D Studio Max to NVIDIA Omniverse.

Conclusion

In conclusion, NVIDIA Omniverse is a powerful platform that enables the creation and management of Digital Twins. The platform provides a comprehensive set of tools and features for creating highly realistic virtual replicas of physical products and systems. The integration of physics-based simulation and advanced visualization capabilities makes NVIDIA Omniverse an ideal platform for testing, optimization, and marketing purposes. The collaboration tools provided by the platform also make it easy for teams to work together on Digital Twin projects. As technology continues to advance, we can expect to see more companies adopting Digital Twin technology, and platforms like NVIDIA Omniverse will play a critical role in facilitating this transition.