Patent Application
20250074355
This application claims priority to EP Application Serial No. 23194219.4 filed Aug. 30, 2023, the disclosure of which is hereby incorporated in its entirety by reference herein.
The present disclosure relates to a chip architecture, in particular, a chip architecture for incorporating metaverse in automobile cabins.
The concept of metaverse started to expand in 2021, stemming from the fusion of the words “meta” and “universe”, implying an existence that transcends our current universe. This expansion of the metaverse signifies the transition from a primarily physical existence to a predominantly digital life, culminating in a world of the Internet of Things (IoT) that seamlessly blends the virtual and the tangible.
Metaverse unifies a multitude of virtual realities, augmented reality, and the vast realm of the IoT via a single platform or multiple interfaces, to create experiences for gaming, online meetings, education, sales, entertainment, healthcare, and even basic necessities like food, clothing, transportation, and leisure. Virtually everything in the daily lives can integrate with metaverse.
One of the many potential uses of the metaverse is the automobile sector. As the advent of completely autonomous vehicles would be expected in the next decades, metaverse is predicted to drastically shape in-car information and entertainment systems, creating a world where one can superimpose digital content onto the external environment or entirely transform the car windows to depict an alternate universe. Such a revolution could dramatically boost traveling experience of passengers.
A first aspect of the present disclosure relates to a chip architecture for metaverse incorporated automobile system, comprising: an input module to provide other modules in the chip architecture with one or more inputs; an output module to output data received from other modules in the chip architecture to various actuators (in the system); an object module to manage data and/or structures relating to objects in the metaverse incorporated automobile system, in particular currency, payments, history, or any other item belonging to the metaverse; a view module to present and/or visualize the data/structures of the metaverse incorporated automobile system, and realizes user interactions; a control module to control the data and/or structures of the object module and the view module; and a communication infrastructure to enable exchange of information across the modules. The chip architecture according to the present disclosure provides a safe and secure infrastructure for automotives, in which metaverse use cases or applications are incorporated.
In the present disclosure, a metaverse incorporated automobile system refers to an automobile system which incorporates metaverse in the cabin of the automobile, to provide passengers onboard with metaverse experience. The metaverse refers to a virtual reality space where users can interact with a computer-generated environment and other uses in real time. It is typically visualized as a collective virtual shared space, including the sum of all virtual worlds, augmented reality, and the internet. Metaverse involves creating digital spaces that allow for social interaction, commerce, entertainment, and potentially even new forms of work and education.
Such chip architecture meets the massive real-time rendering, processing, and networking requirements of the metaverse, and enables users to have seamless metaverse experience, such as gaming, shopping, meeting, learning, or the like, in a cockpit, or passenger sits of an automobile.
Preferably, the chip architecture may further comprise a safety module to manage automotive functional safety of the system. The safety module may process and implement a series of protective measures tailored to the automotive domain. These measures are meticulously designed to address potential risks associated with the operation of the chip, such as any malfunction, system failure, or unanticipated behavior which could result in hazardous situations or compromised vehicle operations. The safety module may detect anomalies early enough to prevent or reduce their impact.
More preferably, the safety module may fulfill the ISO 26262 standard. ISO 26262 is an international standard for functional safety of electrical and/or electronic systems that are installed in serial production road vehicles (excluding mopeds), defined by the International Organization for Standardization (ISO). By adhering to the ISO 26262 standard, the chip architecture can ensure the highest levels of safety in automotive electronic systems.
Preferably, the chip architecture may further comprise a security module to minimize and/or eliminate potential cyber-attacks. The security module takes different measures to control the flow of data to minimize or eliminate the impact of a potential cyber-attack.
More preferably, the security module may be designed and implemented to adhere to the requirements of ISO/SAE 21434 standard. ISO/SAE 21434 is a standard for cyber security in motor vehicles. By complying with ISO/SAE 21434, the security module ensures that it identifies potential threats, assesses the risks, and implements robust countermeasures with regard the metaverse incorporated automobile system.
Preferably, the chip architecture may further comprise an artificial intelligence/machine learning (AI/ML) module to support the other modules to carry out specific tasks. The AI/ML module may constantly learn, adapt, and evolve based on the vast data streams the AI/ML module encounters, offering predictive insights, pattern recognition, and real-time decision-making capabilities. The AI/ML module may be kept separate from the other modules, so that in case of any issue (either technical or legal), the AI/ML module can be switched off while still keeping the other modules active, as well as keeping the use cases/applications active.
Preferably, the input module may comprise a gesture recognition submodule to interpret specific hand or body movements; a natural language processing submodule to processes human language; and a feedback submodule to confirm user inputs. During development, each submodule can be developed, tested, and maintained independently, simplifying the development process. Furthermore, as requirements change or grow, new submodules can be added or existing ones can be extended without overhauling the entire input module.
Preferably, the object module may comprise an avatar management submodule to handles data associated with user avatar; a currency management submodule to manage creation, distribution and transaction of currency in the metaverse; and a payment management submodule to manage various payment gateways in the metaverse. Other submodules can be added to the object module as needed according to actual requirements.
Preferably, the communication infrastructure may be a bus system, a matrix, a network or point-to-point communication. The choice of communication concepts depends on specific requirements and scale of interactions, taking into account calculation power, data transfer rate or the like of the system.
Preferably, one or more of the modules are realized as IP cores or blocks, microcontroller units, system on chip or a single chip.
A further aspect of the present disclosure relates to a Central Control Unit (CCU) of a metaverse incorporated automobile system, the CCU comprises the chip architecture as described above. By integrating the chip architecture into the CCU, the core processing capability is centralized, leading to potentially faster data processing and system response. Furthermore, latency can be reduced, and cost can be saved.
A further aspect of the present disclosure relates to a zone of a metaverse incorporated automobile system, the zone comprises the chip architecture as described above. By integrating the chip architecture in a zone of the metaverse incorporated automobile system, high-performance operations tailored to specific needs of that zone can be achieved, ensuring efficient and rapid data processing. The systems can be designed more scalable, easier to upgrade, and simpler to troubleshoot.
A further aspect of the present disclosure relates to a metaverse incorporated in an automobile system, the system comprises the chip architecture as described above.
A further aspect of the present disclosure relates to an automobile, the automobile comprises the chip architecture as described above.
A further aspect of the present disclosure relates to a method of designing a chip architecture for metaverse incorporated automobile system, comprising: providing an input module to provide other modules of the system with one or more inputs; providing an output module to output data received from other modules the chip architecture to various actuators; providing an object module to process all data and structures related to objects in the metaverse incorporated automobile system, in particular currency, payments, history, or any other item belonging to the metaverse; providing a view module to present and/or visualize the data/structures of the metaverse incorporated automobile system, and realizes any user interaction; providing a control module to control data or structures of the object module and the view module; and providing a communication infrastructure to enable exchange of information across the modules.
In the following description, various embodiments of the present disclosure will be presented. It should be understood that these embodiments are provided by way of example only and are not intended to be limiting. The embodiments described herein can be combined in any suitable manner. The scope of the present disclosure is defined in the independent claims, and all modifications and equivalents that fall within the scope of the claims are intended to be embraced therein.
With the rise of metaverse, it is anticipated that in-car information and entertainment systems will be drastically shaped. As the boundaries between the digital and physical worlds blur, vehicles could become an extension of this interconnected universe. For example, a windshield with augmented reality features can offer immersive navigation to the passenger, where directions are overlaid directly onto the road. Also, the windshield can provide the passenger 102 with virtual reality experiences such as joining a meeting from the vehicle, watching a live concert, or engaging in an online gaming. This fusion of the metaverse with vehicular systems can provide a more engaging and multi-dimensional travel experience.
In some embodiments, the metaverse incorporated automobile system 100 may include a virtual reality or augmented reality headset 104 that the passenger 102 can wear. Once inside the metaverse, the passenger 102 could watch movies in a virtual cinema, play immersive games, attend virtual meetings, or learn new things in a virtual environment. This essentially transforms the vehicle into a moving extension of homes, offices and schools, offering endless possibilities for productive or entertaining uses of time during travelling.
In some embodiments, the metaverse incorporated automobile system 100 may function as a virtual co-pilot, which is an artificial intelligence powered (AI-powered) entity that provides real-time guidance and information about a journey. The virtual co-pilot may keep passengers updated on routes, estimated arrival time, and any adjustments made due to traffic or road conditions. It may also provide updates on the status of the vehicle, including fuel or battery levels, maintenance needs, etc. The virtual co-pilot may interact with the passenger 102 via voice commands or through a virtual interface in the metaverse.
In some embodiments, passengers traveling in separate vehicles can share a common metaverse environment to interact and communicate with each other during the journey. The passengers may play virtual games together, watch the same movie in a shared virtual cinema, or simply chat in a more immersive way than a phone call would allow.
In some embodiments, automakers may create virtual showrooms within the metaverse, allowing potential customers to explore and customize vehicles in a 3D environment. The customers may even take virtual test drives, experiencing the vehicle's handling, interior features or the like, all within the metaverse. This may revolutionize the vehicle shopping experience, making it more immersive and convenient.
It goes without saying that the applications of incorporating metaverse in automobile systems are non-exhaustive, which are not limited to the embodiments listed above. While some potential uses are mentioned, the convergence of metaverse and automobile opens the door to endless possibilities. Collaborations between automotive engineers and metaverse developers can drive both industries forward in ways that enrich the user experience and redefine the very concept of travel.
To incorporate metaverse in automobile cabins, specific hardware components are needed.
In the chip architecture 200, the hardware module input IF 202 represents an input interface for the system 100. The module input IF 202 is configured to input data into a virtual, augmented, or mixed reality environments of the metaverse. The module 202 takes all inputs from the automobile and provides them via a communication infrastructure 218 to the other modules in the chip architecture. The inputs can be for example from motion sensors, haptic sensors, voice recognition sensors, eye-tracking sensors, brain-computer interfaces, cameras, radars, lidars, or the like. The module 202 may also include submodules such as gesture recognition submodule, which interprets specific hand or body movements; natural language processing submodule, which processes human language and enabling users to converse with virtual entity of the metaverse using speech, or feedback submodules, which confirms user inputs, ensuring that the system has actually received the user's intended actions.
The chip architecture further includes an object module 216. The object module 216 is configured to manage, analyze and process diverse data structures associated with virtual objects in the metaverse, such as avatars, currency, payments, history, or any other item belonging to the metaverse, to ensure the integrity and consistency of the retrieved data. In some embodiments, the object module includes an avatar management submodule, which handles data associated with user avatars; a currency management submodule, which manages the creation, distribution and transaction of currency; and a payment management submodule, which manages various payment gateways relating to purchasing of virtual goods or services.
A view module 214 is provided in the chip architecture to present and visualize data or object from the object module 216 to a user. The primary function of the view module 214 may be to render avatars, currency symbols, or other virtual items and entities in a visually appealing manner, adapting to the user's environment, preferences, and device capabilities. The view module 214 may also allow users to interact with the visual representations. It may perform real-time updates to reflect changes in the state of the objections, or allow users to customize how they view certain elements, giving users control over aspects such as layout, information density, color scheme, etc.
The chip architecture 200 may further include a control module 212 to control the data or structures of the object module and the view module. The control module 212 processes user inputs, such as commands, gestures, or voice inputs, translating them into actions within the metaverse, like moving an avatar, initiating transactions, or customizing settings. The control module 212 may take inputs from the input IF module 202 for its control procedures and provides output data for an output IF module 204 based on the outcome of the control procedures.
The chip architecture may further comprise an output IF module 204. The output IF module 204 outputs the data received through the communication infrastructure from other modules and sends it to various actuators, such as Heating, Ventilation, and Air conditioning (HVAC), brakes or the like, or user interfaces, such as the In-Vehicle Infotainment (IVI), cluster or any other (data) receiver in the automobile network. Once data is received via the communication infrastructure 218 from other interconnected modules, the output module 204 outputs it to a variety of actuators and user interfaces designed for specific functions. For instance, it might relay essential commands to HVAC systems, ensuring the desired cabin temperature is maintained, or to braking systems, enabling responsive control and safety.
In some embodiments, the chip architecture 200 may further comprise a safety module 206. The safety module 206 may be provided to oversee and manage automotive functional safety considerations. The safety module 206 may process and implement a series of protective measures tailored to the automotive domain. These measures are meticulously designed to address potential risks associated with the chip's operation. For instance, any malfunction, system failure, or unanticipated behavior of the chip architecture 200 could result in hazardous situations or compromised vehicle operations. The safety module 206 may detect anomalies early on, and either prevent or reduce their impact. Several layers of safety protocols might be embedded within this module. These can range from continuous system health checks, monitoring of data integrity, and ensuring the synchronization of processes, to more advanced measures such as failover mechanisms, redundancy provisions, and emergency response actions.
In some preferred embodiments, the safety module 206 may be designed and implemented to adhere to the requirements of ISO 26262 standard. ISO 26262 is an international standard that addresses the functional safety of electrical and electronic systems in production automobiles.
In complying with ISO 26262, the safety module 206 undergoes rigorous testing, validation, and certification processes. This involves various stages, from the conceptualization of its design to its integration into the larger system, ensuring its resilience against potential hazards and malfunctions. By adhering to the ISO 26262 standard, the chip architecture can ensure the highest levels of safety in automotive electronic systems. This entails not just detecting and managing irregularities or faults, but also having predefined mechanisms and strategies in place to counteract unforeseen events or failures. For instance, the module would incorporate fail-safe mechanisms, redundancy protocols, and self-check capabilities, all aligned with the standard's requirements. Furthermore, to maintain compliance, the performance of the safety module would be periodically reviewed, tested, and updated as necessary. Any modifications, enhancements, or updates to the module would also be subjected to a thorough safety assessment, ensuring that the module consistently aligns with the evolving demands of the ISO 26262 standard.
In some embodiments, the chip architecture 200 may further comprise a security module 208. The security module 208 controls the flow of data to minimize or eliminate the impact of a potential cyber-attack. The security module 208 may comprise a multi-layered security mechanism to fortify against potential cyber threats. For example, advanced intrusion detection systems are used to monitor the network traffic against any malicious patterns. Data, both in transit and at rest, is encrypted using the latest cryptographic standards, such as Advanced Encryption Standard (AES), Elliptic-curve cryptography (ECC), Digital Signature Algorithm (DSA) or the like, to safeguard user information and in-world transactions.
In some preferred embodiments, the security module 208 may be designed and implemented to adhere to the requirements of ISO/SAE 21434 standard. This international standard ISO/SAE 21434 defines the comprehensive requirements and guidelines for cybersecurity risk management pertaining to road vehicles and their interfaces. By complying with ISO/SAE 21434, the security module 208 ensures that it identifies potential threats, assesses the risks, and implements robust countermeasures with regard the metaverse incorporated automobile system.
In some preferred embodiments, the chip architecture may further comprise a machine learning (ML) module 210. The AI/ML module 210 may support other modules to carry out specific tasks. It may constantly learn, adapt, and evolve based on the vast data streams it encounters, offering predictive insights, pattern recognition, and real-time decision-making capabilities. For example, the AI/ML module 210 may refine avatar interactions based on user preferences to optimizing virtual economies or enhance the realism of simulations. The AI/ML module enhances the user experience and drives the continuous evolution of the metaverse, ensuring it remains adaptive, immersive, and engaging for all users. The AI/ML module may be kept separated from the other modules, so that in case of any issue (either technical or legal), the AI/ML module can be switched off while still keeping the other modules active, as well as keeping the use cases/applications active.
The communication infrastructure 218 of the chip architecture 200 enables the exchange of information (data) across the modules. For example, the communication infrastructure 218 can transfer user commands to avatars in real-time, in an efficient, and secure way. The communication infrastructure 218 may be designed to handle varying data loads, ensuring latency-free experiences. It may also incorporate security protocols to protect user data and transactions, safeguarding against potential cyber threats. The communication infrastructure 218 may be kept versatile, allowing the communication infrastructure 218 to adapt to various architectural designs and technological demands. Depending on the specific requirements and scale of interactions, the communication infrastructure 218 may be realized through various communication concepts. The communication infrastructure 218 may manifest as a bus system, wherein multiple modules and systems share a single communication line, ensuring a simplified yet effective data transfer mechanism. Alternatively, the communication infrastructure 218 may be designed as a matrix, allowing for more complex data routing and multi-dimensional connectivity. Still alternatively, for broader and more distributed scenarios, it may be designed as a network, facilitating interconnectivity across disparate metaverse regions or platforms. Additionally, point-to-point communication may be used in cases where high-speed data transfer is required.
In the present disclosure, each module may be kept as flexible and open as possible. For example, one or more modules may be realized as hardware IP cores or blocks. IP cores or blocks are reusable units of logic or functionality that can be integrated seamlessly into different designs, allowing for optimized, dedicated functions and high-speed performance. Such a realization is particularly beneficial for tasks that demand real-time responses or heavy computational power. Alternatively, one or more modules may be realized as a Microcontroller Unit (MCU). MCUs are compact, integrated systems that come with their own memory, processor, and I/O peripherals. MCUs are particularly suitable for applications that need both control and moderate computational capabilities, balancing efficiency with versatility. The one or more modules may also be realized as System on Chip (SoC). SoCs integrate all components of a computer or other electronic system into a single chip. This provides immense power, as the SoC encapsulates multiple functionalities and components, like processors, memory, and more, into one unified system, making the SoC ideal for more complex tasks and multi-faceted operations. Alternatively, one or more modules may be realized as a standalone computer chip. Such realization may maximize computational capacity, flexibility, and scalability, and ensure high performance. The SoC is especially suitable for tasks that demand extensive processing resources, multitasking, and intricate operations. In essence, by allowing each module to be realized diversely, maximum flexibility in architecture can be ensured.
In the present disclosure, based on specific requirements, the chip architecture 200 may be realized as a part of a Central Control Unit (CCU) 302 of an automobile 300, as depicted in
As can be seen from
| Number | Date | Country | Kind |
|---|---|---|---|
| 23194219.4 | Aug 2023 | EP | regional |
