Patent Application
20230179441
This invention relates to the field of smart infrastructure management - also known as Smart City solutions.
The field of infrastructure management has historically been limited to “silo-based” applications and operations, in which data captured through devices and human data entry stays in the applications in which it was captured or entered. Where integration across systems is required, custom solutions are made by solution integrators. These custom solutions are typically designed to serve trained operators, such as facilities managers or building engineers.
Since the advent of the internet, city system infrastructures have grown in sophistication and variety. During last decade, many internet-based devices have made their way to marketplace - ushering in the era of the Internet of Things (IoT), and building, building complex, neighborhood, town and city infrastructures are evolving from ‘systems and devices’ to the “Internet of Everything” (IoE) -which includes every device, system, network, application, cloud service, and even person, that is connected to the internet. Indeed, any person with a smartphone is now a node of the IoE. In order to be effective and efficient, system integration must no longer be merely between ‘systems’-- it must include IoE.
The widely growing adoption of digital technologies to manage cities, with the goal of enhancing performance of city operations, reducing cost and improving service to citizens (end-users of systems), demands a sophisticated approach to infrastructure management. These infrastructures include both physical systems such as energy and water supply, transportation, etc., and software systems such as supervisory and data acquisition (SCADA), maintenance management, lease management, etc.
The complexity of smart city implementation comes from two fundamental challenges: first, a smart city according to the invention is a ‘living system’ which must constantly evolve to serve city needs and to reflect constantly changing and moving citizenry/population; and secondly, people and skills involved in modeling city services are different from people and skills involved in system implementation and integration. That is, the smart city solution of the invention must be highly intuitive, as mobile users have a very short operation/attention timespan -- they expect a system to provide ONLY what they need to carry out their operation.
The present smart city solution addresses this by modeling ‘user journeys’- providing a template containing the anticipated exact steps (interactions with the system of the invention) that a particular type of user will require or carry out achieve an objective.
Building Information Modelling (“BIM”) software and files can present the entire structure and infrastructure of a building in a single file or file set representing the 3D environment of the building. BIM software and files allow a user to zoom in and ‘walk through’ a 3D visualization of a building, forward, backward, up and down (navigate in 3D space). Moreover, current BIM software and files and extend this visualization into a ‘time dimension’ by providing different ‘versions’ of drawing elements associated with ‘time’. This allows someone to stop in 10th floor (while navigating through 3D space) and take a walk through ‘time’ to see how, for example, the 10th floor was constructed step by step - or go towards future to see how it will look after construction is finished. BIM software and files with the time capability is referred to as 4D BIM.
BIM software and files that can also accommodate ‘materials and costing’ information to BIM drawing elements, allowing users to plan and optimize resource usage and logistics for undisrupted building construction process, are often referred to as 5D BIM.
6D BIM software and files include information such as Assets installed in the building, and recommended operating procedures, such as emergency evacuations, that allow building lifecycle management and sustainability.
The present invention implements a “metadata model of a city’s IoE,” and using the metadata model of the city’s IoE to implement citizen engagement through ‘user journeys’ and system intelligence through automated response logic. Furthermore, present invention extracts building structure and device information from conventional CAD BIM files to automatically populate and update the metadata model thereby providing building modeling functionality and and 3D-walkthrough capability for operations of building infrastructure. The improved metadata model, including building structure and device information extracted from conventional CAD BIM files, and the corresponding additional functionality, may sometimes herein be referred to as “Operational BIM-Activated Metadata Model” or simply ‘Operational BIM’. More specifically, the present invention provides a framework that allows system integrators to build a metadata model of a city’s IoE, which metadata model then enables city managers to implement user journeys and system intelligence, in incremental fashion, as the business needs of the city evolves.
The following is a summary of some of the features of the present invention:
A metadata model of a smart city having been built using the present invention, including all the devices, systems, subsystems, devices, networks, applications, data, etc., that are connected to the internet, the system receives metadata from a building control system reflecting a room temperature of 26 degrees, within normal parameters. However, the system of the present invention is monitoring metadata related to user-devices (mobile phones) of building occupants [metadata model] who are sending out Tweets indicating ‘office is hot.’ The system correlates the control system metadata concerning room temperature with metadata from other systems, i.e., mobile phones on the IoE [event processing & analytics], and the system qualifies the situation as “important to respond” and sends an SMS [unified communication] to the facilities manager of the building indicating that building occupants feel uncomfortably warm. The facilities manager opens his mobile app, and it will show a single page with the details of the problem, and also what system thinks the root cause is --- for example, perhaps an “energy saving” mode has automatically kicked in due to a sudden rise of dynamic energy price. The same mobile app page can give the facilities manager the option to turn off the energy saving mode, and will show how much extra it will cost the company, without the facilities manager having to navigate to various systems to gather such knowledge. This is an example of modeling a user (in this case, a facilities manager) journey.
Accordingly, there is provided according to the invention the following features:
Accordingly, there is provided according to the invention a system for building a meta-data model representing the infrastructure of a community, including buildings, building systems, building devices, building wide and tenant specific enterprise applications, (etc.), and using that meta-data model for the integrated management of the infrastructure of a community, where the infrastructure includes a plurality of network-connected infrastructure systems, subsystems, devices, and applications, and where the systems, subsystems, devices and applications are independent and unintegrated, the system including:
There is further provided according to an embodiment of the invention a system wherein the community is a city.
There is further provided according to an embodiment of the invention a system wherein the community is a single building.
There is further provided according to an embodiment of the invention a system in which one or more buildings are represented by BIM files generated during building construction which indicate various equipment and systems in the building and their relationships to one-another which may be used as a source of metadata
There is further provided according to an embodiment of the invention a system wherein the community is a collection of buildings.
There is further provided according to an embodiment of the invention a system wherein the community is a workplace or single-tenant space within a multi-tenant building where some subsystems (such as air conditioning) belong or are operated by the building and/or building management, while other subsystems (such as card access & CCTV) are owned and/or managed by the tenant.
There is further provided according to an embodiment of the invention a system according to which cross references are established between existing metadata nodes.
There is further provided according to an embodiment of the invention a system wherein the event engine is configured to build the dynamic metadata map automatically, continuously, and dynamically, modifying the metadata map as collected metadata changes.
There is further provided according to an embodiment of the invention connector software module that periodically scans evolving BIM files and compares updated versions to earlier versions of the same BIM file in order to extract any modified metadata and send said metadata to the event engine for processing.
There is further provided according to an embodiment of the invention a system wherein the event engine is configured to use metadata relationships to extract metadata from existing nodes or from subsystems to build and update metadata nodes.
There is further provided according to an embodiment of the invention the ability of the metadata model to publish information for third party systems and applications by means of API (application programming interface) to receive and exchange metadata.
There is further provided according to an embodiment of the invention a system wherein the dynamic metadata map/model contains metadata notes for a plurality of infrastructure systems, subsystems, devices, and/or applications that are involved in a service workflow, wherein a service workflow comprises a set of pre-selected information that presented to an end user and said predefined operation sequences.
There is further provided according to an embodiment of the invention a system wherein the predefined operation sequences comprise an end-to-end service workflow for end-users of the system by having metadata model undertake integration with multiple subsystems involved in the service workflow.
There is further provided according to an embodiment of the invention a system further including an analytics engine configured to identify and correlate events and to update existing events or create new events based on identified correlations.
There is further provided according to an embodiment of the invention a system wherein the event engine is configured to send commands to one or more of said infrastructure systems, subsystems, devices, and applications based on the receipt of one or more event conditions received by the event engine.
There is further provided according to an embodiment of the invention a system wherein the network comprises the Internet.
There is further provided according to an embodiment of the invention a system wherein the network comprises only private networks.
The present invention may be used in any community (workplace, building, building complex, neighborhood, town, or city) that might benefit from the integrated management of infrastructure systems, the use or operation of which impacts or is impacted by devices, systems, networks, applications, and persons (smartphones) in the IoE. “Workplace” as used herein refers to a single-tenant space within a multi-tenant building where some subsystems (such as air conditioning) belong or are operated by the building and/or building management, while other subsystems (such as card access & CCTV) are owned and/or managed by the tenant. However, the invention is most useful in the case of a city wide IoE or other large collection of devices, systems, and networks, the members of which (in particular the people using smartphones) are constantly in flux, moving within the system, and moving into and out of the system. Using the case of a Smart City as an example, the system of the invention includes a metadata map or “model” that is made up of metadata nodes that represent the IoE -- each internet-aware device/entity in the city. Since this comprehensive metadata model includes people with smartphones who are constantly moving around and going in and out of the city, also referred to herein as the solution boundary, the metadata model is adapted to be constantly, dynamically, and automatically changing as the available metadata changes. According to a preferred embodiment, there will be only one metadata map for the entire city with millions of metadata nodes connected to each other directly or indirectly.
The metadata model is made up of metadata nodes. Each node is a collection of metadata. For example, ‘Meeting Room One’ is a Facility-type metadata node, and it carries metadata such as ‘Location’, ‘Seating Capacity’, ‘Amenities’, etc. Other types of metadata nodes include: Visitor, Staff, Equipment, Work Order, and Room Booking. In the case of Work Order and Room Booking types of metadata nodes, these metadata nodes represent transactions that are dynamically introduced while system is in operation. Further, a Room Booking metadata node may contain metadata of Facility (meeting room), Staff (host) and Visitor (attendee). Visitor-type metadata nodes might be identified by a telephone number or an email address; in this way, a visitor’s mobile phone constitutes a physical representation of the visitor, which in turn is represented in the invention as a metadata node on the system with, for example, the mobile phone’s telephone number or email address(es). Likewise, a Staff-type metadata node might also be identified by a telephone number, email address, name, organization, department and/or work location. An Equipment-type metadata node might include Asset-ID, Asset Category, Installed Location, Serving Locations (where it serves, which may be different from installed location), Assignee (User), Related Equipment, Make, Model and/or Serial number. A Work Order type metadata node might include Work Order ID, Service Category, Location, Work Description, Assigned Vendor/Technician, Supervisor and/or Deadline to Complete.
According to a preferred embodiment of the invention, a significant source of metadata for building of the metadata model is Building Information Modelling files, with the more advanced BIM (i.e., 5D and 6D) being more preferred; however 3D and 4D BIM files may be used.
Constructing and modifying of metadata nodes is done by an ‘event engine’. It receives metadata in the form of ‘events’. Events are generated by one or more Connector modules, described below.
Another exemplary event might be ‘Visitor V001 arrived at Gate G001’. Upon receiving new or modified metadata, the event engine either creates new metadata nodes or modifies existing nodes to absorb newly received information into metadata model. In another example, a staff member in the office may be represented by a Staff metadata node carrying two metadata (among others): Staff ID and Current Location.
Event filtering and processing is also done by the event engine, based on the logic defined in metadata model template. The metadata model is configured to allow a user to establish an ‘action sequence’ that defines how engine will process incoming events.
A central feature of the invention is a metadata node template, which is defined by the user (e.g., system integrator) and which defines how a metadata node should represent an entity, how metadata nodes should be connected to one-another. A metadata node template is a passive element that describes the contents of a node, which contents constitute instructions to the event-engine for assembly of the nodes and the relationships between them (the metadata nodes and relationships together constituting the metadata model/map). All instructions to the event engine go into the template which is editable by system integrator to modify behavior of the entire system.
The user builds the metadata template using a metadata model editor, see
Referring to
Thus, the Metadata map or model is the mesh or network of metadata nodes (built using metadata organized according to metadata note templates), which nodes are “connected” using the relationships that are reflected in the metadata. For example, Staff-JohnSmith node forms a metadata map (on a microscale) with Building-1251ndustrialDrive node when CurrentLocation metadata (of Staff-JohnSmith) carries the reference to Building-125IndustrialDrive. According to a preferred embodiment of the invention, the metadata map encompasses an entire city, with millions of metadata nodes constructed using collected metadata and assembled using metadata node templates, and the relationships between which are also defined by common metadata elements between metadata nodes.
Referring to
Alternatively, certain subsystems may be programmed to ‘push’ their metadata to the event engine directly, for example using their own Connectors configured to communicate with the system of the invention.
Examples of subsystems according to the invention include: air conditioning and heating systems, elevator systems, card access systems, CCTV systems and BMS (building management systems). Applications are addressed in the same way as any other subsystem. Mobile devices (such as a temperature sensor inside transportable refrigerator) report their status to a central server, and a Connector specific to that device’s system will pick it up from the central server. With respect to IoT-devices, they carry an IP-address that allows a Connector to directly address them. Smartphones and Networks are not considered a subsystem, although, a smartphone could represent a device or system through a connector (app) installed in the smartphone - for example, a connector placed in the smartphone could detect the proximity of iBeacon thereby determine current location of the Visitor and notify event engine.
In the above example, the Connector may receive information reflecting that Visitor V001 is leaving the building, i.e., Visitor V001’s entry/exit card was presented at a security device-monitored exit point. At that point, Connector will raise ‘Visitor V001 left’ event. The event engine will correlate this event to existing ‘Visitor In Building’ node for V001 and make the necessary updates to reflect the current status.
Referring to
For example, Location metadata (‘Room 001’) of equipment (‘A001’) gets attached to Location (‘Room 001’) attribute (metadata). Location hierarchy makes it immediately inherit parent location (‘Floor 001’) of Room 001. In this way, for example, any service that is looking for ‘photocopy machines’ (equipment of category ‘photocopy’) at Location ‘Floor 001’ will now find A001. The metadata model is a machine readable structure that does not require a human to browse through it.
The invention allows system implementation to take place progressively. When the system (the metadata map) is expanded by adding new metadata node templates and connectors, the system of the invention generates/builds cross references between new nodes and existing nodes, without having to modify existing nodes. Referring, for example, to
As explained in the example above, ‘people count’ node has a list of people (both employees and visitors) who are currently in floor F001. The present invention allows a system integrator to define a single action for an event associated with a metadata node, e.g., “notify emergency evacuation” to everyone in F001 in the event that metadata is received reflecting an emergency situation, for example, a terrorist threat, fire or smoke, regardless of the silo that captured their presence on floor F001, whether it was Room Booking systems managing visitors, Work Order system managing contractors, or Space Management system managing seating locations for internal staff, and/or regardless of what silo captured the existence of an event, whether it was twitter feed from personal devices, email applications, a building/workplace fire detection/suppression system, and/or any other system or application.
In some cases, it is required to automatically execute a series of actions, upon receiving an event. For example, People Count node may also be programmed to send SMS notification to all ‘persons’ in the particular floor when evacuation is ordered. More importantly, additional metadata that would identify a person as ‘handicapped’ could enable people count node to send a notification to facilities managers about locations of handicap persons in the building with instructions to provide special handling and assistance during the evacuation.
The invention includes an analytic engine that is able to process events and event---data to identify patterns and correlations. Examples are as follows:
The present invention also processes events through analytics, allowing identifying meaningful events - both threats and opportunities, therefore enabling “operational intelligence.” For example, when one smoke detector reports a fire in a particular location, a new metadata node may be initiated to build necessary connections between the city’s metadata elements and to orchestrate certain actions such as notification to first responders. A few minutes later, more smoke detectors (in that Location) will likely report a fire. The system’s response to 2nd and 3rd smoke detected events should be different from the first event. They act as “confirmation” to first event, rather than initiating new fire---incident scenarios.
Similarly, additional metadata can be appended to an event to add “more sense.” In the same example above, if the first smoke detected alarm came from a detector that has produced a ‘false alarm’ a week ago, it would be wise to look for a 2nd confirmation from another detector, or a human confirmation, before the orchestration is carried out to respond to a fire.
According to a preferred embodiment of the invention, operational intelligence produced through this invention is ‘machine readable’. As a result, it allows predetermined orchestrations to be executed based on complex event conditions. This allows operators to manage the infrastructure predictively. For example, in a large space such as the lobby of a shopping mall, it takes a long time for the air-conditioning system to react when more shoppers come into the mall. This is due to the natural delay in warmed air reaching air-conditioning ducts on the roof, where sensors are placed to recognize the rising temperature. The present invention allows system integrators to setup an orchestration that acts on ‘people count’ that is monitored through an independent system to influence air- conditioning system to produce more air-conditioning, predictively.
Scanner - Extracting Metadata from CAD BIM. See
The Scanner is a special Connector with specific knowledge of the CAD BIM file format. Different scanners are built for popular CAD BIM formats.
The Scanner is responsible for following primary tasks:
When the Scanner updates the Metadata Model, it sends metadata extracted from the CAD BIM file, it also associates two additional drawing metadata to identify the elements that represent a device or piece of equipment: BIM File ID and Element ID. These two pieces of information are used to ‘bind’ metadata nodes with the CAD BIM file’s drawing elements.
When scanning for ‘changes’ in the CAD BIM file relative to a scan of an earlier version of the same file, the Scanner uses this binding to match CAD BIM elements with metadata nodes of the invention.
The Scanner can be deployed either as a ‘manually executable’ process (to update on demand) or as an ‘automated background process’ to periodically synch Metadata Model with changes in CAD BIM.
Deployment architecture and exact procedure for identifying and extracting metadata from each type of CAD BIM file is different, but that is public knowledge, published by the manufacturer of the BIM design software under ‘SDK’ - software development kit.
As shown in
As shown in
The API (Application Programming Interface) comes in multiple forms:
The Asset Register carries information about Assets and related information necessary for asset lifecycle management (maintenance/warranty/insurance information) and also for servicing of occupants with those assets (assignment/ownership/billing information).
Typically, this data is manually entered into the computer system. Although most of Asset Register software allows ‘data import’ electronically, such data should be collected and imported in one go. However, the building construction phase is very long and this information do not arrive at the same time to allow ‘full data import’. Also, some of the data changes often during operations.
The following example illustrates the significance of using Operational BIM-Activated Metadata Model for populating the Asset Register.
As part of construction phase, the physical details of an Asset (say, wall mounted TV) such as Installed Location, Make, Model and Serial Number will get populated in Operational BIM.
As a new Asset is added to the metadata model (say, imported from CAD BIM), the Asset Register will get notified about the new Asset through the connector (that connects Asset Register to Metadata Model), so it will extract information about the new Asset, automatically.
At a later point, when the accounting system updates the TV’s warranty certificate, the Connector for the accounting system will extract the metadata (such as Vendor, Expiry Date, etc.) and will generate an event and transmit it to the Metadata Model server, which allows the Metadata Model to update the Asset’s metadata. This action in turn will trigger a notification to Connectors that listen to changes in the Metadata model. The Connector that represents the Asset Register will grab that information and update it again, automatically.
Once the building goes into operation and Tenants are coming in, a leasing agreement is signed with a Tenant for leasing of the location where the TV is installed. This agreement transfers the Asset’s ownership to the Tenant for the duration of the leasing agreement. The Connector picks up the metadata about the new assignment (Tenant, effective date, etc.) and send it to the Metadata Model by raising an event.
This enables the Metadata Model to update its metadata (assignment) and also to make a new ‘metadata relationship’ between the Asset (TV) and the Metadata Node representing the Tenant.
When an operator searches for ‘assignments’, this relationship comes up thereby listing the TV as an asset assigned to this Tenant.
The SCADA system of the building (often known as “BMS” - building management system) requires information about equipment (assets) in order to communicate with them. Two key pieces of information are: point-address (how to address a particular operational attribute) and protocol (how messages should be crafted such that device will understand).
Often, this data is manually entered into SCADA system when operations are setup. These are very ‘cryptic’ information and small mistake of a single character could result in a disaster. Therefore, it is highly desirable to automatically obtain this data from ‘system commissioning’ repository where this information is already captured in electronic form and verified during handing over phase.
As the building goes into the system commissioning phase, real-time communication with the equipment is established. When testing and commissioning a Light, for example, commissioning agent will verify that system is able to read the Light’s brightness and operating status (ON or OFF), and also will verify the system’s ability to change the brightness the light and turn it on/off.
The Metadata involved in this communication (how to address the equipment, what data to be sent to the device to perform control, etc.) gets stored in the Operational BIM-Activated Metadata Model. The process of metadata capture is the same as explained in previous section -‘automated construction of asset register’.
When a user wants to monitor the status of the Light, or to control the Light, the SCADA system will use the metadata extracted from Operational BIM to figure out how to communicate with the device, without having to ‘manually configure’ it.
Apart from allowing operators to monitor and control devices on demand, the SCADA system is also responsible of ‘keeping track’ of health of equipment and systems. For example, if room temperature rises beyond 27 degrees while the AC-Unit’s status is ON, it could indicate an equipment failure. This is known as ‘alarm’ and operators should be notified about it. Often, this notification is automated based on ‘maintenance contract’ information, and will be directly routed to the maintenance vendor.
However, if this alarm occurs while system maintenance is performed in that location, it could be part of their testing work. Therefore, it is desirable to ‘mask’ such false alarms to avoid unwanted confusions.
With access to Operational BIM, the SCADA system is able to find if the equipment that generated the alarm is under maintenance at that point, hence mask the alarm notification.
Incident Management is one of the critical operations where ‘related information’ has an enormous value. The following scenario illustrates how Operational BIM-Activated Metadata Model can be used.
Referring to
The Fire system is programmed to enunciate the fire (activate strobes and sirens) in two minutes unless the building operator acknowledges the fire alarm by clicking a button on the fire panel (equipment mounted on the wall) or through the SCADA system. Normal practice is to acknowledge the alarm from control centre through the SCADA system.
Once the alarm is acknowledged, the fire system is programmed to extend the enunciation deadline by another 5 minutes to allow building operators to clear the smoke or disable the detector - if it is due to a false alarm.
This five-minute window is very critical, especially where false alarm could cause a heavy business damage (say, in a shopping mall), or when it has a potential to create a life-safety issue due to panic (say, in a hospital), or when it could create a security threat by opening unwanted door locks (say, in a prison or airport).
As described in section ‘Scanner - Extracting Metadata from CAD BIM’, asset metadata in Operational BIM also contains drawing metadata of CAD BIM.
With that, an operator is able to click on the smoke detector that generated the alarm and automatically zoom in to view the location where smoke is detected in. Then, the operator finds that there is a camera in that area. With Operational BIM’s ability to execute a process, the operator can zoom/pan the camera to the smoke detector’s location and visually verify that there is no smoke.
As the operator ‘walks through’ the 3D BIM, he may also notice there are two other smoke detectors in that room. With Operational BIM’s metadata relationships, the operator can quickly monitor the real-time status of the other two smoke detectors, and double-confirm that it is a sensor failure.
This allows the operator to disable a smoke detector to prevent enunciation of false alarm, and dispatch a technician to replace it.
In an alternative situation of the above smoke detection scenario, if it happened to be an actual fire, the operator sitting in the control center needs to quickly find out who are the building operators near the fire area to execute a guided evacuation of building occupants. See
Operational BIM has been dynamically forming metadata relationships as indoor positioning data picks up presence of pre-registered personnel (building operators, fire wardens and others who are trained to assist public). As the incident management dashboard opens, it extracts metadata from Operational BIM, and it will assemble the most appropriate list of people to form the first responder team.
When systems are commissioned, commissioning agents run various procedures to ensure systems work according to operational specifications. However, with time, these systems could deteriorate thereby reducing operational efficiency, lifespan and safety of people around them. This problem is solved by repeating these test procedures at predetermined time intervals — a procedure known as ‘continuous commissioning’. One well known example is fire drills where the entire system - including occupants’ ability to respond - is tested.
However, this requires complex logistics. First, it is necessary to dispatch operators and safety wardens to appropriate locations to handle any unexpected situation. Secondly, occupants need to be notified to ensure they do not overreact to the service breakdown. Thirdly, the SCADA system that is monitoring the building needs to be put on standby to avoid undesired notifications to go out to people and 3rd party systems.
As such, continuous commissioning requires so many people with specific knowledge of each vertical; therefore; the performance of certain required activities are sometimes neglected.
This problem is significantly reduced with the Operational BIM-Activated Metadata Model which maintains the central repository of all information in the building, and more importantly, their relationships.
For example, when an elevator’s backup battery has to be tested, the procedure requires cutting elevator power supply during its move, and check if it safely was brought down to nearest floor and the door is opened.
With Operational BIM, the operator has access to the elevator’s CCTV feed to ensure there are passengers inside, has the list of duty personal available in the building at that point in time to ensure they are alerted about the procedure to keep an eye, and access to SCADA systems alarm configuration to temporary disable them. See, e.g.,
| Number | Date | Country | Kind |
|---|---|---|---|
| 10201702888X | Apr 2017 | SG | national |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 16603461 | Oct 2019 | US |
| Child | 17939302 | US |
