Patent Grant
9919426
The present invention relates to a system, method, computer program and data signal for the registration, monitoring and control of machines and devices. Embodiments of the invention find specific, but not exclusive, use in the registration, monitoring and control of robotic devices, autonomous vehicles, “smart” devices, and other programmable and computer controlled devices.
The following discussion of the background art is intended to facilitate an understanding of the present invention only. The discussion is not an acknowledgement or admission that any of the material referred to is or was part of the common general knowledge as at the priority date of the application.
Science fiction predicted the eventual development of robotic devices and “smart” devices which are arranged to autonomously perform one or more functions. In the past, due to limitations in computing power and the ability to create reliable, cost-effective electronics, robotic devices have largely been used in very specialized applications (such as in manufacturing applications) or as “show-pieces” (e.g. the development of ASIMO, a humanoid robot developed by Honda Corporation).
However, the explosion in and frenzied development of telecommunications and computing technology (such as the development of cell phone technology, the Internet, wireless Internet, the release of Global Positioning System technology for consumer use and miniaturised computing technology) now provides a platform for the development and creation of consumer robots.
For example, robot vacuum cleaners, small remote controlled robotic devices (e.g. helicopters and multicopters) and the more recent development of autonomous ‘self-driving’ vehicles are examples of practical and increasingly accessible robots and smart devices available to average consumers.
It is against this background that embodiments of the present invention have been developed.
In a first aspect, the present invention provides system for controlling at least one robotic device, comprising a computing device capable of communication with at least one robotic device and arranged to receive at least one command from a command module, the command being arranged to contain at least one instruction which is arranged to effect an operation on the robotic device and identification information to identify the at least one robotic device, wherein the computing device includes a processor and a database, the processor being arranged to receive the command and review the command against information in the database to determine whether the command is suitable for execution by the at least one robotic device, wherein the command is provided to the robotic device if the command is suitable for execution.
In one embodiment, the processor determines whether the command is associated with at least one authorisation code.
In one embodiment, the at least one authorisation code is received independently of the at least one command.
In one embodiment, the processor determines whether the command is one of a predetermined set of commands by accessing a set of predetermined commands stored in the database.
In one embodiment, wherein at least one of the at least one command, the authorisation code and the identification code is encrypted.
In one embodiment, the processor decrypts the at least one of the at least one command, the authorisation code and the identification code prior to reviewing the command to determine whether the command is suitable for execution.
In one embodiment, at least one of the at least one command, the authorisation code and the identification code includes a checksum, wherein the checksum is utilised to determine the correctness of the at least one command, the authorisation code and the identification code.
In one embodiment, the robotic device is a programmable device.
In one embodiment, the robotic device includes at least one processor arranged to receive and execute the at least one command.
In one embodiment, the robotic device is capable of performing at least one physical function.
In a second aspect, the present invention provides a system for controlling a robotic device, comprising a computing device capable of receiving at least one instruction, a processor capable of generating a command based on the at least one instruction, wherein the command is communicated via the computing device to initiate a response based on the at least one generated command.
In one embodiment, the processor requests further information to further assess the instruction, prior to initiating a response.
In a third aspect, the present invention provides a method for controlling a robotic device, comprising the steps of, receiving at a computing device at least one command arranged to effect an operation on the robotic device, reviewing the command to determine whether the command is suitable for execution, wherein the command is provided to the device only if the command is suitable for execution.
In one embodiment, the step of reviewing the command includes the step of determining whether the command is associated with at least one authorisation code.
In one embodiment, the at least one authorisation code is received independently of the at least one command.
In one embodiment, the step of reviewing the command includes the further step of determining whether the command is one of a predetermined set of commands.
In one embodiment, the invention provides the further step of the computing device receiving at least one identification code arranged to identify the robotic device.
In one embodiment, the invention provides the further step of receiving the identification code with the at least one command.
In one embodiment, at least one of the at least one command, the authorisation code and the identification code is encrypted.
In one embodiment, the invention provides the further step of decrypting the at least one of the at least one command, the authorisation code and the identification code prior to reviewing the command to determine whether the command is suitable for execution.
In one embodiment, at least one of the at least one command, the authorisation code and the identification code includes a checksum, wherein the checksum is utilised to determine the correctness of the at least one command, the authorisation code and the identification code.
In a fourth aspect, the present invention provides a system for controlling a robotic device, comprising a computing device in communication with the robotic device and arranged to receive at least one command which is arranged to effect an operation on the robotic device, wherein the computing device reviews the command to determine whether the command is suitable for execution, and the command is provided to the device only if the command is suitable for execution.
In a fifth aspect, the present invention provides a computer program including at least one command, which, when executed on a computing system, is arranged to perform the method steps in accordance with the third aspect of the invention.
In one embodiment, the invention provides a computer readable medium incorporating a computer program in accordance with the fifth aspect of the invention.
In a sixth aspect, the present invention provides a data signal encoding at least one command and being arranged to be receivable by at least one computing device, wherein, when the encoded command is executed on the computing system, the computing system performs the method steps in accordance with the third aspect of the invention.
Further features of the present invention are more fully described in the following description of several non-limiting embodiments thereof. This description is included solely for the purposes of exemplifying the present invention. It should not be understood as a restriction on the broad summary, disclosure or description of the invention as set out above. The description will be made with reference to the accompanying drawings in which:
a are example systems in accordance with an embodiment of the invention;
The present invention relates generally to a system, method, computer program and data signal for the registration, monitoring and control of machines and devices. In particular, embodiments of the invention relate to the registration, monitoring and control of robotic devices, autonomous vehicles, “smart” devices, and other programmable and computer controlled devices.
In more detail, one aspect of the embodiments described herein provides a method for controlling a robotic device. The method comprises the steps of, receiving at a computing device at least one command arranged to effect an operation on the robotic device. When the command is received, it is reviewed to determine whether the command is suitable for execution and is directed to the correct robotic device. The command is only provided to the device if the command is suitable for execution and directed to the correct robotic device.
In other words, one broad aspect of the embodiments described herein provides a system for controlling and monitoring the commands issued to autonomous or “smart” devices. Such a system is particularly useful for situations where the autonomous or smart devices are to be operated in a public space, where inappropriate operation of such devices may pose safety, security and financial risks to other members of the public.
One embodiment of the method is codified in a computing system, such as the computing system shown at
In
With reference to
The server 100 includes instructions that may be installed in ROM 104, RAM 106 or disc drives 112 and may be executed by the processor 102. There may be provided a plurality of communication links 114 which may variously connect to one or more computing devices 110 such as servers, personal computers, terminals, wireless or handheld computing devices, or mobile communication devices such as a mobile (cell) telephone. At least one of a plurality of communications links 114 may be connected to an external computing network through a telecommunications network.
In one particular embodiment the device may include a database 116 which may reside on the storage device 112. It will be understood that the database may reside on any suitable storage device, which may encompass solid state drives, hard disc drives, optical drives or magnetic tape drives. The database 116 may reside on a single physical storage device or may be spread across multiple storage devices.
The server 100 includes a suitable operating system 118 which may also reside on a storage device or in the ROM of the server 100. The operating system is arranged to interact with the database and with one or more computer programs to cause the server to carry out the steps, functions and/or procedures in accordance with the embodiments of the invention described herein.
Broadly, the invention relates to a computing method and system arranged to interact with one or more remote devices via a communications network. The remote devices may take the form of computing devices as described above, but may also take the form of robotic devices, as will be described in more detail later.
The system, in one embodiment, utilises a server including a database arranged to contain biometric or other identifying information regarding one or more entities. The database is arranged to receive the information via the communications network from the one or more remote devices and to subsequently communicate information to one or more remote robotic devices.
Other aspects of the broad inventive concept relate to a corresponding method, computer program, computer readable media and data signal. The method facilitates the transfer of commands regarding the desired instructions to be sent to an autonomous or “smart” device (also referred to as a “robot” device) between one or more remote devices and a centralized database. The centralized database receives a request to provide the command to the one or more remote devices, and forwards the information via a communications network to the one or more remote robotic devices.
Initial Interaction with the System
For a user to interact with the system in one embodiment, it is necessary for the user to identify themselves and register with the system. This is achieved through a registration process that is analogous to many other consumer product registration processes, such as the registering of a vehicle.
Preferably, a user may be required to prove or verify their identity by undertaking an identification check.
In one embodiment, prospective users (‘prospective registrants’) are required to set up a “profile account” or obtain an “eLicence”. For the purpose of the broader invention described herein, a “profile account” or “eLicence” is any type of digital and/or electronic identifying means utilised to verify the identity of a user. In a co-pending application filed by Digital (ID) entity Limited, a Hong Kong company, novel and inventive embodiments of eLicence and Profile Accounts are described in more detail and are incorporated herein by reference.
The user uses their eLicence or Profile Account (along with other identifying information, such as a password) to connect or log-in a device (such as device 110) to a registry server such as server cluster 100a (
Once the user is registered with the system, the user then registers their robotic device. As with a vehicle or boating license, the user, preferably, only has one license, but is able to register a plurality of robotic devices, as the user may have more than one robotic device.
Command and Control System
Each robotic device includes an internal secure computing device which is arranged to validate, authenticate and execute commands which are generated either from/by the robotic device itself or by external parties.
That is, each robotic device includes an internal “logic” where, before an action is taken (whether physical or otherwise), the robotic device firstly receives an “Intention”. The Intention must then be Validated against some internal logic or rules (i.e. a policy). Once the Intention is Validated, the Intention can then be Communicated to the relevant section of the robotic device and consequently Approved.
The Command and Control (CC) structure is now described in more detail with reference to
Referring now to
When a robotic device 108a receives or generates a request (intention), the request originates from an “intention” application 110a. The intention application passes the intention request from the intentions application 110a to a validation controller (not shown). The validation controller ensures the robotic device and software have not been tampered with. The validation controller also ensures that all the physical components of the robotic device are in working order and approved for use within policy guidelines.
The request is then encrypted and transferred over a secure protocol (Virtual Private Network (VPN) connection 112a) to the server cluster 100a. All data packets are encrypted before transmission using key pair authentication or any other suitable encryption methodology, as may be required.
Once connected the robotic device establishes a secure Virtual Private Network (VPN) tunnel over a public communications mesh such as a mobile telecommunications network 114a, which may utilise a 3G (3rd Generation GSM standard) and/or 4G (4th Generation GSM standard) cellular network standard.
All communication to the server cluster 100a is via a secure firewall service 116a that limits VPN endpoints and ports that are available within the VPN. An appropriate standard, such as Secure Socket Layer (SSL) is utilised for the tunnel.
Once packets sent by the intention application (via the communications manager application) are encrypted and the VPN communication is secured and passes through the firewall, the robotic device authenticates to establish a connection with the CC system (i.e. the server cluster 100a). In the embodiment described herein, an authentication server 118a is utilised to authenticate the robotic device, through the exchange of certificates, although it will be understood that other authentication methods or systems may be utilised.
The CC system 100a further includes an application platform 120a that manages communication, requests, manifest distribution and ultimately the control of the robotic device. In the context of the embodiment described herein, no action can be executed by the robotic device without first passing through the application platform 120a.
The application platform 120a interfaces with a policy engine 120b as the capabilities and allowed actions of each registered robotic device are stored as policies within the system 100a. The policy engine 120b allows each robotic device to be controlled uniquely as required by law, device capabilities and end user requirements. Policies are transmitted to the robotic device via the server cluster 100a as a base set of guidelines that cannot be breached. The workings of the application platform and the policy engine are described in more detail below.
The CC system is also responsible for recording an audit trail of all actions taken and all data received from the end device. This includes data such as the make and model of the robotic device, the capabilities of the device, flight manifests (where the robotic device is capable of flight), previous approvals for flight or movement, GPS movement and position data, including associated times and dates, instrument metrics, etc. Such information is stored in the Data Archive 122a, so that it may be accessed if necessary for auditing purposes.
A website 124a is provided as an interface to administer the application logic and all associated data/metadata. This includes information such as updating device policies, capabilities manifests, and other services, including, special services such as “ghosting” and “shrouding” (which are described in more detail below).
A fast, high Input/Output (10) data retention service 126a caches incoming data feeds from the robotic device. Once cached the data is subsequently moved to the Data Archive 122a where it can be data mined as required for auditing purposes.
Returning to end users 102a, a web interface provides access from both a client device 104a and a mobile client device 106a. Through these interfaces the end user is able to securely instruct a robotic device to take actions, receive feedback on the status of the device and track results. All communication into the client device 104a and mobile client device 106a are secured via standard secure web protocols and firewall access 128a.
Once a connection is established the consumer may authenticate against the platform using password or biometric exchange to prove their identity, as previously described (i.e. using a “Profile Account” or “eLicence”). Depending on policies set by the end user and the platform the authentication can be more or less strict as required.
All end user (customer) data is secured and protected within a series of databases generally denoted by computing system 130a. In one embodiment, users may be required to pay for use of some services provided by the server cluster 100a. In such an embodiment, end users can carry out purchases or cash transactions via a standard, secure payment gateway 132a. This service is in the form of a shopping cart style interface with account management.
The end user interacts with the web application to execute robotic device commands and receive feedback. This application provides a series of interfaces and logic for robotic device administration.
At no time does the end user have direct access to a robotic device. While an end user would provide a series of policies about their robotic device and what they wish the device to do in certain instances these policies are not applied directly, but are vetted by the policy engine.
That is, all policies are exchanged and validated with the policy engine 120b to ensure that server cluster 100a has ultimate control of the device. As such, the server cluster 100a utilizes a “Command and Control” type structure to control the movement and action of robotic devices, to prevent unauthorised or illegal use.
Internal Robotic Device Validation
Each robotic device is capable of performing a predefined set of actions (physical or otherwise) and each action in the set of actions are associated with one or more instructions. Taken together, these allowable actions and associated instructions form the “policy” for the robotic device.
For example, a robotic drone (pilotless flying device) may be capable of flying in any direction, but may be limited to flying only in certain predefined air space, due to privacy, security or safety reasons.
One set of instructions necessary for the robotic device to be allowed to perform a command would be to undertake a diagnostic test each time the robotic device is activated.
The system would include a range of ‘tests’, beginning with regular (e.g. fixed date) tests. However, as problems can occur between regular checks (e.g. consumers may unintentionally or unknowingly damage their robots), it will be understood that diagnostic tests may also occur at random time intervals.
In addition to performing tests, test data can be gathered and in one embodiment, test data is communicated to the server cluster 100a as shown in
An example of the types of data collected during tests is described below:
The system's approved diagnostic tests are undertaken (completed) by an object or component written for the express purpose of performing one or more of the following operations:
Once the system receives confirmation that all registration, diagnostic and/or health tests have been successfully completed, the system issues relevant clearance codes to allow the robotic device to perform the function.
Command and Control Structure in More Detail
In other words,
The Server 300 includes various modules and databases which provide functionality that enables robotic devices to be monitored, controlled and managed in order to uphold, for example, safety, security and privacy considerations. A short summary of each module is provided below:
301 Administration Module
An Administration Module 301 is provided to transmit system administrative function commands to one or more robotic devices. The commands can include commands that would be common to a number of robotic devices, including powering on and off, setting access controls, software updates, performance monitoring, statistic gathering and/or other administrative functions not directly related to diagnostic tests, clearance codes and/or policing ‘rogue’ robots.
302 Communications Module
The system further provides a Communications Module 302 which enables communication between the robotic devices and the Registry's Server 300, and/or other registry servers (not shown).
Further, in another use of the Communications Module 302, clearance codes (that may be generated, outputted or stored in a Clearance Code Database 331) may be transmitted to users' robots, devices or vehicles.
Additionally, the Communications Module 302 may also facilitate communications with the Server 300 by other entities (or the same entity) to facilitate operations, activities or tasks such as:
A Transaction Module 303 is employed to process financial transactions to pay for services provided by the Server 300 or associated, related third party.
In one embodiment, a Transaction Module 303 is responsible for issuing or processing product and/or service subscription accounts for users, owners or clients, which may be listed in a User/Owner/Client Account Database 321. Such subscriptions may be for the initiation or execution of exclusion zones or privacy constraints (not limited to shrouding or ghosting).
Moreover, the Transaction Module 303 may be responsible for issuing or processing fines, infringements or penalties in relation to, for example, not limited to, inappropriate, unauthorised, unregistered, illicit, illegal or unruly use, control, management or ownership of a robot, device or vehicle. Such fines, infringements or penalties may be communicated to the relevant party or parties using a Tasks/Activities/Programs Module 312, a Seizure Protocol Module 309, and/or a Communications Module 302.
304 Controller
In some embodiments, the modules and databases listed operate autonomously. However, in the embodiment described herein, a central controller 304 is utilised to provide a “supervisory” function. The central controller may operate autonomously or may be controlled by a user, to “override” the autonomous functioning of any individual module. For example, if a breach or compromise of the system is detected, a user may use the controller to override any given clearance code or other permission given to a particular robotic device.
305 Proposed & Active Assignments/Functions & Travel Plans & Tracks Module
A Proposed & Active Assignments/Functions & Travel Plans & Tracks Module 305 is responsible for a variety of processes or tasks, including receiving, analysing, storing and outputting, as necessary, an end user's proposed robotic device commands.
The Proposed & Active Assignments/Functions & Travel Plans & Tracks Module 305 utilises data or information stored in a User/Owner/Client Account Database 321 or an Ineligible Database 322 to confirm that a proposed assignment or function is permissible, i.e. can be authorized.
The Proposed & Active Assignments/Functions & Travel Plans & Tracks Module 305 also utilises data or information stored in a Registered Robot Database 323 to determine, whether a particular class, model or type of robotic device possesses the hardware, software or capabilities (including functional or operational) to undertake and successfully complete a proposed assignment, operation or function.
The Proposed & Active Assignments/Functions & Travel Plans & Tracks Module 305 also confirms that proposed or active assignments, operations or functions are permissible or authorised according to information in the Robot ‘Apps’/Functions Database 325 and/or information in the Operational Spaces Database 326, which contains approved, ‘restricted’ or disallowed assignments, operations or functions.
The Proposed & Active Assignments/Functions & Travel Plans & Tracks Module 305 may consult with, check and/or confirm information and data contained or stored in one or more of a Server's 300 databases or modules for the purpose or running any necessary test such as those that may be conducted by a Diagnostic Tests Database 327.
306 Clearance Code Module
A Clearance Code Module 306 allows the generation of suitable authorisation codes, for the purposes of allowing or approving a user robot, device or vehicle to undertake or successfully complete an assignment, operation or function.
In some instances, the Clearance Code Module 306 may cause a robot, device or vehicle to perform another type of assignment, operation or function that may not have been proposed or initiated by a user.
Where there is a requirement to perform a diagnostics test prior to conducting a task, the Clearance Code Module may be instructed by a Tasks/Activities/Programs Module 312 following the successful passing of the diagnostic test.
307 Exclusion/Privacy Protocol Module
An Exclusion/Privacy Protocol Module 307 may be a Server 300 component that is responsible for processing all privacy-related matters such as, but not limited to, exclusion zones, shrouding and ghosting, which may otherwise be known as privacy constraints.
In one embodiment, the Exclusion/Privacy Protocol Module 307 includes a web-based interface which allows users to access or interact with available server tools or features leading to the creation, setting up, amendment, removal and/or payment of a privacy constraint and/or a subscription or an associated user account. Such an account may be listed in or stored by a User/Owner/Client Account Database 321. The Exclusion/Privacy Protocol Module 307 may communicate with a User/Owner/Client Account Database 321 when necessary to allow, for example, a user to create, set up, amend or remove an account which may only exist for the purposes of enacting privacy constraints that may be conveyed to other's robots, devices or vehicles for execution or implementation. The Communications Module 302 may facilitate the connection between a user's (remotely located) device that is used to connect to a Server's 300 Exclusion/Privacy Protocol Module 307. The Communications Module 302 may also facilitate the distribution of privacy constraints to other Servers and/or user robots, devices or vehicles.
The Exclusion/Privacy Protocol Module 307 may also impose changes to a Robot ‘Apps’/Functions Database 325, for example, by altering or amending aspects of robot apps or functions, not limited to disallowing robots to travel into or within a particular space when executing a particular assignment, operation or function.
In this context, an Operational Spaces Database 326 may be altered to reflect changes to travel spaces.
In another instance, an Exclusion/Privacy Protocol Module 307 may communicate with a Diagnostic Tests Database 327, not limited to the following, for the purposes of altering, amending, analysing, reviewing and/or confirming that a Diagnostic Tests Database 327 would appropriately and accurately instruct or command a robot, device or vehicle to perform all necessary tests before, during or after an assignment, operation or function with respect to any existing, or newly imposed changes to, privacy constraints listed in or stored on an Exclusion/Privacy Protocol Module 307.
For example, a user may form or create a new privacy constraint that may pose a particular challenge or be a different set of parameters not previously dealt with by a robot, device or vehicle. Accordingly, amendments and/or additions are made to relevant or applicable diagnostic tests on a Diagnostic Tests Database 327 which would cause all relevant or applicable robots, devices or vehicles to undertake or successfully complete an updated diagnostic test when next required.
The Exclusion/Privacy Protocol Module 307 may communicate with a Payload Database 328 for the purpose of forming a new or altering or amending an existing list of authorised payloads that is carried in, on or by a robot, device or vehicle. Certain privacy constraints may dictate which robots, devices or vehicles can carry or transport particular payloads. Authorised payloads may be dictated also by any restrictions placed upon, a user, which is listed in the User/Owner/Client Account Database 321.
The Exclusion/Privacy Protocol Module 307 may also communicate with a Surveillance Database 330 for the purpose of altering or amending authorised and unauthorised surveillance areas. Further to a Surveillance Database 330, an Operational Spaces Database 326 may be utilised for the same purpose.
The Exclusion/Privacy Protocol Module 307 also communicates with a Profiles Database 332 for the purpose of implementing any privacy constraints that may involve Profiles. For example, a user may upload to a Server 300 using their robot, device or vehicle, with the assistance of a Communications Module 302, a new or updated Profile to a Profile Database 332, with any updates to a Master Profile that relate to a privacy constraint communicated with an Exclusion/Privacy Protocol Module 307 which is then distributed out to any relevant or applicable robots, devices or vehicles.
308 Rendezvous (Sensor/Scanner) Protocol Module
A Rendezvous (Sensor/Scanner) Protocol Module 308 is a Server 300 component responsible for processing all rendezvous protocols (not limited to the example of a ‘Sense/Scan’ operation). These protocols may relate to the relocation of robots, devices or vehicles to specified locations. Rendezvous protocols may be formed, set up, amended or removed from one or more Server 300 databases or modules, not limited to a Rendezvous (Sensor/Scanner) Protocol Module 308 by either the registry party, users, governing bodies or other stakeholders or third parties.
Using the ‘sense/scan’ rendezvous protocol as an example scenario, such a protocol utilises a software application that executes on dedicated or shared hardware or other peripherals, which cause robots, devices and vehicles to perform, in one example, a predefined operation. Further, a Rendezvous (E.g. Sensor/Scanner) Protocol Module 308 may communicate with one or more databases such as a User/Owner/Client Account Database 321 or a Registered Robot Database 323 to determine which users, owners or clients require their robots, devices or vehicles to be tested or examined by sensors or scanners.
The Rendezvous (E.g. Sensor/Scanner) Protocol Module 308 may also communicate with a Proposed & Active Assignments/Functions & Travel Plans & Tracks Module 305 in order to plan, program, calculate, anticipate or expect which, when or where robots, devices or vehicles may be ‘called upon’ (appropriately) for the purposes of a rendezvous protocol. For example, a robot may be in the vicinity of a sense/scan station and, thus, an opportune moment to activate or initiative the relevant rendezvous protocol.
In another example, using a practice of concealing (final) rendezvous locations or positions, the Rendezvous (Sensor/Scanner) Protocol Module 308 when activating a robot, device or vehicle for sensing or scanning may communicate with a robot, device or vehicle using a Communications Module 302 in order to regulate its surveillance or tracking capabilities, particularly, with respect to data that may be viewed or stored by an unauthorised party.
An Operational Spaces Database 326 or a Surveillance Database 330 may provide (the most up-to-date) privacy constraints that may need to be obeyed to protect the confidentiality, for example, of a sensor/scanner station's location or position. An Operational Spaces Database 326 may also have a station's location or position updated from time to time.
Certain clearance codes are generated by a Clearance Code Module 306 or retrieved from, or confirmed against a Clearance Code Database 331. Clearance codes sent may be the data signal which causes robots, devices or vehicles to initiate or execute a particular assignment, operation or function either for the purposes of this instance or another.
In another example, the Rendezvous (Sensor/Scanner) Protocol Module 308 may lead to a positive detection result of a robot, device or vehicle which may then cause another protocol, for example, a ‘seizure protocol’ run by a Seizure Protocol Module 309 to be initiated. A seizure protocol overrides any prior commands or instructions conveyed to a robot, device or vehicle by any party to, for example, instruct that a robot, device or vehicle instead execute new commands or instructions that may be issued by the Registry or a governing body.
The seizure protocol commands program a robot, device or vehicle to undertake or successfully complete a particular assignment, operation or function. For example, the assignment may be relocated to a designated space (e.g. Police impound) that may be listed in an Operational Spaces Database 326. Further, different types or models of robots, devices or vehicles (that may be specified in a Registered Robot Database 323) may have different designated space.
309 Seizure Protocol Module
This module commands a user's robot, device or vehicle be relocated to a registry designated space. This space may be stored in an Operational Spaces Database 326. In various embodiments, a command may be communicated or uploaded to a robot, device or vehicle (using a Communications Module 302). The specific data signal communicated (e.g. an output file) may trigger an installed software application on a robot, device or vehicle and that application will perform most of the computations. In other embodiments, a Server 300 runs the applicable software application (as a host) and the remotely located robots, devices and vehicles (clients) are simply a vessel which receives the commands from the Server 300.
310 Orders (Maintenance/Service/Inspection) Module
Module 310 commands that a user's or owner's robot be relocated to a designated space. This space may be listed in an Operational Spaces Database 326. The Module 310 may also incorporate all necessary information, data, tasks, activities or requirements related to maintenance, service, inspection or other matters. This Module 310 may also be a principal module for use by a robot, device or vehicle testing facility; therefore, not only used to instigate relocation of a robot, device or vehicle but able to be used after the robot has arrived at the relocation destination by dictating, undertaking or actioning (e.g. outstanding) orders.
In another aspect, Module 310 may communicate with (perhaps, by using the Communications Module 302, for example) an Approved & Historical Assignments/Functions & Travel Plans & Tracks Database 329 to inform the Server 300 of a robot's, device's or vehicle's location or position relative to a designated space as described in the above paragraph. The server and any relevant user, owner or client may be kept updated on any order matters.
312 Tasks/Activities/Programs Module
This module may be utilised for numerous applications. For example, the module may be responsible for operating, managing, monitoring and controlling (herein this specification these may be referred to as ‘running’ or ‘run(s)’, depending on the context) one or more of the Server's 300 databases or other modules.
Some non-exhaustive examples are provided as follows:
In another example, the 312 Module is also responsible for interacting with the Profiles Database 332 for the purpose of determining the percentage accuracy of a Master Profile.
In other words, the Module 312 may cause changes in the percentage accuracy assigned to a particular Master Profile. This change may then be distributed to an applicable or associated user, which would signal to the user the requirement to update the subject Profile. Distribution or communication with a remote device (e.g. a robot) by the server may be instigated by the Communications Module 302.
If Profiles also apply or are, for example, registered for privacy protection, e.g. Profile Exclusion Zone, Profile Shrouding or Profile Ghosting, then the applicable Profiles from the Profiles Database 332 may interface with Operational Spaces Database 326 (in the case of excluding spaces) and these privacy constraints are implemented in conjunction with the Exclusion/Privacy Protocol Module 307. Further, in performing the above mentioned programs, for example, the Module 312 and Profiles Database 332 may interact with the User/Owner/Client Account Database 321 where necessary. For example, Profiles in the Profiles Database 332 may be linked or associated with specific accounts in the User/Owner/Client Account Database 321.
In one embodiment, a Tasks/Activities/Programs Module 312 facilitates the policing of ‘rogue robots’ or unregistered, unlicensed, illegal, non-participating, etc. robots, devices and vehicles. In one aspect, using data and information from sensors, scanners or other surveillance capturing devices (installed on a user's robots, devices or vehicles) and transmitted to a Server 300 (using a Communication Module 302) and received by or stored in a Surveillance Database 330, a Tasks/Activities/Programs Module 312 may run a software program that monitors, for example, surveillance information or data that consists or comprises of identifying factors of a ‘rogue robot’ that is present or operating in a space that does not correlate with any active assignments, operations or functions listed in or stored on a Proposed & Active Assignments/Functions & Travel Plans & Tracks Module 305, which interfaces with an Operational Spaces Database 326.
In other words, a Surveillance Database 330 receive data; the data may be a captured video image of a robot, for example; where and when the image was captured may be recorded against or with the image file in the Tasks/Activities/Programs Module 312 and is then cross-indexed with data contained in a Proposed & Active Assignments/Functions & Travel Plans & Tracks Module 305, a Surveillance Database 330, a Registered Robot Database 323, a Manufacturer And Robot Database 324, a Robot ‘Apps’/Functions Database 325, an Operational Spaces Database 326 and a Profiles Database 332.
In more detail:
A User/Owner/Client Account Database 321 includes a data structure of all users that participate with the system described herein. The User/Owner/Client Account Database 321 contains data concerning, but not limited to:
The Ineligible Database 322 comprises a list of ineligible users, owners or clients. Ineligibility may be for a variety of applications, activities or requests, for example, particular users, owner or clients may not be mature enough (e.g. under a statutory age limit) to possess or have the authority to instruct, command or convey assignments, operations or functions to a particular type or model of robot, device or vehicle. In one embodiment, the Ineligible Database 322 is operated by a Tasks/Activities/Programs Module 312, and receives collaborative information or data from a Manufacturer And Robot Database 324 which specifies the hardware, software and capabilities (i.e. specifications) of a particular type or model of robot, device or vehicle.
Robots, devices or vehicles that are deemed ineligible may be listed in an Ineligible Database 322 and the corresponding ineligibility rating or status linked to or associated with a user's account which may be listed in the User/Owner/Client Account Database 321.
323 Registered Robot Database
The Registered Robot Database 323 includes information on robots, devices or vehicles that have been registered by their users, owners or clients. Robots, devices or vehicles that may be or become ineligible for registration may instead be listed in or stored on an Ineligible Database 322.
324 Manufacturer and Robot Database
A Manufacturer and Robot Database 324 includes data or information regarding manufacturers of robots, devices and vehicles. In more detail, the Manufacturer and Robot Database 324 lists all robots, devices and vehicles recognised by the Server 300 system.
Further, the Manufacturer and Robot Database 324 includes data with regard to ‘Type Approval’ methods, typically associated with compliance identifications/certifications. For example, part of a compliance process may establish a performance baseline for a particular robot type, which would need to be above the required standards for compliance approval.
Further, data in relation to compliance matters may be stored on a Manufacturer and Robot Database 324. When diagnostic tests are being undertaken, e.g. by a Tasks/Activities/Programs Module 312 in collaboration with a Diagnostic Tests Database 327, the Manufacturer And Robot Database 324 may be referenced for any required data or information.
325 Robot ‘Apps’/Functions Database
A Robot ‘Apps’/Functions Database 325 includes data regarding approved or unapproved robot, device or vehicle software applications.
In more detail, a Server 300 utilises the Robot ‘Apps’/Functions Database 325 for the purpose of listing or storing all available ‘apps’ for download and/or purchase by users, owners or clients, perhaps, for their respective robot, device or vehicle. Those parties may access or log-in to their registry account, navigate a web-based interface in order to search for, select and purchase, then download a desired app.
The ‘app marketplace’ may interface with any one or more of the aforementioned modules and/or databases. For example:
An Operational Spaces Database 326 is provided which includes an entire inventory of environments, places, areas or spaces approved for particular robots, devices or vehicles. The Tasks/Activities/Programs Module 312 interfaces with this database to transmit information to the robots, devices or vehicles.
In more detail, the Operational Spaces Database 326 regulates particular assignments, operations or functions of robots, devices or vehicles. For example, a particular (air) space may be permanently excluded-for-use by all or particular robots, devices or vehicles for safety, security or privacy considerations.
327 Diagnostic Tests Database
The Diagnostic Tests Database 327 includes a plurality of tests for execution on robots, devices and vehicles. In one embodiment, a robot, device or vehicles utilises the Server 300 Diagnostic Tests Database 327 when required to perform a test, and/or the Server 300 may reference its Diagnostic Tests Database 327 to confirm that a robot, device or vehicle possesses the correct tests on its own database(s) and/or module(s).
The Server's 300 Tasks/Activities/Programs Module 312 and/or Communications Module 302 is utilised to, respectively:
An Approved & Historical Assignments/Functions & Travel Plans & Tracks Database 329 contains registry, governing body or other third party approved robot, device or vehicle assignments, operations or functions, which includes permissible operational travel spaces for robots, devices and vehicles.
The Proposed & Active Assignments/Functions & Travel Plans & Tracks Module 305 communicate or references information or data contained in the Approved & Historical Assignments/Functions & Travel Plans & Tracks Database 329 before approving or amending any proposed or active assignments, operations or functions.
A Clearance Code Module 306 is utilised to generate any relevant or appropriate clearance codes after a Proposed & Active Assignments/Functions & Travel Plans & Tracks Module 305 has deemed an assignment, operation or function is acceptable or approved following collaboration with an Approved & Historical Assignments/Functions & Travel Plans & Tracks Database 329.
330 Surveillance Database
In addition to aspects already described herein, a Surveillance Database 330 may be utilised to collect particular surveillance data or information from a plurality of robots, devices or vehicles.
A user may run a particular software application (that may be listed in or stored on a Robot ‘Apps’/Functions Database 325), and that application carry a stipulation that particular surveillance data (e.g. with respect to time, date or location) be relayed to the Server 300 for analysis, processing, storage, etc. A Communications Module 302 may facilitate the transmissions between the Server 300 and remote robots, devices or vehicles. A Tasks/Activities/Programs Module 312 may then route (after, for example, filtering) relevant, appropriate or of value data to a Surveillance Database 330.
331 Clearance Code Database
A Clearance Code Database 331 may list or store all historical or currently active codes issued and their respective assignment, operation or function particulars (which robot it was issued to, the user involved, time/date, etc.) and is run by a Clearance Code Module 306. The Clearance Code Database 331 can also be used for the purposes of ensuring that particular codes are not reused again, or are only recycled following suitable quarantine periods.
Referring to
At step 402, a user typically wishes to utilize a robot for a particular function. If the consumer does not have a robot, the consumer may wish to purchase or otherwise obtain a robot at step 404, before proceeding to step 406, where a user makes a determination as to whether the robot will be used in a public space.
If the consumer wishes to operate their robot in a public space, the process flow continues to step 412, where a determination is made as to whether the consumer possesses a valid robot account. If the consumer does not have an account, a further determination is made at step 414 to determine whether the consumer is eligible for an account. If the consumer is not eligible for an account, the consumer may register to determine whether they may be eligible for an account at a later date at step 416.
If the consumer is eligible for an account, at step 418, the consumer obtains a valid account.
Once the system has determined that the consumer has a valid account, a check is made to determine whether the consumer's robot is registered at step 420. If the robot is not registered, then at step 422, a determination is made as to whether the consumers robot is registration eligible. If the robot is not registrable, the consumer may obtain a robot at step 424, and the eligible robot can then proceed through the flow process at step 406. If the consumer's robot is registration eligible, the consumer registers the robot at step 426 and the process flow may continue as shown in
Referring now to
Referring to
Referring to
At step A002, the command or request (which includes any attached information or metadata) is communicated 8 to the remote server 2 (or Robot Registry). At step A003, the remote server 2 receives and assesses the command or request, then generates an assessment determination.
At step A004 and A005, if the determination was to conditionally approve the command or request subject to approval of results or responses from outstanding assessment requirements, the remote server 2 communicates 9 outstanding requirements or instructions (e.g. assessment, diagnostic or health test instructions) to the robot's 6 receiver/transmitter module 10.
Therefore, in one embodiment, the command or request (e.g. operation mission) is first approved, in-principle, then the robot may need to be vetted (e.g. to ensure it is up to the task) before it is authorised to effect the command or request (e.g. like a two-part procedure or assessment).
The receiver/transmitter module 10 transmits the requirements or instructions to the robot's 6 regulating ‘chip’ 3 (e.g. a robot-local Robot Registry device, which may be software, hardware and/or firmware, and may comprise one or more devices functioning together, separately, dependently or independently; such devices may be integrated into robot modules or units, e.g. central processing units, CPUs).
At step A006 and A007, the regulating chip 3 facilitates and/or supervises robot testing or querying, communicating 9 results or responses to the remote server 2 (e.g. via the robot's 6 receiver/transmitter module 10).
At step A008, the remote server 2 receives and assesses the results or responses, then generates an assessment determination. (Note: steps A004 to A008 may go through numerous rounds or be repeated e.g. to facilitate further querying or testing following receipt, assessment and determination of prior query or test results or responses).
At step A009, if a determination was to approve the command or request following receipt and assessment of the robot's 6 results or responses, then the remote server 2 communicates 9 the command or request to the robot's 6 receiver/transmitter module 10. The receiver/transmitter module 10 transmits the command or request to the robot's 6 regulating chip 3.
At step A010, the regulating chip 3 facilitates the command or request, transmitting to the robot's 6 output 5, which essentially results in the command or request being effected.
Referring to
In another embodiment, an internal command or request generator 7 (e.g. the robot's 6 autonomous logic or ‘brain’) generates a command or request. In one example, the robot detects something in its environment such as unfavourable or bad weather, which upon internal logic assessment results in the robot needing to generate a request such as permission to implement a detour to the destination, i.e. to avoid the bad weather.
Since the detour would involve a new travel path the robot would first need it approved before the robot is permitted to pursue the new route. At step B002, the command or request is communicated 12 (or transmitted 11, according to the other embodiment in step B001) to the robot's 6 input module 4. The robot's 6 input module 4 transmits the command or request to the robot's 6 regulating ‘chip’ 3.
At step B003, the robot's 6 regulating chip 3 assesses the command or request, then generates an assessment determination. At step B004 and B005, if the determination was to conditionally approve the command or request subject to approval of results or responses from outstanding assessment requirements, the regulating chip 3 then facilitates and/or supervises robot testing or querying to establish said results or responses.
At step B006, should the results or responses be satisfactory, in conjunction with the command and request, then the regulating chip 3 communicates 9 this data information to the remote server 2 (e.g. via the robot's 6 receiver/transmitter module 10) for further or final assessment and determination. In other words, the regulating chip 3 may pre-vet the command or request and/or robot's results or responses, i.e. before communicating with the remote server 2.
An advantage of this includes reducing the probability of the remote server 2 receiving and processing extraneous, redundant or purposeless data information traffic, e.g. the regulating chip 3 may act as a ‘first screen’, rejecting any commands or requests (which may be in conjunction with the robot's results or responses) that do not cut muster or are determined as not (likely) to be approved by the remote server 2.
At step A007, the remote server 2 receives and assesses the command or request and/or results or responses, then generates an assessment determination. At step A008, if the determination was to approve the command or request, the remote server 2 communicates 9 the command or request approval to the robot's 6 receiver/transmitter module 10.
The receiver/transmitter module 10 transmits the command or request approval to the robot's 6 regulating chip 3. At step A009, the regulating chip 3 facilitates the command or request, transmitting to the robot's 6 output 5, which equates to the command or request being effected by the robot 6. Note, as described in Scenario 1 (i.e.
Referring to
At step C002, the command or request is communicated 12 (or transmitted 11) to the robot's 6 input module 4. The robot's 6 input module 4 transmits the command or request to the robot's 6 regulating ‘chip’ 3.
At step C003, the robot's 6 regulating chip 3 assesses the command or request, then generates an assessment determination. At step C004 and C005, if the determination was to conditionally approve the command or request subject to approval of results or responses from outstanding assessment requirements, the regulating chip 3 then facilitates and/or supervises robot testing or querying to establish said results or responses.
At step C006, it is here that the process may differ from Scenario 2, in that the regulating chip 3 may determine that there is no communication with a remote server 2 (e.g. the robot 6 may be ‘out of range’, in a wireless signal denied environment or not ‘permanently’ online), so the regulating chip 3 assesses to the best of its ability and/or programming the command or request and/or results or responses (e.g. it may take on the same or similar role as what the remote server 2 would have performed).
In one embodiment, the regulating chip 3 may occasionally communicate 9 with a remote server 2, e.g. via the robot's 6 receiver/transmitter module 10, and in doing so may facilitate the renewal of the robot's regulatory chip service subscription (or licence key) and/or the updating of relevant software, patches or flashes such as the latest Orders and/or Protocols that are to be obeyed or complied with by the robot 6.
In a further embodiment, should the robot's 6 regulating chip 3 not communicate with the remote server 2 within a specified time period or in respect of a currently proposed command or request the robot's 6 regulating chip 3 may cause commands or requests to not be approved in whole or in part—essentially, limiting or restricting the robot 6 or preventing it from operating.
A limitation or restriction example includes the command or request being ‘travel around [this] location, taking surveillance footage’; however, since the robot had not been in communication with the remote server within a specified period of time the command or request was reduced or limited, for example, as follows: the robot was only permitted to travel within a specified radius of its ‘home base’ (or registered address) or was not permitted to travel within public spaces.
At step C007, if the determination was to approve the command or request, the regulating chip 3 facilitates the command or request, transmitting to the robot's 6 output 5, which equates to the command or request being effected by the robot 6.
Upon receiving the instructions, at step 445, the robot commences diagnostic tests and subsequently, at step 450, the robots test data is sent to the registry where, at step 455, a determination is made to determine whether the robot has successfully completed the diagnostic tests. If not, the process is terminated at step 460 as the consumer's robot is not issued a clearance code required to execute assignment and/or perform a restrictive function in a public space. If the robot successfully completes the diagnostic tests, the registry issues the consumer's robot with the appropriate instructions or clearance codes at step 465 and at step 470 the consumer's robot executes the assignment or function. Subsequent to the assignment or function being executed, the registry may optionally be updated at step 475 and a further determination is made to determine whether the consumer has other assignments or function instructions for the robot at step 480. If not, the process ends at step 485, but if there are further instructions, the process returns to step 406 at
Profile Updating
Referring now to
Once the customer has obtained profile capture products and a subscription account, at step A515 the customer forms their profile. Once the profile is formed at step A520 the customer logs onto their account and subsequently at step A525 the customer may upload their profile update (or alternatively, the profile may be updated automatically).
In some embodiments, the file updates may be sent to a master profile and added to the master profile as shown at step A530. Subsequently, the customer's updated master profile is distributed.
Referring now to
Conditions or Constraints
In the previous section, and with reference to
With reference to
At step D02, the condition or constraint (which includes any attached information or metadata) is communicated to the remote server CC11 (or Robot Registry). At step D03, the remote server CC11 receives and assesses the condition or constraint, then generates an assessment determination.
At step D04, the remote server CC11 may approve the condition or constraint being imposed. At step D05, the remote server CC11 communicates the condition or constraint to the robot's CC12 one or more ‘regulating chips’. At step D06, the robot's CC12 regulating chip facilitates the condition or constraint being imposed.
At step D07, the condition or constraint may be amended or revoked for one or more reasons, e.g. the condition/constraint creator CC10 may no longer require the condition or constraint to be active, so it may be cancelled.
With regard to Scenario 2 in
Privacy Issues and Ghosting
Privacy constraints may take the form of software code downloaded onto the robot, which then self-regulates its navigation movements. In one embodiment, the robot's navigation system may have ‘blocks’ or ‘no go’ zones placed upon its ‘internal’ map. These no go zones may be added, subtracted or amended accordingly, from time to time. Compatible navigation software code may utilise or feature multi-dimensional (geographical) coordinate systems/techniques. Such as, the robot's system is informed to avoid or not to travel within certain zones.
For example, unregistered robots would not typically receive clearance codes but they may receive privacy constraints. Being able to receive these constraints despite not being registered may be due to robot manufacturers supporting the ‘privacy protocol standard’, thereby, by default, designing robots to automatically receive these relevant transmissions—in whatever form they may be relayed to the robot (and whether from the registry or another party)—irrespective of the user being aware of such receipt.
In an alternative example, a registered robot may receive clearances codes; however not receive any privacy constraints. One reason for this may be because the consumer's instructed assignment or function did or would not cause the robot to venture in, on or near an exclusion zone (a zone that may be registered with the Registry or another party).
In more detail, the steps for facilitating exclusion zones (e.g. adjust travel path or plan):
The broad steps for facilitating shrouding or ghosting (e.g. augmenting of surveillance data and regulating disclosure):
Condition or constraint position data information is received by robot and/or remote server in the following manners:
The systems, methods, computer programs, data signals and apparatuses which may facilitate exclusion zones include those technologies deployed for Airborne Collision Avoidance Systems (http://en.wikipedia.org/wiki/Airborne Collision Avoidance System). Examples (non-limiting), include:
The main differences with exclusion zones being facilitated via Profiles rather than collocated location module or fixed coordinates and altitude include the substitution of position announcing transmitting devices and signals with Profile recognition technology and the generation of a prescribed exclusion zone around that recognised Profile, which may be facilitated within the robot's Geospatial Information System (GIS), navigation and path planning system.
Returning now to
Similarly, user device 110b which is a non participant does not receive the imposed privacy constraints. However, user devices 110a and 110n do receive the privacy constraints.
Referring now to
In one embodiment, the Registry's server implements an exclusion zone and all robots must adjust their subsequent travel plans and tracks accordingly.
In addition there may be provided a software application for robots that prevent any (on-board) equipment (e.g. cameras) from conducting surveillance operations when faced or encountered with a shrouded zone to protect, for example, public privacy.
In more detail, before explaining the process flow of
In another embodiment, select Profile data is stored (and processed) locally of a user's devices or robots. For example, the registry may transmit to participating robots and devices Profile data of registered clients (subscribers) present in the vicinity of those participating robots and devices. So if they were to encounter each other (i.e. come into capture view) then these privacy measures could be effected. Then, once participating robots or devices have moved away from a registered client, that client's Profile data is deleted from those robots and devices. This embodiment would help ease the latency problem, by storing and processing data locally, and slightly solve the issue of having sensitive Profile data (of others) stored and processed locally, rather than just by the Registry's servers.
Profile data is stored, processed and filtered (with respect to database queries) in any number of manners. For example, the end user's device or robot may locally store a copy (and receive relevant updates) of all database Profiles; and the device or robot may subsequently process and filter such queries as well.
In one embodiment of the invention, the user may be presented with a map (that is at least one-dimension), however, preferably a three-dimensional mapping computer program like one or more of the following: Bing 3D Maps; Placebase; Google Tour Guide or Maps GL, 3D Maps Mobile; Amazon's UpNext app; Whereis 3D City Models; Uniden TRAX 5000; Aviation Mapper; Nokia 3D Maps (e.g. Ovi), and so on.
The user would be presented with the opportunity to select in the map (by directing a mouse cursor or equivalently operable pointer) areas or spaces, perhaps, by dragging the cursor or pointer across a screen or by using gesture control, voice commands or other effecting action that may be or become applicable methods from time to time.
One example of shrouding would be a resident wishing to shroud their apartment's windows and balconies from robot surveillance (perhaps, aerial robotic devices), would access the system, locate their property's windows and balconies, drag the cursor over those areas or spaces to shroud (that may be illustrated on-screen by a size-changing rectangle or rectangular prism), confirm selection, enter and accept payment terms, confirm transaction, shrouding effected with specified timeframe.
Only applicable or allowable zones may be available for selection by the user. For example, parties may not have the option available to select a space that they do not have control over, do not own or are not named on the land's title. If a party selects a zone for exclusion (i.e. to prevent other robots from travelling in this space or on this area) and this zone not be a space legally associated with the party, for example, then to prevent robots that are related to a party legally associated with that space (due to the exclusion imposed by the other disassociated party) those robots may be approved to override such imposition. There is a need to account for antagonists, i.e. dissociated parties that may attempt to prevent other parties' robots from genuinely accessing or travelling in a particular space.
Referring now to
Referring now to
The remote server (Robot Registry) is responsible for ensuring all exclusion zone data is updated and maintained. This data arrives from various sources such as public aviation departments, defense departments, LEO groups, corporations and individuals. An exclusion zone (e.g. ‘no fly zone’) consists of various pieces of metadata that includes the following:
The remote server allows data to be submitted via secure web forms or via automated Application Programming Interfaces, within the remote server web form interface individuals could enter data in several ways. Data is entered as a series of GPS locations creating a closed loop or a map could be displayed allowing the individual to plot the closed loop over the map. Once submitted a remote server validates geo-data and approves it for submission to the exclusion zone database. Additional screens allow for the editing and configuration of this data as required by remote server staff. Alternatively, once submitted, the remote server's automated assessment programs may determine suitability for approval.
To create an accurate three-dimensional volume all data is GPS based including altitude values. A default height may be applied to the zone.
The user can use any tool that creates a valid set of GPS coordinates which form a loop, or alternatively they could utilise an online mapping interface provided by a remote server (Robot Registry).
One example of an online mapping interface is Google Maps developer API. Specific information on the capabilities can be found here https://developers.google.com/maps/.
Referring to
The remote server processes this request to identify if it exists within its known database of exclusion zones.
If the destination is within an exclusion zone a rejection is sent and alternatives offered, such as nearest possible safe point or choice to abandon the request. If the destination is approved then a confirmation is sent approving the request.
This process is described further with reference to
At step 525, the client confirms their choices and any payments necessary. At step 530 the registry receives the client's application and step 535 the registry reviews the application and consults with third parties if relevant.
At step 540 the registry may approve, deny or propose amendments the client's application and, once all relevant parties accept the application at step 545, the client is invoiced and the application finalised.
At step 550 upon payment receipt, the registry takes the necessary action to update the database to exclude, shroud or ghost the area, device, machine or robot.
The devices, machines or robots are informed and begin obeying the updated privacy constraints.
In very select circumstances, waivers are obtainable to allow robots and participating devices to not be constrained by the ‘privacy protocol’. In one embodiment, ‘exclusion zones’, ‘shrouding (fixed)’ and ‘Profile shrouding (mobile)’ may be waived for approved parents (running approved dedicated apps), so parents may monitor their children, for example:
Another aspect of the invention to ultimately restrict unencumbered operation or function (e.g. surveillance) opportunitiesit will be understood that the user may be provided with a time-limiting camera viewing (perhaps, when travelling in a particular class of exclusion zone). After the permissible time has expired a latency period may apply before the user is permitted another timed viewing, for example.
Turning to
At step 585, the registry may relay clearance codes and relevant privacy constraints if any to the devices, machines or robots and at step 590 the user's device, machine or robot executes the assignment or function and conforms to any privacy constraints.
It will be understood that in other embodiments, indicated generally by arrows 2, 3, and 4, variations on the process flow are possible. For example, in accordance with process flow 2, when a user initiates an assignment or function instructions, at step 570 the relevant privacy constraints may be relayed to the user's device, machine or robot directly, at step 584 and the robot subsequently executes the assignment or function, conforming to any privacy constraints at step 590.
Alternate embodiments are also envisaged, where the user is not required to log into their account or to initiate an assignment or function. As shown generally by process flows 3 and 4, privacy constraints may be relayed to the user's device, machine or robot. These privacy constraints may then be utilised to either perform a function as shown at step 590 or may simply be added to the robots store of information, for future use. The process flows shown generally by process flows 3 and 4 may be used in embodiments where instructions may be issued directly to the robot and not sent via the registry.
Referring to
Broad Process Steps (Policing)
1. Generate
Generate Policing Initiative
2. Assess (Optional)
In another example, if a sample matches an enrolled normal or acceptable robometric, then this does not result in an enforcement trigger event.
In yet another example, if a sample does not match an enrolled normal or acceptable robometric i.e. is outside tolerance or exceeding threshold, then this results in an enforcement trigger event.
3. Respond
Respond with Enforcement and/or Notify
Passive enforcement strategies may include:
Active enforcement strategies may include:
4. Amend (Optional)
Amending or revoking enforcement may include cancel enforcement pursuit (e.g. ‘call off the chase’), call in reinforcements (e.g. more policing robots to assist), and/or revert to enforcement (e.g. trigger event)
Broad Process Steps (Conditions)
1. Generate
Generating condition/constraint may include generating performance/functionality/capability constraint (e.g. regulations may stipulate constraints due to license class, inadequate supporting infrastructure, network or systems, emergency provision trigger event and/or enforcement trigger event generates constraint).
Generated exclusion zone conditions may include:
Generated shrouding conditions may include:
Generated ghosting conditions may include:
Generated condition/constraint waiver or exemption may include:
2. Assess (Optional)
Assessing generated condition/constraint may include detecting:
3. Respond
Responding by imposing condition/constraint may include:
With reference to
Referring now to
At step 608 or 610, the user's robot becomes aware of a situation where a sense/scan operation is required.
If a positive result is achieved, at step 615 a seizure protocol may be initiated. If a seizure protocol is initiated at step 617 then the process of sense/scan ends at step 630. If a seizure protocol is not initiated then the user's robot proceeds as normal at step 620 and the user's robot may complete the user assignment or function at step 625 or alternatively the user's robot may return to the location prior to the sense/scan protocol being initiated at step 623 prior to the process ending at step 630.
Referring now to
A system that allows the registry to police the environment of robots that may be unregistered, unlawful, a danger to the public, an invasion to people's privacy or operating improperly, perhaps, even stolen. Herewith these robots will be referred to as ‘rogue robots’.
The Registry's system utilises its monitoring, controlling and managing capabilities (not limited to the fact that it may oversee the travel paths and tracks of a vast variety of robots)—may, in one embodiment, grant it special abilities or opportunities to police the environment for rogue robots.
In another embodiment, the registry may ‘discover’ rogue robots by employing the assistance of non-registry robots (e.g. consumers' robots) operating in or near particular spaces. Preferably, consumers' robots may feed environmental data back to the registry or process environment locally then send results to the registry. In either method, the registry may utilise the on-board sensors and scanners of consumers' robots, specifically monitor their travel environment for rogue robots, i.e. robots that should not be encountered, perhaps, by the consumers' robot's ‘anti-collision and avoid’ detector or module.
This initiative of employing the help of consumer's robots may also allow the tracking of rogue robots that are initially detected. In one embodiment, as each consumer robot detects the presence or signature of a particular rogue robot the current position of each respective consumer robot that detected the rogue robot and/or the position the rogue robot was detected at by each consumer robot may allow track plotting (and this may be in multiple dimensions).
However, ghosting may also be applied to, preferably, tangibles that are recognised via other means, that is, not by Profile recognition methods. For example, a tangible may not be identified or tagged if it is located within a zone that has been excluded (perhaps, by a client selecting it on a 3D map) or if it is associated (e.g. collocated) with a location module device (e.g. smartphone with geo-location features).
In other words, the Registry's system may, in a first instance, ‘spot’ or identify all robots in a particular environment, then eliminate from its electronic field of view all robots that are registered or that are known to be operating there following the issue of clearance codes (i.e. removing robots that are not rogues). Thus, if there are any robots remaining after this elimination process, by deduction, the remaining robots may be deemed ‘off the grid’ or rogue operators.
In addition, the registry may conduct such investigations using a variety of methods. In one method, the registry deploys at least one robot of its own to track the rogue robot to its destination (hopefully, to where the user is located) or, perhaps, to (safety) deactivate the rogue robot using a number of means (e.g. one ideal embodiment may be overriding or ‘spoofing’ its communications system to inflict the Registry's ‘seizure protocol’).
The registry, over time, builds up a library or inventory of signatures (not limited to acoustic) for a variety of robots, such signatures may be assigned to particular robot types.
Referring to
Referring to
Robots 735 and 740 remain in view. These robots are deemed not registered. Further, rogue robot 740 has been listed as having an unrecognizable signature (illustrated by its unique symbol). Whereas 735 was detected as having a familiar signature, was listed as being a ‘Concentric Circle’ Type robot, according to the Registry's database analysis.
Referring to
Participants may act as detector nodes and/or (mobile) repeater stations—respectively, facilitating detection of non-participants and assisting to relay any alert notices or other relevant data to other participants. The mechanism of action may resemble (signal) propagation. [In the figure, ‘R’ may equate to a ‘registered’ status.]
In the first instance, 803 captures a greater than partial match of a non-participating device, machine or robot 801. This match percentage value is deemed significant enough to warrant an alert be transmitted (in) directly to nearby participants 805 and 807, which both are positioned with the alert radius.
In another instance, multiple participants 809, 811 and 813 capture only partial matches of a non-participating device, machine or robot 815. Individually these partial matches may not be enough to elicit an alert be distributed, however considering there are multiple, these matches synergistically equate to a percentage value deemed significant enough to warrant an alert be transmitted (in) directly to nearby participants 819 and 817, which both are positioned within the alert radius; further, original detectors 809, 811 and 813 would similarly be alerted, advising them that, yes, their partial matches were relevant.
Meanwhile, participants 821, 823 and 825, since outside any alert radius or positioned away from any non-participant, have not been involved in any of the above instances.
Advantages
One of the advantages of the embodiments and broader invention described herein is that the invention removes the onus or control from consumers (i.e. owners of the robots) to assume full responsibility for the actions of the robots at any given time. So long as the commands are filtered or processed through a central server system, then illegal, immoral or accidental use of autonomous and robotic devices is greatly reduced.
If robots were unrestricted in their activities (operations or functions), were able to venture into particular spaces unhindered (with or without explicit instructions or control from the user), then this would cause or provoke contentious issues—some of which include, privacy, safety, security, liability, technical and ethical issues. Therefore, the system provides a mechanism and framework by which robots and/or their controllers are required to obtain the relevant clearance before an operation is allowed, particularly if that operation is to occur in a public space.
Moreover, as consumers are required to register and identify themselves, the system provides an ability to monitor, control or manage the actions or activities of consumers, in so far as it is related to their robot use. This reflects society's general interest in requiring people to obtain licenses to drive cars, pilot aircraft or own and use firearms.
That is, there is a public interest in preventing users from allowing their robots to be used by unapproved users, or from having their robots unknowingly used by unapproved users. Such robots may possess the capacity and capabilities to perform restricted (e.g. dangerous) functions. These occurrences would first and foremost present safety issues, e.g. robots (owned by parents) being used by underage children that are not listed as or approved to be registered users.
As a corollary, there are fewer onuses on robot and autonomous system providers for being responsible for facilitating vital robot updates. Instead, all updates would be processed by and issued from the system described herein, irrespective of the robot's origin of manufacture, country or space of operation. This ameliorates the legal liability of robot and autonomous system providers.
Advantages of the embodiments described herein where the embodiment is fulfilled by a separate device to the robot or by a remote server include:
Further advantages of a user and/or the user's ‘smart’ device or robot pendant communicating indirectly, or not at all, with the robot, e.g. via the remote server, include no ‘lost in translation’ events as the remote server (Robot Registry) receives proposed commands or operation requests, directly from the source, i.e. the user or the user's device. In other words, the remote server acts as an intermediary between the user (or their device) and the robot.
Disclaimers
Throughout this specification, unless the context requires otherwise, the word “comprise” or variations such as “comprises” or “comprising”, will be understood to imply the inclusion of a stated integer or group of integers but not the exclusion of any other inte.g.er or group of integers.
Those skilled in the art will appreciate that the invention described herein is susceptible to variations and modifications other than those specifically described. The invention includes all such variation and modifications. The invention also includes all of the steps, features, formulations and compounds referred to or indicated in the specification, individually or collectively and any and all combinations or any two or more of the steps or features.
Other definitions for selected terms used herein may be found within the detailed description of the invention and apply throughout. Unless otherwise defined, all other scientific and technical terms used herein have the same meaning as commonly understood to one of ordinary skill in the art to which the invention belongs.
Although not required, the embodiments described with reference to the method, computer program, data signal and aspects of the system can be implemented via an application programming interface (API), an application development kit (ADK) or as a series of program libraries, for use by a developer, for the creation of software applications which are to be used on any one or more computing platforms or devices, such as a terminal or personal computer operating system or a portable computing device, such as a smartphone or a tablet computing system operating system, or within a larger server structure, such as a ‘data farm’ or within a larger transaction processing system.
Generally, as program modules include routines, programs, objects, components and data files that perform or assist in the performance of particular functions, it will be understood that the functionality of the software application may be distributed across a number of routines, programs, objects or components to achieve the same functionality as the embodiment and the broader invention claimed herein. Such variations and modifications are within the purview of those skilled in the art.
It will also be appreciated that where methods and systems of the present invention and/or embodiments are implemented by computing systems or partly implemented by computing systems then any appropriate computing system architecture may be utilised. This includes standalone computers, network computers and dedicated computing devices (such as field-programmable gate arrays).
Where the terms “computer”, “computing system” and “computing device” are used in the specification, these terms are intended to cover any appropriate arrangement of computer hardware for implementing the inventive concept and/or embodiments described herein.
Where the terms “robotic device”, “autonomous device” and “smart device” are used in the specification, these terms are intended to cover any appropriate device which is capable of receiving a command and utilising the command to perform a function, which may be either a “physical” function (i.e. movement) or a “virtual” function (e.g. interact with another device via electronic commands).
Where reference is made to communication standards, methods and/or systems, robots or devices may transmit and receive data via a variety of forms: 3G, 4G (CDMA/GSM), Wi-Fi, Bluetooth, other radio frequency, optical, acoustic, magnetic, GPS/GPRS, or any other form or method of communication that may become available from time to time.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2012904989 | Nov 2012 | AU | national |
| 2013204965 | Apr 2013 | AU | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/AU2013/001303 | 11/12/2013 | WO | 00 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2014/071465 | 5/15/2014 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 5075853 | Luke, Jr. | Dec 1991 | A |
| 5483440 | Aono et al. | Jan 1996 | A |
| 5920678 | Watanabe | Jul 1999 | A |
| 5959423 | Nakanishi et al. | Sep 1999 | A |
| 6061561 | Alanara et al. | May 2000 | A |
| 6122572 | Yavnai | Sep 2000 | A |
| 6175206 | Ueno | Jan 2001 | B1 |
| 6493609 | Johnson | Dec 2002 | B2 |
| 6650224 | Weigl et al. | Nov 2003 | B1 |
| 6739556 | Langston | May 2004 | B1 |
| 6816753 | Sakamoto | Nov 2004 | B2 |
| 6856894 | Bodin et al. | Feb 2005 | B1 |
| 6860206 | Rudakevych | Mar 2005 | B1 |
| 6868181 | Feiten | Mar 2005 | B1 |
| 7042185 | Yang | May 2006 | B2 |
| 7072740 | Iribe et al. | Jul 2006 | B2 |
| 7104495 | McGeer | Sep 2006 | B2 |
| 7121507 | Dennis et al. | Oct 2006 | B2 |
| 7228232 | Bodin et al. | Jun 2007 | B2 |
| 7299007 | Eskin | Nov 2007 | B2 |
| 7343232 | Duggan et al. | Mar 2008 | B2 |
| 7373377 | Altieri | May 2008 | B2 |
| 7376511 | Szabo et al. | May 2008 | B2 |
| 7408439 | Wang | Aug 2008 | B2 |
| 7412325 | Tannenbaum et al. | Aug 2008 | B1 |
| 7510141 | Nonami et al. | Mar 2009 | B2 |
| 7539561 | Nonami et al. | May 2009 | B2 |
| 7603207 | Abraham et al. | Oct 2009 | B2 |
| 7610122 | Anderson | Oct 2009 | B2 |
| 7693624 | Duggan et al. | Apr 2010 | B2 |
| 7720572 | Ziegler et al. | May 2010 | B2 |
| 7737878 | van Tooren et al. | Jun 2010 | B2 |
| 7835824 | Matos | Nov 2010 | B2 |
| 7903099 | Baluja | Mar 2011 | B2 |
| 7924927 | Boesjes | Apr 2011 | B1 |
| 7940259 | Wright et al. | May 2011 | B2 |
| 7957837 | Ziegler et al. | Jun 2011 | B2 |
| 7991517 | Matos | Aug 2011 | B2 |
| 8014573 | Boomer et al. | Sep 2011 | B2 |
| 8050688 | Jacob | Nov 2011 | B2 |
| 8060295 | Estkowski et al. | Nov 2011 | B2 |
| 8068949 | Duggan et al. | Nov 2011 | B2 |
| 8068950 | Duggan et al. | Nov 2011 | B2 |
| 8082074 | Duggan et al. | Dec 2011 | B2 |
| 8082102 | Ravenscroft | Dec 2011 | B2 |
| 8103398 | Duggan et al. | Jan 2012 | B2 |
| 8121729 | Blanc et al. | Feb 2012 | B2 |
| 8169328 | Duvall et al. | May 2012 | B2 |
| 8175913 | Checketts et al. | May 2012 | B2 |
| 8184423 | Rothkopf | May 2012 | B2 |
| 8186589 | Ben Asher et al. | May 2012 | B2 |
| 8225220 | Barbaro Altieri | Jul 2012 | B2 |
| 8228325 | Barbaro Altieri | Jul 2012 | B2 |
| 8229467 | Root et al. | Jul 2012 | B2 |
| 8281127 | Hayes | Oct 2012 | B2 |
| 8286236 | Jung et al. | Oct 2012 | B2 |
| 8612051 | Norman | Dec 2013 | B2 |
| 8718822 | Hickman | May 2014 | B1 |
| 9160783 | Pinter | Oct 2015 | B2 |
| 9198728 | Wang | Dec 2015 | B2 |
| 9251313 | Ross | Feb 2016 | B2 |
| 9261578 | Im | Feb 2016 | B2 |
| 9317110 | Lutnick | Apr 2016 | B2 |
| 20020019783 | Chol | Feb 2002 | A1 |
| 20020103575 | Sugawara | Aug 2002 | A1 |
| 20020120361 | Kuroki | Aug 2002 | A1 |
| 20020137427 | Peters | Sep 2002 | A1 |
| 20020140814 | Cohen-Solal et al. | Oct 2002 | A1 |
| 20020161489 | Johnson | Oct 2002 | A1 |
| 20030046304 | Peskin et al. | Mar 2003 | A1 |
| 20030097202 | Fujita | May 2003 | A1 |
| 20030114959 | Sakamoto | Jun 2003 | A1 |
| 20030125963 | Haken | Jul 2003 | A1 |
| 20030137435 | Haddad et al. | Jul 2003 | A1 |
| 20030216833 | Mukai | Nov 2003 | A1 |
| 20040034571 | Wood et al. | Feb 2004 | A1 |
| 20040083274 | Katiyar | Apr 2004 | A1 |
| 20040098167 | Yi et al. | May 2004 | A1 |
| 20040181327 | Tsosie | Sep 2004 | A1 |
| 20040182614 | Wakui | Sep 2004 | A1 |
| 20040210345 | Noda | Oct 2004 | A1 |
| 20040224676 | Iseki | Nov 2004 | A1 |
| 20040232243 | Silverbrook et al. | Nov 2004 | A1 |
| 20040245378 | Nonami et al. | Dec 2004 | A1 |
| 20050004723 | Duggan et al. | Jan 2005 | A1 |
| 20050027406 | Nonami et al. | Feb 2005 | A1 |
| 20050080799 | Harnden | Apr 2005 | A1 |
| 20050162119 | Landry | Jul 2005 | A1 |
| 20050210267 | Sugano et al. | Sep 2005 | A1 |
| 20050218852 | Landry | Oct 2005 | A1 |
| 20050234592 | McGee | Oct 2005 | A1 |
| 20050274845 | Miller et al. | Dec 2005 | A1 |
| 20050281439 | Lange | Dec 2005 | A1 |
| 20060009879 | Lynch | Jan 2006 | A1 |
| 20060032978 | Matos et al. | Feb 2006 | A1 |
| 20060049790 | Yang | Mar 2006 | A1 |
| 20060060694 | Nonami et al. | Mar 2006 | A1 |
| 20060070558 | Chiu | Apr 2006 | A1 |
| 20060071116 | Quenneville | Apr 2006 | A1 |
| 20060129638 | Deakin | Jun 2006 | A1 |
| 20060146048 | Wright et al. | Jul 2006 | A1 |
| 20060287913 | Baluja | Dec 2006 | A1 |
| 20070015518 | Winter et al. | Jan 2007 | A1 |
| 20070016328 | Ziegler | Jan 2007 | A1 |
| 20070025809 | Lee et al. | Feb 2007 | A1 |
| 20070029449 | Matos | Feb 2007 | A1 |
| 20070042716 | Goodall | Feb 2007 | A1 |
| 20070042803 | Anderson | Feb 2007 | A1 |
| 20070061040 | Augenbraun | Mar 2007 | A1 |
| 20070061043 | Ermakov | Mar 2007 | A1 |
| 20070073439 | Habibi | Mar 2007 | A1 |
| 20070074182 | Hinchey | Mar 2007 | A1 |
| 20070093258 | Steenstra et al. | Apr 2007 | A1 |
| 20070119326 | Rudakevych | May 2007 | A1 |
| 20070133846 | Andersson | Jun 2007 | A1 |
| 20070138345 | Shuster | Jun 2007 | A1 |
| 20070145182 | Page | Jun 2007 | A1 |
| 20070156286 | Yamauchi | Jul 2007 | A1 |
| 20070162196 | Nonami et al. | Jul 2007 | A1 |
| 20070192910 | Vu | Aug 2007 | A1 |
| 20070210953 | Abraham et al. | Sep 2007 | A1 |
| 20070244599 | Tsai | Oct 2007 | A1 |
| 20070250212 | Halloran | Oct 2007 | A1 |
| 20070273557 | Baillot | Nov 2007 | A1 |
| 20070284474 | Olson et al. | Dec 2007 | A1 |
| 20070293989 | Norris | Dec 2007 | A1 |
| 20080027604 | Oesterling | Jan 2008 | A1 |
| 20080121097 | Rudakevych | May 2008 | A1 |
| 20080133052 | Jones | Jun 2008 | A1 |
| 20080194311 | Cage et al. | Aug 2008 | A1 |
| 20080227435 | Six et al. | Sep 2008 | A1 |
| 20080247549 | Blanc et al. | Oct 2008 | A1 |
| 20080255711 | Matas | Oct 2008 | A1 |
| 20080263628 | Norman | Oct 2008 | A1 |
| 20080269949 | Norman et al. | Oct 2008 | A1 |
| 20080291849 | Ostermeier et al. | Nov 2008 | A1 |
| 20090027253 | van Tooren et al. | Jan 2009 | A1 |
| 20090033112 | Westerlind et al. | Feb 2009 | A1 |
| 20090045290 | Small et al. | Feb 2009 | A1 |
| 20090084761 | Bortolus | Apr 2009 | A1 |
| 20090107386 | Sampson et al. | Apr 2009 | A1 |
| 20090112387 | Kabalkin et al. | Apr 2009 | A1 |
| 20090112388 | Yeager et al. | Apr 2009 | A1 |
| 20090125163 | Uggan et al. | May 2009 | A1 |
| 20090125169 | Edwards et al. | May 2009 | A1 |
| 20090125221 | Estkowski et al. | May 2009 | A1 |
| 20090151617 | Lambertus | Jun 2009 | A1 |
| 20090165126 | Jung et al. | Jun 2009 | A1 |
| 20090167787 | Bathiche et al. | Jul 2009 | A1 |
| 20090175599 | Grim et al. | Jul 2009 | A1 |
| 20090189015 | Alavi | Jul 2009 | A1 |
| 20090210109 | Ravenscroft | Aug 2009 | A1 |
| 20090216831 | Buckner | Aug 2009 | A1 |
| 20090224097 | Kariv | Sep 2009 | A1 |
| 20090235870 | Troy | Sep 2009 | A1 |
| 20090251354 | Zahavi | Oct 2009 | A1 |
| 20090271131 | Ashton | Oct 2009 | A1 |
| 20090289778 | King | Nov 2009 | A1 |
| 20090298539 | Anderson | Dec 2009 | A1 |
| 20090299582 | Anderson | Dec 2009 | A1 |
| 20090306840 | Blenkhorn et al. | Dec 2009 | A1 |
| 20090327885 | Aoki et al. | Dec 2009 | A1 |
| 20100006034 | Van Den Berg | Jan 2010 | A1 |
| 20100042269 | Kokkeby et al. | Feb 2010 | A1 |
| 20100070124 | Yeager et al. | Mar 2010 | A1 |
| 20100070450 | Barker | Mar 2010 | A1 |
| 20100084513 | Gariepy et al. | Apr 2010 | A1 |
| 20100085153 | Smith | Apr 2010 | A1 |
| 20100100269 | Ekhaguere et al. | Apr 2010 | A1 |
| 20100118147 | Dorneich et al. | May 2010 | A1 |
| 20100131121 | Garlock | May 2010 | A1 |
| 20100133849 | Allaei | Jun 2010 | A1 |
| 20100163621 | Ben-Asher et al. | Jul 2010 | A1 |
| 20100178903 | Tofighbakhsh et al. | Jul 2010 | A1 |
| 20100194762 | Latta et al. | Aug 2010 | A1 |
| 20100197390 | Craig et al. | Aug 2010 | A1 |
| 20100198514 | Miralles | Aug 2010 | A1 |
| 20100199230 | Latta et al. | Aug 2010 | A1 |
| 20100208032 | Kweon | Aug 2010 | A1 |
| 20100228985 | Kim et al. | Sep 2010 | A1 |
| 20100235017 | Peltonen et al. | Sep 2010 | A1 |
| 20100249545 | Copeland et al. | Sep 2010 | A1 |
| 20100250022 | Hines et al. | Sep 2010 | A1 |
| 20100277278 | Courouble et al. | Nov 2010 | A1 |
| 20100277723 | Rezac et al. | Nov 2010 | A1 |
| 20100286859 | Feigh et al. | Nov 2010 | A1 |
| 20100292873 | Duggan et al. | Nov 2010 | A1 |
| 20100292874 | Duggan et al. | Nov 2010 | A1 |
| 20100301057 | Tattam et al. | Dec 2010 | A1 |
| 20100302395 | Mathe et al. | Dec 2010 | A1 |
| 20100303289 | Polzin et al. | Dec 2010 | A1 |
| 20100306713 | Geisner et al. | Dec 2010 | A1 |
| 20110025689 | Perez et al. | Feb 2011 | A1 |
| 20110035149 | McAndrew et al. | Feb 2011 | A1 |
| 20110035337 | Choi et al. | Feb 2011 | A1 |
| 20110055046 | Bowen et al. | Mar 2011 | A1 |
| 20110061963 | Farwell et al. | Mar 2011 | A1 |
| 20110062281 | Woolley et al. | Mar 2011 | A1 |
| 20110071704 | Matos | Mar 2011 | A1 |
| 20110073707 | Bossert et al. | Mar 2011 | A1 |
| 20110080336 | Leyvand et al. | Apr 2011 | A1 |
| 20110083600 | Whitten et al. | Apr 2011 | A1 |
| 20110085705 | Izadi et al. | Apr 2011 | A1 |
| 20110130913 | Duggan et al. | Jun 2011 | A1 |
| 20110143811 | Rodriguez | Jun 2011 | A1 |
| 20110145059 | Baluja | Jun 2011 | A1 |
| 20110181689 | Kweon | Jul 2011 | A1 |
| 20110184590 | Duggan et al. | Jul 2011 | A1 |
| 20110186657 | Haviland | Aug 2011 | A1 |
| 20110192338 | Goudeau et al. | Aug 2011 | A1 |
| 20110216088 | Leung | Sep 2011 | A1 |
| 20110290937 | Salkeld | Dec 2011 | A1 |
| 20110296505 | Perez et al. | Dec 2011 | A1 |
| 20110314515 | Hernoud | Dec 2011 | A1 |
| 20110315817 | Miralles et al. | Dec 2011 | A1 |
| 20120001474 | Stokes | Jan 2012 | A1 |
| 20120023166 | Kim et al. | Jan 2012 | A1 |
| 20120028624 | Jedlicka | Feb 2012 | A1 |
| 20120038669 | Lee et al. | Feb 2012 | A1 |
| 20120041971 | Kim et al. | Feb 2012 | A1 |
| 20120044043 | Nettleton et al. | Feb 2012 | A1 |
| 20120046818 | Nettleton et al. | Feb 2012 | A1 |
| 20120048171 | Kalwa | Mar 2012 | A1 |
| 20120053703 | Nettleton et al. | Mar 2012 | A1 |
| 20120055390 | Kalwa et al. | Mar 2012 | A1 |
| 20120062123 | Jarrell et al. | Mar 2012 | A1 |
| 20120069131 | Abelow | Mar 2012 | A1 |
| 20120072046 | Tattam | Mar 2012 | A1 |
| 20120073889 | Franzen et al. | Mar 2012 | A1 |
| 20120075433 | Tatzgern et al. | Mar 2012 | A1 |
| 20120076397 | Moresve | Mar 2012 | A1 |
| 20120078882 | Boldyrev et al. | Mar 2012 | A1 |
| 20120092208 | LeMire et al. | Apr 2012 | A1 |
| 20120120361 | Yanagawa et al. | May 2012 | A1 |
| 20120123628 | Duggan et al. | May 2012 | A1 |
| 20120143482 | Goossen et al. | Jun 2012 | A1 |
| 20120158280 | Ravenscroft | Jun 2012 | A1 |
| 20120173018 | Allen et al. | Jul 2012 | A1 |
| 20120185090 | Sanchez | Jul 2012 | A1 |
| 20120185356 | Barron et al. | Jul 2012 | A1 |
| 20120188064 | Mahaffey | Jul 2012 | A1 |
| 20120213212 | Moore et al. | Aug 2012 | A1 |
| 20120223182 | Gilchrist et al. | Sep 2012 | A1 |
| 20120253581 | Anderson | Oct 2012 | A1 |
| 20120267473 | Tao et al. | Oct 2012 | A1 |
| 20120299761 | Tung | Nov 2012 | A1 |
| 20130035790 | Olivier | Feb 2013 | A1 |
| 20130290234 | Harris | Oct 2013 | A1 |
| 20140012417 | Zelivinski | Jan 2014 | A1 |
| 20140032034 | Raptopoulos et al. | Jan 2014 | A1 |
| 20160066189 | Mahaffey | Mar 2016 | A1 |
| 20160234342 | Oonk | Aug 2016 | A1 |
| Number | Date | Country |
|---|---|---|
| 2008203834 | Aug 2008 | AU |
| 2009201713 88 | May 2009 | AU |
| 2009217390 62 | Oct 2009 | AU |
| 2011100304 | Mar 2011 | AU |
| 2011235925 | Oct 2011 | AU |
| 2012100373 | Apr 2012 | AU |
| 2011277414 | Oct 2012 | AU |
| 2011332763 | Oct 2012 | AU |
| 2011337752 | Oct 2012 | AU |
| WO1997038548 | Oct 1997 | WO |
| WO2008097264 | Aug 2008 | WO |
| WO2011075707 | Jun 2011 | WO |
| WO2011104716 | Sep 2011 | WO |
| WO2011142859 | Nov 2011 | WO |
| Entry |
|---|
| International Search Report for PCT/AU213/001303, ISA/AU, Woden ACT, dated Dec. 20, 2013. |
| Number | Date | Country | |
|---|---|---|---|
| 20150336270 A1 | Nov 2015 | US |
