Entities are increasingly using digital video to monitor various locations. This can be used to monitor occurrences such as traffic congestion or the actions of people in a particular location. One downside to such an approach is that many approaches still require at least some amount of manual review, which can be expensive and prone to detection errors. In other approaches the video can be analyzed by a set of servers to attempt to detect specific information. Such an approach can be very expensive, however, as a significant amount of bandwidth is needed to transfer the video to the data center or other location for analysis. Further, the analysis is performed offline and following capture and transmission of the video data, which prevents any real-time action from being taken in response to the analysis.
Various embodiments in accordance with the present disclosure will be described with reference to the drawings which are described as follows.
In the following description, various embodiments will be described. For purposes of explanation, specific configurations and details are set forth in order to provide a thorough understanding of the embodiments. However, it will also be apparent to one skilled in the art that the embodiments may be practiced without the specific details. Furthermore, well-known features may be omitted or simplified in order not to obscure the embodiment being described.
Systems and methods in accordance with various embodiments of the present disclosure may overcome one or more of the aforementioned and other deficiencies experienced in conventional approaches to detecting physical objects. In particular, various embodiments provide mechanisms for locating objects of interest, such as people, vehicles, products, logos, fires, and other detectable objects. Various embodiments enable these items to be detected, identified, counted, tracked, monitored, and/or otherwise accounted for through the use of, for example, captured image data. The image data (or other sensor data) can be captured using one or more detection devices as described herein, among other such devices and systems. Various other functions and advantages are described and suggested below as may be provided in accordance with the various embodiments.
There can be many situations where it may be desirable to detect a presence of one or more objects of interest, such as to determine the number of objects in a given location at any time, as well as to determine patterns of motion, behavior, and other such information. This can include, for example, detecting the number of people in a given location, as well as the movement or actions of those people over a period of time. Conventional image or video analysis approaches require the captured image or video data to be transferred to a server or other remote system for analysis. As mentioned, this requires significant bandwidth and causes the data to be analyzed offline and after the transmission, which prevents actions from being initiated in response to the analysis in near real time. Further, in many instances it will be undesirable, and potentially unlawful, to collect information about the locations, movements, and actions of specific people. Thus, transmission of the video data for analysis may not be a viable solution. There are various other deficiencies to conventional approaches to such tasks as well.
Accordingly, approaches in accordance with various embodiments provide systems, devices, methods, and software, among other options, that can provide for the near real time detection and/or tracking of specific types of objects, as may include people, vehicles, products, and the like. Other types of information can be provided that can enable actions to be taken in response to the information while those actions can make an impact, and in a way that does not disclose information about the persons represented in the captured image or video data, unless otherwise instructed or permitted. Various other approaches and advantages will be appreciated by one of ordinary skill in the art in light of the teachings and suggestions contained herein.
In various embodiments, a detection device 100 can be used such as that illustrated in the front view of
The example detection device 100 of
As illustrated, the housing 102 in some embodiments can also be shaped to fit within a mounting bracket 204 or other such mounting apparatus. The mounting bracket can be made of any appropriate materials, such as metal or aluminum, that is sufficiently strong to support the detection device. In this example the bracket can include various attachment mechanisms, as may include openings 206, 212 (threaded or otherwise) for attachment screws or bolts, as well as regions 204, 214 shaped to allow for mounting to a wall, pole, or tripod, among other such options. The bracket illustrated can allow for one-hand installation, such as where the bracket 204 can be screwed to a pole or wall. The detection device 202 can then be installed by placing the detection device into the mounted bracket 204 until dimples 208 extending from the bracket are received into corresponding recesses in the detection device (or vice versa) such that the detection device 202 is held in place on the bracket. This can allow for relatively easy one-handed installation of the device in the bracket, particularly useful when the installation occurs from a ladder to a bracket mounted on a pole or other such location. Once held in place, the device can be securely fastened to the bracket using one or more safety screws, or other such attachment mechanisms, fastened through corresponding openings 210 in the mounting bracket. Various other approaches for mounting the detection device in a bracket, or using a bracketless approach where the device is mounted directly to a location, can be used as well within the scope of the various embodiments. As discussed in more detail later herein, another example mounting approach involves using double-sided tape, or another such adhesive material, with a pre-cut stencil. One side of the tape can be applied to the casing of the detection device during manufacture and assembly, for example, such that when installation is to occur one can peel off or remove an outer silicone paper and press the exposed adhesive on the tape carrier material directly to a window or other light-transmissive surface. As discussed, such an approach can enable the face or lip region of the front of an example device to a window in order for the two cameras 104, 106 to capture light passing through the window glass. The adhesive will also help to form a seal such that external light does not leak into the camera region and get detected by the relevant sensors. Further, while in some embodiments the detection device will include a power cord (or port to receive a power cord), in other embodiments the bracket can function as a docking station wherein a power port on the device mates with a power connection on the bracket (or vice versa) in order to power the device. Other power sources such as battery, solar cells, or wireless charging can be used as well within the scope of the various embodiments.
The detection device can include at least one display element 710. In various examples this includes one or more LEDs or other status lights that can provide basic communication to a technician or other observer of the device. It should be understood, however, that screens such as LCD screens or other types of displays can be used as well within the scope of the various embodiments. In at least some embodiments one or more speakers or other sound producing elements can also be included, which can enable alarms or other type of information to be conveyed by the device. Similarly, one or more audio capture elements such as a microphone can be included as well. This can allow for the capture of audio data in addition to video data, either to assist with analysis or to capture audio data for specific periods of time, among other such options. As mentioned, if a security alarm is triggered the device might capture video data (and potentially audio data if a microphone is included) for subsequent analysis and/or to provide updates on the location or state of the emergency, etc. In some embodiments a microphone may not be included for privacy or power concerns, among other such reasons.
The detection device 702 can include various other components, including those shown and not shown, that might be included in a computing device as would be appreciated to one of ordinary skill in the art. This can include, for example, at least one power component 714 for powering the device. This can include, for example, a primary power component and a backup power component in at least one embodiment. For example, a primary power component might include power electronics and a port to receive a power cord for an external power source, or a battery to provide internal power, among solar and wireless charging components and other such options. The device might also include at least one backup power source, such as a backup battery, that can provide at least limited power for at least a minimum period of time. The backup power may not be sufficient to operate the device for length periods of time, but may allow for continued operation in the event of power glitches or short power outages. The device might be configured to operate in a reduced power state, or operational state, while utilizing backup power, such as to only capture data without immediate analysis, or to capture and analyze data using only a single camera, among other such options. Another option is to turn off (or reduce) communications until full power is restored, then transmit the stored data in a batch to the target destination. As mentioned, in some embodiments the device may also have a port or connector for docking with the mounting bracket to receive power via the bracket.
The device can have one or more network communications components 720, or sub-systems, that enable the device to communicate with a remote server or computing system. This can include, for example, a cellular modem for cellular communications (e.g., LTE, 5G, etc.) or a wireless modem for wireless network communications (e.g., WiFi for Internet-based communications). The device can also include one or more components 718 for “local” communications (e.g., Bluetooth) whereby the device can communicate with other devices within a given communication range of the device. Examples of such subsystems and components are well known in the art and will not be discussed in detail herein. The network communications components 720 can be used to transfer data to a remote system or service, where that data can include information such as count, object location, and tracking data, among other such options, as discussed herein. The network communications component can also be used to receive instructions or requests from the remote system or service, such as to capture specific video data, perform a specific type of analysis, or enter a low power mode of operation, etc. A local communications component 718 can enable the device to communicate with other nearby detection devices or a computing device of a repair technician, for example. In some embodiments, the device may additionally (or alternatively) include at least one input 716 and/or output, such as a port to receive a USB, micro-USB, FireWire, HDMI, or other such hardwired connection. The inputs can also include devices such as keyboards, push buttons, touch screens, switches, and the like.
The illustrated detection device also includes a camera subsystem 722 that includes a pair of matched cameras 724 for stereoscopic video capture and a camera controller 726 for controlling the cameras. Various other subsystems or separate components can be used as well for video capture as discussed herein and known or used for video capture. The cameras can include any appropriate camera, as may include a complementary metal-oxide-semiconductor (CMOS), charge coupled device (CCD), or other such sensor or detector capable of capturing light energy over a determined spectrum, as may include portions of the visible, infrared, and/or ultraviolet spectrum. Each camera may be part of an assembly that includes appropriate optics, lenses, focusing elements, shutters, and other such elements for image capture by a single camera, set of cameras, stereoscopic camera assembly including two matched cameras, or other such configuration. Each camera can also be configured to perform tasks such as autofocusing, zoom (optical or digital), brightness and color adjustments, and the like. The cameras 724 can be matched digital cameras of an appropriate resolution, such as may be able to capture HD or 4K video, with other appropriate properties, such as may be appropriate for object recognition. Thus, high color range may not be required for certain applications, with grayscale or limited colors being sufficient for some basic object recognition approaches. Further, different frame rates may be appropriate for different applications. For example, thirty frames per second may be more than sufficient for tracking person movement in a library, but sixty frames per second may be needed to get accurate information for a highway or other high speed location. As mentioned, the cameras can be matched and calibrated to obtain stereoscopic video data, or at least matched video data that can be used to determine disparity information for depth, scale, and distance determinations. The camera controller 726 can help to synchronize the capture to minimize the impact of motion on the disparity data, as different capture times would cause some of the objects to be represented at different locations, leading to inaccurate disparity calculations.
The example detection device 700 also includes a microcontroller 706 to perform specific tasks with respect to the device. In some embodiments, the microcontroller can function as a temperature monitor or regulator that can communicate with various temperature sensors (not shown) on the board to determine fluctuations in temperature and send instructions to the processor 704 or other components to adjust operation in response to significant temperature fluctuation, such as to reduce operational state if the temperature exceeds a specific temperature threshold or resume normal operation once the temperature falls below the same (or a different) temperature threshold. Similarly, the microcontroller can be responsible for tasks such as power regulation, data sequencing, and the like. The microcontroller can be programmed to perform any of these and other tasks that relate to operation of the detection device, separate from the capture and analysis of video data and other tasks performed by the primary processor 704.
The data from the devices can be received to the communication interface and then directed to a data aggregation server 806, or other such system or service, which can correlate the data received from the various detection devices 802 for a specific region or location. This can include not aggregating the data from the set of devices for a location, but potentially performing other tasks such as time sequencing, device location and overlap determinations, and the like. In some embodiments, such an approach can provide the ability to track a single object through overlapping fields of view of different devices as discussed elsewhere herein. Such a process can be referred to as virtual stitching, wherein the actual image or video data is not stitched together but the object paths or locations are “stitched” or correlated across a large area monitored by the devices. The data aggregation server 806 can also process the data itself, or in combination with another resource of (or external to) the environment 804, to determine appropriate object determination, correlation, count, movement, and the like. For example, if two detection devices have overlapping fields of view, then some objects might be represented in data captured by each of those two devices. The aggregation server 806 can determine that, based on the devices providing the data, the relative orientation and field overlap of the devices, and positions where the object is represented in both sets of data, that the object is the same object represented in both data sets. As mentioned elsewhere herein, one or more descriptor values may also be provided that can help correlate object between frames and/or different fields of view. The aggregation server can then correlate these representations such that the object is only counted once for that location. The aggregation server can also, in at least some embodiments, correlate the data with data from a previous frame in order to correlate objects over time as well. This can help to not only ensure that a single object is only counted once even though represented in multiple video frames over time, but can also help to track motion of the objects through the location where object tracking is of interest. In some embodiments, descriptors or other contextual data for an object (such as the determined hair color, age, gender, height, or shirt color) can be provided as well to help correlate the objects, since only time and coordinate data is otherwise provided in at least some embodiments. Other basic information may be provided as well, such as may include object type (e.g., person or car) or detection duration information. Information from the analysis can then be stored to at least one data store 810. The data stored can include the raw data from the devices, the aggregated or correlated data from the data aggregation server, report data generated by a reporting server or application, or other such data. The data stored in some embodiments can depend at least in part upon the preferences or type of account of a customer of the data service provider who pays or subscribes to receive information based on the data provided by the detection devices 802 at the particular location. In some embodiments, basic information such as the raw data is always stored, with count, tracking, report, or other data being configurable or selectable by one or more customers or other such entities associated with account.
In order to obtain the data, a request can be submitted from various client devices 816, 818 to an interface layer 812 of the data service provider environment. The interface can include any appropriate interface, such as may correspond to a network address or application programming interface (API). The communication interface 808 for communicating with the detection devices 808 can be part of, or separate from, this interface layer. In some embodiments the client devices 816, 818 may be able to submit requests that enable the detection device data to be sent directly to the client devices 816, 818 for analysis. The client devices can then use a corresponding user interface, application, command prompt, or other such mechanism to obtain the data. This can include, for example, obtaining the aggregated and correlated data from the data store or obtaining reports generated based on that data, among other such options. Customized reports or interfaces can be provided that enable customers or authorized users to obtain the information of interest. The client devices can include any appropriate devices operable to send and receive requests, messages, or information over an appropriate network and convey information back to a user of the device. Examples of such client devices include personal computers, smart phones, handheld messaging devices, wearable computers, desktop computers, notebook computers, tablets, and the like. Such an approach enables a user to obtain the data of interest, as well as to request further information or new types of information to be collected or determined. It should be understood that although many components are shown as part of a data service provider environment 804 that the components can be part of various different environments, associated with any of a number of different entities, or associated with no specific environment, among other such options.
In at least some embodiments at least one valid credential will need to be provided in order to access the data from the data service provider environment 804. This can include, for example, providing a username and password to be authenticated by the data service environment (or an identity management service in communication with the environment, for example) that is valid and authorized to obtain or access the data, or at least a portion of the data, under the terms of the corresponding customer account. In some embodiments a customer will have an account with the data service provider, and user can obtain credentials under permission from the customer account. In some embodiments the data may be encrypted before storage and/or transmission, where the encryption may be performed using a customer encryption key or asymmetric key pair, among other such options. The data may also be transferred using a secure transmission protocol, among other such options.
In this example, the cameras capture video data which can then be processed by at least one processor on the detection device. The object recognition process can detect objects in the video data and then determine which of the objects correspond to objects of interest, in this example corresponding to people. The process can then determine a location of each person, such as by determining a boundary, centroid location, or other such location identifier. The process can then provide this data as output, where the output can include information such as an object identifier, which can be assigned to each unique object in the video data, a timestamp for the video frame(s), and coordinate data indicating a location of the object at that timestamp. In one embodiment, a location (x, y, z) timestamp (t) can be generated as well as a set of descriptors (d1, d2, . . . ) specific to the object or person being detected and/or tracked. Object matching across different frames within a field of view, or across multiple fields of view, can then be performed using a multidimensional vector (e.g., x, y, z, t, d1, d2, d3, . . . ). The coordinate data can be relative to a coordinate of the detection device or relative to a coordinate set or frame of reference previously determined for the detection device. Such an approach enables the number and location of people in the region of interest to be counted and tracked over time without transmitting, from the detection device, any personal information that could be used to identify the individual people represented in the video data. Such an approach maintains privacy and prevents violation of various privacy or data collection laws, while also significantly reducing the amount of data that needs to be transmitted from the detection device.
As illustrated, however, the video data and distance information will be with respect to the cameras, and a plane of reference 906 of the cameras, which can be substantially parallel to the primary plane(s) of the camera sensors. For purposes of the coordinate data provided to a customer, however, the customer will often be more interested in coordinate data relative to a plane 908 of the region of interest, such as may correspond to the floor of a store or surface of a road or sidewalk that can be directly correlated to the physical location. Thus, in at least some embodiments a conversion or translation of coordinate data is performed such that the coordinates or position data reported to the customer corresponds to the plane 908 (or non-planar surface) of the physical region of interest. This translation can be performed on the detection device itself, or the translation can be performed by a data aggregation server or other such system or service discussed herein that receives the data, and can use information known about the detection device 902, such as position, orientation, and characteristics, to perform the translation when analyzing the data and/or aggregating/correlating the data with data from other nearby and associated detection devices. Mathematical approaches for translating coordinates between two known planes of reference are well known in the art and, as such, will not be discussed in detail herein.
The locations of the specific objects can be tracked over time, such as by monitoring changes in the coordinate information determined for a sequence of video frames over time. As an example,
In other embodiments the occurrence may be logged for subsequent analysis, such as to determine where such occurrences are taking place in order to make changes to reduce the frequency of such occurrences. If in a store situation, such movement data can alternatively be used to determine how men and women move through a store, such that the store can optimize the location of various products or attempt to place items to direct the persons to different regions in the store. The data can also help to alert when a person is in a restricted area or otherwise doing something that should generate an alarm, alert, notification, or other such action.
In various embodiments, some amount of image pre-processing can be performed for purposes of improving the quality of the image, as may include filtering out noise, adjusting brightness or contrast, etc. In cases where the camera might be moving or capable of vibrating or swaying on a pole, for example, some amount of position or motion compensation may be performed as well. Background subtraction approaches that can be utilized with various embodiments include mean filtering, frame differencing, Gaussian average processing, background mixture modeling, mixture of Gaussians (MoG) subtraction, and the like. Libraries such as the OPEN CV library can also be utilized to take advantage of the conventional background and foreground segmentation algorithm.
Once the foreground portions or “blobs” of image data are determined, those portions can be processed using a computer vision algorithm for object recognition or other such process. Object recognition typically makes use of one or more classifiers that have been trained to recognize specific types of categories of objects, such as people, cars, bicycles, and the like. Algorithms used for such purposes can include convolutional or other deep neural networks (DNNs), as may utilize one or more feature extraction libraries for identifying types of feature points of various objects. In some embodiments, a histogram or oriented gradients (HOG)-based approach uses feature descriptors for object detection, such as by counting occurrences of gradient orientation in localized portions of the image data. Other approaches that can be used take advantage of features such as edge orientation histograms and shape contexts, as well as scale- and rotation-invariant feature transform descriptors, although these approaches may not provide the same level of accuracy for at least some data sets.
In some embodiments, an attempt to classify objects that does not require precision can rely on the general shapes of the blobs or foreground regions. For example, there may be two blobs detected that correspond to different types of objects. The first blob can have an outline or other aspect determined that a classifier might indicate corresponds to a human with 85% certainty. Certain classifiers might provide multiple confidence or certainty values, such that the scores provided might indicate an 85% likelihood that the blob corresponds to a human and a 5% likelihood that the blob corresponds to an automobile, based upon the correspondence of the shape to the range of possible shapes for each type of object, which in some embodiments can include different poses or angles, among other such options. Similarly, a second blob might have a shape that a trained classifier could indicate has a high likelihood of corresponding to a vehicle. For situations where the objects are visible over time, such that additional views and/or image data can be obtained, the image data for various portions of each blob can be aggregated, averaged, or otherwise processed in order to attempt to improve precision and confidence. As mentioned elsewhere herein, the ability to obtain views from two or more different cameras can help to improve the confidence of the object recognition processes.
Where more precise identifications are desired, the computer vision process used can attempt to locate specific feature points as discussed above. As mentioned, different classifiers can be used that are trained on different data sets and/or utilize different libraries, where specific classifiers can be utilized to attempt to identify or recognize specific types of objects. For example, a human classifier might be used with a feature extraction algorithm to identify specific feature points of a foreground object, and then analyze the spatial relations of those feature points to determine with at least a minimum level of confidence that the foreground object corresponds to a human. The feature points located can correspond to any features that are identified during training to be representative of a human, such as facial features and other features representative of a human in various poses. Similar classifiers can be used to determine the feature points of other foreground object in order to identify those objects as vehicles, bicycles, or other objects of interest. If an object is not identified with at least a minimum level of confidence, that object can be removed from consideration, or another device can attempt to obtain additional data in order to attempt to determine the type of object with higher confidence. In some embodiments the image data can be saved for subsequent analysis by a computer system or service with sufficient processing, memory, and other resource capacity to perform a more robust analysis.
After processing using a computer vision algorithm with the appropriate classifiers, libraries, or descriptors, for example, a result can be obtained that is an identification of each potential object of interest with associated confidence value(s). One or more confidence thresholds or criteria can be used to determine which objects to select as the indicated type. The setting of the threshold value can be a balance between the desire for precision of identification and the ability to include objects that appear to be, but may not be, objects of a given type. For example, there might be 1,000 people in a scene. Setting a confidence threshold too high, such as at 99%, might result in a count of around 100 people, but there will be a very high confidence that each object identified as a person is actually a person. Setting a threshold too low, such as at 50%, might result in too many false positives being counted, which might result in a count of 1,500 people, one-third of which do not actually correspond to people. For applications where approximate counts are desired, the data can be analyzed to determine the appropriate threshold where, on average, the number of false positives is balanced by the number of persons missed, such that the overall count is approximately correct on average. For many applications this can be a threshold between about 60% and about 85%, although as discussed the ranges can vary by application or situation.
The ability to recognize certain types of objects of interest, such as pedestrians, bicycles, and vehicles, enables various types of data to be determined that can be useful for a variety of purposes. As mentioned, the ability to count the number of cars stopped at an intersection or people in a crosswalk can help to determine the traffic in a particular area, and changes in that count can be monitored over time to attempt to determine density or volume as a factor of time. Tracking these objects over time can help to determine aspects such as traffic flow and points of congestion. Determining irregularities in density, behavior, or patterns can help to identify situations such as accidents or other unexpected incidents.
The ability to obtain the image data and provide data regarding recognized objects could be offered as a standalone system that can be operated by agencies or entities such as traffic departments and other governmental agencies. The data also can be provided as part of a service, whereby an organization collects and analyzes the image data, and provides the data as part of a one-time project, ongoing monitoring project, or other such package. The customer of the service can specify the type of data desired, as well as the frequency of the data or length of monitoring, and can be charged accordingly. In some embodiments the data might be published as part of a subscription service, whereby a mobile app provider or other such entity can obtain a subscription in order to publish or obtain the data for purposes such as navigation and route determination. Such data also can be used to help identify accidents, construction, congestion, and other such occurrences.
As mentioned, many of the examples herein utilize image data captured by one or more detection devices with a view of an area of interest. In addition to one or more digital still image or video cameras, these devices can include infrared detectors, stereoscopic cameras, thermal sensors, motion sensors, proximity sensors, and other such sensors or components. The image data captured can include one or more images, or video, indicating pixel values for pixel locations of the camera sensor, for example, where the pixel values can represent data such as the intensity or color of ambient, infrared IR, or ultraviolet (UV) radiation detected by the sensor. A device may also include non-visual based sensors, such as radio or audio receivers, for detecting energy emanating from various objects of interest. These energy sources can include, for example, cell phone signals, voices, vehicle noises, and the like. This can include looking for distinct signals or a total number of signals, as well as the bandwidth, congestion, or throughput of signals, among other such options. Audio and other signature data can help to determine aspects such as type of vehicle, regions of activity, and the like, as well as providing another input for counting or tracking purposes. The overall audio level and direction of the audio can also provide an additional input for potential locations of interest.
In some embodiments, a detection device can include an active, structured-light sensor. Such an approach can utilize a set of light sources, such as a laser array, that projects a pattern of light of a certain wavelength, such as in the infrared (IR) spectrum, that may not be detectable by the human eye. One or more structured light sensors can be used, in place of or in addition to the ambient light camera sensors, to detect the reflected IR light. In some embodiments sensors can be used that detect light over the visible and infrared spectrums. The size and placement of the reflected pattern components can enable the creation of a three-dimensional mapping of the objects within the field of view. Such an approach may require more power, due to the projection of the IR pattern, but may provide more accurate results in certain situations, such as low light situations or locations where image data is not permitted to be captured, etc.
It should be understood that information about the objects themselves can also be determined using approaches discussed and suggested herein. For example,
If, however, one or more objects of interest are detected in the image data, the objects can be analyzed to determine relevant information. In the example process the objects will be analyzed individually for purposes of explanation, but it should be understood that object data can be analyzed concurrently as well in at least some embodiments. An object of interest can be selected 1410 and at least one descriptor for that object can be determined 1412. The types of descriptor in some embodiments can depend at least in part upon the type of object. For example, a human object might have descriptors relating to height, clothing color, gender, or other aspects discussed elsewhere herein. A vehicle, however, might have descriptors such as vehicle type and color, etc. The descriptors can vary in detail, but should be sufficiently specific such that two objects in similar locations in the area can be differentiated based at least in part upon those descriptors. The disparity data for the object, from the image feature data correlated from each of the stereo cameras in this example, can be utilized to determine 1412 distance information for the object. As mentioned, a centroid or other point may be determined as a tracking point for the object, and the disparity information used to determine a distance from the detection device to that representative point. In some embodiments the disparity data can be used to determine dimensional data as well, such as height or length data, which can be returned as some of the descriptor data in at least some embodiments. The disparity data can also be used along with the location of the object in the image data to determine 1416 coordinates for the object in a reference plane for the monitored area. As mentioned, the image plane of the cameras will be different than the plane of interest for the area, as may correspond to the ground or a floor plane, such that some coordinate transform may need to be performed to determine the coordinates for the object with respect to the plane of reference. As mentioned, the area of interest can have been mapped during a calibration or setup process such that the distance and point location information can be used to determine the coordinates in the relevant coordinate system. The process can be repeated for the next object if it is determined 1418 that there are more objects of interest in the image data. Otherwise, the coordinate, descriptor, and timestamp data for the objects can be transmitted 1420 from the detection device to the specified location, such as an address associated with a remote monitoring service. The information in at least some embodiments will be transmitted in one batch per analyzed image frame, although other groupings can be used as well within the scope of the various embodiments. The image data for the frame can also be deleted 1422 once analyzed, either immediately or after some period of time, such that no personal or identifying data can be extracted from the device by an unauthorized entity.
For each object detected in the captured data, the object can be selected 1508 for further analysis. As with the prior described process of
Client devices used to perform aspects of various embodiments can include any appropriate devices operable to send and receive requests, messages, or information over an appropriate network and convey information back to a user of the device. Examples of such client devices include personal computers, smart phones, handheld messaging devices, wearable computers, laptop computers, and the like. The network can include any appropriate network, including an intranet, the Internet, a cellular network, a local area network (LAN), or any other such network or combination thereof. Components used for such a system can depend at least in part upon the type of network and/or environment selected. Protocols and components for communicating via such a network are well known and will not be discussed herein in detail. Various aspects can be implemented as part of at least one service or Web service, such as may be part of a service-oriented architecture. In embodiments utilizing a Web server, the Web server can run any of a variety of server or mid-tier applications, including HTTP servers, FTP servers, CGI servers, data servers, Java servers, and business application servers. The server(s) also may be capable of executing programs or scripts in response requests from user devices, such as by executing one or more Web applications that may be implemented as one or more scripts or programs written in any appropriate programming language.
Storage media and other non-transitory computer readable media for containing code, or portions of code, can include any appropriate media known or used in the art, including storage media and communication media, such as but not limited to volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules, or other data, including RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by the a system device. Based on the disclosure and teachings provided herein, a person of ordinary skill in the art will appreciate other ways and/or methods to implement the various embodiments.
The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense. It will, however, be evident that various modifications and changes may be made thereunto without departing from the broader spirit and scope of the invention as set forth in the claims.