Patent Grant
11671423
Embodiments of the present disclosure relate generally to user authentication and, more particularly, but not by way of limitation, to authentication via a camera of a computing device accessing an identity verification image.
Quick Response (QR) codes, and other optical barcodes, are a convenient way to share small pieces of information with users of mobile devices, wearable devices, and other smart devices. Typically, an optical barcode uses a finder pattern for identification of the optical barcode. Conventional finder patterns commonly use multiple generic markings conspicuously placed within the optical barcode. Such conspicuous and generic markings can be unsightly and often serve no purpose other than to function as a finder pattern.
Users typically access web-based resources by creating a username and a password with a provider of the resource. A user typically accesses a website associated with the provider and enters his/her username and password to view the resource. However, as users create more and more passwords at more and more providers, users tend to forget their passwords, reducing the security provided by the passwords and creating additional difficulty in accessing the resources. As the foregoing illustrates, a new approach to user authentication for access to a web-based resource may be desirable.
Various ones of the appended drawings merely illustrate example embodiments of the present disclosure and should not be considered as limiting its scope.
The description that follows includes systems, methods, techniques, instruction sequences, and computing machine program products that embody illustrative embodiments of the disclosure. In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide an understanding of various embodiments of the inventive subject matter. It will be evident, however, to those skilled in the art, that embodiments of the inventive subject matter may be practiced without these specific details. In general, well-known instruction instances, protocols, structures, and techniques are not necessarily shown in detail.
QR codes, and other optical barcodes (e.g., Universal Product Code (UPC) barcodes, Aztec code, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code), are a convenient way to share small pieces of information with users of mobile devices, wearable devices, and other smart devices. For instance, QR codes are two-dimensional optical barcodes that encode information readable by a device (e.g., a smart phone) equipped with a camera sensor. Typically, a QR code includes one or more functional patterns such as a finder pattern used for identification and recognition of the QR code or an alignment pattern used to facilitate decoding. Conventional finder patterns comprise multiple markings that are generic in design such as square marks placed in all corners except the bottom right corner (as is the case with a QR code). These finder patterns are absent aesthetic elements such as curves, non-uniformities, and other stylistic elements and often conform to a particular standard to promote open use of the optical barcode.
In various example embodiments, an optical barcode that uses custom patterns, or non-standard functional patterns, provides users with an aesthetically pleasing, branded barcode that allows for an exclusive experience associated with the optical barcode. For example, an entity logo (e.g., a logo of a company, organization, or individual) can be used as a finder pattern, and in some instances an alignment pattern, to create a branded and exclusive optical barcode that is machine-readable using software provided by the entity. In a specific example, a “snapcode” is an optical barcode that uses the SNAPCHAT® logo as a functional pattern. Alternatively, a “snapcode” may include an image of a face, such as a photograph of a human face or an avatar of a human face, in place of the SNAPCHAT® logo.
In an example embodiment, a custom pattern system receives image data representing an image from a user device. For example, the custom pattern system receives the image data from an optical sensor (e.g., a camera sensor) of a smart phone of the user. In various embodiments, the image data from the user device is received in response to a user-initiated image capture, a periodic monitoring of image data being detected by the optical sensor of the user device, an access of stored image data, or a combination thereof. A portion of the image data can include data representing an optical barcode employing a custom graphic for a particular functional pattern (e.g., a finder pattern). In some scenarios, the image data includes extraneous or irrelevant data along with the data pertaining to the optical barcode (e.g., an image of an optical barcode includes a background that is not pertinent to decoding the optical barcode). In a specific example, the optical sensor of the user device captures an image of a promotional poster that includes a particular optical barcode. The image of the promotional poster can include the particular optical barcode along with irrelevant portions of the promotional poster or background that surrounds the particular optical barcode.
After the custom pattern system receives the image data, the custom pattern system searches the image data of the image for the custom graphic to determine whether the image includes the optical barcode. That is to say, the custom graphic, which may be an image of a face, such as a photograph of human face or an avatar or drawing of a human face, is used as a finder pattern for recognition, identification, or detection of the optical barcode within the image. In an example embodiment, the custom pattern system searches for the custom graphic by extracting a candidate shape feature, or multiple candidate shape features, from the image data. For example, the custom pattern system performs an edge detection technique, or another image processing technique, to identify the candidate shape feature such as a contour line of the image. The custom pattern system then determines whether the candidate shape feature satisfies shape feature rules or criteria. For instance, if a particular candidate shape feature is a contour line, the custom pattern system can determine whether the contour line is an enclosed line that encircles a portion of the image. Consistent with some embodiments, the shape feature rules filter out irrelevant or extraneous candidate shape features or candidate shape features with a low probability of being the custom graphic.
In response to the candidate shape feature satisfying the shape feature rules, the custom pattern system identifies the custom graphic by comparing the candidate shape feature with a reference shape feature of the custom graphic. For example, the custom pattern system can compare an area or size of the candidate shape feature with a reference area or size of the reference shape feature. In this example, the custom pattern system identifies the custom graphic based on a match or near match (e.g., a percentage match above a threshold) between the candidate shape feature and the reference shape feature. In this way, the custom pattern system uses the custom graphic as a finder pattern to identify the presence of the optical barcode within a portion of the image.
In further example embodiments, the custom graphic or the recognized face functions as an alignment pattern to facilitate the custom pattern system decoding the data encoded in the optical barcode. In an example embodiment, the custom pattern system extracts spatial attributes of the custom graphic in the image from the image data. For example, the custom pattern system extracts a position, scale, or orientation of the custom graphic from the image data. The custom pattern system decodes data encoded in the image from the image data using the spatial attributes of the custom graphic in the image. For instance, the custom pattern system can perform an image transform using the spatial attributes (e.g., a de-skew, a rotation, a scale, or another type of image transform) to improve detectability/readability of data encoded in a portion of the image. In this way, the custom pattern system uses the custom graphic as an alignment pattern to facilitate decoding the optical barcode.
Accordingly, the custom pattern system uses the custom graphic as a functional pattern of the optical barcode without utilizing conventional functional patterns. Using the custom graphic as a functional pattern allows for an aesthetically pleasing design and can provide exclusivity to a particular software application as the functional pattern does not necessarily conform to an open standard and thus is readable exclusively by the particular software application.
Some aspects of the subject technology relate to user authentication for access to a web-based resource. Some aspects may be implemented in conjunction with a web server that hosts the web based resource and an authentication server that authenticates the user. The user may access the web-based resource through a computing device (e.g., a public computer at a library) different from the user's mobile device. The authentication server may be associated with a service that the user accesses on his/her mobile device, such as an instant messaging program that stores some identity information (e.g., name, telephone, email, etc.) of the user.
According to some implementations, the authentication server transmits, to a mobile device of the user (e.g., via the instant messaging application) an identity verification image. The identity verification image may be transmitted in response to a request by the user either through a computing device accessing the web server or through the mobile device (e.g., through the instant messaging application of the mobile device). The authentication server transmits, to the computing device via the web server, an instruction to scan, via a camera of the computing device, the identity verification image transmitted to the mobile device. The user, in response to instructions presented via a display unit of either the mobile device or the computing device, places the display unit of the mobile device in front of the camera of the computing device for scanning. The authentication server receives, from the computing device and via the web server, a scanned image that was scanned via the camera of the computing device. The authentication server determines whether the scanned image includes data from the identity verification image. If the scanned image includes the data from the identity verification image, the authentication server transmits, to the web server, an indication that the user's identity has been verified. If the scanned image does not include the data from the identity verification image, the authentication server transmits, to the web server, an indication that the user's identity has not been verified.
As used herein, the phrase “the scanned image includes (or comprises) data from the identity verification image” encompasses its plain and ordinary meaning. The scanned image includes the data from the identity verification image if the data (e.g., dots, lines, or other markings) in the scanned image can be decoded to yield indicia that the scanned image corresponds to the identity verification image. The scanned image includes the data from the identity verification image if data from the scanned image can be decoded to yield data which was encoded into the identity verification image.
The scanned image may be said to include (or comprise) the data from the identity verification image if it includes an image of a display unit that shows the image. However, the scanned image may be blurred, tilted, rotated, or the like, relative to the original identity verification image as the scanned image is created by a user who may be inexperienced in photography, using low quality equipment, working in non-ideal lighting conditions for photography, trying to make the scan as quickly as possible, and so forth.
As shown in
As shown in
The client device(s) 110 can execute conventional web browser applications or applications (also referred to as “apps”) that have been developed for a specific platform to include any of a wide variety of mobile computing devices and mobile-specific operating systems (e.g., IOS™, ANDROID™, WINDOWS® PHONE). In an example, the client device(s) 110 are executing the client application(s) 112. The client application(s) 112 can provide functionality to present information to a user 106 and communicate via the network 104 to exchange information with the social messaging system 130. Each client device 110 can comprise a computing device that includes at least a display and communication capabilities with the network 104 to access the social messaging system 130. The client device(s) 110 comprise, but are not limited to, remote devices, work stations, computers, general purpose computers, Internet appliances, hand-held devices, wireless devices, portable devices, wearable computers, cellular or mobile phones, personal digital assistants (PDAs), smart phones, tablets, ultrabooks, netbooks, laptops, desktops, multi-processor systems, microprocessor-based or programmable consumer electronics, game consoles, set-top boxes, network personal computers (PCs), mini-computers, and the like. Users 106 can include a person, a machine, or other means of interacting with the client devices 110. In some embodiments, the users 106 interact with the social messaging system 130 via the client device(s) 110.
As shown in
An individual can register with the social messaging system 130 to become a member of the social messaging system 130. Once registered, a member can form social network relationships (e.g., friends, followers, or contacts) on the social messaging system 130 and interact with a broad range of applications provided by the social messaging system 130.
The application logic layer 126 includes various application logic modules 150, which, in conjunction with the interface module(s) 140, generate various user interfaces with data retrieved from various data sources or data services in the data layer 128. Individual application logic module(s) 150 may be used to implement the functionality associated with various applications, services, and features of the social messaging system 130. For instance, a social messaging application can be implemented with one or more of the application logic module(s) 150. The social messaging application provides a messaging mechanism for users of the client device(s) 110 to send and receive messages that include text and media content such as pictures and video. The client device(s) 110 may access and view the messages from the social messaging application for a specified period of time (e.g., limited or unlimited). In an example, a particular message is accessible to a message recipient for a predefined duration (e.g., specified by a message sender) that begins when the particular message is first accessed. After the predefined duration elapses, the message is deleted and is no longer accessible to the message recipient. Of course, other applications and services may be separately embodied in their own application logic module(s) 150.
As illustrated in
The communication module 210 provides various communications functionality. For example, the communication module 210 receives, accesses, or otherwise obtains image data of an image from a user device. In a specific example, the communication module 210 receives substantially real-time image data from a camera sensor of a smart phone (e.g., a single frame of image data or a continuous stream of frames captured by a camera sensor of the smart phone). The communication module 210 exchanges network communications with the database server(s) 132, the client device(s) 110, and the third party server(s) 120. The information retrieved by the communication module 210 includes data associated with the user (e.g., user 106) (e.g., member profile data from an online account or social network service data) or other data to facilitate the functionality described herein.
The presentation module 220 provides various presentation and user interface functionality operable to interactively present and receive information to and from the user. For instance, the presentation module 220 is utilizable to present user interfaces generated in response to decoding the optical barcode. In other instances, the presentation module 220 generates user interfaces that include optical barcode(s). In various embodiments, the presentation module 220 presents or causes presentation of information (e.g., visually displaying information on a screen, acoustic output, haptic feedback). The process of interactively presenting information is intended to include the exchange of information between a particular device and the user. The user may provide input to interact with the user interface in many possible manners, such as alphanumeric, point based (e.g., cursor), tactile, or other input (e.g., touch screen, tactile sensor, light sensor, infrared sensor, biometric sensor, microphone, gyroscope, accelerometer, or other sensors). The presentation module 220 provides many other user interfaces to facilitate functionality described herein. The term “presenting” as used herein is intended to include communicating information or instructions to a particular device that is operable to perform presentation based on the communicated information or instructions.
The finder module 230 provides image processing functionality to identify, recognize, or detect the custom graphic being employed as a finder pattern in the optical barcode. For example, the finder module 230 extracts and analyzes candidate shape features or candidate contour characteristics from image data of the image received from the user device (e.g., the client device(s) 110). The finder module 230 determines satisfaction of various rules or criteria associated with the extracted candidate shape features. The finder module 230 compares the extracted candidate shape features with reference shape features of the custom graphic, or another reference image, to identify the custom graphic included in the image. The finder module 230 can employ a wide variety of schemes and techniques to extract the candidate shape features from the image data of the image and subsequently identify the custom graphic based on an analysis of the candidate shape features. Examples of those techniques are illustrated later with respect to
The alignment module 240 provides image processing functionality to determine an alignment of the optical barcode using the custom graphic. The custom pattern system 160 can use the alignment to facilitate decoding of data encoded in the optical barcode. In this way, the custom graphic functions as an alignment pattern for the optical barcode. For example, the alignment module 240 extracts spatial attributes of the custom graphic in the image from the image data. In various embodiments, the spatial attributes include at least one of position, orientation, scale, or another spatial aspect of the optical barcode. The alignment module 240 determines an alignment of the optical barcode based on the spatial attributes (e.g., a particular orientation of the optical barcode). In an example, the alignment module 240 can determine an alignment including position and orientation based on the spatial attributes and generate a transformed image according to the alignment. The custom pattern system 160 can then use the transformed image to decode data encoded in a portion of the transformed image.
The decoder module 250 provides functionality to decode data encoded in the image using the spatial attributes or the determined alignment of the custom graphic in the image. For instance, the decoder module 250 can decode the data encoded in the image from an image transformed according to the spatial attributes of the custom graphic extracted from image data. In an embodiment, the decoder module 250 detects markings (e.g., high contrast dots, squares, or other marks in the image) representing data encoded in a portion of the image from the image data. In a specific example, the decoder module 250 employs a Reed-Solomon error correction scheme (or other error correction scheme) to decode data encoded in the image. The Reed-Solomon error correction scheme (or other error correction scheme) allows for a successful or valid decoding even when a certain percentage of data could not be decoded from the optical barcode (e.g., damaged bits or incorrectly decoded bits). In some embodiments, the user or an administrator of the custom pattern system 160 configures a tolerance value for an amount of damaged or incorrectly decoded data acceptable when decoding the optical barcode. In some embodiments, the decoder module 250 also provides image processing functionality to improve decoding of the optical barcode. For instance, the decoder module 250, as well as the alignment module 240, can perform image transforms of the image (e.g., perform image sharpening, de-noise processing, other digital filtering, or other image processing techniques to improve decoding accuracy).
The action module 260 provides functionality to perform a variety of actions based on decoding the data encoded in the image. For example, the data encoded in a portion of the image can indicate a particular action or include information to be used in conjunction with a particular action. In a specific example, the data encoded in a portion of the image can comprise a user name, or other user identification, of a member of a social networking service and based on decoding the user name, the action module 260 can perform an action on the social networking service corresponding to the user name (e.g., sending a message to the member associated with the user name). In some embodiments, the action module 260 performs an action specific to a particular app that scans the image (e.g., a function available to a user of the app but otherwise unavailable). In some instances, the action module 260 performs the action without communicating with an external server (e.g., an action locally performed on the user device that scanned the snapcode).
The encoder module 270 provides functionality to generate and encode data into an optical barcode that employs the custom graphic as one or more functional patterns (e.g., generating snapcodes). As discussed above in connection with the decoder module 250, in a specific example, the encoder module 270 can employ a technique such as Reed-Solomon error correction (or other error correction scheme) to encode data. In an example embodiment, the encoder module 270 renders a machine-readable arrangement of marks that represents the data to be encoded. The encoder module 270 can then generate the machine-readable optical barcode using the rendered arrangement of marks and the custom graphic to be used as a functional pattern.
As shown in the diagram 300, the markings 320 are dots that are arranged in a pattern with a particular spacing or positioning readable by a machine. Although the diagram 300 shows the markings 320 as dots, other shapes and marks can be employed (e.g., squares or asymmetric shapes of various geometries). The markings 320 can be arranged in a uniform pattern or a non-uniform pattern. In some instances, the marks can be of different sizes or a uniform size. Additionally, the markings 320 can be in a predetermined arrangement or an arrangement that is dynamically determinable when decoding data from the markings. In some embodiments, the custom graphic 310 and the markings 320 can be surrounded by a bounding shape, such as an outer box 325. Although the outer box 325 of the diagram 300 is shown as a square with rounded corners, the outer box 325 can be in the form of a variety of other shapes with various geometries. Diagram 330 in
Turning now to
In an example embodiment, the user device 414 captures an image of the poster 404 that includes the optical barcode 406. The custom pattern system 160 receives the image data representing the image from the user device 414. In this example embodiment, the custom pattern system 160 is included in the user device 414 (e.g., an application executing on a smart phone of the user 410), although in other example embodiments, the custom pattern system 160 can reside on a server (e.g., a server of the social messaging system 130) that is communicatively coupled with the user device 414. Callout 416 portrays example image processing performed by the finder module 230 to identify the custom graphic in the image and use the custom graphic as an alignment pattern for decoding data included in the optical barcode 406. In the callout 416, the finder module 230 extracts candidate shape features from the image data of the image. Subsequently, the finder module 230 determines if the candidate features meet certain rules and criteria to filter out irrelevant shape features or shape features that have a low probability of being the custom graphic. The finder module 230 can then compare the candidate shape features that meet the shape feature criteria or rules with reference shape features of the custom graphic. In an example, the finder module 230 identifies the custom graphic based on a match between the candidate shape features and the reference shape feature (e.g., a match score that exceeds a threshold).
Subsequent to the finder module 230 identifying the custom graphic, the custom pattern system 160 can use the custom graphic as an alignment pattern for decoding. For instance, the alignment module 240 extracts spatial attributes of the custom graphic in the image and compares the extracted spatial attributes to reference spatial attributes to determine an alignment of the custom graphic. The alignment module 240 or the decoder module 250 may then generate a transformed image of the image according to the alignment (e.g., a rotation or de-skew) as shown in callout 418. After generating the transformed image, the decoder module 250 decodes the data encoded in a portion of the transformed image as shown in callout 420. In the callout 420, the dots of the optical barcode 406 are transformed into data shown as ones for dots and zeros for non-dots, although this is merely an illustrative example and other schemes can be employed. In this way, the custom pattern system 160 uses the custom graphic included in the optical barcode 406 as one or more functional patterns such as a finder pattern or an alignment pattern.
At operation 510, the communication module 210 receives image data of an image from a user device. For example, the communication module 210 receives the image data from an optical sensor (e.g., a camera sensor) of a smart phone of the user (e.g., client device 110 or user device 414). In various embodiments, the image data from the user device (e.g., client device 110 or user device 414) is received in response to a user-initiated image capture, a periodic monitoring of image data being detected by the optical sensor of the user device, or a combination thereof. In some embodiments, the image data represents an image or video being captured by the user device in substantially real time (e.g., a live image feed from a camera sensor of a smart phone). In other embodiments, the image data represents an image captured by the user device, or another device and stored on the user device, from a time in the past (e.g., a still image or video stored on the user device or downloaded from a social networking service). In embodiments where the image data comprises video image data, the custom pattern system 160 can analyze individual frames of the video or a combination of multiple frames of the video to detect and decode the optical barcode. A portion of the image data can include data representing an optical barcode employing a custom graphic, custom symbol, or specific graphic for a particular functional pattern (e.g., a finder pattern or alignment pattern).
In some scenarios, the image data includes extraneous or irrelevant data along with the data pertaining to the optical barcode (e.g., an image of an optical barcode includes a background that is not pertinent to decoding the optical barcode). In a specific example, the optical sensor of the user device captures an image of a movie poster that includes a particular optical barcode. The image of the movie poster can include the particular optical barcode along with irrelevant portions of the movie poster or background that surrounds the particular optical barcode.
At operation 520, the finder module 230 extracts a candidate shape feature or candidate characteristic of the image from the image data. The candidate shape feature can be indicative of an identification of the custom graphic (e.g., include certain traits or characteristics that indicate the custom graphic). For example, the finder module 230 performs an edge detection technique, or another image processing technique, to identify shape features such as contour lines or localized concentrations of color or shading of the image. In some embodiments, the finder module 230 extracts multiple candidate shape features from the image data. In some embodiments, the candidate shape feature includes various shape feature data such as a position of the candidate shape feature relative to a boundary of the image, a brightness of the candidate shape feature relative to the image, an average color of the candidate shape feature, and so forth.
In further example embodiments, the finder module 230 generates a low resolution copy of the image. The finder module 230 can perform various image processing on the low resolution copy of the image, such as a blur (e.g., a Gaussian blur function or another blur function) and a thresholding, to generate a modified low resolution image. The thresholding image process can include adjusting lighter colors (e.g., as determined by a threshold or threshold range) of the low resolution copy of the image to a white color and darker colors (e.g., as determined by a threshold or threshold range) of the low resolution copy of the image to a black color. The finder module 230 can then extract candidate shape features from the modified low resolution image to improve detection of the custom graphic in the image and improve computational efficiency of identifying the custom graphic in the image.
In still further example embodiments, the finder module 230 generates a high resolution copy of a portion of the image. For instance, the finder module 230 can generate the high resolution copy of a particular portion of the image corresponding to the extracted candidate shape feature. The finder module 230, the alignment module 240, or the decoder module 250 can use the high resolution copy for subsequent analysis, as described below, to improve detection, alignment, and decoding results.
At operation 530, the finder module 230 determines that the candidate shape feature satisfies one or more shape feature criteria or rules. For instance, if a particular shape feature is a contour line, the finder module 230 can determine whether the contour line is an enclosed line that encircles a portion of the image. Consistent with some embodiments, the shape feature rule filters out irrelevant or extraneous features. Particular shape feature rules can be directed to or purposed for various objectives. For example, a particular shape feature rule can be purposed to filter out candidate shape features with a low probability of being the custom graphic. In this example, the particular shape feature rule can be specific to the custom graphic. In other examples, some shape feature rules can be purposed to filter out candidate shape features that are unlikely to be associated with the optical barcode. In these examples, the shape feature rule is not necessarily specific to the custom graphic.
At operation 540, in response to the candidate shape feature satisfying the shape feature rule, the finder module 230 identifies the custom graphic or custom symbol in the image by comparing the candidate shape feature with a reference shape feature of the custom graphic or custom symbol. For example, the finder module 230 can compare an area or size of the candidate shape feature with a reference area or size of the reference shape feature. In this example, the finder module 230 identifies the custom graphic based on a match or near match (e.g., a percentage match above a threshold) between the candidate shape feature and the reference shape feature. In this way, the finder module 230 uses the custom graphic, or at least a portion of the custom graphic, as a finder pattern to identify the presence of the optical barcode within a portion of the image.
In some embodiments, the finder module 230 extracts multiple candidate shape features from the image data. In these embodiments, the finder module 230 scores each candidate shape feature and ranks the multiple candidate shape features according to respective scores. For example, the finder module 230 determines a shape feature score for respective candidate shape features based on a count, or weighted count, of shape feature rules satisfied by the respective candidate shape feature. The finder module 230 can iterate through the ranked candidate shape features starting with the highest scoring candidate shape feature and perform further analysis (e.g., comparing the candidate shape feature to a reference shape feature) to determine that the candidate shape feature is the custom graphic.
In some embodiments, the reference shape feature is predetermined, and in other embodiments, the reference shape feature is dynamically determined. For instance, the finder module 230 can dynamically determine the reference shape feature by analyzing a reference image of the custom graphic. For example, the finder module 230 can perform analysis techniques similar to those for analyzing the image data on the reference image such as calculating the reference area value for a particular feature or characteristic of the reference image. In these embodiments, the finder module 230 dynamically determining the reference shape feature allows for dynamic use of a particular custom graphic as a functional pattern in an optical barcode. For instance, the custom pattern system 160 can be provided (e.g., received at the communication module 210) data representing the reference image or data representing the reference features when the method 500 is performed. In this way, the custom functional patterns do not necessarily have to be fixed prior to performing the method 500.
In further example embodiments, the finder module 230 searches for multiple custom graphics in the image data of the image (e.g., where multiple versions or different custom graphics are employed as functional patterns). In a specific example, the custom graphic can comprise a first company logo and the company may change logos to a second company logo. The custom pattern system 160 can be operable to use the first company logo as a finder pattern and the second company logo as a finder pattern and the custom pattern system 160 can search for each logo when performing the method 500.
In further example embodiments, the finder module 230 identifies the custom graphic in the image in conjunction with other candidate shape features extracted from the image data. For example, the finder module 230 can search for both the custom graphic (e.g., a logo) and an outer box (e.g., the outer box 325) surrounding the custom graphic. In these embodiments, the finder module 230 identifies a combination of the custom graphic and one or more additional candidate shape features extracted from the image data.
At operation 550, in response to identifying the custom graphic, the alignment module 240 extracts a spatial attribute, geometry attribute, or spatial property of the custom graphic or custom symbol in the image from the image data. For example, the alignment module 240 extracts a position, scale, or orientation of the custom graphic from the image data. In various example embodiments, the spatial attribute is indicative of an orientation of the custom graphic in the image. The alignment module 240 or the decoder module 250 can use the spatial attribute to facilitate decoding the optical barcode.
In further embodiments, the alignment module 240 extracts a spatial attribute, geometry attribute, or spatial property of another candidate shape feature extracted from the image data of the image. For example, the alignment module 240 extracts a spatial attribute of the outer box (e.g., the outer box 325 of
At operation 560, the decoder module 250 decodes data encoded in a portion of the image from the image data using the spatial attribute of the custom graphic in the image. For instance, the decoder module 250 can perform an image transform using the spatial attributes (e.g., a de-skew, a rotation, a scale, or another type of image transform) to improve detectability or readability of data encoded in a portion of the image. In an example embodiment, the decoder module 250 decodes the data encoded in the portion of the image by detecting marking (e.g., dots, squares, or another marking) indicative of data included in the image. In this way, the decoder module 250 uses the custom graphic, or at least a portion of the custom graphic, as an alignment pattern to facilitate decoding the optical barcode. In various embodiments, the decoder module 250 employs a Reed-Solomon error correction scheme (or other error correction scheme) to decode data encoded in the image. The Reed-Solomon error correction scheme (or other error correction scheme) allows for a successful decoding of the data encoded in the image with a certain percentage of data encoded in the image being corrupt, damaged, or incorrectly decoded. In further embodiments, the decoder module 250 uses a small checksum to verify that the value decoded from the image data is a value that includes real data rather than just random data (e.g., random bits).
In further example embodiments, the decoder module 250 rejects certain results of the decoded data (e.g., results of data decoded from the image data known to be invalid as specified by an administrator of the custom pattern system 160). For example, the decoder module 250 can reject decoded data that includes all zeros, all ones, or another specified result even though the decoded data passed other data integrity tests (e.g., error correction and checksumming). For example, this can occur when the custom pattern system 160 scans the custom graphic without any associated markings that indicate data (e.g., where the custom graphic is a logo, simply scanning the logo may yield all zeros in the decoded data and may be rejected by the decoder module 250). In a specific example, scanning the icon associated with social messaging app 1908, shown below in
At operation 610, the finder module 230 determines that the candidate shape feature comprises an enclosed line from the image data. That is to say, the shape feature rule comprises a path rule, and the finder module 230 determines that the candidate shape feature satisfies the path rule. The finder module 230 can employ a variety of techniques to determine that the candidate shape feature satisfies the path rule.
At operation 630, the finder module 230 determines whether the candidate shape feature is an enclosed line by determining that the candidate shape feature encircles a portion of the image by having a path that starts at a particular point and returns to the same particular point. In an example embodiment, if the candidate shape feature does not satisfy the path rule (indicated by “no” in
To illustrate the concepts of
Callout 740 shows a particular candidate shape feature of the candidate shape features 730. The callout 740 shows a contour line 750 (illustrated as a dotted line) of the particular candidate shape feature, a path 760, and a point 770 of the particular candidate shape feature. In the callout 740, the finder module 230 determines that the path rule is met if the path 760 that starts at the point 770 can follow the contour line 750 and return to the point 770. In the diagram 700, the particular candidate shape feature shown in the callout 740 does satisfy the path rule since the path 760 can follow the contour line 750 and return to the point 770.
At operation 810, the finder module 230 calculates an area value or size approximation of the candidate shape feature. For example, the finder module 230 uses a proxy shape such as a polygon (e.g., a square, a rectangle, or a quadrilateral) or a non-polygonal shape (e.g., an ellipse) to approximate the shape of the candidate shape feature. The finder module 230 fits or nearly fits the proxy shape to the outer edges or outer perimeter of the candidate shape feature so that the proxy shape is representative of an area of the candidate shape feature. Subsequently, the finder module 230 calculates the area value of the proxy shape to determine the area value or size approximation of the candidate shape feature. In some embodiments, the finder module 230 employs such a technique (e.g., polygonal area approximation) to avoid a computationally expensive area calculation of the candidate shape feature in situations where the candidate shape feature is likely to be complex in shape (e.g., an area calculation for a non-uniform or irregular shaped feature is typically more computationally expensive). In some embodiments, other techniques such as pixel-based counting can be employed to determine the area value.
At operation 820, the finder module 230 determines an area score or size score of the candidate shape feature. The finder module 230 determines the area score by comparing the area value of the candidate shape feature with a reference area value. In some embodiments, the reference area value comprises an area value of a corresponding proxy shape fitted to a reference image of the custom graphic (e.g., the area value of a proxy shape fitted to the ghost logo from a front view perspective). In other embodiments, the reference area value comprises the area value of the custom graphic (e.g., the area value of the ghost logo). The finder module 230 calculates the area score, for example, by determining a match percentage between the candidate shape feature area value and the reference area value. The finder module 230 can employ a wide variety of other schemes and techniques to calculate the area score.
At operation 830, the finder module 230 determines whether the area score exceeds a threshold. The threshold can be predefined or dynamically determined (e.g., statistically determined based on a rolling historical average of scans).
At operation 840, based on the area score exceeding the threshold (indicated by “yes” in
To further illustrate the concepts of
Callout 908 shows an enlarged portion of the image 902 that includes the candidate shape feature being analyzed by the finder module 230 to identify the custom graphic. In the callout 908, the polygon 910 (e.g., a quadrilateral) is shown fitted to a perimeter of the candidate shape feature. Area value 912 is the area of the polygon 910.
Callout 914 shows a reference image of the custom graphic. Callout 916 shows the spatial orientation of the reference image. In this example, the reference image is shown from the front view perspective. Polygon 918 is shown fitted to a perimeter of the reference image. Reference area value 920 is the area of the polygon 918. Although
The finder module 230 compares the area value 912 with the reference area value 920 to determine that the candidate shape feature satisfies the area rule. Another candidate shape feature of the image 902, such as one of the musical notes of the image 902, would not have an area value that is similar to the reference area value and therefore would not satisfy the area rule. In this way, the finder module 230 can quickly remove or skip certain candidate shape features that are unlikely to be identified as the custom graphic.
At operation 1010, the alignment module 240 extracts a distinctive feature of the custom graphic from the image data where the distinctive feature is indicative of an alignment of the custom graphic (e.g., a particular asymmetry of the custom graphic that can be used to determine an orientation of the custom graphic). For example, the distinctive feature can comprise a distinctive point of the custom graphic, a distinctive curve, a particular asymmetry, a particular non-uniformity, or another characteristic of the custom graphic.
At operation 1020, the alignment module 240 determines an orientation of the custom graphic in the image by comparing the distinctive feature with a reference distinctive feature of the custom graphic. For example, the alignment module 240 maps the extracted distinctive feature of the custom graphic to a reference distinctive feature to determine spatial differences between the distinctive features. In this way, the alignment module 240 can determine an alignment of the custom graphic as compared to a reference image of the custom graphic based on the determined spatial differences.
At operation 1030, the alignment module 240 generates a transformed image by transforming the image according to the orientation of the custom graphic. For instance, the alignment module 240 can rotate, de-skew, scale, or otherwise spatially transform the image to allow for a more accurate decoding of the data in the image.
At operation 1040, the decoder module 250 decodes the data encoded in the image using the orientation and a position of the custom graphic in the image. For example, the decoder module 250 decodes the data encoded in the image from the transformed image. In a specific scenario, the image is transformed to a front view to increase visibility and uniformity of marks in the image that represent data encoded in the image.
To assist in understanding the disclosure of
Callout 1108 shows an enlarged portion of the image 1102 that includes the candidate shape feature being analyzed by the alignment module 240. Callout 1110 shows an enlarged portion of the callout 1108 showing a distinctive feature of the candidate shape feature.
Callout 1112 shows a reference image of the custom graphic. Callout 1114 shows the spatial orientation of the reference image. In this example, the reference image is shown from the front view perspective. Callout 1116 shows an enlarged portion of the callout 1112 showing a reference distinctive feature of the reference image.
The alignment module 240 compares the distinctive feature and the reference distinctive feature to determine an alignment including an orientation, scale, or position. For example, if the image that includes the custom graphic is shown from the front perspective, the distinctive feature of the custom graphic in the image should match the reference distinctive feature. The alignment module 240 can determine perspective changes based on a mismatch between the distinctive feature and the reference distinctive feature. The alignment module 240 uses the mismatch to infer or determine a perspective of the image or other spatial attributes of the image that can be utilized by the decoder module 250 to more accurately decode data from the image.
At operation 1310, the decoder module 250 determines a failure to decode the data encoded in the portion of the image using the transformed image. For instance, if the data decoded from the image is corrupted, incomplete, or garbled, the decoder module 250 determines the failure to decode the data. In another instance, a portion of the data encoded in the image can be for the purposes of data validation. That is to say, a known or determinable value can be encoded into the data such that the data is valid if the value is decoded from the image. The decoder module 250 can employ a variety of other schemes and techniques to determine the failure to decode the data encoded in the portion of the image.
At operation 1320, the alignment module 240 generates another transformed image by transforming the image according to a different orientation of the custom graphic. For example, the alignment module 240 generates a transformed image that is rotated 180 degrees, and the decoder module 250 attempts to decode the data a second time. The alignment module 240 can perform common transforms that may resolve the failure to decode such as 90 degree rotations or another transform that has frequently resolved the failure to decode in past scans. In some embodiments, the alignment module 240 performs another analysis of the image data of the image to determine another alignment to use to use when generating another transformed image. The alignment module 240 can perform other types of image transformations by applying different types of filters (e.g., orientation, color reduction, brightness manipulation, etc.) to the custom graphic.
At operation 1330, the decoder module 250 decodes the data encoded in the portion of the image using another transformed image. The alignment module 240 and the decoder module 250 can attempt any number (e.g., a set number of attempts or an unlimited number of attempts) of iterations of alignments that ends when the data is successfully decoded from the image. In this way, the custom pattern system 160 can use the markings for self-alignment.
To further explain the discussion in connection with
Turning now to
In further example embodiments, the action is exclusive to software that provides scanning functionality for the optical barcode that uses the custom functional pattern (e.g., a snapcode). In some embodiments, the software that scans the optical barcode can perform certain exclusive actions without communicating with a server. This is due to the exclusive, branded nature of the custom functional pattern that is not necessarily openly decodable by other third-party software applications. The snapcode can specify such actions since it is likely that the software (e.g., a mobile computing software such as an app) that scans the branded optical barcode is associated with the branded optical barcode.
At operation 1710, the communication module 210 receives a request to generate a machine-readable image such as an optical barcode that uses custom functional patterns. In some embodiments, the request includes user specified data to encode into the image.
At operation 1720, the encoder module 270 renders a machine-readable arrangement of marks that encodes the user-specified data. For instance, the marks can comprise dots, squares, or other markings that are arranged in a predetermined pattern. In an example embodiment, the presence of a mark at a particular location in the arrangement is indicative of data.
At operation 1730, the encoder module 270 generates the machine-readable image by positioning the machine-readable arrangement of marks in the machine-readable image with respect to a position of the custom graphic included in the machine-readable image. For example, the custom graphic can be centered in the optical barcode or positioned elsewhere (e.g., the example optical barcodes of
At operation 1740, the communication module 210 stores or transmits the machine-readable image. For instance, the communication module 210 can store the machine-readable image on the user device, a server, or another storage repository (either locally or remotely stored). In other instances, the communication module 210 transmits the machine-readable image to the user device, a server, or one or more other devices.
Many varieties of applications (also referred to as “apps”) can be executed on the mobile device 1900, such as native applications (e.g., applications programmed in Objective-C, Swift, or another suitable language running on IOS™, or applications programmed in Java running on ANDROID™), mobile web applications (e.g., applications written in Hypertext Markup Language-5 (HTML5)), or hybrid applications (e.g., a native shell application that launches an HTML5 session). For example, the mobile device 1900 includes a messaging app, an audio recording app, a camera app, a book reader app, a media app, a fitness app, a file management app, a location app, a browser app, a settings app, a contacts app, a telephone call app, or other apps (e.g., gaming apps, social networking apps, biometric monitoring apps). In another example, the mobile device 1900 includes a social messaging app 1908 such as SNAPCHAT® that, consistent with some embodiments, allows users to exchange ephemeral messages that include media content. In this example, the social messaging app 1908 can incorporate aspects of embodiments described herein.
Certain embodiments are described herein as including logic or a number of components, modules, or mechanisms. Modules can constitute either software modules (e.g., code embodied on a machine-readable medium) or hardware modules. A “hardware module” is a tangible unit capable of performing certain operations and can be configured or arranged in a certain physical manner. In various example embodiments, one or more computer systems (e.g., a stand-alone computer system, a client computer system, or a server computer system) or one or more hardware modules of a computer system (e.g., a processor or a group of processors) can be configured by software (e.g., an application or application portion) as a hardware module that operates to perform certain operations as described herein.
In some embodiments, a hardware module can be implemented mechanically, electronically, or any suitable combination thereof. For example, a hardware module can include dedicated circuitry or logic that is permanently configured to perform certain operations. For example, a hardware module can be a special-purpose processor, such as a Field-Programmable Gate Array (FPGA) or an Application Specific Integrated Circuit (ASIC). A hardware module may also include programmable logic or circuitry that is temporarily configured by software to perform certain operations. For example, a hardware module can include software executed by a general-purpose processor or other programmable processor. Once configured by such software, hardware modules become specific machines (or specific components of a machine) uniquely tailored to perform the configured functions and are no longer general-purpose processors. It will be appreciated that the decision to implement a hardware module mechanically, in dedicated and permanently configured circuitry, or in temporarily configured circuitry (e.g., configured by software) can be driven by cost and time considerations.
Accordingly, the phrase “hardware module” should be understood to encompass a tangible entity, be that an entity that is physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a certain manner or to perform certain operations described herein. As used herein, “hardware-implemented module” refers to a hardware module. Considering embodiments in which hardware modules are temporarily configured (e.g., programmed), each of the hardware modules need not be configured or instantiated at any one instance in time. For example, where a hardware module comprises a general-purpose processor configured by software to become a special-purpose processor, the general-purpose processor may be configured as respectively different special-purpose processors (e.g., comprising different hardware modules) at different times. Software accordingly configures a particular processor or processors, for example, to constitute a particular hardware module at one instance of time and to constitute a different hardware module at a different instance of time.
Hardware modules can provide information to, and receive information from, other hardware modules. Accordingly, the described hardware modules can be regarded as being communicatively coupled. Where multiple hardware modules exist contemporaneously, communications can be achieved through signal transmission (e.g., over appropriate circuits and buses) between or among two or more of the hardware modules. In embodiments in which multiple hardware modules are configured or instantiated at different times, communications between such hardware modules may be achieved, for example, through the storage and retrieval of information in memory structures to which the multiple hardware modules have access. For example, one hardware module can perform an operation and store the output of that operation in a memory device to which it is communicatively coupled. A further hardware module can then, at a later time, access the memory device to retrieve and process the stored output. Hardware modules can also initiate communications with input or output devices, and can operate on a resource (e.g., a collection of information).
The various operations of example methods described herein can be performed, at least partially, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors constitute processor-implemented modules that operate to perform one or more operations or functions described herein. As used herein, “processor-implemented module” refers to a hardware module implemented using one or more processors.
Similarly, the methods described herein can be at least partially processor-implemented, with a particular processor or processors being an example of hardware. For example, at least some of the operations of a method can be performed by one or more processors or processor-implemented modules. Moreover, the one or more processors may also operate to support performance of the relevant operations in a “cloud computing” environment or as a “software as a service” (SaaS). For example, at least some of the operations may be performed by a group of computers (as examples of machines including processors), with these operations being accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., an API).
The performance of certain of the operations may be distributed among the processors, not only residing within a single machine, but deployed across a number of machines. In some example embodiments, the processors or processor-implemented modules can be located in a single geographic location (e.g., within a home environment, an office environment, or a server farm). In other example embodiments, the processors or processor-implemented modules are distributed across a number of geographic locations.
The modules, methods, applications and so forth described in conjunction with the figures above are implemented in some embodiments in the context of a machine and an associated software architecture. The sections below describe representative software architecture(s) and machine (e.g., hardware) architecture that are suitable for use with the disclosed embodiments.
Software architectures are used in conjunction with hardware architectures to create devices and machines tailored to particular purposes. For example, a particular hardware architecture coupled with a particular software architecture will create a mobile device, such as a mobile phone, tablet device, or so forth. A slightly different hardware and software architecture may yield a smart device for use in the “internet of things,” while yet another combination produces a server computer for use within a cloud computing architecture. Not all combinations of such software and hardware architectures are presented here as those of skill in the art can readily understand how to implement the subject technology in different contexts from the disclosure contained herein.
In the example architecture of
The operating system 2014 may manage hardware resources and provide common services. The operating system 2014 may include, for example, a kernel 2028, services 2030, and drivers 2032. The kernel 2028 may act as an abstraction layer between the hardware and the other software layers. For example, the kernel 2028 may be responsible for memory management, processor management (e.g., scheduling), component management, networking, security settings, and so on. The services 2030 may provide other common services for the other software layers. The drivers 2032 may be responsible for controlling or interfacing with the underlying hardware. For instance, the drivers 2032 may include display drivers, camera drivers, BLUETOOTH® drivers, flash memory drivers, serial communication drivers (e.g., Universal Serial Bus (USB) drivers), WI-FI® drivers, audio drivers, power management drivers, and so forth depending on the hardware configuration. In an example embodiment, the operating system 2014 includes an imaging service 2033 that can provide image processing services, such as hardware-accelerated image processing, or image capture services, such as low-level access to optical sensors or optical sensor data.
The libraries 2016 may provide a common infrastructure that may be utilized by the applications 2020 or other components or layers. The libraries 2016 typically provide functionality that allows other software modules to perform tasks in an easier fashion than to interface directly with the underlying operating system 2014 functionality (e.g., kernel 2028, services 2030, or drivers 2032). The libraries 2016 may include system libraries 2034 (e.g., C standard library) that may provide functions such as memory allocation functions, string manipulation functions, mathematic functions, and the like. In addition, the libraries 2016 may include API libraries 2036 such as media libraries (e.g., libraries to support presentation and manipulation of various media format such as Motion Picture Experts Group Layer-4 (MPREG4), H.264, Motion Picture Layer-3 (MP3), Advanced Audio Coding (AAC), Adaptive Mesh Refinement (AMR), Joint Photographic Experts Group (JPG), or Portable Network Graphics (PNG)), graphics libraries (e.g., an OpenGL framework that may be used to render two-dimensional (2D) and three-dimensional (3D) in a graphic content on a display), database libraries (e.g., SQLite that may provide various relational database functions), web libraries (e.g., WebKit that may provide web browsing functionality), and the like. The libraries 2016 may also include a wide variety of other libraries 2038 to provide many other APIs to the applications 2020 and other software components/modules. In an example embodiment, the libraries 2016 include imaging libraries 2039 that provide image processing or image capture functionality that can be utilized by the custom pattern system 160.
The frameworks 2018 (also sometimes referred to as middleware) may provide a higher-level common infrastructure that may be utilized by the applications 2020 or other software components/modules. For example, the frameworks 2018 may provide various graphic user interface (GUI) functions, high-level resource management, high-level location services, and so forth. The frameworks 2018 may provide a broad spectrum of other APIs that may be utilized by the applications 2020 or other software components/modules, some of which may be specific to a particular operating system or platform. In an example embodiment, the frameworks 2018 include an image processing framework 2022 and an image capture framework 2023. The image processing framework 2022 can provide high-level support for image processing functions that can be used in aspects of the custom pattern system 160. Similarly, the image capture framework 2023 can provide high-level support for capturing images and interfacing with optical sensors.
The applications 2020 include built-in applications 2040 or third party applications 2042. Examples of representative built-in applications 2040 may include, but are not limited to, a contacts application, a browser application, a book reader application, a location application, a media application, a messaging application, or a game application. Third party applications 2042 may include any of the built-in applications 2040 as well as a broad assortment of other applications. In a specific example, the third party application 2042 (e.g., an application developed using the ANDROID™ or IOS™ software development kit (SDK) by an entity other than the vendor of the particular platform) may be mobile software running on a mobile operating system such as IOS™, ANDROID™, WINDOWS® Phone, or other mobile operating systems. In this example, the third party application 2042 may invoke the API calls 2024 provided by the mobile operating system such as operating system 2014 to facilitate functionality described herein. In an example embodiment, the applications 2020 include a messaging application 2043 that includes the custom pattern system 160 as part of the application. In another embodiment, the applications 2020 include a stand-alone application 2045 that includes the custom pattern system 160.
The applications 2020 may utilize built-in operating system functions (e.g., kernel 2028, services 2030, or drivers 2032), libraries (e.g., system libraries 2034, API libraries 2036, and other libraries 2038), frameworks/middleware 2018 to create user interfaces to interact with users of the system 2000. Alternatively, or additionally, in some systems, interactions with a user may occur through a presentation layer, such as presentation layer 2044. In these systems, the application/module “logic” can be separated from the aspects of the application/module that interact with a user.
Some software architectures utilize virtual machines. In the example of
The machine 2100 can include processors 2110, memory/storage 2130, and I/O components 2150, which can be configured to communicate with each other such as via a bus 2102. In an example embodiment, the processors 2110 (e.g., a Central Processing Unit (CPU), a Reduced Instruction Set Computing (RISC) processor, a Complex Instruction Set Computing (CISC) processor, a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), an ASIC, a Radio-Frequency Integrated Circuit (RFIC), another processor, or any suitable combination thereof) can include, for example, processor 2112 and processor 2114 that may execute instructions 2116. The term “processor” is intended to include multi-core processor that may comprise two or more independent processors (sometimes referred to as “cores”) that can execute instructions contemporaneously. Although
The memory/storage 2130 can include a memory 2132, such as a main memory, or other memory storage, and a storage unit 2136, both accessible to the processors 2110 such as via the bus 2102. The storage unit 2136 and memory 2132 store the instructions 2116 embodying any one or more of the methodologies or functions described herein. The instructions 2116 can also reside, completely or partially, within the memory 2132, within the storage unit 2136, within at least one of the processors 2110 (e.g., within the processor's cache memory), or any suitable combination thereof, during execution thereof by the machine 2100. Accordingly, the memory 2132, the storage unit 2136, and the memory of the processors 2110 are examples of machine-readable media.
As used herein, the term “machine-readable medium” means a device able to store instructions and data temporarily or permanently and may include, but is not limited to, random-access memory (RAM), read-only memory (ROM), buffer memory, flash memory, optical media, magnetic media, cache memory, other types of storage (e.g., Erasable Programmable Read-Only Memory (EEPROM)) or any suitable combination thereof. The term “machine-readable medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, or associated caches and servers) able to store instructions 2116. The term “machine-readable medium” shall also be taken to include any medium, or combination of multiple media, that is capable of storing instructions (e.g., instructions 2116) for execution by a machine (e.g., machine 2100), such that the instructions, when executed by one or more processors of the machine 2100 (e.g., processors 2110), cause the machine 2100 to perform any one or more of the methodologies described herein. Accordingly, a “machine-readable medium” refers to a single storage apparatus or device, as well as “cloud-based” storage systems or storage networks that include multiple storage apparatus or devices. The term “machine-readable medium” excludes signals per se.
The I/O components 2150 can include a wide variety of components to receive input, provide output, produce output, transmit information, exchange information, capture measurements, and so on. The specific I/O components 2150 that are included in a particular machine will depend on the type of machine. For example, portable machines such as mobile phones will likely include a touch input device or other such input mechanisms, while a headless server machine will likely not include such a touch input device. It will be appreciated that the I/O components 2150 can include many other components that are not shown in
In further example embodiments, the I/O components 2150 can include biometric components 2156, motion components 2158, environmental components 2160, or position components 2162 among a wide array of other components. For example, the biometric components 2156 can include components to detect expressions (e.g., hand expressions, facial expressions, vocal expressions, body gestures, or eye tracking), measure biosignals (e.g., blood pressure, heart rate, body temperature, perspiration, or brain waves), identify a person (e.g., voice identification, retinal identification, facial identification, fingerprint identification, or electroencephalogram based identification), and the like. The motion components 2158 can include acceleration sensor components (e.g., an accelerometer), gravitation sensor components, rotation sensor components (e.g., a gyroscope), and so forth. The environmental components 2160 can include, for example, illumination sensor components (e.g., a photometer), temperature sensor components (e.g., one or more thermometers that detect ambient temperature), humidity sensor components, pressure sensor components (e.g., a barometer), acoustic sensor components (e.g., one or more microphones that detect background noise), proximity sensor components (e.g., infrared sensors that detect nearby objects), gas sensor components (e.g., machine olfaction detection sensors, gas detection sensors to detect concentrations of hazardous gases for safety or to measure pollutants in the atmosphere), or other components that may provide indications, measurements, or signals corresponding to a surrounding physical environment. The position components 2162 can include location sensor components (e.g., a Global Positioning System (GPS) receiver component), altitude sensor components (e.g., altimeters or barometers that detect air pressure from which altitude may be derived), orientation sensor components (e.g., magnetometers), and the like.
Communication can be implemented using a wide variety of technologies. The I/O components 2150 may include communication components 2164 operable to couple the machine 2100 to a network 2180 or devices 2170 via a coupling 2182 and a coupling 2172, respectively. For example, the communication components 2164 include a network interface component or other suitable device to interface with the network 2180. In further examples, communication components 2164 include wired communication components, wireless communication components, cellular communication components, Near Field Communication (NFC) components, BLUETOOTH® components (e.g., BLUETOOTH® Low Energy), WI-FI® components, and other communication components to provide communication via other modalities. The devices 2170 may be another machine or any of a wide variety of peripheral devices (e.g., a peripheral device coupled via a USB).
Moreover, the communication components 2164 can detect identifiers or include components operable to detect identifiers. For example, the communication components 2164 can include Radio Frequency Identification (RFID) tag reader components, NFC smart tag detection components, optical reader components (e.g., an optical sensor to detect one-dimensional bar codes such as a UPC bar code, multi-dimensional bar codes such as a QR code, Aztec Code, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code, Uniform Commercial Code Reduced Space Symbology (UCC RSS)-2D bar codes, and other optical codes), acoustic detection components (e.g., microphones to identify tagged audio signals), or any suitable combination thereof. In addition, a variety of information can be derived via the communication components 2164, such as location via Internet Protocol (IP) geo-location, location via WI-FI® signal triangulation, location via detecting a BLUETOOTH® or NFC beacon signal that may indicate a particular location, and so forth.
In various example embodiments, one or more portions of the network 2180 can be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), the Internet, a portion of the Internet, a portion of the Public Switched Telephone Network (PSTN), a plain old telephone service (POTS) network, a cellular telephone network, a wireless network, a WI-FI® network, another type of network, or a combination of two or more such networks. For example, the network 2180 or a portion of the network 2180 may include a wireless or cellular network, and the coupling 2182 may be a Code Division Multiple Access (CDMA) connection, a Global System for Mobile communications (GSM) connection, or other type of cellular or wireless coupling. In this example, the coupling 2182 can implement any of a variety of types of data transfer technology, such as Single Carrier Radio Transmission Technology (1×RTT), Evolution-Data Optimized (EVDO) technology, General Packet Radio Service (GPRS) technology, Enhanced Data rates for GSM Evolution (EDGE) technology, third Generation Partnership Project (3GPP) including 3G, fourth generation wireless (4G) networks, Universal Mobile Telecommunications System (UMTS), High Speed Packet Access (HSPA), Worldwide Interoperability for Microwave Access (WiMAX), Long Term Evolution (LTE) standard, others defined by various standard setting organizations, other long range protocols, or other data transfer technology.
The instructions 2116 can be transmitted or received over the network 2180 using a transmission medium via a network interface device (e.g., a network interface component included in the communication components 2164) and utilizing any one of a number of well-known transfer protocols (e.g., HTTP). Similarly, the instructions 2116 can be transmitted or received using a transmission medium via the coupling 2172 (e.g., a peer-to-peer coupling) to devices 2170. The term “transmission medium” shall be taken to include any intangible medium that is capable of storing, encoding, or carrying the instructions 2116 for execution by the machine 2100, and includes digital or analog communications signals or other intangible medium to facilitate communication of such software.
The mobile device 2240 is, according to some examples, a mobile phone, a tablet computer, a PDA, a digital music player, or a laptop computer. As shown, the mobile device 2240 has, among other things, a camera 2242 and a display unit 2244. The camera 2242 is used to scan images, take photographs, record or stream videos, etc. The display unit 2244 includes, according to one or more examples, one or more screens (e.g., touchscreens), one or more monitors, or one or more projector displays. The display unit 2244 is used to display images or information. As shown, the display unit 2244 displays an identity verification image for scanning into the computing device 2230 to verify the user's identity. The camera 2242 and the display unit 2244 may be components of the mobile device 2240 or may be coupled with the mobile device 2240 via a wired or wireless connection.
The computing device 2230 is, according to some examples, a mobile phone, a tablet computer, a PDA, a digital music player, a laptop computer, or a desktop computer. As shown, the computing device 2230 has, among other things, a camera 2232 and a display unit 2234. The camera 2232 is used to scan images, take photographs, record or stream videos, and the like. The display unit 2234 includes, according to one or more examples, one or more screens (e.g., touchscreens), one or more monitors, or one or more projector displays. The display unit 2234 is used to display images or information. According to some examples, the display unit 2234 displays instructions for the user to scan the identity verification image from the display unit 2244 of the mobile device 2240 into the camera 2232 of the computing device 2230. The camera 2232 and the display unit 2234 may be components of the computing device 2230 or may be coupled with the computing device 2230 via a wired or wireless connection.
As shown in
The web server 2220 hosts one or more web-based secure resources (e.g., webpages or webpage content) that are accessible to users of computing devices (e.g., computing device 2230) after verifying the identity of the user. The web-based resources include, for example, stored account information such as, messages in a messaging service, resources in a social networking service, financial institution account information, or resources (e.g., stored credit card number, shipping address, etc.) for shopping in an online store.
The authentication server 2210 verifies the identity of the user of the computing device 2230 and instructs the web server 2220 to provide, to the computing device 2230, access to the secure resource after verifying the identity of the user of the computing device 2230. According to some implementations, the authentication server 2210 transmits, to the mobile device 2240, an identity verification image. The authentication server 2210 transmits, to the computing device 2230, an instruction to scan, via the camera 2232 of the computing device, an image for verifying a user's identity. The authentication server 2210 receives, from the computing device 2230, a scanned image. The authentication server 2210 determines whether the scanned image includes data from the identity verification image. The authentication server 2210 transmits, to the web server 2220, an indication that the user's identity has been verified upon determining that the scanned image includes the data from the identity verification image. Operations of the authentication server 2210 are described in greater detail in conjunction with
The identity verification image generator 2320 is configured to generate an identity verification image for a user. In some examples, the identity verification image includes a geometric shape with a custom pattern and an optical barcode within the geometric shape (for example, as illustrated in
Any image can be the identity verification image. In some examples, the identity verification image includes encoded information (e.g., encoded in binary with dots representing 1 and blank spaces representing 0) for identifying the identity of a user. For example, the identity verification image can include a code that the authentication server 2210 maps (e.g., within a stored hash table or data repository) to the associated user's identity.
The identity verification module 2325 is configure to receive, from the computing device 2230, a scanned image. In some cases, the scanned image is scanned, at the computing device 2230, using the camera 2232 of the computing device 2230. The identity verification module 2325 is configured to determine whether the scanned image includes data from the identity verification image. For example, the identity verification module 2325 reads the encoded information from the scanned image and determines whether this encoded information corresponds to the encoded information of the identity verification image that was transmitted to the mobile device 2240. Upon determining that the scanned image includes the data from the identity verification image, the identity verification server is configured to transmit, to the web server 2220, an indication that the user's identity has been verified. In response, the web server provides, to the computing device 2230, access to a secure resource.
The identity verification data 2330 stores information about identity verification image(s) that were provided to mobile device(s) (e.g., mobile device 2240) such that a scanned image, received by the authentication server 2210, can be determined to correspond to an identity verification image. As used herein, the phrase “corresponds to” encompasses its plain and ordinary meaning. In some cases, the scanned image corresponds to the identity verification image if the same information is encoded (e.g., using dots and blank spaces) in both the scanned image and the identity verification image. In some cases, when transmitting the scanned image to the authentication server 2210, the user of the computing device 2230 provides an indication of his/her identity (e.g., username or telephone number), and the authentication server 2210 verifies the user's identity by comparing the data from the scanned image with data corresponding to the identity verification image provided to the mobile device 2240 of the user. For example, the user enters his/her username, telephone number or other identity information (but, in some cases, not password) to identify him/herself. Based on the entered username or telephone number, the authentication server 2210 can identify a specific identity verification image that was sent to the mobile device 2240 of the user, and a scanned image corresponding to which should now be received from the computing device 2230.
Alternatively, the user may cause the computing device 2230 to transmit, to the authentication server 2210, the scanned image without indicating his/her identity. In accordance with this embodiment, the authentication server 2210 compares the data in the transmitted scanned image with data from all of the identity verification images that were generated by the authentication server 2210 during a predetermined time period (e.g., the last ten minutes, hour, two hours, etc.). Alternatively, in some implementations, the scanned image (created using the identity verification image) is decoded to yield both information setting forth the identity (e.g., username or telephone number) of the user and information for verifying the identity of the user.
For example, during a predetermined time period, the authentication server 2210 sends three identity verification images to mobile devices with telephone numbers 111-2222, 333-4444, and 555-6666. According to one implementation, when transmitting the scanned image, the user identifies him/herself by telephone number (e.g., 111-2222) such that the authentication server 2210 knows to which identity verification image the scanned image should correspond.
According to other implementations, the user does not specify his/her telephone number. In one case, upon receiving a scanned image from the computing device 2230, the authentication server compares the scanned image with all three identity verification images to determine to which identity verification image, if any, the scanned image corresponds. In another case, specified bits (e.g., the top-leftmost n bits, where n is a predetermined positive integer) of the identity verification image and, therefore, also the scanned image, specify the telephone number. Upon reading the telephone number from the scanned image, the authentication server knows with which identity verification image the scanned image should be compared. The bits of the identity verification image and the scanned image can correspond to dots and blank spaces, with dots representing 1 and blank spaces representing 0.
As used herein, the term “configured” encompasses its plain and ordinary meaning. A module (e.g., identity verification image generator 2320 or identity verification module 2325) may be configured to carry out operation(s) by storing code for the operation(s) in memory (e.g., memory 2315). Processing hardware (e.g., processor 2305) may carry out the operations by accessing the appropriate locations in the memory. Alternatively, the module may be configured to carry out the operation(s) by having the operation(s) hard-wired in the processing hardware.
The method 2400 begins at step 2410, where the authentication server 2210 transmits, to the mobile device 2240, an identity verification image. In some cases, the identity verification image is transmitted to the mobile device 2240 in response to a request for the authentication image. The request for the authentication image comes from the mobile device 2240. Alternatively, the request from the authentication image comes from the computing device 2230. If the request comes from the computing device 2230, the request includes an identification of the mobile device 2240, which is provided as input by the user of the computing device 2230. The identification of the mobile device may be a username in an application, such as an instant messaging application, of the mobile device, or a telephone number of the mobile device. In some examples, the identity verification image includes encoded data which is to be decoded to verify the identity of the user of the mobile device 2240. In some examples, the identity verification image includes a geometric shape (e.g., a square or rectangle with rounded corners) with a custom pattern and an optical barcode within the geometric shape. The optical barcode includes the encoded data.
The authentication server 2210 identifies the mobile device 2240 to which to transmit the identity verification image based on a request, for the identity verification image, transmitted either by the mobile device 2240 or by the computing device 2230. In some cases, the user of the mobile device 2240 has an account with the authentication server 2210 (which may also be used for other purposes, such as accessing an instant messaging application). If the computing device 2230 transmits the request for the identity verification image, the request, which is entered into the computing device 2230 by the user of the computing device 2230, includes an indication of the mobile device 2240 (e.g., telephone number or username in instant messaging service) to which to transmit the identity verification image.
In some cases, the identity verification image is transmitted, from the authentication server 2210 to the mobile device 2240, in conjunction with an indication of a validity period (e.g., 5 minutes, 10 minutes, 1 hour, 2 hours, or 24 hours) for the identity verification image. The validity period indicates for how much time the identity verification image can be used before it stops being valid for identifying the user's identity. A shorter validity period decreases the probability that the identity verification image will be compromised, for example, by a stranger who takes a photograph of the identity verification image, by the user accidentally sharing the identity verification image with a contact, or by the user losing or forgetting his/her mobile device. Upon expiration of the validity period, the authentication server 2210 removes, from the memory 2315 of the authentication server 2210 (e.g., from the identity verification data 2330), data associated with the identity verification image. After expiration of the validity period, the authentication server 2210 foregoes transmitting an indication that the user's identity has been verified using the data associated with the identity verification image.
After the identity verification image is transmitted to the mobile device 2240, the identity verification image is displayed on the display unit 2244 of the mobile device 2240. The user then scans the display unit 2244 displaying the identity verification image into the computing device 2230 using the camera of the computing device 2230, in order to authenticate the computing device 2230 to allow the computing device 2230 to access a secure web-based resource at the web server 2220. In some cases, the authentication server 2210 transmits, for display at the computing device 2230, an instruction for the user. The instruction instructs the user to place the display unit 2244 of the mobile device 2240, which displays the identity verification image, in front of the camera 2232 of the computing device 2230 for scanning the identity verification image into the computing device 2230. In some cases, the authentication server 2210 is able to identify the computing device 2230 based on the computing device 2230 requesting access to the secure web-based resource at the web server 2220. The web server 2220 either informs the authentication server 2210 of an address of the computing device 2230 or allows the authentication server 2210 to communicate with the computing device 2230 via the web server 2220.
The computing device 2230 generates a scanned image by scanning the display unit 2244 of the mobile device 2240 (which displays the identity verification image) into the camera 2232 of the computing device. The computing device 2230 transmits the scanned image to the authentication server 2210.
At step 2420, the authentication server 2210 receives, from the computing device 2230, the scanned image. In some cases, the authentication server 2210 decodes encoded information in the scanned image.
At step 2430, the authentication server 2210 determines whether the scanned image includes data from the identity verification image. If the identity verification image (and the scanned image generated from the identity verification image) includes the geometric shape with the custom pattern and the optical barcode, the authentication server 2210 decodes encoded information in the optical barcode within the geometric shape based on an orientation of the geometric shape identified by the custom pattern. The authentication server 2210 verifies whether the decoded information corresponds to the stored identity of the user. In some cases, determining whether the scanned image includes data from the identity verification image includes decoding data encoded in the scanned image and determining whether the decoded data corresponds to the data that was encoded into the identity verification image.
According to some aspects, the authentication server 2210 receives, from the web server 2220 or the computing device 2230, an indication of the user's identity (e.g., a username or telephone number). In some examples, the indication of identity is entered into the computing device 2230 by the user of the computing device 2230 and provided to the authentication server 2210 by the web server 2220. The authentication server 2210 then determines whether the scanned image includes data from the identity verification image that was transmitted to the mobile device 2240 associated with the user's identity (e.g., being logged into an application with the username of the user's identity or having the telephone number of the user's identity).
According to other aspects, prior to receiving the scanned image, the authentication server 2210 is not informed of the identity of the user sending the scanned image. However, the authentication server 2210 stores information, generated at the authentication server 2210, about users to whom identity verification images were transmitted, and can use this information to identify the user sending the scanned image, as discussed below. In some implementations, the authentication server 2210 identifies the user by determining whether the scanned image includes data from one of multiple identity verification images provides to multiple users. For example, if the scanned image decodes to 1-0-1-1 and the authentication server 2210 provides identity verification images with the codes 0-0-1-1, 1-1-0-0, and 1-0-1-1, the scanned image corresponds to the user associated with the last identity verification image.
In other implementations, the authentication server 2210 decodes data from a first portion of the scanned image (e.g., the first n bits) to determine the user's identity. The authentication server 2210 decodes a second portion (e.g., everything after the first n bits) of the scanned image (either simultaneously with decoding the first portion or after decoding the first portion). The authentication server 2210 compares the decoded data from the second portion with data that was encoded in the identity verification image for the user to determine whether the scanned image includes data from the identity verification image. The first portion is different from the second portion.
Upon determining, at the authentication server 2210, that the scanned image includes data from the identity verification image, the method 2400 continues to step 2440. Upon determining that the scanned image does not include the data from the identity verification image, the method 2400 continues to step 2450.
At step 2440, upon determining that the scanned image includes data from the identity verification image, the authentication server 2210 transmits, to the web server 2220, an indication that the user's identity has been verified together with an indication of the user's identity, which may correspond to a telephone number or a username (e.g., in an instant messaging service). In response, the web server 2220 provides access to the secure web-based resource to the computing device 2230. After step 2440, the method 2400 ends.
At step 2450, upon determining that the scanned image does not include data from the identity verification image, the authentication server 2210 transmits, to the web server 2220, an indication that the user's identity has not been verified. In response, the web server 2220, does not provide access to the secure web-based resource to the computing device 2230, and informs the computing device 2230 that access to that resource is denied. The user may then attempt to obtain access to the resource using another authentication method (e.g., entering a password or re-scanning the identity verification image). Alternatively, upon determining that the scanned image does not include data from the identity verification image, the authentication server 2210 may forego notifying the web server 2220 that the user's identity has not been verified. The web server 2220 may deny the computing device 2230 access to the secure web-based resource until the user's identity is verified using the scanned image or using another method. After step 2450, the method 2400 ends.
The method 2500 begins at step 2505, where the computing device 2230 accesses example.com (or any other webpage stored at the web server 2220). The computing device 2230 accesses example.com, for example, in response to the user entering the uniform resource locator (URL) example.com into a web browser.
At step 2510, the web server 2220 provides, to the computing device 2230, data from example.com, in response to the computing device 2230 accessing example.com. The data includes any information presented on the website example.com, such as text, images, videos, and the like.
At step 2515, the user at the computing device 2230 requests to login to his/her account in example.com. For example, the user may wish to place a comment on an item (e.g., video, article, etc.) on example.com, and may log in to do so.
At step 2520, in response to the user's request to log in to his/her account, the web server 2220 requests a scanned image from the user's computing device 2230. The web server 2220 may request the scanned image as an alternative to another method for logging in (e.g., entering a password).
At step 2525, upon noticing that the scanned image has been requested via the computing device 2230, the user requests an identity verification image for the mobile device 2240. As shown, the user enters the request into the mobile device 2240, which transmits the request to the authentication server 2210. In an alternative implementation, the user enters the request via the computing device 2230, and provides identifying information, such as a username or a telephone number (but, in some cases, not a password) of the mobile device 2240. The computing device then forwards the request and the identifying information of the mobile device 2240 to the web server 2220, which forwards the request and the identifying information to the authentication server 2210.
At step 2530, the authentication server 2210 transmits the identity verification image to the mobile device 2240. The identity verification image is transmitted over one or more networks. In some examples, the identity verification image is accessed, at the mobile device 2240, via an application, such as an instant messaging application or a special-purpose application for communicating with the authentication server.
At step 2535, after the mobile device receives the identity verification image, the user scans the identity verification image from the display unit 2244 of the mobile device 2240 to the camera 2232 of the computing device 2230.
At step 2540, the computing device 2230 provides the scanned image to the web server 2220 over one or more networks. The web server 2220 forwards the scanned image to the authentication server 2210 for identifying the user.
At step 2545, the authentication server 2210 identifies the user based on the scanned image. The authentication server 2210 identifies the user by comparing information encoded (e.g., with dots and blank spaces) the scanned image with information that was encoded into the identity verification images provided to various mobile devices. Upon determining that the scanned image includes the same information as one of the identity verification images, the authentication server 2210 identifies the user as the owner of the mobile device to which that identity verification image was provided. The authentication server 2210 notifies the web server 2220 of the user's identity and confirms to the web server 2220 that the user's identity was verified.
At step 2550 upon receiving, from the authentication server 2210, the user's identity information and a confirmation that the identity has been verified, the web server 2220 provides, to the computing device 22230, access to the account of the identified user. After step 2550, the method 2500 ends.
In some implementations, the web server 2220 provides a nonce to the computing device 2230 together with the request for the scanned image of step 2520 so that when the request for an identification verification image is sent to the authentication server 2210 at step 2525, the nonce is encoded into the image. Thus, when the image is provided back to the authentication server 2210 in step 2540, the authentication server 2210 can return the nonce back to the web server 2220 along with the user identity in step 2545. This transmission, in step 2545, verifies not only the user's identity but the fact that the request for verification came from the web server 2220. This may help eliminate certain types of spoofing.
In some cases, the request 2525 contains a nonce (which is in some cases encrypted with a private key of the mobile device 2240), and the nonce is encoded into the identity verification image provided in step 2530 in a way that cannot be decoded, for example, using the public key of the authentication server 2210. When the scanned image is sent to the authentication server in step 2540, the authentication server 2210 checks the nonce in the scanned image against the nonce encoded into the identity verification image. This reduces the likelihood of a replay attack or someone stealing the identity verification image and using it for authentication, since authentication using the identity verification image depends on the image (which has the encoded nonce) and the nonce itself.
In some implementations, there could be some check on the part of the authentication server 2220, so that the authentication server 220 is satisfied that the mobile device 2240 has the right to receive an identity verification image on behalf of a particular user. In some cases, this check comes from the authentication server knowing the telephone number of the mobile device 2240 beforehand and double-checking it with the source of the request 2525. If the source of the request 2525 can be spoofed, something else can be used to double-check the telephone number.
According to one implementation for creating the identity verification image, a user accesses a webpage, hosted at the web server 2220, on the computing device 2230. The user provides a telephone number or username for identifying his/her mobile device 2240 and selects a button for receiving the identity verification image at the mobile device 2240. The web server 2220 transmits the telephone number or username to the authentication server 2210, which generates the identity verification image and provides the identity verification image to the mobile device 2240. The mobile device 2240 provides a user output notifying the user that it has received the identity verification image. The user opens an application on the mobile device 2240 to access the identity verification image. The user scans the identity verification image from the mobile device 2240 into the computing device 2230 via the camera 2232 of the computing device 2230. The computing device 2230 provides the scanned image to the web server 2220, which provides the scanned image to the authentication server 2210. The authentication server 2210 confirms that the user's identity has been verified based on data encoded in the scanned image. After a threshold time period (e.g., one or two minutes) the identity verification image expires, and the authentication server 2210 will no longer confirm that the user's identity has been verified based on data from a scanned image corresponding to data from the generated identity verification image.
According to another implementation, to generate the identity verification image, the user accesses an application (e.g., an application for generating the identity verification image, which may also be used for other purposes, such as instant messaging) on the mobile device 2240. Within the application, the user navigates to a workflow for generating the identity verification image and selects a button for generating the identity verification image. The mobile device 2240 sends, to the authentication server 2210, a request for the identity verification image. The authentication server 2210 generates a new identity verification image and provides the generated identity verification image to the mobile device 2240. The user displays the identity verification image on a display unit 2244 of the mobile device 2240 and scans the displayed identity verification image into the computing device 2230 via the camera 2232 of the computing device 2230. The computing device 2230 provides the scanned image to the web server 2220, for example, via an interface at a webpage. The web server 2220 forwards the scanned image to the authentication server 2210, which verifies the identity of the user using the scanned image. After a threshold time period (e.g., one or two minutes) the identity verification image expires, and the authentication server 2210 will no longer confirm that the user's identity has been verified based on data from a scanned image corresponding to data from the generated identity verification image.
In some cases, a portion of the identity verification image encodes a user's telephone number or username (e.g., in an instant messaging application). The encoded telephone number or username is used to look up information, such as whether the user has permission to access the resource at the web server 2220 that the user is attempting to access.
Throughout this specification, plural instances may implement components, operations, or structures described as a single instance. Although individual operations of one or more methods are illustrated and described as separate operations, one or more of the individual operations may be performed concurrently, and nothing requires that the operations be performed in the order illustrated. Structures and functionality presented as separate components in example configurations may be implemented as a combined structure or component. Similarly, structures and functionality presented as a single component may be implemented as separate components. These and other variations, modifications, additions, and improvements fall within the scope of the subject matter herein.
Although an overview of the inventive subject matter has been described with reference to specific example embodiments, various modifications and changes may be made to these embodiments without departing from the broader scope of embodiments of the present disclosure. Such embodiments of the inventive subject matter may be referred to herein, individually or collectively, by the term “invention” merely for convenience and without intending to voluntarily limit the scope of this application to any single disclosure or inventive concept if more than one is, in fact, disclosed.
The embodiments illustrated herein are described in sufficient detail to enable those skilled in the art to practice the teachings disclosed. Other embodiments may be used and derived therefrom, such that structural and logical substitutions and changes may be made without departing from the scope of this disclosure. The Detailed Description, therefore, is not to be taken in a limiting sense, and the scope of various embodiments is defined only by the appended claims, along with the full range of equivalents to which such claims are entitled.
As used herein, the term “or” may be construed in either an inclusive or exclusive sense. Moreover, plural instances may be provided for resources, operations, or structures described herein as a single instance. Additionally, boundaries between various resources, operations, modules, engines, and data stores are somewhat arbitrary, and particular operations are illustrated in a context of specific illustrative configurations. Other allocations of functionality are envisioned and may fall within a scope of various embodiments of the present disclosure. In general, structures and functionality presented as separate resources in the example configurations may be implemented as a combined structure or resource. Similarly, structures and functionality presented as a single resource may be implemented as separate resources. These and other variations, modifications, additions, and improvements fall within a scope of embodiments of the present disclosure as represented by the appended claims. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.
This application is a continuation of and claims the benefit of priority to U.S. patent application Ser. No. 16/671,800, filed Nov. 1, 2019, which is a continuation of and claims the benefit of priority to U.S. patent application Ser. No. 16/186,236, filed Nov. 9, 2018, which is a continuation of and claims the benefit of priority to U.S. patent application Ser. No. 15/085,033, filed Mar. 30, 2016, each of which are hereby incorporated by reference herein in their entireties.
| Number | Name | Date | Kind |
|---|---|---|---|
| 6038295 | Mattes | Mar 2000 | A |
| 6980909 | Root et al. | Dec 2005 | B2 |
| 7173651 | Knowles | Feb 2007 | B1 |
| 7411493 | Smith | Aug 2008 | B2 |
| 7535890 | Rojas | May 2009 | B2 |
| 8131597 | Hudetz | Mar 2012 | B2 |
| 8199747 | Rojas et al. | Jun 2012 | B2 |
| 8332475 | Rosen et al. | Dec 2012 | B2 |
| 8392975 | Raghunath | Mar 2013 | B1 |
| 8718333 | Wolf et al. | May 2014 | B2 |
| 8724622 | Rojas | May 2014 | B2 |
| 8874677 | Rosen et al. | Oct 2014 | B2 |
| 8909679 | Root et al. | Dec 2014 | B2 |
| 8995433 | Rojas | Mar 2015 | B2 |
| 9040574 | Wang et al. | May 2015 | B2 |
| 9055416 | Rosen et al. | Jun 2015 | B2 |
| 9100806 | Rosen et al. | Aug 2015 | B2 |
| 9100807 | Rosen et al. | Aug 2015 | B2 |
| 9191776 | Root et al. | Nov 2015 | B2 |
| 9204252 | Root | Dec 2015 | B2 |
| 9411950 | Olsen-Kreusch | Aug 2016 | B1 |
| 9443227 | Evans et al. | Sep 2016 | B2 |
| 9489661 | Evans et al. | Nov 2016 | B2 |
| 9491134 | Rosen et al. | Nov 2016 | B2 |
| 9781132 | Ramakrishnan | Oct 2017 | B2 |
| 9805182 | Kayyidavazhiyil et al. | Oct 2017 | B1 |
| 9985786 | Bhabbur et al. | May 2018 | B1 |
| 10164975 | Son et al. | Dec 2018 | B1 |
| 10432618 | Poder | Oct 2019 | B1 |
| 10462118 | Ren | Oct 2019 | B2 |
| 10498730 | Son et al. | Dec 2019 | B1 |
| 10521623 | Rodriguez et al. | Dec 2019 | B2 |
| 11063937 | Son et al. | Jul 2021 | B1 |
| 11233799 | Whyte et al. | Jan 2022 | B1 |
| 20070112968 | Schwab | May 2007 | A1 |
| 20080127331 | Seidman et al. | May 2008 | A1 |
| 20100042669 | Cohen et al. | Feb 2010 | A1 |
| 20110202598 | Evans et al. | Aug 2011 | A1 |
| 20120209924 | Evans et al. | Aug 2012 | A1 |
| 20130031623 | Sanders | Jan 2013 | A1 |
| 20130124292 | Juthani | May 2013 | A1 |
| 20130124855 | Varadarajan et al. | May 2013 | A1 |
| 20140075551 | Heo | Mar 2014 | A1 |
| 20140282961 | Dorfman et al. | Sep 2014 | A1 |
| 20140372303 | Gafni | Dec 2014 | A1 |
| 20150089615 | Krawczyk | Mar 2015 | A1 |
| 20150156198 | Sabin | Jun 2015 | A1 |
| 20150310251 | Wyrwas et al. | Oct 2015 | A1 |
| 20160156635 | Liu et al. | Jun 2016 | A1 |
| 20160188859 | Lee et al. | Jun 2016 | A1 |
| 20160314462 | Hong | Oct 2016 | A1 |
| 20200253400 | Born et al. | Aug 2020 | A1 |
| Number | Date | Country |
|---|---|---|
| 2887596 | Jul 2015 | CA |
| Entry |
|---|
| “U.S. Appl. No. 16/798,053, Notice of Allowance dated Sep. 24, 2021”, 10 pgs. |
| “U.S. Appl. No. 16/798,053, Corrected Notice of Allowability dated Oct. 6, 2021”, 2 pgs. |
| U.S. Appl. No. 15/085,033 U.S. Pat. No. 10,164,975, filed Mar. 30, 2016, Authentication Via Camera. |
| U.S. Appl. No. 16/186,236 U.S. Pat. No. 10,498,730, filed Nov. 9, 2018, Authentication Via Camera. |
| U.S. Appl. No. 16/671,800 U.S. Pat. No. 11,063,937, filed Nov. 1, 2019, Authentication Via Camera. |
| U.S. Appl. No. 16/798,053, filed Feb. 21, 2020, Scan to Login. |
| “U.S. Appl. No. 15/085,033, Examiner Interview Summary dated Jan. 19, 2018”, 3 pgs. |
| “U.S. Appl. No. 15/085,033, Final Office Action dated May 23, 2018”, 16 pgs. |
| “U.S. Appl. No. 15/085,033, Non Final Office Action dated Nov. 16, 2017”, 12 pgs. |
| “U.S. Appl. No. 15/085,033, Notice of Allowance dated Aug. 2, 2018”, 5 pgs. |
| “U.S. Appl. No. 15/085,033, Response filed Feb. 13, 2018 to Non Final Office Action dated Nov. 16, 2017”, 12 pgs. |
| “U.S. Appl. No. 15/085,033, Response filed Jul. 23, 2018 to Final Office Action dated May 23, 2018”, 14 pgs. |
| “U.S. Appl. No. 16/186,236, Notice of Allowance dated Aug. 8, 2019”, 11 pgs. |
| “U.S. Appl. No. 16/671,800, Non Final Office Action dated Nov. 27, 2020”, 14 pgs. |
| “U.S. Appl. No. 16/671,800, Notice of Allowance dated Mar. 10, 2021”, 8 pgs. |
| “U.S. Appl. No. 16/671,800, Response filed Feb. 12, 2021 to Non Final Office Action dated Nov. 27, 2020”, 11 pgs. |
| Leyden, John, “This SMS will self-destruct in 40 seconds”, [Online] Retrieved from the Internet: <URL: http://www.theregister.co.uk/2005/12/12/stealthtext/>, (Dec. 12, 2005), 1 pg. |
| Number | Date | Country | |
|---|---|---|---|
| 20210328983 A1 | Oct 2021 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 16671800 | Nov 2019 | US |
| Child | 17360655 | US | |
| Parent | 16186236 | Nov 2018 | US |
| Child | 16671800 | US | |
| Parent | 15085033 | Mar 2016 | US |
| Child | 16186236 | US |
