Patent Grant
9516466
Systems to implement audio fingerprinting exist. Audio fingerprinting refers to capturing a brief sample of audio being played, creating an acoustic fingerprint based on the sample, and comparing the sample against a central database of known audio samples. Audio fingerprinting is presently used to identify songs.
According to an embodiment of the disclosed subject matter, a method may include capturing, in a first device, a sample of audio streaming from a server, transmitting the sample to the server, and receiving, in the first device, access to a secure service provided by the server when the server verifies that the first device is at a location of the audio based on the sample.
According to an embodiment of the disclosed subject matter, a system may include a server to stream data of a first audio sound and to receive a captured sample of the first audio sound. The server may be configured to compare the sample to one or more streaming data channels managed by the server and, based at least in part on the comparison, provide a first device which transmitted the sample with access to a secure service of the server.
According to an embodiment of the disclosed subject matter, means for capturing, in a first device, a sample of audio data streaming from a server, means for transmitting the sample to the server, and means for receiving, in the first device, access to a secure service provided by the server when the server verifies that the first device is at a location of the audio based on the sample are provided.
Additional features, advantages, and embodiments of the disclosed subject matter may be set forth or apparent from consideration of the following detailed description, drawings, and claims. Moreover, it is to be understood that both the foregoing summary and the following detailed description are illustrative and are intended to provide further explanation without limiting the scope of the claims.
The accompanying drawings, which are included to provide a further understanding of the disclosed subject matter, are incorporated in and constitute a part of this specification. The drawings also illustrate embodiments of the disclosed subject matter and together with the detailed description serve to explain the principles of embodiments of the disclosed subject matter. No attempt is made to show structural details in more detail than may be necessary for a fundamental understanding of the disclosed subject matter and various ways in which it may be practiced.
Various aspects or features of this disclosure are described with reference to the drawings, wherein like reference numerals are used to refer to like elements throughout. In this specification, numerous details are set forth in order to provide a thorough understanding of this disclosure. It should be understood, however, that certain aspects of disclosure may be practiced without these specific details, or with other methods, components, materials, etc. In other instances, well-known structures and devices are shown in block diagram form to facilitate describing the subject disclosure.
A media distribution system may store a plurality of media files which are available to users of the system for download, viewing, listening, etc. The system may include one or more servers to provide data services and one or more databases storing information including media files and user data. User data may include, for example, user account data and history data.
A user of the system may have an individual account with the system. The system may provide various services to a user in accordance with permissions and rights associated with the user's account. For example, a user may have purchased or licensed a number of copyrighted works, such as songs or movies. The system may thereby provide the user with a “streaming” service through which the user may remotely listen to the songs or watch the movies which have been purchased or licensed with the user's account by receiving a stream of data from the system at a device of the user's choosing.
When the system streams media files to a user device, the server may track the data being streamed and may determine that the data stream was initiated by a specific user account, i.e., a host user. The present disclosure provides embodiments of a system which uses the host user account and data stream correspondence to determine co-presence of two or more users at a given location. When co-presence is established between a host user and a guest user, the system may provide various services to the guest user at the location, as will be discussed in the embodiments below.
In situations in which the systems discussed herein collect personal information about users, or may make use of personal information, the users may be provided with an opportunity to control whether programs or features collect user information (e.g., information about a user's social network, social actions or activities, profession, a user's preferences, or a user's current location), or to control whether and/or how to receive content from the content server that may be more relevant to the user. In addition, certain data may be treated in one or more ways before it is stored or used, so that personally identifiable information is removed. For example, a user's identity may be treated so that no personally identifiable information can be determined for the user, or a user's geographic location may be generalized where location information is obtained (such as to a city, ZIP code, or state level), so that a particular location of a user cannot be determined. In this sense, a determination of co-presence of users does not strictly require a determination of physical location of the users. Thus, the user may have control over how information is collected about the user and used by a system as disclosed herein.
Embodiments of the presently disclosed subject matter may be implemented in and used with a variety of component and network architectures.
The bus 21 allows data communication between the central processor 24 and one or more memory components, which may include RAM, ROM, and other memory, as previously noted. Typically RAM is the main memory into which an operating system and application programs are loaded. A ROM or flash memory component can contain, among other code, the Basic Input-Output system (BIOS) which controls basic hardware operation such as the interaction with peripheral components. Applications resident with the device 20 are generally stored on and accessed via a computer readable medium, such as a hard disk drive (e.g., fixed storage 23), an optical drive, floppy disk, or other storage medium.
The fixed storage 23 may be integral with the device 20 or may be separate and accessed through other interfaces. The network interface 29 may provide a direct connection to a remote server via a wired or wireless connection. The network interface 29 may provide such connection using any suitable technique and protocol as will be readily understood by one of skill in the art, including digital cellular telephone, WiFi, Bluetooth®, near-field, and the like. For example, the network interface 29 may allow the device 20 to communicate with other computers via one or more local, wide-area, or other communication networks, as described in further detail below.
Many other devices or components (not shown) may be connected in a similar manner (e.g., document scanners, digital cameras and so on). Conversely, all of the components shown in
Referring now to
An example system 300 may include one or more servers 310 which store music media files, user account data, and use history data. In this example, host user 320, e.g., “Lauren” may have a user account with the system 300. With the user account, Lauren may have purchased or licensed a number of songs through the system and may maintain a library of songs associated with her account. The system 300 may provide Lauren with various services related to her library of songs. For example, the system 300 may provide an interface and functionality by which Lauren may create playlists and stream songs to a device of her choice in a designated order from the one or more servers 310.
In
The system 300 includes a processor and instructions configured to operate the system 300 to manage user accounts, user libraries and user playlist services in a secure manner. For example, the system 300 may require Lauren to enter a user name and password in order for Lauren to receive access to her account information, library, playlists, and the like. Lauren may enter security credentials through a device of her choosing, for example, her phone 365. Thus, neither Grayson nor Everly can independently view or adjust Lauren's playlist immediately upon their arrival at the location 380 unless they know Lauren's security information or use Lauren's phone 365. Depending upon the circumstances, Grayson and Everly may confront significant inconveniences if either of them wishes to modify the contents or order of music playing at the location 380, such as to share music different from that selected by Lauren.
For example, Lauren may have left her phone 365 elsewhere in the house, may have otherwise created the playlist 360 with a host user device which is presently not accessible, or Lauren herself may be inaccessible if the occasion is a party in which she is busy attending to hosting duties. Distributing Lauren's security information among multiple parties would introduce a severe security risk. However, conventional media distribution systems provide no means for other users to access content purchased by one another, thus resulting in such insecure sharing of credentials in such situations. Even in the event that both Lauren and her phone 365 are accessible, it may not be convenient to pass Lauren's phone 365 around to multiple parties for the purpose of viewing and/or editing the playlist 360.
The system 300 of the present disclosure includes features which address these and other problems. Upon arrival at the location 380 where Lauren is playing the songs from her playlist 360, Grayson may use his own guest user device 370, e.g., phone 370, to capture a sample of the audio playing, e.g., song #6, “Get Up and Dance.” Accordingly, Grayson may transmit the captured sample to the system 300. The transmission may include identification data which identifies Grayson's guest user device 370. Furthermore, the transmitted sample may be captured in any suitable form, for example, as a continuous transmission sample or a sample segment of audio data with a timestamp indicating a time when the sample was captured.
Referring to
Referring back to
Alternatively, the system may use audio fingerprinting to identify the sample with as being a portion of the song “Get Up and Dance.” Having identified the sample, the system 300 may execute a search for instances of “Get Up and Dance” streaming and find an instance associated with Lauren's user account and presently streaming in sync with the timing of the sample. Accordingly, the system 300 may determine that the Grayson is a guest user 330 co-present with host user 320 Lauren at the location 380.
Based on the determination of co-presence, the system 300 may execute any of several optional actions. Therefore, generally, server 310 of the system 300 may be configured to receive the sample and compare the sample to one or more streaming data channels managed by the server 310 and, based at least in part on the comparison, provide the first device with access to a secure service of the server 310. The guest user device 370 may, upon receiving a notification from the server 310, provide the guest user with access to the secure service of the server 310 based on the server 310 processing the transmitted sample of captured audio
For example, in an embodiment, the system 300 may automatically grant the ability to access various song queue or playlist management functions that otherwise may not be accessible to users other than Lauren, i.e., other than the host user. A playlist management function may be, without limitation, the system granting Grayson the ability to view Lauren's playlist 360. If Lauren is streaming songs based on a song queue instead of a playlist, a song queue management function may be, without limitation, the system granting Grayson the ability to add one or more songs or albums to the song queue.
In another embodiment, the system 300 may transmit an alert to Lauren or transmit an access approval request to her device 365. Lauren may accordingly deny or approve the request. Upon approval, the system 300 may allow Grayson further privileges, e.g., to add songs, delete songs or rearrange song order in Lauren's playlist 360. More generally, systems and techniques as disclosed herein may provide any suitable song queue or playlist management service or functionality to a guest user that is determined to be co-present with a host user with any additional restrictions or limitations imposed by the host user, as disclosed herein.
An embodiment of the system 300 may be configured to provide functions in which user account data may include personal designations of other users or classes of users. In this embodiment, Lauren may preapprove the system 300 providing certain services according to categorical designations. For example, using an account setting Lauren may inform the system 300 that any guest user whom she has assigned a certain designation (e.g., “friends” or “family”) and is co-present with Lauren may view or edit a playlist associated with Lauren's account which contains a song that is presently streaming and used by the system 300 to establish co-presence. Such user categories designations may be predefined and/or defined by the user, and assignments of particular contacts to categories may be selected by the user, or may be determined based on another data source. For example, user associations within a social network or similar service may be used to determine to which group or groups each of a user's known associates may belong. As a specific example, a host user may be provided an opportunity to import contacts, groups, and the like from an existing social network, and/or to assign various permissions as disclosed herein to those groups and/or contacts. Furthermore, default permissions and/or limitations may be assigned to previously unknown guest users.
The management function provided to the host user by the system may include other control functions. For example, the host user may adjust or revoke permissions from a guest user or category of guest users at any time. The host use may optionally set a pre-determined time period for which the permission is valid or a cut-off time at which permissions automatically terminate. In this manner the host user may, for example, allow all guest users arriving at a party to add songs to a song queue up until 11:00, when the party ends.
While attempting to determine co-presence based on a received sample, the system 300 may encounter a situation in which multiple channels are streaming identical media files which match the sample. For example, a plurality of user accounts of the system 300 may be playing the same, extremely popular song. In order to de-dupe multiple synchronized streams, the server 310 may send a message to the host user device 365 and/or guest user device 370 requesting information identifying a secondary source to validate which of the channels is the correct channel matching the received audio sample. The secondary source may be, for example, a satellite navigation system such as global positioning system (GPS), local area network information such as WiFi network information, a device configured to communicate using near-field communication or short-wavelength radio waves, such as Bluetooth®, or other secondary sources. Upon receiving data regarding the secondary source, the system 300 may determine which of the plurality of user accounts streaming the matching media file is associated with the stream that is the source of the received sample.
The system 300 may use the secondary data source to verify the correct host user in any number of ways depending upon the availability of the data. For example, the secondary source data may be used to verify that a host user device 365 and guest user device 370 are on the same WiFi network, within the same physical GPS location, or in communication with the same Bluetooth device.
In another embodiment, to de-dupe multiple synchronized streams the server 310 may introduce a momentary delay in one or more candidate streams to identify which stream corresponds to the received sample. Such a delay may be relatively short, for example, such that it does not substantially interrupt user enjoyment of the audio stream, or such that it is too short to be generally audible. Similarly, a non-audible modification may be made to the audio stream, such as where a sound above or below the normal range of human hearing is added to the audio stream at a known point.
Further to the scenario of
The services are not limited to those discussed above. The system 300 may execute any service based on a determination of co-presence. For example, a service provided to the host user and guest users may include a receipt and recordation of data. In this example, one or more guest users arriving at an event may establish co-presence with the host user. An application running on the server may automatically track the guest users which establish co-presence as a check-in service that indicates a guest user is present at the location. The server may maintain a list of checked-in guest users for the host user to review, or maintain a running total count of guest users who have established co-presence. In this example, the system 300 may automatically check-in guest users and keep a headcount without the host user needing to intervene or directly attend to the task.
One or more functions of the system 300 may be implemented in either the user devices (e.g., 365, 370 in
At operation 530, the server compares the sample against active streams to search for a match. The server may identify the media file using audio fingerprinting and search for presently streaming instances of the media file. The server may execute one or more measures to de-dupe the results if the search produces multiple matching instances. Once a match is found, the server determines that the guest user is co-present with the host user who initiated the stream. At operation 540, the server provides the guest user with access to a service based on the determination of co-presence. The service may be, for example, a check-in service, viewing a playlist, viewing a queue, editing a playlist, adding media files to a queue, or other services. The service may be based on a status of the guest user, for example, a designation or title assigned to the guest user or to categories of guest users by the host user.
Although illustrative examples provided herein are described in terms of audio playback, it will be understood that such audio may be part of an audiovisual or other combined work, such as a short film, movie, presentation, or the like. Such a media stream may be used as described herein using appropriate fingerprinting or other identification techniques without departing from the scope of the present disclosure.
The user interface 53, database 52, and/or processing units 54 may be part of an integral system, or may include multiple computer systems communicating via a private network, the Internet, or any other suitable network. One or more processing units 54 may be, for example, part of a distributed system such as a cloud-based computing system, search engine, content delivery system, or the like, which may also include or communicate with a database 52 and/or user interface 53. In some arrangements, an analysis system 55 may provide back-end processing, such as where stored or acquired data is pre-processed by the analysis system 55 before delivery to the processing unit 54, database 52, and/or user interface 53. For example, a machine learning system 55 may provide various prediction models, data analysis, or the like to one or more other systems 52, 53, 54.
More generally, various embodiments of the presently disclosed subject matter may include or be embodied in the form of computer-implemented processes and apparatuses for practicing those processes. Embodiments also may be embodied in the form of a computer program product having computer program code containing instructions embodied in non-transitory and/or tangible media, such as floppy diskettes, CD-ROMs, hard drives, USB (universal serial bus) drives, or any other machine readable storage medium, such that when the computer program code is loaded into and executed by a computer, the computer becomes an apparatus for practicing embodiments of the disclosed subject matter. Embodiments also may be embodied in the form of computer program code, for example, whether stored in a storage medium, loaded into and/or executed by a computer, or transmitted over some transmission medium, such as over electrical wiring or cabling, through fiber optics, or via electromagnetic radiation, such that when the computer program code is loaded into and executed by a computer, the computer becomes an apparatus for practicing embodiments of the disclosed subject matter. When implemented on a general-purpose microprocessor, the computer program code segments configure the microprocessor to create specific logic circuits.
In some configurations, a set of computer-readable instructions stored on a computer-readable storage medium may be implemented by a general-purpose processor, which may transform the general-purpose processor or a device containing the general-purpose processor into a special-purpose device configured to implement or carry out the instructions. Embodiments may be implemented using hardware that may include a processor, such as a general purpose microprocessor and/or an Application Specific Integrated Circuit (ASIC) that embodies all or part of the techniques according to embodiments of the disclosed subject matter in hardware and/or firmware. The processor may be coupled to memory, such as RAM, ROM, flash memory, a hard disk or any other device capable of storing electronic information. The memory may store instructions adapted to be executed by the processor to perform the techniques according to embodiments of the disclosed subject matter.
The foregoing description, for purpose of explanation, has been described with reference to specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit embodiments of the disclosed subject matter to the precise forms disclosed. Many modifications and variations are possible in view of the above teachings. The embodiments were chosen and described in order to explain the principles of embodiments of the disclosed subject matter and their practical applications, to thereby enable others skilled in the art to utilize those embodiments as well as various embodiments with various modifications as may be suited to the particular use contemplated.
| Number | Name | Date | Kind |
|---|---|---|---|
| 7013301 | Holm et al. | Mar 2006 | B2 |
| 7706838 | Atsmon et al. | Apr 2010 | B2 |
| 8078136 | Atsmon et al. | Dec 2011 | B2 |
| 20070169142 | Claassen et al. | Jul 2007 | A1 |
| 20100281108 | Cohen | Nov 2010 | A1 |
| 20120047156 | Jarvinen et al. | Feb 2012 | A1 |
| 20130191454 | Oliver et al. | Jul 2013 | A1 |
| 20140075308 | Sanders | Mar 2014 | A1 |
| 20140108441 | Samari et al. | Apr 2014 | A1 |
| 20140115463 | Reznor et al. | Apr 2014 | A1 |
| 20140168071 | Ahmed et al. | Jun 2014 | A1 |
| Entry |
|---|
| PCT/US2015/060142, International Search Report and Written Opinion issued in PCT/US2015/060142 on Jan. 21, 2016., 12. |
| Number | Date | Country | |
|---|---|---|---|
| 20160174036 A1 | Jun 2016 | US |
