Patent Grant
11800190
The present disclosure generally relates to a method and system for content switching and content control based on facial identification and/or facial recognition.
Cable service providers, which are also referred to as Multiple System Operators (“MSO”), or any communication or content distribution business that operates through a cable network, renders its services to its subscribers. The services can include, but are not limited to, different subscription plans for broadband Internet access and telephony. In order to consume these services, subscribers connect to a private network owned (or co-owned or rented) by the broadband cable operator which is implemented according to the Data Over Cable Service Interface Specification (DOCSIS) standard.
To provide television and data services, a MSO typically uses a Cable Modem Termination System (“CMTS”) for the data services and a quadrature amplitude modulation (“QAM”) multiplexer for downstream broadcast television, narrow cast and video-on-demand (VoD) traffic signals. These devices may be located at one or more hubs, which are typically connected to a head end via a network according to a networking protocol, such as Ethernet or SONET. A residential cable customer is typically connected to the CMTS and hub via a cable modem and a wireless router that may be combined in a single box which is called a gateway. In order to view video content that is transmitted through the cable modem or gateway, residential customers connect their televisions to a set-top box (STB). Since set-top boxes have recently gained additional features and functionality, they are often referred to as a smart media device (“SMD”) and can be considered as the network interface controller for their peripheral devices.
The set-top box can be used by the subscriber to access a variety of multimedia services, including but not limited to live or linear television, digital video recorder (DVR) content, video-on-demand (VoD) content, over-the-top (OTT) content, and others. For example, set-top boxes usually play user selected content, for example, either live or with a digital video recorder (DVR).
It would be desirable to be able to provide content based on facial identification or facial recognition of two or more users watching the media play device, and wherein the content can include both a variety of multimedia services and advertising, which can be, for example, age appropriate content.
In accordance with exemplary embodiments, a method and system are disclosed that creates multi-user profiles based on facial information captured by customer-premise equipment or customer-provided equipment (CPE) broadband devices, for example, a set-top box, that can recommend content for the two or more users based on machine learning.
In accordance with an aspect, a method is disclosed for content switching and content control based on multi-user profiles, the method comprising: capturing, by a set-top box, facial information of a plurality of users; forwarding, by the set-top box, the captured facial information of the plurality of users to a server, the server configured to receive the captured facial information of the plurality of users and identify each of the plurality of users based on the facial information captured by the set-top box and creating a plurality of multi-user profiles for each combination of the plurality of users; capturing, by the set-top box, the facial information of two or more users of the plurality of users viewing content on a media play device, the media play device configured to display content from the set-top box; sending, by the set-top box, the captured facial information of the two or more users of the plurality of user viewing content on the media play device to the server; and receiving, on the set-top box, recommended content for the two or more users viewing the media play device based on a multi-user profile for the two or more users from the plurality of user profiles.
In accordance with another aspect, a customer-premises equipment device is disclosed comprising: a camera configured to capture facial information of a plurality of users viewing content on a media play device, the media play device configured to display content from the customer-premises equipment device; and a processor configured to: forward the captured facial information of the plurality of users to a server, the server configured to receive the captured facial information of the plurality of users and identify each of the plurality of users based on the facial information captured by the set-top box and creating a plurality of multi-user profiles for each combination of the plurality of users; send the captured facial information of the two or more users of the plurality of user viewing content on the media play device to the server; and receive recommended content for the two or more users viewing the media play device based on a multi-user profile for the two or more users from the plurality of user profiles.
In accordance with an aspect, a non-transitory computer readable medium having instructions operable to cause one or more processors to perform operations comprising: capturing, by a set-top box, facial information of a plurality of users; forwarding, by the set-top box, the captured facial information of the plurality of users to a server, the server configured to receive the captured facial information of the plurality of users and identify each of the plurality of users based on the facial information captured by the set-top box and creating a plurality of multi-user profiles for each combination of the plurality of users; capturing, by the set-top box, the facial information of two or more users of the plurality of users viewing content on a media play device, the media play device configured to display content from the set-top box; sending, by the set-top box, the captured facial information of the two or more users of the plurality of user viewing content on the media play device to the server; and receiving, on the set-top box, recommended content for the two or more users viewing the media play device based on a multi-user profile for the two or more users from the plurality of user profiles.
Further areas of applicability of the present disclosure will become apparent from the detailed description provided hereinafter. It should be understood that the detailed description of exemplary embodiments are intended for illustration purposes only and are, therefore, not intended to necessarily limit the scope of the disclosure.
For simplicity and illustrative purposes, the principles of the embodiments are described by referring mainly to examples thereof. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the embodiments. It will be apparent however, to one of ordinary skill in the art, that the embodiments may be practiced without limitation to these specific details. In some instances, well known methods and structures have not been described in detail so as not to unnecessarily obscure the embodiments.
In accordance with an exemplary embodiment, the media device 130 may be any type of computing device configured to connect via a wireless network, for example, wireless network utilizing an IEEE 802.11 specification, including a smart phone, a smart TV, a computer, a mobile device, a tablet, or any other device operable to communicate wirelessly with the set-top box 120. In accordance with an alternative embodiment, the media device 130 can be a television, for example, that is directly connected to the set-top box 120.
In accordance with an exemplary embodiment, the set-top box 120 may communicate with the cable provider server 110 over a wired or a wireless connection. A wireless connection between the cable provider server 110 and the set-top box 120 may be established through a protected setup sequence (for example, Wi-Fi protected setup (WPS)). The protected setup sequence may include the steps of scanning multiple wireless channels for an available access point, exchanging one or more messages between a station and access point, exchanging key messages (for example, pre-shared key (PSK)) between the station and access point, and installing a key (for example, PSK) at the station.
As set forth, the set-top box 120, for example, customer premise equipment (CPE) device, typically provides access to a variety of multimedia services, including but not limited to live or linear television, digital video recorder (DVR) content, video-on-demand (VoD) content, over-the-top (OTT) content, and others. Alternatively, the set-top box 120 (e.g., customer premise equipment) can receive Internet Protocol-based (IP-based) streaming content from through the upstream wide area network (WAN) 112 and cable provider server 110. In embodiments, various data, multimedia, and/or voice services may be delivered to the set-top box 120 including but not limited to, live or broadcast television, video-on-demand (VoD) content, pay-per view content, recorded content (e.g., DVR content), audio-only content, streaming content, and others. The set-top box 120 may process and output content to one or more media play device 130 such as a television, mobile device, tablet, computer, and any other device operable to receive video, voice, and/or data services.
In accordance with an embodiment, the set-top box 120 can include a camera or sensor 240 (
In accordance with an embodiment, the system 100 can also include a server 150, for example, a cloud server that can be configured to receive a list of each of the users 140 and corresponding content that is being watched or viewed by the users 140 to providing the training data to the machine learning application or program. In accordance with an embodiment, if the users 140 are part of an existing group 142 of users 140 having a multi-user profile that has been created in the server 150, the server 150 can recommend content for the group 142 of users 140. Alternatively, if the group 142 of users 140 do not have a multi-user profile that has been created within the server 150, a new multi-user profile can be created based on the facial recognition and/or facial identification of the users 140. In addition, for both existing multi-user profiles and new multi-user profiles, the corresponding multi-user profile can be updated within the server 150 as disclosed herein.
If programmable logic is used, such logic may execute on a commercially available processing platform configured by executable software code to become a specific purpose computer or a special purpose device (for example, programmable logic array, application-specific integrated circuit, etc.). A person having ordinary skill in the art may appreciate that embodiments of the disclosed subject matter can be practiced with various computer system configurations, including multi-core multiprocessor systems, minicomputers, mainframe computers, computers linked or clustered with distributed functions, as well as pervasive or miniature computers that may be embedded into virtually any device. For instance, at least one processor device and a memory may be used to implement the above described embodiments.
A processor unit or device as discussed herein may be a single processor, a plurality of processors, or combinations thereof. Processor devices may have one or more processor “cores.” The terms “computer program medium,” “non-transitory computer readable medium,” and “computer usable medium” as discussed herein are used to generally refer to tangible media such as a removable storage unit 218, a removable storage unit 222, and a hard disk installed in hard disk drive 212.
Various embodiments of the present disclosure are described in terms of this representative computer system 200. After reading this description, it will become apparent to a person skilled in the relevant art how to implement the present disclosure using other computer systems and/or computer architectures. Although operations may be described as a sequential process, some of the operations may in fact be performed in parallel, concurrently, and/or in a distributed environment, and with program code stored locally or remotely for access by single or multi-processor machines. In addition, in some embodiments the order of operations may be rearranged without departing from the spirit of the disclosed subject matter.
A processor device 204 may be processor device specifically configured to perform the functions discussed herein. The processor device 204 may be connected to a communications infrastructure 206, such as a bus, message queue, network, multi-core message-passing scheme, etc. The network may be any network suitable for performing the functions as disclosed herein and may include a local area network (“LAN”), a wide area network (“WAN”), a wireless network (e.g., “Wi-Fi”), a mobile communication network, a satellite network, the Internet, fiber optic, coaxial cable, infrared, radio frequency (“RF”), or any combination thereof. Other suitable network types and configurations will be apparent to persons having skill in the relevant art. The computer system 200 may also include a main memory 208 (e.g., random access memory, read-only memory, etc.), and may also include a secondary memory 210. The secondary memory 210 may include the hard disk drive 212 and a removable storage drive 214, such as a floppy disk drive, a magnetic tape drive, an optical disk drive, a flash memory, etc.
The removable storage drive 214 may read from and/or write to the removable storage unit 218 in a well-known manner. The removable storage unit 218 may include a removable storage media that may be read by and written to by the removable storage drive 214. For example, if the removable storage drive 214 is a floppy disk drive or universal serial bus port, the removable storage unit 218 may be a floppy disk or portable flash drive, respectively. In one embodiment, the removable storage unit 218 may be non-transitory computer readable recording media.
In some embodiments, the secondary memory 210 may include alternative means for allowing computer programs or other instructions to be loaded into the computer system 200, for example, the removable storage unit 222 and an interface 220. Examples of such means may include a program cartridge and cartridge interface (e.g., as found in video game systems), a removable memory chip (e.g., EEPROM, PROM, etc.) and associated socket, and other removable storage units 222 and interfaces 220 as will be apparent to persons having skill in the relevant art.
Data stored in the computer system 200 (e.g., in the main memory 208 and/or the secondary memory 210) may be stored on any type of suitable computer readable media, such as optical storage (e.g., a compact disc, digital versatile disc, Blu-ray disc, etc.) or magnetic storage (e.g., a hard disk drive). The data may be configured in any type of suitable database configuration, such as a relational database, a structured query language (SQL) database, a distributed database, an object database, etc. Suitable configurations and storage types will be apparent to persons having skill in the relevant art.
The computer system 200 may also include a communications interface 224. The communications interface 224 may be configured to allow software and data to be transferred between the computer system 200 and external devices. Exemplary communications interfaces 224 may include a modem, a network interface (e.g., an Ethernet card), a communications port, a PCMCIA slot and card, etc. Software and data transferred via the communications interface 224 may be in the form of signals, which may be electronic, electromagnetic, optical, or other signals as will be apparent to persons having skill in the relevant art. The signals may travel via a communications path 226, which may be configured to carry the signals and may be implemented using wire, cable, fiber optics, a phone line, a cellular phone link, a radio frequency link, etc.
The computer system 200 may further include a display interface 202. The display interface 202 may be configured to allow data to be transferred between the computer system 200 and external display 230. Exemplary display interfaces 202 may include high-definition multimedia interface (HDMI), digital visual interface (DVI), video graphics array (VGA), etc. The display 230 may be any suitable type of display for displaying data transmitted via the display interface 202 of the computer system 200, including a cathode ray tube (CRT) display, liquid crystal display (LCD), light-emitting diode (LED) display, capacitive touch display, thin-film transistor (TFT) display, etc.
Computer program medium and computer usable medium may refer to memories, such as the main memory 208 and secondary memory 210, which may be memory semiconductors (e.g., DRAMs, etc.). These computer program products may be means for providing software to the computer system 200. Computer programs (e.g., computer control logic) may be stored in the main memory 308 and/or the secondary memory 210. Computer programs may also be received via the communications interface 224. Such computer programs, when executed, may enable computer system 200 to implement the present methods as discussed herein. In particular, the computer programs, when executed, may enable processor device 204 to implement the methods illustrated by
The processor device 204 may comprise one or more modules or engines configured to perform the functions of the computer system 200. Each of the modules or engines may be implemented using hardware and, in some instances, may also utilize software executed on hardware, such as corresponding to program code and/or programs stored in the main memory 208 or secondary memory 210. In such instances, program code may be compiled by the processor device 204 (e.g., by a compiling module or engine) prior to execution by the hardware of the computer system 200. For example, the program code may be source code written in a programming language that is translated into a lower level language, such as assembly language or machine code, for execution by the processor device 204 and/or any additional hardware components of the computer system 200. The process of compiling may include the use of lexical analysis, preprocessing, parsing, semantic analysis, syntax-directed translation, code generation, code optimization, and any other techniques that may be suitable for translation of program code into a lower level language suitable for controlling the computer system 200 to perform the functions disclosed herein. It will be apparent to persons having skill in the relevant art that such processes result in the computer system 200 being a specially configured computer system 200 uniquely programmed to perform the functions discussed above.
In accordance with an exemplary embodiment, for example, when the computer system 200 is a set-top box 120, the set-top box 120 can include a camera 240, which can be contained within the set-top box 120, or an external camera 240 in communication with the set-top box 120. In accordance with an embodiment, the camera 240 can be a sensor, for example, a RGB/infrared sensor or other facial recognition system that is capable of obtaining an image, for example, a digital image or video frame from a video source that can be used to identify a user 140 by comparing, for example, selected facial features from given images with faces of users 140 within a database. Alternatively, the facial recognition system can be a biometric artificial intelligence based application that can uniquely identify a user by analyzing patterns based on a user's facial textures and shape, for instance.
In accordance with an embodiment, content recommendations can further include age appropriate recommended content for the multi-user profile for the two or more users, which is received on the set-top box. In addition, the recommended content may include advertisement content. The method may also generate the recommended content for the two or more users viewing the media play device with a machine learning algorithm. In accordance with an embodiment, a machine learning algorithm, for example, can update the recommended content while the two or more users are viewing content on the media play device, or alternatively, the machine learning algorithm can be trained to provide recommended content, at preset intervals, for example, daily, weekly, etc.
In accordance with an embodiment, a new multi-user profile can be received on the set-top box for the two or more users when a combination of the captured information on the two or more users has not previously been captured by the set-top box with recommended content for the new multi-user profile. In addition, content currently being watched by the two or more users can be sent by the set-top box to the server, and the set-top box can receive further content recommendations based on the content being currently watched.
In accordance with an aspect, advertisements based on the multi-user profile for the two or more users viewing the media play device can be received on the set-top box, and the advertisements can be sent by the set-top box to the media display device for display on a display.
In accordance with another aspect, access to certain content can be restricted on the set-top box for the two or more users of the plurality of users viewing the content based on the multi-user profile of the two or more users. For example, the restricting of the access to certain content may be based on age of one of the two or more users.
In accordance with an aspect, the plurality users may be three or more, and when the facial information of one of the three or more users has not previously been captured by the set-top box, the method can include receiving, on the set-top box, the recommended content for two or more users having the multi-user profile, and receiving, on the set-top box, a new multi-user profile for the three or more users with recommended content for the new multi-user profile.
In accordance with another embodiment, facial information of a new user different from the two or more users viewing content on the media play device may be captured by the set-top box, for example, a child enters the room while two or more adults are watching content on the media play device. In accordance with an embodiment, the facial information of the new user is forwarded by the set-top box to the server, and the set-top box may receive recommended content based on a new multi-user profile from the new user and the two or more users viewing content on the media play device. In addition, the set-top box may restricted he two or more users from continuing viewing content on the media play device when the new multi-user profile has content restrictions, for example, an age restriction on the new user different from the two or more users.
Techniques consistent with the present disclosure provide, among other features, methods and systems for content switching and content control based on facial identification and facial identification profiling. While various exemplary embodiments of the disclosed system and method have been described above it should be understood that they have been presented for purposes of example only, not limitations. It is not exhaustive and does not limit the disclosure to the precise form disclosed. Modifications and variations are possible in light of the above teachings or may be acquired from practicing of the disclosure, without departing from the breadth or scope.
| Number | Name | Date | Kind |
|---|---|---|---|
| 20090138805 | Hildreth | May 2009 | A1 |
| 20140215504 | Hsiao et al. | Jul 2014 | A1 |
| 20150082330 | Yun | Mar 2015 | A1 |
| 20150271571 | Laksono | Sep 2015 | A1 |
| 20160337696 | Lee et al. | Nov 2016 | A1 |
| 20180262582 | Benedetto | Sep 2018 | A1 |
| 20200178027 | Keller | Jun 2020 | A1 |
| Number | Date | Country |
|---|---|---|
| 10-1963200 | Mar 2019 | KR |
| 10-2019-0066402 | Jun 2019 | KR |
| Entry |
|---|
| Notification of Transmittal of the International Search Report (Forms PCT/ISA/220 and PCT/ISA/210) and the Written Opinion of the International Searching Authority (Form PCT/ISA/237) dated Dec. 3, 2021, by the International Application Division Korean Intellectual Property Office in corresponding International Application No. PCT/US2021/047264. (9 pages). |
| Number | Date | Country | |
|---|---|---|---|
| 20220070536 A1 | Mar 2022 | US |
| Number | Date | Country | |
|---|---|---|---|
| 63074152 | Sep 2020 | US |
