Patent Application
20140078039
Typically, while a user consumes media with a traditional television device, the user is actively engaged in another activity (e.g., surfing the web or operating another user equipment device). During these instances, the attention of the user is mainly directed at the other activity, and not the television device. This makes the other activity become the primary focus for the user rather than the consumption of media with the television device. As a result, oftentimes information provided by the television device that may be of greater interest to the user is missed because these systems lack a suitable mechanism to inform the user about such information.
In view of the foregoing, systems and methods for recapturing attention of the user when content meeting a criterion is being presented are provided. In particular, when content being presented on a display of a user equipment device meets a criterion and the user is determined to not be gazing towards the display, a setting of the user equipment device is adjusted to alert the user about the content.
In some embodiments, content is caused to be presented on a display of a user equipment device. In particular, a given program received from a content source (e.g., the Internet or television channel) may be displayed. A first state of a setting of the user equipment device is stored. In particular, a current level of the volume may be determined and stored.
A determination is made as to whether the content being presented meets a criterion. In particular, a data structure or closed caption information associated with the content being presented may be processed to identify one or more attributes or characteristics of the content being presented. A profile may be retrieved to identify an alert condition to retrieve a criterion (e.g., a commercial segment ending or a comedy). The retrieved criterion or criteria are compared with the attributes or characteristics of the content to determine whether the content meets the criterion or criteria.
In some embodiments, a facial feature of the user is processed to determine whether the user is gazing towards the display when the content being presented meets the criterion. In particular, an image of the user may be captured using a camera or other biometric response device. The image of the user may be processed to identify and extract a facial feature of the user (e.g., a nose, eyes, ears, mouth, etc.). The facial feature may be processed by determining the position of the eyes and whether they are pointed towards the lens of the camera or the display on which the content is being presented. When the position of the eyes is substantially pointed towards the camera or the display, the user may be determined to be gazing towards the display. When the position of the eyes is not substantially pointed towards the camera or the display, the user may be determined to not be gazing towards the display. In some implementations, a first normal axis of a lens of the camera is substantially aligned with a second normal axis of the display of the user equipment device. The facial feature is identified from the snapshot. A determination is made as to whether a third normal axis of the facial feature is substantially parallel to the second normal axis of the display. The user is determined not to be gazing towards the display when the third normal axis is not substantially parallel to the second normal axis.
In response to determining the user is not gazing towards the display when the content being presented meets the criterion, the first state of the setting of the user equipment device is adjusted to a second state to alert the user about the content. In particular, a volume level may be increased or decreased from one value to another.
In some embodiments, after the setting is adjusted, a determination is made as to whether the user is gazing towards the display. In response to determining the user is now gazing towards the display, the setting may be adjusted or returned to the original state before being adjusted.
The above and other objects and advantages of the invention will be apparent upon consideration of the following detailed description, taken in conjunction with the accompanying drawings, in which like reference characters refer to like parts throughout, and in which:
The amount of content available to users in any given content delivery system can be substantial. Consequently, many users desire a form of media guidance through an interface that allows users to efficiently navigate content selections and easily identify content that they may desire. An application that provides such guidance is referred to herein as an interactive media guidance application or, sometimes, a media guidance application or a guidance application.
Interactive media guidance applications may take various forms depending on the content for which they provide guidance. One typical type of media guidance application is an interactive television program guide. Interactive television program guides (sometimes referred to as electronic program guides) are well-known guidance applications that, among other things, allow users to navigate among and locate many types of content or media assets. Interactive media guidance applications may generate graphical user interface screens that enable a user to navigate among, locate and select content. As referred to herein, the terms “media asset” and “content” should be understood to mean an electronically consumable user asset, such as television programming, as well as pay-per-view programs, on-demand programs (as in video-on-demand (VOD) systems), Internet content (e.g., streaming content, downloadable content, webcasts, etc.), video clips, audio, content information, pictures, rotating images, documents, playlists, websites, articles, books, electronic books, blogs, advertisements, chat sessions, social media, applications, games, and/or any other media or multimedia and/or combination of the same. Guidance applications also allow users to navigate among and locate content. As referred to herein, the term “multimedia” should be understood to mean content that utilizes at least two different content forms described above, for example, text, audio, images, video, or interactivity content forms. Content may be recorded, played, displayed or accessed by user equipment devices, but can also be part of a live performance.
With the advent of the Internet, mobile computing, and high-speed wireless networks, users are accessing media on user equipment devices on which they traditionally did not. As referred to herein, the phrase “user equipment device,” “user equipment,” “user device,” “electronic device,” “electronic equipment,” “media equipment device,” or “media device” should be understood to mean any device for accessing the content described above, such as a television, a Smart TV, a set-top box, an integrated receiver decoder (IRD) for handling satellite television, a digital storage device, a digital media receiver (DMR), a digital media adapter (DMA), a streaming media device, a DVD player, a DVD recorder, a connected DVD, a local media server, a BLU-RAY player, a BLU-RAY recorder, a personal computer (PC), a laptop computer, a tablet computer, a WebTV box, a personal computer television (PC/TV), a PC media server, a PC media center, a hand-held computer, a stationary telephone, a personal digital assistant (PDA), a mobile telephone, a portable video player, a portable music player, a portable gaming machine, a smart phone, or any other television equipment, computing equipment, or wireless device, and/or combination of the same. In some embodiments, the user equipment device may have a front facing screen and a rear facing screen, multiple front screens, or multiple angled screens. In some embodiments, the user equipment device may have a front facing camera and/or a rear facing camera. On these user equipment devices, users may be able to navigate among and locate the same content available through a television. Consequently, media guidance may be available on these devices, as well. The guidance provided may be for content available only through a television, for content available only through one or more of other types of user equipment devices, or for content available both through a television and one or more of the other types of user equipment devices. The media guidance applications may be provided as on-line applications (i.e., provided on a website), or as stand-alone applications or clients on user equipment devices. Various devices and platforms that may implement media guidance applications are described in more detail below.
One of the functions of the media guidance application is to provide media guidance data to users. As referred to herein, the phrase, “media guidance data” or “guidance data” should be understood to mean any data related to content, such as media listings, media-related information (e.g., broadcast times, broadcast channels, titles, descriptions, ratings information (e.g., parental control ratings, critic's ratings, etc.), messages from a crowd of users on a social network, messages from a crowd of users posted to a blog or website, genre or category information, actor information, logo data for broadcasters' or providers' logos, etc.), media format (e.g., standard definition, high definition, 3D, etc.), advertisement information (e.g., text, images, media clips, etc.), on-demand information, blogs, websites, and any other type of guidance data that is helpful for a user to navigate among and locate desired content selections. In some implementations, this data may be referred to as a data feed. As referred to herein the term “crowd” should be understood to mean any number of users greater than one.
As referred to herein, the phrase “in response” should be understood to mean automatically, directly and immediately as a result of, without further input from the user, or automatically based on the corresponding action where intervening inputs or actions may occur.
In addition to, or alternative to, providing access to linear programming (e.g., content that is scheduled to be transmitted to a plurality of user equipment devices at a predetermined time and is provided according to a schedule), from content sources (e.g., television channels or scheduled streaming sources) the media guidance application may also provide access to non-linear programming (e.g., content accessible to a user equipment device at any time and not provided according to a schedule). Non-linear programming may include content from different content sources including on-demand content (e.g., VOD), Internet content (e.g., streaming media, downloadable media, etc.), locally stored content (e.g., content stored on any user equipment device described above or other storage device), or other time-independent content. On-demand content may include movies or any other content provided by a particular content provider (e.g., HBO On Demand providing “The Sopranos” and “Curb Your Enthusiasm”). HBO ON DEMAND is a service mark owned by Time Warner Company L. P. et al. and THE SOPRANOS and CURB YOUR ENTHUSIASM are trademarks owned by the Home Box Office, Inc. Internet content may include web events, such as a chat session or Webcast, or content available on-demand as streaming content or downloadable content through an Internet website or other Internet access (e.g. FTP).
Grid 102 may provide media guidance data for non-linear programming including on-demand listing 114, recorded content listing 116, and Internet content listing 118. A display combining media guidance data for content from different types of content sources is sometimes referred to as a “mixed-media” display. Various permutations of the types of media guidance data that may be displayed that are different from display 100 may be based on user selection or guidance application definition (e.g., a display of only recorded and broadcast listings, only on-demand and broadcast listings, etc.). As illustrated, listings 114, 116, and 118 are shown as spanning the entire time block displayed in grid 102 to indicate that selection of these listings may provide access to a display dedicated to on-demand listings, recorded listings, or Internet listings, respectively. In some embodiments, listings for these content types may be included directly in grid 102. Additional media guidance data may be displayed in response to the user selecting one of the navigational icons 120. (Pressing an arrow key on a user input device may affect the display in a similar manner as selecting navigational icons 120.)
Display 100 may also include video region 122, advertisement 124, and options region 126. Video region 122 may allow the user to view and/or preview programs that are currently available, will be available, or were available to the user. The content of video region 122 may correspond to, or be independent from, one of the listings displayed in grid 102. Grid displays including a video region are sometimes referred to as picture-in-guide (PIG) displays. PIG displays and their functionalities are described in greater detail in Satterfield et al. U.S. Pat. No. 6,564,378, issued May 13, 2003 and Yuen et al. U.S. Pat. No. 6,239,794, issued May 29, 2001, which are hereby incorporated by reference herein in their entireties. PIG displays may be included in other media guidance application display screens of the embodiments described herein.
Advertisement 124 may provide an advertisement for content that, depending on a viewer's access rights (e.g., for subscription programming), is currently available for viewing, will be available for viewing in the future, or may never become available for viewing, and may correspond to or be unrelated to one or more of the content listings in grid 102. Advertisement 124 may also be for products or services related or unrelated to the content displayed in grid 102. Advertisement 124 may be selectable and provide further information about content, provide information about a product or a service, enable purchasing of content, a product, or a service, provide content relating to the advertisement, etc. Advertisement 124 may be targeted based on a user's profile/preferences, monitored user activity, the type of display provided, or on other suitable targeted advertisement bases.
While advertisement 124 is shown as rectangular or banner shaped, advertisements may be provided in any suitable size, shape, and location in a guidance application display. For example, advertisement 124 may be provided as a rectangular shape that is horizontally adjacent to grid 102. This is sometimes referred to as a panel advertisement. In addition, advertisements may be overlaid over content or a guidance application display or embedded within a display. Advertisements may also include text, images, rotating images, video clips, or other types of content described above. Advertisements may be stored in a user equipment device having a guidance application, in a database connected to the user equipment, in a remote location (including streaming media servers), or on other storage means, or a combination of these locations. Providing advertisements in a media guidance application is discussed in greater detail in, for example, Knudson et al., U.S. Patent Application Publication No. 2003/0110499, filed Jan. 17, 2003; Ward, III et al. U.S. Pat. No. 6,756,997, issued Jun. 29, 2004; and Schein et al. U.S. Pat. No. 6,388,714, issued May 14, 2002, which are hereby incorporated by reference herein in their entireties. It will be appreciated that advertisements may be included in other media guidance application display screens of the embodiments described herein.
Options region 126 may allow the user to access different types of content, media guidance application displays, and/or media guidance application features. Options region 126 may be part of display 100 (and other display screens described herein), or may be invoked by a user by selecting an on-screen option or pressing a dedicated or assignable button on a user input device. The selectable options within options region 126 may concern features related to program listings in grid 102 or may include options available from a main menu display. Features related to program listings may include searching for other airtimes or ways of receiving a program, recording a program, enabling series recording of a program, setting program and/or channel as a favorite, purchasing a program, or other features. Options available from a main menu display may include search options, VOD options (e.g., fast-access playback operations), parental control options, Internet options, cloud-based options, device synchronization options, options to set up criteria for triggering alerts to recapture a user's attention, second screen device options, options to access various types of media guidance data displays, options to subscribe to a premium service, options to edit a user's profile, options to access a browse overlay, or other options.
The media guidance application may be personalized based on a user's preferences. A personalized media guidance application allows a user to customize displays and features to create a personalized “experience” with the media guidance application. This personalized experience may be created by allowing a user to input these customizations and/or by the media guidance application monitoring user activity to determine various user preferences. Users may access their personalized guidance application by logging in or otherwise identifying themselves to the guidance application. Customization of the media guidance application may be made in accordance with a user profile. The customizations may include varying presentation schemes (e.g., color scheme of displays, font size of text, etc.), aspects of content listings displayed (e.g., only HDTV or only 3D programming, user-specified broadcast channels based on favorite channel selections, re-ordering the display of channels, recommended content, etc.), desired recording features (e.g., recording or series recordings for particular users, recording quality, etc.), parental control settings, customized presentation of Internet content (e.g., presentation of social media content, e-mail, electronically delivered articles, etc.) and other desired customizations.
The media guidance application may allow a user to provide user profile information or may automatically compile user profile information. The media guidance application may, for example, monitor the content the user accesses and/or other interactions the user may have with the guidance application. Additionally, the media guidance application may obtain all or part of other user profiles that are related to a particular user (e.g., from other websites on the Internet the user accesses, such as www.allrovi.com, from other media guidance applications the user accesses, from other interactive applications the user accesses, from another user equipment device of the user, etc.), and/or obtain information about the user from other sources that the media guidance application may access. As a result, a user can be provided with a unified guidance application experience across the user's different user equipment devices. The user profile information may include identifiers of one or more social networks or blogs the user is associated with. For each social network or blog a username and password (e.g., log-in information) may be stored in the user profile information. Control circuitry 304 may utilize this information in the user profile to determine whether content being presented meets a criterion and trigger an alert if the focus of the user (a facial feature of the user) is not on the display of the user equipment device on which the content is being presented. This type of user experience is described in greater detail below in connection with
Another display arrangement for providing media guidance is shown in
The listings in display 200 are of different sizes (i.e., listing 206 is larger than listings 208, 210, and 212), but if desired, all the listings may be the same size. Listings may be of different sizes or graphically accentuated to indicate degrees of interest to the user or to emphasize certain content, as desired by the content provider or based on user preferences. Various systems and methods for graphically accentuating content listings are discussed in, for example, Yates, U.S. Patent Application Publication No. 2010/0153885, filed Dec. 29, 2005, which is hereby incorporated by reference herein in its entirety.
Users may access content and the media guidance application (and its display screens described above and below) from one or more of their user equipment devices.
In some embodiments, a viewing history may be stored in storage 308 for a user. The viewing history may include indications of which media assets have been viewed by a given user. The viewing history may also include, for each media asset, which portion or portions have or have not been viewed by the user. In some implementations, the viewing history may include indications of which users in a group of users have seen or viewed a media asset or a particular segment of a media asset. The group of users may be users in a certain geographical location (e.g., in the same home) or users that are associated with each other on a social network.
In some embodiments, control circuitry 304 may monitor the content being presented on a content source currently being accessed to determine whether a criterion is met (e.g., whether the attributes of the content meet settings of a user profile). Control circuitry 304 may store in storage 308 a first state of a setting (e.g., a first volume level) of the user equipment device on which the content is being presented. In response to determining that the content does meet the criterion, control circuitry 304 may capture a snapshot of the user using a camera associated with the user equipment device on which the content is being presented and process the snapshot to identify a facial feature of the user. In some implementations, control circuitry 304 may process the facial features to determine whether the user is gazing towards the display on which the content is being presented. When the user is determined to not be gazing towards the display, control circuitry 304 may adjust the first state of the setting to a second state (e.g., a second volume level that is higher than the first volume level) to alert the user about the content. In some embodiments, control circuitry 304 may adjust a setting of a second user equipment device (e.g., mobile phone or tablet) that is different from the user equipment device (e.g., television) on which the content is being presented to alert the user. Control circuitry 304 may capture another snapshot of the user and process the facial features to determine whether the user is gazing towards the display on which the content is being presented. Alternatively or in addition, control circuitry 304 may receive a user input confirming the user received the alert instead of or in addition to capturing another snapshot. When the user is determined to be gazing towards the display, control circuitry 304 may adjust the second state of the setting back to the first state that is stored in storage 308 by retrieving from storage 308 the value of the first state.
Control circuitry 304 may be based on any suitable processing circuitry such as processing circuitry 306. As referred to herein, processing circuitry should be understood to mean circuitry based on one or more microprocessors, microcontrollers, digital signal processors, programmable logic devices, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), etc., and may include a multi-core processor (e.g., dual-core, quad-core, hexa-core, or any suitable number of cores) or supercomputer. In some embodiments, processing circuitry may be distributed across multiple separate processors or processing units, for example, multiple of the same type of processing units (e.g., two Intel Core i7 processors) or multiple different processors (e.g., an Intel Core i5 processor and an Intel Core i7 processor). In some embodiments, control circuitry 304 executes instructions for a media guidance application stored in memory (i.e., storage 308). Specifically, control circuitry 304 may be instructed by the media guidance application to perform the functions discussed above and below. For example, the media guidance application may provide instructions to control circuitry 304 to generate the media guidance displays. In some implementations, any action performed by control circuitry 304 may be based on instructions received from the media guidance application.
In some embodiments, control circuitry 304 may include voice or verbal input processing circuitry, such as a voice recognition engine 316. This voice or verbal input processing circuitry may include any interface that performs a voice recognition process. Control circuitry 304 may continuously or periodically access the microphone of user input interface 310 to detect the presence of voice input. Upon detecting voice input, control circuitry 304 may utilize voice recognition engine 316 (implemented in software or with dedicated circuitry) to identify words/phrases in the voice input. As used herein, voice input and verbal input are interchangeable terms and should be understood to have the same meaning. The voice recognition process may perform analog-to-digital conversion on the voice input and process the digital information to form a digital representation of the voice input (e.g., a digital sentence with the identified words/phrases). The voice recognition process may output to another component of control circuitry 304 the identified or extracted words or phrases for further processing. In some implementations, voice recognition engine 316 may be implemented to only respond to a voice received from a particular user or group of users. For example, in an environment where multiple users are present and from whom all of their voices are received by user input interface 310, control circuitry 304 may only respond to one or a selected few of the voices. This way, only one or the selected members of the users can be authorized to provide instructions using verbal commands to control circuitry 304.
In some embodiments, voice recognition engine 316 may include a voice command database. Voice command database may include a list of words/phrases and a corresponding set of instructions for each word/phrase. In response to receiving a verbal input from a user, voice recognition engine 316 may process the verbal input to extract words/phrases from the verbal input. Voice recognition engine 316 may cross-reference the identified words/phrases with the voice command database to retrieve the corresponding set of instructions for the identified words/phrases. For example, the voice command database may include an entry with the phrase “last channel” or any variation thereof. The phrase “last channel” may correspond to a set of instructions that instruct control circuitry 304 to return to one or more content sources included in a list of content sources from which content meeting the criterion was previously presented to the user (discussed below in
In some embodiments, control circuitry 304 may include a biometric response engine 318. Biometric response engine 318 may include circuitry that remotely or directly monitors a user's physiological and physical responses to content being presented. For example, biometric response engine 318 may include a heart pulse rate monitor, facial expression or position detection circuitry, eye movement and position detection circuitry, breathing pattern detection circuitry, user position and orientation detection circuitry, and/or any combination thereof. Control circuitry 304 may continuously or periodically access biometric response engine 318 to detect any changes (or interest) in physiological and/or physical attributes of a user at or during specified sections of a media content being presented (e.g., when the media content is determined to match a criterion). Upon detecting a change in the biometric response of the user (e.g., indicating the user is not gazing towards the display on which the media content is being presented), control circuitry 304 may utilize a biometric response database to identify a corresponding set of instructions to perform.
The biometric response database may include a mapping between a given biometric response and a set of instructions for processing by control circuitry 304. For example, a biometric response database may include an entry that indicates the position of the user's face, nose or eyes is not towards the display on which content is being presented. When the biometric response database entry matches the biometric response, biometric response engine 318 may determine that the user is no longer gazing towards the display on which the content is being presented. As a result, control circuitry 304 may generate an alert (e.g., by adjusting a state of a setting of a user equipment device) when the content meets a criterion. In some implementations, the alert may be triggered on a second user equipment device (e.g., a mobile phone or tablet device) that is not the same first user equipment device (e.g., television) on which the content satisfying the criterion is being presented. In such circumstances, the alert may be provided as a text message or by changing a setting of the second user equipment device (e.g., setting off a ringer). After biometric response engine 318 indicates the user is gazing towards the content being presented, control circuitry 304 may retrieve a previous state of the setting from storage 308 and return the state of the setting back to the value it was before control circuitry 304 adjusted the state.
In client-server-based embodiments, control circuitry 304 may include communications circuitry suitable for communicating with a guidance application server or other networks or servers. The instructions for carrying out the above-mentioned functionality may be stored on the guidance application server. Communications circuitry may include a cable modem, an integrated services digital network (ISDN) modem, a digital subscriber line (DSL) modem, a telephone modem, Ethernet card, or a wireless modem for communications with other equipment, or any other suitable communications circuitry. Such communications may involve the Internet or any other suitable communications networks or paths (which are described in more detail in connection with
Memory may be an electronic storage device provided as storage 308 that is part of control circuitry 304. As referred to herein, the phrase “electronic storage device” or “storage device” should be understood to mean any device for storing electronic data, computer software, or firmware, such as random-access memory, read-only memory, hard drives, optical drives, digital video disc (DVD) recorders, compact disc (CD) recorders, BLU-RAY disc (BD) recorders, BLU-RAY 3D disc recorders, digital video recorders (DVR, sometimes called a personal video recorder, or PVR), solid state devices, quantum storage devices, gaming consoles, gaming media, or any other suitable fixed or removable storage devices, and/or any combination of the same. Storage 308 may be used to store various types of content described herein as well as media guidance information, described above, and guidance application data, described above. Nonvolatile memory may also be used (e.g., to launch a boot-up routine and other instructions). Cloud-based storage, described in relation to
Control circuitry 304 may include video generating circuitry and tuning circuitry, such as one or more analog tuners, one or more MPEG-2 decoders or other digital decoding circuitry, high-definition tuners, or any other suitable tuning or video circuits or combinations of such circuits. Encoding circuitry (e.g., for converting over-the-air, analog, or digital signals to MPEG signals for storage) may also be provided. Control circuitry 304 may also include scaler circuitry for upconverting and downconverting content into the preferred output format of the user equipment 300. Circuitry 304 may also include digital-to-analog converter circuitry and analog-to-digital converter circuitry for converting between digital and analog signals. The tuning and encoding circuitry may be used by the user equipment device to receive and to display, to play, or to record content. The tuning and encoding circuitry may also be used to receive guidance data. The circuitry described herein, including for example, the tuning, video generating, encoding, decoding, encrypting, decrypting, scaler, and analog/digital circuitry, may be implemented using software running on one or more general purpose or specialized processors. Multiple tuners may be provided to handle simultaneous tuning functions (e.g., watch and record functions, picture-in-picture (PIP) functions, multiple-tuner recording, etc.). If storage 308 is provided as a separate device from user equipment 300, the tuning and encoding circuitry (including multiple tuners) may be associated with storage 308.
A user may send instructions to control circuitry 304 using user input interface 310. User input interface 310 may be any suitable user interface, such as a remote control, mouse, trackball, keypad, keyboard, microphone, touch screen, touchpad, stylus input, joystick, voice recognition interface, camera, infrared sensors, or other user input interfaces. Display 312 may be provided as a stand-alone device or integrated with other elements of user equipment device 300. Display 312 may be one or more of a monitor, a television, a liquid crystal display (LCD) for a mobile device, or any other suitable equipment for displaying visual images. In some embodiments, display 312 may be HDTV-capable. In some embodiments, display 312 may be a 3D display, and the interactive media guidance application and any suitable content may be displayed in 3D. A video card or graphics card may generate the output to the display 312. The video card may offer various functions such as accelerated rendering of 3D scenes and 2D graphics, MPEG-2/MPEG-4 decoding, TV output, or the ability to connect multiple monitors. The video card may be any processing circuitry described above in relation to control circuitry 304. The video card may be integrated with the control circuitry 304. Speakers 314 may be provided as integrated with other elements of user equipment device 300 or may be stand-alone units. The audio component of videos and other content displayed on display 312 may be played through speakers 314. In some embodiments, the audio may be distributed to a receiver (not shown), which processes and outputs the audio via speakers 314.
The guidance application may be implemented using any suitable architecture. For example, it may be a stand-alone application wholly implemented on user equipment device 300. In such an approach, instructions of the application are stored locally, and data for use by the application is downloaded on a periodic basis (e.g., from an out-of-band feed, from an Internet resource, or using another suitable approach). In some embodiments, the media guidance application is a client-server based application. Data for use by a thick or thin client implemented on user equipment device 300 is retrieved on-demand by issuing requests to a server remote to the user equipment device 300. In one example of a client-server based guidance application, control circuitry 304 runs a web browser that interprets web pages provided by a remote server.
In some embodiments, the media guidance application is downloaded and interpreted or otherwise run by an interpreter or virtual machine (run by control circuitry 304). In some embodiments, the guidance application may be encoded in the ETV Binary Interchange Format (EBIF), received by control circuitry 304 as part of a suitable feed, and interpreted by a user agent running on control circuitry 304. For example, the guidance application may be an EBIF application. In some embodiments, the guidance application may be defined by a series of JAVA-based files that are received and run by a local virtual machine or other suitable middleware executed by control circuitry 304. In some of such embodiments (e.g., those employing MPEG-2 or other digital media encoding schemes), the guidance application may be, for example, encoded and transmitted in an MPEG-2 object carousel with the MPEG audio and video packets of a program.
User equipment device 300 of
A user equipment device utilizing at least some of the system features described above in connection with
In system 400, there is typically more than one of each type of user equipment device but only one of each is shown in
In some embodiments, a user equipment device (e.g., user television equipment 402, user computer equipment 404, wireless user communications device 406) may be referred to as a “second screen device.” For example, a second screen device may supplement content presented on a first user equipment device. The content presented on the second screen device may be any suitable content that supplements the content presented on the first device. In some embodiments, the second screen device provides an interface for adjusting settings and display preferences of the first device. In some embodiments, the second screen device is configured for interacting with other second screen devices or for interacting with a social network. The second screen device can be located in the same room as the first device, a different room from the first device but in the same house or building, or in a different building from the first device.
The user may also set various settings to maintain consistent media guidance application settings across in-home devices and remote devices. Settings include those described herein, as well as channel and program favorites, programming preferences that the guidance application utilizes to make programming recommendations, display preferences, and other desirable guidance settings. For example, if a user sets a channel as a favorite on, for example, the website www.allrovi.com on their personal computer at their office, the same channel would appear as a favorite on the user's in-home devices (e.g., user television equipment and user computer equipment) as well as the user's mobile devices, if desired. Therefore, changes made on one user equipment device can change the guidance experience on another user equipment device, regardless of whether they are the same or a different type of user equipment device. In addition, the changes made may be based on settings input by a user, as well as user activity monitored by the guidance application.
The user equipment devices may be coupled to communications network 414. Namely, user television equipment 402, user computer equipment 404, and wireless user communications device 406 are coupled to communications network 414 via communications paths 408, 410, and 412, respectively. Each user of the user equipment devices may be associated with different users in a crowd of users. Communications network 414 may be one or more networks including the Internet, a mobile phone network, mobile voice or data network (e.g., a 4G or LTE network), cable network, public switched telephone network, or other types of communications network or combinations of communications networks. Paths 408, 410, and 412 may separately or together include one or more communications paths, such as a satellite path, a fiber-optic path, a cable path, a path that supports Internet communications (e.g., IPTV), free-space connections (e.g., for broadcast or other wireless signals), or any other suitable wired or wireless communications path or combination of such paths. Path 412 is drawn with dotted lines to indicate that in the exemplary embodiment shown in
Although communications paths are not drawn between user equipment devices, these devices may communicate directly with each other via communication paths, such as those described above in connection with paths 408, 410, and 412, as well as other short-range point-to-point communication paths, such as USB cables, IEEE 1394 cables, wireless paths (e.g., Bluetooth, infrared, IEEE 802-11x, etc.), or other short-range communication via wired or wireless paths. BLUETOOTH is a certification mark owned by Bluetooth SIG, INC. The user equipment devices may also communicate with each other directly through an indirect path via communications network 414.
System 400 includes content source 416 and media guidance data source 418 coupled to communications network 414 via communication paths 420 and 422, respectively. Paths 420 and 422 may include any of the communication paths described above in connection with paths 408, 410, and 412. Communications with the content source 416 and media guidance data source 418 may be exchanged over one or more communications paths, but are shown as a single path in
Content source 416 may include one or more types of content distribution equipment including a television distribution facility, cable system headend, satellite distribution facility, programming sources (e.g., television broadcasters, such as NBC, ABC, HBO, etc.), intermediate distribution facilities and/or servers, Internet providers, on-demand media servers, and other content providers. NBC is a trademark owned by the National Broadcasting Company, Inc., ABC is a trademark owned by the American Broadcasting Company, Inc., and HBO is a trademark owned by the Home Box Office, Inc. Content source 416 may be the originator of content (e.g., a television broadcaster, a Webcast provider, etc.) or may not be the originator of content (e.g., an on-demand content provider, an Internet provider of content of broadcast programs for downloading, etc.). Content source 416 may include cable sources, satellite providers, on-demand providers, Internet providers, over-the-top content providers, or other providers of content. Content source 416 may also include a remote media server used to store different types of content (including video content selected by a user), in a location remote from any of the user equipment devices. Systems and methods for remote storage of content, and providing remotely stored content to user equipment are discussed in greater detail in connection with Ellis et al., U.S. Pat. No. 7,761,892, issued Jul. 20, 2010, which is hereby incorporated by reference herein in its entirety.
Media guidance data source 418 may provide media guidance data, such as the media guidance data described above. Media guidance application data may be provided to the user equipment devices using any suitable approach. In some embodiments, the guidance application may be a stand-alone interactive television program guide that receives program guide data via a data feed (e.g., a continuous feed or trickle feed).
Program schedule data and other guidance data may be provided to the user equipment on a television channel sideband, using an in-band digital signal, using an out-of-band digital signal, or any other suitable data transmission technique. Program schedule data and other media guidance data may be provided to user equipment on multiple analog or digital television channels.
In some embodiments, guidance data from media guidance data source 418 may be provided to users' equipment using a client-server approach. For example, a user equipment device may pull media guidance data from a server, or a server may push media guidance data to a user equipment device. In some embodiments, a guidance application client residing on the user's equipment may initiate sessions with source 418 to obtain guidance data when needed, e.g., when the guidance data is out of date or when the user equipment device receives a request from the user to receive data. Media guidance may be provided to the user equipment with any suitable frequency (e.g., continuously, daily, a user-specified period of time, a system-specified period of time, in response to a request from user equipment, etc.). Media guidance data source 418 may provide user equipment devices 402, 404, and 406 the media guidance application itself or software updates for the media guidance application.
Media guidance applications may be, for example, stand-alone applications implemented on user equipment devices. For example, the media guidance application may be implemented as software or a set of executable instructions which may be stored in storage 308, and executed by control circuitry 304 of a user equipment device 300. In some embodiments, media guidance applications may be client-server applications where only a client application resides on the user equipment device, and a server application resides on a remote server. For example, media guidance applications may be implemented partially as a client application on control circuitry 304 of user equipment device 300 and partially on a remote server as a server application (e.g., media guidance data source 418) running on control circuitry of remote server 415. When executed by control circuitry of remote server 415 (such as media guidance data source 418), the media guidance application may instruct the control circuitry to generate the guidance application displays and transmit the generated displays to the user equipment devices. The server application may instruct the control circuitry of the media guidance data source 418 to transmit data for storage on the user equipment. The client application may instruct control circuitry of the receiving user equipment to generate the guidance application displays.
Content and/or media guidance data delivered to user equipment devices 402, 404, and 406 may be over-the-top (OTT) content. OTT content delivery allows Internet-enabled user devices, including any user equipment device described above, to receive content that is transferred over the Internet, including any content described above, in addition to content received over cable or satellite connections. OTT content is delivered via an Internet connection provided by an Internet service provider (ISP), but a third party distributes the content. The ISP may not be responsible for the viewing abilities, copyrights, or redistribution of the content, and may only transfer IP packets provided by the OTT content provider. Examples of OTT content providers include YOUTUBE, NETFLIX, and HULU, which provide audio and video via IP packets. Youtube is a trademark owned by Google Inc., Netflix is a trademark owned by Netflix Inc., and Hulu is a trademark owned by Hulu, LLC. OTT content providers may additionally or alternatively provide media guidance data described above. In addition to content and/or media guidance data, providers of OTT content can distribute media guidance applications (e.g., web-based applications or cloud-based applications), or the content can be displayed by media guidance applications stored on the user equipment device.
Media guidance system 400 is intended to illustrate a number of approaches, or network configurations, by which user equipment devices and sources of content and guidance data may communicate with each other for the purpose of accessing content and providing media guidance. The embodiments described herein may be applied in any one or a subset of these approaches, or in a system employing other approaches for delivering content and providing media guidance. The following four approaches provide specific illustrations of the generalized example of
In one approach, user equipment devices may communicate with each other within a home network. User equipment devices can communicate with each other directly via short-range point-to-point communication schemes described above, via indirect paths through a hub or other similar device provided on a home network, or via communications network 414. Each of the multiple individuals in a single home may operate different user equipment devices on the home network. As a result, it may be desirable for various media guidance information or settings to be communicated between the different user equipment devices. For example, it may be desirable for users to maintain consistent media guidance application settings on different user equipment devices within a home network, as described in greater detail in Ellis et al., U.S. patent application Ser. No. 11/179,410, filed Jul. 11, 2005. Different types of user equipment devices in a home network may also communicate with each other to transmit content. For example, a user may transmit content from user computer equipment to a portable video player or portable music player.
In a second approach, users may have multiple types of user equipment by which they access content and obtain media guidance. For example, some users may have home networks that are accessed by in-home and mobile devices. Users may control in-home devices via a media guidance application implemented on a remote device. For example, users may access an online media guidance application on a website via a personal computer at their office, or a mobile device such as a PDA or web-enabled mobile telephone. The user may set various settings (e.g., recordings, reminders, or other settings) on the online guidance application to control the user's in-home equipment. The online guide may control the user's equipment directly, or by communicating with a media guidance application on the user's in-home equipment. Various systems and methods for user equipment devices communicating, where the user equipment devices are in locations remote from each other, is discussed in, for example, Ellis et al., U.S. Pat. No. 8,046,801, issued Oct. 25, 2011, which is hereby incorporated by reference herein in its entirety.
In a third approach, users of user equipment devices inside and outside a home can use their media guidance application to communicate directly with content source 416 to access content. Specifically, within a home, users of user television equipment 402 and user computer equipment 404 may access the media guidance application to navigate among and locate desirable content. Users may also access the media guidance application outside of the home using wireless user communications devices 406 to navigate among and locate desirable content.
In a fourth approach, user equipment devices may operate in a cloud computing environment to access cloud services. In a cloud computing environment, various types of computing services for content sharing, storage or distribution (e.g., video sharing sites, blogs, news sites, or social networking sites) are provided by a collection of network-accessible computing and storage resources, referred to as “the cloud.” For example, the cloud can include a collection of server computing devices, which may be located centrally or at distributed locations, that provide cloud-based services to various types of users and devices connected via a network such as the Internet via communications network 414. These cloud resources may include one or more content sources 416 and one or more media guidance data sources 418. In addition or in the alternative, the remote computing sites may include other user equipment devices, such as user television equipment 402, user computer equipment 404, and wireless user communications device 406. For example, the other user equipment devices may provide access to a stored copy of a video or a streamed video. In such embodiments, user equipment devices may operate in a peer-to-peer manner without communicating with a central server.
The cloud provides access to services, such as content storage, content sharing, access to messages posted by users in a crowd, or social networking services, among other examples, as well as access to any content described above, for user equipment devices. Services can be provided in the cloud through cloud computing service providers, or through other providers of online services. For example, the cloud-based services can include a content storage service, a content sharing site, a social networking site, or other services via which user-sourced content is distributed for viewing by others on connected devices. These cloud-based services may allow a user equipment device to store content to the cloud and to receive content from the cloud rather than storing content locally and accessing locally stored content.
A user may use various content capture devices, such as camcorders, digital cameras with video mode, audio recorders, mobile phones, and handheld computing devices, to record content. The user can upload content to a content storage service on the cloud either directly, for example, from user computer equipment 404 or wireless user communications device 406 having content capture feature. Alternatively, the user can first transfer the content to a user equipment device, such as user computer equipment 404. The user equipment device storing the content uploads the content to the cloud using a data transmission service on communications network 414. In some embodiments, the user equipment device itself is a cloud resource, and other user equipment devices can access the content directly from the user equipment device on which the user stored the content.
Cloud resources may be accessed by a user equipment device using, for example, a web browser, a media guidance application, a desktop application, a mobile application, and/or any combination of access applications of the same. The user equipment device may be a cloud client that relies on cloud computing for application delivery, or the user equipment device may have some functionality without access to cloud resources. For example, some applications running on the user equipment device may be cloud applications, i.e., applications delivered as a service over the Internet, while other applications may be stored and run on the user equipment device. In some embodiments, a user device may receive content from multiple cloud resources simultaneously. For example, a user device can stream audio from one cloud resource while downloading content from a second cloud resource. Or a user device can download content from multiple cloud resources for more efficient downloading. In some embodiments, user equipment devices can use cloud resources for processing operations such as the processing operations performed by processing circuitry described in relation to
In some embodiments, control circuitry 304 may monitor attributes or characteristics of a media asset currently being presented to the user. Control circuitry 304 may retrieve criterion or criteria set by a user and compare the retrieved attributes or characteristics of the media asset currently being presented to the user on display 312 with the retrieved criterion or criteria. In response to determining that the attributes or characteristics of the media asset matches the criterion or criteria set by the user, control circuitry 304 may determine whether the user is gazing at display 312 on which the media asset is being presented. For example, control circuitry 304 may utilize biometric response engine 318 to the user's eyes to identify whether the user's eyes are directed towards (or pointed at) display 312 or another device. A discussion of how biometric response engine 318 determines whether the user is gazing towards display 312 is provided in connection with
In some embodiments, control circuitry 304 may receive a user input selecting an option to customize criterion/criteria and/or ways for alerting the user when content meeting the criterion is being presented on a display towards which the user is not currently gazing. In response to receiving the user input, control circuitry 304 may navigate the user to screen 500 (
Region 510 may include a display that indicates the alert type used to alert the user when the user is not gazing towards the display on which content meeting the criteria is being presented. Specifically, region 510 may list a set of settings that can be modified to alert the user about content. The user may select any one or more of the settings listed in region 510. Control circuitry 304 may use the alert type(s) or settings type specified in region 510 to determine what type of alert to use (e.g., what setting(s) to adjust) to inform the user about content being presented on a display matching criteria. For example, one setting may be noise generation setting. In response to the user enabling this type of setting, control circuitry 304 may adjust the state of the setting to generate an audible signal that includes one or more steady tone(s) for output through speakers 314 when the user is not gazing towards the display on which content meeting criteria is being presented. Another setting may be a volume setting. In response to the user enabling the volume setting, control circuitry 304 may adjust the volume of the content being presented (e.g., raise the volume steadily or continuously) when the user is not gazing towards the display on which content meeting criteria is being presented. Another setting may be a flash setting. In response to the user enabling the flash setting, control circuitry 304 may enable a visual alarm (e.g., turn ON a dedicated lamp) when the user is not gazing towards the display on which content meeting criteria is being presented.
In some embodiments, region 510 may include one or more options to adjust the settings of another device (e.g., a second screen device) to alert the user. Specifically, a first user equipment device may present content that meets criteria and control circuitry 304 may determine whether the user is gazing towards the display associated with the first user equipment device. In response to determining that the user is not gazing towards the display associated with the first user equipment device, control circuitry 304 may adjust the setting of a second user equipment device to alert the user about the content being displayed on the first user equipment device. For example, a setting for alerting the user on the second user equipment device may be an SMS, instant, social network, or MMS message setting. In response to the user enabling the SMS, instant, social network, or MMS message setting, control circuitry 304 may automatically transmit a SMS, instant, social network, or MMS message to a specified second user equipment device(s) when the user is not gazing towards the display on which content meeting criteria is being presented. When the SMS, instant, social network, or MMS message setting is selected, control circuitry 304 may request input from the user that identifies the second user equipment device (e.g., a telephone number, email address, IP address, domain name or other suitable identifier). Another setting for alerting the user on the second user equipment device may be ringer setting. In response to the user enabling the ringer setting, control circuitry 304 may automatically call a specified second user equipment device(s) when the user is not gazing towards the display on which content meeting criteria is being presented. When the ringer setting is selected, control circuitry 304 may request input from the user that identifies the second user equipment device (e.g., a telephone number, email address, IP address, domain name or other suitable identifier).
Region 520 may include various options/settings to change and control what criteria are analyzed and monitored from the content being presented to trigger the measurement of viewing time. Specifically, control circuitry 304 may determine whether attributes/characteristics of the content being presented from a content source matches the criteria specified and selected in region 520. In response to determining that the content being presented has attributes/characteristics that match the criteria selected in region 520, control circuitry 304 may request biometric response engine 318 to determine whether the user is currently gazing towards the display on which the content is being presented.
Region 520 may include a user profile option 521, a genre option 522, a type option 523, a length option 524, a rating option 525, a time of day option 526, a recommended-by-others option 527 and a team option 528. Other options not shown in region 520 may be provided without departing from the teachings of this invention but are not being discussed for the sake of brevity. Options shown in region 520 are illustrated as radio buttons to indicate that any one or all of the options in region 520 may be selected. The criteria defined by those options in region 520 with a radio button selected are used by control circuitry 304 to match against attributes/characteristics of content being presented to the user. The criteria defined by those options in region 520 with a radio button not selected are ignored or not used by control circuitry 304 to match against attributes/characteristics of content being presented to the user.
User profile option 521 may allow a user to specify one or more user profiles. In response to selecting user profile option 521, control circuitry 304 may present a user profile selection menu 560. User profile selection menu 520 may list all of the profiles stored in storage 308, profiles retrieved from a social network, the cloud, or any other profile. The user may select any one or all of the profiles listed in menu 520. As part of the process of determining whether the attributes/characteristics of the content being presented matches the criteria specified in region 520, control circuitry 304 may determine whether attributes of the content meet or exceed preferences stored in the selected profile(s) when option 521 is selected.
In some implementations, a percent match selection option 562 may be provided. The user may input a percentage in option 562 to indicate a threshold value for which attributes/characteristics of content being presented have to exceed in matching preferences stored in the profiles selected in menu 560. For example, control circuitry 304 may compute a Euclidian distance between attributes/characteristics of content being presented and preferences stored in the profile(s) selected in menu 560. Control circuitry 304 may derive a percentage match from the computed Euclidian distance and compare the percentage to the percentage specified in option 562. In response to determining that the percentage match exceeds the percentage specified in option 562, control circuitry 304 may determine that the attributes/characteristics of the content being presented matches the criteria of the user profile option 521. When no value is specified in option 562, control circuitry 304 may determine whether an average number of attributes/characteristics of the content being presented matches the preferences specified in the profile(s) selected in menu 560.
Genre option 522 may allow a user to specify a genre for use in comparing against attributes/characteristics of content being presented to the user. In response to selecting genre option 522, control circuitry 304 may present a list of various genres (e.g., comedy, action, drama, sci-fi, sports, etc.), categories and/or combination categories. The user may select any one or all of the genres, categories and/or combination categories listed for option 522. As part of the process of determining whether the attributes/characteristics of the content being presented matches the criteria specified in region 520, control circuitry 304 may determine whether a genre, category, and/or combination category of the content being presented meets the selected genres, categories and/or combination categories for option 522 when option 522 is selected.
Type option 523 may allow a user to specify a type for use in comparing against attributes/characteristics of content being presented to the user. In response to selecting type option 523, control circuitry 304 may present a list of content types (e.g., animated, high-definition, standard definition, on-demand delivery, recording, satellite content, cable content, internet content, antenna content, etc.). The user may select any one or all of the types listed for option 523. As part of the process of determining whether the attributes/characteristics of the content being presented matches the criteria specified in region 520, control circuitry 304 may determine whether a type of the content being presented meets the selected type for option 523 when option 523 is selected.
Length option 524 may allow a user to specify a duration for use in comparing against attributes/characteristics of content being presented to the user. In response to selecting length option 524, control circuitry 304 may present a drop-down menu 570 with a list of durations for content (e.g., 30 minutes, 1 hour, 2 hours, user specified duration, etc.). The user may select any one or all of the durations listed in menu 570. As part of the process of determining whether the attributes/characteristics of the content being presented match the criteria specified in region 520, control circuitry 304 may determine whether duration of the content being presented meets the duration specified in menu 570 when option 524 is selected.
Rating option 525 may allow a user to specify a rating for use in comparing against attributes/characteristics of content being presented to the user. In response to selecting rating option 525, control circuitry 304 may present a list of ratings (e.g., G, PG, R, PG-13, TV-G, TV-T, TV14, TVMA, TVY7, etc.). The user may select any one or all of the ratings listed for option 525. As part of the process of determining whether the attributes/characteristics of the content being presented match the criteria specified in region 520, control circuitry 304 may determine whether a rating assigned to the content being presented meets or exceeds the selected rating for option 525 when option 525 is selected.
Time of day option 526 may allow a user to specify a time of day for use in comparing against attributes/characteristics of content being presented to the user. In response to selecting time of day option 526, control circuitry 304 may present a list of times of day (e.g., morning, afternoon, night, user specified time range, etc.). The user may select any one or all of the times of day listed for option 526. As part of the process of determining whether the attributes/characteristics of the content being presented match the criteria specified in region 520, control circuitry 304 may determine whether a time of day in which the content is being presented meets the selected time(s) of day for option 526 when option 526 is selected.
Recommended-by-others option 527 may allow a user to select whether to identify content recommended-by-others to compare against attributes/characteristics of content being presented to the user. As part of the process of determining whether the attributes/characteristics of the content being presented match the criteria specified in region 520, control circuitry 304 may determine whether the content being presented meets was Recommended-by-others when option 527 is selected. For example, control circuitry 304 may query a social network associated with the user to retrieve a list of recommendations made by friends of the user on the social network. Control circuitry 304 may determine whether the content being presented matches any one of the recommendations made by friends to determine whether the content being presented meets was recommended-by-others when option 527 is selected.
Team option 528 may allow a user to specify a team for use in comparing against attributes/characteristics of content being presented to the user. In response to selecting team option 528, control circuitry 304 may present a list of contestants, teams or athletes and/or may allow the user to specify a specific contestant, team or athlete using text entry box 580. The user may select any one or all of the teams listed for option 528. As part of the process of determining whether the attributes/characteristics of the content being presented match the criteria specified in region 520, control circuitry 304 may determine whether a contestant, team or athlete associated with or playing in the content being presented meets the selected contestant, team or athlete for option 528 when option 528 is selected. The contestant, team or athlete may be associated with the content being presented if the contestant, team or athlete is playing in the content or is a rival of another team playing in the content being presented.
Region 520 may include a commercial option (not shown). In response to selecting the commercial option, control circuitry 304 may, as part of the process of determining whether the attributes/characteristics of the content being presented match the criteria specified in region 520, determine whether the content is in a commercial break (e.g., a portion of the program that is an advertisement and not a plot portion). Specifically, content presented to the user may be a program that has multiple plot portions with commercial breaks or interruptions between the plot portions. When the commercial option is selected, control circuitry 304 may determine whether the content being presented is in the plot portion or is in a commercial break. If the content being presented is in a commercial break and the commercial option is selected, control circuitry 304 may determine that the content does not match criteria set in region 520. Alternatively, if the content being presented is in a commercial break and the commercial option is not selected, control circuitry 304 may determine that the content does match criteria set in region 520.
Region 520 may include a biometric response condition option (not shown). In response to selecting the biometric response condition option, control circuitry 304 may, as part of the process of determining whether the attributes/characteristics of the content being presented match the criteria specified in region 520, determine biometric response of the user (e.g., a heart rate, attention, face direction, breathing rate, focus, eye gazing, user's presence within range of the screen, etc.) to determine whether the user is interested or is paying attention to the content being presented. If the biometric response of the user indicates that the user is interested in or is paying attention to the content being presented and the biometric response condition option is selected, control circuitry 304 may determine that the content does match criteria set in region 520.
Region 520 may also include a criteria combination selection option 530. Option 530 may allow a user to specify how the options discussed above for region 520 (e.g., options 521-528) are logically combined in comparing against attributes/characteristics of content being presented to the user. Specifically, option 530 may include a logical OR option and a logical AND option. In response to receiving a user selection of the logical OR option in option 530, control circuitry 304 may determine that the content being presented to the user matches the criteria/criterion set in region 520 if the attributes/characteristics of the content match any one of selected options 521-528 in region 520. For example, control circuitry 304 may determine that the content being presented to the user matches the criteria set in region 520 if the genre of the content matches the genre selected for option 522 even though the content being presented does not match a rating specified for option 525 (e.g., the content may be rated R and the selected rating for option 525 may be G or lower).
In response to receiving a user selection of the logical AND option in option 530, control circuitry 304 may determine that the content being presented to the user matches the criteria set in region 520 if the attributes/characteristics of the content match every one of selected options 521-528 in region 520. For example, control circuitry 304 may determine that the content being presented to the user matches the criteria set in region 520 if the genre of the content matches the genre selected for option 522 and the content being presented matches a rating specified for option 525. Similarly, control circuitry 304 may determine that the content being presented to the user fails to match the criteria set in region 520 if the genre of the content matches the genre selected for option 522 and the content being presented does not match a rating specified for option 525.
Control circuitry 304 may first determine the outcome of every option selected for region 520 (e.g., option 521-528) when compared with the content being presented to the user. The outcome of every option 521-528 may be logically TRUE when an attribute of the content being presented matches the criterion set for a given option 521-528. For example, the outcome may be TRUE when time of day option 526 is selected to be morning and the time at which the content being presented to the user is between 8 AM-11 AM. The outcome of every option 521-528 may be logically FALSE when an attribute of the content being presented fails to match the criterion set for a given option 521-528. For example, the outcome may be FALSE when team option 528 is selected and specifies the team “Gators” and the teams playing in the content being presented to the user is “Seminoles” and “Hurricanes.”
After determining the outcome of every option from region 520 that is selected (e.g., option 521-528), control circuitry 304 may determine whether option 530 is selected to be a logical OR or a logical AND. When option 530 specifies a logical OR was selected, control circuitry 304 may determine whether the outcome of any selected option 521-528 was TRUE. If the outcome of any selected option 521-528 was TRUE, control circuitry 304 may determine that the content being presented to the user matches criteria set in region 520. Consequently, control circuitry 304 may determine (e.g., using biometric response engine 318) whether the user is currently gazing towards the screen or display on which the content is being presented. When option 530 specifies a logical AND was selected, control circuitry 304 may determine whether the outcome of every selected option 521-528 was TRUE. If the outcome of any selected option 521-528 was FALSE, control circuitry 304 may determine that the content being presented to the user does not match criteria set in region 520. Consequently, control circuitry 304 may not determine (e.g., using biometric response engine 318) whether the user is currently gazing towards the screen or display on which the content is being presented.
In some embodiments, control circuitry 304 may sequentially determine whether an outcome of a given option 521-528 when compared with the content being presented to the user meets the selected logical operation specified in option 530. This is to avoid having to first compute and determine whether the content being presented to the user matches all of the selected options and then determining whether the logical operation is met. Specifically, control circuitry 304 may first determine whether option 530 is selected to be a logical OR or a logical AND. After determining whether option 530 is selected to be a logical OR or a logical AND operation, control circuitry 304 may determine an outcome of a first of options 521-528 selected for region 520 when compared with the content being presented to the user. If option 530 is selected to be a logical OR and in response to determining that the outcome of the first of selected options 521-528 is FALSE, control circuitry 304 may next determine an outcome of a second of options 521-528. If option 530 is selected to be a logical OR and in response to determining that the outcome of the first of selected options 521-528 is TRUE, control circuitry 304 may stop testing outcomes of the remaining selected options 521-528 and may determine that the content being presented meets criteria/criterion set in region 520. If option 530 is selected to be a logical AND and in response to determining that the outcome of the first of selected options 521-528 is FALSE, control circuitry 304 may stop testing outcomes of the remaining selected options 521-528 and may determine that the content being presented fails to meet criteria/criterion set in region 520. If option 530 is selected to be a logical AND and in response to determining that the outcome of the first of selected options 521-528 is TRUE, control circuitry 304 may next determine an outcome of a second of options 521-528 until outcomes of all of the selected options 521-528 are determined to be TRUE.
Control circuitry 304 may continue to make the determinations of whether the content being presented matches the selected options 521-528 until all of the selected options are tested or until control circuitry 304 determines that the logical operation is TRUE. Specifically, in the case of option 530 being selected to be an OR operation, control circuitry 304 may continue to make the determinations of whether the content being presented matches the selected options 521-528 until all of the selected options are tested or until control circuitry 304 determines that the logical operation is TRUE. In the case of option 530 being selected to be an AND operation, control circuitry 304 may continue to make the determinations of whether the content being presented matches the selected options 521-528 until all of the selected options are tested and determined to be TRUE or until control circuitry 304 determines that the logical operation is FALSE (i.e., any one of the outcomes of selected options 521-528 is determined to be FALSE).
A save option 540 may be included in display 500. In response to receiving a user selection of save option 540, control circuitry 304 may store all of the options selected in regions 510 and 520 and return the user to the previous screen (e.g., the content being presented previously). An exit option 550 may be included in display 500. In response to receiving a user selection of exit option 550, control circuitry 304 may discard all of the options selected in regions 510 and 520 and return the user to the previous screen (e.g., the content being presented previously).
In some embodiments, while content is being presented on a user equipment device, control circuitry 304 may determine whether the content presented matches a criterion or criteria (e.g., the criteria set in region 520 of screen 500 (
In some embodiments, a user may select a content source (e.g., a television channel) and control circuitry 304 may access the selected content source (e.g., tune to the selected television channel). For example, control circuitry 304 may receive a user selection of a program listing from grid 102 and in response may tune to the television channel corresponding to the selected listing. Control circuitry 304 may display content received from the content source as video 610. For example, control circuitry 304 may display as video 610 the video of the program received from the television channel.
Control circuitry 304 may automatically, or in response to a user request (e.g., a selection of an INFO key), display content information overlay 630. Content information overlay 630 may include information about content 610 being presented on the selected content source. For example, content information overlay 630 may include a content source identifier (e.g., a channel name or number), a title of content 610, a scheduled transmission time interval for content 610, a detailed description region 632 and a content attribute region 640. Detailed description region 632 may include a brief description about content 610 being presented. Detailed description region 632 may also include a list of other content related to content 610 being presented.
To populate the information for overlay 630, control circuitry 304 may retrieve from storage 308 a data structure associated with the content being presented. Specifically, control circuitry 304 may generate an SQL query that includes the current time and the content source being accessed. Control circuitry 304 may transmit the SQL query to a local or remote database. The local or remote database may identify a data structure associated with content provided on the content source at the specified time and may return back to control circuitry 304 the data structure. The data structure may include multiple fields. For example, the data structure may include a title field that identifies the title of the accessed content, a related content field identifying related content, a scheduled transmission time field identifying the period of time during which the content is available from the content source, a detailed description field, and content attributes fields which specify different attributes associated with the content. Control circuitry 304 may retrieve the data structure for content 610 before, during or after accessing the content source from which content 610 is being received and presented.
Content attribute region 640 may display attributes received in the data structure for content 610. Specifically, content attribute region 640 may display a list of attributes of the content currently being presented. For example, the data structure for content 610 may indicate that at the current time, the content being presented may be a commercial or advertisement portion of a program. Accordingly, content attribute region 640 may indicate that an advertisement is currently being presented as content 610.
In response to accessing the selected content source, control circuitry 304 may continuously or periodically determine whether attributes of content 610 (e.g., attributes in region 640) match criteria selected in region 520 (
In response to determining that the attributes of content 610 match the criteria selected in region 520, control circuitry 304 may utilize biometric response engine 318 to determine whether the user is gazing towards the display on which the content is being presented. In some implementations, the determination of whether the user is gazing towards the display may be performed before or simultaneously with the determination of whether the content being presented matches the criteria. Control circuitry 304 may periodically or continuously monitor the attributes of content 610. For example, control circuitry 304 may analyze closed-caption data or may re-transmit and request updated data structures for the content being presented on the selected content source from a local or remote database. When attributes of content 610 being presented match the criteria selected in region 520 and the user is determined not to be gazing towards the display, control circuitry 304 may generate an alert (e.g., adjust a setting selected in region 510). Control circuitry 304 may continue to monitor whether the user is gazing towards the display and generating the alert (e.g., adjusting the selected setting) until determining that the user is now gazing towards the display. When attributes of content 610 being presented match the criteria selected in region 520 and the user is determined to be gazing towards the display, control circuitry 304 may not generate the alert (or stop generating the alert) and continue monitoring where the user is gazing and the attributes of the content being presented. If the alert was being generated and control circuitry 304 was waiting for the user to gaze towards the display, control circuitry 304 may return the state of the alert (e.g., the state of the setting) back to the original state before the alert was generated upon determining that the user has now gazed towards the display.
For example, a user may select the FOX television channel as a content source. Control circuitry 304 may tune to the FOX television channel and display video of the show “Family Guy” currently being received on that channel. Control circuitry 304 upon tuning to the selected channel may determine that the content on the selected channel is associated with an advertisement attribute. Control circuitry 304 may determine that criteria selected in region 520 are: genre option 522 with a setting of comedy and type option 523 with a setting of VOD and criteria combination selection option 530 is selected to be OR. Accordingly, control circuitry 304 may determine that the attribute of the show currently being presented does not match the criteria set in region 520. In response to determining that the attributes of the show do not match the criteria, control circuitry 304 may not utilize biometric response engine 318 to alert the user about the show. Specifically, control circuitry 304 may not adjust the settings selected in region 510. For example, the setting selected may be a volume setting and as shown in screen 600, the volume setting 660 remains on the value 20 and is not adjusted.
While the show “Family Guy” is being presented on the selected television channel FOX, control circuitry 304 may monitor the attributes of the show. Control circuitry 304 may determine that after 20 seconds of presenting the show, the attribute of being an advertisement changes because the show returned from a commercial break.
Content attribute region 750 may display attributes received in the data structure for content 710. For example, content attribute region 750 may include a genre field 752 (e.g., comedy), a type field 754 (e.g., animated), a length or duration field 756 (e.g., 1 hour long), a rating field 758 (e.g., PG) and a profile match field 759 (e.g., illustrating how close the attributes of content 710 match preferences in a user profile). Other fields may also be included in region 750. In some implementations, region 750 may list all the fields that match or that do not match those criteria selected in region 520 (
In some embodiments, control circuitry 304 may determine that the show currently being displayed is no longer an advertisement and is now in the plot portion of the show “Family Guy” which is associated with a genre attribute, comedy, and is associated with a show type, animated. Control circuitry 304 may determine that criteria selected in region 520 are: genre option 522 with a setting of comedy and type option 523 with a setting of VOD and criteria combination selection option 530 is selected to be OR. Accordingly, control circuitry 304 may determine that the genre attribute of the show “Family Guy” matches the criteria set in region 520 but that the other attributes of the show do not match the criteria set in region 520. However, because the combination selection option 530 was selected to be OR logic, the logical combination of the outcomes of the comparison between the criteria selected in region 520 and the attributes of the show is a logical TRUE (i.e., only one of the show attributes needs to match the criteria selected in region 520) and thus control circuitry 304 may determine that the attributes of the show match the criteria set in region 520. In response to determining that the attributes of the show do match the criteria, control circuitry 304 may utilize biometric response engine 318 to determine whether the user is gazing towards display 312 on which the show is being presented.
Biometric response engine 318 may determine whether the user is gazing towards display 312 in any suitable way. For example, biometric response engine 318 may perform facial recognition to determine which of a plurality of users in a room correspond to the profile enabled on the user equipment device. In some implementations, biometric response engine 318 may skip this step and select any user in the room. Biometric response engine 318 may track eyes of the selected user to determine whether the eyes are directed towards display 312. For example, biometric response engine 318 may include a camera and may capture a snapshot or image of a selected user. Biometric response engine 318 may process the captured image to retrieve a facial feature of the user in the image. Biometric response engine 318 may determine whether the axis normal to the facial feature is substantially parallel (e.g., within 10 degrees or less) to the normal axis of display 312. In response to determining, the axes are substantially normal to each other, biometric response engine 318 may indicate to control circuitry 304 that the user is gazing towards display 312. In response to determining, the axes are not substantially normal to each other, biometric response engine 318 may indicate to control circuitry 304 that the user is not gazing towards display 312.
After receiving a communication from biometric response engine 318 indicating that the user is not gazing towards display 312 on which the content is being presented, control circuitry 304 may retrieve a list of settings selected in region 510 (
In some embodiments, instead of utilizing biometric response engine 318 to determine whether the user is gazing towards display 312 after adjusting the setting, control circuitry 304 may await user input to confirm the user is gazing towards display 312. For example, after adjusting the volume setting to second state 760, control circuitry 304 may display a prompt indicating to the user that the volume setting was automatically adjusted to recapture the user's attention because the content being displayed meets criteria. After the user selects an onscreen option confirming receipt of the message, control circuitry 304 may retrieve the first state from memory and automatically adjust the state of the setting back to the first state from the second state (e.g., reduce the volume back to the original value 20 before the alert was generated).
In some embodiments, after determining that the user is not gazing towards the display while content meeting the criteria is being presented, control circuitry 304 may automatically pause the content being displayed and/or begin buffering or recording the content being presented. The buffering and recording may continue to be performed automatically until the user is determined to be gazing towards display 312 on which the content is being presented or the user confirms receiving the alert (e.g., selects an option displayed on display 312). Upon receiving selection from the user of the on screen option or determining that the user is now gazing towards display 312, control circuitry 304 may resume playback of the content or provide an option for the user to view the portion of the content that was missed by playing back a recorded version of the content.
In some embodiments, control circuitry 304 may determine whether content being presented is in a commercial break. Control circuitry 304 may monitor data associated with the content being presented to determine when the commercial break ends. Upon determining that the commercial break ended, control circuitry 304 may determine whether the user is gazing towards the display on which the content is being presented. In response to determining the user is not gazing towards the display, control circuitry 304 may generate the alert by adjusting the selected setting. This may allow a user to watch a given content until the commercial portion of the content begins. The user may focus his/her attention on another device or activity and control circuitry 304 may automatically inform the user by generating the alert that the content has returned to the plot portion from the commercial portion if the user is still focusing his/her attention on the other device or activity.
In some embodiments, control circuitry 304 may determine whether content (e.g., a news program) being presented includes important information (e.g., “Breaking News”). Control circuitry 304 may monitor data associated with the content being presented to determine whether a news program is showing important news (e.g., “Breaking News”). Upon determining that the news program is showing important news, control circuitry 304 may determine whether the user is gazing towards the display on which the content is being presented. In response to determining the user is not gazing towards the display, control circuitry 304 may generate the alert by adjusting the selected setting. This may allow a user to disassociate from content being displayed until the important portion of the content begins. The user may focus his/her attention on another device or activity and control circuitry 304 may automatically inform the user by generating the alert that the content being presented includes important information if the user is still focusing his/her attention on the other device or activity.
Biometric response engine 318 may instruct camera 820 to capture an image or snapshot of user 830. Biometric response engine 318 may process the image or snapshot of user 830 to identify one or more facial features of user 830. For example, biometric response engine 318 may identify a nose or eyes of user 830 from the image or snapshot. Biometric response engine 318 may compute a third normal 832 of the facial feature of user 830 that has been identified. Biometric response engine 318 may compare a direction of third normal 832 relative to first normal 812 and second normal 822. In response to determining that the direction of third normal 832 is substantially parallel (e.g., within 10 degrees or less) to first normal 812 and second normal 822, biometric response engine 318 may determine that user 830 is gazing towards display 810. In response to determining that the direction of third normal 832 is not substantially parallel (e.g., within 10 degrees or less) to first normal 812 and second normal 822, biometric response engine 318 may determine that user 830 is not gazing towards display 810.
In some embodiments, biometric response engine 318 may utilize eye tracking to identify a position of eyes of user 830 relative to screen 810. In particular, biometric response engine 318 may utilize camera 820 to determine whether eyes of the user are pointed substantially directly at camera 820 and consequently screen 810 (when camera 820 is substantially parallel with screen 810). In response to determining that the eyes of the user are pointed substantially directly at camera 820, biometric response engine 318 may determine that user 830 is gazing towards display 810. In response to determining that the eyes of the user are not pointed substantially directly at camera 820, biometric response engine 318 may determine that user 830 is not gazing towards display 810.
At step 920, content is caused to be presented on a display of a user equipment device. For example, control circuitry 304 may present content 610 or 710 on a display 312 (
At step 930, a first state of a setting of the user equipment device is stored. For example, control circuitry 304 may store a current value for the user equipment device volume setting (e.g., 20).
At step 940, a characteristic of content currently being presented from a content source is retrieved or determined. For example, control circuitry 304 may retrieve a title and content source identifier (or current time and content source identifier) of the content being presented and prepare an SQL query that includes this information. Control circuitry 304 may transmit the SQL query to a local or remote database. The local or remote database may identify a data structure associated with the content identified by the information in the SQL query. The data structure may include characteristics or attributes of the content. The local or remote database may transmit back to control circuitry 304 the data structure with the characteristics or attributes of the content currently presented to the user. In some implementations, control circuitry 304 may perform an on-the-fly or real-time analysis of the video, audio or metadata associated with the content being presented to determine a characteristic or attribute of the content.
At step 950, a determination is made as to whether the characteristic or attribute of the content meets or matches a criterion/criteria. In response to determining the characteristic or attribute of the content meets or matches criterion/criteria, the process proceeds to step 952, otherwise the process proceeds to step 940. For example, control circuitry 304 may retrieve from the data structure associated with the content being presented one or more content characteristics or attributes and compare those characteristics or attributes to the criteria selected or set in region 520.
At step 952, an image of a user is captured with a camera associated with the user equipment device. For example, control circuitry 304 may instruct biometric response engine 318 to identify a user in the room and take an image of the user in the room.
At step 960, the image is processed to extract a facial feature of the user. For example, control circuitry 304 may perform an image recognition process to identify the face of a user in the image and features of the face (e.g., nose, mouth, eyes, etc.). Control circuitry 304 may extract the portion of the image corresponding to the facial feature.
At step 970, a determination is made as to whether a normal axis of the facial feature is substantially parallel to a normal axis of the display. In response to determining the normal axis of the facial feature is substantially parallel to a normal axis of the display, the process proceeds to step 972, otherwise the process proceeds to step 974. For example, control circuitry 304 may determine a first normal of the display on which content is being presented and determine a second normal of the identified facial feature of the user. Control circuitry 304 may compare the first and second normals to determine whether they are pointed in a direction that is substantially parallel (e.g., within 10 degrees or less).
At step 972, an indication that the user is gazing towards the display is stored.
At step 980, a determination is made as to whether the setting is in a different state from the first state. In response to determining that the setting is in a different state from the first state, the process proceeds to step 982, otherwise the process proceeds to step 940. For example, control circuitry 304 may retrieve from storage 308 the value of the volume that was stored (e.g., 20) and compare that value to the current state of the volume.
At step 974, the first state of the setting of the user equipment device is adjusted to a second state to alert the user about the content being presented. For example, control circuitry 304 may increase or adjust the volume level from one level to another (e.g., from 20 to 45).
It should be understood that the above steps of the flow diagram of
The above-described embodiments of the present disclosure are presented for purposes of illustration and not of limitation, and the present disclosure is limited only by the claims which follow. cm 1. A method for capturing user attention, the method comprising:
