Patent Application
20240029191
The present description relates in general to video processing including, for example, applying watermarks to video content.
Video content represents valuable intellectual property for many individuals and companies. Distribution of video content to the end consumer presents opportunities for illegal copying of the video content. Efforts to curtail illegal copying activity while preserving the quality of the video content for legal viewers include encrypting the video content and providing an encryption key for legal viewers to decrypt and view the video content; manipulating a portion of the frames of video content outside of the viewable portions with an identifier; and/or placing watermark over the entire frames of video content in a manner that minimizes any negative impact on the quality for the video content.
Certain features of the subject technology are set forth in the appended claims. However, for purposes of explanation, several embodiments of the subject technology are set forth in the following figures.
The detailed description set forth below is intended as a description of various configurations of the subject technology and is not intended to represent the only configurations in which the subject technology may be practiced. The appended drawings are incorporated herein and constitute part of the detailed description. The detailed description includes specific details for providing a thorough understanding of the subject technology. However, the subject technology is not limited to the specific details set forth herein and may be practiced without one or more of the specific details. In some instances, structures and components are shown in a block-diagram form in order to avoid obscuring the concepts of the subject technology.
Video content represents valuable intellectual property for many individuals and companies. Distribution of video content to the end consumer presents opportunities for illegal copying of the video content. Efforts to curtail illegal copying activity while preserving the quality of the video content for legal viewers include encrypting the video content and providing an encryption key for legal viewers to decrypt and view the video content; manipulating a portion of the frames of video content outside of the viewable portions (e.g., the overscan or vertical blanking interval region) with an identifier; and/or placing a watermark over the entire frame of video content in a manner that minimizes any negative impact on the quality for the video content.
The foregoing solutions each come with their own challenges and limitations. For example, encrypting the video content requires additional hardware and/or software to perform encryption before distribution of the video content and decryption before consumption of the video content. In addition, decryption keys must be distributed to legal viewers while keeping the decryption keys out of the hands of unauthorized viewers. Identifiers placed outside of the viewable portions of the frames can be masked using video manipulation techniques thereby defeating the use of the identifiers to determine the source of an illegal copy. Applying watermarks to video content may be compromised by a third-party modifying a watermark and thereby rendering it useless for identification purposes or preventing the watermark from being applied to the video content.
Watermarks may be applied to video content at the time of production, prior to distribution, or at the point of consumption. Applying watermarks at the time of production and/or prior to distribution may be overly complex when taking into account the potentially large number of consumers that must be taken into account. Applying watermarks at the point of consumption may be preferred since the video content has been distributed, decoded, and made available on a digital video output (e.g., HDMI). The subject technology provides a secure, relatively simple design for applying watermark graphics to video content at the point of consumption. Aspects of the design provided by the subject technology are illustrated in
Briefly, as illustrated in
The example network environment 200 includes content delivery network (CDN) 210 that is communicably coupled to electronic device 220, such as by network 208. CDN 210 may include, and/or may be communicably coupled to, content server 212, antenna 216, and/or satellite transmitting device 218. Content server 212 can encode and/or transmit encoded data streams, such as MPEG AVC (Advanced Video Coding)/ITU-T H.264 encoded video streams, MPEG HEVC (High-Efficiency Video Coding)/ITU-T H.265 encoded video streams, VP9 encoded video streams, AOM AV1 encoded video streams, and/or MPEG VVC (Versatile Video Coding)/ITU-T H.266 encoded video streams, over network 208. Antenna 216 can transmit encoded data streams over the air, and satellite transmitting device 218 can transmit encoded data streams to satellite 215.
Electronic device 220 may include, and/or may be coupled to, satellite receiving device 222, such as a satellite dish, that receives encoded data streams from satellite 215. In one or more implementations, electronic device 220 may further include an antenna for receiving encoded data streams, such as encoded video streams, over the air from antenna 216 of the CDN 210. Content server 212 and/or electronic device 220 may be, or may include, one or more components of the electronic system described below with respect to
Network 208 may be a public communication network (such as the Internet, a cellular data network or dial-up modems over a telephone network) or a private communications network (such as private local area network (LAN) or leased lines). Network 208 may also include, but is not limited to, any one or more of the following network topologies, including a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or hierarchical network, and the like. In one or more implementations, network 208 may include transmission lines, such as coaxial transmission lines, fiber optic transmission lines, or generally any transmission lines, that communicatively couple content server 212 and electronic device 220.
Content server 212 may include, or may be coupled to, one or more processing devices, data store 214, and/or an encoder. The one or more processing devices execute computer instructions stored in data store 214, for example, to implement a content delivery network. Data store 214 may store the computer instructions on a non-transitory computer-readable medium. An encoder may use a codec to encode video streams, such as an AVC/H.264 codec, an HEVC/H.265 codec, a VP9 codec, an AV1 codec, a VVC/H.266 codec, or any other suitable codec.
In one or more implementations, content server 212 may be a single computing device such as a computer server. Alternatively, content server 212 may represent multiple computing devices that are working together to perform the actions of a server computer (such as a cloud of computers and/or a distributed system). Content server 212 may be coupled with various databases, storage services, or other computing devices, that may be collocated with content server 212 or may be disparately located from content server 212.
Electronic device 220 may include, or may be coupled to, one or more processing devices, a memory, and/or a decoder, such as a hardware decoder. Electronic device 220 may be any device that is capable of decoding an encoded video stream, such as a VVC/H.266 encoded video stream, and displaying the decoded video stream on a display.
In one or more implementations, electronic device 220 may be, or may include all or part of, a set-top box, a laptop or desktop computer, a smartphone, a tablet device, a wearable electronic device such as a pair of glasses or a watch with one or more processors coupled thereto and/or embedded therein, a television or other display with one or more processors coupled thereto and/or embedded therein, video game console, or other electronic devices that can be used to receive and decode an encoded data stream, such as an encoded video stream.
In
In the example depicted in
Processor 305 or one or more portions thereof, may be implemented in software (e.g., instructions, subroutines, code), may be implemented in hardware (e.g., an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), a controller, a state machine, gated logic, discrete hardware components, or any other suitable devices) and/or a combination of both.
Secure processor 310 may include suitable logic, circuitry, and/or code that enable processing data and/or securely controlling operations of SGW module 325. In this regard, secure processor 310 may be configured to execute trusted code stored in locations on electronic device 300 that may not be accessible to processor 305 as well as other components of electronic device 300. The trusted code may be considered secure from unwanted modification based on the code being locally stored in a secure location, the code being authenticated using trusted keys, etc. The trusted code may include trusted applications, such as watermark module 340 and configuration module 345 discussed below. Secure processor 310 also may be configured with access to memory locations, such as secure memory 330, that is not accessible to processor 305 or other components of electronic device 300 and is used to store content such as watermark graphics and configuration parameters used by SGW module 325 as described herein.
Secure processor 310 or one or more portions thereof, may be implemented in software (e.g., instructions, subroutines, code), may be implemented in hardware (e.g., an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), a controller, a state machine, gated logic, discrete hardware components, or any other suitable devices) and/or a combination of both.
Video processing hardware 315 may include suitable circuitry, logic, and/or code that enable the processing of video content received by electronic device 300 for display on a display device (not shown). Video processing hardware 315 may include, but is not limited to, decoders, scalers, compositors, graphic feeders, raster-timing generators, color space converters, formatters, output interfaces, etc. utilized in the processing and presentation of video content. These components of video processing hardware 315 may be configured and controlled by processor 305 executing code and/or applications stored in memory 320.
In addition, as depicted in
Memory 320 may include suitable logic, circuitry, and/or code that enable storage of various types of information such as received data, generated data, code, and/or configuration information. Memory 320 may include, for example, random access memory (RAM), read-only memory (ROM), flash memory, magnetic storage, optical storage, etc. As depicted in
According to aspects of the subject technology, secure memory 330 is a portion of memory 320 for which access is restricted to only secure processor 310 and trusted applications executed by processor 310. Other applications, such as applications executed by processor 305, are not allowed to access secure memory 330. The subject technology is not limited to any particular configurations or techniques for controlling access to secure memory 330. For example, the address space used to store items in secure memory 330 may be configured to be addressable only by secure processor 310. While secure memory 330 is represented as being part of memory 320, the subject technology is not limited to this configuration. For example, secure memory 330 may include suitable logic, circuitry, and/or code separate from that of memory 320.
According to aspects of the subject technology, video playback module 335 comprises a computer program having one or more sequences of instructions or code together with associated data and settings. Upon executing the instructions or code, one or more processes are initiated to perform video playback of video content that may include requesting and receiving video content and configuring and managing aspects of video processing hardware 315, with the exception of SGW module 325, to process the video content. Video playback module 335 also may be configured to generate traditional graphics layers to be overlaid on the displayed video content. The traditional graphics layers may include display setting information, navigation menus, graphics generated by third-party applications, etc. Video playback module 335 may be loaded and executed by processor 305.
According to aspects of the subject technology, watermark module 340 comprises a computer program having one or more sequences of instructions or code together with associated data and settings. Upon executing the instructions or code, one or more processes are initiated to manage the generation and storage of watermark graphics for use by SGW module 325. Watermark module 340 is a trusted application that may be executed by secure processor 310. As a trusted application, watermark module 340 may be stored in a secure location, such as secure memory 330, that is not accessible to processor 305, video playback module 335, or other applications that may be executed by processor 305. In addition, the instructions or code of watermark module 340 may be verified or authenticated before execution using a trusted set of keys. In this manner, watermark module 340 is isolated from unwanted interference or modification of the watermark graphics generated and stored for use by SGW module 325.
The subject technology is not limited to any particular type of watermark graphic. For example, the watermark graphics may include recognizable images, random graphic patterns, graphic patterns encoded with information identifying a source of the video content and/or an identifier of the electronic device used to playback the video content or a user logged onto the electronic device used to playback the video content. The watermark graphics may be selected from a library of watermark graphics securely stored on electronic device 300 or may be generated by watermark module 340 using one or more algorithms. The generated watermark graphics may vary in size/resolution and/or in color space. Watermark module 340 may be provided with information on the size/resolution of the video content being played as well as the color space used for the displayed video content. Watermark module 340 may use this information to generate one or more watermark graphics that match the video content in size/resolution and color space. Watermark module 340 is further configured to store the generated watermark graphics in secure memory locations where access is restricted to secure processor 310 and SGW module 325.
According to aspects of the subject technology, configuration module 345 comprises a computer program having one or more sequences of instructions or code together with associated data and settings. Upon executing the instructions or code, one or more processes are initiated to manage the generation and loading of parameter sets used to configure SGW module 325. Configuration module 345 is a trusted application that may be executed by secure processor 310. As a trusted application, configuration module 345 may be stored in a secure location, such as secure memory 330, that is not accessible to processor 305, video playback module 335, or other applications that may be executed by processor 305. In addition, the instructions or code of configuration module 345 may be verified or authenticated before execution using a trusted set of keys. In this manner, configuration module 345 is isolated from unwanted interference or modification of parameter sets used to configured and control SGW module 325.
As noted above, configuration module generates parameter sets used to configure SGW module 325. One or more parameter sets may be generated and loaded into a set of registers in SGW module 325. As discussed further below, SGW module 325 selects and loads one of the one or more parameters sets for configuration and control of SGW module 325 to apply specified watermark graphics to frames of video content. The selection and loading of a parameter set by SGW module 325 are described below. Parameters in a parameter set may include, but are not limited to, a watermark speed which indicates a number of sequential frames from the video content to apply the current parameter set before selecting and loading another parameter set, a watermark address which may be a pointer (e.g., memory address) to a location in the secure memory storing the watermark graphic to be blended with the number of sequential frames indicated by the watermark speed, an alpha bias which is a bias value used by SGW module 325 to attenuate a brightness or luma value of the watermark graphic before the watermark graphic is blended with a frame of video content, and a color palette which may be a pointer (e.g., memory address) to a location in memory of the associated color palette used by a color look-up table in the processing of the video content.
More than one parameter set may be loaded into the set of registers in SGW module 325. The locations of the parameter sets within the set of registers may be indexed to facilitate access to particular parameter sets when configuring SGW module 325. The parameters may include a next index which is an index value identifying the location of the next parameter set that should be selected and loaded by SGW module 325 once the number of frames specified by the watermark speed in the parameter set have been processed. The parameters also may include a defined next index field that validates the next index parameter in the set. The defined next index field is used by SGW module 325 to determine whether to load the parameter set at the register location indicated by the next index parameter based on the validity of the next index parameter. The selection and loading of parameter sets by SGW module 325 are described in more detail below.
As depicted in
According to aspects of the subject technology, SGW module 400 is arranged after raster timing generator 425 and color space converter 430 in a processing pipeline and is configured to receive output canvas size information and frame data at a timing determined by raster timing generator 425 based on an output resolution, where the frame data is converted to an output color space, if needed, by color space converter 430. In this arrangement, SGW module 400 may be placed after a final stage of video manipulation or processing by video processing hardware 315. For example, SGW module 400 may be arranged to apply a watermark graphic to frames of video content after any other graphic overlays, such as traditional graphic layers displaying system information such as volume level, have been applied to the frames. SGW module 400 may be configured to provide the blended frames to digital video formatter 440, which formats the blended frames of video content for output over an interface such as the High-Definition Multimedia Interface (HDMI) and display on a display device. Other processing and/or other interfaces, may be utilized in displaying the blended frame.
As noted above, configuration module 345 may generate and store one or more parameter sets in the SGW module. The one or more parameter sets may be stored in registers 410. Registers 410 may implement a multi-element buffer to store the parameter sets available for selection by control module 405.
As depicted in
As noted above, watermark speed indicates a number of frames of video content for which the current parameter set is used by sGFD 415 and sCMP 420 to process the frames. After the indicated number of frames have been processed, the index value stored as the parameter Next Index in the current parameter set is loaded as the index value Curr Index. The next parameter set is then selected from the multi-element buffer using the Curr Index value, and the parameters stored as ParameterSubsetcurr are replaced with new parameters from the next parameter set. Using the current parameter set to specify the next parameter set creates a linked-list type of system for selecting and loading parameter sets. The parameter sets may be linked to form a loop that repeatedly cycles through the parameters sets that are part of the loop or the parameter sets may be linked to form a chain that has a first parameter set and a last parameter set. The Next Index parameter value in the last parameter set may not be included or may be indicated by the Defined Next Index parameter as being invalid or not defined. In this situation, the parameters stored as ParameterSubsetcurr are used until the Next Index parameter value is provided and is indicated by the Defined Next Index parameter as being valid or defined.
According to aspects of the subject technology, the configuration module may add, remove, replace, and/or update parameters sets stored in the multi-element buffer in registers 410, with the exception of the parameters stored as ParameterSubsetcurr currently being used by the SGW module, during operation of the SGW module. The Defined Next Index parameter may be set to indicate that the Next Index parameter is not defined and therefore operation of the SGW module should not proceed beyond the current parameter set. Accordingly, if the SGW module attempts to use the Next Index parameter value while the Defined Next Index parameter indicates that the Next Index is not defined, the SGW module will stall and continue to use the parameters stored as ParameterSubsetcurr until the Defined Next Index parameter is changed to indicate that the Next Index parameter is valid. This use of the Defined Next Index parameter allows the system to insert a stall in the command chain before starting to modify parameter sets. Once the modifications are complete, the Defined Next Index parameter is updated to indicate that the Next Index parameter is defined and the next parameter set is selected from the multi-element buffer using the Next Index parameter value.
Because the SGW module is able to operate independently to process more than one frame without needing support, the secure processor is freed up to perform other tasks that may be needed in the electronic device. According to aspects of the subject technology, the configuration module may be configured to periodically query status registers on the SGW module to determine if new parameter sets are needed. This querying of the SGW module state is depicted in
According to aspects of the subject technology, the control module may utilize an interrupt to alert the secure processor of certain situations requiring support from the secure processor. For example, if the Defined Next Index parameter value indicates that the Next Index parameter value is missing or is otherwise invalid, an interrupt may be issued to the secure processor to trigger the configuration module to update one or more of the parameters sets already stored in the registers of the SGW module or add new parameter sets to the registers of the SGW module.
Returning to
Resizing the watermark graphic may include upscaling the resolution of the watermark graphic to increase its overall size or downscaling the resolution of the watermark graphic to decrease its overall size. The subject technology is not limited to any particular scaling algorithm or process for either upscaling or downscaling a watermark graphic. It is noted that upscaling involves the use of a single set of filter coefficients. However, downscaling involves the use of multiple sets of filter coefficients, which may be more processing intensive and/or may increase the latency before the watermark graphic is provided to sCMP 420. Accordingly, sGFD 415 may not be configured to automatically downscale the watermark graphic if the canvas size of the frame is smaller than the size of the watermark graphic. In place of downscaling, sGFD 415 may be configured to resize the watermark graphic by cropping the unscaled watermark graphic. For example, sGFD 415 may truncate the lines of the watermark graphic when sufficient pixels have been sent to match the smaller canvas size of the frame.
Changes to the output frame's canvas size as well as changes to the output frame's color space should be infrequent and typically limited to boot-up conditions or reconfiguring the output of the video processing hardware to match a new display setting or a new display device being connected to the electronic device for viewing the video content. The canvas size of the frame and the color space of the frame are provided to sGFD 415 and sCMP 420 by upstream components in the video processing hardware, such as raster timing generator 425 and color space converter 430. When changes to either the canvas size of the frame or the color space of the frame, control module 405 may issue an interrupt to the secure processor to trigger the watermark module to generate one or more new watermark graphics that match the new canvas size of the frame or the new color space of the frame and store the new watermark graphics in secure memory. The parameter sets may then be updated by the configuration module to include pointers to the locations of the new watermark graphics. In this manner, upscaling or cropping the watermark graphics due to differences between the watermark graphics and the frame may be temporary until the new watermark graphics can be generated and the parameter sets updated. According to aspects of the subject technology, a threshold difference between the size of the frame and the size of the watermark graphic may be required before issuing the interrupt to trigger the generation of new watermark graphics.
As depicted in
As depicted in
As noted above, the parameter set may include an alpha bias which is a bias value used to attenuate a brightness or luma value (e.g., alpha value) of the watermark graphic. Like the attenuation factor generated by activity monitor 710, the alpha bias may be a value between 0.0 and 1.0. As depicted in
As depicted in
The operation of the brightness check module is based around observations that linear content does not reveal a luminance change near black, which would indicate that watermark graphics would be undetectable when overlaid on that content. However, the watermark graphic becomes more detectable as the underlying content becomes brighter until the brightness reaches a level where the detectability of the watermark graphic is unacceptable defining a lower limit. A similar upper limit can be defined as the content of the frame approaches white. For example, the lower limit may be ¼ of the maximum brightness and the upper limit may be ¾ of the maximum brightness. As depicted in
Electronic system 1100, for example, can be a set top box, a media converter, a desktop computer, a laptop computer, a tablet computer, a smarthphone, or generally any electronic device that is capable of presenting video content while applying watermark graphics to the video content as described above. Such an electronic system 1100 includes various types of computer readable media and interfaces for various other types of computer readable media. In one or more implementations, the electronic system 1100 may be, or may include, electronic device 220 and/or electronic device 300 with video processing hardware 315 in addition to the components illustrated in
The bus 1108 collectively represents all system, peripheral, and chipset buses that communicatively connect the numerous internal devices of the electronic system 1100. In one or more implementations, the bus 1108 communicatively connects the one or more processing unit(s) 1112 with the ROM 1110, the system memory 1104, and the permanent storage device 1102. From these various memory units, the one or more processing unit(s) 1112 retrieves instructions to execute and data to process in order to execute the processes of the subject disclosure. The one or more processing unit(s) 1112 can be a single processor or a multicore processor in different implementations.
The ROM 1110 stores static data and instructions that are needed by the one or more processing unit(s) 1112 and other modules of the electronic system. The permanent storage device 1102, on the other hand, is a read-and-write memory device. The permanent storage device 1102 is a non-volatile memory unit that stores instructions and data even when the electronic system 1100 is off. One or more implementations of the subject disclosure use a mass-storage device (such as a solid-state drive, or a magnetic or optical disk and its corresponding disk drive) as the permanent storage device 1102.
Other implementations use a removable storage device (such as a flash memory drive, optical disk and its corresponding disk drive, external magnetic hard drive, etc.) as the permanent storage device 1102. Like the permanent storage device 1102, the system memory 1104 is a read-and-write memory device. However, unlike the permanent storage device 1102, the system memory 1104 is a volatile read-and-write memory, such as random access memory. System memory 1104 stores any of the instructions and data that the one or more processing unit(s) 1112 needs at runtime. In one or more implementations, the processes of the subject disclosure are stored in the system memory 1104, the permanent storage device 1102, and/or the ROM 1110. From these various memory units, the one or more processing unit(s) 1112 retrieves instructions to execute and data to process in order to execute the processes of one or more implementations.
The bus 1108 also connects to the input device interface 1114 and the output device interface 1106. The input device interface 1114 enables a user to communicate information and select commands to the electronic system. Input devices used with the input device interface 1114 include, for example, alphanumeric keyboards and pointing devices (also called “cursor control devices”). The output device interface 1106 enables, for example, the display of images generated by the electronic system 1100. Output devices used with the output device interface 1106 include, for example, printers and display devices, such as a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a flexible display, a flat panel display, a solid state display, a projector, or any other device for outputting information. One or more implementations include devices that function as both input and output devices, such as a touchscreen. In these implementations, feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input.
Finally, as shown in
Implementations within the scope of the present disclosure can be partially or entirely realized using a tangible computer-readable storage medium (or multiple tangible computer-readable storage media of one or more types) encoding one or more instructions. The tangible computer-readable storage medium also can be non-transitory in nature.
The computer-readable storage medium can be any storage medium that can be read, written, or otherwise accessed by a general purpose or special purpose computing device, including any processing electronics and/or processing circuitry capable of executing instructions. For example, without limitation, the computer-readable medium can include any volatile semiconductor memory, such as RAM, DRAM, SRAM, T-RAM, Z-RAM, and TTRAM. The computer-readable medium also can include any non-volatile semiconductor memory, such as ROM, PROM, EPROM, EEPROM, NVRAM, flash, nvSRAM, FeRAM, FeTRAM, MRAM, PRAM, CBRAM, SONOS, RRAM, NRAM, racetrack memory, FJG, and Millipede memory.
Further, the computer-readable storage medium can include any non-semiconductor memory, such as optical disk storage, magnetic disk storage, magnetic tape, other magnetic storage devices, or any other medium capable of storing one or more instructions. In some implementations, the tangible computer-readable storage medium can be directly coupled to a computing device, while in other implementations, the tangible computer-readable storage medium can be indirectly coupled to a computing device, e.g., via one or more wired connections, one or more wireless connections, or any combination thereof.
Instructions can be directly executable or can be used to develop executable instructions. For example, instructions can be realized as executable or non-executable machine code or as instructions in a high-level language that can be compiled to produce executable or non-executable machine code. Further, instructions also can be realized as or can include data. Computer-executable instructions also can be organized in any format, including routines, subroutines, programs, data structures, objects, modules, applications, applets, functions, etc. As recognized by those of skill in the art, details including, but not limited to, the number, structure, sequence, and organization of instructions can vary significantly without varying the underlying logic, function, processing, and output.
While the above discussion primarily refers to microprocessor or multicore processors that execute software, one or more implementations are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In one or more implementations, such integrated circuits execute instructions that are stored on the circuit itself.
According to aspects of the subject technology, a method is provided that includes selecting, by a hardware module, a current parameter set; reading, by the hardware module, a watermark graphic from a location in memory indicated by a parameter in the current parameter set, wherein access to the location in memory is restricted to the hardware module and one or more first trusted applications; blending, by the hardware module, the watermark graphic with a frame of video content; and providing, by the hardware module, the blended frame for display.
By restricting access to the location in memory containing the watermark graphic to the hardware module and trusted applications, the subject technology provides a secure approach to applying watermark graphics to video content at the point of consumption with minimized risk of a third party modifying or replacing the watermark graphic. Applying watermarks at the point of consumption is simpler than managing the application of watermarks at points of distribution and/or production. Furthermore, the effective use of watermarks may provide sufficient security for content that encryption/decryption solutions may not be needed thereby preserving computational resources for other processes.
The parameter set may be stored in a set of registers in the hardware module, and access to the set of registers in the hardware module may be restricted to the hardware module and one or more second trusted applications. The current parameter set may be selected from a plurality of parameter sets stored in the set of registers based on an index value indicated in a previous parameter set selected by the hardware module.
As with the secure memory used to store the watermark graphic, restricting access to the set of registers containing the parameter set to the hardware module and trusted applications minimizes the risk of an unauthorized third party manipulating the parameter set to interfere with the watermarking process. In addition, storing and indexing multiple parameter sets the set of registers of the hardware module may facilitate quicker access to selected parameter sets.
The method may further include resizing, by the hardware module, the watermark graphic if a size of the frame is different than a size of the watermark graphic, wherein the resized watermark graphic is blended with the frame of video content. The method may further include cropping, by the hardware module, the watermark graphic if a size of the frame is smaller than a size of the watermark graphic, where the cropped watermark graphic is blended with the frame of video content.
Resizing the watermark graphic at the hardware module if the frame size of the video content is different removes the ability of a third party to interfere with the watermarking process by manipulating the frame size of the video content. The watermark graphic may be resized by upscaling or downscaling the resolution. The subject technology provides the option of cropping the watermark graphic instead of downscaling the resolution of the watermark graphic. Cropping may be more computationally efficient than downscaling allowing the subject technology to provide the resized watermark image more quickly and using less computational resources.
The method may further include attenuating, by the hardware module, a brightness value of the watermark graphic based on a bias value indicated in the current parameter set, wherein the attenuated watermark graphic is blended with the frame of video content. The method may further include determining, by the hardware module, an attenuation factor based on a luma value from the frame of video content; and attenuating, by the hardware module, a brightness value of the watermark graphic based on the determined attenuation factor, wherein the attenuated watermark graphic is blended with the frame of video content.
Attenuation factors may be determined for respective pixels of the frame of video content and are applied to respective brightness values of corresponding pixels of the watermark graphic. The attenuation factor may be determined based on luma values from a plurality of adjacent pixels in the frame of video content. The watermark graphic may be blended by the hardware module with a number of sequential frames of video content, and wherein the number of sequential frames is indicated in the current parameter set.
The brightness or visibility of the watermark graphic in the frame of video content may negatively impact the user experience of viewing the video content. The subject technology utilizes the bias value parameter to control the brightness of the watermark graphic for situations where an owner of the video content, for example, may want the watermark graphic to be displayed more prominently or less prominently over the video content. In addition, attenuating the brightness of the watermark graphic based on luma values of the underlying video content allows the subject technology to avoid disruptions in the user's viewing experience when changes in the video content would otherwise cause the watermark graphic, or portions of the watermark graphic, to be more prominent in the video frame.
According to aspects of the subject technology, a semiconductor device is provided that includes a set of registers, wherein access to the set of registers is restricted to the semiconductor device and a secure processor. The semiconductor device further includes circuitry configured to select a current parameter set from one or more parameter sets stored in the set of registers based on an index value indicated in a previously selected parameter set; read a watermark graphic from a location in memory indicated by a parameter in the current parameter set, wherein access to the location in memory is restricted to the semiconductor device and the secure processor; blend the watermark graphic with a frame of video content; and provide the blended frame for display.
The circuitry may be further configured to resize the watermark graphic if a size of the frame is different than a size of the watermark graphic, where the resized watermark graphic is blended with the frame of video content.
The circuitry may be further configured to attenuate a brightness value of the watermark graphic based on a bias value indicated in the current parameter set, wherein the attenuated watermark graphic is blended with the frame of video content. The circuitry may be further configured to determine an attenuation factor based on luma values from a plurality of adjacent pixels in the frame of video content; and attenuate a brightness value of the watermark graphic based on the determined attenuation factor, wherein the attenuated watermark graphic is blended with the frame of video content. The watermark graphic may be blended with a number of sequential frames of video content, and wherein the number of sequential frames is indicated in the current parameter set.
The features of selecting a parameter set based on an index value from a previous parameter set and/or blending the watermark graphic with multiple sequential frames indicated in the current parameter set allows the subject technology to partially automate the selection and application of watermark graphics to multiple frames of video content which frees up computational resources for other processes.
According to aspects of the subject technology, an electronic device is provided that includes a hardware module including a set of registers; memory; and a processor. The processor is configured to execute one or more trusted applications to store a watermark graphic in a location in the memory, wherein access to the location in the memory is restricted to the one or more trusted applications and the hardware module; and store one or more parameter sets in the set of registers of the hardware module, wherein access to the set of registers is restricted to the one or more trusted applications and the hardware module. The hardware module is configured to select a current parameter set from the one or more parameter sets stored in the set of registers; read the watermark graphic from the location in the memory indicated by a parameter in the current parameter set; blend the watermark graphic with a frame of video content; and provide the blended frame for display.
The processor may be further configured to execute the one or more trusted applications to generate the watermark graphic. The processor may be further configured to execute the one or more trusted applications to query a status indicator of the hardware module, wherein the one or more parameter sets are stored in the set of registers based on the status indicator.
Using trusted applications to generate watermark graphics and storing the watermark graphics in secure memory locations facilitates the advantages of the subject technology to securely and efficiently apply watermark graphics to video content at points of consumption while uniquely identifying the users at those points of consumption.
The hardware module may be further configured to determine an attenuation factor based on one or more luma values from the frame of video content; and attenuate a brightness value of the watermark graphic based on the determined attenuation factor, wherein the attenuated watermark graphic is blended with the frame of video content. The current parameter set may be selected from the one or more parameter sets stored in the set of registers based on an index value indicated in a previous parameter set selected by the hardware module.
The previous description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects. Thus, the claims are not intended to be limited to the aspects shown herein, but are to be accorded the full scope consistent with the language of the claims, wherein reference to an element in the singular is not intended to mean “one and only one” unless specifically so stated, but rather “one or more.” Unless specifically stated otherwise, the term “some” refers to one or more. Pronouns in the masculine (e.g., his) include the feminine and neuter gender (e.g., her and its) and vice versa. Headings and subheadings, if any, are used for convenience only and do not limit the subject disclosure.
The predicate words “configured to”, “operable to”, and “programmed to” do not imply any particular tangible or intangible modification of a subject, but, rather, are intended to be used interchangeably. For example, a processor configured to monitor and control an operation or a component may also mean the processor being programmed to monitor and control the operation or the processor being operable to monitor and control the operation. Likewise, a processor configured to execute code can be construed as a processor programmed to execute code or operable to execute code.
A phrase such as an “aspect” does not imply that such aspect is essential to the subject technology or that such aspect applies to all configurations of the subject technology. A disclosure relating to an aspect may apply to all configurations, or one or more configurations. A phrase such as an aspect may refer to one or more aspects and vice versa. A phrase such as a “configuration” does not imply that such configuration is essential to the subject technology or that such configuration applies to all configurations of the subject technology. A disclosure relating to a configuration may apply to all configurations, or one or more configurations. A phrase such as a configuration may refer to one or more configurations and vice versa.
The word “example” is used herein to mean “serving as an example or illustration.” Any aspect or design described herein as “example” is not necessarily to be construed as preferred or advantageous over other aspects or designs.
All structural and functional equivalents to the elements of the various aspects described throughout this disclosure that are known or later come to be known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be encompassed by the claims. Moreover, nothing disclosed herein is intended to be dedicated to the public regardless of whether such disclosure is explicitly recited in the claims. No claim element is to be construed under the provisions of 35 U.S.C. § 112(f) unless the element is expressly recited using the phrase “means for” or, in the case of a method claim, the element is recited using the phrase “step for.” Furthermore, to the extent that the term “include,” “have,” or the like is used in the description or the claims, such term is intended to be inclusive in a manner similar to the term “comprise” as “comprise” is interpreted when employed as a transitional word in a claim.
Those of skill in the art would appreciate that the various illustrative blocks, modules, elements, components, methods, and algorithms described herein may be implemented as electronic hardware, computer software, or combinations of both. To illustrate this interchangeability of hardware and software, various illustrative blocks, modules, elements, components, methods, and algorithms have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application. Various components and blocks may be arranged differently (e.g., arranged in a different order, or partitioned in a different way), all without departing from the scope of the subject technology.
The predicate words “configured to,” “operable to,” and “programmed to” do not imply any particular tangible or intangible modification of a subject but, rather, are intended to be used interchangeably. For example, a processor configured to monitor and control an operation or a component may also mean the processor being programmed to monitor and control the operation or the processor being operable to monitor and control the operation. Likewise, a processor configured to execute code can be construed as a processor programmed to execute code or operable to execute code.
