1. Field of the Invention
The invention relates to display screen masking for presentation of differing formats of media streams. In aspects thereof, the invention relates to control by a computing device of screen masking in response to information about a media stream to be presented.
2. Related Art
Home theater systems have become very popular due to the high quality that can be attained for a relatively small investment. With the purchase of a large screen television and a surround sound system, a user can get a taste of the movie theatre experience. With the recent reduction in cost of video projectors and consumer desires for a larger more “cineplex” feel, many consumers are moving from big screen televisions to projectors that can produce huge, highly detailed visual presentations; however, this paradigm shift is not without its own set of problems.
A first known problem is that there are distinct aspect ratios for media streams, such as 4×3 (1.33:1) and 16×9 (1.78:1), but the physical screen has only one size. Other common aspect ratios are 1.66:1, 1.85:1, 2.20:1, and 2.35:1; the last three are sometimes called widescreen format. (Some of these aspect ratios are due to use of television, others due to use of screens.)
When an image is for example in 1.85:1 and the screen is 1.33:1 there is a mismatch, and the usual solution is to not use a portion of the screen. When the screen is reflective (such as when a projector is used), this presents a problem because the physical screen is highly reflective, so the part outside the intended media stream displays as an iridescent grey, rather than an unseen black. This diminishes the viewing experience of the viewer.
A first known solution includes velvet masks that can be placed at the top and bottom of the screen to cover the portion that is not being used for display. A second known solution is to letterbox the presentation by adding dark bars at the top and bottom; however, letterboxing doesn't really provide a solution because a TV screen still shows some grayish image.
A second known problem is that it is generally not possible to tell from the media stream itself what the correct aspect ratio should be, thus even when applying the velvet masks solution, manual user adjustment and refinement is required and may not provide the optimal presentation of the media.
A third known problem is that the actual masking used with the media stream might be imperfect and require adjustment, such as at only the top, only the bottom, or both. This is due to some media mastering houses that make errors in transfers of some media from older formats or have to compromise based on an old format or damage to the master media copy. In such cases, an image may be shifted horizontally and/or vertically by a significant number of pixels. In this case velvet masks that have been placed for a first presentation in a first format may obscure part of an image of a second media presentation in the same first format. Manual adjustment is once again an option, but it is an imperfect solution to home theater systems as today's users expect excellence at the touch of a button.
A fourth known problem is that some media streams are permanently reformatted inside an aspect ratio using black bars. This causes the image to be displayed smaller on a display screen with a different aspect ratio. For example, a movie in a ratio of 1.85:1 may have been letterboxed with black bars such that the displayed image including the black bars is now in a ratio of 1.33:1. If a user has a 1.85:1 display screen the image portion of interest to the user is trapped in the black bars.
Accordingly, it would be advantageous to provide a technique for displaying differing formats for media streams that improves on those techniques known in the art.
The invention provides a method and system capable of displaying media streams in a variety of formats on a screen whose aspect ratio is dynamically adjustable to conform to displaying the media format that is the object of the media stream. Specifically, the invention allows the viewable area of a display screen to be dynamically resized using masks and sidebars, thus the resulting viewable area is optimized for the media stream.
In a first embodiment, a database includes metadata for media presentations (such as movies). When a user selects a presentation to view, a server associated with the user queries the database for metadata associated with the presentation selected by the user. The metadata includes aspect ratio and other information for the presentation and sends a response to the server that includes the metadata.
The portion of the metadata returned to the server is used by a mask controller to appropriately format the display screen on which the user will view the presentation. This includes horizontal and vertical adjustments to the size of the viewing area.
In a second embodiment of the invention, additional formatting information is included for controlling the size and placement of the display area For example, a combination of horizontal and vertical offset may be included such that even though the presentation is in a ratio of 1.85:1, the image has been moved down 25 pixels. Additional commands may allow for subtitle space and other anomalies and special feature requirements when viewing some presentations. Resizing of the viewing area can occur at anytime before, during, or after a presentation as required.
In a third embodiment, additional information may be included in the metadata that controls other devices such as audio systems and lighting.
In a fourth embodiment, the user may notify the server and/or the database that the metadata provided is inaccurate. This notification can include the settings that the user has found to be optimal. This may also occur when the database did not initially contain associated metadata for the presentation to be viewed by the user, thus the user is able to provide metadata that may be used by others.
In a fifth embodiment, at least some portion of the database is included at the local server such that the local server is periodically updated. Metadata can be customized by the user to meet their personal taste.
In a sixth embodiment, auto-detection of mask placement is accomplished by analyzing the media stream to determine the aspect ratio of the media stream and/or the location of the boundary between the displayed portion of the media stream of interest to the user and the displayed portion of the video stream not of interest to the user or not displayed.
In a seventh embodiment, masking may be accomplished using non-physical masks in the form of “light masking.” Light masking displays bars, similar to the black bars used to letterbox some media streams, but the bars used for light masking are specifically chosen from a set of colors known to provide relatively equal screen burn-in yet remain unobtrusive.
After reading this application, those skilled in the art would recognize that the invention provides an enabling technology by which substantial advance is made in the art of media streams and digital content representative thereof.
Accordingly, it would be advantageous to provide a technique for presenting differing display formats that is not subject to drawbacks of the known art.
In the description herein, a preferred embodiment of the invention is described, including preferred process steps and data structures. Those skilled in the art would realize, after perusal of this application, that embodiments of the invention might be implemented using a variety of other techniques not specifically described, without undue experimentation or further invention, and that such other techniques would be within the scope and spirit of the invention.
Definitions
The general meaning of each of these following terms is intended to be illustrative and in no way limiting.
The scope and spirit of the invention is not limited to any of these definitions, or to specific examples mentioned therein, but is intended to include the most general concepts embodied by these and other terms.
System Elements
A system 100 includes a database 110, a communication network 120, one or more local servers 130, one or more local players 140, a controller 150, a mask controller 160, and other devices 170.
The database 110 includes a set of digital media metadata 113. The digital content metadata 113 includes technically descriptive information concerning digital content 111 useable in a system 100 as further described herein. For example, but without limitation, digital content metadata 113 can include such information as; aspect ratio, surround sound encoding, pixel offset, equalization sound enhancement, available languages, subtitle availability.
In a preferred embodiment, the database 110 is physically remote to a user 190. In the preferred embodiment, the database 110 is under the supervision of an administrator (not shown but understood by one skilled in the art). In an alternative embodiment, at least some portion of the database 110 is physically local to the user 190. In this embodiment, the database 110 may be included with the local servers 130 or local players 140, and the user 190 may exercise at least some control over the database 110.
Digital content 111 includes audio, video, and combinations thereof as used to present sound and images. For example, but without limitation, digital content 111 can include; movies and songs as might be present on digital media such as Compact Discs, DVDs, Digital Audio Tape, and electronic computer storage devices.
The communication network 120 includes at least a portion of a communication network, such as a LAN, a WAN, the Internet, an intranet, an extranet, a virtual private network, a virtual switched network, or some combination thereof. In a preferred embodiment, the communication network 120 includes a packet switched network such as the Internet, as well as (in addition to or instead of) the communication networks just noted, or any other set of communication networks that enable the elements described herein to perform the functions described herein.
A communication link 121 operates to couple the elements of the system 100 such that the elements can communicate between each other as further described herein.
The system 100 includes one or more local servers 130. Each local server 130 includes a processor, a main memory, and software for executing instructions (not shown, but understood by one skilled in the art). This software preferably includes communications and control software capable of operating the local server 130 consistent with the invention as further explained herein.
The system 100 includes one or more local players 140. Each local player 140 includes a device capable of delivering digital content 111 to a presentation device. For example, but without limitation, a local player 140 may include a DVD player, a digital media stream decoder, a laser disc player, or some combination thereof.
A projector 141 includes any device capable of rendering digital content 111 as humanly viewable media. A display screen 143 includes a surface disposed to display an image. Generally the display screen is flat, smooth, and rectangular; however, there is no requirement that any of these properties exist and the only required property of the display screen 143 is that it has some light reflective property.
Two display screens 143 are illustrated in
The display screen 143 in a system 100 may have practically any dimension. Preferably, the display screen is a rectangle. The unaltered size and horizontal to vertical ratio of the display screen 143 is based on the preference of the user 190 and any physical limitations of the environment that the display screen 143 is in. A first user 190 may prefer an unaltered display screen 143 in a ratio of 4:3 (broadcast television standard). A second user 190 may prefer an unaltered display screen 143 in a 1.78:1 ratio. Regardless of the native size of the display screen 143, the invention may be applied to accommodate viewing of practically any digital content 111.
The controller 150 includes some portion of a computing device capable of interpreting data and issuing commands to a mask controller 160 and other devices 170 responsive to presentation of digital content 111.
The mask controller 160 includes some portion of a computing device capable of interpreting instructions from the controller 150. The mask controller 160 sends commands to move each mask 161 and sidebar 163 to hide or reveal an area of the display screen 143. The mask 161 includes one or more moveable non-reflective surfaces as further described herein. The sidebar 163 includes one or more moveable non-reflective surfaces as further described herein.
Other devices 170 include associative presentation accessories. For example, but without limitation, other devices include; lights, fans, heating and cooling systems, sound systems and combinations thereof. Under the direction of the controller 150, any one of these other devices 170 may be controlled as in integral element of the system 100 as further described herein.
A user 190 includes a human being generally disposed to select and view digital content 111.
Method of Operation
In a preferred embodiment, each display screen 143 includes two masks 161. The first mask 161 runs horizontally across a top portion of the display screen 143 and is capable of reducing the vertical size of the display screen 143 as it obscures successively more screen area starting at the top edge of the display screen 143. A second mask 161 runs horizontally across a bottom portion of the display screen 143 and is capable of reducing the vertical size of the display screen 143 as it obscures successively more screen area starting at the bottom edge of the display screen 143.
The mask 161 is preferably a non-reflective material (such as, black velvet). The mask 161 may be of a relatively rigid material that moves statically retaining its shape and size or it may be flexible such that it stretches or accordions to different dimensions.
In a preferred embodiment, each display screen 143 includes two sidebars 163. The first sidebar 163 runs vertically across a left portion of the display screen 143 and is capable of reducing the horizontal size of the display screen 143 as it obscures successively more screen area starting at the left most vertical edge of the display screen 143. A second sidebar 163 runs vertically across a right portion of the display screen 143 and is capable of reducing the horizontal size of the display screen 143 as it obscures successively more screen area starting at the right most vertical edge of the display screen 143.
The sidebar 163 is preferably a non-reflective material (such as, black velvet). The sidebar 163 may be of a relatively rigid material that moves statically retaining its shape and size or it may be flexible such that it stretches or accordions to different dimensions as would a curtain. In a preferred embodiment each sidebar 163 is lowered into place from an area above the screen and raised when not needed. In an alternative embodiment, each sidebar 163 is moved in from the side. A combination of the preferred and alternative embodiments may also be used.
Due to the fact that the masks 161 and sidebars 163 do not have to be a rigid and static material, only the leading edge of the mask 161 and/or sidebar 163 would have to move to reveal or obscure an area of the display screen 143. For example, but without limitation, a sidebar 163 that included a velvet curtain could fold and unfold to reveal or obscure an area of the display screen 143. In the preferred embodiment, resizing of the sidebars 163 may be accomplished before, during, or after the sidebars 163 are lowered into place.
Resizing of the display screen 143 may occur in conjunction with bookmarks or watchpoints as defined in the definitions section. The user 190 may, by inserting bookmarks and watchpoints, cause video streams of different aspect ratios to be displayed in rapid succession. In such cases, the digital content metadata 113 associated with subsequent media streams is prefetched and processed appropriately. The user 190 is given some control over the implementation. The user 190 may choose to have the resizing take place just prior to presentation of a new media stream so that all video is viewable (not obstructed by masks 161 and sidebars 163) when the new media stream starts or at some time thereafter. An intelligent mode would compare the aspect ratios of the two media streams and move ahead of the transition of the two media streams only those masks 161 and sidebars 163 that would not interfere with viewing of the current media stream.
For some media presentations, absolute masking may be desirable. The absolute position of each sidebar 163 and mask 161 is specified to the mask controller 160 rather than the aspect ratio of the media stream to be presented. When this is done, overscan must be taken into account. Overscan is the part of the frame (at the edge) that is not displayed by the projector 141 and display screen 143. If the system believes that the top 11% of the frame is black, but the projector 141 clips off the top 3%, then the top mask 161 needs to be brought down less than 11%. The same is true for the sidebars 163. Overscan for the system 100 is calibrated manually using standard video test patterns.
Some mask controllers 160 specify a mask's 161 absolute position in “points” and others in units of time. For example, fully closing the masks 161 may take 10 seconds, or may be specified as 100 points. Fully opening the masks may take 0 seconds (from fully open), or be specified as 0 points. In an embodiment of the invention, the local server 130 can communicate to the mask controller 160 the position of the masks 161 in units that the mask controller 160 can understand natively, such as points, time, or other unit of absolute position. In a calibration step the client device 130 is given two or more absolute locations on the screen 143 (using an on-screen calibration utility), each paired with a value that the mask controller 160 can recognize. For each desired location of a mask 161, the client device 130 can linearly interpolate to generate a value that can be used directly. In a preferred embodiment, three points are calibrated and a quadratic curve is used. Other curves could be used, such as linear, cubic, or exponential. A different curve can be used for top, bottom, and each side mask 161, and potentially for any other scenario, such as when changing video modes of a projector 141.
In
Some digital content 111, such as movies, requires an aspect ratio that is not directly related to digital content 111. This occurs, for example, when a foreign film is offered with subtitles. Filmmakers have historically placed the subtitles over the moving images towards the bottom in an effort to make them less obtrusive but readable. With the popularity of widescreen presentations, it is common for filmmakers to place subtitles in the black bar area below the moving images. This allows for easier reading of the text and the text does not obscure or interfere with the moving picture portion.
A problem associated with placing the subtitles in the black bar portion is that if masks 161 and sidebars 163 are placed at the boundaries of the moving picture portion of the digital content 111, the subtitles that are contained in the black bar portion are projected onto the lower mask 161 and not the display screen 143 making them hard or impossible to read. The invention ensures that digital content metadata 113 for every presentation and permutation thereof is available so that masks 161 and sidebars 163 are positioned correctly to allow for subtitles and other idiosyncrasies (for example, icons, ticker symbols, picture in picture portions and combinations thereof).
In an embodiment of the invention, resizing of the digital image and the viewable area of the display screen 143 may be desirable. For example, filmmakers often letterbox a widescreen presentation of a movie by adding black bars at the top and bottom of the movie essentially reformatting the media stream into another aspect ratio, such as 1.33:1 (also known as standard 4:3 television).
In such a case it could be desirable to expand the visual image such that the portion the user 190 is interested in viewing is as large as possible. It will not matter if the black bars are projected on an area outside of the display screen 143 or on a mask 161 or sidebar 163 as the user does not wish to see them anyway. Digital content metadata 113 for positioning the masks 161 and sidebars 163 for media streams formatted in this manner is available in the database 110. The user can determine prior to viewing any digital content 111 whether they want the media stream optimized in this manner or not.
The system provides an onscreen display (OSD) to assist the user 190 in using the system. Whenever the user 190 executes a function, the OSD is activated to display helpful and needed information to the user 190. Generally, it is important that the OSD not interfere with the video presentation. Since the masks 161 and sidebars 163 will usually be at the edge of the video presentation, it is important for the system to take appropriate action to accommodate any OSD. Positional information for the masks 161 and sidebars 163 is maintained and used to calculate new positions for the masks 161 and sidebars 163, thus they can be temporarily moved to accommodate an OSD when it is important that the video presentation not be obscured in any way.
A debugging mode for setting up the system 100 utilizes the OSD for providing guidance to a technician. The OSD debugging mode displays positioning information for the masks 161 and sidebars 163. In a preferred embodiment, the OSD generates a picture frame at a selected aspect ratio, and the technician manually moves the masks 161 and sidebars 163 into place so as to meet the edges of the picture frame. This creates a calibration for the system including a set of values that is stored and ensures that all subsequent video presentations will have masks 161 and sidebars 163 at the optimum positions for a calibrated aspect ratio.
In an embodiment of the invention, auto-detection of mask placement is accomplished. This includes analyzing the media stream to determine the aspect ratio of the media stream and/or the location of the boundary between the displayed portion of the media stream of interest to the user and the displayed portion of the video stream not of interest to the user or not displayed. If auto-detection is successful, digital content metadata 113 can be updated in the database 110, so that auto-detection will not thereafter be necessary. The system 100 may request input from the user 190 to confirm that auto-detection was accurate or to provide additional manual adjustment of masks 161 and sidebars 163.
In an embodiment of the invention, masking may be accomplished using non-physical masks in the form of “light-masking.” Light-masking can provide nonphysical equivalents of masks 161 and sidebars 163. Preferably, colors are chosen than are the least intrusive to the viewing experience of the user yet provide equal burn-in when displayed in conjunction with a media stream.
Light-masking can be applied in conjunction with physical masking such that burn-in of the display is eliminated and the light-masked area is obscured by the physical masks 161 and sidebars 163. Thus, the user 190 benefits by protecting their display device from burn-in and is not distracted by the light-masking as the physical masks 161 and sidebars 163 are in place.
In alternative embodiments, the picture frame may be replaced with a test pattern or any full-framed image with crisply, contrasted edges.
A method 300 includes a set of flow points and steps. Although described serially, these flow points and steps of the method 300 can be performed by separate elements in conjunction or in parallel, whether asynchronously or synchronously, in a pipelined manner, or otherwise. There is no particular requirement that the flow points or steps must be performed in the same order as described, except where explicitly so indicated.
At a flow point 310, the system 100 is ready to process a request from a user 190.
At a step 311, the user 190 makes a selection from available digital content 111 contained on the local servers 130. The user 190 preferably makes their selection by viewing a list on the display screen 143. Alternatively, the user 190 may make their selection at a local player 140 or local server 130.
At a step 313, the local server 130 sends a request 191 to query the database 110 for the digital content metadata 113 associated with the digital content 11 selected by the user 190. In an embodiment of the invention, the user 190 pays a fee for the information retrieved from the database 110. The local server 130 provides identification of the user 190, and the database 10 maintains a transaction history for the user 190 so they can be billed at regular intervals. DVD media are identified by their DVD hash values.
At a step 315, the database 110 locates the digital media metadata 113 associated with the digital content 111 and sends a response 193 back to the local servers 130 that includes the digital media metadata 113.
At a step 317, the local server 130 processes the response 193 by passing identified information to the controller 150. This may include parsing the response 193 into sub-messages for individual processing by elements of the system 100. For example, but without limitation, parsing may extract data to be used by the mask controller 160 and data to be used by the controller 150 to instruct the other devices 170.
At a step 319, the controller identifies the portions of the response 193 that relate to the mask controller 160.
At a step 321, the mask controller 160 interprets the information it has received and issues commands to each mask 161 and sidebar 163 to move to a designated position per the response 193. These actions would preferably take place prior to the start of the presentation of the digital content 111 and would be updated during the presentation of the digital content 111 as designated in the response 193. Some directors have been known to start a movie in one aspect ratio and then move to another.
Currently, about six popular display ratios exist; however, it is within the scope and spirit of the invention to provide for yet unknown display ratios. As described herein, digital content metadata 113 provides the parameter values applied to mask 161 and sidebar 163 placements. Practically any vertical to horizontal ratio display screen 143 can be generated with the associated digital content metadata 113.
At a step 323, the controller 150 directs the other devices 170 consistent with the response 193. For example, but without limitation, the response could include the following instructions; 1) Dim house lights to 5%, 2) Set cooling fans to low/quiet speed, 3) Set surround system to 6.1 compliant, and 4) set DPS mode for audio system to rock concert preset. Some users 190 would have the other devices 170 and some would not. For those users 190 who do not have other devices 170 to control, any control messages for such would be ignored.
At a step 325, playback of the digital content 111 commences.
At a flow point 327, the system 100 has processed at least one request.
Although preferred embodiments are disclosed herein, many variations are possible which remain within the concept and scope of the invention. These variations would become clear to those skilled in the art after perusal of this application.
Those skilled in the art will recognize, after perusal of this application, that these alternative embodiments are illustrative and in no way limiting.
This application incorporates by reference and claims priority of U.S. patent application Ser. No. 10/655,496, filed Sep. 3, 2003, which in turn is based on and claims priority of U.S. Provisional Patent Application 60/488,367, filed Jul. 15, 2003, the contents being incorporated herein by reference. The cited provisional application incorporates a technical appendix that includes “Automatic Masking.” (Paul Rechsteiner, Nik Gervae, Shawn Neely, Michael Malcolm, Ray DePaul, Daniel Collens).
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/US04/22847 | 7/14/2004 | WO | 1/17/2006 |
Number | Date | Country | |
---|---|---|---|
60488367 | Jul 2003 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 10655496 | Sep 2003 | US |
Child | 10564811 | Jan 2006 | US |