The present invention relates to chroma key apparatus and to a method and system using that apparatus.
Chroma key compositing or chroma keying is a post-production technique for combining images or video content. A predetermined colour hue or chroma range is used as a chroma key in an image or video frame to determine where another image or video layer is to be inserted in a compositing process. In a studio set, the chroma key may be created using a background screen of the predetermined hue, such as green or blue, hence this technique is often referred to as ‘green screen’ or ‘blue screen’.
In television studios, chroma keying is often used to add video or computer-generated graphics to video shot in a studio set with a chroma key screen. This technique is particularly suitable for live broadcasting because the compositing can be done in real time using suitable post-production software.
One recent application of chroma keying to live broadcasting is in the coverage of football (soccer) by Sky Sports®.
Due to the spatial constraints of the stadium, the available studio space is relatively small (typically 5 m×5 m), but it is desirable that the broadcast video should appear to be from a wider studio set, of the type that viewers would normally expect in studio broadcasts. To that end, chroma key screens SL, SR are positioned respectively on the left and right sides of the studio set. Another chroma key screen SB may be positioned in an area of the back of the studio set not occupied by the window W. Computer-generated images are then substituted for the chroma key areas during compositing. An example of a composited image is shown in
Details of the above prior art arrangement can be seen for example on the SVG Europe website in the news items ‘And it's live! Sky Sports to bolster Premier League coverage with more AR’ dated 29 Oct. 2018 and available on 31 Jan. 2020 at https://www.svgeurope.org/blog/headlines/and-its-live-sky-sports-to-bolster-premier-league-coverage-with-more-ar/
However, specular reflections R of the side chroma key screens SL, SR in the window W may be visible in the composited video, thus destroying the illusion of the AR studio set. These reflections R may be particularly visible when daylight falls through the window W onto the side chroma key screens SL, SR, or when the view through the window is dark so that the contrast of the reflections R is more noticeable. It may not be practicable to remove these reflections R in post-production, because the hue of the reflections R may be similar to that of parts of the view through the window W.
Another problem is that the diffuse reflected light from the chroma key screens may give an unnatural hue to any person P or object O close to the chroma key screens or may even lead to accidental chroma keying of parts of the people or objects, which constrains the positioning of people or objects within the studio set, particularly where the available studio space is narrow. In a conventional studio, people or objects should normally be positioned at least 1.5 m from a chroma key screen.
The above problems are not specific to live studio broadcasting as described above but may occur in other situations involving chroma key screen reflections.
According to one aspect of the present invention, there is provided a chroma key screen for arrangement along a side of a studio set, the screen comprising a plurality of substantially vertical, mutually spaced surfaces, such as louvres or slats, each angled towards the front of the studio set and therefore towards a studio camera.
At least the part of each of the surfaces that is visible to the camera is within a predetermined suitable chroma key range. An advantage of this arrangement is that the screen appears in frames shot by the studio camera(s) as a substantially continuous chroma key, which allows compositing effects such as an AR studio set to be added. Reflection of the screen from objects or people within the studio set is reduced, because the louvres are directed away from those people or objects.
Preferably, at least one of the louvres has a portion adjacent the leading edge (e.g. the edge of the louvre closest to the centre of the studio set) angled or curved slightly away from the front of the studio set. This reduces edge effects in the chroma key as seen from the camera(s), caused for example by a change of brightness at the edge of the louvres.
Preferably, the lower end of at least one of the louvres is curved or angled towards the front of the studio set, so as to reduce edge effects of the lower end. The lower end is advantageously brought closer to the louvre in front, which may then obscure the lower end.
Preferably, the width of the louvres is selected so that the screen appears to be a substantially continuous chroma key from a range of different camera positions across the front of the studio set. The width and spacing of the louvres is preferably set so that the studio lighting does not cast shadows on the visible parts of the front surfaces of the louvres. A width of at least 20 cm is preferred.
Preferably, at least one of the louvres has a lower part wider than an upper part. This may further assist in obscuring a lower edge of the louvre from the camera(s).
There may be provided means for illuminating the front faces of the louvres, preferably to avoid shadowing such as from an adjacent one of the louvres, thereby giving the chroma key a more even brightness. The means for illuminating may comprise one or more light sources, such as a light-emitting diode (LED) strip, or one or more reflectors. The one or more light sources may be mounted on or connected to the frame. Alternatively, the front faces of the louvres may be light-emitting.
The angles of the louvres may be set according to the configuration of the studio set and/or the position of the camera. Preferably, at least some of the louvres are pivotally mounted about a substantially vertical axis so that their angle may be set or adjusted. The louvres may be mounted on the frame carrying substantially vertical axles or poles about which the louvres are pivotable.
There may be provided a pair of said chroma key screens arranged on either side of the studio set. This allows compositing effects to be added to both sides of the studio set. The pair may be connected together, for example by a cross-piece extending between the chroma key screens. The cross-piece may support a further chroma key screen arranged along the back of the studio set.
A specular reflective surface such as a window may be provided at the rear of the studio. The window may allow light from outside the studio to fall onto the sides of the studio set. Preferably, a back or reverse face of at least one of the louvres is black or of a dark colour, at least for the louvre or louvres closest to the back of the studio set. Advantageously this may reduce or minimise reflection of the reverse face in the reflective surface.
According to another aspect of the present invention, there is provided a chroma key system comprising a studio set as defined above, at least one camera arranged to capture an image or video from the studio set, and a compositing system arranged to replace a chroma key part or parts of the image or video with at least one alternative image, graphic or video so as to generate a composited image or video. The system may include means for broadcasting and/or storing the composited image or video. The broadcasting is preferably live broadcasting.
According to another aspect of the invention, there is provided a method comprising capturing an image or video from the studio set, and replacing a chroma keyed part or parts of the image or video with at least one alternative image, graphic or video so as to generate a composited image or video. The composited image or video may then be stored and/or broadcasted. The broadcast is preferably live.
According to another aspect of the invention, there is provided an image or video signal or data captured from the studio set and including the chroma key.
Advantageously, this signal may include video frames showing the chroma key surfaces as substantially even and continuous chroma key areas, while exhibiting reduced chroma key reflection in other parts of the frame, and therefore embodying technical features of the invention.
There now follows, by way of example only, a detailed description of embodiments of the present invention, with reference to the figures identified below.
In this embodiment there is provided, on each of the left and right hand sides of the studio set, respective left and right chroma key screens SL, SR each comprising a plurality of vertical slats or louvres L, each angled in a direction D towards the front of the studio set i.e. towards the camera(s). The louvres L are approximately rectangular in shape and elongate in the vertical direction.
The facing direction F (i.e. a direction perpendicular to the front surface of at least the upper section Lu) of the louvre L is in the same general direction as direction D, as a result of the louvre L being angled relative to the horizontal axis H. The louvres L of the right screen SR are a mirror image of the louvres L of the left screen SL, about a vertical plane containing the horizontal axis H.
The front faces of the louvres L are within a chroma key range. The range may be green (e.g. with a hue in the range between 100 and 150 and most preferably approximately 120 in ICC colour profile sRGB IEC61966-2.1) or blue (e.g. with a hue in the range between 210 to 250 and most preferably between 225 and 240 in ICC colour profile sRGB IEC61966-2.1). This may be achieved using material or surface coating (e.g. paint) of the required hue. Hence, to the camera the louvres L appear as a substantially continuous chroma key area. The back or reverse faces of the louvres L are preferably black or of a dark colour so as to reduce reflections in the window W. This is preferable for the louvres L closest to the window W, but may not be necessary for louvres L positioned away from the window W, where at least part of the back or reverse faces are obscured by other, closer louvres L relative to the window W.
The louvres L are preferably made of sheet material such as plastic, metal or plywood, or fabric or material on frame, so that they can be easily fabricated and/or transported. In one embodiment, the front faces of the louvres L are of a flexible material of the required colour, attached to the frame of the louvre L. Alternative shapes may be used for the louvres: for example, the louvres may be triangular prisms.
As shown in
The dimensions and mutual spacing of the louvres L1-L7 may depend on the dimensions of the studio set for which they are designed. At least some of the louvres L1-L7 may be between 1.5 and 3 m high, corresponding substantially to the height of the studio set, although the height of some of the louvres L may be reduced to accommodate an object positioned on the side of the studio set. In one specific example, at least some of the louvres L1-L7 have a height h of 180 cm.
The width w of the louvres L1-L7 is preferably at least 20 cm. In one specific embodiment, the width w of the upper sections Lu is 45 cm. The width w may vary between different louvres L: in one example the louvre L7 closest to the camera 10 is narrower, with a width w of 28.5 cm.
A monitor 8 may be provided within the screen SL, SR on either side, and one or more of the louvres L may be dimensioned (e.g. their height reduced) to accommodate the monitor 8. For example, as shown in
The louvres L1-L7 within the same screen SL, SR are mutually spaced apart along the axis H with a spacing or pitch P1-P6 that may vary between different adjacent ones of the louvres L1-L7. The spacing is preferably in the range 20-80 cm, or more preferably within the range 30-60 cm. In one example shown in
As shown in
The louvres L are preferably pivotally mounted about a substantially vertical axis in the respective frame 3, for example by attachment to a respective vertical rod or axle 5 or, in the case of the louvre L closest to the back of the studio set, to a vertical part of the frame 3 itself. The attachment may be by one or more spring clips, hinges or other means allowing rotation around the vertical rod 5, or the rod 5 itself may be rotatably mounted in the frame 3. The louvres L may be pivoted to the optimum horizontal angle α during setup of the studio set, and may be pivoted flat, or removed from the frame 3 altogether, for storage or transport.
The frames 3 on both sides may be joined together by a cross-piece 4 which supports the back screen SB, thus providing a complete chroma key screen apparatus for a studio set, that is self-supporting when assembled and may be easily disassembled for transport. The dimensions of the cross-piece 4 depend on the size of the studio set: in one example, the cross-piece 4 is approximately 400 cm wide, as measured between the frames 3 on either side. The width of the cross-piece 4 may be adjustable and/or different cross-pieces 4 of different widths may be used depending on the desired width of the studio set.
The front face of each louvre L may be illuminated by a respective light source 9, such as an LED strip, and/or by a light reflector, so that the brightness of the front face is approximately even, to assist chroma keying. The respective light sources 9 and/or reflectors for the louvres L may be mounted on the corresponding frame 3. In one example, the light source for each louvre L is mounted on the vertical rod 5 of the louvre L directly in front and extends vertically over the height of the louvre L that it illuminates. A diffuser may be provided to diffuse the light emitted from the light source 9 so as to give more even illumination.
In an alternative embodiment, the front faces of the louvres L may be light-emitting. For example, they may be formed of flexible LED material such as the UNIT flexible LED display, available from Shenzhen Unit LED Co., Ltd and as disclosed for example on the following web page as of 4 Feb. 2020:
https://www.unit-led.com/product/unit-soft-flexible-series-led-display
Preferably, the colour and/or brightness of the light-emitting front faces of the louvres L may be adjusted or set to achieve a uniform chroma key.
As shown in
As shown in
As shown in
As shown in
As shown in
At step S1, the chroma key apparatus, including the screens SL, SR, is set up in the studio as part of the studio set. This step may include setting the louvres L to the optimum angle for the studio set and the position of the camera(s). Where there are multiple cameras 10 at different positions, the louvres L may be adjusted so that the screens SL, SR appear as a substantially continuous chroma key area to each of the cameras.
As part of step S1, the lighting may be adjusted, including the light sources 9 illuminating the louvres L, to achieve optimum chroma keying and to minimise chroma key reflection. The lighting may be adjusted during shooting, for example as the light falling through the window W changes depending on the time of day, or the cameras 10 may be adjusted to compensate for the change in lighting.
At step S2, video is shot from the studio set using one or more cameras 10. In one example, there are three cameras 10 positioned left, centre and right in front of the studio set. The camera(s) 10 are preferably digital studio cameras that generate video signals Sig1, Sig2, Sig3 in a suitable format, such as HEVC/H.265, comprising a plurality of frames that may be inter- and/or intra-frame coded.
The video signal(s) or stream(s) Sig1, Sig2, Sig3 from the camera(s) 10 are input to a computer-implemented post-production system 11, comprising one or computer workstations or processors, which performs compositing at step S3 to replace chroma key areas in frames of the video signals with additional video and/or graphics, for example to create an AR studio set effect. The additional video or graphics may be provided by a secondary source 12 or may be generated within the post-production system 11. The resultant composited video is output to a content store 13 at step S4 and/or to a content delivery network 14 at step S5, from which the composited video may be broadcast or streamed. The broadcast or streaming may be live e.g. provided directly to the viewer.
The compositing may be performed with conventional post-production software and/or hardware, since the screens SL, SR appear in the input video frames as substantially continuous areas of substantially even brightness within the predetermined chroma range. The need to clean up the video frames to remove edges within the chroma key areas is reduced or eliminated, thus enabling real-time compositing suitable for live broadcasts.
A matte may be created to prevent chroma keying of a specified part of the image, such as the window W. The matte may track the position of the specified part as it moves within the image.
During compositing, the position of the monitor 8 may be determined and the monitor 8 may be removed from the image. Preferably a virtual monitor is shown in the AR studio set along the same eye line as the monitor 8 so that the presenter or commentator appears to be looking at the virtual monitor.
Alternative embodiments may be envisaged on reading the present application, which nevertheless fall within the scope of the following claims. For example, although the chroma key surfaces are shown in specific embodiments as being mounted on or forming part of discrete louvres or slats, other arrangements may be envisaged which nevertheless provide the required arrangement of chroma key surfaces. For example, the chroma key surfaces may form part of a continuous folded structure comprising the chroma key screen.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/GB2020/050276 | 2/6/2020 | WO |