Patent Application
20180152630
The present invention relates generally to video capturing with portable video capturing devices like smartphones, tablets, digital cameras or digital camcorders.
Today, a lot of portable devices allow capturing video frames according to different orientations. This is notably the case of the smartphones, the tablets, the digital cameras and the digital camcorders. With such devices, the user can choose capturing video frames according to a landscape mode or a portrait mode by rotating appropriately the video capturing device. If the user holds his video capturing device according to a reference orientation, also called horizontal orientation, the video frames are captured in a landscape mode. And if the user holds his video capturing device according to an orientation perpendicular to the reference orientation, also called vertical orientation, the video frames are captured in a portrait mode.
The video frames are displayed on a display screen of the video capturing device as they are captured. When the user holds the video capturing device in a vertical orientation during the capture (video frames captured in a portrait mode), the video frames are displayed in a portrait mode on the display screen of the video capturing device and are therefore fully in line with the user's expectation.
It makes that the user does not become aware or does not realize that he is capturing video frames in a portrait mode. This can be problematic for post-processing applications, like automatic editing software used for editing video clips from a plurality of video frames captured by one or more video capturing devices. If the video frames do not include metadata on the orientation of the device when capturing said video frames, it can lead to video clips including video frames displayed in a landscape mode although they have been captured in a portrait mode.
In addition, all the big video screens that we use in the every day life, like computer screens, TV screens and movie screens, are conceived for displaying video frames in landscape mode. And you can not turn them for viewing video frames captured in a portrait mode.
The same problem arises when the user holds his video capturing device in an upside down position. The user is not aware that he is capturing video frames in an upside down landscape mode since the video capturing device automatically rotates the captured images 180 degrees before displayed them on its screen.
Some solutions exist for addressing this problem. Some applications such as “Horizon Capture App” prevent the user to capture vertical video clip by recording only a part of the image captured by the device, this part being indicated by a horizontal mask layered with the video really captured by the device. Such a method has two major drawbacks. First, the stored video does not benefit of the device sensor full resolution because only a part of the captured pixels are present in the final horizontal video. Second, because the user has to be focused on a small part of his device, the resulting video clip is more unsteady than video clips captured when the user can focus on the full screen.
Besides, US 2005/0212931 A1 describes a method of detecting and correcting camera rotation in a digital camera. However, the rotation angle of the captured digital image is determined with respect to a set of lines extracted from the image. The method further discloses displaying the rotation angle or modifying captured digital image based on the rotation angle, before storing the modified digital image. However if the set of lines is representative of the horizon in the captured image for exemple, such method fails to inform if one is landcape upside down mode or portrait mode with respect to the orientation of the capturing device.
An object of the invention is to propose a method for avoiding at least partially the above-mentioned drawbacks.
According to the invention, it is proposed to display in real time relevant information to the user capturing some video on whether he is holding his device in vertical position or an upside-down position. The immediate effect is to warn the user that he is capturing video frames which could be difficult to reuse later on within an edited video sequence and which are not adapted to the format of the most widespread screens.
The invention concerns a method for displaying at least one video frame on a display screen coupled to a portable video capturing device, said method comprising displaying, on said display screen, at least one video frame and a visual indicator indicating that the orientation of the portable video capturing device in case where the orientation of said portable video capturing device is portrait or upside down landscape with respect to a reference orientation of said portable video capturing device and were said portable video capturing device is capturing at least one video frame.
Thus the user of the portable video capturing device is warned that he is capturing video frames in a portrait mode or an upside down landscape mode which are not adapted to the format of the most widespread screens and/or which can be difficult to use for video editing. The right portrait orientation, or upside down landcape orientation or left portrait orientation are respectivly determined for an angle of the portable device being substantially equal to 90 or 180 or 270 degrees with respect to the reference landscape orientation of said portable video capturing device.
Then when viewing the visual indication the user may decide or not to rotate his video capturing device in order to put it back in the reference orientation (horizontal position).
With this method, the user is aware that he is capturing video frames in a portrait mode or an upside down landscape mode. Another advantage is that such a warning does not prevent the user to capture vertical video clips in these modes if this is really his intention.
According to a particular embodiment, the orientation of the portable video capturing device is determined by a sensor embedded in the portable video capturing device.
According to a particular embodiment, the sensor is a gyroscope.
According to a particular embodiment, the orientation of the portable video capturing device is determined by analyzing the video content of said at least one video frame.
According to a particular embodiment, the visual indication is a symbol displayed on top of the at least one video frame displayed on the display screen.
According to a particular embodiment, the visual indication is an alteration of the at least one video frame displayed on the display screen. For example a shaded version of the video frame is displayed on the display screen.
The invention concerns also a portable video capturing device comprising
wherein the control unit is configured to:
In a particular embodiment, the portable video capturing device further comprises an orientation sensor for determining the orientation of the portable video capturing device, said orientation being provided to the control unit for determining if the orientation of the portable video capturing device is substantially equal to 90 or 180 or 270 degrees.
In a particular embodiment, the orientation sensor is a gyroscope.
In another embodiment, the control unit comprises a video processing unit for analyzing the video content of the captured video frame and determining the orientation of the portable video capturing device.
The invention can be better understood with reference to the following description and drawings, given by way of example and not limiting the scope of protection, and in which:
While example embodiments are capable of various modifications and alternative forms, embodiments thereof are shown by way of example in the drawings and will herein be described in details. It should be understood, however, that there is no intent to limit example embodiments to the particular forms disclosed, but on the contrary, example embodiments are to cover all modifications, equivalents, and alternatives falling within the scope of the claims. Like numbers refer to like elements throughout the description of the figures.
Before discussing example embodiments in more details, it is noted that some example embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations as sequential processes, many of the operations may be performed in parallel, concurrently or simultaneously. In addition, the order of operations may be re-arranged. The processes may be terminated when their operations are completed, but may also have additional steps not included in the figures. The processes may correspond to methods, functions, procedures, subroutines, subprograms, etc.
Methods discussed below, some of which are illustrated by the flow charts, may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware or microcode, the program code or code segments to perform the necessary tasks may be stored in a machine or computer readable medium such as a storage medium. A processor(s) may perform the necessary tasks. Specific structural and functional details disclosed herein are merely representative for purposes of describing example embodiments of the present invention. This invention may, however, be embodied in many alternate forms and should not be construed as limited to only the embodiments set forth herein.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises”, “comprising,” “includes” and/or “including”, when used herein, specify the presence of stated features, integers, steps, operations, elements and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and/or groups thereof. Similarly, it is to be noticed that the term “coupled” should not be interpreted as being restricted to direct connections only. Thus, the scope of the expression “a device A coupled to a device B” should not be limited to devices or systems wherein an output of device A is directly connected to an input of device B. It means that there exists a path between an output of device A and an input of device B which may be a path including other devices or means.
In addition, the term “orientation” of a portable device is related to the orientation of the image capturing circuit (CCD or CMOS sensor) of the portable device towards the capture scene. Thus, the reference orientation is determined with respect to the portable device position which corresponds to a landcape orientation of the device. If the video frames are captured in a landscape mode, the portable video capturing device is in a reference position (or horizontal position). If the video frames are captured in a portrait mode, the orientation of the portable video capturing device is substantially 90 or 270 degrees from the reference position. If the video frames are captured in an upside-down landscape mode, the orientation of the portable video capturing device is substantially 180 degrees from the reference position.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which example embodiments belong. It will be further understood that terms, e.g., those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
In a step S0, an orientation value and a capturing status, ON or OFF, of the video capturing device are provided. The orientation value can be outputted by a dedicated orientation sensor, for example a gyroscope, embedded in the portable video capturing device. As a variant, the orientation value can be computed by analyzing the video content of the video frames captured by the video capturing device. Such a variant can be implemented by various methods, for example the method disclosed in the publication “Detecting image orientation based on low-level visual content”, Yongmei Michelle Wand and Hongjiang Zhang, Computer Vision and Image Understanding 93 (2004), pp 328-346, available at www.ElsevierComputerScience.com. In this paper, the image orientation can be interpreted as the device orientation since the orientation of the image sensor of the video capturing device is identical to the orientation of the video capturing device.
If the video frames are captured in a landscape mode by the portable video capturing device, the orientation value is about 0 degree. If the video frames are captured in a portrait mode, the orientation value is about 90 or 270 degrees. And if the video frames are captured in an upside-down landscape mode, the orientation value is about 180 degrees.
The capturing status, ON or OFF, can be detected by various ways. This capturing may be for example read from the frame memory of the video capturing device using a dedicated Access Protocol Interface (API). If video frames are currently stored in this memory, it means that the video capturing device is capturing video frames. The capturing status is ON. If video frames are not currently stored in the memory, the capturing status is OFF.
In a step S1, it is determined if the orientation value delivered by the orientation sensor or computed by content analysis of the captured video frames is substantially equal to 90°, 180° or 270° and if the capturing status of the portable video capturing device is ON. If these two conditions are fulfilled, a visual indication on the device orientation is displayed on a screen coupled to the video capturing device at a step S2. This screen may be the screen of the video capturing device or a screen connected to the video capturing device.
Thus, the user is informed that he is capturing video frames in a portrait mode or an upside-down landscape mode which are not adapted to the format of the most widespread screens and which can be difficult to use for video editing.
Then when viewing the visual indication the user may decide or not to rotate his video capturing device in order to put it back in the reference orientation (horizontal position).
The visual indication may be a symbol displayed on top of video frames displayed on the display screen. This embodiment is illustrated by
This symbol may eventually blink to draw the attention of the user.
Of course, the size and the location of the visual indication may vary.
This visual indication may also differ somewhat depending on the device orientation is 90°, 180° or 270°. The visual indication is a V for the device orientation of 90° and 270° and a U (for upside-down) for the device orientation of 180°.
In another embodiment, the visual indication is an alteration of the video frames displayed on the display screen. For example a shaded version of the video frames is displayed on the display screen.
The displaying of this visual information may be an option proposed by the menu of the video capturing device. It may be disabled in case the user does not want to get warnings of his capturing in a portrait mode or in an upside-down landscape mode.
All these elements are connected together via a bus 160 such that the processor (or control unit) 120 can control them.
In a variant, the determination of the orientation of the portable video capturing device is made by analyzing the video content of the captured video frames. This analysis may be implemented by the processor 120 or other dedicated processors.
According to exemplary and non-limitative embodiments, the portable video capturing device 100 is a device that belongs to a set comprising:
The implementations described herein may be implemented in, for example, a method or a process, an apparatus, a software program, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (for example, discussed only as a method or a device), the implementation of features discussed may also be implemented in other forms (for example a program). An apparatus may be implemented in, for example, appropriate hardware, software, and firmware. The methods may be implemented in, for example, an apparatus such as, for example, a processor, which refers to processing devices in general, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as, for example, computers, cell phones, portable/personal digital assistants (“PDAs”), and other devices that facilitate communication of information between end-users.
| Number | Date | Country | Kind |
|---|---|---|---|
| 15305068.7 | Jan 2015 | FR | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/EP2015/080456 | 12/18/2015 | WO | 00 |
