Patent Application
20180048827
This application claims priority from French Patent Application No. 16 57727 filed on Aug. 11, 2016. The content of this application is incorporated herein by reference in its entirety.
The present invention relates to a method for capturing a video using a camera on board a drone with fixed wings, the camera comprising an image sensor, the drone having, during flight, a drift angle between the longitudinal axis of the drone and a flight direction of the drone.
In particular, the fixed-wing drone is of the “sailwing” type. Hereinafter, a “drone” refers to an aircraft with no pilot on board. A drone is autonomous, piloted by automatic pilot in autonomous flight, or piloted remotely assisted by the automatic pilot, in particular using a control stick. The invention also relates to a computer program including software instructions which, when executed by a computer, carry out such a method for capturing a video.
The invention also relates to an electronic system for capturing a video comprising a fixed-wing drone, the camera comprising an image sensor on board the drone, the drone having, during flight, a drift angle between the longitudinal axis of the drone and a flight direction of the drone.
It is possible to provide a drone with equipment for performing specific tasks. It is in particular possible to equip the drone with a camera having an image sensor, to take videos.
Traditionally, the direction in which the camera is pointed corresponds to the longitudinal axis of the drone. In other words, in the absence of wind, the direction targeted by the camera on board the drone corresponds to the direction of flight.
Under the effect of the crosswind, a lateral force is applied on the body of the drone (fuselage). To compensate this lateral force due to the wind and prevent the drone from moving away from its route, the user, or the automatic pilot in autonomous flight, must cause a drift angle between the longitudinal axis of the drone and its direction of flight. The viewing direction of the camera is then shifted by the drift angle relative to the direction of flight of the drone.
In another case, when a gust of wind occurs, the user or the automatic pilot does not have time to compensate the lateral force due to the gust, and the longitudinal axis of the drone and its direction of flight are shifted by a drift angle. The viewing direction of the camera is then shifted relative to the direction of flight of the drone.
Thus, when the fixed-wing drone has drifted under the effect of the wind or the compensation for the wind, the viewing axis of the camera is no longer situated in the axis of the actual direction of flight of the drone.
Such a drift between the longitudinal axis of the drone and its direction of flight deteriorates the piloting capacities of the user, who, in the “immersive piloting” configuration, cannot determine the trajectory of flight of the drone he is piloting, and if applicable correct its piloting to reach the targeted destination.
Such a lack of knowledge of the trajectory of the drone in the presence of wind is also problematic due to the fact that obstacles are sometimes present on this actual flight trajectory of the drone.
As a result, when the fixed-wing drone is outside the visual range of the user, or when the user essentially bases his piloting on the real-time playback of images filmed by the camera, such obstacles will not be able to be avoided, which will result in an interruption of the flight and filming, or even a deterioration of the fixed-wing drone in case of impact with the obstacle or crash.
One aim of the invention is then to propose a method for capturing a video using a camera on board a fixed-wing drone, the camera comprising an image sensor allowing the user to optimize his piloting in the presence of wind.
To that end, the invention relates to a method for capturing a video using a camera on board a drone with fixed wings, the camera comprising an image sensor, the drone having, during flight, a drift angle between the longitudinal axis of the drone and a flight direction of the drone, the method comprising:
determining the drift angle of the drone; and
obtaining video by image acquisition corresponding to a zone with reduced dimensions relative to those of the image sensor, the position of the zone being determined as a function of the drift angle of the drone.
The method for capturing a video according to the invention makes it possible to keep the axis of the video obtained using the camera on board a fixed-wing drone in the direction of flight, and not in the direction of the longitudinal axis of the fixed-wing drone.
In other words, the method according to the invention allows an instantaneous virtual adjustment of the direction targeted by the camera on board the fixed-wing drone based on the intensity of the crosswind.
Furthermore, the user directly perceives the effect of the crosswind on the actual trajectory of the drone through the image he is viewing and, if necessary, corrects his piloting so as to compensate for the effect of drift caused by the wind in order to reach the targeted flight destination and avoid obstacles on this actual trajectory.
According to other advantageous aspects of the invention, the video capture method comprises one or more of the following features, considered alone or according to all technically possible combinations:
the camera comprises a fisheye lens associated with the image sensor,
the zone, the position of which is determined as a function of the drift angle, is configured to capture a region of the scene filmed by the image sensor on board the drone along a viewing direction parallel to the direction of flight of the drone,
the zone, the position of which is determined as a function of the drift angle (α), and the zone obtained in the absence of drift of the drone have a non-zero intersection,
the position of the zone is calculated such that the window corresponding to the overall field of the camera rotates, relative to that without drift, by a drift angle α of the drone around the yaw axis of the drone,
the determination of the drift angle comprises:
obtaining video comprises acquiring image data from the entire surface area of the image sensor, followed by digital processing of the image data delivering video images corresponding solely to the zone,
the acquisition of image data only in the zone during the production of the video, the position of the zone being determined during the production of the video,
when the zone, the position of which is determined as a function of the drift angle, abuts with the image sensor, the image corresponding to the zone comprises a black area,
when the zone, the position of which is determined as a function of the drift angle, abuts with the image sensor, the position is reset such that the center of the zone is on the longitudinal axis of the fixed-wing drone.
The invention also relates to a computer program comprising software set points which, when executed by a computer, carry out a method as defined above.
The invention also relates to an electronic system for capturing a video comprising a fixed-wing drone and a camera on board the fixed-wing drone, the camera comprising an image sensor, the drone having, during flight, a drift angle between the longitudinal axis of the drone and a flight direction of the drone, the electronic system for capturing a video further comprising:
a determining module configured to determine the drift angle of the drone; and
an obtaining module configured to obtain the video by acquiring at least one image corresponding to a zone with reduced dimensions relative to those of the image sensor, the obtaining module being configured to determine the position of the zone as a function of the drift angle of the drone.
These features and advantages of the invention will appear more clearly upon reading the following description, provided solely as a non-limiting example, and done in reference to the appended drawings, in which:
In
The fixed-wing drone 14 comprises a drone body (fuselage) 26 provided in the rear part with a propeller 24 and on the sides with two wings 22, these wings extending the drone body 26 in the illustrated configuration of the “sailwing” type. On the side of the trailing edge, the wings 22 are provided with control surfaces 18 able to be oriented using servo mechanisms to control the path of the drone.
The drone 14 is also provided with a camera fastened on the fixed-wing drone 14 (i.e., no movement of the camera is possible). The camera having an image sensor 28 configured to acquire at least one image of a scene, and a transmission module, not shown, configured to send the image(s) acquired by the image sensor 28, preferably wirelessly, to a piece of electronic equipment, such as the reception module, not shown, of the electronic viewing system 10, the reception module, not shown, of the control stick 16, or the reception module of the multimedia touchscreen digital tablet 70 mounted on the control stick 16, not shown.
The image sensor 28 is for example associated with a hemispherical lens of the fisheye type, i.e., covering a viewing field with a wide angle, of about 180° or more. The projection 300 of the image obtained by the fisheye lens associated with the image sensor 28 is shown in
In the example of
The fixed-wing drone 14 also comprises a determining module 56 configured to determine the drift angle α of the drone. The drift angle α corresponds to the angle formed, during flight and in the presence of cross wind 44, between a longitudinal axis 42 of the drone and a direction of flight 40 of the drone, as shown in
In the example of
The determination module 56 for determining the drift angle α of the drone also comprises a module 60 for determining the direction of flight 40 of the fixed wing drone 14, for example a geolocation module receiving, in real time from a constellation of geolocation satellites 30, the information allowing the determination module 60 to determine the instantaneous geographical position of the fixed-wing drone 14 and to calculate the direction of flight 40 of the fixed-wing drone 14. The direction of flight 40 of the fixed wing drone 14 is expressed by the angle φ between geographical north and the geographical position of the fixed-wing drone 14, as shown in
From the angle β between the longitudinal axis 42 of the fixed-wing drone 14 and magnetic north, delivered by the magnetometer 58, the magnetic breakdown θ corresponding to the angle formed between the direction of the geographical north pole and magnetic north, as shown in
For example, according to the illustration of
The fixed-wing drone 14 also comprises an obtaining module 62 configured to obtain the video by acquiring at least one image using the image sensor 28, the image corresponding to a zone Zc of the image sensor 28, with reduced dimensions relative to those of the image sensor, the position of the zone Zc being determined as a function of the drift angle α of the fixed-wing drone 14.
The image sensor 28 is the photosensitive member of the camera. It is for example a CMOS sensor. The zone Zc is a fraction of the image sensor 28.
The image sensor 28 associated with the lens is able to provide an overall image corresponding to an overall field 200 of the camera as shown in
Obtaining an image from a zone Zc with smaller dimensions of the image sensor makes it possible to virtually orient the viewing axis of the camera in the direction of the window of the overall field of the camera corresponding to the zone Zc with smaller dimensions, without modifying the physical orientation of the camera, which remains immobile relative to the fixed-wing drone 14.
According to a first alternative, the obtaining module 62 comprises a module for acquiring image data dl, not shown, configured to acquire image data from the entire surface area of the image sensor, and a digital processing module for the image data, not shown, configured to deliver video images corresponding only to the zone Zc, the position of the zone Zc being determined as a function of the drift angle α of the fixed-wing drone 14. The output of the acquisition module is connected to the input of the digital processing module.
According to a second alternative, the obtaining module 62 comprises only a module for acquiring image data, not shown, configured to acquire image data only in the zone Zc during the video production, the position of the zone Zc being determined as a function of the drift angle α of the fixed-wing drone 14.
In the example of
The processor 54 of the information processing unit 50 is then able to execute the determining software and the obtaining software using a computer program.
Alternatively, the module 56 for determining the drift angle α of the fixed-wing drone 14 and the obtaining module 62 configured to obtain the video are each made in the form of a programmable logic component, such as an FPGA (Field Programmable Gate Array), or in the form of a dedicated integrated circuit, such as an ASIC (Application Specific Integrated Circuit) mounted on an electronic board 64 on board the fixed-wing drone 14.
An electronic viewing system 10 allows the user 12 to view images, in particular images of the video received from the fixed-wing drone 14.
The electronic viewing system 10 comprises an electronic device, for example a smartphone, provided with a display screen, and a headset 20 including a reception support of the electronic device, a bearing surface against the face of the user 12, across from the user's eyes, and two optical devices positioned between the reception support and the bearing surface.
The headset 20 further includes a maintaining strap 32 making it possible to maintain the headset 20 on the head of the user 12.
The electronic device is removable with respect to the headset 20 or integrated into the headset 20.
The electronic viewing system 10 is for example connected to a control stick 16 via a data link, not shown, the data link being a wireless link or a wired link.
In the example of
The viewing system 10 is for example a virtual-reality viewing system, i.e., a system allowing the user 12 to view an image in his field of view, with a field of view (or field of vision, FOV) angle with a large value, typically greater than 90°, preferably greater than or equal to 100°, in order to procure an immersive view (also called “FPV”, First Person View) for the user 12.
The control stick 16 is known in itself, and for example makes it possible to pilot the fixed-wing drone 14. The control stick 16 comprises two gripping handles 36, each being intended to be grasped by a respective hand of the user 12, a plurality of control members, here including two joysticks 38, each being positioned near a respective gripping handle 36 and being intended to be actuated by the user 12, preferably by a respective thumb.
The control stick 16 also comprises a radio antenna 34 and a radio transceiver, not shown, for exchanging data by radio waves with the fixed-wing drone 14, both uplink and downlink.
Additionally, or alternatively in light of the viewing system 10, a digital multimedia touchscreen tablet 70 is mounted on the control stick 16 to assist the user 12 during piloting of the fixed-wing drone 14.
The operation of the electronic system for capturing a video 1 according to the invention will now be described using
The method for capturing a video according to the invention allows the virtual orientation of the viewing axis of the camera, and therefore the direction filmed by the drone when the fixed-wing drone 14 experiences wind 44.
When the image sensor 28 is associated with a fisheye lens, it is possible to orient the viewing axis of the camera virtually with a sufficient angular travel, in particular to account for the drift of the fixed-wing drone 14.
In particular, the method for capturing a video according to the invention makes it possible to define a virtual image sensor by selecting a zone Zc with smaller dimensions relative to the actual dimensions of the image sensor 28.
In
During a step 100, the drift angle α of the fixed-wing drone 14 is determined by the determining module 56.
According to the embodiment shown in
Furthermore, according to the embodiment shown in
During a step 106, the video filmed during flight by the fixed-wing drone 14 is obtained.
More specifically, the step for obtaining the video 106 comprises an image acquisition step 108.
When the lens associated with the image sensor 28 is of the fisheye type, the image acquisition step 108 in particular carries out a correction, not shown, of the geometric distortions introduced by the image sensor 28.
The image acquisition step 108 comprises a step 110 for determining the position Pzc of the zone Zc with reduced dimensions relative to the actual dimensions of the image sensor 28 as a function of the drift angle α of the fixed-wing drone 14.
According to a first alternative, the zone Zc, the position Pzc of which is determined 110 as a function of the drift angle α, is configured to capture a region of the scene filmed by the image sensor on board the drone along a viewing direction parallel to the direction of flight 40 of the drone.
In other words, the viewing direction is oriented virtually (not mechanically) on the actual direction of flight of the means under the method according to the invention.
According to a second alternative, which can be combined with the first alternative, the zone Zc, the position PZc of which is determined 110 as a function of the drift angle α, and the zone obtained in the absence of drift of the drone have a non-zero intersection.
The center of the zone obtained without drift of the drone is consequently positioned on these two directions 40 and 42, combined. According to the second alternative, the size of the zone is therefore optimized such that the zones determined with and without wind are not separate, which makes it possible to ensure visual continuity when the wind rises abruptly.
According to a third alternative, the position PZc of the zone Zc is calculated such that the window corresponding to the overall field of the camera rotates 112, relative to that without drift, by a drift angle α of the drone around the yaw axis 48 of the drone.
Such a rotation of the center of the window corresponding to the zone Zc amounts to a translation and/or rotation of the zone Zc on the image sensor.
In other words, the window corresponding to the zone Zc is dynamically and virtually moved in the field of the camera produced by the image sensor 28.
Once the position PZc is determined as a function of the drift angle α, it is possible for the zone Zc to abut with the image sensor 28. In this case, according to a first alternative, the image corresponding to the zone Zc comprises a first black area, or according to a second alternative, the position PZc is reset such that the center of the zone Zc is on the longitudinal axis 42 of the fixed-wing drone 14. According to one particular embodiment, the image acquisition step 108 comprises acquiring 114 image data from the entire image sensor, followed by digital processing 116 of the image data delivering video images corresponding solely to the zone Zc.
In other words, according to this embodiment, the digital processing is done after the capture of the image data. Such digital processing makes it possible to shift, in time, the obtaining of corrected images to be played back to the user.
According to another specific embodiment, the image acquisition step 108 comprises the acquisition 118 of image data only in the zone Zc during the production of the video, the position PZc of the zone Zc being determined during the production of the video.
In other words, according to this other embodiment, the selection of the zone Zc is implemented in real time and is made subject in real time to the drift of the fixed-wing drone 14.
The method for capturing a video according to the invention makes it possible to optimize the piloting of the drone owing to the fact that the position of the zone Zc is made subject to the drift of the fixed-wing drone 14 caused by a cross wind.
The user 12 perceives, in particular via the use of the virtual-reality viewing system 10, the actual direction of the flight of the fixed-wing drone 14, even in case of drift thereof, since the video capture method according to the invention seeks to guarantee that the viewing direction of the image sensor and the direction of flight of the drone are parallel.
The method for capturing a video according to the invention consequently makes it possible to improve the ergonomics of the first-person view (FPV).
“The user experience” in the immersive piloting configuration therefore allows the user 12 to optimize the piloting, in order to account for the drift of the fixed-wing drone resulting from the force of the cross wind, to effectively reach the desired flight destination.
This further makes it possible to avoid the crash or deterioration of the fixed-wing drone in the presence of obstacles along the direction of flight resulting from the drift and not visible if the viewing direction of the image sensor points along the longitudinal direction of the fixed-wing drone 14.
| Number | Date | Country | Kind |
|---|---|---|---|
| 16 57727 | Aug 2016 | FR | national |
