Patent Application
20230415894
This application claims the benefit of European Patent Application No. 22180570.8 filed on Jun. 22, 2022, the entire disclosure of which is incorporated herein by way of reference.
The present invention pertains to a system and a method for displaying a view of an external environment of an aircraft inside the aircraft.
The design and size of aircraft windows is usually constrained by engineering considerations. Implementing windows in aircraft can increase weight, drag, manufacturing costs and maintenance efforts for the aircraft. Thus, there is a conflict between engineering boundary conditions and the desire for a higher comfort of the passenger.
Prior art documents US 2016/0090196 A1, EP 3 148 186 A1, and EP 3 296 204 A1 describe virtual window systems for aircraft, which capture the external environment of the respective aircraft via cameras installed on the outside of a fuselage and display the recorded images on one or several screens inside the fuselage, thereby replacing conventional window apertures through the fuselage.
To accomplish a satisfactory view of the environment based on such an approach one needs a large number of cameras. These cameras should be small, which however may impact their image quality. Cameras can also increase air drag and slightly increase field consumption. Moreover, the cameras will be on the outside, exposed to environmental damage and may thus require frequent maintenance or replacement.
Against this background, it is an object of the present invention to find solutions that combine the comfort and user experience of a normal windowed cabin with the practicality and cost effectiveness of a windowless cabin based on virtual windows.
This object may be achieved by a system having the features of one or more embodiments described herein, an aircraft having the features of one or more embodiments described herein, and a method having the features of one or more embodiments described herein.
According to the invention, a system for displaying a view of an external environment of an aircraft inside the aircraft comprises several optical couplers distributed across an outer surface of a fuselage of the aircraft and configured to capture light from the external environment of the aircraft; optical waveguides arranged through the fuselage to transmit the captured light from the optical couplers through the fuselage; at least one display arranged on the inside of the fuselage; and at least one of: a processing unit configured to receive the captured light from several of the optical waveguides, to reconstruct a digital live image of the external environment by combining optical information contained in the captured light across the respective optical couplers and to display the digital live image on the at least one display; and an optical beam expander configured to receive the captured light from at least one of the optical waveguides, to expand the captured light into a light beam carrying a visual live image of the external environment and to project the visual live image on the at least one display.
Further according to the invention, an aircraft is equipped with a system according to the invention.
Further according to the invention, a method for displaying a view of an external environment of an aircraft inside the aircraft comprises capturing light from the external environment of the aircraft with several optical couplers distributed across an outer surface of a fuselage of the aircraft; transmitting the captured light from the optical couplers through the fuselage with optical waveguides arranged through the fuselage; and at least one of: reconstructing a digital live image of the external environment with a processing unit receiving the captured light from several of the optical waveguides by combining optical information contained in the captured light across the respective optical couplers and displaying the digital live image on at least one display arranged inside the fuselage; and expanding the captured light into a light beam carrying a visual live image of the external environment with an optical beam expander receiving the captured light from at least one of the optical waveguides and projecting the visual live image on the at least one display.
Thus, one idea of the present invention is to realize an almost or quasi-windowless cabin, in which some or all of the windows are replaced by virtual windows connected to the outside via one or more optical couplers and waveguides. The idea is to preserve a more or less direct visual connection between the outside and the inside of the cabin, while at the same time avoiding relevant holes in the fuselage. The invention proposes a solution, which will result in virtual windows with high image quality. It provides almost all the advantages of a windowless cabin, in terms of reduced costs, while giving a better passenger experience than with camera-based windowless cabin solutions. To this end, the invention offers two alternative solutions, which may also complement each other. One is based on digital data processing of the optical information contained in the gathered light. The other one is based on manipulation and rerouting of the gathered natural light directly to one or several screens inside the aircraft.
In a normal window, the light is carried from the outside to the inside of a cabin via an aperture of significant size in the fuselage. This large opening however can be avoided with the present invention by distributing multiple optical couplers along the fuselage. The image is then carried over by optical fibers (e.g. a fiber cable) or other optical waveguides. The invention takes inspiration from the fields of compressed sensing, computational imaging and/or lensless cameras, where it was shown that relatively good images can be reconstructed by using only a few (<2%) of the pixels available in the camera sensors. Hence, it is possible to use compressed-sensing or similar algorithms to reconstruct what the passengers would see from different points in the cabin. With the right disposition of the optical waveguides, the same scene could be viewed from different angles, and this could potentially enable 3D displays in the passenger seats, or even holograms, once the related technology will become accessible at lower cost.
The invention offers various advantages, amongst them weight reduction. Windows are heavier than the fuselage, thus by replacing them with smaller apertures the aircraft weights less and consumes less fuel. The solution may also allow cheaper integration as smaller windows do not require reinforcement, so they are much easier and cheaper to integrate. Also it will be easier to route cables (e.g. electricity, data cables etc.) from one part of the aircraft to the other. This task may currently be challenging because cable needs to go around large windows. Moreover, displays/screens themselves might be cheaper than the windows, because windows are made of special materials and require special sealings etc.
Together with seamless and high broadband connectivity, enabled by current and future 5G/6G technologies, the proposed technology could also be an ideal entry point to open the passenger experience to the metaverse. Virtual and hybrid 3D screens, augmented realities and (in a further future) holograms, will allow passengers to interact with the metaverse while flying. The passenger could play games or interact with the images to display more information, see how a certain monument would look in winter or in summer or how it was in the past, and then go back to the real current image. The passenger could interact using virtual touch screens, but also with virtual reality headset, e.g. the passengers could bring their own headset and connect it to the cabin to see a 3D view of the surroundings of the aircraft. Future airborne meshed networks will allow different aircraft to share and fuse data, therefore further increasing the quality of the information and the images produced and displayed to the passengers.
Hence, the invention may open up new revenues, e.g. non-intrusive advertisement. 3D virtual screens and hybrid physical/virtual screens open up the possibility of overlapping small advertisements on top of the image. For example, when flying above some mountains, the virtual screen may show the location of ski facilities, hotels etc. in a similar vein to what search engines do for the map services. In addition, airlines could charge more for seats with 3D virtual screens.
Advantageous embodiments and improvements of the present invention are found in the subordinate claims.
According to an embodiment of the invention, the at least one display may comprise a digital display for displaying the digital live image provided by the processing unit and/or a screen for projecting the visual live image provided by the optical beam expander.
Hence, the digital live image may be shown on one or several digital displays inside the aircraft, e.g. within a passenger cabin. The displays may then replace or supplement conventional windows (i.e. apertures through the fuselage wall). Similarly, natural light routed via the optical waveguides may be magnified/expanded and the corresponding visual live image may be projected onto screens facing inward inside the fuselage. These screens may then also take the place of conventional windows. Both approaches may be combined in a single aircraft, e.g. some windows may be replaced by digital displays provided with computed images from the processing unit, while others are replaced by natural light screens being fed by the optical couplers and respective wave guides.
According to an embodiment of the invention, the at least one display may comprise a hybrid display configured to selectively display on a shared screen the digital live image provided by the processing unit and the visual live image provided by the optical beam expander.
Some or all virtual windows may thus also be hybrid physical-virtual windows. For this purpose, the light (and images) may be collected using a fiber bundle, for example, and then split with a passive device into two (or more) optical channels. One of them may be routed through the processing unit, and the other may be directly coupled to the hybrid window. The hybrid window can be designed to display both the physical and virtual images or to automatically switch to the physical image from the digital image when the electricity fails, for example. Since the diameter of such a fiber bundle will likely be smaller than the size of the actual window, i.e. the size of the respective display/screen, a beam expander of some sort may be employed. This device could be, for example, entirely fiber based (e.g. with a design inspired by photonic lanterns) and/or do the beam expansion in free space optics (e.g. using micro-lenses).
According to an embodiment of the invention, the shared screen may be semi-transparent to transmit the visual live image provided by the optical beam expander.
Hence, in case a hybrid window is designed to display both the physical and virtual images, a semi-transparent screen may be used to display both images on the same screen. The light coming from the optical couplers may first go to a passive beam splitter (e.g. a fiber coupler). Then, part of the light may continue to the processing electronics. The other part may be coupled out from the optical waveguide (e.g. a fiber bundle), magnified to match the screen size (i.e. the virtual window size) and then routed to the hybrid window. If there is an electricity outage, for example, the fail-safe (no-current) mode should be that the natural light is still transmitted through the window. Such a passive channel, which the natural light illuminates, may even be bi-directional. This means that it could be implemented in such a way that an observer from the outside of the aircraft could inspect the inside of the cabin without using any special equipment.
According to an embodiment of the invention, the hybrid display may be configured to dynamically adjust the digital live image provided by the processing unit to match the visual live image provided by the optical beam expander.
The digital image can thus be dynamically adjusted to match the natural image, e.g. increase the contrast, resolution and/or brightness of the image, such that a seamless transition between digital and natural images may be performed.
According to an embodiment of the invention, the at least one display may comprise several displays arranged on an inner wall of the fuselage and shaped to represent aircraft windows.
The number of small windows/apertures need not to be the same as the current number of windows. It could merely be a fraction of it. A detailed evaluation may be performed for each use case to identify an optimal number of windows. In this sense it should be kept in mind that fewer apertures imply cost savings, which are however accompanied by reduced image quality. The latter could potentially be compensated however, e.g. based on machine learning/AI methods, e.g. denoising and filling in gaps using generative adversarial networks, or advanced computational imaging algorithms. Somewhat similar to 3D imaging solutions used in automotive, a 3D image could be created by merging the different optical feeds. The goal is to recreate a high quality (e.g. spherical) image from as few optical couplers as possible. This can be achieved by having a relatively large overlap between the field of views of the different optical couplers so to have redundancy in the data which can be used to digitally improve the images.
In order to prevent total darkness in case of electricity outage, some of the windows (e.g. on the doors) may not be replaced. Also, several of the windows may be hybrid physical/virtual windows. In case of an electricity outage they will still show an image (or alternatively just light) using the light coming from the outside of the aircraft via an optical waveguide. Extra costs associated with cleaning/maintaining the aperture windows (optical couplers, fiber bundles etc.) may arise. However, in general it is likely that the system will work also with relatively dirty windows, since the redundancy/overlap between the images seen from the different optical couplers/waveguides will allow to digitally filter out dust and scratches from the individual optical surfaces. Using couplers and waveguides instead of simple cameras is also safer with respect to lighting.
According to an embodiment of the invention, the optical couplers may be configured as fiber collimators and the optical waveguides comprise optical fibers.
For example, single- and/or multi-core fiber couplers may be combined with one or several bundles of fibers as waveguide (such as provided by a fiber cable). It is to be understood of course that also single multi-core fibers and/or multi-mode fibers or similar may be employed for this purpose.
According to an embodiment of the invention, the optical couplers may be arranged recessed in the outer surface of the fuselage.
Thus, small holes or recesses may be provided in the skin of an aircraft, into which the optical couplers may be mounted such that the flow dynamics above the corresponding outer aircraft surfaces is not significantly affected and drag is not increased relevantly.
According to an embodiment of the invention, the processing unit may be configured to employ compressed sensing, sensor fusion and/or computational imaging to reconstruct the digital live image.
Compressed sensing is a signal processing technique for efficiently acquiring and reconstructing a signal by finding solutions to underdetermined linear systems. This is based on the principle that, through optimization, the sparsity of a signal can be exploited to recover it from far fewer samples than required by the Nyquist-Shannon sampling theorem.
Sensor fusion is the process of combining sensor data or data derived from disparate sources such that the resulting information has less uncertainty than would be possible when these sources were used individually. The term uncertainty reduction in this case can mean more accurate, more complete, or more dependable, but may particularly also refer to the result of an emerging view, such as when combining two-dimensional images from several sources at slightly different viewpoints.
Computational imaging is the process of indirectly forming images from measurements using algorithms that rely on a significant amount of computing. In contrast to traditional imaging, computational imaging systems involve a tight integration of the sensing system and the computation in order to form the images of interest.
The invention will be explained in greater detail with reference to exemplary embodiments depicted in the drawings as appended.
The accompanying drawings are included to provide a further understanding of the present invention and are incorporated in and constitute a part of this specification. The drawings illustrate the embodiments of the present invention and together with the description serve to explain the principles of the invention. Other embodiments of the present invention and many of the intended advantages of the present invention will be readily appreciated as they become better understood by reference to the following detailed description. The elements of the drawings are not necessarily to scale relative to each other. In the figures, like reference numerals denote like or functionally like components, unless indicated otherwise.
Although specific embodiments are illustrated and described herein, it will be appreciated by those of ordinary skill in the art that a variety of alternate and/or equivalent implementations may be substituted for the specific embodiments shown and described without departing from the scope of the present invention. Generally, this application is intended to cover any adaptations or variations of the specific embodiments discussed herein. As used herein, “at least one of” A and B, shall mean A, or B, or A and B.
The aim of the system 10 is to realize a quasi-windowless cabin, in which the windows are replaced by virtual windows (displays 6a, 6b) connected to the outside via optical fibers. The underlying idea is to provide a view of the environment of an aircraft 100 from inside the aircraft 100 without having to weaken the fuselage structure with window apertures. The displays 6a, 6b are arranged on an inner wall 5b of the fuselage 5 and may be shaped to represent conventional aircraft windows (cf.
The system 10 comprises several fiber collimators as optical couplers 1, which are distributed across an outer surface 5a of a fuselage 5 of the aircraft 100, each optical coupler 1 being mounting in a respective recess (or hole) in the outer surface 5a. The optical couplers 1 are configured to capture light 7 from the external environment of the aircraft 100 and to couple it into respective optical waveguides 2, e.g. fiber bundles or cables, which are routed through the fuselage 5 to transmit the captured light 7 from the optical couplers 1 through the fuselage 5. One such optical coupler 1 is depicted in detail in
As can be seen here, light 7 from the environment enters the optical coupler 1 via a phase and/or amplitude mask 1a (similarly to what is done in lensless cameras) and is then focused into the waveguide 2 by beans of a lens 1b. The waveguide 2 (fiber cable) may carry multiple optical signals, which allows it to reconstruct an image, similarly to what is done in lensless imaging. For this purpose the waveguide 2 can be implemented in several possible ways, for example as a bundle of fibers, as a single multi-core fiber or as a multimode fiber. Here, the individual cores of the fibers play a similar role as the pixels of the sensor in a lensless camera.
Some of the waveguides 2 are coupled to a processing unit 3, which is configured to receive the captured light 7 from the optical waveguides 2, to reconstruct a digital live image of the external environment by combining optical information contained in the captured light 7 across the respective optical couplers 1 and to display the digital live image on one or several digital displays 6b. To this end, the processing unit 3 may employ compressed sensing, sensor fusion and/or computational imaging or similar methods to reconstruct the digital live image from one or several optical couplers 1.
The number of fibers required to form an image could be significantly less that the number of pixels required for a normal cameras. This the case, because the high number of pixels usually provided by modern cameras (often >10 megapixel) is not required for the present purpose (no images need to be printed). Moreover, and more importantly, computational imaging and other advanced methods allow to retrieve an image from very few pixels, that is fibers in the present case. Sensor fusion between the data produced by the different cameras/fiber bundles will likely allow to reduce the number of fiber-per-bundle even further, by exploiting redundancies (overlap between images) and/or by using deep learning (e.g. generative adversarial networks).
An alternative approach, perhaps more conventional, is to use several very small cameras and then combine their outputs via sensor fusion techniques in order to combine all the electrical outputs of the mini-cameras into a single big image. Two possible implementations, one more conventional and one relaying on lensless imaging are presented in
Coming back to the embodiment of
The screen 6c may be part of a corresponding (analog) display, for example. In the exemplary embodiment of
As can be seen in
The method M in
The systems and devices described herein may include a controller or a computing device comprising a processor and a memory which has stored therein computer-executable instructions for implementing the processes described herein. The processing unit may comprise any suitable devices configured to cause a series of steps to be performed so as to implement the method such that instructions, when executed by the computing device or other programmable apparatus, may cause the functions/acts/steps specified in the methods described herein to be executed. The processing unit may comprise, for example, any type of general-purpose microprocessor or microcontroller, a digital signal processing (DSP) processor, a central processing unit (CPU), an integrated circuit, a field programmable gate array (FPGA), a reconfigurable processor, other suitably programmed or programmable logic circuits, or any combination thereof.
The memory may be any suitable known or other machine-readable storage medium. The memory may comprise non-transitory computer readable storage medium such as, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. The memory may include a suitable combination of any type of computer memory that is located either internally or externally to the device such as, for example, random-access memory (RAM), read-only memory (ROM), compact disc read-only memory (CDROM), electro-optical memory, magneto-optical memory, erasable programmable read-only memory (EPROM), and electrically-erasable programmable read-only memory (EEPROM), Ferroelectric RAM (FRAM) or the like. The memory may comprise any storage means (e.g., devices) suitable for retrievably storing the computer-executable instructions executable by processing unit.
The methods and systems described herein may be implemented in a high-level procedural or object-oriented programming or scripting language, or a combination thereof, to communicate with or assist in the operation of the controller or computing device. Alternatively, the methods and systems described herein may be implemented in assembly or machine language. The language may be a compiled or interpreted language. Program code for implementing the methods and systems described herein may be stored on the storage media or the device, for example a ROM, a magnetic disk, an optical disc, a flash drive, or any other suitable storage media or device. The program code may be readable by a general or special-purpose programmable computer for configuring and operating the computer when the storage media or device is read by the computer to perform the procedures described herein.
Computer-executable instructions may be in many forms, including modules, executed by one or more computers or other devices. Generally, modules include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. Typically, the functionality of the modules may be combined or distributed as desired in various embodiments.
In the foregoing detailed description, various features are grouped together in one or more examples or examples with the purpose of streamlining the disclosure. It is to be understood that the above description is intended to be illustrative, and not restrictive. It is intended to cover all alternatives, modifications and equivalents. Many other examples will be apparent to one skilled in the art upon reviewing the above specification. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications, to thereby enable others skilled in the art to best utilize the invention and various embodiments with various modifications as are suited to the particular use contemplated.
While at least one exemplary embodiment of the present invention(s) is disclosed herein, it should be understood that modifications, substitutions and alternatives may be apparent to one of ordinary skill in the art and can be made without departing from the scope of this disclosure. This disclosure is intended to cover any adaptations or variations of the exemplary embodiment(s). In addition, in this disclosure, the terms “comprise” or “comprising” do not exclude other elements or steps, the terms “a” or “one” do not exclude a plural number, and the term “or” means either or both. Furthermore, characteristics or steps which have been described may also be used in combination with other characteristics or steps and in any order unless the disclosure or context suggests otherwise. This disclosure hereby incorporates by reference the complete disclosure of any patent or application from which it claims benefit or priority.
| Number | Date | Country | Kind |
|---|---|---|---|
| 22180570.8 | Jun 2022 | EP | regional |
