The present invention relates to improvements in or relating to waveguide display systems, including but not limited to a display integrity scheme.
A display system may be required to operate with a high level of integrity. This is relevant in many situations, including for example for a flight critical display system, for use with avionics or for a targeting system used by the military. Display systems used to present safety critical information to a user are required to operate with a very high level of integrity. The display system must be able to detect when hazardously misleading information is presented to the user and act to mitigate the issue.
Often the display's integrity is monitored solely in the electronic side of the display path with no optical inspection of the display. This is possible where the display path electronics are fully understood from the generation of the image content through to the construction of the image on the display device. This typically requires a full understanding of the design and function of each component in the display path. This is not always possible, especially with the use of complex COTS (commercial off the shelf) components, such as display chipsets. With the use of COTS components within the system full access to the underlying processes and designs of the COTS items may not be possible which may lead to a more basic integrity monitoring system implemented within the display system giving inaccurate information. In these scenarios optical monitoring of the display would provide a method of closing the display path integrity loop.
Optical integrity monitoring systems previously implemented with various display systems can rely on a separate display path within the display projection optic. These optical integrity monitoring systems add mass and volume to the projection system. When used in display systems such as a head up display (HUDs) this is not a major issue as size and weight are not key design parameters. However, in other systems where mass and volume are a key design parameter, for example in a head worn display (HWD). this type of optical integrity monitoring system is inadequate.
Existing integrity monitoring schemes which allow for basic integrity monitoring of the display device within an optical system, can perform this task by checking whether rows or columns of the display are functioning as expected.
There is thus a need for a higher fidelity integrity monitoring system, capable of detecting complex display faults and scaled appropriately such that the integrity monitoring system is usable in smaller form factor applications, such as a head worn display system. The invention provides a scheme which overcomes at least some of the disadvantages of present systems and for implementing a lightweight, low volume optical monitoring package. The invention allows for more detailed pixel level monitoring to take place.
Accordingly, one object of the present invention is to provide a waveguide display system that is able to determine the integrity of an image.
According to one aspect of the present invention there is provided a system for delivering an image to a user via a waveguide display system and for determining integrity of an image, the system comprising: a combination waveguide, in coupling and out coupling assembly comprising: a waveguide region; an in coupling region configured to receive a collimated image from an imaging system and an out coupling region configured to direct the collimated image toward an eye of a user, wherein the waveguide is configured to direct at least a first portion of the collimated image from the in coupling region towards the out coupling region to be directed to the user; and wherein prior to the collimated image being directed towards the eye, a second portion of the collimated image is configured to be directed towards an integrity monitoring module which is configured to determine at least one discrepancy between the collimated image and the image to thereby correct the first portion of the collimated image.
In an aspect the in coupling and out coupling regions each comprise a surface of the waveguide assembly providing a predetermined optical effect.
In an aspect each surface of the waveguide assembly comprises a reflective coating.
In an aspect each surface of the waveguide assembly is a predetermined shape.
In an aspect the in coupling region includes a reflective surface having a saw-tooth profile which splits the light into a +1R diffracted beam and a −1R diffracted beam.
In an aspect the in coupling region includes a reflective surface having a saw-tooth profile which splits the light into different order beams.
In an aspect the out coupling region includes a reflective surface having a saw-tooth profile.
In an aspect the collimated image comes from a collimation optic configured to output an exit pupil comprising the collimated image.
In an aspect the in coupling region directs the first portion of the collimated image towards the out coupling region and the second portion of the collimated image towards the integrity monitoring module.
In an aspect the integrity monitoring module comprises an optical arrangement configured to direct the second portion of the collimated image to a sensor.
In an aspect the integrity monitoring module is further configured to analyse the second portion of the collimated image to determine the at least one discrepancy between the collimated image and the image and to send to the system information related to at least one of the nature of the at least one discrepancy between the collimated image and the image and one or more corrections to mitigate or overcome the at least one discrepancy between the collimated image and the image.
In an aspect the system of is for use in a binocular arrangement comprising one combination waveguide, in coupling and out coupling assembly for each eye.
In an aspect a correction system is configured to implement a correction to the image such that a discrepancy between the collimated image and the image is reduced.
In an aspect the discrepancy is reduced based on comparison with a threshold discrepancy.
In an aspect the integrity monitoring module comprises an optical arrangement configured to direct the second portion of the collimated image to a sensor.
In an aspect the integrity monitoring module comprises a lens element and a sensor to provide the second portion of the collimated image to a monitoring loop in which discrepancies between the collimated image and image are determined.
In an aspect the integrity monitoring module is further configured to analyse the second portion of the collimated image to determine the at least one discrepancy between the collimated image and the image and to send to the system information related to at least one of the nature of the at least one discrepancy between the collimated image and the image and one or more corrections to mitigate or overcome the at least one discrepancy between the collimated image and the image.
According to another aspect of the present invention there is provided a Heads Up Display including the system of another aspect of the invention.
According to another aspect of the present invention there is provided a head worn display system of another aspect of the invention.
According to another aspect of the present invention there is provided an integrity monitoring module for use in the system of any preceding claim, the integrity monitoring module comprising: optics configured to receive the second portion of the collimated image; direct the second portion of the collimated image towards one or more sensors configured to determine at least one discrepancy between the collimated image and the image on the second portion of the collimated image; generate an output indicative of at least one of the at least one discrepancy between the collimated image and the image and a correction to the at least one discrepancy between the collimated image and the image.
According to another aspect of the present invention there is provided a combination waveguide, in coupling and out coupling assembly for a system for delivering an image to a user via a waveguide display system and for determining integrity of an image, the combination waveguide, in coupling and out coupling assembly comprising: a waveguide region; an in coupling region configured to receive a collimated image from a display system and an out coupling region configured to direct the collimated image to an eye of a user.
In an aspect the in coupling and out coupling regions each comprise a surface of the waveguide assembly providing a predetermined optical effect.
In an aspect each surface comprises at least one of: a reflective coating, is of a predetermined shape.
In an aspect the in coupling region includes a reflective surface having a saw-tooth profile which splits the light into different order diffractive beams.
According to another aspect of the present invention there is provided a method for delivering an image to a user via a waveguide display system and for determining integrity of an image, the method comprising: receiving via a combination waveguide, in coupling and out coupling assembly a collimated image from an imaging system; splitting the collimated image into first and second portions; directing via an out coupling region the first portion of the collimated image to an eye of a user, directing the second portion of the collimated image to an integrity monitoring module which determines at least one discrepancy between the collimated image and the image in the collimated image to
In general, the present invention relates to an integrity monitoring scheme (also referred to as an optical integrity monitoring system) used with waveguide-based displays which provides a scheme for checking the optical output of the display system and bypassing any issues with the use of existing components within the display path which cannot be interrogated or modified. A number of examples are presented, which make use of diffractive optical elements for an in coupling element and an out coupling element. The invention could also encompass other optical methods for the coupling of light into and out of the waveguide assembly, such as reflective structures. The image bearing light is then collected by an optical sensing element which provides a way to analyse the display content as part of the optical integrity monitoring system.
Integrity monitoring systems have been implemented in Avionic display systems or imaging which are used as primary flight instruments for some time. In some cases, the displays integrity is monitored solely in the electronic side of the display path with no optical inspection of the display. This is possible where the display path electronics are fully understood from the generation of the image content through to the construction of the image on the display device. This typically requires a full understanding of the design and function of each component in the display path. This is not always possible, especially with the use of complex COTS components, such as display chipsets. Additionally, these methods do not account for any physical changes within the optical system which might affect the display image, for example optical lenses moving causing the display image to become defocussed. In these scenarios optical monitoring of the display provides a method of closing the display path integrity loop. The optical integrity monitoring system then feeds back data which can be used to either correct discrepancies between the commanded or original image from a scene and the image displayed to the user, or inform the user of a failure within the device and removing the risk of potentially misleading information being displayed to the user. The original image may comprise a collimated image which is received from a source, such as a display, camera or any other imaging system. The collimated image comes from a collimation optic configured to output an exit pupil comprising the collimated image. This will be described in greater detail below.
In the figures described below a solid light ray (primary path) represents the part of light directed towards a user and the dotted line (secondary path) represents the part of light directed to an optical integrity monitoring module.
Referring to
The in coupling region directs the majority of the light 110 along a primary path 112 (shown as a solid line) towards the out coupling region 106 which then expands the pupil and directs light 114 out of the waveguide assembly towards the user's eye 116. In the case of the diffractive in coupling regions, the primary path 112 is created by the +1R diffracted order (reflected order). Simultaneously the in coupling region 104 directs a smaller proportion of the incident light along a secondary path 118 within the waveguide assembly (shown by the dashed line) towards the secondary out coupling region 107. This secondary path is created by the −1R diffracted order (reflected order). The secondary out coupling region 107 directs the light out of the waveguide assembly which is then collected by the optical integrity monitoring module 108.
The split of light between the positive and negative diffractive orders can be controlled by the shape of the diffractive structure and any coating applied to the structure. This will be described in more detail below. This aspect of the invention makes use of the +1 and −1 orders; however, any diffractive orders could be utilised to split the light paths. The out coupling regions may be the conjugate of the in coupling region to conserve the angles of the light paths injected into and ejected from the waveguide assembly. The secondary out coupling region could also be configured to be used in transmission rather than reflection. Other optical systems having the ability to split and transmit the light as required may also be used.
The optical integrity monitoring module is configured to analyse portion 118 to determine at least one optical discrepancy between an original collimated and received image and to send to the system information related to at least one of the nature of the optical discrepancies between the original and received image and one or more corrections to mitigate or overcome the optical discrepancies between the original and received image. The manner in which the integrity module analyses and determines discrepancies is discussed further below with reference to
Referring now to
The arrangement 400 in
In
The arrangements shown in
In
In order to provide optical monitoring of the left eye display and right eye display separately the display system may use time multiplexing to split the images or to switch between the left and right displays. For example, each image is displayed at one eye and then the other. In the time the image is being displayed at one eye the other may be blanked In synchronisation with this display drive arrangement the sensor is configured to capture a frame at the same time as the image is displayed to the left eye, and subsequently timed to capture the separate image driven to the right eye. In order to determine the optical discrepancies between the original collimated image and actual image displayed to each eye and caused in the system would thus require a complex analysis.
Similarly, light that passes along the secondary path 118 exits via an out coupling region 106 having a reflective surface 708. This surface again incudes a saw tooth form. Light is passed to the optical integrity monitoring module 108 for processing as discussed above. As can be seen the saw-tooth form of the out coupler has a different form than that of the in coupler. This changes how the light is directed. It should be noted that the form, shape and size of any optical region can be changed to adapt to the nature of the waveguide assembly and the overall system and/or system intent.
The waveguide assembly of the present invention can be coupled to the optical integrity monitoring module, which may or may not be part of the system. As the waveguide assembly is a combined waveguide substrate, in coupler and out coupler, the assembly is formed as a single compact component which is relatively lighter in weight than the individual separate elements.
In all schemes the light extracted for integrity monitoring is analysed to determine if there are any optical discrepancies between the original and received image introduced by the display system. Information relating to any such optical discrepancies between the original and received image is fed back to a correction system (not shown) to apply a correction to the light arriving at the eye of the user.
The integrity monitoring module is able to detect and generate corrections for all optical discrepancies between the original and actual image. There are several possible uses for the integrity monitoring system. In a basic usage scenario, the integrity monitoring system looks for discrepancies between the collimated image and the displayed image, any discrepancy can then be compared to a pre-set threshold value for that discrepancy type. If the discrepancy exceeds the threshold value, then the user is made aware of a fault within the system via a correction or feedback system forming part of the overall system.
In another scenario it may be the case that the fault is a frozen display i.e. the display is not being updated with the latest image. In this case the user may be made aware of the issue by the display channel being switched off. This would prevent the user being exposed to misleading information. In more complex systems it could be that a portion or section of the image is incorrect, due to “stuck pixels” in the display generator. In this case it may be possible to reposition information which would be displayed over those pixels to another position on the display, therefore still providing the correct information to the user albeit in a different location.
The integrity monitoring system may be capable of monitoring the following discrepancies in the display path:
Magnification error resulting in incorrect image scaling.
Image shift causing the image to not align correctly with the outside world.
Image rotation.
Colour channel failure, causing the loss of some commanded colours and the incorrect rendering of mixed colours.
Luminance discrepancies, the image being displayed to the eye not corresponding to the set/commanded brightness level.
Display failure, no image being displayed to the user.
In some cases, a correction may be able to be applied to the driven image to mitigate the detected failure, such as a luminance adjustment, an image shift/rotation. However, in some other cases such as colour channel failure, the user could be made aware of the reduced colour palette being made available. The information may still be displayed to the user, but perhaps in monochrome, removing the advantage of colour coded symbols etc. but still allowing the user to see an accurate representation of the image.
The invention is intended to be used in waveguide HWDs, however its use is not limited to HWDs and could be implemented on the waveguide HUD system. Whilst the invention provides the ability to monitor the integrity of the images displayed to the user, it could also be implemented in a simpler form to monitor display brightness and colour. In these examples the sensor in the optical integrity monitoring module would be appropriate to the intended type of monitoring.
Although the present invention has been described in connection with some embodiments, it is not intended to be limited to the specific form set forth herein. Rather, the scope of the present invention is limited only by the accompanying claims. Additionally, although a feature may appear to be described in connection with particular embodiments, one skilled in the art would recognize that various features of the described embodiments may be combined in accordance with the invention. In the claims, the term ‘comprising’ does not exclude the presence of other elements or steps.
Furthermore, the order of features in the claims does not imply any specific order in which the features must be performed and in particular the order of individual steps in a method claim does not imply that the steps must be performed in this order. Rather, the steps may be performed in any suitable order. In addition, singular references do not exclude a plurality. Thus, references to ‘a’, ‘an’, ‘first’, ‘second’, etc. do not preclude a plurality. In the claims, the term ‘comprising’ or “including” does not exclude the presence of other elements.
| Number | Date | Country | Kind |
|---|---|---|---|
| 22275018.4 | Feb 2022 | EP | regional |
| 2202395.6 | Feb 2022 | GB | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/GB2023/050346 | 2/15/2023 | WO |