Patent Application
20240428713
This application claims priority to European Patent Application No. 23 180 876.7 filed Jun. 22, 2023, European Patent Application No. 23 180 877.5 filed Jun. 22, 2023, European Patent Application No. 23 196 195.4 filed Sep. 8, 2023, and European Patent Application No. 23 214 645.6 filed Dec. 6, 2023, the disclosures of which are incorporated herein by reference.
The present disclosed subject matter relates to a display apparatus comprising a light source configured to emit a light beam, a pulse generator configured to generate a pulse train having a pulse rate, a buffer configured to store pixels of an image, a controller configured to receive the pulse train and the stored pixels, to modulate the pulse train in pulse amplitude and/or pulse width according to the pixels, and to drive the light source according to the modulated pulse train, a mirror assembly with one or more mirrors configured to oscillate and scan the light beam over an image area according to a scan pattern, and a mirror driver configured to drive the mirror assembly according to said scan pattern. The present disclosed subject matter further relates to a method for displaying pixels of an image.
Display apparatus are commonly used in virtual reality (VR) or augmented reality (AR) glasses, helmets or head-up displays (HUDs) for a broad range of applications like navigation, training, entertainment, education or work. A pulse generator provides a pulse train to a controller that modulates the pulse train in amplitude and/or width according to pixels of an image (frame) that are stored in a buffer and successively fed to or retrieved by the controller. According to the modulated pulse train, the controller drives a light source to emit a mono- or multicoloured light beam which carries the image onto a mirror assembly having one or more moving micro-electro-mechanical-system (MEMS) mirrors driven by a mirror driver. The mirror assembly has, e.g., one MEMS mirror oscillating about two axes or two MEMS mirrors each oscillating about a respective axis, to deflect the light beam into subsequent directions (angles) towards an image area, one direction (angle) per pixel of the image to scan the light beam over the image area in order to display the pixels. In VR applications, the image area is typically a (miniature) reflective screen in front of the user's eye. In AR applications the image area is a semi-transparent combiner which redirects the visible light beam towards the user's eye while concurrently superposing it with a light field from a surrounding. In head mounted displays the mirror assembly may even deflect the scanned visible light beam directly into the user's eye, without any reflective screen or semi-transparent combiner.
To adapt the display apparatus to variable ambient lighting conditions or users' preferences, the brightness of the light beam can be adjusted, e.g., in dependence on a brightness signal measured by an ambient light sensor or provided by user input. Typically, the amplitude of the pulse train is scaled to set the brightness of the display.
However, scaling an electric current for driving the light source to set the brightness level of the display may reach two limits when going to low brightness levels. Firstly, the electric current resolution may not suffice to represent all different colour values at some point. Assuming, e.g., an electric current resolution of 10 μA of current laser drivers, downscaling the maximum pulse amplitude to 2 mA would not allow to represent 256 different colour values, as these would require a maximum pulse amplitude of 256·10 μA=2.56 mA, but only about 200 colour values (200·10 μA=2 mA). Reducing display brightness may thus decrease colour depth, impairing image quality. Secondly, driving a laser light source at a low electric current may push it below a threshold for lasing, where stimulated emission is inhibited, resulting in a lower lasing efficiency, light beam and image quality.
It is an object of the present disclosed subject matter to provide a display apparatus and a method which allow for displaying an image with a high quality even at low brightnesses.
In a first aspect of the disclosed subject matter this object is achieved with a display apparatus as specified at the outset, wherein the pulse generator is configured to receive a brightness signal and to generate the pulse train with a low pulse rate when the brightness signal indicates a low brightness and with a high pulse rate when the brightness signal indicates a high brightness.
The disclosed subject matter breaks with the paradigm of adjusting the display brightness via an amplitude scaling of the pulse train and utilises the pulse rate of the pulse train for brightness adjustment. To this end, the pulse generator receives, e.g., from an ambient light sensor or a user-operated brightness control, the brightness signal indicating the desired brightness and switches the pulse rate between a low pulse rate for a low indicated brightness and a high pulse rate for a high indicated brightness and, optionally, one or more further intermediate pulse rates for intermediate brightness/es.
Utilising the pulse rate for brightness adjustment has several advantages. As the maximum amplitudes of the pulses need not be altered for brightness control the full amplitude range can be used for the desired grayscale or colour space. Even at low brightness a high quality image can be obtained. Moreover, decreasing the pulse rate for low brightness also decreases the rate of retrieving the pixels from the buffer and any image memory upstream of the buffer, reducing buffer, controller and processing loads. Furthermore, the present display apparatus saves energy at low brightness levels not only by emitting less light, but also by reducing necessary processing power. Finally, an electric current for laser driving can be maintained above the lasing threshold, preserving lasing efficiency, light beam quality and hence image quality.
The scan pattern may be a non-raster scan pattern, e.g., a spiral pattern, or even a raster scan pattern. In a beneficial embodiment the scan pattern is a Lissajous pattern, which allows to exploit resonances of the mirrors of the mirror assembly and, hence, to achieve higher speeds of the light beam and higher frame rates with low driving powers.
The synchronisation of the pixel retrieval with the mirror assembly position may be established in many ways. The disclosed subject matter provides for at least two advantageous embodiments.
In a first advantageous embodiment the controller or the buffer is configured to receive a current position of the mirror assembly within the scan pattern from the mirror driver and to retrieve that stored pixel that corresponds to the current position from the buffer for modulating. Thereby, the mirror driver provides the current mirror position within the scan pattern, e.g., within the Lissajous pattern, to the controller or buffer which then identifies the currently needed pixel for the indicated mirror position and retrieves that pixel for modulating. This setup ensures that the pixel used by the controller matches the current MEMS mirror position irrespective of the current pulse rate. Moreover, decreasing the pulse rate for low brightnesses allows the mirror assembly to provide the current position less often while still achieving a tight synchronisation of the pixel retrieval with the mirror assembly position.
In a second advantageous embodiment the display apparatus further comprises a processor configured to hold the image in a memory, to determine a sequence of pixels of the image to be successively displayed according to said scan pattern, and to transfer said sequence of pixels in one or more successive segments to the buffer for storing. Thereby, time-critical real-time components like the mirror driver, the controller and the buffer that need to be exactly synchronised to one another are separated from the processor that may only be loosely synchronised to the real-time components. The processor thus has valuable headroom or “slack” for the task of determining the playout-order of the pixels and transferring the pixels in that order to the buffer. The buffer receives the pixels from the processor already in the correct order, i.e. as they are to be played out according to the scan pattern. The buffer pixels can, thus, be retrieved with a fast sequential contiguous (“linear”) buffer access and quickly. Furthermore, as the buffer stores the pixels in the correct order the pixel retrieval need not be synchronised each time a new pixel is to be played-out but, e.g., only when the play-out of several pixels (a “batch” of pixels) shall be (re-)synchronised to the mirror assembly position. Hence, the mirror assembly may send an optional synchronisation or trigger signal less often, obviating the need for a frequent processing of synchronisation signals and identifying scattered memory addresses when retrieving the pixels from the buffer. The pixels can be retrieved at a higher rate, and the display apparatus is capable to display images with a higher resolution and/or frame rate.
The loose synchronisation of the processor to the real-time components may be established in many ways. For instance, the processor may be synchronised by the mirror driver when the mirror assembly is in a given position, by the buffer when its filling level falls below a predetermined threshold, etc. Optionally, the transferring of said segment/s is triggered by each 1-th pulse of the pulse train. In this way, the pulse generator serves as a “system clock” triggering both the pixel transfer (every 1-th pulse) and retrieval (every pulse) and, thus, synchronising the time-critical components to the processor. Thereby, decreasing the pulse rate for low brightness automatically results in lowering the retrieval and transferring rate and, due to the latter, in more time available for the processor to determine the sequence of pixels. Thus, at a low brightness, the processor, the pulse generator, the buffer and the controller are less busy and the display apparatus requires even less electric power.
In a further embodiment the processor is configured to determine the sequence of pixels in successive parts. In this way, the processor determines the pixels of the sequence of pixels at several instances of time, e.g., one part every k-th clock cycle of the processor or every 1-th pulse of the pulse train. Hence, between each two of those instances the processor may perform other tasks like adapting the pixels of the image to dynamically correct for geometrical distortions, e.g., due to a change in image area geometry. To this end, each part optionally comprises at least one segment such that pixels of a determined part may be promptly transferred to the buffer, e.g., within the same or the next processor clock cycle of its determination.
The processor may determine the sequence of pixels based on an on-the-fly calculation of the scan pattern. For a particularly fast determination the processor is optionally configured to store a look-up table of indices of the pixels to be successively displayed according to said scan pattern and to determine the sequence of pixels by retrieving the pixels according to the look-up table from the memory. The processor can easily and quickly determine the sequence of pixels by accessing the look-up-table to obtain the indices of the pixels and then the memory to retrieve the pixels according to the indices.
In a favourable embodiment the high pulse rate is at least twice of the inverse of the shortest duration of stay of the light beam in one pixel of said image when scanning the light beam according to the scan pattern. Employing such a high pulse rate guarantees that each pixel of the image, including the pixel with the shortest duration of stay of the light beam, is displayed by at least two successive pulses. This creates the necessary headroom that at low pulse rates each pixel of the image may be displayed by at least one pulse, preserving image resolution at low brightnesses.
On the other hand, the low pulse rate may be less than the inverse of the shortest duration of stay of the light beam in one pixel of said image when scanning the light beam according to the scan pattern, leading to intentionally left out pixels (“holes”) within one run of the scan pattern. This allows for particularly low brightnesses and a particularly low power consumption, albeit at the cost of image resolution.
Image resolution can, however, be preserved when the pixels are interlaced over successive runs of the scan pattern. To this end, in an optional variant of this embodiment the low pulse rate is an m-th of the inverse of the duration of stay of the light beam in one pixel of said image when scanning the light beam according to the scan pattern, and m successive sections of the pulse train each containing pulses of a respective scan of the scan pattern are mutually offset in time with respect to the beginning of the respective scan of the scan pattern by an integer multiple n of an m-th of the inverse of the low pulse rate, n modulo m being unequal to zero. In this variant, every m-th pixel traversed by the light beam is displayed within each scan (“run”) of the scan pattern, and the mutual time offset of the m sections guarantees that each scan and section displays different pixels. The pixels of different scans of the scan pattern are then interlaced and the image is completed at full resolution after m runs of the scan pattern, all that with a particularly low brightness and full greyscale or colour space. Of course, when the duration of stay of the light beam in one pixel varies within one run of the scan pattern, i.e., when the light beam spends a different amount of time in different pixels, the low pulse rate may vary accordingly to achieve the m-fold interlacing.
This variant may advantageously be combined with the above-mentioned processor when the latter is further configured to transfer at least one segment per scan of the scan pattern. Each segment may then comprise, e.g., all, a half, a third, etc. of the pixels for the modulation of the current or following section of the pulse train. The buffer may store only the pixels of one, two, three, etc. small segments and, hence, be particularly small and fast accessible. Optionally, the processor may also determine the sequence section-wise such that each of the above-mentioned parts corresponds to one section of the pulse train or a unit fraction thereof.
In some embodiments the image may be displayed by a mono-coloured light beam. For displaying a multi-coloured image in optional embodiments each pixel has two or more colours, e.g. the colours red, green and blue, the light source comprises for each of said colours a sub-light source configured to emit a respective sub-light beam of the respective colour, and the controller is configured to perform said modulating and driving separately for each of said colours.
In a second aspect the disclosed subject matter provides a method for displaying pixels of an image, comprising
The method displays pixels of the image adjustably at a low or a high brightness by setting a low or high pulse rate, respectively. For further advantages and advantageous embodiments of the method, reference is made to the advantages and embodiments described above with respect to the display apparatus of the disclosed subject matter.
The disclosed subject matter will now be described by means of exemplary embodiments thereof with reference to the enclosed drawings, in which show:
The image 2 may be part of a movie M, be a single image, e.g., a photo to be displayed for a longer period of time, part of a larger image, etc. Instead of a wall 3, the display apparatus 1 could display the light beam 4 onto any kind of image area, such as a board, projection screen, poster, the retina of an eye, an augmented reality (AR) combiner waveguide, another combiner optics, or the like. Accordingly, the display apparatus 1 may be part of a projector, AR or VR (virtual reality) glasses, a helmet, a head-up display, etc.
With reference to
The mirror assembly 7 may either comprise one MEMS mirror 8 oscillating about the horizontal and vertical axes 10, 11 or two MEMS mirrors 8, one after the other in the optical path of the light beam 4, each of which MEMS mirrors 8 then oscillating about one of the horizontal and vertical axes 10, 11.
Depending on the Lissajous pattern 5 to be displayed, Th and Tv may be chosen such that the trajectory of the light beam 4 on the image plane 2 densely covers the entire image plane 2 during a period Tfr of one image frame 2. Such a “complex” or “dense” Lissajous pattern 5 can be achieved when the frequencies fh=1/Th, fv=1/Tv are greater than the frame rate ffr=1/Tfr, e.g., greater than 1 kHz or tens of kHz, and the beginnings of their respective oscillation periods meet, e.g., only over every one or more images 2, in particular when the frequencies fh, fv are close to each other. To this end, frequencies fh, fv with a small greatest common divisor, e.g. smaller than 10, may be employed, for example.
The light source 6 may be any light source known in the art, e.g., an incandescent lamp, a gas, liquid or solid laser, a laser diode, a vertical-cavity surface-emitting laser (VCSEL), an LED, an SLED, etc. The light source 6 is driven by a controller 12 according to the pixels Pi of the image 2. To this end, the controller 12 receives, on the one hand, a train 13 of pulses 14j that is generated by a pulse generator 15 of the display apparatus 1, and, on the other hand, the pixels Pi of the image 2 that are stored in a buffer 16 of the display apparatus 1. The controller 12 may receive the stored pixels Pi either by retrieving the pixels Pi itself from the buffer 16, e.g. when the controller 12 is part of the buffer 16, or by obtaining them from a buffer controller, e.g. when the controller 12 is separate from the buffer 16. The pixel retrieving is synchronised with the pulses 14j of the pulse train 13.
To synchronise the pixel retrieving with the mirror movement, in the embodiment of
According to the received pixels Pi, i.e. according to the colour of each received pixel Pi, the controller 12 modulates the pulse train 13 in pulse amplitude A (
In case the light source 6 displays a mono-colour, black and white, or grey scale image 2 with a mono-coloured light beam 4, each pixel Pi comprises a single colour, e.g., a brightness or intensity value, and the controller 12 drives the light source 6 according to one modulated train 17. In case the light source 6 displays a multi-colour image 2 with a multi-coloured light beam 4 having a sub-light beam emitted by a sub-light source for each colour, each pixel Pi comprises several colours, e.g., RGB values indicating the brightness or intensity of a red, green, and blue colour, YPbPr values, etc., and the controller 12 drives each sub-light source according to a respective modulated train 17 (not shown).
The display apparatus 1 utilises the pulse rate PR, i.e. the inverse pulse time spacing Δt, also known as “pulse repetition rate” or “pulse frequency”, employed by the pulse generator 15 when generating the pulse train 13 to adjust the brightness of the pixels Pi displayed on the image area 3. The pulse rate PR may be uniform throughout one scan of the scan pattern 5, or varying locally to account for unequal mirror velocities within the scan pattern 5 and thus durations of stay of the light beam 4 in different pixels Pi of the image 2. With reference to
As can be seen in
In the example of
After the first interval T1 has lapsed in the example of
Comparing the amplitudes A, e.g., of the pulses 181 and 1861 used for displaying pixel P6, it is apparent that the amplitudes need not be reduced to achieve the low brightness (albeit they could, e.g. for further brightness reduction). Consequently, the full possible range of amplitudes A may be utilised to display pixel colours.
In the example of
Of course, other values of the high and/or low pulse rates PRh, PRl may be chosen, e.g., as detailed below. Moreover, depending on the required brightness granularity the pulse generator 15 may generate the train 13 with one or more intermediate pulse rates PRi when the brightness signal B indicates a respective intermediate brightness.
The processor 21 may determine the sequence 23 of pixels Pi in any time granularity, e.g., for each image 2 at once or successively in subsequent parts 231, 232, . . . , generally 23p. Each part 23p may comprise one or more segments 24. Moreover, the processor 21 may determine the sequence 23 in many ways, e.g., on-the-fly by matching positions that follow each other in time along the scan pattern 5 to pixels Pi in the image 2 occurring at these positions, or by means of a look-up table as described below with reference to
The transfer of the segment/s 24 may be triggered by each 1-th pulse 14j of the pulse train 13, i.e. the pulse generator 15 may serve as a “system clock” triggering both the pixel retrieval, modulation and display as well as the pixel transfer from the processor 21 to the buffer 16. A pulse counter 25 may count 1 pulses 14j each and then trigger the transfer. The number 1 may, e.g., be the number of pixels Pi of the image 2 or a half, a third, etc. of that number, to transfer one segment 24 per whole, half, third, etc. image (“frame”) 2.
Alternatively, the segment transfer may be triggered by the mirror driver 9 (see broken arrow 26) to transfer segments 24 in synchronicity with the mirror position POS, e.g., every vertical or horizontal oscillation period Tv, Th, or by the buffer 16 each time a filling level of the buffer 16 falls below a predetermined threshold (see arrow 27), or every k-th cycle of the clock of the processor 21, etc.
An interlacing of the pixels Pi for achieving a particularly low brightness shall now be described with reference to
Here, the low pulse rate PRl is less than the inverse of the shortest duration of stay of the light beam 4 in one pixel Pi of the image 2 when the light beam 4 is scanned across the image 2 according to the scan pattern 5 (here: the pixel P6). Hence, the light pulses 19j within one scan (run) of the scan pattern 5 do not display each pixel Pi but leave out pixels, i.e. leave “holes”, see the pixels Pi with an empty circle in the first scan or run of the scan pattern 5 in
In the embodiment shown in
In a first time interval T1′ of
To fill the holes (empty circles) by interlacing, in a second time interval T2 of
The interlacing of the second pixel succession with the first pixel succession on the image area 3 is achieved by offsetting the pixel sections S1, S2 of the pulse train 17 with respect to the beginning tb of the respective first and second scans of the scan pattern 5, e.g., by offsetting the pulse train 13 by the pulse generator 15 and/or the modulated pulse train 17 by the controller 12 and/or the mirror movement by the mirror driver 9, etc. In the example of
Within the first scan of the scan pattern 5 during the first time interval T1′ only the pixels Pi of the first section S1 are required in the buffer 16 for modulating the pulse train 13, and within the second scan of the scan pattern 5 during the second time interval T2′ only the pixels Pi of the second section S2 are required in the buffer 16 for modulating the pulse train 13. The processor 21 may thus transfer in (at least) one segment 24 per scan (run) of the scan pattern 5 only those pixels Pi that are actuals required in the near future. For instance, the processor 21 may transfer the pixels Pi of the next scan of the scan pattern 5 at the end of each current scan of the scan pattern 5.
Instead of a “two-fold” interlacing of two sections S1, S2 as shown in
Optionally, in an embodiment with an m-fold interlacing, the processor 21 may transfer (at least) one segment 24 per scan of the scan pattern 5 such that the buffer 16 only needs to store the pixels Pi of the current (optionally: the current and one or more subsequent) scans of the scan pattern 5.
The look-up table 28 reproduces the Lissajous pattern 5 running over the image 2 and holds, for each of the pixels Pi as they are subsequently passed (scanned) by the scan pattern 5, the corresponding pixel index i, here: the indices of the pixels Pi of the first section S1 or first succession in a first sub-table 281 and the indices of the pixels Pi of the second section S2 or second succession in a second sub-table 282. Hence, the processor 21 can determine the sequence 23 of pixels Pi by retrieving, pixel-for-pixel, the pixels Pi from the memory addresses indicated by the indices i of the look-up table 28.
In the example of
In a first step 32 the pulse generator 15 receives the brightness signal B, e.g. from a user input or the brightness sensor 20.
In a second step 33 the pulse generator 15 generates the pulse train 13 with the low pulse rate PRl when the brightness signal B indicates a low brightness or with the high pulse rate PRh when the brightness signal B indicates a high brightness.
In a third step 34 the pixels Pi of the image 2 are stored in the buffer 16.
In a fourth step 35 the controller 12 successively receives the pixels Pi stored in the buffer 16, e.g. by retrieving the pixels Pi itself or by being fed by a buffer controller of the buffer 16.
As shown in
In a fifth step 36 the controller 12 modulates the pulse train 13 in pulse amplitude and/or pulse width according to the received pixels Pi.
In a sixth step 37 the controller 12 drives the light source 6 according to the modulated pulse train 17 to emit the light beam 4.
Concurrently, in step 38 the mirror driver 9 drives the mirror assembly 7 to oscillate according to the scan pattern 5, so that the mirror assembly 7 scans the light beam 4 over the image area 3 according to the scan pattern 5 (step 39).
Optionally, step 32 may be repeated to receive at least one further brightness signal B and, in case the brightness signal B indicates a different brightness, to generate the pulse train 13 with a different pulse rate PR in step 33.
The disclosed subject matter is not restricted to the specific embodiments described above but encompasses all variants, modifications and combinations thereof that fall within the scope of the appended claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 23180876.7 | Jun 2023 | EP | regional |
| 23180877.5 | Jun 2023 | EP | regional |
| 23196195.4 | Sep 2023 | EP | regional |
| 23214645.6 | Dec 2023 | EP | regional |
