Patent Grant
12167829
This application is a § 371 National Stage Application of PCT International Application No. PCT/EP2020/086131 filed Dec. 15, 2020, which claims the priority of Danish Patent Application No. PA201970799, filed Dec. 19, 2019, each of which are herein incorporated by reference in their entirety.
The present disclosure relates to an endoscope and an endoscope system, more specifically to a method and an endoscope system configured to capture a still image of enhanced quality.
A video feed, e.g. comprising a plurality of frames, from an endoscope may in some instances be transmitted to a processing device, sometimes referred to as a monitor, having an output, e.g. HDMI or SDI, which allow transmission of the received video feed to an external device, such as a video-/image-grabber or an external display, such as a HD-display. A user may choose to grab a still image or a sequence of frames from the video feed by using an external device, such as a video-/image-grabber connected to an output port of the monitor of the endoscope system.
Although, the processing device may do image processing on the video feed and/or frames received from the endoscope, e.g. by applying various filters to the frames and/or video feed and output a processed video feed and/or frames as the output to external devices, in the context of the present disclosure a video feed and/or frames received from the endoscope are referred to as the same video feed and/or frame regardless of having been subject to image processing.
The processing device and/or an external video-/image-grabber may further allow capturing of still images of the received video feed, e.g. by a push of a control button.
It is an object of the present disclosure to provide a solution which at least improve the solutions of the prior art.
It is a further object of the present disclosure to provide a system and method facilitating enhanced quality of captured still images, as well as providing for enhanced output of captured still images.
Accordingly, an endoscope system is disclosed, such as an endoscope system comprising an image sensor and a processing device. The image sensor comprises an image sensor output. The processing device comprises an input connected to the image sensor output. The input of the processing device may be connected to the image sensor output by a wireless connection.
The processing device is configured to: receive, such as consecutively receive, at a first frame rate, via the input, frames captured by the image sensor; receive a first capture signal indicative of an operator performing a first action associated with capturing a still image of the frames received via the input; store, such as temporarily store, a first plurality of frames of the frames received via the input; determine one or more quality properties of each of the first plurality of frames; and select a designated frame of the first plurality of frames based on the one or more quality properties.
Also disclosed is a method for operating an endoscope system, such as the disclosed endoscope system, such as an endoscope system comprising an image sensor and a processing device.
The method comprises: consecutively receiving frames captured by the image sensor; receiving a first capture signal, indicative of an operator performing a first action associated with capturing a still image of the frames captured by the image sensor; storing a first plurality of frames of the frames captured by the image sensor; determining one or more quality properties of each of the first plurality of frames; and selecting a designated frame of the first plurality of frames based on the one or more quality properties.
It will be understood that the frames received may not be the raw input captured by the image sensor electronics but may have been subject to some initial image processing prior to being received.
The one or more quality properties may comprise one or more of sharpness, contrast, and noise. The processing device may be configured to perform a Laplacian operation and/or a Laplacian of Gaussian operation to determine the one or more quality properties of each of the first plurality of frames.
The first plurality of frames may be stored, such as temporarily stored, after receipt of the first capture signal. For example, the first plurality of frames may be stored in response to receipt of the first capture signal. For example, the first plurality of frames may be frames received via the input after receiving the first capture signal and/or captured by the image sensor after receiving the first capture signal. Storing a plurality of frames received via the input after receiving the first capture signal provides for a simple solution avoiding the need for continuously storing captured images for later selection in a buffer.
The first action may correspond to the operator initiating a press of a capture button. Thus, the first capture signal may be indicative of the operator pressing the capture button. For example, the first action may correspond to the operator initiating the press of the capture button, such as a press down on the capture button. Thus, the first capture signal may be indicative of the operator providing a press down on the capture button.
After selection of the designated frame of the first plurality of frames, the first plurality of frames, e.g. except the designated frame, may be deleted or discarded.
A second capture signal may be received after receipt of the first capture signal. For example, the processing device may be configured to receive a second capture signal after receipt of the first capture signal. The second capture signal may be indicative of the operator performing a second action associated with capturing the still image of the frames received via the input. For example, the second action may correspond to the operator releasing the press of the capture button.
The first plurality of frames may be stored, such as temporarily stored, after receipt of the second capture signal. For example, the first plurality of frames may be stored in response to receipt of the second capture signal. For example, the first plurality of frames may be frames received via the input after receiving the second capture signal.
The first plurality of frames may comprise frames received via the input between receipt of the first capture signal and receipt of the second capture signal. The first plurality of frames may consist of frames received via the input between receipt of the first capture signal and receipt of the second capture signal.
The first plurality of frames may be a first number of frames. The first number of frames may be more than 5 frames, such as 10 frames. The first frame rate may be between 20-40 frames per second (fps), such as 30 fps. The first number of frames may correspond to a time span of between 200-500 ms, such as between 300-350 ms, such as approximately 333 ms.
In accordance with more than the first number of frames being received via the first input between receipt of the first capture signal and receipt of the second capture signal, the first plurality of frames may consist of the first number of frames received via the first input immediately prior to receipt of the second capture signal. Thus, in case the time between the first capture signal and the second capture signal is more than the time span of the first number of frames, the first plurality of frames may be selected to be the frames received up to the time of receipt of the second capture signal.
Alternatively, in accordance with more than the first number of frames being received via the first input between receipt of the first capture signal and receipt of the second capture signal, the first plurality of frames may consist of the first number of frames received via the first input immediately after receipt of the first capture signal. Thus, in case the time between the first capture signal and the second capture signal is more than the time span of the first number of frames, the first plurality of frames may be selected to be the frames received immediately after the time of receipt of the first capture signal.
In accordance with less than the first number of frames being received via the first input between receipt of the first capture signal and receipt of the second capture signal, the first plurality of frames may consist of the first number of frames received via the first input after receipt of the first capture signal. Hence, collection of frames to store, such as temporarily store, as the first plurality of frames may be continued after receipt of the second capture signal, in case the time between the first capture signal and the second capture signal is less than the time span needed to receive the first number of frames.
The designated frame may be output via a first output. The processing device may comprise the first output. The first output may be configured to be connected to an external image processing device, such as a video grabber. The processing device may be configured to output the designated frame via the first output. Thereby, the processing device may be configured to output the best available frame of the first plurality of frames via the first output.
The frames received via the input and/or captured by the image sensor may be output via the first output and/or a second output. The processing device may comprise a second output. The second output may be configured to be connected to an external display device, such as an external monitor or HD-display. The processing device may be configured to output, via the second output, the frames received via the input. Thus, the output provided via the second output may be substantially a relay of the frames received and captured by the image sensor. However, it will be understood that in outputting the frames received via the input some image processing may be performed on the received frames in order to output comprehensible frames.
Prior to receiving the first capture signal and/or the second capture signal, the frames received via the input and/or captured by the image sensor may be output via the first output and/or via the second output. After receipt of the first capture signal and/or the second capture signal, such as in response to receiving the first capture signal and/or the second capture signal, the designated frame may be output, such as continuously and/or repeatedly output, via the first output, and/or the frames captured by the image sensor may be output via the second output.
The processing device may be configured to change between a first state and a second state, such as from the first state to the second state. The processing device may be configured to change from the first state to the second state after receipt of the first capture signal and/or after receipt of the second capture signal, such as in response to receipt of the first capture signal and/or in response to receipt of the second capture signal.
In the first state the processing device may be configured to output, via the first output, the frames received via the input. Thus, the output provided via the first output, when the processing device is in the first state, may be substantially a relay of the frames received and captured by the image sensor. However, it will be understood that in outputting the frames received via the input some image processing may be performed on the received frames in order to output comprehensible frames.
In the second state the processing device may be configured to output, e.g. continuously output and/or repeatedly output, via the first output, the designated frame. Thereby, a “best” frame may be provided to an external device, e.g. for storing in a central image database.
In the first state the processing device may be configured to output, via the second output, the frames received via the input. In the second state the processing device may be configured to output, via the second output, the frames received via the input. For example, the processing device may be configured to output, via the second output, the frames received via the input, regardless of whether the processing device is in the first state or in the second state. Thus, continuous performance of the procedure may be allowed while providing a designated frame to an external device, e.g. for storing in a central image database.
The processing device may be configured to change from the second state to the first state after a predetermined time after changing from the first state to the second state. The predetermined time may be more than 5 seconds. The predetermined time may be less than 30 seconds. The predetermined time may be between 5-20 seconds, such as 5 seconds or 10 seconds.
The processing device may be configured to change from the second state to the first state in response to receipt of a third capture signal, e.g. indicative of an operator performing a third action. The third action may correspond to the operator initiating a second press of the capture button, such as a second press down on the capture button. Thus, the third capture signal may be indicative of the operator providing a second press down on the capture button, e.g. after receipt of the first and/or second capture signal and/or after the operator has performed the first and/or second action. Alternatively, the third action may correspond to the operator initiating an input on the capture button in another direction, e.g. a perpendicular direction, than the direction of the input on the capture button corresponding to the first and/or second actions.
The endoscope system may comprise a display unit. The processing device may be configured to display the designated frame via the display unit. The processing device may be configured to display the frames received via the input concurrently with display of the designated frame.
The endoscope system may comprise an endoscope. The endoscope may have an endoscope handle and an elongated flexible member extending from the endoscope handle to a distal end. The image sensor may be positioned at the distal end. Alternatively, the image sensor may be positioned distant from the distal end, e.g. in the handle, and fibreoptic in the elongated flexible member may allow transmission of light from the distal end of the elongated flexible member to the image sensor.
The processing device may be configured to store the designated frame to a memory, e.g. to an internal memory of the processing device and/or to an external memory. For example, the processing device may be configured to cause storing of the designated frame to an external memory. For example, the processing device may issue a signal to an external device, such as an image grabber, to cause the external device to store the designated frame.
It is an advantage of the disclosure that it makes it easier for the user to grab still images and reduces the need for repeatably recapturing a still image because of bad image quality, such as blurring, which may be caused, e.g. by movement of the camera.
It is a further advantage of the disclosure that an enhanced quality image may be provided to certain external devices, while a live image view may be preserved via another connected device.
Embodiments of the disclosure will be described in more detail in the following with regard to the accompanying figures. The figures show one way of implementing the present disclosure and are not to be construed as being limiting to other possible embodiments falling within the scope of the attached claim set.
Various exemplary embodiments and details are described hereinafter, with reference to the figures when relevant. It should be noted that the figures may or may not be drawn to scale and that elements of similar structures or functions are represented by like reference numerals throughout the figures. It should also be noted that the figures are only intended to facilitate the description of the embodiments. They are not intended as an exhaustive description of the disclosure or as a limitation on the scope of the disclosure. In addition, an illustrated embodiment needs not have all the aspects or advantages shown. An aspect or an advantage described in conjunction with a particular embodiment is not necessarily limited to that embodiment and can be practiced in any other embodiments even if not so illustrated, or if not so explicitly described.
The image sensor 4 may be a digital camera unit, e.g. positioned at a distal end 56 of an elongated member 54 of an endoscope 50, as illustrated in
The image sensor 4 comprises an image sensor output 8. The processing device 6 comprises an input 10. The input 10 is connected to the image sensor output 8. The processing device 6 comprises a processing unit 7, e.g. comprising a CPU, GPU or FPGA. The processing device 6 comprises memory 19.
The endoscope system comprises a display unit 18. The display unit 18 may form part of the processing device 6, as illustrated. The display unit 18 may be a touch sensitive display.
The processing device 6 comprise a first output 12 and a second output 14. The first output 12 may be configured to be connected to an external image processing device 20, such as a video grabber. The second output 14 may be configured to be connected to an external display device 24.
The endoscope system 2 comprises a capture button 16 configured to receive an operator input, such as a touch input. The capture button 16 may be a soft button provided on a touch sensitive display 18 of the processing device 6. Alternatively or additionally, a capture button may be provided on the endoscope handle 52 of the endoscope 50.
The processing device 6 is configured to receive a frame 100 of a video feed from the image sensor 4 via the input 10. For example, the processing device 6 may consecutively receive, e.g. at a first frame rate, via the input 10, frames 100 captured by the image sensor 4. The frames 100 may form a video feed. The frames 100 and/or the video feed may form part of an input signal.
The processing device 6 is further configured to receive a first capture signal 102, e.g. indicative of an operator performing a first action associated with capturing a still image of the received frames 100.
The first capture signal 102 may be received from the capture button 16. The processing device 6 may further receive a second capture signal 104, e.g. indicative of an operator performing a second action associated with capturing the still image of the received frames 100. The second capture signal 104 may be configured to be received after receipt of the first capture signal 102. For example, the first action may correspond to a press (e.g. down press) of the capture button 16 and the second action may correspond to release of the press on the capture button 16.
The processing device 6 is configured to store, such as temporarily store, e.g. in the memory 19, a first plurality of frames, e.g. five frames or 10 frames, of the frames 100 received via the input 10. The first plurality of frames may be stored after receipt of the first capture signal 102, e.g. the first plurality of frames may be stored in response to receipt of the first capture signal 102, e.g. receipt of the first capture signal 102 may trigger storing of a first plurality of frames. The first plurality of frames may be frames received via the input after receiving the first capture signal 102. For example, the first plurality of frames may comprise, or consist of, frames received via the input 8 between receipt of the first capture signal 102 and receipt of the second capture signal 104.
The processing device 6, such as the processing unit 7, is configured to determine one or more quality properties of each of the first plurality of frames and select a designated frame of the first plurality of frames based on the one or more quality properties. The one or more quality properties may be sharpness, contrast, and/or noise.
The processing device 6 may cause storing of the designated frame to an external memory (not shown).
The processing device 6 may be configured to output a first output signal 108 via the first output 12, e.g. to the connected external image processing device 20. The first output signal 108 may comprise the designated frame. The processing device 6 may be configured to output a second output signal 110, via the second output 14. The second output signal 110 may comprise the frames 100 received via the input. For example, the second output signal 110 may substantially correspond to the video feed and/or the input signal received from the image sensor 4.
The processing device 6 may be configured to change between a first state and a second state, e.g. in response to receipt of the first capture signal 102 and/or the second capture signal 104. For example, the processing device 6 may be configured to change from the first state to the second state after, such as in response to, receipt of the first capture signal 102 and/or the second capture signal 104. In the first state the processing device 6 may be configured to output, via the first output 12, the frames 100 received via the input 10, and in the second state the processing device 6 is configured to output, via the first output 12, the designated frame. Thus, the output signal 108 via the first output 12 may change based on user input, e.g. based on receipt of the first capture signal 102 and/or receipt of the second capture signal 104.
The processing device 6 may be configured to output, via the second output 14, the frames 100 received via the input 10, both in the first state and in the second state. Thus, the output signal 110 via the second output 14 may be unaffected by the receipt of the first capture signal 102 and/or receipt of the second capture signal 104.
The processing device 6 may be configured to change back to the first state, e.g. from the second state to the first state, after a predetermined time after changing from the first state to the second state. Alternatively, or additionally, the processing device 6 may be configured to change back to the first state, e.g. from the second state to the first state, in response to receipt of a third capture signal 106, e.g. from the capture button 16 (e.g. a press down on the capture button 16, while the processing device is in the second state) or alternatively from another button. The third capture signal 106 may be indicative of an operator performing a third action. Thus, upon receipt of the third capture signal 106, the processing device 6 may be configured to change back to the first state, wherein the first output signal 108 of the first output 12 and the second output signal 110 of the second output 14 corresponds to the frames 100 received via the input 10.
The processing device 6 may be configured to display the designated frame via the display unit 18, e.g. in the first state and/or in the second state. The processing device 6 may be configured to display the frames 100 received via the input 10 concurrently with display of the designated frame. For example, the designated frame may be shown in a first portion of the display unit 18 and the frames 100 received via the input 10 may be displayed in a second portion of the display unit 18.
The method 200 comprises receiving 202, such as consecutively receiving, frames captured by the image sensor.
The method 200 comprises receiving 204 a first capture signal. The first capture signal may be indicative of an operator performing a first action associated with capturing a still image of the frames captured by the image sensor.
The method 200 comprises storing 206 a first plurality of frames of the frames captured by the image sensor. Storing 206 the first plurality of frames may be performed after receipt 204 of the first capture signal, e.g. in response to receipt 204 of the first capture signal. Thus, the first plurality of frames may be frames captured by the image sensor after receiving 204 the first capture signal.
The method 200 optionally comprises receiving 208 a second capture signal. The second capture signal may be indicative of an operator performing a second action associated with capturing the still image of the frames captured by the image sensor.
The method 200 comprises determining 210 one or more quality properties of each of the first plurality of frames. Determining 210 the one or more quality properties may be performed in response to receipt 208 of the second capture signal. Alternatively, determining 210 the one or more quality properties may be performed in response to a predetermined number of frames being stored 206.
The method 200 comprises selecting 212 a designated frame of the first plurality of frames based on the one or more quality properties. For example, the designated frame may be the frame of the first plurality of frames having a highest quality score, such as less noise, highest contrast, or similar. The quality score may be a combination of several factors.
The method 200 optionally comprises outputting 214 the designated frame via a first output. The designated frame may be output 214 via the first output after receipt 204 of the first capture signal and/or after receipt 208 of the second capture signal. For example, the designated frame may be output 214 via the first output in response to receipt 204 of the first capture signal and/or in response to receipt 208 of the second capture signal.
The method 200 optionally comprises outputting 216 the frames captured by the image sensor via a second output. The designated frame may be output 214 via the first output simultaneously with the frames captured by the image sensor being output 216 via the second output.
Prior to receiving 204 the first capture signal, the method 200 optionally comprises outputting 203a, via the first output, the frames captured by the image sensor, and outputting 203b, via the second output, the frames captured by the image sensor. Thus, prior to receiving 204 the first capture signal, the method 200 may comprise outputting the frames captured by the image sensor on both the first output and the second output.
Exemplary systems and methods are disclosed in the following items:
1. An endoscope system comprising an image sensor and a processing device, the image sensor comprising an image sensor output, the processing device comprising an input connected to the image sensor output, the processing device is configured to:
2. Endoscope system according to item 1, wherein the first plurality of frames is stored after receipt of the first capture signal, and wherein the first plurality of frames are frames received via the input after receiving the first capture signal.
3. Endoscope system according to any of the preceding items, wherein the first action corresponds to the operator initiating a press of a capture button.
4. Endoscope system according to any of the preceding items, wherein the processing device is configured to receive a second capture signal after receipt of the first capture signal, wherein the second capture signal is indicative of the operator performing a second action associated with capturing the still image of the frames received via the input.
5. Endoscope system according to item 4 as dependent on item 3, wherein the second action corresponds to the operator releasing the press of the capture button.
6. Endoscope system according to any of items 4-5, wherein the first plurality of frames comprises frames received via the input between receipt of the first capture signal and receipt of the second capture signal.
7. Endoscope system according to any of the preceding items, wherein the first plurality of frames is a first number of frames, optionally wherein the first number of frames are more than 5 frames, such as 10 frames.
8. Endoscope system according to item 7 as dependent on any of items 4-6, wherein, in accordance with more than the first number of frames being received via the first input between receipt of the first capture signal and receipt of the second capture signal, the first plurality of frames consists of the first number of frames received via the first input immediately prior to receipt of the second capture signal.
9. Endoscope system according to any of items 7-8 as dependent on any of items 4-6, wherein, in accordance with less than the first number of frames being received via the first input between receipt of the first capture signal and receipt of the second capture signal, the first plurality of frames consists of the first number of frames received via the first input after receipt of the first capture signal.
10. Endoscope system according to any of the preceding items, wherein the processing device comprises a first output configured to be connected to an external image processing device, and the processing device is configured to output the designated frame via the first output.
11. Endoscope system according to item 10, wherein the external image processing device is a video grabber.
12. Endoscope system according to any of the preceding items, wherein the processing device comprises a second output, and the processing device being configured to output, via the second output, the frames received via the input.
13. Endoscope system according to any of items 10-12, wherein the processing device is configured to change from a first state to a second state after receipt of the first capture signal, wherein in the first state the processing device is configured to output, via the first output, the frames received via the input, and in the second state the processing device is configured to output, via the first output, the designated frame.
14. Endoscope system according to item 13 as dependent on item 12, wherein in the first state the processing device is configured to output, via the second output, the frames received via the input, and in the second state the processing device is configured to output, via the second output, the frames received via the input.
15. Endoscope system according to any of items 13-14, wherein the processing device is configured to change from the second state to the first state after a predetermined time after changing from the first state to the second state.
16. Endoscope system according to any of items 13-15, wherein the processing device is configured to change from the second state to the first state in response to receipt of a third capture signal, indicative of an operator performing a third action.
17. Endoscope system according to any of the preceding items, wherein the endoscope system comprises a display unit, and wherein the processing device is configured to display the designated frame via the display unit.
18. Endoscope system according to item 17, wherein the processing device is configured to display the frames received via the input concurrently with display of the designated frame.
19. Endoscope system according to any of the preceding items, wherein the one or more quality properties comprise one or more of sharpness, contrast, and noise.
20. Endoscope system according to any of the preceding items, wherein the processing device is configured to perform a Laplacian operation and/or a Laplacian of Gaussian operation to determine the one or more quality properties of each of the first plurality of frames.
21. Endoscope system according to any of the preceding items, wherein the first frame rate is between 20-40 fps, such as 30 fps.
22. Endoscope system according to any of the preceding items comprising an endoscope with an endoscope handle and an elongated flexible member extending from the endoscope handle to a distal end, wherein the image sensor is positioned at the distal end.
23. Endoscope system according to any of the preceding items, wherein the processing device is configured to store the designated frame to a memory.
24. Endoscope system according to any of the preceding items, wherein the processing device is configured to cause storing of the designated frame to an external memory.
25. A method for operating an endoscope system comprising an image sensor and a processing device, the method comprising:
26. Method according to item 25, wherein storing the first plurality of frames is performed after receipt of the first capture signal, and wherein the first plurality of frames are frames captured by the image sensor after receiving the first capture signal.
27. Method according to any of items 25-26 comprising outputting the designated frame via a first output.
28. Method according to any of items 25-27 comprising outputting the frames captured by the image sensor via a second output.
29. Method according to any of items 25-28 comprising prior to receiving the first capture signal:
The present disclosure has been described with reference to preferred embodiments. However, the scope of the disclosure is not limited to the illustrated embodiments, and alterations and modifications can be carried out without deviating from the scope of the disclosure.
Throughout the description, the use of the terms “first”, “second”, “third”, “fourth”, “primary”, “secondary”, “tertiary” etc. does not imply any particular order or importance, but are included to identify individual elements. Furthermore, the labelling of a first element does not imply the presence of a second element and vice versa.
| Number | Date | Country | Kind |
|---|---|---|---|
| PA201970799 | Dec 2019 | DK | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/EP2020/086131 | 12/15/2020 | WO |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2021/122536 | 6/24/2021 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 6533721 | Beutter | Mar 2003 | B1 |
| 6549643 | Toklu et al. | Apr 2003 | B1 |
| 6717609 | Sugimoto et al. | Apr 2004 | B2 |
| 7048685 | Sakiyama | May 2006 | B2 |
| 7184100 | Wilf et al. | Feb 2007 | B1 |
| 7492388 | Odlivak et al. | Feb 2009 | B2 |
| 7534205 | Wiklof | May 2009 | B2 |
| 7869856 | Refael | Jan 2011 | B2 |
| 7995045 | Dunki-Jacobs | Aug 2011 | B2 |
| 8199188 | Amling et al. | Jun 2012 | B2 |
| 8199813 | Bendall et al. | Jun 2012 | B2 |
| 8253785 | Abe | Aug 2012 | B2 |
| 8723935 | Kouno | May 2014 | B2 |
| 9007450 | Amling et al. | Apr 2015 | B2 |
| 9030543 | Tsuyuki et al. | May 2015 | B2 |
| 9084529 | Saito | Jul 2015 | B2 |
| 9167160 | King | Oct 2015 | B2 |
| 9289113 | Ozawa | Mar 2016 | B2 |
| 9332890 | Ozawa | May 2016 | B2 |
| 9672619 | Yaguchi | Jun 2017 | B2 |
| 9740939 | Yaguchi | Aug 2017 | B2 |
| 9913575 | Gazdzinski | Mar 2018 | B2 |
| 10004389 | Shigeta | Jun 2018 | B2 |
| 10028645 | Gazdzinski | Jul 2018 | B2 |
| 10028646 | Gazdzinski | Jul 2018 | B2 |
| 10250853 | Yanagidate | Apr 2019 | B2 |
| 10980397 | Ubbesen | Apr 2021 | B1 |
| 11074690 | Jørgensen | Jul 2021 | B2 |
| 11096553 | Sonnenborg | Aug 2021 | B2 |
| 20030120156 | Forrester | Jun 2003 | A1 |
| 20050129108 | Bendall et al. | Jun 2005 | A1 |
| 20050219666 | Ejima et al. | Oct 2005 | A1 |
| 20060015008 | Kennedy | Jan 2006 | A1 |
| 20070013771 | Imaizumi et al. | Jan 2007 | A1 |
| 20070078300 | Zinaty et al. | Apr 2007 | A1 |
| 20070156021 | Morse et al. | Jul 2007 | A1 |
| 20070222865 | Levien et al. | Sep 2007 | A1 |
| 20080058593 | Gu et al. | Mar 2008 | A1 |
| 20080122924 | Tashiro | May 2008 | A1 |
| 20090105544 | Takahira | Apr 2009 | A1 |
| 20090109431 | Delmonico et al. | Apr 2009 | A1 |
| 20090251549 | Meguro | Oct 2009 | A1 |
| 20100165088 | Seo | Jul 2010 | A1 |
| 20110282142 | Refael | Nov 2011 | A1 |
| 20110290887 | Wang et al. | Dec 2011 | A1 |
| 20120057009 | Liao et al. | Mar 2012 | A1 |
| 20120113239 | Krupnik et al. | May 2012 | A1 |
| 20120238810 | Kobayashi | Sep 2012 | A1 |
| 20120307039 | Holmes | Dec 2012 | A1 |
| 20140132746 | King | May 2014 | A1 |
| 20140221749 | Grant et al. | Aug 2014 | A1 |
| 20150164482 | Toji | Jun 2015 | A1 |
| 20150208900 | Vidas et al. | Jul 2015 | A1 |
| 20150320299 | Krupnik | Nov 2015 | A1 |
| 20160128545 | Morita | May 2016 | A1 |
| 20160210411 | Mentis | Jul 2016 | A1 |
| 20160335751 | Sidar et al. | Nov 2016 | A1 |
| 20170034484 | Yanagidate | Feb 2017 | A1 |
| 20170085831 | Hashimoto | Mar 2017 | A1 |
| 20170143310 | Funakubo | May 2017 | A1 |
| 20170215720 | Walker et al. | Aug 2017 | A1 |
| 20170290571 | Funakubo | Oct 2017 | A1 |
| 20170374275 | Shimokura et al. | Dec 2017 | A1 |
| 20180199795 | Maruyama et al. | Jul 2018 | A1 |
| 20180268523 | Takahasi et al. | Sep 2018 | A1 |
| 20180310802 | Gilreath et al. | Nov 2018 | A1 |
| 20190052854 | Kojima | Feb 2019 | A1 |
| 20200184644 | Ueda | Jun 2020 | A1 |
| 20200305700 | Kamon | Oct 2020 | A1 |
| 20210259517 | Ubbesen | Aug 2021 | A1 |
| Number | Date | Country |
|---|---|---|
| 1625751 | Feb 2006 | EP |
| 1500367 | Mar 2008 | EP |
| 2168496 | Mar 2010 | EP |
| 1870827 | Apr 2010 | EP |
| 2054852 | Jun 2010 | EP |
| 1550311 | Aug 2011 | EP |
| 2053862 | Jan 2014 | EP |
| 2823749 | Jan 2015 | EP |
| 1743568 | Aug 2015 | EP |
| 2926717 | Oct 2015 | EP |
| 2994033 | Mar 2016 | EP |
| 3020322 | May 2016 | EP |
| 3050484 | Aug 2016 | EP |
| 2733924 | Apr 2017 | EP |
| 3297515 | Mar 2018 | EP |
| 2868254 | Apr 2018 | EP |
| 3384460 | Oct 2018 | EP |
| H-11339006 | Dec 1999 | JP |
| WO-2015067750 | May 2015 | WO |
| 2016187124 | Nov 2016 | WO |
| 2017094535 | Jun 2017 | WO |
| 2018117468 | Jun 2018 | WO |
| 2019039252 | Feb 2019 | WO |
| 2019130964 | Jul 2019 | WO |
| 2019216084 | Nov 2019 | WO |
| Entry |
|---|
| International Search Report and Written Opinion in PCT/EP2020/086131 dated Mar. 12, 2021, 11 pages. |
| First technical examination report in Danish patent application PA 2019 70799 dated May 15, 2020, 10 pages. |
| Second technical examination report in Danish patent application No. PA 2019 70799 dated Oct. 15, 2020, 5 pages. |
| Number | Date | Country | |
|---|---|---|---|
| 20230025755 A1 | Jan 2023 | US |
