Patent Application
20190209842
The present invention relates to an artificial retinal prosthesis system, and more particularly to a system for artificial retinal prosthesis with color vision.
Currently, among the patients with visual deterioration, some patients choose to implant an artificial retina to improve their vision. At present, expensive artificial retinas of the commercial standard with low pixels have a limited improvement on the quality of life of patients. In view of this, many companies as well as academic and research institutes have begun to actively invest in the improvement of microsystem for artificial retina.
In order to give the users a more comfortable visual experience, many R&D teams are actively making improvements on the image resolution. For example, U.S. Pat. No. 7,751,896 B2, U.S. Pat. No. 6,804,560 B2 improve the signal transmission in the artificial retina by adding components such as an amplifier or a photosensitive reference component to the circuit, so that the electrical stimulation signals of the artificial retina are more even, when the patient wears the above artificial retina, it is just like the response of eyes to ambient light conditions under natural conditions. There are also other teams that focus on the colors of the image, hoping to upgrade the conventional artificial retinas that only show black and white images to color images. For example, in U.S. Pat. No. 7,840,274 B2, the artificial retina comprises a color image receiver for receiving a color image and converting the color image into an electrical signal, and an image processing unit coupled to the color image receiver for processing the electrical signal. In the patent, a plurality of pixel electrodes are driven by data from the image processing unit to stimulate the optic nerve by time mode to produce a perception of color images. As far as we know, however, there is no published evidence that the time mode stimulation scheme as described in said patent works universally, reliably, or at all.
At present, a number of pixel units of the artificial retina continues to increase, which has been greatly advanced for artificial retinas having only a few tens of pixel units in the past. In contrast, related researches on artificial retinal systems that provide color visual perception are still at a very early stage, and even though many manufacturers and teams have proposed various artificial retinal systems that provide color visual perception, there is no corresponding product/system that has been manufactured, or it is not good enough to achieve color visual perception after practical operations. Obviously, there is still a lot of room for development in developing artificial retina systems providing color visual perception, depending on the continuous investment and improvement of relevant teams.
A main object of the present invention is to improve the prior artificial retinal systems for providing color visual perception as well as to provide a better wearing experience for the patients.
In order to achieve the above object, the present invention provides a system for artificial retinal prosthesis with color vision. The system comprises an artificial retinal prosthesis implanted in a user's body and a color shutter disposed on a front side of the artificial retinal prosthesis. The artificial retinal prosthesis comprises a plurality of pixel units for receiving an external visual image entering the eyes of the user and outputting a spatiotemporal electrical stimulation to the user's optic nerve according to the external visual image; the color shutter connects communicatively with the artificial retinal prosthesis, and determines a spectrum of the external visual image that is allowed to enter the eyes according to the spatiotemporal electrical stimulation to allow the user to generate a color image perception.
In another embodiment, a system for artificial retinal prosthesis with color vision is provided, comprising: an artificial retinal prosthesis implanted in a user's body, the artificial retinal prosthesis comprises a plurality of pixel units and a color shutter integrated with the pixel unit structurally. The plurality of pixel units are used to receive an external visual image that enters the eyes of the user, and output a spatiotemporal electrical stimulation to the user's optic nerve according to the external visual image; the color shutter connects communicatively with the plurality of pixel units, and determines a spectrum that is allowed to enter the eyes to allow the user to generate a color image perception.
The system for artificial retinal prosthesis of the present invention causes the stimulation of the pixel electrodes and the spectrum of the external visual image entering the user's eyes to change synchronously along with different time sequences and different spatiotemporal distributions of each retinal cell, thereby stimulating the patient's retinal cells to provide the patient with a color image perception that assists the patient in truly obtaining RGB color vision. The spatiotemporal stimulation creates color perception in essential the same way as the so-called Fechner Color effect.
The detailed description and technical contents of the present invention will now be described below with reference to the drawings.
Please refer to
The artificial retinal prosthesis 10 can send a wireless signal to the goggle 30 to control the color shutter 20 of the goggle 30. For example, when the artificial retinal prosthesis 10 needs a red light stimulus, the artificial retinal prosthesis 10 sends a wireless signal TS1 to the goggle 30 to activate a red color shutter in the color shutter 20, so that only red light can pass through the red color shutter of the goggle 30 to reach the artificial retinal prosthesis 10. If a blue light stimulus is required, the artificial retinal prosthesis 10 sends a wireless signal TS2 to the goggle 30 to activate a blue color shutter in the color shutter 20, so that blue light can pass through the goggle 30 to reach the artificial retinal prosthesis 10. Likewise, when a green light stimulus is required, the artificial retinal prosthesis 10 sends a wireless signal TS3 to the goggle 30 to activate a green color shutter, so that green light can pass through the goggle 30 to reach the artificial retinal prosthesis 10. Subsequently, after the artificial retinal prosthesis 10 receives a specific incident light such as red light, blue light, or green light through the color shutter 20, a pixel electrode array in the artificial retinal prosthesis 10 is electrically stimulated by a spatiotemporal electrical stimulation.
When the pixel electrodes in the artificial retinal prosthesis 10 are defined as Pxy according to the spatial positions, such as P11, P12, P13, P22, P23, the above-mentioned “spatiotemporal electrical stimulation” refers to different stimulations given to corresponding optic nerves by different Pxy at different times, for example, P11 and P12 stimulations are given at time point t1, but the remaining pixel electrodes are not.
The artificial retinal prosthesis 10 is disposed on the retina of the eye structure, and can be disposed on the sub-retina or the epi-retina as needed in actual use without particular limitation. This embodiment is disposed on the sub-retina. The artificial retinal prosthesis 10 comprises a plurality of pixel arrays and a processing module disposed correspondingly to the plurality of pixel arrays. Each of the plurality of pixel arrays comprises a substrate and a plurality of sub-pixels disposed on the substrate for receiving a color image. In this embodiment, the substrate can be a thin flexible silicon substrate that can be deformed and bent as desired, so that it can be bent as much as possible into a structure conforming to the shape of a human eye and disposed in the eye of a patient.
In actual manufacturing, for example, the substrate can be fabricated based on a manufacturing process using a Silicon On Insulator (SOI) chip, and formed by thinning the chip after a Metal-Oxide-semiconductor (MOS) fabrication. The processing module can include a correlated double sampling unit (CDS), an analog-to-digital converter (ADC), a digital core, and a digital-to-analog converter (DAC) to process a signal of the pixel array. However, the components included in the processing module are not limited to the above components, technicians of this field can add or delete based on actual needs and designs.
Each of the plurality of sub-pixels comprises at least one pixel electrode, a photodiode, and a circuit architecture electrically connected to the photodiode. After an incident light emit to the photodiode, the incident light is converted into an electric charge and a photovoltaic potential, and a light-induced electrical stimulation signal is generated according to an intensity ratio of the incident light. The light-induced electrical stimulation signal generates the spatiotemporal electrical stimulation to stimulate the patient's retinal cells, thereby producing a color image.
It should be additionally explained that, in another embodiment of the present invention, the color shutter 20 may not be assembled on the goggle 30, but can be integrated into a single structure with the artificial retinal prosthesis 10. That is, the color shutter 20 can be formed on the pixel array of the artificial retinal prosthesis 10 and can include a plurality of optical shutter units corresponding to different colors. For example, the color shutter 20 can include red shutters formed in a first row to a third row of the pixel array, green shutters formed in a fourth row to a sixth row, and blue shutters formed in a seventh row to a ninth row.
For one of the examples of the color shutter 20, please refer to
In this embodiment, the first substrate 21 and the second substrate 22 are transparent and can be formed with the same or different materials, such as glass, resin, polycarbonate (PC), and the like.
The first fluid layer 25 can be a conductive or polarized water or salt solution; and the second fluid layer 26 can be an oily medium, so that when the first fluid layer 25 and the second fluid layer 26 coexist between the second substrate 22 and the hydrophobic layer 24, a two-layer structure can be formed without being miscible. In this embodiment, the second fluid layer 26 can be a mixture of oils with different colors, such as can be selected from a green oil, a red oil, a blue oil, or any combinations of the above oils.
The hydrophobic layer 24 can be a functional layer with low surface energy and high stability, and specifically, can be made of a polymer or a silicon dioxide layer. For example, the polymer used for the hydrophobic layer 24 may be a fluoropolymer such as Cytop or amorphous Teflon, or a hydrocarbon polymer may also be used. If silicon dioxide is used, its surface needs to be treated hydrophobically.
The electrode 23 is disposed on the first substrate 21 to apply a voltage to the first fluid layer 25. The electrode 23 used in the present embodiment is preferably a transparent electrode made of any suitable conductive material such as indium tin oxide (ITO). The above is merely illustrative, and the present invention is not limited thereto, and the color shutter 20 may employ other devices such as a light filter.
In another embodiment of the present invention, the color shutter 20 can further include an optical sensor for sensing ambient light and/or a variable light filter for automatically controlling light passing through the color shutter 20 according to environmental conditions.
The principle used by the color shutter 20 is an electrowetting effect, that is, a wettability of the oily medium on the substrate is controlled by changing a voltage between the oily medium and the hydrophobic layer 24 (insulating layer). More specifically, the oily to medium is deformed and displaced by changing a contact angle. The term “wetting” used above refers to the process of a fluid on a solid surface being replaced by another fluid. The fluid on the solid surface (i.e., the hydrophobic layer 24) can diffuse, at this time, the adhesion of the fluid on the solid surface is greater than the cohesion, referred to as “wetting.” Conversely, when the fluid on the solid surface (i.e., the hydrophobic insulating layer) cannot diffuse, the contact surface has a tendency to shrink into a spherical shape, which is called “non-wetting”, and “non-wetting” refers to the adhesion of the fluid on the solid surface being smaller than the cohesion.
Returning to the present invention, the first fluid layer 25 and the second fluid layer 26 are immiscible with each other without applying an electric field to the fluids (closed state) to form a two-layer structure in which the first fluid layer 25 is diffused to form as a fluid layer adjacent to the second substrate 22; and the second fluid layer 26 also diffuses to form a fluid layer adjacent to the hydrophobic layer 24 and serves as color pixels. However, when an electric field is applied to the fluids (on state), the second fluid layer 26 is broken into small droplets to cause the color shutter 20 to exhibit a transparent color, as shown in
Therefore, in order to obtain various display results, the second fluid layer 26 (i.e., the oily medium) can be designed to have a desired color, and a surface of the oily fluid can be controlled to change the pixels by controlling the voltage.
In the other embodiment, the anisotropic color pigment particles (say pigment needles) in fluid suspensions could be utilized in an alternative color shutter. Three shutters in tandem, with Yellow, Cyan, and Magenta color pigments, would be needed. Each color shutter would be turned on by applying sufficient large voltage across the fluid to align the particle with the field. Alternatively, another type of color shutter with electrophoretic cells in shutter mode could be used. This is somewhat harder to reach adequate speed, but can work with optimized cells.
When the system for artificial retinal prosthesis with color vision of this embodiment is in use, the color image is converted into the light-induced electrical stimulation signal by the photodiode of the sub-pixel, and the spatiotemporal electrical stimulation is generated to provide the patient with color perception. As a specific example, the spatiotemporal electrical stimulation of about 4 Hz to 8 Hz (preferably 7 Hz) can be divided into seven equally spaced phases within one cycle, producing color sensations of red (R), green (G) and blue (B).
Further explain how to provide the patient with color perception by the spatiotemporal electrical stimulation as below.
In this embodiment, the pixel electrodes in each of the pixel arrays are arranged in 1 column of 9 rows and classified into three groups corresponding to the specific color perceptions. A time series takes 7 equally spaced frames as a cycle, and the cycles per second (cps) can be between 7 and 8, so the frames per second (fps) are between 49 and 56, and the cycle between two of the frames is approximately 20 ms.
Please refer to Table 1, wherein “-” means the pixel electrode is turned off and “|” means the pixel electrode is turned on.
For the 3 rows of R/G/B strips, the span is 240 μm (30 m*8) strip width.
Row 1 to row 9 start rolling at the same time. It can be found from Table 1 that all the pixel electrodes are turned off in frame 1; the pixel electrodes of row 1, row 3 to row 9 are turned on while the other pixel electrodes are turned off in frame 5; while in frame 7, all the pixel electrodes are turned on except for the pixel electrodes of row 5 and row 8 being turned off Based on the arrangement and operation of the pixel electrodes described above, the patient can perceive colors on the corresponding pixel electrodes, for example, in the cycles from frame 6 to frame 7, the patient can perceive red in the pixel electrodes of row 2.
If power attenuation problem is taken into consideration, in other embodiments, electrical stimulations of the above-mentioned “rows” are not simultaneously sent out in the same frame. If all the “rows” in the same frame are enabled at exactly the same time, the artificial retinal prosthesis 10 will consume a very large amount of power and cause a drop in power, even making the artificial retinal prosthesis 10 unable to function properly. In order to avoid the above problem, in the cycles of the same frame, when the state of the pixel electrodes is “|” representing being turned-on, they will be activated row-to-row. That is to say, electrical stimulations of the subsequent rows will slightly lag behind the previous pixel electrode; however, when the state of the pixel electrodes is “-” representing being turned-off, as in the first column to the third column (frame 1 to frame 3) of Table 1 above, the pixel electrodes in the columns cannot be activated and electrical stimulations are not sent out from the columns. Please refer to
Please refer to Table 2. In this embodiment, the pixel electrodes are arranged in 1 column of 6 rows, and the electrodes are classified into three groups with each of the groups respectively corresponding to a specific color. The setting of the time series in this embodiment is the same as that of Embodiment 1.
This embodiment is similar to that of Table 1, the pixel electrodes of row 1 to row 6 are all turned off in frame 1; the pixel electrodes of row 2 to row 6 are all turned on in frame 5, only the pixel electrodes of row 1 are turned off; while in frame 7, all the pixel electrodes are turned on except for the pixel electrodes of row 3 and row 5 being turned off.
Please refer to
If row 2 of column 3 is taken as an example, all the pixel electrodes are turned off in frame 1 to frame 4, and the surrounding dummy electrodes are also turned off; while the pixel electrodes are turned on in frame 5 to frame 6, so only green light can pass through the goggle 30 to reach the artificial retinal prosthesis 10, and at the same time, the surrounding dummy electrodes are also turned on. According to the operation of the pixel electrodes described above, the patient can have a green visual perception in the pixel electrodes of column 3 and row 2, and by the above arrangement of the dummy electrodes, a visual contrast can be generated between the pixel electrodes corresponding to the specific colors and the surrounding areas thereof, and the effect of enhancing the color perception of the patient is achieved.
Referring to
The operation of this embodiment is basically the same as that of Embodiment 3. If row 4 of column 3 is taken as an example, all the pixel electrodes are turned off in frame 1 to frame 5, and the surrounding dummy electrodes are also turned off; while the pixel electrodes are turned on in frame 6 to frame 7, so only red light can pass through the goggle 30 to reach the artificial retinal prosthesis 10, and at the same time, the surrounding dummy electrodes are also turned on.
The electrodes of the present embodiment are arranged in 1 column of 132 rows, and are divided into four groups, which respectively are “D” representing a dummy electrode, “R” representing an electrode group that can perceive red correspondingly, “G” representing an electrode group that can perceive green correspondingly, and “B” representing an electrode group that can perceive blue correspondingly. Wherein, each of the R, G, and B pixel electrodes is surrounded by 2 dummy electrodes. A time series takes 12 frames as a cycle, the cycles per second (cps) are 6, so the frames per second (fps) are 72.
In Table 3, “-” means the pixel electrode is turned off and “|” means the pixel electrode is turned on. For example, if it is desired to enable a blue color shutter during frame 1 to frame 4 so that only blue light can pass through the goggle 30 and generate a specific electrical stimulus, in the cycles of frame 1, blue light is used to activate the pixel electrodes of row 7 to row 9, row 16 to row 18, row 25 to row 27, and row 34 to row 36. The operation of this embodiment is also substantially the same as or similar to that of the foregoing embodiment, and thus will not be described herein again.
The present application claims the benefit of U.S. Provisional Patent Application No. 62/610,004, entitled “System for Artificial Retina Prosthesis,” which was filed on Dec. 22, 2017, and the disclosure of which is incorporated herein by reference in its entirety.
| Number | Date | Country | |
|---|---|---|---|
| 62610004 | Dec 2017 | US |
