This description relates to showing skin lesion information.
Skin lesion information, including scanned images of skin lesions and related data, can be stored on and displayed by computer to a dermatologist or other user to aid in diagnosis and treatment.
In general, in an aspect, on a two-dimensional electronic display are shown simultaneously: (a) at least a partial body view of a surface of the human model on which a location of a skin lesion on a corresponding real human has been indicated, and (b) an image of the lesion area that corresponds to the partial body view of the human model.
Implementations may include one or more of the following features. Simultaneously with the partial body view and the image of the lesion, a thumbnail of the partial body view is shown. The partial body view bears an indication of the location of the skin lesion. The image comprises a reconstructed dermoscopic image. The image comprises a traditional dermoscopic image. A user can choose whether to display the image of the lesion or a model of the lesion.
In general, in an aspect, on a two-dimensional electronic display are shown an image of a lesion location and indications of skin lesions on the skin of a real human corresponding to the shown image. Legends are shown, for the respective skin lesions, each indicating a status of a corresponding skin lesion and a status of a scanned image of the skin lesion. The image comprises a reconstructed dermoscopic image. The image comprises a traditional dermoscopic image. The status of the scanned image comprises progress in completion of an analysis of the image. The legend comprises a color. The legend comprises an icon. The legend comprises a progress bar.
In general, in an aspect, on a two-dimensional electronic display are simultaneously shown: a list of rows where each row is an image representing the lesion location and the next diagnostic or treatment step in a column wise list format.
Implementations may include one or more of the following features. The image comprises a reconstructed dermoscopic image. The image comprises a traditional dermoscopic image. The next diagnostic or treatment step comprises biopsy. The next diagnostic or treatment step comprises observation. The next diagnostic or treatment step is indicated by a color.
These and other aspects and features, and combinations of them, can be expressed as methods, apparatus, program products, methods of doing business, systems, components, means for performing functions, and in other ways.
These and other aspects and features will become apparent from the following description and the claims.
The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with colored drawing(s) will be provided by the patent office upon request and payment of the necessary fee.
By providing novel features in a user interface of a computer-implemented skin lesion information system, a dermatologist or other physician or other medical worker, researcher, or patient (or any other user) can (among other things) acquire, enter, store, review, analyze, annotate, process, manage, and use for diagnosis or treatment (among other things), skin lesion information (including scans of skin lesions, data about the scans, and any other kind of information about skin lesions), effectively, inexpensively, accurately, comfortably, quickly, and safely.
The features discussed here can be used alone or in combination with other user interfaces in a wide variety of skin lesion systems. The skin lesion systems can include general purpose and special purpose computers, machines, devices, equipment, and systems of any kind, including systems that have scanning devices, storage, memory, and display devices, for example. The implementations can use handheld devices, workstations, networks, and any other systems that process or store information or communicate it locally or remotely. When a computer is used in an implementation, the computer can have typical or special memory, storage, displays, keyboards, mice, communications capabilities, and other features. And a wide variety of operating systems, and application software can be used.
In some examples, the features of the user interface can be implemented on a mobile cart of the kind described in U.S. patent application Ser. Nos. 29/341,111, 29/341,114, and 12/512,775, all three filed on Jul. 30, 2009 Skin lesion information may be stored on and retrieved from memory devices of any kind including the memory cards or usage cards described in U.S. patent application Ser. No. 12/512,895, filed Jul. 30, 2009. Usages of the systems associated with usage cards can be monitored and controlled as described, for example, in U.S. patent application Ser. No. 11/761,816, filed Jun. 12, 2007. Additional information concerning user interfaces for skin lesion systems is found in U.S. patent application Ser. No. 11/681,345, filed Mar. 2, 2007. Additional information about identifying a location of a lesion on a human body is found in U.S. patent application Ser. No. 12/204,247, filed Sep. 4, 2008. All of the applications and patents identified here and elsewhere in this application are incorporated here by reference.
In some implementations, the lesion scanner used with the user interface can be a MelaFind probe developed by Electro-Optical Sciences, Inc., a multi-spectral scanner. Some information about the implementation of the MelaFind system can be found in United States patents and patent applications assigned to Electro-Optical Sciences, Inc., and incorporated here by reference, including U.S. Pat. Nos. 6,081,612; 6,208,749; and 6,626,558.
In general, but with some exceptions, the flow in each figure proceeds from left to right. Flow lines 104 in the figures connect user controls 106, such as buttons, in some of the screens, to other screens that appear when the user invokes the controls. In some cases, a flow line indicates a connection by pointing to a box 108 which contains a reference to the screen to which control is shifted. In each of the four figures, each of the thumbnails bears an identifying number to facilitate references to one another.
Turning to the flow to which
In
Invoking the Add button 714 leads to
The Remove button 714 returns the user to the screen of
In
When an account is to be changed, either from
Invoking the Settings tab leads to the screen of
Turning now to the main user flow shown in
If the user inserts a card that is not readable or not authorized for use, the screen of
There are three types of screens represented in the group of
In the screen of
The human model views of
On the left side of the screen of
In
A second set of the screens, in
In the example as shown in
Once the user has identified the location of the lesion, the screen of
On the right side of the multispectral analysis pane 3206 a column 3210 of thumbnail positions 3211 represents a succession of spectral bands for which spectral-band images are taken, for example, when the MelaFind probe is used to scan a lesion. Each of the thumbnail positions represents the spectral-band image for one of the spectral bands. The presence of a thumbnail 3212 in one of the positions in the column indicates that the corresponding spectral-band image has been acquired for that band. The summary button 3214 on the screen of
The scan of
When the image processing has been completed, the screen of
In addition, results of the most recently completed scan are shown in text boxes such as 3604, 3606 in the multi-spectral analysis pane. In this instance, the scan contained two different lesions, which were analyzed separately, and are both included in the image of the scan. The result for the left lesion is “MelaFind Negative” and the lesion area is reported in square millimeters. The result for the right lesion is “MelaFind Positive” indicating a recommendation that a biopsy be performed. These results (positive or negative, indicating the need to biopsy or not) are of the kind described or referred to in U.S. patent application Ser. No. 11/761,816, filed Jun. 12, 2007; Ser. No. 11/681,345, filed Mar. 2, 2007; Ser. No. 11/956,918, filed Dec. 4, 2007; and Ser. No. 12/512,895, filed Jul. 30, 2009, all of which are incorporated here by reference. In each of the Result boxes, one of two icons 3605, 3607 is highlighted to indicate either that the lesion should be biopsied (microscope icon) or that it should be observed (eye icon). The Result boxes are displayed directly on the scanned image in this example, but could also be displayed in a wide variety of other ways.
The completion of the image processing for a scan, as represented in the screen of
Also, with the completion of image processing for the current scan, the general state of the screen is changed so that it is no longer in a scanning mode, as discussed above, but is now in a lesion navigation mode. In the lesion navigation mode, the screen elements are arranged to enable a user to navigate easily from one of the scanned lesions to another and therefore to view the images of different lesions, the multi-spectral images for each lesion, and the results of the analysis of each lesion, among other things.
In the lesion navigation mode, the lesion navigation pane 3102 is updated to show thumbnails of all of the lesions that have been successfully scanned, processed, and stored on the memory card. The scroll buttons 3608, 3610 become brighter and are active to permit easy navigation, and the user can invoke any of the lesion thumbnails in the pane, which then is highlighted to indicate that it has been invoked. (In general, when a user control is displayed more brightly on any of the screens, that control is active and can be invoked, and when it is displayed less brightly (“greyed”), it is inactive and cannot be invoked.)
In the lesion navigation mode, as shown in
The screen of
In some cases, the scan attempted by the user fails because the image analysis cannot be done effectively with an appropriate degree of confidence. The screen of
As illustrated by the screen of
The screens of
In the example screens of
The colors and symbol shapes in the summary bar correspond to symbol shapes and colors on lesion status bars 4110. There is a status bar shown for each lesion. In this case, there are twelve bars corresponding to twelve lesions that were scanned. In this example, a green eye is used to indicate that the next step is observation and a red microscope is used to indicate that the next step is biopsy. Blue is used to indicate that computer processing is not yet complete. In this example, purple identifies a dermascopic image. Progress indications are shown also in the lesion thumbnails. A wide variety of other colors and icons could be used.
In the Show-Results state, each lesion status bar has a color that corresponds to the status of the lesion (in the example shown: positive, negative, dermascope, or in process). The bars for lesions for which there are MelaFind results also show icons at the left end. For images that are in process, the bar shows the state of completion in the form of a progressive completion stripe. In cases in which a single scan contained more than one lesion, the lesion status bar is split to show the results for each of the (up to four) lesions in the scan. Each lesion bar is connected by a lead line to the location of the lesion on the model. On the model, the marker for each lesion takes the form of the icon and color corresponding to the icon and color of the lesion status bar. A wide variety of other configurations of bars or other indicators could be used.
The screen of
In the Hide-Results state, illustrated by the screen of
In some cases, a lesion status bar has an exclamation icon, which indicates that the instruction manual should be consulted for further information.
The lesion thumbnails in the lesion navigation pane all show bars indicating the progress of scanning and image processing.
As shown in the screen of
The screens of
Every four hours, for example, the screen of
The screens of
A Help button 4702 always leads to helpful information about that step. A Back button 4704 always returns the user to the prior step. A Phantom Imaging button 4706 arms the system for phantom imaging from the screen of
Because the actual use of the memory card to store images of skin lesions on patients has an associated cost, the system provides a Practice mode for scanning, the process of which is tracked in
As shown in
When an image of the lesion is available (either one that is taken using a separate digital camera or one that is reconstructed), that “actual” image can be used to replace, on the model displayed to the user, the close-up model image of the lesion location.
To arrange this substitution of real image for model image of the lesion location, the user invokes the clinical button 5902. Doing so, takes the user to a window that shows available images. When the user selects one of those images, the user is returned to the screen of
The real image that is substituted for the model image could be a reconstructed dermascopic image, or a traditional dermascopic image, or other kinds of images.
Other features described above with respect to other figures may also apply to the features illustrated in
For example, a wide variety of other types, arrangements, orientations, appearances, and names of the user controls, thumbnails, views, and arrangements of the user interface could be substituted for the examples described above. The sequence of actions and responses of the user interface could also be altered. The coordination and synchronization of different views could be arranged differently. The human models could be made more or less realistic. In some examples, actual photographs of the real human could be substituted for the model photographs. Different sets of human models could be available in selectable by the user to represent real humans of different sizes, shapes, skin coloring, racial features, ages, and other characteristics so that they more directly represent a real human who is the subject of the skin lesion scanning. The kinds of results displayed for each of the lesions could be other than “positive” and “negative” and could be expressed in displayed in other ways. Other navigational devices could be provided to enable the user to move among scans in other ways.
This application is a continuation-in-part of U.S. application Ser. No. 12/916,656, filed Nov. 1, 2010, which claims priority to U.S. provisional application Ser. 61/280,386, filed on Nov. 3, 2009, the entire contents of which are incorporated herein by reference.
Number | Name | Date | Kind |
---|---|---|---|
4817044 | Ogren | Mar 1989 | A |
6081612 | Gutkowicz-Krusin et al. | Jun 2000 | A |
6208749 | Gutkowicz-Krusin et al. | Mar 2001 | B1 |
6307957 | Gutkowicz-Krusin et al. | Oct 2001 | B1 |
6319201 | Wilk | Nov 2001 | B1 |
6571003 | Hillebrand et al. | May 2003 | B1 |
6626558 | Momot et al. | Sep 2003 | B2 |
6726101 | McIntyre et al. | Apr 2004 | B1 |
6993167 | Skladnev et al. | Jan 2006 | B1 |
7041941 | Faries, Jr. et al. | May 2006 | B2 |
7096282 | Wille | Aug 2006 | B1 |
7102672 | Jacobs | Sep 2006 | B1 |
7127094 | Elbaum et al. | Oct 2006 | B1 |
7215819 | Onno et al. | May 2007 | B2 |
7233693 | Momma | Jun 2007 | B2 |
7324668 | Rubinstenn et al. | Jan 2008 | B2 |
7376346 | Merola et al. | May 2008 | B2 |
7761332 | Capurso et al. | Jul 2010 | B2 |
7920714 | O'Neil | Apr 2011 | B2 |
20010051787 | Haller et al. | Dec 2001 | A1 |
20020010679 | Felsher | Jan 2002 | A1 |
20020016720 | Poropatich et al. | Feb 2002 | A1 |
20020021828 | Papier et al. | Feb 2002 | A1 |
20020049432 | Mukai | Apr 2002 | A1 |
20020059081 | Yasuda et al. | May 2002 | A1 |
20030130567 | Mault et al. | Jul 2003 | A1 |
20040068255 | Short et al. | Apr 2004 | A1 |
20040255081 | Arnouse | Dec 2004 | A1 |
20050049467 | Stamatas et al. | Mar 2005 | A1 |
20060092315 | Payonk et al. | May 2006 | A1 |
20060095297 | Virik | May 2006 | A1 |
20070006322 | Karimzadeh et al. | Jan 2007 | A1 |
20070043594 | Lavergne | Feb 2007 | A1 |
20070078678 | DiSilvestro et al. | Apr 2007 | A1 |
20070109527 | Wenstrand | May 2007 | A1 |
20070127793 | Beckett et al. | Jun 2007 | A1 |
20080214907 | Gutkowicz-Krusin et al. | Sep 2008 | A1 |
20080312952 | Gulfo et al. | Dec 2008 | A1 |
20090060304 | Gulfo et al. | Mar 2009 | A1 |
20090080726 | Cotton et al. | Mar 2009 | A1 |
20110103660 | Butler et al. | May 2011 | A1 |
Number | Date | Country |
---|---|---|
1 505 765 | Feb 2005 | EP |
1 658 817 | May 2006 | EP |
11-070120 | Mar 1999 | JP |
2001-046398 | Feb 2001 | JP |
2002-011106 | Jan 2002 | JP |
2009-506835 | Feb 2009 | JP |
WO 8809973 | Dec 1988 | WO |
WO 03001329 | Jan 2003 | WO |
WO 03060650 | Jul 2003 | WO |
WO 2005109297 | Nov 2005 | WO |
WO 2006060286 | Jun 2006 | WO |
WO 2006122741 | Nov 2006 | WO |
WO 2007022017 | Feb 2007 | WO |
WO 2011056851 | May 2011 | WO |
WO 2012151128 | Nov 2012 | WO |
Entry |
---|
International Preliminary Report on Patentability for App. Ser. No. PCT/US2010/055273, dated May 8, 2012, 7 pages. |
International Search Report and Written Opinion for App. Ser. No. PCT/US2012/035528, dated Nov. 28, 2012, 9 pages. |
Office Action for Japanese App. Ser. No. 2010-512337, dated Dec. 18, 2012, 7 pages. |
“HIPAA Compliance and Smart Cards: Solutions to Privacy and Security Requirements”, Smart Card Alliance, Sep. 2003, pp. 1-46. |
http://www.miaccard.com/miac—ps.html, Jun. 8, 2007, 1 page. |
http://www.doctorsmile.cz/img/erbdiod.pdf, pp. 1-12. Jun. 12, 2007. |
www.lambdascientifica.com/pagina.asp&lingua=&gruppo=9&categoria=39&id=134, KLS Smart Card, Jun. 12, 2007, 2 pages. |
http://www.amo-inc.com, about AMO, Jun. 12, 2007, pp. 1-3. |
Advanced Medical Optics, Inc., http://sec.edgar-online.com/2006/05/10/0001104659-06-033180/Section—8.asp, Mar. 31, 2006, pp. 1-17. |
International Search Report for PCT/US08/66508 dated Sep. 25, 2008 (17 pages). |
2006 Annual Form 10-K Report (Electro Optical Sciences) [online] Mar. 15, 2007 [retrieved on Oct. 9, 2008]. Retrieved from the Internet: URL:http://ccbn.10kwizard.com/xml/download.php?repo=tenk&ipage=4745835&format=PDF, p. 1, para 2-3; p. 8, para 5. |
Electro-Optical Sciences, Inc., Needham & Co. 7th Annual Bio/MedTech Conference, Presentation re MELAFIND (Jun. 11, 2008, 35 pages). |
Electro-Optical Sciences, Inc. 2008 Annual Report (Apr. 15, 2009, 88 pages). |
Electro-Optical Sciences, Inc. 2007 Annual Report (Apr. 15, 2008, 96 pages). |
Van Dusen, A., “Invasive biopsies may soon be a thing of the past if these detection methods prove effective,” Forbes (Aug. 22, 2008, 3 pages). |
Thumbnail image titled “eos-moneyshot.jpg” [retrieved on Jul. 27, 2009]. Retrieved from the Internet: http://images.google.com, and source code indicating that the thumbnail image was originally available at http://www.daniellicalzi.com in Jun. 2009 (2 pages). |
Howard Teacher's Pet—Revolutionary Classroom Management [retrieved on Jul. 30, 2009]. Retrieved from the Internet: http://www.howardcomputers.com/petcart/ (6 pages). |
FreelanceDesigners.org, Daniel LiCalzi, Live Beta [retrieved from the Internet on Jun. 26, 2009] (3 pages). |
International Search Report for App. Ser. No. PCT/US2008/066636, dated Oct. 16, 2008, 16 pages. |
International Preliminary Report on Patentability for App. Ser. No. PCT/US2008/066636, dated Dec. 17, 2009, 14 pages. |
Supplementary European Search Report for App. Ser. No. EP 08 770 774, dated Aug. 23, 2010, 2 pages. |
European Office Action for App. Ser. No. EP 08 770 774, dated Sep. 14, 2010, 11 pages. |
Public Law 104-191 [H.R. 3103] Aug. 21, 1996, Health Insurance Portability and Accountability Act of 1966, 104 P.L. 191; 110 Stat. 1936; 1996 Enacted H.R. 3103; 104 Enacted H.R. 3103. |
International Search Report and Written Opinion for App. Ser. No. PCT/US2010/055273, dated Jun. 23, 2011, 12 pages. |
Office Action for Australian App. Ser. No. 2008266173, dated Feb. 28, 2013, 4 pages. |
U.S. Appl. No. 11/761,816, filed Jun. 12, 2007. |
U.S. Appl. No. 12/916,656, filed Nov. 1, 2010. |
Number | Date | Country | |
---|---|---|---|
20110210984 A1 | Sep 2011 | US |
Number | Date | Country | |
---|---|---|---|
61280386 | Nov 2009 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 12916656 | Nov 2010 | US |
Child | 13100153 | US |