Patent Application
20120139936
The present invention relates to presentations of data streams and to a system and method for presenting in-vivo data.
Known in-vivo imaging devices include ingestible capsules that may capture images from the inside of the gastrointestinal (GI) tract. Captured images may be transmitted to an external source to be examined, for example, for pathology by a healthcare professional. In some embodiments, in-vivo devices may include various other sensors that may transmit data to an external source for monitoring and diagnosis.
An in-vivo device may collect data from different points along a body lumen, for example lumens of the GI tract, and transmit them externally for analysis and diagnosis. The GI tract is a very long and curvy path such that it may be difficult to get a good indication of where along this tract each transmitted datum was obtained.
Time bars are known to be used when reviewing data, so as to indicate to the health professional how far along the image stream he/she may have advanced. However, since the in-vivo device may stall or advance at different speeds through various sections of a body lumen, for example, the GI tract, it may not be positively determined in some cases where or at what distance along the GI tract was a particular datum, for example an image, captured. In addition, on the time bar there may be no indication as to when the device may have reached certain anatomical milestones, for example, the duodenum, the cecum, or other anatomical locations in the GI tract.
Localization methods have been applied. Some localization methods may indicate the spatial position of the device in space at any given time. Although this information together with the time log may give the health professional a better indication of the rate at which the device has advanced it may still be difficult to correlate the spatial position of the device in space to the specific anatomy of, for example, the GI tract.
An in-vivo device may collect data from more than one sensor along the very long and curvy path resulting in multiple data streams captured by the in-vivo sensor. It may be time consuming and difficult to review multiple long streams of data. In addition, it may be difficult for a health profession to get an overall view of the contents of all the data obtained.
Embodiments of the present invention may provide a system and method for generating and displaying a fixed graphical presentation of captured in-vivo data streams. In one embodiment of the present invention, the fixed graphical presentation is a varying visual representation of a quantity or a dimension captured in an in-vivo data stream. In one example the graphical presentation is in the form of a color bar. In other embodiments of the present invention, the fixed graphical presentation may be displayed along side a streaming display of a data stream.
The subject matter regarded as the invention is particularly pointed out and distinctly claimed in the concluding portion of the specification. The invention, however, both as to organization and method of operation, together with objects, features and advantages thereof, may best be understood by reference to the following detailed description when read with the accompanied drawings in which:
It will be appreciated that for simplicity and clarity of illustration, elements shown in the figures have not necessarily been drawn to scale. For example, the dimensions of some of the elements may be exaggerated relative to other elements for clarity. Further, where considered appropriate, reference numerals may be repeated among the figures to indicate corresponding or analogous elements.
The following description is presented to enable one of ordinary skill in the art to make and use the invention as provided in the context of a particular application and its requirements. Various modifications to the described embodiments will be apparent to those with skill in the art, and the general principles defined herein may be applied to other embodiments. Therefore, the present invention is not intended to be limited to the particular embodiments shown and described, but is to be accorded the widest scope consistent with the principles and novel features herein disclosed. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be understood by those skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, and components have not been described in detail so as not to obscure the present invention.
Embodiments of the present invention offer a device, system and method for generating a fixed graphical presentation of a captured data stream, for example image streams, other non-imaged data, or other data such as color coded, possibly imaged data (e.g., pH data, temperature data, etc.) that may have been collected in vivo, for example along the GI tract. The summarized graphical presentation may include, for example, a varying visual representation, for example, a series of colors that may be at least partially representative of a quantity and/or data collected, e.g. a series of colors where each color presented on the bar may representative of a value of a parameter. Other suitable representations may be used, and other visual dimensions or qualities, such as brightness, size, width, pattern, etc. may be used. In some embodiments of the present invention, the summarized graphical presentation may be a fixed display along side a streaming display of the data stream
In one embodiment of the invention, the presentation may map out a varying quantity (e.g. a captured data stream) and may, for example, give indication of the relationship between the data stream captured and the anatomical origin or position relative to a start of the captured data stream, for example, the approximate or exact site, for example, in the GI tract from where various data captured may have originated. In another embodiment of the invention, the mapping may give, for example, an indication of an event (e.g. a physiological event) captured, measured, or otherwise obtained. In yet another embodiment of the invention, the mapping may give for example an indication of change of one or more parameters measured over time, for example, a change occurring due to pathology, a natural change in the local environment, or due to other relevant changes. The location may be relative to other information, for example, anatomical attributes along for example the GI tract. The location may in some embodiments be an absolute location, such as a location based on time or based on position of an in-vivo information capture device, based on an image frame in a sequence of images, etc.
Reference is made to
The in-vivo imaging device 40 may typically, according to embodiments of the present invention, transmit information (e.g., images or other data) to an external data receiver and/or recorder 12 possibly close to or worn on a subject. Typically, the data receiver 12 may include an antenna or antenna array 15 and a data receiver storage unit 16. The data receiver and/or recorder 12 may of course take other suitable configurations and may not include an antenna or antenna array. In some embodiments of the present invention, the receiver may, for example, include processing power and/or a LCD display from displaying image data.
The data receiver and/or recorder 12 may, for example, transfer the received data to a larger computing device 14, such as a workstation or personal computer, where the data may be further analyzed, stored, and/or displayed to a user. Typically, computing device 14 may include processing unit 13, data processor storage unit 19 and monitor 18. Computing device 14 may typically be a personal computer or workstation, which includes standard components such as processing unit 13, a memory, for example storage or memory 19, a disk drive, a monitor 18, and input-output devices, although alternate configurations are possible. Processing unit 13 typically, as part of its functionality, acts as a controller controlling the display of data for example, image data or other data. Monitor 18 is typically a conventional video display, but may, in addition, be any other device capable of providing image or other data. Instructions or software for carrying out a method according to an embodiment of the invention may be included as part of computing device 14, for example stored in memory 19.
In other embodiments, each of the various components need not be required; for example, the in-vivo device 40 may transmit or otherwise transfer (e.g., by wire) data directly to a viewing or computing device 14
In-vivo imaging systems suitable for use with embodiments of the present invention may be similar to various embodiments described in U.S. Pat. No. 5,604,531, assigned to the common assignee of the present application and incorporated herein by reference, and/or Publication Number WO 01/65995, also assigned to the common assignee of the present application and incorporated herein by reference. Other in-vivo systems, having other configurations, may be used. Of course, devices, systems, structures, functionalities and methods as described herein may have other configurations, sets of components, processes, etc.
Embodiments of the present invention include a device, system, and method for generating a typically concise and/or summarized graphical presentation of parameters sensed through or over time in a body lumen, for example the GI tract or any other tract, through which a sensing device may be present and/or traveling. Viewing a data stream captured by an in-vivo device, e.g., viewing an image stream transmitted by an ingestible imaging capsule may be a prolonged procedure. A summarized presentation of the data captured that may, for example, provide a visual representation and/or map of the captured data and may help focus a health professional's attention to an area of interest and/or may promote a health professional's understanding of the origin and contents of the data being viewed.
One or more streams of data obtained from said sensing device may be processed to create one or more summarized presentations that may, for example, be displayed in a graphical user interface, for example a graphical user interface of analysis software.
According to one embodiment, presentation of a data stream (e.g., a stream or set of images, a sequence of pH data, etc.) may be with a bar, for example, a color bar that that may be displayed, for example, on a monitor 18 perhaps through a graphical user interface, or, for example, in real time on an LCD display on a receiver 12 as a data stream is being captured. The presentation may include a varying visual representation of a quantity or a dimension representing, for example, a varying quantity captured in a portion of (e.g., an image frame) an in-vivo data stream. In one example the dimension may be color. The presentation may typically be an abstracted or summarized version of image or other data being presented, for example, streamed on a different portion of the display. The presentation may typically include multiple image items or data items such as bars, stripes, pixels or other components, assembled in a continuing series, such as a bar including multiple strips, each strip corresponding to an image frame. For example, a portion of the presentation may represent a summary of the overall color scheme, brightness, pH level, temperature, pressure, or other quantity on a displayed frame or data item.
Reference is now made to
The color bar may be generated by, for example, assigning a color to each element (e.g., an image frame) or subgroup of elements in the data stream and then processing the series of colors, for example such that it may emphasize variations within the displayed properties. In one embodiment of the invention, it may be processed, for example to emphasize cue points in an accompanying video such that, for example, it may be used as an ancillary tool for indicating points of interest. In one embodiment of the invention, a stream of data 210 may be displayed along side one or more bars and/or graphical presentations (220 and 230) described herein. The data stream 210 may be for example data represented in the graphical presentation 220 (e.g. a captured in-vivo image stream) or other data obtained and/or sampled simultaneously or substantially simultaneously with the data represented in the graphical presentation 220. In one example, a marker or indicator 250 may progress across or along the graphical presentation 220 as the substantially corresponding datum in data stream 210 (e.g., video) may be concurrently displayed to indicate the correspondence between the graphical presentation 220 and the data stream 210. In other embodiments of the invention, the presentation may be of a shape other than a bar, for example a circle, oval, square, etc. According to other embodiments, the presentation may be in the form of an audio tract, graph, and other suitable graphic presentations.
An indicator 250 such as a cursor may advance along the time bar 230 and graphical presentation 220 as the image stream 210 is scrolled on the display 200. In one example, control buttons 240 may be included in the display that may allow the user to, for example, fast-forward, rewind, stop play or reach the beginning or end of, for example, an image stream 210. In other embodiments of the present invention, a user may control the display of a data stream 210, for example, by altering the start position of the streaming display, e.g. skipping to areas of interest, by moving the position of indicator 250, for example with a mouse or other pointing device. In other embodiments of the present invention, a user and/or health professional may insert indications or markers such as thumbnails to mark location along the image stream for easy access to those locations in the future. For example, a health professional may mark these milestones on the graphical presentation 220 (e.g., using a pointing device such as a mouse, a keyboard, etc). Some embodiments described in Published United States patent application US-2002-0171669-A1, entitled “System and method for annotation on a moving image”, published on Nov. 21, 2002, assigned to the assignee of the present invention and incorporated by reference herein in its entirety, include methods and devices to mark or annotate portions of an image stream; such methods may be used in conjunction with embodiments of the present invention. Other suitable methods for marking or annotating a stream of data may be used. A user may then “click” on the thumbnails to advance to the site of datum, for example the image frame, of interest or alternatively click on the graphical presentation 220 to advance or retract to the image frame, of interest and then, for example, continue or begin streaming and/or viewing the data stream from that desired point of interest.
Thumbnails or other markers may be defined based on an image of interest from the data stream 210, based on a location identified on the graphical presentation 220 or based on a time recorded on time bar 230. Other suitable methods of defining thumbnails or other markers or notations may be used. For example, a computer algorithm may be used to identify thumbnails that may be of interest to, for example, the health professional. Algorithm based thumbnails may also, for example, be based on an image of interest from the data stream 210, based on a location identified on the graphical presentation 220 or based on a time recorded on time bar 230, or other methods. In other embodiments, the graphical presentation 220 may be in itself a series of color thumbnails, so that a user may point or “click” on colors in the color bar to restart the display of the data stream from a different point in the stream.
In one embodiment of the present invention, the color scheme of image frames taken of tissue over time may change, for example as an in-vivo imaging device 40 travels along the GI tract. Changes in the color scheme of the images may be used to identify, for example, passage through a specific anatomical site, for example, the duodenal, cecum or other sites, and/or may indicate pathology, for example bleeding or other pathology. When presenting an image stream of a tissue in a summarized, concise color bar, the changes in color streams may be readily identified. For example, passage into the cecum may be identified by a color that may be typical to the large intestine, for example, a color that may indicate content or a color typical of the tissue found in the large intestine. Entrance into the duodenum may be identified by another color that may be typical of the tissue in the small intestine. Other anatomical sites may be identified by observing color and/or changing color streams on a color bar, for example, a tissue color bar. In other embodiments a pathological condition, such as for example, the presence of polyps, bleeding, etc., may be identified by viewing, for example, a tissue graphical presentation 220. A specific area of interest, such as pathology indicated by blood, may be directly identified through the tissue. As such a health professional may first examine the tissue graphical presentation 220 and only afterwards decide what block of images to review. In some embodiments of the present invention, an algorithm may be employed to identify anatomical sites, pathologies, or areas of interest using data from such a color bar and bring them to the attention of a health professional, by for example marking the area of interest along the displayed color bar. A health professional may use the thumbnails or markings along a tissue color bar, for example, markings and/or markings of the first gastric image 320, the first duodenum image 330 and the first cecum image 340 to locate where along the GI tract the data 210 (concurrently being displayed) may be originating from. Knowing the area at which an image was captured may help a health professional decide if an image viewed is representative of a healthy or pathological tissue, and may help a health professional to determine other conditions.
According to some embodiments, different colors or other visual indications, shades, hues, sizes or widths, etc. may be artificially added to a processed data stream, for example, in order to accentuate changes along the data stream. Other processing methods may be used to enhance the information presented to the user. In one embodiment of the invention smoothing may or may not be performed on selected pixels based on decision rules. For example in one embodiment of the invention smoothing may not be performed on dark pixels or on green pixels that may indicate content in the intestines.
Reference is now made to
In one embodiment of the present invention, color bar 226 may give indication of the presence of blood over a period of time. International application published as WO 02/26103 entitled “An immobilizable in vivo sensing device” assigned to the assignee of the present invention which is incorporated by reference herein in its entirety includes, inter alia, descriptions of embodiments of devices, such as capsules, that may be anchored at post-surgical sites. Embodiments described in International application WO 02/26103 may be used in conjunction with the system and methods described herein to capture and transmit data for an in-vivo site over time. A presentation of the captured data, for example a color bar may give indication of any changes occurring over time from a current static situation or may show an overview of how a tissue healed or changed over time without having to review the entire stream image by image.
Reference is now made to
Non-limiting examples of different types of graphical presentations (e.g., color bars, series of brightness levels, etc.) may include:
International application published as WO02102223, entitled “MOTILITY ANALYSIS WITHIN A GASTROINTESTINAL TRACT” assigned to the assignee of the present invention and incorporated by reference herein in its entirety includes, inter alia, a device, system, and methods for determining in-vivo motility that may be used in conjunction with the device, system, and method described herein. Other suitable representations other than bars and other suitable types of data may be implemented using the device, system, and method described herein.
Reference is now made to
Various suitable methods may be use to abstract data from the source data stream (e.g. an image stream, a series of temperature data) to the fixed representation. Reference is now made to
1+2 (alpha*N/Np); alpha=2.2;
where N is the original pixel size and Np is the desired pixel size of the resultant tissue color bar presentation. Other equations or formulae may be used.
In block 540 the pixel size of the resultant tissue color bar presentation may be set by decimating the vector of colors to a desired size, for example, decimating each color vector to the desired size by interpolation.
Other methods of generating a tissue color bar or other data summary may be implemented. In one embodiment, a series of data items, such as for example one or more individual images, may be converted to a data point, such as a color area or a color strip within a larger display area, such as a color bar. An average brightness value for each image or set of images may be found, and a bar or assemblage of strips of widths, patterns, colors or brightnesses corresponding to the averaged values may be generated. The values such as pH, pressure or temperature corresponding to each of an image or set of images (e.g., in a device collecting both image and other data) may be found, and a bar or assemblage of strips or other image units of widths, colors or brightnesses corresponding to the averaged values may be generated. One or more images may be converted or processed to a corresponding stripe of color. Various data items may be combined together to individual data points using, for example, averaging, smoothing, etc. In one embodiment the luminance of the images can be normalized and only normalized chromatic information of the data for example the tissue's color, can be shown, eliminating, for example, the contribution of the light source. Other color bars or other presentations of data obtained in-vivo other than imaged data may be generated.
Color bars and other representations of data may aid in reducing the viewing time necessary to review an image stream. A health professional may, in one embodiment of the present invention, use a pointer, for example, a mouse to point at an area along the color bar that may be of interest. The graphical user interface may in turn skip to the corresponding location on the data stream, so that a health professional may focus into the area of interest without having to review an entire image stream. A health professional may for example, change the rate at which to view different portions defined by a tissue color bar. A specific area of interest, such as pathology indicated by blood, may be directly identified through the tissue. As such a health professional may first examine the tissue color bar and only afterwards decide what block of images is he interested in reviewing. When screening patients it may be possible only to review one or more data presentations, such as a tissue color bar. In other examples, a summarized graphical presentation of a data stream may be generated in real time in for example a recorder 12, and displayed in real time on a display included in recorder 12.
In other embodiments of the present invention, a graphical presentation, for example, color bar may be used for other purposes besides presentation of in-vivo data. For example, a color bar may be used as a summarizing presentation of any stream of frames, for example a video. A summarized graphical presentation, for example a color bar as described herein, of a video may help a viewer to locate different scenes in a video and possibly fast forward, rewind or skip to that scene. For example, a scene in a movie that might have been filmed outdoors may for example have a different color scheme than a later or earlier scene that may have been filmed indoors. The color bar may be analogous to a color table of contents.
The foregoing description of the embodiments of the invention has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. It should be appreciated by persons skilled in the art that many modifications, variations, substitutions, changes, and equivalents are possible in light of the above teaching. It is, therefore, to be understood that the appended claims are intended to cover all such modifications and changes as fall within the true spirit of the invention.
This application is a continuation of prior U.S. patent application Ser. No. 12/615,794, filed Nov. 10, 2009, entitled “SYSTEM AND METHOD FOR PRESENTATION OF DATA STREAMS”, which is a continuation of prior U.S. patent application Ser. No. 11/723,253, filed on Mar. 19, 2007, which in turn is a divisional application of prior U.S. application Ser. No. 10/950,480, filed on Sep. 28, 2004 (now U.S. Pat. No. 7,215,338), which in turn claims benefit from prior U.S. provisional application Ser. No. 60/507,508 filed on Oct. 2, 2003 entitled, “DEVICE, SYSTEM AND METHOD FOR PRESENTATION OF IN-VIVO DATA” all of which are incorporated in its entirety by reference herein.
| Number | Date | Country | |
|---|---|---|---|
| 60507508 | Oct 2003 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 10950480 | Sep 2004 | US |
| Child | 11723253 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 12615794 | Nov 2009 | US |
| Child | 13398386 | US | |
| Parent | 11723253 | Mar 2007 | US |
| Child | 12615794 | US |
