Patent Application
20150201896
This invention relates generally to methods and systems of image analysis. More particularly, in certain embodiments, the invention relates to the automated detection of tissue interior to a mammalian rib cage.
There are a wide array of technologies directed to in vivo imaging of mammals—for example, bioluminescence, fluorescence, tomography, and multimodal imaging technologies. In vivo imaging of small mammals is performed by a large community of investigators in various fields, e.g., oncology, infectious disease, and drug discovery.
In vivo imaging often involves the use of reagents, such as fluorescent probes, for non-invasive spatiotemporal visualization of biological phenomena inside a live animal. For example, fluorescence molecular tomography (FMT) involves in vivo imaging of mammals for quantitative analysis of administered and/or endogenous probes. In vivo microCT imaging, is an x-ray-based technology that can image tissues, organs, and non-organic structures with an extremely high resolution. MicroCT has evolved quickly, requiring low dose scanning and fast imaging protocols to facilitate multi-modal applications and enable longitudinal experimental models. Multi-modal imaging involves the fusion of images obtained in different ways, for example, by combining FMT, PET, MRI, CT, and/or SPECT imaging data.
Acquisition of such in vivo images can be time consuming, and rapid analysis of the acquired images is key to the efficiency of the process. Often, it is desirable to focus imaging efforts on only those portions of the mammal interior to the rib cage, which contains many organs of interest and for which advanced image analysis is needed. Advanced image analysis may involve, for example, advanced tomographic reconstruction for the quantitative analysis of an administered or endogenous probe in one or more target organs of the mammal. The portions of an image outside those target organs may not be important to the analysis, and processing time spent on those portions is wasted and results in reduced efficiency.
There is a need for a highly efficient method for detecting regions of interest of an in vivo mammalian image in order to eliminate unnecessary processing of unimportant regions of the image and reduce overall image processing time without losing important image detail.
Described herein are systems and methods for efficient and accurate automated detection of a region of interest from an in vivo mammalian image. An important region of an imaged mammal is the tissue interior to the ribcage, where key organs lie. By automatically identifying this important region, further analysis and/or processing of the image can be performed on this region without unnecessarily wasting time processing other regions of the image.
An automated procedure for identifying this region is not a simple problem, since the procedure must accurately identify the desired region for a wide range of sizes and shapes of the animals being imaged (must account for variability of the individual animals being imaged), and must be a highly computationally efficient procedure.
It is found that efficient, automated identification of the region of interest interior to the ribcage can be achieved by the use of both a bone distance mask and a surface distance mask. The technique solves the problem of accurate and fast identification of the region of interest for a wide range of sizes and shapes of mammals, e.g., small mammals such as mice.
In one aspect, the invention is directed to a method for automatically detecting (e.g., graphically isolating) a region of interest in a 3D image of a mammal corresponding to (e.g., limited to) soft tissue largely or exclusively interior to the mammal rib cage, the method comprising: automatically determining from the 3D image of the mammal, by a processor of a computing device, a surface distance image comprising intensity values at each of a plurality of points in three dimensions, each of the intensity values corresponding to a distance from a given point in 3D space to the nearest point on the outer surface of the mammal; automatically determining, by the processor, a bone distance image comprising intensity values at each of a plurality of points in three dimensions, each of the intensity values corresponding to a distance from a given point in 3D space to the nearest identified bone tissue of the mammal; and automatically detecting, by the processor, the region of interest corresponding to soft tissue interior to the mammal rib cage using the surface distance image and the bone distance image.
In some embodiments, the detecting step comprises: applying, by the processor, a threshold to the surface distance image to determine a surface distance mask corresponding to a central region of the mammal body (e.g., wherein the surface distance mask identifies one or more discrete volumes within the mammal body); applying, by the processor, a threshold to the bone distance image to determine a bone distance mask corresponding to one or more regions at least a given distance from identified bone tissue of the mammal (e.g., wherein the bone distance mask identifies one or more discrete volumes within the mammal body); applying, by the processor, an AND operation of the surface distance mask and the bone distance mask to identify only a region interior to the rib cage, within the region of interest (e.g., wherein the region identified as a result of the AND operation defines one or more discrete volumes within the mammal body) (and, optionally, applying a filter, e.g., a volume threshold, to the result of the AND operation to remove one or more volumes that lie outside the ribcage); and applying, by the processor, a dilation operation and/or a watershed operation to expand outward a region identified following the AND operation outward (e.g., and following any subsequent filtering step) that is interior to the ribcage, without encroaching the identified bone tissue, thereby detecting the region of interest interior to the mammal rib cage. For example, there may be one discrete volume identified as a result of the AND operation, where the one discrete volume is located interior to the ribcage, and dilation
In some embodiments, the method further comprises determining, by the processor, a transverse plane corresponding to a lower end of the rib cage. In some embodiments, determining the transverse plane comprises estimating a transverse coordinate corresponding to the lower end of the rib cage from the surface distance image and the bone distance image.
In some embodiments, the detecting step comprises: automatically determining, by the processor, a search region mask from values of distance from front (abdominal) surface to nearest bone expressed as a function of nose-to-tail z-coordinate, said search region mask identifying a volume of the mammal excluding regions of the head above the neck and excluding regions below the rib cage; automatically determining, by the processor, a bone distance mask using the bone distance image (e.g., by applying a threshold to the bone distance image), wherein the bone distance mask corresponds to one or more regions at least a given distance from identified bone tissue of the mammal; automatically determining, by the processor, a surface distance mask using the surface distance image (e.g., by applying a threshold to the surface distance image) corresponding to one or more regions at least a given distance from the outer surface of the mammal; applying, by the processor, an AND operation of the search region mask, the bone distance mask, and the surface distance mask, then, optionally, applying a filter to the result of the AND operation to remove small distinct regions smaller than a determined threshold volume, thereby identifying seed a region interior to the rib cage, within the region of interest; and applying, by the processor, a dilation operation and/or a watershed operation to expand the seed region identified following the AND operation outward, without encroaching the identified bone tissue, then applying, by the processor, an AND operation of the result of the dilation and/or watershed operation with the search region mask, thereby detecting the region of interest interior to the mammal rib cage.
In some embodiments, the detecting step comprises: automatically determining, by the processor, a search region mask using values of distance from front (abdominal) surface to nearest bone expressed as a function of nose-to-tail z-coordinate, said search region mask identifying a volume of the mammal excluding regions of the head above the neck and excluding regions below the rib cage; automatically determining, by the processor, a bone distance mask using the bone distance image, wherein the bone distance mask corresponds to one or more regions at least a given distance from identified bone tissue of the mammal; automatically determining, by the processor, a composite distance image computed as the difference between the surface distance image and the bone distance image (e.g., wherein intensity values of the composite distance image reflect the difference between distance from bones and distance from outer surface of the mammal); applying, by the processor, a threshold to the composite distance image to determine a composite distance mask; applying, by the processor, an AND operation of the search region mask, the bone distance mask, and the composite distance mask, then, optionally, applying a filter to the result of the AND operation to remove small distinct regions smaller than a determined threshold volume, thereby identifying a seed region interior to the rib cage, within the region of interest; and applying, by the processor, a dilation operation and/or a watershed operation to expand the seed region identified following the AND operation outward, without encroaching the identified bone tissue, then applying, by the processor, an AND operation of the result of the dilation and/or watershed operation with the search region mask, thereby detecting the region of interest interior to the mammal rib cage.
In some embodiments, the detecting step comprises: automatically determining, by the processor, a bone distance mask using the bone distance image, wherein the bone distance mask corresponds to one or more regions at least a given distance from identified bone tissue of the mammal; automatically determining, by the processor, a composite distance image computed as the difference between the surface distance image and the bone distance image (e.g., wherein intensity values of the composite distance image reflect the difference between distance from bones and distance from outer surface of the mammal); applying, by the processor, a threshold to the composite distance image to determine a composite distance mask; applying, by the processor, an AND operation of the bone distance mask and the composite distance mask, then, optionally, applying a filter to the result of the AND operation to remove small distinct regions smaller than a determined threshold volume, thereby identifying an intermediate result; and automatically determining, by the processor, using the composite distance mask, a transverse plane corresponding to a lower end of the rib cage (e.g., identifying a z-coordinate along the length of the mammal where cross-section of the composite distance mask has maximal area), and removing from the intermediate result all volumes below said transverse plane, thereby identifying a seed region interior to the rib cage, within the region of interest, then applying, by the processor, a dilation operation and/or a watershed operation to expand the resulting seed region outward, without encroaching the identified bone tissue, then, optionally, filtering all volumes below said determined transverse plane, thereby detecting the region of interest interior to the mammal rib cage.
In another aspect, the invention is directed to a method for automatically detecting (e.g., graphically isolating) a region of interest in a 3D image of a mammal corresponding to (e.g., limited to) soft tissue largely or exclusively interior to the mammal rib cage, the method comprising:
at least one of (a) and (b), as follows:
(a) automatically determining using the 3D image of the mammal, by the processor, a surface distance image comprising intensity values at each of a plurality of points in three dimensions, each of the intensity values corresponding to a distance from a given point in 3D space to the nearest point on the outer surface of the mammal, and applying, by the processor, a threshold to the surface distance image to determine a surface distance mask corresponding to a central region of the mammal body (e.g., wherein the surface distance mask identifies one or more discrete volumes within the mammal body);
(b) automatically determining using the 3D image of the mammal, by a processor of a computing device, a body mask corresponding to mammal body, and applying an erosion operation to the body mask (e.g., using a disk kernel of radius R′) to obtain a surface distance mask equivalent;
and at least one of (c) and (d), as follows:
(c) automatically determining using the 3D image of the mammal, by the processor, a bone distance image comprising intensity values at each of a plurality of points in three dimensions, each of the intensity values corresponding to a distance from a given point in 3D space to the nearest identified bone tissue of the mammal, and applying, by the processor, a threshold to the bone distance image to determine a bone distance mask corresponding to one or more regions at least a given distance from identified bone tissue of the mammal (e.g., wherein the bone distance mask identifies one or more volumes within the mammal body);
(d) automatically determining using the 3D image of the mammal, by the processor, a bone mask corresponding to bone tissue, and applying a dilation operation to the bone mask (e.g., using a disk kernel of radius R) (e.g., and invert) to obtain a bone distance mask equivalent;
and automatically detecting, by the processor, the region of interest corresponding to soft tissue interior to the mammal rib cage using (i) at least one of the surface distance mask and the surface distance mask equivalent, and (ii) at least one of the bone distance mask and the bone distance mask equivalent.
In some embodiments, the method further comprises determining, by the processor, a transverse plane corresponding to a lower end of the rib cage. In some embodiments, determining the transverse plane comprises estimating a transverse coordinate corresponding to the lower end of the rib cage from the surface distance image and the bone distance image.
In some embodiments, the detecting step comprises: automatically determining, by the processor, a search region mask from values of distance from front (abdominal) surface to nearest bone expressed as a function of nose-to-tail z-coordinate, said search region mask identifying a volume of the mammal excluding regions of the head above the neck and excluding regions below the rib cage; applying, by the processor, an AND operation of the search region mask, the bone distance mask (or bone distance mask equivalent), and the surface distance mask (or surface distance mask equivalent), then, optionally, applying a filter to the result of the AND operation to remove small distinct regions smaller than a determined threshold volume, thereby identifying a seed region interior to the rib cage, within the region of interest; and applying, by the processor, a dilation operation and/or a watershed operation to expand the seed region identified following the AND operation outward, without encroaching the identified bone tissue, then applying, by the processor, an AND operation of the result of the dilation and/or watershed operation with the search region mask, thereby detecting the region of interest interior to the mammal rib cage.
In some embodiments, the detecting step comprises: automatically determining, by the processor, a search region mask using values of distance from front (abdominal) surface to nearest bone expressed as a function of nose-to-tail z-coordinate, said search region mask identifying a volume of the mammal excluding regions of the head above the neck and excluding regions below the rib cage; automatically determining, by the processor, a composite distance image computed as the difference between the surface distance image and the bone distance image (e.g., wherein intensity values of the composite distance image reflect the difference between distance from bones and distance from outer surface of the mammal); applying, by the processor, a threshold to the composite distance image to determine a composite distance mask; applying, by the processor, an AND operation of the search region mask, the bone distance mask, and the composite distance mask, then, optionally, applying a filter to the result of the AND operation to remove small distinct regions smaller than a determined threshold volume, thereby identifying a seed region interior to the rib cage, within the region of interest; and applying, by the processor, a dilation operation and/or a watershed operation to expand the seed region identified following the AND operation outward, without encroaching the identified bone tissue, then applying, by the processor, an AND operation of the result of the dilation and/or watershed operation with the search region mask, thereby detecting the region of interest interior to the mammal rib cage.
In some embodiments, detecting step comprises: automatically determining, by the processor, a composite distance image computed as the difference between the surface distance image and the bone distance image (e.g., wherein intensity values of the composite distance image reflect the difference between distance from bones and distance from outer surface of the mammal); applying, by the processor, a threshold to the composite distance image to determine a composite distance mask; applying, by the processor, an AND operation of the bone distance mask and the composite distance mask, then, optionally, applying a filter to the result of the AND operation to remove small distinct regions smaller than a determined threshold volume, thereby identifying an intermediate result; and automatically determining, by the processor, using the composite distance mask, a transverse plane corresponding to a lower end of the rib cage (e.g., identifying a z-coordinate along the length of the mammal where cross-section of the composite distance mask has maximal area), and removing from the intermediate result all volumes below said transverse plane, thereby identifying a seed region interior to the rib cage, within the region of interest, then applying, by the processor, a dilation operation and/or a watershed operation to expand the resulting seed region outward, without encroaching the identified bone tissue, then, optionally, filtering all volumes below said determined transverse plane, thereby detecting the region of interest interior to the mammal rib cage.
In another aspect, the invention is directed to a system comprising: a processor; and a memory having instructions stored thereon, wherein the instructions, when executed by the processor, cause the processor to:
(i) at least one of (a) and (b), as follows:
and
(iii) automatically detect, by the processor, the region of interest corresponding to soft tissue interior to the mammal rib cage using (i) at least one of the surface distance image and the surface distance image equivalent, and (ii) at least one of the bone distance image and the bone distance image equivalent.
In another aspect, the invention is directed to a non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed by a processor, cause the processor to:
(i) at least one of (a) and (b), as follows:
and
(ii) at least one of (c) and (d), as follows:
The foregoing and other objects, aspects, features, and advantages of the invention will become more apparent and may be better understood by referring to the following description taken in conjunction with the accompanying drawings, in which:
It is contemplated that systems, devices, methods, and processes of the claimed invention encompass variations and adaptations developed using information from the embodiments described herein. Adaptation and/or modification of the systems, devices, methods, and processes described herein may be performed by those of ordinary skill in the relevant art.
Throughout the description, where articles, devices, and systems are described as having, including, or comprising specific components, or where processes and methods are described as having, including, or comprising specific steps, it is contemplated that, additionally, there are articles, devices, and systems of the present invention that consist essentially of, or consist of, the recited components, and that there are processes and methods according to the present invention that consist essentially of, or consist of, the recited processing steps.
It should be understood that the order of steps or order for performing certain action is immaterial so long as the invention remains operable. Moreover, two or more steps or actions may be conducted simultaneously.
The mention herein of any publication, for example, in the Background section, is not an admission that the publication serves as prior art with respect to any of the claims presented herein. The Background section is presented for purposes of clarity and is not meant as a description of prior art with respect to any claim.
As used herein, an “image”—for example, a 3D image of mammal—includes any visual representation, such as a photo, a video frame, streaming video, as well as any electronic, digital or mathematical analogue of a photo, video frame, or streaming video. Any apparatus described herein, in certain embodiments, includes a display for displaying an image or any other result produced by the processor. Any method described herein, in certain embodiments, includes a step of displaying an image or any other result produced via the method.
As used herein, “3D” or “three-dimensional” with reference to an “image” means conveying information about three dimensions. A 3D image may be rendered as a dataset in three dimensions and/or may be displayed as a set of two-dimensional representations, or as a three-dimensional representation.
As used herein, a “mask” is a graphical pattern that identifies a 2D or 3D region and is used to control the elimination or retention of portions of an image or other graphical pattern.
As used herein, a “seed region” is an identified region of an image or mask which is used to calculate another, larger region, e.g., by using a dilation and/or watershed operation. Example seed regions are shown, for example, in
Described herein are systems and methods for automated detection of a region of interest interior to a mammalian ribcage from an in vivo 3D image. The 3D image may be single image or a set of multiple images.
The purpose of the step illustrated by
Next, an AND-operation is performed among the masks of
Finally, a dilation operation is applied to the mask of
The purpose of the step illustrated by
Next, an AND-operation is performed among the masks of
Finally, a dilation operation is applied to the mask of
Unlike in the preceding Examples i and ii, there is no determination of a search region from a plot of abdomen-surface-to-bone-distance as a function of z-coordinate here in Example iii. Rather, in Example iii, there is a search region defined by a single transverse plane, which is estimated using a different method.
A threshold is applied to the distance from bones minus distance from mouse surface, as determined from the two distance images (distance from mouse surface and distance from mouse bones). The resulting mask is shown in
Next, an AND-operation is performed among the masks of
The purpose of the step illustrated by
This leaves an unwanted region below the ribcage. To remove the region, the z-coordinate corresponding to the base of the ribcage is determined as the coordinate where the cross-section of the mask of
Finally, a dilation operation is applied to the mask of
As illustrated in Examples i and ii, an important feature in exemplary methods for identification of the region of interest interior to the ribcage is the use of both a bone distance mask and a surface distance mask.
A bone distance mask can be determined from a bone distance image, as in the examples above, or, alternatively (and equivalently), a bone distance mask can be determined directly from the bone mask as identified from a 3D image of the mammal. Similarly, a surface distance mask can be determined from a surface distance image, as in Examples i, ii, and iii above, or, alternatively (and equivalently), a surface distance mask can be determined directly from the mammal body (surface) mask as identified from a 3D image of the mammal.
Further clarifying this point,
Then, in step 540, the bone distance mask (or bone distance mask equivalent) and the surface distance mask (or surface distance mask equivalent) is used to determine the desired region of interest interior to the ribcage.
The ribcage is open from below, and regions of the resulting mask below the plane of the ribcage should be removed.
The cloud computing environment 800 may include a resource manager 806. The resource manager 806 may be connected to the resource providers 802 and the computing devices 804 over the computer network 808. In some implementations, the resource manager 806 may facilitate the provision of computing resources by one or more resource providers 802 to one or more computing devices 804. The resource manager 806 may receive a request for a computing resource from a particular computing device 804. The resource manager 806 may identify one or more resource providers 802 capable of providing the computing resource requested by the computing device 804. The resource manager 806 may select a resource provider 802 to provide the computing resource. The resource manager 806 may facilitate a connection between the resource provider 802 and a particular computing device 804. In some implementations, the resource manager 806 may establish a connection between a particular resource provider 802 and a particular computing device 804. In some implementations, the resource manager 806 may redirect a particular computing device 804 to a particular resource provider 802 with the requested computing resource.
The computing device 900 includes a processor 902, a memory 904, a storage device 906, a high-speed interface 908 connecting to the memory 904 and multiple high-speed expansion ports 910, and a low-speed interface 912 connecting to a low-speed expansion port 914 and the storage device 906. Each of the processor 902, the memory 904, the storage device 906, the high-speed interface 908, the high-speed expansion ports 910, and the low-speed interface 912, are interconnected using various busses, and may be mounted on a common motherboard or in other manners as appropriate. The processor 902 can process instructions for execution within the computing device 900, including instructions stored in the memory 904 or on the storage device 906 to display graphical information for a GUI on an external input/output device, such as a display 916 coupled to the high-speed interface 908. In other implementations, multiple processors and/or multiple buses may be used, as appropriate, along with multiple memories and types of memory. Also, multiple computing devices may be connected, with each device providing portions of the necessary operations (e.g., as a server bank, a group of blade servers, or a multi-processor system).
The memory 904 stores information within the computing device 900. In some implementations, the memory 904 is a volatile memory unit or units. In some implementations, the memory 904 is a non-volatile memory unit or units. The memory 904 may also be another form of computer-readable medium, such as a magnetic or optical disk.
The storage device 906 is capable of providing mass storage for the computing device 900. In some implementations, the storage device 906 may be or contain a computer-readable medium, such as a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory or other similar solid state memory device, or an array of devices, including devices in a storage area network or other configurations. Instructions can be stored in an information carrier. The instructions, when executed by one or more processing devices (for example, processor 902), perform one or more methods, such as those described above. The instructions can also be stored by one or more storage devices such as computer- or machine-readable mediums (for example, the memory 904, the storage device 906, or memory on the processor 902).
The high-speed interface 908 manages bandwidth-intensive operations for the computing device 900, while the low-speed interface 912 manages lower bandwidth-intensive operations. Such allocation of functions is an example only. In some implementations, the high-speed interface 908 is coupled to the memory 904, the display 916 (e.g., through a graphics processor or accelerator), and to the high-speed expansion ports 910, which may accept various expansion cards (not shown). In the implementation, the low-speed interface 912 is coupled to the storage device 906 and the low-speed expansion port 914. The low-speed expansion port 914, which may include various communication ports (e.g., USB, Bluetooth®, Ethernet, wireless Ethernet) may be coupled to one or more input/output devices, such as a keyboard, a pointing device, a scanner, or a networking device such as a switch or router, e.g., through a network adapter.
The computing device 900 may be implemented in a number of different forms, as shown in the figure. For example, it may be implemented as a standard server 920, or multiple times in a group of such servers. In addition, it may be implemented in a personal computer such as a laptop computer 922. It may also be implemented as part of a rack server system 924. Alternatively, components from the computing device 900 may be combined with other components in a mobile device (not shown), such as a mobile computing device 950. Each of such devices may contain one or more of the computing device 900 and the mobile computing device 950, and an entire system may be made up of multiple computing devices communicating with each other.
The mobile computing device 950 includes a processor 952, a memory 964, an input/output device such as a display 954, a communication interface 966, and a transceiver 968, among other components. The mobile computing device 950 may also be provided with a storage device, such as a micro-drive or other device, to provide additional storage. Each of the processor 952, the memory 964, the display 954, the communication interface 966, and the transceiver 968, are interconnected using various buses, and several of the components may be mounted on a common motherboard or in other manners as appropriate.
The processor 952 can execute instructions within the mobile computing device 950, including instructions stored in the memory 964. The processor 952 may be implemented as a chipset of chips that include separate and multiple analog and digital processors. The processor 952 may provide, for example, for coordination of the other components of the mobile computing device 950, such as control of user interfaces, applications run by the mobile computing device 950, and wireless communication by the mobile computing device 950.
The processor 952 may communicate with a user through a control interface 958 and a display interface 956 coupled to the display 954. The display 954 may be, for example, a TFT (Thin-Film-Transistor Liquid Crystal Display) display or an OLED (Organic Light Emitting Diode) display, or other appropriate display technology. The display interface 956 may comprise appropriate circuitry for driving the display 954 to present graphical and other information to a user. The control interface 958 may receive commands from a user and convert them for submission to the processor 952. In addition, an external interface 962 may provide communication with the processor 952, so as to enable near area communication of the mobile computing device 950 with other devices. The external interface 962 may provide, for example, for wired communication in some implementations, or for wireless communication in other implementations, and multiple interfaces may also be used.
The memory 964 stores information within the mobile computing device 950. The memory 964 can be implemented as one or more of a computer-readable medium or media, a volatile memory unit or units, or a non-volatile memory unit or units. An expansion memory 974 may also be provided and connected to the mobile computing device 950 through an expansion interface 972, which may include, for example, a SIMM (Single In Line Memory Module) card interface. The expansion memory 974 may provide extra storage space for the mobile computing device 950, or may also store applications or other information for the mobile computing device 950. Specifically, the expansion memory 974 may include instructions to carry out or supplement the processes described above, and may include secure information also. Thus, for example, the expansion memory 974 may be provided as a security module for the mobile computing device 950, and may be programmed with instructions that permit secure use of the mobile computing device 950. In addition, secure applications may be provided via the SIMM cards, along with additional information, such as placing identifying information on the SIMM card in a non-hackable manner.
The memory may include, for example, flash memory and/or NVRAM memory (non-volatile random access memory), as discussed below. In some implementations, instructions are stored in an information carrier and, when executed by one or more processing devices (for example, processor 952), perform one or more methods, such as those described above. The instructions can also be stored by one or more storage devices, such as one or more computer- or machine-readable mediums (for example, the memory 964, the expansion memory 974, or memory on the processor 952). In some implementations, the instructions can be received in a propagated signal, for example, over the transceiver 968 or the external interface 962.
The mobile computing device 950 may communicate wirelessly through the communication interface 966, which may include digital signal processing circuitry where necessary. The communication interface 966 may provide for communications under various modes or protocols, such as GSM voice calls (Global System for Mobile communications), SMS (Short Message Service), EMS (Enhanced Messaging Service), or MMS messaging (Multimedia Messaging Service), CDMA (code division multiple access), TDMA (time division multiple access), PDC (Personal Digital Cellular), WCDMA (Wideband Code Division Multiple Access), CDMA2000, or GPRS (General Packet Radio Service), among others. Such communication may occur, for example, through the transceiver 968 using a radio-frequency. In addition, short-range communication may occur, such as using a Bluetooth®, Wi-Fi™, or other such transceiver (not shown). In addition, a GPS (Global Positioning System) receiver module 970 may provide additional navigation- and location-related wireless data to the mobile computing device 950, which may be used as appropriate by applications running on the mobile computing device 950.
The mobile computing device 950 may also communicate audibly using an audio codec 960, which may receive spoken information from a user and convert it to usable digital information. The audio codec 960 may likewise generate audible sound for a user, such as through a speaker, e.g., in a handset of the mobile computing device 950. Such sound may include sound from voice telephone calls, may include recorded sound (e.g., voice messages, music files, etc.) and may also include sound generated by applications operating on the mobile computing device 950.
The mobile computing device 950 may be implemented in a number of different forms, as shown in the figure. For example, it may be implemented as a cellular telephone 980. It may also be implemented as part of a smart-phone 982, personal digital assistant, or other similar mobile device.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.
These computer programs (also known as programs, software, software applications or code) include machine instructions for a programmable processor, and can be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms machine-readable medium and computer-readable medium refer to any computer program product, apparatus and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term machine-readable signal refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.
The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
While the invention has been particularly shown and described with reference to specific preferred embodiments, it should be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
