This invention relates to sensing systems and methods, and more specifically to system and method for determining an image of a distribution of permittivity of a material.
Knowledge of the spatial distribution of the dielectric permittivity within a material is important for many applications such as microwave imaging, biomicroscopy, medical imaging, through-the-wall imaging (TWI), infrastructure monitoring, and seismic imaging. In particular, determination of permittivity enables the visualization of the internal structure of the material and characterization of its physical properties. For example, in microwave imaging permittivity provides the structure and properties of objects in the material. In biomicroscopy, the permittivity allows to visualize the internal cell structure in three-dimensions. In TWI, the permittivity allows to learn the dielectric properties of the wall and to use that information to compensate for the delay of the signal propagating through the wall.
In a typical scenario, a transmitter emits a signal such as an electromagnetic (EM) or light pulse, which propagates through the material, reflects off various structures inside the material, and propagates to a receiver antenna array. The composition of the material is then visualized by numerically generating an image that represents the distribution of the permittivity in the material. However, depending on the type of material, the received signal often resulted from the multiple reflections of the propagating pulse due to multiple scattering from the structures in the material, which results in artifacts that clutter the reconstructed image.
Accordingly, there is a need for a method determining an image of a distribution of permittivity of a material that accounts for the multiple scattering of the pulse of light propagating through the material. However, the multiple scattering of the pulse affects the pulse in a non-linear manner, making such a determination more difficult.
Some embodiments of the invention are based on realization that scattering of a pulse of wave propagated through a material of an object can be represented by a neural network with varying functions of the nodes of the network and fixed numeric weights connecting the nodes. The functions of the nodes represent permittivity of different portions of the material and the weights of the connections represent the physics of scattering the pulse by corresponding portions of the material. Such a neural network can capture the nonlinearity of scattering the pulse by different portions of the material in a manner allowing determining the permittivity of the material of the object from effects of the permittivity on the result of the scattering.
The neural networks are a family of models inspired by biological neural networks and are used to estimate or approximate functions that can depend on a large number of inputs and are generally unknown. The neural networks are generally presented as systems of interconnected nodes or “neurons” that exchange messages between each other. Each node is associated with a function for transforming the message. This function is usually non-linear to form a non-linear part of message transformation. Each connection between the nodes is associated with a numeric weight for scaling of the messages to form a linear part of message transformation. Typically, the functions are fixed and predetermined for all nodes, e.g., selected by a designer of the neural network. Examples of the functions typically selected for the nodes include the sigmoid and rectifier functions. In contrast, the numeric weights are different and tuned based on experience, making the neural network adaptive to inputs and capable of learning.
However, in the neural network used by different embodiments of the invention, such a relationship is reversed, i.e., the functions of the nodes are varied and determined during the training of the neural network, and the numeric weights of the connections between the nodes are fixed. Some embodiments of the invention are based on recognition that regardless of such a reverse relationships, the values of the nodes of the neural network can be learned using a backpropagation within amount of time comparable with the amount of time required for training the usual neural networks.
Accordingly, one embodiment of the invention discloses a method for determining an image of a distribution of permittivity of a material. The method includes propagating a pulse of wave through the material to receive a set of echoes resulted from scattering the pulse by different portions of the material; simulating a propagation of the pulse in the material using a neural network to determine a simulated set of echoes, wherein each node in a layer of the neural network corresponds to a portion of the material and assigned a value the permittivity of the portion of the material, such that the values of the nodes at locations of the portions form the image of the distribution of the permittivity of the material, and wherein connections between layers in the neural network models scattering events; and updating the values of the nodes by reducing an error between the received set of echoes and the simulated set of echoes to produce the image of the distribution of the permittivity of the material. At least some steps of the method are performed by a processor.
Another embodiment of the invention discloses a permittivity sensor for determining an image of a distribution of permittivity of a material, including at least one transceiver to propagate a pulse of wave through the material and to receive a set of echoes resulted from scattering the pulse by different portions of the material; and a processor to simulate a propagation of the pulse in the material using a neural network to determine a simulated set of echoes, wherein each node in a layer of the neural network corresponds to a portion of the material and assigned a value the permittivity of the portion of the material, such that the values of the nodes at locations of the portions form the image of the distribution of the permittivity of the material, and wherein connections between layers in the neural network models scattering events, and to update the values of the nodes reducing an error between the measured set of echoes and the simulated set of echoes to produce the image of the distribution of the permittivity of the material.
Yet another embodiment of the invention discloses a non-transitory computer readable storage medium embodied thereon a program executable by a processor for performing a method that includes requesting to propagate a pulse of wave through the material to receive a set of echoes resulted from scattering the pulse by different portions of the material; simulating a propagation of the pulse in the material using a neural network to determine a simulated set of echoes, wherein each node in a layer of the neural network corresponds to a portion of the material and assigned a value the permittivity of the portion of the material, such that the values of the nodes at locations of the portions form the image of the distribution of the permittivity of the material, and wherein connections between layers in the neural network models scattering events; and updating the values of the nodes reducing an error between the received set of echoes and the simulated set of echoes to produce the image of the distribution of the permittivity of the material.
For example, the transceiver can include at least one transmitter that transmits the pulse through the material, such that the pulse scattered by the material produces the set of echoes 137. The pulse can be any type of electromagnetic or optical waves, such as one or combination of a microwave pulse, a radar pulse, a laser pulse, an ultrasound pulse, an acoustic pulse. The transceiver can also include at least one receiver arranged at a predetermined location with respect to the transmitter for receiving the set of echoes 137. According to different embodiments, the permittivity sensor can produce a two- or three-dimensional image of the material, where each location in the image provides the value of the dielectric permittivity for a portion of material corresponding to that location.
The permittivity sensor also includes a processor 140 operatively connected with the transceiver 130 to determine the image 110 based on the set of echoes 137. In order to account for multiple scattering, the processor uses a neural network 150, where each node is the value of permittivity. The processor simulates a propagation of the pulse in the material using the neural network 150 to determine a simulated set of echoes and updates the values of the nodes reducing an error between the received set of echoes and the simulated set of echoes to produce the image 110 of the distribution of the permittivity of the material.
The method propagates 210 a pulse of wave 135 through the material to receive a set of echoes 137 resulted from scattering the pulse by different portions of the material. The method also simulates 220 a propagation of the pulse in the material using a neural network 150 to determine a simulated set of echoes 225. In some embodiments, each node in a layer of the neural network 150 corresponds to a portion of the material and assigned a value the permittivity of the portion of the material, such that the values of the nodes at locations of the portions form the image of the distribution of the permittivity of the material. Also, the propagation between two layers in the neural network models a scattering event, as described below. The method further updates 230 the values of the nodes in the network 150 by reducing an error between the received set of echoes and the simulated set of echoes to produce the image of the distribution of the permittivity of the material.
In some embodiments the processor updates the neural network 150 iteratively. For example, for each iteration, the processor simulates the set of echoes using the neural network and compares 270 the results of the simulation with the received set of echoes 255. The processor updates 260 the network 150 to produce a current image 265 of the distribution of permittivity. The iterations are repeated until a termination condition 275 is met. Examples of the termination condition include a maximal number of iterations and/or the size of the error. In some embodiments, additional transmitted and reflected pulses, as well as total variation denoising 280 are used to further improve the image to better fit the measurements.
Some embodiments of the invention are based on realization that scattering of a pulse of wave propagated through a material of an object can be represented by a neural network with varying functions of the nodes of the network and fixed numeric weights connecting the nodes. The functions of the nodes represent permittivity of different portions of the material and the weights of the connections represent the physics of scattering the pulse by corresponding portions of the material. Such a neural network can capture the nonlinearity of scattering the pulse by different portions of the material in a manner allowing determining the permittivity of the material of the object from effects of the permittivity on the result of the scattering.
The neural networks are a family of models inspired by biological neural networks and are used to estimate or approximate functions that can depend on a large number of inputs and are generally unknown. The neural networks are generally presented as systems of interconnected nodes or “neurons” that exchange messages between each other. Each node is associated with a function for transforming the message. This function is usually non-linear to form a non-linear part of message transformation. Each connection between the nodes is associated with a numeric weight for scaling of the messages to form a linear part of message transformation. Typically, the functions are fixed and predetermined for all nodes, e.g., selected by a designer of the neural network. Examples of the functions typically selected for the nodes include the sigmoid and rectifier function. In contrast, the numeric weights are different and tuned based on experience, making the neural network adaptive to inputs and capable of learning.
However, in the neural network used by different embodiments of the invention, such a relationship is reversed, i.e., the functions of the nodes are varied and determined during the training of the neural network, and the numeric weights of the connections between the nodes are fixed. Some embodiments of the invention are based on recognition that regardless of such a reverse relationships, the values of the nodes of the neural network can be learned using a backpropagation within amount of time comparable with the amount of time required for training the usual neural networks.
In some embodiments, the number of portions 320 corresponds to the resolution of the image 310. To that end, one embodiment determines a resolution of the image defining a number of pixels or voxels in the image and determines the set of portions according to the resolution of the image, such that each portion of the material corresponds to a pixel in the image. For example, for the desired resolution of 80×60 pixels in the image 110, the combination 320 has 4800 portions.
The embodiment forms 330 a layer 351 of the neural network using a set of nodes having one-to-one relationship with the set of portions of the material. Each node in the set corresponds to only one portion of the material. For example, the node 360 corresponds to the portion 325. The layers are repeated as many times as desired to represent an additional scattering event. To that end, the layers 351 and 352 are identical. For example, 30 layers of the network 150 represent 30 scattering events from each portion of the material.
The connection 370 between the nodes of the neural network 150 model 340 the scattering of the pulse by convolving the nodes in neighboring layers with a function of physics of the scattering. For example, one embodiment uses Green's function to model the physics of the scattering.
Simulation
Some embodiments of the invention address a scattering problem, where a material of dielectric permittivity distribution ε (X), with x=(x, y, z) ∈ Ω and Ω ⊂3, is immersed into the background medium of permittivity εb. The sources that generate the excitation pulses and sensors collecting the data are located in the sensor region Γ ⊂3. The incident electric field created by the lth source, located at xl ∈ Γ, is denoted as uin(x, xl) for all x ∈ 3.
The Lippmann-Schwinger equation describes the relationship between the permittivity and the wave-field in the material as
u(x,x,l)=uin(x,x,l)+∫Ωg(x−x′)ƒ(x′)u(x′, xl) dx′, (1)
for all x ∈ Ω, where we define the scattering potential
ƒ(x)kb2(εb−ε(x)), (2)
and the Green's function for the homogeneous medium g (x). Similarly, the scattered field in the sensor region can be expressed as
for any x ∈ Γ. Note that the integrals (1) and (4) extend only over Ω because the scattering potential ƒ is zero for all x ∉ Ω.
Various embodiments determine the function ƒ, which is equivalent to dielectric permittivity ε, given transmissions by L sources, where each transmission includes M measurements of {uscl(xm,xl)}m∈[1 . . . M] in Γ. Notably, the internal field u=uin+usc inside the integral depends on usc, which highlights the nonlinear nature of the problem.
The recursive algorithm for simulating the scattered wave at the sensor locations can be specified as follows
Z
m←Σn=1N HmnUnKƒn, (5)
unk←un0+Σi=1NGniu iK−1ƒi (6)
where m=1, . . . , M, k=1, . . . , K, n=1, . . . , N. Here, the vector f ∈ N is the discretization of the scattering potential ƒ, z ∈ M is the predicted scattered wave usc at sensor locations {xm}m∈[1 . . . M], u0 ∈N is the discretization of the input field uin inside Ω, H ∈ M×N is the discretization of the Green's function at sensor locations, G ∈ N×N is the discretization of the Green's function inside Ω. For every k ∈ [1, . . . K], the vector uk ∈N denotes discretized version of the internal field after k scatterings. One embodiment recursively updates the discretized version of the wave field to satisfy the field equation (3-4).
Updating Image
The image is updated by making a step towards minimization of an error function
(f)=D(f)+τ(f), (7)
where and are is the data-fidelity and regularization terms, respectively, and τ>0 controls the amount of regularization. The physical constraints, such as for example non-negativity of the scattering potential, are enforced by projecting the image to a convex set ⊂N.The data-fidelity term is given by
where y ∈ M contains measurements of the scattered wave and z is the field simulated by our network. As a regularization term, we propose to use isotropic total variation penalty
where D: N→N×2 is the discrete gradient operator with matrices Dx and Dy denoting the finite difference operations along x and y directions, respectively.
A single step (7) of optimization is performed by using a proximal-gradient scheme or its accelerated variant as follows
f
t←(ft−1−γ∇(ft−1)), (8)
where γ>0 is a step-size and
is the proximal operator, which corresponds to total variation denoising. Note that, although, the proximal operator for isotropic total-variation does not admit a closed form, it can be numerically computed. The gradient ∇ can be obtained by evaluating
where the Hermitian transposition H is due to the complex nature of quantities. We adopt the following convention for the Jacobian
Then, by differentiating equations in (0) with respect to f and simplifying the resulting expressions, we have for any vectors b ∈ M and r ∈ N
where k=1, . . . K, vector
g
k
←g
K+1
+[G
H
r
k+1
]⊙ū
k (12)
r
k
←[G
H
r
k+1
]⊙f, (13)
where k=K−1, K−2, . . . , 0, with the initialization rK=[HH (z−y)]⊙f and gK=[HH(z−y)]⊙ūK. The final expression for the gradient (10) is finally obtained by returning ∇(f)=Re{g0}.
The error backpropagation allows to efficiently evaluate the gradient of the scattered wave with respect to the scattering potential. Due to the convolutional structure of the matrices, its computational complexity is equivalent to running a forward pass, which is of order (KN log (N)). Equipped with this algorithm, the scattering potential can be optimized by using iteration (8). Note that the algorithm circumvents the need to explicitly evaluate and store the Jacobian (11) by directly computing its product with the residual b=(z−y).
The memory 38 can include a database 90, a training module or trainer 82, the GLN 200, a preprocessor 84. The memory 38 can be any a non-transitory computer readable medium. The database 90 can include the historical data 105, training data, testing data 92. The database may also include results from operational, training or retaining modes of using the neural network. In one embodiment, the training module 82 performs the update of the network 150. The network 150 can be initialized using one or combination of the testing 92, the historical 106 and the training 88 data.
Also shown in memory 38 is the operating system 74. Examples of operating systems include AIX, OS/2, and DOS. Other elements shown in memory 38 include device drivers 76 which interpret the electrical signals generated by devices such as the keyboard and mouse. A working memory area 78 is also shown in memory 38. The working memory area 78 can be utilized by any of the elements shown in memory 38. The working memory area can be utilized by the neural network 150, the trainer 82, the operating system 74 and other functions. The working memory area 78 may be partitioned amongst the elements and within an element. The working memory area 78 may be utilized for communication, buffering, temporary storage, or storage of data while a program is running.
For example, the simulation software 710 is responsible for simulating the propagation of the pulse through the material using the neural network 150. The configuration software 720 is responsible for selecting different parameters of the sensing, such as mutual arrangement between the receivers and transmitters of the permittivity sensor, dimension and resolution of the image 110, number of layers in the network 150. In some implementations, the configuration software directly or with help of the visualization software 740 receives the configuration parameters from the user of the permittivity sensor. Training software 730 performs the iterative updating of the image of the distribution of the permittivity of the material. Visualization software 740 serves to render the final image 110 on the display device to visualize the structure of the material to the user.
The above-described embodiments of the present invention can be implemented in any of numerous ways. For example, the embodiments may be implemented using hardware, software or a combination thereof. When implemented in software, the software code can be executed on any suitable processor or collection of processors, whether provided in a single computer or distributed among multiple computers. Such processors may be implemented as integrated circuits, with one or more processors in an integrated circuit component. Though, a processor may be implemented using circuitry in any suitable format.
Also, the embodiments of the invention may be embodied as a method, of which an example has been provided. The acts performed as part of the method may be ordered in any suitable way. Accordingly, embodiments may be constructed in which acts are performed in an order different than illustrated, which may include performing some acts simultaneously, even though shown as sequential acts in illustrative embodiments.
Use of ordinal terms such as “first,” “second,” in the claims to modify a claim element does not by itself connote any priority, precedence, or order of one claim element over another or the temporal order in which acts of a method are performed, but are used merely as labels to distinguish one claim element having a certain name from another element having a same name (but for use of the ordinal term) to distinguish the claim elements.
Although the invention has been described by way of examples of preferred embodiments, it is to be understood that various other adaptations and modifications can be made within the spirit and scope of the invention. Therefore, it is the object of the appended claims to cover all such variations and modifications as come within the true spirit and scope of the invention.