Patent Application
20160204963
The present disclosure relates to reconstruction of sparse signals in communications and signal processing applications.
In digital communications and signal processing applications, continuous physical phenomena, or signals, such as electromagnetic waves, are used for encoding digital information. Recovery of the digital signal encoded in the continuous physical signal consists in sampling the continuous signal at some sampling rate. Conventional sampling methods typically require a sampling rate of at least twice the maximum frequency in the continuous signal, also known as the Nyquist sampling rate, for reliable digital signal reconstruction. In many applications, however, processing cost is related to the sampling rate, and it is therefore desirable to develop techniques which enable reduction of the measurement sampling rate.
Recovery of a digital signal by sampling an encoded continuous signal at sub-Nyquist sampling rates is possible if it is known or expected that the digital signal is sparse, meaning that many of the coefficients of the signal are equal to zero or nearly so. This technique is known as compressed sensing. The assumption of sparsity is an additional constraint which may enable a solution to the problem.
Recovering a sparse signal from an underdetermined set of observations is a Non-deterministic Polynomial-time hard (NP-hard) problem, but may be rendered a convex optimization problem if an l1 norm is used as the sparsifying norm. A number of algorithms for solving this optimization problem have been proposed. Some known algorithms employ the least absolute shrinkage and selection operator (“Lasso”) version of the least squares method. Two such known algorithms include least-angle regression (LARS) and coordinate descent (CD). These algorithms suffer from certain limitations, however. Firstly, they do not solve the original compressed sensing problem because it is NP-hard. Secondly, because of their sensitivity to certain parameters, they do not necessarily find an optimum solution to the problem after a given number of iterations. In addition, known iterative compressed sensing algorithms tend to increase in complexity with each iteration thus adding processing cost to the attainment of increasingly better estimates of the digital signal elements.
It remains desirable, therefore, to develop improved techniques for recovery of sparse signals using compressed sensing algorithms having lower complexity and processing cost.
Embodiments disclosed herein overcome or ameliorate disadvantages of prior methods, provide additional capabilities or advantages, or provide alternative means for producing desirable results.
In general, a conventional compressed sensing algorithm may be used to obtain a subset of potentially nonzero elements of a signal vector. Then, a non-underdetermined estimation method such as iterative linear minimum mean-square error optimization may be used to find the nonzero elements among the potentially nonzero elements. The non-underdetermined estimation method may be less computationally intensive than the conventional compressed sensing algorithm, and may also more quickly resolve the nonzero elements of the signal vector.
In one embodiment, a signal vector, which may be a sparse signal vector, may be recovered from an observation vector including a set of observations. The signal may encode a signal vector having a number of nonzero elements. A number of elements of the observation vector may be less than a total number of elements of the signal vector. In one embodiment, the number of elements of the observation vector is greater than the number of nonzero elements of the signal vector. A signal vector estimate may be generated based on the observation vector and an observation matrix using compressed sensing optimization. The observation matrix describes a relationship between the observation vector and the signal vector. An index set of most dominant elements of the signal vector estimate may be determined according to a sparsifying norm. A number of the most dominant elements is at most the number of the elements of the observation vector, and in one embodiment is greater than the number of the nonzero elements of the signal vector. In one embodiment, the number of the most dominant elements is less than the number of the elements of the observation vector. An index set of the nonzero elements of the signal vector may be generated by performing a non-underdetermined estimation method, which may be iterative linear minimum mean-square error optimization, based on the observation matrix, the observation vector, and the index set of the most dominant elements of the signal vector estimate.
The general outline provided above enables a better understanding of the detailed description of embodiments which follows. Such embodiments include additional features and advantages, and it will be appreciated by persons of ordinary skill in the art that the particular embodiments described below may be modified or built upon to perform the same functions and provide the same results set forth herein while not departing from the spirit and scope of the subject matter set forth in the appended claims.
Embodiments of the present disclosure will now be described, by way of example only, with reference to the attached Figures.
Particular embodiments will now be described in a context of communications and signal processing systems wherein compressed sensing of signal data is employed. It will be appreciated by persons of ordinary skill in the art, however, that the principles set forth herein may be applied to adapt the embodiments to any content wherein signals or data are compressively sampled.
In general, a digital signal x may be encoded in some physical phenomenon characterized by a continuous signal, and the digital signal x may be recovered from an observation y of the continuous signal by solving the following relation:
y
m×p
=A
m×n
x
n×p (1)
where A is an operator, which may be termed the ‘observation matrix’, describing the relationship between the observation y and the encoded digital signal x. Where the system is non-underdetermined, or in other words fully determined or overdetermined, i.e. m≧n, a unique solution for x is in principle possible if the operator A is full-rank. However, in practice it is ordinarily required to employ optimization methods to determine an estimate of the digital signal x.
In particular, where the number of measurements m in observation y is less than the number of elements n in the digital signal x such that m<n, the problem is underdetermined, and in principle a unique solution for x is not possible. If, however, x is known or expected to be s-sparse, meaning it contains a relatively small number s of nonzero elements such that s<<n, then this additional constraint may enable a solution. Recovery of an s-sparse signal from an underdetermined set of observations is known as compressed sensing. Although the number m of observations is less than the number of elements n of the signal x, and s<m<n, it remains possible to recover the signal x because of the sparsity constraint. For example, compressed sensing is a technique which may be used for digital signal recovery from continuous signals at sampling rates below the Nyquist rate.
The problem of recovering an s-sparse vector x from an underdetermined set of linear observations y may be addressed by using convex-optimization compressed sensing algorithms. Some such algorithms employ the least absolute shrinkage and selection operator (“Lasso”) version of the least squares method, also known as the unconstrained basis pursuit denoising problem (UBPDN):
Two such known algorithms include least-angle regression (LARS) and coordinate descent (CD) algorithms. These algorithms suffer from certain limitations, however, and it remains desirable to provide improved techniques for reconstruction of sparse signals using compressed sensing algorithms having lower complexity and processing cost.
An improved method of signal recovery may use a conventional compressed sensing algorithm to generate an estimate {circumflex over (x)} of a signal x, and based either on a prior knowledge of a number of nonzero elements of the signal x, or a separate method for estimating such number, a subset of k potentially nonzero elements may be selected, where k≦m, where m is the number of observations. As such, the optimization problem is no longer underdetermined. Following this selection of k potentially nonzero elements, a further method different from the compressed sensing algorithm may be used to solve the non-underdetermined optimization problem, but may now be selected without limitation by the constraint of an underdetermined system. In particular, a non-underdetermined estimation method may be used, wherein the non-underdetermined estimation method is any method suitable for estimating the solution to a system of equations that is not underdetermined. The non-underdetermined estimation method may be less computationally intensive than compressed sensing algorithms and may more quickly resolve the nonzero elements of the signal x. In some embodiments, the non-underdetermined estimation method is an iterative linear minimum mean-square error (LMMSE) algorithm used to find the nonzero elements among the k potentially nonzero elements. In other embodiments, other non-underdetermined estimation methods may be used, such as zero-forcing (ZF).
In view of the foregoing, a system 100 for signal recovery using compressive sensing will now be described with reference to
As shown in identifying the nonzero elements of signal x (step 250), and also optionally a signal estimate subset
including a subset of estimates of the nonzero elements of signal x, to a receiver 130.
The processor 110 may be embodied in any device or apparatus required by or appropriate to the circumstances of use. For example, the processor 110 may be included in a computer also having one or more of a volatile memory, a non-volatile storage, a display, a user interface, and one or more communication interfaces. The source 120 and the receiver 130 may respectively be aspects of the computer, or alternatively external to the computer. For example, the source 120 may include the non-volatile storage, and the processor 110 receives data encoding the observation y of signal x from the non-volatile storage. Alternatively, the source 120 may include a communication interface, and the processor 110 receives a signal encoding the observation y of signal x from the communication interface. Similarly, the receiver may include the non-volatile storage or a communication interface, and the processor 110 stores or sends the index subset , and optionally the signal estimate subset
, in or via the non-volatile storage or communication interface, respectively.
The processor 110 may be a digital signal processor or an integrated circuit, and/or may be included in or form an aspect of a general purpose computer, or alternatively a special purpose device. For example, in a communications application the processor 110 may be included in or form an aspect of a base station, a router, an access point, a switch, or a hub configured for wired or wireless communication with one or more devices over a network which may include a wired or wireless network. The processor 110 may be a general-purpose processor configured to execute instructions encoded in a memory to perform the methods described herein, or alternatively the processor 110 may be a special-purpose circuit configured to perform the methods, and may be, for example, an application-specific integrated circuit. The processor 110 may include one or more processors which are configured cooperatively to execute the instructions or perform the methods, or are alternatively configured such that each processor executes a different portion of the instructions or a different step of aspect of the method. Accordingly, the term “processor” as used herein may be understood as including one or more processors, which may be selected based on related design and performance considerations.
The processor 110 may input or receive the observation y in any convenient form or format, and may output or send the index subset , and optionally the signal estimate subset
, in any convenient form or format. For example, the processor 110 may receive the observation y in the form of a data packet, and/or may output or send the index subset
, and optionally the signal estimate subset
, in the form of a packet.
A method 200 performable by the processor 110 is now described.
In general, observation y and signal x are related by:
y=Ax+z (3)
wherein A is an observation matrix, and z represents noise. If the system is noiseless, or may be presumed to be so, or if noise is negligible, then noise z may be omitted or set to zero. Observation y and signal x may, in principle, have any dimensionality, thus determining the dimensionality of observation matrix A and noise z, where the system may be underdetermined. For the sake of convenient exposition, it will be assumed that observation y, signal x, and noise z are one-dimensional vectors, and observation matrix A is a two-dimensional matrix, though it will be appreciated that the principles set forth herein are equally applicable regardless of dimensionality.
Thus, in the method, a signal vector xn×1 is known, determined, or estimated to be s-sparse—that is a relatively small number s<<n elements of signal vector xn×1 are nonzero. A number m of observations ym×1 of a signal encoding vector xn×1 are input, or received (step 210), wherein s<m<n, and are related as follows:
y
m×1
=A
m×n
x
n×1
+z
m×1 (4)
where Am×n is the observation matrix, and zm×1 represents noise.
Using a compressed sensing optimization algorithm, an estimate {circumflex over (x)} of signal x may be generated (step 220). The compressed sensing algorithm may be any conventional or as yet unknown compressed sensing algorithm. The compressed sensing algorithm may be a Lasso algorithm, such as a least-angle regression algorithm or a coordinate descent algorithm. Alternative compressed sensing algorithms may also be used.
Based on the signal estimate {circumflex over (x)}, an index set ={i1, i2, . . . , ik} of a number k of potentially nonzero elements of signal x is generated, wherein k is no greater than the number of observations m in observation vector y, that is k≦m. As such:
⊂[1, 2, . . . , n}, |
|≦|y| (5)
it being recalled that signal x has n elements.
The value of k—that is the number of potentially nonzero elements—may be selected based on any suitable basis, and may be equal to the number of observations m, one less than the number of observations m−1, or any other quantity less than m. In particular, k may be a design parameter determined based on experiment or any other basis related to the system under consideration.
Having selected the number of potentially nonzero elements k, an index set is generated to identify the k most dominant elements of signal estimate {circumflex over (x)} according to the sparsifying norm used (step 230), wherein s<k≦m<n. Thus, if an l1 norm is used, then the k most dominant elements of signal estimate {circumflex over (x)} are determined to be the k elements {circumflex over (x)}i having the greatest norm |{circumflex over (x)}i|. As another example, if an l1,2 norm is used as the sparsifying norm, then the k most dominant elements of signal estimate {circumflex over (x)} consists of the elements in sub-vectors
, having the greatest l2 norm ∥
∥2 with Σ|
|=k.
The index set thus defines a sub-vector {circumflex over (x)}
of signal estimate {circumflex over (x)} including the elements corresponding to index set
, and similarly a sub-matrix A
including columns corresponding to index set
. The number of elements k of signal vector estimate {circumflex over (x)}
is no greater than the number of measurements m in matrix A
—that is, k≦m—and thus the optimization problem is no longer underdetermined and may be solved by a method which is not limited by the constraint of an underdetermined system.
Thus, based on index set and the observation vector y, a non-underdetermined estimation method may be used to find the nonzero elements among the k potentially nonzero elements. In some embodiments, the non-underdetermined estimation method is an iterative linear minimum mean-square error (LMMSE). Other non-underdetermined estimation methods may also be used. In other embodiments, for example, a zero-forcing (ZF) estimation method may be used. In particular, the estimation method generates an iteration index subset
⊂
of the nonzero elements of signal estimate {circumflex over (x)} along with the element estimates
(step 240). In some embodiments, selection of k<m may result in improved performance of the LMMSE algorithm, which may result in a lower estimation error, i.e. a smaller value of [1
−
∥22.
The iterative LMMSE optimization algorithm 300 is now described in more detail with reference to
In an initialization step of the iterative LMMSE algorithm, is set to equal
(step 310)—that is,
←
—and an iteration step-size Δ is set (step 320).
The iteration step-size Δ may be selected on any suitable basis. In some embodiments, it is set to Δ=1, such that only a single potentially nonzero element is eliminated with each iteration. Alternatively, Δ>1. In particular, the iteration step-size Δ may be selected based on any other optimization method, and may be a design parameter determined based on experiment according to the system under consideration.
Having selected an iteration step-size Δ, the following steps are performed in each iteration of the algorithm.
Firstly, a signal estimate subset is generated (step 330), wherein
=W×y and the LMMSE matrix W is given by:
W=(
+Cz)−1 (6)
wherein is the conjugate transpose of
.
Secondly, the q most dominant elements of are then determined, according to the sparsifying norm (step 340), that is:
{i1, . . . , iq}⊂, wherein q=max(|
|−Δ, s)
Finally, the index subset is updated to identify the q most dominant elements of
thus determined (step 350), that is
<{i1, . . . , iq}.
The above steps are performed iteratively until ||≦s—that is, while the number of elements of index subset
is greater than the number of nonzero elements of signal x—or any other stopping criterion has been met (decision 360, end 370). For example, a variance of one or more of the elements of signal estimate subset
may be determined in each iteration, and the stopping criterion may define a threshold value of the variance. Other alternatives are possible.
In a special case where the respective elements of both the noise and signal vectors are internally uncorrelated, then W simplifies to:
wherein σx and σz are the variances of x and z respectively, and wherein I is the identity matrix of appropriate dimensions.
When the iteration terminates, index subset identifies the nonzero elements of signal x, and the signal estimate subset
is the subset of estimates of the nonzero elements of signal x. The index subset
may be output, or sent (step 250). Optionally, the signal estimate subset
may also be output, or sent. In addition, or alternatively, signal x may be recovered (step 260). For example, signal x may be recovered based on on
and
. Alternatively, having identified the nonzero elements of signal x, in index set
, other methods may be used to reconstruct signal x based on observation vector y and index set
.
Use of a non-underdetermined estimation method in accordance with the method above can improve performance of a compressed sensing algorithm. In particular, an iterative LMMSE iteration has less complexity than a compressed sensing iteration, including where the coordinate descent algorithm is used. For example, each iteration of LMMSE involves one matrix multiplication, whereas each iteration of coordinate descent involves n matrix multiplications, and as such each iteration of LMMSE is less computationally intensive. Moreover, each LMMSE iteration may resolve the nonzero elements of signal x more fully than each compressed sensing iteration, resulting in a more rapid solution at a lower computational cost.
The above system and method for reconstructing signal vector x from observation vector y may be used in any application where x is known or determined to be sparse, and following the application of conventional compressed sensing algorithms there exists some basis for selecting a number of most dominant elements which is no less than the number of measurements. The system and method are not limited in their application to any particular field, but may be used in any application wherein vector x and vector y represent respective phenomena related by equation (3). The system and method may be used in any area of communications or signal processing, including wired or wireless telecommunications, image, video, or audio processing, including as part of image, video, or audio compression and transmission methods.
For example, in one embodiment of a system and method, the observation vector y may represent a measurement of a continuous electromagnetic signal encoding symbols of a sparse digital signal vector x by any modulation scheme A. In another embodiment, y may represent compressed data according to compression scheme A, wherein x is the sparse uncompressed data. For example, y may represent a compressed representation of video, image, or audio data in a representation domain (which may be a frequency domain or wavelet domain, for example), compressed according to compression scheme A, and x represents the uncompressed video, image, or audio data in the representation domain.
The inventors have discovered that the above system and method are also useful for identifying active devices in an overloaded, uncoordinated communication system, as follows.
As shown in
For example, the communications devices may include one or more digital radios, cellular telephones, smartphones, tablet computers, laptop computers, smart appliances, smart devices, or any other device having a transmitting capability. The base station is configured to receive transmissions from the communications devices. For example, the base station may include processor 110 and source 120, wherein source 120 includes a communication interface of the base station which receives transmissions from active devices 415.
If in such a system the number of devices is no greater than a number of channels available for communication with the base station, and each of the devices is assigned to a different channel, then the base station is capable at any time of determining which devices are active (i.e. transmitting) by simply observing through which channel a signal is being received. In the case where the number of devices is greater than the number of available channels, however, the system is overloaded, and the base station cannot determine which devices are active in this way.
For example, a communications system employing a code division multiple access (CDMA) channel access method enables synchronous or asynchronous simultaneous transmission of multiple signals over a single communications channel by modulating each signal according to a code, or signature. So long as the set of signatures are mutually orthogonal, all of the signals are in principle recoverable. The set of mutually orthogonal signatures is, however, practically limited in number in a given set of conditions, and thus as the number of communicating devices in such a system increases, and each new device is assigned a new signature, a point is reached where the set of signatures is no longer completely mutually orthogonal. Such a system is said to be ‘overloaded’.
An overloaded system of the kind described above may be modeled by:
y
m×p
=A
m×n
x
n×p (8)
wherein vector ym×p stands for a number p of symbols received in a number m channels, from a number n devices xn×p, according to an encoding operator Am×n. For example, in a CDMA system, operator Am×n stands for the n signatures available for signal modulation and m is the signature length. If n>m devices are assigned n>m corresponding signatures which do not form a mutually orthogonal set, the system is overloaded and it cannot be determined which device is transmitting at any given time simply by reference to the channel.
If, however, it is known or expected that only a relatively small number s of the devices is active (i.e. transmitting) at any given time, that is s<<n, then the above methods for recovering an s-sparse signal x from an underdetermined set of m observations y may be employed to determine the number s active devices in the overloaded system.
Thus, it is assumed that, at a given time, a number s of the n devices is active, and for the sake of convenient exposition it is assumed that each device transmits a single symbol at a given time. This is the situation, for example, in a synchronous communication system. It will be appreciated, however, that the principles herein are equally applicable where more than one symbol is transmitted. The symbols may be encoded in the transmission signal in any desired manner, and thus the symbols may be encoded by quadrature phase-shift keying (QPSK) modulation. An observation ym×1 is thus related to the transmitted symbols xn×1 by:
y
m×1
=A
m×n
x
n×1
+z
n×1 (9)
wherein n is the number of devices, m is the signature length, Am×n is the encoding operator, and zn×1 represents noise. If the system is or may be presumed to be noiseless, or noise is negligible, then zn×1 may be omitted or set to zero. The signature length is less than the number of devices, that is m<n, and the system is overloaded. It is known or expected that the number s of active devices is small, that is s<<n, and is less than the signature length, that is s<m.
Based on the foregoing definitions, therefore, the system and method of recovering a sparse signal from an underdetermined set of measurements is made equally applicable for determining the s<<n active devices transmitting autonomously to a base station at a given time. Thus, with reference to , and optionally signal estimate subset {circumflex over (x)}
, following performance of the method 200 by processor 510. In particular, processor 510 may be further configured to recover transmitted symbols xn×1 from observations ym×1, based on index subset
generated by method 200.
In application of the method, k may initially be set as the number of potentially active devices, and the index set of most dominant elements of vector xn×1 may represent the set of potentially active devices, that is those devices for which the corresponding transmitted symbol is nonzero. The value of k may be set on the bases set forth above, and in the present application may also be set on the basis of any known factors of the set n of devices in the system which would suggest an upper bound on the number of devices that are active at a given time. Following application of the method, and in particular the last iteration of the LMMSE algorithm, index subset
identifies the s devices for which the decoded symbols x
are nonzero, and thus identifies the s devices which are active.
Reference is made to
In another embodiment, a non-volatile computer-readable medium encoding instructions executable by a processor may perform a method as follows. At the processor, an observation vector is received, the observation vector comprising a set of observations of a signal encoding a signal vector, which may be a sparse signal vector. The signal vector has a number of nonzero elements. A number of elements of the observation vector is less than a total number of elements of the signal vector. In some embodiments, and in some embodiments is also greater than the number of nonzero elements of the signal vector. A signal vector estimate is generated by the processor based on the observation vector and an observation matrix using compressed sensing optimization, the observation matrix describing a relationship between the observation vector and the signal vector. An index set of most dominant elements of the signal vector estimate is determined by the processor according to a sparsifying norm, wherein a number of the most dominant elements is at most the number of the elements of the observation vector, and in some embodiments is also greater than the number of the nonzero elements of the signal vector. An index set of the nonzero elements of the signal vector is generated by the processor by performing a non-underdetermined estimation method, such as iterative LMMSE optimization, based on the observation matrix, the observation vector, and the index set of the most dominant elements of the signal vector estimate.
In the preceding description, for purposes of explanation, numerous details are set forth in order to provide a thorough understanding of the embodiments. However, it will be apparent to one skilled in the art that these specific details are not required. In other instances, well-known electrical structures and circuits are shown in block diagram form in order not to obscure the understanding. For example, specific details are not provided as to whether the embodiments described herein are implemented as a software routine, hardware circuit, firmware, or a combination thereof.
Embodiments of the disclosure can be represented as a computer program product stored in a tangible machine-readable medium (also referred to as a computer-readable medium, a processor-readable medium, or a computer usable medium having a computer-readable program code embodied therein). The machine-readable medium can be any suitable tangible, non-transitory medium, including magnetic, optical, or electrical storage medium including a diskette, compact disk read only memory (CD-ROM), memory device (volatile or non-volatile), USB flash disk, or similar storage mechanism. The machine-readable medium can contain various sets of instructions, code sequences, configuration information, or other data, which, when executed, cause a processor to perform steps in a method according to an embodiment of the disclosure. Those of ordinary skill in the art will appreciate that other instructions and operations necessary to implement the described implementations can also be stored on the machine-readable medium. The instructions stored on the machine-readable medium can be executed by a processor or other suitable processing device, and can interface with circuitry to perform the described tasks.
The above-described embodiments are intended to be examples only. Alterations, modifications and variations can be effected to the particular embodiments by those of skill in the art. The scope of the claims should not be limited by the particular embodiments set forth herein, but should be construed in a manner consistent with the specification as a whole.
