Maximum A posterior estimator with fast sigma calculator

Abstract

Forward and backward recursive calculations in a maximum a posteriori decoding process are performed in parallel processes, rather than sequentially, allowing the a posteriori transition probabilities to be calculated in the same time interval as the recursions, thereby reducing decoding latency and required memory.

Description

BACKGROUND OF THE INVENTION

The present invention relates generally to error-correction coding and, more particularly, to trellis codes and to parallel concatenated convolutional coding, i.e., turbo coding.

A new class of forward error control codes, referred to as turbo codes, offers significant coding gain for power limited communication channels. Turbo codes are generated by using two or more recursive systematic encoders operating on different orderings of the same information bits. A subset of the code bits generated by each encoder is transmitted to maintain bandwidth efficiency. Turbo decoding involves an iterative algorithm in which probability estimates of the information bits that are derived for one of the code words are fed back to a probability estimator for the other code word. Each iteration of processing generally increases the reliability of the probability estimates. This process continues, alternately decoding the two code words until the probability estimates can be used to make reliable decisions.

The maximum a posteriori (MAP) algorithm introduced by Bahl, Cocke, Jelinek, and Raviv in "Optimal Decoding of Linear Codes for Minimizing Symbol Error Rate", IEEE Transactions on Information Theory, March 1974, pp. 284-287, is particularly useful as a component decoder in decoding parallel concatenated convolutional codes, i.e., turbo codes. The MAP algorithm is used in the turbo decoder to generate a posteriori probability estimates of the systematic bits (i.e., information bits) in the code word. These probability estimates are used as a priori symbol probabilities for the second MAP decoder. Three fundamental terms in the MAP algorithm are: forward and backward state probability functions (the alpha and beta functions, respectively) and the a posteriori transition probabilities (the sigma function).

It is desirable to reduce the time for computation and memory required in turbo decoding, such as by providing a method for performing forward and reverse recursions simultaneously, thereby improving decoder performance even further.

SUMMARY OF THE INVENTION

Forward and backward recursive calculations in a maximum a posteriori decoding process are performed in parallel processes. Such parallel computations allow for calculation of the a posteriori transition probabilities in the same time interval as that for the forward and backward recursions, thereby reducing decoding latency and required memory.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a simple communications system comprising a Markov source, a memoryless channel and decoder;

FIG. 2 is a time-line representation of the parallel calculation of the .alpha. and .beta. values, illustrating how the .sigma. values are computed when the .alpha. and .beta. values "meet";

FIG. 3 is another time line representation of the parallel calculation of the .alpha. and .beta. values; and

FIGS. 4 and 5 comprise a flow chart illustrating parallel processing of forward and backward recursions and sigma calculations in accordance with preferred embodiments of the present invention;

DETAILED DESCRIPTION OF THE INVENTION

Summary of the MAP Decoding Algorithm

The maximum a posteriori (MAP) algorithm for decoding linear codes was published by Bahl, Cocke, Jelinek, and Raviv, as cited hereinabove. Basically, this algorithm calculates the a posteriori probability of each state in a trellis corresponding to a received code word conditioned on the entire received code word. A variation of this algorithm calculates the a posteriori probability of each state transition in a trellis corresponding to a received code word conditioned on the received code word. Since this variation is of particular interest when decoding recursive systematic convolutional codes, which are typically utilized in turbo coding, the description of the invention will be built around this form of the algorithm. However, the invention also applies to the version of the MAP algorithm that calculates a posteriori state probabilities in a straightforward manner.

Consider the simple communication system block diagram of FIG. 1 for the purpose of describing the MAP algorithm. The source 10 is a discrete-time, finite-state Markov process, shown as comprising a source 11, an encoder 12, and a mapping device 13. As shown, the Markov source's output at time t is X.sub.t and the corresponding channel 14 output is Y.sub.t, which is provided as the input to a decoder 16. The following definitions are also needed to describe the MAP algorithm. Let

M=the number of distinct states of the Markov source; m=the index of the Markov source's states {0,1, . . . , M-1}; S.sub.t =state of the Markov source at time t; S.sub.t.sup.u =S.sub.t, S.sub.t+1, . . . S.sub.u, the state sequence of the Markov source from time t to time u; X.sub.t.sup.u =X.sub.t, X.sub.t+1, . . . X.sub.u, the output sequence of the Markov source from time t to time u; Y.sub.t.sup.u =Y.sub.t, Y.sub.t+1, . . . Y.sub.u, the output sequence of the channel from time t to time u, and L=the length of the Markov source's output sequence. Furthermore, it is assumed that the source's initial state S.sub.0 =0 and its final state S.sub.L =0.

The channel is noisy and memoryless so that a sequence of the source's output symbols X.sub.t.sup.u is transformed into a sequence of channel outputs Y.sub.t.sup.u according to the following channel transition probabilities.

Pr{Y.sub.t.sup.u .vertline.X.sub.t.sup.u }=Pr{Y.sub.t .vertline.X.sub.t }Pr{Y.sub.t+1 .vertline.X.sub.t+1 }. . . Pr{Y.sub.u .vertline.X.sub.u }(1)

The decoder examines the received channel output sequence and estimates the a posteriori probabilities of the Markov source's states or state transitions. The state a posteriori probabilities Pr{S.sub.t =m.vertline.Y.sub.1.sup.L } can be written as

Pr{S.sub.t =m.vertline.Y.sub.1.sup.L }=Pr{S.sub.t =m; Y.sub.1.sup.L }/Pr{Y.sub.1.sup.L } (2) using the definition of conditional probability. Our notation is as follows: Pr{A.vertline.B} means the conditional probability of event A given event B, and Pr{A;B} means the joint probability of events A and B. Similarly, the source's state transition a posteriori probabilities can be written as Pr{S.sub.t-1 =m'; S.sub.t =m.vertline.Y.sub.1.sup.L }=Pr{S.sub.t-1 =m'; S.sub.t =m; Y.sub.1.sup.L }/Pr{Y.sub.1.sup.L }. (3)

It turns out that it is easier to derive an algorithm to calculate the joint probabilities on the right-hand sides of equations (2) and (3). The desired conditional probabilities can then be obtained by simply dividing these joint probabilities by the constant Pr{Y.sub.1.sup.L }. To clarify the derivation, Bahl et al. defined the following probability functions.

.alpha..sub.t (m)=Pr{S.sub.t =m; Y.sub.1.sup.t } (4) .beta..sub.t (m)=Pr{Y.sub.t+1.sup.L .vertline.S.sub.t =m} (5) .gamma..sub.t (m',m)=Pr{S.sub.t =m; Y.sub.t .vertline.S.sub.t-1 =m'}(6)

It can be shown that

Pr{S.sub.t =m; Y.sub.1.sup.L }=.alpha..sub.t (m).beta..sub.t (m)(7) and Pr{S.sub.t-1 =m'; S.sub.t =m; Y.sub.1.sup.L }=.alpha..sub.t-1 (m').gamma..sub.t (m',m).beta..sub.t (m) (8) so all that remains is to calculate the values of .alpha..sub.t (m) .gamma..sub.t (m',m) and .beta..sub.t (m) for all values of m and t. It can be shown that for t=1,2, . . . , L, .alpha..sub.t (m) can be calculated recursively as follows. ##EQU1## For t=0, the boundary conditions are .alpha..sub.0 (0)=1 and .alpha..sub.0 (m)=0 for all m.noteq.0. Similarly, .beta..sub.t (m) can be calculated recursively for t=1,2, . . . , L-1. ##EQU2## For t=L, the boundary conditions are .beta..sub.L (0)=1 and .beta..sub.L (m)=0 for all m.noteq.0. ##EQU3## For the cases of interest to us, the source's mapping of state transitions to output symbols is deterministic; i.e. there is only one possible output symbol X for each state transition, so that Pr{X.sub.t .vertline.S.sub.t =m; S.sub.t-1 =m'}=1 and .gamma..sub.t (m',m) can be written as .gamma..sub.t (m',m)=Pr{S.sub.t =m.vertline.S.sub.t-1 =m'}Pr{Y.sub.t .vertline.X.sub.t }. (12) The calculation of Pr{S.sub.t-1 =m'; S.sub.t =m; Y.sub.1.sup.L } using the algorithm of Bahl et al. can be summarized as follows.

1. .alpha..sub.0 (0)=1 and .alpha..sub.0 (m)=0 for all m.noteq.0.

.beta..sub.L (0)=1 and .beta..sub.L (m)=0 for all m.noteq.0.

2. As soon as Y.sub.t is received, the decoder computes .gamma..sub.t (m',m) using equation (11) and .alpha..sub.t (m) using equation (9). The calculated values of .alpha..sub.t (m) are stored for all t and m. The calculated values of .gamma..sub.t (m',m) are stored for all t and all transitions (m',m).

3. After the complete sequence of channel outputs .gamma..sub.1.sup.L is received, the decoder recursively computes .beta..sub.t (m) using equation (10). When all of the .beta..sub.t (m) have been computed, they can be multiplied by the appropriate .alpha..sub.t-1 (m') and .gamma..sub.t (m',m) to obtain Pr{S.sub.t-1 =m';S.sub.t =m; Y.sub.1.sup.L } as in equation (8).

When decoding trellis codes, the Markov source of FIG. 1 can be modeled as comprising a source of information bits, an encoder for the error correcting trellis code, and a mapping device that converts the encoder's output into symbols X.sub.t, which are transmitted over the channel.

The term Pr{S.sub.t =m.vertline.S.sub.t-1 =m'}, the encoder's conditional state probability, in equation (12) is calculated as ##EQU4## for an (n,k,v) code where Pr{d.sub.i =a.sub.i (m',m)} is the a priori probability that d.sub.i =a.sub.i (m',m) and a.sub.i (m',m) is the i.sup.th bit of the sequence a(m',m) of k bits that drives the encoder from state m' to state m. This equation holds when the code's trellis does not have parallel branches. (If the code's trellis does have parallel branches, then Pr{S.sub.t =m.vertline.S.sub.t-1 =m'} is calculated as the sum of products of bit a priori probabilities in which the sum is over the all of the sequences of source bits a(m',m) that drive the encoder from state m' to state m.) In the decoder, we are interested in event likelihood ratios; specifically, we are interested in Pr{d.sub..tau. =1.vertline.Y.sub.1.sup.L }/Pr{d.sub..tau. =0.vertline.Y.sub.1.sup.L }, where Pr{d.sub..tau. =i.vertline.Y.sub.1.sup.L } represents the a posteriori probability that data bit d.sub..tau. from the source is equal to i.epsilon.{0,1}. ##EQU5## In the equation above A.sub.t (.tau.) is the set of all state transitions (m',m) in the encoder which terminate at time t due to an information bit d.sub..tau. =0. Recall that in general there are k source bits encoded into each n-bit encoder output symbol so that d.sub..tau. for indices .tau.=(t-1)k+1, (t-1)k+2, . . . , tk are associated with the state transition S.sub.t-1 to S.sub.t. ##EQU6##

In the equation above B.sub.t (.tau.) is the set of all state transitions (m',m) in the encoder which terminate at trellis level t due to an information bit d.sub..tau. =1. Since we are working with the likelihood ratio Pr{d.sub..tau. =1.vertline.Y.sub.1.sup.L }/Pr{d.sub..tau. =0.vertline.Y.sub.1.sup.L }, it is not necessary to calculate Pr{S.sub.t-1 =m'; S.sub.t =m.vertline.Y.sub.1.sup.L }; we may use Pr{S.sub.t-1 =m';S.sub.t =m; Y.sub.1.sup.L } directly since Pr{Y.sub.1.sup.L } is common to the numerator and denominator. Therefore, the desired likelihood ratio from the decoder is ##EQU7##

Since the joint probability Pr{S.sub.t-1 =m'; S.sub.t =m; Y.sub.1.sup.L } is used often herein, the following notation from Bahl et aL is introduced for brevity.

.sigma..sub.t (m',m)=Pr{S.sub.t-1 =m'; S.sub.t =m; Y.sub.1.sup.L }=.alpha..sub.t-1 (m').gamma..sub.t (m',m).beta..sub.t (m)(17)

When the encoder's input symbols are binary, i.e. k=1, equations (13) through (16) can be simplified since .tau.=t in this case. The simplified equations are presented below.

Pr{S.sub.t =m.vertline.S.sub.t-1 =m'}=P{d.sub.t =a(m',m)} (18) where a(m',m)=the bit value that drives the encoder from state m' to state m. ##EQU8## A.sub.t =the set of all state transitions (m',m) which terminate at level t in the encoder due to an information bit d.sub.t =0. ##EQU9## B.sub.t =the set of all state transitions (m',m) which terminate at trellis level t in the encoder due to an information bit d.sub.t =1. ##EQU10##

Description of the Log-MAP Decoder

The MAP decoder algorithm presented earlier requires a large number of multiplications. For some implementations wherein a large number of multiplications may be impractical, e.g., integrated circuit implementations, a logarithmic version of the MAP algorithm has been derived that eliminates all multiplication operations. The log-MAP algorithm primarily uses additions, table look-ups, and some simple logic functions. In an integrated circuit implementation of a decoder for trellis codes, the replacement of multiplications with additions enables large savings in gate count (die area) and significant increases in decoding speed. The following is a summary of the basic log-MAP algorithm.

Equations (7) through (10) become

ln Pr{S.sub.t =m; Y.sub.1.sup.L }=ln .alpha..sub.t (m)+ln .beta..sub.t (m),(22) ln Pr{S.sub.t-1 =m'; S.sub.t =m; Y.sub.1.sup.L }=ln .sigma..sub.t (m',m)=ln .alpha..sub.t-1 (m')+ln .gamma..sub.t (m',m)+ln .beta..sub.t (m),(23) ##EQU11## Equations (24) and (25) are derived from equations (9) and (10), respectively, by applying the identity a.multidot.b=exp(ln [a.multidot.b])=exp(ln a+ln b) to the product terms. For example, equation (24) is derived as follows: ##EQU12## The boundary conditions are now ln .alpha..sub.0 (0)=0 and ln .alpha..sub.0 (m)=-.infin. for all m.noteq.0. Similarly, ln .beta..sub.L (0)=0 and ln .beta..sub.L (m)=-.infin. for all m.noteq.0. In a practical implementation, a large negative number is substituted for -.infin..

The sum of exponential terms can be computed by applying the following identity.

ln [exp(a)+exp(b)]=max(a,b)+ln [1+exp(-.vertline.a-b.vertline.)](26) Computational complexity can be significantly reduced by storing a table of values for the function ln [1+exp(-.vertline.a-b.vertline.)] and applying equation (33). Experimentation has shown that 8 or 16 values of this function provide sufficient resolution to prevent performance degradation.

Equation (12) becomes

ln .gamma..sub.t (m',m)=ln Pr{S.sub.t =m.vertline.S.sub.t-1 =m'}+ln Pr{Y.sub.t .vertline.X}. (27)

The log-likelihood ratio is calculated using the identity of equation (26) as follows: ##EQU13##

Preferred Embodiments of the Invention

Turbo codes involve a parallel concatenation of two or more systematic codes. Decoding turbo codes is an iterative operation such that the work in decoding one component code is used as a priori information for decoding another component code. Iterations proceed a finite number of times with the passing of information between decoders comprising feedback in the decoding process.

As set forth in the Summary of the MAP Decoding Algorithm set forth hereinabove, some of the fundamental terms in a MAP decoding algorithm are forward and backward state probabilities, commonly known as the alpha and beta functions, and a posteriori state transition probabilities, commonly known as the sigma functions. Calculations of these probabilities (i.e., the alpha, beta and sigma functions) require symbol transition probabilities, i.e., the gamma functions. From Bahl, Cocke, Jelinek, and Raviv in "Optimal Decoding of Linear Codes for Minimizing Symbol Error Rate", cited hereinabove, ##EQU14## where t is the symbol index and ##EQU15## where J is the number of bits comprising an encoder output symbol, assuming binary signaling.

In a recursive systematic code, a single transition (m', m) determines a specific channel symbol Xd& so that the summation in equation 22 is eliminated and the function q.sub.t (X.sub.t .vertline.m',m) is identically equal to one. Also, for a (2, 1) code, the channel transition probabilities are computed as the product of two one-dimensional transition probabilities for the information and parity bits; that is,

R(X.sub.t .vertline.Y.sub.t)=R(Y.sub.ti .vertline.X.sub.ti)R(Y.sub.tp .vertline.X.sub.tp). (24)

Furthermore, in the turbo decoding procedure, the conditional state probabilities are determined by the a priori bit probabilities as follows:

P.sub.t (m.vertline.m')=AP.sub.t (0) if X.sub.ti .vertline.m,m'=0(25) P.sub.t (m.vertline.m')=AP.sub.t (1) if X.sub.ti .vertline.m,m'=1(26) so that .gamma..sub.t (m',m)=AP(0)R(Y.sub.i .vertline.0)R(Y.sub.p .vertline.X.sub.p) if X.sub.ti .vertline.m,m'=0 (27) .gamma..sub.t (m',m)=AP(1)R(Y.sub.i .vertline.1)R(Y.sub.p .vertline.X.sub.p) if X.sub.ti .vertline.m,m'=1 (28). The time index has been omitted in equations (27) and (28) for brevity.

The output of a MAP component decoder is an a posteriori probability estimate of the systematic symbols. These a posteriori estimates are used as the a priori probabilities AP.sub.t (0) and AP.sub.t (1) for decoding another component code. The a posteriori probability of bit t being equal to 1(0) is found by summing all .sigma..sub.t (m',m) over the state transition pairs (m',m) which correspond to a 1(0) being transmitted where

.sigma..sub.t (m',m)=.alpha..sub.t-1 (m').beta..sub.t (m).gamma..sub.t (m',m). (29)

Thus, for each time interval a vector .sigma..sub.t ={.sigma..sub.t (m',m); all (m',m)} is computed.

The calculation of the .alpha. values requires a forward recursion while the calculation of the .beta. values requires a backward recursion. According to preferred embodiments of the present invention, both forward and backward recursions are performed simultaneously, i.e., in parallel. Moreover, only the first half of the calculated .alpha. and .beta. values need to be stored in memory. Two .sigma. vectors per recursion step may be computed once half the .alpha. and .beta. values have been computed. See FIG. 2 for a time-line representation of the parallel calculation of the .alpha. and .beta. values, illustrating how the .sigma. values are computed when the .alpha. and .beta. values "meet".

After the half-way point has been reached, i.e., the first half of the a values have been computed and the recursion is proceeding to the right half of the time-line (as illustrated in FIGS. 2 and 3), .sigma. vectors may be computed from the current .alpha. values and the corresponding .beta. values which were previously computed and stored in memory. Likewise, as the .beta. recursion proceeds to the left half of the time-line (as illustrated in FIG. 2), .sigma. vectors may be computed from the current .beta. values and the corresponding .alpha. values which were previously computed and stored in memory.

Calculations of the .gamma. values do not involve recursions, as is apparent from the equations for .gamma. set forth hereinabove. In accordance with the processes herein, the .gamma. values need not be calculated separately for every calculation. Instead, previously calculated .gamma. values which were used in the .alpha. and .beta. calculations may be used to calculated the present values of .gamma.. Such dual-use addressing for .gamma. values reduces memory access time and further reduces the computational burden. Alternatively, it may be desirable to use modified .gamma. values in which case new .gamma. values would be calculated.

FIGS. 4 and 5 comprise a flow chart illustrating parallel processing of .alpha., .beta., and .sigma. values in accordance with preferred embodiments of the present invention. In step 30, the trellis level index t is set equal to zero, and the final trellis level index t.sub.f is set to represent the number of levels in the trellis. In the next step 32, .alpha..sub.0 and .beta..sub.tf are initialized.

As soon as the last symbol in the received code word Y.sub.tf is received (step 34), the following operations take place in step 36:

t=t+1 u=t.sub.f -1 v=Int{t.sub.f /2}, where Int{x} comprises the integer part of x, and w=v+1.

Then, in step 38, the decoder computes .gamma..sub.t (m',m) using (12) and .alpha..sub.t (m) using (9) for t=1, 2, . . . t.sub.f-1, beginning with t=1. Simultaneously, it computes .gamma..sub.u+1 (m,m') using (12) and .beta..sub.u (m) using (10) for u=(t.sub.f -1), (t.sub.f -2), . . . , 1, beginning with u=(t.sub.f -1). The calculated values of .alpha..sub.t (m) are stored for t =1, 2, . . . , (t.sub.f /2) and all m. The calculated values of .beta..sub.u (m) are stored for u =(t.sub.f -1), (t.sub.f -2), . . . , t.sub.f -(t.sub.f /2) and all m. To clarify the discussion, let .alpha..sub.t =[.alpha..sub.t (0),.alpha..sub.t (1), . . . .alpha..sub.t (M-1)] and .beta..sub.u =[.beta..sub.u (0), .beta..sub.u (1), . . . .beta..sub.u (M-1)]. The recursive calculation of .alpha..sub.t and .beta..sub.u continues until at least two sets of joint state transition probabilities can be calculated (step 40). Then, calculate .sigma..sub.tf/2) (m',m) and .sigma..sub.(t.sbsb.sf.sub./2)+1 (m',m) are calculated using equation (17) (step 42). The calculation of these two state transition probabilities utilizes the two most recently calculated .alpha. and .beta. vectors. For the remaining .alpha. and .beta. recursions, two state transition probabilities are calculated. In calculating this pair, the current .alpha..sub.t and .beta..sub.u vectors are used so that they do not need to be stored. The other .alpha. and .beta. vectors required are read from memory. Therefore, when .alpha..sub.t.sbsb.f and .beta..sub.1 are calculated, the last two sets of state transition probabilities in this code word segment (.alpha..sub.1 (m',m) and .sigma..sub.t.sbsb.f (m',m)) are immediately computed. The discussion above assumes that t.sub.f is an even number. If t.sub.f is an odd number, .sigma..sub.0 (m',m) are not needed. They may be discarded or not computed.

In step 43, a function (e.g. .lambda..sub.t (m',m)) of the selected probabilities is calculated as the MAP decoder's soft-decision.

In step 44, v is decremented by 1 and w is incremented by 1. Then, a test is made in step 46 to determine whether t=t.sub.f -1. If not, then in step 48, t is incremented by 1 and u is decremented by 1, and the process returns to step 38. But, if the answer to the test of step 46 is positive, then operation proceeds to step 56 where a decision is made as to whether hard-decision or soft-decision decoding is selected.

If hard-decision decoding is selected (step 56), apply the soft-decision values (e.g. .lambda..sub.t (m',m)) corresponding to the information bits to a decision device (step 57) and output hard-decision values for the decoded information bits (step 58). If not, output soft-decision values that are some function of the computed .sigma..sub.t (m',m) (step 59).

While the preferred embodiments of the present invention have been shown and described herein, it will be obvious that such embodiments are provided by way of example only. Numerous variations, changes and substitutions will occur to those of skill in the art without departing from the invention herein. Accordingly, it is intended that the invention be limited only by the spirit and scope of the appended claims.

Claims

1. A method for decoding trellis codes, comprising:

simultaneously calculating, in parallel recursive processes, forward and backward state probabilities for received trellis codewords according to a MAP decoding process;

calculating a posteriori state transition probabilities for the received component codewords beginning when the forward and backward recursive processes coincide in time.

2. The method of claim 1 wherein two a posteriori state transition probabilities are calculated per recursion.

3. The method of claim 1 wherein the trellis codes comprise turbo codes.

4. The method of claim 1 wherein the MAP decoding process comprises a log-MAP decoding process.

5. The method of claim 4 wherein the trellis codes comprises turbo codes.

6. A decoder for decoding trellis codes, comprising:

a forward and backward state probability calculator for simultaneously calculating, in parallel recursive processes, forward and backward state probabilities for received trellis codewords according to a MAP decoding process;

an a posteriori state transition calculator for calculating a posteriori state transition probabilities for the received component codewords beginning when the forward and backward recursive processes coincide in time.

7. The decoder of claim 6 wherein two a posteriori state transition probabilities are calculated per recursion.

8. The decoder of claim 6 wherein the trellis codes comprise turbo codes.

9. The decoder of claim 6 wherein the MAP decoding process comprises a log-MAP decoding process.

10. The decoder of claim 9 wherein the trellis codes comprise turbo codes.