Scalability techniques involving formatting the layers as a function of picture distortion after decoding