1. Field of the Invention
The present invention relates generally to data management in electronic devices, and more particularly to memory and decoder hardware management in such devices.
2. Description of the Related Art
This section is intended to introduce the reader to various aspects of art that may be related to various aspects of the present invention, which are described and/or claimed below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present invention. Accordingly, it should be understood that these statements are to be read in this light, and not as admissions of prior art.
Electronic devices are widely used for a variety of tasks. Among the functions provided by electronic devices, audio playback, such as playback of music, audiobooks, podcasts, lectures, etc., is one of the most widely used. The audio tracks played by such electronic devices may be stored in audio files encoded in a number of different formats. For example, some formats may include compressed formats, such as MPEG-1 Audio Layer 3 (MP3), Advanced Audio Coding (AAC), etc. Typically, the audio may be stored as a file in a non-volatile memory accessible to or integrated in the electronic device. The audio may then be decoded during playback via a specific decoder for each format (the encoder and decoder for a format are commonly referred to as a “codec”).
At any time, an electronic device may store or have access to files encoded in a variety of formats. For example, a device may access an audio file in MP3 format, another audio file in AAC format, etc. The availability and large numbers of formats ensures that different codecs will frequently be used to encode audio files for storage on an electronic device. Similarly, these different codecs may be used to decode the files during playback.
During playback, it may be desirable to have consecutive audio streams (i.e., audio tracks) “fade” in and out of each other. Such a technique is referred to as “crossfading.” A first stream may be slowly faded out, e.g., by decreasing the playback volume of the track, and a second stream may be slowly faded in, e.g., by increasing the playback volume of the track. If the first stream is encoded using a different codec than the second stream, however, both streams are decoded using different codecs. The resources of the electronic device may be insufficient to provide uninterrupted playback of two or more audio streams while decoding two streams using different codecs. Additionally, memory used to store each decoded audio stream may not be sufficiently managed with the decoding processes to ensure uninterrupted playback and elimination of audio artifacts (e.g., skipping, pauses, etc.). As electronic devices increase in portability and decrease in size, the corresponding decrease in available resources such as memory, processing power, battery life, etc. may limit the data decoding and memory management capabilities of the electronic device.
Certain aspects commensurate in scope with the originally claimed invention are set forth below. It should be understood that these aspects are presented merely to provide the reader with a brief summary of certain forms of the invention might take and that these aspects are not intended to limit the scope of the invention. Indeed, the invention may encompass a variety of aspects that may not be set forth below.
In one embodiment, a portable electronic device is provided that includes an audio processor and corresponding audio memory. The portable electronic device includes a storage having one or more audio files stored in various encoded formats. The audio processor decodes audio data from the encoded audio files and transmits the output, decoded data of an audio stream, to a memory buffer of the device. For a crossfade of two audio streams, the buffer may store enough data for each stream to be crossfaded in and out of the real-time output. To minimize size, heat, cost, power usage, and other parameters, the processor may be limited to decoding only one audio stream at a time and incapable of decoding two streams simultaneously. The processor can switch between decoders based on the duration of playback time, i.e., amount of data, stored in the buffer.
In one implementation, data of a first stream is decoded via a first decoder and stored in the buffer. The audio processor may switch to a second decoder based on the amount of decoded data stored in the buffer, and data of a second stream is decoded via the second decoder. A delta may be determined between the empty space of the buffer and the data of the first stream, and the first stream is decoded until the delta is full of the decoded data of the first stream.
Advantages of the invention may become apparent upon reading the following detailed description and upon reference to the drawings in which:
One or more specific embodiments of the present invention will be described below. In an effort to provide a concise description of these embodiments, not all features of an actual implementation are described in the specification. It should be appreciated that in the development of any such actual implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which may vary from one implementation to another. Moreover, it should be appreciated that such a development effort might be complex and time consuming, but would nevertheless be a routine undertaking of design, fabrication, and manufacture for those of ordinary skill having the benefit of this disclosure.
Turning now to the figures,
In certain embodiments the electronic device 10 may be powered by a rechargeable or replaceable battery. Such battery-powered implementations may be highly portable, allowing a user to carry the electronic device 10 while traveling, working, exercising, and so forth. In this manner, a user of the electronic device 10, depending on the functionalities provided by the electronic device 10, may listen to music, play games or video, record video or take pictures, place and take telephone calls, communicate with others, control other devices (e.g., the device 10 may include remote control and/or Bluetooth functionality, for example), and so forth while moving freely with the device 10. In addition, in certain embodiments the device 10 may be sized such that it fits relatively easily into a pocket or hand of the user. In such embodiments, the device 10 is relatively small and easily handled and utilized by its user and thus may be taken practically anywhere the user travels. While the present discussion and examples described herein generally reference an electronic device 10 which is portable, such as that depicted in
In the depicted embodiment, the electronic device 10 includes an enclosure 12, a display 14, user input structures 16, and input/output connectors 18. The enclosure 12 may be formed from plastic, metal, composite materials, or other suitable materials or any combination thereof. The enclosure 12 may protect the interior components of the electronic device 10 from physical damage, and may also shield the interior components from electromagnetic interference (EMI).
The display 14 may be a liquid crystal display (LCD) or may be a light emitting diode (LED) based display, an organic light emitting diode (OLED) based display, or other suitable display. Additionally, in one embodiment the display 14 may be a touch screen through which a user may interact with the user interface.
In one embodiment, one or more of the user input structures 16 are configured to control the device 10, such as by controlling a mode of operation, an output level, an output type, etc. For instance, the user input structures 16 may include a button to turn the device 10 on or off. In general, embodiments of the electronic device 10 may include any number of user input structures 16, including buttons, switches, a control pad, keys, knobs, a scroll wheel, or any other suitable input structures. The input structures 16 may work with a user interface displayed on the device 10 to control functions of the device 10 or of other devices connected to or used by the device 10. For example, the user input structures 16 may allow a user to navigate a displayed user interface or to return such a displayed user interface to a default or home screen.
The electronic device 10 may also include various input and/or output ports 18 to allow connection of additional devices. For example, a port 18 may be a headphone jack that provides for connection of headphones. Additionally, a port 18 may have both input/output capabilities to provide for connection of a headset (e.g. a headphone and microphone combination). Embodiments of the present invention may include any number of input and/or output ports, including headphone and headset jacks, universal serial bus (USB) ports, Firewire or IEEE-1394 ports, and AC and/or DC power connectors. Further, the device 10 may use the input and output ports to connect to and send or receive data with any other device, such as other portable electronic devices, personal computers, printers, etc. For example, in one embodiment the electronic device 10 may connect to a personal computer via a USB, Firewire, or IEEE-1394 connection to send and receive data files, such as media files.
Turning now to
As discussed herein, in certain embodiments the user interface 20 may be displayed on the display 14, and may provide a means for a user to interact with the electronic device 10. The user interface may be a textual user interface, a graphical user interface (GUI), or any combination thereof, and may include various layers, windows, screens, templates, elements or other components that may be displayed in all or some of the areas of the display 14.
The user interface 20 may, in certain embodiments, allow a user to interface with displayed interface elements via the one or more user input structures 16 and/or via a touch sensitive implementation of the display 14. In such embodiments, the user interface provides interactive functionality, allowing a user to select, by touch screen or other input structure, from among options displayed on the display 14. Thus the user can operate the device 10 by appropriate interaction with the user interface 20.
The processor(s) 22 may provide the processing capability required to execute the operating system, programs, user interface 20, and any other functions of the device 10. The processor(s) 22 may include one or more microprocessors, such as one or more “general-purpose” microprocessors, a combination of general and special purpose microprocessors, and/or ASICS. For example, the processor(s) 22 may include one or more reduced instruction set (RISC) processors, such as a RISC processor manufactured by Samsung, as well as graphics processors, video processors, and/or related chip sets.
Embodiments of the electronic device 10 may also include a memory 24. The memory 24 may include a volatile memory, such as RAM, and a non-volatile memory, such as ROM. The memory 24 may store a variety of information and may be used for a variety of purposes. For example, the memory 24 may store the firmware for the device 10, such as an operating system for the device 10 and/or any other programs or executable code necessary for the device 10 to function. In addition, the memory 24 may be used for buffering or caching during operation of the device 10.
The device 10 in
The embodiment in
The device 10 depicted in
The device 10 may also include or be connected to a power source 32. In one embodiment, the power source 32 may be a battery, such as a Li-Ion battery. In such embodiments, the battery may be rechargeable, removable, and/or attached to other components of the device 10. Additionally, in certain embodiments the power source 32 may be an external power source, such as a connection to AC power and the device 10 may be connected to the power source 32 via the I/O ports 18.
To process and decode audio data, the device 10 may include an audio processor 34. In one implementation the audio processor 34 may include and be referred to as a “hardware decoder,” as one of the primary functions of the processor 34 is to decode audio data encoded in a particular format. However, it should be appreciated that the audio processor 34 may also include any other suitable functions and capabilities. Thus, in some embodiments, the audio processor 34 may also be referred to as a codec, an accelerator, etc. In some embodiments, the audio processor 34 may include a memory management unit 36 and a dedicated memory 38, i.e., memory only accessible for use by the audio processor 34. The memory 38 may include any suitable volatile or non-volatile memory, and may be separate from, or a part of, the memory 24 used by the processor 22. In other embodiments, the audio processor 34 may share and use the memory 24 instead of or in addition to the dedicated audio memory 38. The audio processor 34 may include the memory management unit (MMU) 36 to manage access to the dedicated memory 38.
As described above, the storage 26 may store media files, such as audio files. In an embodiment, these media files may be compressed, encoded and/or encrypted in any suitable format. Encoding formats may include, but are not limited to, MP3, AAC, ACCPlus, Ogg Vorbis, MP4, MP3Pro, Windows Media Audio, or any suitable format.
To playback media files, e.g., audio files, stored in the storage 26, the device 10 may decode the audio files before output to the I/O ports 18. As used herein, the term decoding may include decompressing, decrypting, or any other technique to convert data from one format to another format. The decoding is performed via the audio processor 34, and each encoded file may be decoded through the execution of a decoder, i.e., codec, on the audio processor 34. After decoding, the data from the audio files may be streamed to memory 24, the I/O ports 18, or any other suitable component of the device 10 for playback.
In the transition between two audio streams during playback, the device 10 may crossfade audio streams, such as by “fading out” playback of a first audio stream while simultaneously “fading in” playback of a second audio stream. Each audio stream may be a decoded stream from encoded data such as an audio file, and each stream may be decoded from the same or a different format. For example, the first audio stream may be decoded from an MP3 audio file, and the second audio stream may be decoded from an AAC audio file. After the second audio stream is faded in, and the first audio stream is faded out, the transition to any additional audio streams may also include crossfading.
The x-axis of
At point t1, stream B begins to increase in level and stream A begins to decrease in level. Between t1 and t2, the level of stream A is reduced, while the level of stream B increases, crossfading the two streams A and B. At t3, stream A has ended or is reduced to the lowest level, and stream B is at the highest level. As stream B nears the end of its duration, another stream may be added to the mix using the crossfading techniques described above, e.g., stream B is decreased in level and the next stream is increased in level.
Because each audio stream A and B may be decoded from data, e.g., audio files, in different formats, the audio processor 34, in one embodiment, may enable crossfading by switching between decoders, i.e., codecs, in the transition during playback. Switching between decoders during the decoding process may be referred as “multiplexing.” As explained further below, to provide uninterrupted real-time output and crossfading between two decoded audio streams, the decoding for each stream may be faster than the decoding performed for one stream alone. To ensure no interruptions in real-time output, each stream may be decoded at least twice as fast as decoding one stream. In an embodiment, the audio processor 34 may be capable of decoding one audio stream as fast as necessary to maintain uninterrupted real-time output.
Further, as described below, to enable multiplexing the audio processor 34 can stop decoding a stream, store the state, and load a new state and decoder, e.g., decoder 46, for a second stream. This multiplexing may be repeated several times during decode and output of audio streams A and B. Additionally, in an embodiment in which only one of stream A or B is decoded at one time by the audio processor 34, the processor 34 may only include enough processing capability and dedicated memory 38 for decoding one stream, reducing the memory requirements of the processor 34. Further, if the crossfading transition is extended, no additional memory is required, as the processor 34 can switch between decoders 44 and 46 and decode stream A or B to extend the crossfade “on the fly”.
The data from the audio files A and B may be streamed from main memory 24 to the audio processor 34 for decoding, as illustrated by lines 52. In one embodiment, the data from the memory 24 may be transmitted to the audio processor 34 via a DMA request. The audio processor 34 may execute the decoders 44 and 46 to decode the encoded audio streams A and B respectively into a decoded audio stream. In an embodiment, the decoders, e.g., codecs, may be stored in the audio memory 38, the main memory 24, and/or the storage 26. For example, codecs may be stored in the storage 26 and loaded into main memory 24 and/or audio memory 38 upon initiation of the decoding process.
As described above, the audio processor 34 may multiplex decoders 44 and 46, alternately decoding audio streams A and B, as illustrated in area 56. The logic to control multiplexing, e.g. switching, of the decoders 44 and 46 may be implemented in the processor 22 and/or the audio processor 34. For example, the processor 22 may analyze the playback of the decoded audio streams A and B, the memory 24, and signal to the audio processor 34 when to switch decoding from stream A to stream B and vice-versa, as illustrated by line 58. Additionally, a debugging and/or control signal may be provided to the audio memory 38, as illustrated by line 60.
During the decoding process, the audio processor 34 may read or write data into and out of the dedicated audio memory 38. The audio processor 34 may interface with the audio memory 38 though a memory management unit 36, as illustrated by lines 62. The memory management unit 36 manages access to the audio memory 38 and can provide data out the audio memory 38 for decoding, such as decoded streams, codecs, etc., to the audio processor 34. The output from the audio processor 34, decoded output streams A and B may be stored in the audio memory 38.
Output streams A and B from the audio processor 34, i.e., decoded data streams A and B, may be provided to a buffer, such as a circular buffer 66, in the main memory 24 of the device 10, as illustrated by lines 64. As explained further below, the circular buffer 66 stores decoded streams A and B to ensure that an adequate duration of either stream is available for playback and for crossfading during playback. The decoded streams A and B may be read out of the circular buffer 66, such as through a DMA request 68, and output to an digital-to-analog (D/A) converter and/or other processing logic 70. A mix of the streams A and B may be output to an I/O port 18 of the device 10, such as a headphone port, headset port, USB port, etc. In other embodiments, a mix of the streams A and B may be output digitally over I/O ports 18, e.g., omitting the D/A converter of the processing logic 70.
If the process 80 determines to switch decoding to the other stream of the crossfade, e.g., audio stream B, the current decoder is suspended (block 90). The audio processor 34 (or other processor) may load a state and codec for another decoder (block 92), such as from the dedicated audio memory 38. The active stream being decoded is now the other stream of the crossfade, e.g., stream B. After the state and code for the other decoder are loaded, the process 80 continues decoding frames from the active stream (block 84), e.g., audio stream B. In some embodiments, switching of the decoders may also be based on additional parameters of the device 10, such as battery life, the amount of time to switch decoders, the amount of processing overhead to switch decoders, etc. These and similar additional factors may be considered the “cost” of switching between decoders. Additionally, the different frame size of the encoded formats may be considered when multiplexing and switching decoders. For example, the number of samples in a decoded MP3 frame is 1152 samples per frame, and the number of samples in a decoded AAC frame is 1024 samples per frame.
In other embodiments, the penalty of switching of the decoders (e.g., codecs) as illustrated in
The processor(s) 22 (or other processor of the device 10) may determine when to switch decoders, and which stream to decode, based on the data for each decoded stream A and B stored in the circular buffer 66. In one embodiment, the decoders executed by the audio processor 34 may be switched such that the stream that is behind in time since the start of the crossfade is the active stream, i.e., the stream being currently decoded by the audio processor 34. For example, if 5.01 seconds of stream A have been decoded and stored in the buffer 66, and 4.95 seconds of stream B have been decoded and stored in the buffer 66, the audio processor 34 will decode stream B until the amount of data for stream B stored in the circular buffer 66, e.g., the amount of playback time, at least exceeds that of stream A.
In some embodiments, however, the decoder multiplexing and switching may be a relatively resource-intensive process. For example, if DMA requests are used to transmit audio streams to the audio processor 34, the decoder switching process may allow the audio processor 34 to decode multiple frames of data before stopping the input stream (and clearing any buffers), stopping the decoding, switching to another decoder, and starting to transmit the frames of the new stream via DMA requests. Additionally, the amount of time that the decoding is ahead of real-time output also affects the decision to switch decoders. If the read pointer is permitted to get too close to the last time that data has been decoded for both streams (such as near the ACTIVE arrow noted above), the device 10 may “starve” on one of the streams depending on how long it takes to switch decoders, e.g., no data is available for one of the streams stored in the buffer 66. For example, as shown in
As shown in
As shown in