METHOD AND APPARATUS FOR VIDEO TELEPHONY IN PORTABLE TERMINAL

Abstract

Provided is a method and apparatus for video telephony in a portable terminal. In the method, all image frames received from a video telephony counterpart terminal are decoded when a video telephony event is detected. Thereafter, it is determined if the received image frames are accumulated in a buffer of the portable terminal. Thereafter, only a portion of the decoded image frames are selected and displayed when the received image frames are accumulated in the buffer.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which:

FIG. 1A illustrates an apparatus for processing image frames received in a conventional portable terminal:

FIG. 1B illustrates a graph showing the time-dependent number of the received frames in a conventional portable terminal;

FIG. 2A illustrates an apparatus for processing image frames cumulatively received in a conventional portable terminal:

FIG. 2B illustrates a graph showing the time-dependent number of the cumulatively-received frames in a conventional portable terminal:

FIG. 3 is a block diagram of a portable terminal according to the present invention: and

FIG. 4 is a flowchart illustrating a procedure for processing image frames accumulated during a video telephony operation of a portable terminal according to the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Preferred embodiments of the present invention will be described herein below with reference to the accompanying drawings. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail.

The present invention provides a method and apparatus for, when image frames are cumulatively stored during a video telephony operation of a portable terminal selectively displaying the cumulatively-stored image frames to prevent a processing delay of the image frame.

Referring to FIG. 3, the portable terminal includes a controller 300, a camera 302, an image processor 304, a frame determiner 306, a display unit 308, a communication module 310 a voice processor 312, a speaker 314 and a microphone 316.

The controller 300 processes and controls voice communication and data communication. According to the present invention when a video telephony event occurs the controller 300 receives an image signal and a per-second frame rate FR_remote from the communication module 310 and outputs the received image signal and the received frame rate FR_remote respectively to image processor 304 and the frame determiner 306.

The camera 302 includes a camera sensor for converting an optical signal sensed during image photographing into an electrical signal and a signal processor for converting an analog image signal sensed by the camera sensor into digital data. The camera 302 outputs the digital image signal to the image processor 304.

The image processor 304 is also called an image CODEC (coder-decoder). The image processor 304 encodes an image signal received from the camera 302 in a predetermined scheme, and decodes encoded frame image data into original frame image data. In addition the image processor 304 processes an output image signal of the camera 302 on a frame by frame basis to output the resulting signal in accordance with the feature and size of the display unit 308, and decodes an image signal received from the controller 300 into an original image frame to output the original image frame to the display unit 308 during the video telephony operation. In particular the image processor 304 includes a buffer for temporarily storing an image frame. The image processor 304 determines whether the image signals received from the controller 300 (i.e., image frames received from a video telephony counterpart terminal) accumulate in the buffer. When the image frames accumulate in the buffer, the image processor 304 outputs a signal indicating the accumulation of the image frames (hereinafter a frame accumulation indicating signal) to the frame determiner 306. Thereafter, the image processor 304 receives image frames to be skipped, i.e., the number NF_skip of image frames to be not displayed (hereinafter the frame skip number NF_skip) from the frame determiner 306 to select NF_skip image frames among the decoded original image frames, and outputs only the remaining image frames other than the selected image frames to the display unit 308.

Every time when receiving the per-second frame rate FR_remote of the counterpart terminal from the controller 300 the frame determiner 306 divides 1 second by the frame rate FR_remote (1 sec/FR_remote) to calculate the maximum processing time T_{process—max} within which one image frame must be processed. In addition, when a video telephony event occurs, the frame determiner 306 measures the average decoding time T_dec taken to decode one image frame and the average display time T_disp taken to display the decoded image frame. Thereafter when receiving a frame accumulation indicating signal from the image processor 304, the frame determiner 306 measures the number NF_dec of image frames cumulatively stored in the buffer of the image processor 304, calculates the total process time (NF_dec×(T_dec+T_disp)) taken to process all the cumulatively-stored image frames, and calculates the over processing time (T_over=NF_dec×(T_dec+T_disp)−T_{process—max}) that is additionally taken over the maximum processing time T_{process—max} Thereafter the frame determiner 306 divides the over processing time by the average display time (T_over/T_disp) to calculate the frame skip number NF_skip and the calculated image skip number to the image processor 304. Alternatively, the over processing time T_over may be determined to be the total process time (NF_dec×(T_dec+T_disp)) minus the maximum total processing time (NF_dec×T_{process—max}) within which the cumulatively-stored image frames must be processed.

The display unit 308 displays a predetermined number of characters and status information generated during an operation of the portable terminal. In particular, the display unit 308 displays the image frames received from the image processor 304 according to the present invention.

The communication module 310 TX/RX-processes radio-frequency (RF) signals transmitted/received through an antenna (not shown). For example, the communication module 310 converts a baseband signal, which is obtained by performing channel coding and spreading on data to be transmitted into an RF signal and transmits the RF signal through the antenna. In addition, the communication module 310 converts a received RF signal into a baseband signal and performs despreading and channel decoding on the received data to recover original data. In particular according to the present invention, the communication module 310 receives an image signal and a per-second frame rate FR_remote from a video telephony counterpart terminal and provides the same to the controller 300. The per-second frame rate FR_remote changes during the video telephony operation. Therefore, every time when the per-second frame rate FR_remote changes, the communication module 310 receives the per-second frame rate FR_remote and provides the same to the controller 300.

The voice processor 312 is also called a voice CODEC. The voice processor 312 I/O-processes voice signals through the speaker 314 and the microphone 316. For example, the voice processor 312 converts Pulse Code Modulation (PCM) data received from the controller 300 into analog audio signals and outputs the analog audio signals to the speaker 314. In addition, the voice processor 312 converts audio signals received from the microphone 316 into PCM data and provides the PCM data to the controller 300.

FIG. 4 is a flowchart illustrating a procedure for processing image frames cumulated during a video telephony operation of the portable terminal according to the present invention.

Referring to FIG. 4, the portable terminal determines in step 401 whether a video telephony mode is initiated. When the video telephony mode is initiated, the portable terminal decodes and displays a received image frame and measures the resources of a video telephony system in step 403. The resources of the video telephony system are the per-second frame rate FR_remote of a counterpart terminal, the maximum processing time T_{process—max} within which one image frame must be processed, the average decoding time T_dec taken to decode one image frame, and the average display time T_disp taken to display one image frame. The maximum processing time T_{process—max} may be obtained by dividing 1 second by the per-second frame rate FR_remote.

In step 405, the portable terminal determines if image frames received from the counterpart terminal are cumulatively stored in the buffer. If not, the procedure returns to step 403: and if so, the procedure proceeds to step 407.

In step 407, the portable terminal decodes all the cumulatively-stored image frames, measures the number of cumulatively-stored image frames, and determines the frame skip number NF_skip using the measured video telephony system resources. The frame skip number NF_skip may be obtained by calculating the over processing time (T_over=NF_dec×(T_dec+T_disp)−T_{process—max}) that is additionally taken over the maximum processing time T_{process—max} and dividing the over processing time T_over by the average display time T_disp (T_over/T_disp). In addition, because each of the image frames is closely con-elated with the previous image frame and the next image frame, the previous image frame is needed in order to accurately decode the image frame. Therefore it is necessary to decode all of the received image frames.

In step 409, the portable terminal selects NF_skip image frames among the decoded image frames. In step 411, the portable terminal displays the remaining image frames other than the selected image frames on the display unit 308. Thereafter, the procedure is ended.

Alternate embodiments of the present invention can also comprise computer readable codes on a computer readable medium. The computer readable medium includes any data storage device that can store data that can be read by a computer system. Examples of a computer readable medium include magnetic storage media (such as ROM, floppy disks, and hard disks, among others), optical recording media (such as CD-ROMs or DVDs), and storage mechanisms such as carrier waves (such as transmission through the Internet). The computer readable medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. Also, functional programs, codes, and code segments for accomplishing the present invention can be construed by programmers of ordinary skill in the art to which the present invention pertains.

As described above when the image frames received from the counterpart terminal are cumulatively stored during the video telephony operation of the portable terminal the present invention selectively displays the cumulatively-stored image frames to prevent a processing delay of the image frame. Accordingly, it is possible to stabilize the CPU occupation rate of an image decoder of the portable terminal.

While the invention has been shown and described with reference to certain preferred embodiments thereof it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

1. A method for video telephony in a portable terminal, comprising the steps of: decoding all image frames received from a video telephony counterpart terminal:determining if a received image frames are accumulated in a buffer of the portable terminal; andwhen the received image frames are accumulated in the buffer, selecting and displaying only a portion of the decoded image frames.

2. The method of claim 1, further comprising measuring video telephony system resources to select the portion of the decoded image frames when the video telephony event is detected.

3. The method of claim 2, wherein the video telephony system resources include at least one of a per-second frame rate of the counterpart terminal, the maximum processing time within which one image frame must be processed the average decoding time taken to decode one image frame, and the average display time taken to display one image frame.

4. The method of claim 1, wherein selecting and displaying only a portion of the decoded image frames comprises: determining the number of image frames not to be displayed;selecting a said determined number of image frames among the decoded image frames: and displaying the remaining image frames other than the selected image frames among the decoded image frames.

5. The method of claim 4, wherein determining the number of image frames not to be displayed comprises: calculating a total processing time taken to process all the image frames accumulated in the buffer:calculating all over processing time by subtracting the maximum processing time, within which one image frame must be processed from the calculated total processing time: and dividing the calculated over processing time by the average display time taken to display one image frame to determine the number of image frames not to be displayed.

6. The method of claim 4 wherein determining the number of image frames not to be displayed comprises: calculating the total processing time taken to process all the image frames accumulated in the buffer;calculating an over processing time by subtracting the maximum total processing time, within which the accumulated image frames must be processed, from the calculated total processing time; anddividing the calculated over processing time by the average display time taken to display one image frame, to determine the number of image frames not to be displayed.

7. The method of claim 5, wherein the total processing time taken to process all the image frames accumulated in the buffer is calculated by multiplying the sum of the average decoding time taken to decode one image frame and the average processing time taken to display one image frame by the number of the accumulated image frames.

8. The method of claim 6, wherein the total processing time taken to process all the image frames accumulated in the buffer is calculated by multiplying the sum of the average decoding time taken to decode one image frame and the average processing time taken to display one image frame by the number of the accumulated image frames.

9. An apparatus for video telephony in a portable terminal comprising: an image processor for determining if image frames received from a counterpart terminal are accumulated in a buffer of the portable terminal, decoding the accumulated image frames, and outputting the decoded image frames selectively according to a number determined by a frame determiner;the frame determiner for when the image frames are accumulated in the buffer, measuring video telephony system resources to determine a number of image frames not to be displayed: anda display unit for displaying the image frames received from the image processor.

10. The apparatus of claim 8, wherein, for the selective outputting of the decoded image frames the image processor selects a predetermined number of image frames among the decoded image frames and outputs the remaining image frames other than the selected image frames among the decoded image frames.

11. The apparatus of claim 8, wherein the video telephony system resources include at least one of a per-second frame rate of the counterpart terminal, the maximum processing time within which one image frame must be processed, the average decoding time taken to decode one image frame, and the average display time taken to display one image frame.

12. A portable terminal for video telephony, comprising: an image processor for determining if the received image frames are accumulated in a buffer of the portable terminal and for decoding the accumulated image frames: anda frame determiner for determining image frames not to be outputted among the decoded image frames.

13. The portable terminal of claim 12, wherein the frame determiner measures video telephony system resources to determine the image frames not to be outputted among the decoded image frames.

14. The portable terminal of claim 12, wherein the video telephony system resources include at least one of a per-second frame rate of the counterpart terminal, the maximum processing time within which one image frame must be processed, the average decoding time taken to decode one image frame, and the average display time taken to display one image flame.

15. A method for video telephony in a portable terminal, comprising the steps of: determining if the received image frames are accumulated in a buffer of the portable terminal:decoding the accumulated image frames: anddetermining image frames not to be outputted among the decoded image frames.

16. A portable terminal for video telephony, comprising: means for determining if the received image flames are accumulated in a buffer of the portable terminal:means for decoding the accumulated image frames, andmeans for determining image frames not to be outputted among the decoded image frames.

17. A computer-readable recording medium having recorded thereon a program for video telephony in a portable terminal comprising: a first code segment for determining if the received image frames are accumulated in a buffer of the portable terminal;a second code segment for decoding the accumulated image frames: anda third code segment determining image frames not to be outputted among the decoded image frames.