This application claims the priority benefit of Taiwan application serial no. 109136790, filed on Oct. 23, 2020. The entirety of the above-mentioned patent application is hereby incorporated by reference herein and made a part of this specification.
The invention relates to an image processing technique, and more particularly to an image processing device and an image processing method reducing bandwidth requirements to achieve multi-screen display.
Existing TV walls or multi-screen displays usually receive a plurality of image screens from a plurality of image sources and output the image screens to the same screen by partial overlap or reducing the display screen, for example. In particular, the portion of the image that is blocked and not displayed due to the overlap of the display screen still occupies the memory and transmission bandwidth of the hardware equipment. For example, as the number of image sources is increased, the bandwidth requirements of the hardware equipment is increased proportionally, and the computing resources of the hardware equipment are significantly occupied, thus affecting the quality of the display images, and the images may not all be displayed at a size close to full screen.
Accordingly, the invention provides an image processing device and an image processing method for displaying a multi-screen that may reduce the cost of the hardware structure and may reduce bandwidth requirements in the case of a full screen.
The image processing device of the invention includes an image processing circuit, a transmission arrangement circuit, and an image merge circuit. The image processing circuit receives a first image information with a first bandwidth, receives a second image information with a second bandwidth, and processes the first image information and the second image information to generate a first image information packet and a second image information packet. The transmission arrangement circuit receives the first image information packet and the second image information packet. The image merge circuit receives the first image information packet and the second image information packet from the transmission arrangement circuit with a third bandwidth, restores the first image information packet and the second image information packet to the first image information and the second image information, and outputs the first image information and the second image information to a display together. In particular, when the first image information and the second image information are in a full-screen mode, the third bandwidth is less than a sum of the first bandwidth and the second bandwidth.
The image processing method for displaying a multi-screen of the invention is suitable for an image processing device including an image merge circuit and a display, and includes the following steps. A first image information is obtained with a first bandwidth, a second image information is obtained with a second bandwidth, and the first image information and the second image information are processed to generate a first image information packet and a second image information packet. The first image information packet and the second image information packet are transmitted to the image merge circuit with a third bandwidth. The first image information packet and the second image information packet are restored to the first image information and the second image information by the image merge circuit. The first image information and the second image information are outputted to a display together. In particular, when the first image information and the second image information are in a full-screen mode, the third bandwidth is less than a sum of the first bandwidth and the second bandwidth.
Based on the above, the image processing device and the image processing method for displaying a multi-screen of an embodiment of the invention may process a plurality of image sources to reduce the data volume of undisplayed images that are blocked due to overlapped display screens, so that the undisplayed images do not occupy the transmission bandwidth of the hardware equipment. In this way, under the premise of maintaining refresh rate and not affecting display image quality, the number of superimposed input images and the display size of the image screens are not limited. Moreover, the image screens may all be displayed in a size close to full screen, and there is no need to increase the transmission bandwidth and temporary storage space for the image processing device to process and transmit the image screens, thereby saving costs and reducing the complexity of hardware design.
In order to make the aforementioned features and advantages of the disclosure more comprehensible, embodiments accompanied with figures are described in detail below.
The accompanying drawings are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.
In other words, the image processing device 100 may be implemented by being integrated into a single equipment to reduce the wiring area of the circuit board and increase the working efficiency of circuit elements. Alternatively, the image processing device 100 may be implemented by a plurality of electronic devices connected to each other. The type of the display 140 may be liquid crystal display (LCD), light-emitting diode (LED) display, field-emission display (FED), organic light-emitting diode (OLED), or other types of displays.
The image processing circuit 110 is configured to receive and process a plurality of image sources. When a plurality of image sources are stacked on top of each other on the display screen of the display 140, the data volume of image pixels blocked by the images due to the stacking may be reduced, thereby reducing the transmission bandwidth needed. In the present embodiment and
The transmission arrangement circuit 120 may receive and arrange to transmit a plurality of image information packets to the image merge circuit 130. In the present embodiment, the transmission arrangement circuit 120 may transmit the first image information packet PKG1 and the second image information packet PKG2 to the image merge circuit 130 according to a third bandwidth BW3.
The image merge circuit 130 may receive and restore a plurality of image information packets, and then output the plurality of image information packages to the display 140 together for display. In the present embodiment, the image merge circuit 130 may receive and restore the first image information packet PKG1 and the second image information packet PKG2 to the processed first image information IMG1 and the processed second image information IMG2, and then output the processed first image information IMG1 and the processed second image information IMG2 together to the display 140. It should be mentioned here that, at this time, the data volume of image pixels of the processed first image information IMG1 and the processed second image information IMG2 is less than the original unprocessed first image information IMG1 and second image information IMG2. For example, if a portion of the first image information IMG and the second image information IMG2 is stacked on the display 140, the image pixels of the masked region (unviewable region) may not be transmitted to reduce the data volume thereof. In this way, the bandwidth and temporary storage space needed for processing and transmission are still maintained in a state equivalent to transmitting only a single image information. Therefore, when the first image information IMG1 and the second image information IMG2 are in full-screen mode, the third bandwidth BW3 is less than the sum of the first bandwidth BW1 and the second bandwidth BW2 (BW1+BW2>BW3). In other words, when a plurality of input sources are stacked on top of each other or are all in full-screen mode, the image processing device 100 of the invention may make the transmission bandwidth less than the sum of a plurality of input bandwidths without affecting display quality to achieve the effect of saving cost and reducing the complexity of hardware design. In particular, the first, second, and third bandwidths mentioned above may be the maximum bandwidths in the transmission specifications, and are all 5 Gbps, for example. However, the actual bandwidth used when transmitting data may not fully reach the upper limit of the bandwidth, depending on data volume.
In other embodiments, in addition to the image processing circuit 110, the transmission arrangement circuit 120, and the image merge circuit 130 shown in solid lines in
In an embodiment, the image processing circuit 110 may be coupled to the controller 150. The controller 150 may have input and output functions. For example, a display window may be provided for the user to view and adjust the image size and relative positions of a plurality of image sources displayed on the display 140, and to reduce the image pixels that need to be outputted according to the adjustment made by the user. For example, in the present embodiment, the controller 150 may adjust the masks respectively corresponding to the first image information IMG1 and the second image information IMG2 according to the adjustment operation on the display window (i.e., the display window adjustment operation in
In an embodiment, the image processing circuit 110 may be coupled to the first image buffer 160 to temporarily store inputted image information. In an embodiment, the transmission arrangement circuit 120 continues to transmit image information packets to the image merge circuit 130. The image merge circuit 130 may be coupled to the second image buffer 170 to temporarily store the image information packets, and perform subsequent processing after reception is completed. The first image buffer 160 may be provided in the image processing circuit 110, and the second image buffer 170 may be provided in the image merge circuit 130, and the invention is not limited in this regard. The type of the first image buffer 160 and the second image buffer 170 may be any type of fixed or removable random-access memory (RAM), read-only memory (ROM), flash memory, or a combination thereof.
In an embodiment, the image merge circuit 130 may include an image scaler 180 configured to perform image scaling of the inputted image and adjust the display position on the display 140. According to different design requirements, the image processing circuit 110, the transmission arrangement circuit 120, the image merge circuit 130, the controller 150, and the image scaler 180 may be implemented in hardware, firmware, software (i.e., a program), or multiple combinations of the above three.
In terms of hardware, elements such as the image processing circuit 110, the transmission arrangement circuit 120, the image merge circuit 130, the controller 150, and the image scaler 180 may be logic circuits implemented on an integrated circuit. Alternatively, the elements may be implemented as hardware using a hardware description language (for example, Verilog HDL or VHDL) or other suitable programming languages. For example, the related functions of the image processing circuit 110, the transmission arrangement circuit 120, the image merge circuit 130, the controller 150, and/or the image scaler 180 may be implemented in one or a plurality of microcontrollers, microprocessors, application-specific integrated circuits (ASIC), digital signal processors (DSP), field-programmable gate arrays (FPGA), complex programmable logic devices (CPLD), and/or various logic blocks, modules, and circuits in other processing units.
In the form of software and/or firmware, the image processing circuit 110, the transmission arrangement circuit 120, the image merge circuit 130, the controller 150, and/or the image scaler 180 may be implemented as programming codes. For example, the above may be implemented using a general programming language (such as C, C++, or a combination language) or other suitable programming languages. The programming codes may be recorded/stored in a recording medium, and the recording medium includes, for example, a read-only memory (ROM), a storage device, and/or a random-access memory (RAM). A computer, a central processing unit (CPU), a microcontroller, or a microprocessor may read and execute the programming codes from the recording medium to achieve a related function. As the recording medium, a “non-transitory computer-readable medium” may be used. For example, a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, etc. may be used. Moreover, the program may also be provided to a computer (or CPU) via any transmission medium (communication network or broadcast wave, etc.) The communication network is, for example, the Internet, wired communication, wireless communication, or other communication media.
Examples are listed below to illustrate the invention. Please refer to
In the present embodiment, the image processing circuit 210 receives image information IMG3 to IMG6 and temporarily stores the image information IMG3 to IMG6 in the first image buffer 260. The controller 250 may adjust masks MSK1 to MSK4 according to the display window adjustment operation inputted by the user, so that the image information IMG3 to IMG6 may respectively pass through paths PTH1 to PTH4 and the masks MSK1 to MSK4 to reduce the data volume of image pixels outputted and generate image information packets PKG3 to PKG6. Examples are provided below. In an embodiment, the masks MSK1 to MSK4 may be configured to represent a plurality of different image viewable regions relative to the full-screen region, and the plurality of different image viewable regions are not overlapped with each other. Here, the full-screen region may be the maximum display range of the display 240, and the image viewable region may be the display range of the image information IMG3 to IMG6 on the display 240 that is not blocked. In other words, the masks MSK1 to MSK4 may filter out the image pixels of the image information IMG3 to IMG6 that are blocked on the display 240 due to image stacking, so as to reduce bandwidth requirements during transmission.
The transmission arrangement circuit 220 receives the image information packets PKG3 to PKG6 generated by the image processing circuit 210 and transmits the image information packets PKG3 to PKG6 to the image merge circuit 230. The image merge circuit 230 may temporarily store the image information packets PKG3 to PKG6 using the second image buffer 270 and restore the image information packets PKG3 to PKG6 to a plurality of processed image information (the data volume of image pixels at this time is less than the original input image information IMG3 to IMG6), and then merge the plurality of processed image information into an output image OP after the size and display position are adjusted using the image scalar 280. In other words, for the user, the output image OP visually displays a screen in which the image information IMG3 to IMG6 stacked. But in fact, through the processing of the image processing device 200 of the invention, the blocked image pixel data may be reduced or not transmitted. Therefore, the transmission bandwidth needed for the transmission of the image information packets PKG3 to PKG6 may be reduced, and the temporary storage space needed for the temporary storage of the image information packets PKG3 to PKG6 may also be reduced. Based on the above, although the image processing device 200 receives four image information IMG3 to IMG6, the needed internal transmission bandwidth thereof is substantially only equivalent to the image data volume for transmitting any single image information IMG3 to IMG6. As a result, when the image information IMG3 to IMG6 are all displayed in full-screen mode on the display 240, the needed transmission bandwidth inside the image processing device 200 is less than the sum of the input bandwidth of the image information IMG3 to IMG6. Compared with the prior art that needs to increase the internal transmission bandwidth and temporary storage space when more image information is received, the invention may further save costs and reduce the complexity of hardware design.
In the present embodiment, the image information packets PKG3 to PKG6 may include a display image and an image setting information corresponding to the image information IMG3 to IMG6. The image scaler 280 may configure the display positions of the display images in the output image OP according to the image setting information, so as to integrate the processed image information into the output image OP. For example, the image scaler 280 may first configure the display position of the display image corresponding to the image information IMG3 in the output image OP according to the image setting information in the image information packet PKG3, then configure the display positions of a plurality of display images corresponding to the image information IMG4 to IMG6 in sequence to integrate as a complete output image OP. Here, the display image may be a viewable image in which the image information IMG3 to IMG6 are not stacked and blocked on the display 240. In an embodiment, the image setting information may be configured to indicate that the display image is located in different regions of the full-screen region of the display 240, and the regions are not overlapped with each other. In an embodiment, the image setting information may also include displacement information and size information of regions relative to the full-screen region. In other words, the image scaler 280 may configure the display positions of the display images on the output image OP according to the displacement information and the size information of the respective corresponding regions in the image setting information. In this way, the number of superimposed image sources may not be limited, and even under the premise of maintaining the refresh rate and not affecting display image quality, the bandwidth requirements for generating the output image OP is substantially maintained at the bandwidth requirement of a single image, thus significantly saving hardware cost.
Please refer to
In contrast, in the example on the right of
Please refer to
It should be mentioned that the time division multiplexing transmission of the present embodiment means that the transmission arrangement circuit 420 arranges different transmission time intervals for the sub-packets of the different image information packets PKG3 to PKG6 using the image refresh time of the display, and then sequentially transmits the sub-packets to the image merge circuit 430. For example, in the present embodiment, the complete display screen of the output image OP is composed of 18 sub-packets, containing 7, 3, 4, and 4 sub-packets occupied by the image information packets PKG3 to PKG6, respectively. Then the transmission arrangement circuit 420 may calculate the transmission time intervals of the respective sub-packets of the image information packets PKG3 to PKG6 (for example, the transmission time interval of the sub-packets of the image information packet PKG3 may be 16.6 milliseconds/7=2.37 milliseconds) using the image refresh time of the display (for example, 16.6 milliseconds/60 Hz) so as to transmit the plurality of sub-packets of the image information packets PKG3 to PKG6 in average. In this way, the time division multiplexing transmission method described in the present embodiment may avoid issues such as the image information packets PKG3 to PKG6 are divided into different numbers of sub-packets and sent at the same time thus causing the loss of sub-packets due to transmission in an overly concentrated manner at the beginning, and wasted transmission bandwidth after a portion of the sub-packets of the image information packets PKG3 to PKG6 is transmitted (for example, the image information packet PKG4 in
Please refer to both
In the present embodiment, the image processing circuit 510 may separately transmit the image information packets respectively corresponding to the viewable image not blocked due to stacking and the blocked unviewable image on the display screen of the display 540 (such as the position, displacement information, and size information of unviewable image pixels) to be restored to a complete output image by the image merge circuit 530. For example, in the present embodiment, when the display image of the display 540 in
Please refer to
In step S620, the image processing device 100 transmits the first image information packet and the second image information packet to the image merge circuit 130 with a third bandwidth. In step S630, the first image information packet and the second image information packet are restored to the first image information and the second image information by the image merge circuit 130. In step S640, the image merge circuit 130 outputs the first image information and the second image information to the display 140 together, and when both the first image information and the second image information are in full-screen mode, the third bandwidth is less than the sum of the first bandwidth and the second bandwidth. For the implementation details of the steps of the above image processing method, please refer to the description of the above embodiments.
Based on the above, in an embodiment of the invention, the image processing device and the image processing method for displaying a multi-screen may process a plurality of image sources to reduce the data volume of undisplayed images that are blocked due to overlapped display screens, so that the undisplayed images do not occupy the transmission bandwidth of the hardware equipment. In this way, under the premise of maintaining refresh rate and not affecting display image quality, the number of superimposed input images and the display size of the image screens are not limited. Moreover, the image screens may all be displayed in a size close to full screen, and the transmission bandwidth and temporary storage space needed for the image processing device to process and transmit the image screens are not increased, thereby saving costs and reducing the complexity of hardware design.
Although the invention has been described with reference to the above embodiments, it will be apparent to one of ordinary skill in the art that modifications to the described embodiments may be made without departing from the spirit of the disclosure. Accordingly, the scope of the disclosure is defined by the attached claims not by the above detailed descriptions.
Number | Date | Country | Kind |
---|---|---|---|
109136790 | Oct 2020 | TW | national |