This application claims the priority benefit of Taiwan application serial no. 111129017, filed on Aug. 2, 2022. The entirety of the above-mentioned patent application is hereby incorporated by reference herein and made a part of this specification.
The disclosure relates to an electronic apparatus, and more particularly, to a 3D format image detection method and an electronic apparatus using the same method.
With the advancement of display technology, displays supporting three dimension (3D) image playback have gradually become popular. The difference between a 3D display and a two dimension (2D) display is that the 3D display technology allows viewers to experience the three dimension effect in the image screen, such as, the three dimension facial features and depth of field of characters, and the like, and traditional 2D images cannot present such effects. In the principle of the 3D display technology, the viewer's left eye is allowed to view the left-eye image and the viewer's right eye is allowed to view the right-eye image, so that the viewer can experience the 3D visual effect. With the vigorous development of 3D display technology, a visually immersive experience can be provided to people. It is known that a 3D display requires a corresponding 3D display technology to play images of a specific 3D image format, otherwise the 3D display may not be able to display the images correctly. Therefore, how to accurately identify the image content conforming to a specific 3D image format is a concern of those skilled in the art.
In view of this, the disclosure provides a 3D format image detection method and an electronic apparatus using the same method, which may accurately identify whether an input image is a 3D format image.
In an embodiment of the disclosure, a 3D format image detection method is provided. The method includes steps as follows. A first image and a second image are obtained by dividing an input image according to a 3D image format. A 3D matching processing is performed on the first image and the second image to generate a disparity map of the first image and the second image. A matching number between multiple first pixels in the first image and multiple second pixels in the second image is calculated according to the disparity map. Whether the input image is a 3D format image conforming to the 3D image format is determined according to the matching number.
In an embodiment of the disclosure, an electronic apparatus is provided. The electronic apparatus includes a storage device and a processor. The processor is connected to the storage device and configured to execute steps as follows. A first image and a second image are obtained by dividing an input image according to a 3D image format. A 3D matching processing is performed on the first image and the second image to generate a disparity map of the first image and the second image. A matching number between multiple first pixels in the first image and multiple second pixels in the second image is calculated according to the disparity map. Whether the input image is a 3D format image conforming to the 3D image format is determined according to the matching number.
In summary, in the embodiments of the disclosure, an input image is divided based on a 3D image format to obtain the first image and the second image. A disparity map is obtained by performing a 3D matching processing on the first image and the second image. Whether the input image conforms to the 3D image format may be determined based on the matching condition of the disparity map. Accordingly, whether the input image is a 3D format image can be effectively determined, thereby improving the user experience and the application range of the 3D display technology.
In order to make the features and advantages of the disclosure comprehensible, embodiments accompanied with drawings are described in detail below.
Some embodiments of the disclosure will be described in detail with reference to the accompanying drawings. For reference numerals cited in the following descriptions, the same reference numerals appearing in different drawings are regarded as the same or similar elements. The embodiments are only a part of the disclosure and do not disclose all possible implementations of the disclosure. More precisely, the embodiments are merely examples of the device and the method.
In one embodiment, the electronic apparatus 10 and a three dimension (3D) display 20 may form a 3D display system. The 3D display 20 may be a naked-view 3D display or a glasses-type 3D display. On the other hand, the 3D display 20 may be a head-mounted display device or a device that provides a 3D image display function, such as a computer monitor, a desktop monitor, a TV set, or the like. The 3D display system can be a single integrated system or a separate system. Specifically, the 3D display 20, the storage device 110, and the processor 120 in the 3D display system can be implemented as an all-in-one (AIO) electronic apparatus, such as a head-mounted display device, a laptop computer, a smart phone, a tablet computer, a game console, and the like. Alternatively, the 3D display 20 can be connected to the processor 120 through a wired transmission interface or a wireless transmission interface, such as a head-mounted display device, a desktop computer monitor, a television set, an electronic signboard, and the like.
The storage device 110 is configured for storing data such as images, data, program codes (e.g., operating system, application programs, drivers) accessed by the processor 120, and the like, and it can be, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory (flash memory), hard disk, or a combination thereof.
The processor 120 is coupled to the storage device 110, for example, a central processing unit (CPU), an application processor (AP), or other programmable general-purpose or special-purpose microprocessors, a digital signal processor (DSP), an image signal processor (ISP), a graphics processing unit (GPU) or other similar devices, integrated circuits or a combination thereof. The processor 120 can access and execute the code and software modules recorded in the storage device 110, so as to implement the 3D format image detection method in the embodiment of the disclosure.
Generally, for the user to experience the 3D visual effect, the user's left eye and right eye need to view image content (i.e., the left-eye image and the right-eye image) corresponding to different viewing angles, respectively. The left-eye image and the right-eye image may be combined into a 3D format image to display the 3D format image through different 3D display technologies, allowing the viewer's left eye to view the left-eye image and the viewer's right eye to view the right-eye image. In the embodiment of the disclosure, the electronic apparatus 10 can determine whether the input image is a 3D format image conforming to the 3D image format. Thus, in some embodiments, the 3D display 20 may support multiple display modes, such as 2D display modes and 3D display modes associated with one or more 3D display technologies. If the electronic apparatus 10 can accurately determine which 3D format the input image is, the 3D display 20 can automatically switch to a suitable display mode to display the 3D format image content.
In step S210, the processor 120 divides the input image according to the 3D image format to obtain the first image and the second image. In some embodiments, the input image may be a single frame of image in a video stream or a video. In some embodiments, the input image may be an image captured with a screen capture function. In some embodiments, the input image may be, for example, an image generated by an application. In some embodiments, the size of the first image is the same as the size of the second image size. That is, a 3D format image may be split into two images of the same resolution.
In some embodiments, the 3D image format may include a side-by-side (SBS) format, a top and bottom (TB) format, a checkerboard format, or an interlacing format. The processor 120 may extract the first image and the second image from the 3D format image according to the type of the 3D image format. For example,
Referring to the embodiment shown in
Referring to the embodiment shown in
Referring to the embodiment shown in
Referring to the embodiment shown in
Next, in step S220, the processor 120 performs a 3D matching processing on the first image and the second image to generate a disparity map of the first image and the second image. In some embodiments, the processor 120 may perform the 3D matching processing on the first image and the second image according to a block-matching algorithm to estimate disparity information and obtain a disparity map. In some embodiments, the processor 120 may perform 3D matching processing on the first image and the second image according to an optical flow algorithm to estimate disparity information and obtain a disparity map. In some embodiments, the processor 120 may input the first image and the second image to a trained deep neural network model to obtain a disparity map. In some embodiments, the number of elements in the disparity map is equal to the resolution of the first image and the second image. For example, assuming that the resolutions of the first image and the second image are 640*480, the disparity map may include disparity information corresponding to 640*480 pixel positions.
In step S230, the processor 120 calculates the matching number between the multiple first pixels in the first image and the multiple second pixels in the second image according to the disparity map. In some embodiments, the disparity map includes multiple valid disparity values and multiple invalid disparity values, and the matching number is the number of valid disparity values.
Specifically, in the 3D matching processing, if a certain first pixel in the first image can be successfully matched with a certain second pixel in the second image, the processor 120 can obtain a corresponding effective disparity value. Conversely, if a certain first pixel in the first image cannot be successfully matched to any second pixel in the second image, the processor 120 may obtain a corresponding invalid disparity value. Therefore, by counting the number of valid disparity values in the disparity map, the matching number of the multiple first pixels in the first image that are successfully matched to the multiple second pixels in the second image can be obtained. In some embodiments, the invalid disparity value in the disparity map may be set to a negative value, and the valid disparity value in the disparity map may be set to an integer value greater than or equal to 0, but the disclosure is not limited thereto.
In step S240, the processor 120 determines whether the input image is a 3D format image conforming to the 3D image format according to the matching number. Accordingly, if the matching number is large enough, it can be determined that the first image and the second image are the left-eye image and the right-eye image corresponding to the same shooting scene, so the processor 120 can determine that the input image is a 3D format image conforming to the 3D image format.
More specifically,
In some embodiments, the processor 120 may compare the matching number with a preset threshold to determine whether the matching number meets the matching condition. If the matching number is greater than the preset threshold, the processor 120 may determine that the matching number meets the matching condition. If the matching number is not greater than the preset threshold, the processor 120 may determine that the matching number does not meet the matching condition. The preset threshold can be set according to the image resolution of the input image. That is, different image resolutions correspond to different preset thresholds.
In some embodiments, the processor 120 may calculate the matching ratio of the matching number to the number of the pixels of the first image and determine whether the matching ratio is greater than a threshold. That is, the matching ratio is a ratio of the successfully matched first pixels to all the first pixels in the first image, which can be represented by a percentage or a value less than 1 and greater than 0. If the matching ratio is greater than the threshold, the processor 120 may determine that the matching number meets the matching condition. If the matching ratio is not greater than the threshold, the processor 120 may determine that the matching number does not meet the matching condition. In the embodiment of comparing the matching ratio with the threshold, the same threshold can be applied to different image resolutions.
If it is determined “yes” in step S241, in sub-step S242, in response to the matching number meeting the matching condition, the processor 120 determines that the input image is a 3D format image conforming to the 3D image format. Conversely, if it is determined “no” in step S241, in sub-step S243, in response to the matching number not meeting the matching condition, the processor 120 determines that the input image is not a 3D format image conforming to the 3D image format. That is, if the matching number meets the matching condition, it means that the first image and the second image captured from the input image are the left-eye image and the right-eye image corresponding to the same scene, so it can be determined that the input image is a 3D format image.
In step S502, the processor 120 divides the input image IMG1 according to the 3D image format to obtain the first image IMG_L and the second image IMG_R. In step S504, the processor 120 performs a 3D matching processing on the first image IMG_L and the second image IMG_R to generate a disparity map D_map of the first image IMG_L and the second image IMG_R.
In detail,
Next, the processor 120 calculates multiple similarities between the first image block B1 and the multiple second image blocks on the second image IMG_R. In some embodiments, these similarities may also be matching costs or values generated based on matching costs. For example, the processor 120 may sequentially calculate the absolute difference between the grayscale value of each first pixel on the first image block B1 and the grayscale value of the corresponding second pixel on the second image block B2_1, and the reciprocal is taken after summing up all the absolute differences to obtain the similarity between the first image block B1 and the second image block B2_1. Assuming that the size of the first image block B1 is 91*91, the processor 120 can obtain 91*91 absolute difference values.
However, in other embodiments, the processor 120 may also be based on other calculation methods, such as the square difference (SD) algorithm, the pixel dissimilarity measure (PDM) algorithm, the normalized cross correlation (NCC) algorithm, and the like to obtain matching costs corresponding to the multiple second image blocks. In some embodiments, the processor 120 may further perform cost aggregation to obtain matching costs corresponding to the multiple second image blocks.
By repeating the steps of similarity calculation along the horizontal scan line SL1, the processor 120 can obtain the similarities corresponding to the multiple second image blocks respectively, that is, the processor 120 sequentially calculates the similarity between the first image and each of the second image blocks, and the similarity corresponding to each of the multiple second image blocks is obtained. Therefore, the processor 120 may obtain a valid disparity value or an invalid disparity value corresponding to the first target pixel point P1 on the disparity map D_map according to the similarities corresponding to the multiple second image blocks on the horizontal scan line SL1, respectively.
Specifically, the processor 120 can determine whether the first image block B1 matches one of the multiple second image blocks according to the similarities corresponding to the multiple second image blocks on the horizontal scan line SL1, respectively. In the example of
As shown in
Referring back to
Note that the processing procedure of the 3D format image detection method executed by at least one processor is not limited to the examples of the foregoing embodiments. For example, some of the steps (processing) may be omitted, and the steps may be performed in other order. In addition, any two or more of the steps may be combined, and some of the steps may be corrected or deleted. Alternatively, other steps may also be performed in addition to the steps.
In summary, in the embodiments of the disclosure, whether the input image is a 3D format image conforming to various different 3D image formats can be effectively identified, and the user experience and the application range of the 3D display technology can be improved. For example, after determining that the input image is a 3D format image, the 3D display can automatically switch to an appropriate image playback mode, thereby improving the user experience. Alternatively, after determining that the input image is a 3D format image, the 3D display can acquire the blocks occupied by the left-eye image and the right-eye image in the input image, so as to facilitate image processing required for the subsequent 3D display.
Although the disclosure has been described with reference to the above embodiments, they are not intended to limit the disclosure. It will be apparent to one of ordinary skill in the art that modifications and changes to the described embodiments may be made without departing from the spirit and the scope of the disclosure. Accordingly, the scope of the disclosure will be defined by the attached claims and their equivalents and not by the above detailed descriptions.
Number | Date | Country | Kind |
---|---|---|---|
111129017 | Aug 2022 | TW | national |