Patent Grant
11800053
The present application is a 35 U.S.C. 371 national phase application of International Patent Application No. PCT/CN2020/093530 filed on May 29, 2020, the entire disclosure of which is incorporated by reference herein.
The present disclosure relates to the field of information display technology and, specifically, to a method, device, computer readable storage medium, and electronic device for video frame interpolation.
Video frame interpolation is a research area in the field of digital image processing and computer vision where the frame rate of video can be improved using video frame interpolation. The existing video frame interpolation method consists of several sub-modules in parallel, where the accuracy rate of each module is low, resulting that the final interpolation result will be affected by the accuracy rate of each module and the final synthesis module, and that the obtained image containing an interpolation frame has low accuracy.
According to an aspect of the present disclosure, there is provided a video frame interpolation method, comprising:
In an exemplary embodiment of the present disclosure, the iterative residual optical flow estimation process performed on the two input frames to obtain the two initial optical flow maps, comprises:
In an exemplary embodiment of the present disclosure, the updating of the input and the output of the (N+1)th optical flow estimation process using the final output of the Nth optical flow estimation process, comprises:
In an exemplary embodiment of the present disclosure, the two input frames comprises a first input frame and a second input frame, the two initial optical flow maps comprises a first initial optical flow map and a second initial optical flow map, the two target optical flow maps comprises a first target optical flow map and a second target optical flow map, the first input frame corresponds to the first initial optical flow map, and the second input frame correspond to the second initial optical flow map, wherein the processing the two initial optical flow maps using a pixel adaptive convolution joint up-sampling process based on the two input frames to obtain the target optical flow maps, comprises:
In one exemplary embodiment of the present disclosure, the video frame interpolation method comprises:
In an exemplary embodiment of the present disclosure, the training of the initial depth estimation model using the error loss between the reference virtual surface normal generated by the real depth-of-field map of the two input frames and the target virtual surface normal generated by the target depth-of-field map to obtain the target depth estimation model, includes:
In an exemplary embodiment of the present disclosure, the obtaining of the output frame using a pixel adaptive convolution frame synthesis method based on the target optical flow maps, the depth maps, the context feature maps, and the interpolation frame kernel, comprises:
In an exemplary embodiment of the present disclosure, the using the pixel adaptive convolution synthesis method to synthesize the two projected optical flow maps, the interpolation frame kernel, the two deformed depth maps, the two deformed input frames, and the two deformed context feature maps so as to obtain the output frame, comprises:
In an exemplary embodiment of the present disclosure, the frame synthesis process containing the pixel adaptive convolution performed on the synthesized input image to obtain the output frame comprises:
In an exemplary embodiment of the present disclosure, the determination of the projected optical flow maps based on the two target optical flow maps and the two depth maps, and the obtaining of the interpolation frame kernel, the two deformed depth maps, the two deformed input frames, and the two deformed context feature maps, comprise:
In an exemplary embodiment of the present disclosure, the obtaining of the interpolation frame for the two input frames based on the output frame, comprises:
In an exemplary embodiment of the present disclosure, the updating of the output frame with the average deformed frame, comprises:
According to an aspect of the present disclosure, there is provided a video frame interpolation device, comprising:
According to an aspect of the present disclosure, there is provided a computer readable storage medium having a computer program stored thereon, wherein the computer program is configured, when executed by a processor, to implement the video frame interpolation method as described in any one of the foregoing.
According to an aspect of the present disclosure, there is provided an electronic device, comprising.
The accompanying drawings herein are incorporated into and form part of the present specification, illustrate embodiments consistent with the present disclosure, and are used in conjunction with the present specification to explain the principles of the present disclosure. It will be apparent that the accompanying drawings in the following description are only some embodiments of the present disclosure, and that other drawings may be obtained from them without creative effort by one of ordinary skill in the art. In the accompanying drawings:
The example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein. Instead, the provision of these embodiments makes the present disclosure more comprehensive and complete, and communicates the idea of the example embodiments to those skilled in the art in a comprehensive manner. The features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments.
In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. Identical appended markings in the drawings indicate identical or similar parts, and thus repetitive descriptions of them will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different network and/or processor devices and/or microcontroller devices.
In an exemplary embodiment, a video frame interpolation method is first provided, and with reference to that shown in
S110, obtaining two input frames and obtaining, based on the two input frames, two initial optical flow maps corresponding to the two input frames.
S120, performing an up-sampling process on the two initial optical flow maps to obtain two target optical flow maps.
S130, obtaining an interpolation frame kernel, two depth maps corresponding to the two input frames respectively, and two context feature maps corresponding to the two input frames respectively based on the two input frames.
S140, obtaining an output frame using a frame synthesis method based on the two target optical flow maps, the two depth maps, the two context feature maps, and the interpolation frame kernel.
At least one of the following conditions applies:
It is important to note that the interpolation frame herein refers to an image frame that can be interpolated between two input frames, which reduces video motion blur and improves video quality.
According to the video frame interpolation method provided in an exemplary embodiment of the present disclosure, as compared with the prior art, on the one hand, the initial optical flow maps are obtained by performing a motion estimation process on two adjacent input frames through iterative residual refined optical flow prediction, so as to initially improve the accuracy of the interpolation result; and on the other hand, the target optical flow maps are obtained by performing a pixel adaptive convolutional joint up-sampling process on the initial optical flow maps based on the input frames, so as to further improve the accuracy of the interpolation result. Besides, a depth estimation method with geometrical constraints from virtual surface normals is used for depth prediction, which depth prediction is further taken into consideration for projection of the target optical flow maps. In the synthesis module, the pixel adaptive convolution is used to enhance the interpolation results, which can well improve the quality of the interpolation results, leading to the obtained interpolation results of high accuracy. This renders it applicable to video enhancement and upgraded slow motion effects of video post-processing, leading to a variety of application scenarios of the video frame interpolation method.
In the following, the various steps of the video frame interpolation method in an exemplary embodiment of the present disclosure will be described in more detail in conjunction with the accompanying drawings and examples.
In step S110, two input frames are obtained, and two initial optical flow maps corresponding to the two input frames are obtained based on the two input frames.
In an example embodiment of the present disclosure, the two input frames obtained can be a first input frame and a second input frame. Then, a first initial optical flow map and a second initial optical flow map are obtained by performing an optical flow estimation process on the first input frame and the second input frame, where the first initial optical flow map can correspond to the first input frame and the second initial optical flow map can correspond to the second input frame.
In an example implementation, the optical flow estimation process for the above two input frames can be performed by using a new model obtained by training with PWC-Net (CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume). The optical flow estimation process can also be performed by other models for the above two input frames, without any specific limitation in the present example implementation.
In an example embodiment of the present disclosure, the first initial optical flow map and the second initial optical flow map can be obtained by performing an optical flow estimation process only once on the first input frame and the second input frame as described above.
In another example implementation of the present disclosure, with reference to
Specifically, updating of the input and the output of the (N+1)th optical flow estimation with the final output of the Nth optical flow estimation, comprises: the two final outputs of the Nth optical flow estimation and the two inputs of the first optical flow estimation can be summed up respectively so as to obtain the input of the (N+1)th optical flow estimation; the final output of the Nth optical flow estimation and the initial output of the (N+1)th optical flow estimation can be summed up so as to obtain the final output of the (N+1)th optical flow estimation, wherein the final output of the last optical flow estimation can be used as the initial optical flow map.
In an example implementation, taking the value of N being 1 for a detailed explanation, wherein the server can feed the output 230 of the first optical flow estimation to the input 210 of the second optical flow estimation. I.e., the input of the second optical flow estimation can be obtained by summing up the two outputs of the first optical flow estimation with the first input frame and the second input respectively. I.e., the pixel values of the two outputs of the first optical flow estimation are summed up with the pixel values of the first input frame and the second input frame respectively, so as to obtain the input of the second optical flow estimation. After the second optical flow estimation, the first output of the first optical flow estimation is used to update the second initial output of the second optical flow estimation. The second target output is obtained by summing up the pixel values of the first output and the second initial output so as to obtain the second target output, where the second initial output is obtained after optical flow estimation for the input of the second optical flow estimation.
In step S120, two target optical flow maps are obtained by performing an up-sampling process on the two initial optical flow maps.
In a first example implementation of the present disclosure, the server can directly perform feature extraction on each of the two initial optical flow maps, and perform an up-sampling process at least once to obtain the target optical flow map after the feature extraction.
The operation of the convolution layer in the up-sampling process in an example implementation is shown below:
wherein, i can represent the pixel i, vl can be represented as the feature map of the lth layer in the convolutional neural network, Ω(i) can be expressed as the convolutional window around the pixel i, Wl can be used to represent the convolution kernel of the lth layer in the convolutional neural network, pi can be used to represent the pixel coordinate, and bl can be used to represent the bias term of the lth layer in the convolutional neural network.
In another example implementation of the present disclosure, referring to
Specifically, the server can perform feature extraction on the first initial optical flow map and the second initial optical flow map to obtain the first reference optical flow map and the second reference optical flow map respectively, and perform feature extraction on the first input frame and the second input frame to obtain the first reference input map and the second reference input map respectively. After that, the pixel adaptive convolutional joint up-sampling process can be performed at least once on the first reference optical flow map by using the first reference input map as a guide map, and then the feature extraction process is performed to obtain the first target optical flow map. Further, the pixel adaptive convolutional joint up-sampling process can be performed at least once on the second reference optical flow map by using the second reference input map as a guide map, and then the feature extraction process is performed to obtain the second target optical flow map.
In an example implementation, as shown in
In an example implementation, during each time of the pixel adaptive convolutional joint up-sampling process 330 on the above reference optical flow map, it is required to use the above reference input map as a guide map. I.e., constraints are imposed on the above pixel adaptive convolutional up-sampling process. After the pixel adaptive convolutional joint up-sampling process 330 is performed multiple times on the above reference optical flow map, a feature extraction process can be performed on the output result by using the convolutional layer 320 to obtain the target optical flow map 340. This helps to improve the accuracy of the initial optical flow map 310 and to complete the optimization of the initial optical flow map 310.
Specifically, the pixel adaptive convolution is based on an ordinary convolution, multiplying the ordinary convolution by an adaptive kernel function K obtained by a guide feature map f I.e., the convolution operation in the pixel adaptive convolution up-sampling process is as follows:
wherein, i can represent the pixel i, vl can be represented as the feature map of the lth layer in the convolutional neural network, Ω(i) can be expressed as the convolutional window around the pixel i, Wl can be used to represent the convolution kernel of the lth layer in the convolutional neural network, pi can be used to represent the pixel coordinate, and bl can be used to represent the bias term of the lth layer in the convolutional neural network. fi and fj can present the guide feature map. Specifically, the pixel j is a pixel point within a predetermined distance centered on pixel i, where the predetermined distance can be customized as needed and is not specifically limited in the present example implementation.
In an example implementation, the resolution of the initial optical flow map obtained after the above optical flow estimation is one-fourth of the input frame. Thus, in the present example implementation, the pixel adaptive convolutional joint up-sampling process can be performed twice with the sampling multiplicity being 2. Alternatively, the pixel adaptive convolutional joint up-sampling process can be performed once with the sampling multiplicity being 4. There is no specific limitation in the present example implementation. A reference optical flow map can be introduced as a guide map in the pixel adaptive joint up-sampling process, which in turn improves the up-sampling accuracy.
In step S130, an interpolation frame kernel, two depth maps corresponding to the two input frames respectively, and two context feature maps corresponding to the two input frames respectively are obtained based on the two input frames.
In an example implementation of the present disclosure, the depth maps can be obtained using the initial depth estimation module. The interpolation frame kernel as well as the first context feature map and the second context feature map are obtained based on the first input frame and the second input frame.
In an example implementation, a pre-trained model can be used to complete the spatio-temporal context feature extraction of the two input frames, and the feature map of any layer in the model can be used as the two context feature maps obtained. The above-mentioned training model can be a VGG model or a residual network, which is not specifically limited in the present example implementation.
In another example implementation of the present disclosure, the initial depth estimation model can be trained first to obtain the target depth estimation model. Then, the depth estimation model can be used to calculate the first depth map and the second depth map corresponding to the first input frame and the second input frame respectively.
In an example implementation, a pre-trained model of the monocular depth model MegaDepth may be used as the initial depth estimation model described above. Alternatively, other pre-trained models may be used as the initial depth estimation model, without any specific limitation in the present example implementation.
Specifically, the method for training the initial depth estimation model includes: firstly obtaining the real depth-of-field map of the two input frames, and a three-dimensional (3D) point cloud computation is performed on the real depth-of-field map. Specifically, the two-dimensional depth-of-field map is transformed into a three-dimensional map to obtain a relatively simple three-dimensional (3D) point cloud; then the reference virtual surface normal can be generated according to the 3D point cloud; and then referring to
In an example implementation, the first input frame and the second input frame can be input into the target depth estimation model respectively, so as to obtain the first depth map and the second depth map.
In step S140, the projected optical flow map is determined based on the target optical flow maps and the depth maps, and the interpolation frame kernel, the deformed depth map, the deformed input frame, and the deformed context feature map are obtained.
In an example embodiment of the present disclosure, referring to
Among them, the description related to the optical flow estimation 521 has been provided in detail above with reference to
In an example implementation, the first projection optical flow map can be obtained by depth-aware optical flow projection of the first target optical flow map using the first depth map, and the second projection optical flow map can be obtained by depth-aware optical flow projection of the second target optical flow using the second depth map.
Specifically, the time of the first input frame mentioned above can be defined as the time moment 0, the time of the second input frame can be defined as the time moment 1, and a time moment t can be defined which lies between the first and second time moments. The projected optical flow map mentioned above can be calculated by the following equation:
wherein, F0→1(y) denotes the optical flow of the pixel point y from the first input frame to the second input frame; D0(y) represents the depth value of the pixel point y; y∈S(x) denotes the optical flow F0→1(y) of pixel point y, if the optical flow F_(0→1) (y) passes the pixel point x at the time moment t, then F0→1 (x) can be approximated as −t F (0→1) (y); F0→1 (x) denotes the optical flow of the pixel point x from the time moment t to the first input frame.
In an example implementation, the server can obtain two context feature maps by spatio-temporal context feature extraction 523 of the above two input frames 510 respectively, perform interpolation frame kernel estimation 524 on the two input frames to obtain the interpolation frame kernel, and use the interpolation frame kernel to perform adaptive deformation 550 on the above two input frames, two depth maps, and two context feature maps, so as to get two deformed input frames, two deformed depth maps, and two deformed context feature maps.
In an example implementation, the depth estimation can be done using an hourglass model, the context feature extraction is done using a pre-trained ResNet neural network, while the kernel estimation and the adaptive deformation layer are based on the U-Net neural network, which are not specifically limited in the present example implementation.
In an example implementation, a deep learning classical backbone network can be used to generate the interpolation frame kernel at each pixel location based on the two input frames, and at the adaptive deformation layer, the two depth maps, the two input frames, and the two context feature maps are deformed to obtain two deformed input frames, two deformed depth maps, and two deformed context feature maps based on the interpolation frame kernel and the projected optical flow map.
In an example embodiment of the present disclosure, referring to
In an example implementation, referring to
Specifically, the convolution layer in the first residual module can be
wherein, i can represent the pixel i, vl can be represented as the feature map of the lth layer in the convolutional neural network, Ω(i) can be expressed as the convolutional window around the pixel i, Wl can be used to represent the convolution kernel of the lth layer in the convolutional neural network, pi and pj can be used to represent the pixel coordinates, and bl can be used to represent the bias term of the lth layer in the convolutional neural network.
The pixel adaptive convolution layer is used to replace the above convolution layer so as to obtain the second residual module, and the pixel adaptive convolution layer is
wherein, i can represent the pixel i, vl can be represented as the feature map of the lth layer in the convolutional neural network, Ω(i) can be expressed as the convolutional window around the pixel i, Wl can be used to represent the convolution kernel of the lth layer in the convolutional neural network, pi and pj can be used to represent the pixel coordinates, and bl can be used to represent the bias term of the lth layer in the convolutional neural network. fi and fj can be used to represent the guide feature maps Specifically, the pixel j is a pixel point within a predetermined distance centered on pixel i, wherein the predetermined distance can be customized as desired and is not specifically limited in the present example implementation.
The pixel adaptive convolution layer is based on the normal convolution layer multiplied by an adaptive kernel function K obtained from the guide feature map f.
In an example implementation, the feature image output by the first residual module 630 is used as a guide map for the second residual module. I.e., new constraints are added on the pixel adaptive convolution layer in the pixel adaptive residual block based on the feature image, so as to acquire output frames of higher accuracy.
In an example implementation, the number of residual blocks in the residual network may be multiple, such as 2, 3, 4, or more, which is not specifically limited in the present example implementation.
In an example implementation of the present disclosure, again referring to
Specifically, the average deformed frame can be calculated by summing up the pixel values of the two deformed input frames and calculating the average value of them. The new output frame 590 is obtained by summing up the average deformed frame and the output frame 650, i.e., summing up the pixel values of the average deformed frame and the output frame 650 to obtain the new output frame 590.
The following describes embodiments of devices of the present disclosure that can be used to perform the video frame interpolation method described above in the present disclosure. In addition, in an exemplary embodiment of the present disclosure, a video frame interpolation device is provided. Referring to
The motion estimation processor 710 can be used to obtain two input frames and obtain, based on the two input frames, two initial optical flow maps corresponding to the two input frames. The data optimization processor 720 can be used to perform an up-sampling process on the two initial optical flow maps to obtain two target optical flow maps. The depth estimation processor 730 can be used to obtain an interpolation frame kernel, two depth maps corresponding to the two input frames respectively, and two context feature maps corresponding to the two input frames respectively based on the two input frames. Further, the image synthesis processor 740 can be used to obtain an output frame using a frame synthesis method based on the two target optical flow maps, the two depth maps, the two context feature maps, and the interpolation frame kernel.
Since each functional processor of the video frame interpolation device in an example embodiment of the present disclosure corresponds to steps in an example embodiment of the video frame interpolation method described above, for details not disclosed in an embodiment of the device of the present disclosure, please refer to the embodiment of the video interpolation method described above of the present disclosure.
It should be noted that although a number of modules or units of the device for action execution are mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of the present disclosure, features and functions of two or more modules or units described above may be specified in a single module or unit. Conversely, features and functions of one module or unit described above may be further divided to be specified by a plurality of modules or units.
In addition, in an exemplary embodiment of the present disclosure, there is provided an electronic device capable of implementing the above video frame interpolation method.
Those skilled in the art can appreciate that aspects of the present disclosure can be implemented as systems, methods, or program products. Accordingly, aspects of the present disclosure may be specifically implemented in the form of a fully hardware embodiment, a fully software embodiment (including firmware, microcode, etc.), or a combination of hardware and software aspects, which may be collectively referred to herein as a “circuit,” “module” or “system”.
The electronic device 800 according to embodiments of the present disclosure is described below with reference to
As shown in
The memory unit stores program code, which program code may be executed by the processing unit 810 such that the processing unit 810 performs steps according to various exemplary embodiments of the present disclosure described in the “Exemplary Methods” section above of the present specification. For example, the processing unit 810 may perform the steps shown in
As another example, the described electronic device can implement the various steps as shown in
The memory unit 820 may include readable media in the form of volatile memory units, such as random access memory units (RAM) 821 and/or cache memory units 822. Further, the memory unit 820 may include read-only memory units (ROM) 823.
Storage unit 820 may also include a program/utility 824 having a set of (at least one) program modules 825, such program modules 825 including, but not limited to: an operating system, one or more applications, other program modules, and program data, wherein each of these examples or some combination may include an implementation of a network environment.
Bus 830 may be one or more of several types of bus structures, including a memory cell bus or a memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus that using any bus structure among a plurality of bus structures.
The electronic device 800 may also communicate with one or more external devices 870 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), with one or more devices that enable the user to interact with the electronic device 800, and/or with any device that enables the electronic device 800 to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication may be via input/output (I/O) interface 850. And, the electronic device 800 may also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and/or public networks, such as the Internet) through a network adapter 860. As shown, the network adapter 860 communicates with other modules of the electronic device 800 via the bus 830. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device 800, including but not limited to: microcode, device drives, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
With the above description of embodiments, it is readily understood by those skilled in the art that the example embodiments described herein can be implemented by means of software or by means of software in combination with some necessary hardware. Thus, a technical solution according to an embodiment of the present disclosure may be embodied in the form of a software product that may be stored in a non-volatile storage medium (which may be a CD-ROM, USB flash drive, removable hard drive, etc.) or on a network, comprising a number of instructions to cause a computing device (which may be a personal computer, server, terminal device, or network device, etc.) to perform a method according to an embodiment of the present disclosure.
In an exemplary embodiment of the present disclosure, there is also provided a computer readable storage medium having stored thereon a program product capable of implementing the method described above in the present specification. In some possible embodiments, aspects of the present disclosure may also be implemented in the form of a program product comprising program code that, when the program product is run on a terminal device, is used to cause the terminal device to perform the steps described in the “Exemplary Methods” section above of the present specification according to various exemplary embodiments of the present disclosure.
Referring to
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic memory devices, or any suitable combination of the above.
The computer readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave that carries readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals or any suitable combination of the above. The readable signal medium may also be any readable medium other than a readable storage medium that sends, propagates, or transmits a program for use by or in conjunction with an instruction execution system, device, or device.
The program code contained on the readable media may be transmitted by any suitable medium, including but not limited to wireless, wired, fiber optic, RF, etc., or any suitable combination of the above.
Program code for performing operations of the present disclosure may be written in any combination of one or more programming languages, the programming languages including object-oriented programming languages—such as Java, C++, etc., and also including conventional procedural programming languages—such as “C” language or similar programming languages. The program code may be executed entirely on a user computing device, partially on a user device, as a stand-alone package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case involving a remote computing device, the remote computing device may be connected to the user computing device via any kind of network, including a local area network (LAN) or a wide area network (WAN), or, alternatively, may be connected to an external computing device (e.g., using an Internet service provider to connect via the Internet).
Further, the above appended drawings are merely schematic illustrations of the processing included in the method according to exemplary embodiments of the present disclosure, and are not intended to be limiting. It is readily understood that the processing shown above in the accompanying drawings does not indicate or limit the temporal order of such processing. It is also readily understood that these processes may be performed, for example, in multiple modules, either synchronously or asynchronously.
Other embodiments of the present disclosure will readily come to the mind of one skilled in the art upon consideration of the present specification and practice of the present invention disclosed herein. The present disclosure is intended to cover any variation, use, or adaptation of the present disclosure that follows the general principle of the present disclosure, and includes commonly known or customary technical means in the art that are not disclosed herein. The present specification and embodiments are to be considered exemplary only, and the true scope and spirit of the present disclosure is indicated by the claims.
It is to be understood that the present disclosure is not limited to the precise construction already described above and illustrated in the accompanying drawings, and that various modifications and changes can be made without departing from its scope. The scope of the present disclosure is limited only by the appended claims.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/CN2020/093530 | 5/29/2020 | WO |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2021/237743 | 12/2/2021 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 10776688 | Jiang et al. | Sep 2020 | B2 |
| 20100129048 | Pitts | May 2010 | A1 |
| 20170278289 | Marino | Sep 2017 | A1 |
| 20170353670 | Zimmer | Dec 2017 | A1 |
| 20190138889 | Jiang et al. | May 2019 | A1 |
| 20200394752 | Liu et al. | Dec 2020 | A1 |
| Number | Date | Country |
|---|---|---|
| 109145922 | Jan 2019 | CN |
| 109151474 | Jan 2019 | CN |
| 110351511 | Oct 2019 | CN |
| 110392282 | Oct 2019 | CN |
| 110738697 | Jan 2020 | CN |
| 2019168765 | Sep 2019 | WO |
| Entry |
|---|
| Written Opinion of International Application No. PCT/CN2020/093530 dated Feb. 25, 2021. |
| Bao, Wenbo et al. “Depth-Aware Video Frame Interpolation.” 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (Apr. 2019): 3698-3707. |
| Number | Date | Country | |
|---|---|---|---|
| 20220201242 A1 | Jun 2022 | US |
