Patent Application
20250166213
The present application claims priority to Chinese Patent Application No. 202210130360.8 filed in China National Intellectual Property Administration on Feb. 11, 2022, and the entire contents of the Chinese Patent application are incorporated into the present application by reference.
Embodiments of the present disclosure relate to the field of motion simulation technology, for example, to an image display method and apparatus, an electronic device, and a storage medium.
With the continuous development of computer technology, dynamic effects provided by an application to users are becoming more and more abundant, and the interest of generated videos is also constantly increasing. For example, a user may add an effect to a video according to his/her preferences during a process of capturing a video.
However, in the scheme provided by the related technology, when a user moves a terminal device, a picture in a video changes, and in this case, elements in a display interface cannot respond to the change, that is, an object in the display interface may not match a presentation effect in an actual scene during the movement of the terminal device, and therefore, there is a problem that the picture presented in the display interface is poor in texture, thereby affecting the user's usage experience.
The present disclosure provides an image display method and apparatus, an electronic device, and a storage medium, to achieve a motion simulation of an object within an image of a video when the image of the video changes, to reduce a sense of separation between the object within the image and a real world, and to enhance the user's use experience.
In a first aspect, an embodiment of the present disclosure provides an image display method, comprising:
In a second aspect, an embodiment of the present disclosure further provides an image display apparatus, comprising:
In a third aspect, an embodiment of the present disclosure further provides an electronic device, and the electronic device comprises:
In a fourth aspect, an embodiment of the present disclosure further provides a storage medium, comprising computer-executable instructions, the computer-executable instructions, when executed by a computer processor, are used for performing the image display method according to any one of the embodiments of the present disclosure.
Throughout the accompanying drawings, the same or similar reference numerals indicate the same or similar elements. It should be understood that the accompanying drawings are schematic and that components and elements are not necessarily drawn to scale.
It should be understood that the plurality of steps recorded in the implementation modes of the methods of the present disclosure can be performed according to different orders and/or performed in parallel. In addition, the implementation modes of the methods can include additional steps and/or omit performing the steps shown. The scope of the present disclosure is not limited in this aspect.
The term “comprise/include” and variations thereof used in this article are open-ended inclusion, namely “comprising/including but not limited to”. The term “based on” refers to “at least partially based on”. The term “one embodiment” means “at least one embodiment”; the term “another embodiment” means “at least one other embodiment”; and the term “some embodiments” means “at least some embodiments”. Relevant definitions of other terms may be given in the description hereinafter.
It should be noted that the concepts, such as “first” and “second”, mentioned in the present disclosure are only used to distinguish different apparatuses, modules, or units, and are not intended to limit orders or interdependence relationships of functions performed by these apparatuses, modules, or units. It should be noted that the modifications of “one” and “more/plurality” mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise explicitly stated in the context, it should be understood as “at least one”.
The names of messages or information interacted between a plurality of apparatuses in the implementations of the present disclosure are used for illustrative purposes only and are not used to limit the scope of such messages or information.
Before introducing the present technical scheme, an application scenario of the embodiments of the present disclosure may be exemplified first. Exemplarily, when the user captures a video through an application software or makes a video call with another user, some effects (e.g., a three-dimensional (3D) object model in a floating state) may be added to the captured video. When the user moves the terminal device and causes the camera angle of view to change, the captured video picture is also changed. In this case, if the object in the video picture remains still, the user may have a feeling of separation between the object and the real world. According to the technical solution of the present embodiment, the application can simulate the movement of the object within the image during the process of the video image changing, thereby causing the object to present a more realistic dynamic effect and improving the quality of the video image.
As shown in
S110, determining a current movement increment of a target reference point relative to a current to-be-processed video frame.
Here, the apparatus for executing the image display method provided by the embodiment of the present disclosure can be integrated in the application software supporting an effect video processing function, and the application software can be installed in an electronic device, for example, the electronic device can be a mobile terminal or a PC terminal, etc. The application software can be a kind of software for image/video processing, and the specific application software will not be described in detail here, as long as it can realize image/video processing. The application software can also be a specially developed application program, which is integrated in the software that adds the effects and displays the effects, or is integrated in the corresponding page, and the user can achieve to process the effect video through the integrated page in the PC terminal.
It should be noted that the technical solution of the present embodiment can be executed during the process of a user shooting a video, that is, when the user is shooting a video or making a video call with another user, an augmented reality (AR) effect can be added to a captured picture by applying a function provided by the application, and when the user controls a camera device to move, the special effect in the picture can be simulated to have an effect of inertial motion based on the technical solution of this embodiment.
In the embodiment, when the user causes the application to add a target effect (e.g., a 3D object in a floating state) in the video picture by touching a pre-developed effect add control, the application associates the added target effect with a pre-created mount point. The target effect may be any floatable model, for example, be a pre-created 2D or 3D model. This model can be used as the target object in the display interface. When the target object is displayed in the display interface in a floating state, a traction effect may also be simulated for the object to reflect the effect of the user adjusting the target effect. For example, the target object may be a balloon which is in a shape of love heart and is pre-set with certain material properties, and at the same time, when the balloon is displayed on the display interface, the balloon is also associated with a cartoon arm through a connection line, thereby creating the traction effect.
It can be understood that the mount point of the effect is the target reference point. The target reference point is pre-constructed based on a virtual rendering camera, which can be understood as pre-creating the target reference point at a certain position corresponding to the rendering camera and taking the target reference point as a “sub-object” of the rendering camera, and at the same time, keeping the positional relationship between the target reference point and the rendering camera relatively stationary at all times. On this basis, when the user moves the terminal device, a position of the virtual rendering camera in a three-dimensional space coordinate system changes, and accordingly, a coordinate of the target reference point in the three-dimensional space coordinate system also changes in order to keep the relative positional relationship between the target reference point and the rendering camera constant.
Illustratively, when the user moves the terminal device resulting in a change in the captured video picture, not only the coordinate of the virtual rendering camera in the three-dimensional space is changed, but also the coordinate of the target reference point is changed. Based on this, it can be understood that the floating 3D balloon associated with the target reference point in the video picture moves with the movement of the target reference point.
It should be noted that when the user moves the terminal device causing the video picture to change, the three-dimensional coordinates of the target reference point may vary in each video frame. Taking two adjacent video frames in a video as an example for description, if a video frame at a current time is used as a to-be-processed video frame, a distance between a position of the target reference point in the picture of this frame and a position of the target reference point in the picture of a previous frame is the current movement increment of the target reference point. It can be understood that the current movement increment is a three-dimensional vector, and the current movement increment is determined based on the position information of the target reference point in the previous to-be-processed video frame. For example, the target reference point associated with the 3D balloon in the above example is moved from a point A to a point B in the three-dimensional space in the displayed pictures of two adjacent video frames, and a vector (Δx, Δy, Δz) calculated based on the three-dimensional coordinates of the two points is the current movement increment of the target reference point, and the current movement increment can characterize at least the distance and direction when the target reference point moving from the point A to the point B.
In the present embodiment, in order to accurately get the current movement increment of the target reference point, first it is necessary to determine a relative position matrix of the target reference point and the rendering camera, so that according to the relative position matrix, the current movement increment of the target reference point with respect to the current to-be-processed video frame is determined.
It can be understood that the relative position moment between the target reference point and the rendering camera remains unchanged at all times, so that in the case that the coordinate of the rendering camera in the three-dimensional space has been determined, the coordinate of the target reference point in the three-dimensional space can be calculated according to the relative position matrix. The coordinate of the rendering camera within the three-dimensional space may be represented by the camera position information, and the process of determining the camera position information is described below.
For example, the camera position information of the rendering camera is determined based on a simultaneous localization and mapping algorithm; current position information of the target reference point is determined according to the relative position matrix and the camera position information; and the current movement increment is determined according to the current position information and historical position information to which a previous to-be-processed video frame corresponds.
Among them, the basic principle of the simultaneous localization and mapping (SLAM) algorithm in the field of AR technology is to achieve localization and reduction of localization error through multi-feature matching using the method of probability statistics, and mainly includes several parts such as feature extraction, data association, state estimation, state update, and feature update. It can thus be appreciated that the SLAM-based algorithm can construct a map with a more realistic visual effect, thereby rendering a superposition effect of the virtual object for the current perspective, so that the virtual object in the picture is more realistic without any sense of dissonance. Specifically, in the embodiment, the application may determine the coordinate of the rendering camera within the three-dimensional space coordinate system based on the SLAM algorithm regardless of the shooting angle of the camera device.
For example, after the camera position information of the rendering camera is determined, the current position information of the target reference point can be calculated based on the coordinate and the relative position matrix, that is, the coordinate values of the target reference point in the to-be-processed video frame at the current time in the three-dimensional space coordinate system are calculated.
In this embodiment, after the current position information of the target reference point is determined, the difference between the current coordinate of the target reference point and the corresponding coordinate value of the target reference point in the previous to-be-processed video frame is calculated, and this difference is used as the current movement increment of the target reference point. It is understood that the coordinate value corresponding to the target reference point in the previous to-be-processed video frame is historical position information of the target reference point.
Continuing to explain with the above example, when the user moves the terminal device to cause a change in the video picture, the camera position information, i.e., the coordinate value within the three-dimensional space coordinate system, of the rendering camera also changes. For example, when a to-be-processed video frame at the current moment is present in the video picture, the application may determine the camera position information of the rendering camera, i.e., the coordinate value of the rendering camera in the three-dimensional space coordinate system, at this moment based on the SLAM algorithm, and may calculate to obtain the coordinate (x2, y2, z2) of the target reference point in the three-dimensional space coordinate system after the movement of the target reference point according to the determined coordinate value and the relative position matrix. For example, by obtaining the coordinate value (x1, y1, z1) corresponding to the target reference point in the previous to-be-processed video frame and taking the difference between the above two coordinate values, the current movement increment (Δx, Δy, Δz) of the target reference point with respect to the current to-be-processed video frame during the process of changing the video shooting angle by the user can be obtained.
In the present embodiment, because the camera position information of the rendering camera, and the target current position information of the target reference point are both coordinate values within the three-dimensional space coordinate system, in the actual application process, the movement sub-increments in the coordinate axis directions can be determined according to the current space coordinate corresponding to the current position information and the historical space coordinate corresponding to the historical position information; and the movement sub-increments are taken as the current movement increment.
Exemplarily, in the to-be-processed video frame at the current moment, the current space coordinate of the target reference point associated with the 3D balloon is (x2, y2, z2), and in the adjacent to-be-processed video frame before the current moment, the historical space coordinate of the target reference point is (x1, y1, z1), based on this, the variations of the coordinate value of the target reference point in the directions of the three coordinate axes of the space coordinate system can be determined, for example, the variation of the target reference point in the x-axis is Δx=x2−x1, the variation in the y-axis is Δy=y2−y1, and the variation in the z-axis is Δz=x2−z1. It can be understood that the above three coordinate value variations are movement sub-increments of the target reference point in the plurality of coordinate axis directions, and each of the movement sub-increments can be taken as the current movement increment of the target reference point in the three-dimensional space coordinate system, and of course, the vector sum determined based on the above three movement sub-increments can also be taken as the current movement increment, and the embodiments of the present disclosure do not describe details herein again.
S120, determining a current cumulative increment of the target reference point based on the current movement increment and a to-be-called cumulative increment.
For the to-be-processed video frame at the current moment, there may be a plurality of to-be-processed video frames before this frame, it can be understood that when the user moves the terminal device to cause a change in the video picture, there is also a difference in the position of the target reference point in different to-be-processed video frames, so that in the embodiment, when the current movement increment of the target reference point relative to the current to-be-processed video frame is determined, in order for the target reference point to accurately simulate the effect of inertial motion, there is also a need to consider the to-be-called cumulative increment in a plurality of to-be-processed video frames before the current moment, and then a sum of the current movement increment of the target reference point and the to-be-called cumulative increment is calculated, and the calculation result is taken as the current cumulative increment of the target reference point.
The to-be-called cumulative increment is determined based on historical movement increments of historical to-be-processed video frames and corresponding historical target movement amounts, the historical to-be-processed video frames may be a plurality of video frames prior to the current moment during the process in which the user moves the terminal device resulting in a change in the video picture, it can be understood that the to-be-called cumulative increment is updated every time a picture of one to-be-processed video frame is displayed in the display interface in the process in which the target reference point moves in the three-dimensional space.
Exemplarily, during the process of changing the shooting angle, the camera device collects three to-be-processed video frames N1, N2, and N3 in total, and the to-be-processed video frame at the current moment is the video frame N3, and the two video frames N1, N2 before the current moment are historical to-be-processed video frames; meanwhile, the movement increments of the target reference point corresponding to the video frames N1, N2, and N3 are (0, 0, 0), (Δx1, Δy1, Δz1), and (Δx2, Δy2, Δz2), respectively, correspondingly, the movement amount of the target reference point in the display interface corresponding to the video frame Ni is (0, 0, 0) and the movement amount of the target reference point in the display interface corresponding to the video frame N2 is (a1, b1, c1), on this basis, when the picture of the video frame N2 is displayed in the display interface, the corresponding to-be-called cumulative increment (Δx1−a1, Δy1−b1, Δz1−c1) when the target reference point moves to the position in the picture of the video frame N2 is determined. For example, the current cumulative increment is the sum, i.e., (Δx1−a1+Δx2, Δy1−b1+Δy2, Δz1−c1+Δz2), of the to-be-called cumulative increment and the current movement increment corresponding to the to-be-processed video frame (i.e., video frame N3) at the current moment.
In the actual application process, because the target reference point moves a plurality of times, the to-be-called cumulative increment need to be constantly updated, and therefore, an increment cache pool for storing the to-be-called cumulative increment may be constructed in advance within a storage space of the device, and at the same time, because the to-be-called cumulative sub-increments in a plurality of coordinate axis directions are included in the to-be-called cumulative increment, the to-be-called cumulative increment determined each time may be decoupled based on different coordinate axes and stored. For example, when the current movement increment of the target reference point is determined, the to-be-called cumulative increment may be directly retrieved from the increment cache pool; a plurality of cumulative movement sub-increments of the target reference point in a plurality of coordinate axis directions are determined based on the to-be-called cumulative sub-increments and corresponding movement sub-increments; and the plurality of cumulative movement sub-increments are as the current cumulative increment.
Continuing with the above example for description, when the increment cache pool is constructed in advance, it is determined that the target reference point moves to a position in a second to-be-processed video frame, and the corresponding to-be-called cumulative increment is (Δx1−a1, Δy1−b1, Δz1−c1), the to-be-called cumulative increment is decoupled based on the x-axis, the y-axis, and the z-axis in the three-dimensional space coordinate system, thereby obtaining the to-be-called cumulative sub-increment Δx1−a1 in the x-axis direction, the to-be-called cumulative sub-increment Δy1−b1 in the y-axis direction, and the to-be-called cumulative sub-increment Δz1−c1 in the z-axis direction, and furthermore, the above three to-be-called cumulative sub-increments are stored to the increment cache pool. When the current movement increment (Δx2, Δy2, Δz2) of the target reference point in the to-be-processed video frame at the current moment is determined, the coordinate values respectively in the three directions of the x-axis, the y-axis, and the z-axis are respectively added to the corresponding three to-be-called cumulative increments in the incremental cache pool to obtain three cumulative movement sub-increments Δx1−a1+Δx2, Δy1−b1+Δy2, Δz1−c1+Δz2, it can be understood that the above three cumulative movement sub-increments can be taken as the current cumulative increment of the target reference point at the current moment.
S130, determining a target movement amount of the target reference point according to the current cumulative increment and a preset displacement function.
In the embodiment, when the current cumulative increment of the target reference point at the current moment is determined, the current cumulative increment can be inputted into the preset displacement function integrated into the application, thereby calculating the target movement amount of the target reference point. The target movement amount reflects the direction and distance in which the target reference point needs to move in the display interface corresponding to the application.
For example, the preset displacement function is determined based on a traction coefficient for towing the target object, a preset wind resistance coefficient, and a speed coefficient, the traction coefficient is a parameter determined in order to cause the object to present an effect of being pulled by the traction effect during the movement, the preset wind resistance coefficient is a parameter determined in order to cause the object to exhibit an effect of being resisted by the wind during the movement, and the speed coefficient is a parameter determined in order to make the object present the effect of moving based on an initial speed during the movement. In practice, the preset displacement function may be y=(math.abs(x)x)*(k1+k3*x*x)+k2*x, where y is the finally obtained target movement amount of the target reference point in the display interface, k1 is the traction coefficient, k2 is the preset wind resistance coefficient, k3 is the speed coefficient, and x is the current cumulative increment of the target reference point. For example, when it is determined that the current cumulative increment of the target reference point associated with the 3D balloon is (Δx1−a1+Δx2, Δy1−b1+Δy2, Δz1−c1+Δz2), the coordinate value may be input as x into the preset displacement function, thereby obtaining the target movement amount of the reference point in the display interface.
It should be understood by those skilled in the art that, in the actual application, the three coefficients k1, k2, and k3 may be set empirically, and a mapping table characterizing the correspondence between different objects and the corresponding three coefficients may also be established in advance, based on this, the three coefficients may be determined through looking up the table when the target movement amount of a certain object needs to be determined, and the embodiments of the present disclosure do not describe the details herein.
It should be noted that, in the process of determining the target movement amount of the target reference point, in order to prevent a situation in which only a slight change in the shooting angle causes the object in the display interface to generate an inertial motion that does not conform to the actual, it is also possible to set a threshold for the current cumulative increment in advance, such as taking 0.1 as the movement amount threshold for the cumulative movement sub-increment. For example, processing a cumulative movement sub-increment greater than a preset movement amount threshold based on the preset displacement function to obtain a first to-be-moved amount; taking a cumulative sub-increment not greater than the preset movement amount threshold as a second to-be-moved amount; and determining the target movement amount based on the first to-be-moved amount and the second to-be-moved amount.
The first to-be-moved amount/the second to-be-moved amount refer to a theoretical movement amount in the display interface determined by the application for the target reference point, not a target movement amount that determines how the application ultimately renders the target object on the display interface. It can be understood that when the preset movement amount threshold is compared with the size of the cumulative movement sub-increment, when the comparison result is different, the determined target movement value is also different.
Exemplarily, when the cumulative movement sub-increment of the target reference point in the x-axis direction is 0.05 and the preset movement amount threshold is 0.1, because the cumulative movement sub-increment is smaller than the preset movement amount threshold, although the cumulative movement sub-increment is inputted into the preset displacement function to obtain the corresponding target movement amount, the target movement amount needs to be erased because it is too small, thereby avoiding a situation in which the target object associated with the target reference point generates an unrealistic inertial motion, i.e., the target movement amount is finally determined to be 0, and the target object remains relatively stationary in the display interface. Accordingly, when the cumulative movement sub-increment of the target reference point in the x-axis direction is 0.2 and the preset movement amount threshold is 0.1, because the cumulative movement sub-increment is greater than the preset movement amount threshold, in this case, it is necessary to simulate the inertial motion of the target object associated with the target reference point in the display interface, i.e., the cumulative movement sub-increment 0.2 is input to the preset displacement function to obtain the corresponding target movement amount.
S140, rendering a target object to a target display position in the current to-be-processed video frame based on the target movement amount.
In this embodiment, after the target movement amount of the target reference point is determined, the target display position of the target reference point in the current to-be-processed video frame may be determined based on the target movement amount, thereby rendering the target object to the target display position.
For example, when the target movement amount is determined, the target movement amount may be sent down to a vertex shader used for rendering the image, thereby determining a final position, i.e., the target display position, in the display interface where the object associated with the target reference point needs to be displayed. The target display position may be a two-dimensional coordinate, the vertex shader is a programmable program that is used for image rendering and replaces a fixed rendering pipeline, is primarily responsible for the geometric computation of the vertices in the model. It should be understood by those skilled in the art that only one vertex shader can be activated at the same time, and when the vertex shader runs in the GPU, a corresponding image can be rendered in the display interface, and embodiments of the present disclosure do not repeat the relevant contents in detail herein.
In this embodiment, the primary task of the vertex shader is to convert the target movement amount of the target reference point from a three-dimensional model space to a homogeneous clipping space, i.e., to convert the three-dimensional coordinate into the two-dimensional coordinate corresponding to the display interface. It can be understood that the target object in the display interface is formed of two faces, coordinates of vertices of two triangles of one face of the target object are determined after processing the target movement amount by the vertex shader, and the coordinates are also the coordinates obtained by perspective division or homogeneous division on the basis of the homogeneous clipping space and can be used to reflect at least the display position of the target object associated with the target reference point in the display interface. That is, a process showing the inertial motion of the target object as the shooting angle changes can be rendered in the display interface based on the two-dimensional coordinates determined by the vertex shader.
It should be noted that the above-mentioned solution of the present embodiment describes the process of determining the rendering of the target object in the to-be-processed video frame at the current moment, however, the camera device may also continue to generate movement after the current moment, and accordingly, the target object needs to generate inertial motion in the picture of the to-be-processed video frame at the subsequent moment, and therefore, the rendering method of the target object in the subsequent frames can be performed according to the embodiment of the present disclosure.
For example, after determining the target movement amount of the target reference point in the to-be-processed video frame at the current moment, in order to ensure that the application can accurately render the picture of the inertial motion of the target object in the display interface after the moment, it is further necessary to update the to-be-called cumulative increment based on the target movement amount and store the to-be-called cumulative increment into the increment cache pool to determine the target movement amount of a next to-be-processed video frame.
Continuing to explain with the above example, when it is determined that the target movement amount of the target reference point in the to-be-processed video frame at the current moment is (a2, b2, c2), the to-be-called cumulative increment (Δx1−a1, Δy1−b1, Δz1−c1) at the current moment needs to be updated based on the parameter, i.e., subtracting the movement increment and the target movement amount of the target reference point in the to-be-processed video frame at the current moment from the to-be-called cumulative increment, and taking the calculated (Δx1−a1+Δx2−a2, Δy1−b1+Δy2−b2, Δz1−c1+Δz2−c2) as the updated to-be-called cumulative increment, and decoupling it according to the three coordinate axes in the three-dimensional space coordinate system, storing the obtained three values into the increment cache pool, respectively, when it is necessary to determine the target movement amount of the target reference point in the next to-be-processed video frame, according to the scheme of the present embodiment, the to-be-called cumulative increment is called from the increment cache pool to calculate a new target movement amount (a3, b3, c3) of the target reference point.
It should be noted that the technical solution of the present embodiment may be implemented in a mobile terminal in which a relevant video processing application is installed, may also be implemented by a cloud server through uploading relevant data to the cloud by the application, and when the server has finished processing, the processing result may be sent down to a corresponding terminal, thereby rendering an inertial motion picture of the target object in a display interface of the corresponding terminal.
For example, taking a case that a target effect mounted on the target reference point is a balloon as an example, assuming that a position of the target reference point in a N-th frame is A and a position of the target reference point in a (N+1)-th frame is B, i.e., the balloon effect is expected to move from point A to point B, but in practical application, the movement of the balloon is slow, the calculated display position of the balloon effect in the N-th frame is a point C between the point A and the point B, i.e., an actual movement distance of the balloon effect is less than a desired movement distance |AB|. In practice, there is inertia in the movement of any object, when a mobile terminal to which all video frames after a (N+2)-th frame belong does not move, the calculated balloon effect moves to a point D, a distance between A and D is larger than a distance between A and B, and the balloon effect can be returned from point D to point B in subsequent frames due to the action of the traction rope.
The technical solution of an embodiment of the present disclosure first determines a current movement increment of a target reference point relative to a current to-be-processed video frame, that is, determines a movement increment of the target reference point within a three-dimensional space coordinate system, then determines a target movement amount of the target reference point according to the current movement increment and a to-be-called cumulative increment, for example, determines the target movement amount of the target reference point according to the current cumulative increment and a preset displacement function and renders a target object to a target display position in the current to-be-processed video frame based on the target movement amount, thereby achieving the motion simulation of an object in a picture when the picture of the video changes, reducing the sense of severance between the object in the picture and the real world, and improving the quality of the picture, while also enhancing the user's usage experience.
The current movement increment determination module 210 is configured to determine a current movement increment of a target reference point relative to a current to-be-processed video frame, and the current movement increment is determined based on position information of the target reference point in a previous to-be-processed video frame.
The current cumulative increment determination module 220 is configured to determine a current cumulative increment of the target reference point based on the current movement increment and a to-be-called cumulative increment, and the to-be-called cumulative increment is determined based on historical movement increments of historical to-be-processed video frames and corresponding historical target movement amounts.
The target movement amount determination module 230 is configured to determine a target movement amount of the target reference point according to the current cumulative increment and a preset displacement function.
The rendering module 240 is configured to render a target object to a target display position in the current to-be-processed video frame based on the target movement amount.
On the basis of the above technical solution, the image display apparatus further comprises a to-be-called cumulative increment update module.
The to-be-called cumulative increment update module is configured to update the to-be-called cumulative increment based on the target movement amount and store the to-be-called cumulative increment into an increment cache pool to determine a target movement amount for a next to-be-processed video frame.
On the basis of the above technical solution, the image display apparatus further includes a relative position matrix determination module.
The relative position matrix determination module is configured to determine a relative position matrix of the target reference point and a rendering camera, to determine, according to the relative position matrix, the current movement increment of the target reference point relative to the current to-be-processed video frame; the target reference point is for mounting a target effect.
On the basis of the above technical solution, the current movement increment determination module 210 includes a camera position information determination unit, a current position information determination unit, and a current movement increment determination unit.
The camera position information determination unit is configured to determine camera position information of the rendering camera based on a simultaneous localization and mapping algorithm.
The current position information determination unit is configured to determine current position information of the target reference point according to the relative position matrix and the camera position information.
The current movement increment determination unit is configured to determine the current movement increment according to the current position information and historical position information corresponding to the previous to-be-processed video frame.
For example, the current movement increment determination unit is further configured to determine a movement sub-increment in a coordinate axis direction according to a current space coordinate corresponding to the current position information and a historical space coordinate corresponding to the historical position information; and take the movement sub-increment as the current movement increment.
On the basis of the above technical solution, the current cumulative increment determination module 220 includes a to-be-called cumulative increment invoking unit and a current cumulative increment determination unit.
The to-be-called cumulative increment invoking unit is configured to retrieve the to-be-called cumulative increment from the increment cache pool, the to-be-called cumulative increment comprising a to-be-called cumulative sub-increment in a coordinate axis direction; determine a cumulative movement sub-increment of the target reference point in a coordinate axis direction based on the to-be-called cumulative sub-increment and a corresponding movement sub-increment.
The current cumulative increment determination unit is configured to take the cumulative movement sub-increment as the current cumulative increment.
On the basis of the above technical solution, the target movement amount determination module 230 includes a to-be-moved amount determination unit and a target movement amount determination unit.
The to-be-moved amount determination unit is configured to process a cumulative movement sub-increment greater than a preset movement amount threshold based on the preset displacement function to obtain a first to-be-moved amount; and take a cumulative sub-increment not greater than the preset movement amount threshold as a second to-be-moved amount.
The target movement amount determination unit is configured to determine the target movement amount based on the first to-be-moved amount and the second to-be-moved amount.
On the basis of the above technical solution, the preset displacement function is determined based on a traction coefficient for towing the target object, a preset wind resistance coefficient, and a speed coefficient.
On the basis of the above technical solution, the rendering module 240 includes a target display position determination unit and a rendering unit.
The target display position determination unit is configured to determine, based on the target movement amount, the target display position of the target reference point in the current to-be-processed video frame.
The rendering unit is configured to render the target object to the target display position.
The technical solution provided by the embodiment first determines a current movement increment of a target reference point relative to a current to-be-processed video frame, that is, determines a movement increment of the target reference point within a three-dimensional space coordinate system, then determines a target movement amount of the target reference point according to the current movement increment and a to-be-called cumulative increment, for example, determines the target movement amount of the target reference point according to the current cumulative increment and a preset displacement function and renders a target object to a target display position in the current to-be-processed video frame based on the target movement amount, thereby achieving the motion simulation of an object in a picture when the picture of the video changes, reducing the sense of separation between the object in the picture and the real world, and improving the quality of the picture, while also enhancing the user's usage experience.
The image display apparatus provided by the embodiments of the present disclosure can execute the image display method provided by any of the embodiments of the present disclosure, and has corresponding functional modules for executing the method and beneficial effects.
It is worth noting that the various units and modules included in the above-described apparatus are only divided according to functional logics, but not limited to the division manner described above, as long as the corresponding functions can be implemented; in addition, the specific names of the plurality of functional units are only for the convenience of distinguishing between each other, and are not intended to limit the protection scope of the embodiments of the present disclosure.
As shown in
Usually, the following apparatuses can be connected to the I/O interface 305: an input apparatus 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, or the like; an output apparatus 307 including, for example, a liquid crystal display (LCD), a loudspeaker, a vibrator, or the like; a storage apparatus 308 including, for example, a magnetic tape, a hard disk, or the like; and a communication apparatus 309. The communication apparatus 309 may allow the electronic device 300 to be in wireless or wired communication with other devices to exchange data. While
According to the embodiments of the present disclosure, processes described above with reference to the flowchart may be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program codes for performing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication apparatus 309, or installed from the storage apparatus 308, or installed from the ROM 302. When the computer program is executed by the processing apparatus 301, the above functions defined in the method of the embodiment of the present disclosure are performed.
The names of messages or information exchanged between a plurality of apparatuses in the implementations of the present disclosure are used for illustrative purposes only, and are not intended to limit the scope of the messages or information.
The electronic device provided by the embodiments of the present disclosure belongs to the same inventive concept as the image display method provided by the above embodiments, technical details that are not elaborately described in the present embodiment can be referred to the above embodiments, and the present embodiment has the same beneficial effects as the above embodiments.
An embodiment of the present disclosure provides a computer storage medium, a computer program is stored on the computer storage medium, when the computer program is executed by a processor, the image display method provided by the above embodiments is implemented.
It should be noted that the above-mentioned computer-readable medium in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. For example, the computer-readable storage medium may be, but not limited to, an electric, magnetic, optical, electromagnetic, infrared, or semi-conductive system, apparatus, or device, or any combination thereof. More specific examples of the computer-readable storage medium may include but not be limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random-access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination of them. In the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, the computer-readable signal medium may include a data signal that propagates in a baseband or as a part of a carrier wave and carries computer-readable program codes. The data signal propagating in such a manner may take a plurality of forms, including but not limited to an electromagnetic signal, an optical signal, or any appropriate combination thereof. The computer-readable signal medium may also be any other computer-readable medium than the computer-readable storage medium. The computer-readable signal medium may send, propagate, or transmit a program used by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted by using any suitable medium, including but not limited to an electric wire, a fiber-optic cable, radio frequency (RF), and the like, or any appropriate combination of them.
In some implementations, a client and a server may communicate by using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and may be interconnected with digital data communication (for example, the communications network) in any form or medium. Examples of the communication network include a local area network (“LAN”), a wide area network (“WAN”), the Internet (for example, the Internet), and an end-to-end network (for example, an ad hoc end-to-end network), as well as any currently known or future developed network.
The above-mentioned computer-readable medium may be included in the electronic device described above, or may exist alone without being assembled into the electronic device.
The above-mentioned computer-readable medium carries one or more programs, when the one or more programs are executed by the electronic device, the electronic device is caused to:
The computer program codes for performing the operations of the present disclosure can be written in one or more programming languages or a combination thereof. The above programming languages include but not limited to object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the “C” language or similar programming languages. The program code can be executed entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the scenario related to the remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the accompanying drawings illustrate system architectures, functions, and operations that may be implemented by the system, the method, and the computer program product according to the various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, a program segment, or a part of codes, and the module, the program segment, or the part of codes includes one or more executable instructions for implementing specified logic functions. It should also be noted that, in some alternative implementations, functions marked in the blocks may also occur in an order different from the order designated in the accompanying drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and they may sometimes be executed in a reverse order, which depends on involved functions. It should also be noted that each block in the block diagrams and/or flowcharts and combinations of the blocks in the block diagrams and/or flowcharts may be implemented by a dedicated hardware-based system for executing specified functions or operations, or may be implemented by a combination of a dedicated hardware and computer instructions.
The units described and involved in the embodiments of the present disclosure may be implemented by software, or may be implemented by hardware. In some cases, the name of a unit does not constitute a limitation on the unit itself, for example, the first obtaining unit may also be described as “a unit for obtaining at least two Internet Protocol addresses”.
The functions described above in the present disclosure may be executed at least in part by one or more hardware logic components. For example, without limitations, exemplary types of the hardware logic components that can be used include: a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), an application specific standard product (ASSP), a system on chip (SOC), a complex programmable logic device (CPLD), and the like.
In the context of the present disclosure, a machine readable medium may be a tangible medium that may include or store a program for use by or in combination with an instruction execution system, apparatus, or device. The machine readable medium may be a machine readable signal medium or a machine readable storage medium. The machine readable medium may include but not be limited to an electronic, magnetic, optical, electromagnetic, infrared, or semi-conductive system, apparatus, or device, or any appropriate combination of them. More specific examples of the machine readable storage medium may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination of them.
According to one or more embodiments of the present disclosure, [Example One] provides an image display method, and the method includes:
According to one or more embodiments of the present disclosure, [Example Two] provides an image display method, and the method further comprises:
According to one or more embodiments of the present disclosure, [Example Three] provides an image display method, before the determining a current movement increment of a target reference point relative to a current to-be-processed video frame, the method further comprises:
The target reference point is used for mounting a target effect.
According to one or more embodiments of the present disclosure, [Example Four] provides an image display method, and the determining a current movement increment of a target reference point relative to a current to-be-processed video frame comprises:
According to one or more embodiments of the present disclosure, [Example five] provides an image display method, and the determining the current movement increment according to the current position information and historical position information corresponding to the previous to-be-processed video frame comprises:
According to one or more embodiments of the present disclosure, [Example Six] provides an image display method, and the determining a current cumulative increment of the target reference point based on the current movement increment and a to-be-called cumulative increment comprises:
According to one or more embodiments of the present disclosure, [Example Seven] provides an image display method, and the determining a target movement amount of the target reference point according to the current cumulative increment and a preset displacement function comprises:
According to one or more embodiments of the present disclosure, [Example eight] provides an image display method, the preset displacement function is determined based on a traction coefficient for towing the target object, a preset wind resistance coefficient, and a speed coefficient.
According to one or more embodiments of the present disclosure, [Example Nine] provides an image display method, the rendering a target object to a target display position in the current to-be-processed video frame based on the target movement amount comprises:
According to one or more embodiments of the present disclosure, [Example Ten] provides an image display apparatus, and the apparatus include:
In addition, although a plurality of operations have been described in a particular order, it shall not be construed as requiring that such operations are performed in the stated particular order or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussions, these shall not be construed as limitations to the scope of the present disclosure. Some features described in the context of a separate embodiment may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in various embodiments individually or in a plurality of embodiments in any appropriate sub-combination.
| Number | Date | Country | Kind |
|---|---|---|---|
| 202210130360.8 | Feb 2022 | CN | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/CN2023/074499 | 2/6/2023 | WO |
