Patent Grant
11876843
This application claims priority from Chinese Application No. 202111015245.8, filed on Aug. 31, 2021, the contents of which are incorporated by reference herein in its entirety.
The present disclosure relates to the field of image processing, in particular to a method, an apparatus, a medium and an electronic device for generating a round-table video conference.
With the development of Internet technology, a video conference based on Internet technology can realize simultaneous remote communication of more than one person, which can realize both call communication and video communication, thereby perfectly solving the problem of remote conferences in multiple places.
However, in a display part of current video conferences, the videos of multiple participants are often placed on a desktop of a terminal at will, and the background of the participants is messy, which undermines the seriousness of the conference.
Some embodiments of the disclosure provide a method for generating a round-table video conference, comprising:
In some embodiments, the generating and displaying a virtual round-table video in the form of a simulated round-table conference according to the scrolling sequence and target videos corresponding to the target seat numbers comprises:
In some embodiments, wherein the generating and displaying the virtual round-table video according to the scrolling sequence and the virtual participant video corresponding to the target seat number comprises:
In some embodiments, the determining all target seat numbers and a scrolling sequence of the target seat numbers based on the scrolling starting number, the scrolling ending number and the head indication information comprises:
In some embodiments, the acquiring head indication information of a current participant from a currently captured video comprises:
In some embodiments, the determining head rotation information of the current participant in a specified direction based on the two-dimensional positional information and the marking information of all facial structure points comprises:
In some embodiments, the method further comprises:
Some embodiments of the disclosure provide an apparatus for generating a round-table video conference, comprising:
Some embodiments of the disclosure provide a computer-readable storage medium with a computer program stored thereon, when the program is executed by a processor, the method according to any of the above embodiments is implemented.
Some embodiments of the disclosure provide an electronic device, comprising:
Compared with the related art, the above solutions of the embodiments of the present disclosure have at least the following beneficial effects:
The present disclosure provides a method, an apparatus, a medium and an electronic device for generating a round-table video conference. In the present disclosure, in response to the head instruction information of the participant, the scrolling starting number is determined from the round-table seat number sequence through the video of the current presenter, and the scrolling ending number is determined from the round-table seat number sequence through the video of the current speaker, thereby the scrolling sequence of the target seat numbers is obtained. According to the scrolling sequence, a manner of the virtual camera in the center of the round table being rotated and shooting is simulated, and the images of the participants queued between two points are scrolled and displayed, so that a virtual round-table conference scene is created in the video conference.
In order to make the objectives, technical solutions, advantages of the present disclosure clearer, the present disclosure will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by a those skilled in the art without creative work shall fall within the protection scope of the present disclosure.
The terms used in the embodiments of the present disclosure are only for the purpose of describing specific embodiments, and are not intended to limit the present disclosure. The singular forms of “a”, “said” and “the” used in the embodiments of the present disclosure and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, “multiple” generally contains at least two types.
It should be understood that the term “and/or” used in this description is only an association relationship describing associated objects, indicating that there can be three types of relationships. For example, A and/or B can mean that there are three cases where A alone, A and B, and B alone. In addition, the character “/” in this text generally indicates that the associated objects before and after are in an “or” relationship.
It should be understood that although the terms first, second, third, etc. may be used to describe the embodiments of the present disclosure, these descriptions should not be limited to these terms. These terms are only used to distinguish the description. For example, without departing from the scope of the embodiments of the present disclosure, the first may also be referred to as the second, and similarly, the second may also be referred to as the first.
Depending on the context, the words “if” and “whether” as used herein can be interpreted as “when” or “as” or “in response to determination” or “in response to detection”. Similarly, depending on the context, the phrase “if . . . determined” or “if (statement or event) . . . detected” can be interpreted as “when determined” or “in response to determination” or “when detected (statement or event)” or “in response to detection (statement or event)”.
It should also be noted that the terms “include”, “comprise” or any other variants thereof are intended to cover non-exclusive inclusion, so that a commodity or device including a series of elements not only includes those elements, but also includes those elements that are not explicitly listed. Other elements of, or also include elements inherent to this commodity or device. If there are no more restrictions, the element defined by the sentence “includes a . . . ” does not exclude the existence of another same element in the commodity or device that includes the element.
Optional embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.
In the embodiments of the present disclosure, all participants are gathered at a virtual round-table, and a terminal used by each participant can scroll from an image of a current presenter of the terminal to an image of a current speaker according to head instruction information of the participants. In the process of scrolling, a way of rotating and shooting by a virtual camera in the center of the round-table is simulated, and images of the participants queued between the presenter and the speaker are scrolled and displayed, so as to create a virtual round-table conference scene in a video conference.
In this regard, as shown in
The embodiment of the present disclosure will be described in detail below with reference to
In step S101, head indication information of a current participant is acquired from a currently captured video.
The currently captured video is captured by a camera of a current terminal used by the current participant. For example, if the current participant participates in a video conference through a mobile phone, a built-in camera of the mobile phone is used for capturing the video; if the current participant participates in the video conference through a computer, a built-in/external camera of the computer is used for capturing the video.
The currently captured video is not necessarily used for displaying in the current terminal, but is mainly used to analyze whether the current participant has sent head instruction information. The head indication information is used to instruct the current terminal to simulate a round-table conference scene to scroll to an image of the current speaker and display. If the current participant in front of the current terminal is the current speaker, an image of the current participant can be displayed on the current terminal.
Optionally, the head indication information includes information about the head swinging to the left or information about the head swinging to the right. In the embodiment of the present disclosure, a rotation direction of a virtual camera is indicated by a swing direction of the head.
In step S102, a first seat number corresponding to a video of a current presenter and a second seat number corresponding to a video of the current speaker are acquired from a round-table seat number sequence, in response to the head instruction information.
The embodiment of the present disclosure provides the seat number of the round table for the video captured by each participant, and the seat number is a virtual seat number. If the virtual camera rotates to the seat number, the current terminal used by the current participant will display the video corresponding to the seat number.
In order to be able to manage the seat number, the embodiment of the present disclosure provides a round-table seat number sequence. The seat number sequence records the correspondence between the seat numbers and the video of the participants, and the relationship between the seat numbers. Such relationship can be understood as the positional relationship of the videos corresponding to the seat numbers in front of the round table, and can also be understood as the positional relationship of the participants corresponding to the seat numbers in front of the round table.
The round-table seat number sequence can be automatically established according to the order in which the participants enter a video conference room, or can be preset according to identities of the participants. The present disclosure does not make special limitations.
The video of the current presenter can be understood as the video of the participant displayed on the current terminal of the current participant.
The video of the current speaker can be understood as the audio of the current speaker transmitted from the current terminal of the current participant. The audio of the current speaker does not necessarily come from the original video of the current presenter, but may come from an original video of another participant not being displayed, then the original video of the participant not being displayed is the video of the current speaker.
The first seat number is used to represent a scrolling starting number, and the second seat number is used to represent a scrolling ending number.
In step S103, all target seat numbers and a scrolling sequence of the target seat numbers are determined from the seat number sequence based on the scrolling starting number, the scrolling ending number and the head indication information.
The target seat numbers can be understood as all seat numbers that the virtual camera passes through when the virtual camera rotates from the scrolling starting number to the scrolling ending number.
In order to facilitate effective management of target seat numbers, embodiments of the present disclosure provide a scrolling sequence of the target seat numbers. The scrolling sequence records correspondence between the target seat numbers and the videos of the participants, and an order of the target seat numbers passed by the virtual camera when it rotates. The order of the target seat numbers is established based on the relationship of the seat numbers in the seat number sequence.
In a specific embodiment, said that all target seat numbers and a scrolling sequence of the target seat numbers are determined from the seat number sequence based on the scrolling starting number, the scrolling ending number and the head indication information includes the following steps:
In step S103a, in response to the scrolling starting number being smaller than the scrolling ending number, and the head indication information including information about the head swinging to the right, all target seat numbers and a scrolling sequence of the target seat numbers are determined from the seat number sequence from the scrolling starting number to the scrolling ending number orderly.
For example, as shown in
In step S103b, in response to the scrolling starting number being smaller than the scrolling ending number, and the head indication information including information about the head swinging to the left, all target seat numbers and a scrolling sequence of the target seat numbers are determined from the seat number sequence from the scrolling starting number to the a starting number of the seat number sequence in reverse order, and from an ending number of the seat number sequence to the scrolling ending number in reverse order.
For example, as shown in
In step S103c, in response to the scrolling starting number being greater than the scrolling ending number, and the head indication information including the information about the head swinging to the left, all target seat numbers and a scrolling sequence of the target seat numbers are determined from the seat number sequence from the scrolling starting number to the scrolling ending number in reverse order.
For example, as shown in
In step S103d, in response to the scrolling starting number being greater than the scrolling ending number, and the head indication information including the information about the head swinging to the right, all target seat numbers and a scrolling sequence of the target seat numbers are determined from the seat number sequence from the scrolling starting number to an ending number of the seat number sequence orderly, and from a starting number of the seat number sequence to the scrolling ending number orderly.
For example, as shown in
In step S104, a virtual round-table video in the form of a simulated round-table conference is generated and displayed according to the scrolling sequence and target videos corresponding to the target seat numbers.
Herein, the virtual round-table video includes the images of the participants in the target video being scrolled and displayed to the image of the current speaker successively according to the scrolling sequence. For example, it is scrolled and displayed horizontally to the image of the current speaker successively according to the scrolling sequence.
In the embodiment of the present disclosure, in response to the head instruction information of the participant, the scrolling starting number is determined from the round-table seat number sequence through the video of the current presenter, and the scrolling ending number is determined from the round-table seat number sequence through the video of the current speaker, thereby the scrolling sequence of the target seat numbers is obtained. According to the scrolling sequence, a manner of the virtual camera in the center of the round table being rotated and shooting is simulated, and the images of the participants queued between two points are scrolled and displayed, so that a virtual round-table conference scene is created in the video conference.
Since the embodiment of the present disclosure is further optimized based on above-mentioned embodiments, explanations based on the same method and meanings of the same name are the same as the above-mentioned embodiment, and will not be repeated here, as shown in
Said that head indication information of a current participant is acquired from a currently captured video includes the following steps:
In step S201, a video image of the current participant is acquired from the currently captured video.
The video is composed of frames of video images taken continuously. Therefore, when the video is analyzed, that is, changes in the video image are analyzed.
In step S202: two-dimensional positional information and marking information of facial structure points in the video image is determined.
The facial structure points are marked in a facial area of the current participant in the video image. All facial structure points can reflect a facial posture of the current participant. As shown in
In step S203, head rotation information of the current participant in a specified direction is determined based on the two-dimensional positional information and the marking information of all facial structure points.
For example, in space coordinates, the specified direction is a positive direction of an X axis, a positive direction of a Y axis, and a positive direction of a Z axis.
In the embodiment of the present disclosure, the head rotation information of the current participant is determined through facial structure points, and if the head rotation information satisfies an indication condition, the head indication information of the current participant is generated based on the head rotation information. For example, the information about the head swinging to the right or the information about the head swinging to the left. If the facial structure points are limited to the feature points of the five sense organs and/or the feature points of the face shape, the amount of data processing can be reduced, and the efficiency of the data processing can be improved.
In a specific embodiment, said that head rotation information of the current participant in a specified direction is determined based on the two-dimensional positional information and the marking information of all facial structure points includes the following steps:
In step S203-1, in the facial area of the video image, according to a connection relationship rule of the marking information, each two-dimensional positional information is used as vertex position information to obtain facial mesh information.
The connection relationship rule provides the connection relationship between the marking information. For example, the connection relationship rule provides: the marking information “J52” is connected with the marking information “J60”, “J61” and “J62”. Therefore, in the facial area of the video image, the two-dimensional positional information corresponding to the marking information “J52”, “J60”, “J61” and “J62” can be connected as the vertex positional information of the mesh, thereby generating the facial mesh information.
The facial mesh information includes the two-dimensional positional information and the marking information of each mesh vertex and the two-dimensional positional information and the marking information of adjacent mesh vertices connected to the mesh vertex.
In step S203-2, reference mesh information and the facial mesh information corresponding to the current participant are input into a trained neural network model to obtain the head rotation information.
Herein, the reference mesh information is obtained based on the two-dimensional positional information and the marking information of the facial structure points in a reference image of the current participant.
Before the video conference is held, the current terminal uses the reference images and reference mesh information obtained by specific facial images taken by the current participant at multiple specific angles, which are mainly used as control data to analyze movements of the head of the current participant.
The facial structure points in the video image and the facial structure points in the reference image have a corresponding relationship with respect to the facial position of the current participant. For example, the facial structure points of a first image of the current participant and the facial structure points of a second image are both located at the left inner corner of the face of the current participant. Therefore, the facial structure points of the two images have a corresponding relationship in terms of the facial position of the participant.
This specific embodiment uses the corresponding relationship to determine the positional relationship between the facial structure points of the reference mesh information and the facial structure points of the facial mesh information, and then the head rotation information of the current participant in the specified direction is obtained through the trained neural network model. Therefore tedious three-dimensional calculations are reduced, and the processing efficiency of analyzing head movements and the fluency of the video conference are improved.
The reference mesh information is taken as the reference information for determining the facial mesh information of the facial image when determining the head rotation direction. For example, the reference mesh information is generated based on the reference facial image.
When training the neural network model, the head rotation direction and the facial mesh information and reference mesh information with respect to this direction are used as training samples and input into the neural network model until a preset accuracy is reached. For example, if the head rotation direction is left, then the face mesh information and reference mesh information of the facial image with respect to the left direction, and the head rotation direction are used as samples. Several thousand groups of samples in the same direction can be selected and input these samples into the neural network model for training.
In step S204, in response to the head rotation information meeting an indication condition, the head indication information of the current participant is generated based on the head rotation information.
For example, if the head rotation information is that an angle of rotation to the right in the horizontal direction is greater than or equal to 45 degrees, the head indication information includes the information about the head swinging to the right; if the head rotation information is that an angle of rotation to the left in the horizontal direction is greater than or equal to 45 degrees, the head indication information includes the information about the head swinging to the left.
The embodiment of the present disclosure also provides a method for constructing reference information (such as the reference image and the reference mesh information) of each participant before the video conference is held.
In a specific embodiment, the method further includes the following steps:
In step S100-1, specific facial images of the current participant taken at multiple specific angles are acquired in advance.
For example, before the video conference is held, the current participant can take a specific facial image before entering the electronic conference room, or take a specific facial image when registering members of the electronic conference room. The embodiments of the present disclosure do not make specific limitations.
Optionally, the specific facial image includes a front facial image and side facial images on both sides. These three images can not only include the facial features of the current participant, but also reduce the amount of data processing, thereby improving the efficiency of the video conference.
In step S100-2, the two-dimensional positional information and the marking information of the facial structure points in each specific facial image are determined.
The facial structure points are marked in the facial area of the current participant in the specific facial image.
The facial structure points and the facial structure points in the video image of the current participant have a corresponding relationship in terms of the facial position of the current participant.
In step S100-3, each two-dimensional positional information is used as vertex position information to obtain specific mesh information, in the facial area of the specific facial image, according to a connection relationship rule of the marking information.
In step S100-4, a reference image and reference mesh information matched with the facial structure are acquired from a reference facial model library, based on the specific mesh information of the current participant.
In this specific embodiment, the facial mesh information is retrieved in the reference facial model library based on the specific mesh information in multiple specific facial images of the participant. When the specific mesh information matches the facial mesh information in the library, the reference information corresponding to the facial mesh information in the library is taken as the reference information that matches the facial structure of the participant. The reference information is obtained through multiple two-dimensional specific facial images, which simplifies a data acquisition step and improves the versatility and efficiency of data processing.
Since the embodiment of the present disclosure is further optimized based on the above-mentioned embodiments, the explanations based on the same method and the meanings of the same name are the same as the above-mentioned embodiment, and will not be repeated here, as shown in
Said that a virtual round-table video in the form of a simulated round-table conference is generated and displayed according to the scrolling sequence and the target videos corresponding to the target seat numbers includes the following steps:
In step S301, a matting video is acquired according to the target video corresponding to the target seat number.
The matting video only includes the image of the participant in the target video. It can be understood that the matting video is to remove a background image for the participant in the video, and only the image of the participant is retained. In order to be able to convert the video into a matting video in real time and quickly, the background of the participant is usually arranged in a single color (such as green) when the video is captured.
In step S302, a virtual participant video corresponding to the target seat number is acquired, based on the fusion of a foreground image representing the equipment of the virtual conference room, and a background image representing the virtual conference room environment, as well as the image of the participant in the matting video.
The foreground image and the background image are preset images. For example, a round-table image is taken as an image of the equipment of the virtual conference room, and windows and bookshelf images are taken as an image of environment of the virtual conference room. When fusing, the round-table image is used to block an image of the lower body of the participant, and only an image of the upper body of the participant is displayed. Meanwhile, the windows and bookshelf images are placed behind the image of the upper body, so that the virtual conference video corresponding to each target seat number includes the virtual video of the participant sitting in front of the round table in the conference room and participating in the conference.
In step 303, a virtual round-table video is generated and displayed according to the scrolling sequence and the virtual participant video corresponding to the target seat number.
The embodiments of the present disclosure add foreground and background to displayed images of the participant, so that the video conference is closer to a real conference scene, and the seriousness of the conference is increased.
In a specific embodiment, said that a virtual round-table video is generated and displayer according to the scrolling sequence and the virtual conference video corresponding to the target seat number includes the following steps:
In step 303-1, a first virtual participant image and a second virtual participant image corresponding to each time point in a continuous time point sequence are acquired.
The continuous time point sequence is the time point sequence required for scrolling from the scrolling starting number to the scrolling ending number. The continuous time point sequence is used to align the video images in the multiple target videos in accordance with time, so that the target video images related to the same time point can generate a virtual composite image. For example, the continuous time point sequence includes 10 time points: time points T1˜T10; during scrolling, target videos A and B are related to time point T3, and persons in the target videos A and B are needed to synthesize into a virtual composite video.
Herein, the first virtual participant image belongs to a first virtual participant video, and the second virtual participant image belongs to a second virtual participant video.
In the scrolling sequence, the target seat number corresponding to the first virtual participant video is arranged adjacently in front of the target seat number corresponding to the second virtual participant video.
For example, the scrolling sequence includes target seat numbers 4-9, and if the target seat number corresponding to the first virtual participant video is 5, the target seat number corresponding to the second virtual participant video is 6.
In step 303-2, a percentage of a sequence number of the each time point in the continuous time point sequence to total sequence number of the continuous time point sequence is calculated to obtain a first cutting ratio.
For example, the continuous time point sequence includes 10 time points: time points T1˜T10; then total number of the continuous time point sequences is 10; if the time point in the continuous time point sequence is T6, then the sequence number of T6 in the continuous time point sequence is 6, and the cutting ratio=the sequence number/total number=6/10=60%.
In step 303-3, a remaining second cutting ratio is obtained based on the first cutting ratio.
For example, continued above example, the second cutting ratio=1−the first cutting ratio=1−60%=40%.
In step 303-4, a length segmentation is performed on the second virtual participant image corresponding to the each time point, so that a second segmented image segmented based on the first cutting ratio is obtained from a side of a reverse scrolling direction, and a length segmentation is performed on the first virtual participant image corresponding to the each time point, so that a first segmented image segmented based on the second cutting ratio is obtained from a side of the scrolling direction.
For example, continued above example, as shown in
In step 303-5, a virtual composite image matching the seat number sequence is generated based on the target seat number corresponding to the first segmented image and the target seat number corresponding to the second segmented image.
For example, it is assumed that the seat number sequence is arranged from left to right, and if the target seat number corresponding to the first segmented image is 5 and the target seat number corresponding to the second segmented image is 6, then the first segmented image is on the left and the second segmented image is on the right, which is synthesized to generate a virtual composite image; continued above example, a length of the virtual composite image generated after synthesis=400+600=1000, which is collectively called a complete video image.
In step 303-6, a video clip of the virtual round-table video is generated and displayed based on the virtual composite image corresponding to continuous time points.
In the scrolling sequence, every two video clips corresponding to adjacent target seat numbers are synthesized to a virtual composite image through above-mentioned method, than all video clips form a virtual round-table video. The virtual round-table video shows the virtual camera rotating and scrolling the images of the target participants one by one.
The present disclosure also provides an apparatus embodiment inherited from above embodiment, which is used to implement the method and the steps described in above embodiment. The explanation based on the meaning of the same name is the same as above embodiment, and has the same technical effect as above embodiment, which are not repeated here.
As shown in
An acquisition unit 401 is configured to acquire head indication information of a current participant from a currently captured video.
A response unit 402 is configured to acquire a first seat number corresponding to the video of the current presenter and a second seat number corresponding to the video of the current speaker from a round-table seat number sequence in response to the head indication information. The first seat number is used to represent the scrolling starting number, and the second seat number is used to represent the scrolling ending number.
A determination unit 403 is configured to determine all target seat numbers and a scrolling sequence of the target seat numbers from the seat number sequence based on the scrolling starting number, the scrolling ending number, and the head indication information.
A generation unit 404 is configured to generate and display a virtual round-table video in the form of a simulated round-table conference according to the scrolling sequence and the target videos corresponding to the target seat numbers, wherein the virtual round-table video includes the images of the participants in the target video being scrolled and displayed to the images of the current speaker successively according to the scrolling sequence.
Optionally, the generating unit 404 includes:
A matting subunit is configured to acquire a matting video according to the target video corresponding to the target seat number, wherein the matting video only includes the image of the participant in the target video.
An image fusion subunit is configured to acquire a virtual conference video corresponding to the target seat number, based on the fusion of a foreground image representing the equipment of the virtual conference room and a background image representing the virtual conference room environment, as well as the image of the participant in the matting video.
A first generation subunit is configured to generate and display a virtual round-table video according to the scrolling sequence and the virtual conference video corresponding to the target seat number.
Optionally, the first generation subunit includes:
An image acquisition subunit is configured to acquire a first virtual participant image and a second virtual participant image corresponding to each time point in a continuous time point sequence, wherein the first virtual participant image belongs to a first virtual participant video, and the second virtual participant image belongs to a second virtual participant video, and the target seat number corresponding to the first virtual participant video is arranged adjacently in front of the target seat number corresponding to the second virtual participant video.
A ratio calculation subunit is configured to calculate a percentage of a sequence number of the each time point in the continuous time point sequence to total sequence number of the continuous time point sequence to obtain a first cutting ratio.
A ratio obtaining subunit is configured to obtain a remaining second cutting ratio based on the first cutting ratio.
An image segmentation subunit is configured to perform a length segmentation on the second virtual participant image corresponding to the each time point, so that a second segmented image segmented based on the first cutting ratio is obtained from a side of a reverse scrolling direction, and to perform a length segmentation on the first virtual participant image corresponding to the each time point, so that a first segmented image segmented based on the second cutting ratio is obtained from a side of the scrolling direction.
An image synthesis subunit is configured to generate a virtual composite image matching the seat number sequence based on the target seat number corresponding to the first segmented image and the target seat number corresponding to the second segmented image.
A segment generation subunit is configured to generate and display a video clip of the virtual round-table video based on the virtual composite image corresponding to continuous time points.
Optionally, the determination unit 403 includes:
A first response subunit is configured to determine all target seat numbers and a scrolling sequence of the target seat numbers from the seat number sequence from the scrolling starting number to the scrolling ending number orderly, in response to the scrolling starting number being smaller than the scrolling ending number, and the head indication information including information about the head swinging to the right.
A second response subunit is configured to determine all target seat numbers and a scrolling sequence of the target seat numbers from the seat number sequence from the scrolling starting number to the a starting number of the seat number sequence in reverse order, and from an ending number of the seat number sequence to the scrolling ending number in reverse order, in response to the scrolling starting number being smaller than the scrolling ending number, and the head indication information including information about the head swinging to the left.
A third response subunit is configured to determine all target seat numbers and a scrolling sequence of the target seat numbers from the seat number sequence from the scrolling starting number to the scrolling ending number in reverse order, in response to the scrolling starting number being greater than the scrolling ending number, and the head indication information including the information about the head swinging to the left.
A fourth response subunit is configured to determine all target seat numbers and a scrolling sequence of the target seat numbers from the seat number sequence from the scrolling starting number to an ending number of the seat number sequence orderly, and from a starting number of the seat number sequence to the scrolling ending number orderly, in response to the scrolling starting number being greater than the scrolling ending number, and the head indication information including the information about the head swinging to the right.
Optionally, the acquisition unit 401 includes:
A first acquisition subunit is configured to acquire a video image of the current participant from the currently captured video.
A first determination subunit is configured to determine two-dimensional positional information and marking information of facial structure points in the video image, where the facial structure points are marked in a facial area of the current participant in the video image.
A second determination subunit is configured to determine head rotation information of the current participant in a specified direction based on the two-dimensional positional information and marking information of all facial structure points.
A second generation subunit is configured to generate the head indication information of the current participant based on the head rotation information in response to the head rotation information satisfies an indication condition.
Optionally, the second determination subunit includes:
A first obtaining subunit is configured to obtain facial mesh information by using each two-dimensional positional information as vertex position information, according to a connection relationship rule of the marking information, in the facial area of the video image.
A second acquisition subunit is configured to input the reference mesh information and facial mesh information corresponding to the current participant into a trained neural network model to obtain the head rotation information, wherein the reference mesh information is obtained based on the two-dimensional positional information and the marking information of the facial structure points in a reference image of the current participant. The facial structure points in the video image and the facial structure points in the reference image have a corresponding relationship with respect to the facial position of the current participant.
Optionally, the apparatus further includes:
A pre-acquisition unit is configured to acquire in advance specific facial images of the current participant taken at multiple specific angles.
A structure determination unit is configured to determine the two-dimensional positional information and the marking information of the facial structure points in each specific facial image, where the facial structure points are marked in the facial area of the current participant in the specific facial image.
An obtaining unit is configured to obtain specific mesh information by using each two-dimensional positional information as vertex position information, in the facial area of the specific facial image, according to a connection relationship rule of the marking information.
A mesh acquisition unit is configured to acquire a reference image and reference mesh information matched with the facial structure matching from a reference facial model library, based on the specific mesh information of the current participant.
In the embodiment of the present disclosure, in response to the head instruction information of the participant, the scrolling starting number is determined from the round-table seat number sequence through the video of the current presenter, and the scrolling ending number is determined from the round-table seat number sequence through the video of the current speaker, thereby the scrolling sequence of the target seat numbers is obtained. According to the scrolling sequence, a manner of the virtual camera in the center of the round table being rotated and shooting is simulated, and the images of the participants queued between two points are scrolled and displayed, so that a virtual round-table conference scene is created in the video conference.
As shown in
The embodiments of the present disclosure provide a non-volatile computer storage medium, and the computer storage medium stores computer-executable instructions, which can execute the steps of the method described in above embodiments.
Next, referring to
As shown in
Generally, the following devices can be connected to the I/O interface 505: including input devices 506 such as touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, liquid crystal display (LCD), speakers, vibration An output device 507 such as a device; a storage device 508 such as a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device to perform wireless or wired communication with other devices to exchange data. Although
In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from the network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
It should be noted that the aforementioned computer-readable medium in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or a combination of any of the above. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as a part of a carrier wave, and a computer-readable program code is carried therein. This propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium. The computer-readable signal medium may send, propagate, or transmit the program for use by or in combination with the instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to: wire, optical cable, RF (Radio Frequency), etc., or any suitable combination of the above.
The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist alone without being assembled into the electronic device.
The computer program code used to perform the operations of the present disclosure can be written in one or more programming languages or a combination thereof. The above-mentioned programming languages include object-oriented programming languages—such as Java, Smalltalk, C++, and also conventional Procedural programming language—such as “C” language or similar programming language. The program code can be executed entirely on the user's computer, partly on the user's computer, executed as an independent software package, partly on the user's computer and partly executed on a remote computer, or entirely executed on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to pass Internet connection).
The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions, and operations of the system, method, and computer program product according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, program segment, or part of code, and the module, program segment, or part of code contains one or more for realizing the specified logical function Executable instructions. It should also be noted that, in some alternative implementations, the functions marked in the block may also occur in a different order from the order marked in the drawings. For example, two blocks shown in succession can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and/or flowchart, and the combination of the blocks in the block diagram and/or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations Or it can be realized by a combination of dedicated hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented in a software manner, or may be implemented in a hardware manner. Among them, the name of the unit does not constitute a limitation on the unit itself under certain circumstances.
| Number | Date | Country | Kind |
|---|---|---|---|
| 202111015245.8 | Aug 2021 | CN | national |
| Number | Name | Date | Kind |
|---|---|---|---|
| 7336296 | Brown | Feb 2008 | B2 |
| 9460337 | Ohba | Oct 2016 | B2 |
| 20110216153 | Tasker | Sep 2011 | A1 |
| 20210406987 | Kornilov | Dec 2021 | A1 |
| Number | Date | Country | |
|---|---|---|---|
| 20230061662 A1 | Mar 2023 | US |
