This application relates to the field of artificial intelligence, including a bone model combination method, apparatus and device, and a storage medium.
At present, a skeleton-driven deformation technology has been a hot topic studied in the field of cambered surface modeling due to its flexible control measure and simple operations. For example, in mainstream game engines, the skeleton-driven deformation technology is usually used to achieve an animation system with mesh deformation driven by bones. In reality, many animal and plant (including human body) models or industrial product models have obvious skeleton features, so motion simulation or a deformation design of a mesh model can be achieved by applying skeleton-driven deformation. A distinctive feature of animals that are different from other organisms is that animals can drive the muscles on the bones to move through bone motions, resulting in various types of motions. The skeleton-driven deformation technology is precisely a mesh deformation measure generated by simulating this motion form. Deformation of a skeleton is manipulated to drive deformation of a mesh model (also referred to as skinning or skin) attached to the skeleton. At present, the skeleton-driven deformation technology has been widely applied to fields such as animation, games, industrial product design, and biomedicine.
In an animation system that uses the skeleton-driven deformation technology to achieve mesh deformation driven by bones, for the same part in a common skeleton, multiple bone models may simultaneously exist, which can be mapped to the part of the common skeleton, and these bone models may separately correspond to multiple different bone animations to be presented simultaneously. For the above situation, during processing of the multiple bone models mapped to the same common bone in the common skeleton, it is usually necessary to bind the multiple bone models to the common bone. It usually leads to bone conflicts, causing some bone models to be lost or animations to be played in error, resulting in abnormal pictures.
In order to solve the above problems, this disclosure achieves mapping between a plurality of bone models and a bone duplicate of the conflicting bone by duplicating and binding a conflicting bone, thereby achieving efficient combination of complicated bone models.
In an embodiment, a bone model combination method includes obtaining a plurality of to-be-combined bone models, each of the plurality of to-be-combined bone models comprising a pixel set, and each bone model corresponding to a common skeleton. The method further includes determining, for any two bone models among the plurality of bone models, a set of common bones that is an intersection of bones included in both of the two bone models. The method further includes determining, for all the common bones in the intersection, common bone chains, generating common bone chain duplicates on the basis of the common bone chains, and binding the common bone chain duplicates at upper common bones of the common bone chains. The method further includes building a mapping between each of the common bone chains and a pixel set included in a corresponding bone model of the two bone models, and building a mapping between each of the common bone chain duplicates and a pixel set included in a corresponding bone model of the two bone models.
In an embodiment, a bone model combination apparatus includes processing circuitry configured to obtain a plurality of to-be-combined bone models, each of the plurality of to-be-combined bone models comprising a pixel set, and each bone model corresponding to a common skeleton. The processing circuitry is further configured to determine, for any two bone models among the plurality of bone models, a set of common bones that is an intersection of bones included in both of the two bone models. The processing circuitry is further configured to determine, for all the common bones in the intersection, common bone chains, generate common bone chain duplicates on the basis of the common bone chains, and bind the common bone chain duplicates at upper common bones of the common bone chains. The processing circuitry is further configured to build a mapping between each of the common bone chains and a pixel set included in a corresponding bone model of the two bone models, and build a mapping between each of the common bone chain duplicates bound at the upper common bones and a pixel set included in a corresponding bone model of the two bone models.
In an embodiment, a non-transitory computer-readable storage medium stores computer-readable instructions thereon, which, when executed by processing circuitry, cause the processing circuitry to perform a bone model combination method. The method includes obtaining a plurality of to-be-combined bone models, each of the plurality of to-be-combined bone models comprising a pixel set, and each bone model corresponding to a common skeleton. The method further includes determining, for any two bone models among the plurality of bone models, a set of common bones that is an intersection of bones included in both of the two bone models. The method further includes determining, for all the common bones in the intersection, common bone chains, generating common bone chain duplicates on the basis of the common bone chains, and binding the common bone chain duplicates at upper common bones of the common bone chains. The method further includes building a mapping between each of the common bone chains and a pixel set included in a corresponding bone model of the two bone models, and building a mapping between each of the common bone chain duplicates and a pixel set included in a corresponding bone model of the two bone models.
For any two bone models among a plurality of to-be-combined bone models, the method provided by the embodiments of this disclosure can determine an intersection between common bone sets respectively corresponding to the two bone models. If the intersection includes a common bone, it indicates that the two bone models have a bone conflict. Common bone chain duplication and binding can be performed on these bone models having the bone conflicts. Specifically, for all common bones in the intersection, final common bone chains are determined, so that final common bone chain duplicates are generated on the basis of the final common bone chains, and the final common bone chain duplicates are bound to upper common bones of the final common bone chains to complete the duplication and binding of the common bone chains. Afterwards, a mapping between the final common bone chains and a pixel set included a corresponding bone model of the two bone models is built, and a mapping between the final common bone chain duplicates bound with the upper common bones and a pixel set included in a corresponding bone model of the two bone models is built, so as to combine the bone models having the bone conflicts. By the method of the embodiments of this disclosure, bone models can be efficiently and accurately combined in a case that the plurality of to-be-combined bone models have the bone conflicts, so that all bone models corresponding to the same common bone can be integrally maintained, and all corresponding bone animations can be achieved, which avoids bone model loss and animation playing errors due to the bone conflicts, so as to achieve better picture animation displaying to improve the user experience. The method of the embodiments of this disclosure is applicable to combining complicated bone models and multi-character bone models, and is decoupled from a game engine and tool. In addition, the method can also optimize the rendering performance and reduce the complexity of bone model combination.
In order to describe the technical solutions of the embodiments of this disclosure more clearly, the following briefly describes the accompanying drawings describing the embodiments. The accompanying drawings in the following description show only some embodiments of this disclosure, and a person of ordinary skill in the art may still derive other drawings from these accompanying drawings.
In order to make the objectives, technical solutions, and advantages of this disclosure clearer, the following will describe in detail the exemplary embodiments of this disclosure with reference to the accompanying drawings. The described embodiments are merely some but not all of the embodiments of this disclosure. It should be understood that, this disclosure is not limited by the exemplary embodiments described herein.
In this specification and the accompanying drawings, steps and elements that are essentially the same or similar are represented by identical or similar reference numerals, and repeated descriptions of these steps and elements will be omitted. In addition, in the descriptions of this disclosure, the terms “first”, “second”, and the like are only for the purpose of distinguishing the description, and may not be understood as indicating or implying the relative importance or a sequence.
Unless otherwise defined, meanings of all technical and scientific terms used in this specification are the same as those usually understood by a person skilled in the art to which this disclosure belongs. Terms used herein are merely intended to describe objectives of the embodiments of this disclosure, but are not intended to limit this disclosure.
For the convenience of describing this disclosure, the following introduces concepts related to this disclosure.
A bone model combination method of this disclosure can be achieved on the basis of artificial intelligence (AI). AI involves a theory, a method, a technology, and an application system that use a digital computer or a machine controlled by the digital computer to simulate, extend, and expand human intelligence, perceive an environment, obtain knowledge, and use knowledge to obtain an optimal result. In other words, AI is a comprehensive technology in computer science and attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. For example, an artificial intelligence-based bone model combination method can establish association between a plurality of to-be-combined bone models by similar recognition with human eyes and determining their relationships, thereby combining the plurality of bone models. AI studies the design principles and implementation methods of various intelligent machines, so that the bone model combination method of this disclosure has functions of accurately determining conflicting bones between a plurality of to-be-combined bone models in real time and performing bone duplication and mapping on the basis of the conflicting bones, thereby efficiently combining the plurality of bone models having bone conflicts.
The bone model combination method of this disclosure can involve a skeleton-driven deformation technology. The skeleton-driven deformation technology is also referred to as a skeleton-driven mesh deformation method. As an important free-form deformation method, the skeleton-driven deformation technology has a main idea of comparing deformation of a three-dimensional mesh with motions of animals (including human bodies), associating points (a skin) on a surface of the mesh with a skeleton structure, and mesh deformation is driven by motions of bones. The method is easy and intuitive to operate and is suitable for being applied to motion deformation, and the built skeleton model can be repeatedly used, so it is widely used in fields such as human animation and games. The skeleton-driven deformation technology can generally be considered as a deformation technology that is performed to simulate motions of animals. The essence is to drive the geometric model to deform by editing a skeleton of a geometric model, that is, deformation of a surface region of a mesh model is driven by the deformation of a corresponding constraint bone of the mesh model.
The bone model combination method of this disclosure can also involve a skinning deformation technology. Mesh skin deformation caused by bone motion is usually referred to as skinning deformation, where a surface mesh skin is skinning. When skinning vertexes in a skin deformation animation are only affected by a single bone, unnatural skin deformation distortion may occur. To alleviate this phenomenon, multiple bones are generally applied simultaneously to vertexes of a certain mesh skin, which is to bind the mesh skin to the multiple bones to comprehensively generate deformation of the mesh skin. There are various skinning deformation methods that associate multiple bones with a mesh skin, among which, common skinning deformation methods include a Linear Blending Skinning (LBS) deformation method. The LBS deformation method establishes local joint operators for a human skeleton to drive local bone motion to cause local deformation of a mesh skin. Deformation is generated by linear blending of vertexes of the skin and a plurality of associated bones. This method is also commonly referred to as Joint-dependent Local Deformation (OLD) or skeleton subspace deformation. Specifically, in the LBS method, the vertexes of the mesh skin are blended with the bones bound to the vertexes; a weight is assigned to a contribution degree of each bone to the deformation of the vertexes of the skin; and the weight of each bone that affects the skin is linearly blended to obtain a more realistic skinning deformation effect. It is understood that this disclosure only takes the LBS method mentioned above as an example and not as a limitation to perform skinning deformation. Therefore, other skinning deformation algorithms that can achieve similar effects can also be applied to the bone model combination method of this disclosure.
In summary, schemes provided by the embodiments of this disclosure involve technologies such as the AI, the skeleton-driven deformation, and the skinning deformation. It is noted that, the bone model combination method provided in the embodiments of this disclosure can be executed by a bone model combination device. The bone model combination device can be a server or a user terminal. The bone model combination method provided in the embodiments of this disclosure will be further described in conjunction with the accompanying drawings.
In a user terminal that presents two-dimensional or three-dimensional animations to a user, the user can obtain a desired effect by operating displayed pictures on the user terminal. For example, for an online game installed on the user terminal, the user can control game pictures through interaction with the user terminal (such as touch control or peripheral device control), and initiate a combination request for a plurality of bone models by, for example, selecting bone models in the game pictures (for example, one or more of models in a role item replacement interface as shown in
In one possible implementation, the user terminal may specifically include a smart phone, a tablet, a laptop, a vehicle-mounted terminal, a wearable device, and the like. The user terminal may also be a client with a browser or various applications (including a system application and a third-party application). The network may be an Internet of Things based on the Internet and/or telecommunication network. It can be a wired network or a wireless network, for example, an electronic network that can achieve an information exchange function, such as a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), and a cellular data communication network. The server may be an independent physical server, or may be a server cluster or a distributed system formed by a plurality of physical servers, or may be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, Content Delivery Networks (CDNs), big data, and artificial intelligence platforms.
As shown in
The above processing performed on the combination request by the server is usually based on the skeleton-driven deformation technology, which is a deformation technology that simulates a motion way of animals. That is, the deformation of a surface mesh of a bone model is driven by the deformation of a corresponding constraint bone of the bone model, and the motion simulation or deformation design of a bone model is achieved by applying the skeleton-driven deformation. Combination of bone models can include combination of bones and meshes included in the bone models, and respective bone animations of the plurality of combined bone models also need to be displayed on the combined bone models.
However, in an animation system that uses the skeleton-driven deformation technology to achieve mesh deformation driven by bones, for the same part in a common skeleton, multiple bone models may simultaneously exist, which can be mapped to the part of the common skeleton, and these bone models may separately correspond to multiple different bone animations to be presented simultaneously. Usually, during combination bone model sets collected from a scenario, it is necessary to bind each bone model in the bone model sets to a common skeleton separately. For the above multiple-to-one situation, during processing of the multiple bone models mapped to the same common bone in the common skeleton, it is usually necessary to bind the multiple bone models to the common bone. It usually leads to bone conflicts, causing some bone models to be lost or animations to be played in error, resulting in abnormal pictures.
As an example, for a role item replacement scenario as shown in
As shown in
Therefore, there is a need for a method that can achieve correct combination of a plurality of bone models in the presence of bone conflicts. Based on this, this disclosure provides a bone model combination method, which solves the combination conflict caused by the bone conflicts by determining overlapping parts between bone models having bone conflicts and performing corresponding processing on the overlapping parts.
Compared with the bone model combination method provided in the related technology, the method provided in the embodiments of this disclosure can integrally maintain all bone models corresponding to the same common bone and achieve all corresponding bone animations, which avoids bone model loss and animation playing errors due to the bone conflicts.
The method provided by the embodiments of this disclosure determines several bone models corresponding to a same common bone from a plurality of to-be-combined bone models, and duplicates and binds a common bone chain for the bone models to build mappings respectively between pixel sets included in bone models having bone conflicts and corresponding common bones in a common skeleton, thereby achieving combination of a plurality of bone models having bone conflicts. By the method of the embodiments of this disclosure, bone models can be efficiently and accurately combined in a case that the plurality of to-be-combined bone models have the bone conflicts, so as to achieve better picture animation displaying to improve the user experience. The method of the embodiments of this disclosure is applicable to combining complicated bone models and multi-character bone models, and is decoupled from a game engine and tool. In addition, the method can also optimize the rendering performance and reduce the complexity of bone model combination.
As shown in
As shown in
In step 201, a plurality of to-be-combined bone models may be obtained, each of the plurality of to-be-combined bone models may include a pixel set, and each bone model corresponds to a common bone set of a common skeleton. For example, a plurality of to-be-combined bone models are obtained. Each of the plurality of to-be-combined bone models includes a pixel set, and each bone model corresponds to a common skeleton.
In one possible implementation, the execution of step 201 corresponds to the bone model collection and bone model filtering mentioned above. For example, the obtaining of the plurality of to-be-combined bone models may be achieved by a user by performing a selection operation in a user terminal, while unselected bone models are correspondingly filtered. In addition, for clarity, the scenario description construction section in the initialization operation will be described below with reference to
According to the embodiments of this disclosure, the common skeleton may be composed of a plurality of common skeletons, and there is a hierarchical relationship between the plurality of common skeletons.
As mentioned above, the plurality of bone models may be combined on the basis of the common skeleton. As an example,
For example, in the figure, the pelvis is used as the common bone root node, and nodes located at a lower layer of the root node may include three nodes: the body, the left thigh, and the right thigh. These nodes may each include corresponding lower nodes and further lower nodes. Therefore, in the case of the aforementioned example, the common skeleton may include common bone chains, such as pelvis-left thigh-left calf-left foot, pelvis-body-head, and pelvis-body-right upper arm-right lower arm-right hand. It is understood that the definition and selection of root node and its lower nodes may be set according to practical applications and needs. The form of the common skeleton used in this disclosure is only used as an example and not a limitation. Other common skeletons that include more or fewer bones, as well as those common skeletons based on different hierarchical division methods, can also be applied to the bone model combination method of this disclosure.
In one possible implementation, the hierarchical relationship between the plurality of common bones may be based on a motion manner of hierarchical bones between these common bones. Each bone in the same common bone chain may be represented as an abstract representation of a transformation matrix, that is, relative transformation of the bone relative to an upper common bone (for example, motion of the shoulders may drive the arms, the fingers, and the like to move, so transformation of the arms and fingers may be represented as transformation of the arms and fingers relative to the upper common bone). Therefore, calculation of transformation of a lower common bone may be converted into recursive calculation of the transformation of the upper common bones based on hierarchical relationship (namely, from local transformation to global transformation).
In addition, due to the hierarchical relationship between the bones, if an intermediate layer of bone between two layers of bones is lost, the motion of the lower bone may be disordered. Therefore, in subsequent bone model combination processing, subsequent bone model combination processing is performed on the basis of a common bone chain, instead of a single bone.
According to the embodiments of this disclosure, each of the plurality of bone models further includes a bone set. The bone model may correspond to a common bone set of the common skeleton on the basis of the bone set of the bone model.
A skeleton of the example bone models shown in
According to the embodiments of this disclosure, each pixel in one pixel set includes a plurality of vertexes, and each vertex has a bone indicator; and the bone indicator is used for indicating a bone identifier corresponding to the vertex. In one possible implementation, each bone model may include both a bone part and a mesh part, where a bone and a mesh can have mapping relationships based on the correspondence relationship between the bone model and the common skeleton. The bone can drive the mesh to change through motion transformation, and the mesh is a pixel set used for expressing objects in a three-dimensional space.
In one possible implementation, the above-mentioned pixel set may be a set of pixels having a mapping relationship with the common bone set corresponding to the bone model in the common skeleton. The bone identifiers of the vertexes of each pixel in the pixel set may point to bones related to one common bone in the common bone set. A specific description may refer to step 205 described below.
In one possible implementation, the foregoing influence of one or more bones associated with the pixel on the vertexes of the pixel may be as shown in
As mentioned above, when the vertexes of the pixel are only affected by a single bone, unnatural pixel deformation distortion may occur. In order to relieve this phenomenon, a plurality of bones may be usually used to act on one vertex, that is, the pixel is associated with a plurality of bones to jointly generate more natural pixel deformation, thereby generating a more realistic bone animation.
In one possible implementation, the vertexes of each pixel may be blended with one or more bones associated with the pixel. For example, for the vertexes of each pixel of the pixel set, the bones associated with the vertexes are obtained, and influence weight values of the bones associated with the vertexes on the vertexes may be determined. Then, the influence weight value of each bone on the vertex may be blended to determine a vertex after transformation. The method for determining the influence weight values of the bones associated with the vertexes on the vertexes may include determining a contribution degree of each bone to deformation of the vertex as the influence weight value of the bone on the vertex. The blending of the influence weight value of each bone on the vertex may be linear blending, and the embodiments of this disclosure do not limit this.
For example, in a case of the above linear blending, it is assumed that a global transformation matrix of bone j in a bone model is Bj, and vertex vi is transformed due to the influence of a plurality of bones, the vertex vi′ after transformation may be represented as vi′=Σj wijBjvi; and Σj wij=1, where wij is the influence weight of bone j on current vertex i. Of course, the aforementioned linear blending representation is only used as an example and not a limitation in this disclosure. This example is only used for explaining the concept of the influence weight value and is not intended to limit skinning deformation to linear blending skinning.
As shown in
In step 202, for any two bone models among the plurality of bone models, an intersection of common bone sets respectively corresponding to the two bone models is determined. For example, for any two bone models among the plurality of bone models, a set of common bones is determined that is an intersection of bones included in both of the two bone models.
The intersection of the two common bone sets corresponding to the above two bone models may indicate overlapping bones (namely, conflicting bones) of the two bone models relative to the common skeleton. These overlapping bones will cause a combination conflict under a related method. Therefore, in a case that the intersection of the two common bone sets corresponding to the two bone models is not an empty set (that is, there is at least one overlapping bone), bone model combination can be performed for the intersection.
In one possible implementation, for any two bone models (Mesha, Meshb) among the plurality of bone models, assuming that there is common bone set Skel of the common skeleton, the two common bone sets corresponding to the two bone models are SkelA and SkelB, respectively, where SkelA⊆Skel and SkelB⊆Skel; and combination of the two bone models may be regarded as building a mapping from Mesha to SkelA and a mapping from Meshb to SkelB. In one possible implementation, the intersection of SkelA and SkelB may be represented as set S, that is, S=SkelA∩SkelB.
Therefore, in one possible implementation, if S=Ø, it indicates that there is no bone conflict between the two bone models (Mesha, Meshb), so the two bone models may be combined normally, for example, a next stage (for example, a mesh combination stage in
In a case of S≠Ø, the bone model combination may be performed on set S. The combination operation may include conflicting bone chain duplication, redundant bone culling, and animation information repair as shown in
In step 203, in a case that the intersection includes common bones, final common bone chains are determined for all the common bones in the intersection. For example, for all the common bones in the intersection, common bone chains are determined.
In one possible implementation, for set S, considering that set S corresponds to a set of overlapping bones of the aforementioned two bone models relative to the common skeleton, it is necessary to determine a final common bone chain for each overlapping bone one by one, and each final common bone chain includes at least one overlapping bone.
According to the embodiments of this disclosure, step 203 may include: in a case that the intersection includes common bones, determining, for all the common bones in the intersection, the final common bone chains on the basis of the hierarchical relationship between all the common bones.
In one possible implementation, all the common skeletons in the final common bone chains are ordered and sequenced according to the above hierarchical relationship of the common skeleton. For example, serial numbers of layers corresponding to the common bones in one final common bone chain from top to bottom is from large to small.
In one possible implementation, for all the common bones in the above intersection, a to-be-confirmed common bone chain may be determined for each common bone, and then a to-be-confirmed common bone chain set that can cover all the common bones in the intersection and has a smallest quantity of to-be-confirmed common bone chains is selected as the above final common bone chains from all the determined to-be-confirmed common bone chains.
According to the embodiments of this disclosure, the determining, for the all the common bones in the intersection, the final common bone chain on the basis of the hierarchical relationship between all the common bones may include: determining a common bone chain set that can cover all the common bones and has a smallest quantity of common bone chains, so as to determine the final common bone chains according to the common bone chain set, where the same final common bone chain does not contain two or more common bones belonging to the same layer.
In one possible implementation, one to-be-confirmed common bone chain may be determined for each of all the common bones, and the above final common bone chains are determined according to all the determined to-be-confirmed common bone chains. For example, the above to-be-confirmed common bone chain determined for each common bone may be a common bone chain with any length, which takes the common bone as a start point (namely, the common bone is used as the uppermost bone in the common bone chain). The common bone chain includes one or more of lower common bones of the common bone (in a case that there are lower common bones of the common bone). Therefore, there may be intersections among the plurality of determined to-be-confirmed common bone chains. In order to reduce the quantity of the common bone chains in the common bone chain set, it is necessary to combine the to-be-confirmed common bone chains with the intersections to generate the longest common bone chain with the common bones arranged in sequence according to the hierarchical relationship, as one of the final common bone chains mentioned above. If there a plurality of final common bone chains, the common bones included in each common bone chain are arranged in sequence in one direction according to the hierarchical relationship (for example, a gradual increasing or decreasing hierarchical relationship), that is, the common bone chains do not have branches.
In addition, in order to achieve the same purpose as the above method, the determined common bone chains can also be of a tree-like structure different from the above method, that is, each common bone chain may include two or more common bones on the same layer.
According to the embodiments of this disclosure, the determining, for the all the common bones in the intersection, the final common bone chain on the basis of the hierarchical relationship between all the common bones may include: taking, for each common bone among all the common bones, the common bone and all lower common bones corresponding to the common bone in the common skeleton as a to-be-confirmed common bone chain; and for any two to-be-confirmed common bone chains among all the to-be-confirmed common bone chains corresponding to all the common bones, removing, in a case that the two to-be-confirmed common bone chains have an inclusion relationship, a to-be-confirmed common bone chain with fewer common bones from the two to-be-confirmed common bone chains to obtain the final common bone chain. According to the embodiments of this disclosure, the upper common bones and the lower common bones of the final common bone chains are determined on the basis of the hierarchical relationship.
In one possible implementation, the to-be-confirmed common bone chain determined for each common bone may be a common bone chain which takes the common bone as a start point and includes all the lower common bones of the common bone (for example, when the common bone includes a plurality of lower common bones belonging to the same layer, the common bone chain may be of a tree-like structure). Therefore, one to-be-confirmed common bone chain may be determined for each common bone in the intersection. According to the formation of the common bone chains mentioned above, the to-be-confirmed common bone chains corresponding to the common bones in the intersection may overlap (that is, having an inclusion relationship). For example, for the common skeleton shown in
As an example, in a case of S=SkelA∩SkelB≠0 described above, the common skeleton may be regarded as a data structure similar to a tree, and S may be regarded as a sub-node set of the tree. Therefore, n subtrees (n to-be-confirmed common bone chains) are determined according to all nodes (for example, n nodes) in S described above, so as to form to-be-confirmed subtree set T′. For any two subtrees A and B in to-be-confirmed subtree set T′, if A and B have an inclusion relationship, and A⊆B, subset A is removed from set T′, while superset B is maintained. Therefore, through the above operation, new subtree set T can be generated, that is, a set of the above-mentioned final common bone chains.
It is understood that the above two definitions of the form of the final common bone chain are only used as an example for determining the final common bone chains, not a limitation, in this disclosure. Other forms of final common bone chains to achieve the same effects can also be applicable to the bone model combination method of this disclosure.
In step 204, final common bone chain duplicates are generated on the basis of the final common bone chains, and the final common bone chain duplicates are bound to upper common bones of the final common bone chains. For example, common bone chain duplicates are generated based on the common bone chains, and the common bone chain duplicates are bound at upper common bones of the common bone chains.
As mentioned above, in a case of a bone conflict, two or more bone models correspond to the same bone (a conflicting bone) of the common skeleton. For the conflicting bone, two or more bone animations associated with the two or more bone models need to be performed, but this is hardly achieved for the single conflicting bone. Therefore, bone duplicates which are associated with the conflicting bone and do not affect each other. Bone animations can be performed on these bone duplicates. These bone duplicates may correspond to the same common bone, and are jointly bound to upper common bones of the common bone, so that these bone duplicates can be affected by transformation of the upper common bones, and the transformation of the upper common bones can be recursively calculated according to the hierarchical relationship, so as to obtain transformation of these bone duplicates.
According to the embodiments of this disclosure, the generating final common bone chain duplicates on the basis of the final common bone chains includes: duplicating all the common bones in the final common bone chains and the hierarchical relationship of the common bones to obtain the final common bone chain duplicates, where each common bone in the final common bone chain duplicates and each common bone in the final common bone chains have different bone identifiers.
In one possible implementation, the final common bone chain duplicates may include common bone duplicates that correspond to the original common bones but have different bone identifiers, and arrangement (namely, the hierarchical relationship) of these common bone duplicates in the final common bone chain duplicates is consistent with arrangement of the corresponding original common bones in the original common bone chains.
For example, for each subtree Ti in subtree set T mentioned above, a parent node (denoted as Fi) of the subtree is determined, and subtree Ti is duplicated to obtain a subtree duplicate Ti′. Therefore, the subtree duplicate Ti′ can be bound to parent node Fi, thereby generating a new common skeleton corresponding to the to-be-combined bone models, so as to map these bone models to the new common skeleton.
In addition, according to the embodiments of this disclosure, in some cases, the final common bone chain may not have an upper common bone. In this case, in order to achieve the binding operation, in a case that the final common bone chain does not have an upper common bone, a temporary upper common bone can also be established first. In this case, the method for binding the final common bone chain duplicate to the upper common bone of the final common bone chain may be binding the final common bone chain and the final common bone chain duplicate to the temporary upper common bone, respectively.
Due to the hierarchical relationship in the motion transformation between bones and the direct influence of upper bones on motions of lower bones, in a case that there are no upper common bones for a common bone, the common bone and its common bone duplicate cannot be associated. Therefore, in one possible implementation, a temporary upper common bone of the common bone can be established as an association node between the common bone and its common bone duplicate.
For example, the intersection of the two common bone sets mentioned above includes the uppermost common bone in the common skeleton (such as the pelvis as shown in
In one possible implementation, a common bone and a common bone duplicate which correspond to a conflicting bone may have different bone identifiers (for example, the final common bone chain {right upper arm-right lower arm-right hand} and the final common bone chain duplicate {right upper arm 2-right lower arm 2-right hand 2} shown in
In step 205, a mapping between the final common bone chains and the pixel set included in a corresponding bone model of the two bone models is built, and a mapping between the final common bone chain duplicates bound with the upper common bones and the pixel set included in a corresponding bone model of the two bone models is built, so as to combine the two bone models. For example, a mapping is built between each of the common bone chains and a pixel set included in a corresponding bone model of the two bone models, and a mapping is built between each of the common bone chain duplicates bound at the upper common bones and a pixel set included in a corresponding bone model of the two bone models.
In one possible implementation, after the duplication and binding of the final common bone chains corresponding to the conflicting bone are completed, in order to display the bone animations of the respective bone models normally, the pixel sets of the respective bone models may be associated with the generated new common skeleton (including the final common bone chains and the final common bone chain duplicates), that is, the pixel sets of the respective bone models are mapped to the final common bone chains or the final common bone chain duplicate of the new common skeleton.
In one possible implementation, the bones in the bone sets included in the bone models may have different bone identifiers, so that each pixel in the pixel sets of these bone models may be mapped to specific bones through the specific bone identifiers. As mentioned above, the common bones and the common bone duplicates can also have different bone identifiers, and there may be correspondence relationships between the bone identifiers of both the common bones and the common bone duplicates and the bone identifiers of the bones in the bone sets of the bone models, so that after the duplication of a conflicting bone chain is completed, the pixel sets of the respective bone models may be mapped to corresponding common bones or common bone duplicates on the basis of the correspondence relationships.
According to the embodiments of this disclosure, step 205 may include: for each pixel in the pixel set of one of the two bone models, mapping the bone indicator of each vertex of the pixel to a corresponding common bone in a corresponding common bone chain among the final common bone chains, and mapping the bone indicator of each vertex of the pixel to a corresponding common bone in a corresponding common bone chain among the final common bone chain duplicates.
In one possible implementation, each vertex of a pixel may have a bone indicator and weight attribute, where the bone indicator may be used for indicating the bone identifier of the bone corresponding to the vertex, while the weight attribute may indicate the influence weight of the deformations of the respective bones on the vertex (for example, deformation of the head may not have any influence on a vertex of a pixel of the left foot, so that the influence weight value of a head bone on the vertex of the pixel of the left foot may be zero), with reference to
As mentioned above, the pixel sets of the respective bone models are mapped to the final common bone chains or the final common bone chain duplicates, which may include: modifying the bone indicator of each vertex of each pixel in the pixel sets of the respective bone models to be the bone identifier of the corresponding common bone in the corresponding common bone chains or the common bone chain duplicates. It may be regarded as common bone or common bone duplicate skinning, so that the transformation of the common bone leads to the deformation of the vertex of the pixel.
In addition, the weight attributes of these vertexes may also be modified accordingly, including: redetermining weight attributes for the vertexes of the pixels corresponding to the common bone duplicates in the final common bone chain duplicates, such as duplicating the weight attributes of the vertexes of the pixels mapped to the original common bones corresponding to the final common bone duplicates, so that the bone animations of the respective bone models can be presented by the unique common bone or common bone duplicate corresponding to the bone models, thereby repairing animation information in the bone information reconstruction stage shown in
In the above-mentioned process of duplicating the conflicting bone chain, the newly generated common skeleton may include some redundant bones on the basis of the method for determining the final common bone chains. For example, in a case that the determined final common bone chains are of the tree-like structure as described above, the to-be-confirmed common bone chain determined for each common bone may be a common bone chain that takes the common bone as a start point and includes all the lower common bones of the common bone. Although some of the common bone chains have been removed for the case where the inclusion relationship exists, since the to-be-confirmed common bone chain determined for each common bone includes the common bone and all the lower common bones of the common bone, there may be a redundant part at a tail end of the to-be-confirmed common bone chain. As a result, the determined common bone chain and the common bone chain duplicate have redundant parts.
For example, for the common skeleton as shown in
In addition to step 201 to step 205 described above, the bone model combination method of this disclosure may also include step 206 and step 207.
In step 206, in a case that a redundant bone exists in each final common bone chain duplicate, the redundant bone is removed from the final common bone chain duplicate. The redundant bone includes a redundant common bone; the redundant common bone is located in the final common bone chain duplicate; the redundant common bone does not belong to a common bone set corresponding to the bone model, mapped to the final common bone chain duplicate, in the two bone models; and in a case that the redundant common bone has a lower common bone, the lower common bone of the redundant common bone does not belong to the common bone set either.
For example, in a case that the intersection of the two common bone sets includes the body and the right lower arm, the generated original common bone chain duplicate may include the body and all the lower common bones of the body, that is, {body, left upper arm, head, right upper arm, left lower arm, right lower arm, left hand, and right hand}. Assuming that the common bone set corresponding to the bone model, mapped to the final common bone chain duplicates, of the two bone models is {body, right lower arm, left lower arm}, the redundant bone may include a redundant common bone. The redundant common bone is located in the final common bone chain duplicates and does not belong to the common bone set. There are no lower common bones (such as the head, the left hand, and the right hand). The lower common bones of the left upper arm and the right upper arm belong to the common bone set. In order to establish associations between common bones on non-adjacent layers, the left upper arm and the right upper arm are not determined to be redundant bones. Therefore, for the above situation, the head, the left hand, and the right hand may be removed from the final common bone chain duplicate, thereby generating a new common bone chain duplicate {body, left upper arm, right upper arm, left lower arm, right lower arm} subjected to redundant bone culling processing.
By performing the redundant bone culling processing on the generated common skeleton, the complexity of the subsequent calculation can be reduced, and the computing resources are saved.
As shown in
Therefore, in step 207, the pixel sets respectively included in the plurality of bone models may be combined to combine the plurality of bone models. The pixel sets respectively included in the plurality of bone models are combined, which may include: combining all the vertexes of each pixel in the pixel sets respectively included in the plurality of bone models.
In one possible implementation, the combining all the vertexes of each pixel in the pixel sets respectively included in the plurality of bone models, which may include: combining various attributes of all the vertexes, including but not limited to vertex quantity combination described as an example below, that is, vertex index reconstruction of the mesh combination stage of
For example, assuming that there are N to-be-combined bone models (respectively denoted as Mesh1, . . . , MeshN), a vertex quantity respectively corresponding to these bone models is C1, . . . , CN, where a vertex set of each bone model Meshn may be denoted as In=0, . . . , Cn. In addition, assuming that card(P) represents a function used for obtaining a quantity of elements of finite set P, there is card(In)=Cn+1.
It is assumed that a vertex set operation function for vertex set P is as follows:
Accu(P,c)={x0+c,x1+c, . . . ,xn+c} (1)
where (x0, . . . , xn)∈P, n=card(P)−1, so that operation function Accu(P, c) represents adding numerical values of all the elements in vertex set P to c.
In addition, it can also be assumed that a function used for determining a quantity sum of elements of a vertex set in front of vertex set In.
As shown in the above formula (2), PreS(I, 0) and PreS(I, 1) are both 0 because there is no vertex set in front of vertex set I1. From vertex set I2, PreS(I, n) represents a total quantity of elements of vertex sets in front of an nth vertex set.
Therefore, a vertex combination operation performed on n vertex sets I1, . . . , In can be denoted as:
The above formula (3) is a vertex set generated after the n vertex sets I1, . . . , In are combined, where an offset can be added to the index of each element in the vertex sets, so that the indexes of vertexes in the generated vertex set are continuous. The offset may be the total quantity of the elements of the vertex sets in front of the vertex set.
Therefore, through the vertex index reconstruction operation mentioned above, the number of rendering invocations to the graphics API can be significantly reduced, thereby reducing the rendering batches and the computing amount and improving the rendering efficiency. In addition, combination of other attributes of pixels can also be regarded similarly as combination of sets.
As shown in
In addition to the above operations, the post-processing line shown in
Afterwards, for the scenario description construction section in
In
As shown in
Therefore, as mentioned above, according to the bone model combination method of this disclosure, based on the bone combination process shown in
According to the embodiments of this disclosure, the bone model combination apparatus 800 may include a bone model obtaining module 801, a bone intersection determining module 802, a common bone chain determining module 803, a common bone chain duplication module 804, and a bone mapping building module 805.
According to the embodiments of this disclosure, the bone model obtaining module 801 may be configured to obtain a plurality of to-be-combined bone models, each of the plurality of to-be-combined bone models including a pixel set, and each bone model corresponding to a common bone set of a common skeleton.
In one possible implementation, the common skeleton may be composed of a plurality of common skeletons, and there is a hierarchical relationship between the plurality of common skeletons. Therefore, the plurality of bone models may be combined on the basis of the common skeleton and the hierarchical relationship thereof.
In one possible implementation, the above-mentioned pixel set may be a set of pixels having a mapping relationship with the common bone set corresponding to the bone model in the common skeleton. The bone identifiers of the vertexes of each pixel in the pixel set may point to bones related to one common bone in the common bone set.
In one possible implementation, the vertexes of each pixel may be blended with one or more bones associated with the pixel. For example, for the vertexes of each pixel of the pixel set, the bones associated with the vertexes are obtained, and influence weight values of the bones associated with the vertexes on the vertexes may be determined. Then, the influence weight value of each bone on the vertex may be blended to determine a vertex after transformation. The method for determining the influence weight values of the bones associated with the vertexes on the vertexes may include determining a contribution degree of each bone to deformation of the vertex as the influence weight value of the bone on the vertex.
According to the embodiments of this disclosure, the bone intersection determining module 802 may be configured to determine, for any two bone models among the plurality of bone models, an intersection of common bone sets respectively corresponding to the two bone models. In one possible implementation, the bone intersection determining module 802 may perform the operation described with reference to step 202.
In one possible implementation, the intersection of the two common bone sets corresponding to the above two bone models may indicate overlapping bones (namely, conflicting bones) of the two bone models relative to the common skeleton, so that these overlapping bones will cause a combination conflict under a related method. Therefore, in a case that the intersection of the two common bone sets corresponding to the two bone models is not an empty set (that is, there is at least one overlapping bone), bone model combination can be performed for the intersection.
According to the embodiments of this disclosure, the common bone chain determining module 803 may be configured to: in a case that the intersection includes common bones, determine, for all the common bones in the intersection, final common bone chains. In one possible implementation, the common bone chain determining module 803 may perform the operation described with reference to step 203.
In addition, as an example, for each common bone among all the common bones, the common bone and all lower common bones corresponding to the common bone in the common skeleton may also be taken as a to-be-confirmed common bone chain. for any two to-be-confirmed common bone chains among all the to-be-confirmed common bone chains corresponding to all the common bones, removing, in a case that the two to-be-confirmed common bone chains have an inclusion relationship, a to-be-confirmed common bone chain with fewer common bones from the two to-be-confirmed common bone chains to obtain the final common bone chains, where the upper common bones and the lower common bones of the final common bone chains are determined on the basis of the hierarchical relationship.
According to the embodiments of this disclosure, the common bone chain duplication module 804 may be configured to: generate final common bone chain duplicates on the basis of the final common bone chains, and bind the final common bone chain duplicates to upper common bones of the final common bone chains. In one possible implementation, the common bone chain duplication module 804 may perform the operation described with reference to step 204.
In one possible implementation, the final common bone chain duplicates may include common bone duplicates that correspond to the original common bones but have different bone identifiers, and arrangement (namely, the hierarchical relationship) of these common bone duplicates in the final common bone chain duplicates is consistent with arrangement of the corresponding original common bones in the original common bone chains.
In one possible implementation, after the duplication and binding of the common bone chains corresponding to the conflicting bone are completed, in order to display the bone animations of the respective bone models normally, the pixel sets of the respective bone models may be associated with the generated new common skeleton (including the final common bone chains and the final common bone chain duplicates), that is, the pixel sets of the respective bone models are mapped to the final common bone chains or the final common bone chain duplicate.
According to the embodiments of this disclosure, the bone mapping building module 805 may be configured to: build a mapping between the final common bone chains and the pixel set included in a corresponding bone model of the two bone models, and build a mapping between the final common bone chain duplicates bound with the upper common bones and the pixel set included in a corresponding bone model of the two bone models, so as to combine the two bone models. In one possible implementation, the bone mapping building module 805 may perform the operation described with reference to step 205.
In one possible implementation, the action of mapping the pixel sets of the respective bone models to the final common bone chains or the final common bone chain duplicates may include: for each pixel in the pixel set of one of the two bone models, mapping the bone indicator of each vertex of the pixel to a corresponding common bone in a corresponding common bone chain among the final common bone chains, and mapping the bone indicator of each vertex of the pixel to a corresponding common bone in a corresponding common bone chain among the final common bone chain duplicates. This can be regarded as common bone or common bone duplicate skinning, so that transformation of the common bone causes deformation of the vertexes of the pixel.
In addition, as described above with regard to the bone model combination method, the bone model combination apparatus of this disclosure can further include a redundant bone culling module 806 and a mesh combination module 807 which are respectively configured to implement step 206 and step 207.
The redundant bone culling module 806 may be configured to: remove, in a case that it is determined that a redundant bone exists in each final common bone chain duplicate, the redundant bone from the final common bone chain duplicate, the redundant bone including a redundant common bone; the redundant common bone being located in the final common bone chain duplicate; the redundant common bone not belonging to a common bone set corresponding to the bone model, mapped to the final common bone chain duplicate, in the two bone models; and in a case that the redundant common bone has a lower common bone, the lower common bone of the redundant common bone not belonging to the common bone set either. By performing the redundant bone culling processing on the generated common skeleton, the complexity of the subsequent calculation can be reduced, and the computing resources are saved.
The mesh combination module 807 may be configured to combine the pixel sets respectively included in the plurality of bone models to combine the plurality of bone models. The pixel sets respectively included in the plurality of bone models are combined, which may include: combining all the vertexes of each pixel in the pixel sets respectively included in the plurality of bone models.
In one possible implementation, the action of combining all the vertexes of each pixel in the pixel sets respectively included in the plurality of bone models, which may include: combining various attributes of all the vertexes. The attributes of the vertexes include vertex indexes, positions, normals, bone indicators, bone weight attributes, and the like.
According to another aspect of this disclosure, a bone model combination device is further provided.
As shown in
The processor (processing circuitry) in the embodiments of this disclosure may be an integrated circuit chip, which has a signal processing capability. The above processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, a discrete gate or transistor logic device, and a discrete hardware component. may implement or perform the methods, steps, and logic block diagrams that are disclosed in the embodiments of this disclosure. The general-purpose processor may be a microprocessor, any processor, or the like. The general-purpose processor may be an X86 architecture or an ARM architecture.
Generally, the various exemplary embodiments of this disclosure can be implemented in hardware or dedicated circuits, software, firmware, logic, or any combination thereof. Some aspects can be implemented in hardware, while others can be implemented in firmware or software that can be executed by controllers, microprocessors, or other computing devices. When various aspects of the embodiments of this disclosure are illustrated or described as block diagrams, flowcharts, or some other graphical representations, it will be understood that the blocks, apparatuses, systems, techniques, or methods described herein can be implemented as non-limiting examples in hardware, software, firmware, a dedicated circuit or logic, general hardware, a controller, other computing devices, or some combinations thereof.
For example, the method or apparatus according to the embodiments of this disclosure can also be implemented with architecture of a computing device 3000 shown in
According to still another aspect of this disclosure, a non-transitory computer-readable storage medium is further provided.
As shown in
The embodiments of this disclosure further provide a computer program product or a computer program. The computer program product or computer program includes computer instructions which are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the bone model combination method according to the embodiments of this disclosure.
Embodiments of this disclosure provide a bone model combination method, apparatus and device, and a computer-readable storage medium.
Compared with the traditional bone model combination method, the method provided in the embodiments of this disclosure can integrally maintain all bone models corresponding to the same common bone and achieve all corresponding bone animations, which avoids bone model loss and animation playing errors due to the bone conflicts.
The method provided by the embodiments of this disclosure determines several bone models corresponding to a same common bone from a plurality of to-be-combined bone models, and duplicates and binds a common bone chain for the bone models to build mappings respectively between pixel sets included in bone models having bone conflicts and corresponding common bones in a common skeleton, thereby achieving combination of a plurality of bone models having bone conflicts. By the method of the embodiments of this disclosure, bone models can be efficiently and accurately combined in a case that the plurality of to-be-combined bone models have the bone conflicts, so as to achieve better picture animation displaying to improve the user experience. The method of the embodiments of this disclosure is applicable to combining complicated bone models and multi-character bone models, and is decoupled from a game engine and tool. In addition, the method can also optimize the rendering performance and reduce the complexity of bone model combination.
The flowcharts and block diagrams in the accompanying drawings illustrate possible system architectures, functions and operations that may be implemented by a system, a method, and a computer program product according to various embodiments of this disclosure. In this regard, each block in a flowchart or a block diagram may represent a module, a program, or a part of a code. The module, the program, or the part of the code includes at least one executable instruction used for implementing specified logic functions. In some implementations used as substitutes, functions annotated in blocks may alternatively occur in a sequence different from that annotated in an accompanying drawing. For example, actually two boxes shown in succession may be performed basically in parallel, and sometimes the two boxes may be performed in a reverse sequence. This is determined by a related function. Each box in a block diagram and/or a flowchart and a combination of boxes in the block diagram and/or the flowchart may be implemented by using a dedicated hardware-based system configured to perform a specified function or operation, or may be implemented by using a combination of dedicated hardware and a computer instruction.
Generally, the various exemplary embodiments of this disclosure can be implemented in hardware or dedicated circuits, software, firmware, logic, or any combination thereof. Some aspects can be implemented in hardware, while others can be implemented in firmware or software that can be executed by controllers, microprocessors, or other computing devices. When various aspects of the embodiments of this disclosure are illustrated or described as block diagrams, flowcharts, or some other graphical representations, it will be understood that the blocks, apparatuses, systems, techniques, or methods described herein can be implemented as non-limiting examples in hardware, software, firmware, a dedicated circuit or logic, general hardware, a controller, other computing devices, or some combinations thereof.
The term module (and other similar terms such as unit, submodule, etc.) in this disclosure may refer to a software module, a hardware module, or a combination thereof. A software module (e.g., computer program) may be developed using a computer programming language. A hardware module may be implemented using processing circuitry and/or memory. Each module can be implemented using one or more processors (or processors and memory). Likewise, a processor (or processors and memory) can be used to implement one or more modules. Moreover, each module can be part of an overall module that includes the functionalities of the module.
The use of “at least one of” in the disclosure is intended to include any one or a combination of the recited elements. For example, references to at least one of A, B, or C; at least one of A, B, and C; at least one of A, B, and/or C; and at least one of A to C are intended to include only A, only B, only C or any combination thereof.
The foregoing disclosure includes some exemplary embodiments of this disclosure which are not intended to limit the scope of this disclosure. Other embodiments shall also fall within the scope of this disclosure.
Number | Date | Country | Kind |
---|---|---|---|
202210353097.9 | Mar 2022 | CN | national |
This application is a continuation of International Application No. PCT/CN2023/074848, filed on Feb. 7, 2023, which claims priority to Chinese Patent Application No. 202210353097.9, entitled “BONE MODEL COMBINATION METHOD, APPARATUS AND DEVICE, AND STORAGE MEDIUM” filed on Mar. 31, 2022. The disclosures of the prior applications are hereby incorporated by reference in their entirety.
Number | Date | Country | |
---|---|---|---|
Parent | PCT/CN2023/074848 | Feb 2023 | US |
Child | 18376151 | US |