Patent Application
20240275830
This application claims the priority benefit of Taiwan application serial no. 112104706, filed on Feb. 10, 2023. The entirety of the above-mentioned patent application is hereby incorporated by reference herein and made a part of this specification.
The disclosure relates to a communication technology; more particularly, the disclosure relates to a multi-user collaboration system and a multi-user collaboration method.
With the rise of the trend of working from home (WFH), more and more people conduct meetings through video conferencing software. The conventional video conferencing software has many disadvantages. For instance, in most cases, only the moderator of the conventional video conferencing software has the right to share slides or images. The regular participants are unable to share images intended to be displayed with other participants. In order to solve said issue, an interactive flat panel (IFP) may be applied to conduct the video conferencing. The IFP allows the receipt of information from each client and is able to share the information with all of the clients in real time.
The clients may apply the IFP to perform multi-user collaboration operations. However, when the amount of data of the shared image is excessively large, the transmission performance or the collaboration performance may be reduced. For instance, when a client shares a three-dimensional (3D) image with another client based on the IFP, the transmission of the 3D image may occupy most of the communication resources and cause a delay in the network between each client and the IFP.
The information disclosed in this Background section is only for enhancement of understanding of the background of the described technology and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Further, the information disclosed in the Background section does not mean that one or more problems to be resolved by one or more embodiments of the disclosure was acknowledged by a person of ordinary skill in the art.
One or more embodiments of the disclosure provide a multi-user collaboration system and a multi-user collaboration method capable of providing a multi-user collaboration mechanism on a three-dimensional object for a plurality of terminal devices.
Other objectives and advantages of the disclosure may be further understood from the technical features disclosed in the disclosure.
In order to achieve one, some, or all of the aforementioned objectives or other objectives, an embodiment of the disclosure provides a multi-user collaboration system that includes a first terminal device and a second terminal device. The at least one of the first terminal device and the second terminal device at least comprises an input device, a storage unit and a processing unit. The input device receives image information. The image information comprises grid information and dimensional mapping information of a three-dimensional (3D) object. The storage unit stores at least one texture of the 3D object. The processing unit selects the at least one texture of the 3D object stored in the storage unit. The at least one of the first terminal device and the second terminal device displays an image comprising the 3D object according to the image information and the at least one texture.
In an embodiment of the disclosure, the multi-user collaboration system further comprises a server. The server communicates with the first terminal device and the second terminal device and provides the image information to the at least one of the first terminal device and the second terminal device. The server comprises a texture database. The server receives the 3D object. The server determines whether the texture database comprises the at least one texture. In response to determining that the texture database does not comprise the at least one texture, the server transmits the at least one texture of the received 3D object to the at least one of the first terminal device and the second terminal device. The server receives a first operation command on the 3D object from the second terminal device. The server updates the grid information and the dimensional mapping information according to the first operation command, and transmits the updated grid information and the updated dimensional mapping information to the first terminal device. The first terminal device updates the 3D object in the image according to the updated grid information and the updated dimensional mapping information.
In an embodiment of the disclosure, the server transmits the at least one texture to the first terminal device before providing the image information to the first terminal device.
In an embodiment of the disclosure, the multi-user collaboration system further includes a third terminal device communicating with the server. The server receives a first operation command on the 3D object from the second terminal device and receives a second operation command on the 3D object from the third terminal device. In response to a first authority level of the second terminal device being higher than a second authority level of the third terminal device, the server enables the first operation command and disables the second operation command.
In an embodiment of the disclosure, the server disables the second operation command during a predetermined time slot after receiving the first operation command.
In an embodiment of the disclosure, the first operation command includes at least one of following commands: a clicking command, a rotating command, a zooming command, a moving command, a dividing command, and a combining command.
In an embodiment of the disclosure, the server is further configured to obtain a first image layer and determine whether the first image layer includes a dynamic object. In response to determining that the first image layer includes the dynamic object, the server crops a part of the dynamic object from the first image layer. According to the first operation command, the server performs an image processing operation on the part of the dynamic object to update the grid information and the dimensional mapping information.
In an embodiment of the disclosure, the dynamic object includes the 3D object.
In an embodiment of the disclosure, the server is further configured to obtain a second image layer and a third image layer. The server determines whether each of the second image layer and the third image layer includes a dynamic object. In response to determining that neither the second image layer nor the third image layer includes the dynamic object. The server overlaps the second image layer and the third image layer to generate a fourth image layer. The server transmits the fourth image layer to the first terminal device according to the image information.
In an embodiment of the disclosure, the server inputs the 3D object to a machine learning model to generate the at least one texture, the grid information, the dimensional mapping information, and a classification label of the 3D object.
In an embodiment of the disclosure, the server receives a query command. In response to the query command matching the classification label, the server outputs a query response corresponding to the classification label. Here, the query response includes an option corresponding to the 3D object. In response to the option being selected, the server generates the image information according to the 3D object.
In order to achieve one, some, or all of the aforementioned objectives or other objectives, another embodiment of the disclosure provides a multi-user collaboration method adapted for a multi-user collaboration system. The multi-user collaboration system comprises a first terminal device and a second terminal device. The multi-user collaboration method that includes following steps: receiving an image information by at least one of the first terminal device and the second terminal device, wherein the image information comprises grid information and dimensional mapping information of a 3D object; selecting, by the at least one of the first terminal device and the second terminal device, at least one texture of the 3D object stored in the at least one of the first terminal device and the second terminal device; and displaying, by the at least one of the first terminal device and the second terminal device, an image comprising the 3D object according to the image information and the at least one texture.
In view of the above, the server of the multi-user collaboration system provided in one or more embodiments of the disclosure may transmit the texture of the 3D object to each terminal device in the multi-user collaboration system in advance. When the multi-user collaboration task starts to be performed on the 3D object, the server is no longer required to transmit the texture of the 3D object to each terminal device but required to transmit the grid information and the dimensional mapping information of the 3D object to each terminal device, whereby the 3D object displayed by each terminal device may be updated.
Other objectives, features and advantages of the disclosure will be further understood from the further technological features disclosed by the embodiments of the disclosure wherein there are shown and described preferred embodiments of this disclosure, simply by way of illustration of modes best suited to carry out the disclosure.
The accompanying drawings are included to provide a further understanding of the disclosure, and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments of the disclosure and, together with the description, serve to explain the principles of the disclosure.
It is to be understood that other embodiment may be utilized and structural changes may be made without departing from the scope of the disclosure. Also, it is to be understood that the phraseology and terminology used herein are for the purpose of description and should not be regarded as limiting. The use of “including,” “comprising,” or “having” and variations thereof herein is meant to encompass the items listed thereafter and equivalents thereof as well as additional items. Unless limited otherwise, the terms “connected,” “coupled,” and “mounted,” and variations thereof herein are used broadly and encompass direct and indirect connections, couplings, and mountings.
The server 500 is, for instance, a cloud server. The server 500 may be implemented in form of one single computation device or a plurality of computation devices. The server 500 may have a processing unit (e.g., including but not limited to at least one processor. The processor may be a central processing unit (CPU), or other programmable general-purpose or special-purpose micro control unit (MCU), microprocessor, digital signal processor (DSP), programmable controller, application specific integrated circuit (ASIC), graphics processing unit (GPU), image signal processor (ISP), image processing unit (IPU), arithmetic logic unit (ALU), complex programmable logic device (CPLD), field programmable gate array (FPGA), or other similar elements, or a combination of the elements thereof.), a communication unit (e.g., including but not limited to various communication chips, mobile communication chips, Bluetooth chips, Wi-Fi chips, and so on), and a storage unit (e.g., including but not limited to a movable random access memory, a flash memory, a ROM, a PROM, a EPROM and a EEPROM memory, a dynamic random-access memory (DRAM), a static random-access memory (SRAM), a hard disk, and so on), and other necessary components for running the server 500. In an embodiment, the storage unit of the server 500 may store a texture database 510 and a machine learning model 520. The machine learning model 520 includes at least one program that can find patterns or make decisions from a previously unseen dataset.
The terminal device 100 (or the terminal device 200/the terminal device 300) is, for instance, IFP (for example a display device with touch panel for interactive operation), a personal computer, a notebook computer, a smart phone, or a tablet computer. The terminal device 100 (or the terminal device 200/the terminal device 300) may have a processing unit (e.g., including but not limited to at least one processor. The processor may be a central processing unit (CPU), or other programmable general-purpose or special-purpose micro control unit (MCU), microprocessor, digital signal processor (DSP), programmable controller, application specific integrated circuit (ASIC), graphics processing unit (GPU), image signal processor (ISP), image processing unit (IPU), arithmetic logic unit (ALU), complex programmable logic device (CPLD), field programmable gate array (FPGA), or other similar elements, or a combination of the elements thereof.), a communication unit (e.g., including but not limited to various communication chips, mobile communication chips, Bluetooth chips, Wi-Fi chips, and so on), and a storage unit (e.g., including but not limited to a and a movable random access memory, a flash memory, a ROM, a PROM, a EPROM and a EEPROM memory, a dynamic random-access memory (DRAM), a static random-access memory (SRAM), a hard disk, and so on), and other necessary components for running the terminal device 100 (or the terminal device 200/the terminal device 300). In an embodiment, the terminal device 100 (or the terminal device 200/the terminal device 300) may include an input/output device which comprises an input device and output device. The input device is configured to receive information, the output device is configured to output information. The input device may include at least one chip or circuit electrically connected with the communication unit for receiving image information, a keyboard, a mouse, a touch screen, or the like. The output device may include a display or a projector communicatively connected with the terminal device or the like. The terminal device 100, the terminal device 200, and the terminal device 300 may have the same hardware configuration and function above.
The texture database 510 of the server 500 may store various textures of a 3D object. In 3D tech field, the person skilled in the art understands definition regarding to texture. The texture is a data parameter, such as a texture identification code, albedo, metal, glossiness/roughness, opacity, and/or color, etc. The texture database 510 may be synchronized to each terminal device in the multi-user collaboration system 10. Each of the terminal device may store at least one texture of the 3D object transmitted from the texture database 510. If the server 500 and the terminal device 100 are taken as an example, the server 500 may transmit the texture not obtained by the terminal device 100 to the terminal device 100 in a fixed period. On the other hand, the terminal device 100 may periodically access the texture database 510 in a fixed period to check whether the texture database 510 includes any texture that is not yet obtained by the terminal device 100. If the texture database 510 includes a texture that the terminal device 100 has not acquired, the terminal device 100 may request the server 500 to transmit the texture to the terminal device 100. In other words, the textures stored in the texture database 510 will be synchronized to each terminal device. If the specific texture in the texture database 510 has not been synchronized to each terminal device, it means that the specific texture is newly acquired by the server 500.
More specifically, a multi-user collaboration method is adapted for a multi-user collaboration system 10. The terminal device 100 or other terminal device (200, 300) receives an image information from the server 500. The image information comprises grid information and dimensional mapping information of a three-dimensional (3D) object. The terminal device 100 selects at least one texture of the 3D object stored in the terminal device 100. The terminal device 100 displays an image comprising the 3D object according to the image information and the at least one texture.
For instance, in the exemplary terminal device 100, in step S201, the server 500 may receive a 3D object from the network, where the 3D object may include the texture, grid information, dimension mapping information (i.e., UV mapping, which refers to texture mapping to a 3D object and is a term well known to those skilled in the art), and other information. The 3D object is, for instance, uploaded to the server 500 by a terminal device (e.g., the terminal device 100, 200, or 300) by means of the network. The 3D object is, for instance, a 3D object 410 shown in
In step S202, the server 500 may execute a feature capturing operation on the 3D object 410 to obtain a plurality of features of the 3D object 410, where the features may include a texture, grid information, dimensional mapping information, a classification label, and other features of the 3D object 410. The classification label may represent how the server 500 classifies the 3D object 410. For instance, if the 3D object 410 is a pyramid, then the classification label of the 3D object 410 may be a quadrangular pyramid. That is, the server 500 may classify the pyramid into the classification of quadrangular pyramids.
In an embodiment, the server 500 may input the 3D object 410 to the machine learning model 520 to output the features of the 3D object 410 from the machine learning model 520, where the features may include the texture, the grid information, the dimensional mapping information, the classification label, and other features of the 3D object 410. In an embodiment, the server 500 may train the machine learning model 520 according to historical data. For instance, the historical data may include the 3D object and the label corresponding to the 3D object, where the label may include the texture, the grid information, the dimensional mapping information, the classification label, and other features of the 3D object. The server 500 may train the machine learning model 520 based on a supervised learning algorithm according to a plurality of historical data.
In step S203, the server 500 may determine whether the texture database 510 includes the texture of the 3D object 410. If the texture database 510 includes the texture of the 3D object 410, it means that the texture has been synchronized to each terminal device. Accordingly, the server 500 may end the process. If the texture database 510 does not include the texture of the 3D object 410, it means that the texture has not been synchronized to each terminal device. Accordingly, the server 500 may execute step S204.
In step S204, the server 500 may transmit the texture of the received 3D object 410 to each terminal device (i.e., the terminal device 100, 200, and/or 300); that is, the server 500 may preload the texture of the received 3D object 410 to each terminal device.
The server 500 may share an image to the terminal device 100, the terminal device 200, and the terminal device 300, where the image may include a 3D object that can be operated. When executing a multi-user collaborative task on the 3D object, the terminal device with authority may operate the 3D object to change the way in which the 3D object is presented in the image. For instance, the terminal device 100, the terminal device 200, and the terminal device 300 may share images including geometric figures. When a user applies the terminal device 200 to rotate the geometric figures in the image, the geometric figures in the image displayed by the terminal device 100 and the terminal device 300 also rotate accordingly.
In an embodiment, the server 500 may receive a command and select a specific 3D object 410 according to the command, so as to provide the terminal device 100 with the image information (e.g., the grid information and the dimensional mapping information) of the 3D object 410. Specifically, the server 500 may receive from the network, for instance, a query command transmitted by the terminal device 200, for instance. In response to the query command, the server 500 may match a specific classification label with the query command and output a query response corresponding to the classification label to the terminal device 200 that issues the query command, where the query response may include an option corresponding to the classification label. The server 500 may be selected by the terminal device 200 in response to the option of 3D object 410 to generate the image information according to the 3D object 410 and transmit the image information to each terminal device (e.g., the terminal device 100, the terminal device 200, or the terminal device 300).
For instance,
As shown in
In an embodiment, if the server 500 obtains a plurality of static image layers, the server 500 may merge the static image layers into one single merged image layer and then transmit the merged image layer to the terminal device 100 according to image information (e.g., the dynamic or static image layers), so that the terminal device 100 outputs the merged image layer as the background of the image. Since the static image layers obtained by the server 500 do not change continuously, the server 500 only needs to transmit the merged image layer to the terminal device 100 once within a period of time.
The server 500 may inspect whether the image layer 51 and the image layer 52 include any dynamic object, where the dynamic object is, for instance, an image (e.g., a real-time human face image used in a video conference), a two-dimensional (2D) object, or a 3D object. If neither the image layer 51 nor the image layer 52 includes any dynamic object, the server 500 may determine that the image layer 51 and the image layer 52 are the static image layers. Accordingly, the server 500 may overlap the image layer 51 and the image layer 52 to generate an image layer 53. The server 500 may transmit the image layer 53 to the terminal device 100 in advance according to the image information (e.g., the dynamic or the static image layer). The terminal device 100 may directly use the image layer 53 (i.e., obtained by overlapping the static image layers) during the multi-user collaboration instead of overlapping the static image layers during the multi-user collaboration. Thereby, the data processing amount may be reduced.
As shown in
In step S304, the server 500 may update the grid information and the dimensional mapping information of the 3D object 410 according to the operation command(s) and transmit the updated grid information and the updated dimensional mapping information to the terminal device 100. In step S305, the terminal device 100 may update the 3D object 410 in the image according to the updated grid information and the updated dimensional mapping information. Specifically, the terminal device 100 may update the grids of the 3D object 410 according to the updated grid information and the updated dimensional mapping information. After that, the terminal device 100 may render the texture corresponding to the 3D object 410 into the updated grids to update the 3D object 410 in the image.
The server 500 may set the authority level of each terminal device for accessing the 3D object 410, and the authority level may indicate whether the terminal device is allowed to create, modify, or delete the 3D object 410 or may indicate whether the terminal device is allowed to issue the operation command(s) to the 3D object 410. It is assumed that the authority levels of both the terminal device 200 and the terminal device 300 allow the issuance of the operation command(s) to the 3D object 410, where the authority level of the terminal device 200 is higher than that of the terminal device 300. If the server 500 receives the operation command(s) on the 3D object 410 from the terminal device 200 and receives the operation command(s) on the 3D object 410 from the terminal device 300, then the server 500 may decide to enable the operation command(s) of the terminal device 200 and disable the operation command(s) of the terminal device 300 based on the authority levels.
Specifically, the operation command(s) received by the server 500 may include a timestamp, for instance, a coordinated universal time (UTC) stamp. If the server 500 receives a plurality of operation commands on the 3D object 410, the server 500 may determine the order of the operation commands according to the timestamp and apply the operation commands to the 3D object 410 sequentially.
In an embodiment, the server 500 may classify the received operation command(s) into batches corresponding to the timestamp according to the timestamp. If one batch includes a plurality of operation commands, the server 500 selects the operation command corresponding to the highest authority level from the batch as a selected operation command. The server 500 may merely execute the selected operation command and ignore the other operation commands in the batch. In other words, the server 500 may enable the selected operation command and disable the other operation commands in the batch.
For instance, the server 500 may determine the time difference between the operation command from the terminal device 200 and the operation command from the terminal device 300 according to the timestamp. If the server 500 receives the operation command from the terminal device 300 within a predetermined time slot after receiving the operation command from the terminal device 200, then the server 500 may classify the operation command from the terminal device 200 and the operation command from the terminal device 300 into the same batch. It is assumed that the operation command of the terminal device 200 is to control the 3D object 410 to rotate to the left and the operation command of the terminal device 300 is to control the 3D object 410 to rotate to the right. In response to the authority level of the terminal device 200 being higher than that of the terminal device 300, the server 500 enables the operation command of the terminal device 200 and disables the operation command of the terminal device 300. In other words, the 3D object 410 will rotate to the left but will not rotate to the right.
After the server 500 obtains the dynamic image layer including the dynamic object, the server 500 may perform an image processing operation on a part of the dynamic image layer including the dynamic object, so as to save the consumption of computational resources.
After the server 500 receives the operation command on the dynamic object, the server 500 may perform an image processing operation on the part 62 of the image layer 61 according to the operation command to update the grid information and the dimensional mapping information of the dynamic object without the need of performing the image processing operation on the other part of the image layer 61 except for the part 62. As shown in
To sum up, the server of the multi-user collaboration system provided in one or more embodiments of the disclosure may perform the feature capturing operation on a new 3D object when receiving the new 3D object, so as to obtain the information including the texture of the 3D object and transmit the texture to terminal device in the multi-user collaboration system in advance. When the system starts to execute the multi-user collaborative task on the image shared by a plurality of terminal devices, the terminal device may operate the 3D object in the image according to the command to update the information of the 3D object. In the case that each terminal device has obtained the texture of the 3D object in advance, the server is merely required to transmit the updated grid information and the updated dimensional mapping information of the 3D object to each terminal device when the multi-user collaboration task is executed in real time, whereby the 3D object displayed on each terminal device may be updated in real time. Accordingly, the disclosure may be conducive to significantly reducing the communication resources consumed by the multi-user collaborative task performed on the 3D object.
The foregoing description of the preferred embodiments of the disclosure has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the disclosure to the precise form or to exemplary embodiments disclosed. Accordingly, the foregoing description should be regarded as illustrative rather than restrictive. Obviously, many modifications and variations will be apparent to practitioners skilled in this art. The embodiments are chosen and described in order to best explain the principles of the disclosure and its best mode practical application, thereby to enable people skilled in the art to understand the disclosure for various embodiments and with various modifications as are suited to the particular use or implementation contemplated. It is intended that the scope of the disclosure be defined by the claims appended hereto and their equivalents in which all terms are meant in their broadest reasonable sense unless otherwise indicated. Therefore, the term “the disclosure” or the like does not necessarily limit the claim scope to a specific embodiment, and the reference to particularly preferred exemplary embodiments of the disclosure does not imply a limitation on the disclosure, and no such limitation is to be inferred. The disclosure is limited only by the spirit and scope of the appended claims. Moreover, these claims may refer to use “first”, “second”, etc. following with noun or element. Such terms should be understood as a nomenclature and should not be construed as giving the limitation on the number of the elements modified by such nomenclature unless specific number has been given. The abstract of the disclosure is provided to comply with the rules requiring an abstract, which will allow an inquirer to quickly ascertain the subject matter of the technical disclosure of any patent issued from this disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Any advantages and benefits described may not apply to all embodiments of the disclosure. It should be appreciated that variations may be made in the embodiments described by people skilled in the art without departing from the scope of the disclosure as defined by the following claims. Moreover, no element and component in the present disclosure is intended to be dedicated to the public regardless of whether the element or component is explicitly recited in the following claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 112104706 | Feb 2023 | TW | national |
