Patent Application
20240005048
Computer systems can be used to create, use, and manage data for products and other items. Examples of computer systems include computer-aided design (CAD) systems (which may include computer-aided engineering (CAE) systems), visualization and manufacturing systems, product data management (PDM) systems, product lifecycle management (PLM) systems, and more. These systems may include components that facilitate the design and simulated testing of product structures and product manufacture.
Certain examples are described in the following detailed description and in reference to the drawings.
CAD systems and applications can be implemented in various ways. For instance, CAD applications may be offered as a cloud-based service, e.g., as a Software-as-a-Service (“SaaS”) offering in which client computing systems access centrally hosted CAD applications via locally implemented thin clients. In such client-server models, user access to CAD models may be a time-consuming and resource-consuming process, especially as CAD models continue to increase in size and complexity. Receiving and loading 3D CAD models from a server via thin clients may consume significant computing and network resources. Visualization of complex models can be particularly taxing on thin clients (e.g., application browsers) that are oftentimes limited in processing and memory capabilities.
Such limitations may become increasingly apparent for complex CAD models with numerous sub-parts. CAD models may be comprised of CAD parts, which may represent discrete, granular, or atomic objects in a CAD model and complex CAD models may be formed by nested assembly structures in which a given CAD model is formed as combination of CAD parts and CAD assemblies, which are further formed by CAD parts and CAD sub-assemblies, and further so forth. In some implementations, CAD models can be represented as product structure trees, with tree branches representing sub-assemblies and leaf nodes representing CAD parts. Any suitable CAD model formats are contemplated herein, and in modern CAD contexts, CAD models comprising thousands or millions of CAD parts are increasingly common.
Visualization of CAD models is typically performed from specific viewpoints within a 3D space that CAD models are modelled within. For a given view of a CAD model (e.g., from a particular viewpoint and viewing angle in the 3D space), only some, but not all, of the CAD parts of a CAD model may be visible. Some of the CAD parts may occlude other CAD parts of the CAD model in the given view, and in such cases the occluded CAD parts need not be displayed to properly visualize the given view of the CAD model. Determination of occluded CAD parts in CAD model views is not a trivial matter, particularly for resource-limited thin clients in SaaS-based implementations of CAD systems and CAD architectures. Conventional occlusion culling and determination of visible CAD parts and occluded CAD parts rely on algorithmically complex geometrical and spatial analyses of the 3D geometry of CAD parts of the CAD model. For CAD models with thousands or even millions of CAD parts, such complex geometrical and spatial processing of 3D geometries can be time-consuming and resource intensive, greatly limiting the feasibility, usability, and performance of thin clients in cloud-based CAD systems.
The disclosure herein may provide systems, methods, devices, and logic for bounding box-based visualization of CAD models via pixel color analyses. As described in greater detail below, the bounding box-based visualization technology of the present disclosure may support occlusion culling based on bounding boxes instead of full-detail geometry data of CAD parts. In doing so, the bounding box-based visualization features of the present disclosure may assign a unique pixel color to each bounding box of CAD parts of a CAD model and render the bounding boxes instead of the full-detail 3D geometry for a given view of the CAD model, with each CAD part bounding box colored with a unique color value. Then, analysis of a 2D image of the rendered bounding boxes can indicate which pixel color values are included in the 2D image, thus providing an indication of which bounding boxes (and corresponding CAD parts) are visible in the view of the CAD model based on the rendered bounding boxes.
As also described herein in greater detail, iterative determinations of visible CAD parts are supported. Through an iterative model visualization process, renderings of 3D geometry of visible parts and bounding boxes of non-visualized CAD parts can be performed to further determine additional visible CAD parts in subsequent iterations. Thus, the bounding box-based visualization technology of the present disclosure may support determination of visible CAD parts in a view of a CAD model through pixel color analyses of rendered bounding boxes (which may be in combination with rendered 3D geometry of previously determined visible CAD parts).
The bounding box-based visualization technology described herein may provide various technical improvements as compared to conventional CAD model visualization and occlusion culling techniques. Obtaining bounding box data from a cloud server and rendering 3D rectangular shapes may be performed at increased speed and efficiency as compared to the cumbersome loading and rendering full 3D geometry data of all of the CAD parts of a CAD model as in conventional techniques. Pixel color analyses of 2D images may also be performed with significantly lesser resource requirements and reduced computational latency than the complex geometrical spatial analyses of conventional 3D model visualization processes. As such, the bounding box-based visualization technology of the present disclosure may support determination of visible and occluded CAD parts in a view of a CAD model with increased speed, efficiency, and effectiveness, which may be particularly suitable for SaaS-based CAD application implementations. Additionally, the features described herein can support selective loading of visualization data specifically for visible CAD parts determined through bounding boxes renderings and pixel color analyses, reducing network latency and increasing the speed at which views of CAD models are visualized.
These and other bounding box-based visualization features and benefits are described in greater detail herein.
As an example implementation to support any combination of the bounding box-based visualization features described herein, the client computing system 100 shown in
In operation, the model visualization engine 110 may visualize a view of a CAD model in an application window, including by accessing bounding box data for non-visualized CAD parts of the CAD model. Non-visualized CAD parts may include CAD parts of the CAD model for which visualization data has not been retrieved by the client computing system 100 and bounding box data for the non-visualized CAD parts may represent a respective bounding box for the non-visualized CAD parts of the CAD model. The model visualization engine 110 may further visualize the view of the CAD model by assigning a color value to each of the bounding boxes of the non-visualized CAD parts, capturing a 2D image of the view of the CAD model rendered using the bounding boxes for the non-visualized CAD parts instead of rendered using visualization data (e.g., 3D geometry) of the non-visualized CAD parts, and analyzing the 2D image to identify pixel colors present in the 2D image to determine visible CAD parts in the view of the CAD model, wherein the visible CAD parts correspond to bounding boxes of non-visualized CAD parts determined as visible based on the identified pixel colors. Then, the model visualization engine 110 may retrieve visualization data for the visible CAD part and visualize, in the application window, the visible CAD parts in the view of the CAD model via the retrieved visualization data.
These and other bounding box-based visualization features according to the present disclosure are described in greater detail next.
The client computing system 100 as shown in
To visualize a given view of a CAD model, the model visualization engine 110 may determine which of the CAD parts of the CAD model are visible in the given view. In doing so, the model visualization engine 110 may differentiate or otherwise categorize CAD parts of the CAD model as visible CAD parts, non-visualized CAD parts, or any other suitable characterization. Visible CAD parts may refer to CAD parts for which the model visualization engine 110 has determined as visible in a given view of a CAD model (or previously visualized views), responsive to which the model visualization engine 110 may load visualization data to store locally on the client computing system 100. Non-visualized CAD parts may refer to CAD part of a CAD model that the model visualization engine 110 has not retrieved visualization data for, and as such may refer to any CAD parts of a CAD model not previously determined to be a visible CAD part for a visualized view of the CAD model.
Determination of visible CAD parts of the CAD model may be performed through image analyses of rendered bounding boxes. In the example of
The model visualization engine 110 may obtain bounding box data 210 specifically for non-visualized CAD parts of the CAD model, as doing so may allow the model visualization engine 110 to represent such non-visualized CAD parts as bounding boxes instead of via full-detail 3D geometry. As described further herein, visible CAD part determinations may be performed iteratively, and thus the model visualization engine 110 need not obtain bounding box data for visible CAD parts for which visualization data has been previously retrieved, e.g., previously determined visible CAD parts for which corresponding visualization data was retrieved from the server computing system 202. Such prior retrieval may have occurred as part of a previous iteration of a model visualization process for a given view of the CAD model or as part of the visualization of a different view of the CAD model.
In some implementations, the model visualization engine 110 may initially set, flag, or characterize all CAD parts of a CAD model as non-visualized CAD parts (e.g., in an initial iteration of visible CAD part determinations for a given view). As such, the model visualization engine 110 may obtain bounding box data 210 for all CAD parts of the CAD model, even for CAD parts previously determined as visible in the visualization of a different view of a CAD model by the model visualization engine 110. In other implementations, the model visualization engine 110 may maintain characterizations of CAD parts across different view visualizations, and by doing so may deem a particular CAD part of a given view as a visible CAD part when the visualization data of the particular CAD part was previously retrieved in the visualization of a different view of the CAD model.
To support occlusion culling via bounding boxes, the model visualization engine 110 may assign a color value to each of the bounding boxes of the non-visualized CAD parts for a given view of a CAD model. Assignment of color values of may be performed in any suitable manner such that each bounding box (and corresponding CAD part) is assigned a unique color value. In some examples, the model visualization engine 110 may assign sequential RGB pixel values to respective bounding boxes (and, in effect, corresponding CAD parts). In such examples, a first bounding box may be assigned an RGB color value #00001, a next bounding box assigned an RGB color value #00002, and so forth.
The model visualization engine 110 may then render the CAD model (that is, a particular view thereof), doing so by rendering the non-visualized CAD parts of the CAD model as bounding boxes instead of the actual 3D geometry of the non-visualized CAD parts. Each given bounding box may be colored with the particular color value assigned to the bounding box, and thus each bounding box may be uniquely identified by its color value. In an initial iteration, this may include rendering all of the CAD parts of a CAD model as bounding boxes. As shown in
The model visualization engine 110 may render the CAD model via the bounding boxes 220 for a particular view of the CAD model that the model visualization engine 110 is to visualize in an application window of the client computing system 100. In
In the example shown in
The model visualization engine 110 may determine visible CAD parts by analyzing the 2D image 230 of the view of the CAD model. To do so, the model visualization engine 110 may process the 2D image 230 to identify the pixel colors included in the 2D image 230 and then correlate the identified pixel colors to the assigned color values for the bounding boxes 220. Any pixel color included in the 2D image 230 that is an assigned color of a bounding box may indicate that the corresponding CAD part of the bounding box is visible in a given view of the CAD model. As such, the model visualization engine 110 may determine that this corresponding CAD part is a visible CAD part for the view of the CAD model.
In the example shown in
In such a manner, the model visualization engine 110 may utilize bounding box renderings and pixel color analyses of a captured 2D image in order to determine visible CAD parts in a given view of a CAD model. Analyses of a captured 2D image may be performed with increased speed and efficiency as compared to conventional spatial analyses of complex 3D geometries. Instead, by identifying specific pixel color values of a captured 2D image, the model visualization engine 110 may quickly identify bounding boxes visible in the rendered view simply through identification of each different pixel color value present in the 2D image and performing lookup operations to determine the corresponding CAD parts for each identified pixel color value.
Note that the model visualization engine 110 may determine visible CAD parts of a view of CAD model via bounding box renderings and pixel color analyses as a background process that is not displayed via a user interface of a CAD application. That is, the model visualization engine 110 may render CAD models via bounding boxes such that the rendered bounding boxes are not displayed or otherwise visible to CAD application users. In that regard, the rendering of the bounding boxes 220 as well as the capture and analysis of the 2D image 230 may be performed “off-screen” as the 2D image 230 is not visible to a user. Indeed, such an off-screen rendered view of the CAD model via bounding boxes would appear as a collection of overlapping colored boxes, and incrementally assigned color values would make bounding boxes in such an image nearly indistinguishable, providing limited visual cues to a human observer. While such distinguishing of pixel color values (and thus visible bounding boxes and CAD parts) may be near-impossible for a human eye, such an image can be quickly analyzed and processed by the model visualization engine 110 to determine visible CAD parts for a given view of a CAD model.
To visualize the given view of a CAD model in an application window, the model visualization engine 110 may visualize only the CAD parts of a CAD model determined as visible CAD parts for a given view of the CAD model, such as the visible CAD parts 240 of
Upon retrieving visualization data for visible CAD parts, the model visualization engine 110 may visualize (e.g., display) a CAD model in an application window, doing so by visualizing the visible CAD parts for a given view of the CAD model and for which visualization data of the visible CAD parts has been retrieved from a remote server. In
A single iteration of determining visible CAD parts for a view of a CAD model via the bounding box-based visualization technology of the present disclosure may provide an elegant and efficient process to visualize the view of the CAD model. For some 3D part geometries, a particular CAD part may have a relatively large bounding box but the actual 3D geometry of the CAD part may occupy only a relatively small portion of the bounding box. In such cases, there may be other CAD parts visible behind the bounding box of the CAD part as the actual 3D geometry of the CAD part may occlude only a relatively small portion of CAD parts located behind the bounding box in a view of the CAD model. To increase the accuracy of visible part determinations for a CAD model, the model visualization engine 110 may perform multiple iterations of visible CAD part determinations to progressively identify additional visible CAD parts that are not fully occluded by rendered bounding boxes of CAD parts of the CAD model, also referred to herein as iterations of a model visualization process.
In some implementations, the model visualization engine 110 may iteratively determine visible CAD parts for a given view of a CAD model, doing so through off-screen renderings of a combination of bounding boxes for non-visualized CAD parts and 3D geometry of previously determined visible CAD parts. In each given iteration, the model visualization engine 110 may determine additional visible CAD parts through the off-screen combined 3D geometry-bounding box rendering of the given view of a CAD model, doing so in a consistent matter as described herein.
To provide an illustrative example through
As seen in
The model visualization engine 110 may render a given view of the CAD model as a combination of 3D geometry of visible CAD parts and bounding boxes of non-visualized CAD parts. Such a rendering may occur in as a background process, off-screen to a CAD user and thus not visible in the application window 322. In rendering such a view of the CAD model off-screen, the model visualization engine 110 may color each given CAD part (whether rendered as 3D geometry or a bounding box) as the single color assigned to the given CAD part. Thus, the model visualization engine 110 may color the 3D geometry of a visible CAD part entirely as a single color, without having to account for shading, lighting, shadows, or any other visual features of the rendered CAD part. In a consistent manner, the model visualization engine 110 may render each bounding box solely in a single color. Such single-color rendering of CAD parts may allow for proper image analysis and occlusion culling as described herein, but may also provide the technical benefit of reducing (at times significantly) the complexity of off-screen CAD model renderings. As such, the bounding box-based visualization technology described herein may support off-screen view renderings of CAD models as a combination of 3D geometry and bounding boxes with increased speed and reduced resource strain.
Next, the model visualization engine 110 may capture a 2D image of the combined 3D geometry-bounding box rendering, shown in
From a captured 2D image 330 of the view of a CAD model rendered via a combination of 3D geometry and bounding boxes, the model visualization engine 110 may determine visible CAD parts for a given iteration of a model visualization process. Visible CAD parts determined for a given iteration of a CAD model visualization process may also be referred to as additional visible CAD parts that are determined as distinct additions to the visible CAD parts determined in previous iterations for the visualizing of a particular view of a CAD model, determined as visible in any iteration for the visualizing of a different view of the CAD model (and thus with visualization data retrieved for visualizing the different view of the CAD model), or a combination thereof. In
In the 2D image 330 of
The model visualization engine 110 may then retrieve visualization data for the additional visible CAD parts 340 determined in this given iteration of the model visualization process and may visualize the additional visible CAD parts 340 in the application view 322 as part of visualizing the given view of the CAD model 320. The visible CAD parts 340 determined a current iteration of the model visualization process may be visualized in addition to the previously determined visible CAD parts 240. With each progressive iteration, the model visualization engine 110 may categorize additional visible CAD parts, and thus progressively load additional amounts of visualization data for the CAD model unto the client computing system 100. After a given iteration of the model visualization process, the model visualization engine 110 may visualize, in the application window 322, progressively more CAD parts for the given view of the CAD model 320 as long as additional visible CAD parts are determined via the off-screen bounding box rendering, capture, and image analysis of the bounding box-based visualization technology of the present disclosure.
The model visualization engine 110 may iteratively determine additional visible CAD parts for the visualization of given view of a CAD model until an ending criterion is reached. The model visualization engine 110 may set any number of suitable ending criterion in order to control the model visualization process for the given view of the CAD model, and various examples of ending criteria are presented herein. As one example, the ending criterion may be satisfied when the 2D image captured for the rendering of the CAD model via a combination of 3D geometry of visible CAD parts and bounding boxes for non-visualized CAD parts does not include any pixel color values that are assigned to a bounding box. Satisfaction of this ending criterion may indicate that no bounding boxes of non-visualized CAD parts of the CAD model are visible in the rendered view of the CAD model as the 2D image may include only pixel color value(s) of 3D geometry of visible CAD parts. In such cases, the model visualization engine 110 may determine zero (0) additional visible CAD parts for the given iteration and need not continue to iteratively determine visible CAD parts for the given view of the CAD model.
As other examples, the ending criterion may be satisfied based on any number of thresholds to limit how long the model visualization engine 110 performs iterative determinations of additional visible CAD elements. Such ending criterion may be satisfied when a threshold number of iterations are performed by the model visualization engine 110 to determine additional visible CAD parts (e.g., after ten (10) iterations, which can represent a maximum number of iterations to perform), after a threshold amount of time has elapsed in performing the iterations (e.g., a timeout length of 10 seconds), or any other suitable threshold to limit iteration performance. As yet another example, the ending criterion may be satisfied when a user or a CAD application changes the view of the CAD model to a different view. In such cases, the model visualization engine 110 may no longer need to determine visible CAD parts for the given view and instead restart the new model visualization process, this time for the new view of the CAD model.
As yet a further example, the ending criterion for iterations of a particular model visualization process may be satisfied when a render (or re-render) command is issued by the CAD application. In such cases, changes (e.g., user selections, model vanish functions, or other adjustments to the display of the CAD model of the current view) may cause the CAD application to re-render the CAD model through issuance of a render command for the CAD model (which may be the same view). In such cases, the model visualization engine 110 may cease iterations of the current model visualization process and restart the model visualization process to account for the changes. Accordingly, the model visualization engine 110 may visualize the same view of the CAD model, but restart the model visualization process to account for the applied vanish function or any other applicable CAD model change that causes issuance of the render command.
The model visualization engine 110 may apply any combination of various examples of ending criterion described herein in performing iterations of a model visualization process (e.g., to iteratively determine additional visible CAD elements for a given view of a CAD model).
In any of the ways described herein, the model visualization engine 110 may support bounding box-based determinations of visible CAD parts for views of CAD models. Through the use of simplified shapes to represent CAD parts instead of full-detail 3D geometry, the bounding box-based visualization technology of the present disclosure may reduce initial transmission times to obtain CAD model data from cloud servers through transmission of bounding box data instead of full detail 3D geometry. Additionally, the features of the present disclosure can reduce computational requirements and rendering latencies for CAD model views to support determinations of visible CAD parts, as bounding boxes can be rendered with significantly increased speed as compared to complex full-detail 3D geometries. Through the use of 2D image analyses, pixel color comparisons may reduce the time and complexity of occlusion culling for CAD parts as compared to conventional complex geometric operations needed to compare 3D geometries of the many CAD parts that comprise a CAD model. As such, the bounding box-based visualization technology of the present disclosure may provide tangible technical benefits in the visualization of views of CAD models.
In implementing the logic 400, the model visualization engine 110 may visualize a view of a CAD model in an application window (402), including by accessing bounding box data for non-visualized CAD parts of the CAD model (404). As noted herein, the non-visualized CAD parts include CAD parts of the CAD model for which visualization data has not been retrieved by a client computing system and the bounding box data for the non-visualized CAD parts may represent a respective bounding box for the non-visualized CAD parts of the CAD model. In visualizing the view of the CAD model in the application window, the model visualization engine 110 may also assign a color value to each of the bounding boxes of the non-visualized CAD parts (406), capture a 2D image of the view of the CAD model in which the non-visualized CAD parts are rendered as the bounding boxes of the non-visualized CAD parts instead of rendered as a 3D geometry of the non-visualized CAD parts (408), and analyze the 2D image to identify pixel colors present in the 2D image to determine visible CAD parts in the view of the CAD model (410), doing so in any of the ways described herein. In implementing the logic 400, the model visualization engine 110 may visualize the view of the CAD model in the application window further by retrieving visualization data for the visible CAD parts (412) and visualizing, in the application window, the visible CAD parts in the view of the CAD model via the retrieved visualization data (414).
In any of the ways described herein, the model visualization engine 110 may iteratively determine additional visible CAD parts in visualizing the view of the CAD model. In the logic 400 of
In some implementations, in each iteration, the model visualization engine 110 may assign colors for visible CAD parts, non-visualized CAD parts, or a combination of both. In a given iteration, the model visualization engine 110 may assign a pixel color to each of the bounding boxes of the non-visualized CAD parts of the CAD model and assigning a pixel color to each visible CAD part of the CAD model (which in some implementations is the same color for each visible CAD part). Capture of the 2D image in a given iteration may comprise capturing a 2D image of the view of the CAD model rendered using the bounding boxes for the non-visualized CAD parts instead of rendered using the visualization data of the non-visualized CAD parts and also rendered using the visualization data (e.g., 3D geometry) of the visible CAD parts of the CAD model.
Responsive to a determination that the ending criterion is reached, the model visualization engine 110 may cease determination and visualization of additional visible CAD parts for the view of the CAD model visualized in the application window (though the model visualization engine 110 may continue to visualize the view of the CAD model in the application window until a different view of the CAD model is to be visualized or a render command is issued to re-render the view of the CAD model in an application window).
The logic 400 shown in
The system 500 may execute instructions stored on the machine-readable medium 520 through the processor 510. Executing the instructions (e.g., the model visualization instructions 522) may cause the system 500 to perform any of the bounding box-based visualization features described herein, including according to any of the features with respect to the model visualization engine 110.
For example, execution of the model visualization instructions 522 by the processor 510 may cause the system 500 to visualize a view of a CAD model in an application window, including by accessing bounding box data for non-visualized CAD parts of the CAD model, assigning a color value to each of the bounding boxes of the non-visualized CAD parts, capturing a 2D image of the view of the CAD model rendered using the bounding boxes for the non-visualized CAD parts instead of rendered using visualization data of the non-visualized CAD parts, analyzing the 2D image to identify pixel colors present in the 2D image to determine visible CAD parts in the view of the CAD model, wherein the visible CAD parts correspond to bounding boxes of non-visualized CAD parts determined as visible based on the identified pixel colors. Execution of the model visualization instructions 522 by the processor 510 may also cause the system 500 to visualize the view of the CAD model in the application window by retrieving visualization data for the visible CAD parts and visualizing, in the application window, the visible CAD parts in the view of the CAD model via the retrieved visualization data.
Any additional or alternative features of the present disclosure may be implemented via the model visualization instructions 522.
The systems, methods, devices, and logic described above, including the client computing system 100, model visualization engine 110, and server computing system 202, may be implemented in many different ways in many different combinations of hardware, logic, circuitry, and executable instructions stored on a machine-readable medium. For example, the client computing system 100, model visualization engine 110, server computing system 202, or combinations thereof, may include circuitry in a controller, a microprocessor, or an application specific integrated circuit (ASIC), or may be implemented with discrete logic or components, or a combination of other types of analog or digital circuitry, combined on a single integrated circuit or distributed among multiple integrated circuits. A product, such as a computer program product, may include a storage medium and machine-readable instructions stored on the medium, which when executed in an endpoint, computer system, or other device, cause the device to perform operations according to any of the description above, including according to any features of the model visualization engine 110.
The processing capability of the systems, devices, and engines described herein, including the model visualization engine 110, may be distributed among multiple system components, such as among multiple processors and memories, optionally including multiple distributed processing systems or cloud/network elements. Parameters, databases, and other data structures may be separately stored and managed, may be incorporated into a single memory or database, may be logically and physically organized in many different ways, and may be implemented in many ways, including data structures such as linked lists, hash tables, or implicit storage mechanisms. Programs may be parts (e.g., subroutines) of a single program, separate programs, distributed across several memories and processors, or implemented in many different ways, such as in a library (e.g., a shared library).
While various examples have been described above, many more implementations are possible.
