Patent Application
20080088621
a) and 2(b) are highly schematic front elevational views of a viewport and selected part, used in explanation of aspects of the follower operation;
a) and 3(b) are highly schematic front elevational views similar to those of
a), 6(b) and 6(c) are highly schematic perspective views illustrating further operational aspects;
a), 7(b) and 7(c) are highly schematic perspective views used in explanation of three different implementations or embodiments of the follower method; and
As indicated above, according to an important aspect of the invention, a computer-based method is provided which enables a selected part or assembly to be followed throughout a three-dimensional image or computer model. As an initial step, the user selects the desired part to be followed and viewed. In preferred embodiments, to enable the viewing of the desired part, the method provides for automatically removing other objects, layers, or other obstacles or obstructions that prevent direct viewing of the part. For example, if the part to be followed is a particular wiring conduit, pipe or duct in a building, or in part of a building, the method provides for removing from the image as viewed, from the point of view or perspective of the user, all objects, such as walls, insulation, and the like. This enables the user to initially view and then follow the selected part directly in the three dimensional representation. In following the part, other objects and obstacles to viewing of the selected part are automatically removed as the part is being followed. In preferred embodiments, the removal of obstacles or impediments to viewing the part is limited to an area around a center point defined by the projection of the point of view or chosen perspective onto the selected part. As indicated above, the method may also be used to follow an assembly of individual parts. The method may be implemented as a stand-alone tool or mode, or as a non-exclusive mode in conjunction with another modeling tool application, wherein the navigational tools provided by the application are constrained by the method.
Referring to
As shown in
As indicated by block or step 12, in a preferred embodiment, other parts are automatically removed in an initial step when the desired part is selected. The parts that are removed may completely disappear or may remain on the screen in ghost so that the user has an unimpeded view of the selected part but can also view the selected part in context, i.e., in relation to adjacent parts or surroundings. Further, the parts removed may be only those parts in front of the selected part in a chosen or selected area of the screen, or may be all of the parts on the screen. Thus, in the former case, the user may select the size of an area of the image in front of the selected part in which obstacles or impediments are to be removed so that a corresponding area of the selected part can be viewed.
Once the user selects the part to be followed, in one preferred embodiment of the invention, the user may select a perspective or point of view from which the model is viewed and presented on the screen. Preferably, several different modes or options are available to the user. For example, in one mode, the point of view is not changed automatically. In a further mode, a cursor is used and the point of view is always centered on the cursor. In another mode using a cursor, the point of view is changed when the cursor comes close to a boundary of the current view being provided.
In the embodiment of
As discussed above, in a preferred embodiment, any obstacles that impede the view of the selected part within the three-dimensional image are automatically removed in a first step. In one implementation, any view-impeding obstacles or objects are removed, down to the selected part, as determined from the perspective or point of view initially selected by the user. The viewing areas may follow the projection of a cursor or other indicator onto the geometry of the part that is of interest, i.e., a surface, curve, edge or vertex. For example, for a curve, the selected viewing area may be a circle centered on the point nearest the picking line on the curve.
After the part is selected, in a next step, as indicated in step 14, the user moves a cursor, pointer, or some other icon (hereinafter referred to as a cursor), along the selected part, so as to follow the part throughout the three-dimensional image or model. As indicated above, the viewing area or viewport provided may encompass a section of the screen or the entire screen.
As shown in step 16, if, during this movement of the cursor indicated in step 14, other obstacles in the three-dimensional image block viewing of the selected part, or otherwise impede viewing thereof as the user moves the cursor along the selected part, the obstacles impeding the view of the part are automatically removed. This enables the user to view of the selected part as the cursor is moved therealong. As indicated above, in one preferred embodiment, the “removed” obstacles may still be shown in ghost.
As is also indicated by block 16, the screen image display is repositioned to display a further portion of the model that is located in the direction of cursor movement and that was not previously displayed so as to enable the cursor to follow the image beyond an initial screen display. This feature, and further aspects thereof, are discussed in more detail below in connection with
Various known methods may be used in removing or eliminating the obstacles blocking viewing of the selected part. For example, one method concerns testing the depth of the pixels in the area surrounding the selected part by using what is generally referred to as a z-buffer, i.e., a buffer memory of depth z. The use of the z-buffer technique allows revealing of the selected part when hidden, through the use of a memory capacity with dimensions of the image in two-dimensions. Thus, on the 3-D image, a pixel being tested is replaced only if the preceding z value in memory (the depth value) is more than the current z value of the tested pixel. This procedure is usually referred to as a depth test.
Returning to step 14 of the preferred embodiment shown in
In accordance with a further preferred embodiment, an audio output is provided which continually sounds while the cursor is moved along the selected part and which ceases when the part is no longer followed. The feature can be used when the cursor is not directly constrained to the part or in combination with such constraint. In other words, in one preferred embodiment, an audio output is employed to constrain movement of the cursor by alerting the user that the cursor has moved off on the selected part. In this regard, a simple audio alert can be provided if the cursor moves off of the part. In another implementation, an audio alert is provided whenever the cursor encounters a discontinuity or an unwarranted condition occurs.
It will be understood that in some instances, a series of points are sufficient to describe the geometry of the part. For example, in welding points between parts, for reference points on a vehicle, and for a path for a procedure, a series of points may be sufficient.
In cases where the part to be followed has a small cross section (e.g., wherein the part is a wire, a vein, or another elongate shape), the geometry may be defined by a parametric curvilinear model (as represented by x=f(t), y=f(t), z=f(t)). In such cases, in accordance with one advantageous embodiment, in order to determine the viewing plane, the tangent to the curvilinear model is determined, and the viewing plane is defined as a plane orthogonal to the tangent. The viewpoint of the user will thus be included in that viewing plane.
In cases where the selected part, such as a pipe, is sufficiently large, the normal to the surface of the selected pipe part from the viewpoint of the user can be directly computed. Thus, there are a number of different ways in which the selected part can be represented, and a number of different ways in which the viewpoint of the user can be determined.
In preferred embodiments, where the part is defined using a curvilinear model, the orientation of the computer model moves in such a way that a constant angle is maintained between the view or viewpoint and a tangent to the curve. Preferably, where the geometry of the selected part to be followed is a surface, the angle between the viewpoint and a normal to the surface is maintained. On the other hand, where the geometry of the selected part is simply a point, the view is not automatically changed.
Considering an example of the changes in the position and orientation of the view provided to a user, and referring again to the example of a pipe, considering the case wherein the pipe is being followed to left as viewed and there is a bend or turn in the pipe of ninety degrees away from the user (e.g., a 90° elbow), the model will rotate to right as the cursor moves around the bend and will cease to rotate after the ninety degree rotation has been completed.
The point of view is also adjusted when discontinuities are encountered. For example, when a ninety degree turn is encountered in a conduit or pipe, there will be a rapid movement in the orientation of the model from the current orientation to a new orientation. In one preferred embodiment, the user is alerted to a discontinuity by audio signal or by an alert on the screen so as to alert the user that a decision needs to be made and thus assist the user in making the proper decision.
In a preferred implementation, should a discontinuity occur wherein the part being followed divides or separates into multiple parts or paths (e.g., wherein a wiring cable separates into three different smaller cables), a pie chart with the possible choices may be superimposed over the discontinuity so as to enable the user to decide which part or path to follow.
As indicated above, in preferred embodiments, when the cursor comes to a point at the boundary edge of the viewing area, the viewing area presented will automatically be adjusted to bring the next adjacent portion of the image or model into view. Thus, referring to
Similar translations of the viewport 22 are shown in
In preferred embodiments, using a cursor, in the case of an edge, the orientation of the model follows the bisector plane of the “angle” formed by the two faces at the point of a cursor.
For the case of vertex discontinuity, in one implementation, when the cursor nears a vertex, the movement of the cursor is determined by the intersection of the constraints on the vertex and the movement of the mouse or other control device controlling the cursor. The face or edge that is selected will be that which is closest to the movement of the mouse. In the case where only one face is visible and the other faces are hidden, an exception is made, and a movement outside the vertex away from the visible face will be interpreted as a movement towards one of the neighboring hidden faces. The face selected will be the one having the highest unconstrained component of the movement.
Turning now to an example regarding the alignment of a camera used in providing the 3-D image, and, in particular, regarding the positioning and change in orientation of the view provided by such a camera, let C represent the coordinate system of the camera expressed in the world coordinate system (WCS) so Cx is the x-axis of the camera view, Cy is the y-axis of the camera view and Cz is the viewing direction of the camera (the z-axis of the view). C determines, in WCS, the point of view and the orientation of the view. It will be appreciated that the projection of the camera may be perspective or orthographic. In addition, let P represent the picking line (a line in three-dimensions line drawn through the cursor) and Up the absolute vertical of the world.
Now considering R, a point on the curve of the selected part defined in three-dimensional coordinates (R.Vx, R.Vy, and R.Vz), R.Origin is defined as the nearest point of the Cz axis on the curve, and R.Vx equals T, where T is the tangent of the curve at R.Origin. If T is colinear with Up, then R.Vy is an arbitrary axis non-colinear with T. Otherwise, R.Vy is Up. R.Vz is the cross product (R.Vx, R.Vy). Considering Rcursor, a three-dimensional coordinate system computed around the cursor, Rcursor.Origin is the nearest point of P on the curve; Rcursor.Vx equals Tcursor where Tcursor is the tangent of the curve at Rcursor.origin. If Tcursor is colinear with Up, then Rcursor.Vy is an arbitrary axis non-colinear with Tcursor. Otherwise, Rcursor.Vy equals Up. Rcursor.Vz is the cross product (Rcursor.Vx, Rcursor.Vy).
In order to compute the new position of the camera which follows the curve, the following method is used: search to compute Cnew, the three-dimensional coordinate system expressed in WCS which defines the position of the camera necessary to follow the curve. Ctemp is C transformed from the world coordinate system to the R coordinate system. Ctemp is now considered as being expressed in Rcursor, and the “reverse” transformation is computed to express Ctemp in the world coordinate system. Thus, Cnew equals Ctemp as transformed from Rcursor to the WCS. Ccursor determines the new point of a view in the world coordinate system and the new orientation of the view.
In another embodiment, the displacement is “smoothed” and an intermediate point between R.Origin and Rcursor.Origin on the curve is considered. Where the geometry of the part is a surface, the method is the same but T is replaced by N, which is the normal to the face near the cursor.
Referring now to
Referring to
In the embodiment of
In the embodiment of
Referring to
In the next step, step 52, the part is selected using further monitor key, and, as indicated by step 54, a feature of the part (e.g., the extent thereof) is followed throughout by moving a cursor along the part.
As indicated by block 56, while the cursor is moved along the part, any obstacle (other part, object, etc) appearing in front of the part that prevents viewing of the part is removed or otherwise negated so as to enable continued viewing and following of the part. As described above, this removal or negating of an obstacle can be achieved either by actually removing the obstacle completely from the screen image or by showing the obstacle in ghost or phantom lines so that the part can be viewed through the obstacle. Preferably, if, as the cursor continues to be moved along the feature, the obstacle no longer impedes viewing of the part in an area of the cursor, the obstacle (further part or object) is restored and reappears as before, i.e., as a solid part or object.
In accordance with a further feature of this embodiment, as the part is being followed, the view, i.e., the image of the model on the monitor screen, is changed to reveal more of the complete model in the direction of cursor movement so that the part can continue to be followed. In one preferred embodiment, as indicated by block 58, as the cursor is moved along the part, the view of the model displayed on the screen is changed so as to display a further portion of the model located in the direction of cursor movement that was not previously displayed. Further, as is also indicated by block 58, in a preferred embodiment, when the cursor, in following the feature, is moved, i.e., repositioned, to a central area of the screen when the view is repositioned. In one embodiment, such repositioning occurs when the cursor encounters a screen boundary (e.g., one edge of the monitor screen). In other words, when the cursor reaches the screen edge, the cursor is repositioned (e.g., recentered) on the screen so that continued cursor movement can take place and will continue from this new, more central position. In a preferred embodiment, the repositioning or recentering of the cursor occurs simultaneously and automatically with the repositioning (recentering) of the view as the cursor moves in a given direction, and this repositioning of the view and cursor is not dependent on the cursor encountering a screen boundary.
In accordance with a further feature of some preferred embodiments, information of interest is displayed during the follower operation, i.e., while the selected part is being followed. Typically, measurements are carried out in spatial xyz coordinates and not in a curvilinear coordinate system. In one advantageous embodiment, the length of the curve on the surface, or the length of an edge, is computed and displayed. For example, a measurement may be made of the length of a pipe, or the length of a portion of pipe, from the point of pipe entry up to a position at which an obstruction within the pipe may have occurred.
In another implementation, marks are provided along the selected part showing, for example, the length of the curvature along the pipe or the surface to each specific point. This information will again be displayed.
In accordance with a further advantageous feature, follower markers are provided on the 3-D scene to enable the follower operation, i.e., to determine when the follower system is to be enabled and when the system is to be disabled. For example, 3D or 2D markers may be created on the scene, such as 3D rings disposed at a particular position on a curve. A relatively large number of markers may be created at different abscissas of curves, or placed directly on 3D faces, or 3D points. For example, in one implementation, if the user presses the left mouse button when the cursor is on a marker, the follower system is enabled with respect to the linked geometry (i.e., the curve or surface under the marker), and when the user releases the left mouse button, the follower system is disabled, and the marker is displaced to a new position for later use.
Other features of a preferred method include computing measurements of the surfaces and other aspects of the part based on the above calculations. In an advantageous embodiment, the method may compute distances and lengths using curvilinear measurements.
In another advantageous implementation using markers, the user may also apply markers to specific points to compute the distances between the selected points. The user may create markers at different curve abscissa, or place then directly onto three-dimensional faces or points. For example, if a user depresses the left mouse button when the cursor is on a marker, the follower is enabled on the linked geometry (the curve or surface under the marker), and when the user releases the left mouse button, the follower mechanism is disabled, and the marker is displaced to the new position for later use.
As indicated above, obstacles impeding following of the selected part may be removed from the entire screen or only part of the screen. Provision is preferably made for the user to define the size and shape of the area from which obstructions or obstacles are removed. Thus, while the area may be the entire screen, the area may be limited to a circumscribed area around the selected part, depending on the application and the wishes of the user.
According to a further feature, the followed path may be used to generate an animation or AVI. Also, the follower can be combined with a GPS system as discussed in more detail below.
In preferred embodiments, the user can use the follower system to follow a selected part in real time and in its actual location, using a laptop or portable PC, so as, e.g., to permit a user to follow a buried pipe or a pipe disposed within a wall. In this example, when the pipe is accessible, a reset to “zero” is carried out so as to provide a real-location “read” on the 3D image as (i) the position of the pipe and (ii) the depth at which the pipe is buried. As discussed above, similar applications can be used in following ducts in buildings, pipes in refineries or chemical plants and in many other situations.
As was also discussed above, in preferred embodiments, the model can turn or rotate to follow a turn or bend in the selected part (e.g., a ninety degree elbow in a pipe or electrical conduit), and in this case, the three-dimensional image on the screen will enable a user to reference the current viewing coordinates of the model with the physical spatial coordinates of the user.
The follower system can also be used with a six degrees of freedom device that provides an indication of position and orientation in local or global coordinates. Devices such as those made by Pohlhemus can be used to provide the six local degrees of freedom. Global coordinates can also be provided in other ways such as, for example, coupling a GPS to a triaxial accelerometer.
In accordance with a further advantageous feature of the follower system, combined viewing of both the model and the physical objects used implementing the model is provided. In one implementation, a tablet PC is fitted with a follower system and the measurements are used to control the viewport of the software. The tablet PC can be held at a fixed distance (for example, at arms length), and the viewport of the software will cover viewing of the obstructed view of the selected part (such as a pipe) and provide, in situ, the view of the selected part.
In another advantageous implementation, a small display is mounted in proximity to one eye of the viewer. This eye receives an image which is, for example, the unobstructed view of the model of the part, while the naked eye, with a parallax difference, sees the obstructed view.
In another implementation, each eye receives respective images from separate displays. The image generated for each eye is suitably compensated to provide proper registration, so that the perceived image of the model and the perceived image of the physical object coincide in position and orientation. The virtual image of the model and the physical image can be mixed, for example, by using optical elements with a semi-reflective coating.
In some embodiments, when the model turns to follow, for example, a ninety degree elbow in a pipe, a three-dimensional referential object is provided on the screen which enables the user to reference the current viewing coordinates of the model with the physical spatial coordinates of the user.
In a further embodiment, a cutting plane is positioned so as to cut through, i.e., provide a cross sectional view of, the selected object. This would be useful, for example, wherein a plurality of pipes are disposed within a larger tube. Such a cutting plane would be normal to the viewpoint of the user and positioning thereof would follow the movement of a cursor or the like as described above for previous embodiments.
While the invention has been described in connection with a number of preferred embodiments, it is to be understood that the specific methods and systems which have been described are merely illustrative of different aspects and principles of the invention, and that various modifications may be made in the methods and systems described without departing from the spirit and scope of the invention.
