Patent Application
20210233330
The following relates to a system and method for visualizing objects and applying markings within a three-dimensional virtual or augmented reality system.
In the medical field, imaging techniques such as computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), and other three-dimensional (3D) medical imaging modalities are used to visualize a patient's anatomy. Medical imaging data obtained from these medical imaging procedures can be analyzed to identify organs or other structures of interest, and can be reviewed by a medical professional to determine a diagnosis or appropriate treatment for a patient. For example, radiation oncologists and other radiotherapy clinicians may analyze the medical imaging data to plan a course of radiotherapy treatment and assess the calculated dose to radiotherapy targets.
Traditionally, the medical imaging data of a 3D patient obtained through one or more imaging procedures are presented to medical professionals on screens as digital two-dimensional (2D) slices, or cross-sections. To process the imaging data, the medical professional selects a slice of the scan data along a cardinal plane and draws on the slice using a mouse and cursor or a touch-screen. For example, the slice may show a cross-sectional view of the three-dimensional structure, including a cross-section of any organs or other structures of interest within the three-dimensional structure. The medical professional can mark the image to highlight features of medical importance, draw an outline around (contour) one or more of the organs or other structures, or otherwise annotate the cross-sectional image. This process is often repeated for multiple slices. Outlines of an organ or structure on the multiple slices can be combined to form a 3D contour or model.
Various challenges can arise when using 2D slices to annotate a 3D image volume and/or contour in three dimensions. For example, each 2D image slice is often analyzed and drawn on in isolation, without, or with limited, context and/or knowledge of the orientation and position of the 3D structure. As such, the medical professional may have difficulty identifying organ boundaries on the slice, leading to inaccurate annotations or contours. Additionally, the image slices are often only provided along the three anatomical planes with fixed orientation, namely the sagittal, coronal, and transverse planes. Moreover, certain structures, such as the brachial plexus, are not readily visualized on any of these three conventional planes, thus a medical professional may be unable to accurately identify and/or contour these structures if they are provided with only slices in the three conventional planes.
Three-dimensional imaging data are commonly analyzed on touch-screen computer systems, where the user individually selects 2D slices of the 3D image on which to annotate or contour/draw. The physical dimensions of the touchscreen can create a barrier between the image being drawn upon or annotated and the device used for drawing/annotating, decreasing the precision of annotations or contours. Further, the user's hand and stylus may occlude the line of sight of the image, adding time to the contouring process due to periodic repositioning of the image for visual acuity. Lastly, without knowledge of the 3D shape of a structure, the user may be required to frequently switch between slices, for example to draw on a different slice or provide 3D context, which can be cumbersome and time consuming both for annotating/drawing and when reviewing the contours.
Several systems have been previously proposed to address the above-mentioned shortcomings using virtual reality visualization of the 3D image. These systems allow users to view and interact with 3D models of the imaging data in a virtual space. For example, to create 3D models of organs or structures based on the 3D images represented in a virtual space, a user draws contours by manipulating controllers in the air within the virtual image. These controllers may be uncomfortable to use, since drawing in a virtual space is an unusual and unfamiliar task for most users, and for medical professionals in particular. Additionally, the contours may be imprecise, since there is no physical frame of reference nor direct physical anchor or feedback to indicate to the user where they are drawing in the 3D virtual space. As such, these systems can be cumbersome, difficult to use, and therefore prone to errors and large inter- and intra-user variability in drawing precision.
It is an object of the following to address at least one of the above-noted disadvantages.
In one aspect, there is provided a system for applying markings to a three-dimensional virtual image or virtual object, the system comprising: a physical stylus; a surface; and a virtual or augmented reality display; wherein a virtual space is displayed by the virtual or augmented reality display, the virtual space comprising: a three-dimensional target object; at least one plane, including a tracking plane, the tracking plane corresponding to the surface; and a virtual stylus in a virtual reality virtual space, or the physical stylus in an augmented reality virtual space; wherein: a position of the virtual stylus or the physical stylus relative to the tracking plane is correlated to an actual position of the physical stylus relative to the surface; and a cross-section of the target object is displayed on the tracking plane where the tracking plane intersects the target object.
In another aspect, there is provided a method of applying markings to a three-dimensional virtual image or virtual object, the method comprising: displaying a virtual space using a virtual or augmented reality display; providing in the virtual space: a three-dimensional target object, at least one plane including a tracking plane, the tracking plane corresponding to the surface, and a virtual stylus in a virtual reality virtual space, or a physical stylus in an augmented reality virtual space; correlating a position of the virtual stylus or the physical stylus relative to the tracking plane to an actual position of the physical stylus relative to a surface; and displaying a cross-section of the target object on the tracking plane where the tracking plane intersects the target object.
In yet another aspect, there is provided a computer readable medium comprising computer executable instructions for performing the method.
Embodiments will now be described with reference to the appended drawings wherein:
To visualize and apply one or more markings to an object in a virtual 3D space, a virtual or augmented reality system can be utilized, in which the object can be manipulated in a virtual space and such markings (e.g., lines, traces, outlines, annotations, contours, or other drawn or applied marks) can be made on a selected cross-section of the object by moving a physical stylus along a physical surface, which are both also represented in the virtual space.
Turning now to the figures,
The surface 16 can be a rigid or substantially rigid surface of known shape and dimension. In this example, the surface 16 is a rectangle, however the surface 16 may have any other shape such as a circle, triangle, or trapezoid, etc. The surface 16 is illustrated as appreciably planar, however it should be appreciated that the surface 16 may be nonplanar, such as a curved surface or a surface with raised features. In this example, the surface 16 has fixed dimensions, however in other examples, the surface 16 may have adjustable dimensions, such as a frame with telescopic edges placed on a table. In one example, the surface 16 may be a conventional 2D monitor corresponding to the position of the tracking plane which displays a 2D cross-section of the target object, with the system 10 configured to show a perspective view of the target object on the display 14 above the surface 16. The second tracker 18 may be located on a known position on the surface 16, such as a corner of the surface 16. The system 10 is configured to determine the position and orientation of the surface 16 from the position and orientation of the second tracker 18, the known shape and dimension of the surface 16, and the location of the second tracker 18 relative to the surface 16. It may be noted that more than one surface 16 can be provided and interactions with any such additional surface(s) may be tracked using an additional corresponding tracker (not shown). In one embodiment, the system 10 is configured to determine the position and orientation of the surface 16 by way of virtual tracking coordinates that are provided by the user 12 using the stylus 20 to indicate the boundaries of the virtual plane on an existing physical surface 16 such as a tabletop or electronic screen. As described in greater detail below, any one or more surfaces 16 in the physical environment can be represented in the virtual space (which may also be referred to as a virtual “environment”) and used to provide physical interfaces felt by the user 12 while operating the stylus 20 in the physical environment and likewise represented in the virtual space.
The stylus 20 can be a rigid instrument of known dimension held by the user 12 in the manner of a pen or other marking, drawing, writing, etching or interactive instrument, and is used to apply markings such as by drawing contours or annotating the target object and interacting with menus in the virtual space, as described in greater detail below. The stylus 20 is provided with one or more input sensors. In this example, a tip sensor 24 is located on the tip of the stylus 20 and a stylus button 26 is located on the side of the stylus 20. The tip sensor 24 is configured to detect when the tip of the stylus 20 is in contact with the surface 16. The tip sensor 24 may be a capacitive sensor, or a button that is depressed when the stylus 20 is pressed against the surface 16. The tip sensor 24 may also be a photoelectric sensor reflector if the surface 16 is semi-reflective. In some examples, the tip sensor 24 is capable of detecting the force applied by the stylus 20. The stylus button 26 is configured to receive input from the user 12. The stylus button 26 may be a physical button or capacitive sensor to determine a binary state, or the stylus button 26 may be a multi-directional input sensor such as a trackpad or pointing stick. The stylus 20 may also include a haptic motor (not shown) to provide the user 12 with haptic feedback.
The first tracker 22 is located on a known position on the stylus 20, such as on the end opposite of the tip sensor 24. The system 10 is configured to determine the position and orientation of the stylus 20 from the position and orientation of the first tracker 22 using the known dimension of the stylus 20, and the location of the first tracker 22 relative to the stylus 20. It can be appreciated that the stylus tracker 22 depicted in
The controller 28 is configured to receive input from the user 12, in this example with one or more controller buttons 32. The third tracker 30 is located on a known position on the controller 28. The system 10 is configured to determine the position and orientation of the controller 28 from the position and orientation of the third tracker 30, and the location of the third tracker 30 relative to the controller 28. The controller 28 depicted in
In an exemplary use of the system 10, the surface 16 is placed such that it is supported by a table, mount or other supporting structure (not shown for ease of illustration). In another example of the system 10, the surface 16 comprises a table or other rigid surface. The user 12 may manipulate the stylus 20 with a dominant hand, and optionally the controller 28 with the other hand to manipulate images, data, tools, and menus in the virtual space.
The virtual reality or augmented reality display 14 is configured to display a virtual image or virtual object to the user 12. In this example, the display 14 is a virtual reality headset such as the HTC Vive™, and the displayed image is perceived by the user 12 three-dimensionally. While some of the examples described herein are provided in the context of virtual reality, it can be appreciated that the system 10 and principles discussed herein can also be adapted to augmented reality displays, such as see-through glasses that project 3D virtual elements on the view of the real-world. Similarly, the system 10 can use a 3D monitor to provide the surface 16 with shutter glasses to produce a 3D visualization. In general, the display 14 refers to any virtual or augmented reality headset or headgear capable of interacting with 3D virtual elements.
The virtual stylus 120 and the virtual controller 128 are virtual, visual representations of the stylus 20 and controller 28, respectively in virtual-reality. It can be appreciated that in augmented-reality examples, the stylus 20 does not have a corresponding virtual stylus 120 in the augmented reality space, which comprises a view of the virtual space 100 and the physical environment. The position of the virtual stylus 120 and the virtual controller 128 in the virtual space 100 are correlated to the physical position of the stylus 20 and the controller 28 determined by the system 10, as described above. In particular, the virtual stylus 120 relative to the tracking plane 104 correlates to the stylus 20 relative to the surface 16 in the physical environment. As such, the user 12 moves or manipulates the virtual stylus 120 and the virtual controller 128 by physically moving or manipulating the stylus 20 and the controller 28, respectively. Similarly, the tracking plane 104 is a virtual representation of the surface 16. The tracking plane 104 has a shape and dimension corresponding to the real dimensions of the surface 16. In this example, the tracking plane 104 is a rectangle with an identical aspect ratio as the physical surface 16. In another example, a portion of the surface 16 may be associated with the tracking plane 104, e.g. the surface 16 boundary is surrounded by a frame or has a handle that is not represented in the virtual space 100. It should be appreciated that if the surface 16 is curved or has raised features, the tracking plane 104 will also have curvature(s) and/or raised feature(s) with the same dimensions as the surface 16. The virtual stylus 120 can be moved towards the tracking plane 104 by moving the stylus 20 towards the surface 16 in the physical environment, and when the tip of the stylus 20 contacts the surface 16, the tip of the virtual stylus 120 contacts the tracking plane 104 in the virtual space 100.
Other objects in the virtual space 100, such as the target object 102 and the menu 106, may be manipulated using the virtual stylus 120 or the virtual controller 128. For example, the user 12 can move the controller 28 such that the virtual controller 128 is over the target object 102, and press one or more of the controller buttons 32 to select the target object 102. In one example, the menu 106 may be manipulated by bringing the virtual stylus 120 within a defined proximity of the menu 106 without the user 12 pressing one or more of the controller buttons 32. In another example (not shown), the menu 106 or a portion thereof can be aligned with the tracking plane 104 to provide tactile feedback when selecting a menu option using the virtual stylus 120. In another example, the menu 106 can be displayed or hidden from view in the virtual space when the user 12 presses one or more of the controller buttons 32. The system 10 may provide the user 12 with feedback to indicate that the target object 102 is available for selection, for example by changing the color of the target object 102 or providing haptic feedback to the controller 28 when the virtual controller 128 is in proximity to the target object 102. With the controller button 32 pressed, the movement of the target object 102 can be associated with the movement of the virtual controller 128, such that movement or rotation of the controller 28 results in corresponding movement or rotation of both the virtual controller 128 and the target object 102. It should be appreciated that the virtual stylus 120 can be similarly used to manipulate the objects in the virtual space.
The target object 102 comprises information on a 3D object, for example a medical image, that the user 12 wishes to apply markings to, e.g., to draw on, contour, or annotate, using the system 10. One example of the target object 102 is illustrated in
The target object 102 shown in
The target object 102 may store other data, including radiotherapy contours 132 (i.e. specific types of markings) around organs or other structures of interest as shown in
The system 10 is configured to display a cross-section 110 of the target object 102 on the tracking plane 104 derived from the 3D object 130. The underlying 3D object 130 may be stored in a 3D texture (i.e., a collection of voxels in the target object 102). In volume rendering, the system 10 can use a graphics card to trace rays through the 3D texture In order to assign a color and brightness to each pixel on the physical display 14. Along each ray, the associated 3D texture is sampled at regular intervals and color and opacity are accumulated resulting in a final color to display on each individual pixel. The manner by which the accumulation of color and opacity is performed results in the 3D volume being displayed in different ways, the x-ray-like view of
As illustrated in
The angle and positioning of the tracking plane 104 can be anchored such that the user 12 may move the target object 102 through the tracking plane 104 to display a cross section 110 with a fixed orientation. In another example, the user 12 may anchor an angle of intersection of the target object 102 with the tracking plane 104 by registering two virtual coordinates in space using the virtual stylus 120 or menu 106, around which the 3D object 130, or tracking plane 104, as chosen, can be rotated. Similarly, in another example, the user 12 may choose a single point to anchor rotation of the target object 102 by registering a single virtual coordinate in the virtual space 100 using the virtual stylus 120 or menu 106, such that either the tracking plane 104 or 3D object 130 can rotate around this pivot point.
In another implementation, the target object 102 can be manipulated using only the virtual controller 128 and virtual stylus 120. For example, the user 12 can use the virtual stylus 120 to draw or annotate an axis on cross-section 110 and rotate the target object 102 about the axis defined by this line, by moving the virtual controller 128 into the target object 102, pressing the controller button 32, and moving the virtual controller in a circular motion around the line. In another example, the user 12 can anchor the target object 102 on a point-of-interest to the user 12 on cross-section 110 by placing the virtual stylus tip 122 on the point-of-interest, moving the virtual controller 128 into the target object 102, and moving the virtual controller 128 around the point-of-interest while pressing and holding the controller button 32. Changes to position an (movements left, right, up, or down) or only rotations around pre-defined axes of rotation by, for example, selecting the desired option to rotate or translate in the menu 106 using the virtual stylus 120 or virtual controller 128 in a manner as described above.
In another implementation, the user 12 may store one or more orientations of the target object 102 and tracking plane 104 in the menu 106 such that the user 12 may readily switch between these orientations using menu 106. Similarly, the user 12 may store one or more rotation point-of-interests or axes within the target object 102 for selection in menu 106.
The virtual space 100 may also include one or more object manipulation tools, in this example one or more rotation rings 109, as shown in
In one example, the user 12 can scale (increase or decrease size) the target object 102 and 3D object 130. In one implementation, the user may select an option to scale in the menu 106, and place the virtual controller 128 in the target object, and, while pressing controller button 32, move the virtual controller to scale up or down accordingly. In another implementation, scaling can be with the target object anchored at a point on the cross-section 110 with the virtual stylus tip 122 as described above.
As illustrated in
The system 10 may also include within the virtual space 100 one or more virtual planes 108 to change the display of the target object 102. In the example illustrated in
In one example, two different functions may be associated with the same virtual plane 108 (not clearly shown). For example, a single plane may simultaneously hide structures while displaying a cross section 112 or model of a 3D object 130 as the virtual plane 108 is moved through the target object 102. Alternatively, one side of the virtual plane 108 (side A), may have a different function from the second side of the virtual plane 108 (side B). In this example, side A may have the function of hiding structures, while side B may have the function of displaying a model of the 3D object 130, such that the orientation of the plane determines what function the plane will serve. In other words, if side A faces to the right and side B faces to the left, as the virtual plane 108 is moved to the right through the target object 102, models on the left (other side of side B) will be illuminated while structures will be hidden on the right (other side of side B). In this example, if the virtual plane 108 was flipped such that side A now faces to the left and side B now faces to the left, structures will be revealed/hidden differently, i.e. as the plane 108 is moved to the right, tumors on the right will be illuminated while structures on the left will be hidden.
In another example, the virtual plane 108 may be assigned to function as the tracking plane 104 by selecting a function to associate the virtual plane 108 dimensions to that of the physical surface 16. In this example, the tracking plane 104 would convert to a virtual plane 108, and the selected virtual plane 108, coupled with the target object 102, would automatically reorient together in the virtual space 100 to align the virtual plane 108 with the physical surface 16, thereby maintaining the cross-section 112 displayed in the process.
The user 12 can customize the layout of the menu 106 by rearranging or relocating the panels 141, 143, 145. The user 12 can relocate the first panel 141 by moving the virtual controller 128 or virtual stylus 120 over a first relocate option 146, pressing the controller button 32 or the stylus button 26, and physically moving the controller 28 or stylus 20 to move the first panel 141. The user 12 can similarly relocate the second panel 143 by using a second relocate option 147, and the third panel 145 by using a third relocate option 148. As such, the user 12 can customize the placement of the panels 141, 143, 145 such that the checkboxes 140, buttons 142, and sliders 144 are easily accessible.
The virtual stylus 120 can be used to apply markings to the target object 102.
When applying the markings by annotating or drawing is enabled by the system 10, the path of the virtual stylus 120 on the tracking plane 104 is marked with a drawn contour 132 until drawing ceases or is otherwise disabled, for example when the user 12 removes the stylus 20 from the surface 16. Once marking is discontinued, the drawn contour 132 may be saved to the target object 102. Marking such as lines, contours and annotations (e.g., measurements, notes, etc.) can also be delineated using a circular paintbrush that fills a region (not shown) rather than a line defining an outline.
Referring additionally to
The following provides further technical details concerning how the presently described VR contouring can be used to change how identified anatomical volumes of interest are defined, while maintaining an intuitive and fluid input mechanism that is familiar to users of traditional systems. Signed Distance Fields (SDFs) are implicit representations of 3D volumes or 2D areas. In a SDF, a 2D grid of pixels or a 3D grid of voxels is defined and for each point, the minimum distance to the surface of the described object is stored as shown in
SDFs are considered to be useful because they can allow for mathematically simple manipulation of objects (both 2D areas and 3D volumes). For example, constructive solid geometry (CSG) (e.g., when one does “add object A's volume to object B” in CAD software) is done with SDFs. Radiotherapy treatment planning systems use SDFs to interpolate contours that are not defined on every slice: i) currently delineated contours are converted to SDFs, ii) new SDFs are defined on slices without contours by interpolating the SDFs from adjacent slices, then iii) new contours are created on the empty slices by converting the interpolated SDFs into contours. The system 10 can be used for creating contours and object delineations by applying markings via drawing or annotating for example. However, the underlying data structure used can be SDFs.
Using SDFs allows the system 10 to create contours with circular fill paintbrush tools on the primary plane 104. In such an implementation, there is a spherical virtual object attached to the virtual stylus tip 122. When the virtual sphere intersects the tracking plane 104, it defines a circular region on the plane 104. While in a drawing mode, a user sees a “fill-region” contour that is either drawn or erased. In reality, rather than a thick or thin line being drawn, a 2D SDF representing the union (or subtraction for erasing) of all the circular intersection draw positions can be defined. From a user interface perspective, the size of the circular region can be adjusted by holding a stylus button 26 and moving the stylus 20 forwards or backwards. This could also be achieved by a pressure sensing stylus tip sensor 24 or, in another implementation, by pressing the controller button 32 on controller 28 while simultaneously adjusting the size of the circular region by activating the pressure sensing stylus tip sensor 24. It may be noted that the line-drawing mechanism shown herein can still be a useful option for generating a 2D SDF since 2D SDFs can be derived from a closed-loop line contour with a simple and fast algorithm.
The sphere at the end of the virtual stylus tip 122 (or attached to a controller 128) can also be used to define a 3D SDF which now defines a 3D volume (instead of 2D area of the sphere's circular intersection with the plane) in a similar manner. This 3D volume can be visualized with volume rendering or on the tracking plane 104 as a color-fill or contour outline region.
The system 10 can therefore be used to simplify the process for defining a volume that encompasses a given anatomical structure by way of intuitively manipulating 3D SDFs in two ways: (1) using 3D tools to change them directly or (2) using 2D tools to define 2D SDF subsets that describe portions of the final 3D SDF. These 2D SDF subsets (individual contours) are combined to create the resulting 3D SDF volume.
The system 10 can be programmed to take multiple non-parallel 2D SDFs from separate descriptive planes and interpolate them into a 3D SDF that defines the desired 3D structure, as shown in
Returning to
As illustrated by
The drawn contour 132 can remain visible when the target object 102 is manipulated. In the example shown in
The drawn contours 132 may remain visible when the user 12 continues to manipulate the target object 102, as shown in
As shown in
The system 10 may be configured to apply the contour 132 at the point of intersection between the virtual tip extension 123 and the primary plane 104. As illustrated in
Similarly, as illustrated in
The virtual tip extension 123 can additionally be used as a further input to enable an operation that applies markings, e.g. for drawing, annotating, contouring, tracing, outlining, etc. The system 10 may be configured to only enable marking to be applied with the virtual stylus 120 when the tracking plane 104 is within the length of the virtual tip expansion 123, such that the application of markings is not activated when the stylus 20 is away from the surface 16 if the stylus tip sensor 24 falsely detects contact.
As shown in
In another example, the fade effect may be provided by a virtual flashlight-like object (not shown), which produces a cone of effect, such as the fade effect described above. In one example, the cone of effect may reveal underlying fused scan data (such as PET or MRI) and/or hide other data such as contour 132, wireframes or volumes, or volume rendered image data sets. In one example, the user 12 may reposition the virtual flashlight tool (not shown) and hang it in the virtual space to illuminate the target object 102 from a desired angle.
The system 10 and principles of operation thereof can be adapted to various applications, including both medical and non-medical applications.
For example, the system 10 can be used, in any such application for human subjects, virtual characters, or animals. That is, the system 10 can be used in a virtual space 100 that is other than that intended for radiation oncology contouring. For example, surgical planning, gaming, drawing/editing, design, communication/conversing, etc.
In one example, the system 10 can be adapted to provide a drawing application for learning how to draw 3D figures and/or as a means/method for drawing in 3D, using the assistance of the tracking plane 104 corresponding with the volume rendered. This would be useful for people designing games or any 3D object such as furniture design or 3D printing files. It may be noted that, in a “learn to draw” scenario, the 3D volume rendered would be the object being studied, and the user 12, for example a student, can trace the image along different chosen axes or planes to arrive at the final 3D structure.
In another example, the system 10 can be adapted to provide an application for engineers or architects working on 3D building plans or structures, or similarly for 3D printing. For example, a user 12, for example an architect or engineer, could have a 3D blueprint of their plans as the virtual image in the virtual space 100, and any changes they want to make at any place can be accomplished by moving the tracking plane 104 to the desired position to write and at which angle to sketch amendments. In this application, the user 12 could also snap the view to automatically rotate and reorient the image file so that the chosen plane is now mirroring the physical surface 16 and they can draw on it easily in order to manipulate the plans. This can similarly be done for 3D printing structure designs.
In yet another example, a tool for interior design and/or landscape design can be provided, wherein the user 12 can annotate walls/furniture or shrubs/trees by writing notes wherever desired, by inserting a tracking plane 104 into the virtual space 100 that corresponds with a writing surface, or cutting (instead of drawing, make it a selection tool to cut parts out) and moving elements around the virtual space 100.
Further to the above examples, another application includes adapting the system 10 described above for teaching purposes or other collaborative purposes, both networked (live) and asynchronous (recorded), additionally utilizing audio features including microphones for recording and speakers for audio playback. For example, the user 12 drawing can be conducting a lecture/illustration to show one or more students/attendees parts of an anatomy or how to target or not target for radiation therapy. This can be done asynchronously (pre-recorded) or networked (live). All users 12 (e.g. teachers and viewer students) can be placed in the virtual space 100 regardless of whether the lecture/lesson is pre-recorded or live. If the lesson is asynchronous, the viewer can pause the lesson, and move closer to, manipulate, move or otherwise inspect 3D objects around in the virtual space 100. When un-paused, the lesson may revert to a previous setting and the viewing user continues to watch the lecture. In one embodiment, the system 10 can be configured such that if the viewing user 12 draws on or annotates a 3D object 130 when the system is “paused”, the drawing can remain while the user 12 continues to watch the lecture so that the user can see how it fits with the rest of the lecture. In another embodiment, the user 12 can make notes on the lecture that appear and disappear throughout and can save these annotations. In yet another embodiment, teaching can be done so that the student user 12 can practice drawing, and if that user veers off or is wrong, the system 10 can notify the user 12. This can be done in real time, with the system 10 generating alerts as the user 12 makes errors, and can be done as a tutorial where the system 10 guides the user 12 and provides tips as the user 12 progresses through an exercise, or can be done as a test, where the user 12 attempts a contour and when completed, receive a score or other evaluation of performance.
For simplicity and clarity of illustration, where considered appropriate, reference numerals may be repeated among the figures to indicate corresponding or analogous elements. In addition, numerous specific details are set forth in order to provide a thorough understanding of the examples described herein. However, it will be understood by those of ordinary skill in the art that the examples described herein may be practiced without these specific details. In other instances, well-known methods, procedures and components have not been described in detail so as not to obscure the examples described herein. Also, the description is not to be considered as limiting the scope of the examples described herein.
It will be appreciated that the examples and corresponding diagrams used herein are for illustrative purposes only. Different configurations and terminology can be used without departing from the principles expressed herein. For instance, components and modules can be added, deleted, modified, or arranged with differing connections without departing from these principles.
It will also be appreciated that any module or component exemplified herein that executes instructions may include or otherwise have access to computer readable media such as storage media, computer storage media, or data storage devices (removable and/or non-removable) such as, for example, magnetic disks, optical disks, or tape. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. Examples of computer storage media include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by an application, module, or both. Any such computer storage media may be part of the testing tool 12, any component of or related to the computing environment 10, etc., or accessible or connectable thereto. Any application or module herein described may be implemented using computer readable/executable instructions that may be stored or otherwise held by such computer readable media.
Although the above principles have been described with reference to certain specific examples, various modifications thereof will be apparent to those skilled in the art as outlined in the appended claims.
This application claims priority to U.S. Provisional Patent Application No. 62/695,580 filed on Jul. 9, 2018, the contents of which are incorporated herein by reference.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/CA2019/050941 | 7/8/2019 | WO | 00 |
| Number | Date | Country | |
|---|---|---|---|
| 62695580 | Jul 2018 | US |
