VIRTUAL COATINGS APPLICATION SYSTEM

Abstract

A virtual coatings application system has several features to enhance the realism of simulated spray painting. The system generally includes a display screen on which is defined a virtual surface (such as a truck door) that is intended to be virtually painted or coated by the user. Alternatively, the system includes a head-mounted display unit that displays a virtual spray painting environment in which the virtual surface is defined. The user operates an instrumented spray gun controller that outputs one or more signals representing data as to the status of the controls on the spray gun controller. The system also has a motion tracking system that tracks the position and orientation of the spray gun controller with respect to the virtual surface. Simulation software generates virtual spray pattern data in response to at least the data from the spray gun controller and the position and orientation data received from the tracking system. Virtual spray pattern images are displayed in real time on the virtual surface in accordance with the accumulation of virtual spray pattern data at each location on the virtual surface.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic drawing illustrating a person using a virtual coatings application system in accordance with a first embodiment of the invention, wherein a virtual surface is depicted on a display screen (i.e. a non-immersive system).

FIG. 2 is a schematic drawing illustrating a preferred embodiment of an instrumented spray gun controller used to simulate a typical high volume low pressure spray gun in accordance with the invention.

FIG. 3 is a cross-sectional view of the spray gun controller shown in FIG. 2 partially assembled.

FIG. 4 is a block diagram showing the elements and data flow of a virtual reality coatings application system in accordance with the preferred embodiment of the invention.

FIG. 4A is a diagram illustrating the software architecture of the system.

FIG. 5 illustrates a preferred embodiment of a graphical user interface for a virtual coatings application system in accordance with the invention.

FIG. 6 depicts a two-dimensional image of a door defining a virtual surface on the projection screen, wherein overspray is depicted in a color distinct from the color of virtual painted sprayed onto the image of the door.

FIG. 7 is a view similar to FIG. 6 wherein mil thickness of the finish accumulated on the image of the door is depicted in multiple colors.

FIG. 8 is a view similar to FIGS. 6 wherein the user has elected to change the color of the finish during the training session, thus virtually painting a camouflage design on the image of the door.

FIG. 9 is a plot illustrating experimentally obtained data for transfer efficiency in terms of spray distance.

FIG. 10 is a schematic drawing illustrating coverage rate aspects of the preferred paint model.

FIG. 11 is a schematic drawing illustrating various patterns produced on a virtual surface in accordance with the invention.

FIG. 12 is a schematic drawing illustrating a person using a virtual coatings application system in accordance with a second embodiment of the invention wherein the user is wearing a head-mounted display unit (i.e., an immersive system).

FIG. 13 is a perspective view showing a head-mounted display unit used in accordance with the second embodiment of the invention.

FIG. 14 depicts a rearward view of a virtual spray painting environment as might be viewed by a user wearing the head-mounted display unit shown in FIGS. 12 and 13.

FIG. 15 is representative of a generally forward looking view within the virtual spray painting environment as might be viewed by the user of the head-mounted display unit shown in FIGS. 12 and 13.

FIG. 16 is another representative view of the three-dimensional virtual spray painting environment which contains a depiction of a truck door (i.e. the virtual surface) hanging within the virtual environment, as might be viewed by the user of a head-mounted display unit shown in FIGS. 12 and 13.

FIG. 17 is a representative drawing showing the truck door virtual surface within the spray painting environment as depicted in FIG. 16, except from a different perspective as might be viewed by the user of the head-mounted display unit.

FIG. 18 is a view similar to FIG. 17 illustrating the collision control feature.

FIG. 19 is a view from within the head-mounted display unit illustrating a pop-up menu used to set system control settings or access performance records when the menu icon is activated.

Claims

1. A virtual coatings application system comprising: a virtual spray painting environment including a virtual surface;an instrumented spray gun controller outputting one or more signals representing virtual spray gun data;a motion tracking system that tracks the position and orientation of the spray gun controller with respect to the virtual surface;a computer programmed with software which generates virtual spray pattern data in response to at least the virtual spray gun data and the position and the orientation data received from the tracking system; andwherein a virtual spray pattern image is displayed in real time on the virtual surface in accordance with the accumulation of virtual spray pattern data at each location on the virtual surface; andwherein the software comprises a paint model that outputs virtual spray pattern data to characterize the resulting pattern of the virtual spray in terms of spatter size and density on the virtual surface as a function of time in response to at least standoff distance and angular orientation of the spray gun controller relative to the virtual surface as well as virtual spray characteristic data representing settings for spray fan size, air pressure and paint flow rate.

2. A virtual coatings application system as recited in claim 1 wherein the system comprises a head-mounted display unit that displays the virtual spray painting environment to a user wearing the head-mounted display unit, and the display of the virtual spray painting environment appears to the user wearing the head-molted display unit to be a three-dimensional environment in which the user is immersed.

3. A virtual coatings application system as recited in claim 2 wherein the three-dimensional virtual environment comprises a depiction of a spray painting booth including the virtual surface to be virtually painted and a depiction of a spray gun simulating the position and orientation of the instrumented spray gun controller.

4. A virtual coatings application system as recited in claim 3 wherein the system further comprises: means for detecting collision of the simulated spray gin with the virtual surface within the virtual environment; andmeans for alerting the user when such a collision is imminent or has been detected.

5. A virtual coatings application system as recited in claim 2 wherein the display viewed by the user of the head-mounted display unit contains a menu icon that can be activated by the user with the instrumented spray gun controller to display menu options for system control settings, which the user can set using the instrumented spray gun controller.

6. A virtual coatings application system as recited in claim 1 wherein the paint model is based at least on actual data collected from spray patterns generated for various spray gun settings.

7. A virtual coatings application system as recited in claim 1 wherein the paint model is based on empirically derived values for the flow rate of paint sprayed for various spray gum settings and finish type.

8. A virtual coatings application system as recited in claim 7 wherein the amount of paint sprayed per unit time is determined by the paint model in relation to at least the position of a trigger on the instrumented spray gun controller.

9. A virtual coatings application system as recited in claim 1 wherein the paint model is based at least in part on empirically derived data for the transfer efficiency for a given spray gun position and finish type.

10. A virtual coatings application system as recited in claim 1 wherein the paint model simulates coverage and mil thickness in the following manner: inner elliptical radii for width (riw) and height (rih) define an area of constant rate finish coverage;outer elliptical radii for width (row) and height (rih) define the outer extent to which the rate of finish coverage becomes negligible, the rate of finish coverage falling off linearly between inner and outer elliptical radii so that coverage at the outer radii is equal to zero;spatter droplet size is linearly distributed throughout from a minimum spatter droplet size to a maximum spatter droplet size; andwherein overall pattern dimensions are modified linearly with standoff distance and coverage rates -for uniform spray and spatter vary inversely with standoff distance, while spatter size remains constant with respect to standoff distance.

11. A virtual coatings application system as recited in claim 10 wherein the paint model compensates for tilt and rotation of the spray gun controller with respect to the virtual surface by displacing and rotating the coverage pattern and modifying the coverage density according to collected data.

12. A virtual coatings application system as recited in claim 1 wherein each location on the virtual surface has an associated alpha channel, the alpha channel controlling transparency of the coating at that location based on the mathematical accumulation of virtual spray at the given location, thus realistically simulating fade-in for partial cover on the virtual surface.

13. A virtual coatings application system as recited in claim 1 wherein multiple colors are used to depict accumulation level ranges at a given location.

14. A virtual coatings application system as recited in claim 1 wherein shades of a first color are depicted to show accumulation levels lower than 0.5 mil of a selected target thickness, shades of a second color are used to depict accumulation levels within 1 mil of the selected target thickness, and shades of a third color are used to indicate accumulation levels that are more than 0.5 mils of the selected target thickness.

15. A virtual coatings application system as recited in claim 1wherein the virtual spray pattern image depicts virtual overspray distinct from virtual spray on the virtual surface.

16. A virtual coatings application system comprising: a virtual spray painting environment including a virtual surface;an instrumented spray gun controller outputting virtual spray gun data;a motion tracking system that tracks the position and orientation of the spray gun controller with respect to the virtual surface: anda computer programmed with software implementing a paint model which generates virtual spray pattern data for a timing cycle in response to at least the virtual spray gun data and the position and the orientation data received from the tracking system for said timing cycle, wherein the paint model accounts for the effects of transfer inefficiencies due to shear effect when generating the virtual spray pattern data; andwherein a virtual spray pattern image is displayed in real time on the virtual surface in accordance with the accumulation of virtual spray pattern data at each location on the virtual surface.

17. A virtual coatings application system as recited in claim 16 wherein the system comprises a head-mounted display unit that displays the virtual spray painting environment to a user wearing the bead-mounted display unit, and the display of the virtual spray painting environment appears to the user wearing the head-mounted display unit to be a three-dimensional environment in which the user is immersed.

18. A virtual coatings application system as recited in claim 17 wherein the three-dimensional virtual environment comprises a depiction of a spray painting booth including the virtual surface to be virtually painted and a depiction of a spray gun simulating the position and orientation of the instrumented spray gun controller.

19. A virtual coatings application system as recited in claim 18 wherein the system further comprises: means for detecting collision of the simulated spray gun with the virtual surface within the virtual environment; andmeans for alerting the user when such a collision is imminent or has been detected.

20. A virtual coatings application system as recited in claim 17 wherein the display viewed by the user of the head-mounted display unit contains a menu icon that can be activated by the user with the instrumented spray gun controller to display menu options for system control settings, which the user can set using the instrumented spray gun controller.

21. A virtual coatings application system as recited in claim 16 wherein the effects of transfer inefficiencies due to shear effect in the paint model are based on empirically based data collected under normal working conditions.

22. A virtual coatings application system as recited in claim 16 wherein transfer efficiency due to shear effect generally varies inversely with standoff distance between the spray gun controller and the virtual work surface.

23. A virtual coatings application system as recited in claim 16 wherein the instrumented spray gun controller comprises: a variably positionable trigger and a trigger sensor for detecting the position of the trigger and generating a trigger signal representing trigger position data in response thereto, the trigger signal being sent to the computer, wherein the paint model uses said trigger position data to generate the virtual spray pattern data for the respective timing cycle.

24. A virtual coatings application system as recited in claim 23 wherein the instrumented spray gun controller comprises: a flow rate knob that rotates to allow for the adjustment of the maximum virtual fluid flow rate for the spray gun controller; anda sensor that senses the position of the flow rate knob and generates a maximum virtual fluid flow rate signal which is sent to the computer and used to scale the trigger position data.

25. A virtual coatings application system as recited in claim 24 wherein the paint model determines value for flow rate of paint sprayed for various gun settings, finish type and trigger position, these values being based on interpolations of empirically derived data.

26. A virtual coatings application system as recited in claim 25 wherein the paint model determines the maximum amount of paint deposited on the virtual surface by multiplying the transfer efficiency by the flow rate of paint sprayed for the given timing cycle.

27. A virtual coatings application system as recited in claim 26 wherein the virtual spray pattern data generated by the paint model for the respective timing cycle simulates coverage rate over the spray pattern by distributing the total virtually transferred fluid flow for the timing cycle over a virtual spray pattern for the timing cycle.

28. A virtual coatings application system as recited in claim 27 wherein the paint model simulates coverage and mil thickness in the following manner: inner elliptical radii for width (riw) and height (rih) define an area of constant rate finish coverage;outer elliptical radii for width (row) and height (rih) define the outer extent to which the rate of finish coverage becomes negligible, the rate of finish coverage falling off linearly between inner and outer elliptical radii so that coverage at the outer radii is equal to zero;spatter droplet size is linearly distributed throughout from a minimum spatter droplet size to a maximum spatter droplet size; andwherein overall pattern dimensions are modified linearly with standoff distance and coverage rates for uniform spray and spatter vary inversely with standoff distance, while spatter size remains constant with respect to standoff distance.

29. A virtual coatings application system comprising: a virtual spray painting environment including a virtual surface;an instrumented spray gun controller outputting virtual spray gun data, the spray gun being instrumented with a trigger and a sensor which generates a trigger signal sent to the computer;a motion tracking system that tracks the position and orientation of the spray gun controller with respect to the virtual surface;one or more loudspeakers;a computer programmed with software which generates spray pattern data in response to at least the virtual spray characteristic data from the spray gun controller and the position and the orientation data received from the tracking system, and which also interactively generates an output sound signal in response at least to the trigger signal and provides the output sound signal in real time to drive the one or more loudspeakers to simulate the sound of compressed air coming from a spray gun; andwherein a virtual spray pattern image is displayed in real time on the virtual surface in accordance with the accumulation of the virtual spray pattern data at each location on the virtual surface.

30. A virtual coatings application system as recited in claim 29 wherein the software comprises in part digital sound files of actual compressed air noise.

31. A virtual coatings application system as recited in claim 30 wherein the system further comprises the capability of setting an air pressure level for compressed air virtually supplied to the instrumented spray gun controller, and the volume of sound generated by the loudspeakers depends at least on the air pressure setting and the position of the trigger.

32. A virtual coatings application system as recited in claim 29 wherein the system comprises a head-mounted display unit that displays the virtual spray painting environment to a user wearing the head-mounted display unit, and the display of the virtual spray painting environment appears to the user wearing the head-mounted display unit to be a three-dimensional environment in which the user is immersed.

33. A virtual coatings application system comprising: a virtual spray painting environment including a virtual surface;a computer;an instrumented spray gun controller having:a variably positionable trigger and a trigger sensor for detecting the position of the trigger and generating a trigger signal representing trigger position data in response thereto, the trigger signal being sent to the computer;a flow rate knob that rotates to allow for the adjustment of the maximum virtual fluid flow rate for the spray gun controller and a knob sensor that senses the position of the flow rate knob and generates a maximum virtual fluid flow rate signal which is sent to the computer and used to scale trigger position data;a motion tracking system that tracks the position and orientation of the spray gin controller with respect to the virtual surface; andsoftware in the computer that generates virtual spray pattern data in response to at least the trigger signal and the maximum virtual flow rate signal from the spray gun controller and the position and the orientation data received from the tracking system; andwherein a virtual spray pattern image is displayed in real time on the virtual surface in accordance with the accumulation of the virtual spray pattern data at each location on the virtual surface.

34. A virtual coatings application system as recited in claim 33 wherein the instrumented spray gun controller also comprises a fan size knob that rotates to allow for the adjustment of the virtual fan size for the spray gun controller and a sensor that senses the position of the fan size knob and generates a virtual fan size signal in response thereto which is sent to the computer.

35. A virtual coatings application system as recited in claim 33 wherein the tracking system is a combined inertial and ultrasonic, six degree of freedom tracking system and the instrumented spray gun controller further comprises a sensing element the measures linear and angular momentum as well as ultrasonic signals generated by an arrangement of ultrasonic transmitters.

36. A virtual coatings application system as recited in claim 35 further comprising a frame located over or adjacent a virtual work space, and wherein ultrasonic transmitters are mounted to the frame.

37. A virtual coatings application system comprising: a virtual spray painting environment including a virtual surface;an instrumented spray gun controller outputting virtual spray gun data;a motion tracking system that tracks the position and orientation of the spray gun controller with respect to the virtual surface;a computer programmed with software having a paint model that generates virtual spray pattern data for each timing cycle in response to at least the virtual spray gun data from the spray gun controller and the position and the orientation data received from the tracking system; anda graphical user interface that allows the user to select training set-up parameters and settings for the instrumented spray gun controller as well as view performance criteria.

38. A virtual coatings application system as recited in claim 37 wherein the selectable training set-up parameters consist of one or more of the following parameters: finish type, finish color, target mil thickness, surface type and surface color.

39. A virtual coatings application system as recited in claim 37 wherein the system enables the user to select finish color prior to a virtual training session and/or change finish color during the virtual training session.

40. A virtual coatings application system as recited in claim 39 wherein the system monitors average mil thickness of the combined virtual layers and displays this value on the graphical user interface.

41. A virtual coatings application system as recited in claim 40 wherein the display of the average mil thickness changes to a different color in order to alert the user that the average mil thickness has exceeded a targeted mil thickness selected on the graphical user interface.

42. A virtual coatings application system as recited in claim 37 wherein the settings for the instrumented spray gun controller that can be set from the graphical user interface consists of one or more of the following parameters: controller type, fan size, air pressure, and flow rate.

43. A virtual coatings application system as recited in claim 42 wherein the settings for the instrumented spray gun can be adjusted while the system is in use.

44. A virtual coatings application system as recited in claim 37 wherein the graphical user interface includes a prompt allowing the user to select whether the level of virtual paint accumulation at the respective locations on the virtual surface should be depicted via multiple colors.

45. A virtual coatings application system as recited in claim 37 wherein the graphical user interface includes a prompt asking the user whether overspray missing the virtual surface should be depicted.

46. A virtual coatings application system as recited in claim 45 wherein overspray is depicted in a color distinct from the color of the virtual surface and the color of the finish.

47. A virtual coatings application system as recited in claim 37 wherein the graphical user interface prompts the user to log in, and the system records user performance data for the logged-in training session.

48. A virtual coatings application system as recited in claim 47 wherein the graphical user interface includes a display summarizing previous performance scores for the logged-in user.

49. A virtual coatings application system as recited in claim 37 wherein the graphical user interface prompts the user whether to show virtual light beams on the virtual surface to simulate the use of a light beam targeting and positioning system.

50. A virtual coatings application system as recited in claim 37 wherein the graphical user interface prompts the user whether to display performance criteria in the virtual spray painting environment.

51. A virtual coatings application system as recited in claim 50 wherein performance criteria is displayed on a display screen on which the virtual surface is displayed.

52. A virtual coatings application system as recited in claim 50 wherein the performance criteria is displayed as an overlay within the virtual spray painting environment created by a head-mounted display unit, wherein a user wearing the head-mounted display unit is virtually immersed into a three-dimensional virtual spray painting environment.

53. A virtual coatings application system as recited in claim 37 wherein the system includes one or more icons set apart from the virtual surface in the virtual surface painting environment, the icons being toggled by pointing the instrumented spray gun controller at the respective icon and activating a trigger on the spray gun controller.

54. A virtual coatings application system as recited in claim 53 wherein at least one of the icons is an icon for a pause button.

55. A virtual coatings application system as recited in claim 53 wherein one of the icons is an icon prompting the user to select whether the level of virtual paint accumulation at respective locations on the virtual surface should be depicted via multiple colors.

56. A virtual coatings application system comprising: a virtual spray painting environment including a virtual surface;an instrumented spray gun controller outputting virtual spray gun data:a motion tracking system that tracks the position and orientation of the spray gun controller with respect to the virtual surface on the display screen;a computer programmed with software comprising a paint model that generates virtual spray pattern data for each timing cycle in response to at least the virtual spray gun data from the spray gun controller and the position and orientation data received from the tracking system; andperformance monitoring software that monitors the performance of the user at least in terms of virtual transfer efficiency and virtual build efficiency.

57. A virtual coatings application system as recited in claim 48 wherein at least some performance data is displayed in real time within the virtual spray painting environment on which the virtual surface is displayed.

58. A virtual coatings application system as recited in claim 57 wherein performance data is written to a data file on the computer, and is associated with the user via a login procedure.

59. A virtual coatings application system as recited in claim 56 wherein performance is monitored at least in part by monitoring the mil thickness at preselected locations on the virtual surface.

60. A virtual coatings application system comprising: a head-mounted display unit which displays a three-dimensional virtual spray painting environment including a virtual surface;an instrumented spray gun controller outputting virtual spray gin data;a motion tracking system that tracks the position and the orientation of the spray gun controller with respect to the virtual surface;a computer programmed with software that provides part image data for an image of the virtual surface, and that also generates virtual spray pattern data for each timing cycle in response to at least the virtual spray gun data from the spray gun controller and the position and orientation data received from the tracking system,wherein the image of the virtual surface is displayed within the virtual spray painting environment in accordance with the part image data and a virtual spray pattern image is displayed in real time on the virtual surface in accordance with the accumulation of the virtual spray pattern data at each location on the virtual surface, and further wherein the software has the capability of depicting accumulated virtual spray that misses the virtual surface as overspray in a color distinct from spray accumulating on the virtual surface.

61. A virtual coatings application system as recited in claim 60 wherein the system allows the user to change colors of the virtual spray during a training session.

62. A virtual coatings application system as recited in claim 52 wherein the software also provides stencil image data which overlays the past image of the virtual surface, and blocks virtual spray from accumulating over the areas on which the stencil covers the virtual surface.