Patent Application
20140085613
Portions of the disclosure of this patent document contain material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the U.S. Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
1. Field of the Invention
The invention relates to, at least in part, a computer enabled device to control a projection system, set a projection boundary, and provide image correction, a controller for controlling in part an underwater projection system, a method of controlling underwater projection systems, a method of boundary setting in an underwater projection system or controller, a method of projection image correction in an underwater projection system or controller, and a method of using a controller in an underwater projection system.
2. Background of the Invention
In the field of image projection, a number of obstacles are provided to accurate projection of images, especially images on a non-uniform surface. This is further compounded when the image is projected in one composition or fluid environment and the observer is located outside of this composition or fluid environment. Or similarly when the projection of the image and subsequent observation of the image are made in one fluid environment and the projected image target is in another fluid environment, resulting in unacceptable distortion. An example of this distortion can be seen when one observes fish in a body of water, the position of the fish observed is distorted as is the size and sometimes shape of the fish from a vantage outside the water. Additionally, in projecting images in the interior of a body of water like those in water features or in targeted natural water settings, such as but certainly not limited to pools, fountains, spas, sand bars, and the like surface irregularities make correction in combination with the distortion effects difficult.
Some of the technical difficulties and issues in projecting images in these locations include accommodating the variables of transmissivity within the body of water, the variations between the transmissivity of the water and the interface with air and the location of an observer outside of the body of water, undulating contours and angles within the body of water and in the surfaces being projected upon within the body of water, observational variations based on the position of the observer, and similar variables which must be accommodated to provide ideal image viewing.
As such, a need exists for a projection system, a projection system controller, an underwater projection system, an underwater projection system controller, a computer enabled apparatus for controlling a projection system and/or an underwater projection system, a method of controlling a projection system and/or an underwater projection system, and a method of controlling and adjusting a projected image that overcomes these challenges and provides a robust image quality in a projected image with control of the projected light and, potentially, additional lighting. An in-situ projection system with user observed image control and correction is needed to address the myriad of complexities in projecting such quality images.
An aspect of the invention is to provide a projection system with a robust method of correcting an image based on sensed or input variables to accommodate variables in projection surface, fluid changes, and other environmental variables as well as spatial variables.
A further aspect of the invention is to provide a controller in an underwater projection system that can be programmed to sense or receive variable input for the dimension of a pool or water feature and define a target projection boundary and then receive data on variables of known images to provide a correction for the area defined by the target projection boundary and thereby correct images projected from the underwater projection system during operation of same.
Another aspect of the invention is to provide a controller which utilizes an at least one variable regarding the conditions of the water in the water feature to adjust the projection variables for the projection of an image within the water feature.
A further aspect of the invention is to provide an automated data collection system coupled to a controller to provide for automated data collection for one or both of a boundary setting module and an image correction mapping module.
Yet another aspect is to provide a controller for an underwater projector system which is in situ with the fluid environment of a projection boundary area and projects within a water feature, however the observation vantage is outside of the water feature.
The apparatus of the invention includes a computer enabled device controlling image projection calibration of an underwater projection system. The computer enabled device having a system controller communicating with said underwater projection system with a water feature in which said underwater projection system is located under water and projects an image therefrom under the water. Where the system controller has at least one of an at least one boundary mapping module and/or an at least one image correction module. The image is provided to the controller and projected through said underwater projection system after the system controller executes the at least one boundary mapping module to establish a boundary for the display of the image within the water feature. The system controller executes the at least one image correction module to establish image correction data for correcting the image displayed by said underwater projection system and controls projection of the image based on the correction data within the boundary for display of the image within the water feature by the underwater projection system.
The computer enabled device can also include a user interface communicating with the system controller, wherein data representing the mapping of the at least one boundary mapping module is input by a user through the user interface. The user interface can also communicate with the system controller, wherein the correction data of the image is established through observations from a user and the observations are input through the user interface. The user interface can be a graphical user interface. The graphical user interface can include a touch screen. The user input can be drawn onto the touch screen.
The data for the mapping of the at least one boundary mapping module input by a user can be provided through placement of projected boundary image within the water feature through the user interface in communication with the system controller controlling projection of the boundary image in the underwater projection system to establish a projection boundary line within an underwater surface within the water feature. The placement of projected boundary patterns with the water feature through the user interface can include placement around an obstruction within the water. The data for the correction data of the image can be established through observations from the user of a test image of known shape on the user interface and correction of the same test image projected through the underwater image projection system to establish the correction data. The test image can be projected sequentially or simultaneously at multiple points throughout the water feature.
The method of the invention includes a method of operating an underwater image projection system including executing a boundary mapping module for the underwater image projection system to establish a projection area boundary line or edge, executing an image correction module for the underwater image projection system to establish a correction map or calibration table for the area within the projection area boundary line or edge, retrieving image data for an image as input, adjusting the image data and manipulating the image based on the correction map or calibration table through a dynamic content manipulation system and projecting the image underwater in the water feature through the underwater projection system.
The execution of the boundary system module can project through the underwater image projection system an at least one boundary image. The at least one boundary image can displayed on a user interface and a user positions the boundary image at a point of projection. The at least one boundary image can be further manipulated to change its shape to correspond to a contours of the water feature at the point of projection within the water feature. The placement of the at least one boundary image can further comprise the placement of multiple boundary images sequentially. The placement of the at least one boundary image can further comprise the placement of multiple boundary images simultaneously. The placement of the at least one boundary image can include the placement of multiple boundary images around an obstruction.
The method can further include applying additional corrective variables for the water feature analyzed as input by a user or sensed through sensors in communication with the underwater projection system. The additional corrective variables can include at least one of a variable relating to placement, timing, display of the at least one boundary image or at least one variable relating to data collected, stored, programmed or utilized for calculations relating to the water feature.
The execution of the image correction module can project a test image of a known shape into the water feature to produce an uncorrected projected image. The uncorrected projected image can be analyzed and a correction determined by an observation made from outside the water feature. The correction can be applied to the test image of a known shape and the underwater projection system projects the test image of a known shape with the correction to produce an intermediary projected image. The intermediary projected image can be analyzed and the corrections are repeated until the projected intermediary image is substantially similar to the known image as projected. The corrections can be observed by a user and a user interface prompts input of the observed uncorrected and intermediary images by the user. The user interface can be a graphical user interface with a touch screen and the user draws the observed shape and corrections are calculated from the variance of the drawn shape by a controller.
The method of the instant invention also includes a method of establishing a projection area boundary in a water feature defining a projection area for an underwater image projection system, including projecting an at least one boundary image in the water feature; adjusting a position of the at least one boundary image; adjusting or selecting the shape of the at least one boundary image to comport with a surface within the water feature at the position; and drawing and storing a projection area boundary based on the at least one boundary image placement and shape.
The method can also include adjusting the position of the at least one boundary image using a user interface. The user interface can be a touch screen enabled device. The boundary image can be configured through a user changing the shape of the boundary image with the touch screen user interface. The boundary image shape can be adjusted through selection of shapes in a menu or palette of shapes.
The method of the instant invention also includes a method of correcting an image in a water feature projected by an underwater image projection system from an observation point outside of the water feature, including observing an uncorrected projected image of a known or expected shape at a specified target area of said water feature alone or across a defined projection area as set by a defined boundary; inputting observations or sensed variables related to the variance of the uncorrected projected image from the known shape at the specified target area of said water feature alone or across the defined projection area as set by a defined boundary; projecting the uncorrected projected image with an at least one correction data point to produce an intermediary projected image and observing and inputting further corrections until the intermediary projected image is the substantially similar to the known or expected shape; and storing the corrections applied to the image at the specified target area of said water feature alone or across the defined projection area as set by a defined boundary.
The method can include input of observations using a user interface. The user interface is a touch screen enabled device. The observing of an uncorrected projected image and the intermediary projected image can be done by a user from a vantage point outside of the water feature. The underwater projection system can include an at least two underwater image projection devices and the method can include coupling the at least two underwater image projection devices to share data regarding the stored projection area boundary line or edge and correction map or calibration table for each of the at least two underwater projection devices. The methods can be executed by a controller in a computer enabled device.
Moreover, the above objects and advantages of the invention are illustrative, and not exhaustive, of those which can be achieved by the invention. Thus, these and other objects and advantages of the invention will be apparent from the description herein, both as embodied herein and as modified in view of any variations which will be apparent to those skilled in the art.
Embodiments of the invention are explained in greater detail by way of the drawings, where the same reference numerals refer to the same features.
The instant invention is directed to an underwater projection system and controller for configuring and controlling image projection in situ. Water has a different refractive index from air which adds to the optical deformation of the light as it exits a lens surface if it is out of the water, thus in-situ projection simplifies the necessary corrections. In the exemplary embodiment, both the boundary setting and the image correction modules of the controller are executed underwater so that the perspective distortion correction from projector to target is avoided. Thus, in-situ projection reduces the complexity of the corrections applied by the controller. Additionally, the boundary measurements and corrections of the exemplary embodiment of the instant invention being applied in-situ provides consistency and ease of operation. The user views the image at the target as projected in situ and applies the corrections as outlined below from outside the water. Thus, the focus of the projection system is set so that the water is in contact with the exterior lens and participates in the final optical manipulation. In the absence of the water in contact with the lens, the size, position and focus of the objects are different and requires further adjustment calculations, which can be added in additional software in further embodiments of the instant invention. However, the exemplary embodiment of the invention described herein is directed to an underwater projection system. As water features often have complex geometric shapes that cannot be easily modeled onto software, this approach of in-situ projection with a user observing and guiding corrections from outside the water feature allows the user to simply move markers to specific points thus defining the boundary point by point and applying corrections across the defined boundaries as observed without the variation from the projector being out of the water. Thus, the exemplary embodiment provides a novel control function for an underwater projector.
In addition to the image projection elements of the underwater image projection system 100 the exemplary embodiment shown also includes optional ambient lights similar to conventional pool floodlights, that operate in conjunction with the image projection elements of the underwater image projection system 100 to provide optional ambient lighting effects 510. Although shown as a single device, the combination of the underwater projection system with an at least one ambient light (not shown) operating in a further installation is embraced in the instant invention or combined in similar manner. The underwater projection system 100 may also be operated independently without ambient lighting. Similarly, multiple underwater projection systems 100 working in the same water feature 210 are contemplated, as better seen in
In addressing the unique nature of projecting a two dimensional image on the non-uniform surface of most water features 210 it is necessary for the system controller 120 to operate a visual correction system in projecting images in motion in the water feature, along the water feature, and images appearing around the surface of the pool or water feature 210. To this end, the software or modules residing on the controller 120 include an at least one boundary mapping routine or module for the image projection surfaces within the pool in conjunction with or operating independently from an at least one correction routine or module for correcting the projected images.
This is exemplified in
Though a shape is shown, here a right angle corner, the shape of the test images shown in the figures is not exhaustive of what may be used. The test image simply needs to be discernible by the user and/or the system and implemented in the drawing of the boundary. Additional intermediary steps may also be utilized and additional corrective variables may be analyzed as input by a user or sensed through a series of sensors. The placement, timing and display as well as the data collected, stored, programmed or utilized for calculations and/or compensations may be varied without departing from the spirit of the invention and the order shown is provided as a non-limiting example.
The process of placing each projected boundary shape 351-358 in the exemplary embodiment shown is similar in each instance. Again, reference is made to the stepwise exemplary embodiment displaying each projected boundary shape 351-358 in sequence and adjusting placement individually. This is further illustrated in an obstruction in the display area of the water feature is shown here as 230, a Jacuzzi or similar obstruction for example, that is dealt with by implementing an at least one additional marker 360, here shown as multiple markers “E” 357 and “F” 358, to accommodate the changes in the boundary necessary due to the obstruction.
Further additional markers can be added as necessary to outline the display boundary line or edge 370. In other non-limiting examples for instance this type of variation could also simply be caused by the shape of the pool or water feature or could encompass additional water features or other similar obstructions. Additional markers 360 could be used to compensate for these obstructions as well or further additional markers, which could be added to adjust the boundary. These types of obstructions can be typically found in a water feature installation such as pool. Similar obstructions may also occur in other types of water features and bodies of water, for instance water fountains from nozzles, waterfalls, level changes, and similar design aspects of most water features need to be accommodated. Additional obstructions may also be present in naturally occurring environments, such as, but certainly not limited to, those found on sand bars and jetties from rock outcrops and the like. The instant invention allows for adjustment of the underwater projection system 100 projection boundary 370 to compensate for these obstructions, like 230, as shown through the boundary setting component with flexible input of any number of additional marker elements 360.
Each boundary pattern or projected boundary shape 351-358 is selected by a user 450 through the GUI 400 as further described in relation to
In this instance, a series of corners and straights are provided and placed by the user 450 with the aid of the GUI 400 by morphing or configuring a selected boundary shape, for instance the corner shown, to define the projection field or area edge or boundary 370. When selected, the boundary pattern or projected boundary shape 351-358 is selected and moved to a selected edge or boundary or perimeter 370 and fit to define various contour points. In this instance, the projected test shape “A” 353 is projected to a corner and the shape is configured as a right angle. Additional methods of inputting shapes and outlines can be provided and will function to provide the necessary guidance for boundary setting in the underwater projection system. For instance, using a pen or stylus to draw a boundary and then interpolating that boundary on a picture of the desired area is just one further example of boundary definition processes that might be utilized. The result being, as shown in
In this way the additional modules including the automated data acquisition module 370 can capture the information when instructed and create a boundary line 370 setting for the underwater projection system 100. This purpose can be served with other sensors such as ultrasonic or LIDAR or other active or passive sensors as noted. Generally, if the data acquired is to be for the projection border component of the projection border and image correction routine module 610, a passive system to detect the edge of the pool or a passive system alone or in conjunction with an active system can be utilized. This data could then be coupled to the projection border and image correction routine module 610 to set the edge of the water feature and then define a projection area in the boundary as noted above in relation to
An active sensor can also be used for more detailed analysis of the surfaces and an improved initial boundary and projection area determination for the projection border and image correction routine module 610. The can be, for example but is certainly not limited to, providing data in the image correction portion of the module 610 to provide the initial data points similar to the user driven setup or provide for a more automated system for initially mapping the water feature and the projection points of test images. In addition, in a still further automated embodiment a further camera or other image sensing device (not shown) can be used to visualize the boundary images and automatically calculate the correction inputs in a manner similar to the input from the user as explained above. By visual comparison from an observation point which is then compared to a stored known target image by image comparison software.
The data from the automated data collection module 730 can be used in conjunction with the user 450, wherein steps similar to those of the previously disclosed embodiments correct the initial setup from the automated detection modules. Alternatively, the existing modules may function directly from with automated data collection modules 730, providing a user 450 with a more simplified interface, allowing for simple deformation on screen to “adjust” a final image projection determined by the map generated by the boundary setting module and image correction module using data from the automated detection module for instance.
In any case, the result of the process used is delineation in the pool or water feature or target of a boundary 370 within a body of water 220 of a projection area 375 with a projection area boundary edge 370 for the underwater image projection system 100 to project within and for which a correction map 620 is then defined for the water feature or target 210.
In the exemplary embodiment disclosed herein, the GUI 400 is used to interactively shape the boundary such that the corner is opened to the dotted boundary line edge or boundary perimeter 370, as shown. The angle of the boundary line defined by this interaction that maps the corner at the location of projected test shape “A” 353 in the pool. A further edge indicator projected shape is selected by the user 450 and projected as a straight line “B” 354. This is placed as shown and the boundary 370 begins to be constructed by controller 120 joining, without displaying, the edges defined by the projected test shapes A and B 353, 354. The system proceeds with the user 450 selecting and placing components C-G in place 355-351 to define the edge boundary or boundary line 370 of the projection system. The obstruction 230 is avoided by placing the projected test shapes E and F in such a way as to modulate morph or mutate the boundary around the obstruction 230. Each projected test image 351-358 is selected and placed by a user interacting with the GUI 400. This can include the use of a guided or step by step wizard program, as further discussed herein. This placement is stored along with the modulation, morphing or configuration of the selected images and a projection image display boundary 370 is set.
As noted above, a “wizard” or prompting program is used in the exemplary embodiment of the instant invention. The wizard guides the user 450 through a series of onscreen prompts and proceeds through the process of selecting boundary images and then adjusting these shapes to fit the water feature. As shown, an outline of the pool 470 is projected on the screen. This can be, for instance but is certainly not limited to, an image derived from direct input of the user, for instance by providing a subroutine to draw the pool shape and any obstacles or it can be from a stored library of pool shapes or a further embodiment may utilize an actual picture of the pool using an edge detection algorithm to define the shape or similar mechanisms for inputting the image of the shape of the pool 900 and displaying it on the GUI 400. This may be input by the user or may be derived from operation of the GUI 400, such as operation of the GUI through a cellular phone with a camera or similar image capture device.
The user selects projected boundary shapes 351-358, as noted above in relation to
Once placed, the GUI onscreen test image 411 is greyed out or unhighlighted, here shown as being in this state with a dotted line. In the exemplary embodiment shown three of four of the GUI onscreen test images 411, 412, and 413 have been placed and are unselected. User 450 is placing a final GUI onscreen test shape 414 as shown by dragging it into position on the screen 440 a progress bar 420 is shown provides prompts and shows steps or percentage to completion. An additional prompt (not shown) would prompt user 450 to the next software module. This may include the deformation correction module or a further module for operating the pool or the underwater projection display system 100 or a related system. However, to operate properly in an underwater environment or in projecting on a non-uniform surface, the underwater projection system of the instant invention must be able to compensate for both variations in the projected image surface, variations in fluid variables and variations in positional variables within the water feature or target projection area.
Reference is made below to an exemplary method of inputting these deformations by the user 450 through the GUI controller 400. Additional methods of input, interaction, and correction are also contemplated. A comparison to target implementation is made here to compensate for variables in both the shape of the projection surface and other variables, some non-limiting examples being for instance those involving fluid or transmission properties of the fluid or specific fluid flow characteristics, which are in the method described compensated for together. Other schema for determining variance in a projected image could include specific measurement and compensation for the variables in both the projection surface and the projected media as well as spatial variables. These could include triangulation and image distortion based on calculations compensating for density, transmissivity, time of day, water temperature, surface composition, surface color, background color and the like. In either case, the basic operation of the module will result in an image correction or distortion adjustment/control map that provides a corrected image being projected from an underwater source to remain consistent as observed by a user of the projection system based on corrections for observed and/or measured variables to the input caused principally by the projection within the water media, the non-uniform target surface, and the spacial dynamics of a body of moving fluid that interfaces with a fluid of a different density that the point of observation is located within.
In broadest terms, the user interface, here GUI 400, can be used to input observations or sensed variables of the variance of the uncorrected projected image 320 of a known shape and these variances can be used as corrections in the projection for that targeted area of the water feature 210 across a defined projection area as set by a boundary. The process of entering the observed or sensed variances, for example from a point outside the water feature, can be repeated through several intermediary projected results until the projected image 310 is modified to compensate for projection variables and is thus transformed from the uncorrected image 320 to the corrected image 330. These corrections, once entered, can be stored and used during the operation of the underwater projection system 100 to correct these variances in the projected images as they are displayed in real time. The projected image may be a still image, a series of images shown as an animation, a video or the like in any format readable by the controller as desired by the user. More specifically, an animation or series of animations is specifically embraced by the instant invention, which may include the use of one or more projection elements or projection systems to provide for fully animated “shows” or features shown on the surfaces of the underwater feature as desired by a user. Some specific examples would include, but certainly not be limited to, wildlife scenes, advertisements, animated caricatures, visually appealing geometric shapes and displays and the like.
In the exemplary embodiment shown, as best seen in
The objective is to convey what is seen by the user 450 so that a degree of correction can be stored through the series of adjustments made to the output image. These adjustments are then analyzed, stored, and the projected image test pattern in the pool is updated as the adjusted figure on the screen. This is continued until the target shape 411 projected on the screen is satisfactorily met in the projection within the pool, as seen in
In this exemplary embodiment, the operating parameters are adjusted for multiple locations represented by the displayed test figures or images 481-484. Again, these are used as the target for the user input uncorrected shape 490. Once all the corrected shapes are achieved and the data is stored, a calibration table or correction map is compiled by the controller. In cases where the input is sequential, the correction map 620, as shown in
The calibration table or correction map 620 is stored by the controller 120 as best seen in
In
As seen in the figure, the instant invention comprises a GUI controller 400 in communication with a system controller 600. The software in the exemplary embodiment depicted is located on the systems controller 120. However it may also be located on the GUI 400 and its controller. The exemplary embodiment show utilizes a projection border and image correction routine module 610 to calibrate and establish the boundary 370 of the projection area and produce a calibration table or correction map 620 as noted above. The data from this execution of this module in conjunction with the GUI controller 400 is stored as a calibration table or matrix or correction map 620. The stored information can be contained in a non-volatile memory and retained after powering down. In this way the adjustments remain until the user or the system prompts recalibration, as noted above. Alternatively, recalibration may be forced at each startup, for instance if calibration utilizes sensors and is fully automated or may require an active election to save by a user and then saved into non-volatile memory. Content, such as image 520, is sent into the system through a content module 630. This content may be uploaded to or streamed or otherwise provided through the content module 630. The data from the content module together with the calibration map 620 are then provided to the Dynamic Content Manipulator (DCM) module 640.
The DCM 640 is responsible for manipulating the data received form the content module 630 with the measurements made in the calibration table or map 620. The data is essentially corrected for the calculated variances in the image as projected across the entirety of the bounded projection area, as herein described above. The result of the data of the content module 630 being manipulated by the data of the calibration table or map 640 is a control output for the projection system 100 and is communicated through the projection system controller interface 650 to the projection system controller. Additionally, the DCM 640 may also incorporate additional variables representing environmental variances. The can include for instance additional corrective variables including at least one of a variable relating to placement, timing, display of the at least one boundary image or at least one variable relating to data collected, stored, programmed or utilized for calculations relating to the water feature or environmental information on the water feature such as certainly not limited to water temperature, salinity, measure visibility, depth, material data, and the like.
When used in conjunction with one another, the at least two underwater image projectors have some unique control feature issues. Firstly depending on the overall size of the pool and the boundaries circumscribed and stored on each controller, an additional step of handling overlapping projections must be included in any routines conducted by either of the at least two underwater image projection systems 100, 101. For example as A is moved it could pass to locations outside of the boundary of the first underwater image projection systems, it may pass into the bounder of the other system. Thus in addition to the DCM 640 an additional component must be provided to handle “handoffs” between the projection areas. This is provided in a separate module (not shown) that interrogates projection position data from each of the at least two under water projection systems 102A,102B. The result is a seamless projection of a continuous image across boundary lines or zones. It may also be that each of the at least two under water projection systems 102A,102B have defined projection boundaries that overlap. In this instance the at least two under water projection systems 102A,102B must be able to communicate relative position and share markers. This avoids over wash or sudden and erratic fluctuations from projecting the same or conflicting images into a single. Finally, in the configuration having at least two under water projection systems 102A,102B, each system may simply operate as an independent and autonomous control. This would potentially result in distortion on interlacing images if the projections project the same or different things in the same spot.
In each case, the at least two under water projection systems 102A,102B may communicate with the single GUI controller in order to perform the manipulations on their respective images 380,390. Additional information may be shared in the way of edge distortion values and the calibration table or map, such that smooth transitions can be made of the images between each of the at least two underwater projection systems. Additionally, both lights may also be controlled be controlled by a single controller 102, as shown above, controlling multiple underwater image projection systems.
The embodiments and examples discussed herein are non-limiting examples. The invention is described in detail with respect to exemplary embodiments, and it will now be apparent from the foregoing to those skilled in the art that changes and modifications may be made without departing from the invention in its broader aspects, and the invention, therefore, as defined in the claims is intended to cover all such changes and modifications as fall within the true spirit of the invention.
This application claims the priority of U.S. provisional patent 61/678,606, filed Aug. 1, 2012, which is incorporated herein by reference.
