Patent Application
20170124776
1. Field of the Invention
The present invention relates to vehicles, and, more particularly, to an apparatus to direct the maintenance of an agricultural vehicle.
2. Description of the Related Art
Vehicles in general and agricultural vehicles in particular require maintenance and/or adjustments of their subsystems. An agricultural combine, for example, is used to harvest agricultural crops such as corn, soybeans, wheat and other grain crops. As the combine is operated maintenance items arise, either as a passage of operation or as detected by sensors on the combine.
Augmented reality (AR) systems provide a live direct or indirect view of a physical, real-world environment whose elements are augmented or supplemented by way of a computer-generated sensory input such as sound, video, graphics or GPS data. AR systems are used to enhance a person's current perception of reality. The augmentation process is conventionally carried out in real-time and in context with detected or known environmental elements, such as a battlefield environment. The AR technology includes object recognition allowing the information about the viewed environment to become interactive. Information about the environment and its objects can be overlaid on the view of the real world.
What is needed in the art is a vehicular maintenance system that uses AR and allows for easy identification of the item to be maintained and with information on the way to perform the maintenance.
The present invention provides an agricultural vehicle with an onboard maintenance record system that interacts with an AR system to assist the operator with maintenance actions.
The invention in one form is directed to an agricultural vehicle including a chassis, a controller carried by the chassis, and a vehicle maintenance system coupled to the vehicle. The vehicle maintenance system includes a viewing system configured to process an image of at least a portion of the vehicle and an augmentation system. The augmentation system is configured to identify an element of the vehicle in the image; receive information from the vehicle about a maintenance action relative to the element; and place a designator proximate to the element identified in the image, the designator indicating the maintenance action that is to be undertaken involving the element.
The invention in another form is directed to a vehicle maintenance system that includes a viewing system configured to process an image of at least a portion of the vehicle and an augmentation system. The augmentation system is configured to identify an element of the vehicle in the image; receive information from the vehicle about a maintenance action relative to the element; and place a designator proximate to the element identified in the image, the designator indicating the maintenance action that is to be undertaken involving the element.
The invention in yet another form is directed to a vehicle maintenance method including the steps of: viewing a portion of the vehicle with a viewing system, the viewing system displaying an image; identifying an element of the vehicle in the image; receiving maintenance information about the element from the vehicle; and augmenting the image by placing a designator proximate to the element identified in the image, the designator including information received in the receiving step.
An advantage of the present invention is that all of the maintenance information for the vehicle is contained in the vehicle.
Another advantage of the present invention is that the maintenance system uses interactive helps and prompting of the user by way of a device that a lot of people are already familiar with, such as a smart phone.
The above-mentioned and other features and advantages of this invention, and the manner of attaining them, will become more apparent and the invention will be better understood by reference to the following description of embodiments of the invention taken in conjunction with the accompanying drawings, wherein:
Corresponding reference characters indicate corresponding parts throughout the several views. The exemplifications set out herein illustrate embodiments of the invention and such exemplifications are not to be construed as limiting the scope of the invention in any manner.
Referring now to the drawings and more specifically to
The secondary display 24 may be a set of viewing glasses, as shown in
In
When the focus is on one issue represented by a designator, then other designators, such as designator 30, may be temporarily removed from the display, so that a focus on, as in this case, the air filter can be undertaken. More detailed information about the air filter can be shown as the operator approaches the air filter, such as suggested courses of action, what tools are needed, what steps are needed to access the air filter, when the air filter was last serviced or replaced, and when the next regular service is scheduled.
In
In
The present invention uses augmented reality (AR) with a tablet, smart phone, or set of computerized glasses, each being referred to as secondary display 24, communicating with a machine 10 over a wireless network to project a floating indicator 28, 30, 32 pointing out all inspection and service points (wear parts, lube points, etc.) on the machine (as seen on the screen of the tablet by use of the camera), combined with the engine hours of the machine 10 to indicate (for example, during a vehicle walk around) the number of hours left on the service point (service point arrow and indicator turns yellow if less than 20 hours and greater than 8 hours, red if less than 8 hours). When the user selects the service point on the machine 10, the display 24 will go to split screen mode (as shown in
The Augmented Reality App 18 receives information wirelessly from the machine 10 or mechanism 10 for superimposing information thereon. For example, you point your camera on your device directly at the machine 10. On your display you see a red flag that's pointing directly at the device that needs attention, in the body of the red flag, there is shown a data field that is populated by way of the wireless communication from the machine 10. For example, the data may be 3 hours left before the service interval is expired. After the service is completed, the user would touch the red flag on the screen, then a box would appear “service complete”, then the device 24 communicates back to the machine 10 that the service is complete, and that interval would be reset for attention in 500 hours, for example.
As another example of the Augmented Reality App receiving information wirelessly from the machine, there could be an active fault on the machine 10, such as shown in designator 28, with an air filter being partially blocked. When the user points the camera of his device 24 at the machine 10 engine bay area, on the display 24 you would see a red flag that is pointing directly at the air filter. In the body of the red flag, it shows a data field that is populated via wireless communication from the machine 10, which could be, for example, “Filter Partially Plugged, Filter Service Required”.
While this invention has been described with respect to at least one embodiment, the present invention can be further modified within the spirit and scope of this disclosure. This application is therefore intended to cover any variations, uses, or adaptations of the invention using its general principles. Further, this application is intended to cover such departures from the present disclosure as come within known or customary practice in the art to which this invention pertains and which fall within the limits of the appended claims.
