Patent Grant
11772562
Not applicable.
This disclosure generally relates a work vehicle system and method, and more specifically to a multi-camera vision or video display system and method for a work vehicle or fleet of work vehicles.
Work vehicles utilized within construction, agriculture, forestry, mining, and other industries commonly operate in challenging work environments. Operators are often required to carefully navigate such work vehicles, while performing various tasks and avoiding surrounding structures, neighboring work vehicles, and other obstacles. A given work vehicle may be a sizable and complex machine, requiring a relatively high level of operator skill to control the various functions of the work vehicle, in many instances including the movement of boom-mounted implements or other end effectors. Concurrently, visibility from the operator station or cabin of the work vehicle may be limited by the chassis of the work vehicle, by the positioning of implements relative to the cabin, or other visual hinderances. For this reason, certain work vehicles are now equipped with camera systems or “vision systems” providing operators with various views of the surrounding environment.
The disclosure provides a work vehicle vision system and method for a work vehicle and/or a fleet of work vehicles.
In one aspect, the disclosure provides a work vehicle with an operator cabin defining a first operator field of view (FOV) aligned with a longitudinal axis extending in a travel direction of the work vehicle, the operator cabin defining a second operator FOV extending along a transverse axis at an angle to the longitudinal axis; one or more display devices within the operator cabin defining a plurality of display areas arranged at a first side, a second side, and a central location between the first and second sides relative to the first operator FOV; and a plurality of cameras carried by the work vehicle and coupled to the one or more displays to display feeds at the plurality of display areas. The plurality of cameras includes a first set of the plurality of cameras at or trained on the first side, the second side, and the central location between the first and second sides relative to the first operator FOV; and a second set of the plurality of cameras at or trained on a first side, a second side, and a central location between the first and second sides relative to the second operator FOV. The work vehicle further includes a control system having processing architecture configured to control, or receive control instructions regarding, the selection of feeds to display at the plurality of display areas. The control system is configured to: when the work vehicle is in a first mode of operation, display a first set of feeds from the associated first side, second side, and central location of the first set of the plurality of cameras at the plurality of display areas so that the associated feeds are arranged at the first side, the second side, and the central location, respectively, relative to the first operator FOV; and when the work vehicle is in a second mode of operation, display a second set of feeds from the associated first side, second side, and central location of the second set of the plurality of cameras at the plurality of display areas so that the associated feeds are arranged at the first side, the second side, and the central location, respectively, relative to the second operator FOV.
In another aspect, the disclosure provides a vision system for a work vehicle with an operator cabin defining a first operator field of view (FOV) aligned with a longitudinal axis extending in a travel direction of the work vehicle, the operator cabin defining a second operator FOV extending along a transverse axis at an angle to the longitudinal axis. The vision system includes one or more display devices within the operator cabin defining a plurality of display areas arranged at a first side, a second side, and a central location between the first and second sides relative to the first operator FOV and a plurality of cameras carried by the work vehicle and coupled to the one or more displays to display feeds at the plurality of display areas. The plurality of cameras includes a first set of the plurality of cameras at or trained on the first side, the second side, and the central location between the first and second sides relative to the first operator FOV; and a second set of the plurality of cameras at or trained on a first side, a second side, and a central location between the first and second sides relative to the second operator FOV. The vision system further includes a control system having processing architecture configured to control, or receive control instructions regarding, the selection of feeds to display at the plurality of display areas. The control system is configured to: when the work vehicle is in a first mode of operation, display a first set of feeds from the associated first side, second side, and central location of the first set of the plurality of cameras at the plurality of display areas so that the associated feeds are arranged at the first side, the second side, and the central location, respectively, relative to the first operator FOV; and when the work vehicle is in a second mode of operation, display a second set of feeds from the associated first side, second side, and central location of the second set of the plurality of cameras at the plurality of display areas so that the associated feeds are arranged at the first side, the second side, and the central location, respectively, relative to the second operator FOV.
The details of one or more embodiments are set forth in the accompanying drawings and the description below. Other features and advantages will become apparent from the description, the drawings, and the claims.
Like reference symbols in the various drawings indicate like elements.
This disclosure generally relates to work vehicle multi-camera vision or video display system that enables an operator of a work vehicle to intuitively monitor multiple camera feeds providing different views around and/or within the work vehicle. The following describes one or more example embodiments of the disclosed vision system, method, work vehicle, or fleet, as shown in the accompanying figures of the drawings described briefly above. Various modifications to the example embodiments may be contemplated by one of skill in the art.
In the agriculture, construction, and forestry industries, work vehicles are utilized to perform tasks in various types of environments. Operators are often required to carefully navigate such environment, while performing various tasks and avoiding surrounding structures, neighboring work vehicles, and other obstacles. Visibility from the operator station or cabin of the work vehicle may be limited by the chassis of the work vehicle, by the positioning of the work implements relative to the cabin, or other visual hinderances, as well as by unfamiliar displays and systems. For this reason, work vehicles may be equipped with camera systems or “vision systems” providing operators with a display arrangement on which the camera feeds are presented for operator viewing.
This notwithstanding, it can be difficult to design a multi-camera vision system simultaneously providing an operator with multiple camera feeds, while further permitting the operator to readily receive and understand the feeds. Moreover, although many types of work vehicles have cameras, such views are conventionally very specific, such as views of a front hitch, a rear hitch, a discharge spout, and/or a seed or grain tank. However, these views typically only cover a portion of the overall operational need. Such needs may be dramatically different between operators, operations, and/or from one machine or model to the next. Additionally, operator attempts at obtaining preferred views in conventional systems may involve uncomfortable operator positions, leaving the cabin, using a spotter, or making assumptions about the vehicle environment. In a multi-camera system, it may be difficult for an operator to quickly ascertain the context of each view. Thus, if not properly designed, multi-camera vision systems can potentially intensify operator mental workload and increase heads-down time.
Overcoming many, if not all of the above-noted disadvantages, the systems and methods described herein enabling operators to monitor and navigate between multiple camera feeds associated with a work vehicle in a highly intuitive manner. In embodiments, the multi-camera vision system includes multiple cameras mounted at different locations about a given work vehicle to capture camera (live video) feeds of the work vehicle's surrounding environment from various different vantage points, viewpoints or viewing angles. Embodiments of the multi-camera vision system may include at least one, if not multiple rear, forward, left side, right side, and interior (e.g., in operational, filling, loading, unloading areas) vehicle cameras, depending upon work vehicle size, type, and other factors. The vehicle-mounted cameras (or, more simply, “vehicle cameras”) can each capture any desired portion or region of a work vehicle's surrounding environment, whether generally located to the front, to a side, or to the rear of the work vehicle.
Embodiments of the multi-camera vision system further include a controller and at least one display device, which functions within a cabin or other operator station of the work vehicle. During operation of the multi-camera vision system, the controller supplies the display device with appropriate signals (video output signals) to generate a multi-camera display on a screen of the display device. In the manner described herein, a highly intuitive multi-camera display is thus created enabling an operator of a work vehicle to readily appreciate the availability of multiple camera feeds, to simultaneously monitor the content of each camera feed, and to intuitively determine the camera feeds being presented on the various display devices. As such, the present disclosure is directed to an integrated system and method that enhance operational performance by intelligently showing needed information and/or imagery in a consistent and repeatable manner across cab platforms and products.
According to the present disclosure, many large and medium sized agricultural machines have common visibility themes that may accommodated by a platform implemented as a vision system or method across fleets of similar or different machines, thereby creating a familiarity and expectation for the operator from one machine to the other. Selected views may optimally reduce the need for an operator to swivel or leave the cab as often, thereby saving time, mitigating fatigue, and improving effectiveness and efficiency. This further results in improved operator experience that may be mapped and relatable from machine to machine to reduce set up time. Moreover, the system is flexible to accommodate supplemental views for unique visual perspectives without an added secondary video system that would otherwise be required. Thus, operators of different machine platforms can view feeds of similar areas around or features of each machine in similar or common display areas within the operator cabin. As a result, the vision system provides improved visibility during transport and in field to prevent damage to machines and property, aid in operation, and gain trust and validation in automation via a collection of standardized views.
In one example, one or more display devices may be provided along a forward upper headliner within the cabin to provide optimal views in the forward direction within an ergonomic field of view of the operator. The positioning of the display devices enable an operator to focus in the forward view on the task at hand with less distractions of physical contortions or frequent stopping to access job quality, spatial awareness, and the like. The present disclosure provides a modular or scalable solution that may be used across multiple platforms with varying needs to provide a backbone that may then be iterated upon to rapidly adapt to customer display needs. Moreover, the vision system may be configurable and have the capability to determine the operational context and implement an appropriate configuration to automatically render the images and information that are most appropriate or useful for the context. In some examples, information may be provided by the vision system within themes or situations of operation (e.g., planting, tilling, harvest, transport, etc.) to provide the most useful views and/or to reduce the setup time for each specific operation at different operator fields of view (e.g., a forward view for a transport situation, straight ahead view for a working situation, and/or an implement view when using an implement). In particular, different camera groupings for each field of view can be selected and displayed in an intuitive manner for the operator.
As appearing herein, the term “surrounding environment” is utilized in a broad sense to refer to a work environment or area located outside of a work vehicle, whether generally located to the front, to a side, or to the rear of the work vehicle, or any combination thereof. Further, the term “display” refers an image or “picture” generated on the screen of a display device, while the term “display device” refers to an image-generating device on which a display (e.g., the below-described multi-camera display) is presented. The terms “front,” “rear,” “right,” “central,” “left”, “forward”, and “rearward” may be defined relative to the orientation of an operator when seated in the operator station (e.g., cabin) of the work vehicle in a forward-facing position.
Additional details regarding the multi-camera vision or video display system and method for a work vehicle or fleet of work vehicles will be provided below.
Although the vision system 102 is described with reference to a sprayer as the work vehicle 100 in
In the example of
In the example of
Although not shown or discussed in detail, the work vehicle 100 may have additional systems and components typical for such vehicles, including a propulsion system, a steering system, a braking system, and various actuation systems that facilitate operation of the work vehicle 100 based on operator commands (e.g., via operator interface 106).
Returning to the vision system 102, the controller 104 generally implements operation of the vision system 102, as well as other systems and components of the work vehicle 100, including any of the functions described herein. The controller 104 may be configured as computing devices with associated processor devices and memory architectures. The controller 104 may encompass or may be associated with any practical number of processors (central and graphical processing units), control computers, navigational equipment pieces, computer-readable memories, power supplies, storage devices, interface cards, and other standardized components. The controller may also include or cooperate with any number of firmware and software programs or computer-readable instructions designed to carry-out the various process tasks, calculations, and control/display functions described herein. Such computer-readable instructions may be stored within a non-volatile sector of a memory associated with the controller. The memory can encompass any number and type of storage media suitable for storing computer-readable code or instructions, as well as other data utilized to support the operation of the multi-camera vision system. The memory may be integrated into the controller 104 in embodiments as, for example, a system-in-package, a system-on-a-chip, or another type of microelectronic package or module.
As such, the controller 104 may be configured to execute various computational and control functionality with respect to the vehicle 100. The controller 104 may be in electronic, hydraulic, or other communication with various other systems or devices of the vehicle 100, including via a CAN bus (not shown). For example, the controller 104 may be in electronic or hydraulic communication with various actuators, sensors, and other devices within (or outside of) the vehicle 100. The controller 104 may include or otherwise access various types of datastores, including at least one datastore associated with the vision system 102.
In some embodiments, the controller 104 may be configured to receive input commands and to interface with an operator via the operator interface 106, including typical steering, acceleration, velocity, transmission, and wheel braking controls, as well as other suitable controls. The operator interface 106 may be configured in a variety of ways and may include one or more joysticks, various switches or levers, one or more buttons, a touchscreen interface, a keyboard, a speaker, a microphone associated with a speech recognition system, or various other human-machine interface devices.
As introduced above, the vision system 102 may include a set of cameras 108 arranged around and within the work vehicle 100. Although a number of example cameras 108 are discussed below, additional cameras may be provided and/or one or more of the cameras discussed below may be omitted. Further, while primarily described below as video cameras capturing imagery falling within the visible band of the electromagnetic spectrum, other types of imaging devices (e.g., infrared cameras) can also be integrated into the multi-camera vision system in alternative implementations, with the imagery captured by such devices presented on the below-described multi-camera display accordingly. The vehicle cameras 108 are operably coupled to (that is, in signal communication with) the controllers 104 with either wireless or wired data connections. The set of cameras 108 includes a number of depicted cameras 108a-108g, as best shown by
The set of cameras 108 may include a front camera 108a positioned on a relative forward part of the chassis 124 of the vehicle 100 and oriented in a forward direction of travel. The front camera 108a may provide a relatively wide angle FOV to capture views of machine clearance and/or objects immediately in front of the work vehicle 100.
The set of cameras 108 may further include left and right front forward-facing cameras 108b positioned on a relative front or forward part of the side of the vehicle 100 and oriented in a forward direction of travel. The left and right front forward-facing cameras 108b provide and relatively narrow angle FOVs to capture views of machine clearance, obstacle, implements, and the like.
The set of cameras 108 may further include left and right rear forward-facing cameras 108c positioned on a relative rear part of the side of the vehicle and oriented in a forward direction of travel. The left and right rear forward-facing cameras 108c provide relatively wide FOVs for views of machine clearance, particularly when encountering relatively narrow roadways, oncoming attractions, and other obstructions.
The set of cameras 108 may further include left and right “mirror” cameras 108d positioned on a forward or middle part of the side of the vehicle 100 (e.g., in respective positions corresponding to side mirrors of an automobile) and oriented in a rearward direction of travel. The left and right mirror cameras 108d provide views of along the sides of the work vehicle 100.
The set of cameras 108 may further include left and right intersection cameras 108e positioned on a relatively forward or part of the side or hood of the vehicle 100 and oriented in a side-to-side or along transverse axis to the direction of travel. The left and right intersection cameras 108e provide views around any obstructions, particularly at intersections or upon entering roadways.
The set of cameras 108 may further include one or more “C-post” cameras 108f positioned on the sides of the vehicle cabin 128 and oriented in a side-to-side or along transverse axis to the direction of travel. The left and right C-post cameras 108f provide views along an extended boom 122, e.g., during a spraying operation.
The set of cameras 108 may further include at least one rear camera 108g that positioned on a rear portion of the vehicle 100 and oriented rearward. The rear camera 108g may be adjustable and generally provides views of trailing traffic and any center section or portion of towed implements (e.g., sprayers) at the rear of the vehicle.
One or more of the cameras 108 depicted in
In one particular example, the vision system 102 includes the display arrangement 110 with one or more display devices render displays based on commands from the controller 104. Generally, the display devices of the display arrangement 110 may be otherwise standard display devices within the cabin 128, such as LCD or LED screens. The display devices of display arrangement 110 may be mounted on the cab console, window-mounted, or head-mounted. In addition to display arrangement 110, other output devices may be utilized to convey all or a portion of the information discussed below, including analog or digital gauges, other graphical or textual displays, and audio or haptic mechanisms. Examples of the displays generated by the controller 104 and rendered on the display arrangement 110 are discussed in greater detail below.
As introduced above, the vision system 102 may, in some examples, include one or more sensors 112 that facilitate any of the operation described herein. For example, the sensors 112 may include environment sensors that collect information about the environment of the work vehicle 100; position sensors that collect information associated with the location, velocity, acceleration, heading, and/or orientation of the work vehicle 100; and/or vehicle sensors that collect information associated with the various components, systems, and/or implements on the work vehicle 100.
As described below, the vision system 102 operates to display selected feeds from the cameras 108 on the various display devices of display arrangement 110 based on display signals from the controller 104, which in turn are based on operator inputs via the operator interface 106, sensor data from sensors 112, stored data, and/or other information. As noted, the vision system 102 may be implemented across similar or otherwise dissimilar work vehicles, including the work vehicle discussed below with reference to
Generally,
Unless otherwise noted, the vision system 202 of vehicle 200 performs the functions and generates analogous views to the vision system 102 of vehicle 100. As such, many of the analogous components (e.g., controller, operator interface, display arrangement, and sensors) will not be described again. The vision system 202 is merely presented as a further example of how the vision system according to the disclosure herein may be applied across vehicles and vehicle models. In some examples, settings for the vision systems 102, 202, some of which are described below, may be shared via a cloud-based data transfer and/or via key fobs or other operator-based identification. Due to the disparate functions and vehicle configurations, the vision system 202 of
In addition to the work vehicles 100, 200 of
The views of
Generally, the primary display devices 140a, 140b, 140c provide intuitive views of the surrounding environment and/or within the work vehicle based in a manner that improves the spatial awareness and understanding of the operator. A number of additional or auxiliary display devices 142, 144 may be provided that depict various types of information associated with the work vehicle, the task, the environment, and/or the fleet. The display devices 142, 144 may render information such as job quality, job settings, automation status, visibility, control or convenience configuration, state or configuration feedback, management data, platform or vehicle-specific information, infotainment, and the like.
Referring to
Generally, the controller 104 may receive input data in a number of forms and/or from a number of sources, including the operator interface 106, cameras 108, and sensors 112, as well as various datastores, stored separately or in memory 116. Moreover, such input data may also come in from other systems or controllers, either internal or external to the work vehicle 100. This input data may represent any data sufficient to select and display the camera feeds on the appropriate display device 140a, 140b, 140c in a manner that provides an intuitive operator experience across work vehicles.
In particular, the controller 104 may receive camera feeds from the various cameras 108 representing the live video feed from each camera 108. At times, the controller 104 may receive input data from a number of cameras 108 that may be stitched together into a single video feed. Moreover, an individual camera 108 may be used to provide more than one video feed (e.g., breaking up a field of view into multiple feeds and/or separating out a feed into feeds with various levels of zoom or focus). The controller 104 may also receive sensor information from sensors 112 and/or operator input information from the operator interface 106; and/or platform or framework data from a datastore or memory 116.
In addition to the video feeds and images discussed below, the controller 104 may also provide and/or consider the camera feeds with respect to other systems associated with the work vehicle 100 (or fleet of vehicles). For example, the images of the camera feeds may be used in autonomous or semi-autonomous control operations. Moreover, such images may be remotely viewed or evaluated by a control or fleet operation.
Generally, the vision system 102 may have a preset or baseline setting that is common from vehicle to vehicle and/or from model to model. The preset baseline setting may result in analogous views being rendered on particular display devices, including modifying such views based on common conditions and/or situations. Such preset or baseline settings may be modified by the operator and/or owner of the work vehicle 100. In particular, the vision system 102 may receive operator information via the operator interface 106 or from another source (e.g., a key fob, smartphone, or remote communication). The operator settings may specify operator preferences for the displays discussed herein.
The controller 104 further receives sensor information, and such information may be evaluated to determine a present situation (or “mode”) for the vehicle 100. Situation information may also be provided via the operator interface 106. The situations may include, as examples, a transport situation, an intersection situation, a folding/unfolding/parking situation, a working situation, and/or a filling/levels situation, which are discussed in greater detail below. Additional situations may be considered, including a fault situation, a preparation situation, a warm-up situation, and the like. The sensor information used to determine the situation may include vehicle speed, vehicle direction, gear, status of implements, and the like.
Generally, the controller 104 selects from the camera feeds and renders selected camera feeds on selected display devices 140a, 140b, 140c. In some examples, the controller 104 may activate a selected feed (e.g., activate or power the assigned camera) upon selecting the feed. As noted above, the display devices 140a, 140b, 140c are generally arranged in a left, right, and center configuration, respectively. Generally, the feeds assigned to each display device are selected to collectively maintain an intuitive left, right, and center configuration. For example, “center” views (e.g., either forward or rearward facing) are typically rendered on the third display device 140c, and “side” views (e.g., forward or rearward facing) are typically rendered on the first and second display devices 140a, 140b, as appropriate for the left and right views.
In one example, the controller 104 selects the camera feeds based on the determined situation, discussed above. In particular, as the situation changes, the displays rendered on the display devices 140a, 140b, 140c may be modified to present the feeds that are most appropriate or relevant to the situation. Examples are provided below with reference to
As such, in effect, the vision system 102 may be applicable to a work vehicle fleet in which each work vehicle (e.g., vehicles 100, 200) has a set of display devices within the operator cabin, each defining a plurality of display areas arranged at a first side of the respective work vehicle (e.g., device 140a), a second side of the respective work vehicle (e.g., device 140b), and a central location of the respective work vehicle (e.g., device 140c) between the first and second sides relative to an operator FOV (e.g., each operator FOV for the respective vehicle); a set of cameras (e.g., cameras 108) carried by each work vehicle, each set of cameras having a respective at least one camera at or trained on the first side, the second side and the central location between the first and second sides relative to each operator FOV; and a controller 104 configured to display a standard set of feeds from the associated first side, second side, and central location of each set of cameras at the first set of display areas so that the associated feeds are arranged at the first side, the second side and the central location, respectively, relative to the respective operator FOV. In other words, the vision system 102 may be implemented across different models or platforms to provide a consistent “look and feel” for an operator or operators. In the examples of
Moreover, the vision system 102 may be applicable to a work vehicle with an operator cabin defining a first operator field of view (FOV) aligned with a longitudinal axis extending in a travel direction of the work vehicle and a second operator FOV extending along a transverse axis at an angle to the longitudinal axis; one or more display devices within the operator cabin defining a plurality of display areas arranged at a first side, a second side and a central location between the first and second sides relative to the first operator FOV; a plurality of cameras carried by the work vehicle and coupled to the one or more displays to display feeds at the plurality of display areas, the plurality of cameras including: a first set of the plurality of cameras at or trained on the first side, the second side and the central location between the first and second sides relative to the first operator FOV; and a second set of the plurality of cameras at or trained on a first side, a second side and a central location between the first and second sides relative to the second operator FOV; and a controller having processing architecture configured to control, or receive control instructions regarding, the selection of feeds to display at the plurality of display areas, the control system is configured to: when the work vehicle is in a first mode of operation, display a first set of feeds from the associated first side, second side and central location of the first set of the plurality of cameras at the plurality of display areas so that the associated feeds are arranged at the first side, the second side and the central location, respectively, relative to the first operator FOV; and when the work vehicle is in a second mode of operation, display a second set of feeds from the associated first side, second side and central location of the second set of the plurality of cameras at the plurality of display areas so that the associated feeds are arranged at the first side, the second side and the central location, respectively, relative to the second operator FOV.
As such, upon determining the appropriate settings, the situation, and/or context information, the controller 104 generates display commands for each display device 140a, 140b, 140c that to render the selected view, particularly of one or more work apparatuses performing a function. Upon generating the display signals and rending the displays, the controller 104 continues to collect information and update the display devices 140a, 140b, 140c.
As shown in
Referring to
Referring to
Referring to
Referring to
Accordingly, the vision system 102 depicted in
The views of
The vision system discussed herein may further be embodied as a method for a work vehicle. In particular, the method includes initiating operation of the vision system; determining settings associated with the vision system; determining a situation of the work vehicle associated with the vision system; identifying selected feeds based on the situation and/or operator input; identifying context information associated with the selected feeds; generating display signals for selected display devices for the selected feeds; and displaying the selected feeds on selected display devices.
Embodiments of the multi-camera vision system can be utilized in conjunction with various different types of work vehicles; and, further, that the vehicle cameras can capture views of any portions or regions of the environment surrounding or adjacent a work vehicle. Further, description of the manner in which the multi-camera display and the associated display arrangement, display devices, and/or graphical user interfaces appear in the illustrated examples is provided purely by way of non-limiting illustration; noting that the “look and feel” of the displays on the display devices will inevitably vary among embodiments and may be customizable to customer or operator preferences.
Accordingly, the present disclosure provides a multi-camera vision system and method for a work vehicle that provide more intuitive and consistent operator experience.
Also, the following examples are provided, which are numbered for easier reference.
1. A work vehicle comprising: an operator cabin defining a first operator field of view (FOV) aligned with a longitudinal axis extending in a travel direction of the work vehicle, the operator cabin defining a second operator FOV extending along a transverse axis at an angle to the longitudinal axis; one or more display devices within the operator cabin defining a plurality of display areas arranged at a first side, a second side, and a central location between the first and second sides relative to the first operator FOV; a plurality of cameras carried by the work vehicle and coupled to the one or more displays to display feeds at the plurality of display areas, the plurality of cameras including: a first set of the plurality of cameras at or trained on the first side, the second side, and the central location between the first and second sides relative to the first operator FOV; and a second set of the plurality of cameras at or trained on a first side, a second side, and a central location between the first and second sides relative to the second operator FOV; and a control system having processing architecture configured to control, or receive control instructions regarding, the selection of feeds to display at the plurality of display areas, the control system is configured to: when the work vehicle is in a first mode of operation, display a first set of feeds from the associated first side, second side, and central location of the first set of the plurality of cameras at the plurality of display areas so that the associated feeds are arranged at the first side, the second side, and the central location, respectively, relative to the first operator FOV; and when the work vehicle is in a second mode of operation, display a second set of feeds from the associated first side, second side, and central location of the second set of the plurality of cameras at the plurality of display areas so that the associated feeds are arranged at the first side, the second side, and the central location, respectively, relative to the second operator FOV.
2. The work vehicle of example 1, wherein the first set of the plurality of cameras includes a first forward facing camera positioned on the first side of the work vehicle and a second forward facing camera positioned on the second side of the work vehicle; and wherein the first mode of operation is a transport mode in which the first set of feeds includes the feed from the first forward facing camera provided on the display area at the first side and the feed from the second forward facing camera provided on the display area at the second side.
3. The work vehicle of example 2, wherein the second set of the plurality of cameras includes a first transverse facing camera positioned on the first side of the work vehicle and a second transverse facing camera positioned on the second side of the work vehicle; and wherein the second mode of operation is an intersection mode in which the second set of feeds includes the feed from the second transverse facing camera provided on the display area at the first side and the feed from the second transverse facing camera provided on the display area at the second side.
4. The work vehicle of example 3, further comprising at least one sensor coupled to the control system and configured to collect mode data, and wherein the control system is configured to automatically transition between the transport mode and the intersection mode based on the mode data.
5. The work vehicle of example 1, wherein the first set of the plurality of cameras includes a first camera positioned on a first C-post on the first side of the work vehicle and a second camera positioned on a second C-post on the second side of the work vehicle; and wherein the first mode of operation is a set-up mode in which the first set of feeds includes the feed from the first camera provided on the display area at the first side and the feed form the second camera provided on the display area at the second side.
6. The work vehicle of example 5, wherein the first set of the plurality of cameras further includes a third camera positioned on a rear portion of the work vehicle and facing rearward; and wherein, in the set-up mode of operation, the first set of feeds includes the feed from the third camera provided on the display area in the central location.
7. The work vehicle of example 5, wherein the second set of the plurality of cameras includes a third camera positioned on a forward portion of the work vehicle on the first side and facing forward and a fourth camera positioned on the forward portion of the work vehicle on the second side and facing forward; and wherein the second mode of operation is a working mode in which the second set of feeds includes the feed from the third camera provided on the display area at the first side and the feed from the fourth camera provided on the display area at the second side.
8. The work vehicle of example 5, wherein the second set of the plurality of cameras includes a third camera positioned on the first side of the work vehicle and facing in a first transverse direction and a fourth camera positioned on the second side of the work vehicle and facing in a second transverse direction; and wherein the second mode of operation is a working mode in which the second set of feeds includes the feed from the third camera provided on the display area at the first side and the feed from the fourth camera provided on the display area at the second side.
9. The work vehicle of example 1, further comprising a tank positioned behind the operator cabin; wherein the first set of the plurality of cameras includes a first camera oriented towards the tank; and wherein the first mode of operation is a filling mode in which the first set of feeds includes a first camera feed from the first camera provided on the center display area.
10. The work vehicle of example 1, wherein the first operator cabin includes a first headliner and the display area at the central location of the first set of display devices is positioned on or in the first headliner.
11. The work vehicle of example 10, wherein the first operator cabin further includes a first windshield at least partially bounded by a first side C-post and a second side C-post, and the display area at the first side location of the first set of display devices is positioned on the first side C-post in the first cabin and the display area at the second side location of the first set of display devices is positioned on the second side C-post in the first cabin.
12. The work vehicle of example 10, wherein the display area at the first side location of the first set of display devices and the display area at the second side location of the first set of display devices are positioned on the first headliner with the display area at the central location in between.
13. The work vehicle of example 1, wherein at least one of the cameras of the first set of the plurality of cameras is positioned on a work implement.
14. A vision system for a work vehicle having an operator cabin defining a first operator field of view (FOV) aligned with a longitudinal axis extending in a travel direction of the work vehicle, the operator cabin defining a second operator FOV extending along a transverse axis at an angle to the longitudinal axis, the vision system comprising: one or more display devices within the operator cabin defining a plurality of display areas arranged at a first side, a second side, and a central location between the first and second sides relative to the first operator FOV; a plurality of cameras carried by the work vehicle and coupled to the one or more displays to display feeds at the plurality of display areas, the plurality of cameras including: a first set of the plurality of cameras at or trained on the first side, the second side, and the central location between the first and second sides relative to the first operator FOV; and a second set of the plurality of cameras at or trained on a first side, a second side, and a central location between the first and second sides relative to the second operator FOV; and a control system having processing architecture configured to control, or receive control instructions regarding, the selection of feeds to display at the plurality of display areas, the control system is configured to: when the work vehicle is in a first mode of operation, display a first set of feeds from the associated first side, second side, and central location of the first set of the plurality of cameras at the plurality of display areas so that the associated feeds are arranged at the first side, the second side, and the central location, respectively, relative to the first operator FOV; and when the work vehicle is in a second mode of operation, display a second set of feeds from the associated first side, second side, and central location of the second set of the plurality of cameras at the plurality of display areas so that the associated feeds are arranged at the first side, the second side, and the central location, respectively, relative to the second operator FOV.
15. The vision system of example 14, wherein the first set of the plurality of cameras includes a first forward facing camera positioned on the first side of the work vehicle and a second forward facing camera positioned on the second side of the work vehicle; and wherein the first mode of operation is a transport mode in which the first set of feeds includes the feed from the first forward facing camera provided on the display area at the first side and the feed from the second forward facing camera provided on the display area at the second side.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
As will be appreciated by one skilled in the art, certain aspects of the disclosed subject matter can be embodied as a method, system (e.g., a work machine control system included in a work machine), or computer program product. Accordingly, certain embodiments can be implemented entirely as hardware, entirely as software (including firmware, resident software, micro-code, etc.) or as a combination of software and hardware (and other) aspects. Furthermore, certain embodiments can take the form of a computer program product on a computer-usable storage medium having computer-usable program code embodied in the medium.
As will be appreciated by one skilled in the art, aspects of the disclosed subject matter can be described in terms of methods, systems (e.g., control or display systems deployed onboard or otherwise utilized in conjunction with work machines), and computer program products. With respect to computer program products, in particular, embodiments of the disclosure may consist of or include tangible, non-transitory storage media storing computer-readable instructions or code for performing one or more of the functions described throughout this document. As will be readily apparent, such computer-readable storage media can be realized utilizing any currently-known or later-developed memory type, including various types of random access memory (RAM) and read-only memory (ROM). Further, embodiments of the present disclosure are open or “agnostic” to the particular memory technology employed, noting that magnetic storage solutions (hard disk drive), solid state storage solutions (flash memory), optimal storage solutions, and other storage solutions can all potentially contain computer-readable instructions for carrying-out the functions described herein. Similarly, the systems or devices described herein may also contain memory storing computer-readable instructions (e.g., as any combination of firmware or other software executing on an operating system) that, when executed by a processor or processing system, instruct the system or device to perform one or more functions described herein. When locally executed, such computer-readable instructions or code may be copied or distributed to the memory of a given computing system or device in various different manners, such as by transmission over a communications network including the Internet. Generally, then, embodiments of the present disclosure should not be limited to any particular set of hardware or memory structure, or to the particular manner in which computer-readable instructions are stored, unless otherwise expressly specified herein.
A computer readable signal medium can include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal can take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium can be non-transitory and can be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
As used herein, unless otherwise limited or modified, lists with elements that are separated by conjunctive terms (e.g., “and”) and that are also preceded by the phrase “one or more of” or “at least one of” indicate configurations or arrangements that potentially include individual elements of the list, or any combination thereof. For example, “at least one of A, B, and C” or “one or more of A, B, and C” indicates the possibilities of only A, only B, only C, or any combination of two or more of A, B, and C (e.g., A and B; B and C; A and C; or A, B, and C).
As used herein, the term module refers to any hardware, software, firmware, electronic control component, processing logic, and/or processor device, individually or in any combination, including without limitation: application specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or group) and memory that executes one or more software or firmware programs, a combinational logic circuit, and/or other suitable components that provide the described functionality. The term module may be synonymous with unit, component, subsystem, sub-controller, circuitry, routine, element, structure, control section, and the like.
Embodiments of the present disclosure may be described herein in terms of functional and/or logical block components and various processing steps. It should be appreciated that such block components may be realized by any number of hardware, software, and/or firmware components configured to perform the specified functions. For example, an embodiment of the present disclosure may employ various integrated circuit components, e.g., memory elements, digital signal processing elements, logic elements, look-up tables, or the like, which may carry out a variety of functions under the control of one or more microprocessors or other control devices. In addition, those skilled in the art will appreciate that embodiments of the present disclosure may be practiced in conjunction with any number of work vehicles.
The description of the present disclosure has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the disclosure in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the disclosure. Explicitly referenced embodiments herein were chosen and described in order to best explain the principles of the disclosure and their practical application, and to enable others of ordinary skill in the art to understand the disclosure and recognize many alternatives, modifications, and variations on the described example(s). Accordingly, various embodiments and implementations other than those explicitly described are within the scope of the following claims.
This application is a nonprovisional application, and claims priority to, of both of U.S. Provisional Patent Application No. 63/165,677 filed Mar. 24, 2021 and U.S. Provisional Patent Application No. 63/175,334 filed Apr. 15, 2021, each of which is hereby incorporated by reference.
| Number | Name | Date | Kind |
|---|---|---|---|
| 10259383 | Campbell | Apr 2019 | B1 |
| 10324461 | Ogura et al. | Jun 2019 | B2 |
| 10391940 | Izumikawa et al. | Aug 2019 | B2 |
| 10450169 | Oishi | Oct 2019 | B2 |
| 10589679 | Askeland | Mar 2020 | B1 |
| 10668854 | Imaizumi et al. | Jun 2020 | B2 |
| 11040660 | Yamaguchi | Jun 2021 | B2 |
| 11447928 | Hatake et al. | Sep 2022 | B2 |
| 11533835 | Kurogi | Dec 2022 | B2 |
| 20150217691 | Tanuki et al. | Aug 2015 | A1 |
| 20170315771 | Kerr | Nov 2017 | A1 |
| 20180027179 | Matsuzaki et al. | Jan 2018 | A1 |
| 20180370432 | Imaizumi et al. | Dec 2018 | A1 |
| 20190166760 | Palla et al. | Jun 2019 | A1 |
| 20200407948 | Seki | Dec 2020 | A1 |
| 20210002863 | Sakuta et al. | Jan 2021 | A1 |
| 20210043085 | Kreiling et al. | Feb 2021 | A1 |
| 20210155167 | Lynam | May 2021 | A1 |
| 20210246635 | Tsukamoto et al. | Aug 2021 | A1 |
| 20210291733 | Ding et al. | Sep 2021 | A1 |
| 20210307230 | Kurogi | Oct 2021 | A1 |
| 20210350681 | Imaizumi et al. | Nov 2021 | A1 |
| 20210377492 | Zieser | Dec 2021 | A1 |
| 20220074171 | Izumikawa et al. | Mar 2022 | A1 |
| 20220118915 | Pastoor | Apr 2022 | A1 |
| 20220212602 | Watanabe et al. | Jul 2022 | A1 |
| Number | Date | Country |
|---|---|---|
| 112013000566 | Nov 2014 | DE |
| 102020211905 | Apr 2021 | DE |
| 102021200028 | Aug 2021 | DE |
| 102021201728 | Sep 2021 | DE |
| Entry |
|---|
| USPTO Non-Final Office Action issued in U.S. Appl. No. 17/673,038 dated Feb. 21, 2023. (24 pages). |
| USPTO Non-Final Office Action issued in U.S. Appl. No. 17/673,026 dated Jan. 18, 2023. (30 pages). |
| YouTube, 360 Camera with Split-View Display—Ford How-To, https://www.youtube.com/watch?v=2asWY4qcJtl, Mar. 21, 2017 (2 pages). |
| YouTube, 2020 Silverado HD Invisible Trailer Mode and its 15 Cameras, https://www.youtube.com/watch?v=xox-Ym9ZW8k, Oct. 30, 2019 (1 page). |
| YouTube, How to Use the Different Camera Views of Surround View—BMW How-To, https://www.youtube.com/watch?v=RziW6FPfLGQApr. 17, 2019 (1 page). |
| YouTube, F-150 Backup Camera, https://www.youtube.com/watch?v=xtMCvXxH15o, Jul. 17, 2017 (1 page). |
| YouTube, Honda's Multi-Angle Rearview Camera, https://www.youtube.com/watch?v=MxO8k0a1nWg, Sep. 15, 2020 (2 pages). |
| German Search Report for application No. 102022206814.9 dated Feb. 28, 2023. |
| German Search Report for application No. 102022206817.3 dated Mar. 3, 2023. |
| German Search Report for application No. 102022206818.1 dated Mar. 6, 2023. |
| AGCO News Releases “AGCO Debuts the FendtOne Operator Interface—Next Generation of Popular Fendt 700 Series Tractors Introduced” at https://news.agcocorp.com/news/agco-debuts-the-fendtone-operator-interface, Aug. 25, 2020. (9 pages). |
| FENDT Smart Farming “FendtOne is Your Future—Jump Onboard” at https://www.fendt.com/us/smart-farming/fendtone, 2001. (6 pages). |
| New Holland iVT magazine article “Inspiring Style New Holland and Pininfarina are Making this a Vintage Year for Aesthetics with their Vineyard Straddle-Tractor Concept—Raise a Glass!” at iVTInternational.com, Mar. 2022. (6 pages). |
| USPTO Final Office Action issued in U.S. Appl. No. 17/673,026 dated May 2, 2023. (44 pages). |
| USPTO Final Office Action issued in U.S. Appl. No. 17/673,038 dated Aug. 18, 2023. |
| Number | Date | Country | |
|---|---|---|---|
| 20220305999 A1 | Sep 2022 | US |
| Number | Date | Country | |
|---|---|---|---|
| 63175334 | Apr 2021 | US | |
| 63165677 | Mar 2021 | US |
