Patent Application
20240052601
The present disclosure relates to a monitoring system for a work machine and a method of generating awareness of a surrounding area of the work machine.
A work machine, such as, a loader, a dozer, and the like may be used to perform various work operations at a worksite. While the work machine is operating at the worksite or travelling from one place to another, it may be desirable to monitor a surrounding area of the work machine to detect a presence of objects or people. For this purpose, one or more imaging devices may be mounted on the work machine. The imaging devices may generate images or videos of the surrounding area of the work machine.
The imaging devices are typically mounted on the work machine in a fixed orientation, and a field of view of the imaging devices may not cover each area surrounding the work machine, thereby creating blind spots for machine operators. In some cases, a mounting device may be used to mount a corresponding imaging device on the work machine. The mounting devices may be used to adjust an orientation of the corresponding imaging device. However, such mounting devices offer limited adjustment capability and may not cover each area surrounding the work machine. Further, as the presence of the blind spots may be dynamic in nature, new blind spots may be created each time the imaging devices are adjusted. Thus, in order to efficiently operate the work machine and to create awareness of the surrounding area of the work machine, it may be imperative that the machine operator is aware of the blind spots around the work machine.
CN100364334C describes a monitoring system and its method using mobile communication terminal dead angle area. A camera is mounted at a rotatable driving part set at the dead angle area of a transport means. One communicating manner is selected from the close range wire/wireless communicating manners set at the mobile communicating terminal and is connected with a camera driving controller which is used for controlling the driving part and is switched-in camera images. The filmed images mentioned above are displayed at a mobile communicating terminal display screen. Once obstacles appear, the camera driving part mentioned above is started-up to change the pointing angle of the camera, and the distance from the obstacle is displayed at the mobile communicating terminal display screen according to the pointing angle to ensure that users can make sure the dead angle area through visions and can grasp the distance to the obstacle. Hence, on the basis of the more specific distance data, the effects of transport means driving safely are obtained.
In an aspect of the present disclosure, a monitoring system for a work machine is provided. The monitoring system includes at least one imaging device disposed on the work machine. The at least one imaging device has a field of view. The monitoring system also includes at least one mounting device adapted to support the at least one imaging device. The at least one mounting device is adjustable to a plurality of orientations. The at least one mounting device includes at least one sensor configured to generate an input signal indicative of a current orientation of the at least one mounting device. The monitoring system further includes a controller communicably coupled to the at least one imaging device and the at least one mounting device. The controller includes one or more memories and one or more processors communicably coupled with the one or more memories. The one or more processors are configured to receive the input signal from the at least one sensor. The one or more processors are also configured to determine a location of the at least one imaging device on the work machine and the field of view of the at least one imaging device. The one or more processors are further configured to determine a presence of one or more blind spots around the work machine based on the current orientation of the at least one mounting device, the location of the at least one imaging device on the work machine, and the field of view of the at least one imaging device. The one or more processors are configured to generate a pictorial view depicting the one or more blind spots around the work machine. The monitoring system includes a user interface configured to receive the pictorial view from the one or more processors. The user interface is further configured to display the pictorial view thereon.
In another aspect of the present disclosure, a monitoring system for a work machine is provided. The monitoring system includes a controller communicably coupled to at least one imaging device associated with the work machine and at least one mounting device adapted to support the at least one imaging device. The controller includes one or more memories and one or more processors communicably coupled with the one or more memories. The one or more processors are configured to receive an input signal indicative of a current orientation of the at least one mounting device. The one or more processors are also configured to determine a location of the at least one imaging device on the work machine and a field of view of the at least one imaging device. The one or more processors are further configured to determine a presence of one or more blind spots around the work machine based on the current orientation of the at least one mounting device, the location of the at least one imaging device on the work machine, and the field of view of the at least one imaging device. The one or more processors are configured to generate a pictorial view depicting the one or more blind spots around the work machine.
In another aspect of the present disclosure, a method of generating awareness of a surrounding area of a work machine is provided. The method includes receiving, by a controller, an input signal indicative of a current orientation of at least one mounting device adapted to support at least one imaging device on the work machine. The method also includes determining, by the controller, a location of the at least one imaging device on the work machine and a field of view of the at least one imaging device. The method further includes determining, by the controller, a presence of one or more blind spots around the work machine based on the current orientation of the at least one mounting device, the location of the at least one imaging device on the work machine, and the field of view of the at least one imaging device. The method includes generating, by the controller, a pictorial view depicting the one or more blind spots around the work machine. The method also includes displaying, on a user interface, the pictorial view depicting the one or more blind spots.
Other features and aspects of this disclosure will be apparent from the following description and the accompanying drawings.
Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts.
Referring to
As shown in
Referring to
The monitoring system 200 includes one or more imaging devices 202, 204, 206, 208 disposed on the work machine 102. In the illustrated example of
In an example, the imaging devices 202, 204, 206, 208 may include a camera. In another example, the imaging devices 202, 204, 206, 208 may include a digital video camera, such as, an ethernet camera to provide an electronic motion picture acquisition. It should be noted that the imaging devices 202, 204, 206, 208 may include any other type of conventional imaging device that captures still images or videos.
The monitoring system 200 also includes one or more mounting devices 210 that supports the one or more imaging devices 202, 204, 206, 208. It should be noted that each imaging device 202, 204, 206, 208 is mounted on the work machine 102 by a corresponding mounting device 210. Thus, the monitoring system 200 may include four mounting devices 210 to support the four imaging devices 202, 204, 206, 208. The one or more mounting devices 210 are adjustable to a number of orientations. Based on the adjustment of the one or more mounting devices 210 to different orientations, the field of view “F1”, “F2”, “F3”, “F4” of the imaging devices 202, 204, 206, 208, respectively, may vary so as to capture images or videos of different portions of the surrounding area 122. The one or more mounting devices 210 includes one or more sensors 212 that generate an input signal “Il” indicative of a current orientation of the one or more mounting devices 210. It should be noted that the mounting devices 210 may include any number of the sensors 212, without any limitations. In some examples, the one or more sensors 212 may include, but is not limited to, a position sensor.
The mounting devices 210 may include brackets, coupling elements, and the like for movably coupling the mounting devices 210 with a stationary element, such as, a frame of the work machine 102. Further, each mounting device 210 may include an adjustment assembly (not shown). The adjustment assembly may receive signals for adjusting the mounting device 210 to different orientations relative to the stationary element of the work machine 102, as per application requirements. In an example, the adjustment assembly may include one or more actuators. It should be noted that each mounting device 210 may include any combination of elements that may allow adjustment of the mounting device 210 to different orientations.
The monitoring system 200 further includes a controller 214 communicably coupled to the one or more imaging devices 202, 204, 206, 208 associated with the work machine 102 and the one or more mounting devices 210 that supports the one or more imaging devices 202, 204, 206, 208. In some examples, the imaging devices 202, 204, 206, 208 and the mounting devices 210 may be communicably coupled to the controller 214, via a communication medium (not shown). Examples of the communication medium may include, but are not limited to, a Wide Area Network (WAN), a Local Area Network (LAN), an internet, an intranet, a cellular network, a satellite network, or any other known network for transmitting and receiving data. In various examples, the communication medium may include a combination of two or more of the aforementioned networks and/or other types of networks known in the art. Further, the communication medium may be implemented as a wired network, a wireless network, or a combination thereof. In an example, the controller 214 may be present onboard the work machine 102. In another example, the controller 214 may be present at the remote operator station 120 (see
The controller 214 includes one or more memories 218 and one or more processors 220 communicably coupled with the one or more memories 218. Further, the one or more processors 220 may be communicably coupled to the one or more sensors 212. The one or more processors 220 may be any device that performs logic operations. It should be noted that the one or more processors 220 may embody a single microprocessor or multiple microprocessors for receiving various input signals. Numerous commercially available microprocessors may be configured to perform the functions of the one or more processors 220. The one or more processors 220 may further include a general processor, a central processing unit, an application specific integrated circuit (ASIC), a digital signal processor, a field programmable gate array (FPGA), a digital circuit, an analog circuit, a controller, a microcontroller, any other type of processor, or any combination thereof. The one or more processors 220 may include one or more components that may be operable to execute computer executable instructions or computer code that may be stored and retrieved from the one or more memories 218.
In some examples, the mounting devices 210 are adjustable based on an input received from the operator. More particularly, the processors 220 may receive the input from the operator to adjust the mounting devices 210 to a desired orientation based on a preference of the operator. Accordingly, the processors 220 may control the adjustment assembly associated with the corresponding mounting device 210 for adjusting the mounting devices 210 to the desired orientation.
In another example, the mounting devices 210 may be automatically adjustable based on the preference of the operator. For example, the mounting devices 210 may be automatically adjusted based on a change in a state of the work machine 102. More particularly, if the work machine 102 switches to a parked state, the mounting devices 210 may be automatically adjusted so that the imaging devices 202, 204, 206, 208 can provides improved ground coverage. Further, when the work machine 102 is not in the parked state, the mounting devices 210 may be automatically adjusted so that the imaging devices 202, 204, 206, 208 may provide optimum views of the surrounding area 122 of the work machine 102. In such examples, the memories 218 may contain prestored information related to preferences of multiple operators. Further, the processors 220 may identify the operator who is currently in charge of the work machine 102. Based on the identification, the processors 220 may automatically adjust the mounting devices 210 to a desired orientation, as per the preference of the identified operator.
The monitoring system 200 further includes a user interface 222 communicably coupled to the controller 214. In an example, the user interface 222 is present within the operator cabin 118 of the work machine 102. In another example, the user interface 222 is present at the remote operator station 120 that is externally located relative to the work machine 102. In yet another example, the user interface 222 may be present with the personnel 108 (see
The user interface 222 includes a display screen 224 (shown in
Further, the one or more processors 220 receive the input signal “Il” indicative of the current orientation of the one or more mounting devices 210. Specifically, the one or more processors 220 receive the input signal “Il” from the one or more sensors 212. It should be noted that the input signal “Il” may be generated each time the one or more mounting devices 210 are adjusted, so that the processors 220 have a real-time data of the current orientation of the mounting devices 210.
Further, the one or more processors 220 determine a location of the one or more imaging devices 202, 204, 206, 208 on the work machine 102 and the field of view “F1”, “F2”, “F3”, “F4” of the one or more imaging devices 202, 204, 206, 208. The location of the imaging devices 202, 204, 206, 208 may be prestored within the memories 218 of the controller 214. Further, the processors 220 may retrieve information pertaining to the location of the imaging devices 202, 204, 206, 208 from the memories 218. In some examples, the memories 218 may also store information related to dimensions of the work machine 102. The dimensions of the work machine 102 may be used to determine the location of the imaging devices 202, 204, 206, 208 with respect to one or more portions of the work machine 102, such as the operator cabin 118 or one or more work implements of the work machine 102. In an example, the field of view “F1”, “F2”, “F3”, “F4” of the imaging devices 202, 204, 206, 208, respectively, may be prestored within the memories 218 of the controller 214 and may be retrieved from the processors 220 as and when required. In another example, the field of view “F 1”, “F2”, “F3”, “F4” may be a dynamic parameter that may be updated in real-time each time the mounting devices 210 are adjusted.
Further, the one or more processors 220 determine the presence of one or more blind spots 226, 228 (shown in
Further, the one or more processors 220 generate a pictorial view “P1” depicting the one or more blind spots 226, 228 around the work machine 102. Specifically, the one or more processors 220 generate the pictorial view “P1” depicting the one or more blind spots 226, 228 each time the one or more mounting devices 210 are adjusted. More particularly, when the mounting devices 210 are adjusted, the field of view “F1”, “F2”, “F3”, “F4” of the imaging devices 202, 204, 206, 208, respectively, may change due to which a location of the blind spots 226, 228 may vary. Thus, the processors 220 may generate an updated pictorial view depicting the blind spots 226, 228 each time the mounting devices 210 are adjusted.
Furthermore, the user interface 222 receives the pictorial view “P1” from the one or more processors 220. Moreover, the user interface 222 displays the pictorial view “P1” thereon. It should be noted that the pictorial view “P1” may include a still image or a video that is displayed on the display screen 224 of the user interface 222. In addition to the pictorial view “P1”, the user interface 222 may also provide audible alerts (such as, voice messages) to the operator regarding the presence of the blind spots 226, 228. In some examples, the user interface 222 may flash warning lights to alert the operator regarding the presence of the one or more blind spots 226, 228. It may be noted that the user interface 222 may use a single technique or a combination of techniques to alert the operator regarding the presence of the blind spots 226, 228, and the present disclosure is not limited by a technique of alerting the operator regarding the presence of the blind spots 226, 228.
As shown in
The pictorial view “P1” further depicts the field of view “F1”, “F2”, “F3”, “F4” of the one or more imaging devices 202, 204, 206, 208. As illustrated herein, the field of views “F1”, “F3” of the imaging devices 202, 206 overlap with each other, and the field of views “F 1”, “F4” of the imaging devices 202, 208 overlap with each other. In some examples, each field of view “F 1”, “F2”, “F3”, “F4” may be depicted by a unique indicating feature. For example, the field of views “F1”, “F2”, “F3”, “F4” may be depicted by different colors, different hatchings, or any other indication features, without any limitations.
Referring again to
Further, in some examples, the operator seated at the remote operator station 120 (see
Further, the one or more processors 220 transmit the one or more pictorial views “P1”, “P2”, “P3” from the number of pictorial views “Pn” based on an operator input “I2”. More particularly, the processors 220 may receive the operator input “I2” via the user interface 222. Accordingly, one or more of the pictorial views “P1”, “P2”, “P3” may be displayed on the user interface 222. Thus, when the operator is in charge of operating the work machines 102, 104, 106, it may be possible to switch between the pictorial views “P1”, “P2”, “P3” of the work machines 102, 104, 106, based on the operator input “I2”. In some examples, the user interface 222 may display two or more of the pictorial views “P1”, “P2”, “P3” adjacent to each other, based on an input received from the operator so that the operator is aware of the blind spots around each work machine 102, 104, 106.
The present disclosure relates to the monitoring system 200 associated with the work machine 102. The monitoring system 200 includes the imaging devices 202, 204, 206, 208, the mounting devices 210 for supporting the corresponding imaging device 202, 204, 206, 208, and the controller 214 communicably coupled to the imaging devices 202, 204, 206, 208 and the mounting devices 210. The monitoring system 200 may generate awareness of the surrounding area 122 of the work machine 102. More particularly, based on the current orientation of the mounting devices 210, the location of the imaging devices 202, 204, 206, 208, and the field of view “F1”, “F2”, “F3”, “F4” of the imaging devices 202, 204, 206, 208, the monitoring system 200 may be used to determine the presence of the one or more blind spots 226, 228 around the work machine 102.
The monitoring system 200 described herein may provide a real time update of the blind spots 226, 228 present around the work machine 102. The monitoring system 200 may improve an accuracy with which the work machine 102 may be monitored. The monitoring system 200 may further allow adjustment of the mounting devices 210 to a desired orientation so that the area of the one or more blind spots 226, 228 is minimal. Thus, the monitoring system 200 may ensure that a maximum region of the surrounding area 122 of the work machine 102 lies in the field of view “F1”, “F2”, “F3”, “F4” of the imaging devices 202, 204, 206, 208, thereby improving monitoring of the work machine 102. Furthermore, the monitoring system 200 may also generate the pictorial view “P1” depicting the one or more blind spots 226, 228 that can be displayed on the user interface 222 present in the operator cabin 118 or the remote operator station 120. The monitoring system 200 may improve a confidence of the operator, improve operator awareness, and may reduce a scope of errors while operating the work machine 102.
Further, the monitoring system 200 may be embodied as a modular monitoring system that may be used with different types of work machines. The monitoring system 200 may be embodied as an aftermarket kit that may be used to monitor blind spots around various work machines. Further, the monitoring system 200 includes usage of conventional imaging devices that may be already present on existing work machines or can be easily mounted on existing work machines with minimum modifications. Moreover, the mounting devices 210 may include low-cost elements that may be easily retrofitted on existing work machines. Thus, the monitoring system 200 may be cost-effective to incorporate on various work machines.
Referring to
At step 404, the controller 214 determines the location of the one or more imaging devices 202, 204, 206, 208 on the work machine 102 and the field of view “F1”, “F2”, “F3”, “F4” of the one or more imaging devices 202, 204, 206, 208.
At step 406, the controller 214 determines the presence of the one or more blind spots 226, 228 around the work machine 102 based on the current orientation of the one or more mounting devices 210, the location of the one or more imaging devices 202, 204, 206, 208 on the work machine 102, and the field of view “F1”, “F2”, “F3”, “F4” of the one or more imaging devices 202, 204, 206, 208.
At step 408, the controller 214 generates the pictorial view “P1” depicting the one or more blind spots 226, 228 around the work machine 102. In some examples, the controller 214 generates the pictorial view “P1” depicting the one or more blind spots 226, 228 each time the one or more mounting devices 210 are adjusted. At step 410, the user interface 222 displays the pictorial view “P1” depicting the one or more blind spots 226, 228.
In some examples, the controller 214 generates the number of pictorial views “Pn” corresponding to the number of work machines 102, 104, 106. The controller 214 further transmits the one or more pictorial views “P1”, “P2”, “P3” from the number of pictorial views “Pn” based on the operator input “I2”.
In some examples, the controller 214 further generates the output signal “O1” for adjusting the one or more mounting devices 210 in order to minimize the area of the one or more blind spots 226, 228 based on the determination of the one or more blind spots 226, 228 around the work machine 102.
While aspects of the present disclosure have been particularly shown and described with reference to the embodiments above, it will be understood by those skilled in the art that various additional embodiments may be contemplated by the modification of the disclosed work machines, systems and methods without departing from the spirit and scope of the disclosure. Such embodiments should be understood to fall within the scope of the present disclosure as determined based upon the claims and any equivalents thereof.
