Patent Application
20250153742
This application claims priority to Japanese Patent Application No. 2023-194048 filed on Nov. 15, 2023, incorporated herein by reference in its entirety.
The present disclosure relates to control devices and control systems.
Conventionally, a technique of remotely controlling a vehicle to perform autonomous driving has been known in the art (Japanese Unexamined Patent Application Publication (Translation of PCT Application) No. 2017-538619 (JP 2017-538619 A)).
When a mounted device mounted on a moving object such as a vehicle fails, an operator sometimes directly moves the moving object by riding and operating the moving object or by transporting the moving object using a towing device etc. However, when the operator directly moves the moving object, the number of man-hours required to move the moving object may increase.
The present disclosure can be implemented in the following aspects.
When in the remote manual driving mode, the moving object receives a manual control signal and moves using the received manual control signal, the manual control signal being a signal generated according to an operation of a maneuvering device by an external operator to control movement of the moving object, and the maneuvering device being located at a different location from the moving object. According to this aspect, the acquisition unit can acquire the result of inspection of the mounted device. The determination unit can determine based on the result of inspection of the mounted device whether the moving object is movable in the remote manual driving mode. The setting unit can set the driving mode of the moving object to the remote manual driving mode when determination is made that the moving object is movable in the remote manual driving mode. With the above configuration, when the mounted device fails, the operator can move the moving object in the remote manual driving mode without directly moving the moving object. This can reduce the number of man-hours required to move the moving object compared to the case where the operator directly moves the moving object.
The present disclosure can be implemented in various forms other than the control device and the control system. For example, the present disclosure can be implemented in the following forms: a control device, a control system, and a method for manufacturing a moving object; a control device, a control system, and a method for controlling a moving object; a computer program that implements the control method; and a non-transitory recording medium recording the computer program.
Features, advantages, and technical and industrial significance of exemplary embodiments of the disclosure will be described below with reference to the accompanying drawings, in which like signs denote like elements, and wherein:
The external sensor 300 is a sensor located outside the vehicle 100. The external sensor 300 in the present embodiment is a sensor that captures the vehicle 100 from the outside of the vehicle 100. The external sensor 300 includes a communication device (not shown), and can communicate with another device such as the server 200 by wired communication or wireless communication.
Specifically, the external sensor 300 is constituted by a camera. The camera as the external sensor 300 captures a captured image including the vehicle 100, and outputs the captured image as a detection result. Hereinafter, a camera as the external sensor 300 is also referred to as an “external camera”.
In the present disclosure, the “moving object” means a movable object, and is, for example, the vehicle 100 or an electric vertical takeoff and landing machine (so-called flying vehicle). The vehicle 100 may be a vehicle traveling by a wheel or a vehicle traveling by an infinite track, and is, for example, a passenger car, a truck, a bus, a two-wheeled vehicle, a four-wheeled vehicle, a tank, a construction vehicle, or the like. Vehicles 100 include battery electric vehicle (BEV), gasoline-powered vehicles, hybrid electric vehicle, and fuel cell electric vehicle. When the moving object is other than the vehicle 100, the expressions of “vehicle” and “vehicle” in the present disclosure can be appropriately replaced with “moving object”, and the expression of “traveling” can be appropriately replaced with “moving”.
The vehicle 100 is configured to be able to travel by unmanned driving. The term “unmanned driving” means driving that does not depend on the traveling operation of the occupant. The traveling operation means an operation related to at least one of “running”, “turning”, and “stopping” of the vehicle 100. The unmanned driving is realized by automated or manual remote control using a device located outside the vehicle 100 or by autonomous control of the vehicle 100. An occupant who does not perform the traveling operation may be on the vehicle 100 traveling by the unmanned driving. The occupant who does not perform the traveling operation includes, for example, a person who is simply seated on the seat of the vehicle 100 and a person who performs a work different from the traveling operation such as an assembling operation, an inspection operation, and an operation of switches while riding on the vehicle 100. Driving by the traveling operation of the occupant is sometimes referred to as “manned driving”.
Herein, “remote control” includes “full remote control” in which all of the movements of the vehicle 100 are completely determined from the outside of the vehicle 100, and “partial remote control” in which part of the movements of the vehicle 100 is determined from the outside of the vehicle 100. Also, “autonomous control” includes “full autonomous control” and “partial autonomous control”. In “fully autonomous control”, the vehicle 100 autonomously controls its own movement without receiving any information from a device external to the vehicle 100. In “partial autonomous control”, the vehicle 100 autonomously controls its own movement by using information received from a device outside the vehicle 100.
In this embodiment, the control system 50 is used in a factory that manufactures the vehicle 100. The factory reference coordinate system is a global coordinate system. That is, any position in the plant is represented by the coordinates of X, Y, Z in the global coordinate system. The factory includes a first location and a second location. The first location and the second location are connected by a track on which the vehicle 100 can travel. In the factory, a plurality of external sensors 300 are installed along the runway. The position of each external sensor 300 in the factory is adjusted in advance. The vehicles 100 move from the first location to the second location through the runway in accordance with at least one of the traction driving mode, the manned driving mode, the remote manual driving mode, and the automated driving mode according to the result of inspection of the mounted device DE.
The mounted device DE is a device mounted on the vehicle 100. The mounted device DE is, for example, a motion control system device, a mounting sensor, and an electric parking brake. The motion control system device is a mounted device DE for implementing the following three functions of the vehicle 100: “run,” “turn,” and “stop.” The motion control system device is, for example, a disc brake device having an electric power steering, a brake pump, and the like, an engine, a traveling motor, and an electric shift. The mounted sensor is a sensor mounted on the vehicle 100. Examples of the mounted sensor include an acceleration sensor, a yaw rate sensor, a millimeter wave radar, an in-vehicle camera, an in-vehicle LiDAR, and a steering angle sensor. Note that the type of the mounted device DE is not limited to the above.
In the towing driving mode, the vehicle 100 moves by being towed by a device other than the host vehicle 100, such as a towing device, without making any movement.
In the manned driving mode, the vehicle 100 travels by the traveling operation of the occupant. In the automated driving mode, the vehicle 100 travels in one of a remote automated driving mode in which automated remote control is performed using a device located outside the vehicle 100 and an autonomous driving mode in which autonomous control of the vehicle 100 is performed. In the present embodiment, of the automated driving mode, the vehicle 100 travels in the remote automated driving mode in which automated remote control is performed using the server 200 as a device located outside the vehicle 100. In the remote manual driving mode, the vehicle 100 travels by manual remote control using a device located outside the vehicle 100. In the present embodiment, in the remote manual driving mode, the vehicle 100 travels by manual remote control using the server 200 and the remove maneuvering device 400.
The vehicle 100 includes a vehicle control device 110 for controlling each unit of the vehicle 100, an actuator group 120 including one or more actuators driven under the control of the vehicle control device 110, and a communication device 130. The communication device 130 communicates with an external device such as the server 200 by wireless communication. The actuator group 120 includes an actuator of a driving device for accelerating the vehicle 100, an actuator of a steering device for changing a traveling direction of the vehicle 100, and an actuator of a braking device for decelerating the vehicle 100.
The vehicle control device 110 includes a computer including a processor 111, a memory 112, an input/output interface 113, and an internal bus 114. The processor 111, the memory 112, and the input/output interface 113 are bidirectionally communicably connected via an internal bus 114. An actuator group 120 and a communication device 130 are connected to the input/output interface 113. The processor 111 executes the program PG1 stored in the memory 112 to realize various functions including the functions of the acquisition unit 115, the determination unit 116, and the vehicle control unit 117.
The acquisition unit 115 acquires the result of inspection of the mounted device DE. In the present embodiment, in the inspection process in the manufacturing process of the vehicle 100, the plurality of mounted device DE are inspected while the vehicle 100 is traveling in the remote automated driving mode. Therefore, the acquisition unit 115 acquires the result of inspection of each mounted device DE.
The result of inspection of the mounted device DE is, for example, information in which the device identifier for identifying the mounted device DE is associated with the pass/fail information indicating the pass/fail of the inspection. The result of inspection of the mounted device DE may further include a state quantity indicating a state of the mounted device DE. The acceptance/rejection of the mounted device DE is determined by comparing a reference set in advance with a state quantity in order to determine the acceptance/rejection according to, for example, whether or not the state of the mounted device DE is good. In this case, if the quantity of condition satisfies the criterion, the mounted device DE is determined to be acceptable. If the quantity of condition does not meet the criterion, the mounted device DE is determined to be rejected. The acceptance or rejection of the mounted device DE may be determined by other methods. The acceptance or rejection of the mounted device DE may be determined by, for example, calculating and clustering the similarities of the state quantities of the mounted device DE of the same type at the same acquiring timing acquired for each of the plurality of vehicles 100. Further, the acceptance/rejection of the mounted device DE may be determined, for example, by relatively comparing the time-series data representing the time-series change of the state quantities of the same type of mounted device DE acquired for each of the plurality of vehicles 100.
The determination unit 116 determines whether or not the vehicle can travel in the remote automated driving mode, using the result of inspection of the mounted device DE. In the present embodiment, when it is determined that all of the plurality of mounted device DE inspected as the inspection target pass, the determination unit 116 determines that the vehicle can travel in the remote automated driving mode. On the other hand, when it is determined that any one of the plurality of mounted device DE inspected as the inspection target fails, the determination unit 116 determines that the vehicle cannot travel in the remote automated driving mode. When it is determined that the driving in the remote automated driving mode is impossible, the determination unit 116 executes a class determination process for determining whether or not the driving in the remote manual driving mode is possible.
In the class determination process, the determination unit 116 classifies one of the plurality of classes, for example, according to the degree of the effect on the traveling by the remote manual driving mode, using the result of inspection of the mounted device DE. In this way, the determination unit 116 determines the class according to the result of inspection of the mounted device DE. In other words, in the class determination process, the determination unit 116 classifies one of the plurality of classes according to the degree of the failure of the mounted device DE, that is, the operating state of the mounted device DE. For example, the determination unit 116 classifies the vehicle into one of a plurality of classes according to DE of the mounted device that contributes to determination that the vehicle cannot travel in the remote automated driving mode. That is, the determination unit 116 classifies the mounted device DE into one of a plurality of classes according to the type and combination of the mounted devices determined to be rejected in the inspection. The determination unit 116 may further classify the state quantity of the mounted device DE determined to be rejected in the inspection into any of a plurality of classes.
In the present embodiment, in at least one of the first case and the second case, the determination unit 116 classifies the plurality of classes into a possible class indicating that the vehicle can travel in the remote manual driving mode.
In the first case, the function of the mounted device DE that contributes to determination that the vehicle cannot be driven by the remote automated driving mode, can be compensated by the feeling and operation of the external operator. The sense of the external operator may be, for example, five senses, such as the visual and auditory sense of the external operator, or may be a sense based on experience with a traveling operation. For example, when the mounted device DE that contributes to determination that the vehicle cannot be driven in the remote automated driving mode is an in-vehicle camera that is a kind of an in-vehicle sensor, the vehicle 100 can be driven as follows. In this case, the external operator visually recognizes the captured image displayed on the display of the remove maneuvering device 400 and acquired by imaging the vehicle 100 equipped with the in-vehicle camera determined to be rejected in the inspection by the external camera. As a result, the external operator recognizes the surrounding situation of the vehicle 100 and operates the remove maneuvering device 400, so that the vehicle 100 can be caused to travel by remote control by the external operator. Further, for example, when the mounted device DE that contributes to determination that the vehicle cannot travel in the remote automated driving mode is the steering angle sensor, the vehicle 100 can travel in the following manner. In this case, the external operator visually recognizes the captured image displayed on the display of the remove maneuvering device 400 and acquired by imaging the vehicle 100 equipped with the steering angle sensor determined to be rejected in the inspection by the external camera. As a result, the external operator recognizes the steering angle of the vehicle 100 and the surrounding situation of the vehicle 100, and operates the steering of the remove maneuvering device 400, whereby the vehicle 100 can be caused to travel by remote control by the external operator. As described above, when the mounted device DE that contributes to determination that the vehicle cannot travel in the remote automated driving mode is the mounted sensor, the vehicle can travel in the remote manual driving mode by supplementing it by an external operator. Further, for example, the mounted device DE that contributes to determination that the vehicle cannot travel in the remote automated driving mode, may be an electric parking brake. In this case, when no problem occurs in the electric shift and the disc brake device, the vehicle 100 can be stopped in the following manner. In this case, the external operator depresses the brake pedal of the remove maneuvering device 400 or operates the lever of the remove maneuvering device 400 to set the position of the electric shift mounted on the vehicle 100 as a parking position. Thus, the vehicle 100 can be stopped by remote control of the external operator. Therefore, in the first case, for example, the mounted device DE that contributes to determination that the vehicle cannot travel in the remote automated driving mode is at least one of the mounted sensor and the electric parking brake.
In the second case, the function of one mounted device DE that contributes to determination that the vehicle cannot travel in the remote automated driving mode, can be compensated by another mounted device DE different from one mounted device DE. For example, the mounted device DE that contributes to determination that the vehicle cannot be driven by the remote automated driving mode, may be an electric parking brake. In this case, when no problem occurs in the electric shift or the disc brake device, the vehicle 100 can be stopped as follows. In this case, the external operator depresses the brake pedal of the remove maneuvering device 400 or operates the lever of the remove maneuvering device 400 to set the position of the electric shift mounted on the vehicle 100 as a parking position. Thus, the vehicle 100 can be stopped by remote control of the external operator. Therefore, in the second case, for example, the mounted device DE that contributes to determination that the vehicle cannot be driven by the remote manual driving mode, is an electric parking brake.
When neither of the first case nor the second case corresponds, the determination unit 116 classifies the plurality of classes into an impossible class indicating that the vehicle cannot travel in the remote manual driving mode. The cases that do not correspond to either of the first case and the second case are cases in which, for example, the mounted device DE determined to be rejected in the examination is a motion control system device.
Note that each of the possible class and the impossible class may be constituted by a plurality of classes corresponding to the type, combination, state quantity, and the like of the mounted device DE determined to be rejected in the examination, or may be constituted by a single class. The plurality of classes corresponding to the operating state of the mounted device DE are also referred to as “fail classes”.
The determination unit 116 transmits the determination result to the server 200. Specifically, when it is determined that the vehicle can travel in the remote automated driving mode, the determination unit 116 transmits confirmation information indicating that the vehicle can travel in the remote automated driving mode. When it is determined that the vehicle cannot travel in the remote automated driving mode, the determination unit 116 transmits, to the server 200, class information indicating which of the possible class and the impossible class is classified. When at least one of the possible class and the impossible class is composed of a plurality of classes, the determination unit 116 may transmit a class identifier for identifying the plurality of classes to the server 200. For example, a possible class is composed of a first class, a second class, and a third class. And, the impossible class is composed of the fourth class, the fifth class, the sixth class, and the seventh class. In this case, the determination unit 116 transmits a class identifier indicating which class is classified to the server 200.
The vehicle control unit 117 controls the actuator group 120 to cause the vehicle 100 to travel. In the manned driving mode, the vehicle control unit 117 generates a travel control signal in response to an operation of the vehicle maneuvering device 140 mounted on the vehicle 100 by the occupant. The vehicle maneuvering device 140 includes, for example, a steering, an accelerator pedal, a brake pedal, and an electric shift for operating the vehicle 100. Accordingly, the vehicle control unit 117 can cause the vehicle 100 to travel by controlling the actuator group 120 using the generated travel control signal. The travel control signal is a control signal that defines the movement of the vehicle 100 in order to control the movement of the vehicle 100 and cause the vehicle 100 to travel. In the present embodiment, the travel control signal includes the acceleration and the steering angle of the vehicle 100 as parameters. In other embodiments, the travel control signal may include the speed of the vehicle 100 as a parameter in place of or in addition to the acceleration of the vehicle 100. In the remote manual driving mode, the vehicle control unit 117 receives a travel control signal generated according to an operation of the remove maneuvering device 400 provided at a location different from the vehicle 100 by the external operator. That is, the vehicle control unit 117 receives the travel control signal manually generated by the external operator via the remove maneuvering device 400. Thus, the vehicle control unit 117 controls the actuator group 120 by using the received travel control signal, so that the vehicle 100 can travel by remote control by an external operator. In the remote automated driving mode, the vehicle control unit 117 receives the travel control signal automatically generated by the server 200 not according to an operation by an external operator. Thus, the vehicle control unit 117 can control the actuator group 120 by using the travel control signal received from the server 200, thereby allowing the vehicle 100 to travel by remote control by the server 200.
Hereinafter, the travel control signal generated according to an operation of the vehicle maneuvering device 140 by the occupant is also referred to as a “manned control signal”. The travel control signal generated according to an operation of the remove maneuvering device 400 by the external operator is also referred to as a “manual control signal”. The travel control signal generated without using the operation amount of the vehicle maneuvering device 140 and the operation amount of the remove maneuvering device 400 is also referred to as an “automated control signal”.
The server 200 includes a computer including a processor 201, a memory 202, an input/output interface 203, and an internal bus 204. The processor 201, the memory 202, and the input/output interface 203 are bidirectionally communicably connected via an internal bus 204. A communication device 205 for communicating with various devices external to the server 200 is connected to the input/output interface 203. The communication device 205 can communicate with the vehicle 100 by wireless communication, and can communicate with each external sensor 300 by wired communication or wireless communication. The processor 201 executes the program PG2 stored in the memory 202 to realize various functions including the functions of the setting unit 211 and the remote control unit 212.
The setting unit 211 sets the driving mode of the vehicle 100 using the determination result of the determination unit 116 received from the vehicle 100. When it is determined that the vehicle can travel in the remote automated driving mode, the setting unit 211 sets the driving mode of the vehicle 100 to the remote automated driving mode. In a case where it is determined that the vehicle cannot travel in the remote automated driving mode, and in a case where it is determined that the vehicle can travel in the remote manual driving mode, the setting unit 211 sets the driving mode of the vehicle 100 to the remote manual driving mode. When it is determined that the vehicle cannot travel in the remote automated driving mode and the remote manual driving mode, the setting unit 211 sets the driving mode of the vehicle 100 to one of the traction driving mode and the manned driving mode.
In the present embodiment, it is determined which driving mode can be used in accordance with the result of inspection of the mounted device DE performed during driving in the remote automated driving mode. Then, the server 200 receives, as a determination result, one of the determination information and the class information from the vehicle 100. Therefore, when the determination information is received from the vehicle 100, the setting unit 211 maintains the driving mode of the vehicle 100 without changing from the remote automated driving mode. In a case where the class information is received from the vehicle 100 and the class specified by the class information is a possible class, the setting unit 211 switches the driving mode of the vehicle 100 from the remote automated driving mode to the remote manual driving mode. When the class information is received from the vehicle 100, the class specified by the class information may be an impossible class. In this case, the setting unit 211 switches the driving mode of the vehicle 100 from the remote automated driving mode to one of the traction driving mode and the manned driving mode.
When the class specified by the class information received from the vehicle 100 is an impossible class, the setting unit 211 executes, for example, the following processing. In this situation, the setting unit 211 sets the driving mode of the vehicle 100 to one of the traction driving mode and the manned driving mode according to the type of the mounted device DE determined to be rejected in the inspection. In the present embodiment, the setting unit 211 determines whether or not the occupant can drive the vehicle 100 by operating the vehicle maneuvering device 140 according to the type of the mounted device DE determined to be rejected in the inspection. When it is determined that the occupant can drive the vehicle 100 by operating the vehicle maneuvering device 140, the setting unit 211 sets the driving mode of the vehicle 100 to the manned driving mode. On the other hand, when it is determined that the vehicle 100 cannot be caused to travel even if the occupant operates the vehicle maneuvering device 140, the setting unit 211 sets the driving mode of the vehicle 100 to the traction driving mode.
For example, the mounted device DE determined to be rejected in the examination may be a driving device related to driving of the vehicle 100 such as an engine, a traveling motor, or a hybrid system among the motion control system devices. In this case, even if the occupant operates the vehicle maneuvering device 140, the vehicle may not be able to travel. In addition, in some cases, the mounted device DE determined to be rejected in the inspection is electrically shifted among the operation control devices. In this case, the traveling direction of the vehicle 100 may not be switched between the forward direction and the backward direction, or the driving mode of the vehicle 100 may not be switched. Accordingly, even if the occupant operates the vehicle maneuvering device 140, the vehicle may not be able to travel on a desired route. Further, even when the mounted device DE determined to be rejected in the examination is a power supply device, the drive device or the like cannot be driven, and even if the occupant operates the vehicle maneuvering device 140, the vehicle may not be able to travel. Therefore, in at least one of a case where the mounted device DE determined to be rejected in the examination is a driving device, a case where the mounted device is electrically shifted, and a case where the mounted device is a power supply device, the setting unit 211 sets, for example, the driving mode of the vehicle 100 to the traction driving mode.
On the other hand, for example, when the mounted device DE determined to be rejected in the examination is an electric power steering device among the driving control system devices, the steering angle of the vehicle 100 can be changed by the occupant operating the steering of the vehicle maneuvering device 140. Further, when the mounted device DE determined to be rejected in the examination is a brake pump in the operation control system device, the vehicle 100 can be decelerated or stopped by the occupant depressing the brake pedal of the vehicle maneuvering device 140. In addition, when the mounted device DE determined to be rejected in the examination is the communication device 130, the vehicle 100 may be allowed to travel by the occupant operating the vehicle maneuvering device 140. That is, when the mounted device DE determined to be rejected in the examination is at least one of the electric power steering device, the brake pump, and the communication device 130, the occupant may travel by operating the vehicle maneuvering device 140. Therefore, when the mounted device DE determined to be rejected in the inspection is at least one of the electric power steering device, the brake pump, and the communication device 130, the setting unit 211 sets, for example, the driving mode of the vehicle 100 to the manned driving mode.
When the driving mode of the vehicle 100 is set to the remote manual driving mode, the setting unit 211 may further execute various processes for enabling the vehicle to travel in the remote manual driving mode. In this case, for example, the setting unit 211 may notify the external operator of information indicating that the driving mode of the vehicle 100 is set to the remote manual driving mode via a notification unit (not shown), and prompt the external operator to operate the remove maneuvering device 400. In addition, the setting unit 211 may be activated by turning on the power of the remove maneuvering device 400 in order to enable the vehicle to travel in the remote manual driving mode. The setting unit 211 may start communication between the remove maneuvering device 400 and the server 200 or start communication between the server 200 and the vehicle 100 in order to enable traveling in the remote manual driving mode. In this way, it is possible to smoothly start traveling in the remote manual driving mode.
When the driving mode of the vehicle 100 is set to at least one of the manned driving mode, the remote manual driving mode, and the remote automated driving mode, the remote control unit 212 generates a travel control signal for controlling the actuator group 120 of the vehicle 100. Then, the remote control unit 212 transmits a travel control signal to the vehicle 100. Thus, the remote control unit 212 causes the vehicle 100 to travel by remote control.
In the remote manual driving mode, the remote control unit 212 generates a manual control signal in accordance with an operation of an external operator located outside the vehicle 100. Specifically, in the remote manual driving mode, an external operator operates the remove maneuvering device 400. Then, the remote control unit 212 of the server 200 acquires the operation amount applied to the remove maneuvering device 400 to generate a manual control signal corresponding to the operation applied to the remove maneuvering device 400. The remove maneuvering device 400 includes, for example, a display for displaying a captured image, steering for remotely controlling the vehicle 100, an accelerator pedal, a brake pedal, and a communication device for communicating with the server 200 through wired communication or wireless communication. The captured image is an image output from at least one of the external camera and the in-vehicle camera.
In the remote automated driving mode, the remote control unit 212 acquires the detection result by the sensor, and generates a travel control signal for controlling the actuator group 120 of the vehicle 100 using the detection result. The remote control unit 212 transmits the generated travel control signal to the vehicle 100 to cause the vehicle 100 to travel by remote control.
In some cases, the driving mode of the vehicle 100 is set to at least one of the manned driving mode and the towing driving mode. In this case, the remote control unit 212 may further notify the operator that the operator needs assistance when the vehicle 100 is moved via a notification unit (not shown). In this way, the operator can be called to the vehicle 100 that needs to be directly moved by the operator.
In S101, the processor 201 of the server 200 acquires the vehicle position information of the vehicle 100 using the detection result outputted from the external sensor 300. The vehicle position information is position information that is a basis for generating a travel control signal. In the present embodiment, the vehicle position information includes the position and orientation of the vehicle 100 in the global coordinate system of the factory. Specifically, in S101, the processor 201 acquires vehicle-position data using captured images acquired from cameras that are the external sensors 300.
Specifically, in S101, the processor 201 detects the external shape of the vehicle 100 from the captured images, for example. Thereafter, the processor 201 calculates the coordinates of the positioning point of the vehicle 100 in the coordinate system of the captured image, that is, the local coordinate system. Thereafter, the processor 201 obtains the position of the vehicle 100 by converting the calculated coordinates into coordinates in the global coordinate system. The outline of the vehicle 100 included in the captured image can be detected by, for example, inputting the captured image into a detection-model DM using artificial intelligence. The detection-model DM is prepared in the control system 50 or outside the control system 50, for example, and stored in advance in the memory 202 of the server 200. The detected model DM may be, for example, a learned machine learning model learned to implement either semantic segmentation or instance segmentation. As the machine learning model, for example, a convolutional neural network (hereinafter, CNN) learned by supervised learning using a learning dataset can be used. The training data set includes, for example, a plurality of training images including the vehicle 100 and a label indicating which of the regions in the training image indicates the vehicle 100 and the regions other than the vehicle 100. When CNN is learned, the parameters of CNN are preferably updated by back propagation so as to reduce the error between the output-result and -label due to the detected-model DM. Further, the processor 201 estimates, for example, based on the orientation of the movement vector of the vehicle 100 calculated from the positional change of the feature point of the vehicle 100 between the frames of the captured image by using the optical flow method. Thus, the processor 201 can acquire the orientation of the vehicle 100.
In S102, the processor 201 of the server 200 determines the target location to which the vehicles 100 should be heading next. In the present embodiment, the target position is represented by the coordinates of X, Y, Z in the global coordinate system. In the memory 202 of the server 200, reference route RR that is a route on which the vehicles 100 should travel is stored in advance. The route is represented by a node indicating a starting point, a node indicating a passing point, a node indicating a destination, and a link connecting the respective nodes. The processor 201 uses the vehicle position information and the reference route RR to determine the target position to which the vehicle 100 is to be directed next. The processor 201 determines the target position on the reference route RR ahead of the current position of the vehicles 100.
In S103, the processor 201 of the server 200 generates a travel control signal for causing the vehicle 100 to travel toward the determined target position. The processor 201 calculates the traveling speed of the vehicle 100 from the transition of the position of the vehicle 100, and compares the calculated traveling speed with the target speed. The processor 201 generally determines the acceleration so that the vehicle 100 accelerates when the travel speed is lower than the target speed, and determines the acceleration so that the vehicle 100 decelerates when the travel speed is higher than the target speed. When the vehicle 100 is located on the reference route RR, the processor 201 determines the steering angle and the acceleration so that the vehicle 100 does not deviate from the reference route RR. On the other hand, when the vehicle 100 is not located on the reference route RR, in other words, when the vehicle 100 deviates from the reference route RR, the processor 201 determines the steering angle and the acceleration so that the vehicle 100 returns to the reference route RR.
In S104, the processor 201 of the server 200 transmits the generated travel control signal to the vehicles 100. The processor 201 repeats acquisition of vehicle position information, determination of a target position, generation of a travel control signal, transmission of a travel control signal, and the like at predetermined intervals.
In S105, the processor 111 of the vehicle 100 receives the travel control signal transmitted from the server 200. In S106, the processor 111 of the vehicle 100 controls the actuator group 120 using the received travel control signal, thereby causing the vehicle 100 to travel at the acceleration and the steering angle represented by the travel control signal. The processor 111 repeatedly receives the travel control signal and controls the actuator group 120 at a predetermined cycle. According to the control system 50 of the present embodiment, the vehicle 100 can be driven by remote control, and the vehicle 100 can be moved without using a conveyance facility such as a crane or a conveyor.
As illustrated in
When all of the plurality of mounted device DE pass (S202: Yes), the determination unit 116 determines that the vehicle can travel in the remote automated driving mode in S203. When it is determined that the vehicle can travel in the remote automated driving mode, the determination unit 116 transmits the determination information to the server 200 in S204. When the determination data is received from the vehicle 100 (S205: Yes), the setting unit 211 of the server 200 sets the driving mode of the vehicle 100 to the remote automated driving mode in S206. In S207, the remote control unit 212 acquires the position information of the vehicle by using the detection result outputted from the camera as the external sensor 300. In S208, the remote control unit 212 determines a target position to which the vehicles 100 are to be directed next. At S209, the remote control unit 212 generates an automated control signal for causing the vehicle 100 to travel toward the determined target position. At S210, the remote control unit 212 transmits the generated automated control signal to the vehicles 100. In S211, the vehicle control unit 117 of the vehicle control device 110 mounted on the vehicle 100 controls the actuator group 120 using the automated control signal received from the server 200. Thus, the vehicle control unit 117 causes the vehicle 100 to travel at the acceleration and the steering angle represented by the automated control signal.
As illustrated in
In the classifying process, in at least one of the first case and the second case (S213: Yes), the determination unit 116 determines that the vehicle can travel in the remote manual driving mode in S214. As a result, the determination unit 116 classifies S215 into the feasible classes. In S216, the determination unit 116 transmits, to the server 200, class data indicating that the classes are classified as possible classes. In a case where the class information is received from the vehicles 100 (S217: Yes), and in a case where the class specified by the class information is a possible class (S218: Yes), the setting unit 211 executes S219. In S219, the setting unit 211 sets the driving mode of the vehicles 100 to the remote manual driving mode. In S220, the remove maneuvering device 400 receives an input by an operation of an external operator. In S221, the remove maneuvering device 400 transmits the manipulated variable of the external operator to the server 200. In S222, the remote control unit 212 of the server 200 generates a manual control signal using an operation amount of an external operator received from the remove maneuvering device 400. At S223, the remote control unit 212 transmits the generated manual control signal to the vehicles 100. In S224, the vehicle control unit 117 of the vehicle control device 110 mounted on the vehicle 100 controls the actuator group 120 using the manual control signal received from the server 200. Thus, the vehicle control unit 117 causes the vehicle 100 to travel at the acceleration and the steering angle represented by the manual control signal.
When neither of the first case nor the second case is satisfied in the classifying process (S213: No), as illustrated in
According to the first embodiment, the control system 50 may acquire the result of inspection of the mounted device DE. The control system 50 may determine whether the vehicle can travel in the remote automated driving mode by using the result of inspection of the mounted device DE. If it is determined that the driving in the remote automated driving mode is impossible, the control system 50 can determine whether or not the driving in the remote manual driving mode is possible. Then, the control system 50 can set the driving mode of the vehicle 100 to the remote manual driving mode when it is determined that the driving in the remote automated driving mode is impossible when it is determined that the driving in the remote manual driving mode is possible. Accordingly, the vehicle 100 can travel by executing the following processing when it is determined that the vehicle cannot travel in the remote automated driving mode and that the vehicle can travel in the remote manual driving mode. In this case, the vehicle 100 can travel by remote control by the external operator by controlling the actuator group 120 using a manual control signal generated in response to an operation by the external operator of the remove maneuvering device 400. With this configuration, even when the vehicle 100 cannot be caused to travel in the remote automated driving mode in which automated remote control is performed using a device located outside the vehicle 100 due to the occurrence of a defect in the mounted device DE, the following can be performed. In this case, the vehicle 100 can be moved by the remote manual driving mode in which the operator performs manual remote control by the external operator without directly moving the vehicle 100. That is, the operator can move the vehicle 100 without riding on the vehicle 100 to operate the vehicle maneuvering device 140 or the operator transporting the vehicle 100 using a traction device or the like. As a result, it is possible to reduce the number of man-hours required to move the vehicle 100 as compared with the case where the operator directly moves the vehicle 100.
In some cases, the external operator may be able to compensate for the function of the mounted device DE that contributes to determination that the vehicle cannot be driven by the remote automated driving mode. In this case, according to the first embodiment, the control system 50 can determine that the vehicle can travel in the remote manual driving mode.
Further, according to the first embodiment, a plurality of mounted devices DE are inspected. In this case, the function of one mounted device DE that contributes to determination that the vehicle cannot be driven by the remote automated driving mode may be compensated by the function of another mounted device DE. In this case, the control system 50 can determine that the vehicle can travel in the remote manual driving mode.
In addition, according to the first embodiment, the mounted device DE that contributes to determination that the vehicle cannot travel in the remote automated driving mode, may be at least one of the mounted sensor and the electric parking brake. In this case, the control system 50 can determine that the vehicle can travel in the remote manual driving mode.
Further, according to the first embodiment, the vehicle control device 110 can classify the state of the vehicle 100 into any of a plurality of classes according to the operating state of the mounted device DE and determine whether or not the vehicle can travel in the remote manual driving mode. In this way, it is possible to determine whether or not the vehicle 100 can travel in the remote manual driving mode without preparing a data base in which the propriety of the driving in the remote manual driving mode is associated with all types and all combinations of the various mounted devices DE that differ depending on the type of the vehicle 100. That is, it is only necessary to store a data base corresponding to the type of the mounted device DE of the host vehicle 100 in the respective vehicles 100. The type of the vehicle 100 is, for example, the type of the vehicle 100 when classified according to a trade name, a model, a specification, and the like. Accordingly, it is possible to reduce the load required for preparing the database used for determining whether or not the vehicle can travel in the remote manual driving mode.
In the present embodiment, the processor 111v of the vehicle control device 110v functions as the acquisition unit 115, the determination unit 116v, the setting unit 118, and the vehicle control unit 117v by executing the program PG1v stored in the memory 112v.
The determination unit 116v determines whether or not the vehicle can travel in the autonomous driving mode. When it is determined that the vehicle cannot travel in the autonomous driving mode, the determination unit 116v determines whether or not the vehicle can travel in the remote manual driving mode. When it is determined that the vehicle cannot travel in the remote manual driving mode, the determination unit 116v determines whether or not the vehicle can travel in the manned driving mode.
When the determination unit 116v determines that the vehicle can travel in the autonomous driving mode, the setting unit 211 sets the driving mode of the vehicle 100v to the autonomous driving mode. When the determination unit 116v determines that the vehicle cannot travel in the autonomous driving mode but can travel in the remote manual driving mode, the setting unit 211 sets the driving mode of the vehicle 100v to the remote manual driving mode. When the determination unit 116v determines that the vehicle cannot be driven in the autonomous driving mode and the remote manual driving mode, but the vehicle can be driven in the manned driving mode, the setting unit 118 sets the driving mode of the vehicle 100v to the manned driving mode. When the determination unit 116v determines that the vehicle cannot travel in the autonomous driving mode, the remote manual driving mode, and the manned driving mode, the setting unit 118 sets the driving mode of the vehicle 100v to the traction driving mode.
In the autonomous driving mode, the vehicle control unit 117v acquires an output result of the sensor and generates a travel control signal using the output result. Thus, the vehicle control unit 117v can cause the vehicle 100v to travel by autonomous control by outputting the generated travel control signal and operating the actuator group 120. In the present embodiment, in addition to the program PG1v, the detection-model DM and the reference route RR are stored in advance in the memory 112v. Note that the functions of the vehicle control unit 117v in the manned driving mode and the remote manual driving mode are the same as those of the vehicle control unit 117 of the server 200 described in the first embodiment.
In S301, the processor 111v of the vehicle control device 110v acquires the vehicle position information using the detection result outputted from the camera as the external sensor 300. In S302, the processor 111v determines the target position to which the vehicle 100v should be headed next. In S303, the processor 111v generates a travel control signal for causing the vehicle 100v to travel toward the determined target position. In S304, the processor 111v controls the actuator group 120 by using the generated travel control signal, thereby causing the vehicle 100v to travel in accordance with the parameter represented by the travel control signal. The processor 111v repeats acquiring the vehicle position information, determining the target position, generating the travel control signal, and controlling the actuator at a predetermined cycle.
According to the second embodiment, the control system 50v can cause the vehicle 100v to travel by autonomous control of the vehicle 100v without remotely controlling the vehicle 100v by the server 200 while the control of the autonomous driving mode is executed.
Further, according to the second embodiment, even when the vehicle 100v cannot be driven by the autonomous driving mode in which autonomous control of the vehicle 100v is performed due to the failure of the mounted device DE, the following can be performed. Thus, the vehicle 100v can be moved by manual remote control by an external operator without the operator directly moving the vehicle 100v. As a result, it is possible to reduce the number of man-hours required to move the vehicle 100v as compared with a case where the operator directly moves the vehicle 100v.
At least some of the functions of the server 200 may be one function of the vehicle control device 110, 110v or one function of the external sensor 300. Further, at least some of the functions of the vehicle control device 110, 110v may be one function of the server 200 or one function of the external sensor 300. That is, the control device including the acquisition unit 115, the determination unit 116, and the setting unit 118,211 may be the server 200 or the vehicle control device 110, 110v. Further, each function of the server as the control device may be realized by a plurality of servers. In a case where each function of the control device is realized by a plurality of servers, the control system 50 may further include a server 200 having a function other than the function described in the first embodiment. In this case, the control system 50 may include, for example, a self-propelled transport server, a self-propelled control server, a remote driving server, an inspection server, and a notification server. The self-propelled transport server has the same functions as those of the acquisition unit 115, the determination unit 116, and the setting unit 118,211, for example. That is, the self-propelled transport server has a function of determining which of the manned driving mode, the remote manual driving mode, and the automated driving mode the vehicle 100 is caused to travel in, and setting the driving mode according to the determination result. The self-propelled control server has a function of automatically generating an automated control signal using vehicle position information and the like among the functions of the remote control unit 212 and transmitting the automated control signal to the vehicle 100. That is, the self-propelled control server is used for traveling in the remote automated driving mode. The remote driving server has, among the functions of the remote control unit 212, a function of generating a manual control signal in response to an operation by an external operator of the remove maneuvering device 400 and transmitting the manual control signal to the vehicle 100. The inspection server has a function of executing an inspection of the mounted device DE. The notification server transmits, at a predetermined cycle, information on an object existing in the vicinity of the runway and the vehicle 100 to at least one of the self-propelled transport server and the automated control server. Objects present on the runway and around the vehicle 100 may be moving objects such as, for example, humans and AGV, and may be stationary objects such as manufacturing facilities and signboards. Even in such a configuration, when the vehicle 100 cannot be caused to travel in the automated driving mode due to a failure of the mounted device DE, the following can be performed. In this case, the vehicle 100 can be moved by manual remote control by an external operator without the operator directly moving the vehicle 100.
The mounted device DE may be inspected prior to driving the vehicles 100 in the automated driving mode. For example, in the assembly process of assembling the mounted device DE to the vehicle 100, the mounted device DE may be inspected after the mounted device DE is assembled to the vehicle 100. With this configuration, the control system 50, 50v can be set to the driving mode according to the operating state of the mounted device DE prior to starting the travel in the automated driving mode.
The determination unit 116, 116v may be one function of the servers 200. In this case, the determination unit 116, 116v may determine whether or not the vehicle is allowed to travel in the remote manual driving mode by using a data base in which the type and combination of the mounted device DE and whether or not the vehicle is allowed to travel in the remote manual driving mode are associated with each other. The database in which the types of the mounted device DE and the combinations and the propriety of the driving in the remote manual driving mode are associated with each other is a database stored in the memory 202 of the server 200. Even in such a configuration, the vehicle 100 can be moved by manual remote control by an external operator without the operator directly moving the vehicle 100.
The vehicle 100, 100v may have a remote manual driving mode, a manned driving mode, and a towing driving mode without having an automated driving mode. In this case, for example, the acquisition unit 115 acquires the result of inspection of the mounted device DE performed while the vehicle 100, 100v is traveling in the manned driving mode. The determination unit 116, 116v determines whether the vehicle can be moved in the manual driving mode without determining whether the vehicle can be moved in the automated driving mode by using the result of inspection of the mounted device DE. When it is determined that the vehicle can be moved in the remote manual driving mode, the setting unit 118,211 sets the driving mode of the vehicle 100, 100v to the remote manual driving mode. Even in such a configuration, when a failure occurs in the mounted device DE, the vehicle 100, 100v can be moved by the remote manual driving mode without the operator directly moving the vehicle 100, 100v.
In each of the above embodiments, the external sensor 300 is a camera. On the other hand, the external sensor 300 may not be a camera, and may be a distance measuring device such as a LiDAR (Light Detection And Ranging). In this case, the detection result output by the external sensor 300 may be three-dimensional point cloud data representing the vehicle 100. In this case, the server 200 or the vehicle 100 may acquire the vehicle position information by template matching using three-dimensional point cloud data as a detection result and reference point cloud data prepared in advance.
In the first embodiment, the server 200 executes processing from acquisition of vehicle position information to generation of a travel control signal. On the other hand, at least a part of the processing from the acquisition of the vehicle position information to the generation of the travel control signal may be executed by the vehicle 100. For example, the following forms (1) to (3) may be used.
In the second embodiment, an internal sensor may be mounted on the vehicle 100v, and a detection result outputted from the internal sensor may be used for at least one of generation of a route and generation of a travel control signal. For example, the vehicle 100v may acquire the detection result of the internal sensor and reflect the detection result of the internal sensor in the route when generating the route. The vehicle 100v may acquire the detection result of the internal sensor and reflect the detection result of the internal sensor in the travel control signal when generating the travel control signal.
In the second embodiment, the vehicle 100v acquires the vehicle position information using the detection result of the external sensor 300. On the other hand, an inner sensor is mounted on the vehicle 100v. The vehicle 100v acquires the vehicle position information by using the detection result of the inner sensor, and determines a target position to which the vehicle 100v is to be directed next. Then, the vehicle 100v generates a route from the current position of the vehicle 100v to the target position, which is represented by the acquired vehicle position data. The vehicle 100v may further generate a travel control signal for traveling on the generated route, and control the actuator group 120 using the generated travel control signal. In this case, the vehicle 100v can travel without using the detection result of the external sensor 300 at all. The vehicle 100v may acquire the target arrival time and the traffic jam information from the outside of the vehicle 100v and reflect the target arrival time and the traffic jam information on at least one of the route and the travel control signal. In addition, all the functional configurations of the control system 50v may be provided in the vehicle 100v. In other words, the process implemented by the control system 50v may be implemented by the vehicle 100v alone.
In the above-described embodiments, the vehicle 100, 100v may have a configuration that can be moved by unmanned driving, and may be, for example, in the form of a platform that includes the configuration described below. Specifically, the vehicle 100, 100v may include at least a vehicle control device 110, 110v and an actuator group 120 in order to perform the following three functions by unmanned driving: “run,” “turn,” and “stop.” When the vehicle 100, 100v acquires information from the outside for unmanned driving, the vehicle 100, 100v may further include a communication device 130. That is, the vehicle 100, 100v that can be moved by the unmanned driving may not be equipped with at least a part of the interior components such as the driver's seat and the dashboard. In addition, at least a part of an exterior component such as a bumper or a fender may not be attached to the vehicle 100, 100v. The vehicle 100, 100v may not be fitted with a body shell. In this instance, the remaining components, such as the body shell, may be mounted on the vehicle 100, 100v until the vehicle 100, 100v is shipped from the factory. In addition, the vehicle 100, 100v may be equipped with the remaining components such as the body shell after the vehicle 100, 100v is shipped from the factory while the remaining components such as the body shell are not mounted on the vehicle 100, 100V. Each component may be attached from any direction, such as the upper, lower, front, back, right or left side of the vehicle 100, 100v, may be attached from the same direction, each may be attached from different directions. It should be noted that the position determination can be performed in the same manner as in the vehicle 100, 100v according to the first embodiment.
The vehicle 100, 100v may be manufactured by combining a plurality of modules. The modular means a unit composed of a plurality of components arranged according to a part or a function of a vehicle 100, 100v. For example, a vehicle 100, 100v may be manufactured by combining a front module, a central module, and a rear module. The front module constitutes the front of the platform. The central module constitutes the central part of the platform. The rear module constitutes the rear of the platform. The number of modules constituting the platform is not limited to three, and may be two or less or four or more. In addition to or instead of the components constituting the platform, components constituting parts of the vehicle 100, 100v that differ from the platform may be modularized. Further, the various modules may include any exterior parts such as bumpers and grills, and any interior parts such as sheets and consoles. In addition, the present disclosure is not limited to a vehicle 100, 100v, and a moving object of any aspect may be manufactured by combining a plurality of modules. Such a module may be manufactured, for example, by joining a plurality of parts by welding, a fixture, or the like, or may be manufactured by integrally molding at least a part of the parts constituting the module as one part by casting. Molding techniques for integrally molding one part, in particular a relatively large part, are also called gigacasting or megacasting. For example, the front module, the central module, and the rear module described above may be manufactured using gigacasting.
Transporting the vehicle 100, 100v by using the traveling of the vehicle 100, 100v by the unmanned driving is also referred to as “self-propelled conveyance”. A configuration for realizing self-propelled conveyance is also referred to as a “vehicle remote control autonomous driving conveyance system”. Further, a production method of producing a vehicle 100, 100v by using self-propelled conveyance is also referred to as “self-propelled production”. In self-propelled manufacturing, for example, at least a part of conveyance of the vehicle 100 is realized by self-propelled conveyance in a factory that manufactures the vehicle 100, 100v.
In each of the above-described embodiments, some or all of the functions and processes implemented in software may be implemented in hardware. In addition, some or all of the functions and processes implemented in hardware may be implemented in software. For example, various circuits such as an integrated circuit and a discrete circuit may be used as hardware for realizing various functions in the above-described embodiments.
The present disclosure is not limited to each of the above embodiments, and can be realized by various configurations without departing from the spirit thereof. For example, the technical features of the embodiments corresponding to the technical features in the respective embodiments described in the Summary can be appropriately replaced or combined in order to solve some or all of the above-described problems or to achieve some or all of the above-described effects. Further, when the technical features are not described as essential in the present specification, these can be deleted as appropriate.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2023-194048 | Nov 2023 | JP | national |
