Patent Application
20230311332
This disclosure relates to a thin mobile communication terminal capable of operating as a robot and automatically being recharged, and a control method and control program thereof.
Various efforts have been made to provide a cell phone, which has become a familiar item for users, with a character to cause the users to feel more familiar. For example, PCT/JP2019/023711 (WO 2020/250434 A1) discloses a thin mobile communication terminal that can be used in a thin form, but can also serve as a robot. Also, Japanese Laid-Open Patent Publication No. 2006-236333 discloses a vacuum cleaner configured to be able to move independently and communicate sequentially with a charger so that if communication with the charger is lost, the cleaner can return to the position where communication was last established and return itself to the charger.
When the thin mobile communication terminal as described in PCT/JP2019/023711 (WO '434) above is driven as a robot, it naturally consumes power. Accordingly, it is desirable to have a configuration that automatically charges the battery in the thin mobile communication terminal. It is possible to allow a charger to have a communication function and communicate with a thin mobile communication terminal to guide it to the charger to perform charging as in JP2006-236333A above. However, a problem is in that it may be hard to employ the configuration of JP2006-236333A above if the automatic charging above is using an existing OS or application for a thin mobile communication terminal in a manner compliant with existing communication devices such as smart phones.
It could therefore be helpful to provide a thin mobile communication terminal that can be driven as a robot that can be automatically recharged, and a control method and control program thereof.
We thus provide a thin mobile communication terminal that includes: a rectangular enclosure; two arms wherein one end of each arm is rotationally supported by a rotary shaft against corresponding one of both side surfaces of long sides of the enclosure; a hand at another end of each arm; a leg provided on one side surface of short sides of the enclosure in an independently movable manner; a rotary motor that rotates the rotary shaft that supports corresponding one of the two arms on one end of the corresponding one of the arms; an imaging unit provided on a main surface of the enclosure; and a controller that controls the rotary motor. The controller determines whether a charger is included in an image captured by the imaging unit, and if it determines that the charger is not included in the captured image, the controller drives the rotary motor to change an angle of view of the imaging unit in a vertical direction to search for the charger.
A method of controlling a thin mobile communication terminal includes: a rectangular enclosure; two arms wherein one end of each arm is rotationally supported by a rotary shaft against corresponding one of both side surfaces of long sides of the enclosure; a hand at another end of each arm; a leg provided on one side surface of short sides of the enclosure in an independently movable manner; a rotary motor that rotates the rotary shaft that supports corresponding one of the two arms on one end of the corresponding one of the arms; an imaging unit provided on a main surface of the enclosure; and a controller that controls the rotary motor, where the method includes the steps of: performing imaging by the imaging unit; determining whether a charger is included in a captured image obtained in the imaging step; changing an angle of view of the imaging unit in a vertical direction by driving the rotary motor if it is determined that the charger is not included in the captured image; moving the terminal toward the charger if the charger is included in the captured image; and receiving charging by the charger.
A program causes a computer of a thin mobile communication terminal including: a rectangular enclosure; two arms wherein one end of each arm is rotationally supported by a rotary shaft against corresponding one of both side surfaces of long sides of the enclosure; a hand at another end of each arm; a leg provided on one side surface of short sides of the enclosure in an independently movable manner; a rotary motor that rotates the rotary shaft that supports corresponding one of the two arms on one end of the corresponding one of the arms; an imaging unit provided on a main surface of the enclosure; and a controller that controls the rotary motor, to embody the functions of: performing imaging by the imaging unit; determining whether a charger is included in a captured image obtained by the imaging function; changing an angle of view of the imaging unit in a vertical direction by driving the rotary motor if it is determined that the charger is not included in the captured image; moving the terminal toward the charger if the charger is included in the captured image; and receiving charging by the charger.
In addition, in the thin mobile communication terminal above, the leg may include: a first rotary shaft with an axis direction in a direction of the short sides of the enclosure; a second rotary shaft coaxial with the first rotary shaft; a right wheel at one end of the first rotary shaft; a left wheel at one end of the second rotary shaft; a first traveling motor provided at another end of the first rotary shaft and drives the right wheel; and a second traveling motor provided at another end of the second rotary shaft and drives the left wheel. If the controller further determines that the charger is not included in the image captured by the imaging unit, the controller may drive the first traveling motor and second traveling motor to change an angle of view of the imaging unit in a horizontal direction to search for the charger.
Further, if the controller determines that the charger is included in the captured image, the controller may drive the first traveling motor and second traveling motor to move the terminal toward the charger on the basis of a position of the charger in the captured image.
Moreover, the imaging unit may perform sequential imaging while the terminal moving toward the charger, and the controller may rotate the rotary motor so that the charger is included in a captured image each time the imaging unit images.
Furthermore, the controller may stop the first and second traveling motors if a rotation angle of the rotary motor reaches a predetermined angle.
In addition, the controller may stop the first traveling motor and second traveling motor if a predetermined object on the charger is present in the image captured by the imaging unit.
Further, the thin mobile communication terminal may further include: a storage battery; and a charging unit that charges the storage battery with electric power. The charging unit may receive a wireless power supply from the charger and charge the storage battery.
The thin mobile communication terminal may include a hand and a leg, and the leg is configured to be able to move independently, thus providing a mobile communication terminal that is capable of operating as a robot without compromising portability.
Features, advantages, and technical and industrial significance of examples will be described below with reference to the accompanying drawings, in which like numerals denote like elements.
A mobile communication terminal 100 is directed to a thin portable communication device represented by a smartphone or tablet terminal as well as a device also capable of serving as a robot, which can also be described as a mobile portable terminal. This example describes a method that allows automatic recharging to be performed using existing software in mobile communication terminals in which new configuration is added as little as possible. An example of a mobile communication terminal will be described below with reference to the drawings.
First, with reference to
As shown in
As shown in
In the same manner, the enclosure 10 of the mobile communication terminal 100 includes a rotary motor 16 inside it, which freely rotates the rotary shaft 12a in clockwise and counterclockwise directions. In the same manner as in the rotary motor 15, the rotary motor 16 is also driven by electric power supplied from the power supply (not shown) of the mobile communication terminal 100 and controlled by the controller 190 of the mobile communication terminal 100 (described in detail below). The rotary shaft 12a is connected (fixed) to one end of an arm plate 12b. The arm plate 12b corresponds to a flat plate member parallel to the long side of the enclosure 10. When the rotary shaft 12a is rotated by the rotary motor 16, the arm plate 12b is also rotated about the rotary shaft 12a. A rotary shaft 12c is connected (fixed) to the other end of the arm plate 12b, and a hand 12d is connected to the rotary shaft 12c in a rotational manner. The hand 12d freely rotates about the rotary shaft 12c. The arm 12 serves as a right arm in the mobile communication terminal 100.
The mobile communication terminal 100 includes a protrusion 13a configured to protrude from the enclosure 10. As shown in
A right wheel 13b is connected (fixed) to the rotary shaft 13e, and the right wheel 13b rotates in accordance with the rotation of the rotary shaft 13e. In the same manner, a left wheel 13c is connected (fixed) to the rotary shaft 13g, and the left wheel 13c rotates in accordance with the rotation of the rotary shaft 13g. The leg 13 is configured by the protrusion 13a, the first traveling motor 13d and the second traveling motor 13f, the corresponding rotary shaft 13e and rotary shaft 13g, and the right wheel 13b and the left wheel 13c that are rotated by the corresponding rotary shafts 13e and 13g.
As shown in
Further, as shown in
Further, the hand 12d is configured to have a size, i.e., the diameter L3 less than or equal to the thickness L1 of the enclosure 10. In the same manner, the hand 11d is configured to have a diameter less than or equal to the thickness L1 of the enclosure 10, and the diameter of the hand 11d is the same as that of the hand 12d. The hands 11d and 12d are configured to have a size less than or equal to the thickness of the enclosure 10, configuring the entire mobile communication terminal 100 thinner and improving the portability of the mobile communication terminal 100.
Such a mobile communication terminal 100 may be driven in the manner shown in
As mentioned above, the mobile communication terminal 100 corresponds to a general mobile communication terminal such as a smartphone or tablet terminal, and as shown in
The communication unit 110 is directed to a communication interface that serves to receive information from other devices via a network. The communication unit 110 communicates the received information to the controller 190. The communication unit 110 also transmits information instructed by the controller 190 to external devices. The communication unit 110 is capable of exchanging electronic information such as e-mail where it serves as a mobile communication terminal, website information where the user browses the Internet, and audio signals where the user makes a phone call.
The storage 120 serves to store various programs and data required by the mobile communication terminal 100 to operate. The storage 120 may be, for example, various recording media such as hard disk drive (HDD), solid state drive (SSD), and flash storage. The storage 120 stores an operation program 121 for the mobile communication terminal 100 to operate as a robot. The operation program 121 includes a process that searches for a charger that is capable of wirelessly charging the mobile communication terminal 100 and moves to the location of the charger.
The input unit 130 serves to receive input from the user. The input unit 320 can be a touch panel, for example, but is not limited to the touch panel. The input unit 130 communicates the input contents received from the user to the controller 190. The input unit 130 may accept voice input, where the input unit 130 can be a microphone. Further, the input unit 130 may be other hardware keys.
The audio output unit 150 outputs the audio signals in accordance with the instructions from the controller 190. The audio output unit 150 may be, for example, a speaker. The audio output unit 150 may, for example, output audio as lines of a robot. The audio output unit 150 may also output audio from a phone call.
The imaging unit 160 is directed to a camera that captures images of the surroundings in accordance with the instructions from the input unit 130 and the controller 190. An example in which the imaging unit 160 is provided on the front side of the mobile communication terminal 100 is illustrated, but this may be provided on the rear side or on both sides.
The power supply 170 is directed to a battery that serves to supply power to drive the various portions of the mobile communication terminal 100. The power supply 170 is directed to a chargeable/dischargeable battery, typically, a lithium-ion battery, but it is not limited thereto.
The charging unit 180 serves to receive transmitted power supply from an external charger (wireless charger) and charge the power supply 170 with the received power supply. The external charger performs the charging by wireless charging.
The controller 190 controls various portions of the mobile communication terminal 100 and may be, for example, a central processing unit (CPU), microprocessor, ASIC, and FPGA. The controller 190 is not limited to these examples, and may be any portion if it controls the mobile communication terminal 100. The controller 190 reads and executes various programs including the operation program 121 stored in the storage 120 to embody the functions to be performed by the mobile communication terminal 100. The controller 190 executes the process of moving toward the charger and executing charging under predetermined conditions. The controller 190 may also execute the process in accordance with the instructions by the user entered through the input unit 130. The controller 190 may also execute the process of information communicated by the communication unit 110.
The controller 190 includes a drive controller 191 and a determination unit 192 as functions to be implemented by the mobile communication terminal 100.
The drive controller 191 controls the arms 11, 12, and leg 13 when the mobile communication terminal 100 serves as a robot. The drive controller 191 controls the rotary motor 15 that operates the arm 11, the rotary motor 16 that operates the arm 12, the first traveling motor 13d that operates the right wheel 13b, and the second traveling motor 13f that operates the left wheel 13c in accordance with the movement information 123 that defines a predetermined movement pattern to allow the mobile communication terminal 100 to operate as if it is a robot. For example, the drive controller 191 is capable of advancing the mobile communication terminal 100 by rotating the first traveling motor 13d and the second traveling motor 13f both counterclockwise when viewed from the right side, and is capable of retreating the mobile communication terminal 100 by rotating them both clockwise when viewed from the right side. The drive controller 191 is also capable of rotating the mobile communication terminal 100 in place by rotating the first traveling motor 13d and the second traveling motor 13f in opposite directions. The drive controller 191 is capable of changing the angle between the main surface of the mobile communication terminal 100 including the imaging unit 160 and the ground surface by controlling the rotary motor 15 that operates the arm 11 and the rotary motor 16 that operates the arm 12, thereby changing the vertical angle of view (imaging direction) of the imaging unit 160. The angle of view in the horizontal direction (imaging direction) may be changed by rotating the right wheel 13b and left wheel 13c.
The determination unit 192 determines whether a charger capable of charging the power supply 170 of the mobile communication terminal 100 is present in the captured image taken by the imaging unit 160. The determination unit 192 determines whether a charger is included in the captured image by determining whether a marker provided on the charger is included in the captured image, but the method of determining whether a charger is included in the captured image is not limited to this. Other methods are described below as variations.
The functional configuration of the mobile communication terminal 100 is described above.
The mobile communication terminal 100 may move toward the charger to perform charging under various conditions. Various conditions may include, for example, where a predetermined amount of time elapses without any input from the user of the mobile communication terminal 100, or where the amount of power in the power supply 170 falls below a predetermined threshold, or even any random situation.
If the controller 190 of the mobile communication terminal 100 determines to perform charging, the drive controller 191 causes the mobile communication terminal 100 to stand in an upright state (step S601).
The controller 190 causes the imaging unit 160 to perform imaging (step S602). The imaging unit 160 communicates the captured image obtained through the imaging to the controller 190.
The determination unit 192 determines whether a charger is present in the transmitted captured image (step S603). If the determination unit 192 determines that the charger is not present in the captured image (NO in step S603), the controller 190 determines whether the imaging has been performed at all angles of the arms 11 and 12 (step S607). The imaging at all angles of the arms 11 and 12 may be a determination of whether imaging has been performed at all of a predetermined number of angles so that imaging by the mobile communication terminal 100 in the vertical direction can be performed in a somewhat exhaustive manner.
If imaging has been performed at all angles of the arms (YES in step S607), the drive controller 191 rotates the main body of the mobile communication terminal 100 by a predetermined angle by rotating the right wheel 13b and the left wheel 13c in opposite directions, respectively, by a predetermined angle (step S608), and returns to the process in step S602. If imaging at all angles of the arms is not performed (NO in step S607), the drive controller 191 rotates the rotary motors 15 and 16 of the arms 11 and 12 by a predetermined angle, respectively (step S609) to change the vertical imaging direction by the imaging unit 160 of the mobile communication terminal 100. The process then returns to step S602.
In contrast, if the determination unit 192 determines that a charger is present in the captured image (YES in step S603), the controller 190 determines whether the charger in the captured image satisfies a predetermined condition (step S604). The predetermined condition is directed to a condition for determining whether the mobile communication terminal 100 is within a range of receiving wireless power supply from the charger. As an example, the predetermined condition may be such that the marker is captured at a predetermined size or greater in the captured image, and that the arms 11 and 12 are at a predetermined angle (the mobile communication terminal 100 is in a state close to a prone state). The state in which the arms 11 and 12 are at a predetermined angle close to the prone state and the marker can be imaged at a predetermined size or greater may be recognized to correspond to a state in which the mobile communication terminal 100 is almost directly above the marker on the charger and is close enough from the charger.
If it is determined that the charger in the captured image does not satisfy the predetermined condition (NO in step S604), the drive controller 191 drives the first traveling motor 13d and the second traveling motor 13f to move the terminal 100 by a predetermined distance toward the charger (step S606) and the process returns to step S602. If it is determined that the charger in the captured image satisfies the predetermined condition (YES in step S604), the drive controller 191 controls the arms 11 and 12 so that the mobile communication terminal 100 is in the prone state, and the charging unit 180 receives wireless power supply from the charger to charge the power supply 170 (step S605). The process is then terminated when charging is complete. The automatic charging process by the mobile communication terminal 100 is described above. In
With reference to
In contrast,
The dotted line in
In the example shown in
In contrast,
That is, with respect to the examples of
As described above, the mobile communication terminal 100 can change the angles of the terminal in the vertical and horizontal directions by changing the angles of the arms 11 and 12 and rotating the right wheel 13b and left wheel 13c, and change the direction of imaging by the imaging unit 160 to embody the search for the charger by imaging.
According to the examples of the mobile communication terminal 100, the arms 11 and 12 corresponding to the arms can be driven to automatically change the imaging direction of the imaging unit 160 so that even if the charger 70 does not include a communication function, the mobile communication terminal 100, which functions as a robot, can search for the charger 70 and automatically move toward the charger 70 for charging with only the functions provided in the mobile communication terminal 100. In situations where the mobile communication terminal 100 is allowed to act freely as a robot, it naturally consumes power, but the mobile communication terminal 100 can perform charging by itself, reducing the time and efforts of the user. In addition, conventional mobile communication terminals may fail to be usable in an emergency if the user forgets to charge the battery, but this form of mobile communication terminal 100 reduces such risk.
The mobile communication terminals according to the above examples are not limited to the above examples, and may be other methods. Various examples of variations will be described below.
(1) In the above examples, the charger 70 may include a communication unit for teaching the location of the charger 70 to the mobile communication terminal 100 in place of the marker 71. The mobile communication terminal 100 may then identify the positional relationship between the terminal 100 and the charger 70 by executing communication with the charger 70, and move to the charger 70 to execute charging.
The charger 70 may include a device such as an ultrasonic wave generator in place of a communication unit, wherein the mobile communication terminal 100 is configured to be able to detect ultrasonic waves. The terminal may search for the charger 70 on the basis of the reception strength of the ultrasonic waves (the stronger the reception strength is, the closer the terminal is to the charger 70).
In place of the ultrasonic device, the charger 70 may be capable of performing infrared communication, wherein the mobile communication terminal 100 may retain infrared communication capability in the same manner to identify the positional relationship between it and the charger 70.
(2) In the above example, a configuration is described in which the marker 71 is attached to the mobile communication terminal 100 to allow the terminal 100 to easily search for the charger 70, but the marker 71 may be omitted. In this example, the mobile communication terminal 100 may retain information on the appearance of the charger 70 (images of the charger 70 captured from multiple different angles) and determine whether the charger 70 is included in the captured image by performing pattern matching with the captured image.
In place of the marker 71, the charger 70 itself may have a pattern painted on it that could serve as a marker. The mobile communication terminal 100 may read the pattern to identify the location of the charger 70. Alternatively, the charger 70 itself may be given a special coloring (coloring in a special color that is hard to be found in nature), and the terminal may be configured to determine whether the charger 70 is included in the captured image on the basis of whether the color is included in the captured image.
The marker 71 may correspond to a QR code, and the QR code may include an instruction to perform charging, and the mobile communication terminal 100 may be configured to start charging from the charger 70 upon receiving the charging instruction included in the QR code if the QR code is correctly read in the captured image. The correct reading of the QR code can be achieved by causing the imaging unit 160 to perform the image capturing in a manner as directly facing the QR code as possible. This can be achieved by a configuration in which the mobile communication terminal 100 is sufficiently close to the charger 70, and the state of the mobile communication terminal 100 is caused to be as close to the prone position as possible to capture the QR code. Therefore, in this example, the mobile communication terminal 100 may be recognized to be in a position to receive charging power supply from the charger 70, and charging can be started and performed without problems.
(3) In the example above, the processing shown in
(4) In the processing in the example above shown in
(5) In the examples above, the charger 70 is assumed to be a flat plate, but if the mobile communication terminal 100 is apart away from the charger 70, it may be captured in a state where it is hard to be recognized as the charger 70 from the captured image as shown in
(6) The programs according to the examples may be provided as stored in a computer-readable storage medium. The storage medium is directed to a “non-transitory tangible medium” capable of storing programs. The storage medium may include any suitable storage medium such as an HDD or SSD, or a suitable combination of two or more of these. The storage medium may be volatile, nonvolatile, or a combination of volatile and nonvolatile. The storage medium is not limited to these examples and may be any device or medium if it is capable of storing programs.
The mobile communication terminal 100 is capable of embodying the functional units illustrated in the examples by reading programs stored in a storage medium and executing the read programs, for example. The programs may be provided through any transmission medium (such as communication network and broadcast wave) to the mobile communication terminal 100. The mobile communication terminal 100 may, for example, embody the functions of the multiple functional units illustrated in the examples by executing programs downloaded via the Internet or other means.
The programs may also be implemented using, for example, scripting languages such as ActionScript, and JavaScript (registered trademark), object-oriented programming languages such as Objective-C, and Java (registered trademark), and markup languages such as HTML5.
At least a part of the processing in the mobile communication terminal 100 may be by cloud computing configured by one or more computers. Each of the functional units of the mobile communication terminal 100 may be one or more circuits that embody the functions illustrated in the above examples, or multiple functional units may be a single circuit.
(7) Although the examples have been described on the basis of various drawings and examples, those skilled in the art would be able to make various changes and modifications on the basis of this disclosure. Accordingly, these variations and modifications are included within the scope of this disclosure. For example, the functions included in each method and step, for example, may be rearranged to be avoided to be logically inconsistent, and multiple methods and steps may be integrated or divided. The configurations shown in the examples above may be combined as appropriate.
This application is a continuation application of International Application No. PCT/JP2020/045546, filed on Dec. 7, 2020, the contents of which are hereby incorporated by reference in its entirety.
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/JP2020/045546 | Dec 2020 | US |
| Child | 18206663 | US |
