Movable robot capable of providing a projected interactive user interface

Abstract

The present invention discloses a moveable robot that includes a processing control system; a rotation mechanism, a projection system that can be titled by the rotation mechanism to a first position to project a first image on a horizontal surface outside the body of the moveable robot, wherein the projection system is configured to be titled by the rotation mechanism to a second position to project a second image on a wall surface, and an optical sensing system configured to detect the user's movement, location, facial expression or gesture over the first image. The processing control system can interpret user's inputs based on the user's movement, location, facial expression or gesture over the first image projected on the horizontal surface outside the body of the moveable robot. The processing control system can control one or more outputs of the moveable robot based on the interpreted user's inputs.

Description

BACKGROUND OF THE INVENTION

The present application relates to robotic technologies, and in particular, to a robot having a novel mechanism for providing an interactive user interface.

Robotic technologies have seen a revival in recent years. Various robots are being developed for a wide range of applications such as care of senior citizens and patients, security and patrol, delivery, hotel services, and hospitality services. Typically, the robots not only need to independently move around a service area; they are also required to deliver messages to and receive commands from people. It has been a challenge to provide technologies that can effectively fulfill the above needs with simple and low-cost components and designs. With domestic technology on the rise, the quantity and complexity of social robots are becoming an important interaction design challenge to promoting the user's sense of control and engagement. Additionally, a wide range of interaction modalities for the social robots have been designed and researched, including GUI, voice control, gesture input, and augmented reality interfaces.

Social robots provide for an alternative mode of interaction, a way to simulate the way of communication between humans or between human and a pet using gestures, facial expression, body language and nonverbal behavior. Furthermore, by sharing physical space and objects with their users, people like to involve physical action on the part of the human for more user experience, better learning and higher engagement.

Other major social robot is to accompany 0-6 year old kids at home or a kindergarten. At this period of ages, kids are not good at verbal communication; instead they prefer physical body languages. There is therefore a need for a robot that can effectively interact with users and deliver content at a place and a time convenient to the users.

SUMMARY OF THE INVENTION

The presently application discloses a moveable robot having simple multi-purpose mechanisms for movement and user interface. A multi-purpose projection system can display an image with a body of the moveable robot, or project an image served as an interactive user interface on an external surface. A multi-purpose optical sensing system can detect objects in the environment as well as detecting user's movement, location, and gesture for receiving user inputs.

In one general aspect, the present invention relates to a moveable robot that includes a processing control system, a rotation mechanism under the control of the processing control system, a projection system that can be tilted by the rotation mechanism to a first position to project a first image on a horizontal surface outside the body of the moveable robot, wherein the projection system that can be tilted by the rotation mechanism to a second position to project a second image on a wall surface, an optical sensing system that can detect the user's movement, location, facial expression or gesture over the first image, wherein the processing control system is configured to interpret user's inputs based on the user's movement, location, facial expression or gesture over the first image projected on the horizontal surface outside the body of the moveable robot, wherein the processing control system can control one or more outputs of the moveable robot based on the interpreted user's inputs, wherein the optical sensing system is configured to detect objects surrounding the moveable robot; and a transport system that can produce, under the control of the processing control system, a movement on the horizontal surface in a moving path that avoids the objects detected by the optical sensing system.

Implementations of the system may include one or more of the following. The moveable robot can further include a head that houses the projection system; and a head tilt system under the control of the processing control system, wherein the head tilt system includes the rotation mechanism. The optical sensing system can detect the user's positions on the horizontal surface at a first time and at a second time, wherein the processing control system can calculate a first coordinate of the user at the first time and a second coordinate of the user at the second time, wherein the processing control system can determine if the displacement of the foot exceeds a predetermined threshold. The processing control system can calculate a direction of movement of the user's foot if the displacement of the foot exceeds a predetermined threshold. The processing control system can interpret a user input based on the direction of movement of the user's foot. The predetermined threshold can depend on the user's height. The second image can be a two-dimensional image formed on a surface. The rotation mechanism can tilt the projection system a third position to project a three-dimensional image in the air in the front the moveable robot. The optical sensing system can emit light beams to an object or a person and detect light reflected from the object or the person, wherein the processing control system can calculate locations of the objects based on the reflected lights. The optical sensing system can include a camera that detects light reflected from the surface where the first image is formed to detect the user's movement, location, facial expression or gesture over the first image. The optical sensing system can include an IR emitter and a depth camera configured to sense user's movement, location, facial expression or gesture and the objects surrounding the moveable robot. The optical sensing system can include a laser emitter and a rotating mirror configured to sense user's movement, location, facial expression or gesture and the objects surrounding the moveable robot.

In another general aspect, the present invention relates to a moveable robot, comprising: a processing control system; a projection system that can project a first image on a surface outside the body of the moveable robot; an optical sensing system that can detect the user's movement, location, facial expression or gesture over the first image, wherein the processing control system can interpret user's inputs based on the user's movement, location, facial expression or gesture over the first image projected on the surface outside the body of the moveable robot, wherein the optical sensing system can detect objects surrounding the moveable robot; and a transport system that can produce motion relative to the surface to plan its moving path or avoid the objects detected by the optical sensing system.

Implementations of the system may include one or more of the following. The moveable robot can further include a main housing body; an upper housing body comprising an optical window at a lower surface, wherein at least a portion of the project system is inside the upper housing body; a sliding platform comprising a sliding mechanism that can slide the upper housing body relative to the main housing body to expose a lower surface of the upper housing body, wherein the project system can emit light through the optical window at the lower surface of the upper housing body to project the first image on a surface outside the body of the moveable robot. The sliding mechanism can slide the upper housing body to a home position and a slide-out position, wherein the projection system is configured to display the second image on or inside a body of the moveable robot when the upper housing body is at the home position, wherein the projection system can project the first image on a surface outside the body of the moveable robot when the upper housing body is at the slide-out position.

The moveable robot can further include a rotation mechanism that can rotate the main housing body relative to the transport system. The projection system can include a projector configured to produce images; a mirror configured to reflect the images; and a steering mechanism configured to align the mirror to a first angle to produce the first image, and to align the mirror to a second angle to produce the second image. The projection system can display a second image on or inside a body of the moveable robot. The second image can a two-dimensional image formed on a surface. The second image can be a three-dimensional image formed inside the moveable robot. The optical sensing system can emit light beams to an object or a person and detect light reflected from the object or the person, wherein the processing control system can calculate locations of the objects based on the reflected lights. The optical sensing system can include a camera that detects light reflected from the surface where the first image is formed to detect the user's movement, location, facial expression or gesture over the first image. The optical sensing system can include an IR emitter and a depth camera configured to sense user's movement, location, facial expression or gesture and the objects surrounding the moveable robot. The optical sensing system can include a laser emitter and a rotating mirror configured to sense user's movement, location, facial expression or gesture and the objects surrounding the moveable robot.

These and other aspects, their implementations and other features are described in detail in the drawings, the description and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a front view of a moveable robot in accordance with the present application.

FIG. 2 shows a front view of a moveable robot in a configuration for projecting an image on the ground in accordance with the present application.

FIG. 3 is a bottom view of the upper housing body showing a sliding platform comprising a sliding mechanism between the head and the main body of the moveable robot in FIGS. 1 and 2.

FIG. 4A is a schematic diagram illustrating the sensing system in the moveable robot in accordance with the present application.

FIG. 4B is a schematic diagram illustrating an exemplified sensing system in the moveable robot in accordance with the present application.

FIG. 4C is a schematic diagram illustrating another exemplified sensing system in the moveable robot in accordance with the present application.

FIG. 5A shows a configuration of the projection system in the disclosed moveable robot for projecting an image on the surface as a part of an interactive user interface.

FIGS. 5B and 5C respectively show two display configurations for the projection system compatible with the disclosed moveable robot.

FIG. 6 is a system diagram of the moveable robot in accordance with the present application.

FIG. 7A shows a front view of another moveable robot in accordance with the present application.

FIG. 7B shows a side view of the moveable robot of FIG. 7A projecting an image on a wall.

FIG. 7C shows a side view of the moveable robot of FIG. 7A projecting an image on the ground.

FIG. 8 is a system diagram of the moveable robot in FIGS. 7A-7C.

DETAILED DESCRIPTION OF THE INVENTION

Referring to FIGS. 1 and 2, a moveable robot 100 includes a transport platform 120, an optical sensing system 130, a main housing body 140, a sliding platform 150, an upper housing body 160, and a microphone 170 in the upper housing body 160. The transport platform 120 includes a transport mechanism including wheels 125, which can move the moveable robot 100 to different locations. The main housing body 140 and the transport platform 120 are connected by a rotation mechanism that allows the main housing body 140 to rotate relative to the transport platform 120 so that components in the main housing body 140 and the upper housing body 160 can detect people and objects in the environment and display information and interact with people in the right directions.

The optical sensing system 130 can detect objects in the environment and assist the moveable robot 100 to design the best movement path and to avoid obstacles during movement. For example, the optical sensing system 130 can emit laser beams to the environment, receive bounced back laser signals, and detect objects and their locations by analyzing the bounced back signals. As described below, an important aspect of the presently disclosed robot is that the optical sensing system 130 can also detect user's movements, locations, gestures, facial expression, body language and nonverbal behavior over a projected user interface.

The sliding platform 150 enables the upper housing body 160 to slide relative to the main housing body 140 to expose a lower surface of the upper housing body 160. A project system (400, 610 in FIGS. 4 and 7, described below) can project a project image 180 on a surface 190 that the moveable robot 100 stands on. The surface can be a ground surface, a floor surface, a work platform, a tabletop, etc.

Referring to FIGS. 1-3, the sliding platform 150 includes a sliding mechanism 200 that includes a motor, a drive, a controller, a power supply, and mechanical components. The sliding platform 150 includes an optical window 205 for the projection optical path of the projection system (400, FIG. 4A).

In some embodiments, as shown in FIG. 3, the sliding mechanism 200 can further include a projector sliding plate 210, a projector sliding reference plate 220, a sliding block 230, a sliding rail 240, a pulley transmission mechanism 260, a timing belt locking bridge 270, a travel limit column 280, and a photoelectric sensor 290. The sliding mechanism 200 has two states: a home position and a slide-out position, whose positions are sensed by the photoelectric sensor 290 and locked by the travel limit column 280. The slide rail 240, the timing pulley drive mechanism 260, the travel limit column 280, and the photoelectric sensor 290 are mounted on the projector slide reference plate 220. The slide block 230 and the timing belt lock clamp 270 are installed on the projector sliding plate 210. The projector sliding plate 210 is further provided with a device for fixing the projector 250. In the present implementation, at least one slide/slide-fit structure is provided; the projector sliding plate 210 is sleeved on the slide rail 240 by the slide block 230. The timing belt locking bridge 270 is connected to the belt of the pulley transmission mechanism 260. The driving pulley of the pulley transmission mechanism 260 is fixedly connected with the rotor shaft of a motor mounted on the projector sliding reference plate 220. The stroke limiting post 280 engages with the notch on the projector slide-out plate 210. The motor can be implemented using a DC motor.

The pulley transmission mechanism 260 can include a driving pulley, a driven pulley, a belt, and a timing belt clamping plate 270. When the sliding mechanism 200 is in operation, the motor rotates according to the instruction sent by a processing control system (620 in FIG. 6, described below), drives the driving wheel to rotate, the driving wheel drives the belt to move left and right, the belt drives the synchronous belt clamping plate 270 to slide around, the synchronous wheel clamping plate 270. The projector slides out of the board 210 and the projector 250 can slide corresponding to the left and right of the belt.

Referring to FIGS. 4A and 5A, the moveable robot 100 includes a projection system 400 that includes a mirror 403 and a projector 408 in the upper housing body (160 in FIGS. 1 and 2). The mirror 403 can rotate with a steering mechanism (not shown in figures) to change the angle of the mirror 403 relative to the ground 190 such that the projector 408 can project, via the mirror 403, the image 180 on the surface 190. One purpose of the mirror 403 can allow more flexibility for the projector 408 to be installed in very limited upper housing body 160. If the upper housing body 160 is big enough and a slim projector 408 can be found, the mirror 403 and the steering mechanism can be optional. The image 180 can include common features of an interactive user interface such as a border, functional symbols, selection buttons, a grid, a table, text and graphics, etc. The image 180 can also include product and other commercial information. A camera 406 is installed on the upper position in the moveable robot 100, which can detect the bounced lights from the surface 190 comprising user 450's movement over the image 180.

Referring to FIGS. 2-3, the components of the sliding mechanism 200 (except for the slide rail 240, FIG. 3), the projector 408 and the mirror 403 can be mounted in the upper housing body 160. As the upper housing body 160 is shifted relative to the main housing body 140 (shown in FIG. 2), the optical window 205 for the projection optical path is exposed at the bottom of the upper housing body 160. An image produced by the projector 408 is reflected by the mirror 403. The steering mechanism changes the angle α of the mirror 403 relative to the surface 190, to direct the light to casts on the surface 190 to form a projected image 180.

In accordance with an important aspect of the present application, the project image 180 can provide a user interface that includes functional input areas. A user 450 can move about the project image 180 and produce gestures, facial expression, body language and other nonverbal behaviors at different functional input areas. The optical sensing system 130 has dual functions: in addition to detecting objects in the environment, it can detect user's movements, locations, gestures, facial expression, body language and other nonverbal behaviors and interpret them as inputs to the processing control system (620 in FIG. 6, described below). The laser optical sensing system 470 can emit a light beam 460, which is reflected and bounced off from surrounding objects or by a user. The camera 406 is installed on the upper position in the moveable robot 100, which can detect the bounced lights from the surfaces. The processing control system (620 in FIG. 6, described below) conducts computations to calculate the positions and distances of the surrounding objects, or user's movement, location, facial expression or gesture.

In some embodiments, LIDAR (light detection and ranging) can be used as sensors for both sensing human movement for interaction purpose and building the mapping to move in a complex environment. LIDAR is a surveying method that measures distance to a target by illuminating the target with pulsed laser light and measuring the reflected pulses with a sensor. Differences in laser return times and wavelengths can then be used to make digital 2D or 3D representations of the target. Time-of-flight (TOF) cameras are sensors that can measure the depths of scene-points, by illuminating the scene with a controlled laser or LED source, and then analyzing the reflected light. The ability to remotely measure range is extremely useful and has been extensively used for mapping and surveying.

One type of LIDAR system uses a single laser or multiple line laser fire onto a rotating mirror spinning at high speed and hence to view objects in a single plane or multiple planes at different heights, a 360-degree field of view is generated. As shown in FIG. 4B, a 3D image sensing system 413 comprising a spinning mirror for scanning laser beam can sense all the objects in its plane to detect objects in the environments and user movement.

Another type of LIDAR system includes IR emitter and depth camera. IR emitter projects a pattern of infrared light. As the light hits a surface, the pattern becomes distorted and the depth camera reads the distortion. The depth camera analyzes IR patterns to build a 3D map of the room and peoples action within it. As shown in FIG. 4C, a 3D image sensing system 480 comprising an IR emitter and a depth camera can view all the objects in its range to detect environments and user movement.

The optical sensing system using LIDAR can fulfill two functions: 1. building a map for the complex environment for robot to plan it path through obstacles; 2. collecting the body movement of humans relating to the position of the projected image on the surface and analyze the motion to activate corresponding response. In this case, one component of LIDAR can replace both the camera 406 and the optical sensing system 130 because it usually integrates the light emitter and sensor into one component.

In some embodiments, the projection system 400 can display images on a surface of the upper housing body 160 or inside the upper housing body 160 while the upper housing body 160 is at its home position. Referring to FIG. 5B, the upper housing body 160 includes a winder 510. The angle β of the mirror 403 relative to the surface 190 (FIG. 5B) is adjusted by the steering mechanism. An image is projected and displayed at the winder 510 viewable by users. Moreover, referring to FIG. 5C, the angle β1 of the mirror 403 relative to the surface 190 (FIG. 5A) is adjusted by the steering mechanism. An image is projected and displayed on a holographic film 520 inside the upper housing body 160. Alternatively, a three-dimensional hologram image is formed inside the upper housing body 160 by the projection system 400. The image on the holographic film 520 or the hologram image can be viewed by a user through a transparent window in the upper housing body 160.

FIG. 6 shows a system diagram of the moveable robot in accordance with the present application. The moveable robot 100 includes a projection system 610, a processing control system 620, an optical sensing system 630, a transport system 640, and a rotation mechanism 650. The project system 610 can display two-dimensional images or three-dimensional holograms in the upper housing body 160, viewable by users under the control of the processing control system 620. The project system 610 can also project an image on a surface outside of in the upper housing body 160 under the control of the processing control system 620. The projected image is not only viewable by users, but also serves as an interactive user interface for taking input from users. Under the control of the processing control system 620, the optical sensing system 630 can not only scan and determine locations of objects in the environment (for motion path planning and object avoidance during movement); it can also detect user's movement, locations, gestures, facial expression, body language and other nonverbal behaviors over the project area, which are interpreted as user inputs by the processing control system 620. The processing control system 620 can also controls a rotation mechanism 650 to rotate the main housing body (140, FIGS. 1-2, 5-6B) relative to the transport platform (120, FIGS. 1-2, 5-6B).

In some embodiments, Referring to FIGS. 7A-7C, a moveable robot 700 includes a transport & rotation system 720, an optical sensing system 730, a housing body 740, a head 750, a rotation mechanism 760, and a projection system 770 in the head 750. The transport & rotation system 720 includes wheels 725 and can move the moveable robot 700 to different locations on a surface 790.

The transport & rotation system 720 can also rotate the moveable robot 700 to different directions to allow the optical sensing system 730 detect people and objects in the environment in the right directions. The optical sensing system 730 can detect objects in the environment and assist the moveable robot 700 to design the best movement path and to avoid obstacles during movement. For example, the optical sensing system 730 can emit laser beams to the environment, receive bounced back laser signals, and detect objects and their locations by analyzing the bounced back signals. As described below, an important aspect of the presently disclosed robot is that the optical sensing system 730 can also detect user's movements, locations, gestures, facial expression, body language and other nonverbal behaviors over a projected user interface.

The rotation of the moveable robot 700 by the transport & rotation system 720 allows the projection system 770 to face right polar direction. Furthermore, the rotation mechanism 760 can tilt the head 750 up and down for projection on different surfaces. For example, the head 750 can be tilted by the rotation mechanism 760 to a position to project on a wall 780 (FIG. 7B). In some embodiments, the head 750 can be tilted up to another position by the rotation mechanism 760, which allows the projection system 770 to project a 3D image in the air in front of the moveable robot 700. In another example, the head 750 can be tilted down to yet another position by the rotation mechanism 760 to project on the ground 790 (FIG. 7C).

FIG. 8 shows a system diagram of the moveable robot 700. The moveable robot 700 includes the projection system 770, a processing control system 820, the optical sensing system 730, the transport and rotation system 720, and a head tilt system 850 comprising the rotation mechanism 760. The project system 770 can project an image on a surface outside of in the housing body 740 under the control of the processing control system 820.

The projected image is not only viewable by users, but also serves as an interactive user interface for taking input from users. Under the control of the processing control system 620, the optical sensing system 730 can not only scan and determine locations of objects in the environment (for motion path planning and object avoidance during movement); it can also detect the movements, locations, gestures, facial expression, body language and other nonverbal behaviors of a user 840 over the project area, which are interpreted as user inputs by the processing control system 620. Based on the interpreted user inputs, the processing control system 820 can employ a decision-making algorithm to make decisions to further control the outputs of the moveable robot 700 to interact with or give instructions to the user 840. The outputs of the moveable robot 700 can include one or a combination: a projected content by the projection system 770, a sound, or a rotation or a movement by the transport and rotation system 720.

Referring to FIG. 9 and FIG. 7C, and similar to the descriptions above relating to FIG. 4C, the position of a user's foot is measured by the optical sensing system (step 910). When the foot is detected to be inside a projected area, a first coordinate of the user's foot at a first time is calculated by the processing control system (step 920). In some embodiments, the projected area can be divided into zones. Different users can stand and move in different zones. After a movement is detected by the user's foot, a second coordinate of the user's foot at a second time is calculated by the processing control system when the foot is inside a projected area (step 930). The processing control system determines if the displacement of the foot exceeds a predetermined threshold (step 940). The threshold can be dependent on the user's height and the specific application (e.g. the game that the user is playing, or the imaging program that the user is using). The direction of movement of the user's foot is calculated by the processing control system if the displacement of the foot is more than the predetermined threshold (step 950). A user input is interpreted from the direction of movement by the processing control system (step 960).

It should be noticed that the above examples are intended to illustrate the concept of the present invention. The present invention may be compatible with many other configurations and variations: for example, the shape of the moveable robot is not limited to the examples illustrated. In addition, the moveable robot can include fewer or additional housing bodies. For example, the main housing body and the transport platform can be combined and the rotation of the robot relative to the ground surface can be accomplished by the transport mechanism in the transport platform rather than accomplished by a separate rotation mechanism. Furthermore, the sliding mechanism can be realized by other mechanical and electronic components. Moreover, the optical scanning system can be implemented in other configurations to provide the capability of sensing both an object and a person's location, movements, gestures, facial expression, body language and other nonverbal behaviors.

While this document contains many specifics, these should not be construed as limitations on the scope of an invention that is claimed or of what may be claimed, but rather as descriptions of features specific to particular embodiments. Certain features that are described in this document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable sub-combination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a sub-combination or a variation of a sub-combination.

Only a few examples and implementations are described. Other implementations, variations, modifications and enhancements to the described examples and implementations may be made without deviating from the spirit of the present invention.

Claims

1. A moveable robot, comprising: a processing control system;a rotation mechanism under the control of the processing control system;a projection system configured to be titled by the rotation mechanism to a first position to project a first image on a horizontal surface outside the body of the moveable robot, wherein the projection system is configured to be titled by the rotation mechanism to a second position to project a second image on a wall surface;an optical sensing system configured to detect the user's movement, location, facial expression or gesture over the first image,wherein the processing control system is configured to interpret user's inputs based on the user's movement, location, facial expression or gesture over the first image projected on the horizontal surface outside the body of the moveable robot,wherein the processing control system is configured to control one or more outputs of the moveable robot based on the interpreted user's inputs,wherein the optical sensing system is configured to detect objects surrounding the moveable robot; anda transport system under the control of the processing control system and configured to produce a movement on the horizontal surface in a moving path that avoids the objects detected by the optical sensing system.

2. The moveable robot of claim 1, further comprising: a head that houses the projection system; anda head tilt system under the control of the processing control system, wherein the head tilt system includes the rotation mechanism.

3. The moveable robot of claim 1, wherein the optical sensing system is configured to detect the user's positions on the horizontal surface at a first time and at a second time, wherein the processing control system is configured to calculate a first coordinate of the user at the first time and a second coordinate of the user at the second time, wherein the processing control system is configured to determine if the displacement of the foot exceeds a predetermined threshold.

4. The moveable robot of claim 3, wherein the processing control system is configured to calculate a direction of movement of the user's foot if the displacement of the foot exceeds a predetermined threshold.

5. The moveable robot of claim 4, wherein the processing control system is configured to interpret a user input based on the direction of movement of the user's foot.

6. The moveable robot of claim 3, wherein the predetermined threshold is dependent on the user's height.

7. The moveable robot of claim 1, wherein the second image is a two-dimensional image formed on a surface.

8. The moveable robot of claim 1, wherein the rotation mechanism is configured to tilt the projection system a third position to project a three-dimensional image in the air in the front the moveable robot.

9. The moveable robot of claim 1, wherein the one or more outputs of the moveable robot include one or more of a facial expression or a projected content by the projection system.

10. The moveable robot of claim 1, wherein the one or more outputs of the moveable robot include one or more of a sound, a rotation of the rotation system, or a movement by the transport system.

11. The moveable robot of claim 1, wherein the optical sensing system is configured to emit light beams to an object or a person and detect light reflected from the object or the person, wherein the processing control system is configured to calculate locations of the objects based on the reflected lights.

12. The moveable robot of claim 11, wherein the optical sensing system includes a camera that detects light reflected from the surface where the first image is formed to detect the user's movement, location, facial expression or gesture over the first image.

13. The moveable robot of claim 11, wherein the optical sensing system includes an IR emitter and a depth camera configured to sense user's movement, location, facial expression or gesture and the objects surrounding the moveable robot.

14. The moveable robot of claim 11, wherein the optical sensing system includes a laser emitter and a rotating mirror configured to sense user's movement, location, facial expression or gesture and the objects surrounding the moveable robot.