Patent Grant
10104278
1. Field of the Invention
The present invention relates to a pan and tilt operation device, a camera system, a pan and a tilt operation program, and a pan and tilt operation method, and particularly, to a technology for remotely operating a camera comprising a pan and tilt mechanism.
2. Description of the Related Art
Conventionally, a pan and tilt camera system that remotely operates a pan and tilt camera using a pan and tilt device is known (JP2000-209575A).
The pan and tilt camera system described in JP2000-209575A includes a pan and tilt camera including a camera unit that outputs a video signal and a pan and tilt mechanism that changes an imaging direction of a camera according to an input direction control signal, and a pan and tilt operation device including direction operation means for designating a movement direction of the pan and tilt camera and a control unit that controls a direction control signal according to an operation of the direction operation means, and comprises discrimination means for detecting a top and bottom of an installation posture of the pan and tilt camera and discriminating whether the top and bottom is the same as a predetermined top and bottom posture. The control unit maintains up, down, left, and right movement directions of the direction control signal to be up, down, left, and right directions as they are or inverts up and down directions and left and right directions according to a result of the discrimination of the discrimination means and outputs the resultant direction control signal to the pan and tilt mechanism.
Thus, the up, down, left, and right movement direction of the pan and tilt camera designated by the direction operation means is caused to match up, down, left, and right movement directions of the actually moving pan and tilt camera regardless of the installation posture of the top and bottom of the pan and tilt camera to achieve improved operability.
Further, JP2006-145761A describes a camera that detects a posture of a camera body and changes assignment of an up key, a down key, a left key, and a right key of a cross key provided on a rear surface of the camera body according to the detected posture of the camera body when the camera performs horizontal imaging or vertical imaging that is performed while holding the camera body in a normal direction or in a direction substantially perpendicular to the normal direction. Specifically, the left and right keys are assigned as zoom keys at the time of horizontal imaging, and the up and down keys are assigned as the zoom keys at the time of vertical imaging.
In the pan and tilt camera system described in JP2000-209575A, up, down, left, and right movement directions of the pan and tilt camera designated by the direction operation means is caused to match up, down, left, and right movement directions of the actually moving pan and tilt camera regardless of the installation posture of the top and bottom of the pan and tilt camera to achieve improved operability. However, in a case where the pan and tilt mechanism of the pan and tilt camera satisfies a specific condition (for example, in a case where an imaging optical axis direction of the camera unit substantially matches a central axis of rotation of a pan mechanism of the pan and tilt mechanism), there is a problem in that even when a direction control signal for a left and right movement direction (pan direction) is output to the pan and tilt mechanism, a video to be captured rotates (rolls) instead of moving in a left and right direction, and an operator remotely operating the pan and tilt camera feels discomfort between a pan and tilt operation instruction and the obtained video.
Further, in JP2006-145761A, the camera in which functions of the respective up, down, left and right keys of the cross key provided on the rear surface of the camera body can be changed between the time of horizontal imaging and the time of vertical imaging has been described, but this camera is a camera which is grasped by the hand, but is not a pan and tilt camera that is installed in a specific position and direction.
The present invention has been made in view of these circumstances, and an object thereof is to provide a pan and tilt operation device, a camera system, a pan and tilt operation program, and a pan and tilt operation method capable of remotely operating a camera without discomfort regardless of an installation posture of the camera.
In order to achieve the above object, an aspect of the present invention provides pan and tilt operation device that is wirelessly or wiredly connected to a camera, the camera including an imaging unit including an imaging lens and an imaging element, and a pan and tilt mechanism that rotates the imaging unit in a horizontal direction and a vertical direction with respect to a camera body, and remotely operates the camera, the pan and tilt operation device comprising: a posture information acquisition unit that acquires posture information indicating an installation posture of the camera; a touch panel including a display panel, and an operation panel that detects touched coordinates on the display panel; a display control unit that displays a cross key including icons indicating an up key, a down key, a left key, and a right key on the display panel; a key operation detection unit that detects whether any of the up key, the down key, the left key, and the right key of the cross key is touched on the basis of the coordinates detected by the operation panel; a motor control assignment setting unit that sets motor control assignment indicating a correspondence relationship between the touch operation of the up key, the down key, the left key, or the right key of the cross key detected by the key operation detection unit and assignment of the pan motor or the tilt motor that drives the pan and tilt mechanism and a motor rotation direction on the basis of the posture information acquired by the posture information acquisition unit; and a motor driving command output unit that outputs a motor driving command corresponding to the detected key to the camera when the touch operation of the up key, the down key, the left key, or the right key of the cross key is detected by the key operation detection unit, the motor driving command output unit outputting the motor driving command obtained by selecting the pan motor or the tilt motor according to the motor control assignment set by the motor control assignment setting unit and specifying the rotation direction of the selected motor, in which the display control unit discriminates a specific condition under which a video captured by the camera rotates when the pan motor is driven, on the basis of a tilt angle of the pan and tilt mechanism, and switches the corresponding icon among the icons indicating the up key, the down key, the left key, and the right key of the cross key displayed on the display panel from a first icon indicating an up and down direction or a left and right direction to a second icon indicating a rotation direction when the specific condition is discriminated.
According to the aspect of the present invention, since the assignment and the rotation direction of the pan motor and the tilt motor of each key of the cross key are set on the basis of the installation posture of the camera, it is possible to operate the cross key without discomfort when the camera is caused to perform the pan operation or the tilt operation using the cross key. Further, if the camera is caused to perform the pan operation using the cross key and the specific condition under which the video rotates is discriminated from the tilt angle, since the display of the corresponding icon of the cross key is switched from the first icon indicating the up and down direction or the left and right direction to the second icon indicating the rotation direction, it is possible not to cause discomfort even when the video rotates.
In pan and tilt operation device according to another aspect of the present invention, it is preferable that the pan and tilt operation device further comprises an image acquisition unit that acquires a live view image captured by the camera from the camera, and the display control unit displays the live view image acquired by the image acquisition unit on the display panel. Accordingly, it is possible to operate the camera while viewing the live view image.
In pan and tilt operation device according to still another aspect of the present invention, it is preferable that the posture information acquisition unit acquires posture information indicating an installation posture of the camera from an installation posture input unit in which the installation posture is set, through wireless or wired communication from the camera or a manual operation.
In pan and tilt operation device according to still another aspect of the present invention, it is preferable that the pan and tilt operation device further comprises an angle acquisition unit that acquires a pan angle and a tilt angle of the pan and tilt mechanism from the camera, and the motor control assignment setting unit sets the motor control assignment on the basis of the posture information acquired by the posture information acquisition unit and the pan angle and the tilt angle acquired by the angle acquisition unit. This is because it is necessary to change the motor control assignment for the operation of the cross key according to the pan angle or the tilt angle of the pan and tilt mechanism.
In pan and tilt operation device according to still another aspect of the present invention, it is preferable that the posture information indicating the installation posture of the camera is information indicating any of a posture when the camera is installed on an upper surface of a horizontal plane, a posture when the camera is installed on a lower surface of the horizontal plane, and a posture when the camera is installed in a vertical plane.
In pan and tilt operation device according to still another aspect of the present invention, it is preferable that the pan and tilt operation device further comprises an angle acquisition unit that acquires a pan angle and a tilt angle of the pan and tilt mechanism from the camera, and the display control unit graphically displays the pan angle and the tilt angle acquired by the angle acquisition unit on the display panel. Thus, it is possible to confirm how much the pan and tilt mechanism can be moved in a certain direction, and this can be used as a reference when the cross key is operated.
In pan and tilt operation device according to still another aspect of the present invention, it is preferable that the display control unit changes a form of the graphical display according to whether a specific condition under which a video captured by the camera rotates is satisfied. Accordingly, it is possible to confirm whether a video moves in the pan direction or rotates from a change in a form of a graphic display for displaying the pan angle and the tilt angle, as well as the switching of the display form of the icon indicating the cross key.
Still another aspect of the present invention provides a camera system including a camera and a pan and tilt operation device that is wirelessly or wiredly connected to the camera and remotely operates the camera, in which the camera comprises an imaging unit including an imaging lens and an imaging element, a pan and tilt mechanism that rotates the imaging unit in a horizontal direction and a vertical direction with respect to a camera body, a pan driving unit and a tilt driving unit that drives the pan and tilt mechanism using a pan motor and a tilt motor, an angle detection unit that detects each of a pan angle and a tilt angle of the pan and tilt mechanism, a motor driving command input unit that receives a motor driving command for the pan motor and the tilt motor from the pan and tilt operation device, and a pan and tilt control unit that controls the pan and tilt mechanism via the pan driving unit and the tilt driving unit on the basis of the motor driving command input from the motor driving command input unit, the pan and tilt operation device comprises a posture information acquisition unit that acquires posture information indicating an installation posture of the camera, an angle acquisition unit that acquires angle information indicating a pan angle and a tilt angle of the camera, a touch panel including a display panel, and an operation panel that detects touched coordinates on the display panel, a display control unit that displays a cross key including icons indicating an up key, a down key, a left key, and a right key on the display panel, a key operation detection unit that detects whether any of the up key, the down key, the left key, and the right key of the cross key is touched on the basis of the coordinates detected by the operation panel, a motor control assignment setting unit that sets motor control assignment indicating a correspondence relationship between the touch operation of the up key, the down key, the left key, or the right key of the cross key detected by the key operation detection unit and assignment of the pan motor or the tilt motor that drives the pan and tilt mechanism and a motor rotation direction on the basis of the posture information acquired by the posture information acquisition unit, and a motor driving command output unit that outputs a motor driving command corresponding to the detected key to the camera when the touch operation of the up key, the down key, the left key, or the right key of the cross key is detected by the key operation detection unit, the motor driving command output unit outputting the motor driving command obtained by selecting the pan motor or the tilt motor according to the motor control assignment set by the motor control assignment setting unit and specifying the rotation direction of the selected motor, and the display control unit discriminates a specific condition under which a video captured by the camera rotates when the pan motor is driven, on the basis of the tilt angle acquired by the angle acquisition unit, and switches the corresponding icon among the icons indicating the up key, the down key, the left key, and the right key of the cross key displayed on the display panel from a first icon indicating an up and down direction or a left and right direction to a second icon indicating a rotation direction when the specific condition is discriminated.
In a camera system according to still another aspect of the present invention, the camera includes a live view image output unit that outputs the live view image captured by the imaging unit to the pan and tilt operation device, the pan and tilt operation device includes an image acquisition unit that acquires the live view image captured by the camera from the camera, and the display control unit displays the live view image acquired by the image acquisition unit on the display panel.
In a camera system according to still another aspect of the present invention, the live view image output unit outputs a live view image in which the live view image is erect in a case where the installation posture of the camera is a reference installation posture, and the display control unit has a function of rotating and displaying the acquired live view image, and rotates the acquired live view image to erect the live view image to be displayed on the display panel in a case where the installation posture of the camera is different from the reference installation posture.
In a camera system according to still another aspect of the present invention, it is preferable that the camera includes a posture sensor that detects the installation posture of the camera; an angle detection unit that detects each of the pan angle and the tilt angle of the pan and tilt mechanism; and an information output unit that outputs posture information indicating the installation posture detected by the posture sensor and angle information indicating the pan angle and the tilt angle detected by the angle detection unit to the pan and tilt operation device, and the posture information acquisition unit and the angle acquisition unit acquire the posture information and the angle information from the information output unit, respectively.
In a camera system according to still another aspect of the present invention, it is preferable that the pan and tilt operation device comprises an installation posture input unit in which the installation posture is set by a manual operation, and the posture information acquisition unit acquires posture information indicating the installation posture of the camera from the installation posture input unit.
In a camera system according to still another aspect of the present invention, it is preferable for the motor control assignment setting unit to set the motor control assignment on the basis of the posture information, and the pan angle and the tilt angle respectively acquired by the posture information acquisition unit and the angle acquisition unit.
In a camera system according to still another aspect of the present invention, it is preferable that at least one of the camera and the pan and tilt operation device comprises a recording unit that records the image captured by the imaging unit.
Still another aspect of the present invention provides a pan and tilt operation program that causes a portable terminal including a wireless communication unit, a touch panel including a display panel and an operation panel that detects touched coordinates on the display panel, and a computer to function as a pan and tilt operation device that remotely operates a camera including an imaging unit including an imaging lens and an imaging element, and a pan and tilt mechanism that rotates the imaging unit in a horizontal direction and a vertical direction with respect to a camera body, the pan and tilt operation program causing the computer to realize: a function of acquiring posture information indicating an installation posture of the camera; a function of displaying a cross key including icons indicating an up key, a down key, a left key, and a right key on the display panel; a function of detecting whether any of the up key, the down key, the left key, and the right key of the cross key is touched on the basis of the coordinates detected by the operation panel; a function of setting motor control assignment indicating a correspondence relationship between the touch operation of the up key, the down key, the left key, or the right key of the cross key and assignment of the pan motor or the tilt motor that drives the pan and tilt mechanism and a motor rotation direction on the basis of the acquired posture information; and a function of outputting a motor driving command corresponding to the detected key to the camera when the touch operation of the up key, the down key, the left key, or the right key of the cross key is detected, the function including outputting the motor driving command obtained by selecting the pan motor or the tilt motor according to the set motor control assignment and specifying the rotation direction of the selected motor, in which the function of displaying a cross key includes discriminating a specific condition under which a video captured by the camera rotates when the pan motor is driven, on the basis of a tilt angle of the pan and tilt mechanism, and switching the corresponding icon among the icons indicating the up key, the down key, the left key, and the right key of the cross key displayed on the display panel from a first icon indicating an up and down direction or a left and right direction to a second icon indicating a rotation direction when the specific condition is discriminated.
By further installing the pan and tilt operation program according to the aspect of the present invention in an existing portable terminal such as a smartphone or a tablet terminal, it is possible to cause the existing portable terminal to function as the pan and tilt operation device that remotely operates the camera.
In the pan and tilt operation program according to still another aspect of the present invention, it is preferable that the pan and tilt operation program further causes the computer to realize a function of acquiring a live view image captured by the camera from the camera, and a function of displaying the acquired live view image on the display panel.
In the pan and tilt operation program according to still another aspect of the present invention, it is preferable that the pan and tilt operation program further causes the computer to realize a function of acquiring the pan angle and the tilt angle of the pan and tilt mechanism from the camera, and the function of setting the motor control assignment includes setting the motor control assignment on the basis of the acquired posture information, the pan angle, and the tilt angle. A computer-readable non-transitory recording medium having the program recorded thereon is also included in the aspect of the present invention.
Still another aspect of the present invention provides a pan and tilt operation method that includes wirelessly or wiredly connecting to a camera, the camera including an imaging unit including an imaging lens and an imaging element, and a pan and tilt mechanism that rotates the imaging unit in a horizontal direction and a vertical direction with respect to a camera body, and remotely operating the camera, the pan and tilt operation method comprising: a step of acquiring posture information indicating an installation posture of the camera; a step of displaying a cross key including icons indicating an up key, a down key, a left key, and a right key on a display panel of a touch panel including the display panel, and an operation panel that detects touched coordinates on the display panel; a step of detecting whether any of the up key, the down key, the left key, and the right key of the cross key displayed on the display panel is touched on the basis of the coordinates detected by the operation panel; a step of setting motor control assignment indicating a correspondence relationship between a touch operation of the up key, the down key, the left key, or the right key of the cross key and assignment of a pan motor or a tilt motor that drives the pan and tilt mechanism and a motor rotation direction on the basis of the acquired posture information; and a step of outputting a motor driving command corresponding to the detected key to the camera when the touch operation of the up key, the down key, the left key, or the right key of the cross key is detected, the step including outputting the motor driving command obtained by selecting the pan motor or the tilt motor according to the set motor control assignment and specifying the rotation direction of the selected motor, in which the step of displaying a cross key includes discriminating a specific condition under which a video captured by the camera rotates when the pan motor is driven, on the basis of a tilt angle of the pan and tilt mechanism, and switching the corresponding icon among the icons indicating the up key, the down key, the left key, and the right key of the cross key displayed on the display panel from a first icon indicating an up and down direction or a left and right direction to a second icon indicating a rotation direction when the specific condition is discriminated.
According to the present invention, it is possible to operate the cross key without discomfort when the camera is caused to perform the pan operation or the tilt operation using the cross key regardless of the installation posture of the camera. In particular, since the display of the corresponding icon of the cross key is switched to the icon indicating the rotation direction in the case of a specific condition under which the video rotates if the camera is caused to perform the pan operation using the cross key, it is possible not to cause discomfort even when the video rotates.
FIGS. 10A1 to 10B3 are diagrams illustrating that a movement direction (including rotation) of the angle of view for imaging is changed according to a range of a tilt angle of the pan and tilt mechanism when a pan and tilt camera is installed on an upper surface of the horizontal plane.
Hereinafter, embodiments of pan and tilt operation device, a pan and tilt camera system, a pan and tilt operation program, and a pan and tilt operation method according to the present invention will be described with reference to the accompanying drawings.
[Pan and Tilt Camera System]
The pan and tilt camera system includes a pan and tilt camera 10, and a pan and tilt operation device 100 (a smartphone in this example) according to the present invention.
<Pan and Tilt Camera>
The pan and tilt camera 10 mainly includes a camera body 12, a pedestal 14, a holding portion 16 that is fixed to the pedestal 14 and rotatably holds an imaging unit 20, and a dome cover 18 that covers the imaging unit 20.
The pedestal 14 is disposed to be rotatable about an axis in a vertical direction Z of the camera body 12, and is rotated about the axis in the vertical direction Z by a pan driving unit 34 (
The holding portion 16 includes a gear 16A provided coaxially with an axis in a horizontal direction X, and rotates the imaging unit 20 in an up and down direction (a tilt operation) using a driving force transmitted from a tilt driving unit 36 (
The dome cover 18 is a cover for dust-proof and drip-proof, and is preferably a spherical shell shape having a constant thickness, which has an intersection between an axis in the horizontal direction X and an axis in the vertical direction Z as a center of curvature so that optical performance of the imaging unit 20 does not change regardless of an optical axis direction L of the imaging unit 20.
Further, it is preferable for a tripod attachment portion (for example, tripod screw hole) (not illustrated) to be provided on a rear surface of the camera body 12.
An imaging start button 19 for instructing imaging start and a power switch (not illustrated) are provided in the pan and tilt camera 10, but the pan and tilt camera 10 includes a wireless communication unit 50 (
This pan and tilt camera 10 captures at least one of a still image and a moving image, and roughly includes the imaging unit 20, a pan and tilt device 30, a control unit 40, and the wireless communication unit 50.
The imaging unit 20 includes, for example, an imaging lens 22 and an imaging element 24. The imaging lens 22 includes a single focus lens or a zoom lens, and forms a subject image on an imaging surface of the imaging element 24. A focus lens, a variable magnification lens (in the case of a zoom lens), and a diaphragm 23 included in the imaging lens 22 is driven by a lens driving unit 26.
The imaging element 24 in this example is a color imaging device in which primary color filters of three primary colors including red (R), green (G), and blue (B) are arranged in each pixel in a predetermined pattern (for example, a Bayer array, G stripe R/G full checkered, an X-Trans (registered trademark) array, or a honeycomb array), and includes a complementary metal oxide semiconductor (CMOS) type image sensor. The imaging element 24 is not limited to the CMOS image sensor and may be a charge coupled device (CCD) type image sensor.
The imaging element 24 is driven by a CMOS driver 28 including, for example, a vertical driver and a horizontal driver, and a timing generator (TG) 29. From the imaging element 24, a pixel signal according to an incident light amount of subject light (a digital signal corresponding to signal charge accumulated in each pixel) is read.
The pan and tilt device 30 includes, for example, a pan mechanism that rotates the imaging unit 20 with respect to the camera body 12 in a horizontal direction (pan direction), and a tilt mechanism that rotates the imaging unit 20 in a vertical direction (tilt direction) (hereinafter referred to as a “pan and tilt mechanism”) 32, a pan driving unit 34, and a tilt driving unit 36, as illustrated in
The pan driving unit 34 and the tilt driving unit 36 include a pan motor and a tilt motor including a stepping motor, and motor drivers that drive the motors, respectively, and outputs a driving force to the pan and tilt mechanism 32 to drive the pan and tilt mechanism 32.
The control unit 40 mainly includes a signal processing unit 41, an imaging control unit 42, a lens control unit 43, a pan and tilt control unit 44, and a camera work control unit 46.
The signal processing unit 41 performs signal processing such as offset processing, a gain control process including white balance correction and sensitivity correction, a gamma correction process, demosaic processing (demosaicing process), and RGB/YC conversion process on a digital image signal input from the imaging unit 20. Here, the demosaic processing is a process of calculating all of color information for each pixel from a mosaic image corresponding to a color filter array of a single plate type color imaging element, and is also referred to as a synchronization process. For example, in the case of an imaging element including color filters for three colors including RGB, the demosaic processing is a process of calculating color information of all of RGB for each pixel from a mosaic image consisting of RGB. Further, the RGB/YC conversion process is a process of generating luminance data Y and color difference data Cb and Cr from RGB image data subjected to demosaic processing.
The imaging control unit 42 is a unit that instructs discharge of charge accumulated in a capacitor of each pixel of the imaging element 24 via the CMOS driver 28 and the TG 29 or reading of a signal corresponding to the charge accumulated in the capacitor, and performs imaging control of a still image or a moving image.
The lens control unit 43 is a unit that controls the focus lens, the variable magnification lens, and the diaphragm 23 included in the imaging lens 22 via the lens driving unit 26, and performs, for example, autofocus (AF) control to move the focus lens to a focusing position. The AF control is performed by integrating an absolute value of a high-frequency component of the digital signal corresponding to the AF area, detecting the focusing position at which the integrated value (AF evaluation value) is maximized, and moving the focus lens to the detected focusing position.
The pan and tilt control unit 44 is a unit that controls the pan and tilt device 30, and outputs a motor driving command input from the smartphone 100 via the wireless communication unit 50 and the camera work control unit 46 (motor driving command input unit), to the pan driving unit 34 and the tilt driving unit 36 of the pan and tilt mechanism 32.
The posture sensor 45 is a sensor that detects an installation posture of the pan and tilt camera 10. A gyro sensor can be used as the posture sensor 45. In this example, the posture sensor 45 detects any one of a posture when the pan and tilt camera 10 is installed on an upper surface of a horizontal plane, a posture when the pan and tilt camera 10 is installed on a lower surface of the horizontal plane, and a posture when the pan and tilt camera 10 is installed on a vertical plane. A posture when the pan and tilt camera 10 is installed on the upper surface of the horizontal plane corresponds to, for example, a posture when the pan and tilt camera is placed on a desk or the pan and tilt camera is attached to a tripod, and a posture when the pan and tilt camera 10 is installed on the vertical plane corresponds to, for example, a posture when the pan and tilt camera is installed on a wall. Posture information indicating the installation posture detected by the posture sensor 45 can be transmitted to the smartphone 100 via the wireless communication unit 50.
The camera work control unit 46 outputs a command signal for controlling the lens control unit 43 and the pan and tilt control unit 44 in order to realize imaging of a desired still image or moving image.
The angle detection unit 48 is a unit that detects each of the pan angle and the tilt angle of the pan and tilt mechanism 32, and includes a first counter that up and down counts the number of pulse signals for driving the pan motor of the pan driving unit 34, and a second counter that up and down counts the number of pulse signals for driving the tilt motor of the tilt driving unit 36. In the first counter and the second counter, the count value is reset to zero or a defined value is preset by a detection signal of a home position sensor that detects a reference position in the pan direction provided in the pan and tilt mechanism 32 and a detection signal of a home position sensor that detects a reference position in the tilt direction. Accordingly, the count value of the first counter and the count value of the second counter after the count value is reset or preset correspond to the pan angle (pan position) and the tilt angle (tilt position) of the pan and tilt mechanism 32, respectively.
The wireless communication unit 50 (an information output unit or a live view image output unit) is a unit that performs wireless communication with an external portable terminal such as the smartphone 100 illustrated in
The operation unit 60 includes, for example, an imaging start button 19 provided in the camera body 12 (
The display unit 61 functions as an image display unit that displays a live view image, a played image, or the like, and functions as a user interface (UI unit) for displaying a menu screen and setting and inputting various parameters in cooperation with an operation unit 60.
The memory 62 includes a recording unit such as a synchronous dynamic random access memory (SDRAM) including a storage area that temporarily stores a captured still image or moving image, and a work area in which various calculation processes are performed, a read only memory (ROM) in which, for example, a program for imaging and various data necessary for control are stored, or a flash ROM that records the captured still image or moving image.
<Pan Tilt Operation Device>
Next, an embodiment of the pan and tilt operation device according to the present invention will be described.
In this example, a smartphone 100 in which pan and tilt operation program for causing a general-purpose smartphone to have various functions to be described below is installed functions as a pan and tilt operation device according to the present invention, as illustrated in
As illustrated in
The wireless communication unit 110 performs wireless communication with the base station device BS accommodated in the mobile communication network NW according to an instruction of the main control unit 101. Using this wireless communication, the wireless communication unit 110 performs transmission and reception of various types of file data such as audio data and image data, e-mail data, or the like, or reception of web data, streaming data, or the like. In this example, the wireless communication unit 110 of the smartphone 100 constitutes a wireless LAN between a station (for example, a pan and tilt camera) and an access point (for example, a router), and transmits an instruction input for various operations to the pan and tilt camera 10 or receives a live view image, an image for recording, or the like from the pan and tilt camera 10.
The touch panel 120 is a touch panel that displays an image (a still image and a moving image), text information, or the like to visually deliver information to a user and detects a user operation for the displayed information under the main control unit 101. The touch panel 120 includes a display panel 121 and an operation panel 122. In a case where a 3D image is viewed, it is preferable for the display panel 121 to be a 3D display panel.
The display panel 121 uses a liquid crystal display (LCD), an organic electro-luminescence display (OELD), or the like as a display device.
The operation panel 122 is a device that is placed so that an image displayed on a display surface of the display panel 121 can be viewed, and detects one or a plurality of coordinates operated by a finger of a user or a stylus. If this device is operated by the finger of the user or the stylus, the operation panel 122 outputs a detection signal generated by the operation to the main control unit 101. Then, the main control unit 101 detects an operation position (coordinates) on the display panel 121 on the basis of the received detection signal.
The display panel 121 and the operation panel 122 of the smartphone 100 are formed integrally to constitute the touch panel 120 as illustrated in
A size of the display area and a size of the display panel 121 fully match, but it is not always necessary for both to match. Further, the operation panel 122 may include two sensitive areas including an outer edge portion and an inner portion other than the outer edge portion. Further, a width of the outer edge portion is appropriately designed according to, for example, a size of a housing 102. Further, a position detection scheme adopted in the operation panel 122 may include a matrix switch scheme, a resistance film type, a surface acoustic wave scheme, an infrared scheme, an electromagnetic induction scheme, an electrostatic capacitance scheme, and the like, and any of the schemes may be adopted.
The calling unit 130 includes a speaker 131 or a microphone 132. The calling unit 130 converts voice of the user input via the microphone 132 into audio data which can be processed by the main control unit 101 and outputs the audio data to the main control unit 101, or decodes the audio data received by the wireless communication unit 110 or the external input and output unit 160 and outputs the audio data from the speaker 131. Further, as illustrated in
The operation unit 140 is a hardware key using a key switch or the like, and receives an instruction from the user. For example, the operation unit 140 is a push button switch that is mounted on a lower surface in a lower portion of a display portion of the housing 102 of the smartphone 100, and is turned ON when pressed by a finger or the like and turned OFF due to a restoring force of a spring or the like when the finger is released.
The storage unit 150 stores a control program or control data of the main control unit 101, address data associated with, for example, a name or a telephone number of a communication partner, transmitted and received e-mail data, web data downloaded by web browsing, or downloaded content data, and temporarily stores streaming data or the like. The storage unit 150 includes an internal storage unit 151 built in the smartphone and an external storage unit 152 having a slot for a detachable external memory. Each of the internal storage unit 151 and the external storage unit 152 constituting the storage unit 150 is realized using a storage medium, such as a flash memory type, hard disk type, multimedia card micro type, or card type of memory (for example, a Micro SD (registered trademark) memory), a random access memory (RAM), or a read only memory (ROM).
The external input and output unit 160 serves as an interface with all of external devices connected to the smartphone 100, and is directly or indirectly connected with other external devices through, for example, communication (for example, Universal Serial Bus (USB) or IEEE1394) or a network (for example, the Internet, wireless LAN, Bluetooth (registered trademark), radio frequency identification (RFID), infrared communication (infrared Data Association: IrDA; registered trademark), UWB (Ultra Wideband; registered trademark), or ZigBee (registered trademark)).
Examples of the external device connected to the smartphone 100 include a wired/wireless headset, a wired/wireless external charger, a wired/wireless data port, a memory card or a subscriber identity module card (SIM)/user identity module card (UIM) connected via a card socket, an external audio and video device connected via an audio and video input/output (I/O) terminal, a wirelessly connected external audio and video device, a wiredly/wirelessly connected smartphone, a wiredly/wirelessly connected personal computer, a wiredly/wireless connected PDA, and a wiredly/wireless connected earphone. The external input and output unit can transfer data received from such an external device to each component inside the smartphone 100, or send internal data of the smartphone 100 to the external device.
The GPS reception unit 170 receives GPS signals that are transmitted from GPS satellites ST1 to STn, executes a positioning calculation process based on a plurality of received GPS signals, and detects a position including a latitude, a longitude, and an altitude of the smartphone 100 according to an instruction of the main control unit 101. When the GPS reception unit 170 can acquire position information from the wireless communication unit 110 or the external input and output unit 160 (for example, wireless LAN), the GPS reception unit 170 can also detect the position using the position information.
The motion sensor unit 180 includes, for example, a triaxial acceleration sensor, and detects a physical motion of the smartphone 100 according to an instruction of the main control unit 101. By detecting a physical movement of the smartphone 100, a movement direction or an acceleration of the smartphone 100 is detected. A result of the detection is output to the main control unit 101.
The power supply unit 190 supplies power accumulated in a battery (not illustrated) to each unit of the smartphone 100 according to an instruction of the main control unit 101.
The main control unit 101 includes a microprocessor. The main control unit 101 operates according to the control program or the control data stored in the storage unit 150 and generally controls each unit of the smartphone 100. Further, the main control unit 101 has a mobile communication control function of controlling each unit of a communication system, and an application processing function in order to perform audio communication or data communication through the wireless communication unit 110.
The application processing function is realized by the main control unit 101 operating according to application software stored in the storage unit 150. Examples of the application processing function include an infrared communication function of controlling the external input and output unit 160 and performing data communication with a facing device, an e-mail function of performing transmission and reception of an e-mail, and a web browsing function of viewing web pages.
Further, the main control unit 101 has an image processing function of, for example, displaying a video on the touch panel 120 on the basis of image data (data of a still image or a moving image) such as received data or downloaded streaming data. The image processing function refers to a function of the main control unit 101 decoding the image data, performing image processing on a result of the decoding, and displaying the image on the touch panel 120.
Further, the main control unit 101 executes display control for the display panel 121, and an operation detection control for detecting a user operation through the operation unit 140 or the operation panel 122.
Through the execution of the display control, the main control unit 101 displays an icon for starting up the application software or a software key such as a scroll bar, or displays a window for creating an e-mail. The scroll bar refers to a software key for receiving an instruction for moving an image display portion for a large image that cannot be fitted in the display area of the display panel 121.
Further, through the execution of the operation detection control, the main control unit 101 detects a user operation through the operation unit 140, or receives an operation for the icon or an input of a character string to an input field of the window or receives a request for scroll of a display image using the scroll bar via the operation panel 122.
Further, the main control unit 101 has a touch panel control function of determining whether an operation position for the operation panel 122 is the overlapping portion (display area) that overlaps the display panel 121 or the outer edge portion (non-display area) that does not overlap the display panel 121, other than the overlapping portion, and controlling the sensitive area of the operation panel 122 or a display position of the software key, through the execution of the operation detection control.
Further, the main control unit 101 can also detect a gesture operation for the operation panel 122, and execute a preset function according to the detected gesture operation. The gesture operation refers to an operation of drawing a trajectory with a finger or the like, designating a plurality of positions simultaneously, or combining these and drawing a trajectory for at least one of a plurality of positions, rather than a conventional simple touch operation.
The camera unit 141 is a digital camera that performs electronic imaging using an imaging element such as a complementary metal oxide semiconductor (CMOS) or a charge-coupled device (CCD).
Further, the camera unit 141 can convert the image data obtained through imaging into, for example, compressed image data such as joint photographic coding experts group (JPEG), and record the compressed image data in the storage unit 150 or output the compressed image data via the external input and output unit 160 or the wireless communication unit 110 under the control of the main control unit 101.
In the smartphone 100 illustrated in
Further, the camera unit 141 can be used for various functions of the smartphone 100. For example, the image acquired by the camera unit 141 can be displayed on the display panel 121, or the image of the camera unit 141 can be used as one operation input of the operation panel 122. Further, when the GPS reception unit 170 detects the position, the GPS reception unit 170 can also detect the position by referring to the image from the camera unit 141. Further, the optical axis direction of the camera unit 141 of the smartphone 100 can be determined or a current use environment can be determined by referring to the image from the camera unit 141 without using a triaxial acceleration sensor, or in combination with the triaxial acceleration sensor. Of course, the image from the camera unit 141 can also be used within the application software.
In this embodiment, by downloading application software for operating the pan and tilt camera 10 (a pan and tilt operation program according to the present invention) over a network or the like, storing the application software in the storage unit 150, and operating the main control unit 101 according to the downloaded application software using the application processing function of the smartphone 100, the general-purpose smartphone 100 functions as a pan and tilt operation device for operating the pan and tilt camera 10.
[Operation and Installation Posture of Pan and Tilt Camera 10]
As illustrated in
Further, as illustrated in
If the pan and tilt camera 10 is powered ON, the pan and tilt camera 10 moves to an initial pan position at which the pan angle θ is 0° and an initial tilt position at which the tilt angle ϕ is 0° and waits for.
In a case where the pan and tilt mechanism 32 is caused to perform a pan operation and a tilt operation in the forward rotation direction when the pan and tilt camera 10 takes posture A as illustrated in
In a case where the pan and tilt mechanism 32 is caused to perform the pan operation and the tilt operation in the forward rotation direction when the pan and tilt camera 10 takes posture B as illustrated in
In a case where the pan and tilt mechanism 32 is caused to perform the pan operation in the forward rotation direction when the pan and tilt camera 10 takes posture C as illustrated in
[Smartphone Display Screen]
The smartphone 100 includes the touch panel 120 including the display panel 121 and the operation panel 122 (see
As illustrated in
Further, the main control unit 101 (display control unit) displays a cross key 124 including icons indicating an up key 124U, a down key 124D, a left key 124L, and a right key 124R on the display panel 121, as illustrated in
The icons indicating the up key 124U and the down key 124D illustrated in
The icons indicating the up key 124U and the down key 124D illustrated in
That is, the main control unit 101 displays the first icon indicating the tilt direction using an arrow as the icons indicating the left key 124L and the right key 124R in a case where the installation postures of the pan and tilt camera 10 are postures A and B, and displays the second icon indicating the rotation direction (rolling) using an arrow as the icons indicating the left key 124L and the right key 124R in a case where the installation posture of the pan and tilt camera 10 is posture C.
Further, the main control unit 101 functions as a key operation detection unit that detects which of the up key 124U, the down key 124D, the left key 124L, and the right key 124R of the cross key 124 is touched on the basis of touched coordinates on the display panel 121 which is detected by the operation panel 122. The main control unit 101 recognizes respective display areas of the icons corresponding to the up key 124U, the down key 124D, the left key 124L, and the right key 124R of the cross key 124 displayed on the display panel 121 and detects the touched key according to the display area to which coordinates detected by the operation panel 122 belong.
Further, the main control unit 101 functions as a motor control assignment setting unit and a motor driving command output unit shown below.
That is, the main control unit 101 functioning as the motor control assignment setting unit sets motor control assignment indicating a correspondence relationship between a touch operation of the up key 124U, the down key 124D, the left key 124L, or the right key 124R of the cross key 124, and assignment of the pan motor or the tilt motor driving the pan and tilt mechanism 32 and motor rotation direction (forward rotation direction or reverse rotation direction) on the basis of the posture information indicating the installation posture of the pan and tilt camera 10 received via the wireless communication unit 50 (posture information acquisition unit or angle acquisition unit) from the pan and tilt camera 10, and the pan angle and the tilt angle of the pan and tilt mechanism 32.
Further, if the up key 124U, the down key 124D, the left key 124L, or the right key 124R of the cross key 124 is touched, the main control unit 101 functioning as the motor driving command output unit selects the pan motor or the tilt motor of the pan and tilt mechanism 32 and outputs a motor driving command obtained by specifying a rotation direction of the selected motor according to the motor control assignment set for up, down, left and right keys of the cross key 124.
Although the display form of the icon indicating the respective up, down, left, and right keys of the cross key 124 is basically an icon (first icon) indicting a pan direction and a tilt direction using an arrow, the first icon indicating the left key and the right key is switched to a second icon indicating the rotation direction using an arrow under a specific condition (in this example, posture C) in which a video captured by the pan and tilt camera 10 rotates if the pan motor is driven (see
Accordingly, when the left key or the right key is operated under a specific condition under which a video rotates, it is possible not to cause discomfort even when the video rotates.
Further, when the installation postures of the pan and tilt camera 10 are postures A and B, each of the up key and the down key is assigned for control of the tilt motor, and each of the left key and the right key is assigned for control of the pan motor, but in posture A and posture B, motor control is assigned so that a motor rotation direction (CW direction and CCW direction) is reversed.
Thus, in a case where the respective up, down, left, and right keys of the cross key 124 are touched, a movement direction of an angle of view of the video to be captured can be set to the direction indicated by the respective up, down, left, and right keys regardless of whether the installation posture of the pan and tilt camera 10 being posture A or posture B, and no discomfort can be caused in a touch operation of the respective up, down, left, and right keys of the cross key 124.
FIGS. 10A1 to 10B3 are diagrams illustrating that a movement direction (including rotation) of the angle of view for imaging is changed according to a range of a tilt angle ϕ of the pan and tilt mechanism 32 when the installation posture of the pan and tilt camera 10 is posture A.
FIG. 10A1 illustrates a state in which the tilt angle ϕ of the pan and tilt mechanism 32 is 0°, and FIG. 10B1 is a diagram illustrating the movement directions a and b of the angle of view of the video to be captured in a case where the pan and tilt mechanism 32 is caused to perform a pan operation in a CCW direction (forward rotation direction) from the state illustrated in FIG. 10A1 and the pan and tilt mechanism 32 is caused to perform a tilt operation in a CW direction (forward rotation direction).
FIG. 10A2 illustrates a state in which the tilt angle ϕ of the pan and tilt mechanism 32 is 90°, and FIG. 10B2 is a diagram illustrating the movement directions a and b of the angle of view of the video to be captured in a case where the pan and tilt mechanism 32 is caused to perform a pan operation in a CCW direction from the state illustrated in FIG. 10A2 and the pan and tilt mechanism 32 is caused to perform a tilt operation in a CW direction.
FIG. 10A3 illustrates a state in which the tilt angle ϕ of the pan and tilt mechanism 32 is 180°, and FIG. 10B3 is a diagram illustrating the movement directions a and b of the angle of view of the video to be captured in a case where the pan and tilt mechanism 32 is caused to perform a pan operation in a CCW direction from the state illustrated in FIG. 10A3 and the pan and tilt mechanism 32 is caused to perform a tilt operation in a CW direction. As illustrated in
If the tilt angle ϕ of the pan and tilt mechanism 32 is different as illustrated in FIGS. 10A1, 10A2, and 10A3, in a case where the pan and tilt mechanism 32 is caused to perform the pan operation in the CCW direction and the pan and tilt mechanism 32 is caused to perform a tilt operation in the CW direction, the movement directions a and b of the angle of view of the video to be captured is different as illustrated in FIGS. 10B1, 10B2, and 10B3. In particular, if the tilt angle ϕ of the pan and tilt mechanism 32 is 90°, the angle of view of the video to be captured does not move in the pan direction and rotates (rolls) as illustrated in FIG. 10B2.
Accordingly, the main control unit 101 sets motor control assignment indicating a correspondence relationship between a touch operation of each of the respective up, down, left and right keys of the cross key 124, and assignment of the pan motor or the tilt motor for driving the pan and tilt mechanism 32 and the motor rotation direction according to the current tilt angle of the pan and tilt mechanism 32, in addition to the installation posture of the pan and tilt camera 10, selects the pan motor or the tilt motor of the pan and tilt mechanism 32 according to the set motor control assignment, and outputs a motor driving command obtained by specifying the rotation direction of the selected motor.
As illustrated in
Further, when the range of the tilt angle ϕ is from 60° to 120°, a display form of the icons indicated by the left key and the right key of the cross key 124 is switched from the first icon indicating the pan direction using an arrow to the second icon indicating the rotation direction using an arrow.
In this example, for the range of the tilt angle switched to the second icon (that is, a specific conditions under which the video captured by the pan and tilt camera 10 rotates when the pan motor is driven), the tilt angle ϕ is 60° to 120°, but the present invention is not limited thereto and it is preferable for a user to be able to set an arbitrary angle range with the tilt angle ϕ of 90° as a center angle.
Since the embodiment illustrated in
A pan angle display portion 126A1 (126A2) and a tilt angle display portion 126B for graphically displaying a current pan angle and a current tilt angle of the pan and tilt mechanism 32 are provided on the lower side and the left side of a display area 121A for displaying a live view image on the display panel 121 of the smartphone 100 as illustrated in
The pan angle display portion 126A1 (126A2) is a portion for graphically displaying the current pan angle. A range (−170° to 170°) of the pan angle θ is displayed using a length of a bar or a length of an arc, and the current pan angle is displayed by a marker on the bar or the arc M.
It can be confirmed that a video captured by the pan and tilt camera 10 rotates if the pan motor is driven by switching the display form of the range of the pan angle θ from the bar to the arc.
On the other hand, the tilt angle display portion 126B is a portion for graphically displays the current tilt angle. A range (−30° to 120°) of the tilt angle ϕ is displayed by a length of a bar, and the current tilt angle is displayed by a marker M on the bar.
In the tilt angle display portion 126B, a range (from 60° to 120°) of the tilt angle at which a video captured by the pan and tilt camera 10 rotates is displayed so that the range is identifiable from the other angle range if the pan motor is driven. Accordingly, it can be visually (intuitively) confirmed whether the current tilt angle satisfies specific conditions in which an video rotates due to driving of the pan motor.
Further,
If the pan and tilt mechanism 32 is tilt-driven in a state illustrated in
That is, it is necessary to set motor control assignment indicating a correspondence relationship between a touch operation of each of the respective up, down, left and right keys of the cross key 124, and assignment of the pan motor or the tilt motor for driving the pan and tilt mechanism 32 and the motor rotation direction according to the current pan angle, in addition to the installation posture of the pan and tilt camera 10 and the current tilt angle of the pan and tilt mechanism 32.
In the case of the state illustrated in
Further, in the state illustrated in
Therefore, in this case, it is preferable that the up and down keys of the cross key 124 are assigned for motor control of the pan motor and the left and right keys are assigned for motor control of the tilt motor.
[Pan and Tilt Operation Method]
Next, a pan and tilt operation method according to the present invention will be described with reference to a flowchart illustrated in
In
If the pan and tilt mechanism 32 is moved to the initial position, the posture information indicating the installation posture of the pan and tilt camera 10 is transmitted to the smartphone 100 (step S14). Subsequently, a moving image is captured by the imaging unit 20, a live view image processed by the signal processing unit 41 is transmitted to the smartphone 100, and angle information indicating the current pan angle and the current tilt angle detected by the angle detection unit 48 is transmitted to the smartphone 100 (step S16).
Meanwhile, the smartphone 100 receives the posture information transmitted from the pan and tilt camera 10 (step S20), and receives the live view image transmitted from the pan and tilt camera 10 and the current pan angle and the current tilt angle (step S22). The main control unit 101 of the smartphone 100 directly displays the live view image in the display area 121A of the display panel 121 on the basis of the posture information received in step S20 so that the received live view image is displayed to erect in the display area 121A of the display panel 121, or appropriately rotates and displays the live view image.
Subsequently, the main control unit 101 of the smartphone 100 sets motor control assignment indicating a correspondence relationship between a touch operation of each of the respective up, down, left and right keys of the cross key 124, and assignment of the pan motor or the tilt motor for driving the pan and tilt mechanism 32 and the motor rotation direction on the basis of the received posture information of the pan and tilt camera 10 and the current pan angle and the current tilt angle of the pan and tilt mechanism 32 (step S24).
Further, the main control unit 101 of the smartphone 100 determines the display form of the icons indicating the respective up, down, left, and right keys of the cross key 124 on the basis of the received posture information of the pan and tilt camera 10 and the current pan angle and the current tilt angle of the pan and tilt mechanism 32, and displays the icons in the determined display form (step S26). The display form for the icons indicating the respective up, down, left, and right keys of the cross key 124 is, for example, an icon (first icon) indicating the pan direction and the tilt direction using an arrow, or an icon (second icon) indicating the rotation direction using an arrow in a case where the video rotates if the pan operation is performed, as illustrated in
Thereafter, when the cross key 124 is touched (step S28), the main control unit 101 of the smartphone 100 transmits a motor driving command (a motor driving command in which the rotation direction is specified) for the motor assigned to the touched key to the pan and tilt camera 10 according to the motor control assignment set in step S24 (step S30).
If the pan and tilt camera 10 receives a command to drive the pan motor or the tilt motor from the smartphone 100, the pan and tilt camera 10 drives and controls the pan motor or the tilt motor on the basis of the received driving command, moves the pan and tilt mechanism 32, and detects the pan angle and the tilt angle of the pan and tilt mechanism 32 (step S18).
The pan and tilt camera 10 transitions to step S16 after performing the process of step S18, the smartphone 100 transitions to step S22 after performing the process in step S30, and the pan and tilt camera 10 and the smartphone 100 repeat the respective processes described above.
Accordingly, a photographer can perform pan and tilt operations of the pan and tilt mechanism 32 using the smartphone 100 so that the pan and tilt camera 10 can image a desired subject. In this case, it is possible not to cause discomfort between an instruction resulting from a touch operation of the cross key 124 and the video to be captured. In particular, in the case of a specific condition under which the video rotates, since the display of the icon of the corresponding key of the cross key 124 is switched to an icon indicating the rotation direction, it is possible not to cause discomfort even when the video rotates.
[Others]
In this embodiment, the posture information indicating the installation posture of the pan and tilt camera is acquired from the pan and tilt camera, the present invention is not limited thereto, and the posture information may be acquired on the basis of a manual operation using the touch panel 120 or the operation unit 140 (installation posture input unit) of the smartphone 100. For example, the posture information can be acquired through an operation of an icon or a button for selecting the installation posture of the pan and tilt camera. Further, when the pan and tilt camera moves to the initial position and waits for, it is possible to rotate the live view image so that the live view image erects while viewing the live view image displayed on the display panel and to acquire the posture information on the basis of the presence or absence of the rotation and the rotation angle at that case.
Further, the smartphone 100 may transmit an imaging command for a still image or a moving image through a switch operation of the operation unit 140 assigned to an icon button indicating an imaging button (not illustrated) or the imaging button or transmit a zoom instruction through a switch operation of the operation unit 140 assigned to an icon button indicating a zoom button (not illustrated) or a zoom button in a case where the imaging unit 20 is a zoom lens. Further, after a still image or a moving image has been captured in the pan and tilt camera 10, the captured still image or moving image may be received and the received still image or moving image may be stored in the external storage unit 152 or the like.
Further, the pan and tilt operation device according to the present invention is not limited to the smartphone in which the pan and tilt operation program is installed. A tablet terminal having a touch panel, or a personal digital assistant (PDA) may be applied as the pan and tilt operation device, or the pan and tilt operation device may be a dedicated pan and tilt operation device for operating the pan and tilt camera. Further, the pan and tilt operating unit and the pan and tilt camera are not limited to the wireless connection or may be connected using a cable.
Further, the icons indicating the respective up, down, left and right keys of the cross key are not limited to this embodiment and, for example, since the pan direction and the tilt direction can be visually recognized from an arrangement position of the respective up, down, left, and right keys, arrows, A, or the like may not be displayed. If at least the pan motor is driven and a specific condition under which the video to be captured rotates is discriminated, the corresponding icon among the icons indicating the up key, the down key, the left key, and the right key of the cross key may be switched to a icon indicating a rotation direction.
Further, in a case of a camera that cuts out and outputs a partial image from an image captured at a wide angle without using the pan and tilt mechanism, the cameral being capable of electronic panning and tilting by moving a cut-out position of the image according to a pan instruction or a tilt instruction, a video is not rotated with a pan operation. Thus, in the case of pan and tilt operation device that operates such a camera, the display form of an icon indicating the respective up, down, left, and right keys of the cross key for performing panning and tilting instructions is different from the display form illustrated in
Further, the present invention is not limited to the above-described embodiments, and it is understood that various modifications can be made without departing from the spirit of the present invention.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2014-181034 | Sep 2014 | JP | national |
This application is a Continuation of PCT International Application No. PCT/JP2015/067869 filed on Jun. 22, 2015, which claims priority under 35 U.S.C. § 119(a) to Japanese Patent Application No. 2014-181034 filed on Sep. 5, 2014. Each of the above applications is hereby expressly incorporated by reference, in their entirety, into the present application.
| Number | Name | Date | Kind |
|---|---|---|---|
| 8838276 | Desai | Sep 2014 | B1 |
| 20020191080 | Terada | Dec 2002 | A1 |
| 20070109417 | Hyttfors | May 2007 | A1 |
| 20090041378 | Yamaoka | Feb 2009 | A1 |
| 20110085016 | Kristiansen | Apr 2011 | A1 |
| 20120007999 | Horii | Jan 2012 | A1 |
| 20120057037 | Ono | Mar 2012 | A1 |
| 20120307052 | Thiruvengada | Dec 2012 | A1 |
| Number | Date | Country |
|---|---|---|
| 2000-209575 | Jul 2000 | JP |
| 2006-145761 | Jun 2006 | JP |
| 2012-23434 | Feb 2012 | JP |
| 2012-60215 | Mar 2012 | JP |
| Entry |
|---|
| International Search Report, issued in PCT/JP2015/067869, dated Aug. 11, 2015. |
| Written Opinion of the International Searching Authority, issued in PCT/JP2015/067869, dated Aug. 11, 2015. |
| Number | Date | Country | |
|---|---|---|---|
| 20170150031 A1 | May 2017 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/JP2015/067869 | Jun 2015 | US |
| Child | 15423832 | US |
