Patent Grant
9921054
This application claims the benefit under 35 U.S.C. § 119(a) of a Korean patent application filed on Jun. 4, 2013 in the Korean Intellectual Property Office and assigned Serial number 10-2013-0063774, the entire disclosure of which is hereby incorporated by reference.
The present disclosure relates to a camera shooting method for three dimensional modeling and an electronic device supporting the same.
In general, 3 Dimensional (3D) images in a virtual space are created through 3D modeling based on mathematical models.
To create a 3D image through 3D modeling, shape, appearance (e.g., color), and position information of a real object are first obtained.
Information for composing a 3D image may be obtained generally in two ways. First, multiple photographs of a physical object (target object) are taken by multiple optical instruments (such as cameras or electronic devices having a camera module) installed at fixed sites around the target object. Second, multiple photographs of a target object are taken in sequence by a single optical instrument being rotated around the target object, and information on the position, shape and appearance is obtained from the multiple photographs. Here, the information on the position between an optical instrument and a target object is related to the distance, height, and angle of the optical instrument or the target object.
Use of multiple optical instruments enables composition of a precise 3D image because information on the position between the optical instrument and the target object can be known in advance. In the case of using one optical instrument, a 3D image of the target object is composed by estimating information on the position between the optical instrument and the target object mainly using information on planes with a static or fixed background.
Use of multiple optical instruments to obtain position information of a target object may cause a problem in terms of space and costs.
In the case of using one optical instrument to obtain position information of a target object, it may be difficult to compute relative values in position information obtained by the optical instrument depending upon conditions in the background of the target object. When a complex background (e.g., a three dimensional background that is not flat) is used, the obtained 3D image tends to be distorted. To avoid such a problem, a large amount of position information is collected by taking many photographs. This may increase the amount of information to be sent to the processor and lengthen processing time.
The above information is presented as background information only to assist with an understanding of the present disclosure. No determination has been made, and no assertion is made, as to whether any of the above might be applicable as prior art with regard to the present disclosure.
Aspects of the present disclosure are to address at least the above-mentioned problems and/or disadvantages and to provide at least the advantages described below. Accordingly, an aspect of the present disclosure is to provide a shooting method for effective three dimensional modeling and an electronic device supporting the same.
In accordance with an aspect of the present disclosure, a shooting method for three-dimensional modeling is provided. In an electronic device having a sensor unit for sensing position information and a camera module for image capture, the shooting method includes performing sensor calibration, obtaining position information used for three dimensional modeling using the sensor unit, capturing multiple images of a target object used for three dimensional modeling according to the obtained position information, and storing the position information as metadata of each captured image.
In accordance with another aspect of the present disclosure, an electronic device supporting a shooting method for three-dimensional modeling is provided. The electronic device includes a sensor unit configured to sense a current position of the electronic device, a camera module to capture images used for three dimensional modeling, a control unit configured to control a process of performing sensor calibration, to obtain position information used for three dimensional modeling using the sensor unit, to capture multiple images of a target object used for three dimensional modeling according to the obtained position information, and to store the position information as metadata of each captured image, and a touchscreen configured to display the position information and captured images.
In a feature of the present disclosure, when an image for 3 Dimensional (3D) modeling is created, position information obtained using multiple sensors of the electronic device is included as metadata. Hence, it is possible to create an elaborate 3D image at low cost.
Other aspects, advantages, and salient features of the disclosure will become apparent to those skilled in the art from the following detailed description, which, taken in conjunction with the annexed drawings, discloses various embodiments of the present disclosure.
The above and other aspects, features, and advantages of certain embodiments of the present disclosure will be more apparent from the following description taken in conjunction with the accompanying drawings, in which:
The same reference numerals are used to represent the same elements throughout the drawings.
The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of various embodiments of the present disclosure as defined by the claims and their equivalents. It includes various specific details to assist in that understanding but these are to be regarded as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the various embodiments described herein can be made without departing from the scope and spirit of the present disclosure. In addition, descriptions of well-known functions and constructions may be omitted for clarity and conciseness.
The terms and words used in the following description and claims are not limited to the bibliographical meanings, but, are merely used by the inventor to enable a clear and consistent understanding of the present disclosure. Accordingly, it should be apparent to those skilled in the art that the following description of various embodiments of the present disclosure is provided for illustration purpose only and not for the purpose of limiting the present disclosure as defined by the appended claims and their equivalents.
It is to be understood that the singular forms “a,” “an,” and “the” include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to “a component surface” includes reference to one or more of such surfaces.
Referring to
The control unit 110 controls operations implementing a shooting method for 3 Dimensional (3D) modeling in the electronic device. For 3D modeling, the control unit 110 obtains position information of the electronic device and position information between the electronic device and target object through the sensor unit 140 and outputs the obtained information on the display unit 121. The control unit 110 may obtain image information for 3D modeling from the camera module 130 and display the image information on the display unit 121. The control unit 110 may store position information obtained from the sensor unit 140 and image information obtained from the camera module 130 in the storage unit 160. The control unit 110 may send position information obtained from the sensor unit 140 and image information obtained from the camera module 130 to an external server through the communication unit 150. The external server may generate 3D image data using image information and position information received from the electronic device through 3D modeling and then send the 3D image data to the electronic device. The control unit 110 may receive 3D image data from the external server through the communication unit 150 and output the 3D image data on the display unit 121.
The control unit 110 may separately store position information obtained from the sensor unit 140 and image information obtained from the camera module 130 in the storage unit 160. The control unit 110 may also store the position information together with the image information as an image file in the storage unit 160 so that the position information is included as metadata of the image information.
The touchscreen 120 is an input and output means for input and display and may include a display unit 121 and a touch sensor 122. The touchscreen 120 may display various screens used for operation of the electronic device (such as a content playback screen, call handling screen, messenger screen, gaming screen, and gallery screen) through the display unit 121. In particular, for 3D modeling, the touchscreen 120 may display position information obtained from the sensor unit 140 and image information obtained from the camera module 130 on the display unit 121. When a user touch event is detected by the touch sensor 122 during screen display on the display unit 121, the touchscreen 120 may send an input signal corresponding to the touch event to the control unit 110, which may then recognize a touch event and perform a control operation according to the touch event.
The display unit 121 may output or display information processed by the electronic device. For example, when the electronic device is in a call processing mode, the display unit 121 may display a User Interface (UI) or Graphical User Interface (GUI) related to a call. When the electronic device is in a shooting mode for 3D modeling, the display unit 121 may display a UI or GUI related to image information and position information. The display unit 121 may operate in a landscape mode or portrait mode and may switch between the landscape mode and portrait mode according to rotation or placement of the electronic device.
The display unit 121 may be realized using one or more of display techniques based on Liquid Crystal Display (LCD), Thin Film transistor LCD (TFT-LCD), Light Emitting diodes (LED), Organic LED (OLED), Active Matrix OLEDs (AMOLED), flexible display, bendable display, 3D display, and any similar and/or suitable display technique. The display unit 130 may also use a transparent display technology so as to be seen from the outside.
The touch sensor 122 may be placed on the display unit 121 and may sense a touch event generated by the user on the surface of the touchscreen 120 (for example, a long press touch gesture, short press touch gesture, single-touch gesture, multi-touch gesture, and touch and move gesture such as drag). Upon sensing of a touch event on the touchscreen 120, the touch sensor 122 may identify coordinates of the touch point and send the coordinate information to the control unit 110. The control unit 110 may perform a function corresponding to the touch point in response to a signal from the touch sensor 122. The touch sensor 122 may be configured to convert a pressure change or capacity change at a portion of the display unit 121 into an electrical signal. The touch sensor 122 may be configured to sense not only a touch point and touch area but also a touch pressure according to applied touch techniques. When a touch input is sensed by the touch sensor 122, a corresponding signal may be sent to a touch controller (not shown). The touch controller may process the signal and send the processing result to the control unit 110. Hence, the control unit 110 may identify the touched portion on the touchscreen 120.
The camera module 130 is configured to capture an image of a target object, and may include a lens and a Charge Coupled Device (CCD) or Complementary Metal Oxide Semiconductor (CMOS) image sensor. In particular, the camera module 130 may capture an image of a target object and generate corresponding image information. In response to a user selection signal, the control unit 110 may store the image information and associated metadata as an image file in the storage unit 160.
The sensor unit 140 may include a gyro sensor for sensing movement direction and acceleration of the electronic device, a geomagnetic sensor for sensing the azimuth using earth's magnetic fields to identify the orientation of the electronic device, an acceleration sensor for sensing spatial movement of the electronic device, a proximity sensor for measuring the distance between the electronic device and an object, and any similar and/or suitable sensor. The sensor unit 140 may obtain position information of a target object for 3D modeling by use of at least one of the gyro sensor, geomagnetic sensor, acceleration sensor, and proximity sensor. The obtained position information may be used as metadata of image information under control of the control unit 110.
The communication unit 150 may establish a communication channel for a voice call, video call and data call under control of the control unit 110. That is, the communication unit 150 may establish a voice call channel, video call channel or a data communication channel with a service operator network. To this end, the communication unit 150 may include a radio frequency transmitter for upconverting the frequency of a signal to be transmitted and amplifying the signal, and a radio frequency receiver for low-noise amplifying a received signal and downconverting the frequency of the received signal. Here, the data communication channel may be used for a video call service, an instant messaging service, a chat service, and a data transmission and download service. In particular, the communication unit 150 may send 3D modeling information to an external server and receive a 3D image as a result of 3D modeling from the external server under control of the control unit 110.
The storage unit 160 may store programs used to perform overall operation and specific functions of the electronic device, and data generated during program execution. For example, the storage unit 160 may store an Operating System (OS) for booting the electronic device, application programs realizing specific functions of the electronic device, and data generated in the course of operating the electronic device. The storage unit 160 may include a Read Only Memory (ROM), a Random Access Memory (RAM), and the like. In particular, the storage unit 160 may store image information of a target object obtained by the camera module 130 and position information of the target object obtained by the sensor unit 140 under control of the control unit 110.
The audio processing unit 170 may output various audio data generated during operation of the electronic device, audio data generated by playback of an audio file stored in the storage unit 160, and audio data received from an external entity. The audio processing unit 170 may also support collection of audio data. To this end, the audio processing unit 170 may include a speaker SPK and a microphone MIC. In particular, the audio processing unit 170 may output sound effects or sound notifications configured for acquisition of position information for 3D modeling. Output of a sound effect or sound notification may be skipped according to user or designer settings.
Referring to
The sensor unit 140 may measure the height of the electronic device as part of position information first. For example, when the user raises the electronic device from a reference plane to a desired height, the acceleration sensor may determine the height of the electronic device from the reference plane on the basis of the sensed acceleration and the time taken.
Alternatively, when the electronic device is fixed at a desired height, the proximity sensor may measure the distance between the electronic device and the ground.
At operation 203, the electronic device obtains position information for 3D modeling using the sensor unit 140. Here, the position information may include information regarding the height of the electronic device, the distance between the electronic device and target object, the tilt of the electronic device, and the angle between the electronic device and target object relative to the start position of image capture. The electronic device may sense the tilt thereof using the gyro sensor, and may determine the distance between the electronic device and target object on the basis of the tilt and height thereof.
Image information and position information of the target object is used for 3D modeling. The number of images of the target object used for 3D modeling may be determined in advance by an external server or the user. Here, the angle between the electronic device and target object relative to the start position of image capture may be determined according to the number of target object images used for 3D modeling. For example, when front, rear, left and right images of a target object are used for 3D modeling, as the number of target object images used for 3D modeling is four, the electronic device is operated to measure the angle at a step of 90 degrees.
As the sensor unit 140 may sense the angle between the electronic device and target object, a minimum number of target object images may be captured for 3D modeling without duplicate or unnecessary images.
At operation 205, the electronic device generates an image using the camera module 130 according to the position information obtained by the sensor unit 140. Here, the electronic device outputs position information obtained through the sensor unit 140 on the display unit 121 in real time. The electronic device may identify position information used for 3D modeling using information obtained in real-time through the sensor unit 140 and generate image information accordingly. Position information obtained when the image information is generated may be stored separately or as metadata of the image information in the storage unit 160. The electronic device may output position information obtained through the sensor unit 140 to the display unit 121 in real time, and may also output the same as a sound effect or notification through the speaker SPK of the audio processing unit 170 in real time.
When position information obtained through the sensor unit 140 is output in real time on the display unit 121, the display unit 121 may display a UI or GUI related to image information and position information.
At operation 207, the electronic device stores the position information obtained when the image information is generated as metadata of the image information. The electronic device may also separately store the image information and the position information obtained when the image information is generated. Here, metadata is structured data related to an image. For example, when a photograph is taken, a digital camera may store information regarding the camera itself, the date and time photographed, exposure, flash utilization, resolution, photo size, and other similar and/or suitable information together with the image data. In one embodiment, metadata may be stored in the Exchangeable image file format (Exif).
Referring to
First, the electronic device 310 obtains information on the height 330 set by the user through sensor calibration.
The tilt 340 of the electronic device 310 may be sensed through the gyro sensor, and the distance 350 between the electronic device 310 and target object 320 may be determined on the basis of the tilt 340 and height 330 of the electronic device 310.
For example, when the user leans the electronic device 310 toward the target object 320, the electronic device 310 may compute the distance 350 between the electronic device 310 and target object 320 on the basis of the height 330 thereof (already measured) using Equation 1.
height*tan(tilt)=distance Equation 1
Referring to
Referring to
Referring to
To this end, the electronic device 511 may sense the angle between the electronic device 511 and target object 510 relative to the start position of image capture and notify the user of the sensed angle.
Here, the angle between the electronic device and target object may be determined according to the number of target object images used for 3D modeling. For example, when front, rear, left and right images of a target object are used for 3D modeling, as the number of target object images used for 3D modeling is four, the electronic device may be operated to sense the angle at a step of 90 degrees from the start position.
As another example, when the target object 510 is a person, front and rear images may suffice. In this case, as the number of target object images used for 3D modeling is two, the electronic device 511 may be operated to sense the angle at a step of 180 degrees from the start position.
In
The electronic device 511 does not have to start image generation for 3D modeling from the front (a) of the target object 510. For example, the electronic device 511 may sense the angle between the electronic device 511 and the target object 510 at a step of 90 degrees from the start position of image capture, or may sense angles of 90, 180 and 270 degrees regardless of the start position of image capture and notify the user of the sensed angle.
When a target object image not matching a preset angle between the electronic device 511 and the target object 510 is generated, the electronic device 511 may delete the generated target object image or stop image generation for 3D modeling.
Referring to
In
The electronic device 513 does not have to start image generation for 3D modeling from the front (a) of the target object 512. For example, the electronic device 513 may sense the angle between the electronic device 513 and the target object 512 at a step of 90 degrees from the start position of image capture, or may sense angles of 90, 180 and 270 degrees regardless of the start position of image capture and notify the user of the sensed angle.
When a target object image not matching a preset angle between the electronic device 513 and the target object 512 is generated, the electronic device 513 may delete the generated target object image or stop image generation for 3D modeling.
Referring to
As shown in
The x-axis line segment of the guide region 611 indicates the height of the electronic device 610, the y-axis line segment thereof indicates the angle between the electronic device 610 and target object 601 relative to the start position of image capture, and the area thereof indicates the distance between the electronic device 610 and target object 601. The tilt of the electronic device 610 is represented by the color of line segments constituting the action region 612. Here, display of position information related to the angle and distance may be omitted at the start position of image generation for 3D modeling. The electronic device 610 may display position information as character strings in a designated region 613 of the touchscreen 120.
For example, referring to
When the angle between the electronic device 610 and the target object 601 is small, the y-axis line segment of the action region 612 may be displayed on the left side of the y-axis line segment of the guide region 611.
When the distance between the electronic device 610 and target object 601 is less than that at the start position of image generation for 3D modeling, the action region 612 may be displayed within the guide region 611 (the area of the action region 612 is smaller than that of the guide region 611).
When the tilt of the electronic device 610 is larger than that at the start position of image generation for 3D modeling, the line segments constituting the action region 612 may be displayed in a red color. In one embodiment, information on the tilt of the electronic device 610 may be output as a sound effect or notification through the speaker SPK of the audio processing unit 170.
In
When the angle between the electronic device 610 and the target object 601 is small, the y-axis line segment of the action region 612 may be displayed on the left side of the y-axis line segment of the guide region 611.
When the distance between the electronic device 610 and target object 601 is less than that at the start position of image generation for 3D modeling, the action region 612 may be displayed within the guide region 611.
The user may identify optimal position information by placing the electronic device 610 so that the action region 612 is matched with the guide region 611.
Referring to
The x-axis line segment of the guide region 621 indicates the height of the electronic device 620, the y-axis line segment thereof indicates the angle between the electronic device 620 and target object 603 relative to the start position of image capture, and the area thereof indicates the distance between the electronic device 620 and target object 603. The tilt of the electronic device 620 is represented by the color of line segments constituting the action region 622. Here, display of position information related to the angle and distance may be omitted at the start position of image generation for 3D modeling. The electronic device 620 may display position information as character strings in a designated region 623 of the touchscreen 120.
For example, in
When the angle between the electronic device 620 and the target object 603 is large, the y-axis line segment of the action region 622 may be displayed on the right side of the y-axis line segment of the guide region 621.
When the distance between the electronic device 620 and target object 603 is greater than that at the start position of image generation for 3D modeling, the action region 622 may be displayed outside the guide region 621 (the area of the action region 622 is larger than that of the guide region 621). In other words, when the distance between the electronic device 620 and target object 603 is greater than that at the start position of image generation, the guide region 621 may be displayed within the action region 622.
When the tilt of the electronic device 620 is the same as that at the start position of image generation for 3D modeling, the line segments constituting the action region 622 may be displayed in a green color.
The user may identify optimal position information by placing the electronic device 620 so that the action region 622 is matched with the guide region 621.
Referring to
The x-axis line segment of the guide region 631 indicates the height of the electronic device 630, the y-axis line segment thereof indicates the angle between the electronic device 630 and target object 605 relative to the start position of image capture, and the area thereof indicates the distance between the electronic device 630 and target object 605. The tilt of the electronic device 630 is represented by the color of line segments constituting the action region 632. Here, display of position information related to the angle and distance may be omitted at the start position of image generation for 3D modeling. The electronic device 630 may display position information as character strings in a designated region 633 of the touchscreen 120.
For example, in
When the angle between the electronic device 630 and the target object 605 matches a preset value, the y-axis line segment of the action region 632 may be displayed so as to coincide with the y-axis line segment of the guide region 631.
When the distance between the electronic device 630 and target object 605 is the same as that at the start position of image generation for 3D modeling, the action region 632 may be displayed so as to coincide with the guide region 631 (the area of the action region 632 is the same as that of the guide region 631).
When the tilt of the electronic device 630 is larger than that at the start position of image generation for 3D modeling, the line segments constituting the action region 632 may be displayed in a red color. The user may identify optimal position information by placing the electronic device 630 so that the action region 632 matches the guide region 631.
Referring to
The x-axis line segment of the guide region 641 indicates the height of the electronic device 640, the y-axis line segment thereof indicates the angle between the electronic device 640 and target object 607 relative to the start position of image capture, and the area thereof indicates the distance between the electronic device 640 and target object 607. The tilt of the electronic device 640 is represented by the color of line segments constituting the action region 642. Here, display of position information related to the angle and distance may be omitted at the start position of image generation for 3D modeling. The electronic device 640 may display position information as character strings in a designated region 643 of the touchscreen 120.
For example, in
When the angle between the electronic device 640 and the target object 607 matches a preset value, the y-axis line segment of the action region 642 may be displayed so as to coincide with the y-axis line segment of the guide region 641.
When the distance between the electronic device 640 and target object 607 is the same as that at the start position of image generation for 3D modeling, the action region 642 may be displayed so as to coincide with the guide region 641 (the area of the action region 642 is the same as that of the guide region 641).
When the tilt of the electronic device 640 is the same as that at the start position of image generation for 3D modeling, the line segments constituting the action region 642 may be displayed in a green color.
The user may obtain image information 608 of the target object 607 using the camera module 130 when the action region 642 matches the guide region 641.
When an image of the target object is captured by the camera module 130, the control unit 110 may store the target object image together with the position information, in such a way that the position information may be used as metadata of the target object image, as an image file in the storage unit 160, and may send the image file to a specific server through the communication unit 150. Alternatively, the control unit 110 may separately store the position information and the target object image in the storage unit 160, and may separately send the same to a specific server through the communication unit 150.
Referring to
As described above, embodiments of the present disclosure may be provided in a form of program commands executable through various computer means and recorded in a non-transitory recording media readable by a computer. Here, the non-transitory recording media readable by a computer may include a program command, data file, data structure, and any combination thereof. In the meantime, the program command recorded in the non-transitory recording media may be one specially designed for the embodiments of the present disclosure or one disclosed in the art of computer software. The non-transitory recording media readable by a computer may include hardware devices specially designed to store and execute programs such as magnetic media (hard disk, floppy disk, and magnetic tape), optical media (Compact Disc-Read Only Memory (CD-ROM) and Digital Versatile Disc (DVD)), magneto-optical media (floptical disk), Read Only Memory (ROM), Random Access Memory (RAM), and flash memory. Further, the program command may include a high level language code executable by a computer having an interpreter as well as a machine language code provided by a complier. The hardware device may be configured with at least one software module to perform operations according to various embodiments of the present disclosure.
While the present disclosure has been shown and described with reference to various embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure as defined in the appended claims and their equivalents.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2013-0063774 | Jun 2013 | KR | national |
| Number | Name | Date | Kind |
|---|---|---|---|
| 4794417 | Sekiguchi | Dec 1988 | A |
| 6253160 | Hanseder | Jun 2001 | B1 |
| 20030202089 | Alhadef | Oct 2003 | A1 |
| 20040189813 | Tanaka | Sep 2004 | A1 |
| 20060119864 | Lindner | Jun 2006 | A1 |
| 20110304750 | Lee | Dec 2011 | A1 |
| 20120121135 | Kotake | May 2012 | A1 |
| 20120162220 | Sakurai | Jun 2012 | A1 |
| 20130258117 | Penov | Oct 2013 | A1 |
| 20130329014 | Obata | Dec 2013 | A1 |
| 20140063220 | Taylor | Mar 2014 | A1 |
| Number | Date | Country |
|---|---|---|
| 2000253393 | Sep 2000 | JP |
| Number | Date | Country | |
|---|---|---|---|
| 20140354784 A1 | Dec 2014 | US |
