Patent Application
20210250500
This patent application is based on and claims priority under 35 U.S.C. § 119(a) to Japanese Patent Application No. 2020-019934, filed on Feb. 7, 2020 in the Japan Patent Office, the entire disclosure of which is incorporated herein by reference.
The present disclosure relates to an information processing method, a non-transitory computer-readable medium, and an information processing apparatus.
Still images and videos are used to introduce and promote goods or facilities in industries such as tourism, real estate, used cars and weddings. Further, in recent years, in addition to still images taken by a general-purpose camera, wide-angle images such as spherical images are used that enhance the feeling of actually being there. Furthermore, contents created by combining the wide-angle images and the still images with text information are used for the promotion or introduction of goods or services as described above. As a technique of capturing the wide-angle images for creating such contents, a technique of using an application installed in an information terminal such as a smartphone in combination with a celestial-sphere camera is known.
For example, a technique regarding generating contents using wide-angle images is known. More specifically, a technique is known of performing an operation of setting hot spots for switching scenes at desired locations in a wide-angle image, while a user is shooting photographs, in order to generate contents more efficiently.
According to one or more embodiments, an information processing method is performed by an information processing apparatus including a communication interface and an imaging device. The method includes acquiring a wide-angle image from an external imaging device via the communication interface; invoking an imaging function implemented by the imaging device; acquiring a captured image imaged by the imaging device; and associating the acquired wide-angle image with the acquired captured image.
Amore complete appreciation of the disclosure and many of the attendant advantages and features thereof can be readily obtained and understood from the following detailed description with reference to the accompanying drawings, wherein:
The accompanying drawings are intended to depict embodiments of the present disclosure and should not be interpreted to limit the scope thereof. The accompanying drawings are not to be considered as drawn to scale unless explicitly noted.
In describing embodiments illustrated in the drawings, specific terminology is employed for the sake of clarity. However, the disclosure of this patent specification is not intended to be limited to the specific terminology so selected, and it is to be understood that each specific element includes all technical equivalents that have the same function, operate in a similar manner, and achieve a similar result. Referring now to the drawings, wherein like reference numerals designate identical or corresponding parts throughout the several views thereof, embodiments of this disclosure are described. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. In embodiments, a description is given of an example in which an information processing apparatus is an information terminal that communicates with a spherical imaging device. Further, in embodiments, a description is given of an example in which a program executable by a computer implementing the information processing apparatus is an application program to be installed in the information terminal.
Any suitable information terminal is used as the information terminal 50, and a mobile information terminal such as a smartphone, a tablet computer, or a laptop computer is preferable. In the information terminal 50, an application program is installed that controls the spherical imaging device 10 to capture a spherical image and creates content using the captured spherical image. The content created as above is hereinafter referred to as a “spherical image content”. The information terminal 50 constitutes an information processing apparatus or a computer according to the present embodiment, which has a spherical image content creation function. Further, the information terminal 50 and the spherical imaging device 10 in combination constitute the spherical image content creation system 1.
In the embodiment illustrated in
In the embodiment illustrated in
In the embodiment illustrated in
In an example embodiment, the content publishing server 70 is configured as a web server or a web application server. In such embodiment, the content publishing server 70 receives a request for a content registration request including a spherical image content to be registered in accordance with HyperText Transfer Protocol (HTTP), stores the spherical image content in an appropriate storage area, and manages the content in a database, for example. The content publishing server 70 further receives a request for an image viewing request designating a spherical image content, reads the spherical image content as the request target, and transmits a response including the read content. The request source information terminal that receives the response makes the received spherical image content viewable on the display device using a web browser or a specific application. In the present embodiment, the content or the spherical image content is a collection of one or a plurality pieces of data as material. Examples of the material of this content or the spherical image content include, but are not limited to, a spherical image, a normal image, a spherical video, a normal video, text (characters), and sound. Although, in the embodiment, a description is given of an example in which a content is a spherical image content including at least a spherical image, in another example, instead of the spherical image, a content uses a wide-angle image other than the spherical image.
A description is now given of examples of configurations of the spherical imaging device 10 and the information terminal 50, according to the present embodiment, with reference to
The imaging body 12 illustrated in
The relative positions of the optical elements (lenses, prisms, filters, and aperture stops) of the two image forming optical systems 20A and 20B are defined with reference to the image sensors 22A and 22B. More specifically, positioning is made such that the optical axis of the optical elements of each of the image forming optical systems 20A and 20B is positioned at the central part of the light receiving area of corresponding one of the image sensors 22 orthogonally to the light receiving area, and such that the light receiving area serves as the imaging plane of corresponding one of the fisheye lenses.
In the embodiment illustrated in
As described below in detail, the images captured by the respective image sensors 22A and 22B are combined so as to generate an image over a solid angle of 4π steradian (hereinafter, such an image is referred to as a “spherical image”). The spherical image is obtained by capturing images of all directions that can be seen from an image capturing point. Although in the embodiments, a description is given of an example case in which a spherical image is preferably generated, in another example, a so-called panoramic image obtained by photographing 360 degrees only in a horizontal plane or an image that is a part of the image obtained by photographing omnidirectionally or 360 degrees in a horizontal plane is generated. The spherical image is either a still image or a moving image (movie).
The CPU 112 controls operation of each of hardware components and overall operation of the spherical imaging device 10. The ROM 114 stores a control program described in a code readable by the CPU 112 and various kinds of parameters. The image processing block 116 is connected to a first image sensor 130A and a second image sensor 130B (corresponding to the image sensors 22A and 22B in
The moving image compression block 118 is a codec block configured to compress and decompress a video in a moving picture experts group (MPEG)-4 advanced video coding (AVC)/H.264 format, for example. The moving image compression block 118 is used to generate video data of the generated spherical image. The still image compression block 119 is a codec block configured to compressing and decompress a still image in a joint photographic experts group (JPEG) or tagged image file format (TIFF) format, for example. The still image compression block 119 is used to generate still image data of the generated spherical image. The DRAM 132 provides a storage area for temporarily storing data when performing various types of signal processing and image processing. The acceleration sensor 136 detects acceleration components of three axes, which are used to detect the vertical direction to perform zenith correction on the spherical image. The acceleration sensor 136 is just one example. In another example, in addition to the acceleration sensor, at least one of functions of a triaxial angular velocity sensor and an electronic compass is provided.
The spherical imaging device 10 further includes an external storage interface 122, a USB interface 126, a serial block 128, and a video output interface 129. The external storage interface 122 is connected to an external storage 134. The external storage interface 122 controls reading and writing of data from and to the external storage 134 such as a memory card inserted in a memory card slot.
The USB interface 126 is connected to a USB connector 138. The USB interface 126 controls USB communication with an external device such as the information terminal 50 or a personal computer connected via the USB connector 138. The serial block 128 controls serial communication with an external device such as the information terminal 50 or a personal computer, and is connected to a wireless module 140. The video output interface 129 is an interface to connect the spherical imaging device 10 to an external display such as a high-definition multimedia interface (HDMI, registered trademark), through which captured images are output to the external display as a video.
When the power is turned on by an operation to a power switch, the control program mentioned above is loaded to a main memory. The CPU 112 controls operation of each hardware component of the spherical imaging device 10 according to the control program loaded into the main memory, while temporarily saving data required for the control in the memory. Thus, the spherical imaging device 10 implements functional units and operations described below.
The CPU 152 controls operation of each of hardware components and overall operation of the information terminal 50. The RAM 154 provides a work area for the CPU 152. The built-in storage 156 stores an operating system and a control program, such as an application program that executes processes in the information terminal 50 according to the present embodiment, each of the operating system and the control program being written in codes interpretable by the CPU 152.
The input device 158 is an input device such as a touch screen, and provides a user interface. The input device 158 constitutes input means in the present embodiment. The external storage 160 is a removable storage medium mounted, for example, in a memory card slot, and records various types of data, such as image data in a video format and still image data. The wireless module 164 establishes a connection for wireless LAN communication between the information terminal 50 and an external device such as the spherical imaging device 10. The USB connector 166 establishes a USB connection between the information terminal 50 and an external device.
Although, a description given hereinafter is of an example in which the wireless module 164 is used as communication means that allows the information terminal 50 to communicate with the spherical imaging device 10, limitation to any specific standard is not intended. In another example, the connection to an external device is established through other wireless communication such as Bluetooth (registered trademark) or wireless USB, the USB connector 166, or through wired communication such as a wired LAN. The wireless module 164, the other wireless communication, the USB connector 166, and the wired communication are each an example of a communication interface.
The display 162 displays an operation screen that accepts operation inputs from a user, displays a monitor image of an image captured by the spherical imaging device 10 that is ready to be captured or being captured, and displays a moving image or still image stored for playback or viewing. The display 162 and the input device 158 allows a user, through the operation screen, to make instructions for image capturing or changing various kinds of setting in the spherical imaging device 10. The display 162 constitutes built-in display means in the present embodiment. In another example, the information terminal 50 further includes a video interface such as HDMI (registered trademark). In this example, the information terminal 50 outputs a screen to external display means such as a display device or a projector device via the video interface.
The built-in camera 168 is a standard camera that the information terminal 50 includes, which captures a normal still image or a moving image, to generate still image data and moving image data. The built-in camera 168 constitutes imaging means in the present embodiment.
When power is supplied to the information terminal 50 and the power thereof is turned on, the programs are read from the ROM 114 or the built-in storage 156, and loaded to the RAM 154. The CPU 152 controls operation of each hardware element of the information terminal 50 according to the programs loaded into the RAM 154, while temporarily saving data required for the control in the memory. Thus, the information terminal 50 implements functional units and operations described below. The above-mentioned application program is a program that causes a computer to operate as each means in the present embodiment. The above-mentioned program is installed in a computer, and the computer executes the installed program, thereby implementing the information terminal 50 according to the present embodiment.
The content publishing server 70 has the similar or substantially the similar hardware configuration as that of the information terminal 50 illustrated in
A detailed description is now given of the spherical image content creation function, according to the present embodiment, with reference to
As illustrated in
The communication unit 202 controls the wireless module 164, which is an interface with an external device, to perform communication with the spherical imaging device 10. The built-in camera control unit 204 controls the built-in camera 168 as the imaging means.
The display control unit 206 controls the display 162 as the display means. The input unit 208 controls the input device 158 as the input means.
In a preferable example embodiment, the spherical image content creation unit 220 is provided as an application that integrates a function of controlling the spherical imaging device 10 to capture a spherical image and an imaging function of the built-in camera 168 of the information terminal 50, and collectively manages plural types of images (e.g., a spherical image and a normal image) captured with these functions.
The spherical image content creation unit 220 includes a spherical imaging device control unit 222, a built-in imaging function invoking unit 226, an image information management unit 230, and a user interface (UI) unit 234.
The spherical imaging device control unit 222 controls the spherical imaging device 10 as an extraneous source via the communication unit 202. The spherical imaging device control unit 222 issues commands to the spherical imaging device 10 to control the spherical imaging device 10 to capture a spherical image and transmit the captured spherical image to the information terminal 50. Further, the spherical imaging device control unit 222 acquires a spherical image from the spherical imaging device 10 via the communication unit 202. The spherical imaging device control unit 222 constitutes external device control means and first acquisition means in the present embodiment. The spherical image is either a still image or a moving image (movie). A description is given hereinafter basically is of an example in which the spherical image is a still image.
The built-in imaging function invoking unit 226 invokes an imaging function of the built-in camera 168 by using the built-in camera control unit 204. Further, the built-in imaging function invoking unit 226 acquires an image captured by the built-in camera 168 from the built-in camera control unit 204. The built-in imaging function invoking unit 226 constitutes invoking means and second acquisition means in the present embodiment.
There are three example implementation ways for directly or indirectly controlling the built-in camera 168 from the application. The first example way is to invoke a camera application itself for capturing an image using the built-in camera 168. The second example way is to invoke a camera function for capturing an image using the built-in camera 168. The third example way is to implement a function itself for controlling the built-in camera 168 in the spherical image content creation unit 220. From the viewpoint of user operability, the second and third example ways are preferable. However, both implementation ways have a common feature of controlling the built-in camera 168 directly or indirectly from the application.
Further, the configuration illustrated in
Although in the embodiment, a description is given of an example in which a normal still image (also referred to as a “still image”) is acquired as an image captured by the built-in camera 168, in another example the image captured by the built-in is a moving image. Further, in the embodiment, in a case in which a spherical image is in an image folder (camera roll), the built-in camera 168 can be used to capture a normal image.
The image information management unit 230 manages a spherical image acquired by the spherical imaging device control unit 222 in association with a normal image acquired from the built-in imaging function invoking unit 226. Thus, the image information management unit 230 stores a spherical image content in the content storage unit 240. The spherical image content includes one or more spherical images and one or more captured images. The image information management unit 230 constitutes associating means in the present embodiment. In another example, in addition to or in alterative to storing the spherical image content in the content storage unit 240, the image information management unit 230 transmits (so-called uploads) the created spherical image content to the content publishing server 70 as an extraneous source to register the transmitted content therein.
The image information management unit 230 illustrated in
In an example embodiment, the association position specifying unit 232 specifies a central area of a display area of the spherical image when displayed on the display 162 as the particular position in the spherical image. In another example embodiment, the association position specifying unit 232 specifies a designated area designated by an input to the input device 158 in the display area when the spherical image is displayed on the display 162 as the particular position in the spherical image.
In another example, in addition to associating the normal image with the particular position in the spherical image, the image information management unit 230 associates the normal image with a content including the spherical image, the content being a higher-level unit than the spherical image. Further, in addition to the normal image acquired by the built-in imaging function invoking unit 226, the image information management unit 230 can manage other images in association with the spherical image or the content. For example, the image information management unit 230 manages either one or both of images stored in the built-in storage 156 of the information terminal 50 or the external storage 160 and images acquired from extraneous sources such as images received from an image management system (e.g., the content publishing server 70 or another service) communicable with the information terminal 50, in association with the spherical image or the content. Furthermore, the image information management unit 230 can also manage a second spherical image in association with a particular position of a first spherical image. Moreover, in this case, the image information management unit 230 can manage the first spherical image in association with a particular position of the second spherical image.
The UI unit 234 manages the transition of screens relating to spherical image content creation, and provides a graphical user interface (GUI) such as a screen and buttons constituting the screen. The UI unit 234 constitutes first providing means in the present embodiment for providing a first instruction unit as a GUI component that receives an instruction for controlling the spherical imaging device 10 to perform imaging. Further, the UI unit 234 constitutes second providing means in the present embodiment for providing a second instruction unit as a GUI component that receives an instruction for invoking the imaging function of the built-in camera 168.
A detailed description is now given of an operation of creating a spherical image content, according to the present embodiment, with reference to
In step S101, the information terminal 50 displays a content information input screen 410 illustrated in
In step S102, the information terminal 50 branches the operation depending on an instruction received from the user on the content information input screen 410 illustrated in
In step S103, as illustrated in
Although in the embodiment, the description given is of an example in which the content information input screen 410 illustrated in
In step S104, the information terminal 50 performs a floor plan registration process. The floor plan is a base image to which spherical images are to be associated in the application. In the present embodiment, the floor plan (also referred to as a “floor map” or a “sketch drawing”) is used as a base image as a base image associating the entire contents with each other. In another example, an image other than the floor plan is used as the base image. In still another example, no floor plan is provided, and the entire content is configured by two types of images, i.e., the spherical image and the normal image.
In step S203, the information terminal 50 invokes the built-in camera control unit 204 by the built-in imaging function invoking unit 226.
By contrast, when the determination result in step S202 indicates that imaging by the built-in camera is not selected (S202: NO), the operation proceeds to step S207. In step S207, the information terminal 50 determines whether the information terminal 50 is selected as the source. When the determination result in step S207 indicates that the “Select Image from Camera Roll” button 424 is tapped in the source selection menu and therefore the information terminal 50 is selected (S207: YES), the operation proceeds to step S208.
In step S208, the information terminal 50 reads a list of images stored in the information terminal 50 (e.g., in an image folder) and displays the list in a manner that a desired image is selected. In step S209, the information terminal 50 receives selection from the list of images stored in the information terminal 50, acquires the selected image. Then, the operation proceeds to step S205.
In step S205, the information terminal 50 determines whether the image is acquired (acquired by imaging, or acquired from the information terminal itself or an extraneous source). When the determination result indicates that the image is acquired (S205: YES), the operation proceeds to step S206. In step S206, the information terminal 50 registers the acquired image as a floor plan in association with the content, and records “with floor plan”. In step S210, the operation returns to the operation of
By contrast, when the determination result in step S205 indicates that no image is acquired (S205: NO), such as when no imaging is performed or when no selection of the desired image is performed, the operation proceeds to S210 and returns to the operation of
Referring again to
In step S105, the operation by the information terminal 50 branches depending on the presence or absence of the floor plan. When the determination result in step S105 indicates that the floor plan is registered (S105: YES), the operation proceeds to step S106.
In step S106, the information terminal 50 displays a content creation screen (with floor plan) 440 to register a spherical image in association with a position on the floor plan. In step S107, the operation by the information terminal 50 branches depending on an instruction received from the user on the content creation screen (with floor plan) 440. When the floor plan is registered and no spherical image is registered yet, the content creation screen (with floor plan) 440 is in the state as illustrated in
The content creation screen (with floor plan) 440 illustrated in
In the content creation screen (with floor plan) 440 displayed in step S106, a function to “save” a content and a function to confirm a registered spherical image in the content (“image confirmation”) can be invoked, in addition to the registration of the spherical image (“image registration”). However, when no spherical image is registered yet, a GUI component for the “image confirmation” is not displayed. On the other hand, a GUI component to “save” a content may be displayed so that the content can be temporarily saved.
When the determination result in step S107 indicates that an instruction for “image registration” is received on the content creation screen (with floor plan) 440, the operation proceeds to step S108. In step S108, the information terminal 50 displays the source selection menu illustrated in
In step S109, the information terminal 50 sets the content and the position coordinate value on the image of the floor plan corresponding to the designated position as an association destination to which the spherical image is to be registered. In step S110, the information terminal 50 performs a spherical image registration process.
By contrast, when the determination result in step S105 indicates that no floor plan is registered (S105: NO), the operation proceeds to step S111. In step S111, the information terminal 50 displays a content creation screen (without floor plan) 460. In step S112, the operation by the information terminal 50 branches depending on an instruction received from the user on the content creation screen (without floor plan) 460. When no floor plan is registered and no spherical image is registered yet, the content creation screen (without floor plan) 460 is in the state as illustrated in
In the content creation screen (without floor plan) 460 displayed in step S111, a function to “save” a content and a function to confirm a registered spherical image in the content (“image confirmation”) can be invoked, in addition to the registration of the spherical image (“image registration”). Although in a state in which no spherical image is registered yet, a GUI component for the “image confirmation” is not displayed, a GUI component to “save” the content may be displayed for temporary storage.
When the determination result in step S112 indicates that an instruction for “image registration” is received on the content creation screen (without floor plan) 460, the operation proceeds to step S113. In step S113, the information terminal 50 sets the content itself as an association destination with which a spherical image to be captured is to be associated. In step S110, the information terminal 50 performs the spherical image registration process.
In the embodiment, the description given is of an example in which the screen illustrated in
In step S302, the information terminal 50 controls the spherical imaging device 10 by the spherical imaging device control unit 222 to capture a spherical image.
By contrast, when the determination result in step S301 indicates that imaging by the spherical imaging device 10 is not selected (S301: NO), the operation proceeds to step S304. In step S304, the information terminal 50 determines whether a captured image stored in the spherical imaging device 10, which is an external imaging device, is selected as the source. When the determined result in step S304 indicates that the captured image stored in the spherical imaging device 10 is selected as the source (S304: YES), the operation proceeds to step S305.
In step S305, the information terminal 50 acquires a list of spherical images that have been captured (and has not been transferred to the information terminal 50 yet) from the spherical imaging device 10 by the spherical imaging device control unit 222, and displays the list in a manner that a desired spherical image is selected. In step S306, the information terminal 50 receives a selection of a particular image from the list of the spherical images stored in the spherical imaging device 10. In step S303, the information terminal 50 receives the selected spherical image from the spherical imaging device 10, and the operation proceeds to step S313.
By contrast, when the determination result in step S304 indicates that a captured image stored in the spherical imaging device 10, which is an external imaging device, is not selected as the source (S304: NO), the operation proceeds to step S307. The determination in step S304 is performed in a case in which the menu illustrated in
In step S307, the information terminal 50 determines whether images stored in the information terminal 50 are selected as the source. When the determination result in step S307 indicates that the images stored in the information terminal 50 are selected as the source (S307: YES), the operation proceeds to step S308.
In step S308, the information terminal 50 reads out a list of spherical images stored in the information terminal 50. A format of a spherical image is same as that of a general-purpose image, such as JPEG. Further, to an image, information for identifying whether the image is a spherical image is attached as meta information. For example, when a user selects an image other than a spherical image, the information terminal 50 prompts the user to select a different image, based on this meta information. In another example, based on the meta information, the information terminal 50 displays, from the acquired list of images, spherical images, and other normal captured images in a manner that two types of images are distinguishable. In still another example, the information terminal 50 extracts only spherical images in advance based on the meta information and include the extracted spherical images in the list to be displayed. Further, it is assumed that spherical images that have already been captured and has been acquired are stored in the information terminal 50. In step S309, the information terminal 50 receives a selection of a particular spherical image from the list of the spherical images stored in the information terminal 50, and reads the selected image. Then, the operation proceeds to step S313.
By contrast, when the determination result in step S307 indicates that images stored in the information terminal 50 are not selected as the source (S307: NO), the operation proceeds to step S310. In step S310, the information terminal 50 acquires a list of spherical images from an image management system (e.g. the content publishing server 70) communicable with the information terminal 50 via a network. It is assumed that spherical images that have already been captured have been uploaded to the image management system. In step S311, the information terminal 50 receives a selection of a particular spherical image from the list of the spherical images stored in the image management system, and in step S312, receives the selected spherical image via the network. Then, the operation proceeds to step S313.
The processes from step S310 to step S312 are performed in a case in which the source selection menu illustrated in
In step S313, the information terminal 50 displays a screen for confirming a spherical image to be used. In step S314, the information terminal 50 determines whether the spherical image to be used is confirmed.
When the determination result in step S314 indicates that the image is confirmed (S314: YES), as the button 494 is tapped on the spherical image confirmation screen 490, the operation proceeds to step S315. In step S315, the information terminal 50 registers the content by associating the confirmed spherical image with the association destination that is set in step S109 or S113 of
When the spherical image registration process illustrated in
The content creation screen (without floor plan) 460 illustrated in
The display area 466 of a particular spherical image is a GUI component for invoking confirmation of a registered spherical image (“image confirmation”). The “Save” button 468 is a GUI component for invoking a content saving process. As one or more spherical images are registered, “image confirmation” is displayed, and the GUI component of “save” of the content is displayed as needed.
The content creation screen (with floor plan) 440 illustrated in
In the display area 446, the spherical image currently being selected is displayed. The display area 446 further includes a GUI component 445 for invoking a screen for entering information indicating a place where the spherical image is captured. Further, an icon 442c having a fan shape indicating an angle at which the spherical image is being displayed is superimposed on the display area 442 of the floor plan. The display area 446 of the spherical image is a GUI component for invoking confirmation (“image confirmation”) of the spherical image that is registered and currently being selected. The content creation screen (with floor plan) 440 illustrated in
Hereinafter, a description is given of an embodiment in which a normal image as an annotation is added to a spherical image using the content creation screen (without floor plan) 460 illustrated in
When the display area 466 corresponding to a particular spherical image is tapped on the content creation screen (without floor plan) 460 illustrated in
In step S114, the information terminal 50 displays an image confirmation screen 500 as illustrated in
When the menu button 508 is tapped on the image confirmation screen 500 illustrated in
When the determination result in step S115 that the instruction of “Add annotation” is accepted on the image confirmation screen 500 illustrated in
The description given above is of an example in which the “Add Annotation” button 510 is displayed in response to tapping the menu button 508 on the image confirmation screen 500 illustrated in
When displaying such spherical image on a device having a flat surface such as the display 162 of the information terminal 50, the spherical image is mapped to a spherical object 310, and a virtual camera 312 is arranged, for example, at the center of the spherical object 310. Thereby, the spherical image is displayed as a projection image 314 viewed from the virtual camera 312. In other words, the display area 504 is displayed on the display in a predetermined size, and the projection image 314 generated from the spherical image is arranged in the display area 504. A position in the spherical image corresponding to the center position of the projection image 314 currently being displayed is acquired from a direction of the virtual camera 312. On the other hand, a certain position (e.g., a tapped position) 322 in the display area 504 is converted to a coordinate 332 on the spherical image format 330, from relative positions (X1/(X1+X2), Y1/(Y1+Y2)) as illustrated in
Referring again to
In step S119, the information terminal 50 displays a source selection menu that accepts a selection of a source from which a normal image is to be acquired, as illustrated in
In step S120, the information terminal 50 performs a normal image registration process.
In step S402, the information terminal 50 invokes the built-in camera control unit 204 by the built-in imaging function invoking unit 226.
By contrast, when the determination result in step S401 indicates that imaging by the built-in camera 168 is not selected (S401: NO), the operation proceeds to step S404. In step S404, the information terminal 50 determines whether the information terminal 50 is selected as the source. When the determination result in step S404 indicates that the information terminal 50 is selected as the source (S404: YES), the operation proceeds to step S405.
In step S405, the information terminal 50 reads a list of images stored in the information terminal 50 and displays the list in a manner that a desired image is selected. In step S406, the information terminal 50 receives a selection of a particular image from the list of the images stored in the information terminal 50, and reads the selected image. Then, the operation proceeds to step S410.
By contrast, when the determination result in step S404 indicates that images stored in the information terminal 50 are not selected as the source (S404: NO), the operation proceeds to step S407. In step S407, the information terminal 50 acquires a list of normal images from an image management system (e.g. the content publishing server 70) communicable with the information terminal 50 via a network, and displays the acquired list. In step S408, the information terminal 50 receives a selection of a particular normal image from the list of the normal images stored in the image management system, and in step S409, receives the selected normal image via the network. Then, the operation proceeds to step S410. The processes from step S407 to step S409 are performed in a case in which the source selection menu illustrated in
In step S410, the information terminal 50 displays a screen for confirming a spherical image to be used. In step S410, the information terminal 50 determines whether the normal image to be used is confirmed.
When the determination result in step S411 indicates that the image is confirmed (S411: YES), as the button 542 is tapped on the image confirmation screen 540, the operation proceeds to step S412. In step S412, the information terminal 50 registers the content by associating the confirmed image with the association destination. In step S413, the operation returns to the operation of
When the operation returns to the operation of
When the determination result in step S115 indicates that the information terminal 50 receives an instruction for returning to an upper level, as the “Back” button 502 is tapped on the image confirmation screen 500 illustrated in
It is assumed that the content creation screen (without floor plan) 460 as illustrated in
In step S121, the information terminal 50 stores the created spherical image content in the content storage unit 240. Then, the operation ends in step S122. In another example, instead of storing the spherical image content in the content storage unit 240 or in addition to storing the spherical image content in the content storage unit 240, the spherical image content is uploaded to the content publishing server 70.
In another example, any other suitable functions are provided, although a detailed description thereof is omitted below. Examples of the other suitable functions include deleting and replacing the registered spherical image and the normal image, before storing or uploading the spherical image content. Examples of the other suitable functions further include performing image processing such as blurring a particular area of the spherical image or the normal image, stamping, and reducing, before storing or uploading the spherical image content. Examples of the other suitable functions still further include inputting, changing, or deleting the name of the content or the spherical image, before storing or uploading the spherical image content.
Although the description given heretofore is of an example in which a normal image is used as an annotation to be associated with a particular position in a spherical image, the annotation is not limited to the normal image. In another example, another spherical image is added as an annotation, or other types of information such as text or audio are added as annotations. Further, although the description given heretofore is of an example in which a normal image is associated with a particular position in the spherical image, in another example, the normal image is associated with an entire spherical image instead of the particular position, or is associated with a content, which is a higher unit including the spherical image.
For example, in a content that introduces a piece of real estate and equipment, a normal image obtained by photographing equipment provided in each of plural rooms is associated with a particular position of a spherical image of the corresponding room. On the other hand, in some cases, a normal image is obtained by photographing view from an entrance or a veranda. In this case, such normal image can be associated with the entire content rather than the spherical image of each room. In still another example, spherical images of plural rooms are associated with each other at a predetermined position (e.g., example, a position corresponding to the door).
By the above-described operations, the spherical image content having a data structure schematically illustrated in
As described heretofore, according to the described embodiment, in an information processing apparatus (a computer) including imaging means, a program implementing the information processing apparatus configured to integrate a function of controlling an external imaging device to capture an image and a function of imaging by the imaging means that the information processing apparatus (the computer) includes, and manage a plurality of types of captured images collectively, a storage medium storing the program, and the information processing apparatus are provided.
In the above-described embodiment, a function of controlling an external imaging device to capture an image and a function of capturing an image with imaging means that the information processing apparatus itself includes are integrated into a single application. Therefore, a user is saved from having to use the function of controlling the external imaging device to capture an image and the function of capturing an image by the imaging means in separate applications.
Further, when using the different imaging functions in different applications respectively, images are to be temporarily stored in an image folder such as a camera roll and registered in a content via the image folder. However, in this case, although the images can be stored in chronological order in the image folder, the images to be registered in different contents are sometimes stored in a mixed manner. In some cases, this makes it difficult to correctly register the images in the content. By contrast, in the above-described embodiment, since a plurality of types of images are collectively managed, complications caused by the intervening image folder are reduced.
Further, integrating the capturing of the spherical image and the capturing of the normal image for content creation reduces time and efforts of a creator of the content. Since the creator can perform up to photographing a normal image on the content creation application installed in the information terminal 50, as long as the spherical imaging device 10 and the information terminal 50 are provided, photographic equipment such as a general-purpose is not required. Thus, capturing a normal image or movie, which is required in the related art, is no more required. Furthermore, a content, a floor plan (optional), a spherical image, and a normal image are associated without time and efforts. This reduces time and efforts of classifying and managing the captured normal images.
In the embodiment, the description given above is of an example in which a content to be created is a spherical image content including a spherical image as a wide-angle image. However, the spherical image is just one example of the wide-angle image. In another example, the wide-angle image is a panoramic image obtained by photographing 360 degrees in a horizontal plane or an image that is a part of the image obtained by photographing omnidirectionally or 360 degrees in a horizontal plane. In the disclosure, the wide-angle image refers to an image having a relatively large angle of view as compared with a normal image, and preferably an image having an angle of view of at least a horizontal angle of view of 180 degrees or more. In still another example, the wide-angle image is an image having a large angle of view captured by one or more imaging devices. The normal image is either a still image or a moving image. The normal image an image having a narrower angle of view than the wide-angle image, and more specifically, an image having an angle of view of 100 to 25 degrees, preferably 70 to 45 degrees.
In the embodiment, the description given above is of an example in which the spherical image and the normal image are each a still image. However, the spherical image and the normal image are each not limited to a still images. In another example, the spherical image and the normal image is a semi-moving image (e.g., a time lapse movie, an interval-shot images, animation), which includes a series of still images. In this case, image association is performed by using time information such as a time and a frame interval as the association destination, in addition to the particular position in the spherical image.
According to well-known techniques, when creating contents by associating a still image with a wide-angle image, images captured by an external digital camera are to be copied to an information terminal and a desired image selected from an image folder in the information terminal is to be associated with the wide-angle image. Such capturing images with the external camera and copying the images to the information terminal take time and efforts. Further there is a possibility that an image wrongly selected from the image folder is associated with the wide-angle image, and therefore there is room for improvement.
According to one or more embodiments of the present disclosure, in an method performed by an information processing apparatus including imaging means, a function of controlling an external imaging device to capture an image and a function of imaging by the imaging means that the information processing apparatus includes are integrated, and a plurality of types of captured images are managed collectively.
Each of the functions of the described embodiments may be implemented by one or more processing circuits or circuitry. Processing circuitry includes a programmed processor, as a processor includes circuitry. A processing circuit also includes devices such as an application specific integrated circuit (ASIC), digital signal processor (DSP), field programmable gate array (FPGA) and conventional circuit components arranged to perform the recited functions.
The functional blocks as described above are implemented by a computer-executable program written by programming languages such as an assembler language, C, and object-oriented programming languages such as C++, C#, and Java (registered trademark). The program may be distributed through a telecommunication line or as being stored in a computer-executable storage medium such as a ROM, an electrically erasable and programmable read only memory (EEPROM), an electrically programmable read only memory (EPROM), a flash memory, a flexible disk, a compact disk read only memory (CD-ROM), a compact disk rewritable (CD-RW), a digital versatile disk read only memory (DVD-ROM), a digital versatile disk random access memory (DVD-RAM), a digital versatile disk rewritable (DVD-RW), a Blu-ray disk, an SD card, and a magneto-optical disk (MO).
The above-described embodiments are illustrative and do not limit the present disclosure. Thus, numerous additional modifications and variations are possible in light of the above teachings. For example, elements and/or features of different illustrative embodiments may be combined with each other and/or substituted for each other within the scope of the present disclosure. Any one of the above-described operations may be performed in various other ways, for example, in an order different from the one described above.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2020-019934 | Feb 2020 | JP | national |
