Field of the Invention
The present invention relates to a service system, an information processing apparatus, and a service providing method.
Description of the Related Art
Information service using a network has been one of key areas of marketing and many pictures and videos are posted on Web pages. A user who browses the Web pages can see the scenery of a place taken in images and grasp surrounding circumstances thereof by browsing pictures and videos posted in consideration of a theme and descriptions of the Web pages. For example, if the theme of Web pages is tourism, a user sees the scenery of the tourist sites and grasps surrounding circumstances thereof by thinking that pictures and videos posed on the Web pages are related to tourist sites.
However, generally-used cameras have a limited imaging range and a user who browses pictures and videos can see scenery and grasp surrounding circumstances only in a range of the pictures and videos that have been taken. By contrast, if the surroundings are captured in an image using a camera with a wide angle of view such as a fish-eye lens, a user who browses pictures can see wider scenery and grasp wider surrounding circumstances just by browsing a single image (see Patent Document 1, for example). Patent Document 1 discloses a method for forming a panoramic image, in which an image taken using a fish-eye optical device is converted into a panoramic image. A part of the panoramic image and an imaging location indicated by a point are displayed on a map.
However, since the method for forming a panoramic image disclosed in Patent Document 1 employs a single fish-eye lens, an imaging range thereof is still limited in many respects. Further, how to present taken pictures and videos to users is not considered. [Patent Document 1] Japanese Translation Publication No. H7-504285 of PCT International Application
It is a general object of at least one embodiment of the present invention to provide a service system that can easily provide pictures and videos that have a wide imaging range.
In an embodiment, a service system including a mobile terminal and an information processing device capable of communication via a network is provided. The mobile terminal includes a first transmission unit that transmits a plurality of spherical images taken in respective imaging locations and positional information about the imaging locations to the information processing device. The information processing device includes a reception unit that receives the spherical images transmitted by the first transmission unit; a map data obtaining unit that obtains map data from a map data storage, the map data including the imaging locations of the spherical images; a path information creation unit that creates information about a path made by connecting the imaging locations in the map data obtained by the map data obtaining unit; and a content providing unit that makes content available for a request through the network, the content including the map data, the information about the path, and the spherical images.
According to an embodiment of the present invention, it is possible to provide a service system that can easily provide pictures and videos that have a wide imaging range.
Other objects and further features of embodiments will become apparent from the following detailed description when read in conjunction with the accompanying drawings, in which:
In the following, embodiments of the present invention will be described with reference to the accompanied drawings.
An outline of Web service in the present embodiment will be described with reference to FIGS. 1 to 3.
(1) A photographer 8 walks while gripping an imaging device 11 capable of taking an image of 360 degrees of surroundings (hereafter “omnidirectional surroundings”). The imaging device 11 is set to take an image of surroundings at regular intervals or at each fixed distance. The imaging device 11 associates the taken image (spherical image to be described later) with an imaging location and stores the taken image. Accordingly, it is possible to obtain an image along a route that the photographer 8 has walked.
(2) When the photographer 8 has walked to a destination and taking images on the route ends, the photographer 8 transmits a plurality of images to a content creation server 12 via an access point 9.
(3) The content creation server 12 creates content using a map including the route that the photographer 8 has walked and the images and registers the content with a content providing server 13. In an example to be described later, a mobile terminal that receives an image from the imaging device 11 transmits the image to the content providing server 13.
(4) The content creation server 12 creates content to be described later and registers the content. The content creation server 12 transmits a Uniform Resource Identifier (URI) of the content to a person 7 in charge in the content-using company, the person 7 in charge being registered in advance. In addition, the content creation server 12 may also transmit the URI to the photographer 8 who transmitted the image as indicated in a dotted line of (4).
(5) The person 7 in charge in the content-using company operates a Personal Computer (PC) 15 of the content-using company to input (copy and paste, for example) the URI of the content in HTML data about a Web page (such as a Web site) of the company. In accordance with this, the content-using company can link surrounding images taken along the route that the photographer 8 has walked to the company's Web page.
(6) The person 7 in charge in the content-using company registers (transmits) the Web page with a Web server 14 of the company. Accordingly, an end user 6 can browse the Web page of the content-using company, in which contents are linked.
(7) The end user 6 who browses the Web page of the content-using company operates a user's PC 16 to display the content.
Accordingly, the end user 6 may have a simulated experience as if the end user 6 were actually walking on the above-mentioned route 63. Further, the route 63 is emphatically displayed on the map 62 such that the end user 6 can understand, at first sight, the imaging location 64 of the image 61 currently displayed and an imaging location 64 of an image to be displayed. Further, the end user 6 can operate the PC 16 to display an image of a desired location on the route 63.
In this manner, with a Web service system 100 according to the present embodiment, everyone can easily create and publish content with presence in which space of omnidirectional surroundings is expressed. The Web service system 100 can provide the end user 6 with a simulated experience of touring on a route by replaying an omnidirectional panoramic image.
The content-using company refers to a corporate body, a natural person, a group, or an individual that wishes to introduce a route using content. The content-using company need not be a commercial company.
The Web service system 100 provides the content-using company with a service to support the creation and publication of content.
Examples of the route (an example of a path or path information) include a tourist route at a tourist site, a walking course for mountain climbing or hiking, a running course for a marathon, a course to guide how to get to a destination such as from a station to an apartment, a course to introduce how to get to an attraction site or facilities in a theme park, and a course to introduce a shopping mall or stores in a department store. However, any place may be a route according to the present embodiment as long as people can walk to the place.
The end user 6 is a user who browses content via a Web page (and a Web application) operated by the content-using company.
Content has at least a plurality of images of omnidirectional surroundings. The map 62 and the route 63 may be included in the content. In addition, audio data collected at an imaging location may be included in the content.
Link information refers to information for a given information processing terminal such as the user's PC 16 to obtain content from a resource connected to a network. In the present embodiment, a URI is used for the descriptions as an example of the link information. The URI is information to identify the resource on the network and a Uniform Resource Locator (URL) is information to specify a place of the resource on the network. The URL may be used instead of the URI.
Public information refers to information published on the network. The public information is accessible to the user's PC 16, for example. In the present embodiment, the public information is called a Web page in the descriptions. The Web page is a document published on the Internet. The Web page may be prepared in advance or dynamically generated by a Web application. A URI may be embedded in the Web page.
The network 18 is built with a LAN, a WAN, or the Internet, for example. The network 18 is mainly built with wires but a part or an entire portion of the network 18 may be built wirelessly. Further, the network 18 may include a mobile phone network such as Personal Digital Cellular (PDC), Global System for Mobile communications (GSM™), CDMA, or LTE.
As mentioned above, the content creation server 12 is a server that creates content in which images are used and the content providing server 13 is a server that provides content to the end user 6. The Web server 14 is a server operated by the content-using company and a Web page of the content-using company is registered with the Web server 14. Information to be published by the content-using company is described in HTML or JavaScript™ on the Web page.
The imaging device 11 is an omnidirectional imaging device capable of taking an image of 360 degrees of surroundings with two imaging elements disposed on the imaging device 11. The imaging device 11 does not need to be an imaging device dedicated for omnidirectional image taking. An omnidirectional imaging unit to be installed afterword may be attached to a normal digital camera or a smartphone, for example, to have substantially the same function as in the imaging device 11.
The imaging device 11 is capable of communicating with the mobile terminal 17 in a wired or wireless manner. This is because the mobile terminal 17 supports a function that the imaging device 11 does not have. In other words, if the imaging device 11 does not have a function of detecting positional information (such as a GPS reception function), the mobile terminal 17 detects an imaging location. In the present embodiment, the mobile terminal 17 adds the imaging location to an image taken by the imaging device 11 and transmits the image to the content creation server 12. If the imaging device 11 has a function of detecting positional information, the mobile terminal 17 may be eliminated. Accordingly, the imaging device 11 and the mobile terminal 17 may be provided by a single device.
The mobile terminal 17 has a display screen to display an image. The display screen is used when the photographer 8 confirms the image. However, the confirmation of the image is not mandatory.
Examples of the mobile terminal 17 include a smartphone, a Personal Digital Assistant (PDA), a tablet terminal, a mobile phone, a wearable PC, and a general-purpose PC. However, the mobile terminal 17 is not limited to these examples.
The content-using company's PC 15 is a PC operated by the person 7 in charge in the content-using company when content is linked to a Web page to be published by the content-using company. Examples of the content-using company's PC 15 may be a PC, a smartphone, a PDA, a tablet terminal, a mobile phone, or a wearable PC.
The user's PC 16 is a PC operated by the end user 6 and displays a Web page provided by the content-using company. A specific device of the user's PC 16 may be the same as the content-using company's PC 15. The user's PC 16 is an example of an information processing terminal.
In
Further, the functions of at least one of the content creation server 12, the content providing server 13, and the Web server 14 may be provided as a cloud service in which cloud computing is used. In cloud computing, a plurality of servers are connected using L1 to L3 switches and each server is connected to at least one storage via a fiber channel or iSCSI, for example. In cloud computing, a virtualized server is added or released to or from each server depending on load, so that a physical configuration will be dynamically changed.
In this manner, the configuration of the Web service system 100 shown in the drawings is an example. The Web service system 100 may be any system as long as it has a function of an information processing apparatus capable of performing a process in the present embodiment to be described below.
<<Imaging Device 11>>
An appearance of the imaging device 11 is described with reference to
As shown in
The image unit 101 includes wide-angle lenses (so called “fish-eye lenses”) 102a and 102b each having an angle of view of 180° or more in order to image a hemispheric image, and the two imaging elements 103a and 103b disposed for the wide-angle lenses. The imaging elements 103a and 103b include an image sensor, a timing creation circuit, a register group, and the like. The image sensor may be a Complementary Metal Oxide Semiconductor (CMOS) sensor or a Charge Coupled Device (CCD) sensor that converts an optical image imaged by the fish-eye lens into image data of an electric signal. The timing creation circuit creates a horizontal or vertical synchronizing signal and a pixel clock of the image sensor. In the register group, various commands, parameters, and the like necessary for an operation of the imaging element are set.
The imaging elements 103a and 103b are each connected to the image processing unit 104 via a parallel I/F bus. The imaging elements 103a and 103b are connected to the imaging control unit 105 via a serial I/F bus (such as an I2C bus). The image processing unit 104 and the imaging control unit 105 are connected to the CPU 111 and the like via a bus 110.
The image processing unit 104 receives sets of image data output from the imaging elements 103a and 103b via the parallel I/F bus, performs a predetermined process on each set of image data, and combines the sets of image data to generate image data about a Mercator image as shown in
The imaging control unit 105 sets commands in the register group of the imaging elements 103a and 103b using the I2C bus. The imaging control unit 105 receives necessary commands and the like from the CPU 111. Further, the imaging control unit 105 uses the I2C bus to receive status data about the register group of the imaging elements 103a and 103b and sends the status data to the CPU 111.
The imaging control unit 105 instructs the imaging elements 103a and 103b to output image data at a time when the shutter button of the operation unit 115 is pressed. The imaging device 11 may have a preview display function or a function for movie display in a display unit. In this case, the output of image data from the imaging elements 103a and 103b is continuously performed at a predetermined frame rate (frame/minute).
Further, the imaging control unit 105 also functions as a synchronization control unit that synchronizes output timing of the image data from the imaging elements 103a and 103b in cooperation with the CPU 111. In addition, while the display unit is not disposed on the imaging device 11 in the present embodiment, the display unit may be disposed.
The CPU 111 controls an entire operation of the imaging device 11 and performs a necessary process. The ROM 112 stores various programs for the CPU 111. The SRAM 113 and the DRAM 114 are working memories and store a program to be executed by the CPU 111 and data being processed, for example. The DRAM 114 stores image data being processed by the image processing unit 104 and image data about a processed Mercator image.
The operation unit 115 is a generic term for various operation buttons, a power supply switch, the shutter button, and a touch panel that serves both as display and operation functions. The photographer 8 operates the operation buttons to input various imaging modes and imaging conditions.
The network I/F 116 is an interface to connect to the network 18 via an Ethernet (registered trademark) card, for example. Further, the network I/F 116 may also be used as an interface for an external recording medium such as a USB memory or an SD card. Data about a Mercator image stored in the DRAM 114 may be stored in the recording medium from the network I/F 116.
The communication unit 117 is a network interface to connect to the network 18 via the antenna 117a. Examples of the communication unit 117 include a communication device for a wireless LAN, a communication chip for BlueTooth (registered trademark), and Near Field Communication (NFC). However, the communication unit 117 is not limited to these examples. In the present embodiment, data about a Mercator image stored in the DRAM 114 is transmitted to the mobile terminal 17 via the communication unit 117.
The electronic compass 118 calculates a direction and an inclination (Roll angle of rotation) of the imaging device 11 from terrestrial magnetism, and outputs direction and inclination information. This direction and inclination information is attached to an image in accordance with a metadata format called Exif. The direction and inclination information is used for image processing such as image correction. In addition, the Exif includes an imaging date and time, a thumbnail of image data, a data volume of the image data, and the like.
<<Hardware Configuration of Mobile Terminal 17>>
In addition, the EEPROM 204 stores an Operating System (OS) to be executed by the CPU 201, and other programs and various data. Further, a CCD sensor may be used instead of the CMOS sensor 205.
The mobile terminal 17 includes an audio input unit 211 that converts audio into an audio signal, an audio output unit 212 that converts an audio signal into audio, an antenna 213a, a communication unit 213 that uses the antenna 213a to communicate with a nearest access point 9 via a radio communication signal, a GPS reception unit 214 that receives a GPS signal including positional information (latitude, longitude, and altitude) about the imaging device 11 by Global Positioning Systems (GPS) satellites or Indoor MEssaging System (IMES) as indoor GPS, a display screen 215 such as a liquid crystal display or an organic EL display that displays an image of an object and various types of icons, a touch panel 216 that is disposed on the display screen 215 in an integrated manner and, the touch panel 216 detecting a touch location on the display screen 215 in response to a touch by the finger, a touch pen, or the like, and a bus line 210 such as an address bus or a data bus which electrically connects each of the above-mentioned elements.
In addition, the audio input unit 211 includes a microphone that inputs audio and the audio output unit 212 includes a loud speaker that outputs audio. In accordance with this, it is possible to obtain audio data upon taking an image.
<<Hardware Configurations of Content Creation Server 12, Content Providing Server 13, Web Server 14, Content-Using Company's PC 15, and User's PC 16>>
The content creation server 12 includes a CPU 301, a ROM 302, a RAM 303, a Hard Disk Drive (HDD) 305, a display screen 308, a network I/F 309, a keyboard 311, a mouse 312, a media drive 307, and an optical drive 314, being connected to a bus 310. The CPU 301 executes a program 320 stored in an HD 304 to control an entire operation of the content creation server 12. The ROM 302 stores a program such as an IPL used to drive the CPU 301. The RAM 303 is used as a work area for the CPU 301. The HD 304 is a storage device on which a non-volatile memory is installed. The HD 304 stores the program 320 to create content, an OS, and the like.
The HDD 305 controls reading or writing of various types of data from or into the HD 304 in accordance with control of the CPU 301. The display screen 308 displays various types of information such as a cursor, menus, a window, characters, and an image. The network I/F 309 is an interface for the network 18 such as a LAN or the Internet.
The keyboard 311 and the mouse 312 are input and output devices. The keyboard 311 has a plurality of keys for inputting characters, numerical values, and various types of instructions and receives an input from the keys. The mouse 312 receives a movement of a mouse pointer, a selection and execution of various types of instructions, and a selection of an object to be processed.
The media drive 307 controls data reading or writing (storing) from or into a medium 306 such as a flash memory. The optical drive 314 controls reading or writing of various types of data from or into a Compact Disc (CD) 313 serving as a removable recording medium, for example.
In addition, the program 320 may be stored distributed in a file of an installable format or in a file of an executable format in a computer-readable recording medium such as the medium 306 or the CD 313. Or the program 320 may be delivered in a form downloaded from a given server-type information processing apparatus.
Since the hardware configuration of the content providing server 13 may be the same as the hardware configuration of the content creation server 12, illustration is omitted. However, in the case of the content providing server 13, the HD 304 stores the program 320 to provide content. Further, since the hardware configuration of the Web server 14 may be the same as the hardware configuration of the content creation server 12, illustration is omitted. However, in the case of the Web server 14, the HD 304 stores the program 320 to provide a Web page and a Web application and the Web page (Web application). Further, since the hardware configuration of the content-using company's PC 15 may be the same as the hardware configuration of the content creation server 12, illustration is omitted. However, in the case of the content-using company's PC 15, the HD 304 stores the program 320 to edit HTML data. Further, since the hardware configuration of the user's PC 16 may be the same as the hardware configuration of the content creation server 12, illustration is omitted. However, in the case of the user's PC 16, the HD 304 stores browser software capable of browsing a Web page and content, for example, as the program 320.
<Example of Imaging by Imaging Device 11>
In the following, a range to be imaged by the imaging device 11 and an image are described with reference to
In this manner, each of the imaging elements 103a and 103b takes an image of objects around the photographer 8, so that two hemispheric images are obtained.
As shown in
<Functions of Web Service System 100>
The imaging device 11 includes an image creation unit 31, an imaging information collection unit 32, and a transmission and reception unit 33. Each of the units (the image creation unit 31, the imaging information collection unit 32 and the transmission and reception unit 33) included in the imaging device 11 is a function implemented or a unit caused to function when any one of constituent elements shown in
The image creation unit 31 is implemented by an instruction from the CPU 111, the image unit 101, the image processing unit 104, and the imaging control unit 105 shown in
The imaging information collection unit 32 is implemented by an instruction from the CPU 111 and the electronic compass 118 shown in
This imaging information is attached to an image in a file format called Exif, for example. In addition, the direction is information that indicates north, south, east and west in the image. In addition, the roll angle may be corrected to be zero when an image is taken. In this case, the roll angle may not be included in the imaging information.
The transmission and reception unit 33 is implemented by an instruction from the CPU 111 and the communication unit 117 shown in
<<Mobile Terminal 17>>
In the following, functions of the mobile terminal 17 are described. The mobile terminal 17 includes an image with data transmission unit 34, a positional information obtaining unit 35, an image reception unit 36, an operation input reception unit 37, and a storing and reading unit 39. Each of the units (the image with data transmission unit 34, the positional information obtaining unit 35, the image reception unit 36, the operation input reception unit 37, and the storing and reading unit 39) included in the mobile terminal 17 is a function implemented or a unit caused to function when any one of constituent elements shown in
Further, the mobile terminal 17 also includes a storage 2000 constructed with the RAM 203 or the EEPROM 204 shown in
The operation input reception unit 37 is implemented by an instruction from the CPU 201 and the touch panel 216 shown in
The image reception unit 36 is implemented by an instruction from the CPU 201 and the communication unit 213 shown in
The positional information obtaining unit 35 is implemented by an instruction from the CPU 201 and the GPS reception unit 214 shown in
Imaging information: “imaging date and time, direction, and the imaging location”
The image with data transmission unit 34 is implemented by an instruction from the CPU 201 and the communication unit 213 shown in
The storing and reading unit 39 is implemented by an instruction from the CPU 201 shown in
<<Content Creation Server 12>>
In the following, functions of the content creation server 12 are described. The content creation server 12 includes an image correction unit 20, a route creation unit 21, a map obtainment unit 22, an image with data reception unit 23, a URI transmission unit 24, a content transmission unit 25, an image embedding unit 26, and a storing and reading unit 29. Each of the units (the image correction unit 20, the route creation unit 21, the map obtainment unit 22, the image with data reception unit 23, the URI transmission unit 24, the content transmission unit 25, the image embedding unit 26, and the storing and reading unit 29) included in the content creation server 12 is a function implemented or a unit caused to function when any one of constituent elements shown in
Further, the content creation server 12 also includes a storage 3000 constructed with the RAM 303 or the HDD 305 shown in
The map DB 3001 stores data to draw the map 62 and data that represents a structure of roads where walkers can walk. Since the data to draw the map 62 contains many display objects including boundaries such as prefectures, green space, rivers, roads, railroads, symbols, and notes, the data is classified into groups of similar properties, so that and each group can be separately drawn. An image where display objects classified into each group or display objects are drawn is called a layer. A map is drawn when several layers are superposed. Map data in each layer is either vector data or raster data described in a format suitable for the display objects. Further, in the map data, longitude and latitude are sectioned into meshes and one or more meshes are coupled to create the map 62. In the case of the vector data, points, polylines, and polygons are positioned based on longitude and latitude. In the case of the raster data, data is prepared in association with longitude and latitude depending on a scale.
The data that represents the structure of roads has a node table and a link table. In the node table, nodal points when road networks are represented are registered in association with longitude and latitude. The nodal points are called nodes. Examples of nodes include intersections, junctions, confluences, and curves. In the link table, roads where walkers can walk (sidewalks, crosswalks, pedestrian bridges, underpasses, roads where walkers can pass through) are registered in association with a node number of a node. Not only the roads where walkers can walk but also roads where cars can run may also be registered. Further, in the link table, a link type, a width, a link length, and the like are registered. A road between two nodes is called a link. The link can be a line segment that connects nodes.
In the account management DB 3002, an account table for content-using companies is registered as shown in Table 1. In the account table, IDs of the content-using companies, passwords, and mail addresses are registered, for example. The IDs of the content-using companies are unique identification information to identify the content-using companies. The passwords are used when the content creation server 12 authenticates the content-using companies. The mail addresses are mail addresses of persons 7 in charge in the content-using companies and are destinations when a URI of content is transmitted. In addition, instead of the mail addresses or in addition to the mail addresses, FAX numbers or phone numbers may be registered.
The image with data reception unit 23 is implemented by an instruction from the CPU 301 and the network I/F 309 shown in
The image correction unit 20 is implemented by an instruction from the CPU 301 shown in
The map obtainment unit 22 is implemented by an instruction from the CPU 301 shown in
The route creation unit 21 is implemented by an instruction from the CPU 301 shown in
The image embedding unit 26 embeds images in imaging locations arranged when the route 63 is created. Embedding refers to associating imaging locations with images on a map. In accordance with the association, when an end user 6 specifies an imaging location on the map, an image taken at the imaging location is displayed. The image embedding unit 26 is an example of an image association unit.
Content refers to data in which the route 63 is created on the map and images are embedded in imaging locations. As shown in
The content providing server 13 transmits an image of the selected location to the user's PC 16. Further, a plurality of still images may be converted into a movie file such as MPEG.
The content transmission unit 25 is implemented by an instruction from the CPU 301 and the network I/F 309 shown in
The domain name is known because it is a domain of the content providing server 13. The directory name is a known folder name (such as “4000” for a storage 4000). The content name may be provided by the photographer 8 before transmission of an image or may be provided by the content creation server 12. Accordingly, a URI may be “content providing server/4000/ID of content-using company/content name,” for example.
The URI transmission unit 24 is implemented by an instruction from the CPU 301 and the network I/F 309 shown in
The storing and reading unit 29 is implemented by an instruction from the CPU 301 shown in
<<Content Providing Server 13>>
In the following, functions of the content providing server 13 are described. The content providing server 13 includes a browsing information obtaining unit 55, a content reception unit 56, a content providing unit 57, and a storing and reading unit 54. Each of the units (the browsing information obtaining unit 55, the content reception unit 56, the content providing unit 57, and the storing and reading unit 54) included in the content providing server 13 is a function implemented or a unit caused to function when any one of constituent elements shown in
The content providing server 13 also includes the storage 4000 constructed with the RAM 303 or the HDD 305 shown in
The content management DB 4001 stores a content management table as shown in Table 2. In the content management table, IDs of content-using companies, content names, URIs, numbers of provisions, a browsing time, and time periods are registered, for example. In addition, for ease of description, it is assumed that content is stored in the content management DB 4001. However, the content may be stored at a location accessible via the network 18. If a URI is specified, the content providing server 13 can identify content. The numbers of provisions indicate how many times each content is provided to an end user 6. The browsing time indicates a staying time in the content providing server 13 and a number of people is registered in each of time periods divided into 5-10 minute intervals. In addition, it is possible to measure the staying time in the content providing server 13 from a period of time when a TCP session is maintained, for example. The time periods indicate in which time period content is browsed and a number of people registered in each of the time periods divided into one hour intervals like 0-1 o'clock and 1-2 o'clock, for example.
The content reception unit 56 is implemented by an instruction from the CPU 301 and the network I/F 309 shown in
The content providing unit 57 is implemented by an instruction from the CPU 301 shown in
The browsing information obtaining unit 55 is implemented by an instruction from the CPU 301 shown in
The storing and reading unit 54 is implemented by an instruction from the CPU 301 shown in
<<Content-Using Company's PC 15>>
In the following, functions of the content-using company's PC 15 are described. The content-using company's PC 15 includes a URI reception unit 41, an operation input reception unit 42, a Web page transmission unit 43, and a storing and reading unit 44. Each of the units (the URI reception unit 41, the operation input reception unit 42, the Web page transmission unit 43, and the storing and reading unit 44) included in the content-using company's PC 15 is a function implemented or a unit caused to function when any one of constituent elements shown in
The content-using company's PC 15 also includes a storage 5000 constructed with the RAM 303 or the HDD 305 shown in
The Web page DB 5001 stores Web pages of the content-using company's PC 15. These Web pages are to be registered with the Web server 14 and are the same as Web pages to be provided by the Web server 14.
The URI reception unit 41 is implemented by an instruction from the CPU 301 and the network I/F 309 shown in
The operation input reception unit 42 is implemented by an instruction from the CPU 301, the keyboard 311, and the mouse 312 shown in
The Web page transmission unit 43 is implemented by an instruction from the CPU 301 and the network I/F 309 shown in
The storing and reading unit 44 is implemented by an instruction from the CPU 301 shown in
In addition, the content-using company's PC 15 may include the map obtainment unit 22, the route creation unit 21, and the image embedding unit 26 of the content creation server 12. In accordance with this, the person 7 in charge in the content-using company can create content.
<<Web Server 14>>
In the following, functions of the Web server 14 are described. The Web server 14 includes a Web page reception unit 51, a Web page providing unit 52, and a storing and reading unit 53. Each of the units (the Web page reception unit 51, the Web page providing unit 52, and the storing and reading unit 53) included in the Web server 14 is a function implemented or a unit caused to function when any one of constituent elements shown in
Further, the Web server 14 also includes a storage 6000 constructed with the RAM 303 or the HDD 305 shown in
The Web page DB 6001 stores Web pages to be provided by the Web server 14 in response to a request from the user's PC 16.
The Web page reception unit 51 is implemented by an instruction from the CPU 301 and the network I/F 309 shown in
The Web page providing unit 52 is implemented by an instruction from the CPU 301 and the network I/F 309 shown in
The storing and reading unit 53 is implemented by an instruction from the CPU 301 shown in
<<User's PC 16>>
In the following, functions of the user's PC 16 are described. The user's PC 16 includes a Web page request unit 45, a Web page display unit 46, and an operation input reception unit 47. Each of the units (the Web page request unit 45, the Web page display unit 46, and the operation input reception unit 47) included in the user's PC 16 is a function implemented or a unit caused to function when any one of constituent elements shown in
The Web page request unit 45 is implemented by an instruction from the CPU 301 and the network I/F 309 shown in
The Web page display unit 46 is implemented by an instruction from the CPU 301 and the display screen 308 shown in
The operation input reception unit 47 is implemented by an instruction from the CPU 301, the keyboard 311, and the mouse 312 shown in
In addition, the user's PC 16 may include the map obtainment unit 22, the route creation unit 21, and the image embedding unit 26 of the content creation server 12. In accordance with this, the end user 6 can create content.
<Embedding of URI>
In the URI shown in
Iframe tags are a kind of tags for HTML data by which a resource (content in the present embodiment) specified by “src” is displayed in line on a screen. A width of an image to be displayed on the user's PC 16 is specified by “width” and a height of the image to be displayed is specified by “height.” In the “src,” a domain of the site and the URI are described. A width of a boundary line when an image is displayed is specified by “frameborder.” Allowing display in a full screen mode is specified by “allowfullscreen.”
In addition, the Iframe tags are an example. Any kind of tags may be used in compliance with a description format of Web pages (description format of HTML in this case).
<Transmission of Content when Content is Provided>
When the content providing unit 57 transmits content to the user's PC 16, the content providing unit 57 may transmit at least the map 62 and one or more images at first. It is not necessary to transmit all the images included in the content. Further, when the end user 6 successively specifies imaging locations or replays the content as a movie, the end user 6 does not see the images of omnidirectional surroundings.
Accordingly, the content providing unit 57 transmits only a partial image of the same area as displayed on the user's PC 16 instead of transmitting all portions the image to the user's PC 16. In this manner, the content providing server 13 may transmit only a part of omnidirectional panoramic images that have a relatively large data size, so that a transmission time can be reduced.
On the spherical image, a pixel of a given point is specified in accordance with coordinates such as longitude and latitude. In
The Web page request unit 45 transmits the coordinates of the partial image Im to the content providing server 13. The coordinates of the partial image Im are an example of image range information and specify an area (angle of view) when a part of the spherical image is displayed on the display screen 308. The content providing unit 57 of the content providing server 13 transmits only the partial image defined by the coordinates from an image to be transmitted next to the user's PC 16. The image to be transmitted next is an image in a next imaging location along the route 63 if the end user 6 has browsed a plurality of images. If the end user 6 has browsed only a single image, the image to be transmitted next is a next image arranged in a direction perpendicular (in a direction that the end user 6 sees) to the partial image displayed in the browsed image.
In accordance with this, the content providing server 13 can transmit the image in a short time. In addition, if there is a sufficient communication band (if there is sufficient time until an image in the next imaging location is to be transmitted), the content providing unit 57 preferentially transmits another partial image adjacent to the transmitted partial image to the user's PC 16. Accordingly, if there is a sufficient communication band, it is possible to transmit all portions of a spherical image to the user's PC 16. Further, if the image in the next imaging location is requested from the user's PC 16 before all the portions of spherical image are transmitted to the user's PC 16, only a partial image defined by four coordinates in the next imaging location is transmitted.
<Embedding of Spherical Image>
<Operational Procedure>
In the following, a procedure for transmitting an image is described with reference to steps of
S1: When the photographer 8 operates the mobile terminal 17, the operation input reception unit 37 receives the operation and transmits operation contents to the imaging device 11. In this case, it is assumed that an operation to start taking images is performed. Further, a time to specify a cycle as imaging intervals or a distance to specify a fixed distance may be transmitted.
S2: The image creation unit 31 takes an image at regular intervals or at each fixed distance and transmits the image to the imaging information collection unit 32. If the image is taken at each fixed distance, as shown in S1-1, the mobile terminal 17 reports, to the imaging device 11, that the photographer 8 has moved to the fixed distance. The movement to the fixed distance may be estimated from positional information or may be estimated from a number of steps detected by the acceleration and direction sensor 206. In accordance with this, it is possible to obtain images at substantially equal intervals on a route, so that when the end user 6 displays content, the end user 6 is likely to have a feeling as if he is actually walking.
S3: The imaging information collection unit 32 collects an imaging date and time and a direction, attaches them as metadata to the image, and sends the image and the metadata to the transmission and reception unit 33.
S4: The transmission and reception unit 33 transmits the image to which the imaging date and time and the direction are attached to the image reception unit 36 of the mobile terminal 17.
S5: The image reception unit 36 of the mobile terminal 17 receives the image to which the imaging date and time and the direction are attached and sends them to the positional information obtaining unit 35.
S6: Upon receiving the image, the positional information obtaining unit 35 obtains an imaging location and attaches it to the image. In accordance with this, the image to which imaging information is attached is obtained. The positional information obtaining unit 35 stores the image in the image DB 2001.
The imaging device 11 and the mobile terminal 17 repeat taking an image and storing the image in the image DB 2001 until the photographer 8 ends the image taking. In addition, the photographer 8 may take images a plurality of times on a single route. For example, if images are taken at morning, noon, and night, content having images taken in different time periods on the route is created. Further, the photographer 8 may take images in different situations such as the four seasons, fine weather, rainy weather, and cloudy weather on the same route.
S1: The photographer 8 performs an operation to end taking images.
S2: When the operation to end taking images is performed, the mobile terminal 17 displays a dialog for inputting a content name, so that the photographer 8 inputs the content name. In addition, the content name may be input before ending taking images. Further, the photographer 8 can add an attribute of a start point of a route to any image and an attribute of an end point of the route to any image. In other words, the photographer 8 can specify the start point and the end point of the route. When these inputs end, the operation input reception unit 37 reports an end of taking images to the image with data transmission unit 34.
S3: The image with data transmission unit 34 reads out a serious of images from the start to end of taking images from the image DB 2001.
S4: The image with data transmission unit 34 transmits the series of images to the content creation server 12. The image with data transmission unit 34 may transmit an ID of a content-using company and a password input by the photographer 8 together with the series of images (or before or after the transmission). The content creation server 12 creates content if the photographer 8 is authenticated based on the ID of the content-using company and the password. In addition, if there is no fee charging, the authentication of the content-using company is unnecessary.
S5: The image with data reception unit 23 sends the series of images to the map obtainment unit 22.
S6: The map obtainment unit 22 reads out the map 62 that includes all imaging locations of the series of images from the map DB 3001 and attaches the series of images to the map 62. The map obtainment unit 22 sends the series of images to which the map 62 is attached to the route creation unit 21.
S7: The route creation unit 21 maps all the imaging locations of the series of images to the map 62 and creates a route 63 by successively connecting the imaging locations in the order of imaging date and times. The route 63 is emphatically displayed with a bold line, for example. The route creation unit 21 sends the series of images to which the map 62 and the route 63 is attached to the image embedding unit 26.
S8: The image embedding unit 26 embeds the series of images in the imaging locations on the route 63. Then the image embedding unit 26 sends the map 62 and the route 63 in which the images are embedded to the content transmission unit 25 as one content.
S1: The content transmission unit 25 specifies a URI and transmits the content to the content providing server 13.
S2: The content reception unit 56 of the content providing server 13 receives the content and stores the content in the content management DB 4001 specified in the URI. In addition, the content providing server 13 may store the content in any location and transmit a URI of the storage location to the content creation server 12.
S3: The content transmission unit 25 sends the URI where the content is stored to the URI transmission unit 24.
S4: The URI transmission unit 24 transmits the URI to a mail address of the person 7 in charge in the content-using company.
S5: The URI reception unit 41 of the content-using company's PC 15 receives mail where the URI is described and temporarily stores the mail in the RAM 303, for example. The person 7 in charge in the content-using company opens the mail to display the URI on the display screen 308, so that the operation input reception unit 42 copies the URI, for example, in response to this operation.
S6: The person 7 in charge in the content-using company performs an operation to read out a desired Web page from the Web page DB 5001 and the operation input reception unit 42 reads out the Web page in response to this operation.
S7: The person 7 in charge in the content-using company performs an operation to paste a copied URI on the Web page and the operation input reception unit 42 sends the Web page on which the URI is pasted to the Web page transmission unit 43.
In addition, in the procedure shown in
S1: The Web page transmission unit 43 of the content-using company's PC 15 transmits the Web page on which the URI is pasted to the Web server 14.
S2: The Web page reception unit 51 of the Web server 14 receives the Web page and stores it in the Web page DB 6001.
S3: The end user 6 operates the user's PC 16 to display the Web page of the content-using company. In accordance with this, the Web page request unit 45 of the user's PC 16 specifies an address of a web site, for example, and sends a request for the Web page to the Web server 14.
S4: The Web page providing unit 52 reads out the requested Web page from the Web page DB 6001. On this Web page, the URI of the content is pasted.
S5: The Web page providing unit 52 transmits the Web page to the Web page request unit 45 of the user's PC 16.
S6: The Web page request unit 45 of the user's PC 16 receives the Web page and sends the Web page to the Web page display unit 46. In accordance with this, the user's PC 16 can display the Web page on which the URI of the content is pasted.
S7: The end user 6 moves a mouse cursor and clicks on a field prepared as an area for displaying a spherical image or clicks on an image or a character to which content is linked on the Web page, for example. If such an operation to display content is performed, the Web page request unit 45 specifies the URI and sends a request for content to the content providing server 13.
S8: The content providing unit 57 of the content providing server 13 reads out the specified content from the content management DB 4001.
S9: The content providing unit 57 transmits the content to the user's PC 16.
S10: The Web page request unit 45 of the user's PC 16 receives the content and sends the content to the Web page display unit 46. In accordance with this, the user's PC 16 can display the content as shown in
S11: To the browsing information obtaining unit 55, the content providing unit 57 sends a report that the content is provided. The browsing information obtaining unit 55 increments a number of provisions of content. Further, the browsing information obtaining unit 55 determines a browsing time and a current time period and increments a number of people in a corresponding time period of the account management DB 3002. The content providing server 13 calculates an amount of money to charge based on the number of provisions of content and a size of content and charges the content-using company at the end of the month, for example. The browsing time and the time period of the account management DB 3002 may be provided to the content-using company with or without charge. The content-using company can use the browsing time and the time period to improve its Web page, for example.
In accordance with such a process, the end user 6 that has displayed the content can operate a menu 66 in
As mentioned above, with the Web service system 100 according to the present embodiment, the photographer 8 can take a spherical image in a single imaging. Everyone can easily create and publish content with presence in which space of omnidirectional surroundings is expressed. Since spherical images are embedded in the imaging locations 64 on the map 62, the end user 6 can understand which imaging location the spherical image corresponds to at a glance. Further, since the route 63 is displayed on the map 62, the end user 6 can understand at a glance that the route 63 can be checked with the spherical images. It is possible to provide the end user 6 with a simulated experience of touring on the route 63 by replaying an omnidirectional panoramic image. Further, an operator of the content providing server 13 can charge the content-using company depending on a size of content and the number of provisions of content.
While the photographer 8 can take an image of surroundings in any location as described in Embodiment 1, the faces of passersby may be captured in images. If personal specific information about the passersby is captured in the images, this is undesirable in terms of protection of privacy. In view of this, in this embodiment, the Web service system 100 performs an image process on the faces of passersby in order to make identification or determination of persons more difficult.
The image processing unit 27 is implemented by an instruction from the CPU 301 shown in
S4-2: The image processing unit 27 performs an image process on all of a series of images. Further, it is not necessary to perform the image process before obtaining the map 62. The image process may be performed before transmitting the content.
While the image process is performed only on the faces of the passersby 67 in
As mentioned above, in addition to the effects of Embodiment 1, the Web service system 100 according to the present embodiment can protect privacy even if the photographer 8 takes an image of the passersby 67.
When the end user 6 displays a spherical image on the user's PC 16, various objects are captured in an image. It is useful if the end user 6 can display information about these objects. Accordingly, in the present embodiment, the Web service system 100 embeds information about the captured objects in the spherical image.
Further, the content creation server 12 according to the present embodiment includes the advertisement information DB 3003 in the storage 3000. In the advertisement information DB 3003, an advertisement information table is registered as shown in Table 3.
In the advertisement information table, IDs of buildings, IDs of advertisement companies, advertisement information, and numbers of provisions are registered. IDs of the buildings are identification information (an example of object identification information) to uniquely identify buildings on a map. The IDs of advertisement companies are identification information to uniquely identify advertisement companies that place an advertisement. The advertisement information describes contents of an advertisement (URL to display advertisement contents in Table 3) that an advertising company displays. The numbers of provisions indicate how many times advertisement information is selected by the end user 6.
In the following, embedding of object information is described with reference to
Further, as shown in
When the object information embedding unit 28 identifies the buildings 71 to 73 on the map, the object information embedding unit 28 obtains building information about respective buildings from the map DB 3001 and embeds the building information in the buildings of the spherical image. The building information includes a name of the building and names of tenants of the building, for example.
Specifically, a JavaScript™ code, for example, is described in content such that the building information is displayed when the end user 6 clicks on, taps, or hovers a mouse cursor over the buildings 71 to 73 in an image included in the content.
If advertisement information for the buildings 71 to 73 is registered in the advertisement information DB 3003, the advertisement information is embedded in the same manner. However, if the advertisement information is embedded, a JavaScript™ code, for example, is described in the content such that an ID of an advertising company and a fact that an advertisement is selected are reported to the content providing server 13.
The object information embedding unit 28 detects buildings from a spherical image (S10).
The object information embedding unit 28 determines a direction of each building seen from an imaging location based on directions of the spherical image (S20).
The object information embedding unit 28 determines a location of the building on the map based on the imaging location and the direction of each building (S30).
From the map DB 3001, the object information embedding unit 28 obtains building information about the building identified on the map (S40).
From the advertisement information DB 3003, the object information embedding unit 28 reads out advertisement information associated with an ID of the building (S50).
The object information embedding unit 28 embeds the building information and the advertisement information in the buildings of the spherical image (S60).
Further, when the advertisement button 75 is selected, the user's PC 16 executes JavaScript™ code, for example, and reports an ID of an advertising company and the fact that an advertisement is displayed to the content providing server 13. The browsing information obtaining unit 55 of the content providing server 13 counts a number of reports and updates the number of provisions in the advertisement information DB 3003. In accordance with this, an operator of the Web service system 100 can charge the adverting company for an advertisement cost depending on the number of provisions of advertisement.
As mentioned above, in addition to the effects of Embodiment 1, the Web service system 100 according to the present embodiment embeds object information in objects of a spherical image, so that the end user 6 can display information about the objects.
<Application Examples of Web Service System 100>
While preferred embodiments of the present invention are described with examples as mentioned above, the present invention is not limited to these examples at all. Various modifications and replacements can be added without departing from the scope of the present invention.
For example, in the present embodiment, the content providing server 13 specifies the content-using company. However, a user of content may not be specified. In other words, any consumer can transmit an image to the content creation server 12 and provide the image on her/his own Web page.
Further, in the above-mentioned examples, outdoor imaging and outdoor images are described as examples. However, locations of imaging may be indoors. In this case, imaging locations may use positional information based on iBeacon (registered trademark) in addition to IMES mentioned above.
When the Web service system 100 publishes indoor images, it is possible to create content in the same manner as outdoor images.
First, since imaging locations of spherical images are known, if coordinates on the indoor map (such as coordinates of a two-dimensional coordinate system with an origin at an upper left corner of the indoor map) are associated with positional information, it is clear which spherical image corresponds to which coordinates on the indoor map. Further, directions of north, south, east and west are attached to the spherical images, and north, south, east and west of the indoor map are usually known. Accordingly, the content creation server 12 can identify both an imaging location on the indoor map and a front side of the imaging device 11, so that an imaging direction becomes clear.
In the display field 402 for displaying the indoor map, these types of information are used to display a
Accordingly, depending on the imaging location of the spherical image being displayed in the display field 401, the user's PC 16 can emphatically display the
Further, a
Further, the content creation server 12 associates (by embedding a link, for example) in advance the adjacent spherical image having a
When the user selects the
Further, if the content creation server 12 displays the
In the same manner as outdoor images, if the user selects the
Further, in the above-mentioned examples, the photographer is assumed to walk. However, the imaging device 11 may be installed on a vehicle and take an image of surroundings from the vehicle.
Further, a place of imaging is not limited to the ground. It is possible to perform aerial imaging while falling by parachute or flying a paraglider. Further, it is possible to take an image in a pool or in the sea. In the case of taking an image in a place other than on the ground in this manner, preferably, not only longitude and latitude but also altitude information are measured and attached to an image.
Further, in the above-mentioned examples, the user's PC 16 displays content via a Web page. However, the contents may be directly displayed without using the Web page.
Further, in the above-mentioned examples, still images are mainly used as examples. However, if a movie is to be provided as content, the content providing server 13 may extract some scenes from the movie and emphatically display them on a corresponding imaging location on the map 62. The scenes may include a spherical image whose evaluation value of picture quality is equal to a standard or more. Alternatively, the person 7 in charge in the content-using company may select some scenes.
In addition, the image with data transmission unit 34 is an example of a first or second transmission unit, the image with data reception unit 23 is an example of a reception unit, the map obtainment unit 22 is an example of a map data obtaining unit, the route creation unit 21 is an example of a path information creation unit, and the content providing unit 57 is an example of a content providing unit. The URI transmission unit 24 is an example of a link information transmission unit, the Web page reception unit 51 is an example of a public information reception unit, and the Web page providing unit 52 is an example of a public information providing unit. The browsing information obtaining unit 55 is an example of a content counter or an advertisement counter, the image processing unit 27 is an example of an image processing unit, the image embedding unit 26 is an example of an image association unit, and the object information embedding unit 28 is an example of an object information association unit. The image creation unit 31 is an example of an imaging unit, the transmission and reception unit 33 is an example of a second transmission unit, the positional information obtaining unit 35 is an example of a position detection unit, and the operation input reception unit 37 is an example of a setting reception unit. The map DB 3001 is an example of a map data storage and the advertisement information DB 3003 is an example of an advertisement information storage. The Web service system 100 is an example of a service system and a method performed by the Web service system 100 is an example of a service providing method.
Further, the present invention is not limited to these embodiments, and various variations and modifications may be made without departing from the scope of the present invention.
The present application is based on and claims the benefit of priorities of Japanese Priority Application No. 2014-247212 filed on Dec. 5, 2014 and Japanese Priority Application No. 2015-227585 filed on Nov. 20, 2015, the entire contents of which are hereby incorporated by reference.
Number | Date | Country | Kind |
---|---|---|---|
2014-247212 | Dec 2014 | JP | national |
2015-227585 | Nov 2015 | JP | national |
Number | Name | Date | Kind |
---|---|---|---|
5563650 | Poelstra | Oct 1996 | A |
20130263016 | Lehtiniemi | Oct 2013 | A1 |
20130326419 | Harada et al. | Dec 2013 | A1 |
Number | Date | Country |
---|---|---|
H07-504285 | May 1995 | JP |
2014-006880 | Jan 2014 | JP |
Number | Date | Country | |
---|---|---|---|
20160165136 A1 | Jun 2016 | US |