Patent Application
20110199511
Present invention relates to an image photographing system and an image photographing system methods provided with a photographing function for taking pictures in an information processing terminal, such as portable telephone terminals or PDA (Personal Digital Assistance).
These years, since digital camera and personal digital assistants, such as a cellular phone with camera are spread, a lot of users are taking a photograph on various scenes. The memory card used as a recording medium of the personal digital assistant get large capacity, and low cost, it makes users can carry a lot of graphics files easily.
Thereby, for the user, there is the tendency for the management burden of the graphics file stored in the recording medium to also increase, particularly, with the information equipment which has comparatively few manual operation buttons and a small display interface such as a digital camera or a cellular phone, there was problems that complicatedness of handling for management of those files were increased, and the user's burden were increased further.
For this problem, there was some arts which lessen the user's burden in management of the graphics file by classifying a plurality of graphics files semi-automatically according to the geographical location information included in the graphics file, and distributing and storing up according to the classification (for example, refer to patent Literature 1).
[Patent Literature 1]
JP, 2003-298987, A
However, as for the above patent documents 1, since the graphics file was only classified according to the position information in which the user took a photograph, it was impossible to classify according to content of the shot. Therefore, for example, even in cases where the user takes the daily growth picture of their the pet or their child at their house everyday, and takes the daily cooking picture at the house, the pictures were classified only according to the position information as at house, in this case, all pictures were classified with the same position information and stored into the same folder, the user needed to do sub classification.
The present invention has been made in order to solve the problems as described above, and it is an object to provide an image photographing system and an image photographing system method which can be reducible of the user's burden in management of the graphics file, by considering with the facts of photography in addition to position information or the hour entry, and classifying the graphics file automatically, about the information processing terminal provided with the photographing function for taking pictures, such as the portable telephone terminal and PDA.
In order to accomplish the object as described above, the present invention provides an image photographing system comprising:
The other invention provides an image photographing method using the photographing unit for taking pictures, comprising steps as follow:
According to these inventions, since the photographing mode information at the time of the shot, the own location information, and photographing time information can be added to the image data about the taken picture, according to the attribution information, the image data can be stored for searching. For example, it is possible to presume roughly the attribute of the object, based on optical photographing modes according to the place and the time zone, such as figure photographing mode in the specific place and time zone, scenery shot mode, close-up mode of the still life shot, the taken image data can be classified according to the presumption. As a result, the burden of classifying of the image data by the user will be reduced.
Preferably, in accordance with in the above invention, the image photographing system comprising further:
In this case, since the photographing setting storage unit relates with position information and the time information and memorizes photographing mode information previously, at the time of photography, if position information and the time information are acquired, photographing mode can be set up promptly, the burden of the manipulation which selects the photographing mode by the user will be eased more.
In addition, about the photographing mode memorized by this photographing setting storage unit, in cases where the user changes the setup of the photographing condition manually, it is possible to add or to change the photographing mode with relation to the position information and the time information.
Preferably, in accordance with in the above invention, the image photographing system comprising further: a schedule memory unit for storing the position information, time information, and the type of the user's schedule as a schedule information; wherein case of referring to the mode setting, the photographing mode setting unit sets up the photographing unit into the photographing mode based on the schedule information include the position information, the time information, and the type of the schedule.
In this case, the photographing mode can be set up giving priority to the user's schedule information, the attribute of the image data can be presumed more accurately, with reflecting the user's action plan into consideration. In addition, since the schedule like the “birthday” and “traveling abroad” is recorded to schedule information, for example in addition to location information and time information, it is available of the search based on the keyword about the mode setting table T1, and the distribution of the image data can be performed accurately.
Preferably, in accordance with in the above invention, wherein the display control unit comprising the calendar display function with showing the current date, the calendar display function splits the calendar by the current date, displays a list of the image data stored in the image data accumulating unit in the split part of the past from the current date, displays a schedule information stored in the schedule memory unit in the split part of the future from the current date.
In this case, since the image data for each date can be browsed about the date part before the current date, the time and effort searched from much image data in image data retrieval can be saved, and discovery of the picture made into the purpose can be made easy. Since the action of the user for each date can also be browsed, it is possible to have the function as the diaries. On the other hand, about the date part after the current date, since it is available to record and display the user's schedule, it is possible to have the schedule function as a notebook. And since these displays are unified into the one calendar with dividing line on the current date, the GUI for the diary and the notebook can be summarized to one, the area for the display can be reduced with raising clearness, and the limited area for the display can be utilized effectively.
Preferably, in accordance with in the above invention, the image photographing system comprising further; an operation history recording unit for recording a history of the photographing mode set by the photographing mode setting unit set;
In this case, since the different guidance and operation explanation about unused photographing mode for each user are added to the schedule as the information about the example of how to use operation description, the operation lessons of ignorance or the unknown photographing function for the user can be scheduled as the user's plan. As a result, in the suitable place and the time zone, the tutorial mode is started automatically, and he can recommend using the unused function to the user.
Preferably, in accordance with in the above invention, the image photographing system comprising further:
In this case, the face of the person in the images taken by the user are detected partially, and since can be stored as useful face seals for the icon, each user can use the person's face part for various purposes. Further, because photographing mode including the detection processing and the conversion process is also memorized to the photographing setting storage unit, the automatic setup of the photographing mode at the time of photography and the unused tutorial setup are also attained.
Discussing above, in accordance with the present invention, with the information processing terminal provided with the photographing function for taking pictures, such as the portable telephone terminal and PDA. It is possible to reduce the user's burden in management of the graphics file by considering with the facts of photography in addition to position information or the hour entry, and classifying the graphics file automatically.
In the following, with reference to accompanying drawings, an embodiment of the image photographing system related to the present invention will be explained in detail.
The contents server 3 is a server which distributes additional contents and tutorial information in the case of the present embodiment, comprises the equipment, such as the Web server, for example. The content server is a document system of the WWW (World Wide Web) or the like, and is implemented as a server computer or software capable of transmitting content such as HTML (HyperText Markup Language) files, image files, music files and the like, and serves to deliver content data accumulated in it through the Internet 2 in response to that request from the personal digital assistants 1.
Moreover, All or some of contents data distributed by this contents server 3 can also be stored in the personal digital assistant 1 by installing in the personal digital assistant 1 previously, even if the personal digital assistant 1 searches for local contents data in itself, when it is not stored locally, it is possible to request distribution of the data to the contents server 3 and to download it.
In addition, the contents server 3 as this data accumulation unit has a regional image storing unit 31 to store, relating contents data such as regional image data with position information, a performs the function as the regional image distribution server which distributes the contents including regional image data unique in each region by limited distribution, the background data of the famous place and special of each region, wallpaper data, the frame, the template such as the “face set-in signboard” of the symbol characters of each region, etc. are contained in the regional image.
Moreover, the “recommended information” which is ranking about the topics (places of interest, the specialty, store information, other news) about the zone, and the map data in addition to the above regional image data are included in unique contents in each of this region.
The personal digital assistant 1 with camera is a portable telephone uses wireless communications, and communicates by the relay point and radio such as the base station 22, and can use communications services, such as the phone call and data communications with moving. As a communication mode of this cellular phone, there is the PHS (Personal Handy phone System) method in addition to the FDMA method, the TDMA system, the CDMA system, and W-CDMA etc., for example. This cellular phone has an execution function of application software in addition to the digital camera function, a GPS (Global Positioning System) function, etc., and also performs the function as a Personal Digital Assistant (PDA).
The camera function of this personal digital assistant 1 is a photographing function which photos digital images optically. In addition, the position information acquiring function is the function which acquires and records the position information which points the own position at the time of photography, as this position information acquiring function, as shown in
And, as shown in
First, the personal digital assistant 1 is provided with a photographing unit 106, a shutter control unit 107, a face detection unit 109, a photographing mode setting unit 104, an additional information processing unit 105, a photographing condition information receiving unit 102, a photographing setting storage unit 103, and a data accumulation unit 122 as the image photographing system module.
The photographing unit 106 is a module which takes the digital image optically, executes photographing processing according to the control from the shutter control unit 107, and stores up the taken picture in the data accumulation unit 122 as the image data D1. In addition, although the shutter control unit 107 executes a photographing processing with the manipulate signal based on the user's operation inputted from the operation device via the manipulate signal receiving unit 108, for example, it is possible to add an automatic photographing mode which executes a photographing processing automatically at the moment of the expression recognition unit 109a in the face detection unit 109 has recognized differential expression, such as the smiling face of the object.
The additional information processing unit 105 is a module which adds attribution information to the image data D1 taken by the photographing unit 106 as tag information, and adds a photographing mode setting when the image data D1 acquired from the photographing mode setting unit 104 is taken, and the photographing condition information (position information, time, etc.) acquired from the photographing condition information receiving unit 102 as attribution information about the facts of each image data D1. For adding this tag information, there is the way to store it directly in the image data D1, or the way to store it in another file data such as the management data (metadata) related with the image data D1.
In addition, the content (the place, the participant, the content of the event) of the schedule according to the photographing time and the attribution information of the taken image data can be related and recorded on this tag information. Further, the face of the person reflected to the picture is recognized, and it is possible to include the feature of the face in this tag information. For example, after the person specifies by referring to the face seal (face photograph information) which are set as the address book, the feature of the face may be related with the specified information, and may be recorded in the address book etc.
In addition, as for this facial recognition, it is possible to recognize an expression like a smiling face, and to include the information of the expression in the tag information. In this case, for example, to classify the pictures taken with smiling face recognition by the tag information of the “smile”, the album as a collection of the picture of those smiling faces is automatically generable.
In addition, this tag information can be displayed in a list for each picture, can be edited like a modification, deletion, and addition by user's operation. And the attribution information included in tag information is applicable to an album classification of the pictures, an e-mail automatic generation for attaching and sending to the E-mail, etc.
The photographing mode setting unit 104 is a module which sets up the photographing mode of photography by the photographing unit 106. As this photographing mode, there is setting of shutter speed, exposure (iris) and the focal length, the necessity of the flash, the filter, etc., in detail, it has a portrait mode (with adjusting the iris, and a little shading off the background so that the person may be looked nice), a scenery mode (with adjusting a focus equally from the short distance to the long distance, the commemorative photos with a plurality of objects are included), a close-up mode (with close up to the objects), a sport mode (by the high speed shutter: continuous shooting etc. are included), night view portrait modes (for example a person photography in the night view as the background, with adjusting a shutter speed and an iris, to the light volume of the narrow area like a center object), strobe light inhibit mode (photography in the open air) etc. And setting of image size, and image quality (resolution) and the preservation destinations (the body or the archive medium) is included in basic setting included in this photographing mode.
In addition, with this photographing mode, in order to make the user's understanding easy, for example, the suitable selection (put together) pattern of photographing settings and functions may be displayed as a kind of camera, and the user may be made to select them. For example, a mode in which the user can set up all the photographing functions mounted self can be displayed as a “dainty pro camera”, a mode in which photography by high sensitivity setting in night or at indoor can be displayed as a “high sensitivity camera”, a mode in which attach the frame or the character and the drawing are added to the picture after photography with the function which amends the skin-colored can be displayed as a “Print Club camera”, a mode which transforming the taken picture or mixing other pictures can be displayed as a “party camera”, a mode in which effect processing, such as the sepia tone, the oil painting touch and the suspected fish-eye lens, and the filter effect, can be displayed as a “art camera”, a mode in which image analyses, such as analysis of the two dimensions bar code or QR Code, character recognition functions, such as OCR, the facial recognition function, can be displayed as a “search camera”, and the mode in which taking of movies and uploading the taken movies/pictures can be displayed as a “video camera.” As other photographing modes, the mode which addition comments to the taken pictures, and uploading to the Web site on the Internet can be displayed as a “blog camera”, as the mode for making the attention to the camera of the animal by outputting the voice of the animals, such as a dog and a cat can be displayed as a “the pet camera”, a mode adjusting a short-distance shot for a still lives, such as foods can be displayed as the “cooking camera.”
In addition, in case of the reference the mode settings, the above photographing mode setting unit 104 can adjust setting of the photographing mode to the photographing unit 106 based on current time, position information, time information and the type of the schedule which are included in schedule information, and can also display the message which recommends setting of better photographing mode. For example, if current time is during the travel record on the schedule, the portrait mode is set as the default setting, if it is during the party, the “party camera”, the “Print Club camera”, etc. are set up preferentially, or a message which recommends to use of that is outputted. When in the night time, the above the “high sensitivity camera” may be recommended. Further, with setting of this photographing mode, the message which recommends other functions, such as recommending to use of the regional frame and to use of the navigation function by GPS, may be outputted.
The photographing setting storage units 103 are a memory storage, such as the nonvolatile memory which stores of the table data for selecting the photographing mode according to the place and time, when the photographing mode setting unit 104 sets up photographing mode. In detail, it has stored the table data which related position information and time information, and photographing mode as the mode setting table T1, refers to the mode setting table T1 based on position information and time information according to the request from the photographing mode setting unit 104, and reads the photographing mode which should be used for the shot, and sends it out to the photographing mode setting unit 104. The photographing mode setting unit 104 refers to the mode setting for according to the photographing condition information which the photographing condition information receiving unit 102 acquired, and sets up the referred photographing mode to the photographing unit 106.
In the case of the present embodiment, when taking pictures, the photographing condition information receiving unit 102 has an acquiring positional information unit 102a which acquires the position information which points the own position, and a clock part 102b which acquires the current time information as photographing condition information, at the time of completing of the shot, and inputs each data acquired from the acquiring positional information unit 102a and the clock part 102b as photographing condition information into the additional information processing unit 105, responds to the request from the additional information processing unit 105.
The acquiring positional information unit 102a is a module which acquires and records the position information which points the own position when the shot, for example, it detects the position of the self-opportunity with the signal from the satellite 21 such as GPS, and it detects the position with the radio field intensity from the base transceiver station 22 of the cellular phone, etc. The clock part 102b is a module which measures current time, can manage the time zone according to the position information acquired by the acquiring positional information unit 102a, and can also display the time which considered the time difference etc.
The data accumulation unit 122 is memory storage which stores up the various data, The taken image data D1, the data for processing acquired from the contents server 3, the composite image data with which it was mixed of these, the editing data (for example, face seal) which processed it into each image data D1, etc. are stored up.
In this data accumulation unit 122, the image data D1 is distributed and stored in a plurality of folder 122a-c according to the attribution information (tag information) added by the additional information processing unit 105. In addition, In the case of the present embodiment, even though it distributes to the folder according to tag information and stored up the picture, the present invention is not limited thereto, but also, the actual storing position is used as the ordinary folders (for example, “My Pictures” etc.) appointed by basic setting, and it is preferred at in the case of the list display or the album display to execute in dummy the distribution based on the tag information added to each picture, and to indicate by the classification.
About this folder 122a-c, as shown in
This photograph album is a view which classified the picture automatically according to tag information, and each picture is classified according to each attribute of picture included in tag information. In addition, the same picture seems to be stored in a plurality of folders (album), because same image data are repeatedly classified over a plurality of albums by the number of the tag information.
As a kind of this photograph album, it can classify by the item in the tag information, for example, pictures taken during the category (“travel” etc.) of the schedule can be collected, pictures can be collected by each photographing mode (kind of camera) , pictures can be collected by the person in the picture, or by his attributes (for example, “family” etc.), and pictures can be collected by date (specific dates, such as the birthday) in the calendar.
The face detection unit 109 is a module for recognition person's face parts by finding out the specified geometric shape formed of the eyes, the nose, the mouth, etc., and computes the coordinate's position of the detected face part in base images. This face detection processing is started before photographing operation, as shown in
This face detection unit 109 has an expression recognition unit 109a which recognizes the specified expression such as the smiling face of the person who is the target of the shot, and is provided with the function to recognize the face part and its expression in the monitor picture at the shot, and to input the recognized result into the shutter control unit 107. The shutter control unit 107 inputs the control signal which makes the photographing unit 106 execute shutter operation according to the inputted recognizing result. By containing the information about the feature and expression of the face which have been recognized by this Face detection unit 109 and expression recognition unit 109a to the tag information, each picture can be classified according to the person in the picture, or can be classified according to the type of expression. About the feature of the face, for example, the person may be specified by comparing the face seal registered into the address book.
The personal digital assistant 1 is provided with an editing processing unit 110 and an editing data generating unit 111 as a module of the editing processing system.
The editing processing unit 110 is a module which executes various editing processing according to the operator's handling, can execute the image synthesizing process which composes the pictures, such as the frame image and the background image, to the taken base images in addition to photo retouch processing which adds the drawing on the picture with the touch pen etc., or fills in the character etc. in conjunction with GUI. This editing processing unit 110 is provided also with the function to display the list of the tag information linked with each picture, and to edit it by addition, deletion, modification, etc.
In addition, this editing processing unit 110 is provided also with an E-mail generating unit 110a which generates the E-mail and sends via communication I/F 101.
This E-mail generating unit 110a is a function which attaches the picture stored up in the image data accumulating part 122, and the taken picture D1 to the E-mail, and sends it, and can generate automatically the E-mail include a necessary information based on the tag information of the image data. For example, the function recognizes the person contained by tag information from the picture, searches the person from the address book, and fills the mail address and person's name found out from the address book into the E-mail as a destination address automatically. In addition, the function generates the text by extracting the place, date and the details of the event when the picture was taken, from the information contained by tag information or the related schedule information (hysteresis information), and may use the text as the body test and the title of the E-mail.
The E-mail generating unit 110a is provided also with the function which inserts the face seal as a pictorial symbol while editing the mail text. For the function, for example, as for the dictionary conversion function, the face seal insertion function is related with the person's name, and when the person's name is written in the text, it may change to or insert the face seal automatically.
In addition, E-mail generating unit 110a is provided also with a function which inserts the content of the schedule to the E-mail automatically in the case of generating an E-mail. For example, the function searches the address book using the name of the person related to the schedule written on the schedule, and fills in the person's mail address as an address, and makes the title and the body text of the E-mail by using the content of the selected event from the schedule.
The editing data generating unit 111 is a module which generates the editing data which is the result of the editing operation by the editing processing unit 110, and the generated data is stored in the data accumulation unit 122. This editing data generating unit 111 also stores up an edited image data into the data accumulation unit 122, with the tag information relevant to it. The editing data generating unit 111 is provided with an image synthesis unit 111a, this image synthesis unit 111a sets up the picture taken by the photographing unit 106 as the base images D11, and composes other image data D12 and the base images D11. The compositing process by this image synthesis unit 111a is also controllable by the editing operation to the above editing processing unit 110.
In addition, as shown in
In addition, the user can also create the face seal manually purposely, moreover, whenever the face part is taken, the face seal is created and stored automatically.
In addition, making of this face seal is faced, as shown in
Moreover, the face seal created by this image synthesis unit 111a can be related with the person registered into the address book, and can be stored, thereby the person's face seal can be used for the manipulation and data relevant to each one. For example, the face seal can be displayed as an index of the address book, or can be icon displayed as an index of the photograph album. In addition, this face seal can also be used as a pictorial symbol signifying each one, when creating the E-mail, if the person name registered into the address book is written in the E-mail, the person name is detected automatically and it may be made to add the face seal in the text.
As the module of the user interface system, the personal digital assistant 1 is provided with a manipulate signal receiving unit 108, a display control unit 112, an image display unit 113, a schedule memory unit 121, an operation history recording unit 123, a tutorial storage unit 124, and a tutorial setting unit 114.
The manipulate signal receiving unit 108 is a module which acquires the manipulate signal from the operation device based on user's operation, and inputs the executive instruction according to the manipulate signal into Shutter control unit 107 or the editing processing part 110. At this point, as this operation device, the touch panel, the manual operation button, the acceleration sensor, etc. are included. This manipulate signal is sent out to each module as the operation target, is sent out to the operation history recording unit 123, and is stored as the log data D3.
The display control unit 112 is a module which creates the image data for displaying on the image display unit 113 as GUI, when in taking pictures, it displays the finder image caught by the photographing unit 106, and when in editing the picture, it displays change of the pictures, such as the icon based on the user's operation signal acquired by the manipulate signal receiving unit 108. Moreover, as the way of the icon display by this display control unit 112, as shown in
The display control unit 112 is provided with a slide show function which plays back pictures stored in the image data accumulating part 122 order by and continuous in order and continuously. In this slide show function, it is possible to playback order by photograph album, and various rendering can be carried out when playing back this slide show.
In this slide show rendering, for example, it is possible to display together with the information relevant to a tag information (photographing location, photographing time, photographing mode, the schedule of the time, and the regional information about the location, etc.), by comparing the tag information used as the basis of the classification of the photograph album. In detail, when the photograph album is related to the travel, the moving track based on the regional information on the travel destination, the map, and the navigation history, etc. may be displayed as an introduction of the slide show. And also it is possible to make the display order of the picture in the slide show by photographing time, and it is possible to execute produce according to photographing mode, for example, when photographing mode is the Print Club camera, frames of the slide show is changed like the Print Club note.
In addition, the display control unit 112 in this embodiment is provided with a function which displays the attribution information of the data stored in the data accumulation unit 122 as the face seals 501a-501c. For example, incase the person in the picture taken by the camera is recorded as attribution information (tag information), the face seal is displayed as an icon for signifying the person.
Relation of this tag information and the face seal, for example, it is possible to use the data in the address book. Accordingly, it sets up on the person registered into the address book, and the taken image and the face seal are matched and recorded for the person.
In addition, the display control unit 112 is provided also with a function which searches and reads out the data stored into the data accumulation unit 122, the corresponding data can be read by operation selection on calendar UI (
The schedule memory unit 121 is a memory storage which stores the position information, the time information, and the schedule type about the user's schedule as the schedule information D2. This schedule information can be customized by each user, and is displayed on calendar UI402 as shown in
In addition, as shown in
As other examples of this schedule information D2, as shown in
The tutorial storage unit 124 is a memory storage which relates the information about the example of how to use of photographing mode with location information and time information, and stores them as the tutorial information 124a. In the case of the present embodiment, this tutorial stored in Tutorial storage unit 124 is the teaching materials for explaining how to use about the photographing function, etc., and is contents data including the animation, the still picture, the writing text, and the script for executing each functional module of the personal digital assistant 1 (program).
As this tutorial, the explanation about the function of recommendation and the display of the information data (recommendation research) distributed from the Internet in addition to the information about the example of how to use of photographing mode is included. As this recommendation research function, for example, and may measure the current position by the position information acquiring means, such as GPS, and may display the area information of the recommendation spot, store information, gourmet information, the specialty, etc., around the current position. This area information may be the regional information distributed from the contents server 3, and it is preferred to be displayed as ranking by the importance (the degree of recommendation). As this ranking, picture show record, CD and the number of downloads of the ring tone, television viewership, and the retrieval number may be included.
Further, this tutorial may select the function suitable for the situation according to the contents (the place, the date, and the contents of the event) of the schedule registered into the schedule, and may explain the example of use which recommends it. As the information which tied up with this schedule, the related information data may be collected from the Internet based on the contents of the schedule, and it may display as push information.
This tutorial may recommend using other functions mounted in the personal digital assistant, and may explain the how to use it. For example, the operating procedures, such as the purchase of the ticket, the purchase of goods, and download of contents, are explained by the communication function and browser function with the personal digital assistant. The information about these tickets, and goods and contents may be made to link with the ranking display function, and may be displayed as a result of the recommendation research.
In the case of the present embodiment, when a script of this tutorial is executed, as shown in
In addition, when the tutorial is the above recommendation research, the browser function is started automatically, it accesses the local information data which accessed the Internet etc. or was previously stored into the body, and executes displaying of advice of the recommendation function, and explanation and pertinent information (ranking information etc.).
In the example as shown in
And the tutorial analyzes the history of user's operation, it will be executed to the unused function, and the execution is set up by the tutorial setting unit 114.
In addition, in execution of the tutorial, the content of the registered schedules are watched periodically, and if the time registered into the schedule comes, the alert of the schedule time will be displayed with explanation of the recommendation function, and the relevant information by recommendation research. Further, at the time of execution of this tutorial, current time and the current position may be acquired from the photographing condition information receiving unit 102, and explanation of the function according to the situations (the time, the place, and etc.) and the relevant information by recommendation research may be displayed.
The tutorial setting unit 114 is a module which adds the location information and time information included in the tutorial information 124a acquired by the tutorial receiving unit 114a, and the type of the schedule according to the example of how to use, to the schedule information as the user's schedule. In the case of the present embodiment, it has a tutorial receiving unit 114a and an operation history analyzing unit 114b. The operation history analyzing unit 114b is a module which analyzes the history of the photographing mode and extracts unused photographing mode. The tutorial receiving unit 114a is a module which acquires the tutorial of the unused photographing mode and the other functions, by referring the tutorial information 124a, based on the unused photographing mode extracted by the operation history analyzing unit 114b, the content of the schedule registered into the schedule 121, and the function according to the situation acquired by the photographing condition information receiving unit 102.
The operation history recording unit 123 is a memory storage which records the history of the photographing mode set up by the photographing mode setting unit, as the log data D3, the history of the mode setting based on user's operation is stored as the log data D3.
The image photographing method of the present invention can be enforced by operating the image photographing system which has the above composition.
To begin with, while acquiring the own location information and the current time information periodically by loop processing (S101 and S102), with reference to the schedule information D2, checking whether it is in the place and time as the schedule, or at unexpected place and time, checking whether the user started the handling selfish (S103 and S104). It follows, it will be in the waiting state until the user operates it, or until required time comes (“N” in S103 and S104).
In addition, the schedule in this step S104 may be the starting time of the tutorial for the lesson operation mentioned above. That is, to the tutorial information 124a, the information about the example of how to use of the photographing mode relates with location information and time information, and is memorized, by adding the location information, the time information and the type of the schedule according to the example of how to use included in this tutorial information 124a, to the schedule information as the user's schedule, the operation lessons of unused photographing mode will be scheduled as the user's plan.
And in case the user starts photographing operation selfish, before the time of the schedule came (“Y” in S103), the shot is executed by the photographing mode selected by the user (S105 and S107), and the photographing mode is added to the log data D3 (S108). In addition, in the case of the present embodiment, previously, the table data relates the location information the time information, and the photographing mode is store into the photographing setting storage unit 103 as the photographing mode setting T1, in the photographing mode setting step (S105),according to the photographing condition information acquired by the photographing condition information acquiring step (S101), the mode setting T1 is referred and the referred photographing mode T1 is set up to the photographing unit 106.
On the other hand, in Step S104, when the scheduled place and time arrive (“Y” in Step S104), photographing operation is urged by selecting photographing mode automatically (S106), if photographing operation is executed (S107), and its photographing mode are added to the log data D3 (S108). In addition, the location information, the time information, and the type of the schedule about the user's schedule are recorded on schedule information, in the photographing mode setting step (S106), when mode setting is referred, the photographing mode to the photographing unit is set up, based on the location information, the time information, and the type the schedule included in the schedule information D2. Since the content of the schedule, for example, like the “birthday” and “traveling abroad” is recorded in the schedule information D2, more exact photographing mode can be set up by using the type of the content of the schedule as a keyword.
And according to attribution information including the place, time, and photographing mode of the shot, the image data D1 will be distributed and stored into each folder 122a-c of the data accumulation unit 122. The image data D1 stored in each of this folder 122a-c can be displayed as the list in calendar UI with the icon display according to the attribution information of each picture. In detail, in this calendar display, it splits the calendar by the current date, displays a list of the image data D1 stored in the image data accumulating unit 122 in the split part of the past from the current date, displays a schedule information stored in the schedule memory unit 121 in the split part of the future from the current date. As shown in
In addition, by this shot, the face seal will be generated by changing the person's face part into the image data of specific shape, this face seal can be freely pasted in calendar UI as an icon image.
The face seal can be related with the tag information of the picture, the address book, the schedule, etc. For example, it is possible to relate with the attribution information of the taken image data, and the content (the place, the participant, the content of the event) of the schedule according to the photographing time, and it possible to record them on tag information. Further, by recognizing the face of the person in the picture, the feature of the face can be included in this tag information. About the feature of this face, for example, by referring the face seal set in the address book, the person can be specified, and it can be related with the address book and can also be recorded. In addition, in case of this facial recognition, expression, such as the smiling face, is recognized and the recognized expression can be included in the tag information. In this case, for example, by classifying the picture taken by recognition of the smiling face, the album collected the pictures of the smiling face, will be generated automatically based on the tag information of the “smile.”
Further, the image synthesis unit 111a downloads an indigenous attached data in the specific region by the data transmission and reception with the contents server 3, and, is provided with the function which composes the indigenous image data in the specific region to the image data of the face seal, this image compositing function can be used together at the time of generation of the face seal.
First, the image is taken by the photographing unit 106 (S201). When taking a photograph, in the case of the present embodiment, “automatic photographing mode” which always watches the finder image by the photographing unit 106 may be executed.
With this mode, in the finder image taken by the photographing unit 106, by the expression recognition unit 109a performs the expression recognition for recognizing a specified expression, for example, such a person's smiling face as a target for photography, and at the moment of recognizing the smiling face, photographing processing is executed automatically.
In Step S201, the image data taken by the photographing unit 106 is recorded on the data accumulation unit 122 as base images (S202), and face detection is performed to base images (S203). When in this detection of face part, the face detection unit 109 computes the coordinates position in the base images of the detected face part.
In addition, by this photographing operation, and while the own position information at the time of photography is acquired and recorded, this position information is sent to the contents server 3 (S204). With the contents server 3 which received this position information (S301), based on the position information at the time of the received photography, the regional image data related with position information is searched for from the regional image storage database (S302), and the selected image is replied (S303). In addition, if regional image data is previously installed in the personal digital assistant 1 and is stored in the personal digital assistant 1, it is not necessary to distribute the regional image data from this contents server 3. It follows, in the step S204 and S301-303, the personal digital assistant 1 searches for the contents data in the local, and if it does not store in the local, then the steps may be executed.
The “recommended information” which carried out ranking of the topics (places of interest, the specialty, store information, other news) about the region in addition to the above regional image data as contents unique with the every place region distributed from this contents server 3, and the map data are included, and these information are useful to the tutorial, the photograph album of the taken image, and the rendering of the slide show by image restoration, etc.
The camera side which received regional image data, or read the data in the local (S205), and acquired regional image data composes the acquired regional image data D12 to and the base images D11 (S206). In case of this picture composition, the image synthesis unit 111a is composite of the region image data D12 to the base images D11 based on the coordinates position of the face part found out by the face detection unit 109.
This composite edit can be operated with the operation devices 116, such as GUI and the ten key, as shown in
Thus, the editing operation such as other photo retouch processing can be done to the stored image data taken or edited (S207). If editing work is completed (“N” in S208), then the edited image data will be stored (S209).
When storing this image data, in the case of the present embodiment, it is related with location information and stored in the data accumulation unit 122. Then the location information of the stored pictures are generated as index information data (tag information) (S210), and are displayed as icons on GUI. And, by selecting the index (tag information) displayed on GUI, it is possible to search, to read or to start the selected picture, the related data, or the program can be searched from the data accumulation unit 122. Thus, in GUI, it is possible to read and to browse the generated face seal, by selection operation, such as touch operation to the index icon.
According to the embodiment explained above, about the taken picture, since the photographing mode information of the shot, the own location information, and photographing time information can be added to the image data D1, according to the attribution information, the image data D1 can be distributed and stored automatically. For example, it is possible to presume roughly the attribute of the object, based on optical photographing modes according to the place and the time zone, such as figure photographing mode in the specific place and time zone, scenery shot mode, close-up mode of the still life shot, the taken image data D1 can be classified according to the presumption. As a result, the burden of classifying of the image data by the user will be reducing.
In addition, in the case of the present embodiment, when referring mode setting, in case schedule information exists, since the photographing mode is set up to the photographing unit based on the location information, the time information and the type of the schedule included in this schedule, photographing mode can be set up giving priority to the user's schedule information, the attribute of the image data D1 can be presumed more accurately, with reflecting the user's action plan into consideration. And, since the schedule like the “birthday” and “traveling abroad” is recorded to schedule information, for example in addition to location information and time information, it is available of the search based on the keyword about the mode setting table T1, and the distribution of the image data D1 can be performed accurately.
In the case of the present embodiment, since calendar UI is split by dividing line on the current date, displays a list of the image data D1 stored in the image data accumulating unit in the split part of the past from the current date, and displays a schedule information stored in the schedule memory unit in the split part of the future from the current date, about the past, it is possible to have the function as the diaries with the images data D1 for each date, and about the future, it is available to record and display the user's schedule, and it is possible to have the schedule function as a notebook. And since these displays are unified into the one calendar with dividing line on the current date, the GUI for the diary and the notebook can be summarized to one, the area for the display can be reduced with raising clearness, and the limited area for the display can be utilized effectively.
Further, in the case of the present embodiment, since the different guidance and operation explanation about unused photographing mode for each user are added to the schedule as the information about the example of how to use operation description, the operation lessons of ignorance or the unknown photographing function for the user can be scheduled as the user's plan. As a result, in the suitable place and the time zone, the tutorial mode is started automatically, and he can recommend using the unused function to the user.
In the case of the present embodiment, the face of the person in the images taken by the user are detected partially, and since can be stored as useful face seals for the icon, each user can use the person's face part for various purposes. Further, because photographing mode including the detection processing and the conversion process is also memorized to the photographing setting storage unit, the automatic setup of the photographing mode at the time of photography and the unused tutorial setup are also attained.
D1—Image data
D2—Schedule information
D3—Log data
T1—Mode setting table
1—Personal digital assistant
2—Internet
3—Contents server
21—Satellite
22—Base transceiver station
102—Photographing condition information receiving unit
102
a—Position information receiving unit
102
b—Clock unit
103—Photographing setting storage unit
104—Photographing mode setting unit
105—Additional information processing unit
106—Photographing unit
107—Shutter control unit
108—Manipulate signal receiving unit
109—Face detection unit
109
a—Expression recognition unit
110—Editing processing unit
110
a—E-mail generating unit
111—Editing data generating unit
111
a—Image synthesis unit
112—Display control unit
113—Image display unit
114—Tutorial setting unit
114
a—Tutorial receiving unit
114
b—Operation history analyzing unit
116—Operation device
121—Schedule memory unit
122—Data accumulation unit
122
a-c—Folder
123—Operation history recording unit
124—Tutorial storage unit
124
a—Tutorial information
| Number | Date | Country | Kind |
|---|---|---|---|
| 2008-270360 | Oct 2008 | JP | national |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/JP2009/068076 | 10/20/2009 | WO | 00 | 4/19/2011 |
