Embodiments of the present invention will be described in detail based on the following figures, wherein:
A description will now be given, with reference to the accompanying drawings, of embodiments of the present invention.
A description will now be given of exemplary embodiments employed in the present invention.
The presentation system 10 is provided with a video camera and a microphone. The presentation system 10 captures a presenter and one or more participants, the presenter delivering a presentation to participants by using multiple materials or documents, and sends conference data including capture data to the information recording and delivering apparatus 20. Here, capture data is the data obtained by capturing the presenter and the participants by the video camera. Further, the captured data is also the data obtained by capturing sound from the presenter and the participants by the microphone. The captured data is sent to the information recording and delivering apparatus 20 via the PC 12. Here, an example is shown in such a manner that the presenter delivers the presentation to multiple participants while using multiple slides (materials) in a conference. Also, a slide data file is composed of multiple pages of slide elements. While only one presentation system 10 is shown in the figure, each presentation system 10 is set respectively in multiple conference rooms, if there are the multiple conference rooms.
The presenter operates the PC 12 and projects data of PowerPoint as a material by the projector 13. The data of PowerPoint is sent to the information recording and delivering apparatus 20 via the PC 12. The information recording and delivering apparatus 20 can acquire the material data. The slides projected by the projector 13 may be captured by a video camera, and data obtained by capturing the slides may be accumulated as material data in the information recording and delivering apparatus 20.
The information recording and delivering apparatus 20 accumulates conference data sent from the presentation system 10, and delivers data obtained by processing the accumulated conference data to the user terminal 30. The user terminal 30 has a function of receiving delivery data sent from the information recording and delivering apparatus 20 and displaying the delivery data. The user terminal 30 is composed of a portable terminal such as a notebook computer and a mobile telephone.
The characteristic information extracting portion 22 extracts characteristic information of the presenter or participants when the presenter delivers the slides based on the conference data obtained by capturing the presenter or the participants. In addition, the characteristic information extracting portion 22 utilizes an image processing technique, a sound recognition technique or the like, when extracting the above-described characteristic information. Here, the characteristic information extracting portion 22 extracts information on an intention of the presenter as characteristic information of the presenter. The characteristic information extracting portion 22 accomplishes a function by executing a given program on a computer. For example, the characteristic information extracting portion 22 extracts the characteristic information such as a slide presentation time of the presenter, an attribute of the presenter, the number of the presenters, the number of the descriptions of the slide of the presenter, a keyword mentioned by the presenter, a region of interest in the slide pointed by the presenter as the information on the intention of the presenter.
The characteristic information extracting portion 22 can determine the presentation time of the slide of the presenter by implementing sound signal processing on the data produced by the presenter presenting the slide. In addition, the characteristic information extracting portion 22 can determine an attribute of the presenter by referring to the data of the job title written in a predetermined presenters list. The characteristic information extracting portion 22 recognizes sound produced by the presenter by sound signal processing, detects that, for example, the same key word and the same sentence are repeatedly described by utilizing the results of the sound recognition, and thereby determines the number of descriptions of the slide of the presenter. The characteristic information extracting portion 22 determines a position in the slide pointed by the presenter by using the image processing technique, and thereby determines the region of interest in the slide pointed by the presenter.
For example, when presentation time of the slide is long, it is possible to understand that the slide is important as an intention of the presenter. By contrast, when the presentation time of the slide is short, it is possible to understand that the slide is not important as an intention of the presenter. In addition, when the attribute of a presenter is, for example, a corporate executive or the like, it is possible to understand that the slide is important as an intention of the presenter. Further, when the number of presenters is large, it is possible to understand that the slide is important as an intention of the presenter.
The characteristic information extracting portion 22 extracts the information on reactions of the participants as characteristic information of the participants. For example, the characteristic information extracting portion 22 extracts the number of references to the slide of the participant or an viewing rate of the slide of the participant as the characteristic information of the participant. For example, the characteristic information extracting portion 22 detects a direction of a sight line of the participant by using the image processing technique, and can determine the number of references to the slide of the participant. Also, the characteristic information extracting portion 22 divides reference time of the slide of the participant by the description time of the slide of the presenter, and can obtain the viewing rate of the slide of the participant. When the number of references to the slide of the participant is large, it is possible to learn that the participant is interested in the slide. Meanwhile, when the number of references of the slide of the participant is small, it is possible to learn that the participant is not interested in the slide. In addition, the characteristic information extracting portion 22 may extract, based on the conference data obtained by capturing the presenter or participants, a slide projection start time, a slide projection finish time, a text character string included in the slide, word appearance coordinates, the pointed number of characters, a speech segment and the like as the characteristic information. For example, it is possible to convey the intention of the slide creator with respect to the slide by means of the pointed number of characters.
The position of the region of interest in the slide pointed by the presenter is determined by a pointer position pointed by the presenter during the conference in relation to the position of the slide. Here, a description is given of an example in which the characteristic information extracting portion 22 automatically extracts the characteristic information from the conference data stored in the content DB 21. However, a user may input characteristic information onto the content DB 21 by using an input interface such as a keyboard and a mouse. The accumulation portion 23 stores the characteristic information extracted by the characteristic information extracting portion 22 in the content DB 21 in association with the conference data.
The search portion 24 searches for the conference content data stored in the content DB 21. The search portion 24 produces a search formula from a search inquiry given from the user terminal 30, executes the inquiry to the content DB 21 based on the search formula, and obtains the search result. Here, the search inquiry is given in the form of a keyword, a document, sound, an image, a combination thereof, or the like. The material content analysis portion 25 analyzes the content of the slide based on the keyword, sound, or the image included in the slide data by using the image processing technique or the sound recognition technique.
The importance level determining portion 26 determines the importance level of each slide based on characteristic information extracted by the characteristic information extracting portion 22 and the slide content analyzed by the material content analysis portion 25. Also, when the slide content is not analyzed by the material content analysis portion 25, the importance level determining portion 26 can determine the importance level of each slide based on only the characteristic information extracted by the characteristic information extracting portion 22. The importance level determining portion 26 stores such determined importance level of each slide in the content DB 21 in association with conference data.
The summary creation portion 27 processes multiple pieces of the slide data based on the importance level of each slide determined by the importance level determining portion 26. Specifically, the summary creation portion 27 creates data obtained by composing multiple slides based on the importance level of each slide. For example, the summary creation portion 27 creates data obtained by composing multiple slides based on the importance level of each slide. At this time, the summary creation portion 27 creates a composition by changing the regions on which the slides are placed in accordance with the importance levels of the slides. The summary creation portion 27 creates a stained glass like summary image when producing a summary.
Specifically, the summary creation portion 27 automatically extracts a Region of Interest (hereinafter, referred to as ROI) by using the characteristics of the slide image obtained by the search result of the search portion 24. A method of extracting ROI is described as follows. The summary creation portion 27 extracts a rectangle including a region with a high density in the slide image as ROI. Then, the summary creation portion 27 automatically extracts ROI by performing an image processing calculation such as changing the area of ROI utilizing an importance level corresponding to slide image data. This can extract ROI reflecting the slide content. Next, the summary creation portion 27 composes a stained glass like image by arranging each ROI respectively extracted from multiple slide images to produce an image. Here, the size of the composed image, the number of slides used, and the layout may be changed in accordance with a screen size of a display portion of the user terminal 30. The transmission portion 28 sends the summary data created by the summary creation portion 27 to the user terminal 30. The display portion of the user terminal 30 displays the composed stained glass like summary image as a result for a user.
The importance level determining portion 26 refers to the summary creation table 60 stored in the content DB 21, and determines the importance level of the slide based on the slide classification composed of “headline”, “browsing”, “listening”, and “intensive reading” (step S1). In the example shown in
The summary creation portion 27 determines the size of ROI of the slide image in accordance with the importance level (high or low) of the slide (step S2). The summary creation portion 27 determines the size of ROI as “small” for the slide having the importance level of “low”, and determines the size of ROI as “large” for the slide having the importance level of “high”. The summary creation portion 27 creates the stained glass like summary image in accordance with the determined size of ROI (step S3).
The summary creation portion 27 determines a size of ROT of the slide image by the importance level (high or low) of the slide (step S12). Here, the summary creation portion 27 determines the size of ROI as “small” for the slide having the importance level of “low”, and determines the size of ROI as “large” for the slide having the importance level of “high”. The summary creation portion 27 creates the stained glass like summary image based on the determined size of ROI (step S13).
If there is a fetch item (“Y” at step S24), the summary creation portion 27 determines whether or not there is a region of interest in an image. If there is the region of interest in the image (“Y” at step S26), the summary creation portion 27 sets the region of interest as initial ROI (step S27). When there is no region of interest in the image (“N” at step S26), the summary creation portion 27 extracts the initial ROI from characteristics of the image (step S28). The importance level determining portion 26 calculates an importance level score from index data (step S29). The summary creation portion 27 cuts out ROI with the size corresponding to the importance level score centering on the initial ROI (step S210), and the procedure goes back to step S23. If there is no fetch item (“N” at step 824), the summary creation portion 27 creates the summary image from such cut out ROI (step S25).
At this time, the summary creation portion 27 reflects the importance level (high or low) of each slide to the size (large or small) of ROI. This allows the user to obtain the search results of the slides related to the slide designated at step S41.
In this manner, when a user views the stained glass like summary image 74 and the icon 74a of “listening” is displayed, the user clicks the icon of “listening” with a mouse, so that the user can comprehend and listen to the content of the slide by reproducing the sound captured while the slide is being presented. Also, when a user view the stained glass like summary image 74 and the icon 74b of “intensive reading” is displayed, the user clicks the icon of “intensive reading” with a mouse, so that the user can comprehend the content of the slide S5 and intensively read sentences included in the slide S5 by magnifying the slide.
Next, a description will be given of an example of a case where a stained glass like summary reflecting the importance level based on slide classification is applied to a newspaper summary.
The material content analysis portion 25 analyzes the content of the slide on the basis of the keyword, sound, or image included in the slide data, and classifies the slide into “headline”, “browsing”, “listening”, or “intensive reading” in accordance with the analysis results. The summary creation portion 27 refers to the table for setting the maximum number of the slides 62. With respect to the slides classified into “headline”, the summary creation portion 27 includes 5 slides at the maximum in the stained glass summary image since it is easily understand the contents thereof. With respect to the slides classified into “browsing”, the summary creation portion 27 includes four slides at the maximum in the stained glass like summary image since it is slightly difficult to comprehend the content. With respect to the slides classified into “listening” or “intensive reading”, the summary creation portion 27 includes three slides at the maximum in the stained glass like summary image since it is difficult to comprehend the content. The newspaper summary 75 is thus created. In
In accordance with the above-described exemplary embodiment, ROI, namely, an attention region is extracted by utilizing meta data other than image characteristics such as an intention of a presenter, a reaction of a participant, a presentation content, and a feedback from the participant in a conference. This enables extraction based on the content. It is therefore possible to produce a summary reflecting intentions of the presenter and the participants and the importance level of the participants. Further, by composing the stained glass like summary image with such extracted ROIs, the user can easily understand a conference point, shorten the time for comprehending the content of a whole conference, and easily find a region for which the user searches. For a person who looks back on a material later, the intention of the presenter and the reactions of the participants can be conveyed so as to support the person who looks back on the material.
While the exemplary embodiment of the invention has been described in detail, the invention is not limited to the above-described exemplary embodiment, and various variations and modifications may be made without departing from the scope of the invention described in the claims. In the foregoing exemplary embodiment, as an example of a material, the electronic material such as a slide image has been illustratively described. However, the invention is not limited thereto, and can be applied to a paper material delivered in a conference. Further, in the foregoing exemplary embodiment, the example of creating a summary based on index data has been described. However, the invention is not limited thereto, and the invention can be also applied to a method of creating a material such as a method of processing multiple materials. In addition to processing the materials, processing such as sorting out slide images may be performed on the basis of the index data.
In addition, as described heretofore, the description has been given of the example of the case where when slides serving as a material include multiple elements, the importance level determining portion 26 determines the importance level for each of multiple elements, and the summary creation portion 27 processes data of the elements on the basis of the importance levels of the elements determined by the importance level determining portion 26. The invention is not limited thereto, and the invention can be applied to a case where the material does not include multiple elements. In addition to the case of creating a summary by composing the materials as described above, for example, the transmission portion 28 may deliver material data on the basis of the importance level of the material determined by the importance level determining portion 26. Then, the importance level of the material can be changed in accordance with the user who browses the data. For example, it is possible to set importance priority in advance in the order of “headline”, “browsing”, “listening”, and “intensive reading” for a user. In the afore-mentioned case, the transmission portion 28 transmits the material data to the user in accordance with the setting of the user.
Furthermore, the transmission portion 28 may select the material to be processed based on the classification results of the slide elements. For example, when a user sets that slides of “headline” and “browsing” are sent and the slide of “listening” or “intensive reading” is not sent, the transmission portion 28 selects the slide elements to be delivered on the basis of the above-described settings and the classification results of the slide elements. This allows the user to obtain a desired material. Also, the transmission portion 28 may decide a weight level in consideration of the interest of who is to receive the delivery. For example, in the case where it is just enough to know the outline, the importance level determining portion 26 sets the importance level of “intensive reading” classification to low, so that an importance level of “headline” classification becomes high By changing the importance levels classified by the importance level determining portion 26 as described above, the transmission portion 28 is capable of appropriately delivering a material to the user.
An information processing method employed as an aspect of the present invention is realized with a CPU, ROM, RAM, and the like, by installing a program from a portable memory device or a storage device such as an HD device, CD-ROM, DVD, or a flexible disc or downloading the program through a communications line. Then the steps of program are executed as CPU operates the program.
The foregoing-description of the exemplary embodiments of the present invention has been provided for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Obviously, many modifications and variations will be apparent to practitioners skilled in the art. The exemplary embodiments were chosen and described in order to best explain the principles of the invention and its practical applications, thereby enabling others skilled in the art to understand the invention for various embodiments and with the various modifications as are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the following claims and their equivalents.
Number | Date | Country | Kind |
---|---|---|---|
2006-174632 | Jun 2006 | JP | national |