Method and apparatus for automatic annotation of recorded presentations

Abstract

Automatic annotation of recorded lectures includes the annotation of recorded presentations as presented on a computer screen during a presentation. Such annotation is accomplished with metadata that indicates the original slide and the time in the presentation that the slide was presented. Registration of the presentation tags the presentation with this metadata, which uniquely identifies the presentation and each slide. The metadata enables future searching of the recorded presentation for the annotation, such that segments of recorded presentations in which the slide was presented can be recalled. The searching function can also be used in an integrated system with other applications to enable recall of segments of relevant recorded presentations.

Description

BACKGROUND OF THE INVENTION

1. Technical Field

The invention relates to presentations. More specifically, the invention relates to the automatic annotation of recorded presentations.

2. Description of the Prior Art

Presentations are commonly delivered as a sequence of slides presented on a computer screen. These presentations may be recorded for playback as video and audio electronic content, such as in teaching applications or computer-assisted learning applications. In certain cases, a lecture is recorded showing the slides that are presented by the lecturer. This is beneficial for students who want to review a lecture.

Electronic content, such as documents, audio, or video, is often indexed for searching and retrieval. Additionally, the content may be enhanced by tagging it with metadata. This metadata enriches the search functionality, enabling users to search for content based on the metadata. Users of the content can search for the content using metadata terms, such as keywords. Metadata may be authored and added to the electronic content manually. Because the authoring of relevant metadata is a time-consuming process, metadata may also be collected automatically through knowledge management systems.

In multimedia content, such as audio or video content, user-authored annotations and metadata may be temporally synchronized to the content. This metadata is linked to specific time segments of the content. Searching multimedia for such time-synchronized metadata allows the retrieval of specific segments of content. The annotations or metadata may be retrieved and viewed together with the specific time segment of content.

Time-synchronized annotations may be used with recorded presentations to enable time-synchronized capture and review of user-authored notes associated with the presentation. This enables time-synchronized review of recorded lectures and course, with synchronized playback of the recorded notes. Time-synchronized metadata may be used to synchronize a recorded presentation or lecture with a set of source slides. A supplemental metadata file synchronizes the multimedia sources, such as the video of a presentation, with a slide set.

Unfortunately, providing manually authored time-synchronized annotations of the recordings to indicate the source slides in a recorded presentation requires considerable user effort and is a very time-consuming process.

SUMMARY OF THE INVENTION

An embodiment of the invention provides a method and apparatus that generates metadata automatically, and that allows the association of time-synchronized annotations along with recorded presentations and with the original source slides. Embodiments of the invention provide for such a solution to work equally well for multiple recorded presentations derived from the same source slide set. Embodiments of the invention also provide for such a solution to enable searching of the recorded presentations based on the original source slides.

A presently preferred embodiment of the invention provides a method and apparatus for automatic annotation of recorded lectures. One embodiment includes the annotation of recorded presentations, as presented on a computer screen during a presentation, with metadata indicating the original slide and the time in the presentation that the slide was presented. Registration of the presentation tags the presentation with metadata, which uniquely identifies the presentation and each slide. The metadata enables future searching of the recorded presentation for the annotation, such that segments of recorded presentations in which the slide was presented can be recalled. The searching function can be used in an integrated system with other applications to enable recall of segments of relevant recorded presentations.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is schematic diagram of a presentation showing a sequence of slides, where each slide has a unique identification (UID) according to the invention;

FIG. 2 is a schematic diagram of a time line for a presentation, in which different sequences of slides are used to make the presentation according to the invention;

FIG. 3 a schematic diagram showing a sequence of UIDs and times for lecture L1 and a sequence of UIDs and times for lecture L2 according to the invention;

FIG. 4 is a flowchart showing the registration of a sequence of slides as presented by a lecturer during a presentation according to the invention;

FIG. 5 is a flowchart showing the insertion of feedback unique to a student corresponding to a lecture tagged in according to the invention; and

FIG. 6 is a block schematic diagram of a machine in the exemplary form of a computer system within which a set of instructions may be programmed to cause the machine to execute the logic steps of FIGS. 4 and 5 according to the invention.

DETAILED DESCRIPTION OF THE INVENTION

An embodiment of the invention provides a method and apparatus for automatic annotation of recorded lectures. Embodiments of the invention allow the annotation of recorded presentations as presented on a computer screen during a presentation, in which metadata indicates the original slide and the time in the presentation that the slide was presented. Registration of the presentation tags the presentation with metadata, which uniquely identifies the presentation and each slide. The metadata enables future searching of the recorded presentation for the annotation, such that segments of recorded presentations in which the slide was presented can be recalled. The searching function can be used in an integrated system with other applications to allow the recall of segments of relevant recorded presentations.

In an embodiment of the invention, a publisher of a textbook provides a version of at least a portion of the textbook or material that is related to the textbook topic in the form of a presentation on tangible media that is readable by a computer. The presentation also contains one or more slides, also referred to herein as foils. Each slide may be used by a lecturer when the lecturer discusses a respective topic covered by the text book. FIG. 1 is a schematic diagram showing an exemplary presentation 100, also referred to herein as a foil deck or slide deck, comprising a sequence of slides 110, where each slide has a unique identification (UID) 122. A slide, for example slide 110-i, provides the viewable information of the slide. This may include not only the portion of the slide that is viewable by the lecturer and the students, but may also include footnotes that may be used by the lecturer in the course of presenting the specific slide. The slides 110 may be shown in any sequence as desired by the lecturer, which is often the case. Each slide has a UID that uniquely identifies the slide relative to any other slide. The UID is coded as part of the metadata of the slide, for example, metadata 120-i for slide 110-i. Each metadata 120 comprises various types of information including, but not limited to, the UID. For example, the UID 122-i is the unique identification of the slide 110-i, and is registered by the presentation system whenever the slide is shown, as explained in greater detail below. The presentation may be stored in tangible computer readable medium that includes, but is not limited to, a computer memory, computer disk, remote memory, networked attached storage (NAS), remote memory, compact disk (CD), digital video disk (DVD), disk-on-key, read only memory (ROM), non-volatile memory (NVM), and the like.

As noted above, different lecturers who use the same presentation may show the presentation in different sequences of slides that best fit their individual presentation style. FIG. 2 is a schematic timeline 200 of a presentation, in which different sequences of slides are used by different lecturers according to the invention. The lecturer making lecture L1 shows the presentation in the following sequence: slide 2, slide 3, slide 2, slide 4, slide 5. The lecturer making lecture L2 uses the same presentation in a different order: slide 1, slide 2, slide 3, slide 18. According to the invention, as the slides are presented, the computerized system (FIG. 6) registers in memory for each lecture L1 and L2, the respective sequence of UIDs of the slides used, as well as the relative time from the beginning of the lecture when the slide is shown. A sequence in memory for lectures L1 and L2 is shown in FIG. 3. The sequence recorded allows the presentation of the same sequence of slides to a user of the system, for example a student, when accessing the lecture in real-time, i.e. as the lecture takes place, or when accessing the lecture at another time for the purpose of, for example, working on the student's homework or studying for a test.

FIG. 4 is a flow chart 400 that shows the registration of a sequence of slides as presented by a lecturer during a presentation.

In step S410, the lecturer logs-on to the system, which is enabled to register the sequence of slides presented in accordance with the invention.

In step S420, the specific lecture, e.g. lecture L1, is selected from, for example, a list of possible lectures that the lecturer plans to make. In another embodiment, the lecturer determines which lecture is to be presented, for example lecture L1.

In step S430, the system presents a slide, as selected by the lecturer, from the tangible computer readable medium on which the presentation is present, for example a slide from the lecture L1. In one embodiment, the lecturer determines the sequence of the slides to be presented and only those slides are presented in the sequence predetermined by the lecturer.

In step S440, the system registers the slide UID and the time stamp of the starting time of the current presentation of the slide.

In step S450, the system checks whether a following slide is to be presented and, if so, execution continues with step S430; otherwise, execution continues with step S460, where the sequence of slides presentation, for example the sequence shown with respect to lecture L1 shown in FIG. 3, is stored in memory of the system. Thereafter, execution terminates.

As a result, a sequence is stored in memory that enables the recreation of the slide show in exactly the same sequence as presented by the lecturer. It further enables the use of a single presentation, regardless of the number of lecturers using the presentation or the specific sequence the lecturers choose to present the presentation. When a viewer, for example a student, wishes to view the presentation the specific sequence that is unique to the lecturer in the class to which the student is registered is presented. In addition, an audio track of the presentation may be used, providing the student with the correct sequence of the presentation slides, as well as an audio recording of the accompanying lecture. In another embodiment, a video track of the presentation is also provided. Although the slides may be seen in a video recording, it is advantageous to have the sequence of slides available because the display of the specific slides is usually clearer when a full screen display of the slide is available.

FIG. 5 is a flow chart 500 that shows the insertion of feedback unique to a user or specific viewer, for example a student, corresponding to a lecture tagged in accordance with the invention.

In step S510, the user, for example a student, logs on to the system.

In step S520, the user identifies the lectures from, for example, a list of lectures available to the user. In a case of a student, the list may be the list of classes to which the student is registered to attend, and their respective lectures.

In step S530, responsive to the user's selection of a lecture, for example lecture L1, the lecture is presented in sequence using the stored presentation sequence, as shown, for example, in FIG. 3.

In step S540, it is determined if the user wants to provide feedback and, if so, execution continues with step S550; otherwise, execution continues with step S560.

In step S550, the system registers the user's feedback. The user feedback may be received in connection with presentation of an overlay over the slide that allows the user to provide feedback, for example, by depressing one of a plurality of buttons, which may, for example correspond to “not understood,” “review later,” “repeat,” “more information,” and others. The user's feedback may be added to the lecture as tags of user respective metadata, for example, lecture L1. On the next review of the lecture L1 by the same user, the feedback information may be displayed to encourage the user for additional action. In one embodiment of the invention, rather than displaying the feedback buttons on the display used by the user to display the presentation, other input devices are used. For example, a user may use a mobile device, such as a cellular phone or a PDA, to communicate with the system under the user's log on identification, and then associate the feedback with the user's metadata tags as explained above. In yet another embodiment of the invention, additional information is stored, for example, information regarding student identification and lecture identification, in addition to the UID and the timestamp.

In step S460, it is determined whether to continue the viewing of the presentation and, if so, execution continues with step S540; otherwise, execution terminates.

In one embodiment of the invention the steps described in connection with FIG. 5 are performed in real-time, i.e. as the lecture takes place. When using a registered annotation device, such a PDA, a cellular phone, or other mobile devices that are enabled to communicate with the system, the user is identified by the system. A lecturer's schedule is available on the system and therefore a unique identification between the lecturer and a lecture is determined by the system. Similarly, with respect to a student, the student's schedule is available for access by the system and therefore, when a user provides a feedback as discussed with respect to FIG. 5, the user is uniquely identified with respect to the user's schedule and the specific lecture attended by the student. Therefore, any real-time feedback provided can be tagged with respect to the correct presentation.

In yet another embodiment of the invention, the publisher of a textbook can include homework accessible to a user via means of tangible computer readable medium. The homework may include one or more questions that are tagged with metadata that contains the UIDs of the slides that teach the topics covered by the specific question. For example, a question Q1 may have the UIDs of slide 5, slide 7, and slide 12. A user having difficulty in answering the question may request a review of the slides respective of Q1 and be presented with these slides in this predetermined sequence.

The invention may be implemented as hardware, firmware, software, or any combination thereof. Moreover, the software is preferably implemented as an application program tangibly embodied on a program storage unit or computer readable medium. The application program may be uploaded to, and executed by, a machine comprising any suitable architecture. Preferably, the machine is implemented on a computer platform having hardware, such as one or more central processing units (“CPUs”), a memory, and input/output interfaces. The computer platform may also include an operating system and microinstruction code. The various processes and functions described herein may be either part of the microinstruction code or part of the application program, or any combination thereof, which may be executed by a CPU, whether or not such computer or processor is explicitly shown. In addition, various other peripheral units may be connected to the computer platform such as an additional data storage unit and a printing unit.

FIG. 6 is a block schematic diagram of a machine in the exemplary form of a computer system 1600 within which a set of instructions may be programmed to cause the machine to execute the logic steps of FIGS. 4 and 5 according to the invention. In alternative embodiments, the machine may comprise a network router, a network switch, a network bridge, personal digital assistant (PDA), a cellular telephone, a Web appliance or any machine capable of executing a sequence of instructions that specify actions to be taken by that machine.

The computer system 1600 includes a processor 1602, a main memory 1604 and a static memory 1606, which communicate with each other via a bus 1608. The computer system 1600 may further include a display unit 1610, for example, a liquid crystal display (LCD) or a cathode ray tube (CRT). The computer system 1600 also includes an alphanumeric input device 1612, for example, a keyboard; a cursor control device 1614, for example, a mouse; a disk drive unit 1616, a signal generation device 1618, for example, a speaker, and a network interface device 1620.

The disk drive unit 1616 includes a machine-readable medium 1624 on which is stored a set of executable instructions, i.e. software, 1626 embodying any one, or all, of the methodologies described herein below. The software 1626 is also shown to reside, completely or at least partially, within the main memory 1604 and/or within the processor 1602. The software 1626 may further be transmitted or received over a network 1628, 1630 by means of a network interface device 1620.

In contrast to the system 1600 discussed above, a different embodiment uses logic circuitry instead of computer-executed instructions to implement processing entities. Depending upon the particular requirements of the application in the areas of speed, expense, tooling costs, and the like, this logic may be implemented by constructing an application-specific integrated circuit (ASIC) having thousands of tiny integrated transistors. Such an ASIC may be implemented with CMOS (complimentary metal oxide semiconductor), TTL (transistor-transistor logic), VLSI (very large systems integration), or another suitable construction. Other alternatives include a digital signal processing chip (DSP), discrete circuitry (such as resistors, capacitors, diodes, inductors, and transistors), field programmable gate array (FPGA), programmable logic array (PLA), programmable logic device (PLD), and the like.

It is to be understood that embodiments may be used as or to support software programs or software modules executed upon some form of processing core (such as the CPU of a computer) or otherwise implemented or realized upon or within a machine or computer readable medium. A machine-readable medium includes any mechanism for storing or transmitting information in a form readable by a machine, e.g. a computer. For example, a machine readable medium includes read-only memory (ROM); random access memory (RAM); magnetic disk storage media; optical storage media; flash memory devices; electrical, optical, acoustical or other form of propagated signals, for example, carrier waves, infrared signals, digital signals, etc.; or any other type of media suitable for storing or transmitting information.

All examples and conditional language recited herein are intended for pedagogical purposes to aid the reader in understanding the principles of the invention and the concepts contributed by the inventor to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions. Moreover, all statements herein reciting principles, aspects, and embodiments of the invention, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future, i.e. any elements developed that perform the same function, regardless of structure.

Although the invention is described herein with reference to the preferred embodiment, one skilled in the art will readily appreciate that other applications may be substituted for those set forth herein without departing from the spirit and scope of the present invention. Accordingly, the invention should only be limited by the Claims included below.

Claims

1. A computer implemented method comprising: using a computer including a processor and a memory to enable a first lecturer to (i) access a first tangible computer readable medium storing a presentation including slides containing content associated with a textbook, wherein each slide of the presentation stored in the first tangible computer readable medium has a unique identification (UID) that uniquely identifies the slide relative to any other slide of the presentation and is coded as part of the metadata of the slide, (ii) select one or more of the slides of the presentation for presentation during a first lecture, and (iii) present the selected slides during the first lecture;using the computer to register in the memory a first lecture sequence comprising (i) the UID of each of the slides presented during the first lecture and (ii) for each of the slides presented during the first lecture, a timestamp identifying a time at which said slide was presented during the first lecture; andusing the computer to enable a first user to, based on the first lecture sequence, view the slides presented during the first lecture in the order in which the slides presented during the first lecture were presented;wherein the tangible computer readable medium additionally stores a second presentation including slides containing content associated with a second textbook, wherein each slide of the second presentation has a unique identification (UID) that uniquely identifies the slide relative to any other slide of (i) the presentation and (ii) the second presentation, andthe method further comprises: using the computer to enable the first lecturer to select the presentation or the second presentation;using the computer to enable a second lecturer to (i) access the first tangible computer readable medium, (ii) select one or more of the slides of the presentation for presentation during a second lecture, and (iii) present the selected slides during the second lecture;using the computer to register in the memory a second lecture sequence comprising (i) the UID of each of the slides presented during the second lecture and (ii) for each of the slides presented during the second lecture, a timestamp identifying a time at which the slide was presented during the second lecture; andusing the computer to enable a second user to, based on the second lecture sequence, view the slides presented during the second lecture in the order in which the slides presented during the second lecture were presented;wherein the one or more slides of the presentation selected by the first lecturer for presentation during the first lecture include a first slide, the first lecture sequence includes the UID of the first slide, the one or more slides of the presentation selected by the second lecturer for presentation during the second lecture include the first slide, and the second lecture sequence includes the UID of the first slide; andwherein the one or more slides of the presentation selected by the first lecturer for presentation during the first lecture include a second slide, the first lecture sequence includes the UID of the second slide, the one or more slides of the presentation selected by the second lecturer for presentation during the second lecture does not include the second slide, and the second lecture sequence does not include the UID of the second slide.

2. The method of claim 1, further comprising: using the computer to enable the first lecturer to determine a sequence of presentation of the selected slides of the presentation prior to presenting the selected slides during the first lecture; andusing the computer to store the determined sequence of the selected slides.

3. The method of claim 1, further comprising using the computer to associate said first lecture sequence with any of an audio track and a video track.

4. The method of claim 1, further comprising: using the computer to provide the first user with access to a second tangible computer readable medium storing material associated with the textbook, wherein the material is tagged with metadata containing the UID(s) of one or more slides of the presentation;using the computer to receive a request from the first user to view the one or more slides of the presentation uniquely identified by the UID(s) contained in the metadata with which the material is tagged; andusing the computer to enable the first user to view the one or more slides of the presentation uniquely identified by the UID(s) contained in the metadata with which the material is tagged.

5. The method of claim 4, wherein the material associated with the textbook is homework.

6. The method of claim 5, wherein the homework includes one or more questions tagged with metadata containing the UID(s) of one or more slides of the presentation.

7. A computerized system comprising: a memory; anda processor configured to:(a) enable a first lecturer to (i) access a tangible computer readable medium storing a presentation including slides containing content associated with a textbook, wherein each slide of the presentation stored in the tangible computer readable medium has a unique identification (UID) that uniquely identifies the slide relative to any other slide of the presentation and is coded as part of the metadata of the slide, (ii) select one or more of the slides of the presentation for presentation during a first lecture, and (iii) present the selected slides during the first lecture;(b) register in the memory a first lecture sequence comprising (i) the UID of each of the slides presented during the first lecture and (ii) for each of the slides presented during the first lecture, a timestamp identifying a time at which the slide was presented during the first lecture; and(c) enable a first user to, based on the first lecture sequence, view the slides presented during the first lecture in the order in which the slides presented during the first lecture were presented;wherein the tangible computer readable medium additionally stores a second presentation including slides containing content associated with a second textbook, wherein each slide of the second presentation has a unique identification (UID) that uniquely identifies the slide relative to any other slide of (i) the presentation and (ii) the second presentation, andthe processor is further configured to:(d) enable the first lecturer to select the presentation or the second presentation;(e) enable a second lecturer to (i) access the tangible computer readable medium, (ii) select one or more of the slides of the presentation for presentation during a second lecture, and (iii) present the selected slides during the second lecture;(f) register in the memory a second lecture sequence comprising (i) the UID of each of the slides presented during the second lecture and (ii) for each of the slides presented during the second lecture, a timestamp identifying a time at which the slide was presented during the second lecture; and(g) enable a second user to, based on the second lecture sequence, view the slides presented during the second lecture in the order in which the slides presented during the second lecture were presented;wherein the one or more slides of the presentation selected by the first lecturer for presentation during the first lecture include a first slide, the first lecture sequence includes the UID of the first slide, the one or more slides of the presentation selected by the second lecturer for presentation during the second lecture include the first slide, and the second lecture sequence includes the UID of the first slide; andwherein the one or more slides of the presentation selected by the first lecturer for presentation during the first lecture include a second slide, the first lecture sequence includes the UID of the second slide, the one or more slides of the presentation selected by the second lecturer for presentation during the second lecture does not include the second slide, and the second lecture sequence does not include the UID of the second slide.

8. The computerized system of claim 7, wherein the processor is further configured to: provide the first user with access to a second tangible computer readable medium storing material associated with the textbook, wherein the material is tagged with metadata containing the UID(s) of one or more slides of the presentation;receive a request from the first user to view the one or more slides of the presentation uniquely identified by the UID(s) contained in the metadata with which the material is tagged; andenable the first user to view the one or more slides of the presentation uniquely identified by the UID(s) contained in the metadata with which the material is tagged.

9. The computerized system of claim 8, wherein the material associated with the textbook is homework.

10. The computerized system of claim 9, wherein the homework includes one or more questions tagged with metadata containing the UID(s) of one or more slides of the presentation.

11. The computerized system of claim 7, wherein the processor is further configured to: enable the first lecturer to determine a sequence of presentation of the selected slides prior to presenting the selected slides during the first lecture; andstore the determined sequence of the selected slides.

12. The computerized system of claim 7, wherein the processor is further configured to associate said first lecture sequence with any of an audio track and a video track.