Real-time document annotation

Abstract

Aspects of the present disclosure relate to systems and methods for receiving, managing, and displaying annotations on documents in real-time. A user (e.g., an author of a document) uploads a document into a real-time annotation system, which may then generate a composite presentation based on the uploaded document. The composite presentation includes all the content of the document presented in a specially configured graphical user interface to receive and manage annotations from a plurality of user devices.

Description

TECHNICAL FIELD

The subject matter disclosed herein relates to electronic document annotation. In particular, example embodiments may relate to techniques for receiving, allocating, managing, and displaying annotations on electronic documents.

BACKGROUND

Documents are often created and shared with groups of individuals to facilitate the exchange of information. Presentation programs, such as Microsoft PowerPoint, aid this effort by creating a medium in which a presenter (or group of presenters) can create presentation decks made up of slides containing visual and auditory content, which can then be presented to an individual or group of individuals in a slide show format. Individuals viewing the presentation often provide feedback to the presenter after the conclusion of the presentation, in the form of comments and questions, provided in-person, or at a later time, or as a red-lined hardcopy of the presentation itself. This collaborative effort is often the ultimate goal of presenting the information to a group of individuals in the first place, but can be impaired as a result of the inability to effectively provide, receive, and manage the feedback.

For example, in a traditional presentation, a presenter first creates and distributes a set of presentation documents in the form of a digital slide show delivered electronically, or as a printed hardcopy to a group of individuals. In a live presentation of the document, the presenter explains the content of the presentation to the group, pausing as a result of interruptions as a result of questions or comments related to the content of the presentation. These frequent pauses in the presentation can result in a segmented and disconnected explanation of topics, and may also result in the presenter losing their train of thought, slowing down the pace of the presentation, and possibly missing or skipping over important concepts.

In cases where the presentation is simply delivered to individuals to view on their own time, the presenter is then faced with the problem of receiving feedback and comments through multiple mediums (e.g., phone calls, in person conversations, red-lined hardcopies, or emails), as well as duplicative comments which become time consuming and frustrating to repeatedly address. As a result, many comments may remain unresolved, and the presentation may fail to incorporate valuable feedback which the presenter may have missed.

It is therefore valuable to devise systems and methods to facilitate and encourage the collaborative process, while avoiding the current cumbersome, time-consuming, and error-prone methods which exist currently.

BRIEF DESCRIPTION OF THE DRAWINGS

Various ones of the appended drawings merely illustrate example embodiments of the present inventive subject matter and cannot be considered as limiting its scope.

FIG. 1 is an architecture diagram depicting a presentation platform having a client-server architecture configured for exchanging data, according to an example embodiment.

FIG. 2 is a block diagram illustrating various modules comprising a real-time annotation system, which is provided as part of the presentation platform, consistent with some embodiments.

FIG. 3 is a flowchart illustrating a method for receiving and managing annotations on documents in real-time, consistent with some embodiments.

FIG. 4 is a flowchart illustrating a method for displaying annotations and receiving comments on the annotations from one or more users, consistent with some embodiments.

FIG. 5 is an interface diagram illustrating a composite presentation, comprising a graphical user interface overlaid on a composite presentation, according to example embodiments.

FIG. 6 is an interface diagram illustrating a user interface, comprising a newsfeed, and a document repository, according to example embodiments.

FIG. 7 is an interface diagram illustrating the user interface, comprising the newsfeed, updated to display annotation notifications, and an upload window, according to example embodiments.

FIG. 8 is a diagrammatic representation of a machine in the example form of a computer system within which a set of instructions for causing the machine to perform any one or more of the methodologies discussed herein may be executed.

DETAILED DESCRIPTION

Reference will now be made in detail to specific example embodiments for carrying out the inventive subject matter. Examples of these specific embodiments are illustrated in the accompanying drawings, and specific details are set forth in the following description in order to provide a thorough understanding of the subject matter. It will be understood that these examples are not intended to limit the scope of the claims to the illustrated embodiments. On the contrary, they are intended to cover such alternatives, modifications, and equivalents as may be included within the scope of the disclosure.

Aspects of the present disclosure relate to systems and methods for receiving, managing, and displaying annotations on presentation objects in real-time. The presentation objects include electronic documents, such as PowerPoint® presentations, as well as simple word processing documents, and PDF files, and includes an arrangement of one or more presentation elements (e.g., graphics, text, videos, interactive web elements). A user (e.g., a human author of a document) uploads a presentation object(s) into a real-time annotation system, which generates a composite presentation based on the presentation object(s). The presentation object includes one or more user identifiers of users associated with the presentation object data, including, for example, an author of the presentation object, one or more users assigned to the presentation object, as well as a user who uploaded the presentation object into the real-time annotation system. The composite presentation is displayed within a graphical user interface (GUI), and the GUI includes one or more interactive elements overlaid on the composite presentation to receive and manage annotations from user devices, in real-time.

From a user perspective, the users viewing the composite presentation can provide annotations directly onto the GUI through an input component of a user device (e.g., mobile device or desktop computer) by a single or double-click of a mouse, or in the case of touch-enabled devices, a predefined tactile input (e.g., a tap, a swipe, etc.). For example, to provide an annotation on a particular presentation element within the composite presentation, a first user positions a cursor over a location in the GUI corresponding to a desired presentation element of the composite presentation, and provides a user input.

Responsive to receiving the user input, the real-time annotation system indicates receipt of the user input, for example by causing display of a dialogue box within the GUI, where the first user may provide additional annotation data. The annotation data may include, for example, a user identifier of the first user, a string of text (e.g., comments from users), an indication of approval or disapproval, an annotation type (e.g., a question, or a comment) a status indicator flag (e.g., resolved or unresolved), a reference to another user (e.g., a user identifier), as well as multimedia content (e.g., audio and video). The annotation data is then indexed and stored according to the user identifier of the first user, the annotation content, and the composite presentation. The real-time annotation system generates and displays a graphical element representative of the annotation data at the location within the GUI corresponding to the presentation element indicated by the first user. By selecting the graphical element, the annotation data is displayed in a separate window as a user generated annotation, for viewing by a user. For example, a user may select the graphical element by simply placing a cursor over a location of the graphical element within the GUI (e.g., hovering the cursor over the graphical element), as well as by providing an explicit selection of the graphical element (e.g., clicking).

In some embodiments, the graphical element generated by the real-time annotation system includes additional indicators to identify the specific annotation, as well as the annotation type. For example, annotations marked as a “question” may be displayed in red (or any other color), while annotations marked as a “comment” may be displayed as blue (or any other color).

Responsive to receiving the annotation data, the real-time annotation system causes a notification that the annotation data has been received to be displayed at the devices of the one or more users associated with the presentation object of the composite presentation. In this way, the real-time annotation system may notify the one or more users associated with a particular presentation object of annotations received on the composite presentation in real-time. The notification includes an identification of the composite presentation in which the annotation was made and a user identifier of the user who provided the annotation data. Responsive to selecting the notification, the real-time annotation system may cause display of the annotation data as a user generated annotation, which the user may interact with by, for example providing a response. The associated users may then review the user generated annotation, and provide any necessary response or feedback requested.

In some embodiments, the real-time annotation system maintains composite presentations associated with user identifiers, such that all composite presentations associated with a particular user may be accessed through a single GUI at the user device. For example, a composite presentation may be associated to a user identifier based on the user having authored or uploaded the document, the user placing an annotation on the document, or an annotation on the document referencing the user. The real-time annotation system causes display of a user interface which includes a presentation of one or more graphical elements representing each of the associated composite presentations.

In some instances, the user generated annotation may include references to specific users, for example based on a user identifier. Responsive to receiving an annotation with a references to a specific user based on the user identifier, the real-time annotation system may cause display of a notification on a device of the specified user, which the specified user may then respond to.

In additional example embodiments, the user generated annotations may also include additional GUI elements with functionality to receive follow requests, as well as assignment request to assign an annotation to a user based on a user identifier. For example, by selecting the follow request included in the GUI, a user may then be notified of all comments and other activities which occur in reference to the annotation. Similarly, by assigning the annotation to a user based on a user identifier, the assigned user may then be notified of all comments and activities which occur in reference to the annotation.

FIG. 1 is a network diagram illustrating a network environment 100 suitable for operating a real-time document annotation system 150. A presentation platform 102 provides server-side functionality, via a network 104 (e.g., an intranet, the Internet or a Wide Area Network (WAN)), to one or more clients such as the client device 110. FIG. 1 illustrates a web client 112, client applications 114, and a programmatic client 116 executing on respective client device 110.

An Application Program Interface (API) server 120 and a web server 122 are coupled to, and provide programmatic and web interfaces respectively to, one or more application servers 140. The application servers 140 host the real-time annotation system 150. The application servers 140 are, in turn, shown to be coupled to one or more database servers 124 that facilitate access to one or more databases 126.

A real-time annotation system 150 provides real-time document annotation management functions for the presentation platform 102. For example, the real-time annotation system 150 receives a document (or documents) and generates a composite presentation document configured to receive annotation data from user devices in real-time.

As shown, the network system 100 includes a client device 110 in communication with a presentation platform 102 over a network 104. The presentation platform 102 communicates and exchanges data with the client device 110 that pertains to various functions and aspects associated with the presentation platform 102 and its users. Likewise, the client device 110, which may be any of a variety of types of devices that include at least a display, a processor, and communication capabilities that provide access to the network 104 (e.g., a smart phone, a tablet computer, a personal digital assistant (PDA), a personal navigation device (PND), a handheld computer, a desktop computer, a laptop or netbook, or a wearable computing device), may be operated by a user (e.g., a person) of the network system 100 to exchange data with the presentation platform 104 over the network 106.

The client device 110 communicates with the network 104 via a wired or wireless connection. For example, one or more portions of the network 104 may comprises an ad hoc network, an intranet, an extranet, a Virtual Private Network (VPN), a Local Area Network (LAN), a wireless LAN (WLAN), a Wide Area Network (WAN), a wireless WAN (WWAN), a Metropolitan Area Network (MAN), a portion of the Internet, a portion of the Public Switched Telephone Network (PSTN), a cellular telephone network, a wireless network, a Wireless Fidelity (Wi-Fi®) network, a Worldwide Interoperability for Microwave Access (WiMax) network, another type of network, or any suitable combination thereof.

In various embodiments, the data exchanged between the client device 110 and the presentation platform 102 may involve user-selected functions available through one or more user interfaces (UIs). The UIs may be specifically associated with a web client 112 (e.g., a browser) or an application 114, executing on the client device 110, and in communication with the presentation platform 102.

Turning specifically to the presentation platform 102, a web server 122 is coupled to (e.g., via wired or wireless interfaces), and provides web interfaces to, an application server 140. The application server 140 hosts one or more applications (e.g., web applications) that allow users to use various functions and services of the presentation platform 104. For example, the application server 140 may host a real-time annotation system 150 that provides a number of real-time document annotation management functions. In some embodiments, the real-time annotation system 150 runs and executes on the application server 140, while in other embodiments, the application server 140 provides the client device 110 with a set of instructions (e.g., computer-readable code) that causes the web client 112 of the client device 110 to execute and run the real-time annotation system 150. The real-time annotation system 150 receives presentation objects and generates composite presentations based on the presentation objects.

The presentation objects received and ingested by the real-time annotation system 150 may, for example, include a presentation slideshow (e.g., PowerPoint presentation) made up of a set of one or more presentation documents. Each of the presentation documents included within the presentation slideshow may include a set of presentation elements in the form of auditory or visual content (e.g., images, text strings, audio files, etc.).

The real-time annotation system 150 receives the presentation object (e.g., presentation slideshow) as presentation data delivered from a client device (e.g., client device 110), and store the presentation object within a database (e.g., database 126), at a memory location associated with the user of the client device. The real-time annotation system 150 then generates a composite presentation based on the presentation data. The composite presentation generated by the real-time presentation system 150 includes the presentation data received from the client device, overlaid with elements of a GUI configured to receive and manage annotation data from one or more users in real-time.

FIG. 2 is a block diagram illustrating various modules comprising the real-time annotation system 150, which is provided as part of the presentation platform 102, consistent with some embodiments. As is understood by skilled artisans in the relevant computer and Internet-related arts, the modules and engines illustrated in FIG. 2 represent a set of executable software instructions and the corresponding hardware (e.g., memory and processor) for executing the instructions. To avoid obscuring the inventive subject matter with unnecessary detail, various functional components (e.g., modules and engines) that are not germane to conveying an understanding of the inventive subject matter have been omitted from FIG. 2. However, a skilled artisan will readily recognize that various additional functional components may be supported by the real-time annotation system 150 to facilitate additional functionality that is not specifically described herein. Furthermore, the various functional modules and engines depicted in FIG. 2 may reside on a single computer (e.g., a client device), or may be distributed across several computers in various arrangements such as cloud-based architectures.

The real-time annotation system 150 is shown as including a document ingest module 210, a presentation module 220, an annotation module 230, and a notification module 240, all configured to communicate with each other (e.g., via a bus, shared memory, a switch, or application programming interfaces (APIs)). The aforementioned modules of the real-time annotation system 150 may, furthermore, access one or more databases that are part of the presentation platform 102 (e.g., database 126), and each of the modules may access one or more computer-readable storage media of the client device 110.

The document ingest module 210 receives presentation data from various client computing devices, and communicates appropriate responses to the requesting client devices to indicate receipt of the documents. The document ingest module 210 may receive presentation data from client devices. For example, the document ingest module 210 provides a number of interfaces (e.g., APIs or user interfaces that are presented by the client device 102) that allow users to upload data (e.g., documents).

The presentation module 220 is configured to access the ingested presentation data, and generate a composite presentation to display within a GUI at a client device. The composite presentation is displayed within a GUI, and the GUI includes one or more interactive elements overlaid on the composite presentation to receive and manage annotations from user devices, in real-time. The composite presentation includes one or more user identifiers of users associated with the presentation data.

Additionally, the presentation module 220 causes display of additional user interfaces that include graphical representations of the various annotation options provided by the real-time annotation system 150. These user interfaces may include various interface elements to share, assign, save, comment and open composite presentations. The presentation module 220 also receives and processes user input received through such user interfaces. Examples of the user interfaces provided by the presentation module 220 are discussed below in reference to FIGS. 4-7.

The annotation module 230 is configured to receive and manage annotation data received from client devices (e.g., client device 110). For example, users (e.g., user 106) viewing the composite presentation at a client device may provide one or more annotations related to the presentation data of the composite presentation. The annotation module 230 is configured to receive the annotation data and store the annotation data at a memory location in a database (e.g., database 126) associated with the composite presentation. The annotation data includes, for example: location data indicating a location which the annotation is directed to; user identifiers associated with the user providing the annotation; users referenced within the annotation; annotation content such as a text string; and annotation identification information such as data indicating an annotation type.

The notification module 240 is configured to generate and cause display of notifications responsive to the annotation module 230 providing an indication that annotation data has been received from a client device. For example, responsive to receiving an indication that the annotation module 230 has received annotation data from a client device, the notification module 240 generates a notification of the annotation to be displayed at one or more client devices. The notification of the annotation may include, for example, the annotation data, including the user identifier of the user which provided the annotation, as well as an indication of the composite presentation which the annotation applies to. The notification module 240 may then cause display of the notification at a client device of all users associated with the notification. Users associated with the notification include, for example, any users referenced within the annotation (e.g., based on a user identifier), the author of the composite presentation, a user who originally uploaded the document into the real-time annotation system 150, and the users who provide annotations.

FIG. 3 is a flowchart illustrating a method 300 for receiving and managing annotations on documents in real-time, consistent with some embodiments. The method 300 may be embodied in computer-readable instructions for execution by one or more processors such that the operations of the method 300 may be performed in part or in whole by the application server 140. In particular, the operations of the method 300 may be performed in part or in whole by the real-time annotation system 150; accordingly, the method 300 is described below by way of example with reference thereto. However, it shall be appreciated that at least some of the operations of the method 300 may be deployed on various other hardware configurations and the method 300 is not intended to be limited to the application server 140 or the real-time annotation system 150.

At operation 310, the document ingest module 210 receives presentation data from a client device (e.g., client device 110). The presentation data may include one or more presentation documents comprising presentation content. After receiving the presentation data, the document ingest module 210 allocates a memory location for the presentation data within the database 126, and stores the presentation data in the database 126.

At operation 320, responsive to receiving an indication that the real-time annotation system 150 has received presentation data, the presentation module 220 accesses the memory location associated with the presentation data and generates a composite presentation including the presentation data and one or more associated user identifiers. The presentation module 220 causes display of the composite presentation overlaid with a GUI, the GUI including functionality to receive and manage annotations from one or more client devices viewing the composite presentation.

At operation 330, the presentation module 220 causes display of the composite presentation at one or more client devices. The composite presentation may be displayed responsive to receiving an invitation to view the composite presentation from another user (e.g., the author of the presentation, or another user with access to the presentation). For example a user may access the composite presentation through a GUI of the real-time annotation system 150 at their user device. The user may select a graphical element representative of the composite presentation, and responsive to receiving an indication that the graphical element has been selected, the presentation module 220 causes display of the composite presentation.

At operation 340, the annotation module 230 receives annotation data from one or more client devices on which the composite presentation is presented. The annotation data may include a string of text, a user identifier associated with the user providing the annotation, a location within a composite presentation which the annotation applies, and an identifier associated with the composite presentation itself. In some instances, the annotation data may also include one or more user identifiers referenced by the annotating user. In this way, the system may provide notifications to the users associated with the one or more user identifiers of the annotation.

At operation 350, the notification module 240 receives an indication that annotation data has been received, and generates a notification to be displayed at one or more client devices. The notification may be displayed as a text box with a first portion of the text box including a string of text or other indicator (e.g., a graphical element) referencing a corresponding composite presentation, and a second portion of the text box includes a user identifier indicating a user who created the annotation. In some embodiments, the notification may also include an indication of an annotation type (e.g., question or comment), as well as a timestamp indicating a time and date in which the annotation was first received. A user may then access the corresponding composite presentation by selecting the notification. Responsive to a selection of the notification, the real-time annotation system 150 may then cause display of the composite presentation, along with the user generated annotation itself.

In some embodiments, the annotation data may also include a status indicator, indicating whether the user generated annotation has been resolved or unresolved. For example, the user generated annotation may include annotation data indicating a question requiring a response. The status indicator of the annotation may indicate that the annotation is “unresolved,” until a comment is provided, or a user switches the state of the annotation to “resolved” by providing an input from a client device. Once the annotation has been marked resolved, the display of the notification of the annotation may disappear from the GUI.

FIG. 4 is a flowchart illustrating a method 400 for displaying user generated annotations and receiving comments on the user generated annotations from one or more users, consistent with some embodiments. The method 400 may be embodied in computer-readable instructions for execution by one or more processors such that the operations of the method 400 may be performed in part or in whole by the application server 140. In particular, the operations of the method 400 may be performed in part or in whole by the real-time annotation system 150; accordingly, the method 400 is described below by way of example with reference thereto. However, it shall be appreciated that at least some of the operations of the method 400 may be deployed on various other hardware configurations and the method 400 is not intended to be limited to the application server 140 or the real-time annotation system 150.

As discussed above, in FIG. 3, the method 300 describes a method for receiving presentation data, and generating and displaying a composite presentation overlaid with a GUI configured to receive and manage user generated annotations in real-time from one or more user devices. The method 400 may initiate at a point in which a user has received a notification of a user generated annotation, and provided an indication that the user generated annotation be displayed at the user device. At operation 410, the annotation module 230 causes display of the user generated annotation pertaining to a composite presentation within a graphical user interface of the real-time annotation system 150. The user generated annotation may have been received from another user accessing the composite presentation, or alternatively could have been created by the viewing user themselves. The user generated annotation may be displayed as graphical element represented as a text box including a first portion which includes a text string (e.g., a question or comment), a second portion which indicates a source of the annotation (e.g., a user identifier), and a third portion which may include a text field to provide an annotation comment to the annotation. In some instances, the user generated annotation may also include an annotation identifier (e.g., a number, letter, or other form of identification).

At operation 420, the user viewing the user generated annotation provides a comment on the user generated annotation. The user may initiate the comment by selecting the user generated annotation and providing an annotation comment as a text string directly into a text field of the user generated annotation. The annotation comment may include additional text, pertaining to the annotation, as well as one or more references to additional users (e.g., based on user identifiers).

At operation 430, responsive to receiving an indication of the annotation comment from the user device, the notification module 240 generates and causes display of a notification at the user devices of the associated users. The associated users include the author of the composite presentation (or user who uploaded the presentation deck into the real-time annotation system), the user who provided the annotation on which the comment was received, as well as any users referenced in the comment itself, or who may have commented on the annotation. The notification may include a reference to the annotation on which the comment was received, a reference to the corresponding composite presentation, as well as a user identifier associated with the user who provided the comment.

At operation 440, the notification module 240 receives an indication of a selection of the notification from a user device. Responsive to the notification module 240 receiving the selection, at operation 450, the annotation module 230 causes display of the user generated annotation at the user device, including the most recent comment.

FIG. 5 is an interface diagram illustrating a composite presentation 510 overlaid with a GUI 500, according to example embodiments. The composite presentation 510 is shown to include, a set of user generated annotations 515, each of the user generated annotations within the set of user generated annotations 515, and graphical elements (e.g., 520 and 525) to represent a reference location of a corresponding user generated annotation, and a cursor 560 to place annotations on the composite presentation 510.

In some embodiments, the user generated annotations 515 are generated responsive to an input received from a client device (e.g., client device 110), via the cursor 560. For example a user may move the cursor 560 over a location in the composite presentation 510 within the GUI 500 and provide a user input (e.g., a single or double click). Responsive to the received user input, the real-time annotation system 150 generates and places a graphical element (e.g., 520, 525) at the location of the cursor 560 in the GUI 500. Each graphical element may be visually distinct from one another, for example, by numbering, unique colors or patterns, or labels. As shown in FIG. 5, each graphical element includes a unique number identifying an associated annotation among the user generated annotations 515.

The user generated annotations (e.g., 515) created are then displayed within a portion of the GUI 500. The user generated annotation may include an annotation identifier 530, in order to identify a corresponding graphical element (e.g., graphical element 525) associated with the annotation, a source identifier 535 (e.g., a user identifier), identifying a user who created/placed the user generated annotation on the composite presentation 510, a status indicator 540, to indicate a status of the annotation (e.g., resolved or unresolved), an assignment field 545, where a user may assign the user generated annotation to one or more users, based on one or more user identifiers, a comment field 550, where a user may provide a comment onto the user generated annotation, and annotation content 555, which may include a text string, an indication of approval or disapproval (e.g., a like or dislike), or simply a flag to alert a recipient of a particular element within the composite presentation 510.

FIG. 6 is an interface diagram illustrating a GUI 600, comprising a newsfeed 610, and a document repository 615, according to example embodiments. In some embodiments, the newsfeed 610 may include a switch 620 to toggle between various types of content to be displayed within the newsfeed 610. For example, the switch 620 may toggle the content displayed between an updates section and an annotations section, wherein the updates section displays annotation notifications in real-time, and the annotations section displays abbreviated annotations. The newsfeed displayed in FIG. 6 depicts the annotations section.

The annotations section of the newsfeed 610 comprises a list of all user generated annotations which the user accessing the GUI 600 has created, has been assigned, or is following. As shown, the section may be separated into a personal tasks section 625 to display annotations which the user themselves has created, and a following section 630 to display annotations which the user is following (e.g., requested updates for).

A document repository 615 includes an arrangement of graphical representations of all documents which a user is associated with. As stated above, a user may be associated with a document if they uploaded the document into the real-time annotation system 150, created an annotation on a composite presentation, received an assignment on an annotation, or were referenced in a comment or annotation by a user identifier.

FIG. 7 is an interface diagram illustrating the GUI 600, comprising the newsfeed 610, updated to display annotation notifications (e.g., annotation notification 715), and an upload window 710, according to example embodiments. The annotation notifications are received from one or more client devices and displayed in real-time. In some embodiments, the user may sort the annotation notifications in the order they are received, with the most recently received annotations at the top of the newsfeed 610, and the older annotations preceding.

The annotation notifications include an indication of the nature of the notification (e.g., “you were mentioned in the comment,” “you were mentioned in an annotation,” “you were assigned to a task”), an indication of what composite presentation the annotation is regarding (e.g., “Test Document”), as well as a text string. The user may access the corresponding composite presentation by selecting a desired annotation notification from among the set of annotation notifications displayed.

The upload window 710 enables users to upload presentation data into the real-time annotation system 150. Upon uploading the presentation data via the upload window 710, the real-time annotation system 150 stores the document within the database 126, and generates a composite presentation based on the uploaded document. A user may also provide a title for the document, a description for the document, and a file type of the document. In some embodiments, the uploading user may also share the document with one or more additional users by including a set of user identifiers into the upload window 710.

Modules, Components, and Logic

Certain embodiments are described herein as including logic or a number of components, modules, or mechanisms. Modules may constitute either software modules (e.g., code embodied on a machine-readable medium) or hardware modules. A “hardware module” is a tangible unit capable of performing certain operations and may be configured or arranged in a certain physical manner. In various example embodiments, one or more computer systems (e.g., a standalone computer system, a client computer system, or a server computer system) or one or more hardware modules of a computer system (e.g., a processor or a group of processors) may be configured by software (e.g., an application or application portion) as a hardware module that operates to perform certain operations as described herein.

In some embodiments, a hardware module may be implemented mechanically, electronically, or any suitable combination thereof. For example, a hardware module may include dedicated circuitry or logic that is permanently configured to perform certain operations. For example, a hardware module may be a special-purpose processor, such as a Field-Programmable Gate Array (FPGA) or an Application Specific Integrated Circuit (ASIC). A hardware module may also include programmable logic or circuitry that is temporarily configured by software to perform certain operations. For example, a hardware module may include software executed by a general-purpose processor or other programmable processor. Once configured by such software, hardware modules become specific machines (or specific components of a machine) uniquely tailored to perform the configured functions and are no longer general-purpose processors. It will be appreciated that the decision to implement a hardware module mechanically, in dedicated and permanently configured circuitry, or in temporarily configured circuitry (e.g., configured by software) may be driven by cost and time considerations.

Accordingly, the phrase “hardware module” should be understood to encompass a tangible entity, be that an entity that is physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a certain manner or to perform certain operations described herein. As used herein, “hardware-implemented module” refers to a hardware module. Considering embodiments in which hardware modules are temporarily configured (e.g., programmed), each of the hardware modules need not be configured or instantiated at any one instance in time. For example, where a hardware module comprises a general-purpose processor configured by software to become a special-purpose processor, the general-purpose processor may be configured as respectively different special-purpose processors (e.g., comprising different hardware modules) at different times. Software accordingly configures a particular processor or processors, for example, to constitute a particular hardware module at one instance of time and to constitute a different hardware module at a different instance of time.

Hardware modules can provide information to, and receive information from, other hardware modules. Accordingly, the described hardware modules may be regarded as being communicatively coupled. Where multiple hardware modules exist contemporaneously, communications may be achieved through signal transmission (e.g., over appropriate circuits and buses) between or among two or more of the hardware modules. In embodiments in which multiple hardware modules are configured or instantiated at different times, communications between such hardware modules may be achieved, for example, through the storage and retrieval of information in memory structures to which the multiple hardware modules have access. For example, one hardware module may perform an operation and store the output of that operation in a memory device to which it is communicatively coupled. A further hardware module may then, at a later time, access the memory device to retrieve and process the stored output. Hardware modules may also initiate communications with input or output devices, and can operate on a resource (e.g., a collection of information).

The various operations of example methods described herein may be performed, at least partially, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors may constitute processor-implemented modules that operate to perform one or more operations or functions described herein. As used herein, “processor-implemented module” refers to a hardware module implemented using one or more processors.

Similarly, the methods described herein may be at least partially processor-implemented, with a particular processor or processors being an example of hardware. For example, at least some of the operations of a method may be performed by one or more processors or processor-implemented modules. Moreover, the one or more processors may also operate to support performance of the relevant operations in a “cloud computing” environment or as a “software as a service” (SaaS). For example, at least some of the operations may be performed by a group of computers (as examples of machines including processors), with these operations being accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., an API).

The performance of certain of the operations may be distributed among the processors, not only residing within a single machine, but deployed across a number of machines. In some example embodiments, the processors or processor-implemented modules may be located in a single geographic location (e.g., within a home environment, an office environment, or a server farm). In other example embodiments, the processors or processor-implemented modules may be distributed across a number of geographic locations.

Example Machine Architecture and Machine-Readable

FIG. 8 is a block diagram illustrating components of a machine 800, according to some example embodiments, able to read instructions from a machine-readable medium (e.g., a machine-readable storage medium) and perform any one or more of the methodologies discussed herein. Specifically, FIG. 8 shows a diagrammatic representation of the machine 800 in the example form of a computer system, within which instructions 816 (e.g., software, a program, an application, an applet, an app, or other executable code) for causing the machine 800 to perform any one or more of the methodologies discussed herein may be executed. Additionally, or alternatively, the machine 800 may correspond to any one of the client device 102, the web server 110, the application server 112, or the third-party computing system 118. The instructions transform the general, non-programmed machine into a particular machine programmed to carry out the described and illustrated functions in the manner described. In alternative embodiments, the machine 800 operates as a standalone device or may be coupled (e.g., networked) to other machines. In a networked deployment, the machine 800 may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 800 may comprise, but not be limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a PDA, an entertainment media system, a cellular telephone, a smart phone, a mobile device, a wearable device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing the instructions 816, sequentially or otherwise, that specify actions to be taken by the machine 800. Further, while only a single machine 800 is illustrated, the term “machine” shall also be taken to include a collection of machines 800 that individually or jointly execute the instructions 816 to perform any one or more of the methodologies discussed herein.

The machine 800 may include processors 810, memory/storage 830, and I/O components 850, which may be configured to communicate with each other such as via a bus 802. In an example embodiment, the processors 810 (e.g., a Central Processing Unit (CPU), a Reduced Instruction Set Computing (RISC) processor, a Complex Instruction Set Computing (CISC) processor, a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), an ASIC, a Radio-Frequency Integrated Circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor 812 and a processor 814 that may execute the instructions 816. The term “processor” is intended to include multi-core processor that may comprise two or more independent processors (sometimes referred to as “cores”) that may execute instructions contemporaneously. Although FIG. 8 shows multiple processors, the machine 800 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiples cores, or any combination thereof.

The memory/storage 830 may include a memory 832, such as a main memory, or other memory storage, and a storage unit 836, both accessible to the processors 810 such as via the bus 802. The storage unit 836 and memory 832 store the instructions 816 embodying any one or more of the methodologies or functions described herein. The instructions 816 may also reside, completely or partially, within the memory 832, within the storage unit 836, within at least one of the processors 810 (e.g., within the processor's cache memory), or any suitable combination thereof, during execution thereof by the machine 800. Accordingly, the memory 832, the storage unit 836, and the memory of the processors 810 are examples of machine-readable media.

As used herein, “machine-readable medium” means a device able to store instructions and data temporarily or permanently, and may include, but is not limited to, random-access memory (RAM), read-only memory (ROM), buffer memory, flash memory, optical media, magnetic media, cache memory, other types of storage (e.g., Erasable Programmable Read-Only Memory (EEPROM)), and/or any suitable combination thereof. The term “machine-readable medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, or associated caches and servers) able to store the instructions 816. The term “machine-readable medium” shall also be taken to include any medium, or combination of multiple media, that is capable of storing instructions (e.g., instructions 816) for execution by a machine (e.g., machine 800), such that the instructions, when executed by one or more processors of the machine (e.g., processors 810), cause the machine to perform any one or more of the methodologies described herein. Accordingly, a “machine-readable medium” refers to a single storage apparatus or device, as well as “cloud-based” storage systems or storage networks that include multiple storage apparatus or devices. The term “machine-readable medium” excludes signals per se.

Furthermore, the machine-readable medium is non-transitory in that it does not embody a propagating signal. However, labeling the tangible machine-readable medium “non-transitory” should not be construed to mean that the medium is incapable of movement—the medium should be considered as being transportable from one real-world location to another. Additionally, since the machine-readable medium is tangible, the medium may be considered to be a machine-readable device.

The I/O components 850 may include a wide variety of components to receive input, provide output, produce output, transmit information, exchange information, capture measurements, and so on. The specific I/O components 850 that are included in a particular machine will depend on the type of machine. For example, portable machines such as mobile phones will likely include a touch input device or other such input mechanisms, while a headless server machine will likely not include such a touch input device. It will be appreciated that the I/O components 850 may include many other components that are not shown in FIG. 8. The I/O components 850 are grouped according to functionality merely for simplifying the following discussion and the grouping is in no way limiting. In various example embodiments, the I/O components 850 may include output components 852 and input components 854. The output components 852 may include visual components (e.g., a display such as a plasma display panel (PDP), a light emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), acoustic components (e.g., speakers), haptic components (e.g., a vibratory motor, resistance mechanisms), other signal generators, and so forth. The input components 854 may include alphanumeric input components (e.g., a keyboard, a touch screen configured to receive alphanumeric input, a photo-optical keyboard, or other alphanumeric input components), point based input components (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or another pointing instrument), tactile input components (e.g., a physical button, a touch screen that provides location and/or force of touches or touch gestures, or other tactile input components), audio input components (e.g., a microphone), and the like.

In further example embodiments, the I/O components 850 may include biometric components 856, motion components 858, environmental components 860, or position components 862 among a wide array of other components. For example, the biometric components 856 may include components to detect expressions (e.g., hand expressions, facial expressions, vocal expressions, body gestures, or eye tracking), measure biosignals (e.g., blood pressure, heart rate, body temperature, perspiration, or brain waves), identify a person (e.g., voice identification, retinal identification, facial identification, fingerprint identification, or electroencephalogram based identification), and the like. The motion components 858 may include acceleration sensor components (e.g., accelerometer), gravitation sensor components, rotation sensor components (e.g., gyroscope), and so forth. The environmental components 860 may include, for example, illumination sensor components (e.g., photometer), temperature sensor components (e.g., one or more thermometers that detect ambient temperature), humidity sensor components, pressure sensor components (e.g., barometer), acoustic sensor components (e.g., one or more microphones that detect background noise), proximity sensor components (e.g., infrared sensors that detect nearby objects), gas sensors (e.g., gas detection sensors to detect concentrations of hazardous gases for safety or to measure pollutants in the atmosphere), or other components that may provide indications, measurements, or signals corresponding to a surrounding physical environment. The position components 862 may include location sensor components (e.g., a Global Position System (GPS) receiver component), altitude sensor components (e.g., altimeters or barometers that detect air pressure from which altitude may be derived), orientation sensor components (e.g., magnetometers), and the like.

Communication may be implemented using a wide variety of technologies. The I/O components 850 may include communication components 864 operable to couple the machine 800 to a network 880 or devices 870 via a coupling 882 and a coupling 872, respectively. For example, the communication components 864 may include a network interface component or other suitable device to interface with the network 880. In further examples, the communication components 864 may include wired communication components, wireless communication components, cellular communication components, Near Field Communication (NFC) components, Bluetooth® components (e.g., Bluetooth® Low Energy), Wi-Fi® components, and other communication components to provide communication via other modalities. The devices 870 may be another machine or any of a wide variety of peripheral devices (e.g., a peripheral device coupled via a Universal Serial Bus (USB)).

Moreover, the communication components 864 may detect identifiers or include components operable to detect identifiers. For example, the communication components 864 may include Radio Frequency Identification (RFID) tag reader components, NFC smart tag detection components, optical reader components (e.g., an optical sensor to detect one-dimensional bar codes such as Universal Product Code (UPC) bar code, multi-dimensional bar codes such as Quick Response (QR) code, Aztec code, Data Matrix, Dataglyph, MaxiCode, PDF4117, Ultra Code, UCC RSS-2D bar code, and other optical codes), or acoustic detection components (e.g., microphones to identify tagged audio signals). In addition, a variety of information may be derived via the communication components 864, such as location via Internet Protocol (IP) geo-location, location via Wi-Fi® signal triangulation, location via detecting an NFC beacon signal that may indicate a particular location, and so forth.

Transmission Medium

In various example embodiments, one or more portions of the network 880 may be an ad hoc network, an intranet, an extranet, a VPN, a LAN, a WLAN, a WAN, a WWAN, a MAN, the Internet, a portion of the Internet, a portion of the PSTN, a plain old telephone service (POTS) network, a cellular telephone network, a wireless network, a Wi-Fi® network, another type of network, or a combination of two or more such networks. For example, the network 880 or a portion of the network 880 may include a wireless or cellular network and the coupling 882 may be a Code Division Multiple Access (CDMA) connection, a Global System for Mobile communications (GSM) connection, or another type of cellular or wireless coupling. In this example, the coupling 882 may implement any of a variety of types of data transfer technology, such as Single Carrier Radio Transmission Technology (1×RTT), Evolution-Data Optimized (EVDO) technology, General Packet Radio Service (GPRS) technology, Enhanced Data rates for GSM Evolution (EDGE) technology, third Generation Partnership Project (3GPP) including 3G, fourth generation wireless (4G) networks, Universal Mobile Telecommunications System (UMTS), High Speed Packet Access (HSPA), Worldwide Interoperability for Microwave Access (WiMAX), Long Term Evolution (LTE) standard, others defined by various standard-setting organizations, other long range protocols, or other data transfer technology.

The instructions 816 may be transmitted or received over the network 880 using a transmission medium via a network interface device (e.g., a network interface component included in the communication components 864) and using any one of a number of well-known transfer protocols (e.g., hypertext transfer protocol (HTTP)). Similarly, the instructions 816 may be transmitted or received using a transmission medium via the coupling 872 (e.g., a peer-to-peer coupling) to the devices 870. The term “transmission medium” shall be taken to include any intangible medium that is capable of storing, encoding, or carrying the instructions 816 for execution by the machine 800, and includes digital or analog communications signals or other intangible media to facilitate communication of such software.

Language

Throughout this specification, plural instances may implement components, operations, or structures described as a single instance. Although individual operations of one or more methods are illustrated and described as separate operations, one or more of the individual operations may be performed concurrently, and nothing requires that the operations be performed in the order illustrated. Structures and functionality presented as separate components in example configurations may be implemented as a combined structure or component. Similarly, structures and functionality presented as a single component may be implemented as separate components. These and other variations, modifications, additions, and improvements fall within the scope of the subject matter herein.

Although an overview of the inventive subject matter has been described with reference to specific example embodiments, various modifications and changes may be made to these embodiments without departing from the broader scope of embodiments of the present disclosure. Such embodiments of the inventive subject matter may be referred to herein, individually or collectively, by the term “invention” merely for convenience and without intending to voluntarily limit the scope of this application to any single disclosure or inventive concept if more than one is, in fact, disclosed.

The embodiments illustrated herein are described in sufficient detail to enable those skilled in the art to practice the teachings disclosed. Other embodiments may be used and derived therefrom, such that structural and logical substitutions and changes may be made without departing from the scope of this disclosure. The Detailed Description, therefore, is not to be taken in a limiting sense, and the scope of various embodiments is defined only by the appended claims, along with the full range of equivalents to which such claims are entitled.

As used herein, the term “or” may be construed in either an inclusive or exclusive sense. Moreover, plural instances may be provided for resources, operations, or structures described herein as a single instance. Additionally, boundaries between various resources, operations, modules, engines, and data stores are somewhat arbitrary, and particular operations are illustrated in a context of specific illustrative configurations. Other allocations of functionality are envisioned and may fall within a scope of various embodiments of the present disclosure. In general, structures and functionality presented as separate resources in the example configurations may be implemented as a combined structure or resource. Similarly, structures and functionality presented as a single resource may be implemented as separate resources. These and other variations, modifications, additions, and improvements fall within a scope of embodiments of the present disclosure as represented by the appended claims. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.

In this document, the terms “a” or “an” are used, as is common in patent documents, to include one or more than one, independent of any other instances or usages of “at least one” or “one or more.” In the appended claims, the terms “including” and “in which” are used as the plain-English equivalents of the respective terms “comprising” and “wherein.” Also, in the following claims, the terms “including” and “comprising” are open-ended; that is, a system, device, article, or process that includes elements in addition to those listed after such a term in a claim are still deemed to fall within the scope of that claim. Moreover, in the following claims, the terms “first,” “second,” “third,” and so forth are used merely as labels, and are not intended to impose numerical requirements on their objects.

Claims

1. A method comprising: receiving, by a first client device, an update to a set of graphical elements being displayed by the first client device, the update including an annotation identifier corresponding to a first annotation type and data identifying a first graphical element from the set of graphical elements, the update to the set of graphical elements having been transmitted by a cloud-based server in response to the cloud-based server having received annotation data defining the update to the set of graphical elements from a second client device that is displaying the set of graphical elements;updating a presentation of the set of graphical elements being displayed by the first client device based on the update such that the annotation identifier is presented at a location corresponding to the first graphical element;in response to detecting an input selecting the annotation identifier presented at the location corresponding to the first graphical element, transmitting a request to the cloud-based server for the annotation data corresponding to the annotation identifier;in response to receiving the annotation data from the cloud-based server, updating the presentation of the set of graphical elements being displayed by the first client device based on the annotation data;receiving, by the first client device, an assignment of the annotation data to a user based on a user identifier received via an input device of the first client device; andtransmitting, by the first client device, the assignment and the user identifier to the cloud-based server, the transmitting causing a presentation of a notification on the second client device by the cloud-based server, the notification including an indication that the user has been assigned to the annotation data, text of the comment, the user identifier, and an indication of the annotation data.

2. The method of claim 1, wherein the annotation data includes multimedia content.

3. The method of claim 1, wherein updating the presentation of the set of graphical elements being displayed by the first client device comprises: causing presentation of a user identifier that identifies a user profile associated with the second client device.

4. The method of claim 1, further comprising: receiving a second update to the set of graphical elements being displayed by the first client device, the second update including a second annotation identifier corresponding to a second annotation type and data identifying a second graphical element from the set of graphical elements, the second update to the set of graphical elements having been transmitted by the cloud-based server in response to the cloud-based server having received second annotation data defining the second update to the set of graphical elements from a third client device that is displaying the set of graphical elements.

5. The method of claim 4, further comprising: updating presentation of the set of graphical elements being displayed by the first client device based on the second update such that the second annotation identifier is presented at a location corresponding to the second graphical element.

6. A client device comprising: one or more computer processors; andone or more computer-readable mediums storing instructions that, when executed by the one or more computer processors, cause the client device to perform operations comprising:receiving an update to a set of graphical elements being displayed by the client device, the update including an annotation identifier corresponding to a first annotation type and data identifying a first graphical element from the set of graphical elements, the update to the set of graphical elements having been transmitted by a cloud-based server in response to the cloud-based server having received annotation data defining the update to the set of graphical elements from a second client device that is displaying the set of graphical elements;updating presentation of the set of graphical elements being displayed by the client device based on the update such that the annotation identifier is presented at a location corresponding to the first graphical element;in response to detecting an input selecting the annotation identifier presented at the location corresponding to the first graphical element, transmitting a request to the cloud-based server for the annotation data corresponding to the annotation identifier;in response to receiving the annotation data from the cloud-based server, updating presentation of the set of graphical elements being displayed by the client device based on the annotation data;receiving, by the client device, an assignment of the annotation data to a user based on a user identifier received via an input device of the client device; andtransmitting, by the client device, the assignment and the user identifier to the cloud-based server, the transmitting causing a presentation of a notification on the second client device by the cloud-based server, the notification including an indication that the user has been assigned to the annotation data, text of the comment, the user identifier, and an indication of the annotation data.

7. The client device of claim 6, wherein the annotation data includes multimedia content.

8. The client device of claim 6, wherein updating the presentation of the set of graphical elements being displayed by the client device comprises: causing presentation of a user identifier that identifies a user profile associated with the second client device.

9. The client device of claim 6, the operations further comprising: receiving a second update to the set of graphical elements being displayed by the client device, the second update including a second annotation identifier corresponding to a second annotation type and data identifying a second graphical element from the set of graphical elements, the second update to the set of graphical elements having been transmitted by the cloud-based server in response to the cloud-based server having received second annotation data defining the second update to the set of graphical elements from a third client device that is displaying the set of graphical elements.

10. The client device of claim 9, the operations further comprising: updating presentation of the set of graphical elements being displayed by the client device based on the second update such that the second annotation identifier is presented at a location corresponding to the second graphical element.

11. A non-transitory computer-readable medium storing instructions that, when executed by one or more computer processors of a client device, cause the client device to perform operations comprising: receiving an update to a set of graphical elements being displayed by the client device, the update including an annotation identifier corresponding to a first annotation type and data identifying a first graphical element from the set of graphical elements, the update to the set of graphical elements having been transmitted by a cloud-based server in response to the cloud-based server having received annotation data defining the update to the set of graphical elements from a second client device that is displaying the set of graphical elements;updating presentation of the set of graphical elements being displayed by the client device based on the update such that the annotation identifier is presented at a location corresponding to the first graphical element;in response to detecting an input selecting the annotation identifier presented at the location corresponding to the first graphical element, transmitting a request to the cloud-based server for the annotation data corresponding to the annotation identifier; andin response to receiving the annotation data from the cloud-based server, updating presentation of the set of graphical elements being displayed by the client device based on the annotation data;receiving, by the client device, an assignment of the annotation data to a user based on a user identifier received via an input device of the client device; andtransmitting, by the client device, the assignment and the user identifier to the cloud-based server, the transmitting causing a presentation of a notification on the second client device by the cloud-based server, the notification including an indication that the user has been assigned to the annotation data, text of the comment, the user identifier, and an indication of the annotation data.

12. The non-transitory computer-readable medium of claim 11, wherein the annotation data includes multimedia content.

13. The non-transitory computer-readable medium of claim 11, wherein updating the presentation of the set of graphical elements being displayed by the client device comprises: causing presentation of a user identifier that identifies a user profile associated with the second client device.

14. The non-transitory computer-readable medium of claim 11, the operations further comprising: receiving a second update to the set of graphical elements being displayed by the client device, the second update including a second annotation identifier corresponding to a second annotation type and data identifying a second graphical element from the set of graphical elements, the second update to the set of graphical elements having been transmitted by the cloud-based server in response to the cloud-based server having received second annotation data defining the second update to the set of graphical elements from a third client device that is displaying the set of graphical elements; andupdating presentation of the set of graphical elements being displayed by the client device based on the second update such that the second annotation identifier is presented at a location corresponding to the second graphical element.