Modern computing devices, such as smartphones, tablet computers, and laptops can often include a touchscreen as an input/output component. A user can interact with such computing devices via the touchscreen using a stylus, a pen, or even the user's finger. For example, a user can use a stylus to navigate through menus, paint a picture, or perform other operations via the touchscreen.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
In certain computing devices, a reviewer can provide comments, remarks, editing marks, or other annotations to a document on a computing device using a stylus, a pen, the user's finger, or other suitable input device. For instance, a reviewer can handwrite annotations with a stylus on a touchscreen displaying a document on a tablet computer. The handwritten annotations can include, for example, “delete this sentence,” “add [a phrase], “add a space,” or other suitable editing marks. In response to receiving the annotations, the tablet computer can add and save the handwritten annotations as digital images in a layer of the document separate from the underlying content. An editor, for example, an original author of the document, can then manually edit the underlying content of the document based on the saved annotations to delete sentences, add phrases, add spaces, and/or perform other suitable editing operations.
Several embodiments of the disclosed technology can improve efficiencies of the foregoing editing process by allowing direct editing of underlying content in accordance with annotations. In one implementation, a computing device can be configured to execute an application and display underlying content of a document on an input/output component. In response to receiving a user input via, for instance, a stylus, the computing device can be configured to recognize the user input as an annotation. The computing device can also be configured to recognize the underlying content associated with the received annotation as text, images, video recordings, sound recordings, or other suitable types of data. The computing device can then determine whether one or more editing operations can be performed on the underlying content based on the annotation. In one embodiment, the application can automatically perform the one or more available operations. In other embodiments, the application can seek confirmation from the user before performing the one or more available operations. In further embodiments, the application can also be configured to perform the one or more operations upon user actuation, for example, by pressing an “Apply” button. As such, the foregoing disclosed technology can provide more efficient editing capabilities by eliminating or at least reducing manual editing by editors based on a reviewer's annotations.
Certain embodiments of computing systems, devices, components, modules, routines, and processes for implementing smart annotation of digital content are described below. In the following description, specific details of components are included to provide a thorough understanding of certain embodiments of the disclosed technology. A person skilled in the relevant art will also understand that the disclosed technology may have additional embodiments or may be practiced without several of the details of the embodiments described below with reference to
As used herein, the term “content” generally refers to digital data representing information useful to a user or audience. Content examples can include digital images, sound recordings, texts, video recordings, or other suitable digital media. In certain embodiments, a content file or “document” can include multiple data “layers” in which different types of media can be stored. For example, a layer of the content file can include multiple alphanumeric characters. Other layers of the content file can include digital images, sound recordings, video recordings, or other suitable types of media. The foregoing digital media are generally referred to as “underlying content” of a content file. The content file can also include other layers storing data related to authorship, publication information, security information, annotations, or other suitable information related to the underlying content of the content file.
Also used herein, the term “ink” or “digital ink” refers to digital image or other suitable types of data representing one or more strokes received at an input component (e.g., a touchscreen) of a computing device by a user utilizing a stylus, a pen, a user's finger, or other suitable pointing devices related to pen computing. A stroke or a combination of strokes can form a “gesture” with corresponding shapes, lengths, repeating patterns, or other suitable characteristics. One or more gestures can be recognized as an annotation to underlying content of a document. An “annotation” generally refers to digital data representing a note, comment, editing mark, or other suitable categories of remark associated with a content file. An annotation, however, is not a part of the underlying content of the content file. For example, a user can add an annotation that is a comment, explanation, or remark directed to the underlying content of the content file. In another example, a user can add an annotation that is an editing mark to a content file indicating, for example, that a phrase in the content file should be deleted, or a new sentence should be added to the content file. In certain embodiments, annotations can be stored with a content file as metadata of the content file. In other embodiments, the annotations can also be stored as additional content in another layer of the content file.
Further used herein, the term “editing mark” generally refers to a mark recognized as corresponding to one or more editing operations. One example editing mark can include a strike-through as a deletion mark. The term “editing operation” generally refers to an operation that alters the text file, digital images, sound recordings, video recordings, or other suitable types of media in substance or formatting in response to an editing mark. For example, an editing operation to the text file can be deleting at least a portion of the text in the text file in response to a corresponding editing mark such as a strike-through. In another example, an editing operating to a video or sound recording can include deleting a portion, changing a play speed, adding background music, or otherwise modifying the video or sound recording.
Certain editing processes involve a reviewer providing annotations in digital ink to underlying content of a document using a stylus, a pen, the user's finger, or other suitable input pointer. For instance, the reviewer can handwrite annotations with a stylus on a touchscreen displaying a document on a tablet computer. The handwritten annotations can include editing marks or general comments of critique or praise. In response to receiving the reviewer's input, the tablet computer can add and save the annotations to the displayed document, for example, as digital images. An editor can then modify the underlying content of the document based on the saved annotations to delete sentences, add phrases, add spaces, and/or perform other suitable editing operations.
The foregoing editing processes can be inefficient and burdensome by utilizing manual editing to modify the underlying content. Several embodiments of the disclosed technology can improve efficiencies of the foregoing editing process by allowing direct editing of the underlying content based on the reviewer's annotations. In certain implementations, in response to receiving an annotation, a computing device can be configured to determine whether the annotation corresponds to one or more available editing operations that can be performed on the underlying content. In response to determining that one or more editing operations are available, the computing device can automatically perform the one or more available editing operations to modify the underlying content without requiring manual editing. As such, workload on editors can be eliminated or at least reduced to improve editing efficiency, as described in more detail below with reference to
As shown in
Components within a system may take different forms within the system. As one example, a system comprising a first component, a second component and a third component can, without limitation, encompass a system that has the first component being a property in source code, the second component being a binary compiled library, and the third component being a thread created at runtime. The computer program, procedure, or process may be compiled into object, intermediate, or machine code and presented for execution by one or more processors of a personal computer, a network server, a laptop computer, a smartphone, and/or other suitable computing devices. Equally, components may include hardware circuitry. A person of ordinary skill in the art would recognize that hardware may be considered fossilized software, and software may be considered liquefied hardware. As just one example, software instructions in a component may be burned to a Programmable Logic Array circuit, or may be designed as a hardware circuit with appropriate integrated circuits. Equally, hardware may be emulated by software. Various implementations of source, intermediate, and/or object code and associated data may be stored in a computer memory that includes read-only memory, random-access memory, magnetic disk storage media, optical storage media, flash memory devices, and/or other suitable computer readable storage media that exclude propagated signals.
As shown in
The ink analyzer 107 can be configured to process the digital ink 110 captured on the user interface 106. As shown in
The gesture engine 114 can be configured to determine a gesture that corresponds to one or more strokes of the digital ink 110 received from the user 101. In certain embodiments, the gesture engine 114 can include routines that calculate a length, angle, variation, shape, or other suitable parameters of the strokes and correlate the calculated parameters to a gesture based on, for example, a gesture database (not shown). For example, the gesture engine 114 can recognize a stroke as a strike-through gesture when the stroke extends longitudinally for a length with limited variation along the vertical direction. In another example, the gesture engine 114 can correlate a stroke to a check mark when the stroke reverses a direction of extension at certain angles (e.g., about 30° to about 90°). In other embodiments, the gesture engine 114 can be configured to recognize the gesture by implementing other suitable techniques.
The ink recognition module 116 can be configured to recognize one or more editing operations 113 available or corresponding to a recognized gesture for the application 105. For example, the ink recognition module 116 can be configured to recognize that a strike-through gesture in a word processor corresponds to deleting a word, a line, a sentence, a selection, or other parts of the document, as described in more detail below with reference to
In response to the editing operations 113 from the ink analyzer 107, the application 105 can edit or otherwise modify the underlying content by performing the one or more editing operations automatically or selectively. The application 105 can then output the edited content 115 to the rendering engine 109. As shown in
In operation, the application 105 executing on the computing device 102 can receive digital ink 110 having one or more strokes from the user 101 via the user interface 106. The application 105 can then forward the digital ink 110 to the ink analyzer 107 to determine whether the digital ink 110 corresponds to one or more editing marks with corresponding editing operations 113 or to other types of annotations. In response to receiving the digital ink 110, the categorization module 112 can categorize the received digital ink 110 as editing marks, comments, or other suitable categories of annotation. If the received digital ink 110 is categorized as editing marks, the gesture engine 114 can then determine whether the digital ink 110 corresponds to one or more gestures. The gesture engine 114 can then forward the recognized gestures to the ink recognition module 116 which in turn can determine one or more editing operations 113 available or corresponding to the recognized gestures.
The ink recognition module 116 can then forward the determined one or more editing operations 113 to the application 105, which in turn can modify the underlying content of the document by performing the one or more of the editing operations 112 automatically or selectively. The application 105 can then forward the edited content 115 to the rendering engine 109, which in turn renders the edited content 115 on the user interface 106 of the application 105 on the computing device 102. Example operations of the foregoing components of the computing device 102 implemented in a word processor are described in more detail below with reference to
As shown In the illustrated embodiment of
As shown in
As shown in
As shown in
As shown in
The method 150 can then include a decision stage 154 to determine whether the received user input is related to one or more editing marks. In one embodiment, the user input can be categorized into notes, comments, editing marks, explanations, or other suitable categories using, for example, the categorization component 112 in
In response to determining that the user input is not related to an editing mark, the method 150 includes indicating that the user input is a comment, note, explanation, or other suitable categories of information to be stored in as metadata or other suitable data in the document. In response to determining that the user input is related to an editing mark, the method 150 includes recognizing one or more gestures related to the user input by, for example, searching a gesture database or via other suitable techniques. The method 150 can then include correlating the one or more recognized gestures to one or more editing operations at stage 158. In one embodiment, a gesture can be related to a single editing operation. For example, a strike-through can be related to a deletion operation. In other embodiments, a gesture can be related to multiple editing operations. For example, a long tap can be related to operations of copy, cut, and paste.
As shown in
As shown in
At stage 164, in response to determining that the gesture does not correspond to multiple editing operations, the operations include automatically performing the editing operation to the underlying content of the document at stage 165. On the other hand, in response to determining that the gesture corresponds to multiple potential editing operations, the method includes outputting the multiple potential editing operations to the user and prompting the user for a selection of one or more from the outputted editing operations at stage 166. The method can then include another decision stage 167 to determine whether a user selection is received. In response to receiving a user selection of one or more of the editing operations, the method includes performing the selected one or more editing operations at stage 168. In response to not receiving a user selection, the method reverts to prompting for a user selection at stage 166.
Depending on the desired configuration, the processor 204 may be of any type including but not limited to a microprocessor (ρP), a microcontroller (μC), a digital signal processor (DSP), or any combination thereof. The processor 204 may include one more levels of caching, such as a level one cache 210 and a level two cache 212, a processor core 214, and registers 216. An example processor core 214 may include an arithmetic logic unit (ALU), a floating point unit (FPU), a digital signal processing core (DSP Core), or any combination thereof. An example memory controller 218 may also be used with processor 204, or in some implementations memory controller 218 may be an internal part of processor 204.
Depending on the desired configuration, the system memory 206 may be of any type including but not limited to volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.) or any combination thereof. The system memory 206 can include an operating system 220, one or more applications 222, and program data 224. This described basic configuration 202 is illustrated in
The computing device 102 may have additional features or functionality, and additional interfaces to facilitate communications between basic configuration 202 and any other devices and interfaces. For example, a bus/interface controller 230 may be used to facilitate communications between the basic configuration 202 and one or more data storage devices 232 via a storage interface bus 234. The data storage devices 232 may be removable storage devices 236, non-removable storage devices 238, or a combination thereof. Examples of removable storage and non-removable storage devices include magnetic disk devices such as flexible disk drives and hard-disk drives (HDD), optical disk drives such as compact disk (CD) drives or digital versatile disk (DVD) drives, solid state drives (SSD), and tape drives to name a few. Example computer storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data.
The system memory 206, removable storage devices 236, and non-removable storage devices 238 are examples of computer readable storage media. Computer readable storage media include storage hardware or device(s), examples of which include, but not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other media which may be used to store the desired information and which may be accessed by computing device 102. Any such computer readable storage media may be a part of computing device 102. The term “computer readable storage medium” excludes propagated signals and communication media.
The computing device 102 may also include an interface bus 240 for facilitating communication from various interface devices (e.g., output devices 242, peripheral interfaces 244, and communication devices 246) to the basic configuration 202 via bus/interface controller 230. Example output devices 242 include a graphics processing unit 248 and an audio processing unit 220, which may be configured to communicate to various external devices such as a display or speakers via one or more A/V ports 222. Example peripheral interfaces 244 include a serial interface controller 224 or a parallel interface controller 226, which may be configured to communicate with external devices such as input devices (e.g., keyboard, mouse, pen, voice input device, touch input device, etc.) or other peripheral devices (e.g., printer, scanner, etc.) via one or more I/O ports 228. An example communication device 246 includes a network controller 260, which may be arranged to facilitate communications with one or more other computing devices 262 over a network communication link via one or more communication ports 264.
The network communication link may be one example of a communication media. Communication media may typically be embodied by computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and may include any information delivery media. A “modulated data signal” may be a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media may include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), microwave, infrared (IR) and other wireless media. The term computer readable media as used herein may include both storage media and communication media.
The computing device 102 may be implemented as a portion of a small-form factor portable (or mobile) electronic device such as a cell phone, a personal data assistant (PDA), a personal media player device, a wireless web-watch device, a personal headset device, an application specific device, or a hybrid device that include any of the above functions. The computing device 102 may also be implemented as a personal computer including both laptop computer and non-laptop computer configurations.
From the foregoing, it will be appreciated that specific embodiments of the disclosure have been described herein for purposes of illustration, but that various modifications may be made without deviating from the disclosure. In addition, many of the elements of one embodiment may be combined with other embodiments in addition to or in lieu of the elements of the other embodiments. Accordingly, the technology is not limited except as by the appended claims.
This application claims priority to U.S. Provisional Application No. 62/288,546, filed on Jan. 29, 2016, the disclosure of which is incorporated herein in its entirety.
Number | Date | Country | |
---|---|---|---|
62288546 | Jan 2016 | US |