People interact with computer applications through user interfaces. While audio, tactile, and similar forms of user interfaces are available, visual user interfaces through a display device are the most common form of a user interface. With the development of faster and smaller electronics for computing devices, smaller size devices such as handheld computers, smart phones, tablet devices, and comparable devices have become common. Such devices execute a wide variety of applications ranging from communication applications to complicated analysis tools. Many such applications render content from external providers through a display. The applications allow users to access and interact with content that is unavailable locally.
Mobile computing devices provide instantaneous access to content on the go. However, content consumption is limited compared to traditional devices based on hardware limitations of mobile computing devices. Screen size, computing capacity, storage space, and other hardware components may limit a user's ability to view and interact with content through a mobile computing device. Modern user interfaces either disable interaction features to present content in mobile computing devices or implement full featured applications that are slow and ill-suited to the task of content presentation and interaction.
This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This summary is not intended to exclusively identify key features or essential features of the claimed subject matter, nor is it intended as an aid in determining the scope of the claimed subject matter.
Embodiments are directed to providing a layer based reorganization of document components. In some example embodiments, a display application may determine layers of a document. The document may be displayed with mixed content that includes text, image, video, audio, and similar components. A layer may be determined to include one or more sections such as a paragraph, a line, a sentence, a word, a table, a column, a row, a cell, an image, similar sections, or combination of sections. The layer may include multiple sections that share a type such as a paragraph or combination of sections such as multiple paragraphs and multiple images. A user action to select and move one of the layers may be detected. The non-selected layers may be moved away from the selected layer while moving the selected layer during the user action. Furthermore, The selected layer and the non-selected layers may be reflowed upon a completion of the user action which may include a drop action of the selected layer in a new location within the document. The reflow operation may include formatting of the selected layer and the non-selected layers to match respective new locations within the document.
These and other features and advantages will be apparent from a reading of the following detailed description and a review of the associated drawings. It is to be understood that both the foregoing general description and the following detailed description are explanatory and do not restrict aspects as claimed.
As briefly described above, a layer based reorganization of document components may be provided by a display application. A user action may be detected to select and move a layer from a set of determined layers of a document. Non-selected layers may be moved away from the selected layer while moving the selected layer during the user action. The selected layer and the non-selected layers may be reflowed upon a completion of the user action.
In the following detailed description, references are made to the accompanying drawings that form a part hereof, and in which are shown by way of illustrations specific embodiments or examples. These aspects may be combined, other aspects may be utilized, and structural changes may be made without departing from the spirit or scope of the present disclosure. The following detailed description is therefore not to be taken in a limiting sense, and the scope of the present invention is defined by the appended claims and their equivalents.
While the embodiments will be described in the general context of program modules that execute in conjunction with an application program that runs on an operating system on a computing device, those skilled in the art will recognize that aspects may also be implemented in combination with other program modules.
Generally, program modules include routines, programs, components, data structures, and other types of structures that perform particular tasks or implement particular abstract data types. Moreover, those skilled in the art will appreciate that embodiments may be practiced with other computer system configurations, including hand-held devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, and comparable computing devices. Embodiments may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.
Embodiments may be implemented as a computer-implemented process (method), a computing system, or as an article of manufacture, such as a computer program product or computer readable media. The computer program product may be a computer storage medium readable by a computer system and encoding a computer program that comprises instructions for causing a computer or computing system to perform example process(es). The computer-readable storage medium is a computer-readable memory device. The computer-readable storage medium can for example be implemented via one or more of a volatile computer memory, a non-volatile memory, a hard drive, and a flash drive.
Throughout this specification, the term “platform” may be a combination of software and hardware components to provide a layer based reorganization of document components. Examples of platforms include, but are not limited to, a hosted service executed over a plurality of servers, an application executed on a single computing device, and comparable systems. The term “server” generally refers to a computing device executing one or more software programs typically in a networked environment. However, a server may also be implemented as a virtual server (software programs) executed on one or more computing devices viewed as a server on the network. More detail on these technologies and example embodiments may be found in the following description.
In a diagram 100, a display application 104 may provide a layer based reorganization of document components displayed by a computing device 102. The computing device 102 may include a tablet device, a desktop computer, a notebook computer, a smart phone, and similar ones that are capable of rendering content. The computing device 102 may also allow a user 108 to interact with the displayed content through the display application 104. In an example scenario, the user 108 may interact with the displayed content using a touch based action 110. The touch based action 110 may include a number of gestures that may include a selection action, a move action, a copy action, a cut action, a resize action, a reformat action, and similar ones.
The displayed content may be retrieved from a content provider 106. The content provider 106 may include a local or an external server that hosts the displayed content. In addition, the displayed content may include text, image, audio, video, and similar components. Alternatively, the display application 104 may retrieve the displayed content from local storage. An example of the displayed content may include a document such as a word processing document, an html formatted web page, a spreadsheet document, a presentation document, and similar ones. The display application 104 may be a client application of the computing device 102 that renders the displayed content. Alternatively, the display application 104 may include a client application that renders a server application of the content provider 106. An example may include the display application 104 such as a browser application that renders a server application such as a web application of the content provider 106 to display content from the content provider 106.
In another example scenario, the display application 104 may determine layers of a document. The display application 104 may move a selected layer in response to the touch based action 110. The selected layer may be moved to a new location within the displayed content while reorganizing non-selected layers. Components of the non-selected layers may not be processed through a reflow operation prior to a completion of the touch based action 110.
While the example system in
In a diagram 200, a scheme to provide a layer based reorganization of document components may be described. A display application 202 may display a document that includes a number of components. The document may have components such as text, image, audio, video, and similar ones.
The display application 202 may determine layers of the document to allow a user to interact with the document. Examples of user interactions may include edit operations such as a move operation, a copy operation, a cut operation, a paste operation. The display application 202 may partition the document to layers. Examples of a layer may include one or more sections such as a paragraph, a line, a sentence, a word, a table, a column, a row, a cell, an image, a similar section, or combination of sections.
In an example scenario, the display application 202 may partition the document based on identification tags associated with a formatting of the document. The identification tags may encompass and identify the layers. An example may include paragraph tags encompassing a paragraph of a text of the document. Alternatively, a paragraph may be identified by a paragraph format tag. An end of the paragraph layer may be identified by a new line character. Similar schemes may be used to identify layers. Another example may include determining sentences as layers through period characters. Similarly, layers that include tables and components of tables may be identified by tags that encompass the table or the table components such as columns, rows, or cells. Images may also be identified by tags or through object detection schemes. Other schemes may be used to identify layers based on a formatting of the document or a type of the document.
Granularity of the layers may be adjusted based on a user preference, a system setting, or a use history associated with the document or a user who accesses the document. In an example scenario, a use history, that includes a user editing paragraphs in the document, may be analyzed to determine the layers to include paragraphs of the document. In alternative scenario, the layers may be determined to include finer granularity such as sentences or words to allow a user finer control over the document.
According to some embodiments, the display application 202 may detect a user action 220 on one of the layers of the document. The user action may 220 be a touch based action such as a grab and move action. The display application 202 may determine a selected layer 208 based on proximity to the user action 220. The selected layer 208 may be moved based on a direction of the user action 220. Reflow of sub-components of the selected layer 208 may be disabled until a completion of the user action 220. The completion of the user action 220 may be determined to include a drop action by the user to indicate an end of an interaction with the selected layer 208. However, in response to a determination of a format change associated with the selected layer 208 during the user action 220, a reflow operation may be executed within the selected layer 208 during the user action 220 to reflect a new bullet type, a text alignment change, and a text style change.
The display application 202 may determine layers that are outside of a location of a start of the user action 220 to be non-selected layers (204, 206, and 210). The non-selected layers (204, 206, and 210) may also be moved based on the direction of the user action 220. The non-selected layers (204, 206, and 210) may include a layer that includes an image and layers that include text. Reflow of sub-components of the non-selected layers (204, 206, and 210) may be disabled until a completion of the user action 220. Sub-components may include sub-structures of a layer such as a title, a sentence, a line, a sentence, a word, and similar ones associated with the non-selected layers (204 and 210) and similar ones.
In a diagram 300, a display application may have determined layers of a document rendered by the display application. A user may be allowed to interact with the document through the layers. The layers may include non-selected layers (304, 306, and 310) and a selected layer 308. The selected layer 308 may be associated with a user action. The user action may move the selected layer 308 to a new location occupied by one of the non-selected layers (304, 306, and 310). The non-selected layers (304 and 310) may include text content and the non-selected layer 306 may include an image content. Sub-components of the non-selected layers (304, 306, and 310) may not be reflowed prior to a completion of the user action. Sub-components of the selected layer 308 may be reflowed to adjust for a reformatting based change associated with text based content of the selected layer 308.
According to some embodiments, the display application may determine properties (312, 314, 316, and 318) of the selected layer 308 and the non-selected layers (304, 306, and 310). The properties (312, 314, 316, and 318) may include a location and a depth of the layer within the document. The properties (312, 314, 316, and 318) may also include a size and a shape of the layer. If the document does not include metadata that lists the properties (312, 314, 316, and 318) of the selected layer 308 and the non-selected layers (304, 306, and 310), the properties (312, 314, 316, and 318) may be associated with the respective layers. The display application may associate the properties (312, 314, 316, and 318) by inserting the properties (312, 314, 316, and 318) as metadata associated with the document.
In an example scenario, the display application may adjust the properties 314 of the selected layer 308 and the non-selected layers (304, 306, and 310) during the user action to reflect changes to the properties (312, 314, 316, and 318) in accordance with the user action. The location of the selected layer 308 may be updated to reflect its location during the user action. The location of a subset of the non-selected layers (304, 306, and 310) that are moved during the user action may also be updated.
According to other examples, the properties (312, 314, 316, and 318) may also be adjusted to move the selected layer 308 and a subset of the non-selected layers (304, 306, and 308) that are moved as a result of the user action. The display application may determine a direction of the user action. An adjacent layer 306 may be determined based on the direction. A first set of the properties 314 of the selected layer 308 may be adjusted with one or more values of a second set of the properties 316 of the adjacent layer 306 to move the selected layer 308 to a location of the adjacent layer 306. A second set of the properties 316 of the adjacent layer 306 may be adjusted with one or more values of a first set of the properties (314) of the selected layer 308 to move the adjacent layer 306 to a location of the selected layer 308. One or more values used to adjust the properties (314 and 316) may include a location associated with respective layers.
Although an example of a horizontal move of the selected layer 308 was provided, the embodiments are not limited to the horizontal move. The selected layer 308 may be moved in any direction associated with the user action whether the user action is linear, curved, or a combination of both. Similarly, a layer that is adjacent to the selected layer 308 may be determined based on a direction of the user action. The layer that is adjacent to the selected layer 308 may be moved to a previous location of the selected layer 308 during the user action.
In a diagram 400, a display application 402 may render layers (404, 406, 408, and 410) upon a completion of the user action. The completion may be determined to be an end of the user action such as a drop action on a selected layer 408. The layers (404, 406, 408, and 410) may be reflowed to format their respective content to match available resources at new locations. In an example scenario, an adjacent layer 406 may be placed at a left edge of a location previously occupied by the selected layer 408 based on a determination of a left alignment based formatting of the document. Similarly, the selected layer 408 may be placed at a left edge of a location previously occupied by the adjacent layer 406. Alternatively, to maximize utilization of available space in the document, the selected layer 408 may be placed to a right of the adjacent layer 406 based on a determination of a left alignment based formatting of the document.
Sub-components of the selected layer 408 may be reformatted based on properties of the adjacent layer 406. For example, a height of the selected layer 408 may be adjusted to match a height of the adjacent layer 406 upon completion of the user action. The sub-components of the selected layer 408 such as text may be adjusted to fill the additional height (alternatively, the removed height). Other properties of the selected layer 408, the adjacent layer 406, or a subset of non-selected layers (404 and 410), that are moved as a result of the user action, may be adjusted based new respective locations and available space in proximity to the new respective locations.
The reflow operation may also be delayed in response to a determination of a subsequent user action to move one of the non-selected layers (404, 406, and 410) or the selected layer 408. The subsequent user action may be determined based on a use history associated with the user, the user action, or similar criteria. Additionally, the selected layer 408 may be overlaid over the non-selected layers (404, 406, and 410) during the user action and the subsequent user action.
In addition, a move between the adjacent layer 406 and the selected layer 408 may be rendered using an animation. The animation may be used to display a gradual motion of the selected layer 408 and the adjacent layer 406 moving to each other's location. The animation may be matched to a movement of the user action based on one or more system settings. The system settings may define a frame rate of the animation. A system setting that dedicates resources to the animation may allow the display application 402 to display the animation at a high frame rate such as a frame rate above 24 frames per second. A system setting that limits resources to the animation may allow the display application 402 to display the animation at a low frame rate such as a frame rate below 24 frames per second.
Furthermore a decision to activate the reflow operation may be determined based on a determination of a type of change on the selected layer 408 by the user action. The type of change may include a semantic change or an enhancement change. The semantic change may alter a flow of content associated with the selected layer 408 and the non-selected layers (404, 406, and 410). The enhancement change may enhance visuals of the selected layer 408 and the non-selected layers (404, 406, and 410). Additionally, the decision may be determined based on a dynamic computation. The dynamic computation may evaluate resource utilization associated with the reflow operation such as processor and memory resources necessary to execute the reflow operation. In response to a determination that the resource utilization may exceed available resources, the decision may be determined to delay the reflow operation. Alternatively, in response to a determination that the available resources may meet the resource utilization, the decision may be determined to execute the reflow operation.
The example scenarios and schemas in
Client applications executed on any of the client devices 511-513 may facilitate communications via application(s) executed by servers 514, or on individual server 516. A display application may determine layers of a document. A selected layer may be moved through non-selected layers to a new location based on a user action interacting with the document. The display application may store data associated with the document in data store(s) 519 directly or through database server 518.
Network(s) 510 may comprise any topology of servers, clients, Internet service providers, and communication media. A system according to embodiments may have a static or dynamic topology. Network(s) 510 may include secure networks such as an enterprise network, an unsecure network such as a wireless open network, or the Internet. Network(s) 510 may also coordinate communication over other networks such as Public Switched Telephone Network (PSTN) or cellular networks. Furthermore, network(s) 510 may include short range wireless networks such as Bluetooth or similar ones. Network(s) 510 provide communication between the nodes described herein. By way of example, and not limitation, network(s) 510 may include wireless media such as acoustic, RF, infrared and other wireless media.
Many other configurations of computing devices, applications, data sources, and data distribution systems may be employed to provide a layer based reorganization of document components. Furthermore, the networked environments discussed in
The computing device 600 may also include a plurality of processing units that cooperate in execution of programs. Depending on the exact configuration and type of computing device, the system memory 604 may be volatile (such as RAM), non-volatile (such as ROM, flash memory, etc.) or some combination of the two. System memory 604 typically includes an operating system 605 suitable for controlling the operation of the platform, such as the WINDOWS® operating systems from MICROSOFT CORPORATION of Redmond, Wash. The system memory 604 may also include one or more software applications such as program modules 606, the display application 622, and a layer drag module 624.
The display application 622 may determine layers of a document. The layer drag module 624 may move a selected layer in a direction of the user action through non-selected layers. This basic configuration is illustrated in
The computing device 600 may have additional features or functionality. For example, the computing device 600 may also include additional data storage devices (removable and/or non-removable) such as, for example, magnetic disks, optical disks, or tape. Such additional storage is illustrated in
The computing device 600 may also contain communication connections 616 that allow the device to communicate with other devices 618, such as over a wired or wireless network in a distributed computing environment, a satellite link, a cellular link, a short range network, and comparable mechanisms. Other devices 618 may include computer device(s) that execute communication applications, web servers, and comparable devices. Communication connection(s) 616 is one example of communication media. Communication media can include therein computer readable instructions, data structures, program modules, or other data. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media.
Example embodiments also include methods. These methods can be implemented in any number of ways, including the structures described in this document. One such way is by machine operations, of devices of the type described in this document.
Another optional way is for one or more of the individual operations of the methods to be performed in conjunction with one or more human operators performing some. These human operators need not be collocated with each other, but each can be only with a machine that performs a portion of the program.
Process 700 begins with operation 710, where layers of a document may be determined. A layer may include one or more sections such as a paragraph, a sentence, a line, a word, an image, a similar section, or combination of sections. At operation 720, a user action to select and move one of the layers may be detected. The user action may include a touch based action. Next, non-selected layers of a subset of the layers may be moved away from the selected layer while moving the selected layer during the user action, at operation 730. The selected layer may be moved in a place of an adjacent layer from the non-selected layers in a direction of the user action. At operation 740, the selected layer and the non-selected layer may be reflowed upon a completion of the user action.
The operations included in process 700 are for illustration purposes. A display application according to embodiments may be implemented by similar processes with fewer or additional steps, as well as in different order of operations using the principles described herein.
According to some examples, a method that is executed on a computing device to provide a layer based reorganization of document components may be described. The method may include determining layers of a document, detecting a user action to select and move one of the layers, moving non-selected layers of a subset of the layers away from the selected layer while moving the selected layer during the user action, and reflowing the selected layer and the non-selected layers upon a completion of the user action.
According to other examples, the layers of the document may be determined by partitioning the document based on identification tags associated with a format of the document, where the identification tags encompass and identify the layers. The method may further include determining the selected and each of the non-selected layers to include one from a set of: a paragraph, a line, a sentence, and a word, a table, a column, a row, a cell, and an image and adjusting a granularity of the selected and the non-selected layers based on one or more of a user preference, a system setting, and a use history associated with the document.
According to further examples, the method may further include determining properties of the selected layer and the non-selected layers, where the properties include a location and a depth associated with each of the selected layer and the non-selected layers, associating the properties with the selected layer and the non-selected layers, and adjusting the properties of the selected layer and the non-selected layers during the user action to reflect changes to the properties in accordance with the user action. In response to a determination of a format change associated with the selected layer during the user action, a reflow operation may be executed within the selected layer during the user action to reflect one or more of a new bullet type, a text alignment change, and a text style change.
According to other examples, the method may further include determining a direction of the user action, determining an adjacent layer from the non-selected layers based on the direction, rendering a move between the selected layer and the adjacent layer to place the adjacent layer in a first location associated with the selected layer and to place the selected layer in a second location associated with the adjacent layer, rendering the move using an animation to display a gradual motion that is configured based on one or more system settings. A reflow of the non-selected layers may be disabled during the user action.
According to some examples, a computing device to provide a layer based reorganization of document components may be described. The computing device may include a memory, a processor coupled to the memory. The processor may be configured to execute a display application in conjunction with instructions stored in the memory. The display application may be configured to determine layers of a document, where each of the layers include one from a set of: a paragraph, a line, a sentence, and a word, a table, a column, a row, a cell, and an image, detect a user action to select and move one of the layers, where the user action includes a touch based action, move non-selected layers of a subset of the layers away from the selected layer while moving the selected layer during the user action, and reflow the selected layer and the non-selected layers upon a completion of the user action.
According to other examples, the display application is further configured to determine a first set of properties of the selected layer and a second set of properties of an adjacent layer from the non-selected layers, where the first set and the second set of properties include a location and a depth associated with each of the selected and the non-selected layers. The selected layer may be moved by adjusting the first set of properties with one or more values from the second set of properties. The adjacent layer may be moved by adjusting the second set of properties with one or more values from the first set of properties. In response to a determination of the completion, the selected layer may be reformatted based on properties of one of the non-selected layers that previously occupied a location of the selected layer at the completion. In response to a determination of the completion, a first subset of the non-selected layers may be reformatted, that move based on the user action in accordance with properties of a second subset from the non-selected layers and the selected layer that previously occupied locations of the first subset at the completion.
According to some examples, a computer-readable memory device with instructions stored thereon to provide a layer based reorganization of document components may be described. The instructions may include actions that are similar to the method described above.
The above specification, examples and data provide a complete description of the manufacture and use of the composition of the embodiments. Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims and embodiments.
This application claims priority to U.S. Provisional Application Ser. No. 61/916,738 filed on Dec. 16, 2013. The provisional application is hereby incorporated by reference in its entirety.
Number | Date | Country | |
---|---|---|---|
61916738 | Dec 2013 | US |