Patent Application
20100235769
Modern user interface toolkits use layout managers to create user interfaces that can be dynamically resized. Layout managers are software components that have the ability to lay out components (sometimes called widgets) by their relative positions without using distance units. It is often more natural to define component layouts in this manner than to define component positions in pixels or common distance units, so a number of popular widget toolkits include this ability by default. Widget toolkits often allow designers to place widgets in layout containers that have particular layout characteristics when the container is rendered. Windows Presentation Foundation (WPF) and Microsoft Silverlight are two examples of toolkits that provide layout managers. Designers describe user interface (UI) components in Extensible Application Markup Language (XAML) and can define UI states and transitions using an Application Programming Interface (API) such as Microsoft Visual State Manager.
One reason that layout managers are used is to enable automatic resizing of an application, so that UIs can work properly on devices with different display resolutions and with resizable, reconfigurable windows. Layout managers are used to automatically determine the layout of graphical elements and controls on a screen. Layout managers typically operate by forcing their children (graphical elements, UI elements) to conform to a certain layout rule. A stack panel, for example, may force its children into a horizontal or vertical stack, controlling only relative position of the objects with respect to each other. As another example, a uniform grid might force objects to be arranged in a grid, with a fixed size for each child. A table-style layout might force children to conform to row and column definitions, and to define position margins to control positioning and size.
For dynamic user interfaces, it is often useful to transition between different layouts of UI components. For example, when a user clicks on a “More” button the designer may want the area of one component to expand and display more UI controls. For usability reasons, designers like transitions to be smooth and allow for detailed control. In some scenarios, it is sufficient to simply fade between two entirely different UI screens, but this technique does not work when a layout of graphical elements or user interface controls is designed to morph smoothly from one layout to another. For example, a designer may want a panel to appear to fly out from the left side after a user logs in to an application. Current animation systems, such as Microsoft Visual State Manager, automatically animate transitions from one state to another. In the previous example, the animation system causes the panel to animate from the left side of the display through several intermediate locations before reaching the final fully displayed location.
These automatic transitions work well for continuous properties that the animation system can easily vary between two values. For example, an animation system can easily vary opacity that starts at 0% in a first state and ends up at 100% in a second state using intermediate values (e.g., 20%, 40%, and 80%). Unfortunately, many of the properties that define layout in the presence of layout managers are discrete (e.g., changing a column number of a grid) or may even have values that are automatically calculated (e.g., setting a row width to “auto”). Smooth layout transitions (i.e., morphs) between layouts in the presence of such discrete properties are difficult to achieve with current technology. Animation of such properties is either impossible or results in sudden layout switches rather than smooth transitions. Current animation systems cannot smoothly animate transitions for these types of properties.
A layout animation system is described herein that performs smooth transitions for properties of a layout, regardless of whether the layout properties are discrete or automatically calculated. The layout animation system solves the problem of morphing between two given layout states that contain changes to non-continuous properties by extracting the geometric information of the elements in each of the participating layout states. Before a transition is executed, the layout animation system extracts and stores the positioning, sizing, and visibility of participating elements across the layout hierarchy. The system first places the affected elements in the same position, size, and visibility as the outgoing state requests. For each element, the system calculates a smooth animation that transforms the element from the outgoing position, size, and visibility to the incoming position, size, and visibility. For visibility, care is taken to introduce smooth fade-in animations using additional opacity animation. The system then animates the elements over a desired transition time. Thus, the layout animation system provides the ability to create smooth layout transitions in the presence of automatic layout management for properties of various types.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
A layout animation system is described herein that performs smooth transitions for properties of a layout, regardless of whether the layout properties are discrete or automatically calculated. An example of a discrete property is visibility. A UI element might be visible in one state and invisible in another. The layout animation system solves the problem of morphing between two given layout states by extracting the geometric information of the elements in each of the participating layout states. During the execution of a layout transition, the participating UI elements are taken out of the control of their layout managers (e.g., by adding an extra layer of layout that compensates for what the layout manager is doing to animate the UI elements) and animated by the layout animation system directly on a display surface without layout management. Once the elements have reached their destination location and size, the system returns the elements to their place in the layout system, and the layout system takes control again. Thus, the layout animation system provides the ability to create smooth layout transitions in the presence of automatic layout management for properties of various types.
The layout animation system manages transitions as follows. Before a transition (or morph) is executed, the layout animation system extracts and stores the positioning, sizing, and visibility of all participating elements (both primitives, such as graphical elements and controls, and layout containers) across the layout hierarchy in parent-relative coordinates. When the transition is executed, the system may wrap each element that changes a layout property (e.g., size, position, visibility, and so forth) in a drawing container without layout management (in WPF, a Canvas, for example) so that the system can smoothly animate the element (e.g., by gradually varying the Left and Top coordinates relative to the Canvas because those properties are floating point numbers). Alternatively or additionally, the system may request that the layout manager not broadcast new size/position information to containers of elements while the layout animation is in progress (or store such elements in a layout exceptions list). The system first places the affected elements in the same position, size, and visibility as the outgoing state requests. In some embodiments, this step may be performed by a traditional layout manager before the system intervenes to provide smooth animation. For each element, the system calculates a smooth animation that transforms the element from the outgoing position, size, and visibility to the incoming position, size, and visibility. For visibility, care is taken to introduce smooth fade-in animations using additional opacity animation. The system then animates the elements over a desired transition time. After the elements have reached the destination position, the system places the elements back into the appropriate layout containers. Then, the layout animation system releases control of the participating elements, and behavior is again controlled by the layout managers.
Discrete or calculated properties can include various UI element settings. For example, a grid layout may include row and column identifiers rather than coordinates where UI elements are located. A transition of a UI element from one column to another between two states is a discrete property change (e.g., Grid.Column), but may actually move the element some distance as well as changing the element's width and/or height. Although the column property is changing between the states, the layout animation system modifies the position and size properties of the element through several intermediate positions between the start and final location of the element so that the animation is smooth. An example of a calculated property value is changing a height property of an element from an explicit height to a value of “auto” that indicates that the height of the element dynamically adjusts based on other elements around the element. There is no gradual modification native to the platform between, for example, a height of five and a value of auto. However, the layout animation system smoothly animates the transition by modifying the size property of the element.
The state management component 110 stores multiple states in which a UI can exist and receives an indication to transition from one state to another. The states may be stored in a declarative format, such as a XAML file. Designers may create states by visually laying controls on a canvas in a designer tool or may manually add control definitions to the declarative format file. The declarative format may also contain transition information, such as the conditions that cause a transition (e.g., mouse hover, button click, and so forth) and properties of the transition (e.g., a duration). For example, these may be stored as Triggers and/or VisualStates in a XAML file using Microsoft's Visual State Manager. Transitions may be caused by application logic based on a variety of purposes for different applications.
The determine state delta component 120 compares an ingoing and outgoing state to determine the layout properties that change between the states, and the corresponding UI elements that will change. The layout properties may include many different property types, such as columns and rows of a grid layout (and column spans and row spans), top and left properties of a canvas, a visibility property, an orientation of a stack, a docking direction in a dock layout, an expanded property of an expander layout, orientation and item width/height of a wrap panel layout, and individual element positional properties (e.g., width, height, minimum width, minimum height, maximum width, maximum height, margins, padding, horizontal alignment, and vertical alignment). When the determine state delta component 120 determines that layout properties of a UI element change during the transition, the determine state delta component 120 also adds parent and sibling elements of the element to a set of elements that change since these elements may be repositioned as well, due to the new parameters of the element whose layout properties are changing. It is typically unnecessary to handle child elements because they typically change automatically when their parents change, and child elements can construct layout animations of their own for any cases that turn out to be complex.
The state snapshot component 130 takes a snapshot of the rectangle in which each UI element of a layout is positioned. The rectangle is an abstract boundary of the UI element representing the element's dimensions that is not actually visible. Note that the system can also be used to handle three-dimensional animations, so rather than a rectangle the snapshot may include a bounding box. The state snapshot component 130 takes two snapshots, one after informing the UI element that it is in the outgoing state and another after informing the UI element that it is in the incoming state. In some embodiments, the layout animation system provides APIs for requesting that a UI element change states without actually rendering the UI element.
The snapshot comparison component 140 compares the snapshots of the incoming state and outgoing state to determine what UI elements actually change. For each element, the snapshot comparison component 140 determines the value of each property that changed in each of the incoming and outgoing states, and then determines an appropriate property for the element animation component 150 to modify to animate the transition of that property. For example, for the visibility property, the snapshot comparison component 140 may determine that opacity is an appropriate property to modify through several intermediate states to create a smooth transition between the outgoing and incoming layout states. During the transition animation, the system modifies the opacity property, and at the final animation to the incoming state, the system modifies the visibility property to its discrete value (e.g., false, for not visible) and resets the opacity property to its original value.
The determine state delta component 120 and the snapshot comparison component 140 perform similar functions under different conditions. The determine state delta component 120 compares states before a transition occurs to provide an estimate of the layout properties that will change during the transition. However, without actually performing the transition, the determine state delta component 120 may actually be over inclusive in the UI elements that it identifies as changing. The snapshot comparison component 140, on the other hand, compares snapshots of the layout properties when each state is active. The starting snapshot is taken at a time when the outgoing state is active, and the ending snapshot is taken after the transition to the incoming state has been made (e.g., the transition occurs in memory, though it is not yet rendered to the screen). The snapshot comparison component 140 then passes those UI elements that actually changed to the element animation component 150 to smoothly animate the transition, and passes any additional UI elements that did not actually change through a traditional layout path. In addition, a traditional (non-layout) animation path may animate any properties that are changing in the state delta that do not have layout repercussions (e.g., background color).
The element animation component 150 animates an element smoothly between the outgoing state and incoming state. For example, if an element starts at one position and moves to another, then the element animation component 150 moves the element from the start position to the end position through several intermediate positions in between to provide the appearance of smooth movement to the user. Although rectangles and rectangular coordinates are used as examples herein, those of ordinary skill in the art will recognize that the system can be used with various types of coordinates and transition motion. For example, the system may use a polar coordinate system where elements are animated through a rotation angle. As another example, a particular UI layout may define a winding path along which UI elements move to get from the outgoing state to the incoming state.
In some embodiments, the layout animation system 100 includes an element isolation component 160 that isolates the movement of the element from other elements. Because a layout manager dynamically determines element layout and position, changing the size and position of one element may cause the layout of other elements to change. This may be undesirable in the context of transition animation, so the element isolation component 160 may create a temporary container that contains the element the element animation component 150 is animating. The element isolation component 160 sets the container's size and position based on the incoming state, and then animates the element within the container from its outgoing state position and size to its incoming state position and size. Because the container size and position does not change during the transition, other elements are unaffected by the animation. When the transition is complete, the element isolation component 160 removes the temporary container from the layout. Alternatively or additionally, the layout manager may provide a flag that the layout animation system sets to inform the manager that a particular set of changes to an element do not affect other elements.
The computing device on which the system is implemented may include a central processing unit, memory, input devices (e.g., keyboard and pointing devices), output devices (e.g., display devices), and storage devices (e.g., disk drives or other non-volatile storage media). The memory and storage devices are computer-readable storage media that may be encoded with computer-executable instructions (e.g., software) that implement or enable the system. In addition, the data structures and message structures may be stored or transmitted via a data transmission medium, such as a signal on a communication link. Various communication links may be used, such as the Internet, a local area network, a wide area network, a point-to-point dial-up connection, a cell phone network, and so on.
Embodiments of the system may be implemented in various operating environments that include personal computers, server computers, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, programmable consumer electronics, digital cameras, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and so on. The computer systems may be cell phones, personal digital assistants, smart phones, personal computers, programmable consumer electronics, digital cameras, and so on.
The system may be described in the general context of computer-executable instructions, such as program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, and so on that perform particular tasks or implement particular abstract data types. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments.
The use of a temporary container 240 allows the system to animate just the UI element(s) of interest and not have other elements jumping erratically around the display. For example, if UI element 250 has a width set to “auto” so that it fills all available width, then the UI element 250 would reposition itself throughout the animation as the animated UI element moves from the starting position 210 to the intermediate position 220, and finally to the ending position 230. Instead, the temporary container 240 does not move during the animation so UI element 250 stays put. Moreover, UI element 250 may be going through its own layout animation because of the same layout change or even another layout change—and the system avoids any side effects from the current animation interfering with any other animations that are going on. When the transition animation is complete the system removes the temporary container 240 and the layout once again responds according to traditional layout rules unless a state transition occurs that invokes the layout animation system again.
Continuing in block 330, the system stores changed UI elements that have identified changing layout properties in a data structure. For example, the system may store the set of changed UI elements in an in memory list. Continuing in block 340, for each UI element identified, the system adds any related (e.g., sibling and parent) UI elements to the stored UI elements. Sibling and parent UI elements are commonly affected by a change in layout of a particular UI element. Child elements may also be included, though in many layout systems child elements will automatically and appropriately adapt to a layout change of the parent, so this may occur without further intervention by the system. Continuing in block 350, the system smoothly animates the transition of each identified UI element as described further with reference to
Continuing in block 420, the system stops any previously in progress transition. Because a layout UI is dynamic and users can interact with the UI, the system may be mid-transition of a previous state change when a new state change request is received. This is relatively common and just causes the system to animate the transition from the UI element's current position to the UI element's new ending position based on the new incoming state. Continuing in block 430, the system informs the UI element that the layout has changed to the new state. The system does this in a manner that the UI element updates all of its positional and layout properties but does not actually render a transition to the display. In this way, the system can access each of the UI element's properties at their ending values without yet showing any animation to the user.
Continuing in block 440, the system takes a snapshot of the position and other visible properties of the UI element now that the element is in the incoming state. This informs the system of where the UI element will end up when the animation is complete. The system may again store the UI element's effective opacity or may read it as needed from the element's current property values. Continuing in block 450, the system compares the two snapshots to determine whether the UI element moved or changed appearance in some manner (e.g., visibility, alignment). This system uses the comparison results to create a series of intermediate positions/appearances for the UI element to provide a smooth animation transition from the outgoing state to the incoming state.
Continuing in decision block 460, if the appearance of the UI element changed, then the system continues in block 470, else the system completes allowing the UI element to transition using traditional animation. Continuing in block 470, the system wraps the UI element in a temporary container (e.g., a canvas) that isolates the movement of the UI element during animation from other UI elements.
Continuing in block 480, the system animates the transition of the UI element by gradually modifying the values of appearance properties that simulate a change between the outgoing and incoming state in a way that makes the change of appearance of the UI element appear smooth to a user. The system translates from a level of abstraction where qualitative layout properties (e.g. alignment, column) are being animated to a level of abstraction where quantitative layout properties (e.g. width, height, x/y coordinates) are being animated. Inherent in that translation is that the former properties would appear jerky and are not conducive to a good user experience whereas the latter are smooth and are conducive to a good user experience. The system determines a number of intermediate appearances and a speed of transition between intermediate appearances in the series based on a specified duration for the transition between states. For colors the system may animate the red, green, and blue values gradually between the start and end color. In some embodiments, a traditional animation system and the layout animation system may run in parallel (not shown), with the traditional animation system handling non-layout property changes (e.g., color), and the layout animation system handling layout property changes (e.g., alignment). For layout property changes that modify the element's position, the system gradually modifies the coordinates of the UI element to produce smooth movement. For other properties, such as visibility, the system may modify a related property, such as opacity, to create an appearance of smoothly changing visibility.
Continuing in block 490, the system removes the temporary container from the UI element so that the UI element is placed back into its normal position in the layout hierarchy and so that any modifications to it will once again affect surrounding elements. After block 490, these steps conclude.
In some embodiments, the layout animation system invokes pluggable layout containers to perform animation of transitions. A particular container designer may want a type of movement between states other than the default described herein. The layout animation system asks the pluggable container for a start and end state, takes snapshots of the states, informs the container when the snapshots differ, and requests that the container perform the animation from the start state to the end state. The pluggable container may also provide a mapping from a changing layout property to an animation property (or properties) that the system uses for the transition resulting from the layout property.
In some embodiments, the layout animation system operates on some UI elements that use layout management alongside other UI elements that do not use layout management. For example, a UI may include elements positioned according to absolute coordinates alongside UI elements positioned relative to other UI elements. The system may animate changes to the relative positioned elements without affecting the absolute positioned elements. The system may also allow a designer to toggle smooth layout on or off for a particular state group, so that state groups with smooth layout on use the techniques described herein, and groups with smooth layout toggled off use traditional layout animation techniques.
In some embodiments, the layout animation system also takes care of transitions that are interrupted. It is possible to “redirect” a transition to a different morph target state at any given time. A UI layout may change to one state when a user toggles a button and another state when the user toggles the button again. If the user quickly toggles the button back and forth, then the layout may be mid-transition when the user's action reverses the transition. This system handles this smoothly by animating the layout according to the new final state by starting from the mid-transition position.
The following is an example of defining user interface elements along with states and transitions in XAML for use with Microsoft Silverlight. XAML UI elements are typically defined as:
The prefix vsm maps to the System.Windows namespace and assembly. For the prefix vsm, the mapping is declared as xmlns:vsm=“clr-namespace:System.Windows;assembly=System.Windows”.
The text “oneOrMoreVisualStateGroups” refers to one or more object elements of type VisualStateGroup. Designers add VisualStateGroup objects to the VisualStateManager.VisualStateGroups attached property to represent states of a control. Each VisualStateGroup contains a collection of mutually exclusive VisualState objects. Control authors transition between states by calling the VisualStateManager GoToState method. This separate allows designers (who may not have programming experience) to design user interfaces declaratively in XAML, while control authors that are programmers define logic behind the user interface of an application.
The following example creates a simple ControlTemplate for a Button that contains one Grid. The example sets the VisualStateManager.VisualStateGroups property on the Grid. The VisualStateManager.VisualStateGroups contains one VisualStateGroup, called CommonStates, which defines the MouseOver and Normal VisualState objects. When the user puts the mouse cursor over the Button, the Rectangle inside the Grid changes Visibility from Visible to Collapsed over one half second. When the user moves the mouse away from the button, the Rectangle reverts to Visible.
From the foregoing, it will be appreciated that specific embodiments of the layout animation system have been described herein for purposes of illustration, but that various modifications may be made without deviating from the spirit and scope of the invention. Accordingly, the invention is not limited except as by the appended claims.
