Patent Grant
10642416
The disclosed embodiments relate to keyboards and, more specifically, to improved techniques for receiving input via a dynamic input and output (I/O) device.
Conventional keyboards include any number of physical keys for inputting information (e.g., characters) into the computing device. Typically, the user presses or otherwise movably actuates a key to provide input corresponding to the key. In addition to providing inputs for characters, a keyboard may include movably actuated keys related to function inputs. For example, a keyboard may include an “escape” or “esc” key to allow a user to activate an escape or exit function. In many keyboards, a set of functions keys for function inputs are located in a “function row.” Typically, a set of keys for alphanumeric characters is located in a part of the keyboard that is closest to the user and a function row is located is a part of the keyboard that is further away from the user but adjacent to the alphanumeric characters. A keyboard may also include function keys that are not part of the aforementioned function row.
With the advent and popularity of portable computing devices, such as laptop computers, the area consumed by the dedicated keyboard may be limited by the corresponding size of a display. Compared with a peripheral keyboard for a desktop computer, a dedicated keyboard that is a component of a portable computing device may have fewer keys, smaller keys, or keys that are closer together to allow for a smaller overall size of the portable computing device.
Conventional dedicated keyboards are static and fixed in time regardless of the changes on a display. Furthermore, the functions of a software application displayed on a screen are typically accessed via toolbars and menus that a user interacts with by using a mouse. This periodically requires the user to switch modes and move the location of his/her hands between keyboard and mouse. Alternatively, the application's functions are accessed via complicated key combinations that require memory and practice. As such, it is desirable to provide an I/O device (and method for the I/O device) that addresses the shortcomings of conventional systems.
The embodiments described herein address the above shortcomings by providing dynamic and space efficient I/O devices and methods. Such devices and methods optionally complement or replace conventional input devices and methods. Such devices and methods also reduce the amount of mode switching (e.g., moving one's hands between keyboard and mouse, and also moving one's eyes from keyboard to display) required of a user and thereby reduce the number of inputs required to activate a desired function (e.g., number of inputs required to select menu options is reduced, as explained in more detail below). Such devices and methods also make more information available on a limited screen (e.g., a touch-sensitive secondary display is used to provide more information to a user and this information is efficiently presented using limited screen space). Such devices and methods also provide improved man-machine interfaces, e.g., by providing emphasizing effects to make information more discernable on the touch-sensitive secondary display 104, by providing sustained interactions so that successive inputs from a user directed to either a touch-sensitive secondary display or a primary display cause the device to provide outputs which are then used to facilitate further inputs from the user (e.g., a color picker is provided that allows users to quickly preview how information will be rendered on a primary display, by providing inputs at the touch-sensitive secondary display, as discussed below), and by requiring fewer interactions from users to achieve desired results (e.g., allowing users to login to their devices using a single biometric input, as discussed below). For these reasons and those discussed below, the devices and methods described herein reduce power usage and improve battery life of electronic devices.
(A1) In accordance with some embodiments, a method is performed at a computing system (e.g., computing system 100 or system 200,
Displaying application-specific and system-level affordances in a touch-sensitive secondary display in response to changes in focus made on a primary display provides the user with accessible affordances that are directly available via the touch-sensitive secondary display. Providing the user with accessible affordances that are directly accessibly via the touch-sensitive secondary display enhances the operability of the computing system and makes the user-device interface more efficient (e.g., by helping the user to access needed functions directly through the touch-sensitive secondary display with fewer interactions and without having to waste time digging through hierarchical menus to locate the needed functions) which, additionally, reduces power usage and improves battery life of the device by enabling the user to access the needed functions more quickly and efficiently. As well, the display of application-specific affordances on the touch-sensitive secondary display indicates an internal state of the device by providing affordances associated with the application currently in focus on the primary display.
(A2) In some embodiments of the method of A1, the computing system further comprises: (i) a primary computing device comprising the primary display, the processor, the memory, and primary computing device communication circuitry; and (ii) an input device comprising the housing, the touch screen display, the physical input mechanism, and input device communication circuitry for communicating with the primary computing device communication circuitry, wherein the input device is distinct and separate from the primary computing device.
(A3) In some embodiments of the method of any one of A1-A2, the physical input mechanism comprises a plurality of physical keys.
(A4) In some embodiments of the method of any one of A1-A3, the physical input mechanism comprises a touchpad.
(A5) In some embodiments of the method of any one of A1-A4, the application is executed by the processor in the foreground of the first user interface.
(A6) In some embodiments of the method of any one of A1-A5, the least one system-level affordance is configured upon selection to cause display of a plurality of system-level affordances corresponding to system-level functionalities on the touch screen display.
(A7) In some embodiments of the method of any one of A1-A3, the least one system-level affordance corresponds to one of a power control or escape control.
(A8) In some embodiments of the method of any one of A1-A7, at least one of the affordances displayed on the second user interface is a multi-function affordance.
(A9) In some embodiments of the method of A8, the method further includes: detecting a user touch input selecting the multi-function affordance; in accordance with a determination that the user touch input corresponds to a first type, performing a first function associated with the multi-function affordance; and, in accordance with a determination that the user touch input corresponds to a second type distinct from the first type, performing a second function associated with the multi-function affordance.
(A10) In some embodiments of the method of any one of A1-A9, the method further includes, in accordance with a determination that the active user interface element is not associated with the application executed by the computing system, displaying a third user interface on the touch screen display, including: (C) a second set of one or more affordances corresponding to operating system controls of the computing system, wherein the second set of one or more affordances are distinct from the first set of one or more affordances.
(A11) In some embodiments of the method of A10, the second set of one or more affordances is an expanded set of operating system controls that includes (B) the at least one system-level affordance corresponding to the at least one system-level functionality.
(A12) In some embodiments of the method of any one of A1-A11, the method further includes: detecting a user touch input selecting one of the first set of affordances; and, in response to detecting the user touch input: displaying a different set of affordances corresponding to functionalities of the application; and maintaining display of the at least one system-level affordance.
(A13) In some embodiments of the method of A12, the method further includes: detecting a subsequent user touch input selecting the at least one system-level affordance; and, in response to detecting the subsequent user touch input, displaying a plurality of system-level affordances corresponding to system-level functionalities and at least one application-level affordance corresponding to the application.
(A14) In some embodiments of the method of any one of A1-A13, the method further includes: after displaying the second user interface on the touch screen display, identifying a second active user interface element among the one or more user interface elements that is in focus on the primary display; determining whether the second active user interface element corresponds to a different application executed by the computing device; and, in accordance with a determination that the second active user interface element corresponds to the different application, displaying a fourth user interface on the touch screen display, including: (D) a third set of one or more affordances corresponding to the different application; and (E) the at least one system-level affordance corresponding to the at least one system-level functionality.
(A15) In some embodiments of the method of any one of A1-A14, the method further includes: after identifying the second active user interface element, determining whether a media item is being played by the computing system, wherein the media item is not associated with the different application; and, in accordance with a determination that media item is being played by the computing system, displaying at least one persistent affordance on the touch screen display for controlling the media item.
(A16) In some embodiments of the method of A15, the at least one persistent affordance displays feedback that corresponds to the media item.
(A17) In some embodiments of the method of any one of A1-A16, the method further includes: detecting a user input corresponding to an override key; and, in response to detecting the user input: ceasing to display at least the first set of one or more affordances of the second user interface on the touch screen display; and displaying a first set of default function keys.
(A18) In some embodiments of the method of A17, the method further includes: after displaying the first set of default function keys, detecting a swipe gesture on the touch screen display in a direction that is substantially parallel to a major axis of the touch screen display; and, in response to detecting the swipe gesture, displaying a second set of default function keys with at least one distinct function key.
(A19) In another aspect, a computing system is provided, the computing system including one or more processors, memory, a first housing that includes a primary display, and a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display. One or more programs are stored in the memory and configured for execution by one or more processors, the one or more programs including instructions for performing or causing performance of any one of the methods of A1-A18.
(A20) In an additional aspect, a non-transitory computer readable storage medium storing one or more programs is provided, the one or more programs including instructions that, when executed by one or more processors of a computing system with memory, a first housing that includes a primary display, and second a housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, cause the computing system to perform or cause performance of any one of the methods of A1-A18.
(A21) In one more aspect, a graphical user interface on a computing system with one or more processors, memory, a first housing that includes a primary display, a second housing at least partially containing a physical input mechanism and a touch-sensitive secondary display distinct from the primary display, the graphical user interface comprising user interfaces displayed in accordance with any of the methods of claims A1-A18.
(A22) In one other aspect, a computing device is provided. The computing device includes a first housing that includes a primary display, a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, and means for performing or causing performance of any of the methods of claims A1-A18.
(B1) In accordance with some embodiments, an input device is provided. The input device includes: a housing at least partially enclosing a plurality of components, the plurality of components including: (i) a plurality of physical keys (e.g., on keyboard 106,
Displaying application-specific and system-level affordances in a touch-sensitive secondary display in response to changes in focus made on a primary display provides the user with accessible affordances that are directly available via the touch-sensitive secondary display. Providing the user with accessible affordances that are directly accessibly via the touch-sensitive secondary display enhances the operability of the computing system and makes the user-device interface more efficient (e.g., by helping the user to access needed functions directly through the touch-sensitive secondary display with fewer interactions and without having to waste time digging through hierarchical menus to locate the needed functions) which, additionally, reduces power usage and improves battery life of the device by enabling the user to access the needed functions more quickly and efficiently. Furthermore, by dynamically updating affordances that are displayed in the touch-sensitive secondary display based on changes in focus at the primary display, the touch-sensitive secondary display is able to make more information available on a limited screen, and helps to ensure that users are provided with desired options right when those options are needed (thereby reducing power usage and extending battery life, because users do not need to waste power and battery life searching through hierarchical menus to located these desired options).
(B2) In some embodiments of the input device of B1, when the application is in focus on the computing device display the touch screen display is further configured to display at least to one of a power control affordance and an escape affordance.
(B3) In some embodiments of the input device of any one of B1-B2, the input device is a keyboard.
(B4) In some embodiments of the input device of B3, the computing device is a laptop computer that includes the keyboard.
(B5) In some embodiments of the input device of B3, the computing device is a desktop computer and the keyboard is distinct from the desktop computer.
(B6) In some embodiments of the input device of any one of B1-B5, the input device is integrated in a laptop computer.
(B7) In some embodiments of the input device of any one of B1-B6, the plurality of physical keys comprise a QWERTY keyboard.
(B8) In some embodiments of the input device of any one of B1-B7, the alphabet corresponds to the Latin alphabet.
(B9) In some embodiments of the input device of any one of B1-B8, the input device includes a touchpad.
(B10) In some embodiments of the input device of any one of B1-B9, the input device has a major dimension of at least 18 inches in length.
(B11) In some embodiments of the input device of any one of B1-B10, the short-range communication circuitry is configured to communicate less than 15 feet to the computing device.
(B12) In some embodiments of the input device of any one of B1-B11, the short-range communication circuitry corresponds to a wired or wireless connection to the computing device.
(B13) In some embodiments of the input device of any one of B1-B12, the input device includes a fingerprint sensor embedded in the touch screen display.
(C1) In accordance with some embodiments, a method is performed at a computing system (e.g., system 100 or system 200,
Allowing a user to quickly navigate through application-specific affordances in a touch-sensitive secondary display in response to swipe gestures provides the user with a convenient way to scroll through and quickly locate a desired function via the touch-sensitive secondary display. Providing the user with a convenient way to scroll through and quickly locate a desired function via the touch-sensitive secondary display enhances the operability of the computing system and makes the user-device interface more efficient (e.g., by helping the user to access needed functions directly through the touch-sensitive secondary display with fewer interactions and without having to waste time digging through hierarchical menus to locate the needed functions) which, additionally, reduces power usage and improves battery life of the device by enabling the user to access the needed functions more quickly and efficiently. Furthermore, by dynamically updating affordances that are displayed in the touch-sensitive secondary display in response to swipe gestures at the secondary display, the secondary display is able to make more information available on a limited screen, and helps to ensure that users are provided with desired options right when those options are needed (thereby reducing power usage and extending battery life, because users do not need to waste power and battery life searching through hierarchical menus to located these desired options).
(C2) In some embodiments of the method of C1, the second portion is displayed on the primary display in a compact view within the first user interface prior to detecting the swipe gesture, and the method includes: displaying the second portion on the primary display in an expanded view within the first user interface in accordance with the determination that the swipe gesture was performed in the second direction substantially perpendicular to the first direction.
(C3) In some embodiments of the method of C1, the first user interface for the application executed by the computing system is displayed on the primary display in a full-screen mode, and the first set of one or more affordances displayed on the touch screen display includes controls corresponding to the full-screen mode.
(C4) In some embodiments of the method of any one of C1-C3, the second set of one or more affordances and the third set of one or more affordances includes at least one system-level affordance corresponding to at least one system-level functionality.
(C5) In some embodiments of the method of any one of C1-C4, the method includes: after displaying the third set of one or more affordances on the touch screen display: detecting a user input selecting the first portion on the first user interface; and, in response to detecting the user input: ceasing to display the third set of one or more affordances on the touch screen display, wherein the third set of one or more affordances corresponds to the second portion of the application; and displaying the second set of one or more affordances, wherein the second set of one or more affordances corresponds to the first portion of the application.
(C6) In some embodiments of the method of any one of C1-C5, the first direction is substantially parallel to a major dimension of the touch screen display.
(C7) In some embodiments of the method of any one of C1-C5, the first direction is substantially perpendicular to a major dimension of the touch screen display.
(C8) In some embodiments of the method of any one of C1-C7, the first portion is one of a menu, tab, folder, tool set, or toolbar of the application, and the second portion is one of a menu, tab, folder, tool set, or toolbar of the application.
(C9) In another aspect, a computing system is provided, the computing system including one or more processors, memory, a first housing that includes a primary display, and a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display. One or more programs are stored in the memory and configured for execution by one or more processors, the one or more programs including instructions for performing or causing performance of any one of the methods of C1-C8.
(C10) In an additional aspect, a non-transitory computer readable storage medium storing one or more programs is provided, the one or more programs including instructions that, when executed by one or more processors of a computing system with memory, a first housing that includes a primary display, and second a housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, cause the computing system to perform or cause performance of any one of the methods of C1-C8.
(C11) In one more aspect, a graphical user interface on a computing system with one or more processors, memory, a first housing that includes a primary display, a second housing at least partially containing a physical input mechanism and a touch-sensitive secondary display distinct from the primary display, the graphical user interface comprising user interfaces displayed in accordance with any of the methods of claims C1-C8.
(C12) In one other aspect, a computing device is provided. The computing device includes a first housing that includes a primary display, a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, and means for performing or causing performance of any of the methods of claims C1-C8.
(D1) In accordance with some embodiments, a method of maintaining functionality of an application while in full-screen mode is performed at a computing system (e.g., system 100 or system 200,
Providing affordances for controlling an application via a touch-sensitive secondary display, while a portion of the application is displayed in a full-screen mode on a primary display, allows users to continue accessing functions that may no longer be directly displayed on a primary display. Allowing users to continue accessing functions that may no longer be directly displayed on a primary display provides the user with a quick and convenient way to access functions that may have become buried on the primary display and thereby enhances the operability of the computing system and makes the user-device interface more efficient (e.g., by helping the user to access needed functions directly through the touch-sensitive secondary display with fewer interactions and without having to waste time digging through hierarchical menus to locate the needed functions) which, additionally, reduces power usage and improves battery life of the device by enabling the user to access the needed functions more quickly and efficiently. Therefore, by shifting menu options from a primary display and to a touch-sensitive secondary display in order to make sure that content may be presented (without obstruction) in the full-screen mode, users are able to sustain interactions with the device and their workflow is not interrupted when shifting to the full-screen mode. Additionally, fewer interactions are required in order to access menu options while viewing full-screen content, as menu options that may have become buried behind content on the primary display is presented on the touch-sensitive secondary display for easy and quick access (and without having to exit full screen mode and then dig around looking for the menu options), thereby reducing power usage and improving battery life for the device.
(D2) In some embodiments of the method of D1, the second set of one or more affordances is the first set of one or more affordances.
(D3) In some embodiments of the method of any one of D1-D2, the second set of one or more affordances include controls corresponding to the full-screen mode.
(D4) In some embodiments of the method of any one of D1-D3, the method includes: detecting a user touch input selecting one of the second set of affordances displayed on the touch screen display; and, in response to detecting the user touch input, changing the portion of the first user interface for the application being displayed in the full-screen mode on the primary display according to the selected one of the second set of affordances.
(D5) In some embodiments of the method of any one of D1-D4, the method includes: after displaying the portion of the first user interface for the application in the full-screen mode on the primary display: detecting a subsequent user input for exiting the full-screen mode; and, in response to detecting the subsequent user input: displaying, on the primary display in the normal mode, the first user interface for the application executed by the computing system, the first user interface comprising the first set of one or more affordances associated with the application; and maintaining display of at least a subset of the second set of one or more affordances for controlling the application on the touch screen display, wherein the second set of one or more affordances correspond to the first set of one or more affordances.
(D6) In some embodiments of the method of any one of D1-D5, the user input for displaying at least the portion of the first user interface for the application in full-screen mode on the primary display is at least one of a touch input detected on the touch screen display and a control selected within the first user interface on the primary display.
(D7) In some embodiments of the method of any one of D1-D6, the second set of one or more affordances includes at least one system-level affordance corresponding to at least one system-level functionality.
(D8) In another aspect, a computing system is provided, the computing system including one or more processors, memory, a first housing that includes a primary display, and a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display. One or more programs are stored in the memory and configured for execution by one or more processors, the one or more programs including instructions for performing or causing performance of any one of the methods of D1-D7.
(D9) In an additional aspect, a non-transitory computer readable storage medium storing one or more programs is provided, the one or more programs including instructions that, when executed by one or more processors of a computing system with memory, a first housing that includes a primary display, and second a housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, cause the computing system to perform or cause performance of any one of the methods of D1-D7.
(D10) In one more aspect, a graphical user interface on a computing system with one or more processors, memory, a first housing that includes a primary display, a second housing at least partially containing a physical input mechanism and a touch-sensitive secondary display distinct from the primary display, the graphical user interface comprising user interfaces displayed in accordance with any of the methods of claims D1-D7.
(D11) In one other aspect, a computing device is provided. The computing device includes a first housing that includes a primary display, a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, and means for performing or causing performance of any of the methods of claims D1-D7.
(E1) In accordance with some embodiments, a method is performed at a computing system (e.g., system 100 or system 200,
Displaying received notifications at a touch-sensitive secondary display allows users to continue their work on a primary display in an uninterrupted fashion, and allows them to interact with the received notifications via the touch-sensitive secondary display. Allowing users to continue their work on the primary display in an uninterrupted fashion and allowing users to interact with the received notifications via the touch-sensitive secondary display provides users with a quick and convenient way to review and interact with received notifications and thereby enhances the operability of the computing system and makes the user-device interface more efficient (e.g., by helping the user to conveniently access received notifications directly through the touch-sensitive secondary display and without having to interrupt their workflow to deal with a received notification). Furthermore, displaying receiving notifications at the touch-sensitive secondary display provides an emphasizing effect for received notifications at the touch-sensitive secondary display, as the received notification is, in some embodiments, displayed as overlaying other affordances in the touch-sensitive secondary display, thus ensuring that the received notification is visible and easily accessible at the touch-sensitive secondary display.
(E2) In some embodiments of the method of E1, the method includes: prior to detecting the notification, detecting a user input selecting a notification setting so as to display notifications on the touch screen display and to not display notifications on the primary display.
(E3) In some embodiments of the method of any one of E1-E2, the method includes: detecting a user touch input on the touch screen display corresponding to the portion of the detected notification; in accordance with a determination that the user touch input corresponds to a first type, ceasing to display in the second user interface the portion of the detected notification on the touch screen display; and, in accordance with a determination that the user touch input corresponds to a second type distinct from the first type, performing an action associated with the detected notification.
(E4) In some embodiments of the method of any one of E1-E3, the portion of the notification displayed on the touch screen display prompts a user of the computing system to select one of a plurality of options for responding to the detected notification.
(E5) In some embodiments of the method of any one of E1-E4, the portion of the notification displayed on the touch screen display includes one or more suggested responses to the detected notification.
(E6) In some embodiments of the method of any one of E1-E5, the notification corresponds to an at least one of an incoming instant message, SMS, email, voice call, or video call.
(E6) In some embodiments of the method of any one of E1-E5, the notification corresponds to a modal alert issued by an application being executed by the processor of the computing system in response to a user input closing the application or performing an action within the application.
(E7) In some embodiments of the method of any one of E1-E7, the set of one or more affordances includes least one system-level affordance corresponding to at least one system-level functionality, and the notification corresponds to a user input selecting one or more portions of the input mechanism or the least one of a system-level affordance.
(E8) In another aspect, a computing system is provided, the computing system including one or more processors, memory, a first housing that includes a primary display, and a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display. One or more programs are stored in the memory and configured for execution by one or more processors, the one or more programs including instructions for performing or causing performance of any one of the methods of E1-E7.
(E9) In an additional aspect, a non-transitory computer readable storage medium storing one or more programs is provided, the one or more programs including instructions that, when executed by one or more processors of a computing system with memory, a first housing that includes a primary display, and second a housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, cause the computing system to perform or cause performance of any one of the methods of E1-E7.
(E10) In one more aspect, a graphical user interface on a computing system with one or more processors, memory, a first housing that includes a primary display, a second housing at least partially containing a physical input mechanism and a touch-sensitive secondary display distinct from the primary display, the graphical user interface comprising user interfaces displayed in accordance with any of the methods of claims E1-E7.
(E11) In one other aspect, a computing device is provided. The computing device includes a first housing that includes a primary display, a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, and means for performing or causing performance of any of the methods of claims E1-E7.
(F1) In accordance with some embodiments, a method of moving user interface portions is performed at a computing system (e.g., system 100 or system 200,
Allowing a user to quickly move user interface portions (e.g., menus, notifications, etc.) from a primary display and to a touch-sensitive secondary display provides the user with a convenient and customized way to access the user interface portions. Providing the user with a convenient and customized way to access the user interface portions via the touch-sensitive secondary display enhances the operability of the computing system and makes the user-device interface more efficient (e.g., by helping the user to access user interface portions directly through the touch-sensitive secondary display with fewer interactions and without having to waste time looking for a previously viewed (and possibly buried) user interface portion) which, additionally, reduces power usage and improves battery life of the device by enabling the user to access needed user interface portions more quickly and efficiently. Furthermore, displaying user interface portions at the touch-sensitive secondary display in response to user input provides an emphasizing effect for the user interface portions at the touch-sensitive secondary display, as a respective user interface portions is, in some embodiments, displayed as overlaying other affordances in the touch-sensitive secondary display, thus ensuring that the respective user interface portion is visible and easily accessible at the touch-sensitive secondary display.
(F2) In some embodiments of the method of F1, the respective portion of the user interface is a menu corresponding to the application executed by the computing system.
(F3) In some embodiments of the method of any one of F1-F2, the respective portion of the user interface is one of a notification and a modal alert.
(F4) In some embodiments of the method of any one of F1-F3, the predefined action criteria are satisfied when the user input is a dragging gesture that drags the respective portion of the user interface to a predefined location of the primary display.
(F5) In some embodiments of the method of any one of F1-F3, the predefined action criteria are satisfied when the user input is a predetermined input corresponding to moving the respective portion of the user interface to the touch screen display.
(F6) In some embodiments of the method of any one of F1-F5, the method includes: in response to detecting the user input, and in accordance with a determination that the user input does not satisfy the predefined action criteria: maintaining display of the respective portion of the user interface on the primary display; and maintaining display of the set of one or more affordances on the touch screen display.
(F7) In some embodiments of the method of any one of F1-F6, the set of one or more affordances includes at least one system-level affordance corresponding to at least one system-level functionality, the method includes: after displaying the representation of the respective portion of the user interface on the touch screen display, maintaining display of the at least one system-level affordance on the touch screen display.
(F8) In some embodiments of the method of any one of F1-F7, the representation of the respective portion of the user interface is overlaid on the set of one or more affordances on the touch screen display.
(F9) In another aspect, a computing system is provided, the computing system including one or more processors, memory, a first housing that includes a primary display, and a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display. One or more programs are stored in the memory and configured for execution by one or more processors, the one or more programs including instructions for performing or causing performance of any one of the methods of F1-F8.
(F10) In an additional aspect, a non-transitory computer readable storage medium storing one or more programs is provided, the one or more programs including instructions that, when executed by one or more processors of a computing system with memory, a first housing that includes a primary display, and second a housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, cause the computing system to perform or cause performance of any one of the methods of F1-F8.
(F11) In one more aspect, a graphical user interface on a computing system with one or more processors, memory, a first housing that includes a primary display, a second housing at least partially containing a physical input mechanism and a touch-sensitive secondary display distinct from the primary display, the graphical user interface comprising user interfaces displayed in accordance with any of the methods of claims F1-F8.
(F12) In one other aspect, a computing device is provided. The computing device includes a first housing that includes a primary display, a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, and means for performing or causing performance of any of the methods of claims F1-F8.
(G1) In accordance with some embodiments, a method is performed at a computing system (e.g., system 100 or system 200,
Allowing a user to quickly navigate through user interface objects on a primary display (e.g., browser tabs) by providing inputs at a touch-sensitive secondary display provides the user with a convenient way to quickly navigate through the user interface objects. Providing the user with a convenient way to quickly navigate through the user interface objects via the touch-sensitive secondary display (and reducing the number of inputs needed to navigate through the user interface objects, thus requiring fewer interactions to navigate through the user interface objects) enhances the operability of the computing system and makes the user-device interface more efficient (e.g., by requiring a single input or gesture at a touch-sensitive secondary display to navigate through user interface objects on a primary display) which, additionally, reduces power usage and improves battery life of the device by enabling the user to navigate through user interface objects on the primary display more quickly and efficiently. Moreover, as users provide an input at the touch-sensitive display (e.g., a swipe gesture) to navigate through the user interface objects on the primary display, each contacted affordance at the touch-sensitive display (that corresponds to one of the user interface objects) is visually distinguished from other affordances (e.g., a respective contacted affordance is magnified and a border may be highlighted), thus making information displayed on the touch-sensitive secondary display more discernable to the user.
(G2) In some embodiments of the method of G1, the method includes: detecting continuous travel of the swipe gesture across the touch-sensitive secondary display, including the swipe gesture contacting a third affordance that represents a third user interface object. In response to detecting that the swipe gesture contacts the third affordance, the method includes: updating the primary display to display associated content for the third user interface object.
(G3) In some embodiments of the method of any one of G1-G2, each affordance in the set of affordance includes a representation of respective associated content for a respective user interface object of the plurality.
(G4) In some embodiments of the method of any one of G1-G3, the method includes: before detecting the swipe gesture, detecting an initial contact with the touch-sensitive secondary display over the first affordance. In response to detecting the initial contact, the method includes: increasing a magnification level (or display size) of the first affordance.
(G5) In some embodiments of the method of any one of G1-G4, the application is a web browsing application, and the plurality of user interface objects each correspond to web-browsing tabs.
(G6) In some embodiments of the method of G6, the method includes: detecting an input at a URL-input portion of the web browsing application on the primary display. In response to detecting the input, the method includes: updating the touch-sensitive secondary display to include representations of favorite URLs.
(G7) In some embodiments of the method of any one of G1-G4, the application is a photo-browsing application, and the plurality of user interface objects each correspond to individual photos.
(G8) In some embodiments of the method of any one of G1-G4, the application is a video-editing application, and the plurality of user interface object each correspond to individual frames in a respective video.
(G9) In another aspect, a computing system is provided, the computing system including one or more processors, memory, a first housing that includes a primary display, and a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display. One or more programs are stored in the memory and configured for execution by one or more processors, the one or more programs including instructions for performing or causing performance of any one of the methods of G1-G8.
(G10) In an additional aspect, a non-transitory computer readable storage medium storing one or more programs is provided, the one or more programs including instructions that, when executed by one or more processors of a computing system with memory, a first housing that includes a primary display, and second a housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, cause the computing system to perform or cause performance of any one of the methods of G1-G8.
(G11) In one more aspect, a graphical user interface on a computing system with one or more processors, memory, a first housing that includes a primary display, a second housing at least partially containing a physical input mechanism and a touch-sensitive secondary display distinct from the primary display, the graphical user interface comprising user interfaces displayed in accordance with any of the methods of claims G1-G8.
(G12) In one other aspect, a computing device is provided. The computing device includes a first housing that includes a primary display, a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, and means for performing or causing performance of any of the methods of claims G1-G8.
(H1) In accordance with some embodiments, a method is performed at a computing system (e.g., system 100 or system 200,
Allowing a user to quickly navigate through search results on a primary display by providing inputs at a touch-sensitive secondary display provides the user with a convenient way to quickly navigate through the search results. Providing the user with a convenient way to quickly navigate through the search results via the touch-sensitive secondary display (and reducing the number of inputs needed to navigate through the search results, thus requiring fewer interactions from a user to browse through numerous search results quickly) enhances the operability of the computing system and makes the user-device interface more efficient (e.g., by requiring a single input or gesture at a touch-sensitive secondary display to navigate through numerous search results on a primary display) which, additionally, reduces power usage and improves battery life of the device by enabling the user to navigate through search results on the primary display more quickly and efficiently. Moreover, as users provide an input at the touch-sensitive display (e.g., a swipe gesture) to navigate through the search on the primary display, each contacted affordance at the touch-sensitive display (that corresponds to one of the search results) is visually distinguished from other affordances (e.g., a respective contacted affordance is magnified and a border may be highlighted), thus making information displayed on the touch-sensitive secondary display more discernable to the user.
(H2) In some embodiments of the method of H1, changing focus includes modifying, on the primary display, a visual characteristic of the particular search result (e.g., displaying the particular search result with a larger font size).
(H3) In some embodiments of the method of any one of H1-H2, the method includes: detecting a gesture that moves across at least two of the respective representations on the touch-sensitive secondary display. In response to detecting the gesture, the method includes: changing focus on the primary display to respective search results that correspond to the at least two of the respective representations as the swipe gestures moves across the at least two of the respective representations.
(H4) In some embodiments of the method of H3, the method includes: in accordance with a determination that a speed of the gesture is above a threshold speed, changing focus on the primary display to respective search results in addition to those that correspond to the at least two of the respective representations (e.g., if above the threshold speed, cycle through more search results in addition to those contacted during swipe).
(H5) In some embodiments of the method of any one of H3-H4, the gesture is a swipe gesture.
(H6) In some embodiments of the method of any one of H3-H4, the gesture is a flick gesture.
(H7) In some embodiments of the method of any one of H1-H6, the representations are tick marks that each correspond to respective search results of the search results.
(H8) In some embodiments of the method of H7, the tick marks are displayed in a row on the touch-sensitive secondary display in an order that corresponds to an ordering of the search results on the primary display.
(H9) In some embodiments of the method of any one of H1-H8, the request to search within the content is a request to locate a search string within the content, and the plurality of search results each include at least the search string.
(H9) In some embodiments of the method of H8, displaying the plurality of search results includes highlighting the search string for each of the plurality of search results.
(H10) In another aspect, a computing system is provided, the computing system including one or more processors, memory, a first housing that includes a primary display, and a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display. One or more programs are stored in the memory and configured for execution by one or more processors, the one or more programs including instructions for performing or causing performance of any one of the methods of H1-H9.
(H11) In an additional aspect, a non-transitory computer readable storage medium storing one or more programs is provided, the one or more programs including instructions that, when executed by one or more processors of a computing system with memory, a first housing that includes a primary display, and second a housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, cause the computing system to perform or cause performance of any one of the methods of H1-H9.
(H12) In one more aspect, a graphical user interface on a computing system with one or more processors, memory, a first housing that includes a primary display, a second housing at least partially containing a physical input mechanism and a touch-sensitive secondary display distinct from the primary display, the graphical user interface comprising user interfaces displayed in accordance with any of the methods of claims H1-H9.
(H13) In one other aspect, a computing device is provided. The computing device includes a first housing that includes a primary display, a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, and means for performing or causing performance of any of the methods of claims H1-H9.
(I1) In accordance with some embodiments, a method is performed at a computing system (e.g., system 100 or system 200,
Allowing a user to quickly and easily edit event details at a touch-sensitive secondary display provides the user with a convenient way to quickly edit event details without having to perform extra inputs (e.g., having to jump back and forth between using a keyboard and using a trackpad to modify the event details). Providing the user with a convenient way to quickly edit event details via the touch-sensitive secondary display (and reducing the number of inputs needed to edit the event details, thus requiring fewer interactions to achieve a desired result of editing event details) enhances the operability of the computing system and makes the user-device interface more efficient (e.g., by requiring a single input or gesture at a touch-sensitive secondary display to quickly edit certain event details) which, additionally, reduces power usage and improves battery life of the device by enabling the user to edit event details more quickly and efficiently. Additionally, by updating the primary display in response to inputs at the touch-sensitive secondary display (e.g., to show updated start and end times for an event), a user is able to sustain interactions with the device in an efficient way by providing inputs to modify the event and then immediately seeing those modifications reflected on the primary display, so that the user is then able to decide whether to provide an additional input or not.
(I2) In some embodiments of the method of I1, the method includes: detecting, via the touch-sensitive secondary display, an input at the user interface control that modifies the range of time. In response to detecting the input: (i) modifying at least one of the start time and the end time for the event in accordance with the input; and (ii) displaying, on the primary display, a modified range of time for the event in accordance with the input.
(I3) In some embodiments of the method of I2, the method includes: saving the event with the modified start and/or end time to the memory of the computing system.
(I4) in some embodiments of the method of any one of I1-I3, the input that modifies the range of time is a press input that remains in contact with the affordance for more than a threshold amount of time and then moves at least a portion the affordance on the touch-sensitive secondary display.
(I5) in some embodiments of the method of any one of I1-I3, the input that modifies the range of time is a swipe gesture that moves across the touch-sensitive secondary display and causes the computing system to select a new start time and a new end time for the event, wherein the new start and end times correspond to a time slot that is of a same duration covered by the start and end times
(I6) In another aspect, a computing system is provided, the computing system including one or more processors, memory, a first housing that includes a primary display, and a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display. One or more programs are stored in the memory and configured for execution by one or more processors, the one or more programs including instructions for performing or causing performance of any one of the methods of I1-I5.
(I7) In an additional aspect, a non-transitory computer readable storage medium storing one or more programs is provided, the one or more programs including instructions that, when executed by one or more processors of a computing system with memory, a first housing that includes a primary display, and second a housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, cause the computing system to perform or cause performance of any one of the methods of I1-I5.
(I8) In one more aspect, a graphical user interface on a computing system with one or more processors, memory, a first housing that includes a primary display, a second housing at least partially containing a physical input mechanism and a touch-sensitive secondary display distinct from the primary display, the graphical user interface comprising user interfaces displayed in accordance with any of the methods of claims I1-I5.
(I9) In one other aspect, a computing device is provided. The computing device includes a first housing that includes a primary display, a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, and means for performing or causing performance of any of the methods of claims I1-I5.
(J1) In accordance with some embodiments, a method is performed at a computing system (e.g., system 100 or system 200,
Allowing a user to efficiently utilize external devices via a touch-sensitive secondary display provides the user with a convenient way to access functions that may otherwise be buried in menus. Providing the user with a convenient way to access functions for external devices that may otherwise be buried in menus (and, therefore, reducing the number of inputs needed to access the functions, thus requiring fewer interactions in order to use external devices) enhances the operability of the computing system and makes the user-device interface more efficient (e.g., by requiring a single input or gesture at a touch-sensitive secondary display to perform a certain function for an external device) which, additionally, reduces power usage and improves battery life of the device by enabling the user to interact with external devices more quickly and efficiently. In this way, the touch-sensitive secondary display also conveys information about an internal state of the device (by reflecting a connecting status between the device and the external device, and allowing users to easily interact with the external device).
(J2) In some embodiments of the method of J1, the method includes: receiving, via the touch-sensitive secondary display, a selection of a first affordance that corresponds to a first function available via the external device. In response to receiving the selection, the method includes: initiating performance of the first function.
(J3) In some embodiments of the method of any one of J1-J2, the external device is an additional display, distinct from the primary display and the touch-sensitive display.
(J4) In some embodiments of the method of J3, the plurality of affordances include a first affordance that, when selected, causes the computing system to initiate performance of a display mirroring function via the additional display.
(J5) In another aspect, a computing system is provided, the computing system including one or more processors, memory, a first housing that includes a primary display, and a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display. One or more programs are stored in the memory and configured for execution by one or more processors, the one or more programs including instructions for performing or causing performance of any one of the methods of J1-J4.
(J6) In an additional aspect, a non-transitory computer readable storage medium storing one or more programs is provided, the one or more programs including instructions that, when executed by one or more processors of a computing system with memory, a first housing that includes a primary display, and second a housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, cause the computing system to perform or cause performance of any one of the methods of J1-J4.
(J7) In one more aspect, a graphical user interface on a computing system with one or more processors, memory, a first housing that includes a primary display, a second housing at least partially containing a physical input mechanism and a touch-sensitive secondary display distinct from the primary display, the graphical user interface comprising user interfaces displayed in accordance with any of the methods of claims J1-J4.
(J8) In one other aspect, a computing device is provided. The computing device includes a first housing that includes a primary display, a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, and means for performing or causing performance of any of the methods of claims J1-J4.
(K1) In accordance with some embodiments, a method is performed at a computing system (e.g., system 100 or system 200,
Allowing a user to quickly and easily preview how characters will look within an application on a primary display by providing an intuitive input at a touch-sensitive secondary display provides the user with a convenient way to quickly preview how characters will look within the application. Providing the user with a convenient way to quickly preview how characters will look within an application enhances the operability of the computing system and makes the user-device interface more efficient (e.g., by requiring a single input or gesture at a touch-sensitive secondary display to quickly preview how characters will look, thus requiring fewer interactions to preview how these characters will look on the primary display) which, additionally, reduces power usage and improves battery life of the device by enabling the user to preview characters more quickly and efficiently. In this way, users are provided with efficient and sustained interactions with their devices, as the users are permitted to continue previewing characters and then continue providing inputs until a desired character is located.
(K2) In some embodiments of the method of K1, the method includes: detecting, via the touch-sensitive secondary display, a second input over a first affordance that corresponds to a first character of the distinct characters. In response to detecting the second input, the method includes: displaying on the primary display a preview of the first character at the particular location while the input remains in contact with the first affordance.
(K3) In some embodiments of the method of K2, the method includes: detecting, via the touch-sensitive secondary display, movement of the second input from the first affordance and to a second affordance that corresponds to a second character of the distinct characters. In response to detecting the movement of the second input from the first affordance and to the second affordance, the method includes: replacing the preview of the first character with a preview of the second character.
(K4) In some embodiments of the method of any one of K1-K3, the method includes: determining affordances to include in the set of affordances based at least in part on textual content included in the user interface.
(K5) In some embodiments of the method of K4, the determining is conducted in response to detecting that a user has modified textual content included in the user interface.
(K6) In some embodiments of the method of any one of K1-K5, the method includes: detecting liftoff of the second input while it is contact with second affordance. In response to detecting liftoff, the method includes: updating the user interface to include the second user interface element.
(K7) In some embodiments of the method of any one of K1-K5, the method includes: detecting an additional input while second input is in contact with second affordance. In response to detecting the additional input, the method includes: updating the user interface to include the second user interface element.
(K8) In some embodiments of the method of any one of K1-K7, the method includes: as the second input continues to move across the touch-sensitive secondary display, displaying previews for respective characters of the distinct characters as corresponding affordances in the set of affordances are contacted by the second input.
(K9) In some embodiments of the method of any one of K1-K8, the preview of second character remains displayed on the primary display while the input remains in contact with the second affordance
(K10) In another aspect, a computing system is provided, the computing system including one or more processors, memory, a first housing that includes a primary display, and a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display. One or more programs are stored in the memory and configured for execution by one or more processors, the one or more programs including instructions for performing or causing performance of any one of the methods of K1-K9.
(K11) In an additional aspect, a non-transitory computer readable storage medium storing one or more programs is provided, the one or more programs including instructions that, when executed by one or more processors of a computing system with memory, a first housing that includes a primary display, and second a housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, cause the computing system to perform or cause performance of any one of the methods of K1-K9.
(K12) In one more aspect, a graphical user interface on a computing system with one or more processors, memory, a first housing that includes a primary display, a second housing at least partially containing a physical input mechanism and a touch-sensitive secondary display distinct from the primary display, the graphical user interface comprising user interfaces displayed in accordance with any of the methods of claims K1-K9.
(K13) In one other aspect, a computing device is provided. The computing device includes a first housing that includes a primary display, a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, and means for performing or causing performance of any of the methods of claims K1-K9.
(L1) In accordance with some embodiments, a method is performed at a computing system (e.g., system 100 or system 200,
Therefore, users are provided with an intuitive way to modify visual characteristics that are used to render content within a content-editing application on a primary display by providing inputs at a touch-sensitive secondary display. Providing users with an intuitive way to modify visual characteristics in this way enhances the operability of the computing system and makes the user-device interface more efficient (e.g., by requiring a single input or gesture at a touch-sensitive secondary display to quickly preview how certain visual characteristics will look when used to render content on the primary display) which, additionally, reduces power usage and improves battery life of the device by enabling the user to preview changes to visual characteristics in a quicker and more efficient way. In this way, users are provided with efficient and sustained interactions with their devices, as the users are permitted to continue previewing how modifications to a visual characteristic will look on the primary display and then continue providing inputs until a desired modification for the visual characteristic is located.
(L2) In some embodiments of the method of L1, the method includes: detecting, via the touch-sensitive secondary display, an input at the user interface control that selects a first value for the at least one visual characteristic. After detecting the input, the method includes: rendering content in the content-editing application using the first value for the at least one visual characteristic (e.g., all new content added to the content-editing application is rendered using the first value and/or currently selected content is rendered using the first value).
(L3) In some embodiments of the method of any one of L1-L2, the user interface control includes respective controls that each correspond to a respective value for the at least one visual characteristic along a sliding scale of values.
(L4) In some embodiments of the method of L3, the sliding scale of values represents distinct shades of color.
(L5) In some embodiments of the method of L4, the first value corresponds to a first shade of a first color and the method includes: in accordance with a determination that the input satisfies predetermined criteria (remains in contact for more than threshold amount of time), modifying the user interface control on the touch-sensitive secondary display to include options for selecting other shades of the first color, distinct from the first shade of the first color.
(L6) In some embodiments of the method of any one of L1-L5, the method includes: before rendering the content, receiving a selection of the content, and rendering the content includes presenting a preview of the content using the first value for the at least one visual characteristic.
(L7) In some embodiments of the method of L6, the preview is presented while the input remains in contact with the touch-sensitive secondary display.
(L8) In some embodiments of the method of L7, the method includes: in response to detecting liftoff of the input, ceasing to display the preview.
(L9) In some embodiments of the method of L8, the method includes: in response to detecting liftoff of the input, displaying the portion of the editable content with the modified value for the at least one visual characteristic
(L10) In another aspect, a computing system is provided, the computing system including one or more processors, memory, a first housing that includes a primary display, and a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display. One or more programs are stored in the memory and configured for execution by one or more processors, the one or more programs including instructions for performing or causing performance of any one of the methods of L1-L9.
(L11) In an additional aspect, a non-transitory computer readable storage medium storing one or more programs is provided, the one or more programs including instructions that, when executed by one or more processors of a computing system with memory, a first housing that includes a primary display, and second a housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, cause the computing system to perform or cause performance of any one of the methods of L1-L9.
(L12) In one more aspect, a graphical user interface on a computing system with one or more processors, memory, a first housing that includes a primary display, a second housing at least partially containing a physical input mechanism and a touch-sensitive secondary display distinct from the primary display, the graphical user interface comprising user interfaces displayed in accordance with any of the methods of claims L1-L9.
(L13) In one other aspect, a computing device is provided. The computing device includes a first housing that includes a primary display, a second housing at least partially containing a physical keyboard and a touch-sensitive secondary display distinct from the primary display, and means for performing or causing performance of any of the methods of claims L1-L9.
(M1) In accordance with some embodiments, a method is performed at an electronic device (e.g., system 100 or system 200,
Therefore, users are provided with an intuitive way to access a user-specific login page by providing a single input at a biometric sensor. Providing users with an intuitive way to access a user-specific login page in this way enhances the operability of the computing system and makes the user-device interface more efficient (e.g., by requiring a single input or gesture at the biometric sensor to quickly access an appropriate, user-specific login page, thus fewer interactions are required to reach a user-specific login page) which, additionally, reduces power usage and improves battery life of the device by enabling the user to access the login page via single input.
(M2) In some embodiments of the method of M1, the method includes, in response to receiving the biometric information: in accordance with a determination that the biometric information is consistent with biometric information for the first user account of the plurality of user accounts while the first user account has an active session on the device, unlocking the device with respect to the first user account (e.g., without requiring additional user input).
(M3) In some embodiments of the method of any one of M1-M2, the method includes, in response to receiving the biometric information: in accordance with a determination that the biometric information is consistent with biometric information for the second user account of the plurality of user accounts while the second user account has an active session on the device, unlocking the device with respect to the second user account (e.g., without requiring additional user input).
(M4) In some embodiments of the method of any one of M1-M3, the method includes, in response to receiving the biometric information: in accordance with a determination that the biometric information is not consistent with biometric information for the any user account of the device, maintaining the device in the locked state.
(M5) In some embodiments of the method of any one of M1-M4, the log-in user interface includes a plurality of selectable affordances that correspond to the plurality of user accounts.
(M6) In some embodiments of the method of any one of M1-M5, the method includes, while displaying the prompt to input a log-in credential for the first user account, receiving entry of a log-in credential. In response to receiving entry of the log-in credential, the method includes: (i) in accordance with a determination that the log-in credential is consistent with a log-in credential for the first user account, unlocking the device with respect to the first user account; and (ii) in accordance with a determination that the log-in credential is not consistent with a log-in credential for the first user account, maintaining the device in the locked state.
(M7) In some embodiments of the method of any one of M1-M6, the method includes: while displaying the prompt to input a log-in credential for the second user account, receiving entry of a log-in credential. In response to receiving entry of the log-in credential: (i) in accordance with a determination that the log-in credential is consistent with a log-in credential for the second user account, unlocking the device with respect to the first user account; and (ii) in accordance with a determination that the log-in credential is not consistent with a log-in credential for the second user account, maintaining the device in the locked state.
(M8) In some embodiments of the method of any one of M1-M7, the log-in user interface includes instructions to provide biometric information.
(M9) In some embodiments of the method of any one of M1-M8, the device includes a secondary display that is adjacent to the biometric sensor (e.g., the touch-sensitive secondary display); and the method includes, while displaying the log-in user interface on the display of the device, displaying instructions at the secondary display to provide biometric information via the biometric sensor.
(M10) In some embodiments of the method of any one of M1-M9, the biometric sensor is a fingerprint sensor.
(M11) In some embodiments of the method of any one of M1-M9, the biometric sensor is a facial detection sensor.
(M12) In some embodiments of the method of any one of M1-M9, the biometric sensor is a retina scanner.
(M13) In another aspect, an electronic device is provided, the electronic device including one or more processors, memory, a display, and a biometric sensor. One or more programs are stored in the memory and configured for execution by one or more processors, the one or more programs including instructions for performing or causing performance of any one of the methods of M1-M12.
(M14) In an additional aspect, a non-transitory computer readable storage medium storing one or more programs is provided, the one or more programs including instructions that, when executed by one or more processors of a computing system with a display and a biometric sensor, cause the computing system to perform or cause performance of any one of the methods of M1-M12.
(M15) In one more aspect, a graphical user interface is provided on a computing system with one or more processors, memory, a display, and a biometric sensor, the graphical user interface comprising user interfaces displayed in accordance with any of the methods of claims M1-M12.
(M16) In one other aspect, a computing device is provided. The computing device includes a display and a biometric sensor, and means for performing or causing performance of any of the methods of claims M1-M12.
(N1) In accordance with some embodiments, a method is performed at an electronic device (e.g., system 100 or system 200,
Therefore, users are provided with an intuitive way to access an active, user-specific session on an electronic device by providing a single input at a biometric sensor. Providing users with an intuitive way to access an active, user-specific session in this way enhances the operability of the electronic device and makes the user-device interface more efficient (e.g., by requiring a single input or gesture at the biometric sensor to gain immediate access to the active session, therefore requiring fewer interactions to switch user accounts and login to the device) which, additionally, reduces power usage and improves battery life of the device by enabling the user to access the active session via single input.
(N2) In some embodiments of the method of N1, the method includes, in response to receiving the biometric information: in accordance with a determination that the input meets second-user switching criteria while the second user account does not have an active session on the device, wherein the second-user switching criteria include a requirement that biometric information detected during the input with the input element is consistent with biometric information for the second user account of the plurality of user accounts, displaying, on the display, a prompt to input a log-in credential for the second user account.
(N3) In some embodiments of the method of any one of N1-N2, the method includes, in response to receiving the biometric information: in accordance with a determination that the input meets third-user switching criteria while a third user account has an active session on the device, wherein the third-user switching criteria include a requirement that biometric information detected during the input with the input element is consistent with biometric information for the third user account of the plurality of user accounts: (i) unlocking the device with respect to the third user account; (ii) locking the device with respect to the first user account; and (iii) replacing display of the user interface associated with the first account with a user interface associated with the third user account.
(N4) In some embodiments of the method of any one of N1-N3, the method includes, in response to receiving the biometric information: in accordance with a determination that the input meets third-user switching criteria while the third user account does not have an active session on the device, wherein the third-user switching criteria include a requirement that biometric information detected during the input with the input element is consistent with biometric information for the third user account of the plurality of user accounts, displaying, on the display, a prompt to input a log-in credential for the third user account.
(N5) In some embodiments of the method of any one of N1-N2, the input element is a button, and the input via the input element with the integrated biometric sensor includes detecting a press input via the button.
(N6) In some embodiments of the method of N5, the second-user switching criteria include a criterion that the press input lasts for less than a first threshold amount of time; and the method includes, in response to receiving the press input via the input element with the integrated biometric sensor in accordance with a determination that the press input with the button lasts longer than the first threshold amount of time, putting the device into a low power mode (e.g., the low power mode corresponds to a suspended state of the electronic device in which the display is turned off).
(N7) In some embodiments of the method of N5, the second-user switching criteria include a criterion that the button press for less than a first threshold amount of time; and the method includes, in response to receiving the input via the input element with the integrated biometric sensor: (i) in accordance with a determination that the press input with the button lasts longer than the first threshold amount of time and less than a second threshold amount of time, putting the device into a low power mode (e.g., the low power mode corresponds to a sleep/suspended state of the electronic device in which the display is turned off and this low power mode is entered upon an end of the press input); and (ii) in accordance with a determination that the press input with the button lasts longer than the second threshold amount of time, displaying a menu of options for changing a state of the device (e.g., the menu of options include shut down, restart, sleep/suspend options that, when selected, shut down, restart, or sleep/suspend the device, respectively).
(N8) In some embodiments of the method of N7, the method includes, in response to receiving the input via the input element with the integrated biometric sensor: in accordance with a determination that the press input with the button lasts longer than a third threshold amount of time that is greater than the second threshold amount of time, restarting the device.
(N9) In some embodiments of the method of any one of N1-N8, the method includes, after replacing display of the user interface associated with the first account with a user interface associated with the second user account: while displaying the user interface that is associated with the second user account, receiving a second input via the input element with the integrated biometric sensor. In response to receiving the second input via the input element with the integrated biometric sensor: in accordance with a determination that the second input meets first-user switching criteria while the first user account has an active session on the device, wherein the first-user switching criteria include a requirement that biometric information detected during the input with the input element is consistent with biometric information for the first user account of the plurality of user accounts: (i) unlocking the device with respect to the first user account; (ii) locking the device with respect to the second user account; and (iii) replacing display of the user interface associated with the second account with a user interface associated with the first user account.
(N10) In another aspect, an electronic device is provided, the electronic device including one or more processors, memory, a display, and a biometric sensor. One or more programs are stored in the memory and configured for execution by one or more processors, the one or more programs including instructions for performing or causing performance of any one of the methods of N1-N9.
(N11) In an additional aspect, a non-transitory computer readable storage medium storing one or more programs is provided, the one or more programs including instructions that, when executed by one or more processors of a computing system with a display and a biometric sensor, cause the computing system to perform or cause performance of any one of the methods of N1-N9.
(N12) In one more aspect, a graphical user interface is provided on a computing system with one or more processors, memory, a display, and a biometric sensor, the graphical user interface comprising user interfaces displayed in accordance with any of the methods of claims N1-N9.
(N13) In one other aspect, a computing device is provided. The computing device includes a display and a biometric sensor, and means for performing or causing performance of any of the methods of claims N1-N9.
The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.
For a better understanding of the various described embodiments, reference should be made to the Description of Embodiments below, in conjunction with the following drawings in which like reference numerals refer to corresponding parts throughout the figures.
Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the various described embodiments. However, it will be apparent to one of ordinary skill in the art that the various described embodiments may be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.
It will also be understood that, although the terms first, second, etc. are, in some instances, used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first contact could be termed a second contact, and, similarly, a second contact could be termed a first contact, without departing from the scope of the various described embodiments. The first contact and the second contact are both contacts, but they are not the same contact.
The terminology used in the description of the various described embodiments herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of the various described embodiments and the appended claims, the singular forms “a”, “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and/or” as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms “includes,” “including,” “comprises,” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
As used herein, the term “if” is, optionally, construed to mean “when” or “upon” or “in response to determining” or “in response to detecting,” depending on the context. Similarly, the phrase “if it is determined” or “if [a stated condition or event] is detected” is, optionally, construed to mean “upon determining” or “in response to determining” or “upon detecting [the stated condition or event]” or “in response to detecting [the stated condition or event],” depending on the context.
Each physical key of the set of physical keys 106 has at least one associated input. The input may be a printable character, non-printable character, function, or other input. The input associated with a physical key may be shown by a letter, word, symbol, or other indicia shown (e.g., printed) on the surface of the key in Latin script, Arabic characters, Chinese characters, or any other script. For example, the particular physical key indicated at 138 is associated with alphabetic character “z” as indicated by the letter z shown on the key. In another example, a physical key labeled with the word “command” may be associated with a command function. For example, the set of physical keys 106 is associated with a QWERTY, Dvorak, or other keyboard layouts with alphanumeric, numeric, and/or editing/function sections (e.g., standard, extended, or compact) according to ISO/IEC 9995, ANSI-INCITS 154-1988, JIS X 6002-1980, or other similar standards.
A signal corresponding to an input associated with a physical key may be received by the processor of portable computing system 100 (or computing device 202 in
In some embodiments, dynamic function row 104 is a touch screen display (also referred to herein as a touch-sensitive secondary display) that displays one or more user-selectable symbols 142 (sometimes also herein called “user interface elements,” “user interface components,” “affordances,” “buttons,” or “soft keys”). For example, dynamic function row 104 replaces the function row keys on a typical keyboard. A user may select a particular one of the one or more user-selectable symbols 142 by touching a location on the touch screen display that corresponds to the particular one of the one or more user-selectable symbols 142. For example, a user may select the user-selectable symbol indicated by magnifying glass symbol 144 by tapping dynamic function row 104 such that the user's finger contacts dynamic function row 104 at the position of the magnifying glass indicator 214. In some embodiments, a tap contact or a tap gesture includes touch-down of a contact and lift-off of the contact within a predetermined amount of time (e.g., 250 ms or the like). In some embodiments, the touch screen display of dynamic function row 104 is implemented using resistive sensing, acoustic sensing, capacitive sensing, optical sensing, infrared sensing, or the like to detect user inputs and selections.
When a user selects a particular one of the one or more user-selectable symbols 142, a signal corresponding to the particular one of the one or more user-selectable symbols 142 is generated by dynamic function row 104. For example, when a user taps “esc” on dynamic function row 104, dynamic function row 104 transmits a signal indicating a user input corresponding to an escape function to the processor of portable computing system 100 (or computing device 202 in
In some embodiments, when a particular one of the one or more user-selectable symbols 142 is selected, dynamic function row 104 transmits a signal corresponding to a position on the touch screen display where the particular one of the one or more user-selectable symbols 142 is displayed, to the processor of portable computing system 100 (or computing device 202 in
Each of the one or more user-selectable symbols 142 may include an indicator, such as a symbol (e.g., a magnifying glass symbol as shown at 144), an abbreviated word (e.g., “esc”), an unabbreviated word, a character, an image, an animated image, a video, or the like. In some embodiments, a respective one of the one or more user-selectable symbols 142 is capable of receiving user input(s).
An input may be associated with each of the one or more user-selectable symbols 142. The input may be a function, character, numerical value, and the like. A respective one of the one or more user-selectable symbols 142 may include an indicator that corresponds to the input for the respective one of the one or more user-selectable symbols 142. For example, in
In some embodiments, functions may be associated with combinations of movably actuated keys and/or user-selectable symbols. For example, simultaneous actuation of a command key and “c” key (i.e., command+c) may be associated with a “copy” function. In another example, simultaneous actuation of the command key and selection of the user-selectable symbol with the indicator “esc” (i.e., command+esc) may activate a function to open a particular application such as a media player application. In yet another example, simultaneous selection of two user-selectable symbols (e.g., the user-selectable symbol with the indicator “esc” and the user-selectable symbol 144 with the magnifying glass indicator) may result in activation of a function, such as a specialized search function.
In some embodiments, a first subset 146 of the one or more user-selectable symbols 142 of dynamic function row 104 may be associated with one group of functions and a second subset 148 of the one or more user-selectable symbols 142 of dynamic function row 104 may be associated with a second group of functions. For example, the user-selectable symbols in first subset 146 may be global functions (e.g., system-level functions or affordances), and the user-selectable symbols in second subset 148 may be application-specific functions. As such, the user-selectable symbols in second subset 148 change when the focus shifts from a first element of a graphical user interface displayed on primary display 102 (e.g., a first window corresponding to an Internet browser application) to a second element of the graphical user interface (e.g., a second window corresponding to an e-mail application). In contrast, the user-selectable symbols in first subset 146 are maintained when the focus shifts from the first element of the graphical user interface to the second element of the graphical user interface.
In some embodiments, the user-selectable symbols in second subset 148 are determined based on an active user interface element display on primary display 102 that is in focus. In some embodiments, the term “in focus” can refer to the active element of the user interface (e.g., a window associated with an application, a particular toolbar or menu associated with an application, or the operating system) that is currently in the foreground and actively running or is controllable by input received from a user of the computing system such as a key press, mouse click, voice command, gestural motion, or the like.
In some embodiments, the first subset 146 of the one or more user-selectable symbols 142 corresponding to global user-selectable symbols occupies a first area of dynamic function row 104 (e.g., the left half of dynamic function row 104), and the second subset 148 of the one or more user-selectable symbols 142 occupies a second area of dynamic function row 104 (e.g., the right half of dynamic function row 104). It will be realized that other proportions of dynamic function row 104 may be allocated to the first subset 146 and the second subset 148. In some embodiments, when no application has focus, the second area of dynamic function row 104 may not include any user-selectable symbols. In some embodiments, dynamic function row 104 includes three or more subsets of user-selectable symbols. In some embodiments, dynamic function row 104 includes a single set of user-selectable symbols that are not divided into subsets. While a single row of user-selectable symbols are shown in dynamic function row 104 in
In some embodiments, the change in focus changes which element of the graphical user interface displayed on primary display 102 of display portion 110 (or peripheral display device 204,
In some embodiments, the change in focus occurs in response to user input, for example, in response to user selection of an element of a graphical user interface (e.g., a different window) displayed on primary display 102 of display portion 110 (or peripheral display device 204,
In some embodiments, the change in focus may be a change from one element associated with an application to another element associated with the same application (e.g., from an e-mail composition window of an e-mail application to an inbox list window of an e-mail application or from one tab of an Internet browser application to another tab of an Internet browser application). In some embodiments, the change in focus may be a change from an element associated with one application to an element associated with another application (e.g., from an Internet browser window to an e-mail application window). Further, in some embodiments, the change in focus may be a change from an element associated with an application to an element associated with an operating system, such as a system dialog box, a system setting control (e.g., volume control), a window associated with a file/folder navigation application (e.g., Apple Inc.'s FINDER application), etc. Additionally, focus may also be directed to a dialog box, file directory, setting control (e.g., volume control), or any other element of a graphical user interface for which information can be presented to a user and/or user input can be received.
Electronic device 300 typically supports a variety of applications, such as one or more of the following: a drawing application, a presentation application, a word processing application, a website creation application, a disk authoring application, a spreadsheet application, a gaming application, a video conferencing application, an e-mail application, an instant messaging application, an image management application, a digital camera application, a digital video camera application, a web browser application, and/or a media player application.
The various applications that are executed on electronic device 300 optionally use at least one common physical user-interface device, such as the touch-sensitive surface. One or more functions of the touch-sensitive surface as well as corresponding information displayed by electronic device 300 are, optionally, adjusted and/or varied from one application to the next and/or within an application. In this way, a common physical architecture (such as the touch-sensitive surface) of electronic device 300 optionally supports the variety of applications with user interfaces that are intuitive and transparent to the user.
Electronic device 300 includes memory 302 (which optionally includes one or more computer readable storage mediums), memory controller 322, one or more processing units (CPU(s)) 320, peripherals interface 318, RF circuitry 308, audio circuitry 310, speaker 311, microphone 313, input/output (I/O) subsystem 306, other input or control devices 316, and external port 324. Electronic device 300 optionally includes a display system 312 (e.g., primary display 102 of display portion 110,
As used in the specification, the term “intensity” of a contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of a contact (e.g., a finger contact) on the touch sensitive surface, or to a substitute (proxy) for the force or pressure of a contact on the touch sensitive surface. The intensity of a contact has a range of values that includes at least four distinct values and more typically includes hundreds of distinct values (e.g., at least 256). Intensity of a contact is, optionally, determined (or measured) using various approaches and various sensors or combinations of sensors. For example, one or more force sensors underneath or adjacent to the touch-sensitive surface are, optionally, used to measure force at various points on the touch-sensitive surface. In some implementations, force measurements from multiple force sensors are combined (e.g., a weighted average) to determine an estimated force of a contact. Similarly, a pressure-sensitive tip of a stylus is, optionally, used to determine a pressure of the stylus on the touch-sensitive surface. Alternatively, the size of the contact area detected on the touch-sensitive surface and/or changes thereto, the capacitance of the touch-sensitive surface proximate to the contact and/or changes thereto, and/or the resistance of the touch-sensitive surface proximate to the contact and/or changes thereto are, optionally, used as a substitute for the force or pressure of the contact on the touch-sensitive surface. In some implementations, the substitute measurements for contact force or pressure are used directly to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is described in units corresponding to the substitute measurements). In some implementations, the substitute measurements for contact force or pressure are converted to an estimated force or pressure and the estimated force or pressure is used to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is a pressure threshold measured in units of pressure).
As used in the specification and claims, the term “tactile output” refers to physical displacement of a device relative to a previous position of the device, physical displacement of a component (e.g., a touch-sensitive surface) of a device relative to another component (e.g., housing) of the device, or displacement of the component relative to a center of mass of the device that will be detected by a user with the user's sense of touch. For example, in situations where the device or the component of the device is in contact with a surface of a user that is sensitive to touch (e.g., a finger, palm, or other part of a user's hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in physical characteristics of the device or the component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or touch/track pad) is, optionally, interpreted by the user as a “down click” or “up click” of a physical actuator button. In some cases, a user will feel a tactile sensation such as an “down click” or “up click” even when there is no movement of a physical actuator button associated with the touch-sensitive surface that is physically pressed (e.g., displaced) by the user's movements. As another example, movement of the touch-sensitive surface is, optionally, interpreted or sensed by the user as “roughness” of the touch-sensitive surface, even when there is no change in smoothness of the touch-sensitive surface. While such interpretations of touch by a user will be subject to the individualized sensory perceptions of the user, there are many sensory perceptions of touch that are common to a large majority of users. Thus, when a tactile output is described as corresponding to a particular sensory perception of a user (e.g., an “up click,” a “down click,” “roughness”), unless otherwise stated, the generated tactile output corresponds to physical displacement of the device or a component thereof that will generate the described sensory perception for a typical (or average) user.
It should be appreciated that electronic device 300 is only an example and that electronic device 300 optionally has more or fewer components than shown, optionally combines two or more components, or optionally has a different configuration or arrangement of the components. The various components shown in
Memory 302 optionally includes high-speed random access memory and optionally also includes non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Access to memory 302 by other components of electronic device 300, such as CPU(s) 320 and peripherals interface 318, is, optionally, controlled by memory controller 322. Peripherals interface 318 can be used to couple input and output peripherals to CPU(s) 320 and memory 302. The one or more processing units 320 run or execute various software programs and/or sets of instructions stored in memory 302 to perform various functions for electronic device 300 and to process data. In some embodiments, peripherals interface 318, CPU(s) 320, and memory controller 322 are, optionally, implemented on a single chip, such as chip 304. In some other embodiments, they are, optionally, implemented on separate chips.
RF (radio frequency) circuitry 308 receives and sends RF signals, also called electromagnetic signals. RF circuitry 308 converts electrical signals to/from electromagnetic signals and communicates with communications networks and other communications devices via the electromagnetic signals. RF circuitry 308 optionally includes well-known circuitry for performing these functions, including but not limited to an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a subscriber identity module (SIM) card, memory, and so forth. RF circuitry 308 optionally communicates with networks, such as the Internet, also referred to as the World Wide Web (WWW), an intranet and/or a wireless network, such as a cellular telephone network, a wireless local area network (LAN) and/or a metropolitan area network (MAN), and other devices by wireless communication. The wireless communication optionally uses any of a plurality of communications standards, protocols and technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), high-speed downlink packet access (HSDPA), high-speed uplink packet access (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA+, Dual-Cell HSPA (DC-HSPDA), long term evolution (LTE), near field communication (NFC), wideband code division multiple access (W-CDMA), code division multiple access (CDMA), time division multiple access (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and/or IEEE 802.11n), voice over Internet Protocol (VoIP), Wi-MAX, a protocol for e-mail (e.g., Internet message access protocol (IMAP) and/or post office protocol (POP)), instant messaging (e.g., extensible messaging and presence protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions (SIMPLE), Instant Messaging and Presence Service (IMPS)), and/or Short Message Service (SMS), or any other suitable communication protocol, including communication protocols not yet developed as of the filing date of this document.
Audio circuitry 310, speaker 311, and microphone 313 provide an audio interface between a user and electronic device 300. Audio circuitry 310 receives audio data from peripherals interface 318, converts the audio data to an electrical signal, and transmits the electrical signal to speaker 311. Speaker 311 converts the electrical signal to human-audible sound waves. Audio circuitry 310 also receives electrical signals converted by microphone 313 from sound waves. Audio circuitry 310 converts the electrical signals to audio data and transmits the audio data to peripherals interface 318 for processing. Audio data is, optionally, retrieved from and/or transmitted to memory 302 and/or RF circuitry 308 by peripherals interface 318. In some embodiments, audio circuitry 310 also includes a headset jack. The headset jack provides an interface between audio circuitry 310 and removable audio input/output peripherals, such as output-only headphones or a headset with both output (e.g., a headphone for one or both ears) and input (e.g., a microphone).
I/O subsystem 306 couples the input/output peripherals of electronic device 300, such as display system 312 and other input or control devices 316, to peripherals interface 318. I/O subsystem 306 optionally includes display controller 356, optical sensor controller 358, intensity sensor controller 359, haptic feedback controller 361, and one or more other input controllers 360 for other input or control devices. The one or more other input controllers 360 receive/send electrical signals from/to other input or control devices 316. The other input or control devices 316 optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slider switches, joysticks, click wheels, and so forth. In some alternate embodiments, other input controller(s) 360 are, optionally, coupled with any (or none) of the following: a keyboard, infrared port, USB port, and a pointer device such as a mouse. The one or more physical buttons optionally include an up/down button for volume control of speaker 311 and/or microphone 313.
Display system 312 (e.g., primary display 102 of display portion 110,
In some embodiments, display system 312 (e.g., primary display 102 of display portion 110,
Display system 312 (e.g., primary display 102 of display portion 110,
Display system 312 (e.g., primary display 102 of display portion 110,
In some embodiments, in addition to display system 312, electronic device 300 optionally includes a touchpad (e.g., touchpad 108,
Electronic device 300 also includes power system 362 for powering the various components. Power system 362 optionally includes a power management system, one or more power sources (e.g., battery, alternating current (AC), etc.), a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator (e.g., a light-emitting diode (LED)) and any other components associated with the generation, management and distribution of power in portable devices.
Electronic device 300 optionally also includes one or more optical sensors 364 coupled with optical sensor controller 358 in I/O subsystem 306. Optical sensor(s) 364 optionally includes charge-coupled device (CCD) or complementary metal-oxide semiconductor (CMOS) phototransistors. Optical sensor(s) 364 receive light from the environment, projected through one or more lens, and converts the light to data representing an image. In conjunction with imaging module 343, optical sensor(s) 364 optionally capture still images or video. In some embodiments, an optical sensor is located on the front of electronic device 300 so that the user's image is, optionally, obtained for videoconferencing while the user views the other video conference participants on display system 312.
Electronic device 300 optionally also includes one or more contact intensity sensor(s) 365 coupled with intensity sensor controller 359 in I/O subsystem 306. Contact intensity sensor(s) 365 optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electric force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other intensity sensors (e.g., sensors used to measure the force (or pressure) of a contact on a touch-sensitive surface). Contact intensity sensor(s) 365 receives contact intensity information (e.g., pressure information or a proxy for pressure information) from the environment. In some embodiments, at least one contact intensity sensor is collocated with, or proximate to, a touch-sensitive surface (e.g., touchpad 108,
Electronic device 300 optionally also includes one or more tactile output generators 367 coupled with haptic feedback controller 361 in I/O subsystem 306. Tactile output generator(s) 367 optionally includes one or more electroacoustic devices such as speakers or other audio components and/or electromechanical devices that convert energy into linear motion such as a motor, solenoid, electroactive polymer, piezoelectric actuator, electrostatic actuator, or other tactile output generating component (e.g., a component that converts electrical signals into tactile outputs on the device). Contact intensity sensor(s) 365 receives tactile feedback generation instructions from haptic feedback module 333 and generates tactile outputs that are capable of being sensed by a user of electronic device 300. In some embodiments, at least one tactile output generator is collocated with, or proximate to, a touch-sensitive surface (e.g., touchpad 108,
Electronic device 300 optionally also includes one or more proximity sensors 366 coupled with peripherals interface 318. Alternately, proximity sensor(s) 366 are coupled with other input controller(s) 360 in I/O subsystem 306. Electronic device 300 optionally also includes one or more accelerometers 368 coupled with peripherals interface 318. Alternately, accelerometer(s) 368 are coupled with other input controller(s) 360 in I/O subsystem 306.
In some embodiments, the software components stored in memory 302 include operating system 326, communication module 328 (or set of instructions), contact/motion module 330 (or set of instructions), graphics module 332 (or set of instructions), applications 340 (or sets of instructions), and dynamic function row module 350 (or sets of instructions). Furthermore, in some embodiments, memory 302 stores device/global internal state 357 (or sets of instructions), as shown in
Operating system 326 (e.g., DARWIN, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system such as VXWorks) includes various software components and/or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware and software components.
Communication module 328 facilitates communication with other devices (e.g., computing device 202,
Contact/motion module 330 optionally detects contact with display system 312 when it is a touch-sensitive display (in conjunction with display controller 356) and other touch sensitive devices (e.g., a touchpad or physical click wheel). Contact/motion module 330 includes various software components for performing various operations related to detection of contact, such as determining if contact has occurred (e.g., detecting a finger-down event), determining an intensity of the contact (e.g., the force or pressure of the contact or a substitute for the force or pressure of the contact), determining if there is movement of the contact and tracking the movement across the touch-sensitive surface (e.g., detecting one or more finger-dragging events), and determining if the contact has ceased (e.g., detecting a finger-up event or a break in contact). Contact/motion module 330 receives contact data from the touch-sensitive surface. Determining movement of the point of contact, which is represented by a series of contact data, optionally includes determining speed (magnitude), velocity (magnitude and direction), and/or an acceleration (a change in magnitude and/or direction) of the point of contact. These operations are, optionally, applied to single contacts (e.g., one finger contacts) or to multiple simultaneous contacts (e.g., “multitouch”/multiple finger contacts). In some embodiments, contact/motion module 330 also detects contact on a touchpad (e.g., touchpad 108,
In some embodiments, contact/motion module 330 uses a set of one or more intensity thresholds to determine whether an operation has been performed by a user (e.g., to determine whether a user has selected or “clicked” on an affordance). In some embodiments at least a subset of the intensity thresholds are determined in accordance with software parameters (e.g., the intensity thresholds are not determined by the activation thresholds of particular physical actuators and can be adjusted without changing the physical hardware of electronic device 300). For example, a mouse “click” threshold of a trackpad or touch screen display can be set to any of a large range of predefined thresholds values without changing the trackpad or touch screen display hardware. Additionally, in some implementations a user of the device is provided with software settings for adjusting one or more of the set of intensity thresholds (e.g., by adjusting individual intensity thresholds and/or by adjusting a plurality of intensity thresholds at once with a system-level click “intensity” parameter).
Contact/motion module 330 optionally detects a gesture input by a user. Different gestures on the touch-sensitive surface have different contact patterns (e.g., different motions, timings, and/or intensities of detected contacts). Thus, a gesture is, optionally, detected by detecting a particular contact pattern. For example, detecting a finger tap contact includes detecting a finger-down event followed by detecting a finger-up (lift off) event at the same position (or substantially the same position) as the finger-down event (e.g., at the position of an icon). As another example, detecting a finger swipe gesture on the touch-sensitive surface includes detecting a finger-down event followed by detecting one or more finger-dragging events, and in some embodiments also followed by detecting a finger-up (lift off) event.
Graphics module 332 includes various known software components for rendering and causing display of graphics on primary display 102 (e.g., primary display 102 of display portion 110,
Haptic feedback module 333 includes various software components for generating instructions used by tactile output generator(s) 367 to produce tactile outputs at one or more locations on electronic device 300 in response to user interactions with electronic device 300.
Applications 340 optionally include the following modules (or sets of instructions), or a subset or superset thereof:
Examples of other applications 340 that are, optionally, stored in memory 302 include messaging and communications applications, word processing applications, other image editing applications, drawing applications, presentation applications, JAVA-enabled applications, encryption applications, digital rights management applications, voice recognition applications, and voice replication applications.
In conjunction with one or more of RF circuitry 308, display system 312 (e.g., primary display 102 of display portion 110,
In conjunction with one or more of display system 312 (e.g., primary display 102 of display portion 110,
In conjunction with one or more of display system 312 (e.g., primary display 102 of display portion 110,
In conjunction with one or more of display system 312 (e.g., primary display 102 of display portion 110,
In conjunction with one or more of RF circuitry 308, display system 312 (e.g., primary display 102 of display portion 110,
Dynamic function row (DFR) module 350 includes: focus determining module 351, DFR determining module 352, and DFR presenting module 353. In some embodiments, focus determining module 351 is configured to determine an active user interface element that is in focus on the graphical user interface displayed by display system 312 (e.g., primary display 102 of display portion 110,
In some embodiments, the dynamic function row module 350 interfaces with components that allow for providing predicted/proactive/suggested content items (including predicted recipients, suggested text completion strings, proactively suggested applications, etc.). Proactively suggesting content items is discussed in more detail in U.S. application Ser. No. 15/167,713, which is hereby incorporated by reference in its entirety.
Each of the above identified modules and applications correspond to a set of executable instructions for performing one or more functions described above and the methods described in this application (e.g., the computer-implemented methods and other information processing methods described herein). These modules (i.e., sets of instructions) need not be implemented as separate software programs, procedures or modules, and thus various subsets of these modules are, optionally, combined or otherwise re-arranged in various embodiments. In some embodiments, memory 302 optionally stores a subset of the modules and data structures identified above. Furthermore, memory 302 optionally stores additional modules and data structures not described above.
Event sorter 370 receives event information and determines the application 340-1 and application view 391 of application 340-1 to which to deliver the event information. Event sorter 370 includes event monitor 371 and event dispatcher module 374. In some embodiments, application 340-1 includes application internal state 392, which indicates the current application view(s) displayed on display system 312 (e.g., primary display 102 of display portion 110,
In some embodiments, application internal state 392 includes additional information, such as one or more of: resume information to be used when application 340-1 resumes execution, user interface state information that indicates information being displayed or that is ready for display by application 340-1, a state queue for enabling the user to go back to a prior state or view of application 340-1, and a redo/undo queue of previous actions taken by the user.
Event monitor 371 receives event information from peripherals interface 318. Event information includes information about a sub-event (e.g., a user touch on display system 312 when it is a touch-sensitive display, as part of a multi-touch gesture). Peripherals interface 318 transmits information it receives from I/O subsystem 306 or a sensor, such as proximity sensor(s) 366, accelerometer(s) 368, and/or microphone 313 (through audio circuitry 310). Information that peripherals interface 318 receives from I/O subsystem 306 includes information from display system 312 when it is a touch-sensitive display or another touch-sensitive surface (e.g., touchpad 108,
In some embodiments, event monitor 371 sends requests to the peripherals interface 318 at predetermined intervals. In response, peripherals interface 318 transmits event information. In other embodiments, peripheral interface 318 transmits event information only when there is a significant event (e.g., receiving an input above a predetermined noise threshold and/or for more than a predetermined duration).
In some embodiments, event sorter 370 also includes a hit view determination module 372 and/or an active event recognizer determination module 373.
Hit view determination module 372 provides software procedures for determining where a sub-event has taken place within one or more views, when display system 312 displays more than one view, where views are made up of controls and other elements that a user can see on the display.
Another aspect of the user interface associated with an application is a set of views, sometimes herein called application views or user interface windows, in which information is displayed and touch-based gestures occur. The application views (of an application) in which a touch is detected optionally correspond to programmatic levels within a programmatic or view hierarchy of the application. For example, the lowest level view in which a touch is detected is, optionally, called the hit view, and the set of events that are recognized as proper inputs are, optionally, determined based, at least in part, on the hit view of the initial touch that begins a touch-based gesture.
Hit view determination module 372 receives information related to sub-events of a touch-based gesture. When an application has multiple views organized in a hierarchy, hit view determination module 372 identifies a hit view as the lowest view in the hierarchy which should handle the sub-event. In most circumstances, the hit view is the lowest level view in which an initiating sub-event occurs (i.e., the first sub-event in the sequence of sub-events that form an event or potential event). Once the hit view is identified by the hit view determination module, the hit view typically receives all sub-events related to the same touch or input source for which it was identified as the hit view.
Active event recognizer determination module 373 determines which view or views within a view hierarchy should receive a particular sequence of sub-events. In some embodiments, active event recognizer determination module 373 determines that only the hit view should receive a particular sequence of sub-events. In other embodiments, active event recognizer determination module 373 determines that all views that include the physical location of a sub-event are actively involved views, and therefore determines that all actively involved views should receive a particular sequence of sub-events. In other embodiments, even if touch sub-events were entirely confined to the area associated with one particular view, views higher in the hierarchy would still remain as actively involved views.
Event dispatcher module 374 dispatches the event information to an event recognizer (e.g., event recognizer 380). In embodiments including active event recognizer determination module 373, event dispatcher module 374 delivers the event information to an event recognizer determined by active event recognizer determination module 373. In some embodiments, event dispatcher module 374 stores in an event queue the event information, which is retrieved by a respective event receiver 382.
In some embodiments, operating system 326 includes event sorter 370. Alternatively, application 340-1 includes event sorter 370. In yet other embodiments, event sorter 370 is a stand-alone module, or a part of another module stored in memory 302, such as contact/motion module 330.
In some embodiments, application 340-1 includes a plurality of event handlers 390 and one or more application views 391, each of which includes instructions for handling touch events that occur within a respective view of the application's user interface. Each application view 391 of the application 340-1 includes one or more event recognizers 380. Typically, an application view 391 includes a plurality of event recognizers 380. In other embodiments, one or more of event recognizers 380 are part of a separate module, such as a user interface kit (not shown) or a higher level object from which application 340-1 inherits methods and other properties. In some embodiments, a respective event handler 390 includes one or more of: data updater 376, object updater 377, GUI updater 378, and/or event data 379 received from event sorter 370. Event handler 390 optionally utilizes or calls data updater 376, object updater 377 or GUI updater 378 to update the application internal state 392. Alternatively, one or more of the application views 391 includes one or more respective event handlers 390. Also, in some embodiments, one or more of data updater 376, object updater 377, and GUI updater 378 are included in an application view 391.
A respective event recognizer 380 receives event information (e.g., event data 379) from event sorter 370, and identifies an event from the event information. Event recognizer 380 includes event receiver 382 and event comparator 384. In some embodiments, event recognizer 380 also includes at least a subset of: metadata 383, and event delivery instructions 388 (which optionally include sub-event delivery instructions).
Event receiver 382 receives event information from event sorter 370. The event information includes information about a sub-event, for example, a touch or a touch movement. Depending on the sub-event, the event information also includes additional information, such as location of the sub-event. When the sub-event concerns motion of a touch, the event information optionally also includes speed and direction of the sub-event. In some embodiments, events include rotation of the device from one orientation to another (e.g., from a portrait orientation to a landscape orientation, or vice versa), and the event information includes corresponding information about the current orientation (also called device attitude) of the device.
Event comparator 384 compares the event information to predefined event or sub-event definitions and, based on the comparison, determines an event or sub-event, or determines or updates the state of an event or sub-event. In some embodiments, event comparator 384 includes event definitions 386. Event definitions 386 contain definitions of events (e.g., predefined sequences of sub-events), for example, event 1 (387-1), event 2 (387-2), and others. In some embodiments, sub-events in an event 387 include, for example, touch begin, touch end, touch movement, touch cancellation, and multiple touching. In one example, the definition for event 1 (387-1) is a double tap on a displayed object. The double tap, for example, comprises a first touch (touch begin) on the displayed object for a predetermined phase, a first lift-off (touch end) for a predetermined phase, a second touch (touch begin) on the displayed object for a predetermined phase, and a second lift-off (touch end) for a predetermined phase. In another example, the definition for event 2 (387-2) is a dragging on a displayed object. The dragging, for example, comprises a touch (or contact) on the displayed object for a predetermined phase, a movement of the touch across display system 312 when it is a touch-sensitive display, and lift-off of the touch (touch end). In some embodiments, the event also includes information for one or more associated event handlers 390.
In some embodiments, event definition 387 includes a definition of an event for a respective user-interface object. In some embodiments, event comparator 384 performs a hit test to determine which user-interface object is associated with a sub-event. For example, in an application view in which three user-interface objects are displayed on display system 312, when a touch is detected on display system 312 when it is a touch-sensitive display, event comparator 384 performs a hit test to determine which of the three user-interface objects is associated with the touch (sub-event). If each displayed object is associated with a respective event handler 390, the event comparator uses the result of the hit test to determine which event handler 390 should be activated. For example, event comparator 384 selects an event handler associated with the sub-event and the object triggering the hit test.
In some embodiments, the definition for a respective event 387 also includes delayed actions that delay delivery of the event information until after it has been determined whether the sequence of sub-events does or does not correspond to the event recognizer's event type.
When a respective event recognizer 380 determines that the series of sub-events do not match any of the events in event definitions 386, the respective event recognizer 380 enters an event impossible, event failed, or event ended state, after which it disregards subsequent sub-events of the touch-based gesture. In this situation, other event recognizers, if any, that remain active for the hit view continue to track and process sub-events of an ongoing touch-based gesture.
In some embodiments, a respective event recognizer 380 includes metadata 383 with configurable properties, flags, and/or lists that indicate how the event delivery system should perform sub-event delivery to actively involved event recognizers. In some embodiments, metadata 383 includes configurable properties, flags, and/or lists that indicate how event recognizers interact, or are enabled to interact, with one another. In some embodiments, metadata 383 includes configurable properties, flags, and/or lists that indicate whether sub-events are delivered to varying levels in the view or programmatic hierarchy.
In some embodiments, a respective event recognizer 380 activates event handler 390 associated with an event when one or more particular sub-events of an event are recognized. In some embodiments, a respective event recognizer 380 delivers event information associated with the event to event handler 390. Activating an event handler 390 is distinct from sending (and deferred sending) sub-events to a respective hit view. In some embodiments, event recognizer 380 throws a flag associated with the recognized event, and event handler 390 associated with the flag catches the flag and performs a predefined process.
In some embodiments, event delivery instructions 388 include sub-event delivery instructions that deliver event information about a sub-event without activating an event handler. Instead, the sub-event delivery instructions deliver event information to event handlers associated with the series of sub-events or to actively involved views. Event handlers associated with the series of sub-events or with actively involved views receive the event information and perform a predetermined process.
In some embodiments, data updater 376 creates and updates data used in application 340-1. For example, data updater 376 stores a video file used by media player module 344. In some embodiments, object updater 377 creates and updates objects used by application 340-1. For example, object updater 377 creates a new user-interface object or updates the position of a user-interface object. GUI updater 378 updates the GUI. For example, GUI updater 378 prepares display information and sends it to graphics module 332 for display on display system 312 (e.g., primary display 102 of display portion 110,
In some embodiments, event handler(s) 390 includes or has access to data updater 376, object updater 377, and GUI updater 378. In some embodiments, data updater 376, object updater 377, and GUI updater 378 are included in a single module of an application 340-1 or application view 391. In other embodiments, they are included in two or more software modules.
It shall be understood that the foregoing discussion regarding event handling of user touches on touch-sensitive displays also applies to other forms of user inputs to operate electronic device 300 with input-devices, not all of which are initiated on touch screens. For example, mouse movement and mouse button presses, optionally coordinated with single or multiple keyboard presses or holds; contact movements such as taps, drags, scrolls, etc., on touchpads; pen stylus inputs; movement of the device; oral instructions; detected eye movements; biometric inputs; and/or any combination thereof are optionally utilized as inputs corresponding to sub-events which define an event to be recognized.
In some embodiments, peripheral electronic device 400 includes one or more of memory 402 (which optionally includes one or more computer readable storage mediums), memory controller 422, one or more processing units (CPU(s)) 420, peripherals interface 418, RF circuitry 408, audio circuitry 410, speaker 411, microphone 413, input/output (I/O) subsystem 406, other input or control devices 416, and external port 424. Peripheral electronic device 400 includes a touch-sensitive display system 412 (e.g., dynamic function row 104,
Peripheral electronic device 400 optionally includes one or more intensity sensors 465 for detecting intensity of contacts on a touch-sensitive surface such as touch-sensitive display system 412 or a touchpad (e.g., touchpad 108,
Memory 402 optionally includes high-speed random access memory and optionally also includes non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Access to memory 402 by other components of peripheral electronic device 400, such as CPU(s) 420 and peripherals interface 418, is, optionally, controlled by memory controller 422. Peripherals interface 418 can be used to couple CPU(s) 420 and memory 402 to I/O subsystem 406 and other circuitry. The one or more processing units 420 run or execute various software programs and/or sets of instructions stored in memory 402 to perform various functions for peripheral electronic device 400 and to process data. In some embodiments, peripherals interface 418, CPU(s) 420, and memory controller 422 are, optionally, implemented on a single chip, such as chip 404. In some other embodiments, they are, optionally, implemented on separate chips.
RF (radio frequency) circuitry 408 receives and sends RF signals, also called electromagnetic signals. RF circuitry 408 converts electrical signals to/from electromagnetic signals and communicates with communications networks and other communications devices via the electromagnetic signals. RF circuitry 408 optionally includes well-known circuitry for performing these functions, including but not limited to an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a subscriber identity module (SIM) card, memory, and so forth. The wireless communication optionally uses any of a plurality of communications standards, protocols and technologies, including but not limited to near field communication (NFC), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and/or IEEE 802.11n), Wi-MAX, or any other suitable communication protocol, including communication protocols not yet developed as of the filing date of this document.
Optional audio circuitry 410, speaker 411, and microphone 413 provide an audio interface between a user and peripheral electronic device 400. Audio circuitry 410 receives audio data from peripherals interface 418, converts the audio data to an electrical signal, and transmits the electrical signal to speaker 411. Speaker 411 converts the electrical signal to human-audible sound waves. Audio circuitry 410 also receives electrical signals converted by microphone 413 from sound waves. Audio circuitry 410 converts the electrical signals to audio data and transmits the audio data to peripherals interface 418 for processing. Audio data is, optionally, retrieved from and/or transmitted to memory 402 and/or RF circuitry 408 by peripherals interface 418. In some embodiments, audio circuitry 410 also includes a headset jack. The headset jack provides an interface between audio circuitry 410 and removable audio input/output peripherals, such as output-only headphones or a headset with both output (e.g., a headphone for one or both ears) and input (e.g., a microphone).
I/O subsystem 406 couples the input/output peripherals of peripheral electronic device 400, such as touch-sensitive display system 412 (e.g., dynamic function row 104,
Touch-sensitive display system 412 (e.g., dynamic function row 104,
Touch-sensitive display system 412 (e.g., dynamic function row 104,
Touch-sensitive display system 412 (e.g., dynamic function row 104,
Touch-sensitive display system 412 (e.g., dynamic function row 104,
In some embodiments, in addition to touch-sensitive display system 412, peripheral electronic device 400 optionally includes a touchpad (e.g., touchpad 108,
Peripheral electronic device 400 also includes power system 462 for powering the various components. Power system 462 optionally includes a power management system, one or more power sources (e.g., battery, alternating current (AC), etc.), a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator (e.g., a light-emitting diode (LED)) and any other components associated with the generation, management and distribution of power in portable devices.
Peripheral electronic device 400 optionally also includes one or more contact intensity sensors 465 coupled with intensity sensor controller 459 in I/O subsystem 406. Contact intensity sensor(s) 465 optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electric force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other intensity sensors (e.g., sensors used to measure the force (or pressure) of a contact on a touch-sensitive surface). Contact intensity sensor(s) 465 receives contact intensity information (e.g., pressure information or a proxy for pressure information) from the environment. In some embodiments, at least one contact intensity sensor is collocated with, or proximate to, a touch-sensitive surface (e.g., touch-sensitive display system 412 and/or touchpad 108,
Peripheral electronic device 400 optionally also includes one or more tactile output generators 467 coupled with haptic feedback controller 461 in I/O subsystem 406. Tactile output generator(s) 467 optionally includes one or more electroacoustic devices such as speakers or other audio components and/or electromechanical devices that convert energy into linear motion such as a motor, solenoid, electroactive polymer, piezoelectric actuator, electrostatic actuator, or other tactile output generating component (e.g., a component that converts electrical signals into tactile outputs on the device). Contact intensity sensor(s) 465 receives tactile feedback generation instructions from haptic feedback module 433 and generates tactile outputs that are capable of being sensed by a user of peripheral electronic device 400. In some embodiments, at least one tactile output generator is collocated with, or proximate to, a touch-sensitive surface (e.g., touch-sensitive display system 412 and/or touchpad 108,
In some embodiments, the software components stored in memory 402 include operating system 426, communication module 428 (or set of instructions), contact/motion module 430 (or set of instructions), and dynamic function row module 450 (or sets of instructions). Furthermore, in some embodiments, memory 402 stores device state 457 including the display state, indicating what views or other information occupy various regions of touch-sensitive display system 412 (e.g., dynamic function row 104,
Operating system 426 includes various software components and/or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware and software components.
Communication module 428 facilitates communication with other devices (e.g., computing device 202,
Contact/motion module 430 optionally detects contact with touch-sensitive display system 412 and other touch sensitive devices (e.g., a touchpad or physical click wheel). Contact/motion module 430 includes various software components for performing various operations related to detection of contact, such as determining if contact has occurred (e.g., detecting a finger-down event), determining an intensity of the contact (e.g., the force or pressure of the contact or a substitute for the force or pressure of the contact), determining if there is movement of the contact and tracking the movement across the touch-sensitive surface (e.g., detecting one or more finger-dragging events), and determining if the contact has ceased (e.g., detecting a finger-up event or a break in contact). Contact/motion module 430 receives contact data from the touch-sensitive surface. Determining movement of the point of contact, which is represented by a series of contact data, optionally includes determining speed (magnitude), velocity (magnitude and direction), and/or an acceleration (a change in magnitude and/or direction) of the point of contact. These operations are, optionally, applied to single contacts (e.g., one finger contacts) or to multiple simultaneous contacts (e.g., “multitouch”/multiple finger contacts). In some embodiments, contact/motion module 430 also detects contact on a touchpad (e.g., touchpad 108,
In some embodiments, contact/motion module 430 uses a set of one or more intensity thresholds to determine whether an operation has been performed by a user (e.g., to determine whether a user has selected or “clicked” on an affordance). In some embodiments at least a subset of the intensity thresholds are determined in accordance with software parameters (e.g., the intensity thresholds are not determined by the activation thresholds of particular physical actuators and can be adjusted without changing the physical hardware of peripheral electronic device 400). For example, a mouse “click” threshold of a trackpad or touch screen display can be set to any of a large range of predefined thresholds values without changing the trackpad or touch screen display hardware. Additionally, in some implementations a user of the device is provided with software settings for adjusting one or more of the set of intensity thresholds (e.g., by adjusting individual intensity thresholds and/or by adjusting a plurality of intensity thresholds at once with a system-level click “intensity” parameter).
Contact/motion module 430 optionally detects a gesture input by a user. Different gestures on the touch-sensitive surface have different contact patterns (e.g., different motions, timings, and/or intensities of detected contacts). Thus, a gesture is, optionally, detected by detecting a particular contact pattern. For example, detecting a finger tap contact includes detecting a finger-down event followed by detecting a finger-up (lift off) event at the same position (or substantially the same position) as the finger-down event (e.g., at the position of an icon). As another example, detecting a finger swipe gesture on the touch-sensitive surface includes detecting a finger-down event followed by detecting one or more finger-dragging events, and in some embodiments also followed by detecting a finger-up (lift off) event.
Haptic feedback module 433 includes various software components for generating instructions used by tactile output generator(s) 467 to produce tactile outputs at one or more locations on peripheral electronic device 400 in response to user interactions with peripheral electronic device 400.
Dynamic function row (DFR) module 450 includes: focus obtaining module 451, DFR determining module 452, and DFR presenting module 453. In some embodiments, focus obtaining module 451 is configured to obtain an indication of an active user interface element that is the current focus of the graphical user interface displayed on primary display 102 of peripheral display device 204 (
In some embodiments, memory 402 includes event sorter 470 (e.g., in operating system 426). In some embodiments, event sorter 470 performs the same functions as event sorter 370 (
It should be appreciated that peripheral electronic device 400 is only an example and that peripheral electronic device 400 optionally has more or fewer components than shown, optionally combines two or more components, or optionally has a different configuration or arrangement of the components. The various components shown in
Each of the above identified modules correspond to a set of executable instructions for performing one or more functions described above and the methods described in this application (e.g., the computer-implemented methods and other information processing methods described herein). These modules (i.e., sets of instructions) need not be implemented as separate software programs, procedures or modules, and thus various subsets of these modules are, optionally, combined or otherwise re-arranged in various embodiments. In some embodiments, memory 402 optionally stores a subset of the modules and data structures identified above. Furthermore, memory 402 optionally stores additional modules and data structures not described above.
As used herein, the term “focus selector” refers to an input element that indicates a current part of a user interface with which a user is interacting. In some implementations that include a cursor or other location marker, the cursor acts as a “focus selector,” so that when an input (e.g., a press input) is detected on a touch-sensitive surface (e.g., touchpad 355 in
As used in the specification and claims, the term “intensity” of a contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of a contact (e.g., a finger contact or a stylus contact) on the touch-sensitive surface, or to a substitute (proxy) for the force or pressure of a contact on the touch-sensitive surface. The intensity of a contact has a range of values that includes at least four distinct values and more typically includes hundreds of distinct values (e.g., at least 256). Intensity of a contact is, optionally, determined (or measured) using various approaches and various sensors or combinations of sensors. For example, one or more force sensors underneath or adjacent to the touch-sensitive surface are, optionally, used to measure force at various points on the touch-sensitive surface. In some implementations, force measurements from multiple force sensors are combined (e.g., a weighted average or a sum) to determine an estimated force of a contact. Similarly, a pressure-sensitive tip of a stylus is, optionally, used to determine a pressure of the stylus on the touch-sensitive surface. Alternatively, the size of the contact area detected on the touch-sensitive surface and/or changes thereto, the capacitance of the touch-sensitive surface proximate to the contact and/or changes thereto, and/or the resistance of the touch-sensitive surface proximate to the contact and/or changes thereto are, optionally, used as a substitute for the force or pressure of the contact on the touch-sensitive surface. In some implementations, the substitute measurements for contact force or pressure are used directly to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is described in units corresponding to the substitute measurements). In some implementations, the substitute measurements for contact force or pressure are converted to an estimated force or pressure and the estimated force or pressure is used to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is a pressure threshold measured in units of pressure). Using the intensity of a contact as an attribute of a user input allows for user access to additional device functionality that may otherwise not be readily accessible by the user on a reduced-size device with limited real estate for displaying affordances (e.g., on a touch-sensitive display) and/or receiving user input (e.g., via a touch-sensitive display, a touch-sensitive surface, or a physical/mechanical control such as a knob or a button).
In some embodiments, contact/motion module 130 uses a set of one or more intensity thresholds to determine whether an operation has been performed by a user (e.g., to determine whether a user has “clicked” on an icon). In some embodiments, at least a subset of the intensity thresholds are determined in accordance with software parameters (e.g., the intensity thresholds are not determined by the activation thresholds of particular physical actuators and can be adjusted without changing the physical hardware of system 100). For example, a mouse “click” threshold of a trackpad or touch-screen display can be set to any of a large range of predefined thresholds values without changing the trackpad or touch-screen display hardware. Additionally, in some implementations a user of the device is provided with software settings for adjusting one or more of the set of intensity thresholds (e.g., by adjusting individual intensity thresholds and/or by adjusting a plurality of intensity thresholds at once with a system-level click “intensity” parameter).
As used in the specification and claims, the term “characteristic intensity” of a contact refers to a characteristic of the contact based on one or more intensities of the contact. In some embodiments, the characteristic intensity is based on multiple intensity samples. The characteristic intensity is, optionally, based on a predefined number of intensity samples, or a set of intensity samples collected during a predetermined time period (e.g., 0.05, 0.1, 0.2, 0.5, 1, 2, 5, 10 seconds) relative to a predefined event (e.g., after detecting the contact, prior to detecting liftoff of the contact, before or after detecting a start of movement of the contact, prior to detecting an end of the contact, before or after detecting an increase in intensity of the contact, and/or before or after detecting a decrease in intensity of the contact). A characteristic intensity of a contact is, optionally based on one or more of: a maximum value of the intensities of the contact, a mean value of the intensities of the contact, an average value of the intensities of the contact, a top 10 percentile value of the intensities of the contact, a value at the half maximum of the intensities of the contact, a value at the 90 percent maximum of the intensities of the contact, or the like. In some embodiments, the duration of the contact is used in determining the characteristic intensity (e.g., when the characteristic intensity is an average of the intensity of the contact over time). In some embodiments, the characteristic intensity is compared to a set of one or more intensity thresholds to determine whether an operation has been performed by a user. For example, the set of one or more intensity thresholds may include a first intensity threshold and a second intensity threshold. In this example, a contact with a characteristic intensity that does not exceed the first threshold results in a first operation, a contact with a characteristic intensity that exceeds the first intensity threshold and does not exceed the second intensity threshold results in a second operation, and a contact with a characteristic intensity that exceeds the second intensity threshold results in a third operation. In some embodiments, a comparison between the characteristic intensity and one or more intensity thresholds is used to determine whether or not to perform one or more operations (e.g., whether to perform a respective option or forgo performing the respective operation) rather than being used to determine whether to perform a first operation or a second operation.
In some embodiments, a portion of a gesture is identified for purposes of determining a characteristic intensity. For example, a touch-sensitive surface may receive a continuous swipe contact transitioning from a start location and reaching an end location (e.g., a drag gesture), at which point the intensity of the contact increases. In this example, the characteristic intensity of the contact at the end location may be based on only a portion of the continuous swipe contact, and not the entire swipe contact (e.g., only the portion of the swipe contact at the end location). In some embodiments, a smoothing algorithm may be applied to the intensities of the swipe contact prior to determining the characteristic intensity of the contact. For example, the smoothing algorithm optionally includes one or more of: an unweighted sliding-average smoothing algorithm, a triangular smoothing algorithm, a median filter smoothing algorithm, and/or an exponential smoothing algorithm. In some circumstances, these smoothing algorithms eliminate narrow spikes or dips in the intensities of the swipe contact for purposes of determining a characteristic intensity.
In some embodiments one or more predefined intensity thresholds are used to determine whether a particular input satisfies an intensity-based criterion. For example, the one or more predefined intensity thresholds include (i) a contact detection intensity threshold IT0, (ii) a light press intensity threshold ITL, (iii) a deep press intensity threshold ITD (e.g., that is at least initially higher than IL), and/or (iv) one or more other intensity thresholds (e.g., an intensity threshold IH that is lower than IL). In some embodiments, the light press intensity threshold corresponds to an intensity at which the device will perform operations typically associated with clicking a button of a physical mouse or a trackpad. In some embodiments, the deep press intensity threshold corresponds to an intensity at which the device will perform operations that are different from operations typically associated with clicking a button of a physical mouse or a trackpad. In some embodiments, when a contact is detected with a characteristic intensity below the light press intensity threshold (e.g., and above a nominal contact-detection intensity threshold IT0 below which the contact is no longer detected), the device will move a focus selector in accordance with movement of the contact on the touch-sensitive surface without performing an operation associated with the light press intensity threshold or the deep press intensity threshold. Generally, unless otherwise stated, these intensity thresholds are consistent between different sets of user interface figures.
In some embodiments, the response of the device to inputs detected by the device depends on criteria based on the contact intensity during the input. For example, for some “light press” inputs, the intensity of a contact exceeding a first intensity threshold during the input triggers a first response. In some embodiments, the response of the device to inputs detected by the device depends on criteria that include both the contact intensity during the input and time-based criteria. For example, for some “deep press” inputs, the intensity of a contact exceeding a second intensity threshold during the input, greater than the first intensity threshold for a light press, triggers a second response only if a delay time has elapsed between meeting the first intensity threshold and meeting the second intensity threshold. This delay time is typically less than 200 ms in duration (e.g., 40, 100, or 120 ms, depending on the magnitude of the second intensity threshold, with the delay time increasing as the second intensity threshold increases). This delay time helps to avoid accidental deep press inputs. As another example, for some “deep press” inputs, there is a reduced-sensitivity time period that occurs after the time at which the first intensity threshold is met. During the reduced-sensitivity time period, the second intensity threshold is increased. This temporary increase in the second intensity threshold also helps to avoid accidental deep press inputs. For other deep press inputs, the response to detection of a deep press input does not depend on time-based criteria.
In some embodiments, one or more of the input intensity thresholds and/or the corresponding outputs vary based on one or more factors, such as user settings, contact motion, input timing, application running, rate at which the intensity is applied, number of concurrent inputs, user history, environmental factors (e.g., ambient noise), focus selector position, and the like. Example factors are described in U.S. patent application Ser. Nos. 14/399,606 and 14/624,296, which are incorporated by reference herein in their entireties.
For example,
An increase of characteristic intensity of the contact from an intensity below the light press intensity threshold ITL to an intensity between the light press intensity threshold ITL and the deep press intensity threshold ITD is sometimes referred to as a “light press” input. An increase of characteristic intensity of the contact from an intensity below the deep press intensity threshold ITD to an intensity above the deep press intensity threshold ITD is sometimes referred to as a “deep press” input. An increase of characteristic intensity of the contact from an intensity below the contact-detection intensity threshold IT0 to an intensity between the contact-detection intensity threshold IT0 and the light press intensity threshold ITL is sometimes referred to as detecting the contact on the touch-surface. A decrease of characteristic intensity of the contact from an intensity above the contact-detection intensity threshold IT0 to an intensity below the contact-detection intensity threshold IT0 is sometimes referred to as detecting liftoff of the contact from the touch-surface. In some embodiments IT0 is zero. In some embodiments, IT0 is greater than zero. In some illustrations a shaded circle or oval is used to represent intensity of a contact on the touch-sensitive surface. In some illustrations, a circle or oval without shading is used represent a respective contact on the touch-sensitive surface without specifying the intensity of the respective contact.
In some embodiments, described herein, one or more operations are performed in response to detecting a gesture that includes a respective press input or in response to detecting the respective press input performed with a respective contact (or a plurality of contacts), where the respective press input is detected based at least in part on detecting an increase in intensity of the contact (or plurality of contacts) above a press-input intensity threshold. In some embodiments, the respective operation is performed in response to detecting the increase in intensity of the respective contact above the press-input intensity threshold (e.g., the respective operation is performed on a “down stroke” of the respective press input). In some embodiments, the press input includes an increase in intensity of the respective contact above the press-input intensity threshold and a subsequent decrease in intensity of the contact below the press-input intensity threshold, and the respective operation is performed in response to detecting the subsequent decrease in intensity of the respective contact below the press-input threshold (e.g., the respective operation is performed on an “up stroke” of the respective press input).
In some embodiments, the device employs intensity hysteresis to avoid accidental inputs sometimes termed “jitter,” where the device defines or selects a hysteresis intensity threshold with a predefined relationship to the press-input intensity threshold (e.g., the hysteresis intensity threshold is X intensity units lower than the press-input intensity threshold or the hysteresis intensity threshold is 75%, 90%, or some reasonable proportion of the press-input intensity threshold). Thus, in some embodiments, the press input includes an increase in intensity of the respective contact above the press-input intensity threshold and a subsequent decrease in intensity of the contact below the hysteresis intensity threshold that corresponds to the press-input intensity threshold, and the respective operation is performed in response to detecting the subsequent decrease in intensity of the respective contact below the hysteresis intensity threshold (e.g., the respective operation is performed on an “up stroke” of the respective press input). Similarly, in some embodiments, the press input is detected only when the device detects an increase in intensity of the contact from an intensity at or below the hysteresis intensity threshold to an intensity at or above the press-input intensity threshold and, optionally, a subsequent decrease in intensity of the contact to an intensity at or below the hysteresis intensity, and the respective operation is performed in response to detecting the press input (e.g., the increase in intensity of the contact or the decrease in intensity of the contact, depending on the circumstances).
For ease of explanation, the description of operations performed in response to a press input associated with a press-input intensity threshold or in response to a gesture including the press input are, optionally, triggered in response to detecting: an increase in intensity of a contact above the press-input intensity threshold, an increase in intensity of a contact from an intensity below the hysteresis intensity threshold to an intensity above the press-input intensity threshold, a decrease in intensity of the contact below the press-input intensity threshold, or a decrease in intensity of the contact below the hysteresis intensity threshold corresponding to the press-input intensity threshold. Additionally, in examples where an operation is described as being performed in response to detecting a decrease in intensity of a contact below the press-input intensity threshold, the operation is, optionally, performed in response to detecting a decrease in intensity of the contact below a hysteresis intensity threshold corresponding to, and lower than, the press-input intensity threshold. As described above, in some embodiments, the triggering of these responses also depends on time-based criteria being met (e.g., a delay time has elapsed between a first intensity threshold being met and a second intensity threshold being met).
Attention is now directed towards embodiments of user interfaces (“UIs”) and associated processes that may be implemented by portable computing system 100 (
In
In other embodiments, volume slider 5100 is displayed in a separate region from the plurality of affordances corresponding to email A. As such, while volume slider 5100 is activated, the plurality of affordances corresponding to email A (e.g., affordances 582, 584, 586, 588, 590, 592, 594, 596, and 598) are animatedly scrolled or shrunken to ensure enough display space is available to display volume slider 5100. In some embodiments, the circle/thumb is displayed under the user's finger in response to the long press gesture to allow the user to slide the circle/thumb without having to remove the contact from the affordance.
In
For example, the user of portable computing system 100 or desktop computing system 200 is able to adjust the orientation of photo B within window 5200 by performing a left-to-right swipe/drag gesture or a right-to-left swipe/drag gesture at a location originating on slider 5210 or within the set of controls 5209. For example, in response to detecting an upward swipe gesture on dynamic function row 104, dynamic function row 104 displays a set of controls corresponding to crop tool 5210. In another example, in response to detecting a downward swipe gesture on dynamic function row 104, dynamic function row 104 displays a set of controls corresponding to red-eye reduction tool 5206.
In
In some embodiments, the dynamic function row 104 (also referred to herein as the “touch-sensitive secondary display 104”) is populated with affordances that allow a user to easily manipulate a visual characteristic used to render content on the primary display 102. A few examples are shown in
In some embodiments, the affordances that each correspond to various editing functions include: (i) a general editing affordance 5501 that, when selected, causes the touch-sensitive secondary display to display general editing options; (ii) a highlight styles affordance 5502 that, when selected, causes the touch-sensitive secondary display to begin displaying highlighting options; and (iii) a markup affordance 5503 that, when selected (shown as selected in
In some embodiments, in response to a selection of markup affordance 5503, the touch-sensitive secondary display begins displaying the options for modifying visual characteristics. In some embodiments and as shown in
In some embodiments, in response to detecting a selection of a color picker affordance (e.g., selection 5010-A,
In some embodiments, if the user initiates a press-and-hold gesture (e.g., the user contacts a particular color shown in the user interface control 5505 and maintains the contact for more than a threshold amount of time, such as 0.5, 1, 1.5, or 2 seconds) or if the user contact over the particular color and a characteristic intensity of that contact satisfy an intensity threshold, then the user interface control 5505 shown in the touch-sensitive secondary display is updated to include additional options that correspond to the particular color. For example, in response to input 5010-B over a representation of the color pink (e.g., a press-and-hold input or an input that satisfies the intensity threshold), the user interface control is updated to include additional shades of pink that are available for selection. As shown in
In some embodiments, the user is able to slide input 5010-C across the representations of the additional shades of pink on the touch-sensitive secondary display and, in response, the head of the editable object shown on the primary display 102 is rendered using each contacted shade of pink. As shown in
In some embodiments, the user interface control 5505 has at least two display modes that allow for choosing between basic and advanced modifying functions for a particular visual characteristic. In some embodiments, users are able to toggle between these at least two display modes by selecting a mode-switching affordance 5506. For example, in response to input 5003 shown in
In some embodiments, the application-specific section of the touch-sensitive secondary display 104 changes to include different affordances when the user opens up a new application (e.g., switches from the content-editing application shown in
By providing representations of each of the tabs directly within the touch-sensitive secondary display 104, users are able to navigate through tabs very quickly (e.g., instead of having to cycle through tabs using a trackpad or using a complicated sequence of keyboard inputs). In some embodiments, the user is able to provide a single gesture at the touch-sensitive secondary display 104 that allows for quick navigation between each of the tabs shown on the primary display 102. An example of such quick navigation between three different tabs of web content using affordances shown in the touch-sensitive secondary display 104 is shown in
In some embodiments, the representations shown in the touch-sensitive secondary display show a shrunken/mini-view of content associated with a corresponding tab in the web browser on primary display 102 (e.g., as shown in
In some embodiments, users are able to navigate through a hierarchy of favorite sites that is displayed on the primary display 102 using affordances that are shown in the touch-sensitive secondary display 104. For example,
In accordance with a determination that one or more of the favorite sites shown on the primary display 102 are from a same base domain (e.g., URL), the touch-sensitive secondary display includes additional details about the one or more favorite sites. For example, the “Design Milk” and “Dwell” favorites are both from the “twitter.com” domain and, as such, the touch-sensitive secondary display 104 includes additional details (e.g., a TWITTER handle in addition to an icon for TWITTER, as shown in
In some embodiments, the touch-sensitive secondary display 104 also includes affordances for sending payment after a user provides an authenticated fingerprint (as shown in
In some embodiments, the touch-sensitive secondary display 104 also displays affordances that correspond to audio recording, video recording, and screen recording functions available via a media player application. Some examples are shown in
In some embodiments, in response to a selection of the affordance 5060, the touch-sensitive secondary display 104 is updated to include selectable options for each available audio input (e.g., an option for “Built-in Audio” and another option for “Neumann KMS,” as shown in
Turning now to
In some embodiments, if a user begins recording audio, video, or screen content and the user exits the media application, then the touch-sensitive secondary display 104 is updated to include status indicators (such as a file size for the ongoing recording) and user interface controls that allow for controlling the ongoing recording (
In some embodiments, controls and/or affordances presented in the touch-sensitive secondary display 104 also help to provide quick and easy access to useful functions while playing a media item. Some examples of useful functions include seeking through the media item, selecting subtitle and audio configuration options, and/or trimming the media item (example user interfaces shown in the touch-sensitive secondary display 104 to perform the useful functions are provided in
As shown in
In some embodiments, the representations of portions of the media item correspond to a type of the media item. For example, in accordance with a determination that the media item is of a first type (e.g., stored video content or other video content for which thumbnails are available), the touch-sensitive secondary display 104 includes thumbnail representations of particular frames within the media item. In accordance with a determination that the media item is of a second type (e.g., streaming video content for which thumbnail previews are not available), the touch-sensitive secondary display 104 does not include thumbnail representations (as shown in
In some embodiments, in accordance with a determination that subtitles are available (e.g., via a media content provider or other associated entity), the touch-sensitive secondary display 104 includes an audio-configuration affordance 5805 (as shown for
In some embodiments, the media item discussed above is also editable and the touch-sensitive secondary display 104 includes user interface controls that allow for modifying the media item (e.g., to trim the media item, as shown in
Turning now to
In response to detecting the input 5016, the touch-sensitive secondary display 104 displays the new object affordance in an inactive state (i.e., the new object affordance is greyed out and is not selectable), displays a reminder completion affordance (e.g., a checkmark icon that, when selected, causes a computing device in communication with the touch-sensitive secondary display 104 to mark a currently selected reminder on the primary display 102 as complete), a reminder details affordance (e.g., an i included within a circle that, when selected, causes the computing device to display options for editing details associated with the currently selected reminder), and text-entry options for adding text to the currently selected reminder. In some embodiments, the reminder completion affordance and the reminder details affordance are initially shown in the inactive state until the user provides a title for the currently selected reminder. As shown in
In response to an input at the reminder details affordance (e.g., input 5017), the touch-sensitive secondary display 104 is populated to include an “Add Time” affordance and an “Add Location” affordance (
In response to detecting input 5018 at the Add Time affordance, the touch-sensitive secondary display 104 is populated to include options for selecting a date and time for the currently selected reminder on the primary display 102. As shown in
In some embodiments, in a response to a gesture (e.g., a press and hold gesture that remains in contact with the touch-sensitive secondary display 104 for more than a predetermined amount of time, such as 0.5-1.5 seconds) at a date affordance (e.g., showing “June 25” in
In response to detecting input 5019 over the Add Location affordance, the touch-sensitive secondary display 104 is populated to include location-selection options (as shown in
In some embodiments, the touch-sensitive secondary display 104 is populated to include affordances that allow for accessing functions available via a desktop management application (e.g., FINDER application).
In some embodiments, when a user initially accesses the desktop management application, the touch-sensitive secondary display 104 does not display any application-specific affordances for the desktop management application, as shown in
In some embodiments, in response to a selection of a respective file that is displayed within the new window shown (e.g., a selection of a “read-only file,”
In some embodiments, in response to a selection of any one of the options (e.g., in response to any one of the inputs 5021, 5022, and 5023), the touch-sensitive secondary display 104 is updated. For example: in response to input 5021, the touch-sensitive secondary display 104 is updated to include options for tagging the respective file (as shown in
In some embodiments (and as shown in
As another example, in response to input 5024-A (
In some embodiments, while viewing a trash window or while viewing some other window (e.g., an All My Files window), the user is able to begin searching all files by selecting a search box. In response to a selection of the search box, the touch-sensitive secondary display 104 is updated to include searching options (e.g., options for choosing a directory or folder in which the search should be conducted,
In some embodiments, the touch-sensitive secondary display 104 is populated to include affordances that allow for quickly accessing document editing functions (a few examples are provided in
In particular, as shown in
As shown in
In
For example, In accordance with a determination that the type of object that is currently selected on the primary display 102 is of a first type (e.g., is a textual object, such as the “** New Text Object” shown in
In some embodiments, if a user attempts to exit an application (e.g., such as the document-editing application shown on primary display 102 in
In some embodiments, document-editing functions are available to markup objects included within mail messages (e.g., emails, text messages, etc.).
In some embodiments, the touch-sensitive secondary display 104 is populated to include affordances for performing unlock operations. For example,
In some embodiments, the touch-sensitive secondary display 104 is also populated to include instructions for performing an unlock function in response to (i) the user accessing a login screen (e.g., as shown in
In some embodiments, if the computing system is associated with at least two users (e.g., the computing system has been configured to have login profiles for each of the at least two users), then when the computing system is initially started up, a login screen is displayed that allows for selecting whether to login as a first or a second of the at least two users. In some embodiments, in response to detecting that a respective user has provided an authorized fingerprint (e.g., at a region of the touch-sensitive secondary display 104 that accepts fingerprints), the computing system matches the authorized fingerprint to one of the first or the second users and then causes the primary display 102 to show a new login screen that is specific to the matched user (e.g., the new login screen asks the respective user to enter a password to gain access to the computing system). In this way, the respective user simply provides a fingerprint instead of having to navigate using a trackpad or mouse to select one of the at least two users from the login screen.
In some embodiments, providing an authorized fingerprint enables the respective user to gain immediate access to the computing system. In some embodiments, an initial login has already been performed (e.g., during the initial login, a password is entered to gain access to the computing system) and the computing system has been locked at a later point in time. If a user then provides an authorized fingerprint to gain access after that later point in time, then in accordance with a determination that the authorized fingerprint matches a valid user of the computing system, then immediate access to the computing system is granted. In some embodiments, when the computing system is locked at the later point in time, at least two users have active login sessions at the computing system (either of these two users is able to simply provide a fingerprint in order to re-gain access to the computing system). Additional details regarding the use of biometric sensors to gain access and perform other functions at the computing system are provided below for methods 2800 and 2900, described below in reference to
In some embodiments, the touch-sensitive secondary display 104 is populated to include affordances for navigating through photos (or other media items), for modifying photos, and for performing other functions available via a photo-editing or photo-browsing application. Some examples are shown in
In particular,
By providing an input at the touch-sensitive secondary display 104 (e.g., a tap or a swipe gesture), the user is able to quickly navigate through the plurality of photos on the primary display 102. For example, as shown in
In response to a selection of a tool navigation affordance (e.g., via input 5031,
Returning to the discussion of
Turning now to
Turning now to
In some embodiments, after a respective slider control is modified in the touch-sensitive secondary display 104, a checkmark that is highlighted using blue is displayed to indicate that a photo displayed on the primary display 102 has been rendered using the modification. In some embodiments, users are easily able to revert modifications by tapping at the checkmark (e.g., in response to input 5043 at the checkmark for the color slider control, Photo E is no longer rendered using the color modifications and the blue highlighting at the touch-sensitive secondary display 104 is no longer presented, as shown in
Additional examples of controls displayed in the touch-sensitive secondary display 104 and used to browse through photos are shown in
In some embodiments, the representations that are displayed in the touch-sensitive secondary display 104 are presented differently for various types of videos (e.g., an example for a slo-mo video is shown in
In some embodiments, controls displayed within the touch-sensitive secondary display 104 are provided while navigating through an albums tabs in a photos application (e.g., as shown in
In some embodiments, controls displayed within the touch-sensitive secondary display 104 are provided for slideshows and print products that are accessed through a photos application (examples are shown in
In some embodiments, while searching for photos within a photos application, the touch-sensitive secondary display 104 is populated with controls for selecting map viewing options (e.g., to help narrow down a search for photos taken near a particular location,
In some embodiments, the touch-sensitive secondary display 104 is populated with affordances to perform system-wide search functions. Some examples are shown in
In some embodiments, the system-wide search function is activated using a gesture at the touch-sensitive secondary display. For example, in response to a swipe gesture in a substantially downward vertical direction across the touch-sensitive secondary display 104, the primary display is caused to display a user interface for the system-wide search (e.g., the user interface shown in
In some embodiments, the touch-sensitive secondary display 104 is populated with affordances to select between various workspaces (e.g., using a MISSION CONTROL feature available through some operating systems). Some examples are shown in
In some embodiments, the touch-sensitive secondary display 104 is populated with affordances to navigate through numerous search results that are displayed on the primary display 102. Some examples are shown in
In some embodiments, the touch-sensitive secondary display 104 is populated with affordances to quickly access functions while using a messaging application on the primary display 102. Examples are shown in
As shown in
In some embodiments, in response to a selection of a text entry field for the new message (e.g., input 5046-A), the touch-sensitive secondary display 104 begins displaying (shown from left to right in the touch-sensitive secondary display 104 of
In some embodiments, after selecting the microphone affordance, the touch-sensitive secondary display 104 is updated to include a record affordance that, when selected, causes the computing system to begin recording audio. In some embodiments, while audio is being recorded, the touch-sensitive secondary display 104 is updated to include a representation of the recorded audio (as shown in
In some embodiments, after contact by an input with the microphone affordance, the input remains in contact with the microphone affordance to begin an audio recording (as shown in
In some embodiments, in response to a selection of a messaging bubble that is included in a particular conversation thread shown within the messaging application (e.g., a messaging bubble that includes content received from some other user) on the primary display 102, the touch-sensitive secondary display 104 is updated to include controls for selecting bubble acknowledgements (e.g., adjacent to the photo picker affordance are example controls for selecting bubble acknowledgements). In some embodiments, if the selected messaging bubble corresponds to a message sent by the user (i.e., not by some other user), then the touch-sensitive secondary display 104 does not include the controls for selecting bubble acknowledgements.
In some embodiments, the touch-sensitive secondary display 104 is populated with affordances for controlling a contacts application, as shown in
In some embodiments, the optional telephone calling, video calling, instant messaging, and email are each displayed in accordance with a determination that a contact that is currently selected (i.e., has a focus selector on the primary display 102) is associated with contact details that allow for placing a telephone call (e.g., that a telephone number is stored for the contact), placing a video call (e.g., that an appropriate username or email is stored for the contact), sending an instant message (e.g., that an appropriate username or email is stored for the contact), or sending an email (e.g., that an email address is stored for the contact). In some embodiments, if this determination is not satisfied then a respective optional affordance is either not displayed at all or is displayed in an inactive state (i.e., the respective affordance is displayed in a greyed out, un-selectable state).
As another example, in accordance with a determination that a phone number field is being editing (e.g., has the focus selector on the primary display 102, as shown in
In some embodiments, the touch-sensitive secondary display 104 is populated with affordances for controlling a calculator application, as shown in
In some embodiments, the touch-sensitive secondary display 104 is populated with affordances for controlling a calendar application and for modifying events that are associated with the calendar application, as shown in
In some embodiments, the touch-sensitive secondary display 104 is populated with affordances for controlling a mail application, as shown in
In some embodiments, in response to a selection of a move-to option (on either the primary display 102 or the touch-sensitive secondary display 104), the touch-sensitive secondary display 104 is updated to include controls for moving an email to a different folder or repository (examples are shown in
In some embodiments, while a user is editing or composing an email message, the touch-sensitive secondary display 104 is updated to include affordances for text completion (“I,” “The,” and “it” options shown in
In some embodiments, the affordances for text completion include word predictions (an example is shown in
In some embodiments, the touch-sensitive secondary display 104 is populated with affordances for controlling a telephone application, as shown in
In some embodiments, in response to detecting acceptance of the incoming call, the computing system causes the touch-sensitive secondary display 104 to display status information for the incoming call as well as an end affordance for ending the incoming call and a mute affordance for muting the incoming call, as shown in
In some embodiments, if the user was viewing a telephone application (e.g., a FACETIME application) when the incoming call was received, then the touch-sensitive secondary display 104 displays the status information as shown in
In some embodiments, in accordance with a determination that the incoming call is an audio call, then the touch-sensitive secondary display 104 is populated with a video affordance that, when selected, initiates a request to the incoming caller to begin a video call (an example of the video affordance is shown in
In some embodiments, if a new call is received while the incoming call discussed above is ongoing, then the touch-sensitive secondary display is populated to include information and affordances that correspond to the new call (e.g., as shown in
In some embodiments, in accordance with a determination that a call failure has occurred, the touch-sensitive secondary display 104 is populated with affordances for messaging and/or calling back another party from the failed call (e.g., as shown in
In some embodiments, the touch-sensitive secondary display 104 is populated with affordances for controlling an activity monitoring application, as shown in
In some embodiments, the touch-sensitive secondary display 104 is populated with affordances for controlling an activity logging application, as shown in
In some embodiments, the touch-sensitive secondary display 104 is populated with affordances for accessing certain functions while in a different boot mode (such as a mode in which the computing system boots up in an operating system that is not compatible with dynamically rendering controls at the touch-sensitive secondary display 104,
In some embodiments, the touch-sensitive secondary display 104 is populated with affordances for controlling or providing additional information to a virtual assistant application, as shown in
In some embodiments, the touch-sensitive secondary display 104 is populated with affordances for controlling text entry functions and for accessing a set of non-English characters, as shown in
In some embodiments, the touch-sensitive secondary display 104 includes a smart label button that occupies a leftmost region of the touch-sensitive secondary display 104. In some embodiments, the smart label button changes based on what a user is viewing on the primary display 102 and, in some circumstances, based on what controls are available in the touch-sensitive secondary display 104 at a particular point in time. For example, the smart label button may display “esc” (allowing a user to exit a currently displayed application that is shown on the primary display 102), “x” (allowing a user to exit an overlay display mode and revert to displaying controls for an application that has a focus selector on the primary display 102), “cancel” (allowing a user to cancel/revert changes made using various controls at the touch-sensitive secondary display 104), “done” (allowing a user to indicate that they are done making certain changes via the touch-sensitive secondary display 104), and “exit.”
In some embodiments, the touch-sensitive secondary display 104 is associated with multiple display modes. In some embodiments, a three finger (or two finger) gesture is provided at the touch-sensitive secondary display 104 to switch between each of the multiple modes. In some embodiments, the gesture is provided in a substantially horizontal or a substantially vertical direction. In some embodiments, the multiple modes include: (i) a customized mode in which the touch-sensitive secondary display 104 displays affordances that have been customized by the user for use with the customized mode; (ii) a current context mode in which the touch-sensitive secondary display 104 displays affordances that correspond to the primary display and system-level functions; and (iii) a system function mode in which the touch-sensitive secondary display 104 displays system functions only.
In some embodiments, affordances presented in the touch-sensitive secondary display 104 are customizable and users are able to add and/or remove affordances from the touch-sensitive secondary display 104 (in some embodiments, customization is available only when the touch-sensitive secondary display 104 is in the customized mode discussed above). For example, in embodiments in which the touch-sensitive secondary display 104 is displayed above a laptop keyboard and below the laptop's primary display, an affordance (or menus including multiple affordances) presented on the primary display may be dragged down to the touch-sensitive secondary display and then moved horizontally within the touch-sensitive secondary display using one continuous gesture. In some embodiments, as the affordance is moved around within the touch-sensitive secondary display, various sections (e.g., an application-specific and a system-level region) of the touch-sensitive secondary display are emphasized/highlighted.
In some embodiments, a user is able to provide drawing inputs at a trackpad of the computing system and, in response to receiving the drawing inputs, the computing system causes the touch-sensitive secondary display 104 to present candidate shapes for selection by the user.
In some embodiments, a brightness level for the touch-sensitive secondary display 104 is determined based at least in part on a brightness level for the primary display 102 and based at least in part on ambient light conditions around the computing system.
Attention is now directed to
As shown in
In some embodiments, in response to user input at the affordance 7002, the maps application on the primary display 102 is updated to include affordances for selecting nearby categories of places (e.g., nearby restaurants, coffee shops, shopping destinations, travel points of interest, services, fun locations, health locations, and transportation-related services) (
Turning now to
In the example shown in
Turning now to
For example, as shown in
In some embodiments, the touch-sensitive secondary display 104 may also include additional rows within user interface control 7102. Two examples are shown in
FIGS. 71D2-71M are schematics of the touch-sensitive secondary display 104 used to illustrate additional user interface controls that may be displayed for controlling certain functions available in a video-editing application, such as FINAL CUT PRO. For example, FIG. 71D2 shows example user interface controls that may be displayed for controlling functions associated with creating a new video project,
Additional examples are also shown in
Additional details regarding the user interface controls and affordances shown in
Turning now to
In some embodiments and as explained above, to help efficiently utilize limited display screen real estate at the touch-sensitive secondary display 102, users are able to select a leftward pointing caret to cause the touch-sensitive secondary display 102 to cease displaying system-level affordances and to instead display more of a particular user interface control, such as the timeline view shown in the touch-sensitive secondary display of
Example user interface controls for editing text within a video-editing application are provided in
Additional details regarding the user interface controls and affordances shown in
Attention is now directed to
Additional examples are also shown in
Additional details regarding the user interface controls and affordances shown in
Attention is now directed to
Additional details regarding the user interface controls and affordances shown in
Turning now to
In some embodiments, the computing system is portable computing system 100 (
The computing system displays (602) a first user interface on the primary display, the first user interface comprising one or more user interface elements.
In some embodiments, the computing system further comprises (604): (i) a primary computing device comprising the primary display, the processor, the memory, and primary computing device communication circuitry; and (ii) a input device comprising the housing, the touch screen display, the physical input mechanism, and input device communication circuitry for communicating with the primary computing device communication circuitry, where the input device is distinct and separate from the primary computing device. In some embodiments, the computing system is not a laptop, such as portable computing system 100 (
In some embodiments, the physical input mechanism comprises (606) a plurality of physical keys. In
In some embodiments, the physical input mechanism comprises (608) a touchpad. In
The computing system identifies (610) an active user interface element among the one or more user interface elements that is in focus on the primary display. In some embodiments, the term “in focus” can refer to the active element of the user interface (e.g., a window associated with an application, a particular toolbar or menu associated with an application, or the operating system) that is currently in the foreground and actively running or is controllable by input received from a user of the computing system such as a key press, mouse click, voice command, gestural motion, or the like.
In some embodiments, the computing system or a component thereof (e.g., focus determining module 351,
In
The computing system determines (612) whether the active user interface element that is in focus on the primary display is associated with an application executed by the computing system. In some embodiments, the active user interface element is associated with either an application or the operating system. In some embodiments, the computing system or a component thereof (e.g., DFR determining module 352,
In some embodiments, the application is executed (614) by the processor in the foreground of the first user interface. For example, the application is one of an email application, a word processing application, a presentation application, a photo editing application, a music application, a game application, a spreadsheet application, or the like.
In accordance with a determination that the active user interface element that is in focus on the primary display is associated with the application executed by the computing system, the computing system displays (616) a second user interface on the touch screen display, including: (A) a first set of one or more affordances corresponding to the application; and (B) at least one system-level affordance corresponding to at least one system-level functionality. In some embodiments, the first set of one or more affordances includes user selectable symbols/icons and/or indicators and information that may or may not be selectable. In some embodiments, the first set of one or more affordances correspond to basic controls for the application. In some embodiments, at least one system-level affordance is displayed along with the first set of one or more affordances. In one example, in
Displaying application-specific and system-level affordances in a touch-sensitive secondary display in response to changes in focus made on a primary display provides the user with accessible affordances that are directly available via the touch-sensitive secondary display. Providing the user with accessible affordances that are directly accessibly via the touch-sensitive secondary display enhances the operability of the computing system and makes the user-device interface more efficient (e.g., by helping the user to access needed functions directly through the touch-sensitive secondary display with fewer interactions and without having to waste time digging through hierarchical menus to locate the needed functions) which, additionally, reduces power usage and improves battery life of the device by enabling the user to access the needed functions more quickly and efficiently. As well, the display of application-specific affordances on the touch-sensitive secondary display indicates an internal state of the device by providing affordances associated with the application currently in focus on the primary display.
In some embodiments, the user of the computing system is able to modify or customize the affordances included in the first set of one or more affordances. In some embodiments, prior to identifying the active user interface element that is in focus on primary display 102 and displaying the first set of one or more affordances, dynamic function row 104 displays a default interface set by the user of the computing system or set in software. For example, the default interface includes one of: a plurality of function keys (e.g., F1, F2, F3, . . . , F12), a stock ticker, scrolling sports scores, scrolling weather forecasts and information, and/or the like.
In some embodiments, after displaying the first set of one or more affordances for a first predetermined period of time (e.g., 30, 60, 90, etc. seconds), dynamic function row 104 re-displays the default interface set by the user of the computing system or set in software. In some embodiments, after displaying the first set of one or more affordances for a first predetermined period of time (e.g., 30, 60, 90, etc. seconds), dynamic function row 104 turns off until a contact is detected by dynamic function row 104 or a keyboard or touchpad associated with the computing system. In some embodiments, after displaying the first set of one or more affordances for a first predetermined period of time (e.g., 30, 60, 90, etc. seconds), dynamic function row 104 turns off until connected to a power source (e.g., when dynamic function row is implemented in battery powered peripheral keyboard 206,
In some embodiments, after displaying the first set of one or more affordances for the first predetermined period of time (e.g., 30, 60, 90, etc. seconds) and when the computing system is set in a low-power mode, dynamic function row 104 displays a limited set of affordances including, for example, the time, the battery life remaining, the Wi-Fi signal strength, and/or the like. For example, the limited set of affordances are selected by the user of the computing system or set in software, and the user of computing system is able to set the computing into the low-power mode through a system settings panel.
In some embodiments, at least one of the affordances displayed on the second user interface is (618) a multi-function affordance. In some embodiments, a multi-function affordance is capable of performing two or more functions/operations in response to detecting different inputs performed at a location corresponding to the multi-function affordance. For example, persistent volume control 568, in
In some embodiments, the computing system detects (620) a user touch input selecting the multi-function affordance. In accordance with a determination that the user touch input corresponds to a first type, the computing system performs a first function associated with the multi-function affordance. In accordance with a determination that the user touch input corresponds to a second type distinct from the first type, the computing system performs a second function associated with the multi-function affordance. For example, a first function/operation (e.g., mute a media item) is performed in response to detecting a first gesture (e.g., a tap contact) at a location corresponding to the multi-function affordance, and a second function/operation (e.g., display a volume slider or playback controls for the media item) is performed in response to detecting a second gesture (e.g., a long press gesture) at a location corresponding to the multi-function affordance. For example, the first gesture type corresponds to a touch input detected for less than a predetermined period of time (e.g., 500 ms, 1 s, etc.) with one or more contacts (e.g., a tap contact), and the second gesture type corresponds to a touch input detected for greater than or equal to the predetermined period of time (e.g., 500 ms, 1 s, etc.) with one or more contacts (e.g., a long press gesture).
In some embodiments, the least one system-level affordance is configured (622) upon selection to cause display of a plurality of system-level affordances corresponding to system-level functionalities on the touch screen display. In some embodiments, the at least one system-level affordance enables access to a plurality of system-level controls/affordances such as volume and brightness controls, and other system-level functionalities. For example, in
In some embodiments, the at least one system-level affordance corresponds to (624) one of a power control or escape control. In some embodiments, the at least one system-level affordance includes persistent controls that are displayed on dynamic function row 104 regardless of the focus of primary display 102 (e.g., escape affordance 516 and power control 534,
In some embodiments, the computing system detects (626) a user touch input selecting one of the first set of affordances, and, in response to detecting the user touch input, the computing system: displays a different set of affordances corresponding to functionalities of the application; and maintains display of the at least one system-level affordance. In some embodiments, the first set of one or more affordances corresponding to the application includes a single affordance for accessing a set of tools or functions associated with the application.
In some embodiments, the computing system detects (628) a subsequent user touch input selecting the at least one system-level affordance, and, in response to detecting the subsequent user touch input, the computing system displays a plurality of system-level affordances corresponding to system-level functionalities and at least one application-level affordance corresponding to the application.
In some embodiments, after displaying the second user interface on the touch screen display, the computing system identifies (630) a second active user interface element among the one or more user interface elements that is in focus on the primary display and determines whether the second active user interface element corresponds to a different application executed by the computing device. In accordance with a determination that the second active user interface element corresponds to the different application, the computing system displays a fourth user interface on the touch screen display, including: (D) a third set of one or more affordances corresponding to the different application; and (E) the at least one system-level affordance corresponding to the at least one system-level functionality.
In some embodiments, the computing system provides audible cues indicating the different user interface element that is in focus. In some embodiments, primary display 102 displays a visual cue indicating the different user interface element that is in focus. For example, with respect to
In some embodiments, after identifying that the second active user interface element, the computing system determines (632) whether a media (e.g., audio or video) item is being played by the computing system, where the media item is not associated with the different application, and, in accordance with a determination that the media item is being played by the computing system, the computing system displays at least one persistent affordance on the touch screen display for controlling the media item (e.g., volume and/or playback controls). In some embodiments, the at least one affordance is a persistent control that enables the user of the computing system to mute/unmute the media item from the touch screen display even if the focus changes or the media is muted and/or being played in the background.
In some embodiments, the at least one persistent affordance displays (634) feedback that corresponds to the media item (e.g., an equalizer (EQ) bar, a run-time indicator, or the like)
In some embodiments, the computing device detects (636) a user input corresponding to an override key, and, in response to detecting the user input, the computing system: ceases to display at least the first set of one or more affordances of the second user interface on the touch screen display; and displays a first set of default function keys. In some embodiments, the user input corresponding to the override key is detected in response to receiving a signal from the input mechanism (e.g., actuation of a specified physical key such as a function key) or from the dynamic function row (e.g., selection of a virtual key such as “escape” affordance 516). For example, the user actuates an “fn” key to dismiss current affordances displayed by dynamic function row 104 and to display the default F1, F2, F3, . . . , F12 row.
In some embodiments, in response to detecting the user input corresponding to the override key, dynamic function row 104 displays a default interface set by the user of the computing system or set in software. For example, the default interface includes one of: a plurality of function keys (e.g., F1, F2, F3, . . . , F12), a stock ticker, scrolling sports scores, scrolling weather forecasts, or the like.
In some embodiments, after displaying the first set of default function keys, the computing system detects (638) a gesture in a direction substantially parallel to a major dimension of on the touch screen display, and in response to detecting the swipe gesture, displays a second set of default function keys with at least one distinct function key (e.g., a previously undisplayed function key). For example, with respect to
In some embodiments, in accordance with a determination that the active user interface element is not associated with the application executed by the computing system, the computing system displays (640) a third user interface on the touch screen display, including: (C) a second set of one or more affordances corresponding to operating system controls of the computing system, where the second set of one or more affordances are distinct from the first set of one or more affordances. In
In some embodiments, the second set of one or more affordances is (642) an expanded set of operating system controls that includes (B) the at least one system-level affordance corresponding to the at least one system-level functionality.
It should be understood that the particular order in which the operations in
In some embodiments, the computing system is portable computing system 100 (
The computing system displays (702) a first user interface for an application executed by the computing system on the primary display.
The computing system displays (704) a second user interface on the touch screen display, the second user interface comprising a first set of one or more affordances corresponding to the application, where the first set of one or more affordances corresponds to a first portion of the application. In some embodiments, the first set of one or more affordances associated with a top menu or a file menu (i.e., the first portion or sub-section) of the application.
The computing system detects (706) a swipe gesture on the touch screen display. In a first example,
In accordance with a determination that the swipe gesture was performed in a first direction (e.g., horizontal), the computing system displays (708) a second set of one or more affordances corresponding to the application on the touch screen display, where at least one affordance in the second set of one or more affordances is distinct from the first set of one or more affordances, and where the second set of one or more affordances also corresponds to the first portion of the application. In some embodiments, in response to detecting a swipe gesture in the first direction, the dynamic function row displays different a second set of affordances corresponding to the first portion of the application (e.g., a toolset or menu) with at least one different affordance (e.g., a tool or item), as compared to the first set of affordances that also correspond to the first portion of the application. For example, in response to detecting the right-to-left swipe gesture in
In some embodiments, the first direction is (710) substantially parallel to a major dimension of the touch screen display. For example, with reference to portable computing system 100 (
In some embodiments, the first direction is (712) substantially perpendicular to a major dimension of the touch screen display. For example, with reference to portable computing system 100 (
In accordance with a determination that the swipe gesture was performed in a second direction substantially perpendicular to the first direction (e.g., vertical), the computing system displays (714) a third set of one or more affordances corresponding to the application on the touch screen display, where the third set of one or more affordances is distinct from the second set of one or more affordances, and where the third set of one or more affordances corresponds to a second portion of the application that is distinct from the first portion of the application. In some embodiments, in response to detecting a swipe gesture in the second direction, the dynamic function row displays different a third set of affordances corresponding to the second portion of the application (e.g., a different toolset or menu), as compared to the first portion of the application associated with the first set of affordances. For example, in response to detecting the upward swipe gesture in
As such, in one embodiment, a horizontal swipe detected on the dynamic function row 104 moves or scrubs within a group or category and a vertical swipe changes the category/filter. As a result, the user of the computing system is able to move easily navigate tools and menus without moving his/her hands away from the keyboard to a mouse. This also enables more efficient display of information and allows for a more efficient man-machine interaction.
Allowing a user to quickly navigate through application-specific affordances in a touch-sensitive secondary display in response to swipe gestures provides the user with a convenient way to scroll through and quickly locate a desired function via the touch-sensitive secondary display. Providing the user with a convenient way to scroll through and quickly locate a desired function via the touch-sensitive secondary display enhances the operability of the computing system and makes the user-device interface more efficient (e.g., by helping the user to access needed functions directly through the touch-sensitive secondary display with fewer interactions and without having to waste time digging through hierarchical menus to locate the needed functions) which, additionally, reduces power usage and improves battery life of the device by enabling the user to access the needed functions more quickly and efficiently. Furthermore, by dynamically updating affordances that are displayed in the touch-sensitive secondary display in response to swipe gestures at the secondary display, the secondary display is able to make more information available on a limited screen, and helps to ensure that users are provided with desired options right when those options are needed (thereby reducing power usage and extending battery life, because users do not need to waste power and battery life searching through hierarchical menus to located these desired options).
In some embodiments, the second portion is displayed (716) on the primary display in a compact view within the first user interface prior to detecting the swipe gesture, and the system displays the second portion on the primary display in an expanded view within the first user interface in accordance with the determination that the swipe gesture was performed in the second direction substantially perpendicular to the first direction. For example, in
In another example, in
In some embodiments, the first portion is (718) one of a menu, tab, folder, tool set, or toolbar of the application, and the second portion is one of a menu, tab, folder, tool set, or toolbar of the application. In
In some embodiments, after displaying the third set of one or more affordances on the touch screen display, the computing system (720): detects a user input selecting the first portion on the first user interface; and, in response to detecting the user input: ceases to display the third set of one or more affordances on the touch screen display, where the third set of one or more affordances corresponds to the second portion of the application; and displays the second set of one or more affordances, where the second set of one or more affordances corresponds to the first portion of the application. For example, with respect to
In some embodiments, the second set of one or more affordances and the third set of one or more affordances includes (722) at least one system-level affordance corresponding to at least one system-level functionality. For example, in
In some embodiments, the first user interface for the application executed by the computing system is displayed (724) on the primary display in a full-screen mode, and the first set of one or more affordances displayed on the touch screen display includes controls corresponding to the full-screen mode.
In another example, with respect to
It should be understood that the particular order in which the operations in
In some embodiments, the computing system is portable computing system 100 (
The computing system displays (802), on the primary display in a normal mode, a first user interface for the application executed by the computing system, the first user interface comprising at least a first set of one or more affordances associated with an application.
The computing system detects (804) a user input for displaying at least a portion of the first user interface for the application in a full-screen mode on the primary display. For example, the detected user input corresponds to a selection of a full-screen view affordance or a slideshow affordance displayed on primary display 102 or dynamic function row 104.
In some embodiments, the user input for displaying at least the portion of the first user interface for the application in full-screen mode on the primary display is (806) at least one of a touch input detected on the touch screen display and a control selected within the first user interface on the primary display. In a first example,
In response to detecting the user input, the computing system (808): ceases to display the first set of one or more affordances associated with the application in the first user interface on the primary display; displays, on the primary display in the full-screen mode, the portion of the first user interface for the application; and automatically, without human intervention, displays, on the touch screen display, a second set of one or more affordances for controlling the application, where the second set of one or more affordances correspond to the first set of one or more affordances. Continuing with the first example above,
Continuing with the second example above,
In another example, when the web browser application or a portion of the user interface that corresponds to the web browser application is in focus on primary display 102 and the computing system detects a user input to enter full-screen mode, primary display 102 displays a currently active tab in full-screen mode, and dynamic function row 104 displays thumbnail images corresponding to tabs open within the web browser application along with the at least one system-level affordance. For example, the user of the computing system is able to display a respective tab on primary display 102 in full-screen mode by selecting a thumbnail corresponding to the respective tab on dynamic function row 104.
In some embodiments, the second set of one or more affordances is (810) the first set of one or more affordances. For example, the second set of affordances includes at least a portion of the first set of affordances associated with the application. In another example, the second set of affordances includes the first set of affordances associated with the application. In another example, the second set of affordances is the same as the first set of affordances associated with the application. For example, the second set of affordances includes controls associated with the application executed by the computing system such as photo editing controls, gaming controls, slideshow controls and previews, currently opened web page tabs for a web browser, etc.
In some embodiments, the second set of one or more affordances includes (812) controls corresponding to the full-screen mode.
In some embodiments, the second set of one or more affordances includes (814) at least one system-level affordance corresponding to at least one system-level functionality.
Providing affordances for controlling an application via a touch-sensitive secondary display, while a portion of the application is displayed in a full-screen mode on a primary display, allows users to continue accessing functions that may no longer be directly displayed on a primary display. Allowing users to continue accessing functions that may no longer be directly displayed on a primary display provides the user with a quick and convenient way to access functions that may have become buried on the primary display and thereby enhances the operability of the computing system and makes the user-device interface more efficient (e.g., by helping the user to access needed functions directly through the touch-sensitive secondary display with fewer interactions and without having to waste time digging through hierarchical menus to locate the needed functions) which, additionally, reduces power usage and improves battery life of the device by enabling the user to access the needed functions more quickly and efficiently. Therefore, by shifting menu options from a primary display and to a touch-sensitive secondary display in order to make sure that content may be presented (without obstruction) in the full-screen mode, users are able to sustain interactions with the device and their workflow is not interrupted when shifting to the full-screen mode. Additionally, fewer interactions are required in order to access menu options while viewing full-screen content, as menu options that may have become buried behind content on the primary display is presented on the touch-sensitive secondary display for easy and quick access (and without having to exit full screen mode and then dig around looking for the menu options), thereby reducing power usage and improving battery life for the device.
In some embodiments, the computing system detects (816) a user touch input selecting one of the second set of affordances displayed on the touch screen display, and, in response to detecting the user touch input, the computing system changes the portion of the first user interface for the application being displayed in the full-screen mode on the primary display according to the selected one of the second set of affordances. In
In some embodiments, after displaying the portion of the first user interface for the application in the full-screen mode on the primary display, the computing system (818): detects a subsequent user input for exiting the full-screen mode; and, in response to detecting the subsequent user input: displays, on the primary display in the normal mode, the first user interface for the application executed by the computing system, the first user interface comprising the first set of one or more affordances associated with the application; and maintains display of at least a subset of the second set of one or more affordances for controlling the application on the touch screen display, where the second set of one or more affordances correspond to the first set of one or more affordances. In one example,
It should be understood that the particular order in which the operations in
In some embodiments, the computing system is portable computing system 100 (
The computing system displays (902), on the primary display, a first user interface for an application executed by the computing system.
The computing system displays (904), on the touch screen display, a second user interface, the second user interface comprising a set of one or more affordances corresponding to the application. Continuing with the example above,
In some embodiments, prior to detecting the notification, the computing system detects (906) a user input selecting a notification setting so as to display notifications on the touch screen display and to not display notifications on the primary display. In some embodiments, the user of the computing system is able to specify within a settings panel whether received notifications are to be displayed on one or more of primary display 102 and dynamic function row 104. In some embodiments, the computing system displays received notifications on dynamic function row 104 but not on primary display 102 by default.
The computing system detects (908) a notification. In some embodiments, the notification is associated with the application (e.g., the web browser application in
In response to detecting the notification, the computing system concurrently displays (910), in the second user interface, the set of one or more affordances corresponding to the application and at least a portion of the detected notification on the touch screen display, where the detected notification is not displayed on the primary display. For example, in some embodiments, based on the notification setting discussed in operation 906, at least a portion of the detected notification is overlaid on the second user interface displayed on dynamic function row 104. Additionally and/or alternatively, in some embodiments, the notification is overlaid on the first user interface displayed by primary display 102. As such, the user of the computing system is able to view and respond to notifications without shifting his/her hands away from the set of physical keys to another input device (e.g., a mouse) when such selectable controls are displayed on the primary display. This reduction in mode switching, for example, between keyboard and mouse for the user's hands and between keyboard and display for the user's eyes, provides for a more intuitive user experience and a more efficient human-machine interface.
For example,
In yet another example,
In a yet another example,
Displaying received notifications at a touch-sensitive secondary display allows users to continue their work on a primary display in an uninterrupted fashion, and allows them to interact with the received notifications via the touch-sensitive secondary display. Allowing users to continue their work on the primary display in an uninterrupted fashion and allowing users to interact with the received notifications via the touch-sensitive secondary display provides users with a quick and convenient way to review and interact with received notifications and thereby enhances the operability of the computing system and makes the user-device interface more efficient (e.g., by helping the user to conveniently access received notifications directly through the touch-sensitive secondary display and without having to interrupt their workflow to deal with a received notification). Furthermore, displaying receiving notifications at the touch-sensitive secondary display provides an emphasizing effect for received notifications at the touch-sensitive secondary display, as the received notification is, in some embodiments, displayed as overlaying other affordances in the touch-sensitive secondary display, thus ensuring that the received notification is visible and easily accessible at the touch-sensitive secondary display.
In some embodiments, the portion of the notification displayed on the touch screen display prompts (912) a user of the computing system to select one of a plurality of options for responding to the detected notification. For example, modal alert 5156 displayed by primary display 102 and dynamic function row 104, in
In some embodiments, the portion of the notification displayed on the touch screen display includes (914) one or more suggested responses to the detected notification.
In some embodiments, the notification corresponds (916) to an at least one of an incoming instant message, SMS, email, voice call, or video call. In
In some embodiments, the notification corresponds (918) to a modal alert issued by an application being executed by the processor of the computing system in response to a user input closing the application or performing an action within the application. In a first example,
In some embodiments, the set of one or more affordances includes (920) at least one a system-level affordance corresponding to at least one system-level functionality, and the notification corresponds to a user input selecting one or more portions of the input mechanism (e.g., alt+tab or another keyboard combination) or the least one of a system-level affordance (e.g., a power control affordance). In one example,
In some embodiments, the computing system detects (922) a user touch input on the touch screen display corresponding to the portion of the detected notification. In accordance with a determination that the user touch input corresponds to a first type (e.g., swipe to dismiss), the computing system ceases to display in the second user interface the portion of the detected notification on the touch screen display. In accordance with a determination that the user touch input corresponds to a second type (e.g., tap to perform an action) distinct from the first type, the computing system performs an action associated with the detected notification (e.g., open a dialogue for responding to the newly received email, SMS, or IM; save a document; exit a program; or the like). For example,
It should be understood that the particular order in which the operations in
In some embodiments, the computing system is portable computing system 100 (
The computing system displays (1002), on the primary display, a user interface, the user interface comprising one or more user interface elements. For example,
The computing system identifies (1004) an active user interface element of the one or more user interface elements that is in focus on the primary display, where the active user interface element is associated with an application executed by the computing system. In
In response to identifying the active user interface element that is in focus on the primary display, the computing system displays (1006), on the touch screen display, a set of one or more affordances corresponding to the application. In
The computing system detects (1008) a user input to move a respective portion of the user interface. In some embodiments, the portion of the user interface is a menu, toolbar, tool set, notification, or the like. For example, the computing system detects a gesture that drags a menu to the bottom (or other user defined location) of primary display 102 or an instruction to move the menu to dynamic function row 104 via a right-click action or other corresponding affordance. In one example,
In some embodiments, the respective portion of the user interface is (1010) a menu corresponding to the application executed by the computing system. For example, the respective portion of the user interface is a menu or a toolbar for a word processing application. For example, the respective portion of the respective user interface being drug by cursor 504, in
In some embodiments, the respective portion of the user interface is (1012) at least one of a notification or a modal alert. For example, the respective portion of the respective user interface being drug by cursor 504, in
Allowing a user to quickly move user interface portions (e.g., menus, notifications, etc.) from a primary display and to a touch-sensitive secondary display provides the user with a convenient and customized way to access the user interface portions. Providing the user with a convenient and customized way to access the user interface portions via the touch-sensitive secondary display enhances the operability of the computing system and makes the user-device interface more efficient (e.g., by helping the user to access user interface portions directly through the touch-sensitive secondary display with fewer interactions and without having to waste time looking for a previously viewed (and possibly buried) user interface portion) which, additionally, reduces power usage and improves battery life of the device by enabling the user to access needed user interface portions more quickly and efficiently. Furthermore, displaying user interface portions at the touch-sensitive secondary display in response to user input provides an emphasizing effect for the user interface portions at the touch-sensitive secondary display, as a respective user interface portions is, in some embodiments, displayed as overlaying other affordances in the touch-sensitive secondary display, thus ensuring that the respective user interface portion is visible and easily accessible at the touch-sensitive secondary display.
In response to detecting the user input, and in accordance with a determination that the user input satisfies predefined action criteria, the computing system (1014): ceases to display the respective portion of the user interface on the primary display; ceases to display at least a subset of the set of one or more affordances on the touch screen display; and displays, on the touch screen display, a representation of the respective portion of the user interface. In one example,
In some embodiments, the predefined action criteria are satisfied (1016) when the user input is a dragging gesture that drags the respective portion of the user interface to a predefined location of the primary display. In some embodiments, the predefined location is one of a plurality of predefined location set by the user of the computing system or set by default in software. In one example, in
In some embodiments, the predefined action criteria are satisfied when the user input is (1018) a predetermined input corresponding to moving the respective portion of the user interface to the touch screen display. For example, primary display 102 displays a window for a word processing application along with a formatting toolbar overlaid on the window for the word processing application. Continuing with this example, in response to selecting a specific display option after right-clicking on the formatting toolbar or selecting the specific display option while the formatting toolbar is in focus on primary display 102, primary display 102 ceases to display the formatting toolbar and dynamic function row displays the formatting toolbar.
In some embodiments, the representation of the respective portion of the user interface is overlaid (1020) on the set of one or more affordances on the touch screen display. For example,
In some embodiments, the set of one or more affordances includes (1022) at least one system-level affordance corresponding to at least one system-level functionality, and, after displaying the representation of the respective portion of the user interface on the touch screen display, the computing system maintains display of the at least one system-level affordance on the touch screen display. In
In some embodiments, in response to detecting the user input, and in accordance with a determination that the user input does not satisfy the predefined action criteria, the computing system (1024): maintains display of the respective portion of the user interface on the primary display; and maintains display of the set of one or more affordances on the touch screen display. For example, with respect to
It should be understood that the particular order in which the operations in
In some embodiments, the computing system is portable computing system 100 (
As described below, the method 1100 (and associated interfaces) provide(s) an intuitive way to browse through user interface objects on a primary display by providing inputs at a touch-sensitive secondary display. Method 1100 provides users with quick access to user interface controls at the touch-sensitive secondary display so that a user need not move their fingers from positions over keys on the physical input mechanism and can instead simply select controls on the secondary display without having to adjust finger positions to move to a trackpad and then move finger positions back to the physical input mechanism in order to continue working.
As shown in
In response to receiving the request, the computing system displays (1106), on the primary display, a plurality of user interface objects associated with an application executing on the computing system. For example the plurality of user interface objects correspond to tabs in a web browsing application, individual photos in a photo-browsing application, individual frames of a video in a video-editing application, and the like.
In some embodiments, the plurality of user interface objects includes a first user interface object displayed with its associated content and other user interface objects displayed without their associated content. For example, as shown in
In response to receiving the request, the computing system also displays (1108), on the touch-sensitive secondary display, a set of affordances that each represent (i.e., correspond to) one of the plurality of user interface objects. For example, a first affordance 5052-B corresponds to the first user interface object 5052-A,
In some embodiments, the first affordance in the set of affordances that corresponds to the first tab is displayed with a larger display size than other affordances in the set (e.g., the first affordance 5052-B is display with the larger display size relative to other affordances 5050-B and 5051-B). In some embodiments, the other affordances are also displayed with a lower brightness level relative to a brightness level of the first affordance 5052-B.
In some embodiments, before receiving the request to open the application, the touch-sensitive secondary display included a different set of affordances in an application-specific region of the touch-sensitive secondary display (such as affordances for accessing various folders in a Finder application, such as the affordances shown in touch-sensitive secondary display 104 in
The computing system, in accordance with method 1100, detects (1110), via the touch-sensitive secondary display, a swipe gesture (e.g., 5011-A,
In response to detecting the swipe gesture, the computing system updates the primary display (e.g., during the swipe gesture) to cease displaying associated content for the first user interface object and to display associated content for the second user interface object. For example, as shown in
In some embodiments, a tap over a respective affordance displayed on the touch-sensitive secondary display 104 may also be used to facilitate navigation through the plurality of user interface objects on the primary display (instead of or in addition to the swipe gesture). For example, each of the inputs 5011-A, 5011-B, and 5011-C may correspond to discrete inputs or may correspond to a continuous swipe gesture across the touch-sensitive secondary display 104.
In some embodiments, the computing system detects continuous travel of the swipe gesture across the touch-sensitive secondary display, including the swipe gesture contacting a third affordance that represents a third user interface object (as shown in
In some embodiments, each affordance in the set of affordance includes a representation of respective associated content for a respective user interface object of the plurality. For example, each of the affordances 5050-B, 5051-B, and 5052-B include a mini-view/shrunken view of content for a corresponding tab shown on the primary display 102 (e.g., as shown in
Allowing a user to quickly navigate through user interface objects on a primary display (e.g., browser tabs) by providing inputs at a touch-sensitive secondary display provides the user with a convenient way to quickly navigate through the user interface objects. Providing the user with a convenient way to quickly navigate through the user interface objects via the touch-sensitive secondary display (and reducing the number of inputs needed to navigate through the user interface objects, thus requiring fewer interactions to navigate through the user interface objects) enhances the operability of the computing system and makes the user-device interface more efficient (e.g., by requiring a single input or gesture at a touch-sensitive secondary display to navigate through user interface objects on a primary display) which, additionally, reduces power usage and improves battery life of the device by enabling the user to navigate through user interface objects on the primary display more quickly and efficiently. Moreover, as users provide an input at the touch-sensitive display (e.g., a swipe gesture) to navigate through the user interface objects on the primary display, each contacted affordance at the touch-sensitive display (that corresponds to one of the user interface objects) is visually distinguished from other affordances (e.g., a respective contacted affordance is magnified and a border may be highlighted), thus making information displayed on the touch-sensitive secondary display more discernable to the user.
In some embodiments, the computing system, before detecting the swipe gesture (or before detecting movement of the swipe gesture, detects an initial contact with the touch-sensitive secondary display over the first affordance. In response to detecting the initial contact, the computing system increases a magnification level (or display size) of the first affordance. In some embodiments, in accordance with a determination that a user has provided an input at a respective affordance that corresponds to a respective user interface object other than a user interface object that has focus on the primary display 102, the computing device increases a magnification level of the respective affordance.
In some embodiments, the application is a web browsing application, and the plurality of user interface objects each correspond to web-browsing tabs.
In some embodiments, the computing system detects an input at a URL-input portion of the web browsing application on the primary display. In response to detecting the input, the computing system updates the touch-sensitive secondary display to include representations of favorite URLs. An example is shown in
In some embodiments, the application is a photo-browsing application, and the plurality of user interface objects each correspond to individual photos (as shown in
In some embodiments, the application is a video-editing (or video-playback) application, and the plurality of user interface object each correspond to individual frames in a respective video. An example is shown in
It should be understood that the particular order in which the operations in
In some embodiments, the computing system is portable computing system 100 (
As described below, the method 1200 (and associated interfaces) provide(s) an intuitive way to browse through and locate search results on a primary display by providing inputs at a touch-sensitive secondary display. Method 1200 provides users with quick access to user interface controls at the touch-sensitive secondary display for navigating between search results so that a user need not move their fingers from positions over keys on the physical input mechanism and can instead simply select controls on the secondary display without having to adjust finger positions to move to a trackpad and then move finger positions back to the physical input mechanism in order to continue working.
In accordance with the method 1200, the computing system receives (1204) a request to search within content displayed on the primary display of the computing device. For example, the request corresponds to a search for text on a webpage, as shown in
In response to receiving the request, the computing system displays (1206), on the primary display, a plurality of search results responsive to the search. In some embodiments, the request to search within the content is a request to locate a search string within the content, and the plurality of search results each include at least the search string (e.g., the search string is “the,” as shown in
In some embodiments, focus (of the primary display 102) is on a first search result of the plurality of search results (e.g., as shown in
In response to receiving the request, the computing system also displays (1208), on the touch-sensitive secondary display, respective representations that each correspond to a respective search result of the plurality of search results. For example, the representations are tick marks that each correspond to respective search results of the search results (as shown in
In accordance with method 1200, the computing system detects (1210), via the touch-sensitive secondary display, a touch input (e.g., a tap or a swipe) that selects a representation of the respective representations, the representation corresponding to a second search result of the plurality of search results distinct from the first search result. For example, as shown in
In response to detecting the input, the computing system changes focus (1212) on the primary display to the second search result. For example, as shown in
Allowing a user to quickly navigate through search results on a primary display by providing inputs at a touch-sensitive secondary display provides the user with a convenient way to quickly navigate through the search results. Providing the user with a convenient way to quickly navigate through the search results via the touch-sensitive secondary display (and reducing the number of inputs needed to navigate through the search results, thus requiring fewer interactions from a user to browse through numerous search results quickly) enhances the operability of the computing system and makes the user-device interface more efficient (e.g., by requiring a single input or gesture at a touch-sensitive secondary display to navigate through numerous search results on a primary display) which, additionally, reduces power usage and improves battery life of the device by enabling the user to navigate through search results on the primary display more quickly and efficiently. Moreover, as users provide an input at the touch-sensitive display (e.g., a swipe gesture) to navigate through the search on the primary display, each contacted affordance at the touch-sensitive display (that corresponds to one of the search results) is visually distinguished from other affordances (e.g., a respective contacted affordance is magnified and a border may be highlighted), thus making information displayed on the touch-sensitive secondary display more discernable to the user.
In some embodiments, changing focus includes modifying, on the primary display, a visual characteristic of (e.g., a visual characteristic that is used to render) the second search result (e.g., displaying the second search result with a larger font size or displaying the second search result with a different highlight color).
In some embodiments, the computing system detects a gesture that moves across at least two of the respective representations on the touch-sensitive secondary display. In response to detecting the gesture, the computing system changes focus on the primary display to respective search results that correspond to the at least two of the respective representations as the swipe gestures moves across the at least two of the respective representations (e.g., as the contact moves across each of the respective representations, the primary display is updated to show an appropriate search result, allowing quick and easy navigation through all of the search results with a single swipe gesture).
In some embodiments, the computing system, in accordance with a determination that a speed of the gesture is above a threshold speed, the computing system changes focus on the primary display to respective search results in addition to those that correspond to the at least two of the respective representations. For example, if the gesture travels above the threshold speed, then the computing system causes the primary display 102 to cycle through more search results in addition to those contacted during the gesture. In some embodiments, the gesture is a swipe gesture or a flick gesture.
In some embodiments (and as shown in
It should be understood that the particular order in which the operations in
In some embodiments, the computing system is portable computing system 100 (
As described below, the method 1300 (and associated interfaces) provide(s) an intuitive way to modify details for an event that is displayed on a primary display by providing inputs at a touch-sensitive secondary display. Method 1300 provides users with quick access to user interface controls (for modifying event details) at the touch-sensitive secondary display so that a user need not move their fingers from positions over keys on the physical input mechanism and can instead simply select controls on the secondary display without having to adjust finger positions to move to a trackpad (e.g., to waste time navigating through complicated menu hierarchies) and then move finger positions back to the physical input mechanism in order to continue working.
In accordance with method 1300, the computing system displays (1304), on the primary display, a calendar application (e.g., various user interfaces for the calendar application are shown in
The computing system receives (1306) a request to display information about an event that is associated with the calendar application. In response to receiving the request, the computing system displays (1308), on the primary display, event details for the first event, the event details including a start time and an end time for the event. For example, the request corresponds to a selection of an event that is displayed within the calendar application on the primary display (as shown in
In response to receiving the request, the computing system displays (1310), on the touch-sensitive secondary display, an affordance, the affordance indicating a range of time that at least includes the start time and the end time (as shown in
In some embodiments, the computing system detects, via the touch-sensitive secondary display, an input at the affordance that modifies the range of time. In some embodiments, the input that modifies the range of time is a press input that remains in contact with the affordance for more than a threshold amount of time and then moves at least a portion the affordance across the touch-sensitive secondary display (e.g., this press input may also be referred to as a drag gesture that moves the end time for the event to 3 PM, as shown in
In response to detecting the input, the computing system: (i) modifies at least one of the start time and the end time for the event in accordance with the input; and (ii) displays, on the primary display, a modified range of time for the event in accordance with the input (e.g., as shown in
In some embodiments, the input that modifies the range of time is a swipe gesture that moves across the touch-sensitive secondary display and causes the computing system to select a new start time and a new end time for the event. In some embodiments, the new start and end times correspond to a time slot that is of a same duration covered by the start and end times. For example, the computing device causes the touch-sensitive secondary display 104 to jump to a next available block of time that is the same duration as the prior start and end times, e.g., if existing start/end time are 1-2 PM, then a swipe might cause the new start and end time to be selected as 2-3 PM.
In some embodiments, the computing system saves the event with the modified start and/or end time to the memory of the computing system.
Allowing a user to quickly and easily edit event details at a touch-sensitive secondary display provides the user with a convenient way to quickly edit event details without having to perform extra inputs (e.g., having to jump back and forth between using a keyboard and using a trackpad to modify the event details). Providing the user with a convenient way to quickly edit event details via the touch-sensitive secondary display (and reducing the number of inputs needed to edit the event details, thus requiring fewer interactions to achieve a desired result of editing event details) enhances the operability of the computing system and makes the user-device interface more efficient (e.g., by requiring a single input or gesture at a touch-sensitive secondary display to quickly edit certain event details) which, additionally, reduces power usage and improves battery life of the device by enabling the user to edit event details more quickly and efficiently. Additionally, by updating the primary display in response to inputs at the touch-sensitive secondary display (e.g., to show updated start and end times for an event), a user is able to sustain interactions with the device in an efficient way by providing inputs to modify the event and then immediately seeing those modifications reflected on the primary display, so that the user is then able to decide whether to provide an additional input or not.
It should be understood that the particular order in which the operations in
In some embodiments, the computing system is portable computing system 100 (
As described below, the method 1400 (and associated interfaces) provide(s) an intuitive way to present actionable information at a touch-sensitive secondary display about external devices that are connected with a computing system that includes the touch-sensitive secondary display. Method 1400 provides users with quick access to user interface controls (for controlling external devices) at the touch-sensitive secondary display so that a user need not move their fingers from positions over keys on the physical input mechanism and can instead simply select controls on the secondary display without having to adjust finger positions to move to a trackpad (e.g., to waste time navigating through complicated menu hierarchies to located needed functions for controlling external devices) and then move finger positions back to the physical input mechanism in order to continue working.
In accordance with the method 1400, the computing system detects (1404) a new connection between the computing system and an external device distinct from the computing system (e.g., an additional monitor is connected using a physical cable, headphones are connected via Bluetooth or via headphone jack, etc.). In response to detecting the new connection, the computing system displays (1406), on the touch-sensitive secondary display, a plurality of affordances corresponding to functions available via the external device.
In this way, users are able to quickly decide what to do with newly connected external devices by simply selecting an option from the touch-sensitive secondary display. Therefore, users do not need to interrupt their current workflow in order to decide what to do with new external devices (e.g., by having to navigate to a menu and then select an option for the new external device or by having to reposition their hands in order to interact with a trackpad).
In some embodiments, the computing system receives, via the touch-sensitive secondary display, a selection of a first affordance that corresponds to a first function available via the external device. In response to receiving the selection, the computing device initiates performance of the first function (e.g., the computing device begins outputting audio to the headphones or the computing device begins displaying mirroring using the external monitor).
In some embodiments, the external device is an additional display, distinct from the primary display and the touch-sensitive secondary display. In some embodiments, the plurality of affordances include a first affordance that, when selected, causes the computing system to initiate performance of a display mirroring function via the additional display.
Allowing a user to efficiently utilize external devices via a touch-sensitive secondary display provides the user with a convenient way to access functions that may otherwise be buried in menus. Providing the user with a convenient way to access functions for external devices that may otherwise be buried in menus (and, therefore, reducing the number of inputs needed to access the functions, thus requiring fewer interactions in order to use external devices) enhances the operability of the computing system and makes the user-device interface more efficient (e.g., by requiring a single input or gesture at a touch-sensitive secondary display to perform a certain function for an external device) which, additionally, reduces power usage and improves battery life of the device by enabling the user to interact with external devices more quickly and efficiently. In this way, the touch-sensitive secondary display also conveys information about an internal state of the device (by reflecting a connecting status between the device and the external device, and allowing users to easily interact with the external device).
It should be understood that the particular order in which the operations in
In some embodiments, the computing system is portable computing system 100 (
As described below, the method 1500 (and associated interfaces) provide(s) an intuitive way to preview characters that are displayed within an application on a primary display by providing inputs at a touch-sensitive secondary display. Method 1500 provides users with quick access to user interface controls (for easily previewing characters) at the touch-sensitive secondary display so that a user need not move their fingers from positions over keys on the physical input mechanism and can instead simply select controls on the secondary display without having to adjust finger positions to move to a trackpad (e.g., to waste time navigating through complicated menu hierarchies to locate desired characters) and then move finger positions back to the physical input mechanism in order to continue working.
In accordance with the method 1500, the computing system displays (1504), on the primary display, a user interface for an application that is executing on the computing system (e.g., the user interface is for a messaging application, such as that shown in
In response to detecting the first input, the computing system displays (1506), on the touch-sensitive secondary display, a set of affordances that each correspond to distinct characters (e.g., the set of affordances each correspond to individual emojis, as shown in
In some embodiments, the first input may also be provided at the touch-sensitive secondary display in order to activate display of the set of affordances in the touch-sensitive secondary display (e.g., the first input corresponds to input 5046-B,
In some embodiments, displaying the user interface for the application includes updating the touch-sensitive secondary display to include the set of affordances. For example, if the application opens and allows users to begin composing or editing a document, then no additional input is needed to cause the touch-sensitive second display to include the set of affordances.
In some embodiments, the computing system determines affordances to include in the set of affordances based at least in part on textual content included in the user interface (e.g., emojis displayed in the set of affordances change based on what has already been typed, previous emoji selected when particular words have already been typed, etc.). In some embodiments, the determining is conducted in response to detecting that a user has modified textual content included in the user interface (e.g., as user types or removes text, suggested emojis displayed in the touch-sensitive secondary display may change).
In some embodiments, the computing system detects, via the touch-sensitive secondary display, a second input (e.g., input 5047,
In some embodiments, the computing system also detects, via the touch-sensitive secondary display, movement of the second input from the first affordance and to a second affordance that corresponds to a second character of the distinct characters (e.g., the input 5047 travels to a position on the touch-sensitive secondary display that corresponds to input 5048,
In some embodiments, the preview of the second character remains displayed on the primary display while the second input remains in contact with the second affordance.
In some embodiments, the computing system detects liftoff of the second input while it is contact with second affordance. In response to detecting liftoff, the computing system updates the user interface to include a permanent display of (i.e., not a preview of) the second user interface element.
In some embodiments, the computing system detects an additional input (e.g., pressing down harder during the second input at the second affordance) while second input is in contact with second affordance. In response to detecting the additional input, the computing system updates the user interface to include the permanent display of (i.e., not a preview of) the second user interface element.
Allowing a user to quickly and easily preview how characters will look within an application on a primary display by providing an intuitive input at a touch-sensitive secondary display provides the user with a convenient way to quickly preview how characters will look within the application. Providing the user with a convenient way to quickly preview how characters will look within an application enhances the operability of the computing system and makes the user-device interface more efficient (e.g., by requiring a single input or gesture at a touch-sensitive secondary display to quickly preview how characters will look, thus fewer interactions are required to preview how these characters will look on the primary display) which, additionally, reduces power usage and improves battery life of the device by enabling the user to preview characters more quickly and efficiently. In this way, users are provided with efficient and sustained interactions with their devices, as the users are permitted to continue previewing characters and then continue providing inputs until a desired character is located.
In some embodiments, the functionality described above for emoji previews via swipe gestures in the touch-sensitive secondary display is also utilized to perform other modifications to text displayed within a user interface for an application. For example, the touch-sensitive secondary display may display a color picker (e.g., the row of color options shown in
It should be understood that the particular order in which the operations in
In some embodiments, the computing system is portable computing system 100 (
As described below, the method 1600 (and associated interfaces) provide(s) an intuitive way to modify visual characteristics that are used to render content within a content-editing application on a primary display by providing inputs at a touch-sensitive secondary display. Method 1600 provides users with quick access to user interface controls (for easily modifying visual characteristics used to render content on a primary display) at the touch-sensitive secondary display so that a user need not move their fingers from positions over keys on the physical input mechanism and can instead simply select controls on the secondary display without having to adjust finger positions to move to a trackpad (e.g., to waste time navigating through complicated menu hierarchies to locate desired functions that allow for modifying visual characteristics) and then move finger positions back to the physical input mechanism in order to continue working.
In accordance with method 1600, the computing system receives (1604) a request to open a content-editing application (e.g., an application for composing and editing documents, drawings, photos, etc., such as the drawing application shown on
In response to receiving the request, the computing system also displays (1608), on the touch-sensitive secondary display, a user interface control for modifying at least one visual characteristic that is used to render content within the content-editing application (e.g., a color picker that includes a sliding scale of color values used to select colors for content displayed within the content-editing application, such as the color picker 5505 shown in
In some embodiments, the computing system detects, via the touch-sensitive secondary display 104, an input at the user interface control that selects a first value for the at least one visual characteristic (e.g., the input 5010-C selects a shade of pink). After detecting the input, the computing system renders content in the content-editing application using the first value for the at least one visual characteristic (e.g., all new content added to the content-editing application is rendered using the first value and/or currently selected content is rendered using the first value). For example, as shown in
In some embodiments, before rendering the content, the computing system receives a selection of the content (e.g., an input that selects the stick figure's head, such as input receiving via cursor 504,
In some embodiments, the sliding scale of values represents distinct shades of color. In some embodiments, the first value corresponds to a first shade of a first color and the method 1600 further includes: in accordance with a determination that the input satisfies predetermined criteria (remains in contact for more than threshold amount of time or satisfies an intensity-based threshold), modifying the user interface control on the touch-sensitive secondary display to include options for selecting other shades of the first color, distinct from the first shade of the first color. An example is shown on
In some embodiments, the preview is presented while the input remains in contact with the touch-sensitive secondary display. In some embodiments, the computing system, in response to detecting liftoff of the input, ceases to display the preview and instead displays the selected content with the first value for the at least one visual characteristic (e.g., the stick figure's head is rendered in a persistent fashion, not just as a preview that lasts while the input remains in contact with the touch-sensitive secondary display 104).
In some embodiments, the computing system, in response to detecting liftoff of the input, displays the portion of the editable content with the modified value for the at least one visual characteristic.
Therefore, users are provided with an intuitive way to modify visual characteristics that are used to render content within a content-editing application on a primary display by providing inputs at a touch-sensitive secondary display. Providing users with an intuitive way to modify visual characteristics in this way enhances the operability of the computing system and makes the user-device interface more efficient (e.g., by requiring a single input or gesture at a touch-sensitive secondary display to quickly preview how certain visual characteristics will look when used to render content on the primary display) which, additionally, reduces power usage and improves battery life of the device by enabling the user to preview changes to visual characteristics in a quicker and more efficient way. In this way, users are provided with efficient and sustained interactions with their devices, as the users are permitted to continue previewing how modifications to a visual characteristic will look on the primary display and then continue providing inputs until a desired modification for the visual characteristic is located.
It should be understood that the particular order in which the operations in
In some embodiments, the computing system is portable computing system 100 (
As described below, the method 2800 (and associated interfaces) provide(s) an intuitive way to use a biometric sensor to enable efficient logins (logins that require fewer inputs, in some instances only a single input is needed to complete a login). Method 2800 provides users with the ability to login to their devices quickly so that a user need not always type their password or have to navigate to a user-specific login page (instead the user simply provides biometric information and, based on that biometric information, the device identifies the user and allows for fast logins).
In accordance with method 2800, the electronic device, while the device is in a locked state (e.g., the locked state is a state in which one or more features of the device are disabled and access to sensitive information or the ability to change or delete information is prohibited), displays (2804) a respective log-in user interface that is associated with logging in to a plurality of user accounts including a first user account and a second user account. An example log-in user interface is shown on the primary display 102 of
In some embodiments, the log-in user interface includes instructions to provide biometric information (e.g., as shown in
While displaying the log-in user interface, the electronic device receives (2806) biometric information about a user. For example, a user provides a fingerprint at a predefined area of the electronic device (e.g., at a biometric sensor that is integrated with the electronic device, such as a biometric sensor located above a physical keyboard and adjacent to a secondary display). In some embodiments, the biometric sensor is integrated with the secondary display.
In response to receiving the biometric information, the electronic device determines whether the biometric information is consistent with biometric information for the first user account or the second user account of the plurality of user accounts while the first and second user accounts to not have active sessions on the device (e.g., the device has just booted up or the users have not yet logged in to the device). For example, as shown in
In accordance with a determination that the biometric information is consistent with biometric information for the first user account of the plurality of user accounts while the first user account does not have an active session on the device, the device displays (2808), on the display, a prompt to input a log-in credential for the first user account. For example, the prompt includes the text “Touch ID disabled for initial login, please enter your password” (as shown in
In accordance with a determination that the biometric information is consistent with biometric information for the second user account of the plurality of user accounts while the second user account does not have an active session on the device, the device displays (2810), on the display, a prompt to input a log-in credential for the second user account. For example, the prompt is similar to the prompt shown in
In this way, users are provided with an intuitive way to access a user-specific login page by providing a single input at a biometric sensor. Providing users with an intuitive way to access a user-specific login page in this way enhances the operability of the computing system and makes the user-device interface more efficient (e.g., by requiring a single input or gesture at the biometric sensor to quickly access an appropriate, user-specific login page, thus fewer interactions are required to reach a user-specific login page) which, additionally, reduces power usage and improves battery life of the device by enabling the user to access the login page via single input.
In some embodiments, the electronic device, in response to receiving the biometric information: in accordance with a determination that the biometric information is consistent with biometric information for the first user account of the plurality of user accounts while the first user account has an active session on the device, the device unlocks with respect to the first user account (e.g., without requiring additional user input, such as without requiring the first user account to enter a password). For example, the first user account corresponds to the user “Josh Olson” shown in
In some embodiments, in response to receiving the biometric information: in accordance with a determination that the biometric information is consistent with biometric information for the second user account of the plurality of user accounts while the second user account has an active session on the device, the device unlocks with respect to the second user account (e.g., without requiring additional user input). For example, one of the other users, such as “Sally” has already logged in and has an active session on the device, and thus the device unlocks immediately instead of requiring additional input.
In some embodiments, in response to receiving the biometric information: in accordance with a determination that the biometric information is not consistent with biometric information for the any user account of the device, maintaining the device in the locked state. For example, if some other user who does not have a user account on the device attempts to provide biometric information (such as a fingerprint), the device stays in the locked state (e.g., the device continues to display the log-in user interface and may display a message indicating the provided biometric information is not recognized).
In some embodiments, the log-in user interface includes a plurality of selectable affordances that correspond to the plurality of user accounts (as shown in
In some embodiments, the device, while displaying the prompt to input a log-in credential for the first user account (e.g., the prompt shown in
In accordance with a determination that the log-in credential is consistent with a log-in credential for the first user account, the device is unlocked with respect to the first user account. In accordance with a determination that the log-in credential is not consistent with a log-in credential for the first user account, the device remains in the locked state.
In some embodiments, while displaying the prompt to input a log-in credential for the second user account, the device receives entry of a log-in credential (e.g., a password entered by the second user at the “Enter Password” input box of
In accordance with a determination that the log-in credential is consistent with a log-in credential for the second user account, the device is unlocked with respect to the first user account; and in accordance with a determination that the log-in credential is not consistent with a log-in credential for the second user account, the device remains in the locked state.
It should be understood that the particular order in which the operations in
In some embodiments, the computing system is portable computing system 100 (
As described below, the method 2900 (and associated interfaces) provide(s) an intuitive way to use a biometric sensor to enable efficient fast switching between logged in user accounts. Method 2900 provides users with controls and guidance that allows the users to switch into their user accounts by simply providing biometric information (and without having to provide a password or navigate to a user interface for switching users).
In accordance with method 2900, the electronic device, while the device is logged in to a first user account, displays (2904) a user interface that is associated with the first user account (e.g., a home screen or desktop for the first user account). The device is associated with a plurality of user accounts including the first user account and a second user account, and the second user account is associated with biometric information that enables logging in to the second user account.
While displaying the user interface that is associated with the first user account, the device receives (2906) an input via the input element with the integrated biometric sensor. For example, a second user provides the input at the integrated biometric sensor while the first user account is currently logged in.
In response to receiving the input via the input element with the integrated biometric sensor, the device determines whether the input meets second-user switching criteria while the second user account has an active session on the device. In some embodiments, the second-user switching criteria include a requirement that biometric information detected during the input with the input element is consistent with biometric information for the second user account of the plurality of user accounts.
In accordance with a determination that the input meets the second-user switching criteria while the second user account has an active session on the device, the device (2908): (i) unlocks the device with respect to the second user account; (ii) locks the device with respect to the first user account (e.g., logs off the first user account); and (ii) replaces display of the user interface associated with the first account with a user interface associated with the second user account. In this way, the second user is able to gain access to active session on the device by simply providing a single biometric input.
Therefore, users are provided with an intuitive way to access an active, user-specific session on an electronic device by providing a single input at a biometric sensor. Providing users with an intuitive way to access an active, user-specific session in this way enhances the operability of the electronic device and makes the user-device interface more efficient (e.g., by requiring a single input or gesture at the biometric sensor to gain immediate access to the active session, therefore requiring fewer interactions to switch user accounts and login to the device) which, additionally, reduces power usage and improves battery life of the device by enabling the user to access the active session via single input.
In some embodiments, in response to receiving the biometric information: in accordance with a determination that the input meets second-user switching criteria while the second user account does not have an active session on the device, the second-user switching criteria including a requirement that biometric information detected during the input with the input element is consistent with biometric information for the second user account of the plurality of user accounts, the device displays, on the display, a prompt to input a log-in credential for the second user account (e.g., an example prompt is shown on
In some embodiments, in response to receiving the biometric information: in accordance with a determination that the input meets third-user switching criteria while a third user account has an active session on the device, the third-user switching criteria including a requirement that biometric information detected during the input with the input element is consistent with biometric information for the third user account of the plurality of user accounts, the device: (i) unlocks the device with respect to the third user account; (ii) locks the device with respect to the first user account; and (iii) replaces display of the user interface associated with the first account with a user interface associated with the third user account. In this way, the third user is able to gain access to active session on the device by simply providing a single biometric input.
In some embodiments, in response to receiving the biometric information: in accordance with a determination that the input meets third-user switching criteria while the third user account does not have an active session on the device, the third-user switching criteria including a requirement that biometric information detected during the input with the input element is consistent with biometric information for the third user account of the plurality of user accounts, displaying, on the display, a prompt to input a log-in credential for the third user account (e.g., an example prompt is shown on
In some embodiments, the first user is able to quickly and easily log back in to the device using by providing a biometric input. For example, after replacing display of the user interface associated with the first account with a user interface associated with the second user account (or the third user account) and while displaying the user interface that is associated with the second user account (or the third user account), the device receives a second input via the input element with the integrated biometric sensor. In response to receiving the second input via the input element with the integrated biometric sensor: in accordance with a determination that the second input meets first-user switching criteria while the first user account has an active session on the device, the first-user switching criteria including a requirement that biometric information detected during the input with the input element is consistent with biometric information for the first user account of the plurality of user accounts, the device: (i) unlocks the device with respect to the first user account; (ii) locks the device with respect to the second user account; and (iii) replaces display of the user interface associated with the second account with a user interface associated with the first user account. In this way, the first user is able to provide a single biometric input in order to immediately resume use of the electronic device (and without have to enter a password or provide any other inputs to resume user of the device).
In some embodiments, the input element is a button (e.g., a button that is included on the touch-sensitive secondary display 104 or that is located adjacent to the secondary display 104) and detecting the input via the input element with the integrated biometric sensor includes detecting a press input via the button. For example, the input is a push input of a mechanical button, or a press input with an intensity/pressure above a predefined intensity/pressure threshold. In some embodiments, if biometric information is detected without detecting the press input, the device forgoes switching users, even if the received biometric information is consistent with a user account that is not the currently logged in user account.
In some embodiments, the second-user switching criteria include a criterion that the press input lasts for less than a first threshold amount of time.
In some embodiments, in response to receiving the press input via the input element with the integrated biometric sensor in accordance with a determination that the press input with the button lasts longer than the first threshold amount of time, the device enters into a low power mode (e.g., the lower power mode corresponds to a suspended state in which the display is turned off).
In some embodiments, in response to receiving the input via the input element with the integrated biometric sensor: in accordance with a determination that the press input with the button lasts longer than the first threshold amount of time and less than a second threshold amount of time, the device enters into a low power mode. (e.g., the low power mode corresponds to a sleep/suspended state in which the display is turned off and the low power mode is entered upon an end of the press input). In accordance with a determination that the press input with the button lasts longer than the second threshold amount of time, the device displays a menu of options for changing a state of the device (e.g., the menu options include shut down, restart, sleep/suspend options that, when selected cause the device to shut down, restart, or sleep/suspend the device, respectively).
In some embodiments, in response to receiving the input via the input element with the integrated biometric sensor: in accordance with a determination that the press input with the button lasts longer than a third threshold amount of time that is greater than the second threshold amount of time, the device is restarted.
In some embodiments, when the user who is not currently signed in provides biometric information (e.g., places their finger on a biometric sensor of the device, such as a touch ID button that may be integrated with the touch-sensitive secondary display 104), the user is identified by the device based on stored biometric information, if any is available, for that user (e.g., stored fingerprint information previously received via the biometric or Touch ID sensor). In some embodiments, the user is identified before the user presses the button). In accordance with a determination that the biometric information provided by the user is valid (e.g., that it matches stored biometric information for that user), then the touch-sensitive secondary display 104 is updated to include a prompt instructing the user to “click to switch to [User Name]→”. In some embodiments, the arrow in this prompt points the user towards a location of the device where the biometric sensor is located.
In some embodiments, the prompt includes text that is specific to the user (e.g., the [User Name] text is replaced with a user name for the user). For example, the prompt may include the text “click to switch to Sally→” if Sally's fingerprint is detected, and “click to switch to Johnny→” if Johnny's fingerprint is detected.
In some embodiments, if the biometric information provided by the user is associated with a currently signed in account or if the biometric information is not recognized (i.e., is not associated with a user account of the device), then no prompt is displayed on the touch-sensitive secondary display 104.
It should be understood that the particular order in which the operations in
In accordance with some embodiments,
As shown in
The processing unit 1710 is configured to: cause display of (e.g., with the primary display control unit 1712) a first user interface on the primary display unit 1702, the first user interface comprising one or more user interface elements; identify (e.g., with the focus identifying unit 1716) an active user interface element among the one or more user interface elements that is in focus on the primary display unit 1702; and determine (e.g., with the determining unit 1718) whether the active user interface element that is in focus on the primary display unit 1702 is associated with an application executed by the processing unit 1710. In accordance with a determination that the active user interface element that is in focus on the primary display unit 1702 is associated with the application, processing unit 1710 is configured to cause display of (e.g., with the touch screen display control unit 1714) a second user interface on the touch screen display unit 1706, including: (A) a first set of one or more affordances corresponding to the application; and (B) at least one system-level affordance corresponding to at least one system-level functionality.
In some embodiments, the electronic device further includes: (i) a primary computing unit comprising the primary display unit 1702, the processing unit 1710, and a first communication circuitry unit; and (ii) an integrated input unit comprising the touch screen display unit 1706, the touch-sensitive surface unit 1708, the physical input unit 1704, and a second communication circuitry unit for communicating with the first communication circuitry unit, where the integrated input device is distinct and separate from the primary computing unit.
In some embodiments, the physical input unit 1704 comprises a plurality of physical keys.
In some embodiments, the physical input unit 1704 comprises a touchpad.
In some embodiments, the processing unit 1710 is configured to execute the application in the foreground of the first user interface.
In some embodiments, the least one system-level affordance is configured upon selection to cause display of a plurality of system-level affordances corresponding to system-level functionalities on the touch screen display unit 1706.
In some embodiments, the least one system-level affordance corresponds to one of a power control or escape control.
In some embodiments, at least one of the affordances displayed on the touch screen display unit 1706 within the second user interface is a multi-function affordance.
In some embodiments, the processing unit 1710 is configured to: detect (e.g., with the input detecting unit 1720) a user touch input selecting the multi-function affordance on the touch-sensitive surface unit 1708; in accordance with a determination (e.g., with the input type determining unit 1722) that the user touch input corresponds to a first type, perform (e.g., with the performing unit 1724) a first function associated with the multi-function affordance; and, in accordance with a determination (e.g., with the input type determining unit 1722) that the user touch input corresponds to a second type distinct from the first type, perform (e.g., with the performing unit 1724) a second function associated with the multi-function affordance.
In some embodiments, in accordance with a determination that the active user interface element is not associated with the application, the processing unit 1710 is configured to cause display of (e.g., with the touch screen display control unit 1714) a third user interface on the touch screen display unit 1706, including: (C) a second set of one or more affordances corresponding to operating system controls of the electronic device 1700, where the second set of one or more affordances are distinct from the first set of one or more affordances.
In some embodiments, the second set of one or more affordances is an expanded set of operating system controls that includes (B) the at least one system-level affordance corresponding to the at least one system-level functionality.
In some embodiments, the processing unit 1710 is configured to: detect (e.g., with the input detecting unit 1720) a user touch input selecting one of the first set of affordances on the touch-sensitive surface unit 1708; and, in response to detecting the user touch input: cause display of (e.g., with the touch screen display control unit 1714) a different set of affordances corresponding to functionalities of the application on the touch screen display unit 1706; and maintain display of (e.g., with the touch screen display control unit 1714) the at least one system-level affordance on the touch screen display unit 1706.
In some embodiments, the processing unit 1710 is configured to: detect (e.g., with the input detecting unit 1720) a subsequent user touch input selecting the at least one system-level affordance on the touch-sensitive surface unit 1708; and, in response to detecting the subsequent user touch input, cause display of (e.g., with the touch screen display control unit 1714) a plurality of system-level affordances corresponding to system-level functionalities and at least one application-level affordance corresponding to the application on the touch screen display unit 1706.
In some embodiments, the processing unit 1710 is configured to: identify (e.g., with the focus identifying unit 1716) a second active user interface element that is in focus on the primary display unit 1702 after displaying the second user interface on the touch screen display unit 1706; and determine (e.g., with the determining unit 1718) whether the second active user interface element corresponds to a different application executed by the processing unit 1710. In accordance with a determination that the second active user interface element corresponds to the different application, the processing unit 1710 is configured to cause display of (e.g., with the touch screen display control unit 1714) a fourth user interface on the touch screen display unit 1706, including: (D) a third set of one or more affordances corresponding to the different application; and (E) the at least one system-level affordance corresponding to the at least one system-level functionality.
In some embodiments, the processing unit 1710 is configured to: determine (e.g., with the media playback determining unit 1726) whether a media item is being played by the electronic device 1700 after identifying (e.g., with the focus identifying unit 1716) that the second active user interface element, where the media item is not associated with the different application; and, in accordance with a determination (e.g., with the media playback determining unit 1726) that media item is being played by the electronic device 1700, cause display of (e.g., with the touch screen display control unit 1714) at least one persistent affordance on the fourth user interface for controlling the media item on the touch screen display unit 1706.
In some embodiments, the at least one persistent affordance displays feedback that corresponds to the media item.
In some embodiments, the processing unit 1710 is configured to: detect (e.g., with the input detecting unit 1720) a user input corresponding to an override key; and, in response to detecting the user input: cease to display (e.g., with the touch screen display control unit 1714) at least the first set of one or more affordances of the second user interface on the touch screen display unit 1706; and cause display of (e.g., with the touch screen display control unit 1714) a first set of default function keys on the touch screen display unit 1706.
In some embodiments, the processing unit 1710 is configured to: detect (e.g., with the input detecting unit 1720) a gesture on the touch-sensitive surface unit 1708 in a direction that is substantially parallel to a major axis of the touch screen display unit 1706 after displaying the first set of default function keys on the touch screen display unit 1706; and, in response to detecting the substantially horizontal swipe gesture, cause display of (e.g., with the touch screen display control unit 1714) a second set of default function keys with at least one distinct function key on the touch screen display unit 1706.
The operations in the information processing methods described above are, optionally implemented by running one or more functional modules in information processing apparatus such as general purpose processors (e.g., as described above with respect to
The operations described above with reference to
In accordance with some embodiments,
As shown in
The processing unit 1810 is configured to: cause display of (e.g., with the primary display control unit 1812) a first user interface for an application executed by the processing unit 1810 on the primary display unit 1802; cause display of (e.g., with the touch screen display control unit 1814) a second user interface on the touch screen display unit 1806, the second user interface comprising a first set of one or more affordances corresponding to the application, where the first set of one or more affordances corresponds to a first portion of the application; and detect (e.g., with the input detecting unit 1816) a swipe gesture on the touch-sensitive surface unit 1808. In accordance with a determination (e.g., with the input type determining unit 1818) that the swipe gesture was performed in a first direction (e.g., horizontal), the processing unit 1810 is configured to cause display of (e.g., with the touch screen display control unit 1814) a second set of one or more affordances corresponding to the application on the touch screen display unit 1806, where at least one affordance in the second set of one or more affordances is distinct from the first set of one or more affordances, and where the second set of one or more affordances also corresponds to the first portion of the application. In accordance with a determination (e.g., with the input type determining unit 1818) that the swipe gesture was performed in a second direction substantially perpendicular to the first direction (e.g., vertical), the processing unit 1810 is configured to cause display of (e.g., with the touch screen display control unit 1814) a third set of one or more affordances corresponding to the application on the touch screen display unit 1806, where the third set of one or more affordances is distinct from the second set of one or more affordances, and where the third set of one or more affordances corresponds to a second portion of the application that is distinct from the first portion of the application.
In some embodiments, the second portion is displayed on the primary display unit 1802 in a compact view within the first user interface prior to detecting the swipe gesture, and the processing unit 1810 is configured to cause display of (e.g., with the primary display control unit 1812) the second portion on the primary display unit 1802 in an expanded view within the first user interface in accordance with the determination that the swipe gesture was performed in the second direction substantially perpendicular to the first direction.
In some embodiments, the first user interface for the application is displayed on the primary display unit 1802 in a full-screen mode, and the first set of one or more affordances displayed on the touch screen display unit 1806 includes controls corresponding to the full-screen mode.
In some embodiments, the second set of one or more affordances and the third set of one or more affordances includes at least one system-level affordance corresponding to at least one system-level functionality.
In some embodiments, after displaying the third set of one or more affordances on the touch screen display unit 1806, the processing unit 1810 is configured to: detect (e.g., with the input detecting unit 1816) a user input selecting the first portion on the first user interface; and, in response to detecting the user input: cease to display (e.g., with the touch screen display control unit 1814) the third set of one or more affordances on the touch screen display unit 1806, where the third set of one or more affordances corresponds to the second portion of the application; and cause display of (e.g., with the touch screen display control unit 1814) the second set of one or more affordances on the touch screen display unit 1806, where the second set of one or more affordances corresponds to the first portion of the application.
In some embodiments, the first direction is substantially parallel to a major dimension of the touch screen display unit 1806.
In some embodiments, the first direction is substantially perpendicular to a major dimension of the touch screen display unit 1806.
In some embodiments, the first portion is one of a menu, tab, folder, tool set, or toolbar of the application, and the second portion is one of a menu, tab, folder, tool set, or toolbar of the application.
The operations described above with reference to
In accordance with some embodiments,
As shown in
The processing unit 1910 is configured to: cause display of (e.g., with the primary display control unit 1912) a first user interface for the application executed by the processing unit 1910 on the primary display unit 1902 in a normal mode, the first user interface comprising a first set of one or more affordances associated with the application; and detect (e.g., with the input detecting unit 1916) a user input for displaying at least a portion of the first user interface for the application in a full-screen mode on the primary display unit 1902. In response to detecting the user input, the processing unit 1910 is configured to: cease to display (e.g., with the primary display control unit 1912) the first set of one or more affordances associated with the application in the first user interface on the primary display unit 1902; cause display of (e.g., with the primary display control unit 1912) the portion of the first user interface for the application on the primary display unit 1902 in the full-screen mode; and automatically, without human intervention, cause display of (e.g., with the touch screen display control unit 1914) a second set of one or more affordances for controlling the application on the touch screen display unit 1906, where the second set of one or more affordances correspond to the first set of one or more affordances.
In some embodiments, the second set of one or more affordances is the first set of one or more affordances.
In some embodiments, the second set of one or more affordances include controls corresponding to the full-screen mode.
In some embodiments, the processing unit 1910 is configured to detect (e.g., with the input detecting unit 1916) a user touch input selecting one of the second set of affordances on the touch-sensitive surface unit 1908, and, in response to detecting the user touch input, the processing unit 1910 is configured to change (e.g., with the changing unit 1918) the portion of the first user interface for the application being displayed in the full-screen mode on the primary display unit 1902 according to the selected one of the second set of affordances.
In some embodiments, after displaying the portion of the first user interface for the application in the full-screen mode on the primary display unit 1902, the processing unit 1910 is configured to: detect (e.g., with the input detecting unit 1916) a subsequent user input for exiting the full-screen mode; and, in response to detecting the subsequent user input: cause display of (e.g., with the primary display control unit 1912) the first user interface for the application executed by the processing unit 1910 on the primary display unit 1902 in the normal mode, the first user interface comprising the first set of one or more affordances associated with the application; and maintain display of (e.g., with the touch screen display control unit 1914) at least a subset of the second set of one or more affordances for controlling the application on the touch screen display unit 1906, where the second set of one or more affordances correspond to the first set of one or more affordances.
In some embodiments, the user input for displaying at least the portion of the first user interface for the application in full-screen mode on the primary display unit 1902 is at least one of a touch input detected on the touch-sensitive surface unit 1908 and a control selected within the first user interface on the primary display unit 1902.
In some embodiments, the second set of one or more affordances includes at least one system-level affordance corresponding to at least one system-level functionality.
The operations described above with reference to
In accordance with some embodiments,
As shown in
The processing unit 2010 is configured to: cause display of (e.g., with the primary display control unit 2012) a first user interface, on the primary display unit 2002, for an application executed by the processing unit 2010; cause display of (e.g., with the touch screen display control unit 2014) a second user interface, on the touch screen display unit 2006, the second user interface comprising a set of one or more affordances corresponding to the application; detect a notification (e.g., with the notification unit 2016); and, in response to detecting the notification, cause concurrent display of (e.g., with the touch screen display control unit 2014), in the second user interface on the touch screen display unit 2006, the set of one or more affordances corresponding to the application and at least a portion of the detected notification, where the detected notification is not displayed on the primary display unit 2002.
In some embodiments, prior to detecting the notification, the processing unit 2010 is configured to detect (e.g., with the input detecting unit 2018) a user input selecting a notification setting so as to display notifications on the touch screen display unit 2006 and to not display notifications on the primary display unit 2002.
In some embodiments, the processing unit 2010 is configured to detect (e.g., with the input detecting unit 2018) a user touch input on the touch-sensitive surface unit 2008 corresponding to the portion of the detected notification. In accordance with a determination (e.g., with the input type determining unit 2020) that the user touch input corresponds to a first type (e.g., a swipe gesture), the processing unit 2010 is configured to cease to display (e.g., with the touch screen display control unit 2014) in the second user interface the portion of the detected notification on the touch screen display unit 2006. In accordance with a determination (e.g., with the input type determining unit 2020) that the user touch input corresponds to a second type (e.g., a tap contact) distinct from the first type, the processing unit 2010 is configured to perform (e.g., with the performing unit 2022) an action associated with the detected notification.
In some embodiments, the portion of the notification displayed on the touch screen display unit 2006 prompts a user of the electronic device 2000 to select one of a plurality of options for responding to the detected notification.
In some embodiments, the portion of the notification displayed on the touch screen display unit 2006 includes one or more suggested responses to the detected notification.
In some embodiments, the notification corresponds to an at least one of an incoming instant message, SMS, email, voice call, or video call.
In some embodiments, the notification corresponds to a modal alert issued by an application being executed by the processing unit 2010 in response to a user input closing the application or performing an action within the application.
In some embodiments, the set of one or more affordances includes at least one a system-level affordance corresponding to at least one system-level functionality, and the notification corresponds to a user input selecting one or more portions of the input mechanism or the least one of a system-level affordance.
The operations described above with reference to
In accordance with some embodiments,
As shown in
The processing unit 2110 is configured to: cause display of (e.g., with the primary display control unit 2112) a user interface, on the primary display unit 2102, the user interface comprising one or more user interface elements; identify (e.g., with the identifying unit 2116) an active user interface element of the one or more user interface elements that is in focus on the primary display unit 2102, where the active user interface element is associated with an application executed by the processing unit 2110; and, in response to identifying the active user interface element that is in focus on the primary display unit 2102, cause display of (e.g., with the touch screen display control unit 2114) a set of one or more affordances corresponding to the application on the touch screen display unit 2106. The processing unit 2110 is configured to: detect (e.g., with the input detecting unit 2118) a user input to move a respective portion of the user interface; and, in response to detecting the user input, and in accordance with a determination (e.g., with the determining unit 2120) that the user input satisfies predefined action criteria: cease to display (e.g., with the primary display control unit 2112) the respective portion of the user interface on the primary display unit 2102; cease to display (e.g., with the touch screen display control unit 2114) at least a subset of the set of one or more affordances on the touch screen display unit 2106; and cause display of (e.g., with the touch screen display control unit 2114) a representation of the respective portion of the user interface on the touch screen display unit 2106.
In some embodiments, the respective portion of the user interface is a menu corresponding to the application executed by the processing unit 2110.
In some embodiments, the respective portion of the user interface is at least one of a notification or a modal alert.
In some embodiments, the predefined action criteria are satisfied when the user input is a dragging gesture that drags the respective portion of the user interface to a predefined location of the primary display unit 2102.
In some embodiments, the predefined action criteria are satisfied when the user input is predetermined input corresponding to moving the respective portion of the user interface to the touch screen display unit 2106.
In some embodiments, in response to detecting the user input, and in accordance with a determination (e.g., with the determining unit 2120) that the user input does not satisfy the predefined action criteria, the processing unit 2110 is configured to: maintain display of (e.g., with the primary display control unit 2112) the respective portion of the user interface on the primary display unit 2102; and maintain display of (e.g., with the touch screen display control unit 2114) the set of one or more affordances on the touch screen display unit 2106.
In some embodiments, the set of one or more affordances includes at least one system-level affordance corresponding to at least one system-level functionality, and the processing unit 2110 is configured to maintain display of (e.g., with the touch screen display control unit 2114) the at least one system-level affordance on the touch screen display unit 2106 after displaying the representation of the respective portion of the user interface on the touch screen display unit 2106.
In some embodiments, the representation of the respective portion of the user interface is overlaid on the set of one or more affordances on the touch screen display unit 2106.
The operations described above with reference to
In accordance with some embodiments,
As shown in
The processing unit is configured to: receive a request (e.g., with the request receiving unit 2212) to open an application; in response to receiving the request: (i) display, on the primary display (e.g., with the displaying unit 2214 in conjunction with the primary display unit 2202), a plurality of user interface objects associated with an application executing on the computing system, the plurality including a first user interface object displayed with its associated content and other user interface objects displayed without their associated content; and (ii) display, on the touch-sensitive secondary display (e.g., with the displaying unit 2214 in conjunction with the touch-sensitive secondary display unit 2204), a set of affordances that each represent one of the plurality of user interface objects; detect (e.g., with the detecting unit 2216), via the touch-sensitive display, a swipe gesture in a direction from a first affordance of the set of affordances and towards a second affordance of the set of affordances, wherein the first affordance represents the first user interface object and the second affordance represents a second user interface object that is distinct from the first user interface object; and in response to detecting the swipe gesture, update (e.g., with the displaying unit 2214) the primary display to cease displaying associated content for the first user interface object and to display associated content for the second user interface object.
In some embodiments of the computing system 2200, the processing unit is further configured to: detect continuous travel of the swipe gesture (e.g., with the detecting unit 2216) across the touch-sensitive secondary display, including the swipe gesture contacting a third affordance that represents a third user interface object; and, in response to detecting that the swipe gesture contacts the third affordance, update the primary display to display associated content for the third user interface object.
In some embodiments of the computing system 2200, each affordance in the set of affordance includes a representation of respective associated content for a respective user interface object of the plurality.
In some embodiments of the computing system 2200, the processing unit is further configured to: before detecting the swipe gesture, detect an initial contact with the touch-sensitive secondary display over the first affordance (e.g., with the detecting unit 2216); and, in response to detecting the initial contact, increase a magnification level of the first affordance (e.g., with the displaying unit 2214 in conjunction with the touch-sensitive secondary display 2204).
In some embodiments of the computing system 2200, the application is a web browsing application, and the plurality of user interface objects each correspond to web-browsing tabs.
In some embodiments of the computing system 2200, the processing unit is further configured to: detect an input at a URL-input portion of the web browsing application on the primary display (e.g., with the detecting unit 2216); and in response to detecting the input, update the touch-sensitive secondary display to include representations of favorite URLs (e.g., with the displaying unit 2214 in conjunction with the touch-sensitive secondary display unit 2204).
In some embodiments of the computing system 2200, the application is a photo-browsing application, and the plurality of user interface objects each correspond to individual photos.
In some embodiments of the computing system 2200, the application is a video-editing application, and the plurality of user interface object each correspond to individual frames in a respective video.
In accordance with some embodiments,
As shown in
The processing unit is configured to: receive a request to search within content displayed on the primary display of the computing device (e.g., with the request receiving unit 2312); in response to receiving the request: (i) display, on the primary display (e.g., with the displaying unit 2314 in conjunction with the primary display unit 2302), a plurality of search results responsive to the search, wherein focus is on a first search result of the plurality of search results; (ii) display, on the touch-sensitive secondary display (e.g., with the displaying unit 2314 in conjunction with the touch-sensitive secondary display 2304), respective representations that each correspond to a respective search result of the plurality of search results; detect, via the touch-sensitive secondary display, a touch input that selects a representation of the respective representations (e.g., with the detecting unit 2316), the representation corresponding to a second search result of the plurality of search results distinct from the first search result; and in response to detecting the input, change focus on the primary display to the second search result (e.g., with the displaying unit 2316 in conjunction with the primary display unit 2302).
In some embodiments of the computing device 2300, changing focus includes modifying, on the primary display, a visual characteristic of the particular search result.
In some embodiments of the computing device 2300, the processing unit is further configured to: detect a gesture that moves across at least two of the respective representations on the touch-sensitive secondary display (e.g., with the detecting unit 2316); and, in response to detecting the gesture, change focus on the primary display to respective search results that correspond to the at least two of the respective representations as the swipe gestures moves across the at least two of the respective representations (e.g., with the displaying unit 2316 in conjunction with the primary display unit 2302).
In some embodiments of the computing device 2300, the processing unit is further configured to: in accordance with a determination that a speed of the gesture is above a threshold speed, change focus on the primary display to respective search results in addition to those that correspond to the at least two of the respective representations (e.g., with the displaying unit 2316 in conjunction with the primary display unit 2302).
In some embodiments of the computing device 2300, the gesture is a swipe gesture.
In some embodiments of the computing device 2300, the gesture is a flick gesture.
In some embodiments of the computing device 2300, the representations are tick marks that each correspond to respective search results of the search results.
In some embodiments of the computing device 2300, the tick marks are displayed in a row on the touch-sensitive secondary display in an order that corresponds to an ordering of the search results on the primary display.
In some embodiments of the computing device 2300, the request to search within the content is a request to locate a search string within the content, and the plurality of search results each include at least the search string.
In some embodiments of the computing device 2300, displaying the plurality of search results includes highlighting the search string for each of the plurality of search results.
In accordance with some embodiments,
As shown in
The processing unit is configured to: display, on the primary display, a calendar application (e.g., with the displaying unit 2414 in conjunction with the primary display unit 2402); receive a request to display information about an event that is associated with the calendar application (e.g., with the request receiving unit 2412); and in response to receiving the request: (i) display, on the primary display, event details for the first event, the event details including a start time and an end time for the event (e.g., with the displaying unit 2414 in conjunction with the primary display unit 2402); and (ii) display, on the touch-sensitive secondary display, an affordance, the affordance indicating a range of time that at least includes the start time and the end time (e.g., with the displaying unit 2414 in conjunction with the touch-sensitive secondary display unit 2404).
In some embodiments of the computing system 2400, the processing unit is further configured to: detect, via the touch-sensitive secondary display, an input at the user interface control that modifies the range of time (e.g., with the detecting unit 2416); and in response to detecting the input: (i) modify at least one of the start time and the end time for the event in accordance with the input (e.g., with the modifying unit 2418); and (ii) display, on the primary display, a modified range of time for the event in accordance with the input (e.g., with the displaying unit 2414 in conjunction with the primary display unit 2402).
In some embodiments of the computing device 2400, the processing unit is further configured to: save the event with the modified start and/or end time to the memory of the computing system.
In some embodiments of the computing device 2400, the input that modifies the range of time is a press input that remains in contact with the affordance for more than a threshold amount of time and then moves at least a portion the affordance on the touch-sensitive secondary display.
In some embodiments of the computing device 2400, the input that modifies the range of time is a swipe gesture that moves across the touch-sensitive secondary display and causes the computing system to select a new start time and a new end time for the event, wherein the new start and end times correspond to a time slot that is of a same duration covered by the start and end times.
In accordance with some embodiments,
As shown in
The processing unit is configured to: detect a new connection between the computing system and an external device distinct from the computing system (e.g., with the detecting unit 2516); and in response to detecting the new connection, display, on the touch-sensitive secondary display, a plurality of affordances corresponding to functions available via the external device (e.g., with the displaying unit 2514 in conjunction with the touch-sensitive secondary display unit 2504).
In some embodiments of the computing system 2500, the processing unit is further configured to: receive, via the touch-sensitive secondary display, a selection of a first affordance that corresponds to a first function available via the external device (e.g., via the selection receiving unit 2512); and in response to receiving the selection, initiate performance of the first function (e.g., with the performance initiating unit 2518).
In some embodiments of the computing system 2500, the external device is an additional display, distinct from the primary display and the touch-sensitive secondary display.
In some embodiments of the computing system 2500, the plurality of affordances include a first affordance that, when selected, causes the computing system to initiate performance of a display mirroring function via the additional display.
In accordance with some embodiments,
As shown in
The processing unit is configured to: display, on the primary display, a user interface for an application that is executing on the computing system (e.g., with the displaying unit 2612 in conjunction with the primary display unit 2602); detect a first input at a particular location within the user interface (e.g., with the detecting unit 2614); and in response to detecting the first input, display, on the touch-sensitive secondary display, a set of affordances that each correspond to distinct characters (e.g., with the displaying unit 2612 in conjunction with the touch-sensitive secondary display unit 2604).
In some embodiments of the computing system 2600, the processing unit is further configured to: detect, via the touch-sensitive secondary display, a second input over a first affordance that corresponds to a first character of the distinct characters (e.g., with the detecting unit 2614); and in response to detecting the second input, display on the primary display a preview of the first character at the particular location while the input remains in contact with the first affordance (e.g., with the displaying unit 2612 in conjunction with the primary display unit 2602).
In some embodiments of the computing system 2600, the processing unit is further configured to: detect, via the touch-sensitive secondary display, movement of the second input from the first affordance and to a second affordance that corresponds to a second character of the distinct characters (e.g., with the detecting unit 2614); and in response to detecting the movement of the second input from the first affordance and to the second affordance, replace the preview of the first character with a preview of the second character (e.g., with the replacing unit 2616).
In some embodiments of the computing system 2600, the processing unit is further configured to: determine affordances to include in the set of affordances based at least in part on textual content included in the user interface.
In some embodiments of the computing system 2600, the determining is conducted in response to detecting that a user has modified textual content included in the user interface.
In some embodiments of the computing system 2600, the processing unit is further configured to: detect liftoff of the second input while it is contact with second affordance and, in response to detecting liftoff, update the user interface to include the second user interface element.
In some embodiments of the computing system 2600, the processing unit is further configured to: detect an additional input while second input is in contact with second affordance and, in response to detecting the additional input, update the user interface to include the second user interface element.
In some embodiments of the computing system 2600, the processing unit is further configured to: as the second input continues to move across the touch-sensitive secondary display, display previews for respective characters of the distinct characters as corresponding affordances in the set of affordances are contacted by the second input.
In some embodiments of the computing system 2600, the preview of second character remains displayed on the primary display while the input remains in contact with the second affordance.
In accordance with some embodiments,
As shown in
The processing unit is configured to: receive a request to open a content-editing application (e.g., with the request receiving unit 2712); in response to receiving the request: (i) display, on the primary display, the content-editing application (e.g., with the displaying unit 2714 in conjunction with the primary display unit 2702); and (ii) display, on the touch-sensitive secondary display, a user interface control for modifying at least one visual characteristic that is used to render content within the content-editing application (e.g., with the displaying unit 2714 in conjunction with the touch-sensitive secondary display unit 2704).
In some embodiments of the computing system 2700, the processing unit is further configured to: detect, via the touch-sensitive secondary display, an input at the user interface control that selects a first value for the at least one visual characteristic (e.g., with the detecting unit 2716); and after detecting the input, render content in the content-editing application using the first value for the at least one visual characteristic (e.g., with the content rendering unit 2718).
In some embodiments of the computing system 2700, the user interface control includes respective controls that each correspond to a respective value for the at least one visual characteristic along a sliding scale of values.
In some embodiments of the computing system 2700, the sliding scale of values represents distinct shades of color.
In some embodiments of the computing system 2700, the first value corresponds to a first shade of a first color and the processing unit is further configured to: in accordance with a determination that the input satisfies predetermined criteria, modify the user interface control on the touch-sensitive secondary display to include options for selecting other shades of the first color, distinct from the first shade of the first color.
In some embodiments of the computing system 2700, the processing unit is further configured to: before rendering the content, receive a selection of the content, wherein rendering the content includes presenting a preview of the content using the first value for the at least one visual characteristic.
In some embodiments of the computing system 2700, the preview is presented while the input remains in contact with the touch-sensitive secondary display.
In some embodiments of the computing system 2700, the processing unit is further configured to, in response to detecting liftoff of the input, cease to display the preview.
In some embodiments of the computing system 2700, the processing unit is further configured to, in response to detecting liftoff of the input, display the portion of the editable content with the modified value for the at least one visual characteristic.
In accordance with some embodiments,
As shown in
The processing unit is configured to: while the device is in a locked state, display a respective log-in user interface that is associated with logging in to a plurality of user accounts including a first user account and a second user account (e.g., with the displaying unit 3014); while displaying the log-in user interface, receive biometric information about a user (e.g., with the biometric information receiving unit 3012); in response to receiving the biometric information: (i) in accordance with a determination that the biometric information is consistent with biometric information for the first user account of the plurality of user accounts while the first user account does not have an active session on the device, display, on the display, a prompt to input a log-in credential for the first user account (e.g., with the displaying unit 3014); and (ii) in accordance with a determination that the biometric information is consistent with biometric information for the second user account of the plurality of user accounts while the second user account does not have an active session on the device, display, on the display, a prompt to input a log-in credential for the second user account (e.g., with the displaying unit 3014).
In some embodiments of the electronic device 3000, the processing unit is further configured to, in response to receiving the biometric information: in accordance with a determination that the biometric information is consistent with biometric information for the first user account of the plurality of user accounts while the first user account has an active session on the device, unlock the device with respect to the first user account (e.g., with the unlocking unit 3016).
In some embodiments of the electronic device 3000, the processing unit is further configured to, in response to receiving the biometric information: in accordance with a determination that the biometric information is consistent with biometric information for the second user account of the plurality of user accounts while the second user account has an active session on the device, unlock the device with respect to the second user account (e.g., with the unlocking unit 3016).
In some embodiments of the electronic device 3000, the processing unit is further configured to, in response to receiving the biometric information: in accordance with a determination that the biometric information is not consistent with biometric information for the any user account of the device, maintain the device in the locked state.
In some embodiments of the electronic device 3000, the log-in user interface includes a plurality of selectable affordances that correspond to the plurality of user accounts.
In some embodiments of the electronic device 3000, the processing unit is further configured to: while displaying the prompt to input a log-in credential for the first user account, receive entry of a log-in credential (e.g., with the receiving unit 3018); and in response to receiving entry of the log-in credential: in accordance with a determination that the log-in credential is consistent with a log-in credential for the first user account, unlock the device with respect to the first user account (e.g., with the unlocking unit 3016); and in accordance with a determination that the log-in credential is not consistent with a log-in credential for the first user account, maintain the device in the locked state.
In some embodiments of the electronic device 3000, the processing unit is further configured to: while displaying the prompt to input a log-in credential for the second user account, receive entry of a log-in credential (e.g., with the receiving unit 3018); and in response to receiving entry of the log-in credential: in accordance with a determination that the log-in credential is consistent with a log-in credential for the second user account, unlock the device with respect to the first user account (e.g., with the unlocking unit 3016); and in accordance with a determination that the log-in credential is not consistent with a log-in credential for the second user account, maintain the device in the locked state.
In some embodiments of the electronic device 3000, the log-in user interface includes instructions to provide biometric information.
In some embodiments of the electronic device 3000, the electronic device includes a secondary display that is adjacent to the biometric sensor; and the processing unit is further configured to, while displaying the log-in user interface on the display of the device, display instructions at the secondary display to provide biometric information via the biometric sensor.
In some embodiments of the electronic device 3000, the biometric sensor is a fingerprint sensor.
In some embodiments of the electronic device 3000, the biometric sensor is a facial detection sensor.
In some embodiments of the electronic device 3000, the biometric sensor is a retina scanner.
In accordance with some embodiments,
As shown in
The processing unit is configured to: while the device is logged in to a first user account, display a user interface that is associated with the first user account (e.g., with the displaying unit 3114). The device is associated with a plurality of user accounts including the first user account and a second user account, and the second user account is associated with biometric information that enables logging in to the second user account. While displaying the user interface that is associated with the first user account, the processing unit is configured to receive an input via the input element with the integrated biometric sensor (e.g., with the biometric information receiving unit 3112). In response to receiving the input via the input element with the integrated biometric sensor, the processing unit is configured to: in accordance with a determination that the input meets second-user switching criteria while the second user account has an active session on the device, wherein the second-user switching criteria include a requirement that biometric information detected during the input with the input element is consistent with biometric information for the second user account of the plurality of user accounts: (i) unlock the device with respect to the second user account (e.g., with the unlocking unit 3116); (ii) lock the device with respect to the first user account (e.g., with the locking unit 3118); and (iii) replace display of the user interface associated with the first account with a user interface associated with the second user account (e.g., with the displaying unit 3114).
In some embodiments of the electronic device 3100, the processing unit is further configured to, in response to receiving the biometric information: in accordance with a determination that the input meets second-user switching criteria while the second user account does not have an active session on the device, wherein the second-user switching criteria include a requirement that biometric information detected during the input with the input element is consistent with biometric information for the second user account of the plurality of user accounts, display (e.g., with the displaying unit 3114), on the display, a prompt to input a log-in credential for the second user account.
In some embodiments of the electronic device 3100, the processing unit is further configured to, in response to receiving the biometric information: in accordance with a determination that the input meets third-user switching criteria while a third user account has an active session on the device, wherein the third-user switching criteria include a requirement that biometric information detected during the input with the input element is consistent with biometric information for the third user account of the plurality of user accounts: (i) unlock the device with respect to the third user account (e.g., with the unlocking unit 3116); (ii) lock the device with respect to the first user account (e.g., with the locking unit 3118); and (iii) replace display of the user interface associated with the first account with a user interface associated with the third user account (e.g., with the displaying unit 3114).
In some embodiments of the electronic device 3100, the processing unit is further configured to, in response to receiving the biometric information: in accordance with a determination that the input meets third-user switching criteria while the third user account does not have an active session on the device, wherein the third-user switching criteria include a requirement that biometric information detected during the input with the input element is consistent with biometric information for the third user account of the plurality of user accounts, display (e.g., with the displaying unit 3114), on the display, a prompt to input a log-in credential for the third user account.
In some embodiments of the electronic device 3100, the input element is a button, and the input via the input element with the integrated biometric sensor includes detecting a press input via the button.
In some embodiments of the electronic device 3100, the second-user switching criteria include a criterion that the press input lasts for less than a first threshold amount of time; and the processing unit is further configured to, in response to receiving the press input via the input element with the integrated biometric sensor in accordance with a determination that the press input with the button lasts longer than the first threshold amount of time, put the device into a low power mode (e.g., with the low power mode entering unit 3122).
In some embodiments of the electronic device 3100, the second-user switching criteria include a criterion that the button press for less than a first threshold amount of time; and the processing unit is further configured to, in response to receiving the input via the input element with the integrated biometric sensor: in accordance with a determination that the press input with the button lasts longer than the first threshold amount of time and less than a second threshold amount of time, put the device into a low power mode (e.g., with the low power mode entering unit 3122); and in accordance with a determination that the press input with the button lasts longer than the second threshold amount of time, display a menu of options for changing a state of the device (e.g., with the displaying unit 3114).
In some embodiments of the electronic device 3100, the processing unit is further configured to, in response to receiving the input via the input element with the integrated biometric sensor: in accordance with a determination that the press input with the button lasts longer than a third threshold amount of time that is greater than the second threshold amount of time, restart the device (e.g., with the restarting unit 3124).
In some embodiments of the electronic device 3100, the processing unit is further configured to, after replacing display of the user interface associated with the first account with a user interface associated with the second user account: while displaying the user interface that is associated with the second user account, receive a second input via the input element with the integrated biometric sensor (e.g., with the biometric information receiving unit 3112); in response to receiving the second input via the input element with the integrated biometric sensor: in accordance with a determination that the second input meets first-user switching criteria while the first user account has an active session on the device, wherein the first-user switching criteria include a requirement that biometric information detected during the input with the input element is consistent with biometric information for the first user account of the plurality of user accounts: (i) unlock the device with respect to the first user account (e.g., with the unlocking unit 3116); (ii) lock the device with respect to the second user account (e.g., with the locking unit 3118); and (iii) replace display of the user interface associated with the second account with a user interface associated with the first user account (e.g., with the displaying unit 3114).
The foregoing description, for purpose of explanation, has been described with reference to specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in view of the above teachings. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications, to thereby enable others skilled in the art to best use the invention and various described embodiments with various modifications as are suited to the particular use contemplated.
This application is a continuation of U.S. patent application Ser. No. 15/655,707, filed Jul. 20, 2017, which claims priority to U.S. Provisional Application Ser. No. 62/412,792, filed Oct. 25, 2016, and U.S. Provisional Application Ser. No. 62/368,988, filed Jul. 29, 2016. Each of these applications is hereby incorporated by reference in its respective entirety. This application is also a continuation-in-part of U.S. patent application Ser. No. 15/275,298, filed Sep. 23, 2016, which claims priority to U.S. Provisional Application Ser. No. 62/368,988, filed Jul. 29, 2016, and which is a continuation-in-part of U.S. patent application Ser. No. 15/113,779, filed Jul. 22, 2016, which is a national phase entry of PCT/US2015/012694, filed Jan. 23, 2015, which claims priority to U.S. Provisional Application Ser. No. 62/104,023, filed Jan. 15, 2015, and to U.S. Provisional Application Ser. No. 61/930,663, file Jan. 23, 2014.
| Number | Name | Date | Kind |
|---|---|---|---|
| 20020191029 | Gillespie et al. | Dec 2002 | A1 |
| 20090271723 | Matsushima et al. | Oct 2009 | A1 |
| 20090315867 | Sakamoto et al. | Dec 2009 | A1 |
| 20110314405 | Turner et al. | Dec 2011 | A1 |
| 20120068933 | Larsen | Mar 2012 | A1 |
| 20130050135 | Stewart et al. | Feb 2013 | A1 |
| 20130332836 | Cho | Dec 2013 | A1 |
| 20140075373 | Jitkoff et al. | Mar 2014 | A1 |
| 20170010771 | Bernstein et al. | Jan 2017 | A1 |
| 20170315704 | Shaw et al. | Nov 2017 | A1 |
| 20180032203 | Sepulveda et al. | Feb 2018 | A1 |
| 20190220134 | Sepulveda et al. | Jul 2019 | A1 |
| Number | Date | Country |
|---|---|---|
| 2009-258667 | Nov 2009 | JP |
| 2011-018085 | Jan 2011 | JP |
| 2017-504128 | Feb 2017 | JP |
| WO 2015112868 | Jul 2015 | WO |
| Entry |
|---|
| Notice of Allowance, dated Feb. 7, 2019, received in Danish Patent Application No. 2017-70561 (7394DK), which corresponds with U.S. Appl. No. 15/655,707, 2 pages. |
| Patent, filed Apr. 3, 2019, received in Danish Patent Application No. 2017-70561 (7394DK), which corresponds with U.S. Appl. No. 15/655,707, 5 pages. |
| International Preliminary Report on Patentability, dated Jan. 29, 2019, received in International Application No. PCT/US2017/041959 (7394WO), which corresponds with U.S. Appl. No. 15/655,707, 9 pages. |
| iOS 11 Updates, “Demo of iOS 9 Apps: Search, Siri, Photies App”, https://youtube.be.EBnXMM1X6xA, Jun. 12, 2015, 2 pages. |
| MSAppliedSciences, “DisplayCover”, https:/youtube/OSFIvml0Sso, Aug. 20, 2015, 2 pages. |
| MSAppliedSciences, “UIST Student Innovation Contest 2010 Concept”, https:/youtube/ojusRO38Tdc, Aug. 9, 2010, 3 pages. |
| Wagner et al., U.S. application filed on Apr. 15, 2014, titled “Device, Method, and Graphical User Interface for Navigating and Displaying Content in Context”, 140 pages. |
| YouTube, “Steve Jobs Introduces the iPad—2010 (full)”, https://www.youtube.com/watch?v=zZtWISDvb_k, May 5, 2015, 2 pages. |
| Office Action, dated Aug. 10, 2018, received in U.S. Appl. No. 15/655,707, 15 pages. |
| Innovation Patent, dated Sep. 19, 2017, received in Australia Patent Application No. 2017100879, which corresponds with U.S. Appl. No. 15/655,707, 1 page. |
| Office Action, dated Feb. 1, 2019, received in Australia Patent Application No. 2017302420, which corresponds with U.S. Appl. No. 15/655,707, 6 pages. |
| Office Action, dated Aug. 25, 2017, received in Chinese Patent Application No. 201710632979.8, which corresponds with U.S. Appl. No. 15/655,707, 3 pages. |
| Office Action, dated Oct. 10, 2017, received in Danish Patent Application No. 2017-70561, which corresponds with U.S. Appl. No. 15/655,707, 9 pages. |
| Office Action, dated Mar. 14, 2018, received in Danish Patent Application No. 2017-70561, which corresponds with U.S. Appl. No. 15/655,707, 3 pages. |
| Office Action, dated Oct. 31, 2018, received in Danish Patent Application No. 2017-70561, which corresponds with U.S. Appl. No. 15/655,707, 5 pages. |
| Intention to Grant, dated Jan. 2, 2019, received in Danish Patent Application No. 2017-70561, which corresponds with U.S. Appl. No. 15/655,707, 2 pages. |
| International Search Report and Written Opinion, dated Jan. 2, 2018, received in International Patent Application No. PCT/US2017/041959, which corresponds with U.S. Appl. No. 15/655,707, 15 pages. |
| Office Action, dated Jun. 27, 2019, received in European Patent Application No. 17745225.7, which corresponds with U.S. Appl. No. 15/655,707, 6 pages. |
| Office Action, dated Jun. 6, 2019, received in Korean Patent Application No. 2019-7004584, which corresponds with U.S. Appl. No. 15/655,707, 6 pages. |
| Office Action, dated Jul. 31, 2019, received in U.S. Appl. No. 16/361,109, 13 pages. |
| Notice of Acceptance, dated Jul. 15, 2019, received in Australia Patent Application No. 2017302420, which corresponds with U.S. Appl. No. 15/655,707, 3 pages. |
| Notice of Allowance, dated Oct. 24, 2019, received in Korean Patent Application No. 2019-7004584, which corresponds with U.S. Appl. No. 15/655,707, 5 pages. |
| Certificate of Grant, dated Nov. 14, 2019, received in Australia Patent Application No. 2017302420, which corresponds with U.S. Appl. No. 15/655,707, 4 pages. |
| Number | Date | Country | |
|---|---|---|---|
| 20190220135 A1 | Jul 2019 | US |
| Number | Date | Country | |
|---|---|---|---|
| 62412792 | Oct 2016 | US | |
| 62368988 | Jul 2016 | US | |
| 62104023 | Jan 2015 | US | |
| 61930663 | Jan 2014 | US | |
| 62368988 | Jul 2016 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 15655707 | Jul 2017 | US |
| Child | 16361122 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 15275298 | Sep 2016 | US |
| Child | 15655707 | US | |
| Parent | 15113779 | US | |
| Child | 15275298 | US |
