Patent Application
20090237325
The present disclosure relates generally to display devices and more particularly to clustering displays of display devices.
Portable electronic devices including cellular telephone handsets, personal digital assistants (PDAs), and handheld gaming devices among other devices, have become increasingly popular, particularly in mobile societies. Unfortunately, native displays of such devices are not infinitely scalable with respect to practical human usage. At some point, a given display can be too small to be of any practical use. This point can be a function of the limits of human perception, the quantity of information to be displayed at any given time on the display, or both. Such limits represent a significant challenge as products such as cellular telephones, personal digital assistants, small personal computers and the like both shrink in size while gaining increased functionality and/or networking capability.
Others have endeavored to address competing demands for large display areas. Examples of such endeavors are an expandable display having multiple folding sections in a handheld computing device, wherein the display is expandable upon unfolding the multiple display sections. An alternative embodiment includes a retractable e-paper display screen that is supported by a folding panel that may be expanded.
However, with the slow realization of borderless and/or flexible displays in electronic devices, clustering of traditional displays from multiple devices provides an alternative means for obtaining larger, effective displays and can also be applied to borderless and flexible displays in the future.
Accordingly, there is a need for a method and system for Clustering Displays.
The accompanying figures, where like reference numerals refer to identical or functionally similar elements throughout the separate views, together with the detailed description below, are incorporated in and form part of the specification, and serve to further illustrate embodiments of concepts that include the claimed invention, and explain various principles and advantages of those embodiments.
Skilled artisans will appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions of some of the elements in the figures may be exaggerated relative to other elements to help to improve understanding of embodiments of the present invention.
The apparatus and method components have been represented where appropriate by conventional symbols in the drawings, showing only those specific details that are pertinent to understanding the embodiments of the present invention so as not to obscure the disclosure with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein.
Various embodiments of the invention provide a method and a system for clustering displays. The method includes collecting clustering information of display devices. A profile is generated from the collected clustering information. A list of display configurations for the display devices is created based on the generated profile. A particular display configuration is selected from the created list of display configurations for clustering the displays and content is displayed on a display cluster. The display cluster is formed by clustering the displays of display devices in accordance with the selected display configuration.
Before describing in detail the method and system for clustering displays, it should be observed that the present invention resides primarily in combinations of method steps and system components related to a method and system for clustering displays. Accordingly, the system components and method steps have been represented where appropriate by conventional symbols in the drawings, showing only those specific details that are pertinent to understanding the present invention so as not to obscure the disclosure with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein.
The slave display devices (130-150) include a display proxy (260-262) and a user interface (UI) proxy (263-265). The display proxies (260, 261, and 262) are configured to receive pixel streams from the master display device 120. The pixel streams correspond to a portion of the content that is to be displayed 250 on the sub-region formed by the slave display device (130, 140, or 150). The display proxy (260, 261, or 262) then passes the received pixel streams to a rendering layer. The UI proxies (263, 264, and 265) are configured to communicate the UI event occurring at the slave display device (130, 140, or 150) to the master display device 120. Transfer of control between master display device and slave display devices is made using synchronization messages. Synchronization messages provide out-of-the-band channel for meta control data transfer. Such synchronization messages aid frame synchronization as well. Pixel streaming links are used for streaming pixels.
The master display device 120 also includes a display proxy (230) and a UI proxy (235) which functions as discussed above. The master display device further includes an operating system 245, an application platform 255 and a display 240. Examples of application platform include Java® (“JAVA® is a registered trademark of Sun Microsystems, Inc.”), BREW® (“BREW® is a registered trademark of QUALCOMM, Inc.”) etc. The slave display devices (130-150) further include a virtual frame buffer (not shown), a layout coordinator (not shown), a UI coordinator (not shown), and a network dispatcher (not shown) all which functions as discussed above. Each of the slave display devices (130-150) further includes an operating system (266, 267 or 268), an application platform (269,270 or 271), and display (272, 273, or 274). When the master display device 120 transfers control to a slave display device (130, 140, or 150), the slave display device (130, 140, or 150) acts as a master display device and the master display device 120 acts as a slave display device. The master display device 120 and the slave display device (130, 140, and 150) are synchronized with one another through short range communication methods. Examples of short range communication methods are Bluetooth®, WiFi®, UWB®, Zigbee®, or Whispering optical display.
In an embodiment, the master display device 120 creates a prioritized list of display configurations. The display configurations are prioritized based on an analysis of the clustering information of each of the display devices (120-160). In an example, Greedy Heuristic (GH) algorithm is used for prioritizing the display configurations based on analyzing one of several parameters like screen size, border dimensions etc., of each display device. The GH algorithm assumes unified resolution across the display devices. In an example, the GH algorithm calculates a screen size fitness value as follows,
Screen Size Fitness=1−(size_gap_or_extruding_area)/ΣSize(DDi),
size_gap_or_extruding_area—difference between the dimensions of at least two display devices,
Size (DD) is the size of the display of display device (DD), where i=1 to n.
Accordingly, the GH algorithm starts calculating the screen size fitness value starting from the display device that has the largest screen size and iteratively tries out alignment with other display devices that minimize screen size fitness value in each iteration.
At step 340, a particular display configuration is selected from the created list of display configurations for clustering the displays. In an embodiment, the master display device 120 selects a display configuration based on preferences set by a user. The preferences are preset by the user and the master display device 120 automatically selects a display configuration that matches the user's preferences. In another embodiment, the list of display configurations is available to the user such that the user manually selects a display configuration from the list. Once the display configuration is selected, the displays of the display devices (120-160) are clustered in accordance with the selected display configuration to form the display cluster as shown in
In an embodiment, instead of the master display device 120 creating a list of display configurations, the user simply lays the display devices in a configuration of the user's choice with the sides of the display devices touching each other, and the master display device 120 lays the content on the user selected configuration of display cluster.
At step 350, the master display device 120 displays content on the display cluster. In an embodiment, the master display device 120 verifies whether the displays are clustered in accordance with the selected display configuration. If the displays are not clustered in accordance with the selected display configuration, the user is notified of an error until the displays are clustered to match the selected display configuration. In an example, the display cluster is validated using proximity sensors based on optical or RF deployed along the edges of the display devices. The sensors detect a mutual alignment of the display devices and provide information regarding a relative alignment of the display devices. This information is then used for validating the display cluster.
Each display device, master or slave, may be capable of automatic sensing and awareness of the position of the other display devices relative to each other. The relative location can be detected using proximity sensors based on, for example, an optical signal detected by photodiodes or optical fibers on at least one side, and up to all sides, of the display device. For one embodiment, display device A may know that display device B is located at its first side, there is no device at its second side, display device C is located at its third side, and display device D is located at its fourth side. All other display device in the cluster have the same awareness. Thus, even though the connections are wireless, the display devices may know their respective positions, enabling complex composite display functionality.
If a display device operated by a user aligns the other display devices in a format or configuration different from the suggested format or configuration, the system may optimize the content to the available configuration. For example, a display device may decide not to use a particular screen based on the type of content. This also facilitates automatic compensation for a display device joining or leaving the cluster. The proximity sensors may be based on other near line-of-sight technologies such as NFC, IR and whispering displays. Orientation and position can also be obtained by motion sensing with integrated accelerometers, gyroscopes and electronic compass.
In an embodiment, as shown in
In another embodiment as shown in
In yet another embodiment as shown in
An out-of-cluster display device, such as master display device that is outside a cluster, may also alter or control the orientation of the displayed content on display devices that are in-cluster. For example, rotating the master display device that is outside the cluster may also rotate the displayed content of the in-cluster slave devices. In addition to rotation, the master display device may actuate alteration or control in other ways, such as through key press, touch, tilt, or tap at one of the display devices.
In an embodiment, the control from the master display device 120 is transferred to a slave display device (130, 140, 150, or 160) i.e. any of the slave display devices (130, 140, 150, or 160) can behave as a master display device. Transferring control from the master display device 120 to the slave display device (130, 140, 150, or 160) may be performed through activation of a user interface at the slave display device (130, 140, 150, or 160). Examples of activating a user interface are through a single interaction like touch, key press, tap, voice tag, and so forth. Upon activation of a user interface, appropriate control messages are sent through synchronization message channel to be broadcasted within the cluster. Once the control is transferred to the slave display device (130, 140, 150, or 160), the slave display device (130, 140, 150, or 160) behaves as a master display device and shares content to be displayed on the display cluster.
In an embodiment, the orientation of the displayed content can be altered by a single interaction on any display device (120-160) participating in the display cluster. Examples of single interaction are key press, touch, tilt, rotation, or tap. Tilt, rotation, tap etc. may be measured using integrated sensors such as accelerometers and gyroscopes. The orientation of the displayed content is altered such that the content can be optimally viewed from the perspective of the vertex of the device on which the single interaction is performed. In an example, a tap on the second display device would alter the orientation in such a way that the user at the second display device can optimally view the displayed content.
Similarly, the master display device 120 detects an absence of a display device (130-150, or 160) from the display cluster i.e. the master display device 120 detects when a display device leaves an existing display cluster. The master display device 120 detects the absence of a display device (130-150, or 160) through e.g., Bluetooth®, proximity sensors, motion gesture sensors and so forth. Upon detecting that a display device (130-150, or 160) has left the display cluster, the master display device 120 creates a new list of display configurations based on the display devices remaining in the display cluster. A particular display configuration is selected from the new list of display configurations. The displays of the remaining display devices are clustered in accordance with the selected display configuration and the content is displayed on the new display cluster.
In an embodiment, the display cluster supports composite input as well as composite output. For example, the user selects a widget/icon on a first display device in the display cluster and drags his/her finger across the first display device's display, across the edge of the first display device and across the edge of a second display device, and onto the display of the second display device to copy for e.g., a song or a photo represented by that widget/icon. In an example, in a game of chess, the composite input may be used to move a game token on a board represented by the display cluster. A UI event occurring at each display device (120-150) is mapped from a local coordinate system specific to a display device (120-150) to a global coordinate system specific to a virtual frame buffer manager of the master display device 120. The UI event occurring at a display device (120-150) is communicated to the master display device by the UI proxy (263, 264, or 265) of respective display devices (120-150).
In the foregoing specification, specific embodiments have been described. However, one of ordinary skill in the art appreciates that various modifications and changes can be made without departing from the scope of the invention as set forth in the claims below. Accordingly, the specification and figures are to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of present teachings. The benefits, advantages, solutions to problems, and any element(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential features or elements of any or all the claims. The invention is defined solely by the appended claims including any amendments made during the pendency of this application and all equivalents of those claims as issued.
Moreover in this document, relational terms such as first and second, top and bottom, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. The terms “comprises,” “comprising,” “has”, “having,” “includes”, “including,” “contains”, “containing” or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises, has, includes, contains a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by “comprises . . . a”, “has . . . a”, “includes . . . a”, “contains . . . a” does not, without more constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises, has, includes, contains the element. The terms “a” and “an” are defined as one or more unless explicitly stated otherwise herein. The terms “substantially”, “essentially”, “approximately”, “about” or any other version thereof, are defined as being close to as understood by one of ordinary skill in the art, and in one non-limiting embodiment the term is defined to be within 10%, in another embodiment within 5%, in another embodiment within 1% and in another embodiment within 0.5%. The term “coupled” as used herein is defined as connected, although not necessarily directly and not necessarily mechanically. A device or structure that is “configured” in a certain way is configured in at least that way, but may also be configured in ways that are not listed.
It will be appreciated that some embodiments may be comprised of one or more generic or specialized processors (or “processing devices”) such as microprocessors, digital signal processors, customized processors and field programmable gate arrays (FPGAs) and unique stored program instructions (including both software and firmware) that control the one or more processors to implement, in conjunction with certain non-processor circuits, some, most, or all of the functions of the method and/or apparatus described herein. Alternatively, some or all functions could be implemented by a state machine that has no stored program instructions, or in one or more application specific integrated circuits (ASICs), in which each function or some combinations of certain of the functions are implemented as custom logic. Of course, a combination of the two approaches could be used.
Moreover, an embodiment can be implemented as a computer-readable storage medium having computer readable code stored thereon for programming a computer (e.g., comprising a processor) to perform a method as described and claimed herein. Examples of such computer-readable storage mediums include, but are not limited to, a hard disk, a CD-ROM, an optical storage device, a magnetic storage device, a ROM (Read Only Memory), a PROM (Programmable Read Only Memory), an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically Erasable Programmable Read Only Memory) and a Flash memory. Further, it is expected that one of ordinary skill, notwithstanding possibly significant effort and many design choices motivated by, for example, available time, current technology, and economic considerations, when guided by the concepts and principles disclosed herein will be readily capable of generating such software instructions and programs and ICs with minimal experimentation.
The Abstract of the Disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, it can be seen that various features are grouped together in various embodiments for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separately claimed subject matter.
