Today, most computing devices, whether stationary or mobile device, utilize some form of display screen or surface as a user-interface (UI) component. Often these displays are merely output only devices, while a growing number utilize touch-sensitive screens for interactivity and/or input functionality. Recent technological advances both in terms of user-interfaces as well as display surfaces have sparked a growing evolution toward surface computing. In the domain of surface computing, the associated displays are generally touch-sensitive screens of substantially any form factor that often forego many traditional I/O devices such as a keyboard or mouse in favor of tactile-based manipulation.
In a related field, recent technological advances have also led to cheaper displays, some of which can be flexible or malleable, or even in the form of a fabric. Due to the wide proliferation of computing devices with display screens as well as the many new types of display surfaces or materials, new techniques or technologies can be implemented or leveraged in order to enhance functionality, increase productivity, and/or enrich user experiences.
The following presents a simplified summary of the claimed subject matter in order to provide a basic understanding of some aspects of the claimed subject matter. This summary is not an extensive overview of the claimed subject matter. It is intended to neither identify key or critical elements of the claimed subject matter nor delineate the scope of the claimed subject matter. Its sole purpose is to present some concepts of the claimed subject matter in a simplified form as a prelude to the more detailed description that is presented later.
The subject matter disclosed and claimed herein, in one or more aspects thereof, comprises a display that is physically divisible. In accordance therewith and to other related ends, a separable display can be provided that is configured as a user-interface (UI) for an associated multi-node computing architecture. The computing architecture can include a plurality of computing nodes, each of which can be physically embedded within the display and thus dispersed throughout substantially all regions of the display.
Moreover, each computing node can be configured to function as an autonomous computing unit even though typical operation can involve other computing nodes that are physically coupled (e.g., by way of coupled portions of the separable display) and, thus, comprise the multi-node computing component. Furthermore, each computing node can be further configured to manage UI transactions and/or I/O transactions associated with a respective local region of the separable display, e.g. the local region proximal to a location in which a given computing node is embedded.
Generally, each computing node can include a processor, a memory store, and one or more components that can identify other physically coupled computing nodes as well as various attributes associated with other computing nodes such as capabilities, memory or data sets, authorizations, assignments, or licenses, spatial geometry or the like
The following description and the annexed drawings set forth in detail certain illustrative aspects of the claimed subject matter. These aspects are indicative, however, of but a few of the various ways in which the principles of the claimed subject matter may be employed and the claimed subject matter is intended to include all such aspects and their equivalents. Other advantages and distinguishing features of the claimed subject matter will become apparent from the following detailed description of the claimed subject matter when considered in conjunction with the drawings.
The claimed subject matter is now described with reference to the drawings, wherein like reference numerals are used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the claimed subject matter. It may be evident, however, that the claimed subject matter may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to facilitate describing the claimed subject matter.
As used in this application, the terms “component,” “module,” “system,” or the like can, but need not, refer to a computer-related entity, either hardware, a combination of hardware and software, software, or software in execution. For example, a component might be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer. By way of illustration, both an application running on a controller and the controller can be a component. One or more components may reside within a process and/or thread of execution and a component may be localized on one computer and/or distributed between two or more computers.
Furthermore, the claimed subject matter may be implemented as a method, apparatus, or article of manufacture using standard programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof to control a computer to implement the disclosed subject matter. The term “article of manufacture” as used herein is intended to encompass a computer program accessible from any computer-readable device, carrier, or media. For example, computer readable media can include but are not limited to magnetic storage devices (e.g., hard disk, floppy disk, magnetic strips . . . ), optical disks (e.g., compact disk (CD), digital versatile disk (DVD) . . . ), smart cards, and flash memory devices (e.g. card, stick, key drive . . . ). Additionally it should be appreciated that a carrier wave can be employed to carry computer-readable electronic data such as those used in transmitting and receiving electronic mail or in accessing a network such as the Internet or a local area network (LAN). Of course, those skilled in the art will recognize many modifications may be made to this configuration without departing from the scope or spirit of the claimed subject matter.
Moreover, the word “exemplary” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects or designs. Rather, use of the word exemplary is intended to present concepts in a concrete fashion. As used in this application, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” Therefore, unless specified otherwise, or clear from context, “X employs A or B” is intended to mean any of the natural inclusive permutations. That is, if X employs A; X employs B; or X employs both A and B, then “X employs A or B” is satisfied under any of the foregoing instances. In addition, the articles “a” and “an” as used in this application and the appended claims should generally be construed to mean “one or more” unless specified otherwise or clear from context to be directed to a singular form.
As used herein, the terms “infer” or “inference” generally refer to the process of reasoning about or inferring states of the system, environment, and/or user from a set of observations as captured via events and/or data. Inference can be employed to identify a specific context or action, or can generate a probability distribution over states, for example. The inference can be probabilistic-that is, the computation of a probability distribution over states of interest based on a consideration of data and events. Inference can also refer to techniques employed for composing higher-level events from a set of events and/or data. Such inference results in the construction of new events or actions from a set of observed events and/or stored event data, whether or not the events are correlated in close temporal proximity, and whether the events and data come from one or several event and data sources.
As an introductory guideline, it should be noted that
Referring now to the drawings, with reference initially to
The plurality of computing nodes 106 can be included in or embedded in separable display 102. Moreover, each of the computing nodes 106 can be distributed throughout separable display 102 at various locations that can be organized or structured based upon pre-formatted separation boundaries, based upon likely or common separation boundaries, based upon suitability for substantially any separation boundaries, or the like. Regardless, each computing node 106 can be responsible for managing UI transactions associated with a location region 1081-108N of separable display 102 that is local to the associated computing node 106. In other words, UI transactions (e.g., I/O transactions such as rendering graphics or a mouse or cursor click) that occur or originate or relate to a particular region 108 can be handled by the associated computing node 106 that is responsible for that physical space of separable display 102. For instance, computing node 1061 can render graphics or receive inputs when such UI transactions relate to region 1081; computing node 1062 can render graphics or receive inputs when such UI transactions relate to region 1082; and so on.
In accordance with the claimed subject matter, it should be appreciated that although potentially existing in a state of a single, seamless or visibly discrete display, separable display 102 can be physically divided which is discussed in more detail with reference to
Hence, as depicted in
Therefore, it should be appreciated that polling component 114 can be configured to identify physically coupled computing nodes 106, which can be accomplished by way of acknowledgement messages or lists 116. Lists 116 can relate to other types of lists as well directed to additional types of information over membership, which is further discussed in connection with
Turning now to
As discussed previously, each computing node 106 can be responsible for managing UI transactions and/or I/O transactions 208 in connection with an associated local region 108. Typically, for computing node 1061 to interact with computing node 1062, local region 1081 will be physically coupled to local region 1082, either directly or indirectly. In other words, separable display 102 will include both local regions 1081 and 1082 in a continuous surface, whether or not local regions 1081 and 1082 are adjacent to one another or have other local regions 108 in between. Consequently, should local region 1081 be physically severed from local region 1082, then interaction between computing node 1061 and 1062 is effectively terminated, or at least cannot be accomplished by the stated means, but can be accomplished by way of wireless transactions, which are discussed further in connection with
In addition to discovering physically coupled computing nodes 106 as discussed previously, polling component 114 can further identify capabilities of physically coupled computing nodes 106. Discovered capabilities can include, e.g. memory that is shared, sources of power, as well as enumerated I/O capabilities discussed infra in connection with
Moreover, polling component 114 can also identify when a particular computing node 106 is physically decoupled, which is further discussed in connection with
Hence, consider the case in which the physical decoupling of computing node 106 occurs due to a separation (e.g. division boundaries 306 discussed infra) that differs from boundaries associated with a local region 108 for the decoupled computing node 106. In that case, polling component 114 can be appraised of this situation and can determine that other computing nodes 106, at least those that were previously adjacent to the decoupled computing node 106, might need to adjust the size, shape, and/or dimensions of their own respective local regions 108. It should be understood that in some cases, local regions 108 can be configured in accordance with pre-formatted and/or known perforations of separable display 102. In those cases, physical decoupling of a particular computing node 106 will generally not entail re-defining local regions 108 for other computing nodes 106.
Next to be described in more detail, synchronization component(s) 202 can facilitate load balancing 204 and/or processor load or utilization distribution or normalization. In particular, load balancing 204 can be with respect to processor 1101 associated with computing node 1061 and other disparate processors 110 (e.g., processor 1102) associated with other disparate physically coupled computing nodes 106 (e.g., computing node 1062). Hence, if processor 1101 presently has a relatively high utilization, e.g. due to intensive updates to local region 1081, while processor 1102 is relatively idle, then synchronization component 2021 can transfer portions of the load to other processors 110 that are associated with physically coupled computing nodes 106.
Additionally or alternatively, synchronization component 202 can facilitate various types of data operations 206. Data operations 206 can relate to data transfer between various memory stores 112 (e.g., between memory store 1121 and 1122) that are associated with physically coupled computing nodes 106 as well as other transactions with respect to data included in memory store 112. Likewise, synchronization component 202 can facilitate a transfer of data between private memory 118 and public memory 120 for a single computing node 106. In particular, as previously discussed, public memory 120 can be shared among other physically coupled computing nodes 106, however, upon decoupling, in some cases it can be desired that decoupled computing nodes 106 not have access to certain data.
In accordance therewith, synchronization component 202 can tag data propagated from private memory 120 with an identifier associated with the host computing node 106. Appreciably, both data propagated to a disparate memory store 112 of a disparate computing node 106 as well as data propagated to public memory 118 of the same memory store 112 can be tagged with the owner ID. In one or more aspects of the claimed subject matter, synchronization component 202 can delete data from an associated memory store 112 that is tagged with an owner ID associated with a physically decoupled computing node 106. Deletion of data can be applicable to a variety of applications and can be associated with user settings, licensing agreements, digital rights management (DRM) or the like.
For example, consider the case in which separable display 102 is employed for collaboration in connection with an open document. It might be necessary (or even desired) that a particular computing component 106 have access to the entire contents of the document, even though only a portion of the document might be rendered by a local region 108 associated with that computing component 106. Thus, when the instant computing component 106 is decoupled, all data associated with the open document can be maintained on both portions of what was previously the whole separable display 102. However, a user might not desire other information, say, personal information or information associated with other documents included in the desktop to be transferred to the decoupled computing node 106. In that case, synchronization component 202 can delete any such data from the decoupled computing node 106. Again, these settings can be configured by the user, or data can be transferred between public memory 118 and private memory 120 for a given computing node 106 based upon manipulation of objects presented by separable display 102.
For example, an icon representing access to a particular document can be moved/copied from one location on separable display 102 to local region 1081. Thus, the contents of the document can be moved or replicated to memory store 1121, potentially to private memory 120, and owner ID for that associated data can be updated to overwrite or jointly include an ID associated with computing node 1061. Thus, when computing node 1061 is detached, data pertaining to the document can be included with the detached portion, while remaining on or being removed from the other portion, depending upon settings. As another example, consider the case in which proprietary software (e.g., an application or operating system) is required for manipulating the document, yet only one copy is licensed, or the document includes copyrighted media for which only one copy is licensed. In those cases, access to stack information or protected content can be provided to all physically coupled nodes 106, since they can collectively operate as a single multi-node computing component 104, but when decoupled, certain licensing issues can arise. Thus, a public (e.g., view only) application can be provided to one of the separated entities, while the full licensed application can remain on the other entity. Similarly, protected content can remain on only one of the entities, while deleted from the other. In either case, the tag provided to certain data by synchronization component 202 can determine the actual results.
Referring now to
In the depicted aspect of the claimed subject matter, system 300 can include synch component 302 that can manage stored data for a memory store of an associated computing node of the computer-based device. In addition synch component 302 can facilitate data transfer between the memory store and a disparate memory store associated with a second computing node of the device. Appreciably, synch component 302 can be substantially similar to that which has been described in connection with synchronization component 202 of
Furthermore, system 300 can include geometry component 304 that can identify division boundaries 306 when a physical portion 310 of separable display 102 is decoupled from a remainder 312 of separable display 102. Physical separation can relate to tearing, dividing, severing, or otherwise uncoupling a portion of separable display 102, which can, but need not, be along previously structured or predefined division boundaries 306. In addition, system 300 can further include decomposition component 308 that can identify a first subset of computing nodes included in the portion 310. Decomposition component 308 can further identify a second set of computing nodes included in remainder 312, potentially based upon division boundaries 306. Appreciably, features of aspects of geometry component 304 and decomposition component 308 can be substantially similar to all or portions of polling component 114 discussed in connection with
The lower portion of
Regarding example remainders 312B and 312C, the depicted representations can be illustrative of the case in which the remainder 312 is single-layered, such that the absence of portion 310 produces an associated gap in the available screen real estate relative to the original separable display 102. In the case of remainder 312B, data from local regions associated with portion 310 is no longer available, either due to settings, licensing or both. However, for example remainder 312C, the information that existed in portion 310 is still maintained by remainder 312C. This remaining data can be reformatted to suit the changed shape as depicted or, alternatively, can maintain the original structure, even if such would result in a change in size or resolution as is the case with portion 310B. In either case, whether 312B or 312C, the associated portion 310 can be described by portion 310A or portion 310B, depending upon what data is maintained by portion 310.
With reference now to
As a first example, power source 402 can be a battery configured as a separable material strip 402A that constitutes lateral edges of separable display 102. In accordance therewith, the depicted bezel surrounding both representations of separable display 102A and 102P can be a battery. Thus, when separated, either along the depicted dotted lines or substantially any lateral edge of separable display 102, a section of power source 402A can be ported along with the decoupled portion, while another segment can remain with the remainder. Additionally or alternatively, power source 402 can be configured as separable material plate 402B coupled to posterior face 102P. In this case, since plate 402B can exist on posterior face 102P so as not to occlude the UI capabilities. Moreover, plate 402B can be mobile even for decoupled portions that do not include a lateral edge of separable display 102. As still another example, power source 402 can be separable photovoltaic film 402C that covers at least one of anterior face 102A or posterior face 102P. In the case of anterior face 102A, separable photovoltaic film 402C can be made of a substantially transparent material. A transparent material can be employed for photovoltaic film 402C that is utilized for posterior face 102P as well, yet would not tend to be necessary.
Furthermore, it should be understood that power source 402 can be constructed or fabricated by way of printing technologies or techniques. In particular, a battery that supplies power to components detailed herein can be printed directly upon or injected into a surface such as paper or a fabric, potentially with the use of special purpose equipment, “ink,” or other materials. In one or more aspects of the claimed subject matter, similar techniques or technologies can be employed to print all or a portion of separable display 102 (potentially including power source 402) onto a surface. The surface can be a special purpose fabric or paper or other material, and can be separable as well along pre-perforated boundaries or otherwise.
Referring specifically to
With reference now to
Moreover, at reference numeral 506, at least a minimum set of elements necessary to operate as an autonomous computing unit as well as operate in unison with other computing units can be included in each computing node. In particular, at least a processor, a memory, and a polling component can be included in each computing node. The polling component can be configured for identifying other computing nodes physically attached (e.g., nodes that are embedded in a connected portion of the separable display) to the separable display to which the host computing node for that particular polling component is attached. In addition, at reference numeral 508, a computing node in the set of computing nodes can be configured for managing UI transactions and/or I/O transactions in connection with a local region of the separable display that is proximal to the computing node. For instance, the location within the separable display at which a given computing node is embedded, as well as a surrounding area, can be managed, potentially exclusively, by that computing node.
Turning now to
Furthermore, at reference numeral 604, the data obtained in connection with reference numeral 602 can be obtained based upon manipulating of objects between a first region of the separable display associated with (e.g., managed by) the first computing node and the second region that is managed by the second computing node. Moreover, at reference numeral 606 a private portion of any data replicated at reference numerals 602 or 604 can be tagged with a node owner ID. Accordingly, data ownership, or licensing or privacy issues can be mitigated in connection with the node owner ID tag as explained below.
In particular, at reference numeral 608, division boundaries can be identified when a physical portion of the separable display is decoupled from a remainder of the separable display. In addition, at reference numeral 610, a set of capabilities of, a set of data available to, a physical location of, or a region of the separable display managed by computing nodes included in the portion and/or the remainder can be determined. Thus, any given computing node can be appraised of all physically coupled neighbors and their associated capabilities and geometry, as well as those that are no longer coupled and the associated space and capabilities that are therefore no longer available. Moreover, with respect to data tagged in connection with reference numeral 606, at reference numeral 612, replicated data tagged with a node owner ID associated with a computing node included in the remainder can be deleted from computing nodes included in the portion.
Referring to
As another example, at reference numeral 706, the power source can be configured as a separable material cover attached to a posterior side of the separable display. In this case, substantially any portion of the display can be powered even when separated from other portions. In one or more aspects of the claimed subject matter, the power source can be configured as a separable photovoltaic film that coats the posterior side or an anterior side of the separable display, with is detailed in connection with reference numeral 708. Accordingly, ambient light can provide power for any portion of the separable display, which can be produced as a transparent film so as not to obscure or occlude information presented on the anterior face. As another example, at reference numeral 710, the power source included in a remainder of the separable display can be utilized to wirelessly power a decoupled portion of the separable display. Additional features relating to wireless power or energy transmission are discussed in connection with
Referring now to
For example, host display 806 can be attached to host device 802, connected in a wired fashion, or connected wirelessly. Host display 806 can operate as a UI for application 804 that is executed by host device 802, and can thus display visible features of application UI 808. Appreciably, host display 806 can be a touch-screen or other tactile-based display sensitive to contact or gestures, or substantially any type of display surface. Moreover, host display 806 can include or be operatively coupled to other I/O components (e.g., speakers, microphone, keyboard or keypad, network connection . . . ). Hence, host display 806 can present or expose additional features of application UI 808 beyond visual output, such as sound or acoustic aspects of UI 808, connectivity features or apps, as well as interactivity by way of various inputs.
In addition, system 800 can include composition component 810 that can detect second display 812. Similar to host display 806, second display 812 can be substantially any type of display surface, can include additional I/O features or capabilities, and can be operatively coupled to second device 814, for which second display 812 can serve as an UI component for second device 814. Thus, both host display 806 and second display 812 can represent, inter alia, independent display surfaces for associated computing devices (e.g., host device 802 and second device 814, respectively). In many cases, typical computing devices such as laptop computers, tablets, cellular phones, smart phones, digital cameras, media players or recorders, televisions and so forth include a display of some form factor as a single, composite unit. In other cases, as with personal or desktop computers, gaming consoles or the like, the associated display (e.g. a monitor or television) can be a disparate device. In still other cases, as known in the art of surface computing, specialized surfaces, often with a specialized UI that leverages intuitive, hands-on manipulation of objects can represent one or both host display 806 or second display 812. Appreciably, as with host display 806, second display 812 can be all or a portion of separable display 102 such as, e.g., portion 310 or remainder 312.
In the domain of surface computing, the associated displays are generally touch-sensitive screens of substantially any form factor that often forego many traditional I/O devices such as a keyboard or mouse in favor of tactile-based manipulation. However, such I/O devices, if available and/or preferred can of course be employed. In accordance therewith, many ordinary surfaces of today such as tabletops, desks, walls, kiosks, billboards, and so forth can be envisioned to be replaced by interactive surface-based computing displays. Moreover, in addition to large hardware, suitable display surfaces can be developed to be or to simulate paper or fabric. Thus, a paper-like surface that can be folded and placed in one's pocket (or a handkerchief or a fabric worn as apparel) can operate as a display surface or UI for a computer-based application. Such surfaces can themselves be an independent computing unit, with processing features, memory, communication apparatuses, and a power source, or be reliant to some degree on an associated computing device.
For example, advances in miniaturization have enabled such displays to include suitable processors, memory, and communication devices, however, power sources remain today relatively bulky and insufficient for many purposes. Thus, in accordance with one or more aspects of the subject claims, the power source for a given display can be migrated to an associated device. In particular, host display 806 or second display 812 can be powered by an associated device (e.g. host device 802 or second device 814) by way of wireless power transmission and/or wireless energy transfer. Such implementations can include but are not limited to near field techniques such as induction or resonant induction, as well as far field techniques such as radio, microwave, or laser “beaming.” As discussed herein, near field techniques will typically be most suitable. For instance, resonant induction can achieve 75% efficiency at a distance of 80 centimeters or less. In addition, to provide more flexibility or to conserve power, processing or memory can also be farmed out to the associated device, whereby data is exchanged by well-known wireless communication techniques.
It should be understood that host display 806 and second display 812 can be a display from any of the above-mentioned classes of displays or other suitable displays. However, in many instances, the claimed subject matter can relate particularly to the domain of surface computing. Moreover, host display 806 (and second display 812) can include all or a portion of the features normally associated with host device 802 (e.g. processing, memory, power source), or such features can be remote as expressly depicted. Regardless of the particular implementation, composition component 810 can extend application UI 808 to second device 812 based upon a determined display surface geometry, which is further discussed in connection with
While still referring to
The display surface geometry can include a shape of second display 812, a size of second display 812, either absolute or relative to the host display 806, a location of second display 812 relative to host display 806, an orientation of second display 812 relative to host display 806 or the like. As one example, second display 812 can be an adjacent display that shares mutual border 902 with host display 806. Thus, when two or more displays make contact along the boundaries of respective display surfaces, the displays can be characterized as adjacent displays exemplified by diagram 900. In another example, second display 812 can be an overlapping display that overlays all or a portion of host display 806, which is depicted by diagram 910. In contrast, diagram 920 illustrates a situation in which physical contact between the multiple displays does not occur. Rather, second display 812 is a proximal display that is in close proximity to host display 806.
Still referring to
Of course, UPnP 816 and server clustering 818 are merely exemplary technologies that can be employed in connection with the claimed subject matter. Appreciably, other techniques or technologies are envisioned or can be suitable as well. Regardless of how the network or cluster is established between host display 806 (and/or host device 802) and second display 812 (and/or second device 814) as well as additional members, each member can be detected or discovered based upon aspects of proximity. For example, Near Field Communication (NFC), which is inherently low power and short range, can be employed for initial discovery and/or introduction or handshaking. As one example, a NFC packet can carry UPnP 816 discovery algorithm. However, higher bandwidth protocols such as Wi-Fi (IEEE 802.11x specifications) or Bluetooth (IEEE 802.15.x specifications) can be subsequently employed for actual application-based data exchange. Additionally or alternatively, techniques associated with physical contact or line of sight can be employed for proximity-based detection by or for composition component 810.
Next, once one or more disparate displays (e.g. second display 812) are detected, composition component 810 can extend application UI 808 to the disparate displays based upon a determined display surface geometry. Regardless of the techniques or technologies employed (e.g., 816, 818), additional detail relating to determining display surface geometry is provided in connection with
Turning now to
Logistics component 1002 can provide display surface geometry 1004 to composition component 810 by way of a wide range of techniques or technologies, such as transceivers 1008, transducers 1010, or even operator input 8012. For instance, logistics component can automatically determine display surface geometry 1004 based upon a set of transceivers 1008. Such transceivers 1008 can operate in connection with electromagnetic radiation (e.g., light) and can employ techniques associated with trilateration, triangulation, or multipath propagation in order to ascertain the positions of each display relative to others. Likewise, the same techniques can be employed in connection with transducers that are sensitive to acoustic vibration (e.g., sound) in order to ascertain the positions of each display. For example, vibrations that result from physical contact between the displays can be employed to ascertain relative positions. Additionally or alternatively, displays adequately equipped can broadcast an acoustic beacon (e.g., by way of ultrasound or the like) with a particular signature that when received by host display 806 at disparate receivers can be employed to identify the location of the beacon source relative to host display 806. With the position on second display 812 of the acoustic emitter precisely defined, the exact display surface geometry can be ascertained. The above-mentioned techniques can be employed in connection with various topologies exemplified by diagrams 900, 910, or 920.
As another example, a set of tactile-based or piezoelectric transducers (e.g., transducers 1010) can be employed to identify when contact is made between displays as well as the scope or boundaries of such contact. For instance, some or all of the displays can be equipped with a thin magnetic or piezoelectric strip around the edges or other areas of the display, which can sense when and where contact is made. Generally, these techniques can be applicable to adjacent displays exemplified by diagram 900, but can also in some cases be utilized in connection with overlapping displays illustrated by diagram 910.
In one or more aspects of the claimed subject matter, a set of Hall Effect transducers 1010 can be employed to identify features of a magnetic field associated with second display 812. For instance, proximity and position of the second display 812 can be determined based upon a resident magnetic field surrounding the display. Hall effect transducers can be most pertinent to overlapping displays detailed in connection with diagram 910, but can also be employed for adjacent displays such as those exemplified by diagram 900. Another example set of transducers 1010 that can be utilized for overlapping displays can be a set of photodetectors. For example, a thin, transparent layer of photodetectors can exist in host display 806 (or other displays). Thus, based upon a change in the light density of portions overlaid by overlapping displays, display surface geometry 1004 can be determined.
In one or more aspects, logistics component 1002 can determine display surface geometry 1004 based upon operator input 1012. For example, operator input 1012 can relate to a gesture indicative of display surface geometry, such as a gesture that traces the boundaries of overlapping displays or the mutual border(s) for adjacent displays. As another example, operator input 1012 can be a substantially straight line or another shape or mark that spans portions of both host display 806 and second display 812. From such input, the relative topology can be determined as the line spans coordinates that are known to be adjacent between disparate displays. In another aspect, operator input can be explicit coordinate input of substantially any suitable variety. Appreciably, the above examples are intended to provide concrete illustrations, but are not necessarily intended to limit the scope of all appended claims. Rather, other types of sensors, operator input, or means of determining display surface geometry 1004 can also be suitable in connection with the claimed subject matter.
Referring now to
The capabilities enumerated by facility component 1102 can include a set of I/O devices or sensors such as, e.g. keys, a mouse, speakers, microphones, biometric devices, cameras and so on. The capabilities can also include network capabilities (e.g., Wi-Fi, Bluetooth, cellular . . . ), or relevant dimensions or device specifications (e.g., size or shape of associated display, screen resolution or colors, as well as location of relevant features such as speakers/emitters). A substantially similar list enumerating capabilities 1108 of member devices (e.g. 814, 1106) can be constructed or received by facility component 1102 as well.
Furthermore, facility component 1102 can transmit host capabilities list 1104 and member capabilities list 1110 to second device 814 as well as to other member devices 1106. These lists 1104 and 1110 can be combined into a single list or transmitted separately. Moreover, facility component 1102 can leverage UPnP in order to effectuate discovery of the capabilities of all connected devices as discussed supra. Regardless, it should be readily appreciated that an ongoing combined capabilities list or table can be maintained in real time, with capabilities added or removed as member devices join or disconnect. In addition, based upon the available capabilities, composition component 810, in one or more aspects of the claimed subject matter, can expose additional features of application 804, e.g., when certain capabilities are annexed from member devices (e.g., devices 814 or 1106).
Turning now to
Given application object 1202 is now in the process of being presented on second display 812, composition component 810 can facilitate storage of application object data 1204 to second device 814 upon transition from host display 806 to second display 812. Likewise, in the case where application object 1202 is moved from second display 812 to host display 806, composition component can facilitate storage of data 1204 related to application object to host device 802. Typically, upon such transitions, associated data 1204 can be duplicated, however, it should be understood that in some cases, data 1204 can be moved without duplication. Appreciably, what is described herein can facilitate a wide range of use scenarios, many of which can now be described.
For instance, since displays (as well as associated devices) can connect or disconnect in an ad hoc or capricious fashion, potentially at the whim of an operator, the claimed subject matter can provide a natural or intuitive way of understanding and/or tracking data access or flow. Such features are readily appreciated when moving application object 1202 from one display to another, with an inherent understanding that associated data 1204 is now accessible to second device 814, and can (but need not) persist on the second device 814 when or if second display 812 is disconnected or undocked. In some cases, an operator of second device 814 can manipulate object data 1204 (e.g., make changes to a document), then send application object 1202 back to host display 806. Since application object 1202 originated from host display 806, the transition in this case can invoke a move operation rather than a copy operation.
Drawing once more from the domain of surface computing, consider an interactive home or office desktop, where the conventional PC is not a series of boxy artifacts existing on the desk, but rather integrated therein. All or portions of the interactive surface can be overlaid by composable surfaces, such that in the base state many modular displays akin to paper-like sheets or tablets overlay the desk. While the desk itself is an interactive surface, each display can seamlessly overlay a portion of the desk, and present an identical UI. Thus, when interacting with a document, the document can be moved to the area of the desk that is occupied by an overlaying sheet or other display. Then the sheet can be separated from the desk, and the document can remain visible on the detached display (and potentially remain on the lower layer as well, either on the desk surface or another intervening display). In this scenario, an operator can now lean back in her chair or otherwise assume a more comfortable position while working with the document. Moreover, the operator can carry the detached display to another desk or suitable device and share the document, potentially allowing a collaborative session with respect to the document.
In another aspect, public surfaces, e.g. a kiosk, can be available that allow various types of interaction, such as browsing a store catalog or current sales, viewing maps or points of interest, obtaining information about various sites or monuments, etc. A user with, say, a smart phone can bring the smart phone in close proximity to the billboard, then move objects from the kiosk to the phone, for instance, advertisements for products, maps, media guides and so forth. Appreciably, these and other features can be accomplished without installing special software. Rather, UPnP can be employed or other types of connectivity handshaking or introductions such as sending an email or Short Message Service (SMS) message to an address displayed by the surface, and then receiving information in return for establishing a connection; or employing the phone to take a picture of a barcode or address displayed by the surface to facilitate a connection. Additionally or alternatively, once a connection is established, features of the phone, e.g. a keyboard or network connection as well as resident information, can be utilized in connection with the application running on the host surface.
As still another example, a collection of users can physically bring together multiple displays from, say, a tablet PC, a couple cellular phones, and several laptop computers. These collections of multiple displays can not only provide a larger use space or surface, but can also be readily conducive to collaborative endeavors. Moreover, given the composition of multiple devices, capabilities of one device can be shared. For example, consider an application that requires a fingerprint scan for authorization or authentication. However, the party to be authorized is utilizing one of the cellular phones that is not equipped with any biometric devices. Fortunately, one of the connected laptop computers does have an attached fingerprint scanner that can be employed.
Now turning to
Additionally, logistics component 1002 can intelligently determine display surface geometry 1004 based upon suitable approximations as well as a variety of potential parameters that might not always agree or align. Furthermore, facility component 1102 can make intelligent determination or inferences with respect to enumerating capabilities. For example, in some cases or circumstance, certain features might not be available or sharable, in which case, the associated features can (but need not) be ignored or dropped from a capabilities list. Appreciably, any of the foregoing intelligent determinations or inferences can potentially be based upon, e.g., Bayesian probabilities or confidence measures or based upon machine learning techniques related to historical analysis, feedback, and/or other determinations or inferences.
In addition, system 1300 can also include intelligence component 1302 that can provide for or aid in various inferences or determinations. In particular, in accordance with or in addition to what has been described supra with respect to intelligent determinations or inferences provided by various components described herein. For example, all or portions of composition component 810, logistics component 1002, and facility component 1102 can be operatively coupled to intelligence component 1302. Additionally or alternatively, all or portions of intelligence component 1302 can be included in one or more components described herein.
Moreover, intelligence component 1302 will typically have access to all or portions of data sets described herein, such as data store 1304. Data store 1304 is intended to be a repository of all or portions of data, data sets, or information described herein or otherwise suitable for use with the claimed subject. Data store 1304 can be centralized, either remotely or locally cached, or distributed, potentially across multiple devices and/or schemas. Furthermore, data store 1304 can be embodied as substantially any type of memory, including but not limited to volatile or non-volatile, sequential access, structured access, or random access and so on. It should be understood that all or portions of data store 1304 can be included in system 800, or can reside in part or entirely remotely from system 800.
Accordingly, in order to provide for or aid in the numerous inferences described herein, intelligence component 1302 can examine the entirety or a subset of the data available and can provide for reasoning about or infer states of the system, environment, and/or user from a set of observations as captured via events and/or data. Inference can be employed to identify a specific context or action, or can generate a probability distribution over states, for example. The inference can be probabilistic—that is, the computation of a probability distribution over states of interest based on a consideration of data and events. Inference can also refer to techniques employed for composing higher-level events from a set of events and/or data.
Such inference can result in the construction of new events or actions from a set of observed events and/or stored event data, whether or not the events are correlated in close temporal proximity, and whether the events and data come from one or several event and data sources. Various classification (explicitly and/or implicitly trained) schemes and/or systems (e.g. naïve Bayes, Bayesian networks, decision trees, neural networks, fuzzy logic models, and probabilistic classification models providing different patterns of independence can be employed. Classification as used herein also is inclusive of statistical regression that is utilized to develop models of priority.
With reference now to
Moreover, at reference numeral 1404, a host display that is operatively coupled to the host device can be utilized as a UI component for the application. In other words, the host display constitutes at least one portion of a UI adapted to present or interact with the application. For example, the host display can present visualizations associated with application and receive tactile-based inputs such as touch or gestures. Appreciably, other UI components (e.g., speakers, microphone, keys/buttons . . . ) can exist as well, either distinct from or embedded in the host display.
At reference numeral 1406, a second display can be identified based at least in part on physical proximity to the host display, wherein the second display operates as a UI component for a disparate second device. At reference numeral 1408, the UI for the application can be extended to the second display based upon a determined display geometry. The determined display geometry can be employed for understanding where the boundaries of the second display reside, especially relative to the host display.
Referring to
Next to be described, at reference numeral 1506, the second display identified at reference numeral 1406 of
At reference numeral 1510, the display surface geometry can be determined automatically based upon at least one of a set of transceivers, a set of transducers, other suitable sensors or devices, or combinations thereof. Thus, in addition to dimension or other size or shape data that can be provided during initial communications, relative positions can be automatically determined, potentially based upon characteristics of light or acoustics, as detailed in several examples supra. At reference numeral 1512, the display surface geometry can be determined based upon input from an operator. Such input can include boundary tracing gestures, indicia (e.g., lines or shapes) that spans portions of both the host display and the second display, or coordinate parameters of substantially any type of variety.
With reference now to
Accordingly, at reference numeral 1604, one or more capabilities lists can be updated as members connect or disconnect. Appreciably, the combined list or lists can be kept current in real time on the host device. Moreover, at reference numeral 1606, the capabilities list(s) can be transmitted to each member device when capabilities the list(s) are updated, typically due to member connection or disconnection. Thus, the capabilities of the combined devices can be kept current on member devices as well as the host device.
Based upon the above-mentioned data, at reference numeral 1608, additional features of the application can be exposed when capabilities are added from member devices. For example, newly added I/O components, network capabilities, or the like can be employed to operate on substantially any portion of the combined UI. Likewise, at reference numeral 1610, seamless visual transition of application objects can be facilitated between the host display and the second display. Moreover, at reference numeral 1612, storage of data associated with the application object, either through duplication or moving transactions, can be facilitated. In particular, upon a transition of the application object between the host display and the second display, associated data can be stored to the host device or the second device.
Referring now to
Generally, program modules include routines, programs, components, data structures, etc., that perform particular tasks or implement particular abstract data types. Moreover, those skilled in the art will appreciate that the inventive methods can be practiced with other computer system configurations, including single-processor or multiprocessor computer systems, minicomputers, mainframe computers, as well as personal computers, hand-held computing devices, microprocessor-based or programmable consumer electronics, and the like, each of which can be operatively coupled to one or more associated devices.
The illustrated aspects of the claimed subject matter may also be practiced in distributed computing environments where certain tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote memory storage devices.
A computer typically includes a variety of computer readable media. Computer readable media can be any available media that can be accessed by the computer and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer-readable media can comprise computer storage media and communication media. Computer storage media can include both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by the computer.
Communication media typically embodies computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism, and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of the any of the above should also be included within the scope of computer-readable media.
With reference again to
The system bus 1708 can be any of several types of bus structure that may further interconnect to a memory bus (with or without a memory controller), a peripheral bus, and a local bus using any of a variety of commercially available bus architectures. The system memory 1706 includes read-only memory (ROM) 1710 and random access memory (RAM) 1712. A basic input/output system (BIOS) is stored in a non-volatile memory 1710 such as ROM, EPROM, EEPROM, which BIOS contains the basic routines that help to transfer information between elements within the computer 1702, such as during start-up. The RAM 1712 can also include a high-speed RAM such as static RAM for caching data.
The computer 1702 further includes an internal hard disk drive (HDD) 1714 (e.g., EIDE, SATA), which internal hard disk drive 1714 may also be configured for external use in a suitable chassis (not shown), a magnetic floppy disk drive (FDD) 1716, (e.g., to read from or write to a removable diskette 1718) and an optical disk drive 1720, (e.g., reading a CD-ROM disk 1722 or, to read from or write to other high capacity optical media such as the DVD). The hard disk drive 1714, magnetic disk drive 1716 and optical disk drive 1720 can be connected to the system bus 1708 by a hard disk drive interface 1724, a magnetic disk drive interface 1726 and an optical drive interface 1728, respectively. The interface 1724 for external drive implementations includes at least one or both of Universal Serial Bus (USB) and IEEE1394 interface technologies. Other external drive connection technologies are within contemplation of the subject matter claimed herein.
The drives and their associated computer-readable media provide nonvolatile storage of data, data structures, computer-executable instructions, and so forth. For the computer 1702, the drives and media accommodate the storage of any data in a suitable digital format. Although the description of computer-readable media above refers to a HDD, a removable magnetic diskette, and a removable optical media such as a CD or DVD, it should be appreciated by those skilled in the art that other types of media which are readable by a computer, such as zip drives, magnetic cassettes, flash memory cards, cartridges, and the like, may also be used in the exemplary operating environment, and further, that any such media may contain computer-executable instructions for performing the methods of the claimed subject matter.
A number of program modules can be stored in the drives and RAM 1712, including an operating system 1730, one or more application programs 1732, other program modules 1734 and program data 1736. All or portions of the operating system, applications, modules, and/or data can also be cached in the RAM 1712. It is appreciated that the claimed subject matter can be implemented with various commercially available operating systems or combinations of operating systems.
A user can enter commands and information into the computer 1702 through one or more wired/wireless input devices, e.g. a keyboard 1738 and a pointing device, such as a mouse 1740. Other input devices (not shown) may include a microphone, an IR remote control, a joystick, a game pad, a stylus pen, touch screen, or the like. These and other input devices are often connected to the processing unit 1704 through an input device interface 1742 that is coupled to the system bus 1708, but can be connected by other interfaces, such as a parallel port, an IEEE1394 serial port, a game port, a USB port, an IR interface, etc.
A monitor 1744 or other type of display device is also connected to the system bus 1708 via an interface, such as a video adapter 1746. In addition to the monitor 1744, a computer typically includes other peripheral output devices (not shown), such as speakers, printers, etc.
The computer 1702 may operate in a networked environment using logical connections via wired and/or wireless communications to one or more remote computers, such as a remote computer(s) 1748. The remote computer(s) 1748 can be a workstation, a server computer, a router, a personal computer, a mobile device, portable computer, microprocessor-based entertainment appliance, a peer device or other common network node, and typically includes many or all of the elements described relative to the computer 1702, although, for purposes of brevity, only a memory/storage device 1750 is illustrated. The logical connections depicted include wired/wireless connectivity to a local area network (LAN) 1752 and/or larger networks, e.g. a wide area network (WAN) 1754. Such LAN and WAN networking environments are commonplace in offices and companies, and facilitate enterprise-wide computer networks, such as intranets, all of which may connect to a global communications network, e.g. the Internet.
When used in a LAN networking environment, the computer 1702 is connected to the local network 1752 through a wired and/or wireless communication network interface or adapter 1756. The adapter 1756 may facilitate wired or wireless communication to the LAN 1752, which may also include a wireless access point disposed thereon for communicating with the wireless adapter 1756.
When used in a WAN networking environment, the computer 1702 can include a modem 1758, or is connected to a communications server on the WAN 1754, or has other means for establishing communications over the WAN 1754, such as by way of the Internet. The modem 1758, which can be internal or external and a wired or wireless device, is connected to the system bus 1708 via the serial port interface 1742. In a networked environment, program modules depicted relative to the computer 1702, or portions thereof, can be stored in the remote memory/storage device 1750. It will be appreciated that the network connections shown are exemplary and other means of establishing a communications link between the computers can be used.
The computer 1702 is operable to communicate with any wireless devices or entities operatively disposed in wireless communication, e.g., a printer, scanner, desktop and/or portable computer, portable data assistant, communications satellite, any piece of equipment or location associated with a wirelessly detectable tag (e.g., a kiosk, news stand, restroom), and telephone. This includes at least Wi-Fi and Bluetooth™ wireless technologies. Thus, the communication can be a predefined structure as with a conventional network or simply an ad hoc communication between at least two devices.
Wi-Fi, or Wireless Fidelity, allows connection to the Internet from a couch at home, a bed in a hotel room, or a conference room at work, without wires. Wi-Fi is a wireless technology similar to that used in a cell phone that enables such devices, e.g. computers, to send and receive data indoors and out; anywhere within the range of a base station. Wi-Fi networks use radio technologies called IEEE802.11 (a, b, g, etc.) to provide secure, reliable, fast wireless connectivity. A Wi-Fi network can be used to connect computers to each other, to the Internet, and to wired networks (which use IEEE802.3 or Ethernet). Wi-Fi networks operate in the unlicensed 2.4 and 5 GHz radio bands, at an 17 Mbps (802.11b) or 54 Mbps (802.11a) data rate, for example, or with products that contain both bands (dual band), so the networks can provide real-world performance similar to the basic “10 BaseT” wired Ethernet networks used in many offices.
Referring now to
The system 1800 also includes one or more server(s) 1804. The server(s) 1804 can also be hardware and/or software (e.g. threads, processes, computing devices). The servers 1804 can house threads to perform transformations by employing the claimed subject matter, for example. One possible communication between a client 1802 and a server 1804 can be in the form of a data packet adapted to be transmitted between two or more computer processes. The data packet may include a cookie and/or associated contextual information, for example. The system 1800 includes a communication framework 1806 (e.g., a global communication network such as the Internet) that can be employed to facilitate communications between the client(s) 1802 and the server(s) 1804.
Communications can be facilitated via a wired (including optical fiber) and/or wireless technology. The client(s) 1802 are operatively connected to one or more client data store(s) 1808 that can be employed to store information local to the client(s) 1802 (e.g. cookie(s) and/or associated contextual information). Similarly, the server(s) 1804 are operatively connected to one or more server data store(s) 1810 that can be employed to store information local to the servers 1804.
What has been described above includes examples of the various embodiments. It is, of course, not possible to describe every conceivable combination of components or methodologies for purposes of describing the embodiments, but one of ordinary skill in the art may recognize that many further combinations and permutations are possible. Accordingly, the detailed description is intended to embrace all such alterations, modifications, and variations that fall within the spirit and scope of the appended claims.
In particular and in regard to the various functions performed by the above described components, devices, circuits, systems and the like, the terms (including a reference to a “means”) used to describe such components are intended to correspond, unless otherwise indicated, to any component which performs the specified function of the described component (e.g. a functional equivalent), even though not structurally equivalent to the disclosed structure, which performs the function in the herein illustrated exemplary aspects of the embodiments. In this regard, it will also be recognized that the embodiments includes a system as well as a computer-readable medium having computer-executable instructions for performing the acts and/or events of the various methods.
In addition, while a particular feature may have been disclosed with respect to only one of several implementations, such feature may be combined with one or more other features of the other implementations as may be desired and advantageous for any given or particular application. Furthermore, to the extent that the terms “includes,” and “including” and variants thereof are used in either the detailed description or the claims, these terms are intended to be inclusive in a manner similar to the term “comprising.”