Patent Application
20160078840
The present invention relates generally to touch screen technologies, and more particularly, to methods and systems for connecting multiple mobile devices to form a combined virtual touch screen in an easy and intuitive manner.
Driven by the popularity of smartphones, tablets and various information appliances, the display technology in connection with touch screens has advanced tremendously in recent years. As a result, mobile or portable devices, personal computers, TV and many other devices nowadays can be configured with screen displays having narrow edges or no edges, so-called frameless displays. This makes it possible to combine the displays of multiple devices into a single display. For example, if two devices having frameless displays are placed side by side, their displays may be combined to form one big virtual screen with no space or gap in between two displays, thereby providing better viewing experiences for users. However, how to connect the two devices and combine their separate displays into one virtual screen for users to easily operate can be very difficult and complex in terms of actual implementations. Most existing technologies require multiple steps and user inputs to synchronize and consolidate different devices before their displays can be combined into one screen. Therefore, a need exists for an improved solution to this problem.
The presently disclosed embodiments are directed to solving issues relating to one or more of the problems presented in the prior art, as well as providing additional features that will become readily apparent by reference to the following detailed description when taken in conjunction with the accompanying drawings.
In one aspect, the present invention provides a method for connecting multiple mobile devices each of which has a touch screen display, comprising: receiving data from a first device and a second device, respectively, said data including a touch event detected by said first and second devices; based on said received data, determining that said first and second devices are to be connected; and generating instructions for said first and second devices to connect.
In another aspect, the present invention provides a device comprising: a display integrated with a touch screen, said touch screen configured to detect touch inputs in the device; a processor coupled to said touch screen; and a memory accessible to said processor, said memory storing processor-executable instructions, wherein said instructions, while executed, cause said processor to perform: receiving touch information of a touch event detected in said touch screen; sending said touch information and device information to a cloud server, said device information including at least a location of the device; receiving instructions from said cloud server; and based on said instructions, activating a connection mode allowing data exchange with one or more of other devices that are identified by said cloud server.
In another aspect, the present invention provides a device comprising: a display integrated with a touch screen, said touch screen configured for detecting touch inputs in the device; a memory storing process-executable instructions; and a processor having access to said memory, said processor configured for: receiving first touch information of a first touch event detected in said touch screen; receiving second touch information of a second touch event detected in a second device; receiving device information from said second device; and based on said first and second touch information and device information, determining whether to connect said device to said second device.
In another aspect, the present invention provides a non-transitory computer-readable medium comprising processor-executable instructions, which, while executed, cause a processor to perform: receiving data from a first device having a first display and a second device having a second display; based on said received data, determining to combine said first and second displays into a combined virtual screen display; and generating instructions for said first and second devices to connect and form said combined virtual screen display, wherein said data include a touch event detected by said first and second devices.
Further features and advantages of the present disclosure, as well as the structure and operation of various embodiments of the present disclosure, are described in detail below with reference to the accompanying drawings.
The present disclosure, in accordance with one or more various embodiments, is described in detail with reference to the following figures. The drawings are provided for purposes of illustration only and merely depict exemplary embodiments of the disclosure. These drawings are provided to facilitate the reader's understanding of the disclosure and should not be considered limiting of the breadth, scope, or applicability of the disclosure. It should be noted that for clarity and ease of illustration these drawings are not necessarily made to scale.
The following description is presented to enable a person of ordinary skill in the art to make and use the invention. Descriptions of specific devices, techniques, and applications are provided only as examples. Various modifications to the examples described herein will be readily apparent to those of ordinary skill in the art, and the general principles defined herein may be applied to other examples and applications without departing from the spirit and scope of the invention. Thus, embodiments of the present invention are not intended to be limited to the examples described herein and shown, but is to be accorded the scope consistent with the claims.
The word “exemplary” is used herein to mean “serving as an example or illustration.” Any aspect or design described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects or designs.
Reference will now be made in detail to aspects of the subject technology, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like elements throughout.
It should be understood that the specific order or hierarchy of steps in the processes disclosed herein is an example of exemplary approaches. Based upon design preferences, it is understood that the specific order or hierarchy of steps in the processes may be rearranged while remaining within the scope of the present disclosure. The accompanying method claims present elements of the various steps in a sample order, and are not meant to be limited to the specific order or hierarchy presented.
Embodiments disclosed herein are directed to methods and systems for connecting multiple mobile devices to form a combined virtual screen display in an easy and intuitive manner. In one embodiment, this method comprises the steps of receiving data from a first device and a second device, each of said first and second devices having a screen display; based on said received data, determining to combine the screen displays of said first and second devices; and generating instructions for said first and second devices to connect and form a combined virtual screen display, wherein said data include a touch event detected by said first and second devices.
As partial implementation of the embodiments, a device is configured to include the following: a display integrated with a touch screen, said touch screen configure for detecting touch inputs in the device; a processor coupled to said touch screen; and a memory accessible to said processor, said memory storing processor-executable instructions, wherein said instructions, while executed, cause said processor to perform: receiving touch information of a touch event detected in said touch screen; sending said touch information to a cloud server; receiving instructions from said cloud server; and based on said instructions, activating a screen-combining mode allowing said display to be combined with displays of other devices.
Referring to
The devices 120 may comprise various smartphones such as iPhone, Android phones, and Windows phones. However, the devices 120 are not so limited, but may include many other network devices, including a fixed or mobile subscriber unit, a pager, a cellular telephone, a personal digital assistant (PDA), a smart phone, a laptop, a netbook, a tablet computer, a personal computer, a wireless sensor, consumer electronic devices, and the like.
As illustrated in
It should be appreciated that the system 100 in
The memory 220 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM and/or other random access solid state memory devices, and includes non-volatile memory, such as flash memory devices, a magnetic disk storage device, and/or other non-volatile solid state storage devices. The memory 220, or alternately non-volatile memory device(s) within the memory 220, includes a non-transitory computer-readable storage medium.
In some embodiments, the memory 220 stores the following programs, modules and data structures, or a subset thereof: an operating system 222 that includes procedures for handling various basic system services and for performing hardware dependent tasks, communication modules 224 used for communicating with other devices or network controllers, such as a SIM card or phone registration module 224a and a signal processing module 224b, and various applications 226, including one or more downloaded mobile applications, for example, an application allowing the device to be connected with the cloud server as well as other devices to form a combined virtual screen display. Other applications can be included as well, such as social network or messaging applications, security applications and multimedia applications. All these applications may have associated API(s) (not shown) in the memory 220.
The processor 210 is also coupled to one or more motion sensors 230, including an accelerometer 232 for measuring acceleration, a gyroscope 234 for measuring orientation, or a combination thereof, which is sometimes referred as Inertial Measurement Unit (IMU). Besides, a GPS 270 is coupled to the processor 210 for measuring location information. Usually the combination of motion sensors 230 and GPS 270 allows for an accurate measurement of the position of the device.
To determine the location of the device, in addition to or instead of the GPS, various techniques, for example, WiFi access points, Cellular networks and Bluetooth beacons can be implemented.
In addition, the processor 210 is coupled to a user interface 240 by which the processor communicates with external or peripheral devices, including, without limitation, a touch screen 242, a display 244 and a keyboard 246. The touch screen 242 is typically configured with one or more touch sensors underneath. In some embodiments, the touch screen 242, display 244 and keyboard 246 are integrated into one piece, which is typical of today's touch devices, e.g., smartphones. Other peripheral devices coupled to the processor 210 include a camera or video recorder 250 and a microphone or speaker 260. Usually the memory 220 includes software programs or drivers for activating and communicating with each peripheral device.
The processor 210 is further coupled to a Bluetooth or WiFi interface 280 for receiving local network signals, and a communication interface 290 for connecting wireless or wired networks, mostly through an internal component known as transceiver 292.
In one configuration, all different components in
Again, it should be appreciated that the mobile device 200 in
Referring now to
In this process 300, a user may apply a single touch input 340 in the neighboring area of the two devices once they are placed next to each other. This touch input 340 can be detected, almost simultaneously, by each device, or more precisely, the touch sensors embedded underneath the edges of each device. (The manner by which the single touch input in the neighboring area of the two devices is detected will be explained in more detail later with reference to
In some embodiments, the two devices 320 and 330 may establish connections with each other without help from the cloud server 310. For example, in a WiFi environment, the two devices 320 and 330 can directly communicate with each other and decide whether they are placed or aligned in a position ready for combining their displays into one bigger screen.
As seen in
In terms of specific implementations of the above-described functionalities and features, the flow diagram in
As shown in
On the cloud server side, data from both devices are received at steps 520 and 540, respectively. It should be noted that the two steps 520 and 540 can occur simultaneously or in slightly different sequences. Based on the received data, at step 560, the cloud server determines whether the two devices are ready to be combined into one virtual device with one combined screen display. More specifically, the cloud server determines from the data whether the two devices are placed next to each other, whether they are readily aligned, whether each device is enabled with the screen-combining function, whether the touch event detected by each device is a user-intended action to combine the displays, and so forth. Also from the data the cloud server can identify each device by their device profiles, such as their device dimensions, system configurations, etc. In some embodiments, for the purpose of establishing connection between two or more devices, the system can be configured such that these devices need not be aligned horizontally (or vertically or in any particular configuration). In particular, when data transfer or exchange is of interest (i.e., not for establishing a virtual screen), two devices placed side-by-side, but misaligned vertically can receive touch input or sequence of touch inputs to identify the devices.
If the cloud server determines that the two device displays are not to be combined, the process may end at step 590. Otherwise, the process proceeds to step 580, where the cloud server generates and sends instructions for the two devices to connect and combine their individual displays into one virtual screen. In one embodiment, the instructions to one device may include information about the other device, such as screen dimensions, connection interface, and so forth.
Once the devices receive the instructions from the cloud server, they can start the screen-combining process. For example, as shown in
It should be appreciated that the above-described algorithm is for illustration only, and many variations or additional steps may be applied. For example, the two devices can be connected directly in certain network environments without relying on the cloud server. In that case, each device can be configured with such functionalities as determining whether to connect with another device, whether to combine its display with another screen display, how to verify the connection, how to display images in the combined screen, and so forth. In other words, the above-described steps to be performed by the cloud server can be performed by either or both devices.
Further, in some embodiments, in preparation for establishing connections between the two devices, a particular program may be launched in each of the devices. For example, a system can be configured such that unless a particular app is launched in respective smartphones operating under an appropriate OS (such as Android, Windows, or iOS), the smartphones do not send the relevant information to the cloud server 310 (or to adjacent smartphones in case of direct connections) so as to establish the connections.
Also, it should be understood that the algorithm in
In
Alternatively, multiple devices can be connected and combined as shown in
In some embodiments, a user can apply a single touch to connect multiple devices to form a combined virtual screen. In other embodiments, a user may apply a sequence of touches, such as the touches 630 in
After multiple devices are connected to form a combined virtual screen display, a user can operate the combined devices as if it is a single device. This is demonstrated in
Further, at step 830, in addition to or instead of the touch sharing, as described above, any data exchanges or transfer of files/images can be performed between the multiple devices. Moreover, in some embodiments, the virtual screen may not be needed. Once the multiple devices are associated by one or more of the schemes, as described above, these devices are connected (via a cloud server or directly). Thus, data/file exchange or unidirectional data/file transfer can be performed between the connected devices in addition to or instead of establishing a virtual screen.
Again, it should be appreciated that the above-described algorithm is for illustration only, and many variations or additional steps may be applied.
With reference to
While various embodiments of the invention have been described above, it should be understood that they have been presented by way of example only, and not by way of limitation. Likewise, the various diagrams may depict an example architectural or other configuration for the disclosure, which is done to aid in understanding the features and functionality that can be included in the disclosure. The disclosure is not restricted to the illustrated example architectures or configurations, but can be implemented using a variety of alternative architectures and configurations. Additionally, although the disclosure is described above in terms of various exemplary embodiments and implementations, it should be understood that the various features and functionality described in one or more of the individual embodiments are not limited in their applicability to the particular embodiment with which they are described. They instead can be applied alone or in some combination, to one or more of the other embodiments of the disclosure, whether or not such embodiments are described, and whether or not such features are presented as being a part of a described embodiment. Thus the breadth and scope of the present disclosure should not be limited by any of the above-described exemplary embodiments.
In this document, the term “module” as used herein, refers to software, firmware, hardware, and any combination of these elements for performing the associated functions described herein. Additionally, for purpose of discussion, the various modules are described as discrete modules; however, as would be apparent to one of ordinary skill in the art, two or more modules may be combined to form a single module that performs the associated functions according embodiments of the invention.
In this document, the terms “computer program product”, “computer-readable medium”, and the like, may be used generally to refer to media such as, memory storage devices, or storage unit. These and other forms of computer-readable media may be involved in storing one or more instructions for use by processor to cause the processor to perform specified operations. Such instructions, generally referred to as “computer program code” (which may be grouped in the form of computer programs or other groupings), when executed, enable the computing system.
It will be appreciated that, for clarity purposes, the above description has described embodiments of the invention with reference to different functional units and processors. However, it will be apparent that any suitable distribution of functionality between different functional units, processors or domains may be used without detracting from the invention. For example, functionality illustrated to be performed by separate processors or controllers may be performed by the same processor or controller. Hence, references to specific functional units are only to be seen as references to suitable means for providing the described functionality, rather than indicative of a strict logical or physical structure or organization.
Terms and phrases used in this document, and variations thereof, unless otherwise expressly stated, should be construed as open ended as opposed to limiting. As examples of the foregoing: the term “including” should be read as meaning “including, without limitation” or the like; the term “example” is used to provide exemplary instances of the item in discussion, not an exhaustive or limiting list thereof; and adjectives such as “conventional,” “traditional,” “normal,” “standard,” “known”, and terms of similar meaning, should not be construed as limiting the item described to a given time period, or to an item available as of a given time. But instead these terms should be read to encompass conventional, traditional, normal, or standard technologies that may be available, known now, or at any time in the future. Likewise, a group of items linked with the conjunction “and” should not be read as requiring that each and every one of those items be present in the grouping, but rather should be read as “and/or” unless expressly stated otherwise. Similarly, a group of items linked with the conjunction “or” should not be read as requiring mutual exclusivity among that group, but rather should also be read as “and/or” unless expressly stated otherwise. Furthermore, although items, elements or components of the disclosure may be described or claimed in the singular, the plural is contemplated to be within the scope thereof unless limitation to the singular is explicitly stated. The presence of broadening words and phrases such as “one or more,” “at least,” “but not limited to”, or other like phrases in some instances shall not be read to mean that the narrower case is intended or required in instances where such broadening phrases may be absent.
Additionally, memory or other storage, as well as communication components, may be employed in embodiments of the invention. It will be appreciated that, for clarity purposes, the above description has described embodiments of the invention with reference to different functional units and processors. However, it will be apparent that any suitable distribution of functionality between different functional units, processing logic elements or domains may be used without detracting from the invention. For example, functionality illustrated to be performed by separate processing logic elements, or controllers may be performed by the same processing logic element, or controller. Hence, references to specific functional units are only to be seen as references to suitable means for providing the described functionality, rather than indicative of a strict logical or physical structure or organization.
Furthermore, although individually listed, a plurality of means, elements or method steps may be implemented by, for example, a single unit or processing logic element. Additionally, although individual features may be included in different claims, these may possibly be advantageously combined. The inclusion in different claims does not imply that a combination of features is not feasible and/or advantageous. Also, the inclusion of a feature in one category of claims does not imply a limitation to this category, but rather the feature may be equally applicable to other claim categories, as appropriate.
