Patent Grant
9373302
The present disclosure relates to electronic devices and, more particularly, to methods and electronic devices for identifying a relative position of a top electronic device that has been stacked on top of a bottom electronic device.
Increasingly, electronic device users have access to a plurality of electronic devices of varying types. For example, electronic device users often have access to a smartphone, which they use for ultra-portable computing and for communications, and also have access to a tablet computer, a laptop computer and/or a desktop computer.
Such electronic devices are sometimes capable of interacting with one another. For example, when such electronic devices are near one another, they may send and receive data to one another over a short range protocol, such as Bluetooth.
Electronic devices currently provide limited interactive functions for interacting with other electronic devices.
Like reference numerals are used in the drawings to denote like elements and features.
In one example embodiment, the present disclosure describes a processor implemented method of identifying a location of a top electronic device stacked on top of a bottom electronic device. The top electronic device and the bottom electronic device are configured to communicate. The method includes: receiving camera data associated with a camera of the top electronic device; obtaining display data identifying a pixel arrangement on at least a portion of a display of the bottom electronic device; and identifying the position of the top electronic device relative to the bottom electronic device based on the camera data and the display data.
In yet another example embodiment, the present disclosure describes a top electronic device. The top electronic device includes a camera and a communication subsystem for communicating with a bottom electronic device. The top electronic device also includes a processor configured to: receive camera data associated with a camera of the top electronic device; obtain display data identifying a pixel arrangement on at least a portion of a display of the bottom electronic device; and identify the position of the top electronic device relative to the bottom electronic device based on the camera data and the display data.
In yet another example embodiment, the present disclosure describes a bottom electronic device. The bottom electronic device includes a display and a communication subsystem for communicating with a bottom electronic device. The bottom electronic device also includes a processor configured to: receive camera data associated with a camera of the top electronic device; obtain display data identifying a pixel arrangement on at least a portion of a display of the bottom electronic device; and identify the position of the top electronic device relative to the bottom electronic device based on the camera data and the display data.
Other example embodiments of the present disclosure will be apparent to those of ordinary skill in the art from a review of the following detailed description in conjunction with the drawings.
Referring first to
The bottom electronic device 104 has a display 109A and, at least a portion of the display 109A is obscured by the top electronic device 102 when the electronic devices 102, 104 are in the stacked orientation. That is, the top electronic device 102 sits directly over at least a portion of the display 109A of the bottom electronic device.
In at least some example embodiments, the top electronic device 102 may also include a display 109B.
In the example embodiment illustrated in
In the example illustrated, the top electronic device 102 is smaller than the bottom electronic device 104. More particularly, in the example illustrated, the top side 110 of the bottom electronic device 104 has a greater area than a top side 112 of the top electronic device 102. In the example illustrated, the display 109A of the bottom electronic device 104 is larger than the display 109B of the top electronic device 102 and the display 109A of the bottom electronic device 104 is larger than the top side 112 of the top electronic device 102. In the example embodiments of
In the example illustrated, the top electronic device 102 and the bottom electronic device 104 are substantially rectangular. However, the electronic devices 102, 104 could have other form factors in other example embodiments.
In the example embodiment illustrated, the top electronic device 102 is a smartphone and the bottom electronic device 104 is a tablet computer. However, in other example embodiments, other electronic devices may be used. For example, in at least some example embodiments, the electronic devices may both be smartphones or may both be tablet computers. In various example embodiments, one or more of the electronic devices 102, 104 may be a smartphone, tablet computer, wearable computer (such as a watch), a mobile telephone or PDAs (personal digital assistants), a television, a navigation device (such as a global positioning system device) or a computer system. Accordingly, the electronic devices 102, 104 may take a variety of forms.
A smartphone is a communication device that is typically configured for more advanced computing than a non-smart cellular phone. For example, the smartphone may be configured to allow third party applications to be executed and may be configured to access the Internet and Internet-based services. A tablet computer (which may also be referred to as a tablet) is a mobile computer which is generally larger than a mobile phone (such as a smartphone) or personal digital assistant. Many mobile phones or personal digital assistants are designed to be pocket sized. That is, mobile phones (such as smartphones) or personal digital assistants are generally small enough to be carried by a person easily, often in a shirt or pant pocket while tablet computers are larger and may not fit within pant pockets. For example, many tablet computers have a height with is seven inches (7″) or more.
Referring again to
As will be explained below with reference to
Referring now to
A rear facing camera is provided by a camera 403 which is located to obtain images of a subject near a rear side 401 of the top electronic device 102. The rear side 401 is typically a side which does not include the main display 109B (
In at least some example embodiments, the top electronic device 102 may operate in one operating mode in which the display 109B acts as a viewfinder displaying image data associated with camera data received by a rear facing camera. The rear side 401 of the top electronic device 102 is a side of top electronic device 102 that directly opposes the top side 112 of the top electronic device 102. In the example embodiment illustrated, the rear side 401 is substantially parallel to the top side 112.
The camera 403 is capable of capturing camera data such as images in the form of still photographs and/or motion video. The camera data may be captured in the form of an electronic signal which is produced by an image sensor. The image sensor is configured to produce an electronic signal in dependence on received light. More particularly, the image sensor converts an optical image into an electronic signal that represents electronic image data (which may also be referred to as camera data).
While
In some example embodiments, the top electronic device 102 may include a plurality of rear facing cameras 403. For example, in some example embodiments, the top electronic device may include two or more rear facing cameras 403 which may be used, in at least one operating mode, for capturing three-dimensional images such as stereoscopic images. In at least some such example embodiments, the camera data from more than one camera may be compared to the display data to determine the relative position of the top electronic device 102 relative to the bottom electronic device 104.
Reference will now be made to
The electronic device 501 includes a controller including one or more processor 540 (such as a microprocessor) which controls the overall operation of the electronic device 501. The processor 540 interacts with device subsystems such as a wireless communication subsystem 511 for exchanging radio frequency signals with a wireless network 519 to perform long-range communication functions. The processor 540 is communicably coupled with additional device subsystems including one or more output interfaces 505 (such as a display 109 and/or a speaker 556), one or more input interfaces 506 (such as a camera 403, microphone 558, keyboard (not shown), control buttons (not shown), a touch-sensitive overlay (not shown)) associated with a touchscreen display 109 and/or other input interfaces 506), an orientation subsystem 549, memory (such as flash memory 544, random access memory (RAM) 546, read only memory (ROM) 548, etc.), auxiliary input/output (I/O) subsystems 550, a data port 552 (which may be a serial data port, such as a Universal Serial Bus (USB) data port), a short-range communication subsystem 562 and other device subsystems generally designated as 564. Some of the subsystems shown in
In at least some example embodiments, the electronic device 501 may include a touchscreen display 109 which acts as both an input interface 506 (i.e. touch-sensitive overlay) and an output interface 505 (i.e. display). The touchscreen display may be constructed using a touch-sensitive input surface which is connected to an electronic controller and which overlays the display 109. The touch-sensitive overlay and the electronic controller provide a touch-sensitive input interface 506 and the processor 540 interacts with the touch-sensitive overlay via the electronic controller.
As noted above, in some example embodiments, the electronic device 501 may include a communication subsystem 511 which allows the electronic device 501 to communicate over a wireless network 519. In at least some example embodiments, the electronic device 501 may communicate with any one of a plurality of fixed transceiver base stations of the wireless network 519 within its geographic coverage area. The electronic device 501 may send and receive communication signals over the wireless network 519 after the required network registration or activation procedures have been completed.
In some example embodiments, the auxiliary input/output (I/O) subsystems 550 may include an external communication link or interface; for example, an Ethernet connection. The electronic device 501 may include other wireless communication interfaces for communicating with other types of wireless networks; for example, a wireless network such as an orthogonal frequency division multiplexed (OFDM) network. The auxiliary I/O subsystems 550 may include a vibrator for providing vibratory notifications in response to various events on the electronic device 501 such as receipt of an electronic communication or incoming phone call, or for other purposes such as haptic feedback (touch feedback).
In some example embodiments, the electronic device 501 also includes a removable memory module 530 (typically including flash memory, such as a removable memory card) and a memory interface 532. Network access may be associated with a subscriber or user of the electronic device 501 via the memory module 530, which may be a Subscriber Identity Module (SIM) card for use in a GSM network or other type of memory card for use in the relevant wireless network type. The memory module 530 is inserted in or connected to the memory interface 532 of the electronic device 501 in order to operate in conjunction with the wireless network 519.
The data port 552 may be used for synchronization with a user's host computer system (not shown). The data port 552 may enable a user to set preferences through an external device or software application and extends the capabilities of the electronic device 501 by providing for information or software downloads to the electronic device 501 other than through the wireless network 519.
In at least some example embodiments, the electronic device 501 also includes a device orientation subsystem 549, including at least one orientation sensor 551, which is connected to the processor 540 and which is controlled by one or a combination of a monitoring circuit and operating software. The orientation sensor 551 detects the orientation of the electronic device 501 or information from which the orientation of the electronic device 501 can be determined, such as acceleration. In some example embodiments, the orientation sensor 551 is an accelerometer, such as a three-axis accelerometer. An accelerometer is a sensor which converts acceleration from motion and gravity into an electrical. Accelerometers may be available in one, two or three axis configurations. Higher order axis configurations are also possible. Accelerometers may produce digital or analog output signals depending on the type of accelerometer.
An orientation sensor 551 may generate orientation data which specifies the orientation of the electronic device 501. The orientation data, in at least some example embodiments, specifies the orientation of the device 501 relative to the gravitational field of the earth.
In some example embodiments, the orientation subsystem 549 may include other orientation sensors 551, instead of or in addition to accelerometers. For example, in various example embodiments, the orientation subsystem 549 may include a gravity sensor, a gyroscope, a tilt sensor, an electronic compass or other suitable sensor, or combinations thereof. In some example embodiments, the device orientation subsystem 549 may include two or more orientation sensors 551 such as an accelerometer and an electronic compass.
The electronic device 501 may also include one or more cameras 403. The one or more cameras 403 may be of the type described above with reference to
The electronic device 501 also includes or is connectable to a power source, such as a battery 538, which may be one or more rechargeable batteries that may be charged for example, through charging circuitry coupled to a battery interface 536 such as the data port 552. The battery 538 provides electrical power to at least some of the electrical circuitry in the electronic device 501, and the battery interface 536 provides a mechanical and electrical connection for the battery 538. The battery interface 536 is coupled to a regulator (not shown) which provides power V+ to the circuitry of the electronic device 501.
The electronic device 501 includes a short-range communication subsystem 562 which provides for wireless communication between the electronic device 501 and other electronic devices 501. The short-range communication subsystem 562 may be configured to allow the electronic device 501 to communicate with other electronic devices. For example, a top electronic device 102 (shown in
In at least some example embodiments, the short-range communication subsystem 562 is a wireless bus protocol compliant communication mechanism such as a Bluetooth® communication module to provide for communication with similarly-enabled systems and devices.
The electronic device 501 stores data 527 in an erasable persistent memory, which in one example embodiment is the flash memory 544. In various example embodiments, the data 527 includes service data including information required by the electronic device 501 to establish and maintain communication with the wireless network 519. The data 527 may also include user application data such as email messages, address book and contact information, calendar and schedule information, notepad documents, image files, and other commonly stored user information stored on the electronic device 501 by its user, and other data. The data 527 stored in the persistent memory (e.g. flash memory 544) of the electronic device 501 may be organized, at least partially, into one or more databases or data stores. The databases or data stores may contain data items of the same data type or associated with the same application. For example, email messages, contact records, and task items may be stored in individual databases within the device memory.
The processor 540 operates under stored program control and executes software modules 521 stored in memory such as persistent memory; for example, in the flash memory 544. As illustrated in
As will be described in greater detail below with reference to
The software applications 525 on the electronic device 501 may also include a range of additional applications, including for example, a notepad application, Internet browser application, voice communication (i.e. telephony) application, mapping application, or a media player application, or any combination thereof. Each of the software applications 525 may include layout information defining the placement of particular fields and graphic elements (e.g. text fields, input fields, icons, etc.) in the user interface (e.g. the display 504) according to the application.
The software modules 521 or parts thereof may be temporarily loaded into volatile memory such as the RAM 546. The RAM 546 is used for storing runtime data variables and other types of data or information, as will be apparent to those skilled in the art. Although specific functions are described for various types of memory, this is merely one example, and those skilled in the art will appreciate that a different assignment of functions to types of memory could also be used.
A predetermined set of applications that control basic device operations, including data and possibly voice communication applications will normally be installed on the electronic device 501 during or after manufacture. Additional applications and/or upgrades to the operating system 523 or software applications 525 may also be loaded onto the electronic device 501 through the wireless network 519, the auxiliary I/O subsystem 550, the data port 552, the short-range communication subsystem 562, or other suitable subsystem 564. The downloaded programs or code modules may be permanently installed; for example, written into the program memory (i.e. the flash memory 544), or written into and executed from the RAM 546 for execution by the processor 540 at runtime.
One or both of the top electronic device 102 and the bottom electronic device 104 of
The top electronic device 102 may, in at least some example embodiments include a different subset of the components than the bottom electronic device 104. That is, one or more of the components described with reference to
The discussions which follow will refer to a number of example display screens which may, in various example embodiments, be displayed on a display 109A of a bottom electronic device 104, a display 109B of a top electronic device 104, or both. The top electronic device 102 may be configured to display the display screens which are referred to or illustrated as being provided by the top electronic device 102, and the bottom electronic device 104 may be configured to display the display screens which are referred to or illustrated as being provided by the bottom electronic device 104. More particularly, a processor 540 of the top electronic device 102 may be configured to cause the top electronic device 102 to display the display screens which are referred to or illustrated as being provided by the top electronic device 102 and a processor 540 of the bottom electronic device 104 may be configured to cause the bottom electronic device 104 to display the display screens which are referred to or illustrated as being provided by the bottom electronic device 104.
Referring now to
In the example embodiment of
In the example embodiment illustrated, the locating pattern 602 occupies the entirety of the display 109A associated with the bottom electronic device 104. In the example embodiment illustrated, the top electronic device 102 obscures a portion of the display 109A of the bottom electronic device 104. While not visible from
The locating pattern 602 includes a number of unique regions. That is, the locating pattern 602 is selected such that there are portions of the locating pattern 602 that are unlike other portions of the locating pattern 602. When the camera 403 obtains camera data and the electronic devices 102, 104 are in the stacked orientation, due to the uniqueness of various regions of the locating pattern 602, the relative orientation of the top electronic device 102 and the bottom electronic device 104 may be determined. More particularly, one or both of the electronic devices 102, 104 may compare the camera data to the locating pattern 602 and may determine the region of the locating pattern 602 represented by the camera data. That is, the portion of the locating pattern corresponding to the camera data may be determined.
The locating pattern 602 is illustrated as a binary coloured image. More particularly, in the example illustrated, each pixel of the locating pattern 602 is either black or white. This binary colour scheme has been used to allow for reproduction of this document; however, in practice, a locating pattern 602 with a greater number of colours may be used. The use of a greater number of colours in the locating pattern may allow for a greater number of unique regions on the display 109A. That is, by using colour, it may be easier to produce a locating pattern 602 having very unique regions.
In some example embodiments, the locating pattern 602 may be displayed in a manner by which it is invisible to a human eye but is visible to the camera 403. Any one of a variety of techniques may be employed to ensure that the locating pattern is not visible to the human eye but is visible to the camera 403. For example, in some example embodiments, the locating pattern may be displayed at a rate that does not allow a human to perceive it but which allows the camera 403 to detect it.
Referring now to
In the example embodiment of
The locating pattern 702 may be moved across the display 109A in a predetermined pattern. By way of example, it may be moved from left to right across a top row of the display 109A and may then move in a similar manner across a row below the top row and adjacent to the top row. This scanning may continue until the locating pattern 702 has scanned across the entirety of the display 109A. When the locating pattern 702 has scanned across the entirety of the display 109A, the scanning may again continue at the top row.
Thus, while the locating pattern 702 does not, at a given time, cover the entirety of the display 109A, it may be scanned across the entirety of the display 109A so that, over time, the locating pattern 702 occupies every visible region of the display 109A.
Since the locating pattern 702 only occupies a portion of the display 109A at a given time, other features 710 may be displayed on the display 109A concurrently with the locating pattern 702. In some example embodiments, the locating pattern 702 may be scanned across the display 109A while a graphical user interface associated with an active program remains displayed. In the example illustrated, a plurality of icons are displayed concurrently with the locating pattern 702.
In the example embodiment of
In at least some example embodiments, the top electronic device 102 and/or the bottom electronic device 104 may map the display 109B (or a portion thereof) of the top electronic device 102 to a corresponding region of the display 109A of the bottom electronic device 104. More particularly, one or both of the electronic devices may identify a region of the display 109A of the bottom electronic device 102 that is aligned with the display 109B of the top electronic device 104.
After the position of the top electronic device 102 relative to the bottom electronic device 104 is determined, in at least some example embodiments, a further locating pattern 802 may be displayed at a portion of the display 109A of the bottom electronic device 102 that is obscured by the top electronic device 104. Referring now to
In the example embodiment of
The display screen illustrated in
When the position of the top electronic device 102 relative to the bottom electronic device 104 has been determined, the bottom electronic device 104 has identified where the camera 403 of the top electronic device 102 is relative to the display 109A of the bottom electronic device 104. Thus, after identifying the location of the camera 403 relative to the display 109A, the locating pattern 802 may be displayed on a portion of the display 109A that is aligned, at least in part, with the camera 403 associated with the top electronic device 104.
More particularly, the bottom electronic device 104 may display the further locating pattern 802 at a region of the display 109A of the bottom electronic device 102 that is obscured by the top electronic device 102 and that is, at least in part, aligned with the camera 403 of the top electronic device 102. In at least some example embodiments, the further locating pattern 802 is displayed in a manner that would be visible to the human eye. More particularly, the further locating pattern 802 would be visible to the human eye if the top electronic device 102 was not obstructing viewing of the further locating pattern 802.
In some example embodiments, the bottom electronic device 104 may display the further locating pattern 802 on a portion of the display 109A of the bottom electronic device 104 that is aligned with the camera 403 of the top electronic device 102 and that is also aligned with a portion of the top electronic device 102 surrounding the camera 403. That is, the further locating pattern 802 may be larger in size than the camera 403 so that a movement of the top electronic device along the display 109A of the bottom electronic device 104 causes a different portion of the further locating pattern 802 to be captured by the camera 403. In at least some example embodiments, one or both of the electronic devices 102, 104 may identify such movements of the electronic device by comparing camera data to the locating pattern 802. In at least some example embodiments, in response to a movement, the further locating pattern 802 may be moved along the display 109A to remain aligned with the camera. For example, the locating pattern 802 may be moved so that it remains centered with the camera 403.
The further locating pattern 802 of
Referring now to
In at least some example embodiments, the processor 540 (shown in
The method 900 of
At 902, the top electronic device 102 and the bottom electronic device 104 establish communications with one another. The electronic devices 102, 104 may establish wireless communications with one another using short range communication subsystems 562 (of
In at least some example embodiments, at 902, the electronic devices 102, 104 may connect via a Bluetooth™ connection. In other example embodiments, other connection protocols may be used. In at least some example embodiments, the electronic devices 102, 104 may connect together directly (e.g. using compatible short-range communication subsystems 562). In other example embodiments, the electronic devices 102, 104 may connect together through one or more server or network. In at least some example embodiments, the electronic devices 102, 104 may connect together via a Wi-Fi™ connection.
In at least some example embodiments, in order to connect, the electronic devices 102, 104 may undergo a pairing process. A pairing process allows a user to control which electronic devices 102, 104 are permitted to connect with one another. In at least some example embodiments, the pairing process may be completed once in order to establish a relationship between two electronic devices 102, 104. After the pairing process is completed, the paired electronic devices 102, 104 have a relationship in which those electronic devices 102, 104 are able to perform more advanced communications with one another.
The pairing process is an additional level of security which ensures that the users associated with a pair of electronic devices 102, 104 would like those electronic devices 102, 104 to be able to communicate with one another. When electronic devices 501 have been paired, they may be referred to as bonded electronic devices 102, 104.
The pairing process may be triggered by a specific request from a user to create a bond. For example, a user may select an option to pair an electronic device to another electronic device using a user interface associated with one of the electronic devices. The pairing process may involve user interaction to ensure that users of each electronic device 102, 104 confirm that the electronic devices 102, 104 are to be paired. In at least some example embodiments the pairing process may ensure that the positional tracking of stacked electronic devices only occurs when the electronic devices 102, 104 are paired. That is, the positional tracking will only occur if the electronic devices 102, 104 have mutually authenticated one another (i.e. through the pairing process).
During the pairing process, the electronic devices 102, 104 establish a shared secret, which may be referred to as a link key. The link key is stored by both electronic devices 102, 104 which are paired on bonded. Once the pairing process is completed, a bond is formed between the electronic devices which enables those electronic devices to connect to each other in the future without requiring the pairing process in order to confirm the identity of the devices. At the request of a user of either of the electronic devices, this bond can later be severed.
Accordingly, in at least some example embodiments, at 902, two or more electronic devices 501 which have previously undergone a pairing process are communicably connected to one another. That is, the electronic devices 102, 104 establish wireless communications so that data may be shared between the electronic devices 102, 104.
In at least some example embodiments, an operating system 523 or a communication module (such as a Bluetooth™ module) provided on the electronic devices 102, 104 may be configured to perform 902 of
In at least some example embodiments, at 904 one or more of the electronic devices 102, 104 on which wireless communications were established at 902 monitors to determine whether one or more predetermined trigger conditions have occurred. That is, in at least some example embodiments, the electronic devices 102, 104 are configured to only begin trying to determine the relative positioning of the top electronic device 102 and the bottom electronic device 104 if the predetermined trigger condition(s) has occurred.
In at least some example embodiments, the predetermined trigger condition may be a condition which indicates that the top electronic device 102 is (or may be) stacked upon the bottom electronic device 104. Accordingly, in at least some example embodiments, at 903, one or both of the electronic devices 102, 104 attempt to determine whether the electronic devices 102, 104 are in the stacked orientation. By way of example, in at least some example embodiments, the electronic devices 102, 104 compare their orientations specified by their respective orientation subsystems 549 (of
In some example embodiments, one or more of the predetermined trigger conditions may be a user input. For example, the predetermined trigger condition may be the activation of one or more interface elements associated with one or more of the electronic devices 102, 104. For example, in at least some example embodiments, each electronic device may display a selectable interface element, such as an icon which a user may select to instruct the electronic devices 102, 104 to begin trying to determine the relative positions of stacked electronic devices 102, 104.
Accordingly, in at least some example embodiments, at 904 a trigger corresponding to a predetermined trigger condition indicating that the electronic devices may be in a stacked orientation may be received and may cause the electronic devices 102, 104 to begin to determine their relative positions.
At 906, camera data associated with a camera 403 of the top electronic device 102 is received. The manner by which such camera data is received may vary depending on whether the top electronic device 102 or the bottom electronic device 104 is performing the method 900. That is, the manner by which the camera data is received may depend on which of the electronic devices 102, 104 is determining the relative positioning of the electronic devices 102, 104.
Where the method 900 is performed by the top electronic device 102, then the top electronic device 102, which is equipped with the camera 403, receives the camera data from the camera 403. Where the method 900 is performed by the bottom electronic device 104, the bottom electronic device 104 receives the camera data from the top electronic device 102 via a communication subsystem associated with the bottom electronic device 104 (such as the short-range communication subsystem 562 of
In some example embodiments, prior to receiving the camera data, the top electronic device 102 may be configured to automatically adjust the camera 403 to a macro-focus position. In the macro-focus position, a focussing lens associated with the camera is moved to a position suitable for focussing on subjects near the camera 403.
At 908, display data identifying a pixel arrangement on at least a portion of the display 109A of the bottom electronic device 104 is obtained. The manner by which such display data is obtained may vary depending on whether the top electronic device 102 or the bottom electronic device 104 is performing the method 900. Where the method 900 is performed by the bottom electronic device 104, the display data is known to the bottom electronic device 104. Accordingly, in some such example embodiments, the display data may be obtained from memory (such as a cache) or from a video processing unit.
Where the method 900 is performed by the top electronic device 102, the top electronic device 102 receives the display data from the bottom electronic device 104 via a communication subsystem associated with the top electronic device 102 (such as the short-range communication subsystem 562 of
As noted in the discussions of
Where a locating pattern 702 or 802 of the type described with reference to
At 910, the position of the top electronic device relative to the bottom electronic device is identified based on the camera data received at 906 and the display data obtained at 908. More particularly, at 910, the top electronic device 102 and/or the bottom electronic device 104 identifies a portion of the display 109A represented by the camera data.
In some example embodiments where a locating pattern 602 of the type described with reference to
In some example embodiments, where a locating pattern 702 of the type described with reference to
Accordingly, using the camera data and the display data, the position of the top electronic device 102 relative to the bottom electronic device 104 is determined. In at least some example embodiments, the top electronic device 102 and/or the bottom electronic device 104 may map the display 109B (or a portion thereof) of the top electronic device 102 to a corresponding region of the display 109A of the bottom electronic device 104. More particularly, one or both of the electronic devices may identify a region of the display 109A of the bottom electronic device 102 that is aligned with the display 109B of the top electronic device 104. In order to make this determination, it at least some example embodiments, the top electronic device 102 and/or the bottom electronic device 104 may access top electronic device form factor information. The top electronic device form factor information specifies dimensions of the top electronic device 102. For example, the top electronic device form factor information may specify the position of the camera 403 relative to other features of the top electronic device 102 (such as the display 109B). The top electronic device form factor information may, for example, specify the size of the display 109B of the top electronic device 102. In at least some example embodiments, the top electronic device form factor information may be stored in memory of one or more of the electronic devices 102, 104.
In some example embodiments, at 910, a relative orientation of the electronic devices 102, 104 may be determined. The relative orientation of the electronic devices may be specified as an angle. For example, the relative orientation may specify the degree to which the top electronic device 102 is askew when compared with the bottom electronic device 104. For example, the orientation may specify the degree to which the top electronic device 102 is away from a portrait orientation (or a landscape orientation) when its orientation is considered relative to that of the bottom electronic device 104.
In some example embodiments, at 910, the top electronic device 102 and/or the bottom electronic device 104 may identify a region of the display 109A of the bottom electronic device 104 that is obscured by the top electronic device 104. This may be determined based on the camera data, the display data, and also based on top electronic device form factor information that identifies the position of the camera 403 relative to the extremities of the top electronic device 102.
After the position of the top electronic device 102 relative to the bottom electronic device 104 is identified, a function may be performed on one or both of the electronic devices 102, 104 based on that position (at 912).
For example, a function may be performed on one or both of the electronic devices 102, 104 based on any one or a combination of: the relative orientation of the electronic devices, the portion of the display 109A of the bottom electronic device that is obscured by the top electronic device 102, the portion of the display 109A of the bottom electronic device 104 that is covered by the display 109B of the top electronic device 102, or the lateral position of the camera 403 or any other predetermined features of the top electronic device 102 relative to the bottom electronic device 104.
By way of example, in one example embodiment in which the relative orientation of the top electronic device 102 and the bottom electronic device 104 is determined, one or both of the electronic devices 102 may determine whether the relative orientation satisfies first predetermined criteria and, if so, may perform a first function on one or both of the top electronic device 102 and the bottom electronic device 104. Similarly, in some such example embodiments, one or both of the electronic devices 102 may determine whether the relative orientation satisfies second predetermined criteria and, if so, may perform a second function on one or both of the top electronic device 102 and the bottom electronic device 104. The first predetermined criteria and/or the second predetermined criteria may be an orientation or a range of orientations. For example, in at least some example embodiments, such predetermined criteria may specify an angle (or range of angles) at which the electronic devices 102, 104 must be positioned relative to one another.
In at least some example embodiments, the first function may be performed by implementing a first preference profile and the second function may be performed by implementing a second preference profile on one or both of the electronic devices 102, 104. The preference profiles may, for example, specify configuration settings for the top electronic device 102 and/or the bottom electronic device 104. Thus, by changing the orientation of the top electronic device 102 relative to the bottom electronic device 104, a preference profile may be set.
For example, an alarm may be set to a quiet mode when the relative orientation satisfies the first predetermined criteria and the alarm may be set to an audible mode when the relative orientation satisfies the second predetermined criteria.
Where the top electronic device displays a display screen that is related to a display screen associated with the bottom electronic device, the top electronic device may receive the display screen or data that may be used to generate the display screen from the bottom electronic device via the communication subsystem 562. That is, the bottom electronic device 104 may transmit the display screen (or data that may be used to generate the display screen) to the top electronic device 102 via its communication subsystem.
Accordingly, in at least some example embodiments, the complementary user interface mode that may be provided at 912 may allow the top electronic device 102 to display a display screen that is related to a display screen displayed on the bottom electronic device. That is, in at least some example embodiments, the complementary user interface mode may allow the displays 109A, 109B to complement one another. Similarly, in some example embodiments, the complementary user interface mode may allow one or more input interfaces 506 (
For example, as noted above, in some example embodiments, at 910, the top electronic device 102 and/or the bottom electronic device 104 may identify a region of the display 109A of the bottom electronic device 104 that is obscured by the top electronic device 104. In at least some such example embodiments where the display 109A of the bottom electronic device 104 is touch sensitive, the bottom electronic device 104 may, at 912, define one or more input regions near the obscured region of the display 109A that may be used for providing a complementary function. More specifically, the one or more input regions may be used for inputting a command to the bottom electronic device 104 to command the bottom electronic device 104 to perform a function that engages the top electronic device 102. That is, a predetermined gesture applied (at least in part) to the input region may be associated with a function that engages the top electronic device 102.
Since the input region(s) are selected to be near the top electronic device 102, the input regions may be defined based on the identified position of the top electronic device 102 relative to the bottom electronic device 104.
In at least some example embodiments (not illustrated in
In at least some example embodiments, at 912 of
Referring now to
In some example embodiments, at 912, the complementary user interface mode may be provided so that a portion of a display screen meant for display on the display 109A of the bottom electronic device 104 that is obscured by the top electronic device 102 is displayed on a display 109B of the top electronic device 102. In at least some example embodiments, one or both of the electronic devices 102, 104 may identify a portion of the display 109A of the bottom electronic device 104 that is aligned with a display 109B of the top electronic device 102 and may display, on the display 109B of the top electronic device content extracted from a display screen intended for display on the identified portion of the display 109A of the bottom electronic device 104.
In some example embodiments, the top electronic device 102 may have a display 109B that has a greater pixel density than the display 109A of the bottom electronic device 104. Thus, the top electronic device 102 may be used to provide an enhanced viewing environment of content on the bottom electronic device 104.
In some example embodiments, such as that illustrated in
That is, in at least some example embodiments, at 912, the top electronic device may display, on its display 109B, a display screen having one or more features that are aligned with features displayed on a display screen displayed on the display 109A of the bottom electronic device. The display screen on the display of the top electronic device may display a layer of data that is not displayed on the display screen of the bottom electronic device 104.
For example, the top electronic device 102 and the bottom electronic device 104 may display different views of a map (i.e. one of the electronic devices 102, 104 may display a first view and the other of the electronic devices 102, 104 may display a second view). For example, one of the electronic devices 102, 104 may display a traditional map view and the other of the electronic devices 102, 104 may display a satellite map view.
For example, in the example embodiment illustrated, the top electronic device 102 and the bottom electronic device 104 display different views of a map, but the maps on both of the electronic devices 102, 104 are aligned. In the illustrated example, a traditional map view is provided on the display screen 1004 displayed on the display 109A of the bottom electronic device 104 and a satellite map view is provided on the display screen 1002 on the display 109B of the top electronic device 102. At least some of the features of the map are continuous across the displays 109A, 109B. For example, a portion of a road 1006 is represented on the display screen 1004 provided on the bottom electronic device 104 and another portion of the same road is represented on the display screen 1002 provided on the top electronic device 102.
The display screen 1002 displayed on the top electronic device 102 is created based on the relative positioning of the electronic devices. Accordingly, in at least some example embodiments, the display screen 1002 associated with the top electronic device 102 is displayed after the relative position of the top electronic device 102 and the bottom electronic device 104 has been determined (e.g. according to a method described above). In at least some example embodiments, the display screen 1002 associated with the top electronic device may be displayed at 912 of
Referring now to
As shown, the display screen 1102 of the top electronic device 102 is aligned with the display screen 1104 of the bottom electronic device. The display screens 1102, 1104 appear continuous. That is, features provided on the displays may be continuous across the displays. In the example illustrated in
The example embodiment of
In some example embodiments, the bottom electronic device 104 may provide the top electronic device 102 with the display screen 1102 or with information that may be used to generate the display screen 1102 (e.g. via a short-range communication subsystem 562).
The display screen 1102 displayed on the top electronic device is created based on the relative positioning of the electronic devices. Accordingly, in at least some example embodiments, the display screen 1102 associated with the top electronic device 102 is displayed after the relative position of the top electronic device 102 and the bottom electronic device 104 has been determined (e.g. according to a method described above). In at least some example embodiments, the display screen 1102 associated with the top electronic device may be displayed at 912 of
By way of further example, as will be discussed in greater detail below with reference to
Referring now to
In the example embodiment illustrated, the display screen 1202 of the top electronic device 102 provides a magnified view of content intended for display on the display screen 1204 of the bottom electronic device. In the example illustrated, the display screen of the bottom electronic device includes the text: “Hi John, it was very nice to speak with you today. I look forward to our next chat.” A portion of this text is obscured by the top electronic device 104. The top electronic device displays a portion of the obscured text at a higher font size than is used on the display screen 1204 of the bottom electronic device.
The display screen 1202 displayed on the top electronic device is created based on the relative positioning of the electronic devices. Accordingly, in at least some example embodiments, the display screen 1202 associated with the top electronic device 102 is displayed after the relative position of the top electronic device 102 and the bottom electronic device 104 has been determined (e.g. according to a method described above). In at least some example embodiments, the display screen 1202 associated with the top electronic device may be displayed at 912 of
Referring now to
In some example embodiments, a plurality of input regions adjacent to the top electronic device 102 may be defined (at 912 of
For example, as will be described in greater detail below with reference to
Accordingly, in some example embodiments, a plurality of input regions 1306, 1308 may be defined based on the position of the top electronic device 102 relative to the bottom electronic device 104. While broken lines have been used for the purposes of identifying the input regions 1306, 1308 in
In the example embodiment of
A file, folder or content may be identified on the display 109A of the bottom electronic device using one or more identifiers 1330.
When an input of a predetermined type is received at a touchscreen display of the bottom electronic device and the input is received, at least in part at one of the input regions 1306, 1308, the function that is performed in response to the input may depend on the identifier 1320, 1320 that is nearest the input. That is, if the input engages the first input region 1306, then a function associated with the first identifier 1320 may be performed but if the input engages the second input region 1308 then a function associated with the second identifier 1322 may be performed.
In one operating mode, the function associated with an input region of the bottom electronic device is selected based on the identifier 1320, 1322 displayed on the top electronic device 102 at a location that is nearest the input region.
By way of example, in the example embodiment illustrated, a swipe gesture may be performed to send a file to a contact (e.g. via email, etc.). The contact to which the file will be sent is selected based on the input region where the swipe gesture ended. If the swipe gesture ends in the first input region 1306, then the file is sent to a contact identified by a first identifier 1320 and if the swipe gesture ends in the second input region 1308, then the file is sent to a contact identified by a second identifier 1322.
As illustrated in
In at least some example embodiments, a swipe gesture performed on the display 109A of the bottom electronic device 104 may be associated with a complementary function if the swipe gesture engages one or more of the input regions. For example, a swipe gesture that ends in the identified input region (i.e. a swipe gesture that ends near the top electronic device 102) may cause a function to be performed that engages the top electronic device 104. By way of example, a file, folder, or content such as text or an image (or an identifier of a file, folder or content) may be displayed on the display 109A of the bottom electronic device and may be dragged to one or more of the identified input regions using an input interface. That is, it may be dragged to a position adjacent to the top electronic device 102. In response, the file, folder or content may be moved or copied to the top electronic device 102.
An example of the use of such a technique to achieve cutting and pasting of content will be described below with reference to
Referring now to
As noted in the discussion of
In the example embodiment of
In response to receiving the input associated with the selected content via the paste area 1410, the selected content may be transmitted to the top electronic device 102 from the bottom electronic device 104. On the top electronic device, the received content may be displayed on the display of the top electronic device and may automatically be selected so that it may be dragged to a desired position on a document active on the top electronic device 102.
In one operating mode, the function associated with an input region of the bottom electronic device is selected based on the identifier displayed on the top electronic device at a location that is nearest the input region.
While the present disclosure is primarily described in terms of methods, a person of ordinary skill in the art will understand that the present disclosure is also directed to various apparatus such as a handheld electronic device including components for performing at least some of the aspects and features of the described methods, be it by way of hardware components, software or any combination of the two, or in any other manner. Moreover, an article of manufacture for use with the apparatus, such as a pre-recorded storage device or other similar computer readable medium including program instructions recorded thereon, or a computer data signal carrying computer readable program instructions may direct an apparatus to facilitate the practice of the described methods. It is understood that such apparatus, articles of manufacture, and computer data signals also come within the scope of the present disclosure.
Furthermore, methods and electronic devices for providing display screens of the types described above are considered to be within the scope of the present disclosure.
The term “computer readable medium” as used herein means any medium which can store instructions for use by or execution by a computer or other computing device including, but not limited to, a portable computer diskette, a hard disk drive (HDD), a random access memory (RAM), a read-only memory (ROM), an erasable programmable-read-only memory (EPROM) or flash memory, an optical disc such as a Compact Disc (CD), Digital Versatile Disc (DVD) or Blu-ray™ Disc, and a solid state storage device (e.g., NAND flash or synchronous dynamic RAM (SDRAM)).
Example embodiments of the present disclosure are not limited to any particular operating system, system architecture, mobile device architecture, server architecture, or computer programming language.
The various example embodiments presented above are merely examples and are in no way meant to limit the scope of this disclosure. Variations of the innovations described herein will be apparent to persons of ordinary skill in the art, such variations being within the intended scope of the present application. In particular, features from one or more of the above-described example embodiments may be selected to create alternative embodiments included of a sub-combination of features which may not be explicitly described above. In addition, features from one or more of the above-described example embodiments may be selected and combined to create alternative embodiments included of a combination of features which may not be explicitly described above. Features suitable for such combinations and sub-combinations would be readily apparent to persons skilled in the art upon review of the present application as a whole. The subject matter described herein and in the recited claims intends to cover and embrace all suitable changes in technology.
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/CA2012/050545 | 8/10/2012 | WO | 00 | 9/14/2012 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2014/022905 | 2/13/2014 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 7991566 | Maas et al. | Aug 2011 | B2 |
| 8164640 | Vartiainen et al. | Apr 2012 | B2 |
| 8207832 | Yun et al. | Jun 2012 | B2 |
| 8589952 | Wong et al. | Nov 2013 | B2 |
| 20030098845 | Hanson | May 2003 | A1 |
| 20050146518 | Wang et al. | Jul 2005 | A1 |
| 20060047462 | Picciotto et al. | Mar 2006 | A1 |
| 20070001950 | Zhang | Jan 2007 | A1 |
| 20080228433 | Meyer et al. | Sep 2008 | A1 |
| 20090213032 | Newport et al. | Aug 2009 | A1 |
| 20100060572 | Tsern | Mar 2010 | A1 |
| 20100333088 | Rogel et al. | Dec 2010 | A1 |
| 20110016405 | Grob et al. | Jan 2011 | A1 |
| 20110163955 | Nasiri et al. | Jul 2011 | A1 |
| 20110184646 | Wong et al. | Jul 2011 | A1 |
| 20110227827 | Solomon et al. | Sep 2011 | A1 |
| 20110295502 | Faenger | Dec 2011 | A1 |
| 20120007943 | Tytgat | Jan 2012 | A1 |
| Number | Date | Country |
|---|---|---|
| 2757527 | May 2012 | CA |
| Entry |
|---|
| Optical Mouse, http://web.archive.org/web/20100613192052/http://en.wikipedia.org/wiki/Optical—mouse, Jun. 13, 2010. |
| David Dearman, et al., “Determining the Orientation of Proximate Mobile Device Using Their Facing Camera”, May 5, 2012. |
| iTunes Preview, “iTunes is the world's easiest way to organize and add to your digital library media collection”, Mar. 7, 2012. |
| “Determining the Relative Position of 2 Mobile Devices Using Ultrasound”, retrieved Jul. 1, 2012. |
| International Search Report and Written Opinion dated Apr. 29, 2013. |
| Number | Date | Country | |
|---|---|---|---|
| 20140043209 A1 | Feb 2014 | US |
