Patent Application
20180048755
The present invention relates to systems and methods for establishing data communication between electronic devices. More specifically, the present invention includes systems and methods for establishing bi-directional data communication between electronic devices and for establishing uni-directional and bi-directional communication between devices and peripherals.
In the field of computerized devices, there are many different interface ports used for communication. RS-232 serial communication, for inter-terminal communication over copper wires, was standardized in 1969. Since that time, many additional wired and wireless techniques have been developed for inter-device communication.
In recent times, there has been a proliferation of mobile devices. The 1990s saw the emergence of PDAs (personal digital assistants), and the last 10 years have seen the emergence of “smartphones”, phones with a general-purpose central processing unit capable of running multiple simultaneous programs, much like a computer. Similarly, portable media players (PMPs), such as the iPod have developed computer-like functionality. In the last several years, over a billion mobile phones have been sold each year, and an increasing number of these phones are being designed as “smartphones”, incorporating general-purpose computing technology coupled with an always-on Internet connection.
Because these devices are in essence portable computers, they are used for conventional computer functionality, including email, address book, calendaring, and document manipulation. There are various ways to get this sort of information onto portable devices and to connect peripherals to such devices. Unfortunately, all of the current methods have limitations with regards to nearby, immediate, low-latency connectivity.
Because these devices are in essence portable computers, the manufacturers have allowed a controlled level of third-party application development on their platforms. Examples of mobile platforms include the Microsoft® Windows® Mobile platform, the Symbian platform, the Palm® Palm OS® and web OS™ platforms, the Google™ Android™ mobile application platform and the Apple® iPhone® OS platform. Each of these mobile device operating systems resembles in many respects a traditional desktop computer operating system. However, the internals of many of these operating systems are restricted in such a way that new device drivers—the software that allows for additional hardware devices to be connected or for additional network protocols to be implemented—are not permitted.
The smartphone type devices communicate to the Internet using the TCP/IP protocol and to other devices such as computers and printers through local-area and wide-area wireless networking. Hardware transmission standards that support this include Wi-Fi (802.11); cellular wireless networking such as CDMA. UMTS, HSDPA, GPRS and EDGE; and wide-area networking such as WiMAX (IEEE 802.16). However, device-to-device and device-to-peripheral communication is awkward with these standards. To have two devices communicate in this manner, they must either communicate through an intermediary server or website (the most common method), adding delay and complexity of setup. For instance, if one device is to send a single contact of an address book to another device—for example, in exchanging business contact information—there is no straightforward technique for sending the contact directly, and the most common alternative, sending the contact over email, is less direct and not instantaneous, and may not even be supported by one of the devices.
If devices are to communicate in a peer-to-peer manner (i.e., sending network packets directly from one device to another), it is necessary to have both devices communicate with the same network protocols and the same application-layer protocols, and to solve issues such as network peer discovery. In practice, due to concern about viruses inherent in any network protocol, devices are very restricted in what they are designed to do on a direct basis over a network, and direct exchange of data between devices is thus inhibited over a network.
In the case of connecting peripherals, the cost and complexity of adding wireless local-area networking to a simple peripheral device has prevented the attachment of keyboards, mice, joysticks and other peripherals to devices via wireless networking. In practice, there are no known Wi-Fi keyboard peripherals.
Another limitation of Wi-Fi and other wireless techniques is that they require both devices to be wireless network receivers and transmitters, and that they require constant connectivity for devices to work. This consumes much power, requiring portable peripherals and devices to carry larger batteries, and requiring frequent recharging or replacement of batteries.
The Bluetooth wireless networking standard was developed as a point-to-point wireless technique for device-to-device and device-to-peripheral connection. It is specifically designed to minimize power use. Although it overcomes some of the limitations of wireless networking approaches by specifically enabling point-to-point, device-to-device communication, it still requires larger batteries, bulkier devices and requires complex protocol support. Many devices that incorporate some Bluetooth features do not allow application-level developers to access the features. For instance, one device may support transmission of address book entries to another device, but the other device, even though it has the necessary hardware support, does not implement the correct software to accept the address book transmission. Because Bluetooth software support is implemented by the device manufacturer, the device manufacturer can decide, for any reason, not to support parts of the standard.
Additionally, unlike TCP/IP and networking standards, Bluetooth is not designed for ad-hoc creation of new communication protocols. Thus, an application developer who simply wanted to use Bluetooth to implement a proprietary or industry-standard communication scheme on top of Bluetooth would be limited to the protocols the manufacturer enabled on the device. In addition, in the case of peripherals, such as Bluetooth keyboards, if a mobile device is not designed to work with a Bluetooth keyboard by the manufacturer, it is effectively impossible for an application developer to implement support for it.
Infrared light transmission, a line-of-sight wireless communication technique, was popular for one-way and two-way inter-device communication in the 1990s and still sees some use today, but has been eclipsed by Bluetooth for peripheral interconnection. The Palm series of PDA devices traditionally could accomplish the task of sending an address book contact entry from one device to another using serial binary communication over infrared communication standards such as IrDA (a high-speed, infrared serial protocol).
However, most mobile phones, PDAs, smartphones and portable devices only use infrared today as a remote control input standard, and usually only implement support for a single remote. For instance, many Macintosh computers and devices such as iPod docking stations can use an Apple remote control. Even though they have infrared receiver hardware, there is no application-level ability to receive any other infrared protocols or support any other infrared devices than the Apple remote. Many other devices with infrared support have similar limitations. The existence of an infrared receiver or transmitter on a device provides little hope that any device or protocol not supported by the manufacturer will be supported. Even though the vast majority of television and other remotes in market use uni-directional infrared transmission, and even though universal remotes have shown the versatility of this technique for device interoperability, this interoperability has rarely been demonstrated in the area of computing devices. For several years, a number of infrared keyboards were developed, and a brief era of standardization was achieved wherein a dozen or so PDA devices supported perhaps a dozen in-market portable infrared keyboards. However, the current generation of mobile devices, lacking support for the IrDA or general infrared support, cannot use these keyboards.
The Universal Serial Bus (USB) is a wired hardware communication standard and is the most common interface on mobile devices, phones, PDAs, portable computers and smartphones for charging and data communication, but is not commonly used for peripheral interconnection. Most mobile devices can be connected in a master/slave relationship with a personal computer, such that the personal computer recognizes the mobile device as a peripheral. Mobile devices usually change to a “peripheral” mode, and specialized software running on the personal computer, usually provided by the manufacturer, allows custom communication with the device, such as synchronizing address books and contacts, music, calendars and other data Because of the master/slave design of USB, USB cannot be readily used to connect two devices, or to connect peripherals to a mobile device that itself is designed to be a peripheral, without a personal computer intermediary. Thus, address book contact exchange cannot be readily achieved by connecting two devices via USB, nor can a USB keyboard—a ubiquitous and inexpensive device—be connected to a mobile device: even though both the keyboard and the mobile device “have USB,” they are both slave devices.
Proprietary docking connectors are the most common approach to peripheral interconnection for mobile devices. PDA, phone, smartphone and portable media player manufacturers have developed custom physical and electrical connections on their devices to build an ecosystem of peripherals that only work with their devices. For instance, the Apple iPod® dock connector standard has enabled a large aftermarket ecosystem of automobile integration adapters, speakers, video playback devices, microphones, remote controls, pedometers, projectors and other peripherals. Similar approaches are used by mobile phone and portable media player manufacturers to create proprietary peripheral interfaces for remote controls, chargers and audio outputs.
Beyond physical and electrical limitations, manufacturers often develop custom protocols, inaccessible to application developers, so that even if a device exists and can be coupled to the proprietary electrical and mechanical interface, it is impossible for an application developer to support the device or access its functionality. For example, there are microphone input peripherals designed to enable audio input for iPod devices that lack a microphone. Even though they use the standard connector and have even been approved for use with some models of iPod, they cannot work with the latest iPods, merely because Apple has not provided the software support (device drivers) to allow access to them by application developers, and because there is no way for application developers to develop these device drivers or provide these device drivers to consumers.
None of the existing systems have been able to provide mobile device application developers with a reliable technique for accomplishing inter-device communication regardless of hardware design limitations and regardless of manufacturer.
The need in the art is addressed by the present invention, which provides a system and method for coupling a peripheral to a device such as a smartphone. In a most general embodiment, the invention includes a first circuit for converting a data signal output from the peripheral to analog (e.g., audio) signals; a connector for coupling the circuit to an input (e.g. audio) port of the device; and a second circuit disposed within the device for converting the analog signals to (e.g. digital) data signals.
In the illustrative embodiment, the first circuit is a frequency shift key generator and the second circuit is a frequency shift key detector. A host controller is coupled to the peripheral and a microcontroller coupled to the device. A serial port is coupled between the microcontroller and the host controller. A communication controller is coupled between the serial port and the frequency shift key generator.
In a more specific embodiment, the device further includes a converter that transforms frequency-shift-keyed data to binary data, a scan-code-to-character-set mapper and an arrangement for receiving and processing signals wirelessly via an infrared receiver.
Illustrative embodiments and exemplary applications will now be described with reference to the accompanying drawings to disclose the advantageous teachings of the present invention.
While the present invention is described herein with reference to illustrative embodiments for particular applications, it should be understood that the invention is not limited thereto. Those having ordinary skill in the art and access to the teachings provided herein will recognize additional modifications, applications and embodiments within the scope thereof, and additional fields in which the present invention would be of significant utility.
Some portions of the detailed descriptions that follow are presented in terms of algorithms and symbolic representations of operations on data bits within a device memory. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is here, and generally, conceived to be a self-consistent sequence of steps leading to a desired result. The steps are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared and otherwise manipulated. It has proved convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers or the like.
It should be born in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise as apparent from the following discussion, it is appreciated that throughout the description, discussions utilizing terms such as “processing,” “computing,” “calculating,” “determining,” “selecting,” “displaying,” “sending,” “receiving,” “updating,” “modifying,” “assigning,” “requesting,” “notifying,” “communicating”. “interfacing” or the like, refer to the actions and processes of a mobile device or similar electronic device having processing circuitry and components that manipulate and transform data represented as physical (electronic) quantities within the system's registers and memories into other data similarly represented as physical quantities within the device system memories, registers or other such information storage, transmission or display devices.
The present invention also relates to an apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes or it may comprise a general-purpose device selectively activated or reconfigured by a program stored in the device. Such a program may be stored in a computer-readable storage medium, such as, but not limited to, any type of disk—including floppy disks, optical disks, CD-ROMs and magneto-optical disks-read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards or any type of media suitable for storing electronic instructions, each of which may be coupled to a computer system bus.
The algorithms and displays presented herein can be implemented as a particular electronic device, component, computer or other apparatus. However, various general-purpose systems may also be used with the programs in accordance with the teachings herein, or it may prove convenient to construct more specialized apparatus to perform the required method steps. The required structure for a variety of these systems will become apparent from the description below. In addition, the present invention is not described with reference to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the invention as described herein.
A machine or computer readable medium includes any mechanism for storing information in a form readable by a computer or similar machine. For example, a computer-readable medium includes read-only memory (“ROM”), random access memory (“RAM”), magnetic-disk storage media, optical storage media, flash memory devices or other types of machine-accessible storage media.
In accordance with the present teachings, a technique for serial communication is implemented that utilizes the audio input and output facilities of a device. Modulating audio to achieve binary communication is called “modulation-demodulation,” and devices that implement this technique are called “modems.” The first audio modems were implemented in the 1950s, and by the early 1960s, 300-bit-per-second (bps) communication was being achieved over phone lines. One modulation scheme used, called “V.21,” used binary frequency-shift keying (BFSK, or “binary FSK”). In this technique, two tones (pitches) were played alternately, one representing a binary “1” and one representing a binary “0”. In the late 1970s, early personal computers used similar forms of modulation to store and retrieve programs on cassette tape. For instance, the Apple II computer loaded programs at about 180 bytes per second from cassettes by using alternating tones to represent binary digits.
In the 1990s, digital signal processor chips were incorporated into personal computers, allowing them to implement the more complex, higher-bit-rate modem protocols that run at 28.8 kbps and 56 kbps, in software. As computer CPUs have increased in speed, they have gained the ability to implement various modulation schemes in software alone without additional DSP hardware.
Recently, there have been a number of uses of computer audio input and output ports for digital communication. For long-distance radio communication, many of the shortwave modulation schemes have been implemented as software for sound cards on PCs. Sound cards have been used to implement bidirectional communication over several kilometers of copper wire in open-source projects like AuDSL.
For short-distance, line-of-sight applications, the open-source LiRC software project includes support for infrared reception through the microphone port of a PC sound card. Similarly, a product developed by Griffin Technologies several years ago allowed Windows Mobile-based PDAs to transmit infrared codes to control televisions using the audio output facility of the PDA. Similar techniques have been demonstrated for the iPhone device, performing infrared transmission using the audio output facility. Very recently, on 28 Dec. 2008, tellart.com released a video demonstrating the ability to input a single changing value between 0 and 255 by having the iPhone measure the audio amplitude (volume) of a sound looped through a potentiometer, for the purpose of connecting the iPhone to physical sensor peripherals. However, this technique, though effective for real-time measurement of time-varying quantities, is ineffective for uni-directional or bidirectional data transfer.
In the illustrative embodiment, the signals from the keyboard are modulated as frequency-shift-keyed (FSK) audio signals by FSK Generator 006. In accordance with the present teachings, the audio output of the FSK generator 006 is fed to an audio input or microphone port 007 of a mobile device 008, where it is interpreted by a software application 009.
One skilled in the art would understand that any number of additional devices with different functions, without limitation—including computer mice, credit card readers, human interface devices, game controllers and printers-could also be connected to the USB host controller 002.
The microcontroller 004 converts the data to FSK audio and plays it into the microphone port of the mobile device. Application software running on the mobile device decodes the FSK audio into binary digits, assembles the binary digits into byte sequences and interprets sequences as keyboard scan codes, which are then either stored in fixed or temporary memory or displayed, as appropriate.
In the illustrative embodiment, the microcontroller 004 of
Although there are a number of ways to get data into and out of most smartphones and other devices—e.g., the Apple dock connector, Bluetooth, the camera and screen, and Wi-Fi—none of these are well suited for simple, low-speed bidirectional communication with inexpensive components. In addition, several of these inputs are simply inaccessible to independent developers, as in the Apple iPhone SDK. The audio port, however, is accessible from the iPhone SDK—both for playing audio and for recording it. Hence, in accordance with the present teachings, a device design is taught that interfaces with one's device (e.g., iPhone) via the microphone/audio jack and sends bidirectional 1,200-baud serial data.
Whereas the present invention is illustrated herein with respect to the iPhone mobile phone, it should be understood that the present invention is not limited to any particular device.
Wired External Device Connections
For a wired implementation, to provide the microcontroller 004 of
Frequency-shift keying uses alternating analog signals to send information: for example, in binary FSK, a tone of 8,000 hertz might signify a digital ‘1’, and 4,000 hertz a digital ‘0’. Two frequencies are chosen for this project for the following reasons:
To build the circuit, the following electronic components are recommended for the illustrative implementation:
In the illustrative embodiment, the USB host controller 002 of
FSK generators and detectors are known in the art. For example, to build the FSK generator 006 and detector 101, the following papers may be used:
The circuit 006 takes serial data from the computer and converts it to frequency-shift-keyed data. It takes a carrier signal, such as a sine wave, and modulates it, in the illustrative embodiment, toggling it between 4,000 Hz and 8,000 Hz.
Illustrative FSK specifications are:
In the illustrative embodiment, the baud rate to the device is approximately 1266 (to provide a small buffer such that a 1200 baud device connected to the FSK circuit does not jam.) The 1200 baud should be sufficient, inasmuch as the chip will should not send anything at a faster rate than 1200 baud.
The bytes are sent as little-endian ASCII (LSB/MSB). For example, “U” is sent as the following 9 bits:
Bit 0: 4 kHz start bit
Bit 1: 8 kHz
Bit 2: 4 kHz
Bit 3: 8 kHz
Bit 4: 4 kHz
Bit 5: 8 kHz
Bit 6: 4 kHz
Bit 7: 8 kHz
Bit 8: 4 kHz
Standby is at 8 kHz—if the chip is not sending data, it emits a constant 8 kHz tone.
The wired implementation uses the audio port, a method of communicating with a device or second-generation iPod touch. The circuit is capable of communicating with a device with a microphone input, not just the device that is used as an example. The internal microphone of an iPhone mobile device typically has a resistance of about 1KΩ (1 kilo-ohms). This resistance is used by the device to detect whether a microphone is plugged in. When conventional headphones are plugged in, the sleeve on the plug bridges both the microphone and ground pins of the internal jack. Thus, when the iPhone senses zero ohms, it detects that only headphones are connected. If it senses 1K, it detects that a microphone is present.
Using a modified version of the iPhone modem Cypress firmware from the bidirectional serial implementation described above, one can attach an external portable keyboard to one's iPhone mobile device.
Illustrative keyboards 001 that may be used include: a MicroInnovations IrDA Palm keyboard, which has an unusual binary protocol (scan codes) due to its unusual keyboard layout, and a Targus keyboard, whose scan codes match a standard PC keyboard. Both were designed in the PDA (e.g., PalmPilot, iPAQ) era, and use a form of infrared called IrDA to send key presses to a device.
Wireless External Device Connections
The circuit discussed above can detect signals through ambient audio as-is. Adding an IrDA receiver to the circuit and placing the unmodified infrared keyboard in close line-of-sight proximity to the infrared receiver achieves the same functionality.
In operation, the “IrDA” signal from the keyboard provides a short to the audio input via a 5 kΩ pull-up resistor. Thus, the standby voltage is 5 volts (in the illustrative embodiment) and the value goes to 0 volts when shorted for brief 5 microsecond pulses as the keyboard shorts the line. A series of pulses (or lack thereof) indicates what data is being transmitted. In the illustrative embodiment, the pulses are sent at 9,600 baud. Therefore, to read the data sent by the keyboard, the system has to do the following:
Because this keyboard sends data at 9600 baud, the circuit buffers data to some extent, although one cannot typically type faster than 1200 baud. This is because some key presses generate two or three bytes: these are scan codes, not ASCII characters. Because scan codes vary from keyboard to keyboard, it is best to determine what the scan codes are, either by specification or by simply pressing keys. Note that both the press and release of a key generate a scan code, as do modifier keys like SHIFT. A single character typed can generate 2-6 bytes of data. With stop bits, the total could be up to 54 bits for a single key press. A 120-word-per-minute typist can type 10 characters per second. Yet even with overhead, 540 bits per second fits into the 1200 baud bandwidth budget.
It is also worth noting that, in accordance with the illustrative embodiment, the keyboard only uses the microphone input. Thus, by using a simple ‘Y’ splitter and ensuring the software does not open the audio output, one can listen to audio while typing in accordance with the present teachings. Inasmuch as kits with the keyboard preassembled are straightforward to produce, a library that adds keyboard support can be integrated into any iPhone application by one of ordinary skill in the art. Thus, a programmer of ordinary skill could create terminal programs, note-taking programs and even full-screen text editors with arrow controls. Given that some old serial mice originally ran at 1,200 baud, an improved circuit can be produced with both mouse and keyboard support. Thus, the present invention allows an iPhone mobile device to operate as a desktop personal computer with a keyboard by plugging the iPhone into a computer screen for display.
For the illustrative implementation, a simple circuit is wired that takes infrared pulses and converts them to sound. Code is provided to recognize the signals from the IR (infrared) remote that comes bundled with most Macintosh computers. These steps are well within the level of skill of a person of ordinary skill in the art. See, for example, iPhone Hacks: Pushing the iPhone and iPod touch Beyond Their Limits, written by Damien Stolarz et al. and published by O'Reilly Media, first edition (Apr. 16, 2009), ISBN-10: 0596516649, ISBN-13: 978-0596516642, the teachings of which are hereby incorporated herein by reference.
The present invention has been illustrated by way of example, and not by way of limitation, with the Apple iPhone mobile device. However, the present teachings are not limited thereto. Those of ordinary skill in the art will appreciate that the present teachings can be utilized with other devices without departing from the scope thereof. That is, it should be understood that the present teachings can be applied without limitation to any device with audio input capability, such as smartphones, mobile cellular phones, personal computers, tablet PCs, portable media players (PMPs) with audio input, portable and mobile Internet devices (MIDs), ultra mobile PCs (UMPCs), personal navigation devices (PNDs), computing accessories, wearable computers, laptops, handhelds, personal digital assistants (PDAs), system-on-a-chip devices, portable video recorders, portable audio recorders, or any other computing device with an audio input port or microphone.
It is further understood that this invention applies as well to any device that only has audio output capability but is used to communicate uni-directionally to a device with audio input capability as described in the invention. Such devices could include, without limitation, keyboards with audio output, peripheral devices with audio output, mice, bar code scanners, portable media players, portable audio devices, portable video devices, credit card readers, digital storage devices, ambient audio transmitters such as public address (PA) systems, amplifiers, mixers, boom boxes, DJ systems, performance systems, flat-screen televisions, movie projection systems, surround sound systems, in-vehicle audio systems, satellite. HD or terrestrial radio transmission systems, mixing boards, computers, solid-state audio players. CD players, DVD and Blu-ray Disc players, digital media playback systems, or other fixed or portable electronic devices.
In another embodiment, the audio can be transmitted wirelessly—e.g., via RF or Bluetooth—and the same approaches can be used to modulate and transmit data. For instance, a Bluetooth hands-free headset can be disassembled, the microphone and speaker wires of this Bluetooth hands-free headset can be connected to the audio input and output of our exemplary FSK circuit, and the FSK circuit can be connected to an external device, such as a keyboard or bidirectional communications device. If the headset is “paired” (in Bluetooth terminology, connected) to the mobile phone, and the software on the mobile phone is configured to receive and send audio through the Bluetooth hands-free headset, the data will transfer over the air. The Bluetooth headset should be selected to use higher audio quality modes (such as A2DP if available), and the frequencies and speed of transmission can be lowered to decrease the error rate, since Bluetooth adds its own layers of modulation and digital compression to the audio signal. This allows the keyboard or other device to be connected and communicate wirelessly to the mobile device over Bluetooth.
Similarly, any one-way or bidirectional radio frequency-based (non-Bluetooth) analog audio transmitter can be used in the same way as Bluetooth. The sending portion of the transmitter can be connected to the FSK circuit and then to the keyboard, and the receiver to the input of the mobile device. This allows the keyboard to be wirelessly connected to the mobile device via RF.
The teachings provided herein could be used in a variety of ways, for example:
It should also be noted that the present teachings enable device-to-device as well as device-to-peripheral connection. That is, the peripheral could be another mobile device within the scope and meaning of the present teachings.
Thus, a method, a system and an apparatus for connecting peripheral devices to mobile devices and for interconnecting mobile devices have been described. It is to be understood that the above description is intended to be illustrative and not restrictive. Many other embodiments will be apparent to those of skill in the art upon reading and understanding the above description. The scope of the invention should, therefore, be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.
It is therefore intended that the appended claims cover any and all such applications, modifications and embodiments within the scope of the present invention.
This is a U.S. continuation application which claims priority to U.S. nonprovisional application Ser. No. 12/660,995 filed Mar. 8, 2010 and entitled ELECTRONIC DEVICE INPUT/OUTPUT SYSTEM AND METHOD, which claims priority to U.S. provisional application No. 61/158,586 filed Mar. 9, 2009 and entitled ELECTRONIC DEVICE INPUT/OUTPUT TECHNIQUE, both of which are incorporated herein by reference.
| Number | Date | Country | |
|---|---|---|---|
| 61158586 | Mar 2009 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 12660995 | Mar 2010 | US |
| Child | 15791283 | US |
