Patent Application
20170026684
The present disclosure relates generally to multifaceted communications between a peripheral system and a host device and, more particularly, to a peripheral system, such as a plug-in mass-storage system or other portable system, and a host device, such as an automobile head unit, communicating in various and novel ways concerning efficient event-based synchronization of media transfer for real-time display rendering by way of the host device.
Multifaceted communications can include bi-directional, or duplex, communications of instructions and/or data between the peripheral system and host device. Example multifaceted communications include sending of operating instructions, from the host device to the peripheral system, to affect operation of the peripheral system, and media data, from the peripheral system to the host device, for being rendered in real time for a user.
Most modern automobiles are equipped by original equipment manufacturers (OEMs) with infotainment units that can present media including visual media. The units can present audio received over the Internet by way of an audio application running at the unit, for instance, and present video received from a digital video disc (DVD), for instance. While many units can also present visual media such as navigation and weather information received from a remote source, presenting video received from a remote source remains a challenge.
Other display devices or components, such as televisions and computer monitors, can receive video data by way of a high-throughput, or high-transfer-rate interface such as a High-Definition Multimedia Interface (HDMI) or Video Graphics Array (VGA) port. (HDMI is a registered trademark of HDMI Licensing, LLC, of Sunnyvale, Calif.) Digital media routers have been developed for plugging into these high-transfer-rate ports for providing video data to the display device.
Most host devices, such as legacy automobiles already on the road, do not have these high-transfer-rate interfaces. Increasingly, modern vehicles have a peripheral port, such as a universal-serial-bus (USB) port, or a wireless receiver for use in transferring only receiving relatively low-transfer-rate data from a mobile user device such as a smart phone.
Transferring video data efficiently and effectively by way of a lower-transfer-rate connection, such as USB remains a challenge. Streaming video data conventionally requires high data rates. While HDMI data rates can exceed 10 Gbps, USB data rates do not typically exceed about 4 Gbps.
Barriers to transferring video data efficiently and effectively from a remote source to a local device for display also include limitations at the local device, such as limitations of legacy software and/or hardware at the local device. Often, the mobile user devices do not have a video card and/or the vehicles do not have graphics-processing hardware. And, for example, USB video class (UVC) is not supported by either commercial Android ® devices or prevailing infotainment systems. (ANDROID is a registered trademark of Google, Inc., of Mountain View, Calif.)
Another barrier to transferring video data from a remote source to a local display is a high cost of hardware and software required to time-synchronize transmissions between devices to avoid read-write conflict.
Further challenge is a rigidity of arrangements limited to unidirectional communication of data.
There is a need for a peripheral system and a host device configured to exchange various types of information between them as part of an efficient event-based synchronization of media transfer for real-time display rendering by way of the host device.
The peripheral system is referenced primarily herein, in a non-limiting sense, as a portable system.
In various embodiments, the multifaceted communications include bi-directional, or duplex, communications of instructions and/or data between the portable system and host device. For these embodiments, the arrangement can be referred to as a duplex arrangement, and the bi-directional communications as duplexing.
Example multifaceted communications include sending of operating instructions, from the host device to the portable system, and media data, from the portable system to the host device.
Performance of the portable system is affected by instructions received from the host device, in various embodiments.
There is also need for an arrangement to transfer high-speed video streams in an efficient and synchronized manner between connected devices with low latency, and without expensive time-synchronization components. And there is also a need for systems that can transfer high-speed video streams, such as video for streaming at a receiving device, by way of a relatively low-rate connection such as a USB connection.
The present technology solves these and other challenges related to transferring high-throughput media from a source, such as a remote application server, to a destination host device, such as an automobile head unit.
The present technology processes data having a file format, and the result is a novel manner of streaming video and audio. The data being processed at any time includes a volume of still images. The still-image arrangement involving the volume, e.g., thousands, of still images, facilitates delivery of high-speed streaming video, with low latency. The process includes flushing a cache. The implementation includes use of a plug-in mass-storage system, such as one using a USB mass storage class (USB MSC) protocol.
The disclosure presents systems for synchronizing transfer and real-time display of high-throughput media, such as streaming video, between a peripheral, or portable system, such as a USB plug-in mass-storage system, and a destination host device. The media is synchronized in a novel, event-based manner that obviates the need for expensive clock-based synchronization components.
In one aspect, the present disclosure relates to a portable system comprising a processing hardware unit and a non-transitory storage device comprising computer-executable code that, when executed by the processing hardware unit, causes the processing hardware unit to perform various operations of the current technology. The portable system can be referred to by terms such as peripheral, travel system, mobile system, travel or mobile companion, portable device, or the like, by way of example.
The portable system is configured to receive a source streaming video—e.g., video file—from a video source, such as a remote video source (e.g., server), and dividing the source streaming video into a plurality of equal- or non-equal-sized image components. A resulting data-content package is stored at the system such as at a framebuffer thereof. The framebuffer can be, for instance, a transferred video source, such as in the form of a data content package.
The portable system is further configured to generate a meta-index package comprising a plurality of index components, each index component corresponding to a respective one of the equal- or non-equal-sized image components. The portable system is also configured to store the meta-index package to the non-transitory storage device. The operations further comprise sending the data-content package and the meta-index package to the host device for publishing of the image components sequentially, in accord with an order of the meta-index package, for display rendering streaming video, corresponding to the source streaming video, by way of the host device and a display device. This arrangement, involving transfer and real-time display of the data-content package and the meta-index package corresponding thereto can be referred to as a multi-tiered arrangement.
Further regarding embodiments in which the portable system and the host device are configured for bidirectional, or duplex, communications between them, instructions or data can be sent between the two by way of a first, forward channel, from the portable system to the host device, and by way of a second, back channel, from the host device, back to the portable system.
Instructions or data can be configured to change a setting or function of the receiving host device or portable system, for instance. In some implementations, the portable system, host device, and communication channel connecting them are configured to allow simultaneous bidirectional communications.
In some embodiments, the portable system includes a human-machine interface (HMI), such as a button or microphone. The portable system is configured to receive user input by way of the HMI, and trigger any of a variety of actions, including establishing a user preference at the peripheral system, altering a preference established previously at the peripheral system, and generating an instruction for sending from the portable system to the host device.
The instruction sent to the host device can be configured to affect operation of the host device in any of a variety of ways, such as by initiating establishment of a user preference at the host device, or altering a preference established previously at the host device.
Or, instruction from the portable device to the host device can be configured to affect host device operations, such as by affecting a manner by which the host device reads, renders, or acknowledges having read media data sent from the portable device.
In various embodiments, the portable system and the host device comprise a computer-executable code in the form of a dynamic programming language to facilitate interactions between the portable system and the host device.
The portable system in some embodiments uses a first-level cache to store the image components formed. The mentioned framebuffer can be a part of the first-level cache, for instance.
In various embodiments, the host system is configured for implementation as a part of a vehicle of transportation, such as an automobile comprising a communication port and a display screen device mentioned. The portable system can in this case include a communication mass-storage-device-class computing protocol (e.g., USB MSC protocol) for use in communications between the portable system and a processing hardware unit of the host device.
The host device is configured to receive user input by way of any of a variety of HMI, such as the mentioned screen being touch sensitive.
User input to the host device can trigger any of many actions, including establishing a user preference at the host device, altering a preference previously established at the host device, and generating an instruction for sending to the peripheral system.
Instruction from the host device to the portable system can be configured to affect portable system operations, such as by initiating establishment of a user preference at the portable system, or altering a preference previously established at the portable system, or by affecting a manner by which the portable system divides a source video to form indexed image components.
Other aspects of the present technology will be in part apparent and in part pointed out hereinafter.
The figures are not necessarily to scale and some features may be exaggerated or minimized, such as to show details of particular components. In some instances, well-known components, systems, materials or methods have not been described in detail in order to avoid obscuring the present disclosure.
In the figures, like numerals are used to refer to like features.
As required, detailed embodiments of the present disclosure are disclosed herein. The disclosed embodiments are merely examples that may be embodied in various and alternative forms, and combinations thereof. As used herein, for example, exemplary, and similar terms, refer expansively to embodiments that serve as an illustration, specimen, model, or pattern.
Specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a basis for the claims and as a representative basis for teaching one skilled in the art to employ the present disclosure.
While the present technology is described primarily herein in connection with automobiles, the technology is not limited to automobiles. The concepts can be used in a wide variety of applications, such as in connection with aircraft and marine craft, and non-transportation industries such as with televisions.
Other non-automotive implementations can include plug-in peer-to-peer, or network-attached-storage (NAS) devices.
I.
The portable system 110 can take any of a variety of forms, and be referenced in any of a variety of ways—such as by peripheral device, peripheral system, portable peripheral, peripheral, mobile system, mobile peripheral, portable system, and portable mass-storage system.
The portable system 110 can be portable based on being readily removable, such as by having a plug-in configuration, for example, and/or by being mobile, such as by being configured for wireless communications and being readily carried about by a user. The portable system 110 can include a dongle, or a mobile communications device such as a smart phone, as just a couple examples.
Although connections are not shown between all of the components of the portable system 110 and of the host device 150 in
The portable system 110 includes a non-transitory hardware storage device 112. The hardware storage device 112 can be referred to by other terms, such as a memory, or computer-readable medium, and can include, e.g., volatile medium, non-volatile medium, removable medium, and non-removable medium. The term hardware storage device and variants thereof, as used in the specification and claims, refer to tangible or non-transitory, computer-readable storage devices. The component is referred to primarily herein as a hardware storage device 112, or just a storage device 112 for short.
In some embodiments, the storage device 112 includes volatile and/or non-volatile, removable, and/or non-removable media, such as, for example, random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), solid state memory or other memory technology, CD ROM, DVD, BLU-RAY, or other optical disk storage, magnetic tape, magnetic disk storage or other magnetic storage devices.
The portable system 110 also includes a processing hardware unit 114 connected or connectable to the hardware storage device 112 by way of a communication link 116, such as a computer bus.
The processing hardware unit 114 can be referred to by other terms, such as computer processor, just processor, processing hardware unit, processing hardware device, processing hardware system, processing unit, processing device, or the like.
The processor 114 could be or include multiple processors, which could include distributed processors or parallel processors in a single machine or multiple machines. The processor 114 can include or be a multicore unit, such as a multicore digital signal processor (DSP) unit or multicore graphics processing unit (GPU).
The processor 114 can be used in supporting a virtual processing environment. The processor 114 could include a state machine, application specific integrated circuit (ASIC), programmable gate array (PGA) including a Field PGA (FPGA), DSP, GPU, or state machine. References herein to processor executing code or instructions to perform operations, acts, tasks, functions, steps, or the like, could include the processor 114 performing the operations directly and/or facilitating, directing, or cooperating with another device or component to perform the operations.
The portable system 110 in various embodiments comprises one or more complimenting media codec components, such as a processing or hardware component, and a software component to be used in the processing. The hardware or processing component can be a part of the processing device 114.
The hardware storage device 112 includes computer-executable instructions or code 118. The hardware storage device 112 in various embodiments stores at least some of the data received and/or generated, and to be used in processing, in a file-based arrangement corresponding to the code stored therein. For instance, when an FPGA is used, the hardware storage device 112 can include configuration files configured for processing by the FPGA.
The computer-executable code 118 is executable by the processor 114 to cause the processor 114, and thus the portable system 110, to perform any combination of the functions described herein regarding the portable system.
The hardware storage device 112 includes other code or data structures, such as a file sub-system 120, and a framebuffer capture component 122.
As mentioned, the portable system 110 in various embodiments comprises one or more complimenting media codec components, such as a processing, or hardware component, and a software component to be used in the processing. The software media codec component is indicated by reference numeral 124.
As also mentioned, a framebuffer can be a transferred video source, such as in the form of a data content package, captured by the framebuffer capture component 122.
The file sub-system 120 can include a first level cache and in some implementations also a second level cache.
In some embodiments, the hardware storage device 112 includes code of a dynamic programming language 125, such as JavaScript, Java or a C/C++ programming language. The host device 150 includes the same programming language, which is indicated in
The programming language code can define settings for communications between the portable system 110 and the host device 150, such as parameters of one or more application program interfaces (APIs) by which the portable system 110 and the device 150 communicate.
The portable system 110 in some embodiments includes at least one human-machine interface (HMI) component 126. For implementations in which the interface component 126 facilitates user input to the processor 114 and output from the processor 114 to the user, the interface component 126 can be referred to as an input/output (I/O) component. As examples, the interface component 126 can include, or be connected to a sensor for receiving user input, and include or be connected to a visual or audible indicator such as a light, digital display, or tone generator, for communicating output to the user.
The interface component 126 is connected to the processor 114 for passing user input received as corresponding signals to the processor. The interface component 126 is configured in any of a variety of ways to receive user input. In various implementations the interface component 126 includes at least one sensor configured to detect user input provided by, for instance, a touch, an audible sound or a non-touch motion or gesture.
A touch-sensor interface component can include a mechanical actuator, for translating mechanical motion of a moving part such as a mechanical knob or button, to an electrical or digital signal. The touch sensor can also include a touch-sensitive pad or screen, such as a surface-capacitance sensor.
For detecting gestures, the interface component 126 can include or use a projected-capacitance sensor, an infrared laser sub-system, a radar sub-system, or a camera sub-system, by way of examples.
The interface component 126 can be used to affect features—e.g., functions, settings, or parameters—of one or both of the portable system 110 and the host device 150 based on user input. Signals or messages corresponding to inputs received by the interface component 126 are transferred to the processor 114, which, executing code (e.g., code 118) of the hardware storage device 112 can set or alter a feature at the portable system 110. Inputs received can also trigger generation of a communication, such as an instruction or message, for the host device 150, and sending the communication to the host device 150 for setting or altering a feature of the device 150.
The portable system 110 is in various embodiments configured to receive user input via another apparatus, such as a user mobile phone, or a vehicle apparatus, such as a vehicle computer receiving the input from an HMI, such as of the interface components 172 shown in
The portable system 110 is in some embodiments configured to connect to the host device 150 by hard, or wired connection 129. Such connection is referred to primarily herein as a wired connection in a non-limiting sense. The connection can include components connecting wires, such as the USB plug-and-port arrangement described.
In some other embodiments, the connection is configured with connections according to higher throughput arrangements, such as using an HDMI port or a VGA port.
The portable system 110 is in a particular embodiment configured as a dongle, such as by having a data-communications plug 128 for connecting to a matching data-communications port 168 of the host device 150, as indicated in
An example data-communications plug 128 is a USB plug, for connecting to a USB port of the host device 150.
In some embodiments, the portable system 110 is configured for wireless communications with the host device 150 and/or a system 132 external to the portable system 110, such as a remote network or database. A wireless input or input/output (I/O) device—e.g., transceiver—or simply a transmitter, is referenced by numeral 130 in
The wireless device 130 can communicate with any of a wide variety of networks, including cellular communication networks, satellite networks, and local networks—e.g., roadside-infrastructure or other local-wireless transceivers, beacons or hotspots. The wireless device 130 can also communicate with near-field communication (NFC) devices to support functions such as mobile payment processing, or communication setup/handover functions, or any other use cases that are enabled by NFC. The wireless device 130 can include, for example, a radio modem for communication with cellular communication networks.
The remote system 132 can thus in various embodiments include any of cellular communication networks, road-side infrastructure or other local networks, for reaching destinations such as the Internet and remote servers. The remote server may be a part of or operated by a customer-service center or system, such as the OnStar® system (ONSTAR is a registered trademark of Onstar LLC of Detroit, Mich.).
Other features and functions of the portable system 110 are described below, primarily in connection with the algorithm of
The host device 150 is, in some embodiments, part of a greater system 151, such as an automobile.
As shown, the host device 150 includes a hardware storage device 152. The hardware storage device 152 can be referred to by other terms, such as a memory, or computer-readable medium, and can include, e.g., volatile medium, non-volatile medium, removable medium, and non-removable medium. The term hardware storage device and variants thereof, as used in the specification and claims, refer to tangible or non-transitory, computer-readable storage devices. The component is referred to primarily herein as a hardware storage device 152, or just a storage device 152 for short.
In some embodiments, storage media includes volatile and/or non-volatile, removable, and/or non-removable media, such as, for example, random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), solid state memory or other memory technology, CD ROM, DVD, BLU-RAY, or other optical disk storage, magnetic tape, magnetic disk storage or other magnetic storage devices.
The host device 150 also includes an embedded computer processor 154 connected or connectable to the storage device 152 by way of a communication link 156, such as a computer bus.
The processor 154 can be referred to by other terms, such as processing hardware unit, processing hardware device, processing hardware system, processing unit, processing device, or the like.
The processor 154 could be or include multiple processors, which could include distributed processors or parallel processors in a single machine or multiple machines. The processor 154 can include or be a multicore unit, such as a multicore digital signal processor (DSP) unit or multicore graphics processing unit (GPU).
The processor 154 can be used in supporting a virtual processing environment. The processor 154 could include a state machine, application specific integrated circuit (ASIC), programmable gate array (PGA) including a Field PGA (FPGA), DSP, GPU, or state machine. References herein to processor executing code or instructions to perform operations, acts, tasks, functions, steps, or the like, could include the processing device 154 performing the operations directly and/or facilitating, directing, or cooperating with another device or component to perform the operations.
The hardware storage device 152 includes computer-executable instructions or code 158. The hardware storage device 152 in various embodiments stores at least some of the data received and/or generated, and to be used in processing, in a file-based arrangement corresponding to the code stored therein. For instance, when an FPGA is used, the hardware storage device 112 can include configuration files configured for processing by the FPGA.
The storage device 152 includes computer-executable instructions, or code 158. The computer-executable code 158 is executable by the processor 154 to cause the processor, and thus the host device 150, to perform any combination of the functions described in the present disclosure regarding the host device 150.
The host device 150 includes other code or data structures, such as a file sub-system 160, a dynamic-programming-language application framework 162, such as JavaScript, Java or a C/C++ programming language.
The file sub-system 160 can include a first level cache and a second level cache. The file sub-system 160 can be used to store media, such as video or image files, before the processor 154 publishes the file(s).
The dynamic-programming-language (e.g., JavaScript, Java or a C/C++ programming language) application framework 162 can be part of the second level cache. The dynamic-programming-language is used to process image data received from the portable system 110. The programming language code can define settings for communications between the portable system 110 and the host device 150, such as parameters of one or more APIs.
The host device 150 includes or is in communication with one or more interface components 172, such as an HMI component. For implementations in which the components 172 facilitate user input to the processor 154 and output from the processor 154 to the user, the components can be referred to as input/output (I/O) components.
The host device 150 is in various embodiments configured to receive user input via another apparatus, such as a user mobile phone. Transfers, between the other apparatus and the host device 150, can be made via wired or wireless connection.
For output, the interface components 172 can include a display screen 174, which can be referred to as simply a display or screen, and an audio output such as a speaker. In a contemplated embodiment, the interface components 172 include components for providing tactile output, such as a vibration to be sensed by a user (e.g., an automobile driver), such as by way of a steering wheel or vehicle seat.
The interface components 172 are configured in any of a variety of ways to receive user input. The interface components 172 can include for input to the host device 150, for instance, a mechanical or electro-mechanical sensor device such as a touch-sensitive display, which can be referenced by numeral 174, and/or an audio device 176 such as an audio sensor—e.g., microphone—or audio output such as a speaker. In various implementations, the interface components 172 includes at least one sensor. The sensor is configured to detect user input provided by, for instance, touch, audibly, and/or by user non-touch motion, e.g., by gesture.
A touch-sensor interface component can include a mechanical actuator, for translating mechanical motion of a moving part such as a mechanical button, to an electrical or digital signal. The touch sensor can also include a touch-sensitive pad or screen, such as a surface-capacitance sensor. For detecting gestures, an interface component 172 can use a projected-capacitance sensor, an infrared laser sub-system, a radar sub-system, or a camera sub-system, for example.
The interface component 172 can be used to affect functions and setting or parameters of the one or both of the portable system 110 and the host device 150 based on user input. Signals corresponding to inputs received by the interface component 172 are passed to the processor 154, which, executing code of the storage device 152, sets or alters a function at the host device 150, or generates a communication for the portable system 110, such as an instruction or message, and sends the communication to the portable system 110 for setting or altering the function of the portable system 110.
The host device 150 is in some embodiments configured to connect to the portable system 110 by the wired connection 129 mentioned. The host device 150 is in a particular embodiment configured with, or connected to a data-communications port 168 matching the data-communications plug 128 of the portable system 110 for communicating by way of the wired link 129. An example plug/port arrangement provided is the USB arrangement mentioned.
As provided, the connection is in some other embodiments configured with connections according to higher throughput arrangements, such as using an HDMI port or a VGA port.
In some embodiments, the host device 150 is configured for wireless communications 131 with the portable system 110. A wireless input, or input/output (I/O) device—e.g., transceiver—is referenced by numeral 170 in
Other features and functions of the host device 150 are described below, primarily in connection with the algorithm of
II.
The algorithms by which the present technology is implemented are now described in more detail. The algorithms are outlined by flow charts arranged as methods 200, 300 in
It should be understood that operations of the methods 200, 300 are not necessarily presented in a particular order and that performance of some or all the operations in an alternative order is possible and contemplated.
The operations have been presented in the demonstrated order for ease of description and illustration. Operations can be added, omitted and/or performed simultaneously without departing from the scope of the appended claims.
It should also be understood that the illustrated algorithms 200, 300 can be ended at any time. In certain embodiments, some or all operations of this process, and/or substantially equivalent operations are performed by execution by the processors 114, 154 of computer-executable code of the storage devices 112, 152 provided herein.
II.A. Portable System Operations—
The algorithm 200 of
The algorithm 200 commences 201 and flow proceeds to the first operation 202 whereat the portable system 110—i.e., the processor 114 thereof executing code stored at the system storage 112—is placed in communication with the host device 150. Connecting with the host device 150 can include connecting by wire 129 (e.g., plug/port and wires) or wirelessly 131, as described above in connection with the arrangement 100 of
The portable system 110 is in some embodiments configured to connect to the host device 150 by wired connection, referenced by numeral 129 in
The portable system 110 is in a particular embodiment configured as a dongle, such as by having a data-communications plug 128—e.g., USB plug—for connecting to a matching port 168 of the host device 150. For communications between the portable system 110 and the host device 150, each can include in their respective storage devices 112, 512, a protocol operable with the type of connection. With the USB plug/port example, the protocol can be a USB mass-storage-device-class (MSC) computing protocol. Other, more advanced, USB or other, protocols, including Media Transfer Protocol (MTP), could also be supported.
The portable system 110 is in some embodiments configured to connect to the host device 150 by wireless connection, referenced by numeral 131 in
The portable system 110 connected communicatively with the host device 150 performs a handshake process with the host device 150, which can also be considered indicted by reference numeral 203 in
Operation 202 establishes a channel by which data and communications such as messages or instructions can be shared between the portable system 110 and the host device 150.
For embodiments in which both devices include a dynamic programming language, such as JavaScript, the operation 202 can include a handshake routine between the portable system 110 and the host device 150 using the dynamic programming language.
Flow proceeds to block 204 whereat the processor 114 receives, e.g., by way of the wireless communication component 130, a source media file, such as streaming video, from a source, such as a remote video source. The remote source can include a server of a customer-service center or system, such as a server of the OnStar® system.
Turning to
In various embodiments, the source media file referenced is a virtual file, such as in the form of a link or a pointer linked to a memory location containing particular corresponding media files, or a particular subset of the media files.
While the technology can be used to transfer and display render in real time—e.g., render for displaying or display purposes—various types of media files, including those with or without video, and with or without audio, the type of file described primarily herein is a video file representing a graphic output, or data for being output as a corresponding graphical display at a display screen, which in various embodiments does and does not include audio. References in the present disclosure to streaming video, video files, or the like, should for various embodiments be considered to also refer to like embodiments that include any of the various media file types possible.
The operation can include receiving the media—e.g., file—in one piece, or receiving separate portions simultaneously or over time.
In a contemplated embodiment, the video file is received from a local source, such as a virtual video file linked to the framebuffer or associated in the system—e.g., system memory—with the display screen. In embodiments, a primary, if not sole, video source is the framebuffer.
The local source can include, for instance, a smart phone or other mobile device that either receives the video file from a remote source and passes it on to the portable system 110, or has the video stored at the local source. Transfer from the local source to the portable system 110 can be by wire or wireless.
In various embodiments the video stream has any of a variety of formats, such as .mpeg, .wmv, or .avi formats, just by way of example.
Flow proceeds to block 206 whereat the processor 114 divides the source video stream, or other visual media file(s), into a plurality of indexed image components—e.g., consecutively-ordered image components. In various embodiments the image components have any of a variety of formats, such as a .jpeg format for example.
While the image components for various implementations are equal-sized, in other implementations, every image component is not the same size.
As mentioned, the portable system 110 in some embodiments has, in the hardware storage device 112, code of a dynamic programming language 125, such as JavaScript, Java or a C/C++ programming language. The language can be used in system 110 operations including image-processing operations such as the present function of dividing the video stream—e.g., video file—into consecutive image components.
The image components together can be referred to as a data-content package.
The number (N) of image components can be any in a wide range. Only by way of example, the number (N) can be in the range of about 2,500 to about 3,500. In contemplated embodiments, the number (N) is less than 2,500 or above 3,500.
In various embodiments, at block 208, the processor 114 stores the data-content package 412 (
Turning again to
In a contemplated embodiment, the processor 114 at block 208 stores the data-content package 412 (
At block 210, the processor 114 generates a meta-index package 414 (
The meta index components 415 can be the same as or similar to directory entry structures, such as that of a file allocation table (FAT) system.
At block 212, the processor 114 stores the meta index 414 at the portable system 110. The meta index 414 is also stored at the first level cache of the portable system 110.
As provided, operations of the method 200 can be performed in any order, and operations can be combined to a single step, or separated into multiple steps. Regarding the generating operations 206, 210 and storing operations 208, 212, for instance, the generating operations and/or the storing can be performed in respective single steps. The portable system 110 can store the data-content package 412 and the meta-index package 414 to the hardware storage device 112, in single operation, for instance. And the packages 412, 414 can be part of the same packet, stream, or file, or stored or transferred to the host device 150 separately.
In some embodiments, the operations include one or more real-time adjustment functions, referenced by numeral 213 in
As an example of what the adjustment function can include, the adjustment can change particular image content that is associated with a particular meta index. For determining how to adjust the linkage, the processor 114 can use as input its local clock and local copy of the physical address of the video stream. By doing so, static content represented by, for instance, USB mass storage protocol, becomes dynamic, and real-time video streaming through USB mass storage protocol is realized. Benefits of this manipulation include rendering dynamic screen output without requiring advanced classes of USB devices. The process can thus be applied with a much broader range of devices having basic USB mass storage capabilities.
At block 214, the processor 114 sends the data-content package 412 and corresponding meta-index package 414 to the host device 150. The transfer is referenced by reference numeral 215 in
The data-content package 412 and corresponding meta-index package 414 can be sent in a single communication or transmission, or by more than one communication or transmission. The mechanism is referred to at times herein as a packet, stream, file, or the like, though it may include more than one packet or the like.
In one embodiment, each image-component/meta-index-component pair is sent by the processor 114 to the host device 150 individually, in separate transmissions, instead of in packages 412, 414 with other image component/meta index component pairs. This embodiment can be referred to as a single-image file arrangement or management, by way of example.
In a contemplated embodiment, the portable system 110 is configured to determine in real time which of the circular file arrangement and the single-file arrangement to use. This type of determination can be referred to by any of various terms, such as dynamic, real-time, or adaptive—e.g., dynamic selection amongst framebuffering algorithms or techniques. Variables for the determination can include, for instance, one or more characteristics of the media—e.g., video—received for processing (e.g., dividing, storing, and sending). The variable could also include an identification of a relevant application running at the host device 150 to publish the resulting video, a category to which the application belongs, a type of application, or the like.
These arrangements (including circular file and single file arrangements), involving transfer of one or more components of data content at a time and one or more corresponding components of meta index, can be referred to as a multi-tiered arrangement, the meta index features being a first tier corresponding to a second tier of image data features.
The packet(s) is configured and sent to the host device 150 for publishing of the image components sequentially, in accord with an order of the meta-index package. The host device 150 publishes the image components to display render streaming video, corresponding to the original, source video stream, by way of the host device 150 and a display device 174.
An efficient and effective type of synchronization, not requiring synchronized clocks, is provided by this arrangement of sending a meta index 414 of meta index components 415 corresponding in order to image components 413 of a data-content package 412 for sequential display rendering of the image components 413 in accord with the index components 415.
The processing, or reading, of each image component is triggered by the reading first of the corresponding meta index component. Each next index component (e.g., 4152) is read following processing of a prior image component (e.g., 4131), and points the processor 154 to read its corresponding image component (e.g., 4132). The synchronization can be referred to as event-based synchronization, whereby none of the image components will be processed out of order. The event-based synchronization obviates the need for expensive clock or time synchronization components.
The synchronization can also be referred to as distributed device synchronization, as it involves functions of both devices, working together: the generating and sending of the index/data packages according to the present technology at the portable system 110, and the receiving and ordered display rendering of the packages at the host device 150.
The process 200 or portions thereof can be repeated, such as in connection with a new stream or file associated with a new video, or with subsequent portions of the same video used to generate the first image components and meta index components. The subsequent operations would include preparing a second data-content package and corresponding second meta-index package in the ways provided above for the first packages.
As referenced, the portable system 110 and the host device 150 can further be configured for bidirectional communications, which can be referred to as duplexing. The two-way communications can, in some implementations, be made simultaneously.
Each of the portable system 110 and the host device 150 can also be configured for multiplexing, inverse multiplexing, and the like to facilitate the efficient and effective transfer and real-time display of relatively high-throughput data between them.
As provided, in various embodiments, the configuration is arranged to facilitate communications according to the TDMA channel-access method.
The transfer 215 of previous operation 214 can be considered made by way of a first, or forward, channel of the bidirectional arrangement by which data, or data and instructions, is sent from the portable system 110 to the host device 150. Transfer from the host device 150 to the portable system 110 can be considered made by way of a second, or back, channel of the bidirectional arrangement.
At operation 216, the portable system 110 generates, identifies (e.g., retrieves), receives, or otherwise obtains instructions or messages, such as orders or requests for changing of a feature, such as a setting or function, such as from the input device 172 as indicated schematically by reference numeral 311. For adjusting a feature, such as a setting or function, of the portable system 110, based on an instructions obtained (e.g., received, retrieved, or generated), the processor 114 executes the instruction. For adjusting a feature of the host device 150, the processor 114 sends the instruction or message to the host device 150. Regarding the operation is indicated by reference numeral 216 in
In embodiments, the processing hardware device 154 of the host device 150, executing, for instance, the dynamic programming language 164 also captures user inputs, for example, touches, gestures and speech received by way of a machine-user interface 172, translates them to data streams—byte streams, for instance—and then sends them to the portable system 110 through connection 129 or 131, for example.
In various embodiments, the processor 114 receives, such as by way of the wireless communication component 130, a communication, such as an instruction or message from the host device 150. The operation is also indicated by reference numeral 216 in
As mentioned, the transfer 217 is an example of data transfer by way of a second, or back channel of the bidirectional arrangement. Back-channel communications can be, but is not in every implementation, initiated by user input to the portable system 110.
Communications 217 from the host device 150 can take any of a variety of forms, such as by being configured to indicate a characteristic, function, parameter, or setting of the portable system 110. The communication 217 can indicate a manner by which to establish the characteristic, function, parameter or setting at the portable system 110, or to alter such previously established at the portable system 110.
In various embodiments, the portable system 110 can be personalized, such as by various features—e.g., settings or user preferences. These can be programmed to the portable system 110 by any of a variety of methods, including by way of the host device 150—via the second, back, channel mentioned, for instance—a personal computer (now shown), a mobile phone, or the like.
In some embodiments, default settings or preferences of the portable system 110 are provided before any personalization is performed. The settings or preferences for personalization of the portable system 110 can include any of those described herein, such as a manner by which the portable system 110 processes incoming video.
An example features—e.g., setting—of the portable system 110, which can be affected by a message from the host device 150, or user input to the portable system 110, and affect how the portable system 110 processes incoming video, includes a setting controlling a size of equal-sized image snippets into which to divide the incoming video.
Another example feature of the portable system 110, which can be affected by a message from the host device 150, or user input to the portable system 110, and affect how the portable system 110 processes incoming video, includes a setting controlling a number of image components, or snippets, into which to divide incoming video to form each data-image package, and so a number of corresponding meta index components of a meta-index package.
In various embodiments, a playback feature—e.g., setting—is stored at the portable system 110, can be controlled by instruction from the host device 150, and affects a manner by which the portable system 110 delivers data-content packages to the host device 150, such as a timing, speed, rate, or size of the data or data package sent.
Another example feature of the portable system 110, which can be affected by a message from the host device 150, or user input to the portable system 110, and affect how the portable system 110 processes incoming video, includes a setting controlling a size of unequal-sized image components, or snippets, into which to divide the incoming video.
Still another example feature of the portable system 110, which can be affected by a message from the host device 150, or user input to the portable system 110, and affect how the portable system 110 processes incoming video, includes a speed, rate, or quality by which the portable system 110 processes incoming video. The processing affected can include, for instance, reading incoming streaming video or writing the image snippets based on the video.
The speed, rate, or quality of processing can affect other processes (e.g., making available bandwidth for a VOIP call), or video-viewing experience, such as playback qualities at the host device 150.
While the features can take other forms without departing from the scope of the present technology, in various embodiments, the features include at least one setting selected from a group consisting of a quality of the image components formed at the portable system 110 and a playback setting at the host device 150 or at the portable system 110. Example image quality characteristics include a level of zoom, brightness, or contrast.
The playback or screen replication feature—e.g., setting or characteristic—at the host device 150 can affect a speed by which the video display is rendered.
In some embodiments, the playback setting at the host device 150 affects a direction by which the video display is rendered. And in various embodiments, the playback setting at the host device 150 affects whether the video is rendered or played, or not rendered or played at all.
As other examples, playback features—which can be altered at the portable system 110 or the host device 150, such as in response to instruction from the other (the host device 150 or the portable system 110)—include fast-forward (e.g., a fast-forward mode), rewind (e.g., a rewind mode), pause (e.g., a pause mode), stop (e.g., a stop mode), play (e.g., a play mode), or rate of video play, or screen replication.
As another example feature, a setting of the portable system 110 can include a setting controlling whether a single-image file arrangement or a circular-file arrangement should be used, whereby the portable system 110 would send image snippets, and corresponding index components, one at a time or together in a circular file, respectively.
As another example feature, a setting of the portable system 110 and/or the host device 150 includes one or more fractional values, between 0 and 1, of full circular files, associated with a timing of reading and/or writing of the circular files. Example values, described further below, including in connection with
As another example, feature, the feature can include a setting affecting a linkage relationship between the data-content package and the meta-index package in real time at the remote portable system. Adjusting a linkage relationship is described further in connection with blocks 213 and 307.
At block 218, user input is received at the processor 114 of the portable system 110 by way of one or more user input, or I/O interfaces 126 (
The interface component 126 can be used to affect any features, including those mentioned—e.g., functions and settings or parameters—of one or both of the portable system 110 and the host device 150, based on user input provided to either the portable system 110 or the host device 150.
Thus, the processor 114, executing code of the hardware storage device 112 can generate or identify at least one instruction or message. The instruction can take any of a variety of forms, such as by being configured to indicate a feature, such as a characteristic, function, parameter, or setting, of the portable system 110, or to indicate a feature, such as a characteristic, function, parameter, or setting, of the host device 150. The instruction can further indicate a manner by which to establish a feature, such as a characteristic, function, parameter or setting of the portable system 110, or to alter such feature established.
The process 200 of
II.B. Host Device System Operations—
The algorithm 300 of
The host device 150 can be connected by wire or wirelessly to the potable system 110.
The algorithm 300 begins 301 and flow proceeds to the first operation 302 whereat the host device 150—i.e., the processor 154 thereof executing code stored at the device storage 152—is placed in communication with the portable system 110. Connecting with the portable system 110 can include connecting by the wired or wireless connection 129, 131, shown.
The connection of block 302 can include a handshake process between the host device 150 and the portable system 110, which can also be considered indicted by reference numeral 203 in
In embodiments, during this handshake process, meta index components 415 are also exchanged based on, for example, USB mass storage device protocol.
For embodiments in which both devices include a dynamic programming language, such as JavaScript, Java or a C/C++ programming language, the operation 302 can include a handshake routine between the portable system 110 and the host device 150 using the dynamic programming language.
Flow proceeds to block 304 whereat the processor 154 receives from the portable system 110 the data-content package 412 and corresponding meta-index package 414 shown in
Receipt 304 of communications can be made along a forward channel of the bidirectional or arrangement mentioned, by which communications can be sent in both direction between the portable system 110 and the host device 150, including in some implementations simultaneously.
As mentioned, the data-content package 412 and corresponding meta-index package 414 can be sent in a single communication or transmission or by more than one communication or transmission. The mechanism is referred to at times herein as a packet, though it may include more than one packet, stream, or file.
In another embodiment, mentioned above in connection with
Again, the arrangement, involving transfer of one or more components (e.g., components 413) of data content (412) at a time and one or more corresponding components (e.g., components 415) of meta index (414), can be referred to as a multi-tiered arrangement, the meta index features being a first tier corresponding to the image data features being the second tier.
The transfer can be performed by wired connection or wireless connection, which are indicated schematically by reference numerals 129 and 131 in
At block 306, the processor 154 stores the data-content package 412 and the index package 414 received to a portion of memory 152 at the host device 150, such as in memory 152 associated with the dynamic-programming-language (e.g., JavaScript, Java or a C/C++ programming language) application framework 162. The data-content package 412, its constituent parts being the data snippets—e.g., image components 413, and the index package 414, and its constituent index components 415, are referenced in
The memory component including a dynamic-programming-language application framework, such as JavaScript, Java or a C/C++ programming language, referenced 162 in
Continuing with the multi-tiered arrangement referenced, the storing 306 can include saving the data-content package 412 and the meta-index package 414 to a first-level cache of the memory 152, such as a first-level cache of the file sub-system 160.
As provided, in some embodiments, the operations include one or more real-time adjustment functions, referenced by numeral 307 in
As provided, operations of the method 300 can be performed in any order, and operations can be combined to a single step, or separated into multiple steps. Regarding the storing operation 306, for instance, the storing can be performed in one or respective single steps corresponding to each package (data and index). The host device 150 can store the data-content package 452 and the meta-index package 454 to the storage device 152, in a single operation, for instance. And the packages 452, 454 can be part of the same packet, stream, or file, or transferred to the host device 150 by the portable system 110 separately.
Flow proceeds to block 308 whereat the processor 154 publishes the media of the received data package 412 for communication to a user, e.g., vehicle passenger, as a video matching the source video file or virtualized source video received by the portable system 110 (operation 204).
As mentioned, the host device 150 in some embodiments has stored in its storage device 152 code of a dynamic programming language 164, such as JavaScript, Java or a C/C++ programming language. The language in some implementations includes an application framework for facilitating image-processing functions of the host device 150. The programming language code can define features, such as operation settings, for communications between the portable system 110 and the host device 150, such as parameters of one or more APIs, and/or the manner by which the image files are processed at the host device 150 to display render the resulting video for publishing to the user.
As provided, in embodiments, the processing hardware device 154, executing the dynamic programming language 164 also receives user-input data sent from the portable processor 114 executing the dynamic programming language 125 stored there.
The language can be used in operations of the host device 150, including image-processing—e.g., reading and display rendering, operations.
Publishing video at operation 308 comprises rendering the data of the image components 4531-N according to an order of the meta index components 4551-N of the corresponding meta-index package 454.
In embodiments, then, the operations include:
These functions can be performed in a round-robin manner. The configuration of the present technology ensures that the write pointer (PW, or P1) is always one step ahead of the read pointer (PR, or P2)—i.e., ensures that P1 always equals P2-1.
If P1>P2, then images read at the host device (by the index 454 there) will not be valid, if P1=P2, there would be a read-write conflict, and if P1<P2 (e.g., P1<<P2), there would be a large latency in the video streaming.
To achieve the read-write arrangement described (P1=P2-1), conventional systems require relatively expensive, and corresponding software, for fine-timing synchronization between the host device and the portable system. Any frequency offset between clocks of the two apparatus (host device and portable system) would accumulate until P2-P1 does not equal to 1. The event-driven configuration of the present technology achieves the desired result in a much-more efficient manner.
As provided, this arrangement—including receiving a meta index of meta index components corresponding in order to image components for sequential display rendering—provides an efficient and effective form of synchronization without requiring expensive clock synchronizing. The processing of each image component 413 can be triggered by reading the corresponding index component 415. Each next index component (e.g., 4152) is read following processing of a prior image component (e.g., 4131), and points the processor 154 to read its corresponding image component (e.g., 4132). The synchronization can be referred to as event-based synchronization, whereby none of the image components will be processed out of order. The event-based synchronization obviates the need for expensive time or clock synchronization components.
The resulting video is transferred, by wire or wirelessly, to an output 172, such as a display or screen 174, such as an infotainment screen of an encompassing system 151 such as an automobile. The transfer is indicated by numeral 309 in
At block 310, the host device 150 generates, identifies (e.g., retrieves), receives, or otherwise obtains instructions or messages, such as orders or requests for changing of a feature, such as a setting or function. Regarding instructions for adjusting a feature of the host device 150, the processor 154 executes the instruction. Regarding instructions for adjusting a feature, such as a setting or function, of the portable system 110, the processor 154 sends the instruction or message to the portable system 110, as indicated by path 217.
In one implementation, at least one communication, other than those transmitting data-content packages 412 and corresponding meta-index packages 414, is shared between the portable system 110 and the host device 150.
In various embodiments, the processor 154 sends to the portable system 110 a communication, such as an instruction or message, from the host device 150. These potential transmissions are indicated by reference numeral 217 in
For embodiments allowing bidirectional, or duplex, communications, communications or data (e.g., image/index package) sent by the portable system 110 are transmitted along the mentioned first, or forward, channel of the connection to the host device 150, and communications sent by the processor 154 to the portable system 110 are transmitted along the mentioned second, or back, channel.
Communications 217 from the portable system 110 to the host device 150 can take any of a variety of forms, such as by being configured to indicate a feature, such as a characteristic, function, parameter, or setting, of the host device 150. The communication 217 can further indicate a manner by which to establish a feature, or to alter such feature previously established at the host device 150.
The feature adjusted can include any of the features—e.g., settings, parameters, functions—described herein, including those mentioned above in association with features of the portable system 110.
By way of example, while the feature can take other forms without departing from the scope of the present technology, in one embodiment the feature is selected from a group consisting of a setting affecting a quality of the image components and a playback setting. Example image quality characteristics include a level of zoom, brightness, or contrast. The playback characteristic can be a feature that affects speed or direction by which the video display rendered is being played, or whether played at all. The playback characteristics can include, for instance, fast-forward, rewind, pause, stop, play, or rate of video play.
Generation of communications 217 from the host device 150 to the portable system 110 can be triggered by user input to an input device 172. The input can include touch input to a touch-sensitive screen 174, for example, and/or audio input to a vehicle microphone 176, for instance.
Communications 217 from the host device 150 to the portable system 110 can take any of a variety of forms, such as by being configured to indicate a feature—e.g., characteristic, function, parameter, or setting—of the portable system 110. The communication 217 can further indicate a manner by which to establish a feature at the portable system 110, or to alter such feature previously established at portable system 110.
While a feature at the portable system 110 affected by a communication 217 from the host device 150 can take other forms without departing from the scope of the present technology, the communication 217 is in various embodiments configured to affect a manner by which the portable system 110 performs any of its operations described, such as a manner by which the portable system 110 divides the source video stream into image components 413, or generates the meta-index package 414.
The process 300 can end 313, or any portions thereof can be repeated, such as in connection with a new video or media, or with subsequent portions of the same video used to generate the first image components 413 and index components 415 at the portable system 110.
III.
The teachings of
The chart 600 shows, above the line 602, functions of the portable system 110, as referenced by bracket 604, and functions of the host device 150—e.g., vehicle head unit—below the line 602, as referenced by bracket 606.
In the host-device section 606, the chart 600 shows a plurality of circular-file-read-commencement points 608, 610, 612, 614, whereat the host device 150 commences reading respective circular files. Thus between each commencement point is a corresponding circular-file read, such as the read indicated by bracket 616 between the last two commencement points 612, 614 called out.
In various embodiments, at least one algorithm controlling when circular files are written controls the writings according to reading status of an immediately previous circular file. In one of the embodiments, the algorithm provides by a first, ‘for,’ thread:
And by a corresponding second, ‘while,’ thread:
where C0 represents the initial, complete circular file 412, before it is read at the host device 150. When one quarter of the circular file 412 has been read at the host device 150, then the amount of circular file 412 remaining at that point would be ¾C0, and so on.
The fractions shown are only sample values. The fractions could have other values greater than 0 and less than 1. In practice, the values could be set otherwise by users, such as engineers. The setting can be made using a calibration process, such as one in which feedback from test or actual operation of the arrangement 100, or a component thereof, is processed for setting one or more of the values.
The first conditional (if) routine of the first thread [Thread 1] can be a part of the host device 150 reading the circular file, wherein the device 150 reads the data, such as from the portable system 110 in the form of a USB mass storage device, for example, sending a packet such as a USB packet to initiate the reading. By receiving the request from the host device 150, the portable system 110 determines that the reading has occurred or is occurring and can thereby determine a reading time for the file.
The graph 700 shows a timeline 702 along the x-axis, and along the y-axis, an amount of unread portion of the circular file (412 in
(1) a lowest value, U=0, at the x-axis;
(2) a top-most value 706 on the y-axis 704, or U=C0,
(3) a three-quarters value 708, or U=¾C0;
(4) a halved value 710, or U=½C0); and
(5) a one-quarter value 712, or U=¼C0.
The fractions shown are only sample values. The fractions could have other values greater than 0 and less than 1. In practice, the values could be set otherwise by users, such as engineers. The setting can be made using a calibration process, such as one in which feedback from test or actual operation of the arrangement 100, or a component thereof, is processed for setting one or more of the values.
The line 701 shows the amount of circular file left throughout reads at the host device 150 of adjacent circular files. The full content of each circular file has an initial maximum value, where the line 701 starts in each section, corresponding to respective circular file reads, at the highest U value 706, or CO. As each circular file is read, the unread, or U, value, decreases over time, as shown for each read by its descending portion of the line 701.
In the example shown, line 701 is not perfectly symmetric (e.g., it dips below the x-axis, once). This is because intervals between consecutive readings (represented by numeral 616 in
New circular files are written—e.g., generated at the portable system 110 and sent to the host device 150—according to the threads [Thread 1], [Thread 2] described above in connection with
In this way, one circular file is sent at a time, and a next circular file is being received at the host device 150 at the time that the host device is completing reading the immediately preceding circular file. And the next circular file can be read, starting with a first image component (e.g., 4131) of a next circular file 412F, immediately after a final image component (e.g., 413N) of the immediately preceding circular file 412F-1 is read.
V. Select Benefits of the Present Technology
Many of the benefits and advantages of the present technology are described above. The present section restates some of those and references some others. The benefits are provided by way of example, and are not exhaustive of the benefits of the present technology.
The technology in various embodiments includes structure and processing capabilities to exchange multifaceted inter-system communications between the portable system and the host device of.
The aspects add versatility and robustness to the distributed system.
Multifaceted communications enabled by the technology can include bi-directional, or duplex, communications. Instead of a rigid, unidirectional, relationship between the portable system and host device, each can send one of a variety of information or data types to the other. For instance, between allowing either of the portable system and host device to affect operation of the other—i.e., the host device or portable system.
By the exchanges between the portable system and host device, the portable system and host device can affect operation of the other in a desired manner, such as base on user input.
The portable system and host device are in various embodiments configured to generate or pass on instructions or data for use in changing a feature, such as an operation setting, of the host device or the portable system. In embodiments, the portable system and host device are configured to promote functions within the arrangement, such as be freeing up bandwidth for other applications, or improving a viewing experience for the user, for instance.
In various embodiments, the technology allows transfer and real-time display of video data in an efficient and synchronized manner between connected devices without need for expensive time synchronization practices. The synchronization is event-based instead of based on synchronized timing.
The systems and algorithms described can be used to transfer high-speed video streams by way of relatively low-transfer-rate connections, such as a USB connection.
The present technology in at least these ways solves prior challenges to transferring and displaying in real time high-throughput media from a source, such as a remote server, to a destination host device, such as an automotive head unit, without need for expensive time synchronization software or hardware, and without requirements for relatively expensive high-end wireless-communications and graphics-processing hardware at the host device.
The portable systems allow streaming of video data at a pre-existing host device, such as an existing automotive on-board computer in a legacy or on-road vehicle—e.g., a vehicle already in the marketplace.
As another benefit, the capabilities can further be provided using a portable system to include mostly, or completely, parts that are readily available and of relatively low cost.
VI. Conclusion
Various embodiments of the present disclosure are disclosed herein. The disclosed embodiments are merely examples that may be embodied in various and alternative forms, and combinations thereof.
The above-described embodiments are merely exemplary illustrations of implementations set forth for a clear understanding of the principles of the disclosure. Variations, modifications, and combinations may be made to the above-described embodiments without departing from the scope of the claims. All such variations, modifications, and combinations are included herein by the scope of this disclosure and the following claims.
| Number | Date | Country | |
|---|---|---|---|
| Parent | 14808166 | Jul 2015 | US |
| Child | 15065334 | US |
