Patent Application
20100251233
The invention pertains to user interfaces and particularly to simulation of user interfaces.
The invention is an approach for standalone usage of an embedded computing system user interface on a separate heterogeneous computing device.
Viewing and interacting with the user interface of an embedded computing device generally requires a physical presence of the device. For purposes of sales demonstrations, product testing and other scenarios, it may be impracticable, inconvenient or impossible to use the actual device. For example, when demonstrating the device to a large group of people it is desirable to have the capability to project images on a large screen for all to see. This problem has been addressed in the past in different ways, most often by creating a PC-based simulation of the embedded device's user interface. This solution may have several drawbacks. First, two different sets of software must be created and maintained—one for the actual device and one for the PC-based simulator. Second, the PC-based simulator software typically uses different user interface technology resulting in appearance differences between it and the device. Third, the simulator software often does not present the full complement of navigation and functionality as the original device.
The present invention addresses the problem described above by running the same binary software image as the embedded computing device on a PC platform. Because it is virtually the same exact software image as that which is on the device, the PC-hosted version effectively has full fidelity with the device version. Its user interface is virtually identical in appearance and has full navigation and a richer subset of functionality than would be possible with a simulator. Also, it is not necessary to maintain separate simulation and device software since both environments use the same binary image.
The block diagrams of
Various descriptions of the components may be noted. The device operating system 11 may be an embedded computer operating system such as Microsoft's Windows CE Embedded. The application program 12 may be an embedded software application including user interface functionality. The application files 13 may be all file-based resources used by the application program including graphics images and XML configuration files. The filesystem 14 may be a computing file system of the type generally available under virtually all computer operating systems and be capable of storing data as named logical files. The software emulator 22 may be a PC-based program which can emulate another non-native microprocessor instruction set and support cross execution of binary application and operating system images. The emulator 22 needs to have the capability of mapping PC (host) operating system 21 elements such as file system 14 between the emulator 22 and the host for sharing of files. An example of a software emulator 22 is Microsoft's Windows CE Device Emulator which may cross-execute an ARMV4I instruction set on an 80×86 PC. This product may support execution of Windows CE operating system images and associated application programs, and provide for mapping of the host file system 14 to the emulator 22 for file sharing.
The present invention may include the following background assumptions and prerequisites. First, the embedded device may use a computer operating system 11, for example, Microsoft's Windows CE™. Second, the PC-based software emulator 22 may exist for the embedded device's target processor and the embedded device's operating system, for example, Microsoft's Windows CE Device Emulator for the ARMV4I instruction set. Third, the software emulator 22 in the second item above has the capability to map to the file system 14 of the PC 21 for sharing of files between the emulator and the PC. The emulator 22 should also provide graphics display and user interaction through at minimum a pointing device such as a mouse or touchpad.
One version of the invention may consist of the following elements. First, one element may be an operating system image that would execute in the PC-based software emulator 22. It would be ideally very similar to the embedded device's operating system 11 but without support for device-side peripherals not available on a PC. It should have the same pixel resolution and color depth as the embedded device's display for fullest fidelity of rendering. The operating system image would ideally execute the application software image for the embedded device upon completing its initialization and startup sequence within the emulator 22.
Second, another element may be a shared operating system file folder accessible by both the software emulator 22 and the PC 21. This shared file folder would contain the operating system image, the application software and any files needed by the application software. Third, still another element may be application software (same as device application software).
In order to maximize the functionality available on the PC-based emulation of the device application software, it is desirable that the application software be capable of rendering its user interface and carrying out its user interface navigation in a manner which is portable or virtually identical in the two environments. The application software should not rely solely on device-specific definitions. Instead, it should use abstracted data structures and operating system calls which will operate without modification in either environment. In cases where this is unavoidable such the primary storage device name (i.e., “NandFlash” vs “C:\”), the application software will have the capability to determine at runtime which environment it is executing in and make the appropriate abstraction choices.
The following describes a reference implementation of the invention. It describes one way to realize the invention but should not be construed as the only method of implementation. The reference implementation of the invention may consist of the following items. First, there may be the Microsoft Windows CE Device Emulator software program. This program may execute a specified Windows CE image, a so-called NK.BIN file.
Second, there may be a specially constructed Windows CE image (NK.BIN) for execution within Device Emulator. Its purpose is to provide a nearly identical operating system image to the actual device's Windows CE image. It is built using Microsoft's Platform Builder tool for Windows CE. Upon completion of its startup, the image is programmed to launch the application software.
Third, there may be a Windows™ “batch” command file that runs the Device Emulator program with the NK.BIN file in second item above. In addition to launching the Windows CE operating system image, it also sets the display resolution and color depth to match the device's settings and creates a shared file folder between the emulator 22 and the PC 21 so that files from the PC's file system may be used.
Fourth, there may be the application software binary image. The application software has been designed so that the user interface definition including screens, buttons, navigation flow and other elements are contained within a single XML-format file called layout.xml. Additionally, the application software may make use of images such as GIF, JPG, PNG and BMP format files to assign graphics to buttons and other screen elements. Layout.xml and the optional graphics files reside in the shared file folder used by the emulator. A person using the invention may therefore update and modify the device application's user interface by simply copying new sets of files to the shared folder. The application software determines whether it is running on an embedded device or in emulation mode and makes appropriate modifications to its behavior. For example, device-specific items such as communications drivers and LCD backlight control are not started or are disabled in emulation mode.
In the present specification, some of the matter may be of a hypothetical or prophetic nature although stated in another manner or tense.
Although the invention has been described with respect to at least one illustrative example, many variations and modifications will become apparent to those skilled in the art upon reading the present specification. It is therefore the intention that the appended claims be interpreted as broadly as possible in view of the prior art to include all such variations and modifications.
Related patent applications include U.S. patent application Ser. No. ______, Attorney Docket No. H0021716-1161.1398101, filed Mar. 25, 2009, entitled “MECHANISM FOR INTERFACING A DISPLAY SCREEN OF ANOTHER TECHNOLOGY WITH A COMPUTING PLATFORM”; U.S. patent application Ser. No. ______, Attorney Docket No. H0021919-1161.1399101, filed Mar. 25, 2009, entitled, “A SMALL SCREEN DISPLAY WITH A DATA FILTERING AND SORTING USER INTERFACE”; U.S. patent application Ser. No. ______, Attorney Docket No. H0022766-1161.1417101, filed Mar. 25, 2009, entitled “A SYSTEM FOR DEFINING A USER INTERFACE OF A REMOTE DISPLAY DEVICE”; U.S. patent application Ser. No. ______, Attorney Docket No. H0022777-1161.1418101, filed Mar. 25, 2009, entitled “AN APPROACH FOR ADVANCED USER NAVIGATION”; U.S. patent application Ser. No. ______, Attorney Docket No. H0022474-1161.1419101, filed Mar. 25, 2009, entitled “AN AUTOMATIC CONFIGURATOR OF DISPLAY OBJECTS”; all of which are hereby incorporated by reference.
