System, method, and article of manufacture for wireless enablement of the world wide web using a wireless gateway

Abstract

A system, method and article of manufacture are provided for selection and formatting of web content for remote viewing. User-defined information is received and used to retrieve content from one or more web sites. The retrieved content is aggregated at a network server located remotely from the user. The aggregated content is formatted at the network server for display on a wireless device. The formatted content is transmitted to a wireless device for display on the wireless device.

Description

FIELD OF THE INVENTION

[0003] The present invention relates to personalized data manipulation and more particularly to aggregating content and accessing the content remotely.

BACKGROUND OF THE INVENTION

[0004] With the advent of wireless devices, the current shift in communications is to allow wireless access to content on the Internet. Current methods of delivering web-based content to wireless devices, however, requires that each site have wireless enabled content. Accordingly, a significant engineering investment is required on a web-site by web-site basis to enable each web-site to become wireless enabled. This is the biggest bottleneck in the wireless enabling of the web.

[0005] Thus, what is needed is a way to make all content on the web, from any location in the world, wireless enabled by aggregating user selected content at one site and making that selected content wireless enabled. The one site thus behaves as a gateway between the wireless and wired web. Utilizing the present invention, the entire world wide web can be wireless enabled without requiring an engineering investment on a website by website basis.

SUMMARY OF THE INVENTION

[0006] Prior to this invention, a significant engineering investment was required on a web-site by web-site basis to enable each web-site to become wireless enabled. This was the biggest bottleneck in the wireless enabling of the web. By removing that bottleneck, the current invention allows any content from the web to become wireless enabled instantaneously. A user can therefore access and interact with any content from any wireless device in a fashion similar to the wired devices. Accordingly, a system, method and article of manufacture are provided for selection and formatting of network content for remote viewing.

[0007] According to one embodiment, a user is allowed to provide information that specifies general or specific content to be retrieved from one or more network sites for online or offline viewing. For example, the user can specify retrieval of a particular web page when the web page changes/is updated. As another example, the user can make a general request to download articles relating to a particular topic, such as stock exchange information, airline fares, etc. The user can also specify retrieval of other information, such as bank account information to be retrieved from an online database. The user is allowed to submit the user-defined information from at least one of a wireless device and a hardwired device such as a personal computer. It should be noted that such content can be anything on or transmittable via the Internet. By way of example only, such content can be hyperlinks, images, text, tables, secure information such as account information, email, and audio and video data. The user is allowed to select, customize and/or edit the content from any device, wired or wireless.

[0008] The user-defined information is received and used to identify and retrieve the specified content from one or more network sites, such as web sites. If particular content is specified in the user-defined information, the pertinent web page is accessed and the particular content is downloaded. If content is generally requested, a search engine can be used to find the information.

[0009] The retrieved content is aggregated at a network server, preferably located remotely from the user. The network server acts as a gateway through which any content from the world wide web or any other networked source or application is collected and converted into a format amenable to output on the user device. Preferably, the aggregated content is amenable to presentation and content customization through a user interface. At the network server, the aggregated content is formatted for output on a user device such as a wireless device. A wireless device can include any type of device capable of receiving information where at least one of the communication links is wireless, such as a wireless telephone, Personal Digital Assistant (PDA), handheld computer such as a handheld PC, a pager, a device connected to a wireless modem, etc. Exemplary wired devices include a hardwired telephone, personal computer (PC), etc.

[0010] The formatting style can be specific to the device used by the particular user. For example, if the content is to be output on a display of a PDA, the content would be formatted for the screen of the PDA. If the content is to be output via a voice browser on an audio device (such as a telephone), the content is converted to audio, if necessary.

[0011] Preferably, the content is formatted for display on a plurality of user devices so that the user can use any of a plurality of user devices to access the information. The formatted content is transmitted to the user device for output on the user device.

[0012] If the content has been formatted for a plurality of user devices, the user device may send a request for a particular type of formatting associated with that type of device. Some or all of the content can be sent to an application program for further processing.

[0013] In one aspect of the present invention, a user selects which content is aggregated in real time. As an option, customization of the web-based habitat can be performed utilizing the user device. A web page with the specified content can be identified based on the contents of the web page.

[0014] In another aspect of the present invention, the content is aggregated on a portal page unique to the user. The portal page displays the content that the user has specified, and can include the interface that allows the user to specify the information. This portal page is fully customizable by the user and can be accessed by any device, whether wireless or hardwired. As an option, the retrieved content can be updated after a predetermined amount of time has expired. The updated content would then be formatted at the network server for display on the user device and transmitted to the user device for output on the user device. Some or all of the updated content can also be transmitted to an application program for further processing.

[0015] In an aspect of the present invention, the user is allowed to interact with the content. Thus, the user is not only able to receive information, but is also able to send information from the wireless device. For example, the user can be allowed to fill out forms and fields, make purchases (buying and selling), read and reply to emails, etc. using the wireless device. In yet another aspect of the present invention, an alert is sent to the wireless device upon occurrence of a prespecified condition. By way of example only, an alert can be sent at a predetermined time prior to departure of an airline flight. The alert can also be a reminder of an appointment. Further, an alert can be sent upon a stock price reaching a predetermined level or an auction price reaching a certain amount.

[0016] A method for compiling an enhanced query result is also provided. A user query is received, upon which information is extracted from one or more data sites based on the query via a connection. The information extracted from the data site(s) is aggregated at a network server. A determination is made as to the type of output device being used by the user, and the aggregated information is formatted for display and/or audible output on the output device. Finally, the formatted information is transmitted to the output device.

[0017] A further method for accessing network content is provided. According to this method, a form is retrieved, where the form has user-specific identification information used for accessing data on a network site, such as a banking site, email server, etc. The network site is logged in to using the form. At least a portion of the data is retrieved from the network site. The retrieved content is aggregated and formatted for output on a user device. The formatted content is sent to the user device for output on the user device.

[0018] Prior to this invention, a significant engineering investment was required on a web-site by web-site basis to enable each web-site to become wireless enabled. This was the biggest bottleneck in the wireless enabling of the web. By removing that bottleneck, the current invention allows any content from the web to become wireless enabled instantaneously. A user can therefore access and interact with any content from any wireless device in a fashion similar to the wired devices.

BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The invention will be better understood when consideration is given to the following detailed description thereof. Such description makes reference to the annexed drawings wherein:

[0020] FIG. 1 is a schematic diagram of a hardware implementation of one embodiment of the present invention;

[0021] FIG. 2 illustrates a process for selection and formatting of web content for remote viewing;

[0022] FIG. 3 depicts a preferred operating environment for one or more portable wireless devices in connection with a host computer system;

[0023] FIG. 4 illustrates a top level software detail of the control program is shown in accordance with a preferred embodiment of a wireless device;

[0024] FIG. 5 depicts a combined data flow detailing the handling of compressed data objects;

[0025] FIG. 6 illustrates a process for generating a customized network user interface according to one embodiment of the present invention;

[0026] FIG. 7 is a flowchart of a process for allowing a user to customize an information portal according to one embodiment of the present invention;

[0027] FIG. 8 depicts a default mode process for allowing selection and management of preferred content according to one embodiment of the present invention;

[0028] FIG. 9 is a flowchart of an advanced mode process for allowing selection and management of preferred content according to an embodiment of the present invention;

[0029] FIG. 10 depicts a process for recognizing a state based on a current output, according to one embodiment of the present invention;

[0030] FIG. 11 is a flowchart of a process for retrieving network content according to one embodiment;

[0031] FIG. 12 is a flowchart depicting generation of new applications according to one embodiment;

[0032] FIG. 13 is a flow diagram of a process for generating an application according to one embodiment;

[0033] FIG. 14 is a flow diagram depicting a process for creating an application utilizing a graphical user interface in accordance with an embodiment; and

[0034] FIG. 15 provides a flow of execution of the pattern-based application.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0035] A preferred embodiment of a system in accordance with the present invention is preferably practiced in the context of a personal computer such as an IBM compatible personal computer, Apple Macintosh computer or UNIX based workstation. A representative hardware environment is depicted in FIG. 1, which illustrates a typical hardware configuration of a workstation in accordance with a preferred embodiment having a central processing unit 110, such as a microprocessor, and a number of other units interconnected via a system bus 112. The workstation shown in FIG. 1 includes a Random Access Memory (RAM) 114, Read Only Memory (ROM) 116, an I/O adapter 118 for connecting peripheral devices such as disk storage units 120 to the bus 112, a user interface adapter 122 for connecting a keyboard 124, a mouse 126, a speaker 128, a microphone 132, and/or other user interface devices such as a touch screen (not shown) to the bus 112, communication adapter 134 for connecting the workstation to a communication network (e.g., a data processing network) and a display adapter 136 for connecting the bus 112 to a display device 138. The workstation typically has resident thereon an operating system such as the Microsoft Windows NT or Windows/95 Operating System (OS), the IBM OS/2 operating system, the MAC OS, or UNIX operating system. Those skilled in the art will appreciate that the present invention may also be implemented on platforms and operating systems other than those mentioned.

[0036] A preferred embodiment is written using JAVA, C, and the C++ language and utilizes object oriented programming methodology. Object oriented programming (OOP) has become increasingly used to develop complex applications. As OOP moves toward the mainstream of software design and development, various software solutions require adaptation to make use of the benefits of OOP. A need exists for these principles of OOP to be applied to a messaging interface of an electronic messaging system such that a set of OOP classes and objects for the messaging interface can be provided. OOP is a process of developing computer software using objects, including the steps of analyzing the problem, designing the system, and constructing the program. An object is a software package that contains both data and a collection of related structures and procedures. Since it contains both data and a collection of structures and procedures, it can be visualized as a self-sufficient component that does not require other additional structures, procedures or data to perform its specific task. OOP, therefore, views a computer program as a collection of largely autonomous components, called objects, each of which is responsible for a specific task. This concept of packaging data, structures, and procedures together in one component or module is called encapsulation.

[0037] In general, OOP components are reusable software modules which present an interface that conforms to an object model and which are accessed at run-time through a component integration architecture. A component integration architecture is a set of architecture mechanisms which allow software modules in different process spaces to utilize each others capabilities or functions. This is generally done by assuming a common component object model on which to build the architecture. It is worthwhile to differentiate between an object and a class of objects at this point. An object is a single instance of the class of objects, which is often just called a class. A class of objects can be viewed as a blueprint, from which many objects can be formed.

[0038] OOP allows the programmer to create an object that is a part of another object. For example, the object representing a piston engine is said to have a composition-relationship with the object representing a piston. In reality, a piston engine comprises a piston, valves and many other components; the fact that a piston is an element of a piston engine can be logically and semantically represented in OOP by two objects.

[0039] OOP also allows creation of an object that “depends from” another object. If there are two objects, one representing a piston engine and the other representing a piston engine wherein the piston is made of ceramic, then the relationship between the two objects is not that of composition. A ceramic piston engine does not make up a piston engine. Rather it is merely one kind of piston engine that has one more limitation than the piston engine; its piston is made of ceramic. In this case, the object representing the ceramic piston engine is called a derived object, and it inherits all of the aspects of the object representing the piston engine and adds further limitation or detail to it. The object representing the ceramic piston engine “depends from” the object representing the piston engine. The relationship between these objects is called inheritance.

[0040] When the object or class representing the ceramic piston engine inherits all of the aspects of the objects representing the piston engine, it inherits the thermal characteristics of a standard piston defined in the piston engine class. However, the ceramic piston engine object overrides these ceramic specific thermal characteristics, which are typically different from those associated with a metal piston. It skips over the original and uses new functions related to ceramic pistons. Different kinds of piston engines have different characteristics, but may have the same underlying functions associated with it (e.g., how many pistons in the engine, ignition sequences, lubrication, etc.). To access each of these functions in any piston engine object, a programmer would call the same functions with the same names, but each type of piston engine may have different/overriding implementations of functions behind the same name. This ability to hide different implementations of a function behind the same name is called polymorphism and it greatly simplifies communication among objects.

[0041] With the concepts of composition-relationship, encapsulation, inheritance and polymorphism, an object can represent just about anything in the real world. In fact, one's logical perception of the reality is the only limit on determining the kinds of things that can become objects in object-oriented software. Some typical categories are as follows:

[0042] Objects can represent physical objects, such as automobiles in a traffic-flow simulation, electrical components in a circuit-design program, countries in an economics model, or aircraft in an air-traffic-control system.

[0043] Objects can represent elements of the computer-user environment such as windows, menus or graphics objects.

[0044] An object can represent an inventory, such as a personnel file or a table of the latitudes and longitudes of cities.

[0045] An object can represent user-defined data types such as time, angles, and complex numbers, or points on the plane.

[0046] With this enormous capability of an object to represent just about any logically separable matters, OOP allows the software developer to design and implement a computer program that is a model of some aspects of reality, whether that reality is a physical entity, a process, a system, or a composition of matter. Since the object can represent anything, the software developer can create an object which can be used as a component in a larger software project in the future.

[0047] If 90% of a new OOP software program consists of proven, existing components made from preexisting reusable objects, then only the remaining 10% of the new software project has to be written and tested from scratch. Since 90% already came from an inventory of extensively tested reusable objects, the potential domain from which an error could originate is 10% of the program. As a result, OOP enables software developers to build objects out of other, previously built objects.

[0048] This process closely resembles complex machinery being built out of assemblies and sub-assemblies. OOP technology, therefore, makes software engineering more like hardware engineering in that software is built from existing components, which are available to the developer as objects. All this adds up to an improved quality of the software as well as an increased speed of its development.

[0049] Programming languages are beginning to fully support the OOP principles, such as encapsulation, inheritance, polymorphism, and composition-relationship. With the advent of the C++ language, many commercial software developers have embraced OOP. C++ is an OOP language that offers a fast, machine-executable code. Furthermore, C++ is suitable for both commercial-application and systems-programming projects. For now, C++ appears to be the most popular choice among many OOP programmers, but there is a host of other OOP languages, such as Smalltalk, Common Lisp Object System (CLOS), and Eiffel. Additionally, OOP capabilities are being added to more traditional popular computer programming languages such as Pascal.

[0050] The benefits of object classes can be summarized, as follows:

[0051] Objects and their corresponding classes break down complex programming problems into many smaller, simpler problems.

[0052] Encapsulation enforces data abstraction through the organization of data into small, independent objects that can communicate with each other. Encapsulation protects the data in an object from accidental damage, but allows other objects to interact with that data by calling the object's member functions and structures.

[0053] Subclassing and inheritance make it possible to extend and modify objects through deriving new kinds of objects from the standard classes available in the system. Thus, new capabilities are created without having to start from scratch.

[0054] Polymorphism and multiple inheritance make it possible for different programmers to mix and match characteristics of many different classes and create specialized objects that can still work with related objects in predictable ways.

[0055] Class hierarchies and containment hierarchies provide a flexible mechanism for modeling real-world objects and the relationships among them.

[0056] Libraries of reusable classes are useful in many situations, but they also have some limitations. For example:

[0057] Complexity. In a complex system, the class hierarchies for related classes can become extremely confusing, with many dozens or even hundreds of classes.

[0058] Flow of control. A program written with the aid of class libraries is still responsible for the flow of control (i.e., it must control the interactions among all the objects created from a particular library). The programmer has to decide which functions to call at what times for which kinds of objects.

[0059] Duplication of effort. Although class libraries allow programmers to use and reuse many small pieces of code, each programmer puts those pieces together in a different way. Two different programmers can use the same set of class libraries to write two programs that do exactly the same thing but whose internal structure (i.e., design) may be quite different, depending on hundreds of small decisions each programmer makes along the way. Inevitably, similar pieces of code end up doing similar things in slightly different ways and do not work as well together as they should.

[0060] Class libraries are very flexible. As programs grow more complex, more programmers are forced to reinvent basic solutions to basic problems over and over again. A relatively new extension of the class library concept is to have a framework of class libraries. This framework is more complex and consists of significant collections of collaborating classes that capture both the small scale patterns and major mechanisms that implement the common requirements and design in a specific application domain. They were first developed to free application programmers from the chores involved in displaying menus, windows, dialog boxes, and other standard user interface elements for personal computers.

[0061] Frameworks also represent a change in the way programmers think about the interaction between the code they write and code written by others. In the early days of procedural. programming, the programmer called libraries provided by the operating system to perform certain tasks, but basically the program executed down the page from start to finish, and the programmer was solely responsible for the flow of control. This was appropriate for printing out paychecks, calculating a mathematical table, or solving other problems with a program that executed in just one way.

[0062] The development of graphical user interfaces began to turn this procedural programming arrangement inside out. These interfaces allow the user, rather than program logic, to drive the program and decide when certain actions should be performed. Today, most personal computer software accomplishes this by means of an event loop which monitors the mouse, keyboard, and other sources of external events and calls the appropriate parts of the programmer's code according to actions that the user performs. The programmer no longer determines the order in which events occur. Instead, a program is divided into separate pieces that are called at unpredictable times and in an unpredictable order. By relinquishing control in this way to users, the developer creates a program that is much easier to use. Nevertheless, individual pieces of the program written by the developer still call libraries provided by the operating system to accomplish certain tasks, and the programmer must still determine the flow of control within each piece after it's called by the event loop. Application code still “sits on top of” the system.

[0063] Even event loop programs require programmers to write a lot of code that should not need to be written separately for every application. The concept of an application framework carries the event loop concept further. Instead of dealing with all the nuts and bolts of constructing basic menus, windows, and dialog boxes and then making these things all work together, programmers using application frameworks start with working application code and basic user interface elements in place. Subsequently, they build from there by replacing some of the generic capabilities of the framework with the specific capabilities of the intended application.

[0064] Application frameworks reduce the total amount of code that a programmer has to write from scratch. However, because the framework is really a generic application that displays windows, supports copy and paste, and so on, the programmer can also relinquish control to a greater degree than event loop programs permit. The framework code takes care of almost all event handling and flow of control, and the programmer's code is called only when the framework needs it (e.g., to create or manipulate a proprietary data structure).

[0065] A programmer writing a framework program not only relinquishes control to the user (as is also true for event loop programs), but also relinquishes the detailed flow of control within the program to the framework. This approach allows the creation of more complex systems that work together in interesting ways, as opposed to isolated programs, having custom code, being created over and over again for similar problems.

[0066] Thus, as is explained above, a framework basically is a collection of cooperating classes that make up a reusable design solution for a given problem domain. It typically includes objects that provide default behavior (e.g., for menus and windows), and programmers use it by inheriting some of that default behavior and overriding other behavior so that the framework calls application code at the appropriate times.

[0067] There are three main differences between frameworks and class libraries:

[0068] Behavior versus protocol. Class libraries are essentially collections of behaviors that you can call when you want those individual behaviors in your program. A framework, on the other hand, provides not only behavior but also the protocol or set of rules that govern the ways in which behaviors can be combined, including rules for what a programmer is supposed to provide versus what the framework provides.

[0069] Call versus override. With a class library, the code the programmer instantiates objects and calls their member functions. It's possible to instantiate and call objects in the same way with a framework (i.e., to treat the framework as a class library), but to take full advantage of a framework's reusable design, a programmer typically writes code that overrides and is called by the framework. The framework manages the flow of control among its objects. Writing a program involves dividing responsibilities among the various pieces of software that are called by the framework rather than specifying how the different pieces should work together.

[0070] Implementation versus design. With class libraries, programmers reuse only implementations, whereas with frameworks, they reuse design. A framework embodies the way a family of related programs or pieces of software work. It represents a generic design solution that can be adapted to a variety of specific problems in a given domain. For example, a single framework can embody the way a user interface works, even though two different user interfaces created with the same framework might solve quite different interface problems.

[0071] Thus, through the development of frameworks for solutions to various problems and programming tasks, significant reductions in the design and development effort for software can be achieved. A preferred embodiment of the invention utilizes HyperText Markup Language (HTML) to implement documents on the Internet together with a general-purpose secure communication protocol for a transport medium between the client and the Newco. HTTP or other protocols could be readily substituted for HTML without undue experimentation. Information on these products is available in T. Berners-Lee, D. Connoly, “RFC 1866: Hypertext Markup Language-2.0” (November 1995); and R. Fielding, H, Frystyk, T. Berners-Lee, J. Gettys and J. C. Mogul, “Hypertext Transfer Protocol—HTTP/1.1: HTTP Working Group Internet Draft” (May 2, 1996). HTML is a simple data format used to create hypertext documents that are portable from one platform to another. HTML documents are SGML documents with generic semantics that are appropriate for representing information from a wide range of domains. HTML has been in use by the World-Wide Web global information initiative since 1990. HTML is an application of ISO Standard 8879; 1986 Information Processing Text and Office Systems; Standard Generalized Markup Language (SGML).

[0072] To date, Web development tools have been limited in their ability to create dynamic Web applications which span from client to server and interoperate with existing computing resources. Until recently, HTML has been the dominant technology used in development of Web-based solutions. However, HTML has proven to be inadequate in the following areas:

[0073] Poor performance;

[0074] Restricted user interface capabilities;

[0075] Can only produce static Web pages;

[0076] Lack of interoperability with existing applications and data; and

[0077] Inability to scale.

[0078] Sun Microsystem's Java language solves many of the client-side problems by:

[0079] Improving performance on the client side;

[0080] Enabling the creation of dynamic, real-time Web applications; and

[0081] Providing the ability to create a wide variety of user interface components.

[0082] With Java, developers can create robust User Interface (UI) components. Custom “widgets” (e.g., real-time stock tickers, animated icons, etc.) can be created, and client-side performance is improved. Unlike HTML, Java supports the notion of client-side validation, offloading appropriate processing onto the client for improved performance. Dynamic, real-time Web pages can be created. Using the above-mentioned custom UI components, dynamic Web pages can also be created. Sun's Java language has emerged as an industry-recognized language for “programming the Internet.” Sun defines Java as: “a simple, object-oriented, distributed, interpreted, robust, secure, architecture-neutral, portable, high-performance, multithreaded, dynamic, buzzword-compliant, general-purpose programming language. Java supports programming for the Internet in the form of platform-independent Java applets.” Java applets are small, specialized applications that comply with Sun's Java Application Programming Interface (API) allowing developers to add “interactive content” to Web documents (e.g., simple animations, page adornments, basic games, etc.). Applets execute within a Java-compatible browser (e.g., Netscape Navigator) by copying code from the server to client. From a language standpoint, Java's core feature set is based on C++. Sun's Java literature states that Java is basically, “C++ with extensions from Objective C for more dynamic method resolution.”

[0083] Another technology that provides similar function to JAVA is provided by Microsoft and ActiveX Technologies, to give developers and Web designers wherewithal to build dynamic content for the Internet and personal computers. ActiveX includes tools for developing animation, 3-D virtual reality, video and other multimedia content. The tools use Internet standards, work on multiple platforms, and are being supported by over 100 companies. The group's building blocks are called ActiveX Controls, small, fast components that enable developers to embed parts of software in hypertext markup language (HTML) pages. ActiveX Controls work with a variety of programming languages including Microsoft Visual C++, Borland Delphi, Microsoft Visual Basic programming system and, in the future, Microsoft's development tool for Java, code named “Jakarta.” ActiveX Technologies also includes ActiveX Server Framework, allowing developers to create server applications. One of ordinary skill in the art readily recognizes that ActiveX could be substituted for JAVA without undue experimentation to practice the invention.

[0084] The present invention enables personal content management and is able to aggregate content of one's choice and have it accessible on any device. More particularly, the present invention enables wireless access of all services. Accordingly, a broad aspect of the present invention allows any user to send or receive the content of their choice from any device, in particular wireless devices including, but not limited to, Personal Digital Assistants (PDA's) and Wireless phones.

[0085] In this regard, one embodiment of the present invention includes the following parts:

[0086] 1. A customizable information retrieval engine that allows users to aggregate content of their choice from any web site in existence. The content includes but is not restricted to text (i.e. news headlines, hyperlinks in web-pages), secure account information (i.e. email, bank accounts, utilities, and stock portfolios), services (i.e. maps, directions, weather, web searches), electronic mail, financial transactions (i.e. online shopping, buying, selling, trading, auctions, barters, comparisons) and other dynamic tasks that involve interaction of the users with other web-based (client and server side) services. The aggregated content is displayed in a customized web-based habitat, which is amenable to presentation and content customization through an intuitive interface.

[0087] 2. An interface of the above mentioned web-based habitat to a wireless enabling engine that takes the content of the habitat and renders them on all devices, especially wireless devices. All the content in the custom habitat is presented in a format that is supported by the wireless devices and therefore allows any content to become wireless enabled.

[0088] The customized information retrieval engine allows the aggregation of any content into a customized web-based habitat and the wireless enabling engine takes all the content in the habitat and makes it wireless enabled. Therefore, one embodiment of the present invention allows the use of the above mentioned habitat as a conduit that allows any content on the web to become wireless enabled instantaneously.

[0089] FIG. 2 illustrates a process 200 for selection and formatting of web content for remote viewing. A user is allowed to provide information that specifies general or specific content to be retrieved for online or offline viewing. For example, the user can specify retrieval of a particular web page when the web page changes/is updated. As another example, the user can make a general request to download articles relating to a particular topic, such as stock exchange information, airline fares, etc. Preferably, the user is allowed to submit the user-defined information from at least one of the wireless device and a hardwired device such as a personal computer. It should be noted that such content can be anything on or transmittable via the Internet. By way of example only, such content can be hyperlinks, images, text, tables, secure information such as account information, email, and audio and video data. The user is allowed to select, customize and/or edit the content from any device, wired or wireless.

[0090] The user-defined information is received in operation 202 and in operation 204 is used to retrieve content from one or more web sites. If particular content is specified in the user-defined information, the pertinent web page is accessed and the particular content is downloaded. If content is generally requested, a search engine can be used to find the information.

[0091] The retrieved content is aggregated at a network server located remotely from the user in operation 206. The network server acts as a gateway through which any content from the world wide web is collected and converted into a format amenable to the wireless device. Preferably, the aggregated content is amenable to presentation and content customization through a user interface. In operation 208, the content is formatted at the network server for display on a wireless device. The wireless device can include any type of device capable of receiving information where at least one of the communication links is wireless, such as a wireless telephone, Personal Digital Assistant (PDA), handheld computer such as a handheld PC, a pager, a device connected to a wireless modem, etc. The formatting style can be specific to the wireless device used by the particular user. Preferably, the content is formatted for display on a plurality of wireless devices so that the user can use any of a plurality of wireless devices to access the information. In operation 210, the formatted content is transmitted to a wireless device for display on the wireless device. If the content has been formatted for a plurality of wireless devices, the wireless device sends a request for a particular type of formatting associated with that type of device. Preferably, transmission cost and reliability, as well as transmission time, are customizable and are user-customizable.

[0092] In one embodiment of the present invention, a user selects which content is aggregated in real time. As an option, customization of the web-based habitat can be performed utilizing the wireless device or a computer of the user.

[0093] In another embodiment of the present invention, the content is aggregated on a portal page unique to the user. The portal page displays the content that the user has specified, and can include the interface that allows the user to specify the information. This portal page is fully customizable by the user and can be accessed by any device, whether wireless or hardwired. As an option, the retrieved content can be updated after a predetermined amount of time has expired. The content would then be retrieved, formatted, and transmitted to the wireless device.

[0094] In an embodiment of the present invention, the user is allowed to interact with the content. Thus, the user is not only able to receive information, but is also able to send information from the wireless device. For example, the user can be allowed to fill out forms and fields, make purchases (buying and selling), read and reply to emails, etc. using the wireless device. In yet another aspect of the present invention, an alert is sent to the wireless device upon occurrence of a prespecified condition. By way of example only, an alert can be sent at a predetermined time prior to departure of an airline flight. The alert can also be a reminder of an appointment. Further, an alert can be sent upon a stock price reaching a predetermined level or an auction price reaching a certain amount.

[0095] FIG. 3 depicts a preferred operating environment 300 for one or more portable wireless devices 302 in connection with a host computer system 304, where the habitat is stored and manipulated. The host computer is connected to remote sources of data information on the Internet 306.

[0096] The host computer system preferably includes a peripheral interface adapter that provides for the bi-directional transfer of the data via an interconnect line 308 to an external transceiver 310 that supports wireless communications with one or more wireless devices. The transceiver, in a simple embodiment, implements a low-power 900 Mhz or 2.4 Ghz transceiver for short-range communication with the wireless devices. In a preferred embodiment, however, the transceiver is part of a cellular or digital telephone network that allows communication between the host computer and the wireless devices at great distances, including geographically remote locations.

[0097] The wireless device is preferably constructed with a plastic case housing a display panel, a mini keyboard, a pointing device, and pointer selection buttons. The display panel is preferably an active matrix liquid crystal display (LCD) or dual-scan super-twist nematic display suitable for rendering color images at a resolution of about 640. times. 480 pixels or greater. Low cost display panels with reduced resolutions and only monochrome display capabilities can also be utilized. In all events, the display panel is preferably light-weight, reasonably sturdy, and suitable for the graphic display of at least computer video data.

[0098] The mini keyboard may be of any number of conventional configurations providing for alpha-numeric keyed data entry in a relatively small two dimensional form factor. Ultimately, the mini keyboard may be replaced with a smaller number of programmable function keys that programmatically adapt as appropriate to the function of any current application displayed by the display panel. The mini keyboard may be entirely replaced with a virtual keyboard implemented by the display panel in connection with a touch screen sensor mounted in the case of the wireless device. Thus full function and specialized function data entry keys can be created as necessary or desired in support of the use of any application displayed by the display panel.

[0099] A pointing device, such as a power point tracking device or track ball can be provided to allow the wireless device to be easily held while the pointing device is manipulated. Similarly, pointer buttons are preferably configured in close proximity to the pointing device to again allow easy access and use while the wireless device is held. Preferably, pointer buttons may be programmable in defining the function performed or recognized in response to each press of the buttons.

[0100] A preferred embodiment of the present invention may also include an audio pick-up transducer and pair of speakers in support of multimedia utilization of the wireless device.

[0101] A transceiver antenna is preferably mounted within the case. Although the display panel and other electronics located within the case may be electromagnetically shielded, the cross section of such shielding should not significantly affect the efficiency of the antenna. Where the shielding presents a problem or the display table is operated near noise sources or at the near limit of the service area, the antenna may be constructed to permit external extension.

[0102] The flexibility and functionality of the wireless device may be augmented by the addition of a PCMCIA peripheral card. As conventional PCMCIA cards are removable, the function or functions that can be added to the wireless device depends on the implementation of the PCMCIA card itself. A PCMCIA card may implement a cellular phone interface would allow the wireless device to be operated at great distance from the host computer through a combination of air-links and land-lines that route to the host computer system in a conventional manner. The PCMCIA card may also implement an analog or digital modem or other high-speed serial or parallel interface that can connect either directly to the host computer when the wireless device is conveniently close to the host computer or remotely through any combination of air-links and land-lines. The PCMCIA card may also implement supplementary functions to augment the multimedia capabilities of the wireless device, other communications protocols and data connection systems, and upgrades to the basic capabilities present in the wireless device, including new encoding, encryption and compression capabilities.

[0103] A connector can provide external power access that provides operating power and, potentially, power for recharging batteries provided within the case of the wireless device. Other connectors may provide for conventional keyboard, mouse and joysticks, as external peripherals, to be fully integratable into the overall function of the display table.

[0104] While the use of small high energy density rechargeable batteries is preferred, the power consumption requirements of the wireless device can be managed closely to minimize the power load that is required to be supported in the normal operation of the wireless device. The refresh frequency and brightness of the display panel may be reduced during periods of perceived inactivity. The transmission power produced by the on board transceiver connected to the antenna may be selectively reduced to meet a minimum acceptable noise margin. This may have the additional benefit of reducing the effective size of the service area to an area specifically appropriate to the unique location of a particular wireless device, thereby reducing the possibility of signal interception and unnecessary cross-talk. Finally, subsystems such as the PCMCIA card and multimedia support circuitry providing signal and power to the speakers and transducer can be selectively powered down when their use is not needed or desired. As a result, the wireless device should have a battery life of from two to four hours.

[0105] The internal electronic control system of the wireless device is preferably constructed as a low-cost embedded microprocessor control system utilizing a main processor bus to provide a data and control interconnect between a microcontroller CPU and a main memory bank. The microcontroller can be directly implemented utilizing any of a wide number of a existing low-power, high-performance microcontrollers, such as the Intel 80C51 series, the Intel 80C196KB high performance CHMOS microcontroller, or the like.

[0106] The main memory is preferably constructed utilizing approximately two to ten megabytes of low-power dynamic RAM memory. While the wireless device will support the execution of almost any number of complex applications, the resident main memory need not be of significant size. The application programs are executed on the host computer while, in the preferred embodiment, the operation of the wireless device is strictly limited to the terminal display of graphic and related data. Thus, the main memory is preferably sized sufficient to allow execution of a control program implementing primarily the display function of the tablet independent of the actual execution of the application program. Consequently, not only is the size of the main memory both reduced and largely non-critical in relationship to the complexity and type of application programs executed by the host computer, but the microcontroller is not constrained by compatibility issues with regard to the type of CPU utilized by the host computer or the specific type and version of the operating system executed.

[0107] Some combination of non-volatile RAM and ROM memory may be provided to store at least a portion of the control program that is executed by the microcontroller. The non-volatile RAM/ROM memory preferably stores at least a portion of the control program sufficient to enable the microcontroller to download the remaining portions or fall new image of a control program from the host computer. To permit future upgrades of event the permanently resident portion of the control program, non-volatile RAM memory can be utilized to allow field upgradability.

[0108] A conventional power controller provides for the regulation of power to the control system from either an external power source or the onboard battery. The power controller preferably is programmable by the microcontroller to selectively provide power to separate subsystems of the controller. The microcontroller is therefore able to effectively manage power consumption by the control system as a whole. Specifically, independent power regulation may be provided for an audio subsystem, PCMCIA interface and a short range transceiver. Power may be regulated selectively for other components of the control system where continued or excessive power consumption is unnecessary or undesirable.

[0109] A generally conventional video graphics controller is provided as the control interface for managing the operation of the display panel. The video graphics controller may internally implement a simple hardware graphics adaptor or more complex hardware graphics accelerator for enhancing the effective speed of the video graphics controller and, in general, the perceptible performance of the wireless device.

[0110] Depending on the resolution supported by the video graphics controller, including color depth, a conventional video memory array is provided as frame and scratch-pad memory for use by the video graphics controller. Generally, a minimum of 1 megabyte of video memory is sufficient to support a display panel resolution of 640. times. 480 at a color depth of 8 bits. The video memory may be expandable to two, four or more megabytes of memory as appropriate to support the function of video graphics controller.

[0111] A conventional LCD driver circuit is also connected to the video graphics controller to generate the control and driver signals necessary to directly operate the display panel.

[0112] Finally, a touch screen interface may be provided to support a touch screen function in connection with the display panel. The video graphics controller may include circuitry for operating and responding to the touch screen interface as needed to digitally represent a screen touch. This information is then available for use by the microcontroller in much the same manner as any other pointing device information is made available by the microcontroller.

[0113] The audio subsystem preferably includes the conventional functionality of multimedia peripheral cards for personal computers. The preferred supported functions include digital-to-analog conversion and power amplification of stereo audio channels as appropriate to drive the speakers. The audio subsystem preferably also includes an analog-to-digital converter connected to the transducer. Additional analog or digital signal processing circuitry may be provided to reduce noise and eliminate feedback from the speakers prior to or after the analog-to-digital conversion is performed by the audio subsystem.

[0114] Finally, a PCMCIA interface provides a preferably 16-bit wide, high-speed parallel interface between the connector or connectors supported by the PCMCIA slot and the main processor bus. The PCMCIA interface itself is preferably implemented through the use of a conventional PCMCIA interface controller chip.

[0115] Any number of applications can be executed concurrently by the host computer in accordance with the normal operation of the operating system or, as in the preferred embodiment, subject to an effective partitioning of the operating system execution states to support concurrent execution of the multiple applications by an otherwise single-user operating system. In both events, the applications present calls to the operating system including, in particular, calls relating to the display and update of images on a respective display. These displays, however, are logical displays that are mapped by the operating system into a single master logical display space utilizing, as appropriate, windowing and desktop paradigms for the presentation of the composite master logical display. That is, the master logical display is drawn by the operating system by a series of appropriate low-level display driver calls to a display driver.

[0116] In the preferred embodiments of the present invention, a pseudo-display driver is provided to manage the detailed presentation of a master logical display within a window of another master logical display corresponding to another partition of the execution environment supported by the operating system. The pseudo-display driver effectively operates to intercept low-level display driver calls from any or all of the operating system execution partitions. The output of an executing partition may be directed to an independent display, such as the local video display or passed substantially unaltered to a long-range transceiver driver. In an initial implementation of the present invention, the long-range transceiver driver and the low-power transceiver is instantiated once for each wireless device supported by the host computer system. Thus, the display driver calls from a single executing partition of the operating system, or multiple partitions operating in collaboration, are passed as a driver call stream to the long-range transceiver driver for transmission to a corresponding wireless device. Outbound audio data and inbound pointer and audio data are processed through the long-range transceiver driver. Outbound audio data and inbound input and audio data may be transferred directly between the operating system and long-range transceiver driver subject to maintaining the correlation between the applications executing within the execution partition of the operating system associated with the particular instantiation of the long-range transceiver driver corresponding to that partition. Consequently, a proper association both for inbound and outbound data for specific applications is maintained through the operating system as between the host computer system and any number of wireless devices.

[0117] A preferred alternate embodiment of the present invention preferably provides for a single long-range transceiver driver that is effectively aware, as is the pseudo-display driver, of the multiple partition execution space of the operating system. This alternate long-range transceiver driver preferably supports a multi-channel spread spectrum transceiver. Display and analog output data associated with execution partitions of the operating system respectively directed for transmission to a particular wireless device to implement the low-level display driver. Consequently, the appropriate physical display, either the local video display or a wireless device is updated consistent with the ongoing execution of the corresponding operating system partition. The long-range transceiver driver may further provide for variable encryption and decryption of the low-level driver call data streams that pass through the driver. Destination signatures may also be included into the data streams to specifically identify the particular recipient host computer and wireless device that are intended to be exclusively communicating over a particular channel supported by the transceiver. This provides both security over an appropriate interception of the transmitted data as well as secure validation that data streams are being sourced and received by the intended participants.

[0118] Nominally, the client application itself is charged with the responsibility to decode, decrypt or decompress data for presentation. Various graphical images transmitted to browser applications are encoded and/or compressed using various lossee or lossless algorithms to substantially reduce the transmitted data size. In a relationship to this class of application, one embodiment of the present invention implements a processed data bypass that allows the encoded, encrypted or compressed data as received by the host computer to be transferred in an unaltered form to a wireless device. Since data transferred in an encoded, encrypted or compressed form is done subject to a public algorithm specification, no compatibility issues arise by allowing the microcontroller with implementing the unencoding, decrypting or decompression algorithm yet maximizing the effective utilization of the bandwidth connection between the wireless device and host computer system.

[0119] A complication arises particularly in Web browser applications. There, the rendered display is content dependent. Therefore, display dependencies are resolved dynamically by the application based on the final representation of any unencoded, decrypted and decompressed data. In such circumstances, the host computer system must provide for fall processing of the received data in support of the otherwise ordinary operation of the application as needed to produce a finally determined impression of the information to be displayed by a wireless device. This is handled in the present invention on the host computer side through the further implementation of the host system detail. A socket driver, conventionally referred to as a WinSock driver in relationship to Microsoft operating systems, manages a network socket connection to a remote computer system that is the source of encoded, encrypted or compressed data. The WinSock driver effectively supports bi-directional data transfer between the driver and operating system in support of the pseudo-display driver and an exemplary Web browser application. The WinSock driver is typically merged with the operating system to extend the application program interface (API) that is presented to the browser application.

[0120] The WinSock driver is preferably modified to identify objects such as compressed data images from the inbound socket data stream. The object is identified by the driver not only as being subject to immediate bypass to the long-range transceiver driver, but further that the socket data stream carrying the object is destined for a particular application. Thus, the long-range transceiver driver is provided with both the bypassed data object and at least an identification of the particular wireless device that is to receive the object. The data object as passed to the long-range transceiver driver and, in parallel, to the operating system with a unique identification tag generated by the WinSock driver. This tag is associated with the data object in the socket data stream ultimately for use by the pseudo-display driver. Preferably, the data object tag and the communication of the tag to the pseudo-display driver is provided logically separate from the socket data stream that is provided through the operating system to the browser application. Consequently, an entirely conventional browser application may be utilized in connection with the present invention without loss of performance or compatibility. Data objects received by the browser application are therefore conventionally unencoded, decrypted, and decompressed and used if and as necessary to resolve dependencies on, for example, the size and location of a graphic image in relationship to text within the browser applications current logical display. That is, the browser application processes the received socket data stream and produces a series of operating system calls to define the appearance of the logical display window controlled by the browser application.

[0121] Operating system display calls are further reduced by the operating system to low-level display driver calls that are passed to the pseudo display driver. Based on an examination of the various data objects identified to the pseudo-display driver in connection with the low-level display driver calls, respective unique identifying data object tags are identified by the pseudo-display drive. Each tag, as identified, is used to replace the unencoded, decrypted or decompressed representation of the corresponding data object. Thus, only display driver calls referencing data object tags and untagged data are processed through the pseudo-display driver to the long-range transceiver driver for transmission to a wireless device.

[0122] In the preferred embodiment of the present invention, the long-range transceiver driver operates to transmit fixed block size data packets that together convey messages to a wireless device. A message can be a data object received from the WinSock driver. Other messages include a low-level device driver call and as appropriate for the call, a display object tag or an untagged data object as received from the pseudo display driver. A tagged data object identified and provided from the WinSock driver will therefore be at least queued for transmission to a corresponding wireless device prior to a display object tag being provided by the pseudo display driver to the long-range transceiver driver for transmission to the same wireless device. Furthermore, the latency between the transmission of the data object itself and transmission of the tag allows a quite adequate amount of time for the microcontroller to receive and, as appropriate, process the data object into an unencoded, decrypted or decompressed form. The actual latency incurred at different times will be determined by operating system and browser application, executed and latencies that control the generation of display driver calls by the operating system to the pseudo display driver to pass a data object tag to the long-range transceiver driver.

[0123] Referring now to FIG. 4, a top level software detail 400 of the control program is shown in accordance with a preferred embodiment of a wireless device. An embedded control system 402 is the central component of the control program. Preferably, the embedded control system implements an interrupt-driven, light weight threaded core that efficiently supports operation of a long-range transceiver driver 404, a data object decompressor 406, a video driver 408, an input event driver 410, and an audio driver 412. Multiple threads are implemented by the embedded control system primarily in support of overlapping and out-of-order data transfers with respect to the long-range transceiver driver. The long-range transceiver on the wireless device supports a single active transceiver data channel. The tag provided with or in place of a data object includes an identification of the particular application data stream associated with the object. The embedded control system allocates a separate thread for handling each uniquely identified application data stream and thereby conveniently managing overlapping and out-of-order data transmissions. While a non-threaded core could be utilized, the use of a conventional multi-threaded core within the control system is preferred as providing a more convenient transactional platform for supporting multiple application data streams.

[0124] Tagged data objects received by the long-range transceiver driver are transferred to the video image decompressor. A number of different algorithms may be implemented by the video image decompressor to be selectably used as appropriate for the particular nature of the data object passed from the long-range transceiver driver. In many instances the data object will be a graphics image presented in any of a number of compressed forms. Typically, the compressed data object itself identifies the type and version of the algorithm necessary to decompress the data object. The object is decompressed and the resulting video image and unique object identification tag are cached by the embedded control system for subsequent use.

[0125] Untagged data objects and bare data object identification tags are passed directly from the long-range transceiver driver to the embedded control system for evaluation. Decompressed graphics images are substituted for their corresponding unique identifying object tags and an ordered set of low-level drawing commands and data effectively corresponding to the received data stream is then provided to the video driver. In turn, the low-level display driver commands and data are interpreted and executed by the video driver to directly provide for the presentation of a corresponding image by the display panel.

[0126] The data stream received by the long-range transceiver driver may also include low-level commands, data and potentially unique object tags that are to be directed to the audio driver. The commands and data directed to the video driver and audio driver are independently routed by the embedded control system to the appropriate driver. Where the application data stream includes audio driver commands that reference a unique object identification tag, the embedded control system again searches the local object cache for a corresponding expanded object. As with graphic data objects, compressed audio data objects also encode an identification of the decompression or decoding algorithm necessary to process the data object. Thus, the object decompressor processes the compressed audio data object utilizing the appropriate algorithm. The audio commands, untagged data objects and substituted decompressed data objects are then provided to the audio driver for execution and presentation by the speakers.

[0127] The audio driver also produces an inbound application data stream from the audio transducer, when enabled. A resulting digital data stream is passed from the audio driver to the embedded control system for return within a message of an appropriate application data stream through the long-range transceiver driver to the host computer. The appropriate application data stream is identified potentially from the application data stream that last previously enabled operation of the transducer through the audio driver. The digitized audio data gathered by the audio driver may be passed directly through to the long-range transceiver driver for transmission. Alternatively, the embedded control system may direct the digital audio stream through the object decompressor to perform audio compression prior to transmission. By compressing this audio data prior to transmission, a substantial portion of the bandwidth available to the long-range transceiver is preserved while introducing minimal latency due to the operation of the compression algorithm.

[0128] The input event driver receives, manages and provides keyboard, pointer and touch screen input data to the embedded control system. This input data is directed by the embedded control system to the long-range transceiver driver for transmission in the form of call messages to the host computer system. However, such input data is not immediately associated with a particular application data stream by the embedded control system. Rather, the input data is identified only generically by type of information prior to transmission. In accordance with the preferred embodiments of the present invention, the operating system appropriately determines the particular application that is currently set to receive input from the keyboard, printer and touch screen. In certain circumstances, this may be the operating system itself and in other circumstances, the currently active application execution partition of the operating system. Consequently, the control program operates to effectively support any number of applications being executed on the computer system implicitly by the identification of their respective application data streams as received by the long-range transceiver driver. The embedded control system is also capable of associating inputs specifically with the transceiver data streams as appropriate to maintain the integrity of each individual application execution partition of the operating system.

[0129] In the control and data flow relationships between the host computer system and a wireless device, the network operating system component of the operating system executed by the host computer system implements a TCP/IP or similar network communications protocol. A socket intercept module is provided in accordance with the present invention to filter, identify and bypass compressed data objects subject to predefined conditions to a wireless transceiver subsystem. The predefined conditions include the specific data object compression forms that can be handled by a particular model of a wireless device and that a wireless connection has been established with a valid wireless device. All of the data received by the socket intercept module is identified by the number of the particular socket connection that receives the data. All conventionally valid data received by the socket intercept module is provided to the application that owns the corresponding socket connection as determined and defined through the core operating system, including the graphical user interface (GUI) module. An input event multiplexer module, established in accordance with the present invention, receives input event data and associates the events with a corresponding execution partition through the core operating system.

[0130] The application executing within a partition of the core operating system may maintain a private data space. Where the application is a browser, a browser image cache is supported to hold decompressed image data that can be repeatedly referenced on repeated application access requests for the corresponding compressed data object.

[0131] Independent of whether a compressed data object is requested from and immediately received through the socket intercept module or from a browser image cache, particularly as determined by the browser application itself, the core operating system issues low-level graphics calls and accompanying data through a graphics calls intercept module. Where the local display of the host computer system is controlled by an application within the current execution partition of the operating system kernel, the display graphics calls and object data are passed as received to the software display driver for final processing against the display hardware.

[0132] The unique identifying tag assigned to a data object by the socket intercept module is identified by the graphics calls intercept module. The identifying tag is then substituted for the actual data object within the application data stream of low-level graphics calls and data. The resulting modified data stream is then directed to the wireless transceiver subsystem.

[0133] Similarly, the operating system kernel directs low-level audio calls and data to an audio calls intercept module. Any tagged data objects, as previously identified by the socket intercept module, are held in a dedicated cache by the operating system kernel. When a tagged data object is cached, the tag is provided to the audio calls intercept module. The low-level audio calls and data may be passed locally to the software audio driver for final processing by the audio hardware. The low-level audio calls, with all tagged data objects substituted by their tags, may be separately or collaboratively transferred to the wireless transceiver subsystem.

[0134] The wireless transceiver subsystem receives an assigned channel of data. The channel data is provided to a multi-threaded socket data handler that operates to segregate the channel data into separate threads corresponding implicitly to the separate socket data threads as then identified by the socket intercept module. The decompressor cache manager effectively multiplexes between the different data threads to identify compressed data objects. As each compressed data object is encountered, the decompressor cache manager appropriately decompresses the data object and stores the decompressed object in an object cache as established in the main memory by operation of the control program. If the object cache fills, given the finite amount of main memory allocable as the cache, the decompressor cache manager sends an invalidating message to the wireless transceiver subsystem with an identification of the image tag of the least recently used object in the object cache that is being invalidated. This message is returned to the graphics calls intercept module that manages a list of all valid tagged objects within the object cache. Thus, whenever the application and GUI code module directs the transfer of an untagged data object, a tagged compressed data object can be quickly formed from the data provided by the application and GUI code module and forwarded through the wireless transceiver subsystem.

[0135] When the thread handler provides a tagged data object reference to the decompressor cache manager, a list of valid tags maintained by the cache manager is checked to select a corresponding data object from the object cache for transfer to the display control system. These data objects, in connection with the graphics call stream passed through the wireless transceiver subsystem are then utilized by the display control system to compose a composite display image that is presented on the panel by the display hardware.

[0136] Audio sound clips are handled through the decompressor cache manager in a manner generally similar to graphics data objects. The socket intercept module identifies audio clips within the network channel data stream as another specific form of compressed data object. The compressed audio data object is bypassed immediately to the wireless transceiver subsystem while, at the same time, being passed to the application core operating system module. The audio calls intercept module maintains a list of the data object tags associated with bypassed compressed audio data objects. As references to tagged data objects are received by the audio calls intercept module, tag substitution is again performed prior to the audio low-level call being forwarded to the wireless transceiver subsystem.

[0137] The threaded socket data handler receives and distinguishes a message as containing an audio call and a compressed audio data object. The threaded socket data handler 168 passes the object to an appropriate multiplexer channel of the decompressor cache manager for storage in the object cache as any other data object. However, unlike graphics data objects processed through the decompressor cache manager, compressed audio data objects may be stored directly in the object cache where the necessary decompression routine for the audio data is supported directly in the hardware of the audio output module. Alternately, the decompressor cache manager may operate stream decompress and pass the audio data object to the audio output module without storing the object in the object cache. The decompressor cache manager may alternately perform a preprocessing function on compressed audio data objects to transform the compression representation of the object or partially decompress the object to a point where dedicated hardware provided in the audio output module can complete any required further decoding and decompressing as the audio data object is streamed to the audio hardware.

[0138] The audio hardware includes the transducer of the wireless device. Inbound audio is routed by the audio hardware to the audio output module. In a preferred embodiment of the present invention, the audio output module compresses and encodes the inbound audio stream to the extent supported by the hardware of the audio output module. The stream of audio data objects produced from the audio output module are passed to the wireless transceiver subsystem for transmission through the wireless subsystem. These inbound audio data objects are then passed to the audio calls intercept module for selective reintegration into an appropriate application execution partition of the core operating system module. Consequently, the executing application expecting audio input from the audio hardware receives the audio data object stream as though acquired through the software audio driver and audio hardware.

[0139] Finally, an input event module receives and processes keyboard, pointer and touch screen input information into a corresponding set of inbound input event messages that are forwarded ultimately to the input event multiplexer module. The received input events are matched through the input event multiplexer module to the appropriate application execution partition of the core operating system module. Thus again, the partition executing application receives the input events as though locally generated by the host computer system itself.

[0140] A combined data flow 500 further detailing the handling of compressed data objects is shown in FIG. 5. As shown, the flow handles both network received and locally or host generated compressed data objects. Network originated compressed data objects are parsed from the inbound network data steam in operation 502. As individual objects are identified and determined to be of a compressed form that can be handled directly by the control program, the compressed data objects are routed in operation 504 for copy down 506 directly to the control program.

[0141] The inbound network data stream is processed generally by the operating system of the host computer and passed in operation 508, as appropriate, to applications 510, 512 executing on the host computer system. In the case of a browser application, or the like, compressed data objects are frequently requested and received from the inbound data stream. Conversely, a local application may not receive any externally originated compressed data objects that can be directly handled by the control program. Indeed, the local application need not receive any data from the network data stream. In any event, the browser application and local application may manipulate any received data objects, including decompressing and decoding the objects. Both the browser and local applications may function to generate new data objects.

[0142] Both the browser application and local application ultimately issue calls to the operating system executing on the host computer. These calls and accompanying or referenced data objects are received by the operating system in operation 514. A series of low-level calls and data objects responsively generated by the operating system is then passed to the pseudo-display driver, for example. The series of calls and objects are processed to particularly distinguish data objects and call commands in operation 516.

[0143] Data objects are examined in operation 518 to distinguish between tagged and untagged data objects. At a minimum, tagged data objects can be identified by comparing at least a signature of the data object to a local list of known tagged data objects maintained by the pseudo-display driver. Where a signature match is found, the corresponding tag is provided in place of the data object in operation 520. The data object tag is then prepared as part of a message for copy-down to the control program.

[0144] Where a data object fails to match a known tagged object signature, the data object is assumed to have been created locally by the applications. Such local data objects are first compressed in operation 522 and prepared in operation 524 for copy-down to the control program.

[0145] As each message is prepared for copy-down individual messages are identified by thread and multiplexed for transmission to the wireless transceiver subsystem.

[0146] Once received by the control program, a message is examined in operation 526 to determine whether the message includes a data object tag. A control program local list of known object tags is scanned to determine if a corresponding pre-decompressed data object is present in the object cache. When a matching data object tag is found, the tag is replaced in operation 528 with a pointer to the corresponding pre-decompressed data object stored in the object cache. Preferably, the data object is identified by reference thereby reducing memory-to-memory data copies of the tagged data objects. A message is then further examined in operation 530 for the existence of compressed inline data objects. Each identified inline data object is then decompressed in operation 532. The inline data object may then be selectively provided with a local data object tag generated by the control program. By default, inline data objects that are greater than an empirically selected default size of five kilobytes are tagged by the control program. A flag provided as part of a message including a data object may be used to explicitly determine whether the inline data object is to be tagged or not by the control program. The data object, regardless of whether tagged or not is then cached in the object cache in operation 534. Untagged data objects will have the lowest priority for remaining in the cache. Where a tag is generated by the control program, a copy of the new data object tag is added to the local lists of known data object tags that are utilized in identifying tagged data objects on both the host computer system and wireless device. These lists are updated as appropriate whenever tagged data objects are newly cached or expired from the cache immediately or by cache flush. The host list is preferably updated with respect to control program generated tags based on a memory management model executed as part of the pseudo display driver. Alternately whenever the control program updates its tag list, a message directing a corresponding update to the tag list maintained on the host computer system can be generated and returned by the control program.

[0147] Finally, a reference to the cached data object is placed into the stream of application calls within the message. The calls and referenced data objects are then implemented in operation 536 by forwarding to the display control system module. Display calls that reference cached data objects operate to copy at least selected portions of the data objects to the memory as appropriate to carry out the application call. Consequently, the density of data objects transmitted to the wireless device is maximized for both network and local host generated data while minimizing the display latency impact due to the limited available transmission data bandwidth between the host computer system and wireless device.

[0148] The present invention allows a user to create an information portal whose source and content is completely customizable. Information on the web exists in the form of hyperlinks that appear in different web sites. A news site for example may contain headlines that are hyperlinks to their detailed exposition. In typical portals, the user chooses from a pre-determined set of headlines collected from a pre-determined set of web-sites. The user has no control over either the web-sites he/she gets the content from or the headlines that are taken from those web-sites. The present invention allows the user to completely configure both the source and content that he/she wants on his/her own portal.

[0149] FIG. 6 illustrates a process 600 for generating a customized network user interface according to one embodiment of the present invention. A management interface is provided in operation 602. The management interface allows a user to select and manage information that is displayed on an information screen and viewed by the user. The management interface includes information of at least one content source which can be selected. It should be noted that such information can include such things as portions of web pages, links to web pages, or any other type of information. Such a content source can be a web page or any other content source. In operation 604, the user is further allowed to select portions of the information of one or more of the content sources. The selected information of the content source may then be marked in operation 606. In operation 608, such marked information is stored for subsequent retrieval in operation 610. Various changes in the marked information may be checked in operation 612, and in operation 614, the marked information may then be displayed on the information screen.

[0150] In one embodiment of the present invention, the information screen may include a plurality of different views. Each view may contain at least one window for displaying the marked information. Further, the user may be allowed to select, maximize, minimize, refresh and edit the content of the window.

[0151] In another embodiment of the present invention, the user may be allowed to share the views with other users such as via electronic mail or by permitting access to the views. As an option, the marked information may be presented on the information screen over a configurable number of days in the past. Further, the user may be allowed to drag and drop the information from the customizing interface to the information screen. The information may also be marked upon dropping the information in the information screen.

[0152] In still yet another embodiment, the step of marking the selected information may include determining an address of the selected information and a table, row, column, and/or cell of the selected information. Further, the step of checking for change of the marked information may include the steps of determining whether the content of the marked information has changed and determining whether the format of the marked information has changed. As yet another option, the step of checking for change of the marked information may be performed at predetermined intervals.

[0153] According to a preferred embodiment of the present invention, the user is presented with a page that contains the user's information of choice from an arbitrary number of different sources and presented in a completely customizable format. The page consists of different “views” where each view in turn contains multiple windows. The number of views and the number of windows in each view can be configured.

[0154] Each particular window contains hyperlinks that have been selected by the user from web-sites of his/her choice. A window may for instance be dedicated for international news and could contain hyperlinks selected by the user from any number of web-sites of his/her choice. The user has complete freedom in selecting the source of his/her content (i.e. the web-site) and the content from that source (i.e. the hyperlinks).

[0155] FIG. 7 is a flowchart of a process 700 for allowing a user to customize an information portal according to one embodiment of the present invention. When the user wishes to add content, a web-page chosen by the user is presented in operation 702. In operation 704, the user is then allowed to select the headline or hyperlink of his/her choice and simply drags and drops it into his/her portal. From that point on, in operation 706, the content from that headline or hyperlink will be brought to the user's portal regularly. In operation 708, a check for any change or update of the content is made. If the content changes or is refreshed, the new content will be brought to the user. In operation 710, the user is further allowed to edit the content of his/her portal at will by adding or deleting headlines, moving them from one window to another within a view or moving them to other windows in different views.

[0156] Another embodiment of the present invention includes the following parts: (a) An interface that displays the user customized information, (b) an interface that allows the user to select and manage the information of choice, (c) a mechanism for marking selected information contained in a web-page (d) a method for communicating that information to the backend servers that process and store that information, (e) a mechanism for the storage of the selected information (f) a mechanism for regularly retrieving selected information and (g) a mechanism for checking for change in the content or the format of the selected sources of information.

The User Interface to Display Preferred Content

[0157] The user interface comprises “views”, each of which contain multiple windows. The number of windows in a view is completely configurable. The user may create or delete as many views as he/she may desire. This user interface allows a user to cleanly categorize related information within individual windows and views. This provides a user one place to access all of his/her favorite information and content from the web. This content includes (but is not limited to) (a) News and Information headlines (of all sorts) (b) Information about email, bank and other accounts (c) Information about shopping and comparison of rates and prices (d) Graphs, Images, Sounds or any other media.

[0158] This content is presented to the user with an ability to edit and manage it intuitively and interactively. Some of the features of the management process include (a) a presentation of the user's selected information over a configurable number of days in the past (b) an ability to select, maximize, minimize, refresh or edit the content of individual windows (c) to “publish” user's views into a directory of views and (d) to share these views with other people by emailing them the views.

The Interface for Selection and Management of Preferred Content

[0159] The interface that allows the user to create his/her customized portal is based on an intuitive drag and drop capability. The user simply selects the sources or headlines of choice and drags and drops them into windows and views of choice. The drag and drop feature also makes customization very easy for the user, allowing quick compilation and management of their preferred content. There are two levels of selection and management provided, default and advanced.

[0160] Referring to FIG. 8, in a default mode process 800 for allowing selection and management of preferred content according to one embodiment of the present invention, a user is presented with a set of web-sites or other sources of content in operation 802. In operation 804, the user is allowed to select a site and then drag and drop it into a window of choice. Once that is done, pre-selected content from that source is automatically added to the window in operation 806.

[0161] FIG. 9 is a flowchart of an advanced mode process 900 for allowing selection and management of preferred content according to an embodiment of the present invention.

[0162] In operation 902, a user is allowed to select a web-site from a list or specify its URL. A new window is presented in operation 904 that shows the selected web-site. In operation 906, the user is allowed to choose content of choice from the web-site and drag and drop it into a window of choice.

The Mechanism for Tagging Selected Information Contained in a Web-Page

[0163] Web-pages are created using HTML (Hyper Text Markup Language). The content in a web-page is formatted using a tabular format where each table is composed of individual cells distributed into a number of rows and columns. A table may contain other tables within its individual cells. The tagging of selected information within a web-page hinges upon assigning an address to each item of content within the web-page. The addressing scheme takes into account the table(s), row(s), column(s) and cell(s) an item of content belongs to. An item of content can be identified by its address within a web-page and (ii) all the addressing schemes that take into account the table(s), row(s), column(s) and cell(s) an item of content belongs to. The addressing scheme works as follows:

[0164] The page is viewed to be composed of tables that may themselves contain other tables. The tables that are not contained in any other table (highest-level tables) are assigned identifying numbers starting from 1. Tables contained within the highest-level tables are assigned numbers that take into account the tables that contain them. If a table is not contained in any other table, then it may be assigned a number, say 3. If table number 3 contains two tables, then they will be assigned numbers 3-1 and 3-2 respectively. Each table is composed of a unique number of rows and columns. Each item of content resides within a cell that belongs to a specific row and column of a table. The complete address of an item of content is then the unique identifier of the table that contains it and the position of that item of content within that table.

[0165] Once the address of selected content is determined, it is converted into a hyperlink that contains the original content or a hyperlink to it, and its address. When a user drags and drops that selected content into a window of choice, that hyperlink and all of its associated information is sent through the window to the servers where it is entered into a database.

[0166] This mechanism also allows a capture of configurable sections of a web-page, including individual words, lines, paragraphs.

[0167] In the case of secure information like email or bank accounts, the mechanism followed is as follows:

[0168] 1. First forms are created to allow a user to log into their accounts. These forms consist of (a) Dynamic information (like the user name and password) which is captured during the session (b) Static information that is required by the remote account server which is stored in a database and retrieved when an account is selected.

[0169] 2. Using the dynamic and static information, the server logs into the remote server.

[0170] 3. The account information is retrieved.

[0171] 4. The account information is presented in a suitable and configurable format.

A Mechanism for Recognizing a Web Page, Based on its Contents as Specified by the User

[0172] A state's “output structure” description for a state is created and then used during replay to recognize a state of the web page. The term “output structure” refers to the window and frame structure of the output from the remote application. The output structure description contains the information about how many windows were presented to the user at a given state in remote application and how many frames were in each window and the nesting structure of those frames.

[0173] FIG. 10 depicts a process 1000 for recognizing a state based on a current output, according to one embodiment of the present invention. In operation 1002, current output is retrieved from a remote application utilizing a network, the current output corresponding to a state of the remote application. An output structure description of the remote state is generated in operation 1004. The output structure description is preferably of a window and frame structure of the output. The output structure definition is compared to a pre-defined state definition of a recorded state (called a state description) in operation 1006. In operation 1008, a determination is made as to whether the remote state corresponds to the recorded state. Note that the recorded state definition is created during recording of the pattern.

[0174] More detail can be found in copending U.S. patent application entitled SYSTEM, METHOD AND COMPUTER PROGRAM PRODUCT FOR RECOGNIZING A STATE OF A REMOTE APPLICATION, from which priority is claimed and which is incorporated by reference.

A Mechanism for Retrieving Network Content

[0175] A User Agent (UA) is able to fetch web pages of interest and render them correctly. The operation of the User Agent can be likened to remote controlling navigation of a browser on a remote server. For example, the User Agent acts as an intermediary on a remote server that can be commanded to go online and retrieve content, and then send it for rendering for output on a wireless device.

[0176] The User Agent can be used both via a network interface and Interprocessor Communication (IPC). Furthermore, it has performance and scalability befitting a server application.

[0177] FIG. 11 is a flowchart of a process 1100 for User Agent operation. In operation 1102, a network site is accessed utilizing a network. This can be done by navigating to a particular web site using its URL, for example. In operation 1104, a user interaction is simulated for navigating the network site based on client specifications (i.e., directions). An example of such navigation is logging in to a web site. The client is a process that makes requests to and receives output from the Use r Agent. A web page is retrieved from the site in operation 1106. Such content can be part or all of a web page, for example, such as the resulting page that is accessible upon logging in to a web site. In operation 1108, identifiers are assigned to elements of the web page. For example, the third element on the page could be assigned the ID “3”. The web page is rendered for further processing utilizing the identifiers in operation 1110. The rendering can include converting the resulting page into XHTML for processing by another system component and/or output on a user device. Note that XHTML is used herein by way of example; any other XML compliant format may be used; that is, if the page loaded by the User Agent is an XML page, as opposed to an HTML page, it is not converted to XHTML, but rather is passed on to the client in its original XML format. In operation 1112, dynamic content of the web page such as scripting is rendered utilizing the identifiers. The rendered web page is returned to the client in operation 1114. As noted above, the rendered content can be output to a user device, such as a wireless device. The wireless device can include a wireless telephone, Personal Digital Assistant (PDA), handheld computer such as a handheld PC, a pager, a device connected to a wireless modem, or any type of device capable of receiving information where at least one of the communication links is wireless.

[0178] In one aspect of the present invention, navigation from the web page is performed. For example, navigating from the web page can include of clicking links on a web page, submitting forms, and/or firing scripting events (clicking a link being a specific kind of event).

[0179] In another aspect of the present invention, the client is informed of changed frames. The User Agent is able to inform the client which frame(s) have changed since the last event was executed. Navigating to a specified URL would be considered an event in this context. Typically, this would be used by the client to determine which frames have changed since the last event was executed and it would then request only the modified frames. In a further aspect, specified frames are sent to the client. The User Agent is ideally able to send any frames which are requested to the client. Often, a web page will comprise a single frame. However, if the page comprises multiple frames, the User Agent is able to send a selected single frame (or a specified subset of the frames) to the client.

[0180] In a further aspect, a network interface is provided. A runtime engine can use a network interface to make requests to the User Agent. It uses the network interface to navigate the User Agent and to request the web pages from it. As an option, a Component Object Model (COM) interface can be provided. In a preferred environment, the User Agent is embedded as a COM object. The embedding application can then interact with the User Agent's browser component directly. It may then also request the XHTML representations of the current web page. Again, XHTML is used herein by way of example, and any other XML compliant format can be used.

[0181] The User Agent is preferably a Windows application, but can also be a Linux or Unix application, among others. It can use COM and standard Microsoft browser libraries (i.e. MSHTML.DLL) or it can use other browser libraries such as Mozilla. In addition, it can use standard libraries for common functionality (e.g. HTTP communication).

Functionality

[0182] The User Agent is able to fetch the web pages and render them “correctly”. While there is no universally agreed upon definition of “correct”, the User Agent attempts to conform to de facto standards. The User Agent acts, to the degree possible, like a web browser running on a desktop machine and operated by a single user. Broadly speaking, a preferred embodiment of the User Agent:

[0183] satisfies all protocol requirements (e.g. HTTP, FTP, SSL and TLS),

[0184] supports all technologies of interest (e.g. JavaScript, VBScript),

[0185] renders pages in a generally accepted manner

[0186] shares no state information with any other User Agent instance.

[0187] Remote web sites should not be able to determine whether they are interacting with a human user on a dedicated machine or the User Agent. For example, two different User Agents running on the same server should be able to simultaneously log-in to Hotmail using two different user names (and get two different pages).

[0188] The User Agent can be run remotely on a dedicated server. A network interface is exposed so that remote clients can interact with the UA. HTTP can be used to provide this interface. As it is a very widely used protocol, it allows for a very flexible system design (as many diverse clients can speak HTTP) and there are a plethora of standard tools (e.g. performance auditing, load balancing, security tracking) available for it.

[0189] The User Agent can support a COM interface so that it can be embedded into the DT at runtime. It can also support an HTTP interface so that it can be used by remote clients at runtime.

[0190] The User Agent is able to navigate to a desired web page (rendering any dynamic content) and then send an XHTML representation of the rendered web page (which may comprise several frames) to the client. It also allows for the client to be able to direct it to a specified URL, click links, submit forms and fire scripting events on the page. Preferably, it is able to: Note that XHTML is used herein by way of example, and any XML compliant format can be used.

[0191] 1. Navigate to a specified URL and retrieve the associated page over the network. This is the process of actually making network requests and receiving responses. It involves handling all required network protocols, including protocol requirements such as maintaining client-side state information (e.g. HTTP cookies) and having appropriate security certificates.

[0192] 2. Render those pages, tag them and convert them to XHTML. Once the page has been fetched, any dynamic elements (e.g. scripting) are rendered in XHTML for output on the device. While JavaScript is the scripting language of choice, any common scripting language (such as VBScript) could be used. The User Agent also generates and assigns unique IDs to each element on the page. It can use the integer indices of the elements on the page for the IDs (starting from one). For example, the third element on the page would be assigned the ID “3”. It then converts the resulting page into XHTML if it is an HTML page. To render a page, the UA executes any scripting which is to be executed upon the page loading (prior to any user actions). Any scripting which is run as a result of a user action (for example, onMouseOver) is not executed at this point.

[0193] Preferably, no scripting is removed or modified from the page. Thus, if a naive client loads the page into a script-capable browser, it will then execute any automatic events twice. In addition, if an event is fired, whatever script would have been originally called is executed. If the client wishes to modify these events (so that, for example, onClick calls a script of the client's choice instead of the original script), the client modifies the script.

[0194] 3. Navigate from the current web page as specified by other Platform components. This includes clicking links on a web page, submitting forms and firing scripting events. This comprises two parts: filling in forms and firing events (clicking a link being a specific kind of event). Given a specified event (complete with all the parameters needed to execute that event, including the element on which the event is to be fired), the User Agent executes the event on the web page. Typical events include onClick and onSubmit.

[0195] 4. Inform the client of changed frames—The User Agent, upon request, informs the client which frame(s) have changed since the last event was executed. Navigating to a specified URL can be considered an event in this context. Typically, this would be used by the client to determine which frames have changed since the last event was executed and it would then fetch only the modified frames.

[0196] 5. Send specified frames to client—The User Agent can be able to send any frames which are requested to the client. Often, a web page will comprise a single frame. However, if the page comprises multiple frames, the User Agent is able to send only a single frame (or a specified subset of the frames) to the client.

[0197] 6. Provide a network interface. The runtime engine uses a network interface to make requests to the User Agent. It uses the network interface to navigate the User Agent and to request the web pages from it.

[0198] 7. Provide a COM interface. The design-time environment can embed the User Agent as a COM object. It is then able to interact with the browser directly. It may then also request the XHTML representations of the current web page.

[0199] 8. Support scaling easily to multiple server configurations. It is possible to add servers to a UA server farm easily, quickly and without any of interruption of service.

[0200] More information regarding the User Agent can be found in U.S. patent application entitled SYSTEM, METHOD AND COMPUTER PROGRAM PRODUCT FOR A USER AGENT FOR PATTERN REPLAY, from which priority is claimed and which is incorporated by reference.

The Mechanism for Local Storage or Caching of Selected Content

[0201] The selected information is cached or stored locally to enable a faster access. Once a web site is selected by a user, a copy of the site, including text and images, is kept locally in the servers. When any user requests a page that has been requested before, the cached copy is presented if the content of the site has not changed since the time the page was cached. The process is broken down into two: Simple and Customized addition of content:

[0202] Addition of Default content: The addition of default content proceeds as follows:

[0203] 1. Once a site is selected, the backend identifies the headlines that have been pre-selected for that site.

[0204] 2. The server queries the database and picks up the default headlines.

[0205] 3. The headlines that are not included in the pre-selected content are not included.

[0206] 4. The server contacts the ActiveX control that constitutes the administrative page and communicates the selected headlines.

[0207] 5. The selected headlines are visible in the ActiveX control and are also accessible to the main user interface.

[0208] Addition of Customized content: In the case of addition of customized content, the process is as follows:

[0209] 1. The user selects a hyperlink by dragging and dropping them into the ActiveX control on the Administrative page.

[0210] 2. The hyperlink and related information are sent to the servers. The information includes (a) the content of the link, (b) its location on the page, (c) the URL of the site, (d) the identity of the window and the view it has been dropped into and (e) the user name.

[0211] 3. Once the link has been selected, it is added to the database and is accessible to the main user interface.

The Mechanism for Communication of Selected Information to the Backend Servers

[0212] Once a hyperlink is dropped into a window, information is passed by the window to the backend servers. This information includes the address of the hyperlink, as defined above. In addition, the information about the window and the view containing that window is also sent to the server. This information is then used by scripts to generate the front page in HTML.

The Mechanism for Regular Retrieval of Preferred Content from Selected Sites

[0213] The power of the current invention is that refreshed content is retrieved from the selected sources of information as they are updated. The sources of information, or web sites, selected by users are cached locally. The web pages stored locally are categorized according to the number of times they are requested. High request sites are retrieved once every few hours, for example.

The Mechanism to Check for a Change of Content or Format in the Selected Sources of Information

[0214] Once a page has been requested by a user, it is retrieved on a regular basis. There are two checks performed to find out a change in the information in the page. The first involves a change in the content of the page and the second a change in the format in which the content is presented.

Change in a Page's Content

[0215] Every time a page is retrieved, a copy is kept locally on servers. Once a page is automatically retrieved, the content from the newly retrieved version of the page is compared to the content from a previous version of the page. If there is a change in the content, then the updated content is retrieved.

A Change in the Format of the Content

[0216] The formatting of the content in a page is stored in terms of a complete addressing scheme for the page, which specifies the breakdown of the page into its sub-sections. Once there is a change in the formatting of the page, then the relations of different sub-sections of the page to their parent sections change. A mechanism is implemented that keeps track of the number of differences between the format of a previously stored version of the page and the newly retrieved version. An alert is sent to the users if the number of differences is greater than a configurable number.

Return of the Content to an Application Program

[0217] FIG. 12 is a flowchart depicting generation of new applications 1202 according to one embodiment. Accordingly, a developer is able to stitch together components 1204 of existing applications 1206, including web applications, web services, legacy applications, including enterprise applications, and is also able to add new application functionality using a variety of programming languages including Java, JavaScript and SQL, all using a visual paradigm that allows the application to be built very quickly. Using an intuitive user interface, the developer can publish the new application as a web service, accessible via Simple Object Access Protocol (SOAP—described below), through a centralized server. The newly developed application can be made accessible on multiple devices 1208 via any type of network, wireless or Internet connection.

[0218] This process allows the application to be created quickly without requiring the developer to spend days and weeks writing code for a new application. Rather, using the processes set forth herein, selected portions of applications in an existing infrastructure can be reused to provide new functionality and applications. The developer simply interacts with the existing applications and those interactions are recorded to capture the resulting functionality. The developer no longer needs to code to API's. Rather, the methodology presented herein goes above API's, to the interaction level, thus making it easier to call the functionality of the existing infrastructure. The process is described more completely in U.S. Provisional Patent Application entitled SYSTEM, METHOD AND COMPUTER PROGRAM PRODUCT FOR APPLICATION DEVELOPMENT USING A VISUAL PARADIGM TO COMBINE EXISTING DATA AND APPLICATIONS, from which priority is claimed and which is incorporated by reference.

[0219] FIG. 13 is a flow diagram of a process 1300 for generating an application according to one embodiment. An interaction with a first application, preferably at the user-interface level, is recorded in operation 1302 for capturing a functionality of the first application.

[0220] The applications can be any type of application in existence. In operation 1304, an interaction with a second application is also recorded for capturing a functionality of the second application. Note that the interactions may be user interactions, interactions by other applications, etc. A pattern is generated in operation 1306 based on the recorded interactions. The pattern can be saved in memory such as in a file on a hard disk.

[0221] When recording the interaction with one or more of the applications, an interface can be displayed to allow a user to specify interactions with the application(s) via the interface. The user interactions are recorded and become part of the pattern as the user interacts.

[0222] In operation 1308, an application is generated based on the pattern. The interactions recorded in the pattern are repeated upon replay of the pattern for providing the functionalities of the first and second applications. In optional operation 1310, the application is published as a network service accessible via the Internet, an intranet, or other network.

[0223] FIG. 14 is a flow diagram depicting a process 1400 for creating an application utilizing a graphical user interface. In operation 1402, an interface is displayed. A user is allowed to specify interactions with multiple applications via the interface in operation 1404. These interactions are recorded in operation 1406. In operation 1408, a pattern is generated based on the recorded interactions. The pattern is displayed on a graphical user interface in operation 1410. The pattern includes a graphical representation of each interaction. Such graphical representation can be an icon, etc. In operation 1412, the user is allowed to connect representations of the interactions via the graphical user interface for specifying an order of execution of the interactions in the pattern.

[0224] With continued reference to FIG. 14, the user is allowed to add additional functionality to the pattern in operation 1414. In operation 1416, a graphical representation of the additional functionality is displayed on the graphical user interface. Thus, by creating a pattern with recorded interactions and additional functionality, a developer has created an application that can be used and reused.

[0225] As an option, the pattern can include prerecorded interactions. Thus, previously recorded interactions can be used as “building blocks” for the new application, thereby avoiding the need to re-record the interaction. They are merely retrieved from storage and added to the pattern to provide functionality.

[0226] As another option, the pattern can be coupled to another pattern to provide additional functionality. The other pattern can be a previously created pattern that has been stored or another pattern created jointly.

[0227] Preferably, output relating to the pattern can be specified for multiple client devices. This can include specifying which output is actually sent to a particular type of device, as well as the format of the output for particular types of devices.

[0228] Once the application has been created, the pattern-based application can be executed. FIG. 15 provides a flow of execution 1500 of the pattern-based application. Upon receiving a request for data from a user or another application in operation 1502, a pattern for retrieving the requested data is selected in operation 1504 based on stored information associating the pattern with the desired functionality required to obtain the requested data. The pattern has stored therein a recorded interaction with at least one external application. The interaction accesses the functionality of the external application, thereby permitting data to be obtained from the external application. In operation 1506, the pattern is replayed to execute the recorded interaction. In operation 1508, the recorded interaction is conducted as specified in the pattern for obtaining the requested data. The requested data is received in operation 1510. The received data can be further processed and/or output. Note operations 1512 and 1514. For example, the received data can then be processed for output on a particular type of device as specified in the pattern, and/or can be processed for output on multiple types of devices such as a desktop computer, a wireless telephone, a PDA, and/or any other type of device. This can be done by transcoding the output as described above.

EXAMPLE

[0229] Consider the following example of a logistics application. A company is currently using the following system. A truck driver delivers goods and notes the delivery on a sheet, returns to headquarters, and gives sheet to data entry. If there were problems or he was unable to deliver the goods, he writes the details on the sheet and data entry sends an email to a manager.

[0230] The company wants to automate this process. The desired application is to have the following parameters. The driver is to be given a PDA or voice phone (for communicating via a voice portal, as described in copending U.S. patent application entitled VOICE APPLICATIONS AND VOICE-BASED INTERFACE, from which priority is claimed and which is incorporated by reference). The driver is to enter a status or problem via the PDA or voice phone. The information is either entered in database, or sent to report generation, which creates a report and sends it to a manager. The manager is to be able to check status/reports on a desktop.

[0231] To perform all of these functions, the application must interface with a backend resource management system (how company resources are being used), and go into a customer management system (customer database) to see if the customer is not available at a particular delivery time or has billing issues. The application must also be able to communicate with the driver's device, whether connected or disconnected. Also, the application must be capable of generating an alert to the manager.

[0232] To create such a pervasive application in the prior art method, the developer must learn about and write code for the PDA (Palm Pilot, etc). Code must also be written to handle the transactions that occur when the driver interacts and his data comes to the application on the server. A connection to the databases must be opened and the driver's communications written in. Further, the developer must write code that checks for exceptions written by someone (for example, if delivery doesn't happen by this time, send alert) via an administrative interface created by someone. Then if someone (manager) wants to log on through a desktop, the developer must create an interface for that too. Thus, the developer has to write a lot of software.

[0233] According to the methodology presented herein, a pervasive application connecting all of these applications and devices can be quickly built and implemented. Using the IDT, the developer merely needs to interact with the various devices and programs. The IDT captures the interactions, which are then used to create a pattern or patterns.

[0234] A form presented on the PDA can be generated using the publisher for mobile devices described in copending U.S. patent application entitled SYSTEM, METHOD AND COMPUTER PROGRAM PRODUCT FOR A CONTENT PUBLISHER FOR WIRELESS DEVICES filed Jul. 10, 2001 under Ser. No. 09/902,929, and which is herein incorporated by reference. The driver can enter information on the form, which is transmitted to the new application via an existing communications program. The developer interacts with the existing communications program in such a way that the desired data is obtained. The developer's interaction is recorded and placed in a pattern, as described above. Upon replay of the pattern, the desired data is again retrieved. To create the alerts, the developer interacts with an email program, entering delivery and content information. Again, this interaction is recorded and placed in the pattern. The developer is then able to set criteria for which alert is sent upon receipt of information from the driver.

Transaction Object Model (TOM)

[0235] One embodiment allows developers to create transaction objects, which they can use as building blocks to create applications. In sharp contrast to the prior art, the methodology set forth herein overcome the “fixed menu” drawback of prior art systems and allows a developer to use any existing functionality by capturing it in a pattern.

[0236] Any software application can be seen as a set of functional flows. Each transaction can be seen as a transaction object. The transaction objects are captured and converted into building blocks that a developer can use and reuse. The developer no longer needs to select from a predefined set of building blocks, but can capture the blocks from any application. Now, the entire universe of applications provides building blocks. Any portion of any application by anyone can become a building block.

[0237] The developer selects the desired blocks and writes some code around the block (if necessary) to create a new application. Further, the system is flexible, allowing the developer to add and remove blocks at will.

[0238] Simple Object Access Protocol (SOAP) is a way for a program running in one kind of operating system (such as Windows 2000) to communicate with a program in the same or another kind of an operating system (such as Linux) by using the World Wide Web's Hypertext Transfer Protocol (HTTP) and its Extensible Markup Language (XML) as the mechanisms for information exchange. Since Web protocols are installed and available for use by all major operating system platforms, HTTP and XML provide an already at-hand solution to the problem of how programs running under different operating systems in a network can communicate with each other. SOAP specifies exactly how to encode an HTTP header and an XML file so that a program in one computer can call a program in another computer and pass it information. It also specifies how the called program can return a response.

[0239] SOAP was developed by Microsoft, DevelopMentor, and Userland Software and has been proposed as a standard interface to the Internet Engineering Task Force (IETF). It is somewhat similar to the Internet Inter-ORB Protocol (IIOP), a protocol that is part of the Common Object Request Broker Architecture (CORBA). Sun Microsystems'Remote Method Invocation (RMI) is a similar client/server interprogram protocol between programs written in Java.

[0240] An advantage of SOAP is that program calls are much more likely to get through firewall servers that screen out requests other than those for known applications (through the designated port mechanism). Since HTTP requests are usually allowed through firewalls, programs using SOAP to communicate can be sure that they can communicate with programs anywhere.

Voice Functionality

[0241] A Voice Habitat according to one embodiment is a modified Habitat which provides several functions, including:

[0242] Providing a hosting environment for VoiceLets. A VoiceLet is a voice application, or more particularly, a piece of voice application functionality that can be added to the habitat and configured to provide the functionality to the end user. For example, an email access VoiceLet can be dragged into the habitat by the end user, and configured to access the user's email inbox. The user can then access email via the voice habitat. A detailed description of voicelets is provided below.

[0243] Allowing the user to interact with any content/application dragged into a voice-enabled container.

[0244] Creating alerts that cause the user to be notified via a voice telephone call.

[0245] The following features may be added to the Habitat, among others:

[0246] 1. Voice Authentication—The user is authenticated using a PIN, user name, or combination of the two. The user can be authenticated using their voice print (speaker verification), or in the case of a limited set of users, the user can be identified using their voice print (speaker identification).

[0247] 2. Voice Enabling Containers—The user can specify for a given container (View or Window) whether or not that container should be voice enabled when the user dials in.

[0248] 3. Voice Enabled Content—When calling up, the user can select to listen to content items (both text or audio files) dragged into the habitat, and interact with applications aggregated into the habitat using VoiceLets.

[0249] The user may access the voice habitat by calling in to the habitat via telephone, connecting using Internet telephony, etc. Note that the user can also access the habitat using any other device as well, such as computer, PDA, etc.

[0250] VoiceLets are reusable voice applications that can be hosted in the habitat, or accessed independently of the habitat, and are used to quickly provide useful applications to the end user. Each VoiceLet can be implemented as a plug-in module which is configured by the administrator and/or the end user, and which presents its interface entirely in voice, using a VoiceLet API (discussed below). As an option, some or all of the VoiceLets can be optimized and tuned to perform specific tasks.

[0251] VoiceLets can provide, for example, access to internet, POP3 and MAPI email systems, LDAP or MAPI compliant directory systems, text or audio content, etc. Each VoiceLet can be developed and tuned to provide an optimal user experience, thus saving the developer the time and effort of re-creating that interface. VoiceLets can be fully configurable by both administrators and end-users, and can be easily incorporated into a larger voice application, particularly when implemented as VoiceXML sub-dialogs. The VoiceXML for the VoiceLet can be generated by a transcoder at run-time, and can be formatted by a transcoder to maximally utilize the capabilities of the particular VoiceXML browser that is used to access the VoiceLet. This provides more certainty that the VoiceLet will work correctly regardless of the VoiceXML platform used to deploy it, without sacrificing any of the benefits of that particular VoiceXML platform.

[0252] Illustrative VoiceLets that can be used with the voice habitat include:

[0253] 1. User Authentication—Authenticates a user accessing the Habitat or some other voice application.

[0254] 2. Audio Advertisement—Plays back an audio advertisement to the user.

[0255] 3. Email—Provides access to an email inbox and other email functions.

[0256] 4. Address book—Provides access to address book and directory services.

[0257] 5. Content—Wraps a specific piece of content, e.g. news/entertainment/finance, driving directions, etc

[0258] 6. Security—Front-end for any username/password based account. Used to additionally protect access to sensitive user data.

[0259] 7. Exit—Allows the user to gracefully exit the Habitat.

[0260] A more complete description of voice-enabled technology is provided in U.S. patent application entitled VOICE APPLICATIONS AND VOICE-BASED INTERFACE, from which priority is claimed and which is incorporated by reference.

[0261] While various embodiments have been described above, it should be understood that they have been presented by way of example only, and not limitation. Thus, the breadth and scope of a preferred embodiment should not be limited by any of the above described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.

Claims

1. A method for selection and formatting of network content for remote viewing, comprising: receiving user-defined information from at least one of a wireless device and a hardwired device; wherein the user-defined information specifies content to be retrieved from at least one of a web site and an application; wherein the user-defined information being received in real time; wherein the user-defined information includes a general request for content; wherein a search engine is used to retrieve content from the at least one web site and application based on the user-defined information; retrieving content from the at least one web site and application utilizing the user-defined information; wherein the content includes at least one of: text, secure account information, services, and financial transactions; aggregating the retrieved content on a portal page unique to the user at a network server located remotely from the user; formatting the aggregated content at the network server for display on multiple types of wireless devices; wherein the wireless device sends a request for content in a particular type of formatting; wherein a type of the wireless device being used by the user is identified based on the request; wherein the wireless device is at least one of a personal digital assistant (PDA), a handheld computer, a wireless telephone, a device connected to a wireless modem, and a pager; selecting aggregated content for output to the particular type of wireless device; transmitting the formatted content to the particular type of wireless device for display on the particular type of wireless device, wherein the selected content is formatted for output on the particular type of wireless device; updating the retrieved content after a predetermined amount of time has expired; wherein the user is allowed to interact with the content via the wireless device; and sending an alert to the wireless device upon occurrence of a prespecified condition; sending at least a portion of the retrieved content to an application program for further processing wherein the user is allowed to specify whether a particular web page is retrieved upon detection that the web page has changed; wherein a formatting style of the content is specific to the wireless device used by the particular user.

2. A method for selection and formatting of network content, comprising: receiving user-defined information specifying content to be retrieved from at least one network site; identifying the specified content of the at least one network site utilizing the user-defined information; retrieving the identified content from the at least one network site; aggregating the retrieved content at a network server; formatting at least a portion of the aggregated content at the network server for display on a user device; transmitting the formatted content to the user device for output on the user device; and sending at least a portion of the retrieved content to an application program for further processing.

3. A method as recited in claim 2, wherein a user selects which content is aggregated in real time.

4. A method as recited in claim 2, further comprising identifying a web page having the specified content based on contents of the web page.

5. A method as recited in claim 2, further comprising identifying an application view having the specified content based on contents of the application view.

6. A method as recited in claim 2, wherein the content includes at least one of: text, secure account information, services, electronic mail, and financial transactions.

7. A method as recited in claim 2, wherein the user device is at least one of a personal digital assistant (PDA), a handheld computer, a wireless telephone, a device connected to a wireless modem, a pager, and a hardwired telephone.

8. A method as recited in claim 2, further comprising updating the retrieved content after a predetermined amount of time has expired, formatting the updated content at the network server for display on the user device, transmitting the formatted updated content to the user device for output on the user device, and transmitting at least a portion of the updated content to an application program for further processing.

9. A method as recited in claim 2, wherein the user is allowed to interact with the content.

10. A method as recited in claim 2, further comprising the step of sending an alert to the user device upon occurrence of a prespecified condition.

11. A method for selection and formatting of network content, comprising: receiving user-defined information specifying content to be retrieved from at least one network site; retrieving content from the at least one network site utilizing the user-defined information; aggregating the retrieved content at a network server; formatting the aggregated content for output via a voice browser on a user device; and transmitting the formatted content to the user device for output via the voice browser.

12. A method as recited in claim 11, wherein the user is allowed to interact with the content.

13. A method for selection and formatting of web content for remote output, comprising: receiving user-defined information specifying content to be retrieved from at least one web site; retrieving content from the at least one web site utilizing the user-defined information; aggregating the retrieved content at a network server located remotely from the user; formatting the aggregated content at the network server for output on multiple types of user devices; identifying a type of device being used by the user; selecting aggregated content for output to the particular type of user device; and transmitting the formatted content to the particular type of user device for output on the particular type of user device, wherein the selected content is formatted for output on the particular type of user device.

14. A method as recited in claim 13, further comprising updating the retrieved content after a predetermined amount of time has expired, formatting the updated content at the network server for output on the user device, and transmitting the formatted updated content to the user device for output on the user device.

15. A method as recited in claim 13, further comprising transmitting at least a portion of the retrieved content to an application program for further processing.

16. A method as recited in claim 13, further comprising identifying a web page having the specified content based on contents of the web page.

17. A method as recited in claim 13, further comprising retrieving a web page having the specified content, rendering the web page as it would be displayed on a browser, analyzing the rendered web page, and recognizing content within the page based on the analysis.

18. A method as recited in claim 13, wherein the user is allowed to interact with the content.

19. A method as recited in claim 13, wherein the content includes at least one of: text, secure account information, services, electronic mail, and financial transactions.

20. A method for compiling an enhanced query result, comprising: receiving a user query; connecting to at least one network accessible data site; extracting information from the at least one data site based on the query; aggregating the extracted information from the at least one data site at a network server; determining a type of output device being used by the user; formatting the aggregated information for display on the output device; and transmitting the formatted information to the output device.

21. A method for accessing network content, comprising: retrieving a form, the form having user-specific identification information used for accessing data on a network site; logging into the network site using the form; retrieving at least a portion of the data from the network site; aggregating the retrieved content; formatting at least a portion of the aggregated content for output on a user device; and transmitting the formatted content to the user device for output on the user device.

22. A method as recited in claim 21, wherein the user is allowed to interact with the content.

23. A method as recited in claim 21, wherein the content includes at least one of: text, secure account information, services, electronic mail, and financial transactions.

24. A method for selection and formatting of a network user interface, comprising: receiving user-defined information specifying user interface elements to be retrieved from at least one network site; identifying the specified user interface elements of the at least one network site utilizing the user-defined information; retrieving the identified user interface elements from the at least one network site; aggregating the extracted user interface elements from the at least one network site at a network server; formatting at least a portion of the retrieved user interface elements at a network server for display on a user device; transmitting the formatted user interface elements to the user device for output on the user device; and automating at least a portion of the retrieved user interface elements by exposing them to an application program for further processing.

25. A computer program product for selection and formatting of network content, comprising: computer code for receiving user-defined information specifying content to be retrieved from at least one network site; computer code for identifying the specified content of the at least one network site utilizing the user-defined information; computer code for retrieving the identified content from the at least one network site; computer code for aggregating the retrieved content at a network server; computer code for formatting at least a portion of the aggregated content at the network server for display on a user device; computer code for transmitting the formatted content to the user device for output on the user device; and computer code for sending at least a portion of the retrieved content to an application program for further processing.

26. A system for selection and formatting of network content, comprising: logic for receiving user-defined information specifying content to be retrieved from at least one network site; logic for retrieving content from the at least one network site utilizing the user-defined information; logic for aggregating the retrieved content at a network server; logic for formatting the aggregated content for output via a voice browser on a user device; and logic for transmitting the formatted content to the user device for output via the voice browser.