Patent Application
20050138643
The invention relates generally to computer programs, and more particularly to a system and method for controlling a computer operating environment.
Due to the wide use of computers, numerous types of computer applications are now in the consumer marketplace. These computer applications include audio and video applications, Internet-related applications (e.g., browsers, communication tools, and ad blockers), games, and business and educational applications (e.g., word processing, spreadsheets and money management). In general, computer applications cannot easily be modified after these applications have been introduced into the consumer marketplace, other than what the original programmers have written into the applications to be modifiable. Consequently, the end users cannot usually add new features and/or new operations to the computer applications that were not contemplated by the original programmers.
Furthermore, computer applications typically do not allow a user to write simple codes that can be used to control one or more aspects of the applications. One exception to this typical limitation is an Internet browser, which can read an HTML document and create a web page. As an example, using JavaScript programming language, a user can control different aspects of the resulting web page by embedding a JavaScript into an HTML document. Since a JavaScript is written into an HTML document, the JavaScript is part of and is dependent on the HTML document. A JavaScript can be used to add functionalities to a web page, such as image rollovers, form validations, and control elements on the webpage.
However, there is no such mechanism to write simple codes to control other types of computer applications. Thus, there is no way to control any aspect of a computer operating environment provided by, for example, a word processing application by writing a simple code and running that code.
In view of these concerns, there is a need for a system and method for controlling a computer operating environment, such as an environment provided by a computer application, using a simple code that can be written by an end user.
A system and method for controlling a computer operating environment utilizes a non-executable computer program that is dependent on a primary computer program, which provides the computer operating environment. The non-executable computer program is a compiled computer program that is based on a custom scripting programming language. The non-executable computer program can be configured to control various aspects of the computer operating environment using a bytecode interpreter of the primary computer program, including any object that can exist in the computer operating environment.
A system in accordance with an embodiment of the invention comprises a primary computer program including a bytecode interpreter, and a non-executable dependent computer program that can control an object in the computer operating environment. The primary computer program provides the computer operating environment. The non-executable dependent computer program is a scripting language-based computer program including bytecodes that correspond to specific tasks to be performed by the primary computer program. The bytecode interpreter of the primary computer program is configured to correlate the bytecodes of the non-executable dependent program and the specific tasks.
A method in accordance with an embodiment of the invention comprises running a primary computer program including a bytecode interpreter and running a non-executable dependent computer program that can control an object in the computer operating environment.
An embodiment of the invention includes a storage medium, readable by a computer, tangibly embodying a program of instructions executable by the computer to perform method steps for controlling a computer operating environment.
Other aspects and advantages of the present invention will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, illustrated by way of example of the principles of the invention.
A system and method for controlling a computer operating environment in accordance with an embodiment of the invention uses a custom scripting programming language, which allows a user to create a scripting language-based computer program to control the computer operating environment. The custom scripting programming language is referred to herein as the “Object Interaction and Modification” (OIAM) programming language. The computer operating environment may be any computer environment, such as a desktop environment provided by an operating system or a computing environment provided by a computer program running in a computer system. The scripting language-based computer program created using the OIAM programming language can be used to provide additional functionalities to the computer operating environment.
The system and method is described below with respect to a computer operating environment referred to herein as the “Blackspace” environment. The word “Blackspace” is a trademark of the NBOR Corporation. The Blackspace environment presents one universal drawing surface that is shared by all graphic objects within the environment. The Blackspace environment is analogous to a giant drawing “canvas” on which all graphic objects generated in the environment exist and can be applied. Each of these graphic objects can have a user-created relationship to any or all the other objects. There are no barriers between any of the objects that are created for or exist on this canvas. However, the invention is not limited to the Blackspace environment and can be implemented in any computer operating environment, such as a word processing application environment.
As used herein, the term “objects” include recognized graphic objects (e.g., stars, squares, circles, arrows, etc.), free drawn objects (sketches, drawings, lines, etc.), pictures in various file format (.png, .jpg, .bmp, .gif, etc.), graphic control devices (switches, faders, knobs, joysticks, etc.), videos in various file format (.mpg, .avi, .mov, etc.), text, and other graphic objects that are displayed on a display device.
Using the OIAM programming language, a computer program (“an OIAM program”) can be written that can virtually control any aspect of the Blackspace environment. An OIAM program is a computer program that is dependent on the controlled computer environment. An OIAM program can be initiated by a user or the computer program generating the computer operating environment to be controlled, such as a Blackspace program that generates the Blackspace environment. As described in more detail below, an OIAM program is produced by writing a computer code in the OIAM programming language and then compiling the code using a compiler with a custom lookup dictionary composed of bytecodes. The OIAM program can then be initiated by the Blackspace program or a user of the Blackspace program. An OIAM program is repeatedly executed, for example, every {fraction (1/30)} of a second, as described in more detail below, which allows performance currently seen only in video games and fast graphical applications.
An OIAM program can be written to create any graphic object in the Blackspace environment that can be created in that environment, such as stars, rectangles, graphic control device, lines, etc. In addition, an OIAM program can be written to create graphic objects that cannot currently be created in the Blackspace environment by the Blackspace program. This concept is described in more detail below. In addition, the position of a graphic object being created (i.e., the position of the graphic object on a display screen) can be specified by the OIAM program, as well as any attribute of the graphic object, such as size and color. Furthermore, the position of a displayed graphic object can be changed by the OIAM program.
As an example, an OIAM program can be written to incrementally change the position of a graphic object on the screen by one horizontal pixel at each screen refresh. In this situation, the repeated execution of the OIAM program can be set to coincide with the screen refresh. Such an OIAM program would have the effect of horizontally scrolling the graphic object across the screen. As another example, an OIAM program can be written to loop through all graphic objects on the screen and decrease the size of the objects by one pixel increment at each screen refresh. This would have the effect of continuously reducing the sizes of the graphic objects on the screen until the objects disappear from the screen, giving the appearance of all the objects shrinking into nothing.
Other operations that can be performed in the Blackspace environment by using an OIAM program include checking to see if a graphic object has collided with another graphic object on the screen and modifying any attribute of a graphic object on the screen. However, as stated earlier, an OIAM program can be used to control virtually any aspect of the Blackspace environment, including performing operations that do not involve graphic objects or other graphic elements. As an example, an OIAM program can be used to open and read files, including extracting the contents of the files into the Blackspace environment. A more concrete example would be an OIAM program that can open a file from a spreadsheet program, read the contents of the file, and then create a spreadsheet with the contents of the file in the Blackspace environment.
The above source code, when compiled and executed, will load a text object of “This is an example” and position the text object at X and Y coordinates on the screen defined by “50,50”. At each running of the complied code, the text object will be moved by one pixel in the positive X direction until the object reaches a certain X position (i.e., 800) and then the object is deleted. Thus, this source code will have the effect of creating a text object at an initial position and then horizontally moving that object until the object reaches a final position, where the object disappears.
After the source code 12 is written, the code is compiled by an OIAM program language (PL)-to-bytecode compiler 14, referred to herein as the “OIAM compiler”. The OIAM compiler 14 is designed to read the source code 12 and convert the code into bytecodes for control characters. The conversion of control characters into bytecodes involves using a custom lookup dictionary 16 that equates particular control characters to binary values in a predefined range (e.g., [0, 255]). Each binary value represents one or more instructions that will be executed by the Blackspace program. The OIAM compiler 14 also converts each of the value characters in the source code 12 into a corresponding binary value, which represents a specific alphanumeric character.
The compilation of the source code 12 by the OIAM compiler 14 using the lookup dictionary 16 results in an OIAM program 18. The OIAM program 18 can then be loaded into a computer system and executed to control some aspect of a computer operating environment, such as the Blackspace environment. The OIAM program 18 is not an executable, and thus, cannot be independently executed. The OIAM program 18 is dependent on a primary program, which provides the controllable computer operating environment. As described below, the primary program must include a bytecode interpreter the can correlate the bytecodes of an OIAM program to corresponding instructions or tasks to be performed by the primary program. The OIAM program 18 can be distributed to other users of the primary program, such as the Blackspace program, so that those users can run the OIAM program on their systems.
Turning now to
In order for the OIAM program 18 to properly run, the primary program (e.g., the Blackspace program 22) needs to have an OIAM bytecode interpreter 24 that can understand instructions associated with the bytecodes of the compiled OIAM program 18. The OIAM interpreter 24 is part of the Blackspace program 22 and can direct the Blackspace program to execute any function that the Blackspace program can perform. In addition, the OIAM interpreter 24 can be configured to directly execute one or more functions that the Blackspace program 22 cannot perform. For such a function, all the necessary codes to execute that function are written into the OIAM interpreter 24. When the OIAM interpreter 24 is called to perform such a function, the OIAM interpreter reads the bytecode and then executes the function associated with the bytecode, instead of directing the Blackspace program 22 to perform that function.
Since there is an association between the tasks to be performed by the OIAM interpreter 24 and the bytecodes of the compiled OIAM program 18, there is also an association between these tasks and the control characters in the original source code 12 of the OIAM program via the lookup dictionary 16 of the OIAM compiler 14. The following list is an example of such association between control (instruction) characters used in OIAM source codes and the corresponding tasks preformed by the OIAM interpreter 24.
This list illustrates the relationship between the control characters in OIAM source codes and the tasks performed by the OIAM interpreter 24 of the Blackspace program 22 when the source code is compiled into an OIAM program and then executed. The above list is just a small sample of possible control characters and the corresponding interpreter tasks. There can be hundreds, or even thousands or more, of different control characters that correspond to different tasks that the OIAM interpreter 24 can execute, which are only limited by the computer system 20 in which the OIAM program 18 and the Blackspace program 22 are executed.
As mentioned above, each of these control characters is assigned to a specific bytecode by the OIAM compiler 14 using the lookup dictionary 16. That bytecode is then recognized by the OIAM interpreter 24 of the Blackspace program 22 when the OIAM program 18 is executed. In response to the particular bytecode, the OIAM interpreter 24 performs the task or tasks associated with that particular bytecode. The ultimate actions associated with the bytecodes are carried out by the Blackspace program 22 or by the OIAM interpreter 24.
The compiled OIAM programs can be created to perform virtually any type of operations within the Blackspace environment or other computer operating environments, such as allowing communications between networked computers that can be used for playing games on-line or for enabling instant messaging. However, a particular type of operation within the Blackspace environment of interest is inserting recurring graphic objects in “pages” of a document. This operation is similar to inserting headers and/or footers in an electronic document using a conventional word processing application. However, unlike conventional word processing applications that limit the page area in which to insert headers and footers, the use of a special OIAM program allows headers and footers to be inserted anywhere on selected pages of a document.
As illustrated in
In the Blackspace environment 26, a user is also able to print multiple pages using a multi-page mode PAR 32, which is shown in
As shown in
Using a special compiled OIAM program (referred to herein as the “printing OIAM program”), user-defined graphic object (referred to herein as the “recurring graphic object”) can be inserted into one of the rectangles or “print areas” of the multi-page mode PAR 32 such that the same graphic object selectively appears in other selected rectangles of the multi-page mode PAR. The recurring graphic object can include text, numbers, pictures or any graphics that can be displayed in the Blackspace environment 26, e.g., the workspace canvas of the VDACC object 34. The recurring graphic object can be used to selectively insert headers and/or footers into pages of an electronic document, which are represented on the screen 28 as rectangles of the multi-page mode PAR 32. However, unlike conventional word processing applications, headers and footers in the form of recurring graphic objects can be placed anywhere on a page. Thus, the recurring graphic objects created using the printing OIAM program provides more flexibility and control for the user to insert text and other graphic objects in selected pages of a document. This functionality of selectively inserting recurring graphic objects in the print areas of the multi-page mode PAR 32 is part of a feature referred to herein as the recurring graphic object feature.
As an example, the following exemplary OIAM source code can be used to create the printing OIAM program to enable the recurring graphic object feature in a Blackspace environment.
The line numbers “(1)” to “(6)” are not part of the source code. These numbers are used herein for reference. Brief explanation of the above source code (when compiled and ran) is now presented. The line (1) of the source code “CONTROLDATANAME PAGES” names this program as “PAGES” so that the printing OIAM program can be called by other programs, including another printing OIAM program. The line (2) of the source code loads an object called “TEXTCONTROL”, which is a text object and will be the recurring graphic object. The line (3) of the source code “SET TEXTCONTROL” enter text here“ ” inserts the text “enter text here” in the TEXTCONTROL so that a user can enter the desired text for the recurring graphic object.
The “LOOPSTART” on the line (4) of the source code is used to mark the line (4) for reference. The “PSET” on the line (4) instructs the OIAM interpreter to perform two tasks. One of these two tasks is checking to see if the text of the TEXTCONTROL has changed with any respect, such as content, size or color. If the textual content has changed, then the text for all selected pages in which the recurring graphic object is to be inserted are correspondingly changed. The second task is checking to see if the multi-page mode PAR has changed with respect to size or position, which would correspond to a collective change in the size or position of all the printable areas for the different pages. If the multi-page mode PAR has changed in size or position, then the text for all selected pages in which the recurring graphic object is inserted are correspondingly changed.
The “STOP” on line (5) of the source code stops the processing of the printing OIAM program for the current cycle. In the next cycle when the printing OIAM program is again activated, the processing of the printing OIAM program starts at the following line of the program. The “JMP LOOPSTART” on the line (6) jumps the processing of the printing OIAM program to the line of the source code having the reference “LOOPSTART”, which is the line (4) of the source code. The tasks associated with the “PSET” is then again performed by the OIAM interpreter.
As stated above, the printing OIAM program is used the enable the recurring graphic object feature of the Blackspace program. In the Blackspace environment, a user can activate the recurring graphic object feature by entering an appropriate command, e.g., selecting an entry in a menu, when the multi-page mode PAR is the screen. When the recurring graphic object feature is activated, a printing OIAM program is initiated for each print area of the multi-page mode PAR in which the recurring graphic objects is to be inserted. The user can select the print areas in which the recurring graphic objects are to be inserted by entering another command, e.g., selecting another entry in the menu. For example, the choices presented to the user for selection may include:
The sequential order of the print areas of the multi-page mode PAR is user-definable in the Blackspace environment. This sequential order corresponds to the order in which pages of the multi-page mode PAR are to be printed. In the exemplary embodiment, arrow logic and context may be used to select the printing sequence of contents within the rectangles or “pages” of the multi-page mode PAR. In the Blackspace environment, an arrow of a predefined color can be used to execute a predetermined function. In addition, the predetermined function can be dependent on the context in which the arrow is being used. The context is an arrow of predefined color (e.g., yellow) drawn in the multi-page mode PAR. For more information regarding arrow logic and context, see pending U.S. patent application Ser. No. 09/880,397, which is incorporated herein by reference.
As an example, using the described arrow logic and context, the printing sequence for the pages of the multi-page mode PAR 32 can be selected by a user by drawing a yellow arrow that intersects a set of four complete rectangles of the multi-page mode PAR at the uppermost right hand corner of the screen 28 (see
Three exemplary printing sequence selections using an arrow logic and context are now described with reference to
The selected printing sequence can also be applied to the remaining rectangles of the multi-page mode PAR 32 in sets of four rectangles. As an example, if there are additional rectangles of the multi-page mode PAR 32 below the four rectangles 36A-36D, the same printing sequence will be applied to those rectangles in sets of four rectangles. Similarly, if there are additional rectangles of the multi-page mode PAR 32 to the right of the four rectangles 36A-36D, the same printing sequence will be applied to those rectangles in sets of four rectangles. If there are additional rectangles below and to the right of the four rectangles 36A-36D of the multi-page mode PAR 32, the selected the printing sequence can be applied in sets of four rectangles in a raster scanning pattern. The scanning pattern applied to the rectangles of the multi-page mode PAR 32 for the selected printing sequence may be user definable. By selecting a custom printing sequence and the rectangles of the multi-page mode PAR for placing the recurring graphic object, a user can control which printed pages of a document will have the recurring graphic objects, for example, every third printed page.
The process for inserting and modifying recurring graphic objects in accordance with an embodiment of the invention is now described with reference to the flow diagram of
Next, at block 48, recurring graphic objects in the form of text objects are created at each of the selected rectangles of the multi-page mode PAR text object by the sequential execution of the printing OIAM program. The contents of the recurring graphic objects may be blank so that a user can enter a desired text or other graphics. Alternatively, the recurring graphic objects may include a common original content. As an example, the common original content may be the text “enter text here”, as illustrated in
Next, at block 50, the recurring graphic objects in the selected rectangles of the multi-page mode PAR are collectively updated in response to any change in one of the recurring graphic objects. These changes may include any change to the contents of the recurring graphic objects, such as text change, font change, font size change and font color change, and positional change within the respective rectangle of the multi-page mode PAR. In addition, the changes may include any change to the multi-page mode PAR, such as positional change or size change (these changes affect the recurring graphic objects with respect to size and position). As an example, in
As stated above, the printing OIAM program is continuously and serially executed in cycles for the selected rectangles of a multi-page mode PAR. Thus, the process then proceeds back to block 50, where the recurring graphic objects are again collectively updated in response to any visual change in one of the recurring graphic objects. As an example, the printing OIAM program is executed every {fraction (1/30)} of a second. In this example, if there are four selected rectangles of a multi-page mode PAR, the printing OIAM program is executed for all the selected rectangles in {fraction (4/30)} of a second. Every time the printing OIAM program is executed, the OIAM interpreter 16 of the Blackspace program 22 (shown in
A method for controlling a computer operating environment, such as the Blackspace environment, in accordance with an embodiment of the invention is described with reference to the flow diagram of
Although specific embodiments of the invention have been described and illustrated, the invention is not to be limited to the specific forms or arrangements of parts so described and illustrated. The scope of the invention is to be defined by the claims appended hereto and their equivalents.
