Patent Application
20150332482
One of the many challenges in developing computer programs is ensuring that the resulting application provides a good user experience. For broadly distributed applications, providing a good user experience involves considering the many different languages and other capabilities of those users. However, software testing often cannot be performed economically by testers with the same breadth of capabilities as the likely user base.
One of the aspects of the user experience is the graphical output data of the application, such as display data in its graphical user interface or data output to a printer. Such graphical output data can include text, which when rendered is a combination of a character and font properties, such as type, size, weight and color. Other graphical output data can include images and other user interface elements. A developer may define conventions for this graphical output data, to ensure, for example, readability, visibility, and consistency with a theme or branding. Conformance to such conventions may not always occur due to the nature of computer programming and the fact that there may be different conventions for different markets and user types. Therefore, conformance to such conventions generally is verified, if at all, through testing.
Testing for conformance of graphical output data to conventions can be challenging for several reasons. A person may not be able to tell whether output text has a particular font size or other font properties. A person may not be able to tell whether the text is in a particular font type, especially if he or she is not familiar with the characters of the language of the text and how those characters should appear in a given font. A person also may not be able to tell the difference between two very similar colors. An output device also may not accurately present colors. Other characteristics of graphical output data also might not be visibly detectable by a person.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is intended neither to identify key or essential features, nor to limit the scope, of the claimed subject matter.
Conformance of graphical output data on an output device to a convention can be detected by processing instructions to render the graphical output data to determine if the graphical output data rendered in accordance with those instructions conforms to the convention. The graphical output data is then rendered based on this determination of conformance, such that the rendered graphical output data visibly differentiates conforming graphical output data from nonconforming graphical output data. For example, text that conforms to specified font properties, such as font type, size, weight, color, and other properties, can be presented in one color, and text that is not conforming can be presented with another color. This processing can occur within an application when that application renders graphical output data to be directed to an output device. This processing can occur within a renderer of an operating system of a computer, which renders graphical user interfaces for one or more applications for display, in some cases simultaneously, on one or more displays connected to the computer. Such processing can occur in other renderers that generate graphical output data for an application, such as for printing.
Conformance detection can be performed using a variety of techniques. For example, in one implementation, conformance detection can be performed using rule-based processing. In another implementation, a font file can be modified so as to provide glyph data in one color for text that is conforming, and to provide glyph data in another color for text that is nonconforming.
Accordingly, in one aspect, graphical data is received into memory. The graphical data includes instructions to render graphical output data. Data indicating conventions for rendered graphical output data also is received into memory. The instructions, included in the graphical data, are processed to determine if graphical output data rendered in accordance with those instructions conforms to the conventions. The graphical output data is then rendered in accordance with both those instructions and the determination made about conformance, such that the rendered graphical output data visibly differentiates conforming from nonconforming graphical output data when presented on an output device, such as being displayed on a display or printed on a printer.
In another aspect, a rendering module has an input to receive graphical data from a memory. The graphical data includes instructions to render graphical output data. The rendering module also has an input to receive data from the memory indicating conventions for rendered graphical output data. The rendering module processes the instructions to determine if graphical output data rendered in accordance with the instructions conforms to the conventions. The graphical output data is then rendered both in accordance with the instructions and based on the determination made about conformance, such that the rendered graphical output data visibly differentiates conforming from nonconforming graphical output data when presented on an output device, such as being displayed on a display or printed on a printer.
Processing can include rendering the instructions using a font file, wherein glyphs for set of font properties, such as types, sizes, weights and colors, conforming to the graphics conventions, are encoded in a color that, when rendered, indicate conformance to the user.
The graphical output data can include characters with a set of font properties and the conventions include acceptable font properties. For example, the display data can include characters in a font type (also called a face) and the conventions can include acceptable font types. The graphical output data can include characters in a font size and the conventions include acceptable font sizes. The graphical output data can include characters in a color, in which case the conventions include acceptable colors of characters. Other properties, such as weight, style, scaling factors, layers, names, and so on, also can be specified in a convention.
The graphical output data can be for a graphical user interface for an application or operating system on the computer. The instructions can be received from two or more applications on the computer, in which case, the processing can be performed for both applications by a display manager of an operating system. The graphical output data also can be directed to a printer or other output device, and can be for a purpose other than the graphical user interface of the application.
Information about any instructions to render graphical output data can be recorded if it is determined that the graphical output data does not conform to the conventions.
The invention may be embodied as a computer system, as any individual component of such a computer system, as a process performed by such a computer system or any individual component of such a computer system, or as an article of manufacture including computer storage with computer program instructions are stored and which, when processed by computers, configure those computers to provide such a computer system or any individual component of such a computer system.
In the following description, reference is made to the accompanying drawings which form a part hereof, and in which are shown, by way of illustration, specific example implementations of this technique. It is understood that other embodiments may be utilized and structural changes may be made without departing from the scope of the disclosure.
The following section describes an example computer system that supports verification of conformance of graphical output data to a convention.
Referring to
The computer can be any type of general-purpose or special-purpose computer, such as a tablet computer, hand held computer, smart phone, laptop or notebook computer, wearable computing device, or any other computing device that provides graphical output data to an output device, such as a display or printer, regardless of size or environment, more details and examples of which are discussed below in connection with
In the example implementation described in connection with
In the example implementations in
In this environment, the conformance of the display data 112 to conventions defined for display data of the graphical user interfaces of the applications 102 can be verified. A tester, e.g., an individual person, accesses the computer to use the applications 102, through the graphical user interfaces of these applications 102, using display devices 114 and input devices 116. A testing process used by such a tester is described in more detail below in connection with
As will be described in more detail below in connection with
Referring to
A data flow diagram of an example implementation of the display manager will now be described in connection with
In this example implementation, the display manager 300 includes a rendering module 304 that has an input to receive user interface data 302 from an application. A matching module 306 also has an input to receive the user interface data 302. The matching module has another input to receive the conventions 308. The conventions 308 can be defined, for example, using rules for processing the user interface data 302 or any other information suitable to allow the matching module 306 to compare the user interface data 302 to the conventions 308. The matching module has an output to provide results 310, which results indicate whether the user interface data 302, when rendered, produces display data 312 that conforms to the convention. For example, in response to user interface data, the rendered display data can have a first color if the display data is conforming to the convention, and can have a second color if the display data is not conforming to the convention. The first and second colors are substantially different from each other so as to allow a person viewing the display to readily differentiate between conforming and nonconforming display data.
Referring now to
The matching module receives 400 user interface data and validates 402 the user interface data against the convention. The validation performed is dependent on the nature of the convention data, and can involve processing a rule, looking up values in a lookup table, comparing of values, and the like. The matching module outputs results to the rendering module, which renders 404 the user interface data according to the validation results. The process of
An example graphical user interface of a search application is shown in
A data flow diagram of another example implementation of the display manager will now be described in connection with
In particular, a conventional font file 612 that would normally be used by the rendering module 604 is modified, using an editor 610 and input 614. The font file 612 generally includes data for a font type, including bitmaps of glyphs of characters in that font, for different sizes and colors and other variable parameters of the font. A developer may edit the font file 612, for example using a standard text editor, to make modifications to the font file 612. The modified font file 608 is then used by the rendering module 604 to render user interface data 602 to produce the display data 606.
The modifications to be made to the font file generally include changing the color information such that the glyph returned by the font file to the rendering module, in response to a given character in a given font, has a first color if the character conforms to the convention, and has a second color if the character does not conform to the convention. The first and second colors are substantially different from each other so as to allow a person viewing the display to readily differentiate between conforming and nonconforming fonts. For example, the first and second colors can be of different hues, or in substantially different locations in a color space. Thus, the display data 606 visibly differentiates conforming from nonconforming text when displayed on a display.
Referring now to
The process of
It is also possible to detect, and thus convey to a tester, whether rendered graphical output data is partially conforming (or partially nonconforming). For example, given two font properties, text can be rendered in one color if both properties are conforming, in a second color if neither property is conforming, or with a combination of features if one property is conforming and another is not. For example, a “halo” effect in a second color can surround text in first color to indicate such partial conformance. Other features can be used in the rendered graphical output data to indicate partial conformance so long as such features can be visibly distinguishing when the rendered graphical output data is presented on an output device.
By detecting conformance of rendered graphical output data to a convention at the time of rendering, and providing rendered graphical output data that visibly differentiates conforming from nonconforming display data when presented on an output device, many testing problems are addressed. For example, a person can more easily tell whether text conforms to specified font properties, such as an acceptable font type, font size or color, even if he or she is not familiar with the characters of the language of the text and how those characters should appear in a given font.
Having now described an example implementation,
With reference to
A computer storage medium is any medium in which data can be stored in and retrieved from addressable physical storage locations by the computer. Computer storage media includes volatile and nonvolatile memory, and removable and non-removable storage media. Memory 804 and 806, removable storage 808 and non-removable storage 810 are all examples of computer storage media. Some examples of computer storage media are RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optically or magneto-optically recorded storage device, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Computer storage media and communication media are mutually exclusive categories of media.
Computer 800 may also include communications connection(s) 812 that allow the computer to communicate with other devices over a communication medium. Communication media typically transmit computer program instructions, data structures, program modules or other data over a wired or wireless substance by propagating a modulated data signal such as a carrier wave or other transport mechanism over the substance. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal, thereby changing the configuration or state of the receiving device of the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Communications connections 812 are devices, such as a network interface or radio transmitter, that interface with the communication media to transmit data over and receive data from communication media.
Computer 800 may have various input device(s) 814 such as a keyboard, mouse, pen, camera, touch input device, and so on. Output device(s) 816 such as a display, speakers, a printer, and so on may also be included. All of these devices are well known in the art and need not be discussed at length here. Various input and output devices can implement a natural user interface (NUI), which is any interface technology that enables a user to interact with a device in a “natural” manner, free from artificial constraints imposed by input devices such as mice, keyboards, remote controls, and the like.
Examples of NUI methods include those relying on speech recognition, touch and stylus recognition, gesture recognition both on screen and adjacent to the screen, air gestures, head and eye tracking, voice and speech, vision, touch, gestures, and machine intelligence, and may include the use of touch sensitive displays, voice and speech recognition, intention and goal understanding, motion gesture detection using depth cameras (such as stereoscopic camera systems, infrared camera systems, and other camera systems and combinations of these), motion gesture detection using accelerometers or gyroscopes, facial recognition, three dimensional displays, head, eye , and gaze tracking, immersive augmented reality and virtual reality systems, all of which provide a more natural interface, as well as technologies for sensing brain activity using electric field sensing electrodes (EEG and related methods).
The various storage 810, communication connections 812, output devices 816 and input devices 814 can be integrated within a housing with the rest of the computer, or can be connected through input/output interface devices on the computer, in which case the reference numbers 810, 812, 814 and 816 can indicate either the interface for connection to a device or the device itself as the case may be.
Each component (which also may be called a “module” or “engine” or the like), of a system such as described in FIGS. 1-______ above, and which operates on a computer, can be implemented using the one or more processing units of the computer and one or more computer programs processed by the one or more processing units. A computer program includes computer-executable instructions and/or computer-interpreted instructions, such as program modules, which instructions are processed by one or more processing units in the computer. Generally, such instructions define routines, programs, objects, components, data structures, and so on, that, when processed by a processing unit, instruct the processing unit to perform operations on data or configure the processor or computer to implement various components or data structures.
This computer system may be practiced in distributed computing environments where operations are performed by multiple computers that are linked through a communications network. In a distributed computing environment, computer programs may be located in both local and remote computer storage media.
Alternatively, or in addition, the functionality of one or more of the various components described herein can be performed, at least in part, by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Program-specific Integrated Circuits (ASICs), Program-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.
The terms “article of manufacture”, “process”, “machine” and “composition of matter” in the preambles of the appended claims are intended to limit the claims to subject matter deemed to fall within the scope of patentable subject matter defined by the use of these terms in 35 U.S.C. §101.
It should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific implementations described above. The specific implementations described above are disclosed as examples only.
