The present disclosure relates generally to software testing, and more particularly to automatically detecting truncation and overlap defects on a webpage using intersection over union calculations of the webpage elements and the webpage element location relationship.
Truncation defects refer to the situation in which at least a portion of a number string or a text string is shortened. For example, when computing software is adapted to different languages, regional peculiarities and technical requirements of a target locale during internalization and localization, truncation may occur. For instance, when a word is translated from one language to another, it can take more space than in the original version. For example, English “no” in German is “nein,” which has more characters. In another example, the English phrase “Have a nice day!” corresponds to the German equivalent “Ich wünsche Ihnen Einen schönen Tag!”. The words may then be truncated when translated from English to German in order to fit the text into the webpage element, such as within a checkbox graphical user interface element.
Overlap defects refer to the situation in which at least a portion of a webpage element extends over another webpage element so as to at least partially cover that webpage element. Webpage elements, as used herein, refer to portions of the depicted webpage, which may contain a data item, text, image or perhaps nothing. Examples of webpage elements include checkboxes, radio boxes, dropdown lists, list boxes, buttons, toggles, text fields, date fields, sliders, search fields, tags, icons, progress bars, message boxes, notifications, windows, accordions, etc. Such overlap defects may also occur during internalization and localization.
Such defects (truncation and overlap defects) on a depicted webpage make it difficult for the user to understand what is shown on the webpage.
In one embodiment of the present disclosure, a computer-implemented method for detecting truncation and overlap defects on a webpage comprises obtaining location and size information for elements of the webpage. The method further comprises performing an intersection over union (IoU) calculation for a first webpage element and a second webpage element on the webpage using the obtained location and size information for at least one of the first webpage element and the second webpage element. The method additionally comprises determining a location relationship between the first and second webpage elements on the webpage using the obtained location and size information for at least one of the first webpage element and the second webpage element. Furthermore, the method comprises reviewing a table using the IoU calculation for the first and second webpage elements on the webpage and the location relationship between the first and second webpage elements to identify a normal condition, a truncation defect or an overlap defect, where the table defines truncation defect and overlap defect scenarios. Additionally, the method comprises marking a truncation defect or an overlap defect on a screen capture of the webpage in response to identifying the truncation defect or the overlap defect, respectively, in the table.
Other forms of the embodiment of the computer-implemented method described above are in a system and in a computer program product.
The foregoing has outlined rather generally the features and technical advantages of one or more embodiments of the present disclosure in order that the detailed description of the present disclosure that follows may be better understood. Additional features and advantages of the present disclosure will be described hereinafter which may form the subject of the claims of the present disclosure.
A better understanding of the present disclosure can be obtained when the following detailed description is considered in conjunction with the following drawings, in which:
As stated in the Background section, truncation defects refer to the situation in which at least a portion of a number string or a text string is shortened. For example, when computing software is adapted to different languages, regional peculiarities and technical requirements of a target locale during internalization and localization, truncation may occur. For instance, when a word is translated from one language to another, it can take more space than in the original version. For example, English “no” in German is “nein,” which has more characters. In another example, the English phrase “Have a nice day!” corresponds to the German equivalent “Ich wünsche Ihnen Einen schönen Tag!”. The words may then be truncated when translated from English to German in order to fit the text into the webpage element, such as within a checkbox graphical user interface element.
Overlap defects refer to the situation in which at least a portion of a webpage element extends over another webpage element so as to at least partially cover that webpage element. Webpage elements, as used herein, refer to portions of the depicted webpage, which may contain a data item, text, image or perhaps nothing. Examples of webpage elements include checkboxes, radio boxes, dropdown lists, list boxes, buttons, toggles, text fields, date fields, sliders, search fields, tags, icons, progress bars, message boxes, notifications, windows, accordions, etc. Such overlap defects may also occur during internalization and localization.
Such defects (truncation and overlap defects) on a depicted webpage make it difficult for the user to understand what is shown on the webpage.
In software testing, test automation is the use of software separate from the software being tested to control the execution of tests and the comparison of actual outcomes with predicted outcomes.
For example, in the area of internalization and localization, software tools, such as globalization tools, have been developed to ensure that the software product is stable in terms of both its functionalities as well as representation of data in spite of varying cultures/locales. Such globalization tools engage in globalization testing to validate whether an application acknowledges all the language input texts and it can be used across the globe or not. The sole aim of globalization testing is to uncover the potential problems that could hinder the globalization of the application.
For example, globalization testing involves internationalization testing, which is to verify if the code can deal with all the international support with no breaking of functionality that may cause data loss or data integrity issues.
In another example, globalization testing involves localization testing, which involves translation of the software and its presentation to the end user. The translation of the program considers icons, graphics, user manuals, help files, documentation and other cultural specifications.
While such software tools to test software assist in uncovering potential problems that hinder the globalization of the application, such as functional issues, these tools fail to automatically detect truncation and overlap defects on the webpage.
The embodiments of the present disclosure provide a means for automatically detecting truncation and overlap defects on the webpage by performing an intersection over union calculation, such as for adjacent webpage elements, and using such information in connection with the location relationship between such elements as well as whether either of the elements contain text to determine whether or not a truncation or overlap defect was detected.
In some embodiments of the present disclosure, the present disclosure comprises a computer-implemented method, system and computer program product for detecting truncation and overlap defects. In one embodiment of the present disclosure, location and size information for the elements of the webpage are obtained. An intersection over union (IoU) calculation is performed for two webpage elements on the webpage using the obtained location and size information for at least one of these two webpage elements. In one embodiment, the intersection over union (IoU) calculation equals: |A∩B|/|A∪B|, where A and B correspond to a first webpage element and a second webpage element on the webpage, respectively. Furthermore, the location relationship between these two webpage elements on the webpage is determined using the obtained location and size information for at least one of these two webpage elements. In one embodiment, “location relationship,” as used herein, indicates the location scenario of the two webpage elements. A table, referred to herein as the “overlap and truncation situation type table,” which defines truncation defect and overlap defect scenarios, is then reviewed to determine if there are any truncation or overlap defects in the two webpage elements using the IoU calculation, the location relationship and the text condition, which indicates whether text is included in one of the two webpage elements. Such a process is performed for various two webpage elements, such as two adjacent webpage elements, on the webpage. If there were any truncation or overlap defects detected in the webpage, then such truncation and/or overlap defects are marked on a screen capture of the webpage. In this manner, truncation and overlap defects are automatically detected on a webpage.
In the following description, numerous specific details are set forth to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure may be practiced without such specific details. In other instances, well-known circuits have been shown in block diagram form in order not to obscure the present disclosure in unnecessary detail. For the most part, details considering timing considerations and the like have been omitted inasmuch as such details are not necessary to obtain a complete understanding of the present disclosure and are within the skills of persons of ordinary skill in the relevant art.
Referring now to the Figures in detail,
Computing device 101 may be any type of computing device (e.g., portable computing unit, Personal Digital Assistant (PDA), laptop computer, mobile device, tablet personal computer, smartphone, mobile phone, navigation device, gaming unit, desktop computer system, workstation, Internet appliance and the like) configured with the capability of connecting to network 103 and consequently communicating with other computing devices 101 and web server 102.
Network 103 may be, for example, a local area network, a wide area network, a wireless wide area network, a circuit-switched telephone network, a Global System for Mobile Communications (GSM) network, a Wireless Application Protocol (WAP) network, a WiFi network, an IEEE 802.11 standards network, various combinations thereof, etc. Other networks, whose descriptions are omitted here for brevity, may also be used in conjunction with system 100 of
In one embodiment, web server 102 is configured to offer a social networking and/or microblogging service thereby enabling users of computing devices 101 to send and read other users' posts. “Posts,” as used herein, include any one or more of the following: text (e.g., comments, sub-comments and replies), audio, video images, etc. Web server 102, as used herein, is configured to enable various collaborative activities, such as online discussion sites (e.g., public forums).
In one embodiment, web server 102 is configured to host websites (website is a collection of relevant webpages that is addressed to a Uniform Resource Locator (URL)) and serve contents to the World Wide Web. For example, web server 102 may host a website in which its collection of relevant webpages are accessed by a user of computing device 101, such as via a web browser (software application for accessing information on the World Wide Web) on computing device 101. Furthermore, web server 102 is configured to process incoming network requests over HTTP (Hypertext Transfer Protocol) and several other related protocols.
System 100 further includes a device, referred to herein as the “defect detector” 104, configured to detect truncation and overlap defects in a webpage shown to users 101 by web server 102, such as a webpage displayed on the display of computing device 101 that was accessed by a web browser on computing device 101 from web server 102. “Truncation defects,” as used herein, refer to the situation in which at least a portion of a number string or a text string is shortened. “Overlap defects,” as used herein, refer to the situation in which at least a portion of a webpage element extends over another webpage element so as to at least partially cover that webpage element.
In one embodiment, such defects are detected by performing an intersection over union calculation between two webpage elements, such as for adjacent webpage elements. Webpage elements, as used herein, refer to portions of the depicted webpage, which may contain a data item, text, image or perhaps nothing. Examples of webpage elements include, but not limited to, checkboxes, radio boxes, dropdown lists, list boxes, buttons, toggles, text fields, date fields, sliders, search fields, tags, icons, progress bars, message boxes, notifications, windows, accordions, etc.
Such information (intersection over union calculation between two webpage elements) may be used in connection with the location relationship between such webpage elements as well as whether either one of the webpage elements contain text to determine whether or not a truncation or overlap defect was detected as discussed in further detail below. A description of the software components of defect detector 104 is provided below in connection with
System 100 is not to be limited in scope to any one particular network architecture. System 100 may include any number of computing devices 101, web servers 102, networks 103 and defect detectors 104.
A discussion regarding the software components used by defect detector 104 to detect truncation and overlap defects, such as on a webpage, is discussed below in connection with
Referring to
As shown in
In one embodiment, situation type definer engine 201 determines the location relationship between two webpage elements (e.g., webpage elements A and B of
Furthermore, situation type definer engine 201 is configured to utilize a table, that is referred to herein as the “overlap and truncation situation type” table, to determine if there are any truncation and/or overlap defects on a webpage as shown below.
In one embodiment, the overlap and truncation situation type table includes a listing of the IoU value (or range of IoU values) for webpage elements A and B, the location relationship between webpage elements A and B, the text condition (whether webpage element A or B or neither contain text) and the corresponding scenario (discussed below in connection with
The “location relationship” of the table, as discussed below, indicates the various location scenarios of the two neighboring webpage elements, webpage elements A and B, as discussed further below.
In one embodiment, the overlap and truncation situation type table (“table”) is based on six different scenarios of the positioning of one webpage element versus another webpage element as illustrated in
As illustrated in
Referring to scenario (1) 401 of
Referring to scenario (2) 402 of
Referring to scenario (3) 403 of
Referring to scenario (3) 403 of
Referring to scenario (3) 403 of
Referring to scenario (4) 404 of
Referring to scenario (5) 405 of
Referring to scenario (6) 406 of
As discussed further below, such a table will be used by defect detector 104 to determine if there are any truncation and/or overlap defects in the webpage currently being displayed on a display of computing device 101.
In one embodiment, such a table is populated by an expert and stored in a storage device (e.g., memory, disk drive) of defect detector 104.
In one embodiment, defect detector 104 further includes an element locator engine 202 configured to detect the location and size information of the webpage elements, such as the webpage elements of a webpage currently being displayed on a display of computing device 101. In one embodiment, element locator engine 202 identifies such webpage elements using a webpage reading tool (e.g., Java® HttpClient, JSoup, Java® HtmlCleaner, Apache® HttpClient, Jetty® HttpClient and Java® HtmlUnit). In one embodiment, such webpage reading tools extract the HTML (Hypertext Markup Language) code for the webpage, where the HTML code is used to structure the webpage and its content.
In one embodiment, element locator engine 202 identifies webpage elements on a webpage by the element identifier (ID) in the HTML code. The ID is an identifier for the element that is typically assigned by the developer of the code.
In another embodiment, element locator engine 202 identifies webpage elements on a webpage via field names in the HTML code. For example, each input field in a form may have a name, such as username and password fields.
In another embodiment, element locator engine 202 identifies webpage elements on a webpage via text in the HTML code. For example, links and buttons on a webpage that have visible text may be located in this manner.
In another embodiment, element locator engine 202 identifies webpage elements on a webpage via link text in the HTML code.
In another embodiment, element locator engine 202 identifies webpage elements on a webpage via the Cascading Style Sheets (CSS) class attribute in the HTML code. For example, using the CSS class attribute, element locator engine 202 is able to locate items that have a unique style on the webpage. For example, submit and login/logout buttons may be identified on the webpage since there is typically only one such element on the webpage with such a style. In one embodiment, element locator engine 202 locates an element with a specific CSS class attribute, which may be supplied via a list of CSS class attributes in a data structure (e.g., table). Such a data structure may be populated by an expert and stored in a storage device (e.g. memory, disk drive) of defect detector 104.
In another embodiment, element locator engine 202 identifies webpage elements on a webpage via XPath. XPath is a language for traversing the structure of the DOM (document object model) of the webpage.
In another embodiment, element locator engine 202 identifies webpage elements on a webpage from web server 102 providing defect detector 104 user interface metadata that describes the contents and layouts of the user interface elements, including the size of the user interface elements and the location of the user interface elements (e.g., x-y coordinates on the display screen).
In another embodiment, element locator engine 202 identifies webpage elements via the DOM tree of the webpage currently rendered in the browser of computing device 101. In one embodiment, element locator engine 202 utilizes the Chrome DevTools elements panel to identify the elements on the webpage.
Once the webpage element is located, element locator engine 202 determines the location and size information of the webpage elements (e.g., x-y coordinate on the display screen of computing device 101). As discussed above, in one embodiment, such information may be obtained via user interface metadata that describes the contents and layouts of the user interface elements, including the size of the user interface elements and the location of the user interface elements (e.g., x-y coordinates on the display screen).
Furthermore, in one embodiment, element locator engine 202 determines the location and size information of the webpage elements (e.g., x-y coordinates on the display screen of computing device 101) identified in the HTML code via the use of the HTMLElement.offsetWidth and HTMLElement.offsetHeight properties. Other properties that may be used to obtain such information include Element.clientWidth, Element.clientHeight, Element.scrollWidth and Element.scrollHeight properties.
In one embodiment, the extracted location and size information for the webpage elements are stored in a file (e.g., JSON file) by element locator engine 202.
In one embodiment, element locator engine 202 may also obtain the name of the webpage element from the HTML code as well as any text (e.g., text string) within the webpage element based on analyzing the HTML attributes and tags. For example, the name attribute may be used to specify a name. In another example, the <blockquote> tag may indicate a section that is quoted from another source. In a further example, the <textarea> tag defines the text area. In one embodiment, element locator engine 202 may search for such attributes and tags in the HTML code based on a listing of such attributes and tags in a data structure (e.g., table) stored in the storage device (e.g., memory, disk drive) of defect detector 104.
In one embodiment, after extracting the location and size information for the webpage elements, element locator engine 202 obtains a screen capture of the webpage, such as using a screenshot tool (e.g., Snagit®, screenpresso, picpick, Jing®, winsnap, greenshot, etc.).
Defect detector 104 further includes a text border analyzer 203 configured to identify a text border of a text area of a webpage area by performing a horizontal and a vertical projection of the text area.
Defect detector 104 further includes a marker engine 204 configured to mark in the screen capture of the webpage any identified truncation or overlap defects.
A further description of these and other functions is provided below in connection with the discussion of the method for detecting truncation and overlap defects.
Prior to the discussion of the method for detecting truncation and overlap defects, a description of the hardware configuration of defect detector 104 (
Referring now to
Defect detector 104 has a processor 501 connected to various other components by system bus 502. An operating system 503 runs on processor 501 and provides control and coordinates the functions of the various components of
Referring again to
Defect detector 104 may further include a communications adapter 509 connected to bus 502. Communications adapter 509 interconnects bus 502 with an outside network (e.g., network 103 of
In one embodiment, application 504 of defect detector 104 includes the software components of situation type definer engine 201, element locator engine 202, text border analyzer 203 and marker engine 204. In one embodiment, such components may be implemented in hardware, where such hardware components would be connected to bus 502. The functions discussed above performed by such components are not generic computer functions. As a result, defect detector 104 is a particular machine that is the result of implementing specific, non-generic computer functions.
In one embodiment, the functionality of such software components (e.g., situation type definer engine 201, element locator engine 202, text border analyzer 203 and marker engine 204) of defect detector 104, including the functionality for detecting truncation and overlap defects, may be embodied in an application specific integrated circuit.
The present invention may be a system, a method, and/or a computer program product at any possible technical detail level of integration. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention.
The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.
Computer readable program instructions described herein can be downloaded to respective computing/processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing/processing device.
Computer readable program instructions for carrying out operations of the present invention may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, configuration data for integrated circuitry, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++, or the like, and procedural programming languages, such as the “C” programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present invention.
Aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.
These computer readable program instructions may be provided to a processor of a computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the Figures. For example, two blocks shown in succession may, in fact, be accomplished as one step, executed concurrently, substantially concurrently, in a partially or wholly temporally overlapping manner, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.
As stated above, truncation and overlap defects on a depicted webpage make it difficult for the user to understand what is shown on the webpage. In software testing, test automation is the use of software separate from the software being tested to control the execution of tests and the comparison of actual outcomes with predicted outcomes. For example, in the area of internalization and localization, software tools, such as globalization tools, have been developed to ensure that the software product is stable in terms of both its functionalities as well as representation of data in spite of varying cultures/locales. Such globalization tools engage in globalization testing to validate whether an application acknowledges all the language input texts and it can be used across the globe or not. The sole aim of globalization testing is to uncover the potential problems that could hinder the globalization of the application. For example, globalization testing involves internationalization testing, which is to verify if the code can deal with all the international support with no breaking of functionality that may cause data loss or data integrity issues. In another example, globalization testing involves localization testing, which involves translation of the software and its presentation to the end user. The translation of the program considers icons, graphics, user manuals, help files, documentation and other cultural specifications. While such software tools to test software assist in uncovering potential problems that hinder the globalization of the application, such as functional issues, these tools fail to automatically detect truncation and overlap defects on the webpage.
The embodiments of the present disclosure provide a means for automatically detecting truncation and overlap defects on the webpage by performing an intersection over union calculation, such as for adjacent webpage elements, and using such information in connection with the location relationship between such elements as well as whether either one of the elements contain text to determine whether or not a truncation or overlap defect was detected as discussed below in connection with
As stated above,
Referring to
Furthermore, as discussed above, in one embodiment, element locator engine 202 identifies webpage elements on a webpage via the element identifier (ID) in the HTML code, field names in the HTML code, text in the HTML code, link text in the HTML code, Cascading Style Sheets (CSS) class attributes in the HTML code, XPath, user interface metadata, the DOM tree of the webpage currently rendered in the browser of computing device 101, etc.
Additionally, as discussed above, in one embodiment, element locator engine 202 determines the location and size information of the webpage elements (e.g., x-y coordinates on the display screen of computing device 101) via the user interface metadata and/or HTMLElement.offsetWidth, HTMLElement.offsetHeight, Element.clientWidth, Element.clientHeight, Element.scrollWidth and Element.scrollHeight properties.
In one embodiment, element locator engine 202 may also obtain the name of the webpage element from the HTML code as well as any text (e.g., text string) within the webpage element based on analyzing the HTML attributes and tags. For example, the name attribute may be used to specify a name. In another example, the <blockquote> tag may indicate a section that is quoted from another source. In a further example, the <textarea> tag defines the text area. In one embodiment, element locator engine 202 may search for such attributes and tags in the HTML code based on a listing of such attributes and tags in a data structure (e.g., table) stored in the storage device (e.g., memory, disk drive) of defect detector 104.
In step 602, element locator engine 202 of defect detector 104 stores the obtained location and size information for the webpage elements in a file. For example, such obtained location and size information for the webpage elements may be stored in a JSON file. In one embodiment, such a file is stored in the storage medium (e.g., memory, disk drive) of defect detector 104. In one embodiment, information, such as the name of the webpage element and any text within the webpage element, may also be stored in the file.
The following presents an example of such a JSON file storing the obtained location and size information for the webpage elements, including the name, type, etc. of the webpage elements.
After obtaining and storing the location and size information for the webpage elements on the webpage, in step 603, element locator engine 202 of defect detector 104 obtains a screen capture of the webpage, such as using a screenshot tool (e.g., Snagit®, screenpresso, picpick, Jing®, winsnap, greenshot, etc.).
In step 604, situation type definer engine 201 of defect detector 104 extracts the location and size information for the webpage elements from the file (e.g., JSON file). A JSON file is a standard text-based format for representing structured data based on JavaScript® object syntax. In one embodiment, such information is extracted from the JSON file via the json.load( ) method in Python®, which will be stored as a Python® dictionary. Once loaded, particular data can be fetched by specifying the field, which represents the key in the dictionary.
In one embodiment, situation type definer engine 201 opens the JSON file with a text editor (e.g., Microsoft® notepad, GNU nano, TextEdit, etc.). Once opened, situation type definer engine 201 may search and identify location and size information in the JSON file by identify terms, such as “X” for x-coordinate, “Y” for y-coordinate, “Height” for the height of the element, and “Width” for the width of the element. The corresponding element may be identified in the JSON file by identifying the term “element name” associated with such information, where the name of the element is listed after the term “element name.”
For each webpage element on the webpage, the following steps are performed.
In step 605, a determination is made by element locator engine 202 of defect detector 104 as to whether the webpage element contains text.
In one embodiment, element locator engine 202 locates the webpage element on the webpage using the extracted location information and determines if there is text within such a location on the webpage. For instance, element locator engine 202 may search for any alphanumeric characters. Finding such characters may be an indication of text within the webpage element.
In another example, the file (e.g., JSON file) may include information, such as the name of the webpage element and any text within the webpage element. As a result, element locator engine 202 may determine whether the webpage element contains any text based on whether the file (e.g., JSON file) indicates that the webpage element contains text.
If there is a webpage element that contains text, then, in step 606, text border analyzer 203 of defect detector 104 identifies the text border by performing horizontal and vertical projection of the text area as illustrated in
Referring to
After obtaining such coordinates, text border analyzer 203 projects the text area, such as the area of text 701 of
Referring to
Referring to
As shown in
Such an IoU will be used to determine whether there is a truncation or an overlap defect as discussed further below.
Returning to
Referring again to step 605, if however, the webpage element does not contain text, then, in step 609, situation type definer engine 201 performs the calculation of the intersection over union (IoU) for the webpage element and the adjacent webpage element using the location and size information for these webpage elements (obtained in step 604) as previously discussed above in connection with
In step 610, situation type definer engine 201 determines the location relationship between the webpage element and the adjacent webpage element using the location and size information for these webpage elements (obtained in step 604) as previously discussed. For example, situation type definer engine 201 may extract the x, y coordinates of the webpage elements from a file (e.g., JSON file) in step 604. Such information may be used to determine the location and size of each of these webpage elements as well as the location relationship between such webpage elements.
As previously discussed, in one embodiment, the location relationship information is obtained from the file (e.g., JSON file) in step 604 which indicates the coordinates of the webpage elements, such as neighboring webpage elements. Such coordinates enable situation type definer engine 201 to define the location scenario, such as one of the six location scenarios as shown in
After determining the location relationship between the webpage elements in steps 608, 610, in step 611, situation type definer engine 201 of defect detector 104 reviews the table (overlap and truncation situation type table) using the calculated IoU, the location relationship and text conditions to determine if there are any truncation or overlap defects. That is, situation type definer engine 201 reviews the table to identify a normal condition, a truncation defect or an overlap defect using the calculated IoU, the location relationship and text conditions. Step 611, and the following steps 612-617, are performed for each pair of webpage elements identified on the webpage to determine if there are any truncation or overlap defects on the webpage.
In one embodiment, “text conditions,” as used herein, refer to whether text is located within the webpage element. In one embodiment, as previously discussed, element locator engine 202 determines whether there is text located within the webpage element in step 605.
For example, in one embodiment, element locator engine 202 locates the webpage element on the webpage using the extracted location information and determines if there is text within such a location on the webpage. For instance, element locator engine 202 may search for any alphanumeric characters. Finding such characters may be an indication of text within the webpage element.
In another example, the file (e.g., JSON file) may include information, such as the name of the webpage element and any text within the webpage element. As a result, element locator engine 202 may determine whether the webpage element contains any text based on whether the file (e.g., JSON file) indicates that the webpage element contains text.
Such information (determining whether the webpage element contains text) is used by situation type definer engine 201 when reviewing the table to determine if there are any truncation or overlap defects.
Referring to the overlap and truncation situation type table discussed above, situation type definer engine 201 is able to determine if there are any truncation or overlap defects using the calculated IoU, the location relationship and text conditions. For example, referring to
In step 612, situation type definer engine 201 determines if a truncation defect was detected. If a truncation defect was detected, then, for each truncation defect detected, in step 613, marker engine 204 of defect detector 104 marks the truncation defect in the screen capture of the webpage (performed in step 603). In one embodiment, such a marking may involve an annotation in close proximity to the truncation defect, where the annotation indicates the truncation defect, such as via the words “truncation defect.” Other markings may include highlighting around the truncation defect, change in color around the truncation defect, etc.
If, however, there were no truncation defects detected, then, in step 614, situation type definer engine 201 does not identify any truncation defects.
Upon not identifying any truncation defects or upon marking the truncation defect(s) in the screen capture of the webpage, in step 615, situation type definer engine 201 determines if an overlap defect was detected.
If an overlap defect was detected, then, for each overlap defect detected, in step 616, marker engine 204 of defect detector 104 marks the overlap defect in the screen capture of the webpage (performed in step 603). In one embodiment, such a marking may involve an annotation in close proximity to the overlap defect, where the annotation indicates the overlap defect, such as via the words “overlap defect.” Other markings may include highlighting around the overlap defect, change in color around the overlap defect, etc.
If, however, there were no overlap defects detected, then, in step 617, situation type definer engine 201 does not identify any overlap defects.
In this manner, the principles of the present disclosure detect truncation and overlap defects on a webpage.
As a result of the foregoing, embodiments of the present disclosure provide a means for automatically detecting truncation and overlap defects on the webpage by performing an intersection over union calculation, such as for adjacent webpage elements, and using such information in connection with the location relationship between such elements as well as whether either of the elements contain text to determine whether or not a truncation or overlap defect was detected.
Furthermore, the present disclosure improves the technology or technical field involving software testing. As discussed above, truncation and overlap defects on a depicted webpage make it difficult for the user to understand what is shown on the webpage. In software testing, test automation is the use of software separate from the software being tested to control the execution of tests and the comparison of actual outcomes with predicted outcomes. For example, in the area of internalization and localization, software tools, such as globalization tools, have been developed to ensure that the software product is stable in terms of both its functionalities as well as representation of data in spite of varying cultures/locales. Such globalization tools engage in globalization testing to validate whether an application acknowledges all the language input texts and it can be used across the globe or not. The sole aim of globalization testing is to uncover the potential problems that could hinder the globalization of the application. For example, globalization testing involves internationalization testing, which is to verify if the code can deal with all the international support with no breaking of functionality that may cause data loss or data integrity issues. In another example, globalization testing involves localization testing, which involves translation of the software and its presentation to the end user. The translation of the program considers icons, graphics, user manuals, help files, documentation and other cultural specifications. While such software tools to test software assist in uncovering potential problems that hinder the globalization of the application, such as functional issues, these tools fail to automatically detect truncation and overlap defects on the webpage.
Embodiments of the present disclosure improve such technology by obtaining location and size information for the elements of the webpage. An intersection over union (IoU) calculation is performed for two webpage elements on the webpage using the obtained location and size information for at least one of these two webpage elements. In one embodiment, the intersection over union (IoU) calculation equals: |A∩B|/|A∪B|, where A and B correspond to a first webpage element and a second webpage element on the webpage, respectively. Furthermore, the location relationship between these two webpage elements on the webpage is determined using the obtained location and size information for at least one of these two webpage elements. In one embodiment, “location relationship,” as used herein, indicates the location scenario of the two webpage elements. A table, referred to herein as the “overlap and truncation situation type table,” which defines truncation defect and overlap defect scenarios, is then reviewed to determine if there are any truncation or overlap defects in the two webpage elements using the IoU calculation, the location relationship and the text condition, which indicates whether text is included in one of the two webpage elements. Such a process is performed for various two webpage elements, such as two adjacent webpage elements, on the webpage. If there were any truncation or overlap defects detected in the webpage, then such truncation and/or overlap defects are marked on a screen capture of the webpage. In this manner, truncation and overlap defects are automatically detected on a webpage. Furthermore, in this manner, there is an improvement in the technical field involving software testing.
The technical solution provided by the present disclosure cannot be performed in the human mind or by a human using a pen and paper. That is, the technical solution provided by the present disclosure could not be accomplished in the human mind or by a human using a pen and paper in any reasonable amount of time and with any reasonable expectation of accuracy without the use of a computer.
The descriptions of the various embodiments of the present disclosure have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.
Number | Name | Date | Kind |
---|---|---|---|
9239961 | Cole et al. | Jan 2016 | B1 |
11016651 | Black | May 2021 | B1 |
11182178 | Singh | Nov 2021 | B1 |
11455456 | Bikumala | Sep 2022 | B2 |
11494567 | Sethi | Nov 2022 | B2 |
20130014084 | Sahibzada et al. | Jan 2013 | A1 |
20130055117 | Sahibzada et al. | Feb 2013 | A1 |
20170031879 | Xiong et al. | Feb 2017 | A1 |
20190228058 | Shang | Jul 2019 | A1 |
20200387735 | Huang | Dec 2020 | A1 |
20210103515 | Jiang | Apr 2021 | A1 |
Number | Date | Country |
---|---|---|
2008073838 | Jun 2008 | WO |
Entry |
---|
Anonymous, “A Method and System to Automatically Resolve Message Truncation in Web Page,” ip.com, IPCOM000241347D, Apr. 18, 2015, pp. 1-9. |
Number | Date | Country | |
---|---|---|---|
20220327279 A1 | Oct 2022 | US |