Patent Application
20160266878
A wireframe, also known as a website wireframe, is a visual guide that represents the skeletal framework of a website or web based service. Wireframes are created for the purpose of arranging elements to best accomplish a particular purpose. The purpose is usually being informed by a business objective and a creative idea. The wireframe depicts the page layout or arrangement of the website's content, including interface elements and navigational systems, and how they work together. The wireframe usually lacks content, typographic style, color, or graphics, since the main focus lies in functionality, behavior, and priority, rather than on content. Wireframes help establish functionality, and the relationships between different screen elements of a website. Aside from websites, wireframes are utilized for the prototyping of mobile sites, computer applications, or other screen-based products that involve human-computer interaction, such as Software as a Service (SaaS).
SaaS is a software distribution model in which applications are hosted by a vendor or service provider and made available to customers over a network, typically the Internet. Porting legacy application to a SaaS environment typically requires re-implementation of the graphic user interface (UI)) using HTML. HTML design often requires creation of a wireframe from scratch, which is a labor intensive and error-prone process.
A system for creating a wireframe from a user interface of a legacy software application is proposed. This wireframe can be used to port the software application to a SaaS environment. The wireframe is created by running the legacy software application on a computing system such that the user interface is displayed on a monitor. A portion of the user interface (e.g., content) is blocked from view. An image of the user interface, with the portion being blocked from view, is captured and used to automatically create code describing the user interface. For example, one or more shapes in the image are recognized as user interface widgets and HTML code (or other types of code) is created that describes the recognized widgets.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. The claimed subject matter is not limited to implementations that solve any or all disadvantages noted in the Background.
As will be appreciated by one skilled in the art, aspects of the present disclosure may be illustrated and described herein in any of a number of patentable classes or context including any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof. Accordingly, aspects of the present disclosure may be implemented entirely hardware, entirely software (including firmware, resident software, micro-code, etc.) or combining software and hardware implementation that may all generally be referred to herein as a “circuit,” “module,” “component,” or “system.” Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer readable media having computer readable program code embodied thereon for programming a computer/processor.
Any combination of one or more computer readable media may be utilized. The computer readable media may be a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an appropriate optical fiber with a repeater, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, CII, VB.NET, Python or the like, conventional procedural programming languages, such as the “c” programming language, Visual Basic, Fortran 2003, Pert, COBOL 2002, PHP, ABAP, dynamic programming languages such as Python, Ruby and Groovy, or other programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider) or in a cloud computing environment or offered as a service such as a Software as a Service (SaaS).
Aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatuses (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable instruction execution apparatus, create a mechanism for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer readable medium that when executed can direct a computer, processor other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions when stored in the computer readable medium produce an article of manufacture including instructions which when executed, cause a computer or processor to implement the function/act specified in the flowchart and/or block diagram block or blocks. The computer program instructions may also be loaded onto a computer, other programmable instruction execution apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatuses or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
A system for creating a wireframe from a user interface of a software application is proposed herein. The wireframe is created by running software application on a computing system such that the user interface is displayed on a monitor. A portion of the user interface is blocked from view. An image of the user interface, with the portion being blocked from view, is captured and used to automatically create code describing the user interface. For example, one or more shapes in the image are recognized as user interface widgets and HTML code (or other types of code) is created that describes the recognized widgets.
E-ink display 110 is a transparent electrochromic display that includes materials that change/add color (e.g., black or other color) when electricity is applied. While some displays can be used to show black and white, others are operated such that (based on the applied electricity) one or more portions of the e-ink display are black (or dark or have other color) or otherwise opaque, and one or more other portions of the display are transparent. The e-ink display can be implemented using traditional e-ink technologies, as well as transparent LEDs or OLEDs, as well as other technologies known in the art. In one embodiment, computer 106 includes software for configuring e-ink display 110. In one example embodiment, the software configuring e-ink display 110 can black out the entire display and then add shapes of transparent regions, with the shapes being of various sizes and can be placed in any location on the e-ink display 110.
Camera 108, which can be a digital still camera or a digital video camera. Computer 106 can be used to trigger the shutter of camera 108 to take a photograph or a video. Images or video captured by camera 108 can be transmitted to computer 106. In another embodiment, camera 108 includes a wireless trigger such that computer 106 can trigger the shutter via a wireless signal. In another embodiment, e-ink display 110 can include a light trigger which when flashed will be sensed by a sensor on camera 108 in order to trigger camera 108 to take a photograph or video.
Camera 108 is placed in front of monitor 104 and pointed at monitor 104 so that camera 108 can take photos (or video) of whatever is being displayed by monitor 104. In the embodiment when it is desired to create a wireframe for the user interface (UI) of a legacy software application, the legacy software application will be run on computer 102 with the UI being displayed on monitor 104. Camera 108 is positioned and pointed such that camera 108 can take photos (capture images) of the user interface on monitor 104. E-ink display 110 is positioned between monitor 104 and camera 108 so that e-ink display 110 can be configured to selectively block portions of the user interface being displayed on monitor 104. Camera 108 views the UI on monitor 104 through e-ink display 110. As discussed above, the goal in creating the wireframe is to be able to describe the functionality of the UI. It is not necessary that all the content be added to the wireframe too. Therefore, an operator (e.g., human operator) can work with computer 106 to configure e-ink display 110 to block from view some or all content of the UI being displayed on monitor 104, as well as block various UI elements if desired. In one embodiment, e-ink display 110 is configured to block content and not block widgets of the UI. A widget is a UI element that a computer user interacts with through direct manipulation. Examples of UI elements include (but are not limited to) windows (may display still images or videos), dropdown menus, tabs, textboxes, buttons, dials, sliders, dialogue boxes, text insertion points, adjustment handles, icons, etc. No particular set of widgets is required for the technology described herein.
Camera trigger software 172 is used to trigger the shutter of camera 108. In one embodiment, there is a wired connection between computer 106 and camera 108 to control the shutter. In other embodiments, an RF wireless connection could be used. In yet other embodiments, a light can be used to trigger camera 108. In one example, a light on e-ink display 110 can be used to trigger camera 108, with that light being controlled (via a wired or wireless connection) by computer 106 and camera trigger software 172. In one embodiment, a user will use a user interface to engage camera trigger software 172 to control camera 108.
Image transfer software 174 is used to transfer images from camera 108 to computer 106. In one embodiment, images are transferred via a wired connection. In another embodiment, images are transferred via a wireless connection. The images captured and transferred by camera 108 can include jpg format, camera raw format, or any other format known by those of ordinary skill in the art.
Recognition software 176 is used to recognize user interface elements (e.g., widgets) in the UI displayed on monitor 104 and not blocked by e-ink display 110. The user interface elements are recognized in the photographs (images) captured by camera 108. In one embodiment, recognition software 176 will recognize shapes and identify the locations of those shapes in the UI. In one embodiment, the operator will train recognition software 176 to recognize various user interface elements. There are various technologies well known in the art for training software to recognize shapes, and no particular recognition technology is required for this implementation.
Wireframe builder 178 will use the recognition of shapes and the locations of those shapes in order to create the code for an HTML file (or portions of an HTML file) that will comprise the wireframe. In other embodiments, other programming languages, different than HTML, can also be used. While wireframe builder 178 will create the code that represents the recognized user interface elements, that code will then be provided to a wireframe editor (not depicted) to be further edited by a user as part of a design process for the porting of a legacy application to a SaaS environment.
In Step 412, the operator indicates that a photograph (or video) should be taken. The operator can do this by manipulating a user interface on computer 106 or using some type of remote control for computer 106 or camera 108. As discussed above, computer 106 can trigger the shutter of computer 108 using a wired or wireless connection. Additionally, e-ink display 110 can include a set of trigger lights 340 (
Looking back at
In another embodiment, there will only be one computer and no external camera. This one computer will run the legacy software application, perform the image capture, perform the blocking and perform the shape recognition in order to create the wireframe. For example, this embodiment only includes computer 102 and monitor 104 (no camera and no computer 106).
The system of
The components contained in the computer system of
One embodiment of the technology described herein includes a method of creating a wireframe for a user interface of a software application, comprising: blocking a portion of the user interface from view; capturing an image of the user interface with the portion being blocked from view; and automatically creating code describing the user interface from the captured image of the user interface with the portion being blocked from view.
One embodiment includes a system for creating a wireframe for a user interface of a software application, comprising: a computing system including a monitor, the computing system runs the software application and displays the user interface on the monitor; a camera pointed at the monitor; and an e-ink display positioned between the monitor and the camera such that the camera views the user interface on the monitor through the e-ink display, the e-ink display is configurable to selectively block a portion of the user interface from view by the camera, the camera is triggerable to capture an image of the user interface with the portion being blocked from its view and transmit the image to the computing system, the computing system is configured to automatically create an HTML description of the user interface from the captured image.
One embodiment includes a computer program product, comprising: a computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising: computer readable program code configured to receive an image of a user interface of a software application with a portion of the user interface being blocked from view; and computer readable program code configured to automatically create code describing the user interface from the received image of the user interface of the software application with the portion of the user interface being blocked from view.
The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various aspects of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The terminology used herein is for the purpose of describing particular aspects only and is not intended to be limiting of the disclosure. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
The corresponding structures, materials, acts, and equivalents of any means or step plus function elements in the claims below are intended to include any disclosed structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of the present disclosure has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the disclosure in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the disclosure. The aspects of the disclosure herein were chosen and described in order to best explain the principles of the disclosure and the practical application, and to enable others of ordinary skill in the art to understand the disclosure with various modifications as are suited to the particular use contemplated.
