Patent Application
20120311538
A well-known problem among enterprises that build software solutions such as web applications or rich applications is that the software solution that is built often fails to meet the expectations of the customer. For example, a typical scenario is for some customer (e.g., a “stakeholder” using well-known Agile software development terminology, any internal or external application user, a subject matter expert and so forth) to request that a particular software solution be provided. The enterprise's development team puts together the software solution based upon the request, but when it gets sent to the customer, the customer comes to realize that while the product may meet the request as specified, it does not meet the requirements of what the customer actually meant or wanted.
One way to avoid customer dissatisfaction is to review working software with the customer, as various parts are completed, to obtain direct feedback. Today, the procedure to obtain such feedback is to basically carry out a number of manual actions. These include setting up the right software build, inviting the customer for a meeting, taking notes by hand, attempting (after the fact) to reconstruct the path the customer took, and sending the notes and observations to the developers, e.g., via email.
However, this procedure provides the customer and enterprise with a poor experience, and is often unreliable. For example, the feedback sent to the development team may not be actionable, the feedback may be lost or not sent, the feedback may be misinterpreted, and so forth.
This Summary is provided to introduce a selection of representative concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used in any way that would limit the scope of the claimed subject matter.
Briefly, various aspects of the subject matter described herein are directed towards automating the workflow/procedure of requesting and obtaining feedback from a customer (user). A software solution (e.g., application or application piece) is set up to run in an evaluation environment, e.g., a virtual environment such as a virtual machine that runs a rich application, or a hosted web application run on a local machine. A system facilitates creating and sending a request (e.g., via email) for feedback with respect to the software solution, and configures a feedback tool by which feedback is generated by the user. The system receives the feedback on the software solution, and maintains and provides access to the feedback, such as by querying a database in which the feedback is stored, to allow a product owner/development team to take action with respect to the software solution.
In one aspect, the software solution may correspond to one or more pieces of a larger program, each referred to as a user story. The request may specify one or more users and one or more stories (e.g., an Agile user story or a bug/code defect), and may be created based upon interaction with a setup program. For each story, a feedback work item is created and maintained in association with each user in a data store. Feedback state data may be maintained for each work item and associated user. The state data is updated to indicate the state of the feedback request, such as to indicate when the request has been sent to a user prior to feedback being received from that user, when the feedback has been received from that user, when the feedback has been reviewed, or when feedback session has been declined by the user or canceled.
The user may launch the software solution (which may or may not install the software, as needed) via a link and/or using other like information (e.g., file data) provided in the request. The software solution is launched in an evaluation environment or locally on the user's machine, where the software solution is represented as visible information on a display mechanism (e.g., one or more display monitors or the like) for viewing and interacting with the software solution. The feedback tool also may be represented as visible information on the display mechanism, for viewing and interacting with feedback-related content as the user enters such content. The user may use the feedback tool to provide text, zero or more images, arbitrary files, image annotations, and/or recorded audiovisual information, each obtained with respect to the software solution being run in the evaluation environment. The feedback tool may be configured with a mechanism for indicating a bug, and/or a rating mechanism.
A workflow/procedure is described for obtaining feedback, including sending a communication inviting a specified user to return feedback corresponding to evaluating a software solution, and for setting up the software solution to evaluate. A feedback tool is configured to obtain feedback with respect to the evaluation of the software solution by the user, and feedback work items corresponding to received feedback are maintained for subsequent access. The workflow/procedure may create a link corresponding to an action taken on a feedback work item, and each feedback work item may be associated with state indicative of whether for that work item feedback has been requested and not yet received, whether feedback has been received, or whether received feedback has been reviewed. The user may be notified by presenting information to a user that provided the feedback indicative of an action taken based upon the feedback.
Other advantages may become apparent from the following detailed description when taken in conjunction with the drawings.
The present invention is illustrated by way of example and not limited in the accompanying figures in which like reference numerals indicate similar elements and in which:
Various aspects of the technology described herein are generally directed towards automating the procedure of requesting and obtaining and tracking feedback from a customer with respect to a software solution. In general, via the technology described herein, the customer receives a feedback request to evaluate a software solution, and is provided with a mechanism for providing rich, actionable feedback. Also via the technology described herein, the feedback is maintained, such that a development team is able to take action on the feedback and close the loop with the customer.
It should be understood that any of the examples herein are non-limiting. For one, while software solutions are described herein as being the product that is developed, other products and/or services, as well as post-development, pre-development, reverse engineering, planning and so forth may benefit from the technology described herein. As such, the present invention is not limited to any particular embodiments, aspects, concepts, structures, functionalities or examples described herein. Rather, any of the embodiments, aspects, concepts, structures, functionalities or examples described herein are non-limiting, and the present invention may be used in various ways that provide benefits and advantages in computing and product/service related-operations in general.
In one implementation, at a user computing device 102 such as a personal computer or Smartphone, the product owner runs a web application/webpage 104 or the like on a browser 106 that allows the owner to fill in data on a form related to a request for feedback; (note that the feedback request may be made in any suitable way, e.g., a web application, rich application, an email message, and so forth). Examples of the data that may be filled in include the user or users to whom the request is to be sent, information about the software solution for which feedback is being sought, and optionally a story (or set of stories) that are to be evaluated, where “story” (using Agile terminology) is generally an individual unit of work that provides some value to the software solution. Note that it is feasible to not specify a story, in which event the product owner is requesting general feedback about the specified software solution; a feedback work item may be created on demand for each user when such general feedback is received.
As represented in
When the product owner has specified the appropriate information, the product owner signals completion to the server 108, e.g., by clicking a submit button 112 or the like. In addition to saving the user information and any work item or items in the database 110, the server 108 also generates a communication 114 (e.g., one or more email messages) for the specified users. The email generally indicates how the customer is to proceed, and provides one or more links and/or files to facilitate obtaining the user feedback.
The product owner may preview the communication 114 before sending to the users. Also, the server 108 may perform any validation-related operations at this time, e.g., to ensure that any permissions are valid, that the email server is configured, and so forth, so as to detect any errors as soon as possible.
In one implementation, the product owner and/or server 108, manually and/or automatically, or both, may configure an evaluation environment 116 for running the software solution. As described below, the user runs the software solution in the evaluation environment 116 and provides feedback 118.
For example, for a web application, the evaluation environment 116 may comprise a website that a user may access via a link in their invitation email. Such a website may be provided in a virtual environment rather than in the enterprise's production servers, for example.
Alternatively, for a rich application, the evaluation environment 116 may comprise a virtual machine that the user accesses via a remote desktop connection. This protects the user machine, user privacy, and prevents harm to an actual hosting computer. For such a rich application, the server 108 generates a file (e.g., a Remote Desktop Protocol, or RDP file) with appropriate data, e.g., credentials and other data used to access a remote virtual environment, and attaches the file (or a link thereto) to the communication 114. Note that if there are no stories specified, there are no feedback work items, and thus the communication 114 contains the RDP without accompanying story-related information. The application can also be on a different device, e.g., a Smartphone application, an application on a tablet computing device, and so forth.
Turning to additional details of work item generation, a feedback work item is generated for each piece of feedback. For a story, each feedback work item is associated with m users that receive the invitation to provide feedback, where m is at least one user. If the product owner specifies n work items (stories), the system has m times n pieces of feedback to manage per software solution. In other words, there is one work item of type feedback created in the database for each specified (user, work item) entry. The feedback work item has a two-way link to the user story/bug.
For general feedback, a feedback work item is created on demand when such feedback is received, and linked to the user that provided the feedback. Note that a request for general feedback as well as for story-specific feedback may be forwarded by the user to another person, e.g., an assistant. In the event this occurs and the other person provides feedback, a feedback work item is created on demand when the feedback is received, and linked to the other user that provided the feedback. The original user may also provide feedback. This allows delegation and/or distribution of requests for feedback, although it is feasible to disallow delegation and/or distribution by simply deleting feedback received from unspecified other users.
State data is maintained in association with each (user, feedback work item) association, indicating a current state of the feedback in the procedure. As represented in
As described below, when the product owner reviews the feedback, the state data transitions to a reviewed state 224 indicating that the product owner took some action; this state transition may be manually controlled by the product owner. As also represented in
Turning to the user experience and a user interface by which the user provides feedback, when the user receives an email and elects to provide feedback, the software solution is launched. For example, if an RDP file is attached to the communication 114, a remote desktop is launched corresponding to a virtual machine, using the credentials provided on the virtual machine. If there is no RDP file, the software solution launches locally, e.g., as a web application hosted in the user's local browser program. Note the application can be launched on a different box/device.
In one implementation generally represented in
Via the tool 330, the user may choose a general context or a story context (if a story is specified) for providing feedback, which will be used to link the feedback back to a work item, creating one on demand if necessary as described above. The user may select among available stories if more than one has been specified for the user; each new story selection initially starts with a “clean slate” for user-provided feedback with respect to that story.
In one implementation generally corresponding to
As another type of feedback, the user is also able to use a capture tool (e.g., via icon 338) to insert screenshots and/or screen clips into the work area, depending on what the user desires. Screen capturing technology is known, including by capturing the whole screen or via a snipping tool or the like that allows a mouse or other pointing device to select only a portion of the screen for capture. The user may thus insert one or more images into the work area, such as the image 340. Thumbnail previews, scrolling or other mechanisms may be used for images that are too large for the area 334. An annotation tool (e.g., icon 342) allows for adding one or more captions or the like to the image.
Still further, video of the desktop display as well as audio may be captured during the feedback session, as represented by the recording-related data region 344. Note that this may be previewed and/or edited for privacy before sending to the database. In one implementation the video and audio are merged into one file that is associated with the user and feedback work item as one type of received feedback.
A user may also click a bug button 348 to file a bug and thereby place a work item of type “bug” into the database, such as to emphasize a particularly noteworthy observation. Note that this applies to any work item type; bug is a default example.
A rating tool 350 or the like also may be used for providing feedback. When the user has completed entering and capturing of the feedback, the user presses a submit button 346 or the like, which then sends the feedback to the database, where it is associated with the feedback work item and user identity. In this manner, the user may provide feedback in various ways, including stream of consciousness type feedback, which may be associated with other captured feedback via audio and video. As can be readily appreciated, not every exemplified tool need be used by a user, and indeed not every tool may be provided in alternative implementations. Further, additional tools and/or sensors may be provided to capture feedback in other implementations.
Once the feedback 118 is returned and maintained in the database 110, the product owner is able to use the feedback in various ways, including with other feedback. For example, as generally represented in
The product owner may search and/or filter feedback by work item ID, text (e.g., keywords) and/or other means (e.g., time, bugs and so forth). The product owner may also evaluate each item of feedback and use it as desired. For example, the product owner is able to read the text, view the images, replay the audiovisual recording, and so forth. The audio and video may be indexed in time and/or by action, so that the owner can quickly jump to the appropriate place in the recording for some event or the like noted in the feedback.
Moreover, the owner may process the feedback in various ways by parsing it and taking action with respect to individual pieces of the feedback. For example, the owner may highlight content and be given a user interface option to add the highlighted content to a new work item (created on demand) or link the highlighted content to an existing other work item. A link to the source work item from which the content was selected to the new or other work item is created. While reviewing the feedback, the product owner can also create a work item of type bug.
Once the review is complete, the owner or an automated process switches switch the state data for the work item to the reviewed state. The developers (e.g., working with the primary product owner) may then query the reviewed feedback, query for bugs, and so on, in order to make appropriate changes to the software solution based upon the feedback. The user may be notified in some way that his or her feedback was used, possibly with comments on the feedback, e.g., what was valuable, ways to improve, and so forth.
In the example of
The customer receives the invitation as generally described above and as represented in step 506. In the event that the application is a web application, the system creates the feedback request item, generates a feedback invitation for the user, provides a link to the web application to test, provides a link to the feedback tool in the invitation and configures the feedback tool with any previously specified user stories. In the event that the application is a is a rich application, the system creates the feedback request item, generates a feedback invitation for the user, provides a link to the virtual environment, provides a link where the customer can launch the feedback too and configures the feedback tool with any previously specified user stories.
Steps 508 and 510 represent the customer actions based upon providing feedback on a web application or a rich application, respectively. Thus, from the feedback invitation, a web application (step 508), or the virtual machine and application is launched (step 510); also launched is the feedback tool.
Step 312 represents the customer providing rich feedback on the application as described above, e.g., possibly including rich text, screenshots with annotations, audio and video and so forth. Once submitted, the system associates (links) the returned feedback with the corresponding feedback work item or items, and updates the feedback state for this user and work item.
At step 514, the product owner sees the rich feedback arrive, e.g., exposed by the system as feedback items via a web interface on top of TFS, or in some other matter, e.g., a client application. The owner may then take action on the rich feedback, e.g., by creating new bugs, user stories and other work items for the development team to prioritize and take action. The system creates links between the rich feedback, the bugs and the user stories created.
The work item feedback state may be updated to indicate that the customer's feedback has been reviewed, whereby the customer may review the feedback that was created and the actions that were taken, closing the loop. The customer may see the feedback/action-related information, such as via a web interface on top of TFS, for example.
As can be seen, the system provides integration of development and customer evaluation via the workflow automation of requesting feedback, giving feedback and taking action on the feedback, each of which are traceable throughout the feedback-related procedure, thereby providing benefits and advantages over known ways to obtain feedback. The product owner is able to select stories for obtaining feedback, see what has been completed, and trace the feedback throughout the procedure.
The invention is operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well-known computing systems, environments, and/or configurations that may be suitable for use with the invention include, but are not limited to: personal computers, server computers, hand-held or laptop devices, tablet devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
The invention may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and so forth, which perform particular tasks or implement particular abstract data types. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in local and/or remote computer storage media including memory storage devices.
With reference to
The computer 610 typically includes a variety of computer-readable media. Computer-readable media can be any available media that can be accessed by the computer 610 and includes both volatile and nonvolatile media, and removable and non-removable media. By way of example, and not limitation, computer-readable media may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by the computer 610. Communication media typically embodies computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of the any of the above may also be included within the scope of computer-readable media.
The system memory 630 includes computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM) 631 and random access memory (RAM) 632. A basic input/output system 633 (BIOS), containing the basic routines that help to transfer information between elements within computer 610, such as during start-up, is typically stored in ROM 631. RAM 632 typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit 620. By way of example, and not limitation,
The computer 610 may also include other removable/non-removable, volatile/nonvolatile computer storage media. By way of example only,
The drives and their associated computer storage media, described above and illustrated in
The computer 610 may operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 680. The remote computer 680 may be a personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the computer 610, although only a memory storage device 681 has been illustrated in
When used in a LAN networking environment, the computer 610 is connected to the LAN 671 through a network interface or adapter 670. When used in a WAN networking environment, the computer 610 typically includes a modem 672 or other means for establishing communications over the WAN 673, such as the Internet. The modem 672, which may be internal or external, may be connected to the system bus 621 via the user input interface 660 or other appropriate mechanism. A wireless networking component such as comprising an interface and antenna may be coupled through a suitable device such as an access point or peer computer to a WAN or LAN. In a networked environment, program modules depicted relative to the computer 610, or portions thereof, may be stored in the remote memory storage device. By way of example, and not limitation,
An auxiliary subsystem 699 (e.g., for auxiliary display of content) may be connected via the user interface 660 to allow data such as program content, system status and event notifications to be provided to the user, even if the main portions of the computer system are in a low power state. The auxiliary subsystem 699 may be connected to the modem 672 and/or network interface 670 to allow communication between these systems while the main processing unit 620 is in a low power state.
While the invention is susceptible to various modifications and alternative constructions, certain illustrated embodiments thereof are shown in the drawings and have been described above in detail. It should be understood, however, that there is no intention to limit the invention to the specific forms disclosed, but on the contrary, the intention is to cover all modifications, alternative constructions, and equivalents falling within the spirit and scope of the invention.
