Patent Application
20140222480
Product testing can be an expensive and time-consuming process. For example, complex products such as vehicles can require numerous tests during design, development, and other pre-production stages. Further, product tests may require the construction of complex and expensive prototypes. For example, vehicle prototypes may take weeks or months to build, and may cost hundreds of thousands of dollars. Certain tests may damage or even destroy a prototype, or at least certain components and/or systems of the prototype. Unfortunately, current mechanisms are lacking for test planning and scheduling, including for determining an optimal number of prototypes to build to complete a testing process, the configuration of the prototypes, and how to schedule tasks to maximize prototype usage.
In the following figures, the same reference numbers will be used to illustrate the same components. It should be understood that various of the components set forth herein may be changed.
The test planning system 100 includes one or more input devices 105 that generally provide inputs 110, via a network 115, to a server 120, which generates outputs 125. As described further below, inputs 110 may include test requests 111, test calendar 112, buildable configurations data 113, and/or test compatibility data 114. Further, outputs 125 may include a prototype list 121, a control model compatibility list 122, a test compatibility list 123, and a test schedule 124.
The input device 105 may be a client computer, e.g., a device that includes a graphical user interface, e.g., presented via a standard web browser or the like. Accordingly, the input device 105 may be any one of a number of computing devices, such as a laptop computer, a desktop computer, a handheld computing device such as a smart phone, a tablet computer, etc. Although one input device 105 is shown in
The network 115 is generally a digital network such as a packet network and may include any one or more of a variety of known networks, such as a wide area network, a local area network, a wireless network, a cellular network, etc.
The server 120 may include one or more computing devices having a processor and a memory, the memory storing instructions executable by the processor for carrying out operations described herein. Further, implementations are possible in which operations described herein to input devices 105 and the server 120 are performed by single computing device.
Inputs 110 generally include a set of test requests 111, a test calendar 112, buildable configurations data 113, and test compatibility data 114.
Table 1 provides an example of test requests 111.
Test requests 111 generally identify a set of tests requested for a products, including, for each test, a set of parameters for testing the product, in this example of Table 1, a vehicle. “ID” is a record identifier, generally an identifier for a particular test. “Number of business days” indicates a number of business days into a test process by which a particular test must be completed. “Body style” indicates a required body style for a prototype vehicle to be used during a particular test. “Engine” indicates an engine type, e.g., V6, etc. to be used during a particular test. “Transmission” indicates a transmission type required for a vehicle in a particular test. “Destructive” is a binary variable indicating whether a prototype will be destroyed during a particular test. “Test code” is a code for a particular type of test, e.g., a particular crash test, engine test, etc.
It is to be understood that many other requests 111 could be provided for testing a product prototype, e.g., other test parameters including many various attributes to be included on a prototype to be tested, such as a vehicle prototype, could be specified. For example, and not by way of limitation, in the case of a vehicle prototype, wheel sizes, final drive ratios, emissions configurations, electrical system configurations, any deadlines for completing particular individual tests, etc. could be specified. In other words, columns shown in Table 1 could be removed and/or other columns could be added. Where a prototype is of a product other than a vehicle, columns in Table 1 would include different identifications of prototype attributes.
A test calendar 112 may include a beginning date and an ending date for a test process, i.e., for a test schedule. Further, a test calendar 112 may include data specifying individual days during a test schedule on which tests may be conducted. Table 2 provides an example of data that may be included in a test calendar 112; an actual test calendar 112 could likely have significantly more records because an actual test schedule could include more build dates.
“Build date” is a date during a test schedule when prototypes, e.g., prototype vehicles, could be built. “Day of week” specifies the day of the week associated with the build date. “Is holiday?” specifies whether the build date is a holiday. “Day of planning period” sequentially numbers build dates in a test schedule. “Number of vehicles to be built” specifies a number of prototype vehicles for the test planned to be built on each build date. Other information could be included in a test calendar 112. For example, a column could be added to Table 2 to specify one or more “blackout periods,” during which tests are not to be conducted, e.g., a “1” in the column indicating that the date falls within a blackout period. Further, although not shown in Table 2, test calendar 112 generally further includes a test end date. A test duration may then be determined according to a number of days between a first build date indicated in the test calendar 112 and the test end date.
Buildable configurations data 113 specifies possible configurations of features in prototypes that will be available during a test process. Buildable configurations data 113 is sometimes presented in the form of what may be referred to as control model table, i.e., a table that lists various product attributes that can be combined in a particular product model, e.g., a product prototype. Thus, buildable configurations data 113 may be a list of control models, i.e., a list of possible attribute combinations, i.e., buildable configurations, that constitute a definition of a prototype that may be available for testing.
Table 3 provides an example of buildable configurations 113.
In Table 3, “Engine” refers to a vehicle engine type, “Trans.” refers to a vehicle transmission type, and “Body style” refers to a vehicle body style, all possible attributes or features of a prototype vehicle that may be available for a test. Columns could be added to and/or removed from Table 3, or entirely other attributes could be specified, e.g., in cases where a prototype was for a product other than a vehicle.
Further, although Table 3 illustrates 12 combinations of three attributes, i.e., in Table 3 every possible combination of attributes is identified as a buildable configuration, note that, for many prototypes, certain attribute configurations may not be buildable. For example, with reference to Table 3, is conceivable that a V8 engine with a manual transmission and coupe body style, while a possible combination of possible vehicle attributes, might not actually be a buildable configuration of a vehicle prototype. In such a case, the record indicating a V8 engine, manual transmission, and coupe body style would be omitted from buildable configurations data 113, e.g., would not be shown in Table 3.
Test compatibility data 114 is generally a matrix or chart with the tests specified in test requests 111 listed in row and columns of the chart so as to specify the compatibility of tests with one another. Possibilities for test compatibility for any two tests are generally that tests may be performed on a same prototype in any order, tests may be performed on a same prototype but a particular order is required, or tests cannot be performed on the same prototype. In one implementation, test compatibility data 114 is a matrix of vehicle crash test compatibility. For example, different crash tests may affect different parts of a vehicle, and therefore may be able to be performed on a same prototype vehicle if performed in a certain order. Implementations are possible in which all tests are compatible with one another, in which event test compatibility data could be omitted from the system 100.
Table 4 provides an example of test compatibility data 114.
In Table 4, a “0” in a cell of the matrix means that tests specified at the intersection denoted by that cell can be performed on the same prototype, e.g., a vehicle, in any particular order. A “1” means that the two specified tests can be performed on a same vehicle, but that the test specified in the matrix's row heading must be performed before the test specified in the matrix's column heading. A “2” means that the two specified tests can be performed on a same vehicle, but that the test specified in the matrix's column heading must be performed before the test specified in the matrix's row heading. A “3” means that the two specified tests cannot be performed on a same vehicle. Note that the data 114 can indicate whether a particular test can be performed twice on the same vehicle.
The server 120 generally includes instructions for generating outputs 125. Such instructions generally include instructions for generating output data 125 including a prototype list 121, a compatibility list 123, and a test schedule 124. It should be understood that “outputs” as used herein may refer to system generated, displayed, stored, and/or computed data, and does not necessarily, but may, refer to data that is displayed or otherwise provided to a user.
A prototype list 121, in one implementation referred to as a “vehicle list,” is a list of potentially buildable prototypes. Prototype list 121 may be generated from test calendar 112, e.g., for every prototype build indicated in test calendar 112, a prototype p is added to the prototype list 121. Each prototype in the prototype list 121 is associated with the build date that may be stored as an attribute of the prototype, e.g., one of the dates specified in the calendar 112; thus for each prototype in the prototype list 121, a capacity may be assigned and may further or alternatively be stored as an attribute of the prototype. In this context, “capacity” is defined as a number of days that the prototype is available for testing. Accordingly, prototype capacity, or at least a prototype's initial capacity, may be determined according to a number of days between a prototype's build date and the test end date included in the test calendar 112.
Output data 125 further includes a control model compatibility list 122. The list 122 may be a matrix, and in general describes general compatibility relationships between prototype control models. A prototype control model is simply a definition of a prototype having one or more specified attributes based on buildable configurations 113. For example, Table 3, in an example where prototype vehicles are to be tested, illustrates a subset of 12 of the n configurations in the tree 200 in
Accordingly, the control model compatibility list 122 is constructed such that two tests ti, tj are indicated to be compatible where the two tests could use a same prototype. As illustrated by
Although the control model compatibility list 122 is described herein as an output 125, it could alternatively be included in inputs 110. For example, a user could provide a table or other input identifying whether control models were compatible with one another.
Output data 125 further includes a test compatibility list 123. Whereas the control model compatibility list 122 provides a determination of whether tests are compatible in terms of prototype build requirements, the test compatibility list 123 requires as a threshold that tests are compatible in terms of prototype build requirements, e.g., that cti,tj, =1, but also considers other restrictions, such as may be imposed by test compatibility data 114, discussed above. For example, the compatibility list 123 may indicate whether, based on system inputs 110, including the test compatibility data 114, two tests ti, tj can be performed on a single physical prototype.
Note that test compatibility is unidirectional. That is, performing a test ti, and then a test tj on a particular prototype p, may be compatible, whereas performing the test ti and then the test tj is not. For example, a test tj may result in destruction of a prototype pi. In this case, the test tj could be performed after the test ti but not before it.
The compatibility list 123 is generally computed according to test request data 111, test calendar 112, test compatibility data 114, prototype list, and/or control model compatibility list 122.
Output data 125 generally further includes a test schedule 124. In general, a test schedule 124 specifies a schedule for building and testing prototypes, and is based on inputs 110, as well as prototype list 121 and compatibility list 123. That is, each record in a test schedule 124 generally corresponds to a particular prototype, e.g., a prototype vehicle. A portion of an exemplary test schedule 124, i.e., the first 10 days of the exemplary test schedule, is shown below in Table 5.
The leftmost column of Table 5 represents a “Build Finish Day,” sometimes referred to as a “Build Completion Day.” The Build Finish Day is a day in a testing period on which a prototype is completed and ready for assigned tests.
The next column, “ID numbers of tests assigned,” identifies, by a test identification number, the test or tests to be performed on a prototype vehicle completed on the Build Finish Day indicated in the leftmost column of table 5.
The next column, “number of days required per test ID,” identifies a number of calendar days required to complete the test or tests whose identification numbers are indicated.
The next column, “total number of days required,” represents a total number of calendar days required to perform the test or tests indicated in the “ID numbers of tests assigned” column. As can be seen, this column displays a sum of the numbers indicated in the previous column, i.e., a number of days required for each individual test is summed to provide a total number of days required for tests on a prototype vehicle.
The next column, “number of business days remaining,” indicates a number of business days remaining in a test schedule. In the example of Table 5, a 99 business day test schedule is planned, and therefore, on Build Finish Day 1, 98 business days remain. Note that, in some cases, the number of business days remaining is less than the total number of days required. In these cases, the next column, “number of unused business days,” is zero, and the following three columns, “number of Saturdays,” “number of Sundays,” and “number of holidays,” may have positive values, because it is necessary to perform tests on non-business days.
The last three columns in Table 5 describe a configuration of a prototype build used for the indicated tests. For example, for the tests represented in Table 5, a build configuration includes a vehicle body type, an engine type, and a transmission type.
Generally, in an exemplary implementation, to facilitate the process 300, the server 120 includes instructions for creating a test request object for each test of that has been requested, e.g., each record in test requests 111. The process 300 may also include creation of a data structure, such as a list, that includes each of these test objects, the list sometimes being referred to as a “request list.”
Next, in a step 310, the server 120 generates a compatibility list 123. An exemplary process for generating a compatibility list 123 is discussed in detail below with respect to
Next, in a step 315, any required or desired preprocessing may be performed. For example, one or more test objects from test requests 111 may be designated for exclusion from the scheduling processor for inclusion on a different schedule, or for some other reason may be removed from the request list. Similarly, the request list may be partitioned according to priorities, e.g., high, normal, low, etc., assigned to test objects based on designations in test requests 111. In other words, it may be desirable to generate sub-schedules for different partitions of the request list, e.g., tests of different priorities.
Next, in a step 320, a test schedule 124 is generated, e.g., as described below with respect to
Next, in a step 325, outputs, e.g., the test schedule 124, are displayed, e.g., on a screen of input device 105.
Next, in a step 330, the process 300 determines whether a request to modify inputs 110 has been received. For example, an interface may be provided on input device 105 for changing test requests 111 or other parameters. Thus, a test scheduling application may provide “what if” analysis and the like. If inputs 110 are to be modified, the process 300 returns to step 305. Otherwise, the process 300 ends.
The process 400 begins in a step 405, in which a test pair, e.g., tests ti, tj in a set of tests T, such as may be found in test requests 111, are selected for evaluation.
Next, in a step 410, control model compatibility list 122 is consulted to determine whether the two tests in the test pair are compatible in terms of prototype specifications, e.g., buildable configurations 113, such as those listed in Table 3. For example, it may be determined whether, for the test pair, the value cti,tj is zero or one. If the two tests are compatible, then step 415 is executed next. If not, the process 400 proceeds to step 430.
In a step 415, is determined whether the two tests ti, tj are of compatible durations and severity. Determining whether durations are compatible includes determining whether the time required for each test ti, tj will allow each test to be completed within the overall time allotted for testing according to the test calendar 112. For example, if the overall time allotted for testing is 100 days, and a first test requires 60 days, and a second test requires 50 days, then the two tests are not compatible from a timing perspective. Further, as discussed below with respect to
For two tests ti, tj to be of compatible severity means that severity levels of each test will allow both tests to be conducted on a single prototype. For example, as illustrated in Table 1, test requests 111 may specify a severity level for a test; in the example of Table 1, the “Destructive” variable is a binary variable indicating whether a vehicle is destroyed by a test. However, other severity levels are possible, e.g., “light,” “medium,” and “heavy.” In general, tests having greater severity levels cannot be conducted before tests having wider severity levels. Further, tests having certain severity levels, e.g., where Destructive=1, may foreclose any subsequent tests, regardless of the subsequent test's severity level, from being conducted. As noted above, test compatibility is generally not bidirectional at least in part because a test of destructive severity may be able to follow, but not precede, a test of lesser severity.
If the tests are not of both compatible durations and severity, then step 420 is executed next. If the two tests are of compatible durations and severity, then the process 400 proceeds to step 425.
In a step 420, which may be executed if there is the possibility of special factors existing that may allow the tests ti, tj to be compatible even when other determinations, e.g., the determinations of steps 410-415 have determined that the tests are not compatible, it is determined whether such special factors in fact exist with respect to the tests ti, tj, and, if so, whether the tests are in fact compatible. For example, if two tests have been determined to be of incompatible durations, it may be possible that at least portions of the tests can be conducted at the same time. Further, as may be indicated in test compatibility data 114, tests that are otherwise incompatible, e.g., that would otherwise have conflicting severity levels, may nonetheless be conducted on a same prototype. For example, two tests may have “Destructive” severity levels, yet be compatible. For example, a low-speed left-side front crash test may be performed on a vehicle prototype before a right-side pole crash test, although such tests could not be performed in a reverse order. In any case, the determination of step 420 may override determinations of steps 410-415 that would otherwise provide the result that two tests ti, tj are incompatible.
Step 425 is executed following either step 415 or step 420. In step 430, a value indicating whether the two tests ti, tj identified in step 405 are compatible, e.g., a binary variable ati,tj is set to 0 or 1. The value is then saved to the test compatibility list 123.
Next, in a step 430, it is determined whether any additional test pairs ti, tj remain to be evaluated. If so, the process 400 returns to step 405. Otherwise, the process 400 ends.
The process 500 begins in a step 505, in which test requests 111 are sorted in decreasing order of test durations, i.e., a list of test requests 111 begins with the longest test or tests and ends with the shortest test or tests.
Next, in a step 510, the prototype list 121 is sorted in decreasing order of prototype capacity, i.e., a prototype or prototypes having the most capacity are at the beginning of the resulting list of prototypes, which ends with a prototype or prototypes having the least capacity. Further, a prototype counter variable may be initialized; for example, as shown in
Next, in a step 515, a prototype p, is selected from the top of the sorted prototype list 121. Further, a test counter variable may be initialized; for example, as shown in
Next, in a step 520, the value j is incremented a test tj is selected from the top of the sorted test requests 111.
Next, in a step 525, it is determined whether tj has not already been assigned to a prototype and whether the selected prototype p, is able to accommodate the selected test tj, i.e., the prototype has adequate capacity for the test, any tests already assigned to the prototype are compatible with the candidate test, e.g., according to test compatibility list 123, and whether there exists a feasible sequence for performing all tests assigned to the vehicle, including the candidate test. That is, there must be time to perform all tests and all tests must be able to be performed in the assigned order. If the selected prototype is able to accommodate the selected test and the test has not already been assigned, then step 530 is executed next. Otherwise, step 535 is executed next.
In step 530, the test tj is assigned to the prototype p, in a data structure, sometimes referred to as a “prototype assignment list,” stored in a memory of the server 120.
In step 535, which may follow step 525, it is determined whether further tests in the sorted test requests 111 remain to be processed for the current prototype pi. If so, the process 500 returns to step 520. Otherwise, the process 500 proceeds to step 540.
In step 540, it is determined whether further prototypes remain to be processed in the sorted prototype list 121. If so, the process 500 proceeds to a step 550. Otherwise, step 555 is executed next.
Next, in step 550, the variable i is incremented. Then, the prototype assignment list is searched to determine whether there exists a second prototype with less capacity than the current prototype, but nonetheless with sufficient capacity for all tests assigned to the current prototype. If such second prototype is found, it is swapped in the prototype assignment list for the current prototype, and the current prototype once again becomes available in the prototype assignment list at position i. Thus, the system 100 includes mechanisms for minimizing the number of prototypes included in a test schedule. Following step 550, the process 500 returns to step 515.
In step 555, which may follow step 540, a test schedule 124 is generated. Table 5, as discussed above, provides an example of a test schedule 124, providing a schedule for testing vehicle prototypes. With reference to the process 500, fields included in Table 5 may be generated as follows.
“Build Finish Day,” as discussed above, a day in a testing period on which a vehicle prototype is completed and ready for assigned tests. The Build Finish Day is an attribute of a vehicle prototype, generally used for determining prototype capacity discussed above. A Build Finish Day may accordingly be included as output in the test schedule 124.
“ID numbers of tests assigned” includes identifiers for tests assigned to a prototype vehicle represented by a record in the test schedule 124, e.g., tests assigned as described above with respects to steps 530 and/or 550 in the process 500.
“Number of days required per test ID” may be populated by the process 500 reproducing a number of days required to complete the test or tests identified in the “ID numbers of tests assigned” column.
“Total number of days required,” as discussed above, may be computed by the process 500 as a sum of the numbers indicated in the previous column.
“Number of business days remaining,” may be computed by the process 500 as a number of business days remaining in a test schedule, and “number of unused business days” may be computed by subtracting from the total number of days in a test schedule the number of business days remaining.
Where the number of unused business days is less than zero, the process 500 may include rules for assigning tests to Saturdays, Sundays, and/or holidays, in which case, as discussed above, “number of Saturdays,” “number of Sundays,” and “number of holidays,” may have positive values, because it is necessary to perform tests on non-business days.
Remaining fields in a test schedule 124 may be populated by the process 500 to include, as discussed above, attributes of a prototype build used for the indicated tests.
Following step 555, the process 500 ends.
It is to be understood that alternatives to the process 500 could be used to perform test scheduling. In general, the test scheduling problem is an optimization that seeks to minimize a number of prototypes needed to complete a test schedule 124. This optimization could be performed in a number of different ways. For example, although computationally intractable in certain cases, in some cases this optimization could be solved as an integer programming problem. An exemplary integer programming problem is as follows.
The exemplary integer programming problem, assuming that testing of prototype vehicles is being scheduled, has the following parameters:
V is the set of potentially buildable prototype vehicles;
v is a single prototype vehicle;
fv is the number of days that a prototype vehicle v is available for tests;
T is the set of tests requested;
t is a single test;
ct1,t2,v is a binary variable indicating whether tests t1 and t2 are compatible on vehicle v (1 indicating compatibility, and 0 indicating incompatibility);
dt is the number of days required to perform test t.
Decision variables, to which values may be assigned for determining a test schedule 124, are defined as follows:
xt,v is a binary variable indicating whether a test t is assigned to a prototype vehicle v (1 indicating that the test is assigned, and 0 indicating that it is not);
zv is a binary variable indicating whether a prototype vehicle v is used (1 indicating that the vehicle is used; 0 indicating that it is not used).
Accordingly, the integer programming problem may present the following objective function:
subject to satisfying the following constraints:
each test t must be assigned to exactly one prototype vehicle, i.e.,
a prototype vehicle must be built if a test is assigned to it, i.e.,
z
v
≧x
t,v
,∀t εT,vεV;
incompatible tests cannot be assigned to the same prototype vehicle, i.e.,
x
t
,v
+x
t
,v≦1,∀(t1,t2)εT×T,vεV:t1≠t2,ct
when a test is assigned to a prototype vehicle, sufficient testing days must be available with the vehicle, i.e.,
and all binary variables must take either the value 0 or the value 1:
x
t,vε{0,1},∀tεT,vεV.
Further, the foregoing objective function could be modified to consider other goals, e.g., minimizing overtime costs, schedule changes, or the required number of test days subject to a predetermined number of prototypes being available. Moreover, other constraints could be imposed, for example, precedence constraints between tests to ensure that one test is performed before another, or to prevent multiple destructive crash tests from being assigned to a same prototype vehicle. In any case, the solution to the above objective function would provide sets of values zv and xt,v, which could then be used to identify prototype vehicles to be used in respective tests, and to generate a test schedule 124.
Computing devices such as those discussed herein generally each include instructions executable by one or more computing devices such as those listed above. Computer-executable instructions may be compiled or interpreted from computer programs created using a variety of programming languages and/or technologies, including, without limitation, and either alone or in combination, Java™, C, C++, Visual Basic, Java Script, Perl, HTML, etc. In general, a processor (e.g., a microprocessor) receives instructions, e.g., from a memory, a computer-readable medium, etc., and executes these instructions, thereby performing one or more processes, including one or more of the processes described herein. Such instructions and other data may be stored and transmitted using a variety of computer-readable media. A file in a computing device is generally a collection of data stored on a computer readable medium, such as a storage medium, a random access memory, etc.
A computer-readable medium includes any medium that participates in providing data (e.g., instructions), which may be read by a computer. Such a medium may take many forms, including, but not limited to, non-volatile media, volatile media, etc. Non-volatile media include, for example, optical or magnetic disks and other persistent memory. Volatile media include dynamic random access memory (DRAM), which typically constitutes a main memory. Common forms of computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, any other magnetic medium, a CD-ROM, DVD, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, a RAM, a PROM, an EPROM, a FLASH-EEPROM, any other memory chip or cartridge, or any other medium from which a computer can read.
With regard to the media, processes, systems, methods, etc. described herein, it should be understood that, although the steps of such processes, etc. have been described as occurring according to a certain ordered sequence, such processes could be practiced with the described steps performed in an order other than the order described herein. It further should be understood that certain steps could be performed simultaneously, that other steps could be added, or that certain steps described herein could be omitted. In other words, the descriptions of processes herein are provided for the purpose of illustrating certain embodiments, and should in no way be construed so as to limit the claimed invention.
Accordingly, it is to be understood that the above description is intended to be illustrative and not restrictive. Many embodiments and applications other than the examples provided would be apparent to those of skill in the art upon reading the above description. The scope of the invention should be determined, not with reference to the above description, but should instead be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. It is anticipated and intended that future developments will occur in the arts discussed herein, and that the disclosed systems and methods will be incorporated into such future embodiments. In sum, it should be understood that the invention is capable of modification and variation and is limited only by the following claims.
All terms used in the claims are intended to be given their broadest reasonable constructions and their ordinary meanings as understood by those skilled in the art unless an explicit indication to the contrary in made herein. In particular, use of the singular articles such as “a,” “the,” “said,” etc. should be read to recite one or more of the indicated elements unless a claim recites an explicit limitation to the contrary.
