The present disclosure relates to an answer evaluation method, a recording medium and an information processing apparatus.
In recent years, the introduction of ICT (Information and Communication Technology) devices into school education is progressing. Along with this, since chances to acquire, as electronic information significantly increases, answers, which are grading targets for questions, it is expected that automatic grading technology using a computer will be promoted and burdens on teachers will be reduced.
In a CAD examination grading system disclosed in JP-A-2006-251203, when an answer of an examinee exactly matches a model answer prepared in advance, the answer is treated as a correct answer.
In the CAD examination grading system disclosed in JP-A-2006-251203, a partial point can be given by further changing a grading item, which is managed as a setting of the system, to set an item desired by a grader to a grading target. However, since a content of a grading program is fixed, the grader cannot flexibly change a point allocation method for each grading item.
PTL 1: JP-A-2006-251203
An answer evaluation method according to one embodiment of the present invention is an answer evaluation method that is executed by an answer evaluating system, and including acquiring information of an answer that is an evaluation target corresponding to a question, and information of a comprehensive evaluation method, which is a method of determining a comprehensive evaluation of the answer based on an evaluation on the answer with respect to one evaluation item or each of two or more evaluation items for evaluating the answer represented by the information on the answer; and outputting information representing the comprehensive evaluation of the answer and acquired based on the answer represented by the acquired information of the answer and the comprehensive evaluation method represented by the acquired information of the comprehensive evaluation method.
A recording medium according to one embodiment of the present invention is a non-transitory computer-readable recording medium having a program recorded thereon that can be executed by at least one processor of an information processing apparatus, the processor being configured to acquire information of an answer that is an evaluation target corresponding to a question, and information of a comprehensive evaluation method, which is a method of determining a comprehensive evaluation of the answer based on an evaluation on the answer with respect to one evaluation item or each of two or more evaluation items for evaluating the answer represented by the information of the answer; and to output information representing the comprehensive evaluation of the answer and acquired based on the answer represented by the acquired information of the answer and the comprehensive evaluation method represented by the acquired information of the comprehensive evaluation method.
An information processing apparatus according to one embodiment of the present invention includes at least one processor configured to execute a program stored in a storage unit, the processor being configured to cause an acquisition unit to acquire information of an answer that is an evaluation target corresponding to a question, and information of a comprehensive evaluation method, which is a method of determining a comprehensive evaluation of the answer based on an evaluation on the answer with respect to one evaluation item or each of two or more evaluation items for evaluating the answer represented by the information of the answer; and to cause an output unit to output information representing the comprehensive evaluation of the answer and acquired based on the answer represented by the acquired information of the answer and the comprehensive evaluation method represented by the acquired information of the comprehensive evaluation method.
The system 1 is an automatic evaluation system configured to output an evaluation result, in response to an evaluation request for requesting an evaluation of an answer, and more specifically, is an automatic grading system configured to output a grading result, in response to a grading request, for example. A field (subject) of questions pertaining to the evaluation request is not particularly limited. Mathematics, Japanese, a foreign language, science (physics, chemistry, biology), society (history, geography, civics) and the like may be possible, and questions from other specialized subjects may also be possible. In addition, as for a question format, a descriptive question other than a multiple-choice question is desirable. In the below, the configuration and operation of the system 1 are described by taking as an example a case of grading an answer including a mathematical expression in the question field such as mathematics.
As shown in
The client terminal 10 is a terminal having a display device, and is operated by a user of the system 1. The user of the system 1, i.e., a user of the client terminal 10 is not particularly limited. An evaluator who is the user is typically a grader such as a school teacher. However, an answerer himself/herself such as a school student may also use the system 1. The client terminal 10 is configured to transmit an evaluation request for requesting an evaluation of an answer, which is an evaluation target corresponding to a question, for example, a grading request for requesting a grading of an answer, which is a grading target for a question, to the server apparatus 20, in response to a user input, for example. Note that, the grading request is an example of the evaluation request. The evaluation request includes, for example, information (answer information) on an answer of the answerer to the question, information (model answer information) on a model answer to the question, information (evaluation item information) on one or more evaluation items, and information (comprehensive evaluation method information) on a comprehensive evaluation method that is a method of determining a comprehensive evaluation of the answer but may include at least the answer information and the comprehensive evaluation method information. Hereinafter, in the present specification, the answer that is an evaluation target generated by the answerer such as an examinee is simply referred to as ‘answer’ and is distinguished from ‘model answer’ that is a correct answer to the question.
Note that, the answer information may be an answer itself or may also be information for acquiring the answer. For example, the answer information may be information converted so that an answer can be restored or may also be information indicating whereabouts of the answer. The model answer information may be a model answer itself or may also be information for acquiring the model answer. For example, the model answer information may be information converted so that a model answer can be restored or may also be information indicating whereabouts of the model answer. In addition, the evaluation item is a determination element at the time when evaluating an answer, and may be a determination element for adding or deducting a point for grading, for example. The evaluation item information may be an evaluation item itself or may also be information for acquiring the evaluation item. For example, the evaluation item information may be information converted so that an evaluation item can be restored or may also be information indicating whereabouts of the evaluation item. The comprehensive evaluation method is information for specifying a method of determining a comprehensive evaluation of an answer, and is, for example, a grading logic. The comprehensive evaluation of an answer is determined based on an evaluation on an answer with respect to one evaluation item or each of two or more evaluation items for evaluating the answer represented by the answer information. The comprehensive evaluation method information may be a comprehensive evaluation method itself or may also be information for acquiring the comprehensive evaluation method. For example, the comprehensive evaluation method information may be information converted so that a comprehensive evaluation method can be restored or may also be information indicating whereabouts of the comprehensive evaluation method. In addition, the grading logic is a logic for grading an answer based on an evaluation on an answer with respect to one evaluation item or each of two or more evaluation items of the answer. The grading logic information may be a grading logic itself or may also be information for acquiring the grading logic. For example, the grading logic information may be information converted so that a grading logic can be restored or may also be information indicating whereabouts of the grading logic.
As shown in
Although not specifically limited, the client terminal 10 includes, for example, a processor 11, a storage device 12, an input device 13, a display device 14, and a communication device 15, as shown in
The processor 11 is, for example, hardware including a CPU (Central Processing Unit) and the like, and is configured to execute a program 12a stored in the storage device 12. Note that, the processor 11 may also include any electric circuit such as a GPU (Graphics processing unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), and a DSP (Digital Signal Processor).
The storage device 12 is, for example, any semiconductor memory, and includes a volatile memory such as a RAM (Random Access Memory) and a non-volatile memory such as a ROM (Read Only Memory) and a flash memory. In addition, the storage device 12 may also include a magnetic storage device, an optic storage device, and other types of storage devices. In the storage device 12, a program 12a that is executed by the processor 11 is stored. Note that, the program 12a stored in the storage device 12 may be one downloaded from the server apparatus 20 via the network 30 and the communication device 15, for example. The storage device 12 may include a non-transitory computer-readable recording medium having a program recorded thereon that can be executed by the processor.
The input device 13 includes, for example, a keyboard, a mouse, a touch panel and the like but may also include a voice input device such as a microphone, and other types of input devices. The display device 14 is, for example, a liquid crystal monitor, an organic EL display, a plasma display, a CRT display or the like but may also be another type of a display device such as a matrix LED panel. The communication device 15 is, for example, a wireless communication device such as a Wi-Fi (registered trademark) module but may also be a wired communication device.
The server apparatus 20 is an information processing apparatus configured to process an evaluation request and to output information representing a comprehensive evaluation of an answer, and may also be, for example, an information processing apparatus configured to process a grading request and to output a grading result. The information (hereinafter, referred to as ‘evaluation result’) representing a comprehensive evaluation may be output in the server apparatus 20 or may be output to an apparatus different from the server apparatus 20. Note that, an output destination of the information (hereinafter, referred to as ‘evaluation result’) representing a comprehensive evaluation is not limited to this form. An output of the evaluation result may be, for example, generation of a file including evaluation result information, registration of the evaluation result information in a database, and the like. Note that, the server apparatus 20 may be configured to transmit or not to transmit the evaluation result to the client terminal 10 that is a transmission source of the evaluation request.
The evaluation result that is output by the server apparatus 20 includes at least a comprehensive evaluation of an answer. In addition, the evaluation result may also include an evaluation on an answer with respect to one evaluation item or each of two or more evaluation items included in the evaluation request. The comprehensive evaluation and the evaluation may be arithmetically operable scores such as 0 point, 1 point and 2 points, respectively. In addition, the comprehensive evaluation and the evaluation may also be relative evaluations such as good, normal and bad, respectively. Further, one of the comprehensive evaluation and the evaluation may be a score and the other may be a relative evaluation.
The server apparatus 20 may be configured as a single apparatus, or may be a set of a plurality of apparatuses including a Web server apparatus, an application server apparatus, a database server apparatus and the like. In addition, the server apparatus 20 may also be configured as a distributed computing system.
The server apparatus 20 includes, for example, a processor 21, a storage device 22, an input device 23, a display device 24, and a communication device 25, as shown in
The processor 21 is, for example, hardware including a CPU (Central Processing Unit) and the like, and is configured to execute a program 22a and a program 22b stored in the storage device 22. Note that, the processor 21 may also include any electric circuit such as a GPU (Graphics processing unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), and a DSP (Digital Signal Processor).
The storage device 22 is, for example, any semiconductor memory, and includes a volatile memory such as a RAM (Random Access Memory) and a non-volatile memory such as a ROM (Read Only Memory) and a flash memory. In addition, the storage device 22 may also include a magnetic storage device, an optic storage device, and other types of storage devices. In the storage device 22, a program 22a and a program 22b that are executed by the processor 21 are stored. Note that, the program 22a is an automatic evaluation program configured to execute automatic evaluation processing, in response to an evaluation request, for example. In addition, the program 22b is, for example, a program that is called from a variety of programs including an automatic evaluation program, and is executed so as to perform processing that is commonly used in a variety of programs, such as function processing (which will be described later) that is used in the automatic evaluation processing.
Further, the storage device 22 may store the program 12a that is distributed to the client terminal 10 and is executed on the client terminal 10. Further, the server apparatus 20 may also be configured to distribute the program 22a to the client terminal 10, and the client terminal 10 may also be configured to execute the program 22a received from the server apparatus 20, thereby the automatic evaluation processing, which will be described later.
That is, the server apparatus 20 may also be a program distribution server configured to distribute an automatic evaluation program. Further, the server apparatus 20 may also be configured to distribute the program 22b to the client terminal 10, in addition to the program 22a.
The input device 23 includes, for example, a keyboard, a mouse, a touch panel and the like but may also include a voice input device such as a microphone, and other types of input devices. The display device 24 is, for example, a liquid crystal monitor, an organic EL display, a plasma display, a CRT display or the like but may also be another type of a display device such as a matrix LED panel. The communication device 25 may be a wireless communication device or a wired communication device.
In the system 1 configured as described above, the server apparatus 20 is configured to automatically perform evaluation processing and to output an evaluation result, in response to an evaluation request transmitted from the client terminal 10. Therefore, according to the system 1, since the evaluator such as a grader does not have to manually grade and evaluate an answer, it is possible to considerably reduce a burden of an answer evaluation operation (grading operation).
In addition, the evaluation request includes the comprehensive evaluation method information (and the evaluation item information) together with the answer information (and the model answer information). Therefore, even when the system 1 is shared by many users (graders, evaluators), it is possible to securely designate the comprehensive evaluation method such as a grading logic that is used for grading of the answer. Such features of the system 1 are largely different from a system of the related art configured to operate by a fixed grading logic. This allows the system 1 to flexibly respond to requests of evaluators who have different minds about evaluation. In addition, since it is possible to securely designate the comprehensive evaluation method that is used for evaluation of an answer, it is possible to freely select a timing of evaluation processing with respect to a timing of the evaluation request. For this reason, under the environment where the system 1 is shared, for example, it becomes easy to collectively grade evaluation requests received during the day by batch processing at night.
In the system 1, the comprehensive evaluation method may be made different for each answer. For this reason, even when the system is used by the same evaluator, it is possible to make an evaluation criterion different, depending on a level of an answerer, for example. Therefore, according to the system 1, it is possible to perform detailed evaluation, according to an educational concept of the evaluator.
A user who uses an automatic grading function provided by the system 1 first activates a Web browser installed in the client terminal 10 and inputs a predetermined URL. Thereby, the client terminal 10 requests the server apparatus 20 to transmit display data for displaying a grading screen (step S1 in
As shown in
When the grading screen is displayed on the client terminal 10, the user inputs information necessary to execute automatic grading processing, on the grading screen. As a result, the client terminal 10 acquires the information input by the user (step S3 in
The grading logic 141 is a logic of setting a comprehensive evaluation to 1 point or larger when an answer and a model answer are mathematically equivalent, and setting a comprehensive evaluation to 0 point when an answer and a model answer are not mathematically equivalent. More specifically, in the grading logic 141, when an answer and a model answer are mathematically equivalent, a fraction is not included in the answer and the answer is appropriately expanded, 5 points that are the highest point are given. On the other hand, even when an answer and a model answer are mathematically equivalent, if a fraction is included in the answer, 1 point is deducted, and if the answer is not appropriately expanded, 3 points are deducted. In this way, the grading logic 141 includes point allocation information (in this example, 5 points, −1 point, −3 points) allotted to one evaluation item or each of two or more evaluation items. As shown in
Note that, the grading logic 141 that is input to the grading logic field 140 is not limited to the above example, and can be freely described by the grader. Further, a logic using functions other than the above-described three functions may also be described in the grading logic field 140. For example, as shown in
The function isEquationForm( ) is a function for determining whether an answer satisfies a format designated together with the answer. The function Factorized( ) is a function for determining whether an answer is described in a factorized format. The function Simplified( ) is a function for determining whether an answer is described in an organized format that has been reduced, rationalized or the like.
When the user inputs information to the grading screen and pushes a button 150, the client terminal 10 transmits, to the server apparatus 20, a grading request generated based on the information input to the grading screen (step S4 in
When the grading request is transmitted from the client terminal 10, the server apparatus 20 performs automatic grading processing shown in
Note that, the automatic grading processing shown in
When the automatic grading processing is started, the processor 21 first acquires the answer information and the grading logic information (step S10). That is, the processor 21 is an example of the acquisition unit configured to acquire the answer information and the grading logic information. Note that, in step S10, at least the answer information and the grading logic information may be acquired. However, in the below, an example where the model answer information and one or more evaluation item information are acquired, in addition to these information, is described.
In step S10, the processor 21 first receives the grading request. That is, the processor 21 collectively receives the answer information 210, the model answer information 220, the evaluation item information 230 and the grading logic information 240. This allows the server apparatus 20 (processor 21) to recognize that the answer information 210, the model answer information 220, the evaluation item information 230 and the grading logic information 240 are associated with each other. In step S10, at least the answer information 210 and the grading logic information 240 may be collectively received. This processing is an example of the processing in which the server apparatus 20 acquires the information of the grading logic, in response to the reception of the input to the grading screen in step S3 on the client terminal 10 receiving the display data of the grading screen. Then, the processor 21 analyzes the grading request to extract the answer information, the model answer information, one or more evaluation item information and the grading logic information. In addition, the processor 21 specifies the answer, the model answer information, one or more evaluation items and the grading logic, based on the answer information, the model answer information, one or more evaluation item information and the grading logic information.
Next, the processor 21 acquires an evaluation on an answer with respect to one evaluation item or each of two or more evaluation items of the answer (step S20). That is, the processor 21 is an example of the second acquisition unit configured to acquire each evaluation. In step S20, the processor 21 acquires an evaluation with respect to one evaluation item or each of two or more evaluation items of the answer, based on the answer and the one or more evaluation items specified from the grading request in step S10. Note that, the processor 21 may also acquire an evaluation with respect to each of one or more evaluation items of the answer, based on the answer, the model answer and the one or more evaluation items. Specifically, the processor 21 acquires an evaluation with respect to an evaluation item from a common module by calling a function corresponding to the evaluation item provided by the common module (program 22b) from the automatic grading program (program 22a) under execution. More specifically, the processor 21 acquires an evaluation as to whether the answer and the model answer are mathematically equivalent, in a form of being stored in equiv, by designating the answer and the model answer as arguments and calling the function mathEquiv( ). In addition, the processor 21 acquires an evaluation as to whether a fraction is included in the answer, in a form of being stored in the variable hasFrac, by designating the answer as an argument and calling the function hasFraction( ). Further, the processor 21 acquires an evaluation as to whether the answer has an expanded format, in a form of being stored in the variable isExp, by designating the answer as an argument and calling the function isExpanded( ).
At the end of the automatic grading processing, the processor 21 outputs a grading result, which is information (evaluation result) representing a comprehensive evaluation of the answer (step S30). That is, the processor 21 is an example of the output unit configured to output a grading result. In step S30, the processor 21 outputs the grading result, based on the grading logic and the answer. Specifically, the processor outputs the grading result, based on the grading logic acquired in step S10 and the evaluation acquired in step S20 by using the answer and the evaluation item. More specifically, the processor 21 calculates a score, which is a comprehensive evaluation, by executing the grading logic by using the evaluations stored in the variables, and outputs a grading result including the score. In this way, by calculating the score by combining the respective evaluations of the evaluation items and the grading logic, a flexible grading corresponding to the answer is possible, as shown in a table T1 of
The grading result output in step S30 may be output to a file or may also be output and registered in a database.
When the automatic grading processing shown in
As described above, according to the automatic grading method, the automatic grading program that is executed on the server apparatus 20, and the server apparatus 20 described in the present embodiment, the grading logic information (and the evaluation item information) is acquired, in addition to the answer information (and the model answer information), and the grading result generated based on these information is output. This makes it possible to implement the automatic grading flexibly corresponding to the request of the graders who have different minds about evaluation. More specifically, by calculating the comprehensive evaluation as a score by using the grading logic including one or more evaluation items and the point allocation information allotted to one evaluation item or each of two or more evaluation items, it is possible to easily perform the grading corresponding to the answer.
Note that, in the present embodiment, the example has been shown in which one or more evaluation items included in the grading request include a first item (for example, the function mathEquiv( ) for defining, as an evaluation item, whether or not to satisfy the mathematical equivalence between the answer and the model answer and one or more second items (for example, the function hasFraction( ) the function isExpanded( ) and the like) for defining, as an evaluation item, whether or not to satisfy one factor or each of two or more factors that deny an expressive sameness between the answer and the model answer and the grading logic includes a logic of adjusting the comprehensive evaluation (deducting a point) according to the evaluation of one second item or each of two or more second items when the evaluation on the first item is positive. According to this example, even though the answer and the model answer do not exactly match in terms of expression, when they have a mathematically equivalent relationship, it is possible to easily perform grading of giving a certain degree of point allocation. In addition, even an answer having a mathematically equivalent relationship with a model answer can be graded with a different score, depending on the factor that denies the sameness. Note that, it can be said that the second item is an item for defining, as an evaluation item, a condition that is satisfied when an answer is mathematically equivalent but is different in terms of mathematical representation.
However, the evaluation items and the configuration of the grading logic are not limited to the above-described example. For example, one or more evaluation items included in the grading request may also include one or more third items for defining, as an evaluation item, whether or not to satisfy one factor or each of two or more factors that affirm similarity between the answer and the model answer, in addition to the above-described first item and second item. In addition, the grading logic included in the grading request may also include a logic for adjusting the comprehensive evaluation (adding a point) according to an evaluation of one third item or each of two or more third items when an evaluation on the first item is negative. Note that, it can be said that the third item is an item for defining, as an evaluation item, a condition that is satisfied when an answer is not mathematically equivalent but partially matches in terms of mathematical representation.
Note that, in a function list that can be selected from a list box 133 provided in an evaluation item field 130 shown in
Note that, a function mathEquivWithCoordinateCorrect( ) that is included in the function group 137 is a function for determining the number of coordinate values that match a value of a model answer when an answer has a coordinate format consisting of one or more coordinate values. In addition, a function isInequalityMiss( ) is a function for determining whether a reason for lack of mathematical equivalence between an answer and a model answer is only the presence or absence of an equal sign in an inequality. A function isSignMiss( ) is a function for determining whether a reason for lack of mathematical equivalence between an answer and a model answer is only a sign. Further, a function isAngleMiss( ) is a function for determining whether a reason for lack of mathematical equivalence between an answer and a model answer is only the presence or absence of an angle symbol such as “° ” and “π”.
As shown in
In addition, the system 1 may also be configured to provide the user with a function of adjusting a criterion for determining whether or not a mathematical equivalence. For example, an operation of the function mathEquiv( ) for determining whether an answer and a model answer are mathematically equivalent may be adjusted by changing an option setting.
Adjusting the operation of the function mathEquiv( ) in this way allows more flexible grading at the discretion of the grader.
The processing of evaluating the similarity between the answer and the model answer described using
The above-described embodiment shows a specific example so as to easily understand the invention, and the present invention is not limited to the embodiment and should be understood to include a variety of modifications and alternations of the above-described embodiment. For example, it should be understood that each embodiment can be embodied by modifying the constitutional elements without departing from the gist and scope thereof. In addition, it should be understood that various embodiments can be implemented by appropriately combining the plurality of constitutional elements disclosed in the above-described embodiment. Further, one skilled in the art should understand that various embodiments can be implemented by omitting some constitutional elements from all the constitutional elements shown in the embodiment or adding some constitutional elements to the constitutional elements shown in the embodiment. That is, the answer evaluation method, the program, the program transmission server, the information processing apparatus and the answer evaluation system can be diversely modified and changed without departing from the definitions of the claims.
In the above-described embodiment, the example where the server apparatus 20 acquires the answer information 210, the model answer information 220, the evaluation item information 230 and the grading logic information 240 has been shown. However, as for predetermined information among these information, the information may be prepared in advance and may not be newly acquired. For example, if the model answer and the evaluation item may be fixed, the server apparatus 20 may acquire the answer information 210 and the grading logic information 240. By acquiring at least the answer information 210 and the grading logic information 240, it is possible to allow for flexible grading because the grader can freely change the grading logic.
Further, in the above-described embodiment, the example where the server apparatus 20 collectively acquires the answer information 210, the model answer information 220, the evaluation item information 230 and the grading logic information 240 has been shown. However, these information that is used for grading may be just acquired before the grading, and is not necessarily required to be collectively acquired. For example, as shown in FIG. 16, these information may be acquired from a plurality of client terminals 10 (a client terminal 10a and a client terminal 10b). For example, as shown in
Note that, the description “a plurality of information is individually acquired” means not receiving collectively a plurality of information, i.e., receiving any one or more information and other one or more information among the plurality of information at time intervals, receiving any one or more information of the plurality of information and other one or more information from a plurality of different apparatuses or acquiring the plurality of information by a combined method thereof. In the below, descriptions are given according to an actual example.
Further, as shown in
In the above, the case where the answer including a mathematical representation is graded has been described as an example. However, the field of the present application is not limited to the mathematics and the like and can also be applied to grading of English words, Chinese letters and other subjects.
Specifically, a case where an answer to a following question is graded is described as an example.
<Question>
Please answer an English word that applies to the parenthesis( ) in the following sentence.
Japanese: I assess a new technology (in Japanese).
English: I( ) a new technology.
First, in response to a request from the client terminal 10, the server apparatus 20 transmits display data for displaying the grading screen 100 (step S1 and step S2 in
The grading logic 145 is a logic of setting a comprehensive evaluation to 2 points or larger when an answer and a model answer are semantically similar, and setting a comprehensive evaluation to 0 point or 1 point when an answer and a model answer are not semantically similar. More specifically, in the grading logic 145, when an answer and a model answer are semantically similar, an upper case is not included in the answer and a type of the answer is the present tense of the first person, 5 points that are the highest point are given. In addition, even when an answer and a model answer are semantically similar, if an upper case is included in the answer, 1 point is deducted, and if a type of the answer is not the present tense of the first person, 2 points are deducted. On the other hand, when an answer and a model answer are not semantically similar, if a difference in letter between the answer and the model answer is within one letter, 1 point is given, and if the difference is two or more letters, 0 point is given.
In this way, the grading logic 145 includes point allocation information (in this example, 5 points, −1 point, −2 points) allotted to one evaluation item or each of two or more evaluation items. As shown in
Thereafter, when the user inputs information to the grading screen and pushes the button 150, the client terminal 10 transmits, to the server apparatus 20, a grading request generated based on the information input to the grading screen (step S4 in
When the grading request is transmitted from the client terminal 10, the server apparatus 20 performs automatic grading processing shown in
When the automatic grading processing is started, the processor 21 first acquires the answer information and the grading logic information (step S10), and also acquires an evaluation on an answer with respect to one evaluation item or each of two or more evaluation items of the answer (step S20). In step S20, the processor 21 acquires an evaluation with respect to one evaluation item or each of two or more evaluation items of the answer, based on the answer and the one or more evaluation items specified from the grading request in step S10. The processing of step S10 is similar to the case of the mathematics question. The processing of step S20 is also similar to the case of the mathematics question, except that the different functions are used.
The function isSameConjuation( ) which is one of the functions used in step S20, is different from the other functions, in that it accesses the word database 22c shown in
In the word database 22c, data of words is registered. Specifically, for example, for each word, id information for identifying the word, type information for identifying a part of speech of the word, and information predetermined for each part of speech of the word are registered. For example, in a case where the part of speech is a verb, information such as spelling for each type (the present tense of the first person, the present tense of the third person, the past tense, the present participle, the past participle and the prototype) is included, as shown with data D1 and D2 in
In the processing of the function isSameConjuation( ) the processor 21 acquires word data (for example, D1, D2) corresponding to a word of the model answer from the word database 22c, and compares a spelling of a type designated with an argument specified from the word data, and the answer. As a result of the comparison, when both match, True is returned, and when both do not match, False is returned.
At the end of the automatic grading processing, the processor 21 outputs a grading result, which is information (evaluation result) representing a comprehensive evaluation of the answer (step S30). Here, the processor 21 calculates a score, which is a comprehensive evaluation, by executing the grading logic by using the evaluations stored in the variables, and outputs a grading result including the score. In this way, by calculating the score by combining the respective evaluations of the evaluation items and the grading logic, a flexible grading corresponding to the answer is possible, as shown in a table T2 of
When the automatic grading processing shown in
In this way, the above-described automatic grading processing is not limited to the mathematics question, can be applied to a fill-in-blank question of a foreign language such as English, and can flexibly respond to the request of the graders who have different minds about grading.
Note that, in the present embodiment, the example has been shown in which one or more evaluation items included in the grading request include a first item (for example, the function match( ) for defining, as an evaluation item, whether or not to satisfy the semantical similarity between the answer and the model answer and one or more second items (for example, the function hasUpperCase( ) the function isSameConjuation( ) and the like) for defining, as an evaluation item, whether or not to satisfy one factor or each of two or more factors that deny an expressive sameness between the answer and the model answer and the grading logic includes a logic of adjusting the comprehensive evaluation (deducting a point) according to the evaluation of one second item or each of two or more second items when the evaluation on the first item is positive. According to this example, even though the answer and the model answer do not exactly match in terms of expression, when they have a semantically similar relationship, it is possible to easily perform grading of giving a certain degree of point allocation. In addition, even an answer having a semantically similar relationship with a model answer can be graded with a different score, depending on the factor that denies the sameness.
However, the evaluation items and the configuration of the grading logic are not limited to the above-described example. For example, one or more evaluation items included in the grading request may also include one or more third items for defining, as an evaluation item, whether or not to satisfy one factor or each of two or more factors that affirm expressive similarity between the answer and the model answer, in addition to the above-described first item and second item. In addition, the grading logic included in the grading request may also include a logic for adjusting the comprehensive evaluation (adding a point) according to an evaluation of one third item or each of two or more third items when an evaluation on the first item is negative.
Note that, in the grading logic 145 described in the grading logic field 140 shown in
Further,
In the above-described embodiments, the example where the system 1 includes the client terminal 10 and the server apparatus 20 has been shown. However, in the system 1, the server apparatus 20 may also serve as the client terminal 10. That is, the grading request may be generated using the server apparatus 20, and the server apparatus 20 itself may be configured to process the generated grading request to output the grading result. In addition, in the system 1, the client terminal 10 may also serve as the server apparatus 20. That is, the grading result may be output with the single body of the client terminal 10.
Further, in the above-described embodiments, the example where the server apparatus 20 itself performs the function processing by calling the common module (program 22b) from the automatic grading program (program 22a) has been shown. However, the relationship between the automatic grading processing and the function processing is not limited to this example. The function processing may also be performed in an apparatus different from the server apparatus 20, for example. In addition, the automatic grading processing and the function processing may be implemented in the same program.
Further, in the above-described embodiments, the example where the highest points are once given to the answer determined as being mathematically equivalent and the final score is calculated by deducting the point due to the relationship with the factors that deny the expressive sameness has been shown. However, the method of calculating the score is not limited to the point deducting method. For example, a predetermined point may be once given to an answer determined as being mathematically equivalent and a final score may be calculated by a point adding method of adding a point according to conditions.
Further, in the above-described embodiments, the example where the evaluation item and the grading logic are individually designated has been shown. However, the method of designating the evaluation item and the grading logic is not limited to this example. The server apparatus 20 may also be configured to store the evaluation item and the grading logic in advance in association with attributes of a question master (for example, a country, a school, a subject in charge, a teacher in charge, and the like), or when the user (for example, a grader or an answerer) designates the attributes of a question master, the server apparatus 20 may read out and use the evaluation item and the grading logic.
Further, in the above-described embodiments, the example where the client is caused to display the dedicated application screen to edit the comprehensive evaluation method information has been shown. However, the interface for editing the comprehensive evaluation method information is not limited to the dedicated application screen. For example, the editing can be made by a command prompt or other interface.
The present application is based on Japanese Patent Application Nos. 2020-113100 filed on Jun. 30, 2020 and 2021-078509 filed on May 6, 2021, the contents of which are incorporated herein by reference.
Number | Date | Country | Kind |
---|---|---|---|
2020-113100 | Jun 2020 | JP | national |
2021-078509 | May 2021 | JP | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/JP2021/022943 | 6/16/2021 | WO |