Education testing and skills assessment is an important aspect of childhood education. In the context of reading, the de facto method to assess a student's reading ability is the running record. A conventional running record has been a paper and pencil work flow method for the teacher, involving a specialized shorthand notation. The teacher uses a paper-based worksheet to mark data while a student is performing an assessment thus creating a “running record” of the child's reading performance. The running record tallies the student's accuracy in reading and the errors made.
While the teacher gains tacit knowledge about the student when capturing a running record, the process does not easily capture the breadth of possible data. It is only a paper record of the teacher's hand written markings and notes that they can make in real time, and it can vary based on the method used or the purpose for which it is gathered.
This document describes improved methods for gathering much more data during an assessment of a student's reading ability.
A student reading performance assessment is disclosed. A position-identifying pattern which can be read by an image-capturing device is printed on a worksheet, along with other information useful for performing the student reading performance assessment. The worksheet also includes a number of indicator regions. Each region includes at least one indicator portion of the position-identifying pattern. A number of words are printed on the worksheet, each next to at least one of the indicator regions. A note region is also printed on the worksheet that includes a note portion of the position-identifying pattern.
The system also includes an image capturing device having a processor and a computer-readable memory with program instructions that, when executed, cause the processor to execute a method. This method is also described. The processor detects a first indicator portion of the position-identifying pattern that corresponds to a first mark in one of the indicator regions that is associated with a word. Based on whether the first mark indicates that the word was read incorrectly or correctly, the processor determines a first reading assessment result for the first word. A digital document file is stored, by the processor, in a memory. The digital document file includes the first reading assessment result.
This disclosure is not limited to the particular systems, devices and methods described, as these may vary. The terminology used in the description is for the purpose of describing the particular versions or embodiments only, and is not intended to limit the scope.
As used in this document, the singular forms “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise. Unless defined otherwise, all technical and scientific terms used herein have the same meanings as commonly understood by one of ordinary skill in the art. Nothing in this disclosure is to be construed as an admission that the embodiments described in this disclosure are not entitled to antedate such disclosure by virtue of prior invention. As used in this document, the term “comprising” means “including, but not limited to.”
As used in this document, a statement that a device or system is “in electronic communication with” another device or system means that devices or systems are configured to send data, commands and/or queries to each other via a communications network. The network may be a wired or wireless network such as a local area network, a wide area network, an intranet, the Internet or another network.
A “computing device” refers to a computer, a processor and/or any other component, device or system that performs one or more operations according to one or more programming instructions. The term “data” may refer to physical signals that indicate or include information.
An “image,” as a pattern of physical light or a collection of data representing the physical light, may include characters, words, symbologies, graphics and other features, as well as combinations thereof. A “digital image” is by extension an image represented by a collection of digital data. A “digital document” is by extension a document represented by a collection of digital data.
A “digital pen” refers to an input device that is capable of both writing on a substrate and capturing data corresponding to the user's handwriting or strokes. The digital pen contains components that convert the handwritten markings into digital data which can be saved on a computer-readable storage medium. Digital pens may contain accelerometers that can detect the movement of the pen on the substrate. Digital pens may also include positional technology which can physically detect the location of the pen tip during the writing process. Other digital pens can use a camera or other image-capturing sensor to detect the position of the pen tip on special digital paper that includes position identifying information.
An “electronic device” refers to a device that includes an imaging device, a processor and tangible, computer-readable memory. The memory may contain programming instructions in the form of a software application that, when executed by the processor, causes the device to perform various operations according to the programming instructions. Examples of suitable devices include portable electronic devices such as smart phones, personal digital assistants, cameras, tablet devices, electronic readers, personal computers, media players, satellite navigation devices and the like.
This disclosure describes a system that combines digital pen, intelligent paper, and audio recording technologies to create a running record assessment that is capable of capturing teacher annotation and a variety of student response information. Specifically, the system is able to associate detailed teacher annotations with error indicator marks for specific words, to resynchronize the audio record of student reading with the teacher's pen marks, and to infer reading ability metrics from computer aided analysis of the pen marks and audio record of the running record session.
An ink tip 134 extends from an end of the housing 116 and may be used to apply markings on a substrate 140 such as paper. Optionally, the pen may include a timing circuit 112 such as a clock circuit or an interface to an external clock. In some embodiments, the pen 101 may include a timing circuit 112 such as a clock circuit or an interface to an external clock. In some embodiments, the timing circuit 112 may be an independent circuit, programming or another means that enables the pen to track an elapsed time. The pen also may include a position sensor 124 such as an accelerometer or gyroscope.
The substrate 140 also may include an embedded position-identifying pattern 150. The position-identifying pattern 150 will be digitally-readable indicia, such as a pattern or series of codes that uniquely identifies the document and various locations on the document. In situations where the substrates are used for information collection forms, each form (or each type of form) may contain unique elements, and the pattern will vary throughout different locations on each worksheet. For example, the indicia that are printed on an area corresponding to a worksheet's first data field may be different from that printed on the area corresponding to a different data field.
The embedded pattern may be any printable or printed, machine-readable indicia that may be used to provide data to identify a document and a location on the document. For example, the data may take the form of a unique, skewed dot matrix pattern such as that described above. Instead of a pure matrix pattern, in which under typical circumstances each dot is printed at the point where the horizontal and vertical guide lines of the matrix intersect, each dot would be slightly set-off or “skewed” from its intersection. Such dot matrix patterns may be generated using technologies such as those available from Anoto, Inc. The digital pen can see these dots with its sensor, and use the dots to identify the substrate and the location of the pen on the substrate. Alternatively, the embedded data may include a series of glyph codes or other codes, such as glyph codes that are generated using the process of U.S. Pat. No. 6,208,771, the disclosure of which is incorporated herein by reference in its entirety.
Referring now to
Errors are indicated using other notations. For example, the “R” mark 208 indicates that the student re-read a portion of the reading passage. The arrow extending from mark 208 to a check mark three spaces before indicates that the student re-read the three previous words before continuing. Substantive errors (e.g., using the wrong word or misreading a word) are indicated by marks 210, 212, 214, 216, and/or 218. For example, an uncorrected error is indicated by mark 210 showing that the student said “bedroom” instead of the word written in the passage, i.e. “room.” Mark 216 also illustrates an uncorrected error. Mark 212, in contrast, shows a self-corrected error, meaning the student recognized the error and corrected their reading of the word. In the example of mark 212, the student said the word “She” instead of the written word “See.” The “SC” in the upper right corner of mark 212 indicates that that the error was self-corrected. Marks 214 and 218 also illustrate self-corrected errors. The system may use and recognize any suitable notation system, such as a unique system or the known systems of running record symbols and marking notations. Examples of running record symbols may be found at www.readinga-z.com, among other places.
Referring now to
In the scenario shown in
The worksheet 300 also includes a note region 303. Although a defined portion of the page is designated as note region 303 in
As shown in
In one scenario, a student reads the reading passage shown on worksheet 300. As the student reads the passage aloud, an audio capture device records an audio file of the student's reading performance. The audio capture device may be a component of the digital pen or another audio capture device that is, along with the digital pen, in electronic communication with a computing device. The electronic communication may occur in real-time as the audio capture device captures signals, or it may be delayed such as by batch processing or after the audio capture device and computing device are linked. As the teacher listens to the student read (either live or during playback of the recording), the teacher makes a mark for each word in the passage printed on the worksheet 300, by either marking the “correct” region 302 or the “incorrect” region 304 for that word. As the teacher marks each box, the digital pen detects the portion of the position identifying pattern 301 that corresponds with the marked box. For example, when the teacher marks the “incorrect” indicator region 306 for the word “room,” the digital pen detects, associates, and stores data indicating that the student has made an error for the word and data comprising all marks made by the pen on the worksheet until the teacher marks a different indicator region is marked. In other words, after the teacher marks or selects indicator region 306, the system associates the word “room” with all subsequently made marks until the teacher marks or selections another indicator region. This process is described more fully below. In addition, the digital pen may also store a time stamp indicating the point in time during the recording when the teacher marked the region 306 for the word “room.” After the reading session is over, the teacher may then replay the recording starting from the point in time when the student read that word by tapping region 306, again.
In this scenario, after the teacher marks the region 306, and the digital pen detects an indicator portion of the position-identifying pattern corresponding to region 306, all subsequent marks that are not made in an indicator region are associated with the word “room.” For example, the teacher makes note 316 in the note region 303 of worksheet 300. The teacher uses a standard notation to indicate that the student made an uncorrected error in their reading. The notation indicates that, instead of “room” (written on the bottom of the note), the student said “bedroom” (written on the top of the note). The entire notation 316 is associated with the word “room,” and data representing the notation 316 is stored by the digital pen for later processing and review. Similarly, notes 318, 320, 322, and 324 are stored and associated with the incorrect regions 308, 310, 312, and 314 respectively. Therefore, when the teacher marks region 308, all marks (i.e. note 318) is stored and associated with the word “See.” When the teach marks region 310, note 320 is stored and associated with the word “to,” and so on.
In another scenario, the digital pen is in communication with a computing device (not shown in
Optionally, a position-identifying pattern is generated 406. The pattern may be generated using the reading passage or may be pre-generated. The pattern may be pre-printed on the paper or may be printed with the worksheet. As long as the portions of the position-identifying pattern contain the indicator regions, any method of printing the position-identifying pattern is sufficient. The generated worksheet may then be printed out on paper for use as a running record worksheet. The printing may be done by a laser printer, inkjet printer, and/or any other printing device which may use any method of applying a permanent pattern of position-identifying data on the paper so long as the printing is done in a manner that can be detected by the digital pen. The worksheet may also include indicia, such as a barcode or other type of machine readable indicia, that allow the worksheet to be identified. For example, a teacher may wish to recall the worksheet file and/or audio recording of a particular session. In this scenario, the teacher may use the digital pen to read a barcode or other indicia. The digital pen would then access, or instruct a computing device to access, a data storage facility to retrieve the worksheet file and/or audio file for review.
The digital pen, or another audio capture device in communication with the digital pen and/or with a computing device, records an audio file containing ambient sound that includes a student's reading of the imported reading sample 408. The audio file may be recorded by the digital pen, e.g. with digital pen 101 using audio sensor 122 as shown in
An image-capturing sensor on the digital pen or other image-capturing device captures data representing the position of markings in an indicator region associated with a word that is printed on the paper 410. The portion of the position-identifying pattern that corresponds to the indicator region that is marked by the teacher in response to the student's reading of a word. Although the teacher may mark the region, it is not required that they do so. The teacher may tap the indicator region so that the image-capturing sensor is capable of detecting which indicator region is selected. Since each indicator region is associated with a word in the reading passage, the initial mark or tap on an indicator region during a running record session has the effect of associating a particular word with a particular moment in time. Additionally, after a mark has been made in an indicator region, any subsequent mark in the note portion of the worksheet is associated with the word corresponding to the indicator region. All marks made will continue to be associated with that word until an indicator region corresponding to another word is marked.
A time stamp may be stored when the indicator region (e.g. incorrect region 306 as shown in
The image-capturing sensor detects and captures the portions of the handwritten markings made by the teacher in the note portion of the worksheet 414. As discussed above, a tap or mark in an indicator region associates any subsequent mark in the notation portion of the worksheet with the word corresponding to the marked indicator region. Therefore, the handwritten marks are associated with the previously marked word 416. The handwritten markings data is stored in a memory, e.g., as a digital document 418. The digital document may be a portable document format (PDF) document, a word processing format, or some other document format suitable for storing, viewing, searching, and editing handwritten digital documents. Optionally, the digital document or the data representing the handwritten markings may be transferred to a computing device for processing, storage and/or viewing.
All subsequent markings made in the note portion of the worksheet are associated with the selected word 506. Since each indicator region is associated with only one word, the selection of an indicator region, for example region 306 of
The system can use the mark to generate a reading assessment result indicating whether the child's reading of the word was correct or incorrect 508. If so, the error may be categorized by analyzing the teacher's handwritten notations 510. The notations may indicate whether the student self-corrected the error (“SC”), corrected the error after teacher prompting, or did not correct the error. As noted above, common errors can be easily determined based on the notation made. For example, an uncorrected error can be identified when there is no “SC” in the notation to indicate that the student self-corrected the error. Conversely, a self-corrected error can be identified when there is an “SC” in the notation. A re-read can be identified when the teacher marks a series of words as correct, then backtracks and selects the same words again. Referring to
Referring again to
A controller 620 provides an interface between with one or more optional tangible, computer-readable memory devices 625 and the system bus 600. These memory devices 625 may include, for example, an external or internal DVD or CD ROM drive, a hard drive, flash memory, a USB drive or the like. As indicated previously, these various drives and controllers are optional devices. Additionally, the memory devices 625 may be configured to include individual files for storing any software modules or instructions, auxiliary data, common files for storing groups of results or auxiliary, or one or more databases for storing the result information, auxiliary data, and related information as discussed above.
Program instructions, software or interactive modules for performing any of the methods and systems as discussed above may be stored in the ROM 610 and/or the RAM 615. Optionally, the program instructions may be stored on a tangible computer readable medium such as a compact disk, a digital disk, flash memory, a memory card, a USB drive, an optical disc storage medium, such as a Blu-ray™ disc, and/or other recording medium.
An optional display interface 640 may permit information from the bus 600 to be displayed on the display 645 in audio, visual, graphic or alphanumeric format. The information may include information related to a current job ticket and associated tasks. Communication with external devices may occur using various communication ports 650. A communication port 650 may be attached to a communications network, such as the Internet or an local area network.
The hardware may also include an interface 655 which allows for receipt of data from input devices such as a keyboard 660 or other input device 665 such as a digital pen as described above, a mouse, a joystick, a touch screen, a remote control, a pointing device, a video input device and/or an audio input device.
The claims, as originally presented and as they may be amended, encompass variations, alternatives, modifications, improvements, equivalents, and substantial equivalents of the embodiments and teachings disclosed herein, including those that are presently unforeseen or unappreciated, and that, for example, may arise from applicants/patentees and others.
Number | Name | Date | Kind |
---|---|---|---|
4164078 | Goldfarb | Aug 1979 | A |
4261241 | Gould et al. | Apr 1981 | A |
4464118 | Scott et al. | Aug 1984 | A |
4793810 | Beasley, Jr. | Dec 1988 | A |
5281146 | Janney | Jan 1994 | A |
5387107 | Gunter et al. | Feb 1995 | A |
5730602 | Gierhart et al. | Mar 1998 | A |
5794249 | Orsolini et al. | Aug 1998 | A |
RE36028 | Deesen et al. | Jan 1999 | E |
6011949 | Shimomukai | Jan 2000 | A |
6118973 | Ho et al. | Sep 2000 | A |
6154757 | Krause et al. | Nov 2000 | A |
6208771 | Jared et al. | Mar 2001 | B1 |
6215901 | Schwartz | Apr 2001 | B1 |
6515690 | Back et al. | Feb 2003 | B1 |
6606479 | Cook et al. | Aug 2003 | B2 |
6673611 | Thompson et al. | Jan 2004 | B2 |
6678499 | Silverbrook | Jan 2004 | B1 |
6759206 | Rubin et al. | Jul 2004 | B1 |
6953343 | Townshend | Oct 2005 | B2 |
7036075 | Walker | Apr 2006 | B2 |
7147473 | Harpaz | Dec 2006 | B2 |
7618259 | Manos | Nov 2009 | B2 |
7734652 | Tamayo et al. | Jun 2010 | B2 |
7831933 | Marggraff et al. | Nov 2010 | B2 |
7853193 | Marggraff | Dec 2010 | B2 |
1015947 | Hradek et al. | Jun 2011 | A1 |
8226416 | Abrash et al. | Jul 2012 | B2 |
8261967 | Long et al. | Sep 2012 | B1 |
20020087560 | Bardwell | Jul 2002 | A1 |
20030001020 | Kardach | Jan 2003 | A1 |
20030180703 | Yates et al. | Sep 2003 | A1 |
20040093346 | Hochman | May 2004 | A1 |
20050024346 | Dupraz | Feb 2005 | A1 |
20050114160 | Boehme et al. | May 2005 | A1 |
20060018544 | Ohguro | Jan 2006 | A1 |
20060216683 | Goradia | Sep 2006 | A1 |
20060256371 | King et al. | Nov 2006 | A1 |
20070160971 | Caldera | Jul 2007 | A1 |
20080108035 | Warda | May 2008 | A1 |
20080264701 | Radtke et al. | Oct 2008 | A1 |
20080286732 | German et al. | Nov 2008 | A1 |
20090002345 | Burstrom | Jan 2009 | A1 |
20090202971 | Cortez | Aug 2009 | A1 |
20090204821 | Fransson et al. | Aug 2009 | A1 |
20090246744 | Lofthus et al. | Oct 2009 | A1 |
20090271433 | Perronnin et al. | Oct 2009 | A1 |
20100047758 | McCurry et al. | Feb 2010 | A1 |
20100075290 | DeYoung et al. | Mar 2010 | A1 |
20100075291 | DeYoung et al. | Mar 2010 | A1 |
20100075292 | DeYoung et al. | Mar 2010 | A1 |
20100157345 | Lofthus et al. | Jun 2010 | A1 |
20100159432 | German et al. | Jun 2010 | A1 |
20100159437 | German et al. | Jun 2010 | A1 |
20100159438 | German et al. | Jun 2010 | A1 |
20100227306 | Lofthus et al. | Sep 2010 | A1 |
20120021398 | Venable | Jan 2012 | A1 |
20120118964 | Kindberg et al. | May 2012 | A1 |
20120141967 | German et al. | Jun 2012 | A1 |
20120282587 | Lofthus et al. | Nov 2012 | A1 |
20150161903 | Colliander | Jun 2015 | A1 |
Number | Date | Country |
---|---|---|
0556530 | Aug 1993 | EP |
2000123115 | Apr 2000 | JP |
WO-9832107 | Jul 1998 | WO |
WO-0221259 | Mar 2002 | WO |
WO-2004090834 | Oct 2004 | WO |
Entry |
---|
Dymetman et al., “Intelligent Paper”, Xerox Research Centre Europe, Meylan, France, Electronic Publishing, Artistic Imaging, and Digital Typography, Lecture Notes in Computer Science, vol. 1375, 1998, pp. 392-406. |
“Introduction to the Livescribe Platform”, Livescribe SDKs, copyright 2007-2010 Livescribe Inc., Dec. 20, 2010, 11:58 a.m. |
Sebastian Wren, Ph.D. and Jennifer Watts, Ph.D., “The Abecedarian Reading Assessment”, copyright 2002. |
Chuat, “CategoriX & ClusteriX”, Xerox Research Centre Europe, licensing@xrce.xerox.com (May 2005). |
Number | Date | Country | |
---|---|---|---|
20140248591 A1 | Sep 2014 | US |