VIDEO-BASED HANDWRITING INPUT METHOD AND APPARATUS

Description

BRIEF DESCRIPTION OF THE DRAWINGS

In order to further understand of the invention and advantages thereof, the following detailed description of the disclosed embodiment will be considered in conjunction with the accompanying drawings, in which:

FIG. 1 & FIG. 2 respectively illustrate different information processing systems implementing the present invention;

FIG. 3 shows the stylus 130 used for the present invention;

FIG. 4 depicts the flowchart of handwriting input process according to the present invention;

FIG. 5 depicts that the user can interact with a computer system via screen to define the target area on the writing surface;

FIG. 6 depicts a sequence of image frames taken by the digital camera;

FIG. 7 depicts an example of performing step 450; and

FIG. 8 depicts the results of processing the raw strokes shown in FIG. 6 via the step 450, in which there are only the solid lines in FIG. 6 stored in a storage.

Claims

1. A video-based handwriting input method, comprising: capturing images including raw strokes and receiving writing sounds, when a user writes on a writing surface with a stylus having a special mark, said images and sounds being time-stamped;extracting images of raw strokes by comparing the captured images with the special mark on the stylus; andretrieving effective strokes from the raw strokes by utilizing the received writing sounds.
2. The method of claim 1, further comprising a preliminary step of defining a target area for handwriting on the writing surface.
3. The method according to claim 1 wherein said step of retrieving effective strokes from the raw strokes removes strokes corresponding to sound-non-existing periods and leaves effective strokes.
4. The method of claim 3, wherein said step of retrieving effective strokes from the raw strokes compares said time-stamped images of raw strokes with said time-stamped sounds by aligning time-stamps to remove strokes corresponding to sound-non-existing periods and leave effective strokes.
5. The method according to claim 1, wherein said step of capturing images including raw strokes is performed by a digital camera.
6. The method according to claim 1, wherein said step of receiving writing sounds is performed by a microphone.
7. The method according to claim 1, wherein said stylus is basically a pen having a special mark, which can write on a real paper, or a plastic stylus having a special mark, which can not write on a real paper to leave ink traces.
8. The method of claim 7, wherein said special mark is a point having a special color different from the colors of the writing environment, and the special color is changeable.
9. The method of claim 8, wherein said point having the special color is positioned at the tip of the stylus.
10. The method according to claim 1, wherein said special mark is a light point.
11. The method of claim 2, wherein said preliminary step of defining a target area for handwriting includes a step of transforming the image of target area to rectangle.
12. The method of claim 11, further comprising a mathematical transformation and normalization step of images of raw strokes to transform and normalize said time-stamped images of raw strokes in order to correct skew of the images of raw strokes when the digital camera takes images.
13. The method according to claim 1, further comprising a step of feeding said effective strokes to a handwriting recognition engine for recognition.
14. The method according to claim 1, wherein said stylus further comprising a micro-switch and a buzzer.
15. A video-based handwriting input method, comprising: capturing images including effective strokes and receiving writing sounds, when a user writes on a writing surface with a stylus having a special mark, said images and sounds being time-stamped;removing images data corresponding to sound-non-existing periods and recording the remaining images data; andretrieving effective strokes by comparing said recorded images with the special mark on the stylus.
16. The method of claim 15, further comprising a preliminary step of defining a target area for handwriting on the writing surface.
17. The method of claim 15, wherein said step of removing images data corresponding to sound-non-existing periods compares said time-stamped images with said time-stamped sounds by aligning time-stamps to remove the images corresponding to sound-non-existing periods.
18. The method according to claim 15, wherein said step of capturing images including effective strokes is performed by a digital camera.
19. The method according to claim 15, wherein said step of receiving writing sounds is performed by a microphone.
20. The method according to claim 15, wherein said stylus is basically a pen having a special mark, which can write on a real paper, or a plastic stylus having a special mark, which can not write on a real paper to leave ink traces.
21. The method of claim 20, wherein said special mark is a point having a special color different from the colors of the writing environment, and the special color is changeable.
22. The method of claim 21, wherein said point having the special color is positioned at the tip of the stylus.
23. The method according to claim 15, wherein said special mark is a light point.
24. The method of claim 16, wherein said preliminary step of defining a target area for handwriting includes a step of transforming the image of target area to rectangle.
25. The method of claim 24, further comprising a mathematical transformation and normalization step of images of effective strokes to transform and normalize said time-stamped images of effective strokes in order to correct skew of the images of effective strokes when the digital camera takes images.
26. The method according to claim 15, further comprising a step of feeding said effective strokes to a handwriting recognition engine for recognition.
27. The method according to claim 15, wherein said stylus further comprising a micro-switch and a buzzer.
28. A video-based handwriting input apparatus, comprising: a stylus having a special mark;a digital image capture means for capturing images including raw strokes, said images being time-stamped;a writing sound receiving means for receiving writing sounds, when a user writes on a writing surface with said stylus having a special mark, said sounds being time-stamped;a extracting raw strokes image means for extracting images of raw strokes by comparing the captured images with the special mark on the stylus; anda retrieving effective strokes means for retrieving effective strokes from the raw strokes by utilizing the received writing sounds.
29. The handwriting input apparatus of claim 28, further comprising means of defining a target area for handwriting on the writing surface.
30. The handwriting input apparatus according to claim 28, wherein said retrieving effective strokes means removes strokes corresponding to sound-non-existing periods and leaves effective strokes.
31. The handwriting input apparatus of claim 30, wherein said retrieving effective strokes means compares said time-stamped images of raw strokes with said time-stamped sounds by aligning time-stamps to remove strokes corresponding to sound-non-existing periods and leave effective strokes.
32. The handwriting input apparatus according to claim 28 or 29, wherein said digital image capture means is a digital camera.
33. The handwriting input apparatus according to claim 28, wherein said writing sound receiving means is a microphone.
34. The handwriting input apparatus according to claim 28, wherein said stylus is basically a pen having a special mark, which can write on a real paper, or a plastic stylus having a special mark, which can not write on a real paper to leave ink traces.
35. The handwriting input apparatus of claim 34, wherein said special mark is a point having a special color different from the colors of the writing environment, and the special color is changeable.
36. The handwriting input apparatus of claim 35, wherein said point having the special color is positioned at the tip of the stylus.
37. The handwriting input apparatus according to claim 28, wherein said special mark is a light point.
38. The handwriting input apparatus of claim 29, wherein said means of defining a target area for handwriting includes means of transforming the image of target area to rectangle.
39. The handwriting input apparatus of claim 38, further comprising a mathematical transformation and normalization means of images of raw strokes to transform and normalize said time-stamped images of raw strokes in order to correct skew of the images of raw strokes when the digital camera takes images.
40. The handwriting input apparatus according to claim 28, further comprising a handwriting recognition engine for recognition of said effective strokes.
41. The handwriting input apparatus according to claim 28, wherein said stylus further comprising a micro-switch and a buzzer.
42. A video-based handwriting input apparatus, comprising: a stylus having a special mark;a digital image capture means for capturing images including effective strokes, said images being time-stamped;a writing sound receiving means for receiving writing sounds, when a user writes on a writing surface with said stylus having a special mark, said sounds being time-stamped;a recording images data means for recording remaining images data by removing images data corresponding to the sound-nonexisting period; anda retrieving effective strokes means for retrieving effective strokes by comparing said recorded images with the special mark on the stylus.
43. The handwriting input apparatus of claim 42, further comprising means of defining a target area for handwriting on the writing surface.
44. The handwriting input apparatus of claim 42, wherein said recording images data means compares said time-stamped images with said time-stamped sounds by aligning time-stamps to remove the images corresponding to sound-non-existing periods.
45. The handwriting input apparatus according to claim 42, wherein said digital image capture means is a digital camera.
46. The handwriting input apparatus according to claim 42, wherein said writing sound receiving means is a microphone.
47. The handwriting input apparatus according to claim 42, wherein said stylus is basically a pen having a special mark, which can write on a real paper, or a plastic stylus having a special mark, which can not write on a real paper to leave ink traces.
48. The handwriting input apparatus of claim 47, wherein said special mark is a point having a special color different from the colors of the writing environment, and the special color is changeable.
49. The handwriting input apparatus of claim 48, wherein said point having the special color is positioned at the tip of the stylus.
50. The handwriting input apparatus according to claim 42, wherein said special mark is a light point.
51. The handwriting input apparatus of claim 43, wherein said means of defining a target area for handwriting includes means of transforming the image of target area to rectangle.
52. The handwriting input apparatus of claim 51, further comprising a mathematical transformation and normalization means of images of effective strokes to transform and normalize said time-stamped images of effective strokes in order to correct skew of the images of effective strokes when the digital camera takes images.
53. The handwriting input apparatus according to claim 42, further comprising a handwriting recognition engine for recognition of said effective strokes.
54. The handwriting input apparatus according to claim 42, wherein said stylus further comprising a micro-switch and a buzzer.
55. A video-based handwriting input method, comprising enabling a digital image capturing means for capturing images including effective strokes, when a user writes on a writing surface with a stylus having a special mark and said stylus is brought into contact with the writing surface, said images being time-stamped;disabling said digital image capturing means when said stylus is lifted from the writing surface; andextracting effective stroke by comparing the captured images with the special mark on the stylus.
56. The method of claim 55, further comprising a preliminary step of defining a target area for handwriting on the writing surface.
57. The method according to claim 55, wherein said stylus is basically a pen having a special mark, which can write on a real paper, or a plastic stylus having a special mark, which can not write on a real paper to leave ink traces.
58. The method of claim 57, wherein said special mark is a point having a special color different from the colors of the writing environment, and the special color is changeable.
59. The method of claim 58, wherein said point having the special color is positioned at the tip of the stylus.
60. The method according to claim 55, wherein said special mark is a light point.
61. The method of claim 56, wherein said preliminary step of defining a target area for handwriting includes a step of transforming the image of target area to rectangle.
62. The method of claim 61, further comprising a mathematical transformation and normalization step of images of effective strokes to transform and normalize said time-stamped images of effective strokes in order to correct skew of the images of effective strokes when the digital camera takes images.
63. The method according to claim 55, further comprising a step of feeding said effective strokes to a handwriting recognition engine for recognition.
64. The method according to claim 55, wherein said stylus further comprising a micro-switch and a buzzer.
65. A video-based handwriting input apparatus, comprising: a digital image capture means;a stylus having a special mark, said stylus enabling said digital image capturing means for capturing images including effective strokes, when a user writes on a writing surface with a stylus having a special mark and said stylus is brought into contact with the writing surface, said images being time-stamped, and disabling said digital image capturing means when said stylus is lifted from the writing surface; anda extracting effective stroke means for extracting effective stroke by comparing the captured images with the special mark on the stylus.
66. The handwriting input apparatus of claim 65, further comprising means of defining a target area for handwriting on the writing surface.
67. The handwriting input apparatus according to claim 65, wherein said stylus is basically a pen having a special mark, which can write on a real paper, or a plastic stylus having a special mark, which can not write on a real paper to leave ink traces.
68. The handwriting input apparatus of claim 67, wherein said special mark is a point having a special color different from the colors of the writing environment, and the special color is changeable.
69. The handwriting input apparatus of claim 68, wherein said point having the special color is positioned at the tip of the stylus.
70. The handwriting input apparatus according to claim 65 or 66, wherein said special mark is a light point.
71. The handwriting input apparatus of claim 66, wherein said means of defining a target area for handwriting includes means of transforming the image of target area to rectangle.
72. The handwriting input apparatus of claim 71, further comprising a mathematical transformation and normalization means of images of effective strokes to transform and normalize said time-stamped images of effective strokes in order to correct skew of the images of effective strokes when the digital camera takes images.
73. The handwriting input apparatus according to claim 65, further comprising a handwriting recognition engine for recognition of said effective strokes.
74. The handwriting input apparatus according to claim 65, wherein said stylus further comprising a micro-switch and a buzzer.
75. A pervasive device, comprising: a handwriting input apparatus according to anyone of claims 28-54 and claims 65-74.
76. A data processing system having a digital image capture means and a writing sound receiving means, comprising a handwriting input module executing a handwriting input method according to anyone claim 1.
77. A computer recording medium stored a program, said program comprising program codes causing a data processing system to execute a handwriting input method according to claim 1.

Priority Claims (1)

Number	Date	Country	Kind
094147480	Dec 2005	TW	national

VIDEO-BASED HANDWRITING INPUT METHOD AND APPARATUS

Information

Publication Number

Date Filed

Date Published

Inventors

CPC

US Classifications

International Classifications

Abstract

Description

Claims

Priority Claims (1)