Utilization of temporal and spatial parameters to enhance the writing capability of an electronic device

Abstract

The enhanced feature of this invention is the direction of a pen input on an electronic device to a separate canvas where it can be separately manipulated and processed for use. Temporal and spatial analysis can ensure that each canvas contain pen strokes that spatially belong together such as an individual word composed of multiple strokes for easy manipulation of the word on the screen. Another related enhanced feature of the invention for erasing portions of the inputted writing is the process of generating a new curve with different points than the original curve.

Description

FIELD OF THE INVENTION

The invention relates to generally to electronic devices. More particularly, the invention relates to methods and devices for manipulating ink strokes on touch sensitive screens of an electronic device.

BACKGROUND OF THE INVENTION

In numerous electronic touch sensitive devices, a pen or stylus can be used to input writing and display the writing on the screen. In some devices, the method of implementing this includes processing co-ordinate streams from the user's manipulation of a pen near or touching a sensing apparatus attached to a display, and directly drawing on the device's graphics surface lines or curves that approximate the pen's motion. Other devices convert the co-ordinates produced by the motion of a pen to letters or words and then display these letters and words as typewritten text. In other devices, the strokes are interpreted and categorized as simple geometric shapes, and some processing is done to ‘clean up’ those shapes (e.g. ‘straightening’ a line, recognizing and adjusting circles and ovals, adjusting lines with an appropriate spatial relationship to form a triangle, rectangle, square, etc.) and these predefined shapes are displayed. Numerous methods also exist to erase writing on the screen.

The need still exists, however, for a better way to capture, display and edit writing on the screen. Manipulation of writing on the display and in storage could be more streamlined and efficient.

SUMMARY OF THE INVENTION

The enhanced feature of this invention is the direction of a pen input to a separate canvas where it can be separately manipulated and processed for use. Temporal and spatial analysis can ensure that each canvas contain pen strokes that spatially belong together such as an individual word composed of multiple strokes for easy manipulation of the word on the screen. Another related enhanced feature of the invention for erasing portions of the inputted writing is the process of generating a new curve with different points than the original curve.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a flowchart in accordance with the instant invention.

FIG. 2 shows an erasing process in accordance with the instant invention.

FIG. 3 shows an erasing process in accordance with the instant invention.

DETAILED DESCRIPTION OF THE INVENTION

The term canvas refers to a graphics surface. It is typically used as an object within a web page and can initially contain lines and curves when downloaded from the web server, or can be manipulated later locally using a scripting language built into the browser (most commonly JavaScript).

In the instant invention, input pen events on a touch screen or similar device that accepts pen inputs are directed to a canvas object. The object can then be passed to another software component for processing, and the software component then performs filtering, curve-fitting, and similar processing techniques to determine a path the pen is following. The software component then determines and updates the canvas object with lines or curves that approximate the path that the pen took.

The advantage of the invention derives from extending an existing type of object (canvas object) with integrated pen support. This hybrid approach differs from conventional means of supporting pen, where capturing, processing and drawing are all performed in the same software component.

Another novel feature of this device is the method of performing temporal and spatial analysis of the co-ordinate information to group related strokes together, as opposed to attempting to do higher-level analysis of the strokes to recognize handwriting or geometric figures. The benefit is that handwritten words or figures can be selected and manipulated (moved, copy/pasted) as a logical unit (e.g. a word, diagram) rather than individual strokes. A word like ‘little’ written cursively is typically three strokes, and without this ‘superscript’ grouping, selecting and moving the first long cursive stroke would not move the dot over the ‘i’, or the cross across the ‘tt’. The benefits get progressively greater for printed and mixed printed/cursive handwriting.

The flowchart shown in FIG. 1 illustrates how the super strokes are created and manipulated. The process begins with a master canvas that can accept input pen strokes. The first operation 2 is a read of the pen event. The first decision operation 4 is whether the pen input is being received (the pen is down or interacting with the screen). If the pen is down the next operation decision 7 is then whether the pen was down in the previous event. If the pen was down in the previous event, then operations 10 and 11 are executed and the pen coordinates are processed and inserted into the master canvas. The process then returns to operation 2.

If operation decision 4 was false and the pen is not down, the next operation decision 5 checks if the pen was down in the previous event. If it was, operation 3, starting a timer, is executed and the process returns to operation 2. If the pen was not down at operation decision 5, then operation decision 6 determines if the timer set in operation 3 has expired. If it has not expired, the process returns to operation 2. If it has expired then operations 12, 13, 14, 15 and 16 are executed before the process returns to operation 2. Operations 12, 13, 14, 15 and 16 reset the time and create a child canvas from the last set of curves from the master canvas.

If operation decision 4 was true but operation decision 7 was false, operation decision 8, checking whether the timer has expired, is executed. If at operation decision 8 the timer has expired, operations 12, 13, 14, 15 and 16 are executed and the process returns to operation 2.

If, on the other hand, at operation decision 8 it is determined that the timer has not expired, operation decision 9, determining if the new strokes are in close proximity to the previous ones, is executed. If they are, operations 10 and 11, processing the pen input and inserting or updating the curve in the master canvas, are executed and the process returns to operation 2. If at operation decision 8 it is determined that the timer has not expired and then at operation decision 9 it is determined that the new strokes are not in close proximity to the previous strokes then operations 12, 13, 14, 15 and 16 are executed and the process returns to operation 2.

In this way, child canvases are created that are based on temporal and spatial information. The child canvases make screen manipulation of the displayed input much easier and more efficient. For example, in this way, handwritten words or figures can be selected and manipulated (moved, copy/pasted) as a logical unit (e.g. a word, diagram) rather than individual strokes.

In many cases it is desired to edit the displayed canvasses in a similar way to using an eraser. This is typically done in one of two ways. There are several variants of systems that draw lines on the screen according to coordinates generated by a digitizer overlaying the display. In some cases, coordinates are filtered and curves are fitted to the coordinates. The ‘vector’ form of the data (i.e. the series of control points that determine the curve) is retained and portions of the curve can be manipulated by adjusting ‘control points’ or erased by cutting the curve at specific points. Alternatively, the curve can be converted to a bitmap and erasing can be done by pixel-level tools that affect the area of the pixels under the eraser area, irrespective of the means originally used to create the curve.

In the present invention, the improvement is to use an eraser tool on a curve or path wherein the area being erased is determined by a geometric shape (e.g. a circle) that tracks the pointing device (finger, stylus, pen etc.). The path is maintained in a vector format, not flattened to a bitmap. The area erased is determined by calculating the intersection(s) between the eraser shape and the path, inserting (or relocating) points at the intersections, and removing the line segments(s) and control points that fall within the eraser shape. A flowchart of this process is shown in FIG. 2 and FIG. 3.

As can be seen in FIG. 2, and FIG. 3, the eraser tool (1) intersects a given curve with control points (2). As a result, the curve has some of its control points deleted and new points (3) inserted. In particular, in one embodiment, the eraser tool affects a curve that has been fitted to the recorded co-ordinates, not line segments connecting them. For example, the points used to fit a new curve do not necessarily coincide with the original recorded points or even lie on straight segment connecting the original recorded points. A segment of the curve is removed when the area covered by the eraser shape intersects it. New co-ordinates are added at the point of intersection of the eraser shape and the fitted curve, so that the curve is preserved. Inserting points at the intersection of the eraser shape and line segment(s) would alter the fitted curve, in some cases drastically. A curve-fitting algorithm treats endpoints specially. The eraser shape removing a segment of the curve will often create one or two new endpoints (depending on whether the curve was trimmed or bisected, respectively), the algorithm may choose to insert more than the new co-ordinate at the point of intersection. One or more points may be inserted or moved between the new co-ordinate at the point of intersection and nearest recorded co-ordinate, to preserve the original shape of the fitted curve in the absence of the original co-ordinates now erased. Since recorded co-ordinates represent ‘original data’, recorded co-ordinates are only ever deleted (i.e. if the eraser shape occludes them). Inserted points are marked as being synthetic, and the erasing algorithm may insert and move these freely as the erase shape moves and progressively clips larger amounts of the line segment.

Claims

1. A method to be executed at least in part in a pen input receiving application comprising: receiving a pen input;creating a canvas area based on spatial and temporal pen input information; andinserting the canvas area.

2. The method of claim one wherein said canvas area is a smaller area than the original area that accepts said pen input.

3. The method of claim two wherein the said canvas area encloses a single word.

4. A method to be executed at least in part in a pen and eraser input receiving application on a existing digital pen contour comprising: receiving an eraser input;determining the shape of an erasing contour;identifying new points that did not necessarily exist on the original pen contour based on the erasing input;modifying an pen contour based on the erasing contour and said new points; andinserting the new contour or contours.

5. The method of claim four wherein at least one of the new points did not exist on the original pen contour.

6. A computer readable medium capable of storing a program facilitating rendering and electronic pen stroke being intersected by an erasing contour the program comprising the steps of: receiving an eraser input;determining the shape of an erasing contour;identifying new points that did not necessarily exist on the original pen contour based on the erasing input;modifying an pen contour based on the erasing contour and said new points; andinserting the new contour or contours.

7. The computer readable medium of claim six wherein at least one of the new points did not exist on the original pen contour.

8. A computer readable medium capable of storing a program facilitating rendering and electronic pen stroke the program comprising the steps of: receiving a pen input;creating a canvas area based on spatial and temporal pen input information; andinserting the canvas area.

9. The computer readable medium of claim eight wherein said canvas area is a smaller area than the original area that accepts said pen input.

10. The computer readable medium of claim nine wherein said canvas area encloses a single word.