Patent Application
20200218510
The background description includes information that may be useful in understanding the present invention. It is not an admission that any of the information provided herein is prior art or relevant to the presently claimed invention, or that any publication specifically or implicitly referenced is prior art.
Web-based graphing calculator programs for general use that run on a browser and web-based graphing calculator programs optimized for mobile devices such as smartphones and tablets are rapidly increasing in popularity. Such devices are a convenient substitute to carrying a separate specialized graphing calculator (such as the TI-83 Plus or TI-84 Plus or other ubiquitous TI graphing calculators) for students, engineering professionals and others who need to use such a portable device in their daily activities on worksites or in campus classrooms.
Accessibility for visually and hearing-impaired persons to such computing and mathematical applications and devices has long been an area of ongoing research. One challenging research area is finding ways to express graphical output such as curves on a Cartesian, polar or other coordinate system in a comprehensible manner for persons who are blind or low vision. Verbal output systems function well for persons relying on screen reader technology to voice computer activity. A screen reader is software that runs on computer systems which converts text on screen into spoken or Braille output to enable persons who are blind or low vision to operate a computer system. For example, “start fraction, 1 over 2, end fraction” is a reasonable syntax to verbally describe a fraction. However, for following more complex mathematical expressions, verbal output may be more complicated to follow. In addition, hearing and/or visually impaired students who are Braille readers may find verbal output systems inadequate. People who rely on Braille for communication use well-established codes for communicating arithmetic ideas that may be used to adapt calculator functions to Braille input and output devices.
According to the Random House Dictionary, Braille is “a system of writing and printing for the blind in which arrangements of raised dots representing letters and numbers can be identified by touch.” Each Braille character, known as a cell, consists of a combination of six or eight dots, The Braille code is used by people who are low vision or totally blind to read and write, typically with manual writing devices such as the Perkins Brailler, a slate and stylus, specialized embossers, or electronic displays which can operate stand-alone or serve as input and output terminals when connected to computers. More information about Braille is available at https://en.wikipedia.org/wiki/Braille. In addition to literary contexts, codes specific to the domains of math, science and music notation (which, like their print and typeset counterparts to literary text, bear little resemblance to literary Braille) have been developed over time. Common mathematical Braille codes include Nemeth, Unified English Braille (UEB), Computer Braille and UNICODE Braille.
Existing Braille translation systems are designed to translate primarily literary works, and in some limited cases mathematical works, but only as part of desktop software (such as the Duxbury Braille Translator or as part of screen reading programs such as JAWS for Windows and Apple's VoiceOver for the Mac and iOS platforms). There are few, if any, math-to-Braille libraries that exist for use in an online context that translate common mathematical coding formats such as LaTeX to common mathematical Braille codes. One other commonly known online Braille math input system, Pearson's Accessible Equation Editor (https://accessibility.pearson.con/aee), translates only a modified Nemeth syntax to MathML, and is unable to translate LaTeX into Nemeth and UEB, and cannot supply both equations and their answers in the user's preferred Braille code.
The Nemeth Braille code, developed by Dr. Alexander Nemeth, has been the standard set of rules and symbols for describing mathematics in the United States since 1972. It details comprehensive rules to describe many facets of math, including algebra, geometry, trigonometry, calculus and logic. The reference document (see link below) also includes numerous examples of mathematical expressions that have been incorporated in embodiments of the present invention to ensure accurate translation to and from LaTeX in the calculator of the present invention. While it is broadly known, Nemeth Braille code is used primarily in the United States and Canada. See Nemeth overview on Wikipedia (https://en.wikipedia.org/wiki/Nemeth_Braille) and The Nemeth Braille Code for Mathematics and Scientific Notation 1972 (https://nfb.org/images/nfb/documents/pdf/nemeth_1972.pdf)
For decades, English-speaking countries, such as the United States, Great Britain and Australia have written and maintained separate literary and scientific Braille codes. Though the literary specifications differ superficially from one another, the scientific notations are vastly dissimilar. As the electronic interchange of Braille documents has grown in popularity, blindness organizations in the above countries recognized a need to create an internationally recognized standard for creating distributing these files across national boundaries.
Unified English Braille, referred to hereinafter as UEB, was the result of many years' effort. It seeks to create a one-to-one mapping of all literary and scientific symbols to Braille dot patterns (which may encompass one or more cells). UEB is relatively new in comparison to older, more established codes, such as the previous United States literary and Nemeth math systems), and as a result there is much less documentation available aside from the official UEB technical guidelines. Also, while UEB became the standard for literary Braille in the United States in January 2016, it has been left for individual states to decide which system to use for technical material. Anecdotal research has shown that at the time of writing, Nemeth has much broader support than UEB. See UEB overview available at https://en.wikipedia.org/wiki/Unified_English_Braille and UEB Guidelines for Technical Material at http://www.brailleauthority.irg/ueb/Guidelines_for_Technical_Material_2008-10.pdf.
Another form of Braille encoding, Computer Braille, defines an exact 1-to-1 relationship between the standard ASCII character set and a single Braille cell. It is what an end user reads and writes when his or her Braille device operates without an active translator, and it is the only mode of input and output available on every Braille device manufactured throughout the world. While it has been superseded by UEB as an official standard in many countries Computer Braille's uniform implementation across all modern electronic Braille devices makes it ideal for unambiguous input and output. See Overview of Computer Braille available at: https://en.wikipedia.org/wiki/Computer_Braille_Code and Computer Braille reference at http://www.brailleauthority.org/cbc/cbc2000.pdf.
The UNICODE Braille standard defines a block of characters, called “Braille Patterns,” in the range U+2800 through U+28FF. The patterns comprise all possible 64 combinations of six dot Braille characters. See UNICODE Braille Patterns Overview at https://en.wikipedia.org/wiki/Braille_Patterns.
The present invention relates to apparatuses and methods for enabling a user electronic device (e.g. a smart phone, tablet computer, or laptop or desktop computer) to display mathematical expressions and equations in graphical as well as visible typeset or touch-identifiable form in audio, visual and/or Braille formats to enable such graphical and mathematical expressions to be accessible to both able-bodied users as well as users with hearing and vision disabilities.
Some embodiments comprise a method for graphically representing mathematical expressions on a user device in an audiovisual format in accordance with embodiments of the present invention. Exemplary methods may comprise interpreting keyboard input from a user device as a typeset mathematical expression. Exemplary methods may further comprise parsing the typeset mathematical expression into an interpreted mathematical expression and compiling the interpreted mathematical expression into an evaluation function and sampling the evaluation function for at least one coordinate. Exemplary methods may also comprise rendering the sampled evaluation function for the coordinate as a graph configured for display of at least one user device; and generating an audible representation of the rendered graph configured for playback on the user device. Some embodiments of the present invention comprise systems and computer program products for graphically representing mathematical expressions on a user device in an audiovisual format. Such embodiments may be used in verbal output systems, which can be of benefit for persons relying on a screen reader to voice computer activity. Some embodiments of the present invention may graphically represent mathematical expressions as a tactile graphic that may be rendered with a Braille embosser or refreshable tactile Braille display capable of rendering Braille characters and graphics (including but not limited to the Graphiti from Orbit Research).
In some embodiments of the present invention, mathematical expressions are expressed using Braille. In one embodiment of the invention, a math-to-Braille library for use in an online context is described. In another embodiment of the invention, a JavaScript library is developed that translates mathematical expressions written in LaTeX to and from Braille and does not rely on third-party translation algorithms. Because UEB has been adopted as the standard Braille code across the majority of English-speaking countries, one embodiment of the present invention implements UEB as well as Nemeth Braille codes so that users are able to leverage the system with which they are most comfortable. In one embodiment of the invention, Nemeth Braille code rules may be applied for UEB translations in situations which the UEB technical guidelines may not have guidance (such as when adding whitespace after trigonometric function names).
In one embodiment of the invention, Computer Braille encoding is used in translating mathematical expressions from Braille. For example, a Braille keyboard may be used in one embodiment of the invention to transmit text to a personal desktop or laptop computer or other user device, and Computer Braille encoding ensures that one ASCII character is transmitted for each typed symbol. Alternatively, raw keyboard input from a connected display may be received in Computer Braille mode for translation to other Braille codes or to a typewritten mathematical expression (for example, in LaTeX).
In some embodiments of the present invention, a Braille translator receives input consisting exclusively of Braille Patterns when converting from Nemeth to UEB. Likewise, Nemeth and UEB output is returned as a series of Braille Patterns. Such a translator may also include methods to convert between Computer Braille and Braille Patterns to facilitate handling input from standard QWERTY keyboards or other similar devices which emit traditional alpha-numeric characters.
Various objects, features, aspects and advantages of the inventive subject matter will become more apparent from the following detailed description of preferred embodiments, along with the accompanying drawing figures in which like numerals represent like components. The following detailed description is presented to enable any person skilled in the art to make and use the invention, and is provided in the context of particular applications and their requirements.
Various modifications to the exemplary embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.
For the purposes of illustration only, several aspects of particular embodiments of the invention are described by reference to the following figures:
All publications identified herein are incorporated by reference to the same extent as if each individual publication or patent application were specifically and individually indicated to be incorporated by reference. Where a definition or use of a term in an incorporated reference is inconsistent or contrary to the definition of that term provided herein, the definition of that term provided herein applies and the definition of that term in the reference does not apply.
As used in the description herein and throughout the claims that follow, when a system, engine, module, device, server, or other computing element is described as configured to perform or execute functions on data in a memory, the meaning of “configured to” or “programmed to” is defined as one or more processors being programmed by a set of software instructions stored in the memory of the computing element to execute the set of functions on target data or data objects stored in the memory.
As used herein, and unless the context dictates otherwise, the term “coupled to” is intended to include both direct coupling (in which two elements that are coupled to each other contact each other) and indirect coupling (in which at least one additional element is located between the two elements). Therefore, the terms “coupled to” and “coupled with” are used synonymously.
Exemplary System Configurations
Embodiments of the present invention have been tested with commonly available desktop and mobile device, operating system, web browser, and adaptive technology combinations to help ensure that the calculator is compatible with as many commonly used user configurations as possible. For example, features described in embodiments of the present invention have been tested in conjunction with the following system configurations:
For the Windows operating system:
For the iPhone and iPad device platforms:
Note that other previous, present and future versions of these operating system, screen reader, and browsers may also be compatible with accessibility features discussed in conjunction with embodiments of the present invention as described below, but the functionality and features have not been fully tested with respect to these alternative versions. Embodiments of the present invention may also work with other operating system, screen reader, and browser versions outside of those listed above, that are nonetheless enabled to work on the Internet using TCP/IP and other commonly used networking and communication protocols.
The audiovisual, Internet-enabled graphing calculator described in embodiments of the present invention is a rich web application which utilizes both client and server-side components, may be implemented using multiple languages and may leverage existing and home-grown user interface frameworks. Embodiments of the present invention are optimized to run on the client side of the application which is written using a combination of JavaScript, jQuery (www.jquery.org), UnderscoreJS templating (www.underscorejs.org), and an internally developed model/view/controller system called DCGView, which is similar in many respects to ReactJS (www.reactjs.org).
The features described below depend on several calculator components used and included in the present invention shown in
The input interface 110 is configured to transmit a typeset mathematical expression (e.g., in LaTeX typesetting format) to math engine 120. In one embodiment of the present invention, math engine 120 runs in a “web worker” or separate execution thread, thus allowing the graphing calculator to stay responsive to user input while the math engine is working. In embodiments of math engine 120, parser 123 interprets the LaTeX or other typeset mathematical expression to determine what kind of expression it is (e.g., point, function, inequality, polar (expressions with r and theta (θ) variables), piecewise, functions with regressions, expression lists, list expressions, functions with domain and range restrictions). Compiler 125 evaluates the expression, and where feasible, produces a function that represents the expression and that can be efficiently evaluated for different input coordinates, so that sampler 127 can sample the compiled functions at various input values to produce output sketch data, which can be used to produce a graphical representation. This sketch data is transmitted to the grapher 130 which renders sampled representations of expressions to the grapher interface, which in one embodiment of the present invention is a graph window that resembles graph paper in appearance, and renders the graphical form of the expression using HTML and canvas as described further below. In some embodiments of the present invention, a tone generator module 140 utilizes the sketch data to produce an audible representation of the graph that may be played back to the user as described in further detail below.
In an embodiment of the present invention, the input interface component may utilize code from the open-source MathQuill component (www.mathguill.com) which may be adapted provide features needed for screen reader, speech-to-text, and/or Braille accessibility for visually impaired users.
Slider: a draggable control representing a single variable. The slider can be set to animate, causing its value to oscillate between the slider's minimum and maximum values, thereby allowing one to see how the change affects the parent expression and its graph over time. Two examples of sliders are shown at 310 in
Expression List: A list view-style control holding one or more expression items 410 as shown in
An evaluator is a program object which monitors changes to an expression and determines if it is evaluable, graphable, or contains errors. Unlike a built-in JavaScript expression evaluator, the evaluator counterpart present in embodiments of the present invention will also explain the mistake (when possible) and understands a much richer set of mathematical constructs.
A grapher program object is the root object in charge of drawing graphs on the screen used in embodiments of the present invention. This object contains an HTML5 canvas on which lines for each graph's curve are drawn. Each piece of a curve, referenced internally as a sketch segment, is computed by the evaluator and retrieved from the graph's sketch data object. The set of segments representing a curve are stored as a single data object, called a graph layer, within the grapher. The grapher also holds functions for drawing grid lines and dots representing points of interest. Grid lines are drawn first, followed by each graph layer, and finally the POI dots.
The grapher uses two objects to define the scope and resolution of the curves shown on screen. First is the projection data object, which holds properties reflecting the active screen resolution. The second piece, known as the viewport, contains the range of X and Y coordinates (stored as minimums and maximums for each) which are to be displayed. When the window is resized, the limits of the x-axis are kept and the y-axis is scaled as needed. For example,
Sketch: A data object retrieved from the grapher which represents the curve derived from an evaluated expression. The sketch holds additional properties, including the number of branches in the curve, and a list of points of interest (POIs). If an expression has more than one solution, each possible answer (or set of answers) is represented on its own branch within the same sketch. POIs include a graph's minimum, maximum, zero-crossing, X/Y intercept, and intersections with other curves. The following graphs illustrate these concepts:
For example,
For example,
As another example,
Also worth noting is that the sketch updates in real-time based on changes to its associated expression (whether it be a manual edit or influenced by an animating slider).
Embodiments of the present invention also include the following program objects to enable audio representation of mathematical expressions and graphs:
The ARIA object is a program object designed specifically to send ARIA alerts to the browser, which then conveys the text to an active screen reader. The module can either accept one or more strings of text to be queued for later verbalization, or be passed a message to be immediately spoken. This module has been designed to operate with all modern operating systems, browser, and screen reader combinations which support the WAI-ARIA specification (WAI-ARIA specification version 1.1 is available at https://www.w3.org/TR/wai-aria-1.1), and thus is a core component in providing voiced output throughout the calculator. As discussed on the WAI-ARIA Wikipedia entry at https://en.wikipedia.org/wiki/WAI-ARIA, web developers increasingly use client-side scripts to create user interface controls that cannot be created with HTML alone. They also use client-side scripts to update sections of a page without requesting a completely new page from a web server. Such techniques on websites are called rich Internet applications. These user interface controls and content updates are often not accessible to users with disabilities, especially screen reader users and users who cannot use a mouse or other pointing device. WAI-ARIA allows web pages (or portions of pages) to declare themselves as applications rather than as static documents, by adding role, property, and state information to dynamic web applications. ARIA is intended for use by developers of web applications, web browsers, assistive technologies, and accessibility evaluation tools.
WAI-ARIA describes how to add semantics and other metadata to HTML content to make user interface controls and dynamic content more accessible. For example, with WAI-ARIA it is possible to identify a list of links as a navigation menu and to state whether it is expanded or collapsed. Although originally developed to address accessibility issues in HTML, the use of WAI-ARIA is not limited to HTML: in principle, it can also be used in other markup languages such as Scalable Vector Graphics (SVG).
Embodiments of the present invention also include an audio graph program component that is responsible for announcing when focused expressions contain graphs, listening for all audio graphing hotkeys (including those to enter and exit audio trace mode), sample the navigable points of a curve, describe graph points, etc. This module also takes the active browser and operating system into account to ensure the hotkeys and audio behaviors provided are appropriate and feel native. For example, if the operator is using a Mac, the calculator instructs the user to press OPTION+S and OPTION+T to retrieve a summary or activate audio trace, whereas he will be told to press ALT+S and ALT+T on Windows. The calculator also accepts different hotkey modifiers based on the active platform to avoid conflicts with the user's browser. In Windows, the user can press ALT++, ALT+−, and ALT+0, to increase, decrease, and restore the graph's zoom level. On a Mac; CTRL++, CTRL+−, and CTRL+0 adjust the calculator's zoom level.
Finally, if the end user is on an iOS device (such as an iPhone or iPad) and using Apple's built-in screen reader called VoiceOver, the end user is instructed to use the I, J, K, and L keys to navigate (rather than the Arrows) while audio trace is active. This change is used in some embodiments of the present invention because Apple does not currently permit app developers to watch for Arrow keypresses in iOS Safari.
Additional details on how the user interacts with graphs via audio trace in embodiments of the present invention are described further below.
The tone generator program object comprises a module responsible for audio output in embodiments of the present invention. This module utilizes the HTML5 waveform synthesis capabilities found in many popular modern web browsers including Firefox, Chrome, Safari, and Edge. More information regarding the HTML5 audio standard is available at https://www.w3.org/TR/webaudio.
A significant advantage to leveraging HTML5 audio in the calculator is that most modern browsers support the creation and manipulation of simple waveform oscillators including sine, square, triangle, and sawtooth waves through a standardized API described at https://developer.mozilla.org/en-US/docs/Web/API/OscillatorNode. In some embodiments of the present invention, the tone generator preferably generates triangle waves as they sound richer than sine waves, but are not as audibly harsh as square waves.
The tone generator creates three oscillators: the primary triangle wave representing the X/Y position of a point, a secondary triangle wave played at ¼ volume and an octave above the primary wave whenever the independent variable is less-than 0, and a brown noise buffer containing static which is played if the currently selected point cannot be computed, the dependent variable is off-screen, or when the dependent variable is less-than 0.
The following simplified example code demonstrates how an exemplary primary triangle wave is created in some embodiments of the present invention, and connected to the active audio context (which is the user's audio output device):
var audioCtx=new (window.AudioContext∥window.webkitAudioContext)( );
var primaryTriangleWave=audioCtx.createOscillator( );
primaryTriangleWave.type=‘triangle’;
primaryTriangleWave.frequency.value=440; // value in Hz
primaryTriangleWave.connect(audioCtx.destination);
primaryTriangleWave.start( );
In some embodiments of the present invention, an additional sound asset which sounds like a brief “pop” is also loaded and plays when the selected curve intersects another. The tone generator is also responsible for translating a given point's coordinates, based on which variable the evaluator determines is dependent vs. independent and relative to its position in the viewport, into a frequency and panning value which approximates the point's onscreen position.
To illustrate the audiovisual representation of a graphed function in embodiments of the present invention,
Spectrogram for y=1: The spectrogram shown in
Spectrogram for y=−1: The spectrogram shown in
Spectrogram for y=sqrt(x): The spectrogram shown in
Spectrogram for intersecting curves: The spectrogram shown in
The audio graph tracing capability is set forth in
When listening to a graph through audio trace, changes in the stereo field reflect the value of the independent variable, and changes in the pitch reflect the dependent variable. Static will also be heard along with the tone when the dependent variable is less than 0, or by itself if the variable is undefined. In addition, the timbre of the tone changes when the independent variable is less than or greater than 0. A brief “pop” will sound whenever two curves intersect.
For example, consider the equation for a simple sine curve: y=sin(x). When audio trace is active, the tone being played moves from left to right as x increases, and the tone's pitch rises and falls as the y-value changes. In addition, the resonance of the tone morphs once it crosses the X axis, and static (brown noise) plays whenever y moves below 0.
When audio trace is activated, the following activities occur:
After these steps are completed, an announcement that audio trace is activated is spoken, a list of animating variables is given (if applicable), and the coordinates of the selected point are voiced along with its audio representation.
While in audio trace mode, the user can move between computed points within the visible portion of the graph shown on screen using the following commands:
When the user enters Audio Trace Mode, a point is shown on either the graph's origin, or if not on screen, the midpoint of the viewport. In this example, the origin is on the screen, so the point (0,0) is selected and played aloud, as shown in
Navigating to the previous point in
Navigating to the next point in
Navigating to the previous point of interest in
Navigating to the next point of interest in
Navigating to the first point on the left side of the graph in
Navigating to the last point on the right side of the graph in
Navigating back to the origin or the center of the viewport on the graph in
Commands also exist in embodiments of the present invention to report coordinates, descriptions, and branch numbers on demand (all of which are properties of each point):
In some embodiments of the present invention, one may hear an audio representation of either the entire curve or a specific interval, zoom in or out, and adjust playback speed (described in more detail below) with the following commands:
It is also important to note that each point is computed as the user navigates, so the information conveyed in the audio trace is the most up-to-date, accurate information.
When audio trace is deactivated, the following activities occur:
As discussed above, one advantage which comes as part of the calculator's core pedagogical design is that users can adjust factors of an equation (such as a variable with a slider) and see the effect the change has on the curve. Each variable modification causes the evaluator to pass a message to the grapher to recompute (and thus redraw) the curve, and the audio graph module responds in kind and simultaneously recomputes the value of the selected point. This function is shown in
Because of this simultaneous variable adjustment and graph recomputation and redrawing, the user can be confident that the information he hears as he explores the graph is always accurate, even if the graph is changing due to outside influence. This type of real-time graph updating is not available on previous generations of accessible calculator solutions. With older systems, one must enter an equation, and then request a graph to be played. To change a factor in the equation, the user would modify the typed expression (or rewrite it altogether), and command the device to compute a new graph.
When audio trace is active, and an expression contains adjustable factors available via sliders, one can modify their values to hear what effect the changes have on the current equation with the following commands:
For example, take the classic equation for a line: y=mx+b. When audio trace is active, enter slider trace by pressing S, and note that the value of m is set to 1. Pressing H yields an ascending tone which pans from left to right, indicating a line with a positive slope. Next, press the Left Arrow twice until m is −1. Pressing H again now results in a descending tone panning from left to right, indicating a line with negative slope.
One can also animate one or more sliders by selecting them with the Up and Down Arrow keys and pressing P to play or pause them. In a preferred embodiment of the present invention, if any sliders are animating during slider trace, a tone representing the currently selected point will sound and change as the point updates on screen.
Audio playback can be categorized as either single or multi-point. In the single-point system, a newly selected point is sent to the tone generator for immediate translation and playback at a duration of 50 ms. Selection can be performed either by moving the point with the Arrow keys or by tracing the curve with the mouse.
In the multi-point system, invoked when the user presses H to hear the graph or the numbers 1 through 0 to play the respective branches 1 through 10, the audio graph module calculates an array of all navigable points and passes them to the tone generator for playback. Each time a point is played, the following activities occur, as shown in the diagram of
In step 1510, the point's coordinates are translated into pan/frequency values. Each point contains a property called “graph mode,” which is assigned by the evaluator and describes the type of graph being computed. At the time of writing the tone generator understands two graph modes: GRAPHMODE.Y (where Y is the dependent variable and X is its dependent counterpart), and GRAPHMODE.X (where the inverse is true). The pitch of the primary oscillator is computed based on the point's position within the total range of the visible axis multiplied against a base frequency of 500 hz. Similarly, the left-to-right pan value of the independent portion of the point is scaled to the visible portion of the independent axis and assigned a value ranging from −1 (hard left), 0 (center), and 1 (hard right).
In step 1520, required oscillators are muted or unmuted, depending on the values of the dependent and independent variables (see above for details).
In step 1530, oscillator playback is started as necessary.
In step 1540, the selected point is visually highlighted on screen. This is shown at point 1540A of
In step 1550, a timer is set to either move to the next point in the queue or terminate playback. The timer's duration ranges between 12 and 150 ms (dependent on the playback speed selected by the user).
In step 1560, the index of the queue is incremented.
In step 1570, playback continues as described until the end of the array is reached, at which point active oscillators are muted.
Although playing and synchronizing a series of graph points has been done for specialized applications and hardware (including the Orion TI-84 Plus Talking Graphing Calculator, ViewPlus Accessible Graphing Calculator for Windows-based computers, and the Graph-It program for the Braille 'N Speak note taker), no web-based calculator application has developed such audio navigation and playback capabilities for mobile and other networked computing devices. Embodiments of the present invention described in this specification encompass accessibility standards and criteria provided by WAI-ARIA and HTML 5 audio synthesis. Compared to traditional cross-platform development efforts, embodiments of the present invention allow the same code to run on all supported devices with minimal modification and configuration as described in this specification. Furthermore, embodiments of the present invention that feature compliance with the active and relatively new technologies of the WAI-ARIA and HTML 5 web audio standards create an innovative, screen reader accessible, interactive, cross-platform mathematical graphing calculator tool.
When audio tracing, the user moves in equal increments and receives audible feedback scaled to the active viewport at each point, thus providing an accurate picture of the graph's appearance. Note: further details are given in the previous section.
For example, a basic parabola such as y=x{circumflex over ( )}2 is audibly represented as a continuous tone whose pitch increases, decreases, and increases again. A curve which contains discontinuous sections, on the other hand, such as y=sin(sqrt(x)) is represented as an increasing and decreasing tone interrupted periodically by static (indicating no computed value).
While previous generations of audio graphing systems can represent continuous curves (such as the parabola), they cannot interpret discontinuous equations and will often throw errors or entirely skip the non-evaluable points.
When listening to a curve, static is heard alongside the tone when the dependent variable is less-than 0, and an overtone one octave above plays when the independent is less-than 0. A brief “pop” will sound when an intersection is reached. These cues provide additional points of orientation to the audible graph user.
Static+graph tone=dependent less than 0
Overtone+graph tone=independent less than 0
Pop=POIs designated as intersections
Static=function undefined in that region
In some embodiments of the present invention, the audio trace of the graph plays static to indicate the dependent variable is less-than 0, like prior graphing calculator applications (for example, the ViewPlus Accessible Graphing Calculator). Thus, if a user listens to audio playback without the benefit of headphones or stereo speakers, she could gauge whether the current point is above or below the Y axis, but nothing more. The method in which embodiments of the present invention modify the tone itself when the independent variable is less-than 0, however, provide an advantage of being able to convey complete information about a point's quadrant, and thus refine the user's understanding of the graph layout. In addition, other systems do not audibly indicate intersections which is done by embodiments of the present invention.
As one moves between points, the X/Y coordinates are spoken, along with a description of the point (if a POI), and the point's branch number (if a new branch has been selected). Individual properties of a point can be accessed on demand with intuitive commands such as X to speak the X coordinate, Y to speak the Y coordinate, B to voice the branch number, and T to describe the type of point (intercept, maximum, minimum, or intersection). Unlike known graphing calculators in the prior art, the audio trace system described above as implemented in embodiments of the present invention allow one to move through points of a graph in real-time.
Along with providing interactive graph exploration, the user can also request that a verbal graph summary be provided without requiring the need to enter audio trace mode by pressing ALT+S (or OPTION+S on a Mac) when the focused expression has a graph
The verbal graph summary information includes details such as the number of branches, points of interest, and their locations (given as X/Y coordinate pairs). This summary information implemented in embodiments of the present invention, unlike previous generations of graphing calculators, provide these metrics on demand.
In some embodiments of the present invention, a Braille translation system, also known as Abraham, may be used to translate mathematical expressions entered by a user in a typeset format (such as LaTeX) to Braille and vice versa in certain supported environments, set forth below:
The translation system itself functions on any device capable of executing JavaScript code. For practical applications which also accept and deliver input and output from and to a user, the following device and operating system combinations have been used in embodiments of the present invention:
For the above devices used in embodiments of the present invention, the user should disable any active translator or change the input and output translation tables to computer Braille to ensure that the device does not attempt to interpret user input as contracted literary Braille. The methods to activate computer Braille mode used in some embodiments of the present invention are documented on the Desmos accessibility page at https://www.desmos.com/accessibility/. Teachers and students using the Braille translation system used in embodiments of the present invention may use any Braille display that the above screen readers support.
Design Overview
The Abraham Braille translation system set forth in embodiments of the present invention is written in a combination of standard JavaScript, a superset known as TypeScript (https://www.typescriptlang.org/) which enforces type safety, and JISON (https://zaa.ch/jison/), which is a JavaScript-compatible system for designing language parsers based on syntax similar to FLEX/BISON (used in compiler construction).
Components
Abraham defines rules for understanding three languages: LaTeX, Nemeth, and UEB. Each language consists of the following components:
An example Printer Data Flow from LaTeX to Nemeth for LaTeX input: y=sin(x) used in some embodiments of the present invention is as follows:
After running through JISON parser, an initial LaTeX tree becomes:
The initial LaTeX tree is then normalized to:
Tree with 6 nodes is passed to printer
Printer Processes item {type: ‘terminal’, value: ‘y’ }
Output value becomes Y
Printer processes item {type: ‘terminal’, value: ‘=’}
Output value becomes Y .K
Printer processes item {type: ‘command’, value: ‘\\sin’}
Output value becomes Y .K SIN
Printer processes item {type: ‘terminal’, value: ‘(’}
Output value becomes Y .K SIN (
Printer processes item {type: ‘terminal’, value: ‘x’ }
Output value becomes Y .K SIN (X
Printer processes item {type: ‘terminal’, value: ‘)’}
Output value becomes Y .K SIN (X)
Printing completes.
UNICODE Braille module changes ASCII input Y .K SIN (X) to UNICODE Braille
UNICODE Braille is returned as result's value.
An example Printer Data Flow from Nemeth to LaTeX used in some embodiments of the present invention, for the below UNICODE Braille input is as follows:
UNICODE Braille
UNICODE Braille module changes UNICODE to ASCII Y .K SIN (X)
After parsing Nemeth string, initial JISON parsed tree becomes:
Value is passed to printer.
Printer begins processing the tree with 7 nodes.
Printer processes item {type: ‘terminal’, value: ‘y’ }
Output value becomes y
Printer processes item {type: ‘comparison’, value: ‘=’}
Output value becomes y=
Printer processes item {type: ‘command’, value: ‘\\sin’}
Output value becomes y=\sin
Printer processes item {type: ‘terminal’, value: ‘ ’ }
Output value becomes y=\sin
Printer processes item {type: ‘terminal’, value: ‘(’}
Output value becomes y=\sin \left(
Printer processes item {type: ‘terminal’, value: ‘x’ }
Output value becomes y=\sin \left(x
Printer processes item {type: ‘terminal’, value: ‘)’}
Output value becomes y=\sin \left(x\right)
Printer completes processing and returns y=\sin \left(x\right)
Another example Printer Data Flow from LaTeX to UEB used in some embodiments of the present invention, for the LaTeX input y=\sin(x) is as follows:
Parser returns JISON tree:
Tree is passed to normalize LaTeX module which returns:
Normalized tree is passed to printer.
Printer processes tree with 6 nodes.
Printer processes item {type: ‘terminal’, value: ‘y’ }
Output value becomes Y
Printer processes item {type: ‘terminal’, value: ‘=’}
Output value becomes Y“7
Printer processes item {type: ‘command’, value: ‘\\sin’}
Output value becomes Y“7SIN
Printer processes item {type: ‘terminal’, value: ‘(’}
Output value becomes Y“7SIN”<
Printer processes item {type: ‘terminal’, value: ‘x’ }
Output value becomes Y“7SIN”<X
Printer processes item {type: ‘terminal’, value: ‘)’}
Output value becomes Y“7SIN”<X”>
Processing completes.
UNICODE Braille module changes ASCII input Y“7SIN”<X”> to UNICODE Braille
UNICODE output is returned as value property of result object.
Another example Printer Data Flow from UEB to LaTeX for the UEB input listed below is as follows: UNICODE UEB:
UNICODE Braille module changes UNICODE Braille input of to ASCII output Y“7SIN”<X”>
JISON parser returns tree:
Printer begins processing tree with 7 nodes.
Printer processes item {type: ‘terminal’, value: ‘y’ }
Output value becomes y
Printer processes item {type: ‘terminal’, value: ‘=’}
Output value becomes y=
Printer processes item {type: ‘command’, value: ‘\\sin’}
Output value becomes y=\sin
Printer processes item {type: ‘terminal’, value: ‘ ’ }
Output value becomes y=\sin
Printer processes item {type: ‘terminal’, value: ‘(’}
Output value becomes y=\sin \left(
Printer processes item {type: ‘terminal’, value: ‘x’ }
Output value becomes y=\sin \left(x
Printer processes item {type: ‘terminal’, value: ‘)’}
Output value becomes y=\sin \left(x\right)
Processing completes, and output is returned as value property of result object.
Other Components
In addition to each language-specific item, the translator relies on the following general components in embodiments of the present invention:
General Translation Pipeline
At a high level, the Braille translator used in embodiments of the present invention expects a string to translate along with a set of optional parameters which effect translation. Output is a data object containing value and warnings parameters, or an error parameter (described below).
) // returns { value:
}
) // returns { value: ‘1 + 1’, warnings: [ ] }
}
, {strict: true})
Note: the “strict” parameter defines whether the library preprocesses input using a permissive lexer. The permissive lexer attempts to add opening and closing symbols to otherwise invalid Braille input. The permissive lexer will be used automatically unless the user supplies {strict: true} as part of the second options parameter. In some embodiments of the present invention, strict or permissive lexing applies when translating Braille codes to LaTeX as the calculator component already utilizes the basic LaTeX correctness checking which is part of the MathQuill open-source component provided in some embodiments of the present invention.
When translating Braille to LaTeX, the user may also supply a list of keywords which, when encountered, will be written as LaTeX operator names. For example:
More details regarding LaTeX operator names, as well as the list of default keywords, are described in the “Public API” section below.
Nemeth/UEB to LaTeX
Example with Permissive Lexing
Calling Abraham.nemethToLatex(‘’);
Example with Strict Lexing
Calling Abraham.nemethToLatex(, {strict: true}); // note: this is an incomplete fraction and will not parse
Note: if a complete fraction was passed to nemethToLatex( ), the parsing and printing steps would be identical to permissive lexing except that the default JISON lex( ) method would be used.
LaTeX to Nemeth/UEB
Example LaTeX to Nemeth Data Flow
CallingAbraham.latexToNemeth(‘y=\\sin(x)’); // note the \must be escaped
UNICODE Braille is returned as result's value.
Public API
Abraham can be referenced as either a universal JavaScript Module (UMD) or as a global object.
If using the UMD form:
Translating Nemeth to and from LaTeX:
) // returns { value:
}
Translating UEB to and from LaTeX:
) // returns { value: ‘1+1’,
}
nemethToLatex and uebToLatex expect their input to be six-dot Unicode Braille Characters.
nemethToLatex, latexToNemeth, uebToLatex, and latexToUeb return Result objects, which either have a value property, or an error property.
The value property of latexToNemeth and latexToUeb Result is a Unicode Braille String, and the value property of latexToNemeth and latexToUeb Result is a LaTeX string.
When a parse error occurs, the error property of the Result of these methods gives the location of the place where the error occurred in the following format:
{error:
{location: {first_line: 1, last_line: 1, first_column: 0, last_column: 1}}
The error property also has an_diagnostic field with information that's useful for developers for understanding parse errors, but the contents of this property may change and are not part of the public API.
Strict and Permissive Translation
By default, nemethToLatex and uebToLatex will insert missing delimiters like open or close fraction symbols, and open or close radical symbols. This is useful because it allows translating to LaTeX as a user types, even before the expression is complete, and because users may sometimes make mistakes in entering these delimiters.
The result of nemethToLatex and uebToLatex has a warnings property that lists each location at which a missing delimiter was inserted.
For example, the following input is missing a “close fraction” symbol:
)
This behavior can be disabled (in which case missing delimiters will become a parse error) by passing {strict: true} as a second argument to the translators.
{strict: true})
LaTeX Operator Names
LaTeX includes the \operatorname keyword to allow a user to define custom commands. Embodiments of the present invention use this ability to define functions which users can input, even if they are not part of the limited LaTeX standard. For example, a user could write y=sin(x) (which is valid LaTeX), or y=arcsin(x) (which is technically not valid LaTeX). MathQuill transforms the second example to y=\operatorname{arcsin}(x), which is valid LaTeX.
Because neither Nemeth nor UEB do not define an analogous syntax for custom function names, only the value of the custom operatorname is printed when translating LaTeX into Braille. For example:
} AKA FLOOR (X)
In order to translate operator names from Braille to LaTeX, the Braille translator disclosed in embodiments of the present invention uses a list of expected operators to watch for. An extensive list encompassing values used in the calculator program set forth in embodiments of the present invention is included with the library. Developers can overwrite the internal definitions with their own by adding the operatorNames property to the options parameter passed to the translator. The translator expects this value to be an array of strings.
Default Operator Names
Braille Cell Encodings
There are also methods for translating between Braille ASCII (a cell-by-cell encoding of six dot Braille cells into a subset of ASCII), and Unicode Braille cell characters:
) // returns ‘#A’‘6#A’
) // returns
Expanded Braille ASCII is a similar encoding to Braille ASCII intended for eight dot Braille cells (very common on electronic Braille displays such as the HumanWare BrailleNote Touch). A primary difference is that it encodes both upper case and lower case letters as single Braille cells (in Braille ASCII, upper case letters are represented with digraphs: a comma followed by a letter).
UnicodeBraille.coerceToSixDotCells is useful for taking input in a variety of formats and coercing it to six-dot Unicode Braille characters that the translator natively understands. It will convert either Braille ASCII or Expanded Braille ASCII (ignoring case) to six dot Unicode Braille characters, and will also convert any eight dot Unicode Braille characters to the six-dot subset by stripping the bottom pair of dots.
Live Translation
Embodiments of the Braille translator disclosed in this specification include two example web pages demonstrating how to integrate it into a real-world web application: the first presents buttons to toggle between either Nemeth or UEB, an edit box which accepts Braille input, and a standard input field for entering a typeset mathematical expression, as shown in
The second example takes the first example one step farther and in this embodiment of the invention, adds an instance of a browser or mobile app-based graphing calculator to the page as shown in
Braille Normalization and Monitoring
After each keystroke entered in the Braille input field (referenced below as “elt”), the contents of the input box are normalized to Unicode Braille characters. Based on the mode flag, the normalized characters are then translated from either Nemeth or UEB to LaTeX. Finally, the translated LaTeX string is passed to the MathQuill input (referenced below as “mq”) in the following code example:
Mapping input to Unicode is useful to avoid confusion from interpreting Braille ASCII based on the ASCII symbols rather than the Braille cells they represent (e.g. a screen reader might read an open fraction symbol as “question mark”).
Also note the technique for preserving the selection and cursor position when updating the value property of the input.
Typeset Mathematical Expression Monitoring
MathQuill's “edit” event listener is hooked to watch for changes to the field's LaTeX contents and to update the Braille input field accordingly as in the following code snippet:
The above may be implemented into four-function and scientific calculator programs that employ the typeset mathematical expressions used in embodiments of the present invention, in addition to being integrated into the Internet-enabled audio-visual graphing calculator described here as embodiments of the claimed invention.
Tactile Graphics
This section outlines the functionality of the tactile graphic support of the Desmos graphing calculator. Embodiments of the invention described herein include methods to create an embossable and/or tactile image, complete with Braille labels, from a web application.
A tactile graphic is an image that has been rendered with a Braille embosser which can be explored by touch. Such diagrams are rendered at an extremely low resolution compared to standard image files and must be adapted for optimal legibility to the blind or low-vision reader. These adaptations are described below and are based on recommendations described at the Braille Authority of North America's reference for tactile graphic production: http://brailleauthority.org/tg/web-manual/. Alternatively, tactile graphics may be rendered using techniques similar to those used in embodiments of the present invention described herein, with Braille displays such as refreshable Braille displays containing 1,500 to 12,000 pixels, including but not limited to such known graphic braille displays as the Graphiti from Orbit Research, and DV-2 (from KGS) with 1,536 pixels, Hyperbraille with 7,200 pixels and TACTISPLAY Walk/Table (from Tactisplay Corp.) with 2,400/12,000 pixels respectively.
More on tactile graphics can be found at https://en.wikipedia.org/wiki/Tactile_graphic.
Some embodiments of the present invention utilize either or both of two classes of Braille embossers which are supported: (1) the Romeo 60 and Juliet 120, created by Enabling Technologies, and (2) the Tiger VP Max, created by View Plus, Inc. For these devices, two paper size options are available: 8.5×11 and 11.5×11 inches. These embosser models and paper sizes are popular in many schools across the United States, although embodiments of the present invention may utilize and support other Braille embosser devices or other tactile graphic devices. Each of the two specific embossers described above include unique characteristics which are relevant to the following discussion:
(1) The Enabling Technologies embossers support a single dot height. That is, they can produce only one level of dot firmness.
(2) The VP Max is capable of embossing at seven levels of dot firmness (controlled by setting each pixel to a predefined grayscale value—0, 37, 73, 109, 145, 182, 219, or 255). It also supports greater resolution compared to the Enabling Technologies embosser, and embodiments of the present invention scale automated drawing routines to compensate.
End-User Process for Producing Tactile Graphics
The process for creating a tactile graphic within embodiments of the present invention discussed above, where the tactile graphic expresses the output of a graphing calculator is as follows:
1. Create a graph on the Desmos Graphing Calculator.
2. Click the Share menu.
3. On the Share menu for which an exemplary on-screen display is shown in
4. Once the Export Image screen is shown, for which an exemplary on-screen display is shown in
5. Select the desired Braille code (Nemeth or UEB).
6. Click “Download PNG.”
7. If printing to a Tiger embosser, the user may then immediately print the downloaded file from an application such as Paint. If the user is accessing an Enabling Technologies embosser, he must open the image in the Enabling Fire Bird software which communicates with the printer.
8. Optionally, the user may elect to download a list of Braille equations which may be sent to an embosser or stored on a refreshable Braille device.
In some embodiments of the present invention, the combination of the graphic and accompanying expression list provides the student a complete hard-copy picture of a set of equations and their resulting output.
Module Overview
Embosser and Page Dimensions
In one embodiment of the invention, the calculator presently supports four combinations of embosser and paper sizes in one embodiment of the invention, although other types of paper sizes and embossers are contemplated in embodiments of the present invention:
The calculator's graphing module (known internally as the grapher) produces images given a set of width and height values (expressed in pixels). One embodiment of the invention includes an optimal set of dimensions which are passed to the grapher. This and other embodiments utilize a precise 1 to 1 correspondence between a rendered pixel and a Braille dot on the page. Misalignment that does not have such a 1 to 1 correspondence between a rendered pixel and a Braille dot on the page can result in undesired image stretching and distorted Braille characters. Furthermore, the internal data structure for each size option includes data pertaining to desired top, left, right, and bottom margins. While not used directly by the grapher, the margins are used when creating an image to ensure there is sufficient space for a student to add labels, page numbering, or binding to the hard-copy output.
[Internal—this is the code option]
Values for the size options are as follows:
[Internal—this is the non-code option]
Values for the size options are as follows:
VP Max 8.5×11 in
VP Max 11.5×11 in
Enabling Technologies 8.5×11 in
Enabling Technologies 11.5×11 in
Grapher
The grapher is the primary module responsible for transforming a set of mathematical equations into an image. Its screenshot( )method, which is used to obtain raw image data from the calculator, draws the image to a supplied CanvasRenderingContext2D object (from which dimensions are also derived). The method also accepts additional arguments for fine-tuning the resulting image, including Braille size options as defined above. Discussion of the grapher below is limited specifically to how it has been adapted to render Braille in some embodiments of the present invention discussed herein.
Canvas Braille
The Canvas Braille module coordinates all drawing operations for tactile graphics and is described in the greatest detail below.
Low-Level Drawing Functions
Before describing the higher level aspects of what is drawn to the canvas, lower-level setup and drawing functions will be described.
Braille Options
Before an image can be drawn, the Canvas Braille module of some embodiments of the present invention initializes several options depending on the selected embosser model and paper size:
Options are initialized and referred to internally as brailleOpts based on the following functions:
The following two functions are used in some embodiments of the present invention to determine whether the selected device can draw grid lines, as well as which RGB values to use when drawing them.
Drawing Braille Characters
In some embodiments of the present invention, it is not necessary to render separate streams for Braille characters and a graph image, so Braille character data and graph image data may be composited together into a single graphic. Thus, in some embodiments of the present invention, the calculator can “draw” a sequence of Braille characters at any given (x, y) position on the canvas. The following functions are called to determine if the specified area of a canvas can be written to, and if so, draws the Braille dots at the specified coordinates, using the predefined dot spacing and cell padding values derived during the initialization routine. Note that characters are expected to be 6-dot UNICODE Braille characters as used by the Abraham translation library described elsewhere. Note, too, that CTX is a short-hand type alias for CanvasRenderingContext2D.
Drawing Lines, Arrows, and Fill Patterns
The Canvas Braille module uses the following functions for drawing vertical and horizontal lines (for grid lines, axis, and hash marks), arrows (if the user has configured the calculator to render arrow marks on the X and Y axis), and a dotted fill pattern (for inequalities if the user's Braille embosser supports only one dot height) at the specified (x, y) locations and the supplied RGB color values. Note the CanvasRenderingContext2D.fillRect( ) method is used to draw each segment to account for anti-aliasing problems.
Tactile Graph Drawing Procedure
The primary method of the Canvas Braille module, referred to as createBraillelmage, takes the following arguments from which to construct a tactile diagram:
Creating the final image requires that several intermediate pictures be created first:
Drawing each layer requires that different settings in the grapher be modified (such as disabling the grapher's routine to draw grid lines, print labels, etc which have Braille-friendly analogs in this module). Thus, the existing grapher options are stored beforehand and reapplied after drawing completes.
Next, each canvas listed above is created and set to the width and height supplied in the Braille size option map.
After the canvas elements are created, two calls are made to grapher.screenshot( ): the first retrieves an image with only graph curves and no axis markings or gridlines:
A second screenshot is then retrieved with only embosser-compatible shading (if the graph contains inequalities):
Next, the composite context is set to all white before labels are drawn to it:
compositeCtx.fillStyle=‘white’;
compositeCtx.fillRect(0, 0, elt.width, elt.height);
The following comes from section 6.6.2.3 of the tactile graphics standards document linked earlier: “The x-axis line (horizontal line) should be labeled at the right end of the axis line or, if space does not permit, the x can be placed above the x-axis line.” This section also states: “The y-axis line (vertical line) should be labeled at the top of the axis or, if space does not permit, the y can be placed to the left of the y-axis line.”
These rules do not account for custom labels that a user may add to the X and Y axis of a graph, so if either exist, they will be printed in lieu of “X” or “Y” respectively. In some embodiments of the present invention, extra padding is added around the axis labels to make sure they stay legible when curves or fill patterns get close to them. Axis numbers don't need as much padding as axis labels because the axes already have extra padding that covers up fill in the region where axis numbers are placed, and axis numbers can be removed when curves pass too close to them.
Next, if either X or Y axis is shown (dependent on user preference), a set of X and Y steps to print are computed and, if there is sufficient free space available on the braille labels context, written to it. During printing, it may be noted whether the step value being printed is positive or negative due to the following recommendation (also from section 6.6.2.3 of the tactile graphics standard): “On the x-axis, dots 123 of the first digit of the number should align with the grid line or tick mark. If a minus sign is present, it should be placed to the left of the grid line or tick mark. On the y-axis, dots 25 of the number should align with the grid line or tick mark.” Placement of the drawn cells is adjusted as necessary to satisfy both requirements.
Next, if both X and Y axis lines have been drawn, the label for the graph origin is printed based on the following recommendation (also from section 6.6.2.3 of the tactile graphics standard): “Label the point of origin as shown in print (number 0 or letter O). Place below x-axis and to the left of the y-axis. If the point of origin cannot be determined in print and there are numbers on the axes, then the origin will be the number 0. If there are no numbers, then the origin is the letter O.”
Once labels are printed, a series of white grid line borders (4 pixels wide or high depending on orientation) are drawn. Applying the borders adds a slight but important buffer between the grid and a graph segment which may be drawn on top of it. This distinction is especially important for the Enabling embossers which support only a single dot height, but is applied for both devices as the buffer improves tactile legibility of the resulting diagram.
Next, axis borders are drawn. The axis borders are used to make sure that fill patterns do not obscure the axes. They will cover the portion of filled regions which would have otherwise obscured the axis lines. Unlike the grid line borders, the X and Y axis borders are 8 pixels wide or high to encompass tick marks.
Next, the individual grid lines and tick marks are drawn through a series of two loops (one per axis). Depending on the progressive value of the x and y step through each loop, the desired line type and color is added to the grid lines canvas using the line drawing routines defined earlier.
Following the grid lines, the X and Y axis lines are drawn, followed by axis arrows (if the user opts to have them shown).
Once all curves, labels, and grid lines are drawn, and if the selected embosser requires a fill pattern to represent inequalities, the pattern is drawn in areas which still contain free space.
The composite canvas is now layered in the following order:
Finally, with all drawing operations completed, the original grapher settings are restored, and the method returns.
Exporting Braille Equations
Supplemental to exporting embossable images, the Desmos four-function, scientific, and graphing calculators include the ability to render a copy of the expression list as a file suitable for storage on a refreshable Braille device or output to an embosser. To obtain a Braille copy of the expression list in the four-function or scientific calculator:
1. Create a series of equations.
2. As shown in
3. Select the “Download Braille Equations” button.
To export Braille equations from the graphing calculator:
1. Create a series of equations.
2. As shown in
3. In the size option dropdown, choose one of the Braille options.
4. Click the “Download Braille Equations” button.
In both cases, in an embodiment of the present invention, the expression list is sent to an internal library module called “export-braille-equations” which loops through the given set of expressions, translates them into Nemeth or UEB (including evaluations), and returns an ASCII Braille file to the browser. The library also renders complementary constructs (like the start and end of folders, images, and tables) which may be present in a set of graphed expressions.
While the present invention has been particularly described with respect to the illustrated embodiments, it will be appreciated that various alterations, modifications and adaptations may be made based on the present disclosure and are intended to be within the scope of the present invention. While the invention has been described in connection with what are presently considered to be the most practical and preferred embodiments, it is to be understood that the present invention is not limited to the disclosed embodiment but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the scope of the underlying principles of which are illustrated by the examples set forth herein.
This application is a continuation-in-part application of pending U.S. continuation application Ser. No. 16/505,086 titled “Internet-Enabled Audio-Visual Graphing Calculator”, filed Jul. 8, 2019; which claims priority to U.S. non-provisional application Ser. No. 15/852,688 titled “Internet-Enabled Audio-Visual Graphing Calculator”, filed Dec. 22, 2017 and now issued as U.S. Pat. No. 10,346,135; which claims priority to provisional application 62/451,552 titled “Internet-Enabled Audio-Visual Graphing Calculator”, filed Jan. 27, 2017. This application also claims priority to U.S. provisional patent application 62/956,223, titled “Internet-Enabled Audio-Visual Graphing Calculator”, filed Dec. 31, 2019. This and all other extrinsic references referenced herein are incorporated by reference in their entirety.
| Number | Date | Country | |
|---|---|---|---|
| 62451552 | Jan 2017 | US | |
| 62956223 | Dec 2019 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 15852688 | Dec 2017 | US |
| Child | 16505086 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 16505086 | Jul 2019 | US |
| Child | 16740372 | US |
