NSF Award
2315354
Computer Aided Design (CAD) is a multi-billion dollar industry responsible for the digital design of almost all manufactured goods. It leverages parametric modeling, which allows dimensions of a design to be changed facilitating physically-based optimization and design re-mixing by non-experts. But CAD’s potential is diminished by the difficulty of creating parametric models: in addition to mastering design principles, professionals must learn complex CAD software interfaces. To promote effective modeling strategies and creative flow, design educators advocate freehand drawing as a preliminary step to parametric modeling. Unfortunately, CAD systems do not understand these drawings, so designers must re-create their entire design using complex CAD software. This research project explores the question, "Is it possible to automatically convert freehand drawings to parametric CAD models?" By leveraging the visual vocabulary shared by drawing and CAD modeling, this project will develop a system to translate from the natural language of drawing to the formal language of CAD. This technology will increase the productivity of professional CAD designers across multiple industries and make CAD modeling accessible to more people without extensive training in confusing software interfaces.<br/><br/>To handle drawings as input, the researchers will treat them as timestamped sequences of strokes, allowing them to cast the problem as one of machine translation from drawing stroke sequences to CAD program token sequences. Drawing strokes are grouped into coherent drawing operations that are correlated with CAD modeling strategies (e.g. first drawing construction lines and simple primitives shapes, then refining). The researchers propose to extract these drawing operations as an intermediate representation, which helps disambiguate between the (potentially infinitely) many programs which can represent a single shape. Performing this extraction and then producing CAD programs are complex search problems; the researchers will leverage novel deep neural networks to guide the search. They will gather a paired (drawing, CAD program) dataset from professional designers to help develop these networks. They will also develop learning algorithms that do not require such ground-truth paired data. Finally, they will develop metrics to assess CAD programs produced by the system, which will be used both to evaluate the system's efficacy and to guide the program search process.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
