The present disclosure relates generally to systems and methods for data visualization and modification in an immersive three-dimensional (3-D) environment (e.g., via virtual reality). More specifically, the present disclosure relates to analyzing and modifying data (e.g., computer code) relationships via virtual reality (VR) tools.
Data visualization and modification is prevalent in the modern technological world. For example, computer programming (e.g., computer coding) involves analyzing and modifying lines of code (e.g., programming language) to maintain, enable, or change processing methods, where each line of code may include various relationships (e.g., process relationships) with other lines of code. Many digital technologies utilize computer code to run various digital processes associated with data storage, data analyzation, financial transactions, gaming, and the like. As programming continues to evolve, coding may become increasingly complicated. For example, computer programs may continue to increase in size (e.g., more code) and increase in complexity. Unfortunately, relationships between various portions of a data set (e.g., computer code) may be difficult to perceive with traditional data visualization tools. Accordingly, an improved data visualization (e.g., computer programming) environment may be desirable.
In a first embodiment, A computer programming system includes a processor configured to receive computer code in a textual format, and cause a 3-D visual representation of a portion of an immersive 3-D environment to be displayed via a 3-D visual instrumentation, where the 3-D visual representation comprises a visualization of the computer code in the textual format. The system also includes the three-dimensional (3-D) visual instrumentation, communicatively coupled with the processor, and configured to receive the 3-D visual representation, and to present the 3-D visual representation. The system also includes a motion detection instrumentation communicatively coupled with the processor, where the motion detection instrumentation is configured to receive a motion input, and where the 3-D visual representation within the immersive 3-D environment is adjusted based on the motion input. The system also includes an input device communicatively coupled with the processor, where the input device is configured to enable modifications to the computer code, and where the processor is configured to incorporate the modifications to the computer code into the immersive 3-D environment.
A second embodiment includes a tangible, non-transitory, machine-readable medium, having machine-readable instructions for displaying computer programming logic in an immersive three-dimensional (3-D) environment. The machine-readable instructions include instructions to receive, via one or more processors, computer programming logic. The machine-readable instructions also include instructions to generate, via the one or more processors, a 3-D representation of the computer programming logic. The machine-readable instructions also include instructions to present the 3-D representation of the computer programming logic to 3-D visual instrumentation communicatively coupled with the one or more processors, such that the 3-D representation of the computer programming logic is presented by the 3-D visual instrumentation in a 3-D visual representation of a portion of the immersive 3-D environment. The machine-readable instructions also include instructions to receive, from motion detection instrumentation, a motion input from a user. The machine-readable instructions also include instructions to adjust and present, via the one or more processors and based on the motion input from the user, a perspective of the user within the immersive 3-D environment by changing the 3-D visual representation within the immersive 3-D environment, such that a segment of the 3-D representation of the computer programming logic that is not viewable in a first iteration of the 3-D visual representation is viewable by the user in a second iteration of the 3-D visual representation after the adjustment. The machine-readable instructions also include instructions to present relationship data indicative of relationships between portions of the 3-D representation of the computer programming logic to the 3-D visual instrumentation, such that the 3-D visual instrumentation presents an indication of the relationships between the portions of the 3-D representation of the computer programming logic in the 3-D visual representation.
In a third embodiment, a method includes receiving, via one or more processors, computer code. The method also includes generating, via the one or more processors, a three-dimensional (3-D) computer code representation of the computer code, and determining, via the one or more processors, a relationship between portions of the 3-D computer code representation. The method also includes generating viewable relationship data corresponding to the 3-D computer code representation, and presenting visualization data that causes display of the 3-D computer code representation and the viewable relationship data in an immersive 3-D environment. A wearable virtual reality headset communicatively coupled with the one or more processors, and having a 3-D visual instrumentation and a motion detection instrumentation, is configured to display the 3-D computer code representation and the viewable relationship data in a 3-D visual representation of a portion of the immersive 3-D environment. The motion detection instrumentation is configured to enable a user of the virtual reality computer programming system to adjust a perspective of the user within the immersive 3-D environment by changing the 3-D visual representation displayed by the 3-D visual instrumentation. An input device, communicatively coupled with the one or more processors, is configured to enable the user to add to, delete from, or otherwise modify the 3-D computer code to generate a modified 3-D computer code representation disposable and viewable in the immersive 3-D environment.
These and other features, aspects, and advantages of the present disclosure will become better understood when the following detailed description is read with reference to the accompanying drawings in which like characters represent like parts throughout the drawings, wherein:
One or more specific embodiments of the present disclosure will be described below. In an effort to provide a concise description of these embodiments, all features of an actual implementation may not be described in the specification. It should be appreciated that in the development of any such actual implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which may vary from one implementation to another. Moreover, it should be appreciated that such a development effort might be complex and time consuming, but would nevertheless be a routine undertaking of design, fabrication, and manufacture for those of ordinary skill having the benefit of this disclosure.
Present embodiments are directed toward systems and methods for facilitating data visualization and modification in an immersive three dimensional (3-D) environment (e.g., via virtual reality). Specifically, systems and methods in accordance with present embodiments may be directed to virtual reality tools that facilitate computer programming in the immersive 3-D environment. “Immersive 3-D environment” used herein may refer to an environment having a 3-D visual representation in computer graphics viewable by a user. “3-D visual representation” may correspond with a given viewable portion (e.g., window, segment, or slice) of the immersive 3-D environment, and may correspond with a perspective of the user within the immersive 3-D environment. In other words, while the immersive 3-D environment may include a 360 degree environment, the 3-D visual representation may include a portion (e.g., slice) of the 360 degree environment. For example, the user may be situated within the immersive 3-D environment and, at any time, the user may perceive a given 3-D visual representation based on the location and perspective (e.g., orientation) of the user.
In accordance with the present disclosure, the user may be capable of modifying the 3-D visual representation by adjusting his or her position and/or perspective (e.g., through movement of his or her head), such that motion detectors detect the user's movement, and adjust the user's position and/or perspective (e.g., the 3-D visual representation) within the immersive 3-D environment. The user may also be capable of modifying data or other features displayed in the immersive 3-D environment and perceived by the user in the 3-D visual representation. Specifically, the user may be capable of modifying computer code (and corresponding viewable relationships between portions of the computer code) in the immersive 3-D environment, in accordance with the description below.
Systems configured to enable the immersive 3-D environment may include visual instrumentation, such as virtual reality goggles. However, other visual instrumentation may also be used, such as glasses with specially designed lenses that enable the perception of a 3-D visual representation (e.g., a portion of the immersive 3-D environment) projected from a two dimensional (2-D) screen (e.g., television). In general, the visual instrumentation may include one display that receives a first image or video feed (“first feed”) (e.g., associated with a user's right eye) and a second image or video feed (“second feed”) (e.g., associated with a user's left eye). Alternatively, the visual instrumentation may include a first display that receives the first feed (e.g., associated with the user's right eye) and a second display that receives the second feed (e.g., associated with the user's left eye). The visual instrumentation may also include first and second lenses corresponding with the first and second feeds (and, thus, the user's right and left eyes, respectively), where the first and second lenses are positioned between the user and the display(s). The first and second lenses may be positioned, angled, or otherwise designed (e.g., to include variable thickness, curves, etc.) to focus and/or reshape the first and second feeds, creating a stereoscopic 3-D image (or video) viewed by the user.
Further, the system may include motion sensor instrumentation, which may be incorporated in full or in part with the visual instrumentation (e.g., into a single pair of goggles), or may be standalone instrumentation from the virtual reality visual instrumentation. In general, the motion sensor instrumentation may be a wearable device, such as headgear, a glove, or another type of wearable device, configured to capture and/or track motion of the user (e.g., the user's head). For example, the virtual reality goggles having the visual instrumentation may include the motion sensor instrumentation disposed therein or thereon, and as the user of the virtual reality goggles turns his or her head, the motion sensor instrumentation detects the turning of the virtual reality goggles by way of the user's motion.
A processor of the system may receive, from the motion sensor instrumentation, data indicative of the motion. The processor may adjust the first feed, the second feed, the angle of the lenses, or a combination thereof in real time to shift the 3-D visual representation (and, thus, the user perspective within the 3-D environment), in accordance with the user's motion. In other words, as the user turns his or her head, the user's perspective (e.g., the 3-D visual representation) within the immersive 3-D environment changes (e.g., shifts) in accordance with the user's movement.
The system may also include a memory communicatively coupled with the processor. The memory may store instructions that, when executed by the processor, cause the processor to perform certain acts. For example, as previously described, the processor may execute instructions stored to the memory such that the processor feeds the 3-D visual representation of the immersive 3-D environment to the visual instrumentation of the system. Further, the processor may execute instructions stored to the memory such that the processor adjusts the perspective of the user within the immersive 3-D environment in accordance with the user's motion (e.g., in accordance with the data indicative of the user's motion, detected by the motion sensor instrumentation, and received by the processor from the motion sensor instrumentation). Further still, the processor may execute instructions stored to the memory such that the 3-D visual representation fed to the visual instrumentation includes a data set selected by the user. For example, the data set may include a computer program that the user is analyzing and/or modifying. Specifically, the computer program may be rendered such that computer code is displayed in an X-Y plane of the immersive 3-D environment, while relationships between various portions of the code are displayed as representations along a Z direction of the immersive 3-D environment. However, other renderings are also possible and will be described in detail below.
Further, the system may include an input device (e.g., a data entry input device, such as a keyboard, a digital input screen, or the like). For example, the input device (e.g., data entry input device) may be a keyboard that facilitates the user's manipulation of the immersive 3-D environment and/or data within the immersive 3-D environment. In particular, the input device may enable the user to draft, edit, modify, or otherwise manipulate computer code that is fed to the visual instrumentation described above. As previously described, the data set selected by the user may be a computer program having computer code. By visualizing relationships and dependencies between various portions of code in the Z direction of the immersive 3-D environment (or in other axes of the immersive 3-D environment), which may not otherwise be possible in traditional configurations, the user may more readily modify the computer program via the input device to achieve desired goal(s) or intent(s) of the computer program. The input device (or another input device, such as a joystick or the previously described motion detection instrumentation) may also be utilized by the user to adjust a location or orientation of the user within the immersive 3-D environment (e.g., allowing the user to move closer or further from various virtual objects within the immersive 3-D environment, and/or allowing the user to shift or change the 3-D visual representation, as previously described).
To help illustrate,
In the illustrated embodiment, the 3-D visual instrumentation 12 may be goggles, glasses, or other instrumentation that facilitates visualization of a portion of the immersive 3-D environment (e.g., the 3-D visual representation thereof). Specifically, a processor 20 of the system 10 may execute instructions stored to a memory 22 of the system 10, where execution of one or more of the instructions causes the processor 20 to transmit one or more image or video feeds to the 3-D visual instrumentation 12. It should be noted that, in some embodiments, an additional (e.g., separate) service processor 21 may be included to receive and analyze relationship data (e.g., in XML, or in some other format), and to render the relationship data in a 3-D, as described below.
The 3-D visual instrumentation 12 may be a virtual reality tool, such as virtual reality goggles, that includes one display which receives a first image or video feed (“first feed”) (e.g., associated with a user's right eye) and a second image or video feed (“second feed”) (e.g., associated with a user's left eye). Alternatively, the 3-D visual instrumentation 12 (e.g., the virtual reality goggles) may include a first display that receives the first feed (e.g., associated with the user's right eye), and a second display that receives the second feed (e.g., associated with the user's left eye). The 3-D visual instrumentation 12 may also include first and second lenses corresponding with the first and second feeds (and, thus, the user's right and left eyes, respectively), where the first and second lenses are positioned between the user 18 and the display(s) on which the first and second feeds are produced. The first and second lenses may be positioned, angled, or otherwise designed (e.g., to include variable thickness, curves, etc.) to focus and/or reshape the first and second feeds, creating a stereoscopic 3-D image (or video).
The motion sensor instrumentation 14 may be incorporated into a single piece of hardware that includes the 3-D visual instrumentation 12, or the motion sensor instrumentation 14 may be stand-alone hardware. The motion sensor instrumentation 14 includes any suitable type of motion sensor that facilitates detection of 3-D motion of the user 18. In other words, the motion sensor instrumentation 14 may be capable of detecting motion of the user 18 in, and rotatably about, a first direction 24 (e.g., a Y-spatial dimension), a second direction 26 (e.g., an X-spatial dimension), a third direction 28 (e.g., a Y-spatial dimension), or any combination thereof. More specifically, the motion sensor instrumentation 14 may detect motion of the user's 18 head. As will be described in detail below, the system 10 facilitates correspondence of motion of the user's 18 head with a visual perspective (e.g., 3-D visual representation) within the immersive 3-D environment. Accordingly, as the user 18 turns or otherwise moves his or her head, a perspective of the user 18 (corresponding with the 3-D visual representation) within the immersive 3-D environment changes. In some embodiments, as previously described, the input device 16 may be utilized in conjunction with, or independently from, the motion sensor instrumentation 14 to change the 3-D visual representation, such as by zooming in and out.
The motion sensor instrumentation 14 may be communicatively coupled with the processor 20, such that the processor 20 receives feedback relating to (e.g., data indicative of) the user's 18 motion. The processor 20 may process the data indicative of the user's 18 motion and, upon executing instructions stored in the memory 22, may adjust the first and/or second feeds to the 3-D visual instrumentation 12, the first and/or second lenses of the 3-D visual instrumentation 12, or a combination thereof. In doing so, the processor facilitates a change of the user's 18 perspective (corresponding with the 3-D visual representation) within the immersive 3-D environment provided by the 3-D visual instrumentation. The processor 20 may be capable of changing the user's 18 perspective in real time to correspond with the user's 18 motion, such that the user 18 may predictably move to control his or her perspective within the immersive 3-D environment.
Further, the user 18 may modify data presented in the immersive 3-D environment by way of the input device 16 (e.g., handheld input device). For example, the input device 16 may be communicatively coupled with the processor 20, such that the user 18 may enter commands into the input device 16, and the commands are received by the processor 20. The input device 16 may be a keyboard or another device specially configured for data modification (e.g., computer coding or programming). As previously described, the input device 16 may include voice recognition capabilities such that the user 18 can program via spoken word transcribed into written code by the input device 16. Upon receiving the commands from the user 18 by way of the input device 16, the processor 20 may modify the first and/or second feeds to modify the data (e.g., computer code) in correspondence with the user's 18 commands. For example, the user 18 may edit a line of computer code via the input device 16, the processor 20 may receive the edit from the input device 16, and the processor 20 may cause the edits to be shown in the immersive 3-D environment (e.g., by way of the first and/or second feeds transmitted by the processor 20 to the 3-D visual instrumentation).
In some embodiments, the input device 16 may also be utilized to change a position of the user 18 within the immersive 3-D environment (e.g., a user's perspective within the immersive 3-D environment). For example, the user 18 may enter commands to the input device 16 that causes the user 18 to move toward, or away from, virtual objects (e.g., lines of code) disposed in the immersive 3-D environment (e.g., as a zooming tool). It should be noted that the input device 16 may include a corresponding virtual input device rendering within the immersive 3-D environment that corresponds with the input device's physical proximity to the user 18, such that the user 18 may perceive the input device 16 by way of the virtual input device rendering of the input device 16. In other words, the user 18 may adequately utilize the input device 16 without visualizing the physical input device 16, but rather by visualizing the virtual rendering of the input device 16 within the immersive 3-D environment. Of course, in some embodiments, the input device 16 (or one input device of several input devices) may be of a type that is of limited use to be rendered in the immersive 3-D environment, such as embodiments having voice recognition and/or talk-to-text software.
In accordance with present embodiments, at least one dimension of the immersive 3-D environment may be utilized to show relationships between data or code displayed in the immersive 3-D environment. As described above, either the processor 20, or the service processor 21 (e.g., additional or separate service processor), may receive relationship data (e.g., in XML or a similar format) and may render relationships associated with the relationship data in the immersive 3-D environment (e.g., in the at least one dimension of the immersive 3-D environment). Thus, the service processor 21 (or processor 20) may facilitate the ability to recognize a variety of relationship types via a configuration driven system and to transform that input into a common set of primitives that can then be rendered. Further, the service processor 21 (or processor 20) may include, stored or executed thereon, an algorithm for choosing the exact location of rendered objects based on their relationships between and amongst one another. The algorithm extends traditional force-based rendering algorithms to be able to incorporate overall motion of a world to enable 4-D rendering (e.g., 3-D+time) for dynamic systems.
Further, the method 50 includes detecting (block 54) motion of the user corresponding with the user's desire to change a perspective (e.g., change the 3-D visual representation) within the immersive 3-D environment. For example, the user may be virtually immersed in the immersive 3-D environment, and may physically turn his or her head to change the user's visual (e.g., virtual) perspective within the immersive 3-D environment. A motion sensor may be incorporated into the virtual reality goggles worn by the user (or may be a stand-alone component fixed to the user's head), and the motion sensor may detect when the user moves his or her head. The motion sensor may send data indicative of the motion to a processor. As previously described, another input device (e.g., handheld input device), such as a joystick, a trigger, a switch, voice recognition technology, or the like, may be utilized as a zooming tool, although the motion sensor instrumentation may also be utilized as a zooming tool in conjunction with, or independently from, the handheld or voice recognition input device.
The method 50 also includes adjusting (block 56) the 3-D visual representation, based on the data indicative of the user's motion (and/or instruction via the handheld or voice recognition input device) described above, to change the user's visual (e.g., virtual) perspective within the immersive 3-D environment. For example, after receiving the data indicative of the user's motion (or other input, as described above), the processor may adjust the 3-D visual representation (e.g., the image or video feed[s]) transmitted to the visual instrumentation of the system. The adjustment may cause the 3-D visual representation to correspond with a desired visual perspective of the user within the immersive 3-D environment, where the desired visual perspective of the user corresponds with the user's movement (e.g., the motion of the user's head) or other input (e.g., via a handheld or voice recognition input device).
Further still, the method 50 includes receiving (block 58) data (e.g., code) modification commands via an input device (e.g., keyboard). For example, the user may view data (e.g., computer code and/or relationships) shown in the 3-D visual representation, and may enter additions, deletions, or revisions of the data (e.g., computer code) into the input device. The input device may communicate data indicative of the commands to the processor.
After receiving the data modification commands, the processor, in accordance with the present method 50, may modify (block 60) the data (e.g., code) in the immersive 3-D environment based on the modification commands. Further, the method 50 includes displaying (block 62) the modified data (e.g., code) in the 3-D visual representation provided to the user by way of the visual instrumentation (e.g., virtual reality goggles), assuming the user modifies data (e.g., code) within the 3-D visual representation (e.g., within the portion of the immersive 3-D environment being viewed by the user). Thus, the user is able to perform computer programming in real time within the immersive 3-D environment. As previously described, by incorporating the Z-axis in a way that facilitates observance of process relationships or other relationships between various portions of code displayed in the X-Y plane, the system may allow the user to more readily modify the code displayed in the X-Y plane to incorporate desired process relationships, revise or fix broken process relationships, omit undesired process relationships, and so on (e.g., as compared to traditional coding systems). It should be noted that the three dimensions may be utilized in other manners with respect to computer programming as well. For example, the computer code may be warped or bent in a 360 degree sheet surrounding the user's position within the immersive 3-D environment such that, as the user turns (e.g., via motion of the user's head) within the 3-D environment, the computer code is always tangential to the user's perspective. Other configurations may also be possible, and will be described in detail below.
Turning now to
As shown, data (e.g., computer code 80) may be displayed within the immersive 3-D environment 78. The computer code 80 may wrap circumferentially 360 degrees (or fewer degrees) about the user 18, where the user's 18 visualization of a portion of the code 80 (e.g., via the 3-D visual representation 81) is substantially flat (with slight curvature, as shown). The code 80 may be split along the 360 degrees into several pages of code 80. Within the portion of the code 80 viewable in the 3-D visual representation 81, several lines 82 (e.g., portions, segments) of the code 80 may exist. The lines 82 of code may be related to one another in any number of ways. For example, relationships may include process relationships, master-slave relationships, error relationships or other types of relationships. As a specific example, code associated with a particular sub-routine may have a particular shape with a particular color hovering behind the code in the immersive 3-D environment 78 such that the user 18 knows modification of the code will impact the sub-routine. For example, common colors and/or shapes may indicate a relationship. Additionally or alternatively, certain colors and/or shapes may indicate certain types of relationships (e.g., where a color or shape legend indicates which colors/shapes correspond with which types of relationships). The relationships may be perceivable within the immersive 3-D environment 78 via relationship indicators 84. Each relationship indicator 84 may include a different shape, color, or other identifying characteristic. As will be described below, the relationship indicators 84 (and corresponding relationships) may also be interrelated. Thus, secondary relationships 86 may also be included in the 3-D visual representation 81 within the immersive 3-D environment 78.
It should be noted that the relationship indicators 84 and/or secondary relationship indicators 86 may be viewable in any number of ways. For example, the code 80 may be transparent or translucent, and the relationship indicators 84, 86 may be viewable behind the code 80. In some embodiments, the user 18 may be situated above the code 80, and looking down and out toward the code 80, such that the user 18 may see relationship indicators 84, 86 behind the code 80 (e.g., via the user's overhead perspective of the code 80). In some embodiments, the code 80 may be angled toward the direction 28 (e.g., instead of extending directly in the direction 24), and the relationship indicators 84, 86 may be positioned on top of the code 80 and extending upwardly (e.g., directly) in the direction 24. In such embodiments, the relationship indicators 84, 86 may be transparent or translucent, such that the user 18 can see the relationship indicators 84, 86 and the code 80 behind the relationship indicators 84, 86. These and other possibilities will be described in detail below with respect to later figures.
Turning now to
In the illustrated embodiment, the user 18 is positioned at an offset 90 above the code 80. Accordingly, the user 18 can see behind the code 80. The relationship indicators 84, 86 may be in line with or, as shown, situated above the code 80. Accordingly, the user 18 may be able to see the lines 82 (e.g., segments) of the code 80, and which relationship indicators 84 correspond with the lines 82 of the code 80. Thus, the user 18 can see how various lines 82 of code relate to one another. Further, the user 18 can see secondary relationships via the secondary relation indicator(s) 86. It should be noted that ties 92 (e.g., lines) may be drawn between the lines 82 of code 80 and the relationship indicators 84, 86 to indicate which lines 82 of code 80 are related. In other embodiments, other features may be included to indicate which lines 82 of code 80 correspond with which relationship indicators 84, 86 (e.g., via color coding).
As shown in both of
Turning now to
In
In
In accordance with present embodiments, relationships between various lines of code may include process relationships, error signifiers, method interactions, function interactions, library relationships, dependencies, stacktrace on error, and layering relationships. Other object-oriented programming relationships may include inheritance, interface implementation, dependency, composition, and encapsulation. Further, other relationships may include connections between message driven beans (e.g., code) and message queues (e.g., configurable items). Further still, other relationships may include runtime statistics regarding specific objects and portions of code (e.g., number of executions, average time per execution, etc.). Relationship visualization may facilitate the ability to step through (at a user specified speed or ‘at will’ by the user) multiple executing threads within a Java Virtual Machine (JVM) and various locking and synchronization techniques, such that the user may better understand and debug concurrent algorithms. Further, relationship visualization may facilitate the ability to step through code without synchronization algorithms in order to view the code run for each line of code to better understand the code for debugging.
Further, as previously described, at least one dimension of the immersive 3-D environment may be utilized to show relationships between data or code displayed in the immersive 3-D environment. As described above, either the processor, or the service processor (e.g., additional or separate service processor), may receive relationship data (e.g., in XML or a similar format) and may render relationships associated with the relationship data in the immersive 3-D environment (e.g., in the at least one dimension of the immersive 3-D environment). Thus, the service processor (or processor) may facilitate the ability to recognize a variety of relationship types via a configuration driven system and to transform that input a common set of primitives that can then be rendered, which may not otherwise be possible in a traditional 2-D scene. Further, the service processor (or processor) may include stored or executed thereon an algorithm for choosing the exact location to render the objects described above based on their relationships between and amongst one another. The algorithm extends traditional force-based rendering algorithms to be able to incorporate overall motion of a world to enable 4-D rendering (e.g., 3-D+time) for dynamic systems.
Further, in accordance with present embodiments, relationships between portions of data may include, but are not limited to: the topology between hardware on a given network, the changes (additions, modifications, deletions) of attributes connecting entities in fraud investigations, the genealogy of a given person connecting parent-child objects, and interactions between objects that show a relationship.
While only certain features of the disclosure have been illustrated and described herein, many modifications and changes will occur to those skilled in the art. It is, therefore, to be understood that the appended claims are intended to cover all such modifications and changes as fall within the true spirit of the invention.
This application is a continuation of U.S. patent application Ser. No. 15/474,973, filed Mar. 30, 2017, entitled “System and Method for Data Visualization and Modification in an Immersive Three Dimensional (3-D) Environment,” which claims the benefit of U.S. Provisional Application No. 62/316,184, entitled “System and Method for Data Visualization and Modification in an Immersive Three Dimensional (3-D) Environment,” filed Mar. 31, 2016, which are hereby incorporated by reference in their entireties for all purposes.
Number | Name | Date | Kind |
---|---|---|---|
5555354 | Strasnick et al. | Sep 1996 | A |
5720018 | Muller et al. | Feb 1998 | A |
5739824 | Dietrich et al. | Apr 1998 | A |
6373488 | Gasper et al. | Apr 2002 | B1 |
6473080 | Brown et al. | Oct 2002 | B1 |
7168051 | Robinson et al. | Jan 2007 | B2 |
8065658 | Bali et al. | Nov 2011 | B1 |
8434013 | Stevens | Apr 2013 | B2 |
9071520 | Bak et al. | Jun 2015 | B2 |
10452227 | Philbrick | Oct 2019 | B1 |
20020178096 | Marshall | Nov 2002 | A1 |
20070211056 | Chakraborty | Sep 2007 | A1 |
20110131209 | Bechtel et al. | Jun 2011 | A1 |
20120280893 | Halokovszky | Nov 2012 | A1 |
20120326961 | Bromer | Dec 2012 | A1 |
20130205275 | Thomson et al. | Aug 2013 | A1 |
20140282013 | Amijee | Sep 2014 | A1 |
20140282385 | Dattke | Sep 2014 | A1 |
20150156716 | Raffle | Jun 2015 | A1 |
20150187355 | Parkinson et al. | Jul 2015 | A1 |
20150288944 | Nistico et al. | Oct 2015 | A1 |
20160012631 | Kim et al. | Jan 2016 | A1 |
20160077831 | Mihalcea | Mar 2016 | A1 |
20170109933 | Voorhees et al. | Apr 2017 | A1 |
Number | Date | Country |
---|---|---|
2000014574 | Mar 2000 | WO |
Entry |
---|
Elliott, Anthony, Brian Peiris, and Chris Parnin. “Virtual reality in software engineering: Affordances, applications, and challenges.” 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering. vol. 2. IEEE, 2015. |
A. Bragdon, S. P. Reiss, R. Zeleznik, S. Karumuri,W. Cheung, J. Kaplan, C. Coleman, F. Adeputra, and J. J. LaViola, Jr. Code bubbles: Rethinking the user interface paradigm of integrated development environments. In Proc. of the 32nd ACM/IEEE ICSE—vol. 1, ICSE '10, pp. 455-464, New York, NY, USA, 2010. ACM. |
Staples, Mathew L. and Bieman, James M.. 3-D Visualization of Software Structure. Advances in Computers, vol. 49, Academic Press, London, 1999. |
Rekimoto, Jun, and Mark Green. “The information cube: Using transparency in 3d information visualization.” Proceedings of the Third Annual Workshop on Information Technologies & Systems (WITS'93). 1993. |
Porof, Victor, “Debugging and editing webpages in 3D”, Website: https://hacks.mozilla.org/2011/10/debugging-and-editing-webpages-in-3d/#availableasanaddon, Oct. 26, 2011, pp. 1-7. |
“3D view”, Website: https://developer.mozilla.org/en-US/docs/Tools/3D_View, pp. 1-3 (Accessed on Nov. 6, 2015). |
Porof, Victor, et al., “Tilt 3D Add-ons” Website: https://addons.mozilla.org/en-us/firefox/addon/tilt/, pp. 1-2 (Accessed on Nov. 6, 2015). |
Bonyuet, David, Matt Ma, and Kamal Jaffrey. “3D visualization for software development.” Proceedings. IEEE International Conference on Web Services, 2004 .. IEEE, 2004. |
Reiss, Steven P. “An engine for the 3D visualization of program information.” Journal of Visual Languages & Computing 6.3 (1995): 299-323. |
Fittkau, Florian, Alexander Krause, and Wilhelm Hasselbring. “Exploring software cities in virtual reality.” 2015 ieee 3rd working conference on software visualization (vissoft). IEEE, 2015. |
Number | Date | Country | |
---|---|---|---|
62316184 | Mar 2016 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 15474973 | Mar 2017 | US |
Child | 16597658 | US |