Patent Application
20230154347
The present application relates generally to virtual classrooms, and more particularly to methods and systems of providing an interactive learning experience in a virtual classroom.
Video is widely used around the world for video conferences, sharing of information and collaboration. Video is also used for classroom models. However, current classroom models have drawbacks including social isolation. i.e., the student may only see the teacher. Other students in the class ma not have turned on their cameras. Another drawback is any language barrier; for example, the teacher may speak Kannada while the student speaks Portuguese. Another drawback is there is no context for the teacher or student surroundings. And, there is no ability to look around in the space as one could in a traditional classroom. Therefore, improvements are desirable.
In a first aspect of the present invention, a method of presenting a virtual class in a presentation having a start time, normal speed, and end time includes determining if the presentation is being started after the start time and if so, determining a delay time; determining if the presentation is to be presented at the normal speed, and if so, presenting the presentation at the normal speed and finishing the presentation at the end time plus the delay time; if the presentation is not to be presented at the normal speed, presenting the presentation at accelerated speed such that the presentation ends at the end time.
In a second aspect of the present invention, a computer program product includes a non-transitory computer readable medium comprising instructions which, when executed by a processor of a computing system, cause the processor to perform the steps of: determining if the presentation is being started after the start time and if so, determining a delay time; determining if the presentation is to be presented at the normal speed, and if so, presenting the presentation at the normal speed and finishing the presentation at the end time plus the delay time; if the presentation is not to be presented at the normal speed, presenting the presentation at an accelerated speed such that the presentation ends at the end time.
In a third aspect of the present invention, a virtual class presentation on a non-transitory computer readable medium comprising instructions which, when executed by a processor of a computing system, cause the processor to perform the steps of: determining if the presentation is being started after the start time and if so, determining a delay time; determining if the presentation is to be presented at the normal speed, and if so, presenting the presentation at the normal speed and finishing the presentation at the end time plus the delay time; if the presentation is not to be presented at the normal speed, presenting, the presentation at an accelerated speed such that the presentation ends at the end time.
The foregoing has outlined rather broadly the features and technical advantages of the present invention in order that the detailed description of the invention that follows may be better understood. Additional features and advantages of the invention will be described hereinafter that form the subject of the claims of the invention. It should be appreciated by those skilled in the art that the conception and specific embodiment disclosed may he readily utilized as a basis for modifying or designing other structures for carrying out the same purposes of the present invention. It should also be realized by those skilled in the an that such equivalent constructions do not depart from the spirit and scope of the invention as set forth in the appended claims. The novel features that are believed to be characteristic of the invention, both as to its organization and method of operation, together with further objects and advantages will be better understood from the following description when considered in connection with the accompanying figures. It is to be expressly understood, however, that each of the figures is provided for the purpose of illustration and description only and is not intended as a definition of the limits of the present invention.
For a more complete understanding of the disclosed system and methods, reference is now made to the following descriptions taken in conjunction with the accompanying drawings.
In general, the present disclosure is about an interactive Virtual Classroom (IVC). The IVC is a 3D virtual, immersive environment of learning. The PVC seeks to mimic the traditional classroom experience in a virtual world. The IVC preferably includes a teacher, a scholar taking the class, other students in the class and a tutor. The IVC also includes a chat feature, a white board, a bookshelf, and green or black board, and a teacher presence. The IVC also includes time-shifting, translational capabilities, storybook creation for creating composed lectures and assessments of the student.
Referring to
The actors or participants of the IVC 100 include a teacher 104, a scholar (the student) 106, other students 108 and a studious 110. The teacher 104 leads the education or training session. By default, the teacher 104 can be portrayed, perhaps directly form a recorded video, speaking to the class in a standard interaction way. The look and feel can be configured differently. For example, the teacher 104 could be a chosen avatar, celebrity likeness, an animation or an animal. The teacher 104 position could also be selected and configured, including in the front of the class or sitting with the class.
The scholar 106 is the student taking the class. The other students 108 are zero or more other students who appear in the classroom 102 in addition to the scholar 106. The scholar 106 or students 108 could also be real pictures or video or could be selected avatars. The scholar 106 can also select whether or of to see the other students 108. The scholar 106 can be presented in the first-person video such that the scholar 106 does not see herself in the classroom 102—only the teacher 104 and other students 108—or just the teacher 104. The scholar 106 could also be presented in the third-person view such that the scholar 106 also sees herself in the classroom 102.
The studious 110 is the scholar's 106 personalized chatbot or tutor. The scholar 106 may configure the studious 110 according to personal style. The studious 110 monitors the lecture and can answer the scholar's 106 questions in lieu of the teacher 104. The scholar 106 can train the studious 106 and teach it new information, and through artificial intelligence, the studious can learn and gain knowledge. As the studious 110 instance travels from class to class with the scholar 106, it gains additional information and skills. The scholar 106 can choose the language of the studious 110, the avatar and how it interacts (e.g., through cartoon strip balloons). The studious 110 can interact with the scholar 106 to determine if the scholar 106 understands the materials, if they are listening and engaged, if they need a break, may proctor an exam or prepare the scholar 106 for an exam. The studious 110 could also serve as the teacher's 104 assistant.
The IVC 100 allows the scholar 106 to customize the learning experience. For example, the scholar 106 can change the classroom setting, the language, the teacher position, the teacher avatar, scholar avatar, student avatar, whether students are shown or not shown, look and feel of the classroom objects and first or third-person perspective. These settings can be established for a chosen duration, i.e., this lecture, this course, all courses for this scholar 106 or all courses for this teacher 104.
Referring to
The green board 210 has a traditional look and feel of a black board or green board where the teacher 104 can write information to illustrate a point. The bookshelf 212 has the traditional look and feel of a bookshelf that the teacher 104 may pre-populate with resource materials including books, PDFs, PPTs, videos, audio recordings and other items. The bookshelf contains links to the resources. The scholar 106 and students 108 may add additional materials to the bookshelf 212 as desired. The bookshelf 212 can also be segregated into shared and personal portions. In addition, the WC 100 can handle any digital rights management necessary for any object put into the bookshelf 212.
Referring to
Referring to
In
Preferably, the IVC 100 supports and adapts to the scholar's 106 interaction mechanism. The IVC 100 can adapt to an augmented reality (AR) or virtual reality (VR) headset for the most immersive experience. The IVC 100 can adapt to a moveable device such as a tablet or phone to see side-to-side and up-and-down; it can adapt to one or more monitors whose displayed content can be moved or scrolled left, right, up, down—either independently or together—to navigate around the virtual environment. Multichannel audio is used to position the source of the audio, e.g., teacher speaking according to the student's current orientation in the classroom space: above, below, ahead, behind, left, right etc.
Preferably, the IVC 100 experience is independent of time. It can be a live session with a teacher 104, the student 106, and other students 108 participating in the class simultaneously. It can also be a composed session with a teacher 104, the student 106 and other students 108 created by the IVC 100 using previous live modes or composed modes. The IVC 100 allows time shifting, real-time, delayed tin catch-up time (which is not akin to “fast forwarding). A student 106 can join the session late and watch in a delay or accelerated time to catch-up to live. The student 106 can watch later in a passive playback of the class as well.
Referring to
As such, the student can decide to view the presentation in live mode from the start: time to the end time. The student could also join late and view the presentation in normal time and finish at an end time plus the delay time. The student could also join late and choose to “catch” up with the lecture such that the lecture still ends on time. In this matter, the playback is accelerated such that by the end time, the lecture has fully played. Of course, the student could also join, and decide to skip the portion missed. The student can also pause at any time and then make a variation of the above to skip a portion, watch longer or catch-up.
A composed mode format is unique to the IVC 100. After a live session, IVC 100 produces a storyboard containing a sequence of events. Video segments and corresponding transcripts are produced. In addition green-board, white-board and bookshelf artifacts are produced. When, the scholar 106 chooses a lecture for a co posed mode session, the IVC 100 creates a unique composed mode session using these existing storyboard and artifacts. The scholar's 106 interaction during the composed mode session creates additional events and artifacts which are then available for future composed mode sessions.
Referring to
In one example embodiment, the teacher 104 records a class session to video. The IVC 100 processes the video and divides it into the following channels: video, chat, audio and audio transcript. Using artificial intelligence, the IVC 100 process the video and divides it into events, such as lecture, write on green board, show illustration on white board, answer a scholar's question, new entry into the chat, etc. The IVC 100 divides it into sentences and concepts. The results are stored as the storyboard. The storyboard events identify break points in the session, for example, for when a scholar 106 can ask a question. The IVC 100 processes the video and extracts the chat and green-board and white-board content It coordinates the sequencing and timing of these with the storyboard and pre-populates the bookshelf 212 with this content. The teacher may edit the storyboard or objects extracted from the video. The teacher 104 identifies a text book for the students to use and pre-populates the books shelf 212 with auxiliary materials. The teacher 104 identifies class meta-data. The teacher 104 can identify the course topic, unusual jargon and suggest translations to certain languages.
In one example embodiment, the storyboard drives the composed session. During a composed session, the bookshelf 212 is pre-populated with all available content. The scholar 106 sees the teacher 104 speak in the scholar's chosen language—lips move and word formation happens—and the scholar 106 hears the teacher 104 speak in the chosen language. The scholar 106 may ask a question, post on the class chat 206, add to the bookshelf 212, etc. during the composed session. The IVC 100 interrupts the composed session at the end of an event, recognizes the scholar 106 who then asks her question, generates a response and continues with the storyboard. The scholar's 106 interaction generates new artifacts. A future scholar 106 chooses the same lecture and experiences the composed mode session using the modified storyboard, including the previously posed question from the previous composed mode.
In addition, the IVC 100 also manages extensive translation of spoken and textual information. Each scholar 106 can identify a language of choice in which to interact with the lecture. The IVC 100 presents the teacher avatar speaking in the scholar's 106 chosen language and teacher avatar can be chosen by the scholar 106. The teacher avatar appears to speak—lips move, word formation happens, etc. in the scholar's language of choice.
The IVC 100 performs real-time audio translation of the teacher's 104 language into the scholar's 106 chosen language. The IVC 100 also performs real-time textual transcription of the teacher's 104 language into the scholar's 106 chosen language. The IVC 100 also includes real-time transformation of the video of the teacher's 104 speech into the scholar's 106 chosen language. In other words, the teacher's 104 lips move and word formation appear as though the teacher 104 was speaking in the scholar's 106 chosen language, even though the teacher 104 actually spoke in his/her native language. The IVC 100 also performs real-time transformation and translation of interaction between the students 108 and the scholar 106. The IVC 100 can also translate the materials in the bookshelf 212.
Referring to
The IVC 100 uses artificial intelligence driven rendering of the video, in conjunction with the chosen avatar, to construct the teacher presence who speaks to the class in the scholar's 106 chosen language. Furthermore, the teacher 104 can see if the student 106 is in real-time, catch-up, delayed or on pause. The teacher 104 can see if the student 106 is in first or third person viewing. The teacher 104 can initiate a private dialogue with the student, either verbally or in transcript or both.
Whatever experience is selected, the student experience is consistent. It is not exactly the same experience because each student 106 can configure the platform according to his/her needs. In composed mode, the IVC 100 draws from previous class session artifacts and events and constructs an educational experience unique for a particular scholar 106. The scholar 106 can pause the class for a break, and then resume or catch-up on the class after the break.
The scholar 106 may initiate an interaction with another student 108 in the class at any time. The interaction can be verbal or textural. The scholar 106 may pause or continue the lecture during the interaction. The scholar 106 provides input in their language of choice. The scholar 106 receives responses hack in the language of choice. In live mode, the other student 108 can choose to engage. In composed mode, the IVC 100 may reply with a virtual student 108.
The scholar 106 can also participate in class. In live mode, the scholar 106 can raise her hand to ask a question. The teacher 104 can call on the scholar 106 and answer the question, The other students 108 can hear or see or both) the question and answer in their language of choice, which can be different for each student. In composed mode, the scholar 106 can also raise her hand to ask a question in the same manner.
The scholar 106 can present the question verbally or via keyboard. or other input device in the scholar's 106 language of choice. The IVC 100 can answer in a variety of ways including sending the query to the teacher 104 to answer. The query could also go to a teacher's assistant. The IVC 100 can search the bookshelf 212 for an answer or a document corpus. The internet could also be searched taking into account the context and subject matter jargon. Being able to send the query to the teacher 104 in a composed mode is unique to the IVC 100. When the answer is found, the IVC 100 has the teacher 104 avatar speak the answer. The teacher's prompt and scholar questions go into the lecture video, storyboard and transcript as new events. The answer also does. The new event is available for inclusion in a future composed session. The studious 110 could also answer the question. The scholar 106 and the students 108 may also rate the question/answer event. The events with a higher rating have a higher probability of being included in a future composed session. The scholar 106 may augment the answer by providing an addendum or link to the bookshelf 212.
Course assessments i.e., tests, can be conducted either in live or composed modes. The teacher 104 can train the IVC 100 to evaluate the results such that the teacher 104 does not need to “grade” the tests. The teacher's studious 110 could also be used. The IVC 100 can run in a controlled-triode during assessments and limit the scholar's 106 access to the bookshelf 212, the scholar's 106 personalized studious 110 and other resources. The teacher 104 can monitor the scholar 106 during the assessment and provide grades, Course completion certificates can also be issued based on grades. The IVC 100 can integrate with external platforms and issue course completion badges and/or share on social media platforms.
Referring to
The IVC 100 keeps a set of meta-data. In one embodiment, this meta-data includes a course identifier, a lecture identifier and a variant identifier. A course is the overall course, e.g. Physics 101. A lecture is one class in the course, typically identified by a sequence number or a date. A variant is one instance of a class lecture—A new variant may be created each time a student 106 “plays” a lecture in composed mode. The scholar 106 may ask questions or add materials to the book shelf 212, thus generating a new variant of the lecture. Each lecture is composed of one or more events. The meta-data can also include the course topic e.g. physics, The course topic enables customization of the vocabulary for translation of audio to transcript and for translation to the scholar's 106 chosen transcript language and audio language. The meta-data can also include the lecture storyboard and events in the lecture. For each event, the IVC 100 captures the language of the audio, transcript and video.
Another challenge with virtual classrooms is security. The IVC 100 uses authorization and authentication to control student access to a course or lecture. Stealth Identity from Unisys Corporation of Blue Bell, Pa. can be used to implement features of the present disclosure. Stealth can be used to protect the end to end data communications and make the endpoints go dark on the Internet. As with other Stealth applications, not all endpoints require Stealth protection. The IVC 100 uses an encryption and isolation mechanism, such as Stealth Core form Unisys. This can allow or disallow a student to participate in a course or lecture; segregate sets of students into independent groups, based on authorized identify, provide encrypted channels for interaction and allow or disallow a scholar 106 to see or access materials in the bookshelf 112.
In one example embodiment, the scholar 106 chooses a course and a lecture, for example, Banyan Elementary School, Grade 5, Spring 2021, 13 Apr. 2021 lecture. The scholar 106 arrives at 7:55 a.m. and the live session begins at 8:00 a.m. The scholar 106 appears in virtual classroom with the teacher 104 and any other students 108 who have already arrived. The scholar 106 may interact with the teacher 104 and any other students 108 who have arrived. The lecture starts at 8:00. At any time, the scholar 106 may pause and resume the live lecture. In another example, the scholar 106 arrives at 8:05 a.m. and the live session began at 8:00 a.m. The scholar 106 appears in the virtual classroom 102 with the teacher 104 and any other students who have already arrived 108. The scholar 106 can choose between live real-time mode (missing the first minutes of the lecture), live delay mode (finishing the lecture 5 minutes late), or live catch-up mode (seeing the entire lecture and finishing on time).
In another example embodiment, the scholar 106 chooses a course and a lecture for example, Physics 101 Fall 2015, Lecture 3. Because the lecture happened in the past, the playback defaults to composed mode. The IVC 100 selects other students 108 to appear in the classroom 102. The scholar 106 may choose to keep the default students or choose other students W ho have participated in the class in the past, choose more or fewer students or remove one or all of the students, etc. The scholar 106 may choose an avatar for each of the other students 108 in the classroom 102. During this composed session, the scholar 106 may raise her hand to ask a question to the teacher 104. The IVC 100 generates an answer for the question and animates the teacher 104 to appear to answer the question. The scholar 106 may ask a question or engage in dialog with another student 108 in the class. The IVC 100 animates the engaged student to give the illusion that the other student is present in the classroom 102 simultaneous with the scholar 108. The scholar 106 may ask the, studious 110 a question at any time, make use of any materials in the bookshelf 212 at any time, and pause and resume the session. Any questions that the scholar 106 asked become additional events which the IVC 100 may use in future composed sessions.
In another example embodiment, two or more scholars 106 may be viewing a lecture in composed mode at the same time. For example, a team may schedule a composed session (e.g., 5 people in a team want to “take the basic MASM programming class” from 15 Jan. 2021 in a composed mode session). Each of the 5 would have the scholar 106 perspective and personalization capabilities. Each of the 5 would see the other 4 as students 108, and possibly other students chosen by the IVC 100, Each of the 5 could interact in real time while viewing the composed session—asking questions of each other, posting questions or answers in the Class Chat, adding materials to the Book Shelf, and so on. Each of these five experience would be different and each could generate new events or artifacts for possible inclusion in future composed sessions. The IVC 100 would capture the interactions as events which could be chosen for a subsequent composed mode rendering of the lecture.
In one embodiment, the user interface device 910 is referred to broadly and is intended. to encompass a suitable processor-based device such as a desktop computer, a laptop computer, a personal digital assistant (PDA) or tablet computer, a smartphone or other a mobile communication device having access to the network 908. The user interface device 910 may be used to access a web service executing on the server 902. When the device 910 is a mobile device, sensors (not shown), such as a camera or accelerometer, may be embedded in the device 910. When the device 910 is a desktop cot renter the sensors may be embedded in an attachment (not shown) to the device 910. In a further embodiment, the user interface device 910 may access the Internet or other wide area or local area network to access a web application or web service hosted by the server 902 and provide a user it for enabling a user to enter or receive information.
The network 908 may facilitate communications of data, such as dynamic license request messages, between the server 902 and the user interface device 910. The network 908 may include any type of communications network including, but not limited to, a direct PC-to-PC connection, a local area network (LAN), a wide area network (WAN), a modem-to modem connection, the Internet, a combination of the above, or any other communications network ow known or later developed within the networking arts which permits two or more computers, to communicate.
In one embodiment, the user interface device 910 accesses the server 902 through an intermediate sever (not shown). For example, in a cloud application the user interface device 910 may access an application server. The application server may fulfill requests from the user interface device 910 by accessing a database management system (DBMS), In this embodiment, the user interface device 910 may be a computer or phone executing a Java application making requests to a JBOSS server executing on a Linux server, which fulfills the requests by accessing a relational database management system (RDMS) on a mainframe server.
The computer system 1000 also may include random access memory (RAM) 1008, which may be synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous dynamic RAM (SDRAM), or the like, The computer system 1000 may utilize RAM 1008 to store the various data structures used by a software application. The computer system 1000 may also include read only memory (ROM) 1006 which may be PROM, EPROM, EEPROM, optical storage, or the like. The ROM may store configuration information for booting the computer system 1000. The RAM 1008 and the ROM 1006 hold user and system data, and both the RAM 1008 and the ROM 1006 may be randomly accessed.
The computer system 1000 may also include an input/output (I/O) adapter 1010, a communications adapter 1014, a user interface adapter 1016, and a display adapter 1022. The I/O adapter 1010 and/or the user interface adapter 1016 may, in certain embodiments, enable a user to interact with the computer system 1000. In a further embodiment, the display adapter 1022 may display a graphical user interface (GUI) associated with a software or web-based application on a display device 1024, such as a monitor or touch screen.
The I/O adapter 1010 may couple one or more storage devices 1012, such as one or more of a hard drive, a solid state storage device, a flash drive, a compact disc (CD) drive, a floppy disk drive, and a tape drive, to the computer system 1000. According to one embodiment, the data storage 1012 may be a separate server coupled to the computer system 1000 through a network connection to the I/O adapter 1010. The communications adapter 1014 may be adapted to couple the computer system 1000 to the network 1008, which may be one or more of a LAN, WAN, and/or the Internet. The communications adapter 1014 may also be adapted to couple the computer system 1000 to other networks such as a global positioning system (GPS) or a Bluetooth network. The user interface adapter 1016 couples user input devices, such as a keyboard 1020, a pointing device 1018, and/or a touch screen (not shown) to the computer system 1000. The keyboard 1020 may be art on-screen keyboard displayed on a touch panel. Additional devices (not shown) such as a camera, microphone, video camera, accelerometer, compass, and or gyroscope may be coupled to the user interface adapter 1016. The display adapter 1022 may be driven by the CPU 1002 to control the display on the display device 1024. Any of the devices 1002-1022 may be physical and/or logical.
applications of the present disclosure are not limited to the architecture of computer system 1000. Rather the computer system 1000 is provided as an example of one type of computing device that may be adapted to perform the functions of a server 902 and/or the riser interface device 910. For example, any suitable processor-based device may be utilized including, without limitation, personal data assistants (PDAs), tablet computers, smartphones, computer game consoles, and multi-processor servers, Moreover, the systems and methods of the present disclosure may be implemented on application specific integrated circuits (ASIC), large scale integrated (VLSI) circuits, or other circuitry. In fact, persons of ordinary skill in the art may utilize any number of suitable structures capable of executing logical operations according to the described embodiments. For example, the computer system 1000 may be virtualized for access by multiple users and/or applications. The applications could also be performed in a serverless environment, such as the cloud.
If implemented in firmware and/or software, the functions described above may be stored as one or more instructions or code on a computer-readable medium. Examples include non-transitory computer-readable media encoded with a data structure and computer-readable media encoded with a computer program. Computer-readable media includes physical computer storage media. A storage medium may be any available medium that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Disk and disc includes compact discs (CD), laser discs, optical discs, digital versatile discs (DVD), floppy disks and blu-ray discs. Generally, disks reproduce data magnetically, and discs reproduce data optically. Combinations of the above should also be included within the scope of computer-readable media. A serverless environment, such as the cloud, could also be used.
In addition to storage on computer readable medium, instructions and/or data may be provided as signals on transmission media included in a communication apparatus. For example, a communication apparatus may include a transceiver having signals indicative of instructions and data. The instructions and data are configured to cause one or more processors to implement the functions outlined in the claims. A serverless environment, such as the cloud, could also be used.
Although the present disclosure and its advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made herein without departing from the spirit and scope of the disclosure as defined by the appended claims. Moreover, the scope of the present application is not intended to be limited to the particular embodiments of the process, machine, manufacture, composition of matter, means, methods and steps described in the specification. As one of ordinary skill in the art will readily appreciate from the present invention, disclosure, machines, manufacture, compositions of matter, means, methods, or steps, presently existing or later to he developed that perform substantially the same function or achieve substantially the same result as the corresponding embodiments described herein may be utilized according to the present disclosure. Accordingly, the appended claims are intended to include within their scope such processes, machines, manufacture, compositions of matter, means, methods, or steps.
