ONLINE CLASSROOM SYSTEM AND METHOD FOR CONDUCTING BREAKOUT GROUPS

Abstract

Systems and methods for online classroom instruction are described that facilitate the learning and application of course information in an active and synchronous learning environment. Data obtained from monitoring or testing students is used to guide the learning process by providing student-specific questions in quizzes and exams and for forming and conducting breakout groups. The instructor can monitor the breakout groups using a visual display and can rearrange the groups during class. Students and teachers may view all of the students in class using a scrolling display of the student's video feeds.

Description

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention generally relates to education, and more particularly to an apparatus and method for on-line active learning.

Discussion of the Background

Synchronous online learning, also known as live, real-time online learning, uses computer and telecommunications technology to connect teachers and students in real time. Such systems are used, for example, for teaching college and high school courses and for many forms of video conferencing, such as used during college “office hours,” study groups, and discussion sections.

Available synchronous online learning systems are simple evolutions of existing video-conferencing services. Although such systems generally offer a classroom setup where instructors can present slideshows while on live video with students, technological limitations prevent them from offering a richer classroom learning experience.

For example, prior art systems do not promote active learning by keeping all students engaged in the learning process. Thus, prior art systems are typically limited to merely presenting a slideshow, and have limited capabilities of forming or monitoring breakout groups.

• • Other prior art systems treat all students the same, independent of their classroom performance. Thus, prior art systems determine student performance during a class, but do not use this information to adjust lessons or to form breakout groups based on the capabilities that are relevant to that activity. As a result, breakout groups may include mismatched or poorly matched students. Further, the tests provided to the students do not reinforce material that must be mastered.

Thus, there is a need in the art for a system that promotes active learning, determines and uses student performance to automatically tailor instruction, and continually engages students with challenging new material. Such a system and method should be easy for the instructor and student to use, should provide a compelling learning environment that promotes active learning and discussion, should work with standard networked-computer systems, and should be scalable for use by a large number of students.

BRIEF SUMMARY OF THE INVENTION

Certain problems in prior art computer platforms and methods of instruction are solved with a computer platform and methods that provide an active-learning classroom environment that: empowers teachers by giving them access to tools that promote active learning; prepares students for active learning by engaging them with new material.

Certain embodiments provide a method for conducting an active-learning class using a plurality of electronic devices in a computer network, where the plurality of electronic devices includes a first electronic device operated by an instructor, a plurality of second electronic devices each operated by one student of a plurality of students, and one or more servers. The method includes assessing a performance of each student of the plurality of students during the class, where the performance of each student is assessed by one or more electronic devices of the plurality of electronic devices using one or more inputs into the second electronic device operated by each student; assigning each student of the plurality of students to one breakout group (hereafter referred to as “BOGs”) of a plurality of BOGs, where the assigning is performed by one or more electronic devices of the plurality of electronic devices based on the performance of the plurality of students relative to each other; storing, on the server, a shared workspace for each BOG; and providing, over the network, each student computer of the plurality of student computers with the shared workspace, where the shared workspace corresponds to the BOG to which the student is assigned. All students within each BOG can thus communicate with each other through the shared workspace of their BOG.

Certain other embodiments provide an apparatus for conducting an active-learning class using a plurality of electronic devices in a computer network, where the plurality of electronic devices includes a first electronic device operated by an instructor, a plurality of second electronic devices each operated by a corresponding student of a plurality of students, and one or more servers. The apparatus includes one or more processors of at least one electronic device of the plurality of electronic devices programmed to assess a performance of each student of the plurality of students during the class, where the performance of each student is assessed using one or more inputs into the second electronic device operated by each student; assign each student of the plurality of students to one BOG of a plurality of BOGs, where the students are assigned based on the performance of the plurality of students relative to each other; store, on the server, a shared workspace for each BOG; and provide, over the network, each student computer of the plurality of student computers with the shared workspace, where the shared workspace corresponds to the BOG to which the student is assigned. All students within each BOG can thus communication with each other through the shared workspace of their BOG.

Certain other embodiments provide a computer platform and methods that present students with learning tasks that are not too difficult as to be frustrating and not so easy as to be boring; ideally, tasks are at just the right level of difficulty for each student. Thus, certain embodiments assign students automatically to BOGS based on having comparable levels of the relevant capabilities (as determined from previous performance measures, such as quiz scores), and “multileveled” activities are presented to the BOGs that they can perform more or less deeply. For example, in a course on clear communication, students might be asked to go through paragraphs and detect all ambiguity. Some ambiguity might be flagrantly ambiguous words, which everyone can identify; some might be more subtly ambiguous words, which will be identified by more adept students; and some might be very subtle relations among phrases, which only the most adept student will identify. Because the groups are automatically formed so that members have comparable capability, they will “self titrate” so that they are not frustrated or bored, and will drill down to an appropriate level in the materials.

Certain embodiments provide a computer platform and method that facilitates the learning and application of course information, as transmitted from an instructor to a plurality of students, in an active and synchronous learning environment. In certain embodiments the instructor and students communicate over networked computers in either a classroom mode or a breakout mode. In the classroom mode, the instructor presents course material to the students and two-way communication is provided between the instructor and students and between individual students. In the breakout mode, the students are divided into independent BOGs, each consisting of a small number of students, which are presented with group learning activities. In certain embodiments, the students are grouped according to the type and/or requirements of the learning activity which may, for example and without limitation, be debates, role playing, group problem solving, or guided analysis. The learning activities generally are designed to be “multilayered,” so that students of different abilities all can get something out of them.

Certain other embodiments provide a computer platform and methods that promote learning and subsequent memory by inducing students to process relevant information. Thus, certain embodiments form a plurality of BOGs during a class comprising small groups of students. Learning tasks for each BOG are provided with incentives and consequences: students are incentivized to pay attention and process deeply because they know that they will need that information for a subsequent activity (e.g., a follow-on BOG). If students do not pay attention, the consequence will be that they cannot perform a subsequent task and are embarrassed in front of their peers and perform poorly on a quiz.

Certain embodiments provide a computer platform and methods include a scalable videoconferencing and teaching environment and method for teaching classes. Various embodiments are capable of: providing lectures for to up to 200 students, with the lectures punctuated by active learning, which may be in the form of polls, quizzes, and interactions over Chat; forming small BOGs, ranging from 2-8 students. These groups are either configured in advance or created on the fly, as needed for active; earning; specifying sequences of BOGs, with students who played different roles in an initial group automatically being sorted into new groups based on those roles; after a sequence of different groups, allowing students automatically to be placed back into an earlier BOG, so that they can revisit questions posed earlier; providing a dashboard for the instructor, who can see a heat map indicating how much of a specific type of activity (e.g., talking, engagement with writing) is taking place in each BOG; permitting the instructor to visit any specific BOG, interacting with those students; allowing the instructor to “broadcast” to students in all BOGs; permitting students to vote and take polls at any time, including during debriefing after BOGs; using shared files at any point during class, including in BOGs; permitting the work products from BOGs to be shared; providing students with the ability to send messages to the class; providing students and the instructor with screen sharing capabilities; providing individualized quizzes at any point during the class, with the same or different items presented to different students; assigning students to BOGs using quiz performance data, so that students at the same level are grouped together or, alternatively, so that each group has at least one member that understands the material; tagging particular questions a given student received on a quiz and ensuring that students who re-take a quiz automatically receive a new set of relevant questions; and providing a student dashboard that shows students their progress towards achieving specific competencies.

These features together with the various ancillary provisions and features which will become apparent to those skilled in the art from the following detailed description, are attained by the system and method of the present invention, preferred embodiments thereof being shown with reference to the accompanying drawings, by way of example only, wherein:

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING

FIGS. 1 and 2 are schematic diagrams illustrative of one embodiment of an on-line platform;

FIG. 3 is a high-level architecture diagram of the front end/web app detail of the platform;

FIG. 4 is a general schematic of Content Management System (CMS);

FIG. 5 is a schematic of an instructional design view of the CMS;

FIG. 6 illustrates a live classroom view of the CMS for presenting lesson content;

FIG. 7 is a schematic that illustrates the use of the College API Server and a Load Balancer;

FIG. 8 is a schematic of the structure of the Database;

FIG. 9A is a schematic of publish/subscribe services are used in the classroom setting;

FIG. 9B is a schematic that illustrates how publish/subscribe services control student's behavior and views;

FIG. 10 is a schematic that illustrates publish/subscribe services for chat;

FIG. 11 is a schematic that illustrates publish/subscribe services used for learning activities;

FIG. 12 is a schematic illustrating the creation of quizzes, midterms, or final exam;

FIG. 13 is a schematic which illustrates one embodiment for how the system forms BOGs during a lesson;

FIG. 14 illustrates forming BOGs with students having similar ranking;

FIG. 15 illustrates forming BOGS with students based on more than one student metric;

FIG. 16 illustrates the generation of the BOGs Heat Map widget;

FIG. 17A illustrates the display of a plurality of video feeds in the Classroom Videos Grid;

FIGS. 17B, 17C, and 17D illustrate three sequential views of the Classroom Videos Grid of FIG. 17A;

FIGS. 18A and 18B illustrates the effect of the “raise/lower” hand tool on the Classroom Videos Grid;

FIGS. 19A and 19B illustrates the effect of an instructor spotlighting a student from Classroom Videos Grid;

FIG. 20A is a screenshot showing accesses tools to set up rules for forming a BOG;

FIG. 20B is a screenshot showing the elements for organizing BOGs;

FIG. 20C is a screenshot showing the embedding of a shared whiteboard for the group.

FIG. 21 is a screenshot illustrating one embodiment of the Everyone In Class (Instructor View) Instructor View State;

FIG. 22 is a screenshot illustrating one embodiment of the BOG Activity In Progress Instructor View State;

FIG. 23 is a screenshot illustrating one embodiment of the Everyone In Class (Students View) Student View State;

FIG. 24 is a screenshot illustrating one embodiment of the Everyone In Class (Students View) Student View State during a classroom discussion;

FIG. 25 is a screenshot illustrating one embodiment of the Showing Poll Results Student View State;

FIG. 26 is a screenshot illustrating one embodiment of BOG Activity In Progress Instructor Broadcasting/Intercom To All Groups Student View State;

FIG. 27 is a screenshot illustrating one embodiment of the “Spotlit” BOG Student View State;

FIG. 28 as a screenshot illustrating one embodiment of the “Spotlit” BOG Instructor View State; and

FIG. 30 is a screenshot of a document from a breakout session imported into the Everyone In Class (Students View) Student View State.

Reference symbols are used in the Figures to indicate certain components, aspects or features shown therein, with reference symbols common to more than one Figure indicating like components, aspects or features shown therein.

DETAILED DESCRIPTION OF THE INVENTION

In general, the present invention includes a computer network-based system (“platform”), in which an instructor and students interact. Embodiments are presented herein for an Internet-based system that provides webpages to the instructor and to each of a plurality of students via web browsers, and which permits the instructor and students to interact through their browsers.

For purposes of the following discussion a “lesson plan” includes “lesson content” in a number of sequentially accessed “lesson steps” that are presented on the browsers. The lesson content includes both information and events that are designed to enhance learning. Each lesson step presents certain information to the student or requires certain student interactions, either individually or within groups of students. The lesson plans or lesson steps may tagged as be designed for teaching certain subject areas and/or or areas or specific areas of competency, which are termed “Macro competency” (the overarching unit), “Mini competency” (a specific aspect of the unit), and “Nano competency” (an actionable component of the Mini competency, and corresponds to a “Hack” or “Heuristic,” as discussed subsequently.

Lesson steps may include, for example and without limitation, text, graphics, and/or videos, instructions for learning activities, such as breakout groups (“BOGs”), to be performed by students either individually or in groups, and/or quizzes, polls, or tests, an embedded web page, such as a Wikipedia entry, an embedded video, linking to the Internet, an embedded image, providing a URL pointing to an image served from the internet, a student poll, a quiz, or a shared/editable files, referred to herein and without limitation as a “whiteboard” or “whiteboard/documents” and which may be for example, Google Docs, Sheets, or Slides. or other shared workspaces, including but not limited to: a shared coding environment that allows the instructor and students to collaborate on the same piece of software code and test that is runs; a shared drawing tool that allowing the instructor and students to collaborate on building diagrams, org charts, etc.; or a shared “game,” such as battleship, which is used to illustrate topics such as strategy.

FIGS. 1 and 2 are schematic diagrams which illustrate one embodiment of an on-line platform of the present invention as a system 100 including a server 110 for providing programming instructions to a plurality of electronic devices 130 over a network 120. In one embodiment, devices 130 are wireless devices, and network 120 includes wireless communication to the device. In general, server 110 may produce programming instructions, files, or data that may be transmitted over network 120 to operate devices 130. In addition, network 120 may also provide access to instructional material which can be retrieved and displayed on device 130.

In general, a user of device 130 may communicate over network 120 to server 110, which includes programming to receive and transmit information with devices 130. FIG. 2 illustrates one embodiment of platform 100 programmed as a system of the present invention.

Server 110 is a computer, a computer system, or network of computers or systems that may include a network interface 111, a memory 113, and a processor 115. Is to be understood that network interface 111, memory 113, and processor 115 are configured such that a program stored in the memory may be executed by the processor to accept input and/or provide output through network interface 111 over network 120 to devices 130.

Devices 130 may be, for example and without limitation, a desktop or portable computer or a cellular telephone, tablet computer, or a portable digital assistant, and includes a network interface 131, a memory 133, a processor 135, a display 137, and an input device 139. Network interface 131 is used by device 130 to communication over a wireless network, such as a cellular telephone or Wi-Fi network, and then to other telephones through a public switched telephone network (PSTN) or to a satellite, or over the Internet. Memory 133 includes programming required to operate device 130 (such as an operating system or virtual machine instructions) and may include portions that store information or programming instructions obtained over network interface 131, or that are input by the user (such as telephone numbers or images from a device camera (not shown). In one embodiment display 137 is a touch screen, providing the functions of the screen and input device 139. Input device 139 may be a keyboard, a touchscreen, a trackball, a mouse, a microphone or a camera which generates a video feed and a microphone which generates an audio feed.

One use of platform 100 for facilitating of an on-line college platform will now be described. In general, there are at least two types of users of platform 100—instructors and students. Thus, for example, memory 113 and or 133 is provided with software that enables instructors to provide students with instruction, such as lectures and quizzes, allows students to communicate with instructors, such as by answering quizzes or asking questions, and enable students to communicate with each other during breakout sections and have the instructor monitor their progress.

In addition to instructors and students, platform 100 is alternatively programmed with other interfaces, such as: a tech support interface that may be used by technical support (“tech support”) staff to attend to technical issues during class, a recorder interface that allows observation of a lesson without interacting or interfering with a class; a dashboard interface for use by coaches, admissions, tech support, instructors, and other to check on the status of students enrolled in courses, an operations interface for use by software developers and dev/ops personnel to perform maintenance and upgrades, an instruction design interface for use by instructional designers to design lesson plans, enter quiz, midterms, final exams and poll questions, or otherwise provide classroom content, and a school administrator interface for use by school administrators to edit the academic calendar, enroll students, run reports, and perform other school administrative tasks.

The following discussion presents embodiments of platform 100 wherein server 110 and devices 130 include programs stored in memory 113 and 133, respectively, which instruct processor 115 and 135, respectively, to communicate over the network 120, including retrieving data stored in memory 113, and provide output on displays, such as display 137.

System Architecture

FIG. 3 is a high-level architecture diagram of the front end/web app detail of platform 100, which shows interactions between one of a plurality of browser web apps 300, each operating on one of a plurality of devices 130 and server 110. The elements of browser web app 300 include components that permit the browser web app to communicate over network 120, and may include, for example and without limitation, a reactJS/Single-page-app 303, a publish/subscribe client 304, a peer-peer video client 305, chat 306, live tech support 307, remote control 308, share whiteboard/docs 309, screen sharing 310, metrics and monitoring 311, and a classroom videos grid 312.

Using these components, browser web app 300 accesses Content Management System (CMS) 320, Content Delivery Network (CDN) 330, Third-Party API Servers/Services 340, College API Servers 350, Databases 360, Cloud Hosting Services 370, Authentication/Authorization 380, Publish/Subscribe Service 390, and Video Service 392, as well as other browser web apps 300 and other computers on network 120. Unless stated explicitly herein as being provided over the Internet or by third parties, it is understood that the programming and databases described herein reside on servers 110.

College API servers 350 are “stateless” servers that access data by retrieving data in College Database 810 or by retrieving data from calls to other services, as discussed subsequently.

CMS 320 includes College API Servers 350 and CMS Data 410 and Lesson Plans 420 which are stored in Database 360. In certain embodiments, an API call from browser web app 300 of one device 130 are directed to a College API Servers 350 configured to retrieve lesson content from Database 360 that is locally stored on platform 100, or may further call Third-Party API Servers/Services 340 or Cloud Hosting Services 370 to retrieve lesson content stored external to platform 100, such as on the Internet.

Authentication/Authorization 380 is a type of Third-Party API Service 340 that may be used to authenticate and log users onto platform 100.

Publish/Subscribe Service 390 is a Third-Party API Service 340 that is used for a many purposes, such as for chat sessions between browser web apps 300, including classroom chat, BOG chat, person to person chat, or tech support chat.

Publish/Subscribe Service 390 is also used to send commands from a first browser web app 300 to a second browser web app. Thus, for example and described subsequently, an Instructor Browser web app 300A may send commands via Publish/Subscribe Service 390 to one or more Student Browser web apps 300B that “remote control” the Student Browser web apps and/or the device on which the browser is running. Remote control actions include, but are not limited to one of the following: send “current lesson step” meta data and data to display lessons on each student's Student Browser web apps; send “flash messages” to one or more students' Student Browser web app to display a bright popup that covers the rest of the content on the browser and is a “high priority” notification to the student; refresh the Student Browser web app video streams in the event that the student has contacted tech support with problems regarding video streams; “refresh the user's browser,” which reloads browser web apps 300 and to clear intermittent bugs; turn students' microphones and/or videos off for the device 130 running their browser web app; enable/disable students' ability to turn their own microphones on or off for the device running their browser web app; enable/disable students' ability to “raise their hands” using the button along the bottom of their web apps; enable/disable students' ability to “open and use classroom chat” using the button along the bottom of their web apps; transition the students' web apps into different “views” and “states”. For instance, at the end of class, transition students into a quiz—where they see a UI that allows them to answer questions; move students between BOGs; transition students' browsers into a view/state where they see the other members of their BOG and see the shared content their BOG is working on; “spotlight” a BOG. This transitions everyone's web apps into a state where they see a selected BOG, its content, and a grid of the entire classroom; or spotlight a specific student.

Video Service 392 is a Third-Party API Service that provides video chatting between different devices 130. Video Service 392 is used to: allow the instructor and students to “enter into” and “exit from” a virtual meeting space where each attendant is streaming audio and video from their computing device and where others in the space have access (can view and hear) the video and audio streams of all others in that same meeting space; allows platform 100 to turn individual students' audio and/or video streams on and off; present video/audio of the instructor to all students attending class; present video/audio of all other students attending the class; and store and forward student metrics such as: amount of talking, if student turned their video or audio on or off, who is currently “connected” into the shared meeting space.

In addition to retrieving lesson content, College API Servers 350 may also be used to perform one or more of the following tasks: create/drop/add users; enroll/unenroll students; administer academic calendar, instructors; edit courses; generate quizzes, midterms, final exams; generate BOGs content; group students into BOGs during instruction; show user polls and how responses during instruction; monitor student interaction with content during BOGs during instruction; monitor student interaction with chat during BOGs during instruction; monitor student interaction with video/audio during BOGs during instruction; generate reports; provide infrastructure for “operations” of classes during classes; provide storage for all meta data used to describe academic calendar, curriculum, roles, students, etc.; store and report system-level (infrastructure) metrics; and store and report user-level (school, students, instructors, quizzes, etc.) metrics.

Cloud Hosting Services 370 may be used as follows: S3 Content Delivery Network; Deliver Web App to Browsers; Horizontal Scalability; Vertical Scalability; Continuous Integration; Continuous Deployment; Software Versioning; Virtual Servers; Firewalls & Routers; Virtual Private Networks; FERPA-Compliance; Fault Tolerance; Serverless (Functions As A Service); SSL Certificates; Relational Databases; NO SQL Databases; Automated Backup/Restore; Audit Trails; ElasticSearch Log Management; and/or Logging/Monitoring/Metrics.

Browser Configurations

As noted above, platform 100 includes interfaces for different purposes and which provide different functionalities. In certain embodiments, platform 100 includes a number of different browser web apps 300 that provide these interfaces and which interact differently with one or more of Content Management System (CMS) 320, Content Delivery Network 330, Third-Party API Servers/Services 340, College API Servers 350, Databases 360, Cloud Hosting Services 370, Authentication/Authorization 380, Publish/Subscribe Service 390, and Video Service 392 differently, as discussed subsequently.

Certain browser web apps 300 are configured to present the classroom videos grid 312. Classroom videos grid 312 includes the student's video feed of the students in the class—that is, the video feed of device 130 for student's Student Browser web app 300B. In certain embodiments, classroom videos grid 312 is sized to show a portion of all the student's video feeds at a time and scrolls the video feeds through the classroom videos grid. If a student's video feed cannot be displayed for some reason, such as due to network bandwidth limitation, the space intended for a student's video feed may be replaced with an icon or image, such as a photograph of the student.

In certain embodiments, Browser web apps 300 may include but is not limited to:

• • an Instructor Browser web app 300A intended for use by the instructor of the class and used, for example and without limitation, to control, communicate, and monitor students during lectures or BOGs through their Student Browser web apps 300B. Instructor Browser web app 300A may present, for example and without limitation, lesson steps defined by Instruction Design web app 300G, including lecture material, BOGs, and quizzes, classroom videos grid 312 which presents, preferably, video feeds of each student who is logged in to their Student Browser web app 300B, an image or video of the user of Instructor Browser web app 300A, the ability to communication with individual or all Student Browser web apps 300B, and other classroom widgets;
  • a Student Browser web app 300B intended for use by a student to access a course both before, during, and after class. During the presentation of a lesson plan, Student Browser web app 300B accepts commands from Instructor Browser web app 300A that may control the presentation lesson plan steps, and provides communications with the Instructor Browser web app 300A and other Student Browser web apps 300B, an image or video of the user of Instructor Browser web app 300A, and classroom videos grid 312;
  • a Tech Support/Observer Browser web app 300C intended for use by tech support, course observers, or the instructor to answer students' technical questions during class, to move individual students between BOGs, and to control Instructor Brower web app 300A and Student Browser web app 300B during class;
  • a Recorder Browser web app 300D presents a view of the class from the student's perspective, as in Student Browser web app 300B. Recorder Browser web app 300D can also be used to record the class for later viewing.
  • a Dashboard Browser web app 300E intended for use by coaches, admissions, tech support, instructors, and other to check on the status of students enrolled in courses;
  • an Operations Browser web app 300F intended for use by software developers and dev/ops personnel to perform maintenance and upgrades to platform 100;
  • an Instruction Design web app 300G is intended for use by instructional designers to design lesson plans 420 for each lesson, enter quiz, midterms, final exams and poll questions, and to interact with the CMS that drives classroom content; and
  • a School Admin Browser web app 300H intended for use by school administrators to “open class,” which prepares a lesson plan for teaching, edit the academic calendar, enroll students, run reports, and perform other school administrative tasks.

Instructor View States

Instructor Brower web app 300A provides screen 137 of device 130 with views (also referred to herein as “instructor view states”) that may be used by the instructor to run the class, and may include, for example and without limitation, to control the presentation of class material, to interact with students, to monitor student progress, and to use various technologies such as publish and subscribe technology for interaction with the students having their own devices 130B, including remotely controlling the student's devices.

The following discussion describes various instructor view states, followed a list of UI widgets which may be used to generate the various instructor view states.

Instructor view states on Instructor Brower web app 300A may include, but is not limited to the following instructor view states:

Before Class has Started Instructor View State: This view states presents an icon for each of up to 200 students as each student, through their Student Brower web app 300B, “arrive” to class. In the lower right-hand corner of the screen the first step of the lesson plan is displayed. Once the instructor logs in to system 100 and “starts” class, the instructor view state changes to the Everyone In Class (Instructor View) Instructor View State, and the student view state (that is, the view provided on screen 127 of the device 130 running Student Brower web app 300B) changes to the Everyone In Class (Students View) Student View State, both of which are discussed subsequently.

Everyone In Class (Instructor View) Instructor View State: This view state may present, for example and without limitation: a lesson step selector along the left side of the screen; the current lesson step's content in the main content area; the instructor's video feed (that is, the video feed device 130 running Instructor Browser web app 300A) in the upper-right portion of the page; the classroom videos grid 312 along the lower right portion of the page; a top navigation bar along the top of the page; and a page footer along the bottom of the page. FIG. 21 is a screenshot 2100 illustrating one embodiment of the Everyone In Class (Instructor View) Instructor View State;

Everyone in Class With One Student Spotlit Instructor View State: This view state is similar to the Everyone in Class view state, except that, instead of the classroom videos grid 312, the single student video stream for the “currently spotlit” student is shown. Thus, for example and without limitation, an individual student may be selected by clicking on their icon, image, or video feed from classroom videos grid 312, resulting in the spotlit student's video feed expanding to fill the entire space previously occupied by classroom videos grid 312. Clicking on the spotlit student's video feed reverses the process, reinstating the classroom videos grid 312.

BOG Activity In Progress Instructor View State: This view state may present, for example and without limitation: a BOGs Heat Map widget, a Move Students Between BOGs widget, and an Embedded BOG Widget, which shows the instructor the currently selected BOG. FIG. 22 as a screenshot 2200 illustrating one embodiment of the BOG Activity In Progress Instructor View State;

BOG Activity In Progress Instructor “Visiting” One Group I Instructor View State: This view state is similar to the BOG Activity In Progress View State and also shows the instructor's video feed above the embedded BOG view, and to Student Browser web apps 300B of each student of the currently selected BOG, and provides for the instructor to speak, through their respective devices 130, directly and privately to only the members of that BOG.

BOG Activity In Progress Instructor Broadcasting/Intercom To All Groups Instructor View State: This view state is similar to the BOG Activity In Progress View State and also shows the instructor's video feed above the embedded BOG view, which is also seen by all the members of all BOGs, and provides for the instructor to broadcast a message to all BOGs at the same time through their respective devices 130.

“Spotlit” BOG Instructor View State: This view state is displayed when a single BOG has been “spotlit” and asked to present their work to the class, and presents the same content as the BOG Activity In Progress View State and the classroom videos grid 312. FIG. 28 as a screenshot 2800 illustrating one embodiment of the “Spotlit” BOG Instructor View State;

Poll in Progress Instructor View State: This view state differs from the Everyone In Class View State in that the main content area shows a poll question that the instructor broadcasts to Student Browser web apps 300B.

Showing Poll Results Instructor View State: This view state differs from the Everyone In Class View State in that the main content area shows the results of a poll question that all the students have answered. Poll results can be shown in real time, changing as more students respond.

Quiz In Progress Instructor View State This view state may present, for example and without limitation: a message that a quiz is in progress, the instructor's video feed and the classroom videos grid 312, and allows the instructor to watch all the students during the quiz to ensure that the students are not cheating.

UI widgets which may be used to generate the various instructor view states include, but are is not limited to the following UI widgets:

Lesson Step Selector Widget: This widget runs the height of the browser web app 300 and is docked along the left side. It allows the instructor to control what image or video or poll or BOG or other content the students are presented on Student Browser web apps 300B. The instructor may operate Instructor Browser web app 300A to advance the class step-to-step or to a specific lesson plan step using a “lesson step selector” widget along the left-hand side of the screen.

Main Content Widget. This widget is in the center of the screen and displays the currently selected lesson step's content, which may be, for example and without limitation, an image, a video, a PDF file, an editable shared document, or HTML/text.

Instructor Video Feed Widget: This widget is a large, square region of the screen where the instructor's video feed is displayed. This feed is provided to Instructor Browser web app 300A and Student Browser web app 300B so that it may seen by both the instructor and by students.

Classroom Videos Grid Widget: This widget is a grid (1×1, 2×2, 3×3, or 4×4) of video feeds of all the students currently attending class. The video feeds scroll horizontally (either right to left or right to left) so that even if there are as many as 200 students currently logged in, only 16 video feeds (at most) can be seen at the same time. The student can adjust the size of the matrix, which allows them to cope with poor bandwidth or using an older, slower computer.

User Poll (Instructor View) Widget. This widget presents a view of the poll question that the instructor has asked all students to answer.

User Poll Results Widget: This widget is a bar chart showing an aggregated view of the answers that, by the operation of the Student Browser web app 300B, the students have provided to a poll. This view can change in real time as more students respond

“Remote Control” Video (Instructor View) Widget: This widget allows the instructor to control the play, pause, seek of a video. When the instructor uses this widget to start a video or pause a video or seek to a specific play position within a video, the changes the instructor makes are broadcast to all the students' views of the video. This allows the instructor to control the playback of the video.

Embedded Web Content Widget: This widget shows an HTML “embed” of content using an HTML “iframe.” This widget is used to display videos, web pages, audio streams and other types of content that can be embedded into a lesson step.

Image View Widget: This widget displays an image that is hosted on a CDN such as: Amazon Cloud Front/S3, Akami, CloudFlare, or any other web-based content delivery network.

Embedded BOG View Widget: This widget displays the video/audio streams and the shared document/whiteboard of the students in a single BOG. This is used during BOGs as well as during “BOG spotlight”—when a single BOG is presenting their work to the entire class.

BOGs Heat Map Widget: This widget is displayed when the system is in “breakout activity” mode and all of the students are grouped into small, private “BOGs” where they can collaborate. The BOGs Heat Map Widget displays a visual display of the status of each group (different colored rectangles show groups that are highly active, not sufficiently active, too active), etc. The specific measures of “activity” in each BOG include, but are not limited to, the following measures of activity as determined from video, audio, and typing from each of the Student Browser web app 300B in the BOG: how much talking is taking place; how much activity is taking place in the shared work product; how much use is being made of Group Chat; the average “initiative” level (determined by how many times the members have previously raised their hands during class); the average amount of talking; the variance in talking, indicating how evenly talking is distributed across group members; the variance in amount of times cursors are engaged in the shared work product, indicating how evenly work production is distributed across group members; ratios consisting of the person with the greatest activity being divided by the average of the activity of all other members (which provides a measure of whether one person is dominating). The visual display in the BOGs Heat Map Widget of the status of each group is a geometric shape that is color coded according to the activity in the BOG. Examples of the visual display, which are not meant to limit the scope of this patent, are discussed for example and without limitation, in the discussion of FIG. 15, and are shown in FIG. 22.

The heat map is interactive and operates as a “channel changer.” Thus, for example, when the instructor clicks on the geometric shape corresponding to a specific BOG, the instructor will enter the BOG (and then clicks again to leave that group, at will). Further, when the instructor joins a BOG, the instructor may participate as a full member of the group—conversing with students and having access to any shared document/whiteboard. The instructor may also operate Instructor Browser web app 300A to “pin” a group, allowing the instructor to revisit the group at a later time.

Move Students Between BOGs Widget: This widget is displayed at the same time that the BOGs Heat Map is displayed. This widget allows the instructor to move students between BOGs after those BOGs have been created and are in progress.

Students Tool Widget: This widget is a popup window that allows the instructor to see a list of all students who have logged into class, the status of their video and audio (did the student turn these off?), how long the student has been attending class, etc. The instructor can use this tool to turn off a single student's video and/or audio, refresh their browser (this sometimes helps when the student encounters a bug with Student Browser web app 300B), send the student a private “flash message” that shows up in front of all of the content on their screen, and remotely log a student out if that student is being unruly in class.

Top Navigation Widget: This widget is a bar along the top of the screen. It contains the name of the currently taught class, the current step number, and a drop-down menu full of tools the instructor can use to control certain aspects of class.

Page Footer Widget: This widget resides along the bottom of the web application. It allows the instructor to turn on and off their own camera/feed and/or microphone/audio. It also allows the instructor to turn on or off all students' microphones, turn on or off all students' videos, lower all students' hands, enable/disable students' ability to raise their hands, and enable/disable students' ability to access the classroom chat stream.

Quiz In Progress View Widget: This widget is a status message view that lets the instructor know that a quiz is in progress and all the students are currently answering the quiz.

The instructor may operate Instructor Browser web app 300A change/select view states on the Instructor Browser web app 300A and on one or more Student Browser web app 300B by clicking on various pieces of the Instructor Browser web app 300A user interface, such as:

Clicking on the “start BOGs” button in the lesson step chooser changes the instructor's view state to one that shows the embedded/selected BOG, the students mover tool, and the heat map. In addition, this “remotely” causes all students' browsers to be transitioned to a view state that shows them in a BOG.

Clicking a “poll” button in the lesson step chooser changes the instructor's view state to one that shows a poll question that the students will answer. In addition, this “remotely” causes all students' browsers to be transitioned to a view state that shows a question that they can answer.

Clicking a “quiz” button in the lesson step chooser changes the instructor's view state to one that shows a “quiz in progress” message as well as the classroom videos grid 312 of students taking the quiz. In addition, this “remotely” causes all students' browsers to be transitioned to a view state that shows a quiz view—allowing them to take a quiz.

Clicking a lesson step from the lesson step chooser along the side of the UI shows a “viewer” for that type of lesson step (for instance, an image or a video or an embedded web site or some text, etc.) In addition, this “remotely” causes all students' browsers to be transitioned to a view state that that lesson step in the appropriate type of viewer.

Student View States

Student Brower web app 300B provides screen 137 of device 130 with views (also referred to herein as “student view states”) that may be used by the students before, during, or after a lesson. The following discussion describes various student view states, followed a list of UI widgets which may be used to generate the various student view states.

Student view states may include, but is not limited to the following student view states:

Before Class has Started Student View State: This view state shows an icon for each student as they “arrive” to class. The student icons are randomly distributed around the screen. In the lower right-hand corner of the screen the first step in the lesson plan is displayed. Once the instructor logs in and “starts” class, the instructor view state and student view state transitions to the Everyone In Class view state.

Everyone In Class (Students View) Student View State: This view state shows: the current lesson step's content in the main content area; the instructor's video feed in a square in the upper-right portion of the page; the classroom videos grid 312 along the lower right portion of the page; a top navigation bar along the top of the page; and a page footer along the bottom of the page. FIG. 23 is a screenshot 2300 illustrating one embodiment of the Everyone In Class (Students View) Student View State, and FIG. 24 is a screenshot 2400 illustrating one embodiment of the Everyone In Class (Students View) Student View State during a classroom discussion;

Everyone in Class With One Student Spotlit Student View State: This view state is the same as the Everyone in Class state except that instead of an entire grid of many students' videos showing in the lower right area, the single video stream for the “currently spotlit” student is showing.

BOG Activity In Progress Student View State: This view state shows all of the video feeds of the other members of the BOG in which the student has been grouped. He/she can also see one or more tabs of content that are either read-only or editable. These tabs are what allow the members of the BOG to access the shared content/whiteboard that is an important feature of the BOG activity.

BOG Activity In Progress Instructor “Visiting” One Group Student View State: This view state is similar to the BOG Activity In Progress View State but also shows the instructor's video feed above the embedded BOG view. This view state also allows the instructor to speak directly and privately to only the members of that BOG through their respective browser web apps 300.

BOG Activity In Progress Instructor Broadcasting/Intercom To All Groups Student View State: This view state is similar to the BOG Activity In Progress View State, and also shows the instructor's video feed in a circle that sits above the embedded BOG view. This view state also allows the instructor to broadcast a message to all BOGs at the same time. FIG. 26 is a screenshot 2600 illustrating one embodiment of BOG Activity In Progress Instructor Broadcasting/Intercom To All Groups Student View State;

“Spotlit” BOG Student View State: This view state is similar to the BOG Activity In Progress View State, and also shows the entire classroom videos grid 312. This view state is displayed when a single BOG has been “spotlit” and asked to present their work to the class. FIG. 27 is a screenshot 2700 illustrating one embodiment of the “Spotlit” BOG Student View State;

Poll in Progress Student View State: This view state differs from the Everyone In Class View State in that in that the main content area shows a poll question that the instructor wants all the students to answer.

Showing Poll Results Student View State: This view state differs from the Everyone In Class View State in that the main content area shows the results of a poll question that all the students have answered. Poll results can be published in real time, changing as more students respond. FIG. 25 is a screenshot 2500 illustrating one embodiment of the Showing Poll Results Student View State;

Quiz In Progress Student View State: This view state shows the student the Quiz Widget and nothing else. All of the other pieces of the UI that the student saw during class (such as the instructor video, the classroom videos grid, and the main content) are no longer visible on the screen.

UI widgets which may be used to generate the various student view states include, but are is not limited to the following UI widgets:

Main Content Widget: This widget is described above with reference to the instructor view state widgets.

Instructor Video Feed Widget: This widget is a large, square region of the screen where the instructor's video feed is displayed.

Classroom Videos Grid Widget: This widget is described above with reference to the instructor view state widgets.

User Poll (Student View) Widget: This widget is a view of the poll question that the instructor has asked all students to answer. By the operation of the Student Browser web app 300B, students can select and then submit their answers to a poll using this widget.

User Poll Results Widget: This widget is described above with reference to the instructor view state widgets.

“Remote Control” Video (Student View) Widget: This widget that allows the instructor to control the play, pause, seek of a video. When the instructor uses this widget to start a video or pause a video or seek to a specific play position within a video, the changes the instructor makes are broadcast to all the students' views of the video, allowing the instructor to control the playback of the video. The students can watch the video via this widget but cannot control its playback themselves.

Embedded Web Content Widget: This widget is described above with reference to the instructor view state widgets.

Image View Widget: This widget is described above with reference to the instructor view state widgets.

Embedded BOG View Widget: This widget is described above with reference to the instructor view state widgets.

Top Navigation Widget: This widget presents a bar along the top of the screen. It contains the name of the currently taught class, the current step number, and a drop-down menu full of tools the student can use to fine-tune the way their UI looks during class.

Page Footer Widget: This widget allows the student to turn their microphone and video on or off. There are times during class when the instructor disables these buttons. There are other times during class when the instructor “turns off” the students' videos and/or audio feeds. This widget also has a “raise/lower” hand tool. When a student raises their hand using this tool, it causes their classroom videos grid 312 as well as the classroom videos grids of all other students and the instructor to update and show a “hand” in front of the student's video along with an orange border surrounding their icon. This allows everyone to know that the student is raising their hand. The instructor can then “call on” this student by clicking on them in the classroom videos grid, at which point their image expands to fill the area previously occupied by the entire grid. The student also uses the page footer widget to open and close the classroom chat, BOG chat, and technical support chat.

Quiz Widget: This widget allows the student to answer the 6 or more questions in a quiz. Each question is displayed on a page of the screen. Once the student answers the question, they immediately receive the correct answer if they make a mistake, along with a brief explanation as to why that is the correct answer. They then are transitioned to the next question in the quiz. Once the student has answered all questions, they are transitioned to a final page that shows their total quiz results.

The student may changes/selects view states on their Student Browser web app 300B by clicking on various pieces of the Student Browser web app in a manner similar to that described regarding the Instructor Browser web app 300A.

Technical Support View States

Technical Support Brower web app 300C provides screen 137 of device 130 with views (also referred to herein as “technical support view states”) that may be used by the IT personnel to provide technical support. Technical support view states are identical to the instructor view states, except that the “lesson step selector” in the technical support view is “read only”—it shows what the instructor has selected but does not allow the technical support user to choose which step of a lesson all of the students are viewing.

Recorder View States

Recorder Browser web app 300E provides screen 137 of device 130 with views (also referred to herein as “recorder view states”) that may be used to “observe” a class. The recorder view states are identical to the student view state, with the exception that a non-student can log into device 130A to the recorder view in order to “observe” the class from a student's perspective. The recorder view is used by platform 100 to record classes. The recording software brings up a browser with the recorder view and then “records” all screen and the audio during a class. This results in a recording of an entire class from the student's perspective. This recording can then be provided to students—allowing them to review classes they missed or that they had attended previously.

Instruction Design View States

Instruction Design web app 300G provides screen 137 of device 130 with views (also referred to herein as “instructor view states”) that may be used to design lesson plans 420 for each lesson, enter quiz, midterms, final exams and poll questions, and to interact with the CMS that drives classroom content.

Instruction Design web app 300G allows an instructional designer to choose a course and a specific lesson plan by viewing/editing/creating/deleting lesson plan steps that compose the lesson plan using the Instruction Design web app. The following discussion presents examples of the functioning of the Instruction Design web app 300G.

Each lesson plan step has a lesson step type. When a lesson plan step is added to a lesson plan in the Instruction Design web app 300G, the UI prompts the user as to the lesson step type, such as an image, a video, a BOG, a poll, etc., and presents the appropriate type of “lesson plan step editor” based on the lesson step type in Instruction Design web app 300G.

Instruction Design web app 300G permits the user to: drag-and-drop lesson plan steps to rearrange where/when they occur during a lesson; enter all of the questions for a quiz, mid term, or final exam; edit the properties of a BOG activity, where the BOG activity properties may include, but are not limited to: the number of students to be included in the BOG; the time allocated for the group; assign (or not) a role to each BOG user; rules on how to order or group students so that they are grouped with the appropriate members; the number and types of groups will be formed for the BOG activity (See, for example, FIGS. 14 and 15); specify the shared editable documents and/or read-only documents, images, and videos are to be used by the BOG, including which documents may be used by students depending on their role.

Generation, Storage, and Display of Lesson Plans

The generation of lesson content on platform 100 and the presentation of lesson content to Student Browser web apps 300B is illustrated in FIGS. 4-7, where FIG. 4 is a general schematic of Content Management System (CMS) 320, FIG. 5 is a schematic of an instructional design view of the CMS, FIG. 6 illustrates a live classroom view of the CMS for presenting lesson content, and FIG. 7 is a schematic of College API Servers 350.

In certain embodiments, an instructor produces specifications for the lesson content, an instructional designer uses Instruction Design web app 300G to generate or locate the content for use by platform 100, and the lesson plan may then be accesses by an instructor using Instructor Browser web app 300A and each students using Student Browser web app 300B.

FIG. 4 illustrates that a lesson designer may use Instruction Design web app 300G to store documents, videos, and images comprising CMS Data 410 and Lesson Plans 420 in Database 360. Specifically, the lesson content stored in CMS Data 410 includes text, graphics, or videos to be presented, which may be the actual contents or a link to content accessible over and Lesson Plans 420 includes specifications of each of the lesson steps, including metadata of data stored as CMS Data 410.

In certain embodiments, Instruction Design web app 300G is programmed to accept an input file and to identify and work with four different types of events: 1) content shown directly to the students (e.g., slides in a standard PPT format or videos); 2) materials sent to BOGS (with different materials being sent to different groups, if so specified); 3) polls (which can be pre-determined or configured on-the-fly, with the results shown to the students as a bar graph); and 4) Quiz questions which may be, but are not limited to: multiple choice (including simple True/False); “Mark all that apply” multiple answers; instructions to match entries in one column with those in another column.

FIG. 8 is a schematic of the structure of Database 360, which includes data accessible from a server platform 100 and data accessible over an external network, such as the Internet. Data on platform 100 includes CMS Data 410 and Database 810 which may include User Roles 811 (such as student, instructor, etc.), Lesson Plans 420, an academic calendar 813, Questions, Quizzes, and Polls and Results 550, a List 815 of all users such as students, instructors, admins, designers, etc., BOG Configurations 540, Student Metrics 817, and Course Attendance and Online Status for Each Student 819. Data accessible over an external network includes Live Video Sessions 801, Live Sessions and Messages 803, Shared Whiteboard/Documents, Users and Metrics 805, and Authentication Data 710.

As illustrated in FIG. 8, Databases 360 are accessed by calls to College API Servers 350, which access data stored on platform 100, and which further calls Publish/Subscribe Services 390 for accessing Live Sessions and Messages 803, Video Services 392 for accessing Live Video Sessions 801, and Cloud Hosting Services 370 to access Shared Whiteboard/Documents 805, when needed.

With CMS Data 410 and Lesson Plans 420 stored in Database 360, users of Instructor Browser web app 300A and Student Browser web app 300B may access the lesson content via College API Servers 350. Thus, for example and as noted above, Instructor Brower web app 300A includes a lesson step selector which allows the instructor to control the presentation of lesson content to the Student Browser web apps 300B.

In addition to CMS Data 410, which generally includes documents, videos, or images that are stored locally on platform 100, lesson content may include more complex and/or interactive content such as learning activity specifications or content, defined for example, in BOG Definitions, and quiz, poll, or tests and results, which are also stored on platform 100, and external media, such as text, images, or videos that are stored externally, such as on the Internet.

FIG. 5 illustrates one embodiment of the generation of lesson content, which includes CMS Data 410 and Lesson Plans 420, Shared Whiteboard/Documents, Users and Metrics 805, Embed codes 520, web content embed codes and links 540, BOG Definitions 540 and Questions, Quizzes, and Polls 560, which are stored in Database 360 as shown in FIG. 8. In addition, pointers to the Shared Whiteboard/Documents are stored as Whiteboard/Documents in 510 of College Database 810.

Thus, for example, the instructional designer operates Instruction Design web app 300G to define the lesson steps comprising Lesson Plans 420, locate or input and upload lesson content of each lesson step, which is stored as CMS Data 410, configure and store Shared Whiteboard/Documents, Users and Metrics 805, obtain and store embed codes 530 for videos hosted on third party sites, obtain and store web content embed codes and links 530, generate and store BOG Definitions 540 and generate and store Questions, Quizzes, and Polls 560, both of which are stored in College Database 810. During the storing of CMS Data 410, Instruction Design web app 300G stores CMS Data metadata for each lesson step in Lesson Plans 420.

BOG Definitions 540 may include, but are not limited to: how many students will be first be included in the groups, how much time is allotted, assigned roles (or not) to each BOG user, rules on how to order or group students so that they are grouped with the appropriate members, how many steps there are in the breakout activity, and the rules for forming groups for those steps; for each step in the breakout activity, a specification of shared editable documents and/or read-only documents, images, videos, they should see during the group, and allotted time for each step; and specification of which students, based on their role, can access which documents.

Shared Whiteboard/Documents, Users and Metrics 805 includes template and cloned copies of Whiteboard/Documents for the students to share, permissions for students to access the Whiteboard/Documents, and metrics of student use of the shared Whiteboard/Document.

The presentation of lesson content to Student Browser web apps 300B is illustrated with reference to FIG. 6. During the presentation of a lesson plan, the instructor, through Instructor Browser web app 300A sequences through the lesson step. For example, as described above as one embodiment, above, the Everyone In Class (Instructor View) View State of Instructor Browser web app 300A includes the lesson step selector. This selector allows the instructor to sequence through the lesson steps of Lesson Plans 420. Lesson Plans 420 includes the sequence of lesson steps and metadata for the content of each step.

When sequencing to the next step, platform 100 reads the specification of the next lesson step from Lesson Plans 420 and, based on the specification, such as metadata, calls College API Servers 350 to retrieve the required content from CMS Data 420, to retrieve interactive Shared Whiteboard/Documents, Users and Metrics 805, to retrieve BOG Definitions 540, and/or to retrieve Questions, Quizzes, and Polls 560, and by calling Third-Party API Servers/Services to retrieve content from embed codes 520, and to retrieve content from web content embed codes and links 530, when required. College API Servers 350 then call Publish/Subscribe Service 390, which sends the retrieved content to the Instructor Browser web app 300A and each Student Browser web app 300B, and instructs the browser web apps to change their view state to show the retrieved content.

FIG. 7 is a schematic that illustrates the use of College API Server 350 and a Load Balancer 701. College API Server 350 may consist of several identical codes running on one of multiple API servers. Load balancer 701 intercepts calls from browser web apps 300 to the College Server API and determines which specific College API Server 350 will handle the call. This ensures that each server does a portion of the work and no single server is doing all the work of all the College API calls. Load balancer 701 also tracks which API servers are operating, thus ensuring that platform 100 will continue to operate if a server crashes.

Student Metrics

One important aspect of platform 100 is tracking student metrics, which are stored as Student Metrics 817. Student metrics related to whiteboard use are stored in Shared Whiteboard/Documents, Users and Metrics 805.

Student metrics may be determined from the student's current or previous use of their Student Browser web app 300B, examples of which include, but are not limited to:

• • the student's answer to a question or set of related questions prior to forming one or more BOGs, such as the answer to a quiz question or set of related questions about a lecture that was just delivered; answers either to a very specific quiz administered immediately before the BOG or to a set of previous quizzes, where the answers are coded by learning outcome and stored in a data base; responses to poll questions; and chat entries;
  • the student's interaction activity with the on-line class prior to forming the one or more BOGs, as determined by their asking questions during a lecture; the amount of keyboard or microphone activity; responses to polls. and chat entries;
  • a measure of the student's interaction with other students of the plurality of students determined from one or more previous breakout sections, where the measure is: obtained from audio interactions with other members of the previous BOG or groups; determined from interactions in a written document with other members of the previous BOG or groups; or is determined from what specific combinations of students in previous BOGs resulted in a large amount of group interaction;
  • one or more performance capabilities assigned to each student based on prior performance on specific quiz items or other assessment measures that are coded to assess a specific capability (i.e., skill or aspect of knowledge), assign a measure of one or more capabilities to each student. Performance capabilities may include, but are not limited to, specific aspects of: analytical skills; critical thinking skills; self-management skills; personal interaction skills; learning skills; problem solving skills; and/or communication skills;
  • the student's interaction activity with the on-line class after forming the one or more BOGs as determined from the student's microphone activity; whiteboard activity; or cursor activity in shared documents; and
  • the roles played by students in BOGs.

Use of Publish/Subscribe

Publish/subscribe services 390 may be used in platform 100 to perform a variety of tasks, such using an Instructor Browser web app 300A to direct lesson content to Student Browser web app 300B, to permit video or text chat between various browser web apps 300, for conducting interactive learning activities, for controlling another device 130 remotely, as described above in the discussion of FIG. 3.

FIG. 9A is a schematic that illustrates how publish/subscribe services are used in the classroom setting. First, an instructor sequences to the next lesson step in Lesson Plan 420 using the lesson step selector on Instructor Browser web app 300A. This results in Instructor Browser web app 300A calling College API Server 350, which retrieves the content of the next lesson step and a list of students in the class stored in Course Attendance and Online Status for Each Student 819.

In the next step, College API Server 350 calls Publish/Subscribe Service 390 and provides the service with the retrieved lesson content and list of students.

In next, Publish/Subscribe Service 390 provides the Student Browser web app 300B for each student in the retrieved list of students with the content of the next lesson step.

FIG. 9B is a schematic that illustrates how publish/subscribe services control student's behavior and views. First, using Tech Support Browser web app 300C, tech support personnel send a call to College API Server 350, examples of which include but are not limited to: refresh browsers; flash a message to one student or an entire class; move a student between BOGs; set microphones on or off; turn videos off; or force the log off of a students.

In the next step, depending on the call, College API Server 350 updates information in College Database 810, such as lesson and student attendance 819 or BOG configuration 540.

Next, College API Server 350 calls Publish/Subscribe Service 390, which then sends the appropriate message to Student Browser web apps 300B which may be, for the above examples: refreshing browsers; flashing a message to one student or to the entire class; move students between BOGs; turn microphones on or off, videos off, or force a log out.

In the final step, Publish/Subscribe Service 390 provides the Student Browser web app 300B for each student in the retrieved list of students with the content of the next lesson step.

Platform 100 also provides for communication between those using the platform is facilitated by having several types of chat. Classroom chat occurs between Instructor Browser web app 300A and Student Browser web app 300B. Classroom chat is initiated from either Instructor Browser web app 300A or Student Browser web app 300B, and can be viewed from the Instructor Browser web app and all Student Browser web apps. Classroom chat stream is automatically organized thematically, with comments that are responding to the same message are grouped together.

Help chat occurs between Tech Support Browser web app 300C and Instructor Browser web app 300A can only be seen by help desk personnel and teaching assistants. Help chat also includes controls for the viewing mode, including but not limited to: the size of the student matrix and whether scrolling is enabled.

BOG chat occurs between the students using Student Browser web app 300B that are within a BOG, and can be monitored by the instructor with Instructor Browser web app 300A.

Person to person chat occurs between Tech support and the student, between Tech support and the instructor, or between the Instructor and Student.

Tech support chat may be used by Instructor Browser web app 300A to “remote control” one or more Student Browser web apps 300B. Remote control actions include, but are not limited to one of the following: send “current lesson step” meta data and data to each student; send “flash messages” to one or more students—this is a bright popup that covers the rest of the content and is a “high priority” notification to the user; refresh the user's video streams in the event that the user has contacted tech support with problems regarding video streams; “refresh the user's browser”—which reloads the browser web app 300 and can clear away intermittent bugs; turn students' microphones and/or videos off; enable/disable students' ability to turn their own microphones on or off; enable/disable students' ability to “raise their hands” using the button along the bottom of their web apps; enable/disable students' ability to “open and use classroom chat” using the button along the bottom of their web apps; transition the students' web apps into different “views” and “states”. For instance, at the end of class, transition students into a quiz—where they see a UI that allows them to answer questions; move students between BOGs; transition students' browsers into a view/state where they see the other members of their BOG and see the shared content their BOG is working on; “spotlight” a BOG. This transitions everyone's web apps into a state where they see a selected BOG, its content, and a grid of the entire classroom; or spotlight a specific student.

FIG. 10 is a schematic that illustrates how publish/subscribe services are used for chatting. First, a student interacts with a chat widget in Student Browser web app 300B to pose a question to the class. This results in Student Browser web app 300B sending the text to College API Server 350, which stores the messages in Database 810 and retrieves the list of students stored in Course Attendance and Online Status for Each Student 819 and chat message for each lesson 1001 from Database 810.

In the next step, College API Server 350 calls Publish/Subscribe Service 390 and provides the service with the retrieved lesson content and list of students. Publish/Subscribe Service 390 stores the message in Send/Receive Messages 803.

In the final step, Publish/Subscribe Service 390 provides the Instructor Browser web app 300A and Student Browser web app 300B for each student in in the retrieved list of students with the messages, which is displayed within the web app.

Instruction Design

An instructional designer operates Instruction Design web app 300G to design Lesson Plan 420 as described above with reference to FIG. 5. Designing a lesson plan may include, for example and without limitation, uploading slide images for a lesson step, designing polls and quizzes, editing the description/meta-data of one more BOG activities, where each activity may have one or more steps, and making a reference to a URL on the internet that points to: a static image, a static PDF, a video, or an interactive shared whiteboard/document), and creating the templates and rules for shared BOG workspaces. To design a poll, a “poll designer” page is used to design a single-question poll. To design a quiz, Instruction Design web app 300G presents a “quiz designer” page that is used to create the quiz. For both quizzes and polls, a “question editor” widget for each question is used to create and/or edit the question.

FIG. 11 is a schematic that illustrates how publish/subscribe services are used for learning activities. First, an instructional designer operates Instruction Design web app 300G to define a BOG of Lesson Plans 420 by configuring templates of whiteboard/documents and user permissions for a BOG. The whiteboard/documents templates are stored in Shared Whiteboard/Documents 805. The rules for forming BOGs are stored in BOG Definitions 540 and pointers to the Shared Whiteboard/Documents are stored in Whiteboard Templates 510. During class the whiteboard/documents templates are cloned and stored as Cloned Whiteboard/Documents in Shared Whiteboard/Documents 805, with pointer stored in in Shared Whiteboard/Documents 110 of College Database 810. Using Student Browser web app 300B, and according to the BOG definitions, students are directed to their BOG's Cloned Whiteboard/Documents in Shared Whiteboard/Documents 805.

Performance Assessment

Active learning is enhanced in platform 100 by determining how well the students comprehend and remember what is being taught in each lesson. Performance assessments may include, but are not limited to, polls conducted at regular intervals (such as one every 15 minutes or less), polls conducted at key moments (e.g., after a specific concept has been conveyed in lecture), by quizzes (both embedded midstream during a class and at the end of class), midterms, and final exams. Quizzes, polls, midterms and final exams are each a lesson step of Lesson Plan 420.

In certain embodiments, each student must have perfect quiz scores, after taking a version of the quiz (each with different specific questions) up to a criterion number of times (e.g., six). To facilitate this, system 100 monitors the student's responses and forms one or more questions in the quiz to help achieve this goal. Thus, for example and without limitation, during a quiz lesson step, system 100 presents a mixture of questions that are previously unseen by the student and questions that were previously answered incorrectly.

FIG. 12 is a schematic illustrating the creation of quizzes, midterms, or final exams (referred to herein and without limitation as “quizzes”), and showing a quiz that is presented in Student Browser web app 300B, Quiz Creation Logic 1200, and, as stored in Questions, Quizzes, and Polls and Results 550 of Database 810, data including a Per-Course Quiz Question Bank and Previously Taken Quizzes and Questions for Each Student, and, for each question, one or more associated lesson plans 420 and one or more associated Nano competency/subject area.

The order of preference in selecting questions by Quiz Creation Logic 1200 is, first, questions that have not been presented to the student before, followed by questions previously answered incorrectly by the student, followed by questions previously answered correctly by the student.

Quiz Creation Logic, which is programmed into platform 100, includes 3 steps. In the first step, In Step 1201 database 550 is queried to determine the type of quiz: an end-of-lesson quiz, a midterm, or a final exam and how many questions should exist in the quiz based on each Nano competency/subject area being tested by the quiz.

In Step 1203, database 819 is queried to determine whether a student is currently enrolled in the class.

In Step 1205, questions are selected for each student based on the student's performance and the type of quiz. The questions are selected from a question bank of questions from the current lesson plan 420 and prior lesson plans, as stored in database 550, as follows:

• • If the “quiz” is an end-of-lesson quiz, then 5 questions are selected corresponding to the current lesson plan and 1 question is chosen randomly from any prior lesson plan.
  • If the “quiz” is a midterm, then 2 questions are selected for each Nano competency/subject area of the lesson plan.
  • If the “quiz” is a makeup midterm, then the selected questions are all questions that the student answered incorrectly in the previously taken midterm.
  • If the “quiz” is a final exam, then the selected questions are 2 questions for each Nano competency/subject area of the lesson plan and questions in the Nano competency/subject area of the lesson plan that the student answered incorrectly in the previously taken midterm
  • If the “quiz” is a “makeup” final exam, then then the selected questions are all questions that the student answered incorrectly in the previously taken final exam.

With each student's quiz questions 1220 thus determined, CMS 320 provides the determined quiz questions to the each students Student Browser web app 300B.

After the student as answered the quiz questions, platform 100 grades the quizzes and stores each student's results in Questions, Quizzes, and Polls and Results 550.

Learning Activity Groups

In certain embodiments, a lesson step includes forming groups of students that work together on an activity. One such learning activity is a BOG, in which each student is provided with instructions for the learning activity and shared space, such as a document, to which each student may contribute.

FIG. 13 is a schematic which illustrates one embodiment for how system 100 forms BOGs during a lesson, and based on the BOG definitions using Instruction Design web app 300G. BOGS are formed during while executing the lesson plan according to Group Creation Logic 1300, which has access to a list of students 1320 in the course, which is obtained from Course Attendance and Online Status for Each Student 819, shared Whiteboard/Documents templates 805, student quiz scores 550 and the BOG configurations 540.

In certain embodiments, Group Creation Logic 1300 includes step 1301, where the type of group to be created is determined. Examples of how group are arranged may include, but is not limited to, grouping by adjacent ranking, scores, or performance capabilities. In addition, the type of BOG may have predetermined performance capability or capabilities or roles for participants in the BOG for the task to be used in the BOG to which the student is assigned, which is also stored in BOG configurations 540, and the grouping takes the BOG requirements into account to achieve a balance of students in each group.

Next, in Step 1303, students 1320 are ranked, sorted, or ordered according to the type of groups. Thus, for example, there may be no predetermined or preferred ordering of students, and the students may be “ranked” by a random order or in alphabetical order. Alternatively, the type of group may require ranking or sorting according to Student Metrics 817 which may include but is not limited to student's answers to questions, interaction activity prior or during class, interaction with other students, performance capabilities, and/or student roles.

In certain embodiments, groups may be formed of students automatically based on their prior performance on the capabilities required for the task. Thus, for tasks where at least one person must have previously mastered at least one relevant capability, students may be assigned so that at least one such person is present in every BOG. For tasks where the group is engaged in tasks (e.g., problem solving, role playing, debate) where comparable levels of the relevant capabilities are desirable, so that students are not intimidated or looking down at other members of the group, students are assigned with comparable levels of prior performance of the task-relevant capabilities.

Next, in Step 1305, BOGs 1310 are formed into individual BOGs 1311, 1312, 1313 . . . based on the creation criteria and the ranking of students. And lastly, in Step 1307, each student within each group is provided, through their Student Browser web app 300B, with a shared work environment for their group, which may include a shared whiteboard, as discussed herein.

FIG. 14 illustrates forming BOGs with students having similar ranking, such as by the student metric of quiz scores. Students 1320 are arranged in order of increasing quiz scores, and are formed into BOGs 1410 comprising individual groups, shown illustratively as BOGs 1411, 1413, 1415, and 1417. At future time, the BOGs may be formed into larger BOGS 1420, as a BOG 1421 consisting of the students of BOG 1411 and 1413, and a BOG 1423 consisting of the students of BOG 1415 and 1417.

In an alternative embodiment, BOGs 1420 are formed from BOGs 1410 by providing BOGs with instructions to perform some action or have a discussion. Each group spends a few minutes reading and attempting to understand the instructions. BOGs 1420 are formed from pairs of groups from BOGS 1410, where the pairing is random within the role or category of the group. For example, if there are two roles (A and B), only two groups with the same roles will be paired.

The members of each BOG 1420 then attempt to explain the instructions to each other. If any “paired” groups can't come to a consensus/understanding of the instructions, they can “raise their hands”—alerting the instructor to come “visit” their group and explain the instructions in more depth. Or if the instructions are really not being understood, the instructor can do a “broadcast” audio/video chat to the entire class—clarifying for all of them the meaning of the instructions.

FIG. 15 illustrates forming BOGs with students based on more than one student metric. By way of example, students 1320 are arranged in order of increasing quiz scores and identified by their role, indicated as a Role A, B,C, or D. Students 1320 are formed into BOGs 1510 comprising individual groups each having students with the same role. Thus, as illustratively, BOGs 1511 includes students of role A and adjacent quiz scores, 1513 includes students of role B and adjacent quiz scores, 1515 includes students of role C and adjacent quiz scores, and 1517 includes students of role D and adjacent quiz scores, At a future time, follow-on BOGs may be formed into BOGs 1520 from student having different roles and adjacent score of students with those scores. Thus, for example, BOG 1521 consists of one student with each role, such as the A role student with the highest score, the B role student with the highest score, the C role student with the highest score, and the D role student with the highest score. BOGs 1523/1525 consist, respectively, of the A role student with the second/third highest scores, the B role student with the second/third highest scores, the C role student with the second/third highest scores, and the D role student with the second/third highest scores. In general, sequences of BOGs may be created where members of each group each are assigned to a subsequent group that will have one member from each of the previous groups. In addition, early groups may be reconstituted at any time, and students returned to those groups. As many as five groups in sequence may be formed, each created automatically based on criteria specified in the lesson plan.

An example of sequential BOGS will now be discussed with reference to a lesson to teach students negotiating skills, in particular the technique known as Best Alternative To a Negotiated Agreement (BATNA). The purpose of BATNA is to find an advantageous alternative that a negotiating party can take if negotiations fail and an agreement is not reached.

In this example, the students are automatically put into several groups that role play negotiating water rights, where each student is assigned a stakeholder role, such as a farmer, an environmentalist, farmers, home owners, engineers and environmentalists. This example will have 3 different sequential BOGs, Phase I (or BOG 1), followed by Phase II (or BOG 2), followed by Phase III (or BOG 3).

For BOG 1, 4 stakeholder groups are formed—that is, each group consists of students playing the same stakeholder role. During BOG 1, the students are instructed to devise a strategy for negotiating, including a BATNA. During BOG 1, the students know that BOG 2 and BOG 3 will follow.

After 10 minutes, new groups (BOG 2) are formed, which each include one representative from each of the previous groups, such that each student is the sole representative of their stakeholder. The students then simulate negotiations and attempt to discover the BATNAs of other stakeholders.

After another 10 minutes, the original group are reformed in BOG 3. The students report inferences about the BATNAs for each of the other stakeholders.

Lastly, all students return to class. Each of the groups forming BOG 1 summarize one BATNA they inferred, and that group then verifies their inferences.

As this example illustrates, we provide incentives and consequences for each student at every phase, since their groups are relying on them to learn the material. System 100 may also administer a quiz right after the introductory lecture whose answers can be used to automatically compose BOGs so that there is a range of competency.

Bog Definitions

An example of the use of Instruction Design web app 300G to define BOGs is illustrated in the screenshots of FIGS. 20A-20F.

FIG. 20A shows a screen 2000A which provides accesses tools to set up rules for forming a BOG. Screen 2001 includes a viewing area for the steps of the lesson plan and the current lesson plan step being edited as step 2002. Instruction Design web app 300G includes a BOG Widget 2004 which, when clicked, provides possible applications to the BOG, such as how to organize the BOG, which may be for example and without limitation, on how to pair up students, or an activity, which may be for example and without limitation, to view a document, edit a document, or view an image, or to paraphrase provided instructions with another BOG.

Thus, for example, in screen 2001 the BOG Widget was previously selected to produce an application 2011, labeled “Organize, Create BOGs with 2 Students each “Two Roles, Groups of 2,” an application 2012 labeled “Activity” with an indication 2013 that the embedded documents are for the “A” role students, an application 2014 labeled “Activity” with an indication 2015 that the embedded documents are for the “B” role students, an labeled 2016 “Activity” with an indication 2017 that the embedded documents are for all students in the group, an application 2018, labeled “Pair Up, Pairs of BOGs work Together” with an indication 2010 that this activity is for all students in the group, and an application 2020, labeled “Activity” with an indication 2010 that this activity is for all students in the group

From the Organize application, the user may define the composition of BOGs, such as if the students have roles (such as none, A, B, C, etc.), the composition of the group, which may be, for example and without limitation, homogeneous, where the roles are in the same group, such that six students might divide into AAA BBB, AA BB CC, etc., heterogeneous, where the roles are in different groups, such that six students might divide into AAB ABB, ABC, etc., and group size (such as an ideal number of students, a min number of students, or a max number of students).

Another way to organize students is to “pairify,” in which group are initially formed from an even number of groups which may be given activities individually or in pairs of groups, as illustrated in FIG. 14 or FIG. 15.

FIG. 20B shows a screen 2000B which being used to organize the BOG, where the group name is provided in portion 2003, the group is defined a having 2 roles (A and B) in portion 2004, the group is defined as having a homogeneous composition in portion 2005, and the group sizes are defined in portions 2006 and 2007.

FIG. 20C shows a screen 2000C which being used to provide a shared whiteboard for the group. A link to the shared documents is provided in portion 2008, and an image of the document to be embedded is shown in portion 2009.

Instructor Bog View

As discussed above, BOG Activity In Progress View State presents a BOGs Heat Map Widget, a Move Students Between BOGs Widget, and an Embedded BOG Widget, which shows the instructor the currently selected BOG.

FIG. 16 illustrates the generation of the BOGs Heat Map Widget 1600 of Instructor Browser web app 300A. The BOGs Heat Map Widget 1600 presents a graphic representation of activity in BOGS, shown illustratively and without limitation as BOGs 1410. During the breakout session, each BOG interacts with shared whiteboard/documents though Cloud Hosting Service 370 and with each other through Video Services 392 and a chat service provided by Publish/Subscribe Service 390. Instructor Browser web app 300A places calls to College API Servers to extract BOG metrics, which may, for example and without limitation, be metrics of in-group video/audio interactions between students, in-group chat message between students, and in-group document interactions. Specific examples of BOG metrics may include, but are not limited to, how much talking is taking place; how much activity is taking place in the shared work product; how much use is being made of Group Chat; the average “level” (determined by quiz performance) of the group; the average “initiative” level (determined by how many times the members have previously raised their hands during class); the average amount of talking; the variance in talking, indicating how evenly talking is distributed across group members; the variance in amount of time cursors are engaged in the shared work product, indicating how evenly work production is distributed across group members; ratios consisting of the person with the greatest activity (e.g., amount of talking or cursor activity) being divided by the average of the activity (e.g., amount of talking or cursor activity) of all other members (which provides a measure of whether one person is dominating). The BOG metrics are directed, by a call from though Publish/Subscribe Service 390, into BOGs Heat Map Widget 1600. being divided by the average of the activity of all other members (which provides a measure of whether one person is dominating). The BOG metrics are directed, by a call from though Publish/Subscribe Service 390, into BOGs Heat Map Widget 1600.

The BOGs Heat Map presents each BOG as a similarly shaped object, such a rectangle, that is labeled, for example, with the names of students in the group. Thus, for illustrative purposes, BOG 1411 is represented by a rectangle 1601, BOG 1413 is represented by a rectangle 1603, and BOG 1415 is represented by a rectangle 1605,

In one embodiment, the BOG metrics for each BOG are converted into a fill color for the corresponding rectangle of the BOGs Heat Map Widget. In one embodiment, the color scheme is chosen to permit a quick visual determination of any outliers among the different groups.

Thus, for example, the color corresponding to the metric of BOGs 1411/1413/1415 fills the interior of rectangles 1601/1603/1604, respectively. A visual inspection of rectangles 1601/1603/1604 provides a quick way of determining the relative performance of groups, and may provide the basis for the instructor to “drop in” on specific BOGs.

If, for example, the BOG metric is the amount of in-group interactions, the instructor can quickly determine groups that may be having problems with the activity and may switch to the BOG Activity In Progress Instructor “Visiting” One Group View State, which permits the instructor to communicate with members of the selected BOG.

Moving Breakout Whiteboards to the Classroom View

In certain embodiments, platform 100 enables BOG shared whiteboards to be shared with the entire class, as will be discussed with reference to FIG. 28 and FIG. 29 which shows one embodiment of a screenshot 2900 of a document from a breakout session; and FIG. 30 which shows one embodiment of a screenshot 3000 of a document from a breakout session imported into the Everyone In Class (Students View) Student View State. FIGS. 29 and 30 illustrate the moving of breakout whiteboard to the classroom view, where FIG. 29 is a screenshot of a document from a breakout session; and FIG. 30 is a screenshot of a document from a breakout session imported into the Everyone In Class (Students View) Student View State.

Thus, for example FIG. 28 is a screenshot of a classroom in “BOG Spotlight” mode. In this mode a single BOG is sharing the document they have been working on. To share the shared document with the class, the instructor clicks the “Open” button in the upper right corner of the document. This opens a new browser tab with the shared document, as shown in FIG. 29. The instructor then creates a new “Ad Hoc Page” by clicking a button available in the instructor application's footer area. The instructor pastes the URL of the document from the new browser tab into the editor for the Ad Hoc Page.

Lastly, the instructor clicks the lesson step widget that is along the side of her screen and represents the “Ad Hoc Page” to the class. The instructor and all of the students in the class are viewing the document that was originally edited by the BOG, as shown in FIG. 30.

Classroom Videos Grid

Each Student Browser web app 300B generates a video feed from the camera of the student's device 130. To provide the appearance of a classroom of students, it is desirable for the students and the teacher to view as many of these video feeds as possible. FIG. 17 illustrates a Classroom Videos Grid 1700, which is generally similar to classroom videos grid 312, except as explicitly stated, and which is presented on the Instructor Browser web app 300A, any of the Student Browser web apps 300B, or on other browser web apps 300.

FIG. 17A illustrates the display of a plurality of video feeds 1701 in the Classroom Videos Grid 1700. In certain embodiments, each video feed of the plurality student's video feeds is obtained from one of the plurality of Student Browser web apps 300B and are arranged on a rectangular grid for display within an area 1703 of the Classroom Videos Grid. Thus, for example and without limitation, video feeds 1701 in FIG. 17A are shown, for example and without limitation, as being arranged in columns for 4 video feeds and extending as far horizontally as is necessary to fit all of the video feeds, and will repeat to form an endless loop of videos. Of video feeds 1701, video feeds 1720 are at least partially within area 1703, video feeds 1710 are just about to scroll into area 1703, and video feeds 1730 have just previously scrolled through area 1703.

Classroom Videos Grid 1700 scrolls all of the video feeds horizontally, so that a portion of all of the video feeds are visible at one time, such that each video is scrolled across area 1703 within some reasonable time, such as 10 seconds, in an endless loop. Over the course of a specific amount of time, such as two minutes, all of the video feeds will scroll through area 1703. When it is determined that a video feed is to be displayed in Classroom Videos Grid 1700, video service API call is made to retrieve the video feed for display. In various embodiments, area 1703 and video feeds 1701 are arranged to display, for example, 1×1, 2×2, 3×3, or 4×4 of video feeds simultaneously. In certain embodiments, the video feeds are turned on just prior to moving into area 1703 so that the effect is to view a continuous movement of video feeds.

In certain embodiments, the plurality student's video feeds are arranged, for example and without limitation, by virtually arranging the video feeds on a rectangular grid having the height of area 1703 and moving the virtual arranged video feeds across a rectangular area that corresponds to area 1703. Thus, for example, if area 1703 has the height to accommodate 4 video feeds, then the grid will have a height of 4 video feeds. The movement of the video feed across area 1703 is incremental—that is they move incrementally by a distance less that is less than width of the video feed, and thus appear to crawl across the area. As the video feeds move, the video feeds that are streamed to the browser are only those video feeds that: are at least partially in area 1703; or will be at least partially in area 1703 at the next increment of time are provided to the Classroom Videos Grid 1700 for display. Thus, all of the video feeds will appear to move smoothly across area 1703 (moving from left to right or from right to left) while minimizing the number of video feeds sent to the browser for viewing.

In certain embodiments, the virtual arrangement of videos on the grid is fixed as the videos scroll. In certain other embodiments, the arrangement of videos is randomized on the grid. In other embodiments, copies of one or more video feeds are duplicated and placed on the grid, in a fixed or randomized arrangement and the copies may be replaced after moving off of area 1703. The placement of video feed copies on the grid has the effect of the student's not being able to predict when their video feed will be presented.

FIGS. 17B, 17C, and 17D illustrate three sequential views of the Classroom Videos Grid 1700, with adjacent video feed columns 1741, 1742, 1743, 1744, 1745, 1746, and 1747 as they scroll past within area 1703.

FIG. 17B shows video feeds 1701 at a first time, where at least partially viewable video feeds 1720 includes video feed columns 1742, 1743, 1744, 1745, and 1746, where video feeds 1710 includes video feed column 1741, and where video feeds 1730 includes video feed column 1747.

In certain embodiments, the horizontal scrolling of video feeds 1701 is controlled by JavaScript code within the browser web app 300. The JavaScript code operates as a “timer” that is continuously reset after some time interval, at which time the JavaScript code moves the plurality of video feeds 1701 a distance to the left.

FIGS. 17C, and 17D illustrate additional sequential views of the Classroom Videos Grid 1700 after the view of FIG. 17B. When a video feed column has moved so that it is no longer at least partially within area 1703, all of the video streams in that column are shut down. Thus, for example, in going from the view of FIG. 17B to that of FIG. 17C, video feed column 1746 has moved out of area 1703, and thus the video feeds comprising video feed column 1746 are turned off, and in going from the view of FIG. 17C to that of FIG. 17D, no video feed columns have moved out of area 1703, and thus no video feeds are turned off.

When a video feed column is about to move into within area 1703, all of the video streams in that column are turned on. Thus, for example, in going from the view of FIG. 17B to that of FIG. 17C, video feed column 1741 is about to move into area 1703, and thus the video feeds comprising video feed column 1741 are turned on, and in going from the view of FIG. 17C to that of FIG. 17D, no new video feeds are about to move into area 1703, and thus no video feeds are turned on.

During a class a student may “raise their hand” by interacting with the Page Footer Widget of the Student Browser web app 300B, as described above. When the student raises their hand using the “raise/lower” hand tool, the Classroom Videos Grid 1700 brings that student's video feed into the Classroom Videos Grid 1700.

FIGS. 18A and 18B illustrate the effect of the “raise/lower” hand tool on the Classroom Videos Grid 1700. In the example of FIG. 18A, a student whose video feed is not within area 1703 has activate the “raise/lower” hand tool on their Student Browser web app 300B. As illustrated in FIG. 18B, Classroom Videos Grid 1700 responds to this activation by scrolling the video feeds so that that student's video feed is within area 1703, and also superimposes a small raised hand image over the student's video feed and surrounds the icon with an orange border so that it is clear which student is requesting attention. The icons for students with raised hands are not scrolled; they are pinned in the matrix so that they are visible until the instruction either clicks on them or dismisses them.

In one embodiment, Student Browser web app 300B presents a “raise hand” tool in a Page Footer Widget, as discussed above. When a student clicks the “raise hand” tool, a call is placed to College API Servers 350 and then to Publish/Subscribe Service 390, as discussed above with reference to FIGS. 7 and 8. The identity of that Student Browser web app 300B is stored in database 803, and the Publish/Subscribe Service 390 sends a message out to all Instructor Browser web app 300A, Student Browser web apps 300B, Tech Support/Observer Browser web apps 300C and Recorder Browser web apps 300D that are running. When each browser web apps 300 receives this message, the scrolling of Classroom Videos Grid 1700 is stopped, each browser web app determines if the video feed of the Student Browser web app 300B having clicked the “raise hand” tool is within area 1703 and arranges Classroom Videos Grid 1700 such that that video feed is within area 1703. and an orange rectangle and a small orange hand is displayed along the edge of that video feed.

If the instructor operating Instructor Browser web app 300A wants to “call on” the student with the raised hand and “put them in the spotlight,” they click on the student image in the matrix of videos on the Instructor Browser web app. A call is then placed to College API Servers 350 which then calls Publish/Subscribe Service 390, sending instructions to change the Classroom Videos Grid 1700 in each browser web app to replace all of the other videos feeds in Classroom Videos Grid 1700 to display only the student's video feed of the called on student. In addition, the Instructor Browser web app 300A makes a call to the College API Servers 350 telling it that the clicked on student is “in the spotlight.” The server then uses Publish/Subscribe Service 390 to send the “student is spotlit” message out to all of the Student Browser web apps 300B, Tech Support/Observer Browser web apps 300C and Recorder Browser web apps 300D that are running. The browser web apps 300 receive a “student is in the spotlight” and the browser web apps switch to a spotlit view. Thus, for example, the Instructor Browser web app 300A displays the Everyone in Class With One Student Spotlit Instructor View State, and all Student Browser web apps 300B displays the Everyone in Class With One Student Spotlit Student View State.

FIGS. 19A and 19B illustrates the effect of an instructor spotlighting a student from Classroom Videos Grid 1700. During a class the instructor may wish to “spotlight” one student and only show the spotlighted student's video feed. As illustrated in FIG. 19A, the instructor operating Instructor Browser web app 300A may spotlight a student by clicking on the student's video feed 1901 from within area 1703. As illustrated in FIG. 19B, Classroom Videos Grid 1700 responds to this selection by switching to a 1×1 view of that video feed, effectively viewing and broadcasting only the spotlit student to browser web apps 300.

Once the instructor has called on a student and put them “in the spotlight,” they can then take them out of the spotlight. To do this, the instructor operating Instructor Browser web app 300A clicks on the spotlit student's video, resulting in a call being placed to College API Servers 350 which then calls Publish/Subscribe Service 390, which causes student's video to be removed and then replaced with the matrix of the entire class' videos as Classroom Videos Grid 1700. Specifically, messages are sent to all Student Browser web apps 300B, Tech Support/Observer Browser web apps 300C and Recorder Browser web apps 300D that are running to resume showing the Classroom Videos Grid 1700.

Structure and Use of the Platform for Classroom Instruction

The following discussion is centered on the experience of the users of platform 100, and it will be understood that the actions of instructors are by way of interaction with Instructor Browser web app 300A, the actions of students are through their Student Browser web ap 300B, and the actions of instructional designers using Instruction Design web app 300G.

In certain embodiments, the steps of using system 100 may include:

• • 1. an instructional designer uses Instruction Design web app 300G to define lesson plan 420, as described for example, in the discussion of FIG. 5;
  • 2. a school administrator “opens class” using School Admin Browser web app 300H, causing system 100 to prepare for the teaching of lesson plan 420. Thus, for example and without limitation, databases are updated to reflect the upcoming teaching of lesson plan 420 to enrolled students, and if required, clones copies of any shared documents, as discussed above with reference to FIGS. 5 and 11 and prepares quizzes, as discussed above with reference to FIG. 12.
  • 3. the instructor logs onto system 100 using Instructor Browser web app 300A, which displays the Before Class has Started Instructor View State and students log into the system using Student Browser web app 300B, which displays the Before Class has Started Student View State.
  • 4. At the instructor's discretion or at the appointed class start time, the instructor transitions to the Everyone In Class (Instructor View) Instructor View State, which sends publish/subscribe messages to the Student Browser web apps 300B to transition to the Everyone In Class (Students View) Student View State.
  • 5. the instructor sequences through the lesson plan steps and publish/subscribe messages are sent to the Instructor Browser web app 300A and Student Browser web apps 300B to transition the browsers to view states defined by lesson plan 420.
  • 6. when a quiz lesson plan step is selected, system 100 sequences through steps described above with reference to FIG. 12 to determine which question(s) are presented to each student.
  • 7. when a BOG lesson plan step is selected, system 100 uses stored performance data to assign students to specific BOGs, as discussed above with reference to FIGS. 13, 14, and 15. Alternatively, in some embodiments students are assigned to BOGs at random or the instructor manually assigns students to BOGs. Following assignment, publish/subscribe messages are sent to transition the instructor view to BOG Activity In Progress Instructor View State and the student view state to BOG Activity In Progress Student View State. The students may interact with other students of their BOG and the instructor many monitor BOG groups using the BOG heat map and visit individual BOGs, as discussed above with reference to FIG. 16. In addition, the instructor may operate, or request operation, of Tech Support/Observer Browser web app 300C to move students between BOGs, as discussed above, and bring content from the BOG to the entire class, as discussed above regarding FIGS. 28-30.

Platform 100 includes many features which may be used to provide an active-learning class of a course over a computer network. These features enable active learning by empowering teachers by giving them access to tools that promote active learning; prepares students for active learning by engaging them when they are exposed to new material and presenting active-learning tasks; collecting and using data to optimize the active learning environment, automatically tailoring class instruction based on an assessment of at least one student's performance during the course; creates and conducts BOGs partly on the basis of data about previous student performance; and uses may be used to adapt the system to improve the active learning environment over time.

During lecture, students are given a poll, respond, and a graph of the results is shown to the class. Several students are then called on to explain their votes. Students often must answer a query by typing into Group Chat. The instructor then selects some of the responses and asks those students to expand further to the class.

In certain embodiments, lesson plan 420 include a mixture of lectures and BOGs in various proportions, Thus, for example a primality lecture class may have a ratio of 75% lecture/25% BOG, while a primality laboratory class may have a ratio of 25% lecture/75% BOG.

Platform 100 provides multiple ways for students to interact with instructor and class, including but not limited to: “raise their hands,” enter answers into chat and email/tech support/etc.

Instructors are provided with dashboards/reports/very detailed audit trail of: attendance in each lesson: number of minutes attended each lesson; time “on camera” vs “off camera” during lesson; quizzes attempted vs. quizzes passed; number of interactions with “classroom chat” stream; number of interactions with “BOG chat” stream and number of interactions with documents/shared whiteboard during BOGs

Technical support tools allow instructors and/or technical support to: send “flash” messages to individual students; turn individual students' audio/video off if necessary; and refresh/reset browser (web applications) of students who are “stuck” in some way

Platform 100 may be configured to provide each student with a survey at end of each class to gather students' experiences, opinions, etc. These surveys are fed back to the admissions process as well as the AI/ML system that helps allocate future students to BOGs. This data also helps coaches/mentors to proactively help students with issues/problems

In certain embodiments, platform 100 includes the following structure or elements.

• • 1. Platform 100 may accommodate up to 200 students in a combination of lecture and active-learning BOGs.
  • 2. Platform 100 has at least two types of chat:
    • a. Classroom Chat can be viewed by all students and all faculty. The classroom chat stream is automatically organized thematically, so that comments that are responding to the same thing are grouped together.
    • b. Help Chat can only be seen by help desk personnel and teaching assistants. This chat also includes controls for the viewing mode (i.e., the size of the student matrix and whether scrolling is enabled; see below).
  • 3. Platform 100 allows the user to define four different types of events as lesson plans 420:
    • a. Content provided from PowerPoint, Keyonte, PDF, or any drawing program and which are shown directly to the students (e.g., slides in a standard PPT format or videos).
    • b. Materials sent to BOGs (with different materials being sent to different groups, if so specified)
    • c. Polls (which can be pre-determined or configured on-the-fly, with the results shown to the students as a bar graph)
    • d. Quiz questions; all answers are machine graded.
      • i. Multiple choice (including simple True/False)
      • ii. “Mark all that apply” multiple answers
      • iii. Match entries in Column A with those in Column B
  • 4. The view states of Instructor Browser web app 300A includes:
    • a. A Lecture Mode, such as Everyone In Class (Instructor View) Instructor View State, which includes a preview of the sequence of upcoming slides on the left, the slide currently being shown, the faculty member (typically on the upper right of the display) and the classroom videos grid (or “Student Matrix”), typically immediately below the instructor's icon), which is a set of students randomly sampled to be shown in a matrix that is continually changing:
      • i. The default is a 4×4 matrix, with icons of 16 students smoothly scrolling horizontally, as if a camera were panning across the audience.
      • ii. If the student's computer or bandwidth are being taxed, the student can select “Lite Mode” (the control is in Help Chat), where only a 2×2 matrix or even a single icon is shown and/or scrolling is turned off
      • iii. If a student raises his or her hand (by clicking on a “raise hand” button at the bottom of the screen), a hand symbol appears over his or her icon, and this icon is “pinned” to the matrix—not being replaced.
      • iv. The instructor can call on a student by clicking on his or her icon in the Student Matrix. At this point, the student's icon expands to fill the space occupied by the entire matrix. When the instructor later clicks on this enlarged icon, it shrinks back to its original size and the Student Matrix reappears.
    • b. A Breakout Mode, such as a BOG Activity In Progress Instructor View State, which includes a preview of the sequence of upcoming slides appears on the left, and the rest of the screen contains a set of colored bars (one that represents each BOG), which serve as heat maps (see below), and the video feeds for students in one BOG and their work product (or instructions).
      • i. The instructor can click on the bar representing a particular BOG and join that group, seeing those students, their instructions, and any work product. The instructor can then participate as a full member of that group.
  • 5. The view states of Student Browser web app 300B includes:
    • a. A classroom videos grid with a default matrix being 4×4 matrix, with icons of 16 students smoothly scrolling horizontally, as if a camera were panning across the audience. This grid produces a “sense of belonging,” which is known to increase student retention.
    • b. A Lecture Mode, such as an Everyone In Class (Students View) Student View State, in which Student Browser web app 300B displays the slide, video, or anything else that can be accessed from the instructor's desktop or from the internet currently being discussed, a large instructor's video feed, and the Student Matrix under the instructor's video feed.
    • c. A Breakout Mode, such as a BOG Activity In Progress Student View State, in which Student Browser web app 300B displays the other students in their group plus any instructions and any materials (e.g., a Google Doc, Sheet, or Form) that they will collaboratively use to produce a work product.
  • 6. The view states of Tech Support/Observer Browser web app 300C is much like the view states of Instructor Browser web app 300A, but neither the students nor instructor can see the observer. The observer can not only view lectures, but also click on the heat map to join specific BOGs.
  • 7. The system includes a student dashboard that indicates progress on mastering the learning objectives.
  • 8. All learning objectives are defined in terms of “System 1” teaching (this is based on Daniel Kahneman's distinction between System 1 and System 2) (see, Kahneman, D. (2011). Thinking Fast and Slow. New York: Farrar, Straus and Giroux.)
    • a. We have a novel interpretation of System 2: This is the system that loads heavily on IQ tests. System 1 operates by matching and/or filtering input, reflexively.
    • b. Our aim is to teach by drawing most heavily on System 1 processes, which do not involve IQ.
    • c. We achieve this aim by defining two types of learning objectives:
      • i. “Hacks” are templates that solve specific problems. For example, “Five paragraph form” can be used whenever one is asked to write an essay.
      • ii. Heuristics” are processes that often (but not always) will solve a problem. Our aim is to teach specific Hacks or Heuristics so they are automatically triggered via System 1 in the appropriate situation.
      • iii. The teaching steps are
        • 1. Increase comprehension. This is represented by the top part of an hourglass, which serves as a funnel of appropriate situations that will trigger the learned Hack or Heuristic.
        • 2. Enhance memory
        • 3. Illustrate application by the appropriate applications for the Hack or Heuristic that is triggered. This is represented by the bottom part of an hourglass, which indicates a wide range of situations where the Hack of Heuristic can be applied.
        • 4. The conditions and actions of a Hack or Heuristic are broad, allowing them to be used in a wide variety of relevant situations. The use of Hacks and Heuristics solves some aspects of the “problem of transfer”—ensuring that students use material in new, unfamiliar situations.

In certain embodiments, the BOGs promote active learning, as follows:

• • 1. Rectangles represent each BOG on the instructor screen (in Faculty Mode), with the color of each rectangle functioning as a heat map that indicates what is going on in each group:
    • a. The information for the selected BOG is provided in numerical form and/or graphical form. The graphical form may include “heat maps” which use color coding to indicate quantity, with the extremes being red or blue—which are “pop out” colors that are immediately evident. Thus, the instructor can immediately see which groups are behaving differently from the majority.
    • b. Maps can indicate many different quantities, including:
      • i. How much talking is taking place;
      • ii. How much activity is taking place in the associated Doc or other work product;
      • iii. How much use is being made of Group Chat;
      • iv. The average “level” (determined by quiz performance) of the group;
      • v. The average “initiative” level (determined by how many times the members have previously raised their hands during class); or
      • vi. The variance in talking, indicating how evenly talking is distributed across group members.
      • vii. The extent to which one person is dominating either the discussion or production of the work product.
    • c. The instructor can “pin” given groups while they are taking place, allowing them later to be revisited easily during post-BOG debrief sessions.
  • 2. The instructor can click on one of the rectangles representing a BOG and immediately join that BOG.
  • 3. Students can raise their hands in BOGs, which is represented by a hand icon being pinned to the corresponding rectangle (which is immediately evident to the instructor).
  • 4. Each student is associated with many “tags,” indicating a wide range of characteristics (e.g., quiz performance level, quiz performance level for specific learning outcomes, region of residence, type of job, etc.). Any of these tags—or combinations thereof—can be used to create BOGS.
    • a. BOGs can be set up according to these criteria (or randomly or with some other criteria devised by the instructor) prior to class.
    • b. BOGS can be set up on the fly, using the same criteria that can be used in advance (including based on results of a just-administered quiz or poll).
  • 5. The most direct use of tags concerns how to use quiz scores to put people into BOGs automatically, based on grouping them by “relevant” aspects of their performance. “Relevance” requires matching their strengths to the requirements of the activity in that particular BOG. This may be accomplished as follows:
    • a. Step 1. Empirically derive the dimensions that characterize different types of quiz questions.
      • i. First, use Amazon's “Mechanical Turk” (mTurk) group-sourcing system (https://www.mturk.com) to generate many of different questions for many lesson plans.
      • ii. Next, for each separate lesson plan 420, construct a matrix where each item labels one of the rows and each item—in the same order—also labels one of the columns (ignore the diagonal, which would be each item against itself). Use “the method of triads” to get humans raters to decide how similar each pair of questions is, and fill in the corresponding cells of this matrix.
      • iii. Take the resulting similarity matrix (which specifies a value for every possible pair) and submit it to nonmetric multidimensional scaling, Principal Components Analysis, and similar procedures. This will produce a geometric solution where each question is a point in an n-dimensional space and will specify dimensions in that space.
      • iv. This in turn leads to recovering the dimensions that underlie the space. Each dimension in turn becomes a variable in a real-number vector. Each person would have such a vector, which indicates his or her performance levels on each dimension. These values define many of the “tags” we have associated with each student.
      • v. Additional tags are specified (some binary) in terms of demographic variables (e.g., age, years in school).
    • b. Step 2. Using the dimensions to tag new quiz items automatically.
      • i. Start by hand-coding a set of questions, specifying each question with a vector where each value indicates the question's location on one of the dimensions we recovered above.
      • ii. Go through a training set, training a neural net or other machine learning (ML) technique to classify the items appropriately.
      • iii. Then give the neural net (or another ML algorithm) new items and tune it up so that it classifies them appropriately. This method will be used going forward to classify every question.
    • c. Step 3. Classifying the activities
      • i. Use the dimensions recovered in Step 1 to classify each of the BOG activities, again specifying a vector (where each value indicates the location on one of the dimensions).
      • ii. Repeat the procedure in Step 2, but now use descriptions of the activities.
    • d. Step 4: Matching
      • i. For each person, match the question vectors from previous classes to the vector for the new BOG, choosing the question vectors that are closest in the space (none should match completely, given that the BOG is addressing a new learning outcome).
      • ii. Using those questions, compute the quiz scores (based only on the relevant questions) for each student; group students based on similar quiz scores.
      • iii. This procedure will place students with comparable scores on relevant previous measures into the same BOG. It can also be used to ensure that there is a range of relevant abilities in a BOG, as is appropriate for a specific learning activity (see above).
  • 6. BOGs can be terminated at a pre-set time or by intervening. In both cases, a countdown clock appears indicating that the group will end after a specified number of seconds (with the default being 10 seconds).
  • 7. BOGs can be organized into a set of distinct “roles,” each of which receives instructions for just that role.
  • 8. Instructions in BOGs are written to be “multi-level”: They can be approached at different levels of sophistication. When BOGs are defined by performance level on quizzes, this means that students of comparable ability will “self titrate” and interpret the instructions similarly—allowing them to set themselves at the Goldilocks spot: Not too each, not too hard, but just challenging enough (which respects the learning principle of “Desirable Difficulty”—Kosslyn, S. M. (2017). The science of learning. In S. M. Kosslyn & B. Nelson (Eds), Building the intentional university. Cambridge, Mass.: MIT Press.).
  • 9. The instructor can “broadcast” verbal instructions (with his or her picture appearing simultaneously) to all BOGs or just to the subset that corresponds to the set of rectangles (in the heat map) that are clicked on.
  • 10. In order to scale to very large classes, we had to devise mechanisms where students would not need the instructor to intervene. One example is to ensure that students understand the instructions for a BOG. Using the example noted earlier, the students might be role playing different groups who have to negotiate water rights. There might be 4 roles (farmers, home owners, engineers and environmentalists). Groups that have the same roles will have the same instructions (e.g., take the role of farmers when negotiating). After they read them, they then join another group with the same instructions and do the below. In this case:
    • a. Students read the instructions while in a BOG and push a key when they are ready to paraphrase them to another group.
    • b. When another group that received the same instructions has pressed the key, those two groups are temporarily joined.
    • c. One group is randomly chosen to paraphrase the instructions to the other (which has received the identical instructions).
    • d. The other group is charged with vetting the paraphrase. If they feel that the group paraphrasing has not understood the instructions correctly, they correct that group.
    • e. When the two groups agree on the interpretation of the instructions, they press a key. At that point, the activity begins.
    • f. If the groups cannot agree, they press a key to signal the instructor (their rectangle in the heat map starts pulsing).
    • g. If the groups have not responded within a specified period, their rectangle starts pulsing to signal the instructor.
    • h. The instructor intervenes by joining the group and clarifying the problem.
    • i. If it seems likely that other groups have had the same problem, the instructor clarifies them to all groups (using the “broadcast function” noted above).
  • 11. After BOGs end, during debrief a given BOG can be “spotlighted”—which means that those students and their shared work products (e.g., Google docs) are shown to the entire class. A set of BOGs can be specified for spotlighting, and the instructor can control the order of cycling through them as well as how much time is spent with each one.
  • 12. Sequences of BOGs can be specified in advance, where each BOG assigns students from previous ones in specific ways. For example, in a first set of BOGs the students in each BOG could have received a different passage to analyze. The second set of BOGs would then contain one person from each of the first set, and they would be asked to compare/contrast the passages they worked on previously. There could even be a third set of BOGs, where new groups are composed with one student each from the second set of BOGs, who now report what they found. And so on.

In certain embodiments, platform 100 provides synchronous competency-based assessment of the student by ending every class with a quiz which may consist of 6 questions. Five of the questions assess the specific learning outcomes (Hacks or Heuristics) taught in that class, where there is a minimum of 30 items for the Hacks and Heuristics taught in each class. One of the questions is randomly chosen from those for a previous class. The Quiz questions are generated in part via mTurk: The content is presented with the existing quiz question(s), and mTurkers are asked to generate variants of the question. In alternative embodiments, quiz responses are used to train machine learning algorithms, based on latent semantic analysis of the training question, the new variant, and ratings of how acceptable the new question is.

For each student's quiz, items are sampled randomly without replacement—so no two students are likely to have the identical quiz. Each quiz item is tagged by where it falls in a hierarchy that specifies three levels of competency: Macro (the overarching unit), Mini (a specific aspect of the unit), and Nano (an actionable component of the Mini competency, which corresponds to a “Hack” or “Heuristic”).

Students must answer all questions on a quiz correctly (100%) in order to pass. If they do not, they can retake the quiz with a different set of items. If they fail a second time, they then must watch a recording of the lecture before they can take it twice more. If they fail on the fourth attempt, they must watch the lecture again before they have two more tries. When students answer a question incorrectly, they receive feedback that includes the correct answer along with a brief explanation as to why that is the correct answer.

Platform 100 tracks how many times the student required to pass each quiz, and uses that information to place students in BOGs (with other students at the same level). The platform also tracks the student's Nano competency and if they seem problematic for a given student. This allows tutors to know areas in which the student needs help.

One embodiment of each of the methods described herein is in the form of a computer program that executes on a processing system, e.g., a one or more processors that are part of a computer network. Thus, as will be appreciated by those skilled in the art, embodiments of the present invention may be embodied as a method, an apparatus such as a special purpose apparatus, an apparatus such as a data processing system, or a carrier medium, e.g., a computer program product. The carrier medium carries one or more computer readable code segments for controlling a processing system to implement a method. Accordingly, aspects of the present invention may take the form of a method, an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of carrier medium (e.g., a computer program product on a computer-readable storage medium) carrying computer-readable program code segments embodied in the medium. Any suitable computer readable medium may be used including a magnetic storage device such as a diskette or a hard disk, or an optical storage device such as a CD-ROM.

It will be understood that the steps of methods discussed are performed in one embodiment by an appropriate processor (or processors) of a processing (i.e., computer) system executing instructions (code segments) stored in storage. It will also be understood that the invention is not limited to any particular implementation or programming technique and that the invention may be implemented using any appropriate techniques for implementing the functionality described herein. The invention is not limited to any particular programming language or operating system.

Reference throughout this specification to “one embodiment” or “an embodiment” means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, appearances of the phrases “in one embodiment” or “in an embodiment” in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner, as would be apparent to one of ordinary skill in the art from this disclosure, in one or more embodiments.

Similarly, it should be appreciated that in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this invention.

Thus, while there has been described what is believed to be the preferred embodiments of the invention, those skilled in the art will recognize that other and further modifications may be made thereto without departing from the spirit of the invention, and it is intended to claim all such changes and modifications as fall within the scope of the invention. For example, any formulas given above are merely representative of procedures that may be used. Functionality may be added or deleted from the block diagrams and operations may be interchanged among functional blocks. Steps may be added or deleted to methods described within the scope of the present invent

Claims

1. A method for providing an active-learning class using a plurality of electronic devices in a computer network, where the plurality of electronic devices includes a first electronic device operated by an instructor, a plurality of second electronic devices each operated by one student of a plurality of students, and one or more servers, said method comprising: assessing a performance of each student of the plurality of students during the class, where the performance of each student is assessed by one or more electronic devices of the plurality of electronic devices using one or more inputs into the second electronic device operated by each student;assigning each student of the plurality of students to one breakout group (BOG) of a plurality of BOGs, where the assigning is performed by one or more electronic devices of the plurality of electronic devices based on the performance of the plurality of students relative to each other;storing, on the server, a shared workspace for each BOG; andproviding, over the network, each student computer of the plurality of student computers with the shared workspace, where the shared workspace corresponds to the BOG to which the student is assigned,such that all students within each BOG can communicate with each other through the shared workspace of their BOG.

2. The method of claim 1, where said shared workspace has a corresponding shared one or more electronic documents, such that all students within each BOG can communicate with each other through the shared one or more electronic documents of their BOG.

3. The method of claim 1, where the performance of least one student of the plurality of students is assessed during class and prior to providing the shared workspace.

4. The method of claim 1, where the performance of least one student of the plurality of students is determined from an interaction with the first electronic device operated by the instructor, or from an interaction with at least one second electronic device operated by another student.

5. The method of claim 1, where the performance of least one student of the plurality of students is determined from one or more questions asked during a lecture; the amount of keyboard or microphone activity, responses to polls, or chat entries.

6. The method of claim 1, where the performance of least one student of the plurality of students is determined from an interaction with another student of the plurality of students in a previous BOG.

7. The method of claim 1, where the performance of least one student of the plurality of students is determined from audio interactions with other members of a previous BOG.

8. The method of claim 1, where the performance of least one student of the plurality of students is determined from interaction with a shared workspace of a previous BOG.

9. The method of claim 1, where the performance of least one student of the plurality of students is determined from a specific combination of students in one or more previous BOGs having a large amount of group interaction.

10. The method of claim 1, where the method further includes providing a quiz to each student, and where the performance of least one student of the plurality of students is determined from of answers to the quiz.

11. The method of claim 10, where the quiz items are coded to assess a specific capability, and where the performance is a measure of capabilities.

12. The method of claim 11, where each BOG includes a task having a predetermined required performance capability, and where the assigning students includes assigning students based on the predetermined required performance capability and the student's performance on relevant material.

13. The method of claim 11, where the capability is a skill that is an analytical skill, a critical thinking skill, a self-management skill, a personal interaction skill, a learning skill, a problem solving skill, or a communication skill.

14. The method of claim 12, where the assigning students includes assigning students based on their performance relative to the predetermined required performance capability.

15. The method of claim 14, where the task of each BOG requires that at least one person in the BOG have previously mastered at least one relevant capability, and where the assigning assigns at least one student having the predetermined required performance capability.

16. The method of claim 14, where the task requires that each student of each BOG has comparable levels of the relevant capabilities, and where the assigning assigns students of comparable levels of prior performance of the task-relevant capabilities.

17. The method of claim 1, where the performance is assessed after providing the shared workspace, where the performance is determined from the student's microphone activity, the student's whiteboard activity, or the student's cursor activity in shared whiteboard.

18. The method of claim 1, further comprising using the first electronic device to monitor the plurality of breakout sections.

19. The method of claim 18, further comprising operating the first electronic device to move one student of the plurality of students to another BOG of the plurality of BOGs.

20. The method of claim 1, where said plurality of BOGs is a first plurality of BOGs, where the shared workspace is a first shared workspace, and where said method further includes, after the conclusion of first plurality of BOGs, assigning each student of the plurality of students to one BOG of a second plurality of BOGs, where the assigning is performed by one or more electronic devices of the plurality of electronic devices based on the performance of the plurality of students relative to each other;storing, on the server, a second shared workspace for each BOG of the second plurality of BOGs; andproviding, over the network, each student computer of the plurality of student computers with the second shared workspace, where the second shared workspace corresponds to the BOG of the second plurality of BOGs to which the student is assigned,such that all students within each BOG of the second plurality of BOGs can communicate with each other through the shared second workspace of their BOG.

21. The method of claim 20, where said second shared workspace has a corresponding shared one or more electronic documents, such that all students within each BOG of said second plurality of BOGs can communicate with each other through the shared one or more electronic documents of their BOG of said second plurality of BOGs.

22. An apparatus for providing an active-learning class using a plurality of electronic devices in a computer network, where the plurality of electronic devices includes a first electronic device operated by an instructor, a plurality of second electronic devices each operated by a corresponding student of a plurality of students, and one or more servers, said apparatus comprising: one or more processors of at least one electronic device of the plurality of electronic devices programmed to assess a performance of each student of the plurality of students during the class, where the performance of each student is assessed using one or more inputs into the second electronic device operated by each student;assign each student of the plurality of students to one breakout group (BOG) of a plurality of BOGs, where the students are assigned based on the performance of the plurality of students relative to each other;store, on the server, a shared workspace for each BOG; andprovide, over the network, each student computer of the plurality of student computers with the shared workspace, where the shared workspace corresponds to the BOG to which the student is assigned,such that all students within each BOG can communicate with each other through the shared workspace of their BOG.

23. The apparatus of claim 22, where said shared workspace has a corresponding shared one or more electronic documents, such that all students within each BOG can communicate with each other through the shared one or more electronic documents of their BOG.

24. The apparatus of claim 22, where the performance of least one student of the plurality of students is assessed during class and prior to providing the shared workspace.

25. The apparatus of claim 22, where the performance of least one student of the plurality of students is determined from an interaction with the first electronic device operated by the instructor, or from an interaction with at least one second electronic device operated by another student.

26. The apparatus of claim 22, where the performance of least one student of the plurality of students is determined from one or more questions asked during a lecture; the amount of keyboard or microphone activity, responses to polls, or chat entries.

27. The apparatus of claim 22, where the performance of least one student of the plurality of students is determined from an interaction with another student of the plurality of students in a previous BOG.

28. The apparatus of claim 22, where the performance of least one student of the plurality of students is determined from audio interactions with other members of a previous BOG.

30. The apparatus of claim 22, where the performance of least one student of the plurality of students is determined from interaction with a shared workspace of a previous BOG.

31. The apparatus of claim 22, where the performance of least one student of the plurality of students is determined from a specific combination of students in one or more previous BOGs having a large amount of group interaction.

32. The apparatus of claim 22, where said one or more processors are further programmed to provide a quiz to each student, and where the performance of least one student of the plurality of students is determined from of answers to the quiz.

33. The apparatus of claim 32, where the quiz items are coded to assess a specific capability, and where the performance is a measure of capabilities.

34. The apparatus of claim 33, where each BOG includes a task having a predetermined required performance capability, and where said one or more processors are further programmed to assign students includes assigning students based on the predetermined required performance capability and the student's performance on relevant material.

35. The apparatus of claim 33, where the capability is a skill that is an analytical skill, a critical thinking skill, a self-management skill, a personal interaction skill, a learning skill, a problem solving skill, or a communication skill.

36. The apparatus of claim 34, where said one or more processors are further programmed to assign students based on their performance relative to the predetermined required performance capability.

37. The apparatus of claim 36, where the task of each BOG requires that at least one person in the BOG have previously mastered at least one relevant capability, and where said one or more processors are further programmed to assign at least one student having the predetermined required performance capability.

38. The apparatus of claim 36, where the task requires that each student of each BOG has comparable levels of the relevant capabilities, and said one or more processors are further programmed to assign students of comparable levels of prior performance of the task-relevant capabilities.

39. The apparatus of claim 22, where the performance is assessed after providing the shared workspace, where the performance is determined from the student's microphone activity, the student's whiteboard activity, or the student's cursor activity in shared whiteboard.

40. The apparatus of claim 22, said one or more processors are further programmed for the first electronic device to monitor the plurality of breakout sections.

41. The apparatus of claim 41, said one or more processors are further programmed to operate the first electronic device to move one student of the plurality of students to another BOG of the plurality of BOGs.

42. The apparatus of claim 22, where said plurality of BOGS is a first plurality of BOGs, where the shared workspace is a first shared workspace, and where said one or more processors are further programmed to, after the conclusion of first plurality of BOGs, assign each student of the plurality of students to one BOG of a plurality of second BOGs, where the students are assigned based on the performance of the plurality of students relative to each other;store, on the server, a second shared workspace for each BOG of the second plurality of BOGs; andprovide, over the network, each student computer of the plurality of student computers with the second shared workspace, where the shared workspace corresponds to the BOG of the second plurality of BOGs to which the student is assigned,such that all students within each BOG of the second plurality of BOGs can communication with each other through the shared second workspace of their BOG.

43. The apparatus of claim 19, where said second shared workspace has a corresponding shared one or more electronic documents, such that all students within each BOG of said second plurality of BOGs can communicate with each other through the shared one or more electronic documents of their BOG of said second plurality of BOGs.