Patent Application
20240203281
The present invention generally relates to Smart E-Learning system using adaptive video lecture delivery based on attentiveness of the viewer. In particular, the present invention discloses method of learning the correlation between lecture contents and attentiveness of viewer using Artificial Intelligence (AI)/Machine Learning (ML) techniques and dynamically modifying the sequence of the video lecture elements in real time while the viewer is watching the lecture and system thereof.
With the growth of e-learning industry, there comes a need of making online lecture delivery more and more effective as well as personalized. Various computer algorithms are being used to make this possible. Any E-learning with pre-developed content consists one or more of reading materials, notes, MCQ (Multiple Choice Questions), assignments, projects and audio-video content to listen and watch. Various methods are being devised and used for personalizing this content. Personalization is in terms of what content to be displayed, when it is to be displayed, what should be the frequency of particular type of content.
To aid the personalization, various methods of collecting user's information are devised. This includes information about the user, including but not limited to, personal information, family background, content consumption rate, user's answers to MCQ, test marks, audio-video replays, time spent on each topic etc. Some methodologies of detecting faces for students' attendance or detecting attentiveness etc for students based on facial expressions while they are viewing the lectures are being ideated.
All these methods of personalization are used to pre-decide the content flow for a user. They fail to capture last minute cues or run-time mood/tendency of the viewer to adapt the content (video lecture) as and when the content is being viewed or consumed. Thus, they are of no use in increasing the attentiveness or grasping of the topic while the user is viewing the lecture content.
Moreover, some content is being inserted currently between the lectures. This is done in a pre-decided manner. If the attentiveness of the viewer is high, the new content that comes in may disturb him/her. And in turn reduce the effectiveness of the content/lecture. Hence these insertions have to be done tactfully. So, to make these online lectures effective we have designed the process of modifying the sequence of the online lecture elements in real time while the viewer is watching the lecture, based on attentiveness of the viewer using the facial expressions and other cognitive responses.
The present invention generally relates to smart E-learning system for effective lecture delivery. In particular, the present invention discloses method of learning the correlation between lecture contents and attentiveness of viewer using AI/ML techniques and dynamically modifying the sequence of the video lecture elements in real time while the viewer is watching the lecture to increase attentiveness of viewer and system thereof.
This invention is for the E-learning field. It aims to increase the attentiveness of the viewer in real-time while watching video lectures. This is done by capturing and analyzing the attentiveness of the viewer viz. facial expressions, cognitive response, while the user is watching the video lecture. This attentiveness level is analyzed and given as feedback to “Next Segment Predictor with AI/ML techniques” 206. To enable faster processing and get real-time results, this analysis process 205 is completely done on the client browser 201 using the Web Assembly (WASM).
In an embodiment, the Next Segment Predictor with AI/ML techniques 206 is a block that will predict the next segment of content 209 to be displayed to the viewer. The Next Segment Predictor block 206 is developed using incremental AI/ML techniques, whereas a model is incrementally trained with 210, 208, 207 the correlation between lecture content and attentiveness level of viewer. The next lecture segment 209 is recommended using this trained model 206 where this recommended lecture element 209 is added dynamically in online lecture 204 in order to increase attentiveness of viewer.
The Next Segment Predictor 206 runs on the server 202 and has all the information regarding the topic and the content that is being viewed by the user. It also has information about the user's previous attentiveness scores 208 based on facial expressions, cognitive responses etc. and their trends. Using this information, it is able to predict the next content segment 209 that should be presented to the particular user.
All this happens in real-time. And the user has no idea that the content he/she is watching is being customized for him/her and his/her temperament while watching the lectures. But it ultimately results in better attentiveness and hence better grasping of the topic in the video lecture.
An object of the present invention is to
To further clarify advantages and features of the present invention, a more particular description of the invention will be rendered by reference to specific embodiments thereof, which is illustrated in the appended drawings. It is appreciated that these drawings depict only typical embodiments of the invention and are therefore not to be considered limiting of its scope. The invention is described and explained with additional specificity and detail with the accompanying drawings.
In the e-learning scenario, where users watch video lectures in order to learn a subject, the video lectures can get monotonous without any interactivity. Such interactivity is currently being introduced by adding segments similar to MCQ questions, interesting facts etc between the video lectures. But this addition is pre-fixed. The user responds to these segments by answering the question and proceeds further. Such activities may not be useful to each user to boost their concentration while watching the video lecture. Also since the sequence is pre-fixed, the video lecture continues as it is programmed without considering if the user has answered correctly or incorrectly. In such cases, these inserted segments may in turn reduce the concentration level and the user may not be able to grasp the concepts well.
In this solution, we consider that learning behavior for each user is different. The effectiveness of the lecture shall depend on the temperament of the user while watching the lecture. Hence this solution measures the attentiveness of the user, while he/she is watching the video lecture, using sensory inputs, facial expression recognition etc. Based on this input, Next Segment Predictor decides the next segment to be displayed to the user. Next Segment Predictor is a trained model that uses AI/ML techniques to select the next video segment that increase the attentiveness of the user.
This solution gives personalized experience to each individual while watching the video lecture. It increases their ability to grasp the concepts from the video lecture more effectively. It also increases the engagement of the user with the video lecture content and hence gives better learning outcome.
These and other features, aspects, and advantages of the present invention will become better understood when the following detailed description is read with reference to the accompanying drawings in which like characters represent like parts throughout the drawings, wherein:
Further, skilled artisans will appreciate that elements in the drawings are illustrated for simplicity and may not have been necessarily been drawn to scale. For example, the flow charts illustrate the method in terms of the most prominent steps involved to help to improve understanding of aspects of the present invention. Furthermore, in terms of the construction of the device, one or more components of the device may have been represented in the drawings by conventional symbols, and the drawings may show only those specific details that are pertinent to understanding the embodiments of the present invention so as not to obscure the drawings with details that will be readily apparent to those of ordinary skill in the art having benefit of the description herein.
Viewer's attentiveness is received by the Incremental Training Model (302) continuously while the viewer is watching the content in the form of feedback loop. This feedback is used for incrementally learning the impact of changes made to the video content by the decision unit (303) by inserting or not inserting new segments.
The attentiveness received from the viewer triggers the decision unit (303) to check if any insertions are to be made to the main video to maintain the viewer's attention and interest levels in the content to aid grasping more knowledge. If decision unit (303) decides to insert a new supporting video, such video is sent to the client (202) to be displayed to the viewer.
For the purpose of promoting an understanding of the principles of the invention, reference will now be made to the embodiment illustrated in the drawings and specific language will be used to describe the same. It will nevertheless be understood that no limitation of the scope of the invention is thereby intended, such alterations and further modifications in the illustrated system, and such further applications of the principles of the invention as illustrated therein being contemplated as would normally occur to one skilled in the art to which the invention relates.
It will be understood by those skilled in the art that the foregoing general description and the following detailed description are exemplary and explanatory of the invention and are not intended to be restrictive thereof.
Reference throughout this specification to “an aspect”, “another aspect” or similar language means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, appearances of the phrase “in an embodiment”, “in another embodiment” and similar language throughout this specification may, but do not necessarily, all refer to the same embodiment.
The terms “comprises”, “comprising”, or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process or method that comprises a list of steps does not include only those steps but may include other steps not expressly listed or inherent to such process or method. Similarly, one or more devices or sub-systems or elements or structures or components proceeded by “comprises . . . a” does not, without more constraints, preclude the existence of other devices or other sub-systems or other elements or other structures or other components or additional devices or additional sub-systems or additional elements or additional structures or additional components.
Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The system, methods, and examples provided herein are illustrative only and not intended to be limiting.
Embodiments of the present invention will be described below in detail with reference to the accompanying drawings.
In an E-learning scenario, when a user is watching the video content/lecture, the user may not be in a temperament to grasp too much information. In this case, the effectiveness of the video content/lecture is not fully achieved. Hence, it is important to first grab the attention of the listener. In real life scenario with physical delivery of lectures, the lecturer uses the cues from facial expression of the listener to modify the lecture sequence. With our invention we are trying to replicate this scenario even in e-learning environment with no human intervention.
Complete process involves of three mains steps:
This is the first step in the process and is performed while the user is watching the video content. Viewer's attentiveness level is detected based on various parameters like facial expressions, cognitive response mapped based on, but not limited to, eye movement, eye aspect ratio, lip movement, forehead expression etc. These are then compared with a pre-trained model of attentiveness detection. This can be done on the client side 201 or on the server-side 202. We do this on the client side. The trained model of attentiveness detection 205 is combined with the AI/ML technology into a docker. WebAssembly (WASM) is used on the client browser to run this docker. Since this detection are done at the client machine, it saves the huge load of
Since content segments and its sequence are changed dynamically in real time, such technology will provide the desired results as fast as possible. The final attentiveness level score is then sent to the server.
The server 202 once received the attentiveness score as continuous feedback 210. The server 202 uses the steps of process shown in
For this the server 202 already has pre-loaded information 207 about the current topic of the content/lecture. This information consists of a main content video. This main content video is marked for break points. These break points are marked smartly by the content creator using an authoring tool wherein the authoring tool has facility to upload the predicted content video with the facility to mark the break point timing in the main content video; each main content video is accompanied by several supporting content segments wherein these segments may consist of, but not limited to, explanatory video with more examples, MCQ along with their explanatory videos or correct as well as wrong response from the user, solution of certain extra questions, points to remember etc.; each supporting segment is classified into different content type.
The smart decision of what content type should be selected for what kind of attentiveness trend is derived from incremental training model developed with AI/ML technique. For this the viewer's feedback in terms of attentiveness level for the newly inserted content segment 209 is observed. The change in attentiveness is stored for future reference. This data is then fed to the next segment predictor 206 to calculate if sending a particular content segment is desirable or not.
Once this decision is made, new content segments are selected and sent to the client. This process can also be run on the client side. In this case client is preloaded with all the information about the content segments available for insertion.
The new content segments 209 to be inserted are received on the client side 211. As soon as the next break point is hit the new content segments are played. This happens without any notification to the viewer/user. For the user it seems to be a pre-decided sequence but in real it has been modified to suit the temperament of the user/viewer.
Using this invention, the E-learning system can be made more effective. The grasping of the topic will be facilitated for the user/viewer. Every time the user watches the same topic, he may be presented with a new set of content sequence based on its attentiveness.
This analysis is continuously sent to server (210). The Server (202) finds new content segment (209) using the “Next segment Predictor” (206) that uses incremental training AI/ML Technique. It is trained using process shown in
The Next Segment Predictor (206) decides the final content segments (209) that needs to be inserted and sends it to the client. The client (201) modifies the video sequence (204) in run-time. As soon as the current segment is completed, new segment (209) received from the server is first served before proceeding with its pre-decided lecture sequence.
The drawings and the forgoing description give examples of embodiments. Those skilled in the art will appreciate that one or more of the described elements may well be combined into a single functional element. Alternatively, certain elements may be split into multiple functional elements. Elements from one embodiment may be added to another embodiment. For example, orders of processes described herein may be changed and are not limited to the manner described herein. Moreover, the actions of any flow diagram need not be implemented in the order shown; nor do all of the acts necessarily need to be performed. Also, those acts that are not dependent on other acts may be performed in parallel with the other acts. The scope of embodiments is by no means limited by these specific examples. Numerous variations, whether explicitly given in the specification or not, such as differences in structure, dimension, and use of material, are possible. The scope of embodiments is at least as broad as given by the following claims.
Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments. However, the benefits, advantages, solutions to problems, and any component(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential feature or component of any or all the claims.
Example 1 describes an event where user is viewing a lecture online and is not attentive. Low attentiveness is detected by this system and next segment predictor is notified accordingly. Next segment predictor takes action to insert a question or an interesting fact based on its previous knowledge about the content. Here interaction is introduced at the right time to increase the attentiveness of the user. User is now engaged in answering the question. Based on whether user answers correctly or incorrectly, the next segment predictor brings in the correct explanation of the question with few more details. Now attentiveness is measured. If the attentiveness score is increased, next segment predictor does not insert new segment and main video continues to play. If attentiveness is still not up to the mark, next segment predictor will find other content segment for current or next break point. This process continues trying to keep good level of attentiveness from the user.
Example 2 describes an event where user id watching the lecture before the exam to revise the concepts. The user is very attentive here. Hence the next segment predictor does introduce additional segments like questions, extra information etc. to keep the attentiveness intact. This also helps the user to revise the main concepts quickly. Here the content creator takes no extra efforts for such revision lectures separately. The given system handles different scenario based on the attentiveness feedback that it receives from the user
In the process of learning online, interactivity is limited due to lack of feedback loop from the user. Hence it is not as effective as the physical classroom learning. This invention can be used by universities and online self-paced learning platforms to enhance the effectiveness of their online learning content. This invention will bring online self-paced learning to be at par with the offline (one-on-one) learning experience. Using this invention will give personalized learning experience to users and hence they will prefer this mode of learning. This will ultimately lead to increase in number of users learning online and will benefit the online learning industry. For online learning, there will be no limit on number of students that can be admitted to a course at a given time. This will benefit the universities without losing the effectiveness of the course content.
Citation List follows:
PTL 1 discloses the invention of dynamic and adaptive eLearning system for improvement of the productivity of learners and smart attendance monitoring through appropriate interaction and feedback. Particularly this invention is for taking attendance using face recognition and attentiveness detection. It is sent as a feedback but not used anywhere in order to modify the online lecture. Whereas, in our invention this feedback is used for modifying video sequence of pre-recorded lectures.
PTL 2 discloses the real-time portable tele-education method based on cognitive load and learning style from BCI data using CI based techniques and content services. It uses BCI, EEG and brain wave detection to calculate cognitive load of learners and adapts courseware that includes lessons, tasks etc. in runtime. However, it does not focus on dynamic sequencing of learning video lectures. Also, camera feed is not used here for attentiveness detection.
PTL 3 discloses system and method for automated course individualization via learning behaviors and natural language processing. In this invention, content is tagged using NLP and then the most relevant content path for the user from the available content is found. This path is changed at regular intervals like module completion. Moreover, facial expressions, cues or attentive level is not used to detect the learner's response for adaptive real time video content delivery.
PTL 4 discloses invention about personalized and adaptive math learning system. The scope of the invention limits to teaching only maths subject. The invention does not make use of camera feed from students. However, the system makes use of background information of students to create learner's profile. Lesson plans are adapted accordingly. Changing the sequence of video or live lectures are not mentioned anywhere in the said invention.
NPL 1 refers to adaptive video learning by the interactive e-Partner. Here, to motivate students and grab their attention, the interactive e-partner is introduced into the lecture videos. The e-partner's behavior would evolve and animate with the student's learning style and the lecturer's teaching behavior, so that students can be encouraged to focus on the lecture content and enjoy the self-paced learning at the same time. The invention doesn't speak about change in video lecture.
NPL 2 relate to supporting online lectures with adaptive and intelligent features. Authors present different pedagogical aspects for recommending the most suitable learning materials for students based on their learning profiles and preferences, involving students in the learning process from the very early beginning of the lecture, and preparing for the next/upcoming lecture, so students feel the personalization and customization of the lecture to enhance the learning process and students' online learning experience. However, the sequence of lecture content remains pre-decided and fixed for particular student.
| Number | Date | Country | Kind |
|---|---|---|---|
| 202121032292 | Jul 2021 | IN | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/IN2022/050362 | 4/15/2022 | WO |
