Are We There Yet? A Systematic Literature Review on Chatbots in Education

This assessment was aligned with the CHISM scale, which was completed in a post-survey. A minimum interaction of three hours per week with each AIC, or 48 h over a month across all AICs, was requested from each participant. Additionally, tutoring chatbots provide personalized learning experiences, attracting more applicants to educational institutions. Moreover, they contribute to higher learner retention rates, thereby amplifying the success of establishments.

According to an App Annie report, users spent 120 billion dollars on application stores Footnote 8. User-driven conversations are powered by AI and thus allow for a flexible dialogue as the user chooses the types of questions they ask and thus can deviate from the chatbot’s script. One-way user-driven chatbots use machine learning to understand what the user is saying (Dutta, 2017), and the responses are selected from a set of premade answers. In contrast, two-way user-driven chatbots build accurate answers word by word to users (Winkler & Söllner, 2018). Such chatbots can learn from previous user input in similar contexts (De Angeli & Brahnam, 2008).

They engage in a dialogue with each student and determine the areas where they are falling behind. Then, chatbots use this data to compose an entirely personalized learning program that focuses on troubling subjects. Their job is also to follow the students’ advancement from the first to the last lesson, check their assumptions, and guide them through the curriculum. Furthermore, the feedbacks also justified why other variables such as the need for cognition, perception of learning, creativity, self-efficacy, and motivational belief did not show significant differences.

The map, reported in Appendix A, displays the current state of research regarding chatbots in education with the aim of supporting future research in the field. Future studies should explore chatbot localization, where a chatbot is customized based on the culture and context it is used in. Moreover, researchers should explore devising frameworks for designing and developing educational chatbots to guide educators to build usable and effective chatbots. Finally, researchers should explore EUD tools that allow non-programmer educators to design and develop educational chatbots to facilitate the development of educational chatbots.

Among the numerous use cases of chatbots, there are several industry-specific applications of AI chatbots in education. Institutions seeking support in any of these areas can implement chatbots and anticipate remarkable outcomes. Chatbots serve as valuable assistants, optimizing resource allocation in educational institutions.

Pedagogical Roles

We advise that you practice metacognitive routines first, before using a chatbot, so that you can compare results and use the chatbot most effectively. Keep in mind that the tone or style of coaching provided by chatbots may not suit everyone. In modern educational institutions, student feedback is the most important factor for assessing a teacher’s work.

Stanford d.school’s Leticia Britos Cavagnaro is pioneering efforts to extend interactive resources beyond the classroom. She recently has developed the “d.bot,” which takes a software feature that many of us know through our experiences as customers — the chatbot — and deploys it instead as a tool for teaching and learning. Jenny Robinson, a member of the Stanford Digital Education team, discussed with Britos Cavagnaro what led to her innovation, how it’s working and what she sees as its future.

For example, you might guide your students in using chatbots to get feedback on the structure of an essay or to find errors in a piece of programming code. Remember that you and your students should always critically examine feedback generated by chatbots. You can use generative AI chatbots to support teaching and learning in many ways. We also encourage you to access and use chatbots to complete some provided sample tasks.

Advantages for students

Users should be cautious about the information generated by chatbots and not rely solely on them as sources of information. They should critically evaluate and fact-check the responses to prevent the spread of misinformation or disinformation. The advantages and challenges of using chatbots in universities share similarities with those in primary and secondary schools, but there are some additional factors to consider, discussed below. In an experiment in which the chatbot is asked to design a trendy women’s shoe, it offers several possible alternatives and then, when asked, serially and skillfully refines the design. Various design principles, including pedagogical ones, have been used in the selected studies (Table 8, Fig. 8). I should clarify that d.bot — named after its home base, the d.school — is just one member of my bottery (‘bottery’ is a neologism to refer to a group of bots, like a pack of wolves, or a flock of birds).

These programs have one or a few functionalities that tackle specific problems. This article on Chatbots Magazine, written by the creators of Hubert, has pointed out six ways how Artificial Intelligence and chatbots can improve education, and we will list the three most important ones. Today, there are many similar partnerships between corporations and educational institutions that try to make the institutional learning transparent and more efficient.

The results show that the chatbots were proposed in various areas, including mainly computer science, language, general education, and a few other fields such as engineering and mathematics. Most chatbots are accessible via a web platform, and a fewer chatbots were available on mobile and desktop platforms. You can foun additiona information about ai customer service and artificial intelligence and NLP. This choice can be explained by the flexibility the web platform offers as it potentially supports multiple devices, including laptops, mobile phones, etc. By far, the majority (20; 55.55%) of the presented chatbots play the role of a teaching agent, while 13 studies (36.11%) discussed chatbots that are peer agents.

We wanted AI-powered features that were deeply integrated into the app and leveraged the gamified aspect of Duolingo that our learners love. Predicted to experience substantial growth of approximately $9 billion by 2029, the Edtech industry demonstrates numerous practical applications that highlight the capabilities of chatbot in education AI and ML. Users must use chatbots in a manner that respects the rights and dignity of others. They should not be used for malicious purposes, harassment, hate speech, or any activity that violates applicable laws or regulations. Ethical issues such as bias, fairness, and privacy are relevant in university settings.

Use cases of AI chatbots in education industry

It used Artificial Intelligence Markup Language (AIML) to identify an accurate response to user input using knowledge records (AbuShawar and Atwell, 2015). The process of organizing your knowledge, teaching it to someone, and responding to that person reinforces your own learning on that topic (Carey, 2015). For example, you might prompt a chatbot to act as a novice learner and ask you questions about a topic. Try different prompts and refine them so the chatbot responds in a helpful way. However, providing frequent quality feedback requires much time and effort from you and your teaching team. An AI chatbot might help you by giving students frequent, immediate, and adaptive feedback.

• Additionally, investing in research and development to enhance AI chatbot capabilities and address identified concerns is crucial for a seamless integration into educational systems.
• Finally, the seventh question discusses the challenges and limitations of the works behind the proposed chatbots and potential solutions to such challenges.
• The data that support the findings of this study are available from the corresponding author upon reasonable request.

This study focuses on using chatbots as a learning assistant from an educational perspective by comparing the educational implications with a traditional classroom. Therefore, the outcomes of this study reflected only on the pedagogical outcomes intended for design education and project-based learning and not the interaction behaviors. As users, the students may have different or higher expectations of EC, which are potentially a spillover from use behavior from chatbots from different service industries. Moreover, questions to ponder are the ethical implication of using EC, especially out of the learning scheduled time, and if such practices are welcomed, warranted, and accepted by today’s learner as a much-needed learning strategy.

In some cases, the teaching agent started the conversation by asking the students to watch educational videos (Qin et al., 2020) followed by a discussion about the videos. In other cases, the teaching agent started the conversation by asking students to reflect on past learning (Song et al., 2017). Other studies discussed a scenario-based approach to teaching with teaching agents (Latham et al., 2011; D’mello & Graesser, 2013).

By efficiently handling repetitive tasks, they liberate valuable time for teachers and staff. As a result, schools can reduce the need for additional support staff, leading to cost savings. This cost-effective approach ensures that educational resources are utilized efficiently, ultimately contributing to more accessible and affordable education for all. Multilingual chatbots act as friendly language ambassadors, breaking down barriers for students from diverse linguistic backgrounds. Their ability to communicate in various languages fosters inclusivity, ensuring that all students can learn and engage effectively, irrespective of their native language.

This limits their ability to stimulate critical thinking or problem-solving skills. This limitation could impact the overall effectiveness of such tools in promoting creative learning approaches. There are multiple business dimensions in the education industry where chatbots are gaining popularity, such as online tutors, student support, teacher’s assistant, administrative tool, assessing and generating results.

Learning Analytics is defined as the research area that focuses on collecting traces that learners leave behind and using those traces to improve learning (Duval and Verbert, 2012; Greller and Drachsler, 2012). Learning Analytics can be used both by students to reflect on their own learning progress and by teachers to continuously assess the students’ efforts and provide actionable feedback. Intelligent Tutoring Systems are defined as computerized learning environments that incorporate computational models (Graesser et al., 2001) and provide feedback based on learning progress. Educational technologies specifically focused on feedback for help-seekers, comparable to raising hands in the classroom, are Dialogue Systems and Pedagogical Conversational Agents (Lester et al., 1997). These technologies can simulate conversational partners and provide feedback through natural language (McLoughlin and Oliver, 1998).

The same is true of rivals such as Claude from Anthropic and Bard from Google. These so-called “chatbots,” computer programs designed to simulate conversation with human users, have evolved rapidly in recent years. Since chatbots are related to other technologies, the initial literature search also considered keywords such as “pedagogical agents,” “dialogue systems,” or “bots” when composing the search query. However, these increased the number of irrelevant results significantly and were therefore excluded from the query in later searches. Educational Technologies enable distance learning models and provide students with the opportunity to learn at their own pace.

AI chatbots can be attentive to – and train on – students’ learning habits and areas of difficulty. It has been scientifically proven that not everyone understands and learns in the same way. To cater to the needs of every student in terms of complex topics or subjects, chatbots can customize the learning plan and make sure that students gain maximum knowledge – in the classroom and even outside. In the Chat PG supporting learning role (Learning), chatbots are used as an educational tool to teach content or skills. This can be achieved through a fixed integration into the curriculum, such as conversation tasks (L. K. Fryer et al., 2020). Alternatively, learning can be supported through additional offerings alongside classroom teaching, for example, voice assistants for leisure activities at home (Bao, 2019).

Likewise, bots can collect inputs from all involved participants after each interaction or event. Subsequently, this method offers valuable insights into improving the learning journey. AI implementation promotes higher engagement by supplying interactive learning experiences, making the process more enjoyable. The study shows that 90.7% of participants expressed satisfaction with the experiential learning chatbot workshop, while 81.4% felt engaged.

Engagement

When prompting a chatbot, ask it « What more would you need to make this interaction better? » (Chen, 2023). This can in turn prompt you to give more specific details and instructions that can yield better results. All rights are reserved, including those for text and data mining, AI training, and similar technologies.

Only two studies used chatbots as teachable agents, and two studies used them as motivational agents. In terms of the evaluation methods used to establish the validity of the articles, two related studies (Pérez et al., 2020; Smutny & Schreiberova, 2020) discussed the evaluation methods in some detail. However, this study contributes more comprehensive evaluation details such as the number of participants, statistical values, findings, etc.

To structure research topics and findings in a comprehensible way, a three-stage clustering process was applied. While the first stage consisted of coding research topics by keywords, the second stage was applied to form overarching research categories (Table 1). In the final stage, the findings within each research category were clustered to identify and structure commonalities within the literature reviews.

This suggests the need for evolving teaching methods and curricula to more effectively incorporate AICs, emphasizing the enhancement of their capabilities for providing contextually rich and varied linguistic experiences. One practical approach could be the introduction of specific learning modules on different types of chatbots, such as app-integrated, web-based, and standalone tools, as well as Artificial Intelligence, into the curriculum. Such modules would equip students and future educators with a deeper understanding of these technologies and how they can be utilized in language education. The implications of these findings are significant, as they provide a roadmap for the development of more effective and engaging AICs for language learning in the future. Concerning RQ2 (pedagogical roles), our results show that chatbots’ pedagogical roles can be summarized as Learning, Assisting, and Mentoring. The Learning role is the support in learning or teaching activities such as gaining knowledge.

Believe it or not, the education sector is now among the top users of chatbots and other smart AI tools like ChatGPT. Most researchers (25 articles; 69.44%) developed chatbots that operate on the web (Fig. 5). For example, KEMTbot (Ondáš et al., 2019) is a chatbot system that provides information about the department, its staff, and their offices.

Other chatbots acted as intelligent tutoring systems, such as Oscar (Latham et al., 2011), used for teaching computer science topics. Moreover, other web-based chatbots such as EnglishBot (Ruan et al., 2021) help students learn a foreign language. In terms of application, chatbots are primarily used in education to teach various subjects, including but not limited to mathematics, computer science, foreign languages, and engineering. While many chatbots follow predetermined conversational paths, some employ personalized learning approaches tailored to individual student needs, incorporating experiential and collaborative learning principles. Challenges in chatbot development include insufficient training datasets, a lack of emphasis on usability heuristics, ethical concerns, evaluation methods, user attitudes, programming complexities, and data integration issues. According to Schmulian and Coetzee (2019), there is still scarcity in mobile-based chatbot application in the educational domain, and while ECs in MIM has been gaining momentum, it has not instigated studies to address its implementation.

These AICs may cover different aspects of language learning, such as grammar, vocabulary, pronunciation, and listening comprehension, and use various techniques to adapt to the user’s level of proficiency and tailor their responses accordingly. AI chatbots offer a multitude of applications in education, transforming the learning experience. They can act as virtual tutors, providing personalized learning paths and assisting students with queries on academic subjects. Additionally, chatbots streamline administrative tasks, such as admissions and enrollment processes, automating repetitive tasks and reducing response times for improved efficiency. With the integration of Conversational AI and Generative AI, chatbots enhance communication, offer 24/7 support, and cater to the unique needs of each student. Institutional staff, especially teachers, are often overburdened and exhausted, working beyond their office hours just to deliver excellent learning experiences to their students.

Similarly, the chatbot in (Schouten et al., 2017) shows various reactionary emotions and motivates students with encouraging phrases such as “you have already achieved a lot today”. In comparison, chatbots used to teach languages received less attention from the community (6 articles; 16.66%;). Interestingly, researchers used a variety of interactive media such as voice (Ayedoun et al., 2017; Ruan et al., 2021), video (Griol et al., 2014), and https://chat.openai.com/ speech recognition (Ayedoun et al., 2017; Ruan et al., 2019). According to their relevance to our research questions, we evaluated the found articles using the inclusion and exclusion criteria provided in Table 3. The inclusion and exclusion criteria allowed us to reduce the number of articles unrelated to our research questions. Further, we excluded tutorials, technical reports, posters, and Ph.D. thesis since they are not peer-reviewed.

Instead of enduring the hassle of visiting the office and waiting in long queues for answers, students can simply text the chatbots to quickly resolve their queries. This user-friendly option provides convenient and efficient access to information, enhancing the overall student experience and streamlining administrative processes. Whether it’s admission-related inquiries or general questions, educational chatbots offer a seamless and time-saving alternative, empowering students with instant and accurate assistance at their fingertips. Renowned brands such as Duolingo and Mondly are employing these AI bots creatively, enhancing learner engagement and facilitating faster comprehension of concepts.

The need for cognition also indicates positive acceptance towards problem-solving (Cacioppo et al., 1996), enjoyment (Park et al., 2008), and it is critical for teamwork, as it fosters team performance and information-processing motivation (Kearney et al., 2009). Henceforth, we speculated that EC might influence the need for cognition as it aids in simplifying learning tasks (Ciechanowski et al., 2019), especially for teamwork. The findings indicate other key potential areas for AIC improvement to better cater to users’ proficiency levels. The development of LLM-power chatbots could help avoid irrelevant responses often resulting from an over-reliance on pre-set answers, as indicated by Jeon (2021). Chatbot technology has evolved rapidly over the last 60 years, partly thanks to modern advances in Natural Language Processing (NLP) and Machine Learning (ML) and the availability of Large Language Models (LLMs). Today chatbots can understand natural language, respond to user input, and provide feedback in the form of text or audio (text-based and voice-enabled).

An example of Scaffolding can be seen in (Gabrielli et al., 2020), where the chatbot coaches students in life skills, while an example of Recommending can be seen in (Xiao et al., 2019), where the chatbot recommends new teammates. Finally, Informing can be seen in (Kerly et al., 2008), where the chatbot informs students about their personal Open Learner Model. After the first, second, and third filters, we identified 505 candidate publications. We continued our filtering process by reading the candidate publications’ full texts resulting in 74 publications that were used for our review. Compared to 3.619 initial database results, the proportion of relevant publications is therefore about 2.0%. In the case of Google Scholar, the number of results sorted by relevance per query was limited to 300, as this database also delivers many less relevant works.

Thirty years ago, when students wanted a break from study, they would listen to music on cassette players. Alternatively, they would use landline telephones and pagers to arrange dates. If Stretch is asked a question about, say, approaches to social-and-emotional learning, it will only be able to draw on research, articles, and other information that has already been examined by ISTE and any other participating organizations. This implementation will ease data collection for reference and networking purposes.

The SD values show a similar level of variation in the weekly interaction hours across all four AICs for both Spanish and Czech participants, suggesting a comparable spread of interaction frequencies within each group. The research was carried out following the regulations set by each institution for interventions with human subjects, as approved by their respective Ethical Committees. Participants provided written consent for the publication of their interactions with chatbots for academic purposes. Social science research indicates that dialogue represents cultural membership, gender identification, and group membership broadly. How the message is communicated sends a cue of who the message is for and who the speaker is.

I’m also very clear, through what the bot says to the user and what I say when I first introduce the bot, about how the information that is shared will be used. Oftentimes reflections that students share with the bot are shared with the class without identifiable information, as a starting point for social learning. The widespread adoption of chatbots and their increasing accessibility has sparked contrasting reactions across different sectors, leading to considerable confusion in the field of education.

Due to the size of the concept map a full version can be found in Appendix A. Winkler and Söllner (2018) reviewed 80 articles to analyze recent trends in educational chatbots. The authors found that chatbots are used for health and well-being advocacy, language learning, and self-advocacy.

Overall, students appreciate the capabilities of AI chatbots and find them helpful for their studies and skill development, recognizing that they complement human intelligence rather than replace it. As technology continues to advance, AI-powered educational chatbots are expected to become more sophisticated, providing accurate information and offering even more individualized and engaging learning experiences. They are anticipated to engage with humans using voice recognition, comprehend human emotions, and navigate social interactions. This includes activities such as establishing educational objectives, developing teaching methods and curricula, and conducting assessments (Latif et al., 2023).