Walert - Your Open Day FAQ Buddy


Conversational agents based on Large Language Models (LLMs) such as ChatGPT can provide numerous benefits to multiple stakeholders inside and outside an organisation such as RMIT University. One of the primary benefits is that they can help different stakeholders in reducing the cost of tackling various tasks, saving time and resources (e.g., translate data into different formats, prepare draft of documents, etc.). Note that the aim is not to replace humans doing the tasks, but to assist them to be more cost-effective. In this setting, one of the major concerns is the risk of giving away sensitive data, compromising the privacy and security of individuals. This can make it harder for members of the organization to comply with data management policies, as they may be uncertain about what data is being collected by third-parties such as OpenAI.

We wanted to combine the diverse expertise we have within the Interaction, Technology, and Information discipline at the RMIT School of Computing Technologies (SCT) to demonstrate ourselves (and to learn on doing so) how far we can get in designing and deploying our in-house version of a conversational LLM. To make this long-term goal more reachable, we identified a more tangible milestone: can we deploy our own LLM to assist us to build a chatbot for RMIT Open Day? We considered a manually curated Frequently Asked Questions document from the School of Computing Technologies (SCT) as “sensitive data”, and we challenged ourselves to see how we can use our internally deployed LLM to create training phrases (and alternative paraphrases of the correct answers in the FAQ), to create Walert, a conversational agent that answers questions about SCT programs. We also experimented with Retrieval-Augmented Generation (RAG) to better understand how to evaluate and compare the effectiveness of different designs of conversational agents, i.e., intent-based vs. RAG.

Walert at ACM CHIIR 2024

On March 2024, Walert made it all the way to Sheffield, UK.

Our team members Sachin and Futoon presented Walert at CHIIR'24, the 2024 ACM SIGIR Conference on Human Information Interaction and Retrieval.

The Team

The Walert team consists of HDR students, research fellows, and members of ADM+S and RMIT CIDDA (now members of the CHAI STEM Leading Centre), with experience in information access and retrieval, conversational user interfaces, natural language processing, software engineering, and machine learning: Sachin Pathiyan Cherumanal, Kaixin Ji, Lin Tian, Futoon Abu Shaqra, Angel Felipe Magnossão de Paula, Danula Hettiachchi, Halil Ali, Johanne Trippas, Falk Scholer, and Damiano Spina.


Walert was designed and developed in the unceded lands of the Woi Wurrung and Boon Wurrung peoples of the eastern Kulin Nation. We pay our respects to their Ancestors and Elders, past, present, and emerging. This research is partially supported by the Australian Research Council (DE200100064, CE200100005) and is undertaken with the assistance of computing resources from RACE (RMIT AWS Cloud Supercomputing). We thank Amina Hossain and Santha Sumanasekara for their valuable contributions.and the RMIT RACE Hub. We thank Amina Hossain and Santha Sumanasekara for their valuable contributions.