About Me
Womin djeka! This is my personal – not an official RMIT – website. I’m a Senior Lecturer at RMIT University (School of Computing Technologies), an Associate Investigator at the ARC Centre of Excellence for Automated Decision-Making and Society, RMIT Research Leader at the Australian Internet Observatory, and a member of the International Panel on the Information Environment. I was the recipient of an ARC Discovering Early-Career Research Award (DECRA).
My main research interests are Information Retrieval (IR), Text Analytics, and Data Science. I currently work on evaluation of information access systems and interactive information retrieval.
In my free time, I teach Capoeira at the Associação de Capoeira Descendente do Pantera (ACDP) – where I am also known as Contramestre Camaleão – and play samba with Wombatuque.
I acknowledge the people of the Woi wurrung and Boon wurrung language groups of the eastern Kulin Nation on whose unceded lands I live, work, learn, and teach. I respectfully acknowledge their Ancestors and Elders, past and present. I also acknowledge the Traditional Custodians and their Ancestors of the lands, waters, and skies across Australia, and their connection to Country.
Interests
- Information Retrieval
- Human-AI Cooperation
- Evaluation Methodologies
Education
PhD in Computer Science, 2014
UNED (Madrid, Spain)
Affiliations
RMIT University
Senior Lecturer, School of Computing Technologies
ADM+S
Associate Investigator, ARC Centre of Excellence for Automated Decision-Making and Society
AIO
RMIT Research Lead, Australian Internet Observatory
Publications
(2024).
Overview of EXIST 2024 -- Learning with Disagreement for Sexism Identification and Characterization in Tweets and Memes.
Experimental IR Meets Multilinguality, Multimodality, and Interaction.
(2024).
Adapting Generative Information Retrieval Systems to Users, Tasks, and Scenarios.
Information Access in the Era of Generative AI.
(2024).
Advancing Physiological Methods for Human-Information Interaction.
Companion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing.
(2024).
Can Users Detect Biases or Factual Errors in Generated Responses in Conversational Information-Seeking?.
Proceedings of the 2024 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region.
Projects
The Australian Internet Observatory (AIO) is a new, four-year national research infrastructure initiative that will create an interconnected ecosystem of people, data and tools to support innovative approaches to the collection and analysis of digital social data across a range of disciplines and sectors. It will enable researchers to explore topics such as the distribution of misinformation, the patterns of everyday engagement with business, culture and science, flows of communication in emergencies and humanitarian crises, and the dynamics of political conflict and consensus.
AIO will develop the tools and capabilities required to gather and analyse online user experience data, algorithms, and interactions. It will support innovative approaches to the collection and analysis of digital social data and internet platforms and the analytical tools and governance required to support cutting-edge research on social, economic, health, and environmental issues.
Woi wurrung / Boon wurrung word for ‘possum’, Walert is a conversational agent that answers questions about prgrams at the RMIT School of Computing Technolgies (SCT). We used an in-house deployed large language model to rephrase questions and answers in an existing Frequently Asked Questions document for SCT programs to train the conversation model of Walert. We built an evaluation framework to compare the effectiveness of intent-based and Retrieval-Augmented Generation (RAG) architectures for customized chatbots.
EXIST is a series of scientific events and shared tasks on sexism identification in social networks. EXIST aims to capture sexism in a broad sense, from explicit misogyny to other subtle expressions that involve implicit sexist behaviours (EXIST 2021, EXIST 2022). The third and fourth editions of the EXIST shared task was held as a Lab in CLEF 2023. [NEW] Next CLEF 2025 in Madrid, Spain will host the fifth edition of EXIST, including tasks to study sexism characterization in text (microblog posts/tweets), images (memes) and video (TikTok)!
Recent developments in machine learning, information access, and AI communities attempt to define mechanisms to minimise the creation and reinforcement of unintended cognitive biases. This research aims to address research questions related to quantifying and measuring bias and engagement by focusing on information access systems that involve automated decision-making components. By partnering with experts in fact-checking, we use misinformation management as the main scenario of study, given that bias and engagement play an important role in three main elements of the automated decision-making processes: the user, the system, and the information that is presented and consumed.
Cortana Intelligence Institute was a co-funded initiative between Microsoft Research, Cortana Research and RMIT University (2018-2020), which advanced knowledge on new capabilities of digital assistants. The Institute focused on researching work-related tasks, an area that helped to develop more proactive and context-aware digital assistants. Microsoft’s researchers collaborated with RMIT team members on the development of algorithms that used a novel dataset to improve Cortana.
RepLab: Online Reputation Monitoring
RepLab is a competitive evaluation exercise for Online Reputation Management systems organized as an activity of CLEF for the 2012, 2013, and 2014 editions. RepLab focused on the task of monitoring the reputation of entities (companies, organizations, celebrities, etc.) on Twitter. The monitoring task for analysts consists of searching the stream of tweets for potential mentions to the entity, filtering those that do refer to the entity, detecting topics (i.e., clustering tweets by subject) and ranking them based on the degree to which they signal reputation alerts (i.e., issues that may have a substantial impact on the reputation of the entity).
Supervision
Current PhD Students
- Co-Supervision of PhD student Jiaman He, working on Neurophysiological Approaches to Interactive Information Retrieval (together with Johanne R. Trippas and Dana McKay)
- Co-supervision of Phd student Fletcher Scott (Psychology, School of Health and Biomedical Sciences) working on Psychological Effects in Misinformation Sharing (together with Lauren L. Saling)
- Co-supervision of PhD student Devi Mallal (School of Design), working on Human-AI Cooperation for More Accessible Verified Content (together with Maria Nancy Mauro-Flude and Yoko Akama)
- Co-supervision of PhD candidate Angel Felipe Magnossão de Paula, working on Sexism and Hate Speech Characterization using Large Language Models (together with Paolo Rosso).
- Co-supervision of PhD candidate Kaixin Ji, working on Measuring and Quantifying Bias and Engagement in Interactive Information Access (together with Flora D. Salim, Falk Scholer, and Danula Hettiachchi).
- Co-supervision of PhD candidate Rahmad Mahendra, working on Semantic Matching for Clinical Trial Retrieval (together with Karin Verspoor and Lawrence Cavedon).
- Co-supervision of PhD candidate Sachin Pathiyan Cherumanal, working on Fairness-Aware Question Answering for Intelligent Assistants (together with Falk Scholer and W. Bruce Croft).
Current Master and Hounours Students
- Tony Smith (Master of Data Science), working on Sexism Characterization in Social Networks (EXIST @ CLEF'24), co-supervised with Johanne R. Trippas.
- Kun Ran (Master of AI), working on Characterising Information Needs and Query Variants via Large Language Models, co-supervised with Marwah Alaofi and Mark Sanderson.
Former PhD Students
Dr. Ameer Albahem, Evaluating Dynamic Search Systems for Interactive Complex Tasks (co-supervised with Lawrence Cavedon and Falk Scholer).
Dr. Johanne R. Trippas, Spoken conversational search: audio-only interactive information retrieval (co-supervised with Lawrence Cavedon and Mark Sanderson).
Former Master and Honours Students
- Reham Abdullah Altalhi, Search Results Fairness based on Analytical Hierarchy Process, RMIT University, 2022.
- Binh Chon Nut Le, Investigating Algorithmic Bias via Crowdsourcing, RMIT University, 2020. [Paper]
- Xinhuan Duan, Two-Step Classification for Profiling Fake News Spreaders on Twitter, RMIT University, 2020. [Paper]
- Shubhdeep Singh, Spoken DialogWOZ : A Tool to PerformWizard of Oz Experiments in Speech-Only Question Answering Scenarios, RMIT University, 2020.
- Assunta Cerone, Watch ’n’ Check: Towards a Social Media Monitoring Tool to Assist Fact-Checking Experts, Politecnico di Torino, 2020. [Paper]
- Mazhar Morshed, Missing Attribute Imputation through Clustering in New Item Cold Start Job Recommendation, RMIT University, 2017.
- Phanomsinh Homsombath, Applying Phonetic Matching to Spoken Document Retrieval, RMIT University, 2015.
Visitors
- 2023: PhD student Weronika Ławejska (University of Stavanger, Norway)
- 2022-2023: Dr. Jorge Carrillo-de-Albornoz & Dr. Laura Plaza (UNED, Spain)
Awards
Award for Excellence in Reviewing
Sachin Pathiyan Cherumanal, Kaixin Ji, Danula Hettiachchi, Johanne R. Trippas, Falk Scholer, and Damiano Spina. 2023. RMIT_IR at the NTCIR-17 FairWeb-1 Task. In: Proceedings of 17th Conference on Evaluation of Information Access Technologies (NTCIR-17). https://doi.org/10.20736/0002001315
See certificate
Kaixin Ji, Damiano Spina, Danula Hettiachchi, Falk Scholer, and Flora D. Salim. 2023. Towards Detecting Tonic Information Processing Activities with Physiological Data. In: Adjunct Proceedings of the 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing & the 2023 ACM International Symposium on Wearable Computing (UbiComp/ISWC ‘23 Adjunct). Association for Computing Machinery, New York, NY, USA, 1–5. https://doi.org/10.1145/3594739.3610679
See certificate
The RMIT Research Awards and Prizes focus on research excellence and research impact. It is through these research contributions that RMIT continues to bridge the gap between research and impact, benefitting those beyond the academic community. The research awards and prizes acknowledge the accomplishments of individual researchers, research teams, research leaders, supervisors, and HDR candidates.
See certificate
Fair and Transparent Information Access in Spoken Conversational Assistants. This project aims to investigate how rich information needed to answer complex questions can be delivered via a speech-only communication channel. Using laboratory user studies, where users can interact with a smart speaker to ask for information about controversial or multi-perspective topics, the project expects to advance knowledge on how to expose pertinent information without creating or reinforcing biases. Expected outcomes include novel presentation strategies to access rich information via audio in a fair manner. This should significantly benefit the visually impaired and low-literacy communities by enhancing their access to topics with multiple point of views, which would impact decision making such as who to vote for in elections.
See certificate
La Barbera D., Roitero K., Demartini G., Mizzaro S., Spina D. (2020) Crowdsourcing Truthfulness: The Impact of Judgment Scale and Assessor Bias. In: Jose J. et al. (eds) Advances in Information Retrieval. ECIR 2020. Lecture Notes in Computer Science, vol 12036. Springer, Cham. https://doi.org/10.1007/978-3-030-45442-5_26
See certificate
Albahem A., Spina D., Scholer F., Cavedon L. (2019) Meta-evaluation of Dynamic Search: How Do Metrics Capture Topical Relevance, Diversity and User Effort?. In: Azzopardi L., Stein B., Fuhr N., Mayr P., Hauff C., Hiemstra D. (eds) Advances in Information Retrieval. ECIR 2019. Lecture Notes in Computer Science, vol 11437. Springer, Cham. https://doi.org/10.1007/978-3-030-15712-8_39
See certificate
In recognition of the review made for the Elsevier’s international journal Information Processing & Management (IP&M), based on number of reviews, quality of reviews, and timeliness of responses.
See certificate