-
Müller, Hanna authoredMüller, Hanna authored
Advanced Information Retrieval 2023 - Project - Group 20
This is our project for the course Advanced Information Retrieval 2023, TU Graz. In this project, we use BERTopic to extract topic information from documents and queries and incorporate the topic information for re-ranking documents after the initial retrieval with BM25.
Authors: Gatternig Elias, Müller Hanna, Palasser Georg
Dataset
We used the publicly available CISI collection of the University of Glasgow, containing 1460 documents and 112 queries. It contains documents, queries and a "ground-truth" of query-document matchings.
How to run
To run the first make sure all the requirements are met. Simply use the command "pip install -r requirements.txt" in your terminal. This will install all the packages required to run our code. Check if all libraries are installed.
Secondly, for running the "reranker-cosine.ipynb" notebook it is necessary to add the "GoogleNews-vectors-negative300.bin" word-embeddings file into the "models" folder as they were not uploaded to the repository initially, because of its large size. However, they can be downloaded here.
Files
initial-retrieval.ipynb Creating the initial retrieval of the queries using bm25. Retrieves 100 documents out of 1460 documents per query and saves the results in initial_retrieval_with_bm25_scores.pkl.
reranker-cosine.ipynb Re-ranks the retrieval results with the help of cosine similarity and the pre-trained embeddings of GoogleNews-vectors-negative300. Retrieves 50 documents out of the initial 100 per query and saves the results in reranker_embeddings_cosine_results.pkl.
reranker-bertopic.ipynb Creates topics for all documents and queries and re-ranks the initial retrieval results. Retrieves 50 documents per query and saves the results in reranker_bertopic_results_topic_model.pkl.
evaluation.ipynb Takes the results of all three methods and calculates Recall@k, Precicsion@k, F1@k and nDCG@k. Creates plots for visualizing the results.
dataset_and_bertopic_analysis.ipynb Contains analysis of the dataset and BERTopic experiments.