Publications

2022

1. ACL

   Dataset Geography: Mapping Language Data to Language Users

   Faisal, Fahim, Wang, Yinkai,  and Anastasopoulos, Antonios

   In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) May 2022

   Abs Bib PDF Code Poster Slides Website

   As language technologies become more ubiquitous, there are increasing efforts towards expanding the language diversity and coverage of natural language processing (NLP) systems. Arguably, the most important factor influencing the quality of modern NLP systems is data availability. In this work, we study the geographical representativeness of NLP datasets, aiming to quantify if and by how much do NLP datasets match the expected needs of the language speakers. In doing so, we use entity recognition and linking systems, also making important observations about their cross-lingual consistency and giving suggestions for more robust evaluation. Last, we explore some geographical and economic factors that may explain the observed dataset distributions.

   @inproceedings{faisal-etal-2022-dataset,
     title = {Dataset Geography: Mapping Language Data to Language Users},
     author = {Faisal, Fahim and Wang, Yinkai and Anastasopoulos, Antonios},
     booktitle = {Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
     month = may,
     year = {2022},
     address = {Dublin, Ireland},
     publisher = {Association for Computational Linguistics},
     url = {https://aclanthology.org/2022.acl-long.239},
     doi = {10.18653/v1/2022.acl-long.239},
     pages = {3381--3411},
   }

2. AACL

   Phylogeny-Inspired Adaptation of Multilingual Models to New Languages

   Faisal, Fahim,  and Anastasopoulos, Antonios

   Accepted for publication in AACL 2022 May 2022

   arXiv Bib Code Slides

   @article{phylogeny,
     author = {Faisal, Fahim and Anastasopoulos, Antonios},
     keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
     title = {Phylogeny-Inspired Adaptation of Multilingual Models to New Languages},
     journal = {Accepted for publication in AACL 2022},
     publisher = {arXiv},
     year = {2022},
     copyright = {Creative Commons Attribution 4.0 International},
   }

2021

1. NLP4Prog

   Code to Comment Translation: A Comparative Study on Model Effectiveness & Errors

   Mahmud, Junayed,  Faisal, Fahim, Arnob, Raihan Islam, Anastasopoulos, Antonios,  and Moran, Kevin

   In Proceedings of the 1st Workshop on Natural Language Processing for Programming (NLP4Prog 2021) Aug 2021

   Abs Bib PDF

   Automated source code summarization is a popular software engineering research topic wherein machine translation models are employed to “translate” code snippets into relevant natural language descriptions. Most evaluations of such models are conducted using automatic reference-based metrics. However, given the relatively large semantic gap between programming languages and natural language, we argue that this line of research would benefit from a qualitative investigation into the various error modes of current state-of-the-art models. Therefore, in this work, we perform both a quantitative and qualitative comparison of three recently proposed source code summarization models. In our quantitative evaluation, we compare the models based on the smoothed BLEU-4, METEOR, and ROUGE-L machine translation metrics, and in our qualitative evaluation, we perform a manual open-coding of the most common errors committed by the models when compared to ground truth captions. Our investigation reveals new insights into the relationship between metric-based performance and model prediction errors grounded in an error taxonomy that can be used to drive future research efforts.

     title = {Code to Comment Translation: A Comparative Study on Model Effectiveness {\&} Errors},
     author = {Mahmud, Junayed and Faisal, Fahim and Arnob, Raihan Islam and Anastasopoulos, Antonios and Moran, Kevin},
     booktitle = {Proceedings of the 1st Workshop on Natural Language Processing for Programming (NLP4Prog 2021)},
     month = aug,
     year = {2021},
     address = {Online},
     publisher = {Association for Computational Linguistics},
     url = {https://aclanthology.org/2021.nlp4prog-1.1},
     doi = {10.18653/v1/2021.nlp4prog-1.1},
     pages = {1--16},
   }

2. EMNLP

   SD-QA: Spoken Dialectal Question Answering for the Real World

   Faisal, Fahim, Keshava, Sharlina, Alam, Md Mahfuz Ibn,  and Anastasopoulos, Antonios

   In Findings of the Association for Computational Linguistics: EMNLP 2021 Nov 2021

   Abs Bib

   Question answering (QA) systems are now available through numerous commercial applications for a wide variety of domains, serving millions of users that interact with them via speech interfaces. However, current benchmarks in QA research do not account for the errors that speech recognition models might introduce, nor do they consider the language variations (dialects) of the users. To address this gap, we augment an existing QA dataset to construct a multi-dialect, spoken QA benchmark on five languages (Arabic, Bengali, English, Kiswahili, Korean) with more than 68k audio prompts in 24 dialects from 255 speakers. We provide baseline results showcasing the real-world performance of QA systems and analyze the effect of language variety and other sensitive speaker attributes on downstream performance. Last, we study the fairness of the ASR and QA models with respect to the underlying user populations.

   @inproceedings{faisal-etal-2021-sd-qa,
     title = {{SD}-{QA}: Spoken Dialectal Question Answering for the Real World},
     author = {Faisal, Fahim and Keshava, Sharlina and Alam, Md Mahfuz Ibn and Anastasopoulos, Antonios},
     booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2021},
     month = nov,
     year = {2021},
     address = {Punta Cana, Dominican Republic},
     publisher = {Association for Computational Linguistics},
     url = {https://aclanthology.org/2021.findings-emnlp.281},
     pages = {3296--3315},
   }

3. MRQA

   Investigating Post-pretraining Representation Alignment for Cross-Lingual Question Answering

   Faisal, Fahim,  and Anastasopoulos, Antonios

   In Proceedings of the 3rd Workshop on Machine Reading for Question Answering Nov 2021

   Abs Bib

   Human knowledge is collectively encoded in the roughly 6500 languages spoken around the world, but it is not distributed equally across languages. Hence, for information-seeking question answering (QA) systems to adequately serve speakers of all languages, they need to operate cross-lingually. In this work we investigate the capabilities of multilingually pretrained language models on cross-lingual QA. We find that explicitly aligning the representations across languages with a post-hoc finetuning step generally leads to improved performance. We additionally investigate the effect of data size as well as the language choice in this fine-tuning step, also releasing a dataset for evaluating cross-lingual QA systems.

   @inproceedings{faisal-anastasopoulos-2021-investigating,
     title = {Investigating Post-pretraining Representation Alignment for Cross-Lingual Question Answering},
     author = {Faisal, Fahim and Anastasopoulos, Antonios},
     booktitle = {Proceedings of the 3rd Workshop on Machine Reading for Question Answering},
     month = nov,
     year = {2021},
     address = {Punta Cana, Dominican Republic},
     publisher = {Association for Computational Linguistics},
     url = {https://aclanthology.org/2021.mrqa-1.14},
     pages = {133--148},
   }

2020

1. JOI

   Mining Temporal Evolution of Knowledge Graphs and Genealogical Features for Literature-based Discovery Prediction

   Choudhury, Nazim,  Faisal, Fahim,  and Khushi, Matloob

   Journal of Informetrics Nov 2020

   Abs Bib

   Literature-based discovery process identifies the important but implicit relations among information embedded in published literature. Existing techniques from Information Retrieval (IR) and Natural Language Processing (NLP) attempt to identify the hidden or unpublished connections between information concepts within published literature, however, these techniques overlooked the concept of predicting the future and emerging relations among scientific knowledge components such as author selected keywords encapsulated within the literature. Keyword Co-occurrence Network (KCN), built upon author selected keywords, is considered as a knowledge graph that focuses both on these knowledge components and knowledge structure of a scientific domain by examining the relationships between knowledge entities. Using data from two multidisciplinary research domains other than the bio-medical domain, and capitalizing on bibliometrics, the dynamicity of temporal KCNs, and a recurrent neural network, this study develops some novel features supportive for the prediction of the future literature-based discoveries - the emerging connections (co-appearances in the same article) among keywords. Temporal importance extracted from both bipartite and unipartite networks, communities defined by genealogical relations, and the relative importance of temporal citation counts were used in the feature construction process. Both node and edge-level features were input into a recurrent neural network to forecast the feature values and predict the future relations between different scientific concepts/topics represented by the author selected keywords. High performance rates, compared both against contemporary heterogeneous network-based method and preferential attachment process, suggest that these features complement both the prediction of future literature-based discoveries and emerging trend analysis.

   @article{CHOUDHURY2020101057,
     title = {Mining Temporal Evolution of Knowledge Graphs and Genealogical Features for Literature-based Discovery Prediction},
     journal = {Journal of Informetrics},
     volume = {14},
     number = {3},
     pages = {101057},
     year = {2020},
     issn = {1751-1577},
     doi = {https://doi.org/10.1016/j.joi.2020.101057},
     url = {https://www.sciencedirect.com/science/article/pii/S1751157719304468},
     author = {Choudhury, Nazim and Faisal, Fahim and Khushi, Matloob},
     keywords = {Literature-based Knowledge Discovery, Dynamic Supervised Link Prediction, Keyword Co-occurrence Network (KCN), Genealogical Community, Weighted Temporal Citation},
   }

2019

1. ICAEE

   A Framework For Disease Identification From Unstructured Data Using Text Classification And Disease Knowledge Base

   Faisal, Fahim, Bhuiyan, Shafkat Ahmed, Ashraf, Faisal Bin,  and Kamal, Abu Raihan Mostofa

   In 2019 5th International Conference on Advances in Electrical Engineering (ICAEE) Nov 2019

   Bib

   @inproceedings{8975447,
     author = {Faisal, Fahim and Bhuiyan, Shafkat Ahmed and Ashraf, Faisal Bin and Kamal, Abu Raihan Mostofa},
     booktitle = {2019 5th International Conference on Advances in Electrical Engineering (ICAEE)},
     title = {A Framework For Disease Identification From Unstructured Data Using Text Classification And Disease Knowledge Base},
     year = {2019},
     volume = {},
     number = {},
     pages = {547-554},
     doi = {10.1109/ICAEE48663.2019.8975447},
   }

2018