PlotQA

PlotQA is a VQA dataset with 28.9 million question-answer pairs grounded over 224,377 plots on data from real-world sources and questions based on crowd-sourced question templates.

Existing synthetic datasets (FigureQA, DVQA) for reasoning over plots do not contain variability in data labels, real-valued data, or complex reasoning questions. Consequently, proposed models for these datasets do not fully address the challenge of reasoning over plots. In particular, they assume that the answer comes either from a small fixed size vocabulary or from a bounding box within the image. However, in practice this is an unrealistic assumption because many questions require reasoning and thus have real valued answers which appear neither in a small fixed size vocabulary nor in the image. In this work, we aim to bridge this gap between existing datasets and real world plots by introducing PlotQA. Further, 80.76% of the out-of-vocabulary (OOV) questions in PlotQA have answers that are not in a fixed vocabulary. PlotQA paper (WACV 2020)

Download the dataset of 28.9 million question-answer pairs grounded over 224,377 plots.

• Plot Images
• Annotations
• QA Pairs

Ask us questions at nmethani@cse.iitm.ac.in and prithag@cse.iitm.ac.in.

Thank you SQuAD for allowing us to use the code to create this website.

To assess the difficulty of the PlotQA dataset, we report human accuracy on a small subset of the Test split of the dataset. We also evaluate three state-of-the-art models on PlotQA and observe that uur proposed hybrid model significantly outperforms the existing models. It has an aggregate accuracy of 22.52% on the PlotQA dataset. We acknowledge that the accuracy is significantly lower than human performance. This establishes that the dataset is challenging and raises open questions on models for visual reasoning.

We evaluate our model on the test-set of DVQA. Our proposed hybrid model performs better than the existing models (SAN and SANY-OCR) establishing a new state-of-the-art result on DVQA.