Organization of the project's data files
The data is organized within the data/ directory, which is not publicly distributed. The expected organization of the folder is as follows:
data/
├── humor_interpretation
│ ├── humicroedit.json
│ └── puntuguese.json
├── humor_recognition
│ ├── chumor.json
│ ├── clemencio.json
│ ├── HAHA@IberLEF2019.json
│ └── ...
└── large_corpora
├── bookcorpus.txt
├── brwac.txt
├── croissant.txt
└── ...
The data/ directory is organized according to the different tasks that the datasets are used for. In the following sections, we detail the datasets used, where to download them from, and how to prepare them.
All commands below should be run from the root directory of the project.
Humor datasets
We work with several datasets for humor recognition and interpretation, organized in the data/humor_recognition/ and data/humor_interpretation/ directories, respectively.
Humicroedit
Humicroedit can be downloaded from here. In this work, we use the Full Dataset Release.
After the dataset is downloaded and decompressed, we use the scripts utils/conversion/humor_interpretation/humicroedit_to_json.py and utils/conversion/humor_recognition/humicroedit_to_json.py to convert the dataset to our JSON format. To do this, run the following commands:
# Humor interpretation
python utils/conversion/humor_interpretation/humicroedit_to_json.py \
-t [PATH_TO_DATASET]/subtask-1/train.csv \
-d [PATH_TO_DATASET]/subtask-1/dev.csv \
-s [PATH_TO_DATASET]/subtask-1/test.csv
# Humor recognition
python utils/conversion/humor_recognition/humicroedit_to_json.py \
-t [PATH_TO_DATASET]/subtask-1/train.csv \
-d [PATH_TO_DATASET]/subtask-1/dev.csv \
-s [PATH_TO_DATASET]/subtask-1/test.csv
Puntuguese
This corpus can be downloaded from the following Github page: https://github.com/Superar/Puntuguese and converted using the utils/conversion/humor_interpretation/puntuguese_to_json.py and utils/conversion/humor_recognition/puntuguese_to_json.py scripts through the following commands:
# Humor interpretation
python utils/conversion/humor_interpretation/puntuguese_to_json.py \
-c [PATH_TO_DATASET]/data/puns.json
# Humor recognition
python utils/conversion/humor_recognition/puntuguese_to_json.py \
-d [PATH_TO_DATASET]/data/classification_corpus.json
Note that the input file that is passed to the scripts is different for the two tasks.
Clemêncio et al. (2020)
This Portuguese dataset can be downloaded from the project's Github page. Since this is a pun recognition-only dataset, we only use the utils/conversion/humor_recognition/clemencio_to_json.py script to convert the dataset.
# Humor recognition
python utils/conversion/humor_recognition/clemencio_to_json.py \
-d [PATH_TO_DATASET]/Datasets/Balanceados/all.txt
HAHA@IberLEF 2019
This dataset can be downloaded from here, with two separete files: train and test data. We highlight that there are two links for the test data; we use the one with gold annotations. Since this corpus does not contain the pun location task, we only convert it using the utils/conversion/humor_recognition/haha_to_json.py script:
# Humor recognition
python utils/conversion/humor_recognition/haha_to_json.py \
-t [PATH_TO_DATASET]/haha_2019_train.csv \
-s [PATH_TO_DATASET]/haha_2019_test_gold.csv
HAHA@IberLEF 2021
The 2021 version of the HAHA corpus can be obtained from their CodaLab page and follows roughly the same format as the 2019 one. Therefore, we use the same script to convert it, but including the newly-introduced validation dataset. The command to convert the dataset is as follows:
# Humor recognition
python utils/conversion/humor_recognition/haha_to_json.py \
-t [PATH_TO_DATASET]/haha_2021_train.csv \
-d [PATH_TO_DATASET]/haha_2021_dev_gold.csv \
-s [PATH_TO_DATASET]/haha_2021_test_gold.csv
HUHU@IberLEF 2023
The HUHU dataset is unfortunately close-sourced, so we cannot provide an easy way of downloading it. However, if you have access to the dataset, you can convert it using the utils/conversion/humor_recognition/huhu_to_json.py script with the following command:
# Humor recognition
python utils/conversion/humor_recognition/huhu_to_json.py \
-d [PATH_TO_DATASET]/train.csv
Joker CLEF
The JOKER shared task provided datasets in multiple languages, but unfortunately they are not publicly available as the registration for the task is closed. Nonetheless, if you have access to them, you can convert them using the single utils/conversion/humor_recognition/joker_to_json.py script, which already handles all the languages.
# Humor recognition
python utils/conversion/humor_recognition/joker_to_json.py \
-d [PATH_TO_DATASET]/Task\ 1\ -\ detection/
SemEval 2017 Task 7
The data for this task can be downloaded from their official page. Specifically, we use the trial and test data. Then, we convert the data using the utils/conversion/humor_recognition/semeval_to_json.py script:
# Humor recognition
python utils/conversion/humor_recognition/semeval_to_json.py \
-td [PATH_TO_DATASET]/data/test/subtask1-heterographic-test.xml \
-tl [PATH_TO_DATASET]/data/test/subtask1-heterographic-test.gold \
-md [PATH_TO_DATASET]/data/test/subtask1-homographic-test.xml \
-ml [PATH_TO_DATASET]/data/test/subtask1-homographic-test.gold
Chumor
The Chinese humor dataset is available through their HuggingFace page. We can download and convert it using the utils/conversion/humor_recognition/chumor_to_json.py script:
python utils/conversion/humor_recognition/chumor_to_json.py
Large corpora
To train the skipgram model and get word frequency counts, we needed to download a large corpus of texts in each of the languages we work with. The corpora we used are as follows:
- English: BookCorpus
- Portuguese: BRWaC
- Spanish: SBWC
- French: Croissant
- Chinese: Wikipedia
For more details on how to download and preprocess the data, please refer to docs/skipgram.md.
Word Embeddings (GloVe)
For the Quantum Entropy-based metrics, we use GloVe embeddings with 300 dimensions, following the original work. The embeddings are to be included to the data/embeddings/[LANGUAGE]/ directory, all under the name glove_s300.gensim. The embeddings are available from different sources:
- English: GloVe
- Portuguese: NILC Embeddings
- Spanish: SBWCE Project
- French: French Word Embeddings
- Chinese: We trained our own embeddings using the Wikipedia corpus and the original GloVe code.
Humetrix downloads these embeddings automatically form the Hugging Face Hub (NLP-CISUC/humetrix-resources; Portuguese from nilc-nlp/glove-300d). The published files were converted form the original gensim format with scripts/hub/export_embeddings.py.
N-gram counts
To compute the Surprise Ratio metrics, we needed to have 1-gram and 3-gram counts for each language across large corpora. The n-grams counts for English, Spanish, French, and Chinese were obtained from the Google Books n-gram frequency lists. For Portuguese, we use N-gram frequency counts from BRWaC.
Humetrix also automatically downloads these count tables from our Hugging Face Hub repository.