Data and Datasets
This file documents the data KIBA-D is using, both input and reference data, as well as the different data splits and versions. It also aims to document remote copies and reference sources for the individual data items.
Github
Datasets managed by Github are stored in the data/ folder.
| Folder or File | Description |
|---|---|
| external/faktencheck-db | Dump of the FEDA FA literature Postgres database from Nov 5th, 2025. Includes the Entity-Relationship-Diagram to document the DB schema. See USAGE.md and podman/faktencheck-db/README.md for instructions. |
| external/organism_trends/Weighted Vote Count Wald Literatur - Sheet1.csv | Reference / ground truth subset of the Weighted Vote Count trend data created/supervised by Maria (found here), for the forest habitat. The original data was matched via author and year to the Literature DB Overview Excel to add Zotero IDs. |
| external/organism_trends/Weighted Vote Count Agrar- und Offenland Literatur - Sheet1 | Reference / ground truth subset of the Weighted Vote Count trend data created/supervised by Maria (found here), for the agricultural and open landscapes habitat. The original data was matched via author and year to the Literature DB Overview Excel to add Zotero IDs. |
| external/organism_trends/Referenz_Wald_korrigiert_angepasst.csv | manually corrected version of Weighted Vote Count Wald Literatur - Sheet1.csv. See https://github.com/DFKI-NLP/kibad-llm/issues/558 for details. |
| external/zotero/Faktencheck_Artenvielfalt_Literaturdatenbank.csv | export to CSV of the Faktencheck Artenvielfalt Literaturdatenbank Zotero group library |
| external/zotero/Faktencheck_Artenvielfalt.csv | export of the Faktencheck Artenvielfalt Zotero group library |
| interim/faktencheck-db/faktencheck-db-converted_2025-11-05.jsonl | Reference / ground truth annotations for publications exported from the database dump via src/kibad_llm/data_integration/db_converter.py. See Faktencheck Postgres to Json Conversion |
| interim/faktencheck-db/faktencheck_core_corrected.jsonl | Corrected reference / ground truth annotations (as of Feb 2026) for the dev-set-100 PDFs and the core schema. |
| processed/faktencheck/ | Pre-processed versions of the input PDF files containing the extracted markdown-formatted text of each PDF. Stored for error analyses / extraction verification |
| kibad-llm-results/ | Outputs of the LLM pipeline runs, see kibad-llm-results/readme.md |
| raw/ | Currently empty |
DFKI cluster
The main data folder on the DFKI cluster is /ds/text/kiba-d/. It contains the following data folders:
| Folder | Type | Description |
|---|---|---|
| zotero_literaturdatenbank | Source (PDFs) | Contains 3917 PDF files for publications from from the Faktencheck Artenvielfalt Literaturdatenbank that are digitally available, provided by iDiv. PDFs are labeled by their Zotero Item ID, which is also the main key in the Postgres version of the literature database managed by FEDA. The folder is synched with https://cloud.dfki.de/owncloud/index.php/s/AC2XCHfDoza2rkb. See also the synch script https://github.com/DFKI-NLP/kibad-llm/blob/main/src/kibad_llm/data_integration/synch_nextcloud_with_cluster.py. |
| semantic_scholar | Metadata (JSON) and PDFs | (TODO add more details here) Metadata and PDFs retrieved via SemanticScholar as part of our crawling pipeline. See https://github.com/DFKI-NLP/kibad-crawler |
| splits | Source (PDFs) | train/dev/test/unlabeled splits created from the data in /ds/text/kiba-d/zotero_literaturdatenbank, for which PDFs were available. See issue https://github.com/DFKI-NLP/kibad-llm/issues/129 for statistics and the actual script used to create the splits. |
| dev-set-100 | Source (PDFs) | The original development set used for the core schema evaluation of the KIBA-D pipeline. Synched with https://cloud.dfki.de/owncloud/index.php/s/eZQCLcbPFXCf6E3 . See https://github.com/DFKI-NLP/kibad-project-management/issues/2#issuecomment-3350854943 |
| dev-set-Wald-WVC | Source (PDFs) | The original development set used for the OrganismTrend schema evaluation of the KIBA-D pipeline. Contains a total of 468 PDFs- all PDFs that were used in the weighted vote count evaluation for the Forest habitat/chapter (see https://github.com/DFKI-NLP/kibad-llm/blob/main/data/external/organism_trends/Weighted%20Vote%20Count%20Wald%20Literatur%20-%20Sheet1.csv, except rows with #NV), plus all additional publications with no trends that were cited in the Forest chapter of the Faktencheck Artenvielfalt. See https://github.com/DFKI-NLP/kibad-llm/issues/168. See https://cloud.dfki.de/owncloud/index.php/s/XoFaJ3r65m74o8Y for additional PDFs for WVC trends that are not included in the Zotero literature DB. |
| dev-set-Wald-178 | Source (PDFs) | A subset of the dev-set-Wald-WVC containing a sample of the PDFs used in the weighted vote counte evaluation for the Forest habitat/chapter. See issue https://github.com/DFKI-NLP/kibad-llm/issues/139 |
| test-set-AuO-WVC | Source (PDFs) | The original test set used for the OrganismTrend schema evaluation of the KIBA-D pipeline. Contains a total of 430 PDFs - all PDFs that were used in the weighted vote count evaluation for the Agrarian and Open Landscapes habitat/chapter (see here), plus a random sample of 225 additional publications with no trends that were cited in the Agrarian and Open Landscapes chapter of the Faktencheck Artenvielfalt. See https://github.com/DFKI-NLP/kibad-llm/issues/538. See here for additional PDFs for WVC trends that are not included in the Zotero literature DB. |
Owncloud & Web
| Name | URL | Type | Description |
|---|---|---|---|
| PDFs Literaturdatenbank | https://cloud.dfki.de/owncloud/index.php/s/AC2XCHfDoza2rkb | Contains 3917 PDF files for publications from from the Faktencheck Artenvielfalt Literaturdatenbank that are digitally available, provided by iDiv. PDFs are labeled by their Zotero Item ID, which is also the main key in the Postgres version of the literature database managed by FEDA. Synched with ds/text/kiba-d/zotero_literaturdatenbank. See also the synch script https://github.com/DFKI-NLP/kibad-llm/blob/main/src/kibad_llm/data_integration/synch_nextcloud_with_cluster.py |
|
| Testdokumente | https://cloud.dfki.de/owncloud/index.php/s/eZQCLcbPFXCf6E3 | The original development set used for the core schema evaluation of the KIBA-D pipeline, a subset of the 3917 PDFs in the previous folder. Synched with ds/text/kiba-d/dev-set-100 . See https://github.com/DFKI-NLP/kibad-project-management/issues/2#issuecomment-3350854943 |
|
| Zotero Group Library FA Literaturdatenbank | https://www.zotero.org/groups/4699059/faktencheck_artenvielfalt_literaturdatenbank | Zotero/Bibtex (PDF Metadata) | Zotero Group for the Faktencheck Artenvielfalt Literaturdatenbank. Ultimate reference source when it comes to what is part of the FA literature DB. |
| Google Sheet FA Literature DB Overview | https://docs.google.com/spreadsheets/d/1wiBPKG4SmiSNppC1-_qXzz__GIuxa3-C/edit | Google Sheet (Documentation) | Overview of the status, source, file type, etc, of the FA literature database entries. Can be used to obtain additional information on the annotation status of publications. |
| Google Sheet Fragenkatalog | https://docs.google.com/spreadsheets/d/1MyEZTLOm85ZNxuTSHxwJLxaAXiCGLt-mMozUkBlGzJE/edit?gid=0#gid=0 | Google Sheet (Documentation) | Overview / documentation of the annotation fields of the literature database including prompt suggestions, group information etc, as an input for the schema development of the KIBA-D LLM pipeline. |
| faktencheck-db-converted-2025-08-19 | https://cloud.dfki.de/owncloud/index.php/f/109584311) (DFKI Owncloud internal link!) | JSONL | Reference / ground truth annotations for publications exported from the database dump via src/kibad_llm/data_integration/db_converter.py. See Faktencheck Postgres to Json Conversion |
| faktencheck-db-converted-2025-11-05 | https://cloud.dfki.de/owncloud/index.php/f/109584311) (DFKI Owncloud internal link!) | JSONL | Reference / ground truth annotations for publications exported from the database dump via src/kibad_llm/data_integration/db_converter.py. See Faktencheck Postgres to Json Conversion |
| faktencheck_core_corrected | https://cloud.dfki.de/owncloud/index.php/f/109584311) (DFKI Owncloud internal link!) | JSONL | Corrected reference / ground truth annotations for the dev-set-100 |
| Trends-WeightedVoteCount (Organism Trends) | https://cloud.dfki.de/owncloud/index.php/s/5L2BHjc6oTEa3Bs | Excel | Weighted Vote Count reference data for organism trends created/supervised by Maria |
| ÖSL-VoteCount-Vollständig | https://cloud.dfki.de/owncloud/index.php/s/F2M6wqaoCytbt7Q | Excel | Weighted Vote Count reference data for ecosystem service trends created/supervised by Jori |
| Screening Tabellen | https://cloud.dfki.de/owncloud/index.php/s/emiJLtgSqA4xMcP | Excel/CSV | Exports of the relevance screening process conducted in ASReview |
| Screening Tabellen Duplikat? | https://drive.google.com/drive/folders/1v-vkFmrz2BpB0z3OeH9bjgYN6PJiUKcZ | Excel/CSV | Exports of the relevance screening process conducted in ASReview |