1-Aug-2024 Source Code https://fordatis.fraunhofer.de/handle/fordatis/411
http://dx.doi.org/10.24406/fordatis/359

Can Large Language Models (LLMs) compete with Human Requirement Reviewers? - Replication of an Inspection Experiment on Requirements Documents

Seifert, Daniel (Fraunhofer-Institut für Experimentelles Software Engineering IESE); Jöckel, Lisa (Fraunhofer-Institut für Experimentelles Software Engineering IESE); Honroth, Thorsten (Fraunhofer-Institut für Experimentelles Software Engineering IESE); Trendowicz, Adam (Fraunhofer-Institut für Experimentelles Software Engineering IESE); Jedlitschk, Andreas (Fraunhofer-Institut für Experimentelles Software Engineering IESE); Ciolkowski, Marcus  

IESE Fraunhofer-Institut für Experimentelles Software Engineering
Files in This Item:
FileDescriptionSizeFormat 
profes24_published_data.zipSource code and evaluation results434,56 kBZIPDownload/Open

Abstract

This is the data for the paper "Can Large Language Models (LLMs) Compete with Human Requirements Reviewers? – Replication of an Inspection Experiment on Requirements Documents". It contains the source code of the experiment to make our work transparent and reproducible. Furthermore, it contains the evaluation results.

Technical Information

The code is written in python. For each experiment there is one jupyter notebook. We used OpenAI models and open source models. The open source models were run on our infrastructure. To run the code using the open source models, it must be adapted to the infrastructure you have access to.
Classification
000 Informatik, Informationswissenschaft, allgemeine Werke
Keywords
Large Language Models (LLMs)
Requirements Engineering
Inspection Experiment
Python
Quality Assessment
Funder
Bundesministerium für Bildung und Forschung BMBF (Deutschland)
Fraunhofer Group
IUK-Technologie
Show full item record


This item is licensed under a Creative Commons License Creative Commons