Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SVCOMP Benchmark Structure and Naming #29

Open
jckreg opened this issue Jan 17, 2025 · 1 comment
Open

SVCOMP Benchmark Structure and Naming #29

jckreg opened this issue Jan 17, 2025 · 1 comment

Comments

@jckreg
Copy link
Contributor

jckreg commented Jan 17, 2025

Problem

Benchmarks created with the SVCOMP target are not created in a compliant tree structure or naming convention. The current files maintain the original structure, as planned, and should be left as an option for when desired or for other targets.

Goal

Replace the multilevel nest structure with one benchmark folder with folders containing benchmarks renamed to Main.java. In the main folder there should also be all yaml files.
Example Current:

Benchmarks/
    NameA/
        Java/
            src/
                org/
                    com/
                        OriginalFileNameA.java
                        OriginalFileNameA.yml
    NameB/
        Java/
            src/
                org/
                    com/
                        OriginalFileNameB.java
                        OriginalFileNameB.yml

Example Desired:

Benchmarks/
    OriginalFileNameA/
        Main.java
    OriginalFileNameB/
        Main.java
    OriginalFileNameA.yml
    OriginalFileNameB.yml

Naming could be improved or changed to better represent the needs of the project. This is just the basic structure of the SVCOMP benchmarks on SoSy to maintain compliance while also trying to keep some of the original meta information.

@swflint
Copy link
Contributor

swflint commented Jan 17, 2025

A better example for the desired output would be as follows:

benchmarks/
  source-repo/
    OriginalFileNameA/
      Main.java
    OriginalFileNameB/
      Main.java
    OriginalFileNameA.yml
    OriginalFileNameB.yml

This would serve to help us locate license/source information for submission preparation, since autogenerated SPDX comments (#30) may not be correct (not all projects will have SPDX-detectable licence/source/authorship information).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants