Short refactoring + dump comparison script

[AlexxMarkov]

Additional option for unit testing using rvsteel_sim_verilator and dump comparison using the rvsteel_core_unit_tests.py script.

The script runs rvsteel_sim_verilator with the verification program from programs. The generated dump file is then compared to the golden one from references. The result of the work is a report on the execution of tests in this form:

...
TEST PASS : programs/sub-01.hex
TEST PASS : programs/sw-align-01.hex
TEST PASS : programs/xor-01.hex
TEST PASS : programs/xori-01.hex
Total: passed 37, skipped 8, failed 0

The dump file and log are saved to a folder unit-tests/dump

[AlexxMarkov]

However, I still have a question, why when running (these) programs are the beginning and end of the signature the same? As a result, I get an empty dump file.

ebreak.hex
ecall.hex
fence-01.hex
misalign-lh-01.hex
misalign-lhu-01.hex
misalign-lw-01.hex
misalign-sh-01.hex
misalign-sw-01.hex

Temporarily skipping these programs.

[rafaelcalcada]

For some of these files the beginning and the end are not the same. For example, the first signature of fence-01.hex is ffffffff and the last is 00000000. Why is it an issue to have the same start and end signature?

Each line of the signature is the result of a unit test for the instruction. Sometimes the expected results for different unit tests are the same. So they end up filling the signature with the same value. This is expected and not a problem.

[AlexxMarkov]

Where can I find information about these signatures? Are there any options for bringing it together? Or maybe there is a table for the location of signatures? How can I unify this?

[rafaelcalcada]

The RISC-V Test Framework is now at its 3rd version. At the time I compiled the test programs and generated the signatures it was on its 2nd version. This version is available at the old-framework-2.x branch of the repository: https://github.com/riscv-non-isa/riscv-arch-test/tree/old-framework-2.x/doc

It is hard to explain in few words how to compile and generate the signatures. It requires a lot of work. You can find the source code of the unit tests in this folder: https://github.com/riscv-non-isa/riscv-arch-test/tree/old-framework-2.x/riscv-test-suite/rv32i_m/I/src

General instructions on how to compile and generate signatures can be found here: https://github.com/riscv-non-isa/riscv-arch-test/tree/old-framework-2.x/doc#porting-a-new-target

Hope it helps you! To double check that the signatures are indeed correct, you don't need to go through all this process. You can run the test programs in a RISC-V ISA simulator, like QEMU or Spike. The generated signatures will be the same you find in the files.

[rafaelcalcada]

I forgot to mention, the reference signatures for these tests can be found at https://github.com/riscv-non-isa/riscv-arch-test/tree/old-framework-2.x/riscv-test-suite/rv32i_m/privilege/references. They say in the docs that these signatures were generated by running the tests with RISC-V SAIL formal model.

[AlexxMarkov]

Do you plan to use Spike or QEMU for tests?

[rafaelcalcada]

I don't, currently I plan to implement GPIO and SPI interfaces so the SoC IP is more useful for embedded engineers trying to make something out of Steel.

I could not find in your simulation the code to reset the core before running a test. The reset signal must be held high for at least one clock cycle before running each test. This is important because the tests write to CSR registers and the Integer File, and a reset ensure everything is clean before running.

[AlexxMarkov]

Reset is performed by calling function dut_reset();

riscv-steel/ip/core/rvsteel-sim-verilator/main.cpp

Line 209 in 071e703

dut_reset();

[rafaelcalcada]

Sorry I hadn't noticed. Do you get a signature when running those tests? How do they differ from the golder references?

[AlexxMarkov]

When running the tests, memory dumps are obtained that correspond to the format in the golden reference.

[AlexxMarkov]

In the future, you can expand the functionality rvsteel_sim_verilator to generate an execution log. For subsequent comparison with the log of a reference simulator, for example Spike.

[rafaelcalcada]

That would be cool! One way to ensure that the design is correct is to analyse its execution trace against a reference simulator, and generating the execution log can be really helpful.

The tests above were not passing because the end of the simulation was being detected prematurely. A few tests write a value other than 0x00000001 to address 0x00001000 before the test ends. It was needed to compare write_data explicitly with 0x00000001.

Sorry my text editor clang-formatted your source code. Thanks again for contributing!

[AlexxMarkov]

Oops, looks like I wasn't paying attention, thanks for the correction.

By the way, if you want to look at wave you can add another argument to rvsteel_sim_verilator --out-wave=wave.fst. You can use GTKWave to view it *.fst

And one more thing :) If it is not important to you, please make 4 spaces for space 2 in the formatting of your text editor :)

[riscv-steel]

I added you as a colaborator of the repo. I changed the tab space of main.cpp to 4 spaces, though I prefer only 2 ;)

I created a branch called "dev" for development purposes. Everybody can push to this new branch.

Thanks for your contributions.