Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

measure code coverage of runtime verification #88

Open
12 tasks
kiniry opened this issue Feb 18, 2022 · 0 comments
Open
12 tasks

measure code coverage of runtime verification #88

kiniry opened this issue Feb 18, 2022 · 0 comments
Labels
assurance Issues that relate to the assurance of the system, whether via models, code, informal, or formal. ENHANCEMENT New feature or request Q/A Quality assurance issue; validation, testing, and formal assurance. Someday Issues that are not required by our current contract, but we'd like to attend to someday.
Milestone

Comments

@kiniry
Copy link
Member

kiniry commented Feb 18, 2022

The following is written in Assurance.md, our assurance summary.

... we can measure source-level and binary-level code
runtime verification coverage of the system. In general, our RDE
methodology guarantees extremely high statement, block, function,
function call, branch, and condition/decision coverage of the
entire modeled system. It is commonly the case that we achieve
>98% coverage "out of the box", and then use either program
analysis or subject matter expert review to automatically generate
or hand-write the remaining <2% of unit, integration, or system
test cases. This methodology means that we need only manage,
evolve, and maintain two orders of magnitude fewer test cases than
a traditional agile or test-centric development process.

Measure the aforementioned kinds of coverage of our system.

  • source level x statement coverage
  • source level x block coverage
  • source level x function coverage
  • source level x function call coverage
  • source level x branch coverage
  • source level x condition/decision coverage
  • binary level x statement coverage
  • binary level x block coverage
  • binary level x function coverage
  • binary level x function call coverage
  • binary level x branch coverage
  • source level x condition/decision coverage

Note that we have not promised to do this in the SoW, so this is something that we can/should do if we have the hours on the project, or we should do afterwards with IR&D/MOB hours.

@kiniry kiniry added ENHANCEMENT New feature or request To Do assurance Issues that relate to the assurance of the system, whether via models, code, informal, or formal. Q/A Quality assurance issue; validation, testing, and formal assurance. labels Feb 18, 2022
@kiniry kiniry added Someday Issues that are not required by our current contract, but we'd like to attend to someday. and removed To Do labels Apr 18, 2022
@kiniry kiniry added this to the Assurance milestone May 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
assurance Issues that relate to the assurance of the system, whether via models, code, informal, or formal. ENHANCEMENT New feature or request Q/A Quality assurance issue; validation, testing, and formal assurance. Someday Issues that are not required by our current contract, but we'd like to attend to someday.
Projects
None yet
Development

No branches or pull requests

1 participant