Support region coverage on codecov

[taiki-e]

readme says:

Currently, only line coverage is available on Codecov. This is because -Z instrument-coverage does not support branch coverage and Codecov does not support region coverage.

However, we can probably support region coverage on codecov by converting llvm-cov's json coverage format to codecov's json coverage format.

related:

• https://llvm.org/docs/CoverageMappingFormat.html

[flying-sheep]

So it seems that codecov doesn’t support full branch coverage? (“region coverage“ is just branch coverage without ignoring that there can be multiple branch arms in one line, right?)

Is there any tool other than llvm-cov show -format=html itself that visualizes full branch coverage?

[taiki-e]

Sorry, I can't understand the question. (See this LLVM patch for what branch coverage means in the LLVM source-based code coverage and for example report it generates.)

[flying-sheep]

Never mind the definitions: What I mean is that llvm-cov seems (to my knowledge) to be the only tool that is able to visualize which exact parts of the source code have been executed and which haven’t.

Oher tools will show e.g. the following as “covered” or “partially covered” because they work line-based:

[taiki-e]

Ah, thanks for the clarification. I also don't know of other tools that actually show areas that are not covered.

[andrewgazelka]

@taiki-e I am interested in potentially taking a stab at this. Do you already have a Rust deserialized version of CoverageMappingFormat?

[taiki-e]

Do you already have a Rust deserialized version of CoverageMappingFormat?

LlvmCovJsonExport is the data structure for that.

cargo-llvm-cov/src/json.rs

Line 9 in ad889ca

pub struct LlvmCovJsonExport {

[andrewgazelka]

This is because -Z instrument-coverage does not support branch coverage

@taiki-e so

cargo-llvm-cov/src/json.rs

Line 190 in e035f1d

pub(crate) branches: Option<Vec<serde_json::Value>>,

will always be None?

[taiki-e]

will always be None?

IIRC, it will be empty list (Some(vec![]))

EDIT:

cargo-llvm-cov/tests/fixtures/coverage-reports/real1/workspace_root_member2_manifest_path.full.json

Line 6 in 7f9c9ee

"branches": [],

[andrewgazelka]

So we would want to check a separate arg like --codecov-json here, right? If this were true we would convert LlvmCovJsonExport to the codecov JSON format. Is there a better place to do this conversion?

cargo-llvm-cov/tests/auxiliary/mod.rs

Lines 86 to 102 in 4c7c496

	pub fn normalize_output(output_path: &Utf8Path, args: &[&str]) -> Result<()> {
	if args.contains(&"--json") {
	let s = fs::read_to_string(output_path)?;
	let mut json = serde_json::from_str::<cargo_llvm_cov::json::LlvmCovJsonExport>(&s).unwrap();
	if !args.contains(&"--summary-only") {
	json.demangle();
	}
	fs::write(output_path, serde_json::to_vec_pretty(&json)?)?;
	}
	#[cfg(windows)]
	{
	let s = fs::read_to_string(output_path)?;
	// In json \ is escaped ("\\\\"), in other it is not escaped ("\\").
	fs::write(output_path, s.replace("\\\\", "/").replace('\\', "/"))?;
	}
	Ok(())
	}

[andrewgazelka]

@taiki-e here is a draft PR. I still need to test and add unit tests. Is there anything you would change?

#249