Skip to content

Commit

Permalink
fix: use native cargo bench format and only run benchmarks crate
Browse files Browse the repository at this point in the history
  • Loading branch information
timonv committed Jun 28, 2024
1 parent 5f09c11 commit 6430af7
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 53 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/bench.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- name: Run benchmark
run: cargo bench | tee benchmarks/output.txt
run: cargo bench -p benchmarks -- --output-format bencher | tee benchmarks/output.txt

- name: Store benchmark result
uses: benchmark-action/github-action-benchmark@v1
Expand Down
55 changes: 3 additions & 52 deletions benchmarks/output.txt
Original file line number Diff line number Diff line change
@@ -1,55 +1,6 @@
load_1 time: [7.1314 ns 7.1395 ns 7.1484 ns]
change: [-53.094% -52.641% -52.200%] (p = 0.00 < 0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) high mild
1 (1.00%) high severe
test load_1 ... bench: 6 ns/iter (+/- 0)

load_10 time: [7.0139 ns 7.0252 ns 7.0388 ns]
change: [-17.234% -15.784% -14.432%] (p = 0.00 < 0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
5 (5.00%) high mild
3 (3.00%) high severe
test load_10 ... bench: 6 ns/iter (+/- 0)

run_local_pipeline time: [847.33 ns 848.35 ns 849.87 ns]
change: [-18.532% -17.395% -16.332%] (p = 0.00 < 0.05)
Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
9 (9.00%) high severe


running 30 tests
test ingestion::ingestion_pipeline::tests::test_arbitrary_closures_as_batch_transformer ... ignored
test ingestion::ingestion_pipeline::tests::test_arbitrary_closures_as_transformer ... ignored
test ingestion::ingestion_pipeline::tests::test_concurrent_calls_with_simple_transformer ... ignored
test ingestion::ingestion_pipeline::tests::test_simple_run ... ignored
test ingestion::ingestion_pipeline::tests::test_skipping_errors ... ignored
test integrations::aws_bedrock::simple_prompt::test::test_prompt_with_anthropic ... ignored
test integrations::aws_bedrock::simple_prompt::test::test_prompt_with_titan ... ignored
test integrations::fastembed::tests::test_fastembed ... ignored
test integrations::redis::node_cache::tests::test_redis_cache ... ignored
test integrations::redis::persist::tests::test_redis_batch_persist ... ignored
test integrations::redis::persist::tests::test_redis_custom_persist ... ignored
test integrations::redis::persist::tests::test_redis_persist ... ignored
test integrations::scraping::html_to_markdown_transformer::test::test_html_to_markdown ... ignored
test integrations::treesitter::splitter::test::test_chunk_lines ... ignored
test integrations::treesitter::splitter::test::test_empty_text ... ignored
test integrations::treesitter::splitter::test::test_max_bytes_limit ... ignored
test integrations::treesitter::splitter::test::test_range_max ... ignored
test integrations::treesitter::splitter::test::test_range_min_and_max ... ignored
test integrations::treesitter::splitter::test::test_split_single_chunk ... ignored
test integrations::treesitter::supported_languages::test::test_supported_languages_from_str ... ignored
test integrations::treesitter::supported_languages::test::test_supported_languages_from_str_case_insensitive ... ignored
test loaders::file_loader::test::test_with_extensions ... ignored
test persist::memory_storage::test::test_batch_store ... ignored
test persist::memory_storage::test::test_inserting_multiple_nodes ... ignored
test persist::memory_storage::test::test_memory_storage ... ignored
test transformers::metadata_keywords::test::test_metadata_keywords ... ignored
test transformers::metadata_qa_code::test::test_metadata_qacode ... ignored
test transformers::metadata_qa_text::test::test_metadata_qacode ... ignored
test transformers::metadata_summary::test::test_metadata_summary ... ignored
test transformers::metadata_title::test::test_metadata_title ... ignored

test result: ok. 0 passed; 0 failed; 30 ignored; 0 measured; 0 filtered out; finished in 0.00s
test run_local_pipeline ... bench: 846 ns/iter (+/- 7)

0 comments on commit 6430af7

Please sign in to comment.