[OPIK-99] Implement batching for spans creation#298

Merged

andrescrz merged 26 commits intomainfrom OPIK-99-sdk-implement-batching-for-spans-creation

Sep 30, 2024

+623-31

Commits on Sep 30, 2024

Batch processing draft

alexkuzmik
authored and
andrescrz
committed
Batching draft

alexkuzmik
authored and
andrescrz
committed
[DRAFT] Update backend emulator for tests, update message processing to use batching

alexkuzmik
authored and
andrescrz
committed
Fix lint errors

alexkuzmik
authored and
andrescrz
committed
Implement _process_create_span_batch_message method in MessageProcessor

alexkuzmik
authored and
andrescrz
committed
Remove assert, add debug log message

alexkuzmik
authored and
andrescrz
committed
Add base batcher unit test

alexkuzmik
authored and
andrescrz
committed
Make batch manager flush all batches if streamer.flush() was called, add tests for span batcher

alexkuzmik
authored and
andrescrz
committed
Add tests for batch manager

alexkuzmik
authored and
andrescrz
committed
Add test for flushing thread

alexkuzmik
authored and
andrescrz
committed
Add one more test for batch manager

alexkuzmik
authored and
andrescrz
committed
Fix lint errors

alexkuzmik
authored and
andrescrz
committed
Fix lint errors

alexkuzmik
authored and
andrescrz
committed
Fix bug when backend emulator didn't add feedback scores to traces

alexkuzmik
authored and
andrescrz
committed
Fix lint errors

alexkuzmik
authored and
andrescrz
committed
Rename flush_interval to flush_interval_seconds

alexkuzmik
authored and
andrescrz
committed
Enable debug logs for e2e tests. Update debug messages in message_processors.py

alexkuzmik
authored and
andrescrz
committed
Remove export statement from e2e workflow

alexkuzmik
authored and
andrescrz
committed
Enable backend build. Enable debug logs for e2e tests

alexkuzmik
authored and
andrescrz
committed
Update docker compose and e2e workflow file

alexkuzmik
authored and
andrescrz
committed
Make batching disabled by default in Opik client. It is now enabled manually in Opik clients created under the hood of SDK

alexkuzmik
authored and
andrescrz
committed
Add more unit tests for message processing

alexkuzmik
authored and
andrescrz
committed
Add docstring for _use_batching parameter

alexkuzmik
authored and
andrescrz
committed
Add missing _SECONDS suffix

alexkuzmik
authored and
andrescrz
committed
Rename constant

alexkuzmik
authored and
andrescrz
committed
Undone e2e tests infra changes
andrescrz
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[OPIK-99] Implement batching for spans creation#298

Commits on Sep 30, 2024

Batch processing draft

Batching draft

[DRAFT] Update backend emulator for tests, update message processing to use batching

Fix lint errors

Implement _process_create_span_batch_message method in MessageProcessor

Remove assert, add debug log message

Add base batcher unit test

Make batch manager flush all batches if streamer.flush() was called, add tests for span batcher

Add tests for batch manager

Add test for flushing thread

Add one more test for batch manager

Fix lint errors

Fix lint errors

Fix bug when backend emulator didn't add feedback scores to traces

Fix lint errors

Rename flush_interval to flush_interval_seconds

Enable debug logs for e2e tests. Update debug messages in message_processors.py

Remove export statement from e2e workflow

Enable backend build. Enable debug logs for e2e tests

Update docker compose and e2e workflow file

Make batching disabled by default in Opik client. It is now enabled manually in Opik clients created under the hood of SDK

Add more unit tests for message processing

Add docstring for _use_batching parameter

Add missing _SECONDS suffix

Rename constant

Undone e2e tests infra changes