add sse to multipart upload #3
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…uickwit-oss#5428) * Upgrade actions/upload-artifact to match actions/download-artifact * Separate digests by platform * Merge artifacts * Fix artifact name
* add request_timeout config On very large datasets the fixed timeouts are too low for some queries. This PR adds a setting to configure the timeout. Two settings are introduced: - `request_timeout` on the node config - `QW_REQUEST_TIMEOUT` env parameter Currently there are two timeouts when doing a distributed search request, one from quickwit cluster when opening a channel and one from the search client. The timeout is applied to both (That means all cluster connections have the same request_timeout applied, not only search nodes) Related: quickwit-oss#5241 * move timeout to search config, add timeout tower layer * cancel search after timeout * use tokio::timeout * use global timeoutlayer
* Add test showing bulk order issuer * Return action responses in same order as request
…uickwit-oss#5438) The sql that is generated to expand an index name (index_id) into all of the matching index names uses the LIKE clause to do partial text matches. However, the only index on the index_id column utilizes the default btree configuration which can only do exact value comparisons. In environments with a large search volume over many thousands of indexes, this can be a bottle neck as the query will always do an full table scan. This drops and recreates the unique index on index_id to include the `varchar_pattern_ops` which allows for wildcard matching on a varchar column using the LIKE operator Fixes: quickwit-oss#5437
* Garbage collect shards in SQS Filesource * Run shard pruning in a background task * Expose deduplication window to users * Add integration test * Rename cleanup interval config * Address smaller review comments * Change strong_count to Weak * High level design * Remove unpure iterators * Rewrite time operation to rule out underflow * Remove inappropiate unwrap and fix typo * Refactor un-necessary deadline_for_last_extension paramter * Add more details to design document * Clarify what checkpoint_messages does
* Add debounce to shard prune request * Use pruning interval as control plane cooldown * Address smaller review comments * Simplify debounce logic * Use LruCache for debounce map * Refactor and test CooldownMap * Revert grpc number reuse * Rename grpc field interval to interval_secs
* Uncomment some tests * Remove ingest retries * Fix lint and comment flaky test * Remove ingest from basic test * Improve wording * Ignore test known to be flaky * Reference issue instead of TODO * Add back directory cleanup test with link to issue * Assert shutdown timeout
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Describe the proposed changes made in this PR.
How was this PR tested?
Describe how you tested this PR.