Skip to content

Commit

Permalink
Introduce default max number of bucket per connector
Browse files Browse the repository at this point in the history
- Also add the mongodb version that stop supporting mutable
  pipelines in the changestreams, allowing to reuse a resume
  token from another one.

Issue: BB-601
  • Loading branch information
williamlardier committed Sep 11, 2024
1 parent 1378e7b commit f7fd8f0
Showing 1 changed file with 5 additions and 0 deletions.
5 changes: 5 additions & 0 deletions extensions/oplogPopulator/constants.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
const constants = {
bucketMetastore: '__metastore',
defaultConnectorName: 'source-connector',
// Max length in a pipeline is equal to the MongoDB BSON max document size,
// so 16MB. To allow for other parameters in the pipeline, we round the max
// to 16 MB (16777216B) / 64 (max length of a bucket name) ~= 260000
maxBucketPerConnector: 260000,
mongodbVersionWithImmutablePipelines: '6.0.0',
defaultConnectorConfig: {
'connector.class': 'com.mongodb.kafka.connect.MongoSourceConnector',
'pipeline': '[]',
Expand Down

0 comments on commit f7fd8f0

Please sign in to comment.