Configuration Options

This page lays out configuration options for three key migrations:

1. Metadata migration
2. Document migration
3. Live Traffic replication

Notice that each of these migrations may be dependent upon either a snapshot or capture proxy. Also note that other than the snapshot/proxy prerequisite, each of these three migrations are independent of each other. For a complete list of options, please refer to opensearch-migrations options.md

Metadata Migration Options

There are no current options for configuring Metadata migrations, which are performed from the Migration Console. This migration requires an existing snapshot, which can be created when accessing the Migration Console.

Document Migration Options

Name	Example	Description
sourceClusterEndpoint	"https://source-cluster.elb.us-east-1.endpoint.com"	The endpoint for the source cluster from which RFS will take a snapshot
reindexFromSnapshotServiceEnabled	true	Create resources for deploying and configuring the RFS ECS service
reindexFromSnapshotExtraArgs	"--target-aws-region us-east-1 --target-aws-service-signing-name es"	Extra arguments to provide to the Document Migration command with space separation. See RFS Extra Arguments below for available options. Boolean negation is used, so --no-insecure can be passed to remove the --insecure flag if present.

To find all available arguments that can be provided to reindexFromSnapshotExtraArgs see here. At a minimum no extra arguments may be needed

Live Traffic Replication Options

Name	Example	Description
trafficReplayerServiceEnabled	true	Enable deploying the given service, via a new CloudFormation stack
trafficReplayerGroupId	"logging-group-default"	The Kafka consumer group ID the Replayer will use, if not specified a default ID will be used
trafficReplayerUserAgentSuffix	"AWS/test/v1.0.0"	A custom user agent that will be provided to the Replayer using the --user-agent flag. This will append the provided user agent to any existing user agents when requests are made to the target cluster. This setting could also be specified with the trafficReplayerExtraArgs option
trafficReplayerMaxUptime	"P1D"	The maximum uptime for the Traffic Replayer service, specified in ISO 8601 duration format. This controls how long the Traffic Replayer will run before automatically shutting down. Example values: "PT1H" (hourly), "P1D" (daily). When this duration is reached, ECS will initiate the startup of a new Traffic Replayer task to ensure continuous operation. This mechanism ensures that the Traffic Replayer service can manage its resources effectively and prevent issues associated with long running processes. Set to the greater of the given value 5 minutes. When not specified, the replayer will run continuously.
trafficReplayerExtraArgs	"--sigv4-auth-header-service-region es,us-east-1 --speedup-factor 5"	Extra arguments to provide to the Replayer command. This includes auth header options and other parameters supported by the Traffic Replayer.

To find all available arguments that can be provided to trafficReplayerExtraArgs see the @Parameter fields here. At a minimum no extra arguments may be needed, but currently cases such as using SIGV4 auth for the target cluster require additional arguments e.g. --sigv4-auth-header-service-region es,us-east-1.

TODO: Make Replayer args clearer in a README file TODO: Present a clear path for what options need to be provided for different auth patterns (no auth, basic auth, sigv4) that are used for source/target cluster

Owner: @lewijacn