Azure Redis CLI does not Support Geo-Replication Configuration

[archmangler]

az feedback auto-generates most of the information requested below, as of CLI version 2.0.62

Describe the bug

• The azure cli command for configuring redis cache geo-replication only works when the redis instances are a) in the same resource group and therefore b) in the same region. Effectively, this means geo-replication cannot be configure via az cli at the moment since functionality is limited to geo-replication within a single region.

• The azure cli documentation for configuring redis cache geo-replication is not clear on which resource-group should be specified (the resource group of the primary redis instance or the secondary?). Unless this implies that both instances should be in the same resource group (which is not clear from the documentation).

To Reproduce

• Test Code bootstrap #1: Azure cli geo-replication configuration within the same region and same resource-group:

az redis server-link create --debug --name rds-1-region1 --replication-role Primary --resource-group rsg-region1 --server-to-link rds-2-region1

• Test Add current directory to search path to allow load of modules without… #2: Azure cli geo-replication configuration with redis instances in different regions and separate resource groups

az redis server-link create --debug --name rds-1-region1--replication-role Primary --resource-group rsg-region2 --server-to-link rds-2-region2
.
.
{"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Cache/Redis/rds-1-region1' under resource group 'rsg-region2' was not found."}}
.
.

Expected behavior

Azure CLI should allow Geo-replication to be configured between redis caches in separate regions and separate resource groups.

Environment summary

Install Method (e.g. pip, interactive script, apt-get, Docker, MSI, edge build) / CLI version (az --version) / OS version / Shell Type (e.g. bash, cmd.exe, Bash on Windows)

Additional context

When the instances are in the same resource group and same region, replication configuration works. However, that does not achieve our use-case. Putting the instances into separate regions requires separate resource groups. However, once we use separate resource groups the az cli command for setting up replication fails, unwilling to accept either resource group as an argument. So it seems the only conclusion to deduce from that is that the az cli only supports geo-replication within a region and within the same resource group and therefore is not actually "geo" replication, but just "replication"

[alfantp]

CLI for azure redis cache does support replication between multiple regions.
https://docs.microsoft.com/en-us/cli/azure/redis/server-link?view=azure-cli-latest#commands
If you see the documentation for this operation, you can see it accepts name and resource group of primary cache, and accepts resource ID or name of secondary cache. The name of the argument for secondary cache is –server-to-link
If you are replicating to a cache within the same Resource group, you need to provide just the cache name here. But if you are replicating to a cache in a different Resource Group, you can provide the resource ID here. This is in line with design principles for CLI:
https://github.com/Azure/azure-cli/blob/dev/doc/authoring_command_modules/authoring_commands.md#supporting-name-or-id-parameters
You can see details of parameters with the command az redis server-link create -h

[archmangler]

@alfantp - I'm still getting the error which seems to indicate otherwise. Here in more detail:

• Here are my 2 redis instances:

(redis #1)

az1_rediscache_00001_03.azurerm_redis_cache.instance:

  id = /subscriptions/redacted/resourceGroups/rsg-lol-dev-az1-00001/providers/Microsoft.Cache/Redis/rds-lol-dev-az1-000013

(redis #2)

az2_rediscache_00001_02.azurerm_redis_cache.instance:

  id = /subscriptions/redacted/resourceGroups/rsg-lol-dev-az2-00001/providers/Microsoft.Cache/Redis/rds-lol-dev-az2-00001

• Here are the 4 variations of the link command I've used

1) az redis server-link create --debug --name rds-lol-dev-az1-000013 --replication-role Primary --resource-group rsg-lol-dev-az1-00001 --server-to-link rds-lol-dev-az2-00001
2) az redis server-link create --debug --name rds-lol-dev-az1-000013 --replication-role Primary --resource-group rsg-lol-dev-az2-00001 --server-to-link rds-lol-dev-az2-00001
3) az redis server-link create --name rds-lol-dev-az1-000013 --replication-role Primary --resource-group rsg-lol-dev-az2-00001 --server-to-link rds-lol-dev-az2-00001 --replication-role Secondary --resource-group rsg-lol-dev-az1-00001
4) az redis server-link create --name rds-lol-dev-az1-000013 --replication-role Primary --resource-group rsg-lol-dev-az1-00001 --server-to-link rds-lol-dev-az2-00001 --replication-role Secondary --resource-group rsg-lol-dev-az2-00001

All of these give the same type of error:

ERROR: The Resource 'Microsoft.Cache/Redis/rds-lol-dev-az2-00001' under resource group 'rsg-lol-dev-az1-00001' was not found.

NOTE: No matter what permutations of the command, the indication is always that the primary redis instances is being looked for under the resource group of the secondary redis cache.

• Detailed debugging: Running the commands and testing with the resource groups of both redis caches:

1. Specify the primary redis cache' resource group (rsg-lol-dev-az2-00001)

• Snippet of bash code that builds the command:

if [ ${configuration_operation} == "link" ]
then
echo "debug> creating replication link: ${primary_redis_name} => ${secondary_redis_name}"
configure_command="az redis server-link create --debug --name ${primary_redis_name} --replication-role Primary --resource-group ${resource_group_name_primary} --server-to-link ${secondary_redis_name}"
echo "debug> using command: ${configure_command}"
echo "debug> primary_redis_name: ${primary_redis_name}"
echo "debug> secondary_redis_name: ${secondary_redis_name}"
echo "debug> primary_redis resource group: ${resource_group_name_primary}"
echo "debug> secondary_redis resource group: ${resource_group_name_secondary}"
fi

• debug execution output:

----
debug> creating replication link: rds-lol-dev-az1-000013 => rds-lol-dev-az2-00001
debug> using command: az redis server-link create --debug --name rds-lol-dev-az1-000013 --replication-role Primary --resource-group rsg-lol-dev-az1-00001 --server-to-link rds-lol-dev-az2-00001
debug> primary_redis_name: rds-lol-dev-az1-000013
debug> secondary_redis_name: rds-lol-dev-az2-00001
debug> primary_redis resource group: rsg-lol-dev-az1-00001
debug> secondary_redis resource group: rsg-lol-dev-az2-00001
debug> configuring geo-replication with: az redis server-link create --debug --name rds-lol-dev-az1-000013 --replication-role Primary --resource-group rsg-lol-dev-az1-00001 --server-to-link rds-lol-dev-az2-00001

DEBUG: Command arguments: ['redis', 'server-link', 'create', '--debug', '--name', 'rds-lol-dev-az1-000013', '--replication-role', 'Primary', '--resource-group', 'rsg-lol-dev-az1-00001', '--server-to-link', 'rds-lol-dev-az2-00001']
DEBUG: Event: Cli.PreExecute []
DEBUG: Event: CommandParser.OnGlobalArgumentsCreate [<function on_global_arguments at 0x7fc6e887b410>, <function on_global_arguments at 0x7fc6e7f4e8c0>, <function on_global_arguments at 0x7fc6e7f63f50>]
DEBUG: Command arguments: ['redis', 'server-link', 'create', '--debug', '--name', 'rds-lol-dev-az1-00001', '--replication-role', 'Primary', '--resource-group', 'rsg-lol-dev-az1-00001', '--server-to-link', 'rds-lol-dev-az1-000012']
DEBUG: Event: Cli.PreExecute []
DEBUG: msrest.http_logger : {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Cache/Redis/rds-lol-dev-az2-00001' under resource group 'rsg-lol-dev-az1-00001' was not found."}}
DEBUG: msrest.exceptions : The Resource 'Microsoft.Cache/Redis/rds-lol-dev-az2-00001' under resource group 'rsg-lol-dev-az1-00001' was not found.
ERROR: The Resource 'Microsoft.Cache/Redis/rds-lol-dev-az2-00001' under resource group 'rsg-lol-dev-az1-00001' was not found.
----

• Note the error:

ERROR: The Resource 'Microsoft.Cache/Redis/rds-lol-dev-az2-00001' under resource group 'rsg-lol-dev-az1-00001' was not found.

• But this doesn't make sense because why is it looking for secondary redis instance "rds-lol-dev-az2-00001" under primary redi instance's resource group 'rsg-lol-dev-az1-00001'?
• Unless the command is wrongly inferring the resource group for the second redis instance.
• And there is no way to specify to the command the specific resource group of each/both redis caches

2. Specify the secondary redis cache' resource group (rsg-lol-dev-az2-00001)

• Snippet of bash code that builds the command:

if [ ${configuration_operation} == "link" ]
then
echo "debug> creating replication link: ${primary_redis_name} => ${secondary_redis_name}"
configure_command="az redis server-link create --debug --name ${primary_redis_name} --replication-role Primary --resource-group ${resource_group_name_secondary} --server-to-link ${secondary_redis_name}"
echo "debug> using command: ${configure_command}"
echo "debug> primary_redis_name: ${primary_redis_name}"
echo "debug> secondary_redis_name: ${secondary_redis_name}"
echo "debug> primary_redis resource group: ${resource_group_name_primary}"
echo "debug> secondary_redis resource group: ${resource_group_name_secondary}"
fi

• debug execution output:

----
debug> creating replication link: rds-lol-dev-az1-000013 => rds-lol-dev-az2-00001
debug> using command: az redis server-link create --debug --name rds-lol-dev-az1-000013 --replication-role Primary --resource-group rsg-lol-dev-az2-00001 --server-to-link rds-lol-dev-az2-00001
debug> primary_redis_name: rds-lol-dev-az1-000013
debug> secondary_redis_name: rds-lol-dev-az2-00001
debug> primary_redis resource group: rsg-lol-dev-az1-00001
debug> secondary_redis resource group: rsg-lol-dev-az2-00001
debug> configuring geo-replication with: az redis server-link create --debug --name rds-lol-dev-az1-000013 --replication-role Primary --resource-group rsg-lol-dev-az2-00001 --server-to-link rds-lol-dev-az2-00001
DEBUG: Command arguments: ['redis', 'server-link', 'create', '--debug', '--name', 'rds-lol-dev-az1-000013', '--replication-role', 'Primary', '--resource-group', 'rsg-lol-dev-az2-00001', '--server-to-link', 'rds-lol-dev-az2-00001']
DEBUG: Event: Cli.PreExecute []
DEBUG: Event: CommandParser.OnGlobalArgumentsCreate [<function on_global_arguments at 0x7f1c6b8ea410>, <function on_global_arguments at 0x7f1c6afbd8c0>, <function on_global_arguments at 0x7f1c6afd2f50>]
DEBUG: Command arguments: ['redis', 'server-link', 'create', '--debug', '--name', 'rds-lol-dev-az1-00001', '--replication-role', 'Primary', '--resource-group', 'rsg-lol-dev-az1-00001', '--server-to-link', 'rds-lol-dev-az1-000012']
DEBUG: Event: Cli.PreExecute []
ERROR: The Resource 'Microsoft.Cache/Redis/rds-lol-dev-az1-000013' under resource group 'rsg-lol-dev-az2-00001' was not found.
----

• Again, here the az cli is looking for the primary redis instance under the specified secondary resource group ... which means it's wrongly inferring where the instance should be.

3. Try specifying both resource groups:

• Using command:

az redis server-link create --name rds-lol-dev-az1-000013 --replication-role Primary --resource-group rsg-lol-dev-az1-00001 --server-to-link rds-lol-dev-az2-00001 --replication-role Secondary --resource-group rsg-lol-dev-az2-00001

Same error:

 ----<debug>-----
DEBUG: Command arguments: ['redis', 'server-link', 'create', '--debug', '--name', 'rds-lol-dev-az1-00001', '--replication-role', 'Primary', '--resource-group', 'rsg-lol-dev-az1-00001', '--server-to-link', 'rds-lol-dev-az1-000012']
DEBUG: Event: Cli.PreExecute []
DEBUG: Event: CommandParser.OnGlobalArgumentsCreate [<function on_global_arguments at 0x7f2c3da0d410>, <function on_global_arguments at 0x7f2c3d0e08c0>, <function on_global_arguments at 0x7f2c3d0f5f50>]
DEBUG: Event: CommandInvoker.OnPreCommandTableCreate []
DEBUG: Installed command modules ['acr', 'acs', 'advisor', 'ams', 'appservice', 'backup', 'batch', 'batchai', 'billing', 'botservice', 'cdn', 'cloud', 'cognitiveservices', 'configure', 'consumption', 'container', 'cosmosdb', 'dla', 'dls', 'dms', 'eventgrid', 'eventhubs', 'extension', 'feedback', 'find', 'hdinsight', 'interactive', 'iot', 'iotcentral', 'keyvault', 'kusto', 'lab', 'maps', 'monitor', 'network', 'policyinsights', 'profile', 'rdbms', 'redis', 'relay', 'reservations', 'resource', 'role', 'sea:
rch', 'security', 'servicebus', 'servicefabric', 'signalr', 'sql', 'sqlvm', 'storage', 'vm']
DEBUG: Loaded module 'acr' in 0.004 seconds.
DEBUG: Loaded module 'acs' in 0.068 seconds.
DEBUG: Loaded module 'advisor' in 0.001 seconds.
DEBUG: Event: CommandLoader.OnLoadCommandTable []
DEBUG: Loaded module 'ams' in 0.007 seconds.
DEBUG: Loaded module 'appservice' in 0.008 seconds.
DEBUG: Loaded module 'backup' in 0.005 seconds.
DEBUG: Event: CommandLoader.OnLoadCommandTable []
DEBUG: Loaded module 'batch' in 0.010 seconds.
DEBUG: Loaded module 'batchai' in 0.003 seconds.
DEBUG: Loaded module 'billing' in 0.003 seconds.
DEBUG: Loaded module 'botservice' in 0.003 seconds.
DEBUG: Event: CommandLoader.OnLoadCommandTable []
DEBUG: Loaded module 'cdn' in 0.003 seconds.
DEBUG: Loaded module 'cloud' in 0.001 seconds.
DEBUG: Loaded module 'cognitiveservices' in 0.001 seconds.
DEBUG: Loaded module 'configure' in 0.001 seconds.
DEBUG: Loaded module 'consumption' in 0.002 seconds.
DEBUG: Loaded module 'container' in 0.002 seconds.
DEBUG: Loaded module 'cosmosdb' in 0.004 seconds.
DEBUG: Loaded module 'dla' in 0.005 seconds.
DEBUG: Loaded module 'dls' in 0.003 seconds.
DEBUG: Loaded module 'dms' in 0.002 seconds.
DEBUG: Loaded module 'eventgrid' in 0.067 seconds.
DEBUG: Loaded module 'eventhubs' in 0.004 seconds.
DEBUG: Loaded module 'extension' in 0.001 seconds.
DEBUG: Loaded module 'feedback' in 0.001 seconds.
DEBUG: Loaded module 'find' in 0.001 seconds.
DEBUG: Loaded module 'hdinsight' in 0.002 seconds.
DEBUG: Loaded module 'interactive' in 0.000 seconds.
DEBUG: Loaded module 'iot' in 0.006 seconds.
DEBUG: Loaded module 'iotcentral' in 0.001 seconds.
DEBUG: Loaded module 'keyvault' in 0.088 seconds.
DEBUG: Loaded module 'kusto' in 0.002 seconds.
DEBUG: Loaded module 'lab' in 0.003 seconds.
DEBUG: Loaded module 'maps' in 0.001 seconds.
DEBUG: Loaded module 'monitor' in 0.005 seconds.
DEBUG: Loaded module 'network' in 0.193 seconds.
DEBUG: Loaded module 'policyinsights' in 0.001 seconds.
DEBUG: Loaded module 'profile' in 0.001 seconds.
DEBUG: Loaded module 'rdbms' in 0.007 seconds.
DEBUG: Loaded module 'redis' in 0.002 seconds.
DEBUG: Loaded module 'relay' in 0.079 seconds.
DEBUG: Loaded module 'reservations' in 0.002 seconds.
DEBUG: Loaded module 'resource' in 0.008 seconds.
DEBUG: Loaded module 'role' in 0.005 seconds.
DEBUG: Loaded module 'search' in 0.001 seconds.
DEBUG: Loaded module 'security' in 0.003 seconds.
DEBUG: Loaded module 'servicebus' in 0.006 seconds.
DEBUG: Loaded module 'servicefabric' in 0.002 seconds.
DEBUG: Loaded module 'signalr' in 0.001 seconds.
DEBUG: Loaded module 'sql' in 0.074 seconds.
DEBUG: Loaded module 'sqlvm' in 0.002 seconds.
DEBUG: Event: CommandLoader.OnLoadCommandTable []
DEBUG: Loaded module 'storage' in 0.102 seconds.
DEBUG: Loaded module 'vm' in 0.017 seconds.
DEBUG: Loaded all modules in 0.823 seconds. (note: there's always an overhead with the first module loaded)
DEBUG: Extensions directory: '/home/jenkins/.azure/cliextensions'
DEBUG: Event: CommandInvoker.OnPreCommandTableTruncate []
DEBUG: Event: CommandInvoker.OnPostCommandTableCreate [<function add_subscription_parameter at 0x7f2c3ce47578>, <function add_ids_arguments at 0x7f2c3b4292a8>]
DEBUG: Event: CommandInvoker.OnCommandTableLoaded []
DEBUG: Event: CommandInvoker.OnPreParseArgs [<function _documentdb_deprecate at 0x7f2c3ab06cf8>]
DEBUG: Event: CommandInvoker.OnPostParseArgs [<function handle_output_argument at 0x7f2c3d0e0938>, <function handle_query_parameter at 0x7f2c3d0f9050>, <function parse_ids_arguments at 0x7f2c3b429488>, <function handler at 0x7f2c3aa000c8>]
DEBUG: Getting management service client client_type=RedisManagementClient
DEBUG: msrest.universal_http.requests : Configuring retry: max_retries=4, backoff_factor=0.8, max_backoff=90
DEBUG: Getting management service client client_type=RedisManagementClient
DEBUG: msrest.universal_http.requests : Configuring retry: max_retries=4, backoff_factor=0.8, max_backoff=90
DEBUG: attempting to read file /home/jenkins/.azure/accessTokens.json as utf-8-sig
DEBUG: adal-python : 758204c2-ecc0-4a1a-ae8e-c371bcff642f - Authority:Performing instance discovery: ...
DEBUG: adal-python : 758204c2-ecc0-4a1a-ae8e-c371bcff642f - Authority:Performing static instance discovery
DEBUG: adal-python : 758204c2-ecc0-4a1a-ae8e-c371bcff642f - Authority:Authority validated via static instance discovery
INFO: adal-python : 758204c2-ecc0-4a1a-ae8e-c371bcff642f - TokenRequest:Getting token with client credentials.
DEBUG: adal-python : 758204c2-ecc0-4a1a-ae8e-c371bcff642f - TokenRequest:No user_id passed for cache query
DEBUG: adal-python : 758204c2-ecc0-4a1a-ae8e-c371bcff642f - CacheDriver:finding with query keys: {'_clientId': '...'}
DEBUG: adal-python : 758204c2-ecc0-4a1a-ae8e-c371bcff642f - CacheDriver:Looking for potential cache entries: {'_clientId': '...'}
DEBUG: adal-python : 758204c2-ecc0-4a1a-ae8e-c371bcff642f - CacheDriver:Found 0 potential entries.
DEBUG: urllib3.connectionpool : Starting new HTTPS connection (1): login.microsoftonline.com:443
DEBUG: urllib3.connectionpool : https://login.microsoftonline.com:443 "POST /fe72a01e-5c4a-48a4-9b75-7a7c7836ae30/oauth2/token HTTP/1.1" 200 1421
INFO: adal-python : 758204c2-ecc0-4a1a-ae8e-c371bcff642f - OAuth2Client:Get Token Server returned this correlation_id: 758204c2-ecc0-4a1a-ae8e-c371bcff642f
DEBUG: adal-python : 758204c2-ecc0-4a1a-ae8e-c371bcff642f - CacheDriver:Adding entry AccessTokenId: 2BPhSTSJJz3cPsSno5UHgdYsdio2LM7uqAE86eyoakc=
DEBUG: adal-python : 758204c2-ecc0-4a1a-ae8e-c371bcff642f - CacheDriver:Added entry is MRRT
DEBUG: msrest.http_logger : Request URL: 'https://management.azure.com/subscriptions/redacted/resourceGroups/rsg-lol-dev-az1-00001/providers/Microsoft.Cache/Redis/rds-lol-dev-az1-000012?api-version=2018-03-01'
DEBUG: msrest.http_logger : Request method: 'GET'
DEBUG: msrest.http_logger : Request headers:
DEBUG: msrest.http_logger :     'accept-language': 'en-US'
DEBUG: msrest.http_logger :     'Accept': 'application/json'
DEBUG: msrest.http_logger :     'User-Agent': 'python/2.7.5 (Linux-3.10.0-957.el7.x86_64-x86_64-with-redhat-7.6-Maipo) msrest/0.6.4 msrest_azure/0.6.0 azure-mgmt-redis/6.0.0 Azure-SDK-For-Python AZURECLI/2.0.60'
DEBUG: msrest.http_logger : Request body:
DEBUG: msrest.http_logger : None
DEBUG: msrest.universal_http : Configuring redirects: allow=True, max=30
DEBUG: msrest.universal_http : Configuring request: timeout=100, verify=True, cert=None
DEBUG: msrest.universal_http : Configuring proxies: ''
DEBUG: msrest.universal_http : Evaluate proxies against ENV settings: True
DEBUG: urllib3.connectionpool : Starting new HTTPS connection (1): management.azure.com:443
DEBUG: urllib3.connectionpool : https://management.azure.com:443 "GET /subscriptions/redacted/resourceGroups/rsg-lol-dev-az1-00001/providers/Microsoft.Cache/Redis/rds-lol-dev-az1-000012?api-version=2018-03-01 HTTP/1.1" 200 None
DEBUG: msrest.http_logger : Response status: 200
DEBUG: msrest.http_logger : Response headers:
DEBUG: msrest.http_logger :     'Cache-Control': 'no-cache'
DEBUG: msrest.http_logger :     'Pragma': 'no-cache'
DEBUG: msrest.http_logger :     'Transfer-Encoding': 'chunked'
DEBUG: msrest.http_logger :     'Content-Type': 'application/json; charset=utf-8'
DEBUG: msrest.http_logger :     'Content-Encoding': 'gzip'
DEBUG: msrest.http_logger :     'Expires': '-1'
DEBUG: msrest.http_logger :     'Vary': 'Accept-Encoding'
DEBUG: msrest.http_logger :     'x-ms-request-id': '51e0727e-4b11-4be4-b653-c5dd1b8abdad'
DEBUG: msrest.http_logger :     'x-rp-server-mvid': '36d7eb14-b41f-413d-b2ac-e880d98dedd5'
DEBUG: msrest.http_logger :     'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'
DEBUG: msrest.http_logger :     'X-Content-Type-Options': 'nosniff'
DEBUG: msrest.http_logger :     'Server': 'Microsoft-HTTPAPI/2.0'
DEBUG: msrest.http_logger :     'x-ms-ratelimit-remaining-subscription-reads': '11990'
DEBUG: msrest.http_logger :     'x-ms-correlation-request-id': '4c7f58b5-58fe-489e-95e3-9a9388a4abe5'
DEBUG: msrest.http_logger :     'x-ms-routing-request-id': 'CENTRALINDIA:20190507T041117Z:4c7f58b5-58fe-489e-95e3-9a9388a4abe5'
DEBUG: msrest.http_logger :     'Date': 'Tue, 07 May 2019 04:11:17 GMT'
DEBUG: msrest.http_logger : Response content:
DEBUG: msrest.http_logger : {"id":"/subscriptions/redacted/resourceGroups/rsg-lol-dev-az1-00001/providers/Microsoft.Cache/Redis/rds-lol-dev-az1-000012","location":"Southeast Asia","name":"rds-lol-dev-az1-000012","type":"Microsoft.Cache/Redis","tags":{"ApplicationName":"application demo 2","ApplicationOwner":"[email protected]","BusinessUnit":"${var.buAdCode}","Environment":"${var.environment}","Location":"azure southeast asia"},"properties":{"provisioningState":"Succeeded","redisVersion":"3.2.7","sku":{"name":"Premium","family":"P","capacity":1},"enableNonSslPort":false,"redisConfiguration":{"maxmemory-policy":"volatile-lru","maxmemory-reserved":"200","maxclients":"7500","maxfragmentationmemory-reserved":"300","maxmemory-delta":"200"},"accessKeys":null,"hostName":"rds-lol-dev-az1-000012.redis.cache.windows.net","port":6379,"sslPort":6380,"shardCount":3,"linkedServers":[{"id":"/subscriptions/redacted/resourceGroups/rsg-lol-dev-az1-00001/providers/Microsoft.Cache/Redis/rds-lol-dev-az1-000012/linkedServers/rds-lol-dev-az1-00001"}]}}
DEBUG: adal-python : bdad5ac5-e83b-4926-b5d8-79cca3bb165a - Authority:Performing instance discovery: ...
DEBUG: adal-python : bdad5ac5-e83b-4926-b5d8-79cca3bb165a - Authority:Performing static instance discovery
DEBUG: adal-python : bdad5ac5-e83b-4926-b5d8-79cca3bb165a - Authority:Authority validated via static instance discovery
INFO: adal-python : bdad5ac5-e83b-4926-b5d8-79cca3bb165a - TokenRequest:Getting token with client credentials.
DEBUG: adal-python : bdad5ac5-e83b-4926-b5d8-79cca3bb165a - TokenRequest:No user_id passed for cache query
DEBUG: adal-python : bdad5ac5-e83b-4926-b5d8-79cca3bb165a - CacheDriver:finding with query keys: {'_clientId': '...'}
DEBUG: adal-python : bdad5ac5-e83b-4926-b5d8-79cca3bb165a - CacheDriver:Looking for potential cache entries: {'_clientId': '...'}
DEBUG: adal-python : bdad5ac5-e83b-4926-b5d8-79cca3bb165a - CacheDriver:Found 0 potential entries.
DEBUG: urllib3.connectionpool : Starting new HTTPS connection (1): login.microsoftonline.com:443
DEBUG: urllib3.connectionpool : https://login.microsoftonline.com:443 "POST /fe72a01e-5c4a-48a4-9b75-7a7c7836ae30/oauth2/token HTTP/1.1" 200 1421
INFO: adal-python : bdad5ac5-e83b-4926-b5d8-79cca3bb165a - OAuth2Client:Get Token Server returned this correlation_id: bdad5ac5-e83b-4926-b5d8-79cca3bb165a
DEBUG: adal-python : bdad5ac5-e83b-4926-b5d8-79cca3bb165a - CacheDriver:Adding entry AccessTokenId: CKiCUuT+atwI6WO6NJLQY0WPeJhFkoGXC5JtrD3pmWY=
DEBUG: adal-python : bdad5ac5-e83b-4926-b5d8-79cca3bb165a - CacheDriver:Added entry is MRRT
DEBUG: msrest.http_logger : Request URL: 'https://management.azure.com/subscriptions/redacted/resourceGroups/rsg-lol-dev-az1-00001/providers/Microsoft.Cache/Redis/rds-lol-dev-az1-00001/linkedServers/rds-lol-dev-az1-000012?api-version=2018-03-01'
DEBUG: msrest.http_logger : Request method: 'PUT'
DEBUG: msrest.http_logger : Request headers:
DEBUG: msrest.http_logger :     'accept-language': 'en-US'
DEBUG: msrest.http_logger :     'Content-Type': 'application/json; charset=utf-8'
DEBUG: msrest.http_logger :     'Accept': 'application/json'
DEBUG: msrest.http_logger :     'Content-Length': '264'
DEBUG: msrest.http_logger :     'User-Agent': 'python/2.7.5 (Linux-3.10.0-957.el7.x86_64-x86_64-with-redhat-7.6-Maipo) msrest/0.6.4 msrest_azure/0.6.0 azure-mgmt-redis/6.0.0 Azure-SDK-For-Python AZURECLI/2.0.60'
DEBUG: msrest.http_logger : Request body:
DEBUG: msrest.http_logger : {"properties": {"serverRole": "Primary", "linkedRedisCacheLocation": "Southeast Asia", "linkedRedisCacheId": "/subscriptions/redacted/resourceGroups/rsg-lol-dev-az1-00001/providers/Microsoft.Cache/Redis/rds-lol-dev-az1-000012"}}
DEBUG: msrest.universal_http : Configuring redirects: allow=True, max=30
DEBUG: msrest.universal_http : Configuring request: timeout=100, verify=True, cert=None
DEBUG: msrest.universal_http : Configuring proxies: ''
DEBUG: msrest.universal_http : Evaluate proxies against ENV settings: True

• switching resource groups around:

az redis server-link create --name rds-lol-dev-az1-000013 --replication-role Primary --resource-group rsg-lol-dev-az2-00001 --server-to-link rds-lol-dev-az2-00001 --replication-role Secondary --resource-group rsg-lol-dev-az1-00001

• Debug output:

module.rediscache_geo_replication_az1_az2b.null_resource.geo_replication (local-exec): ----<debug>-----
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): ========================================== BEGIN Configure Redis Cache Geo Replication ==================================
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): creating replication link: rds-lol-dev-az1-00001 => rds-lol-dev-az1-000012
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): configuring geo-replication with: az redis server-link create --debug --name rds-lol-dev-az1-00001 --replication-role Primary --resource-group rsg-lol-dev-az1-00001 --s
erver-to-link rds-lol-dev-az1-000012
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Command arguments: ['redis', 'server-link', 'create', '--debug', '--name', 'rds-lol-dev-az1-00001', '--replication-role', 'Primary', '--resource-group', 'rsg-lol
-dev-az1-00001', '--server-to-link', 'rds-lol-dev-az1-000012']
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Event: Cli.PreExecute []
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Event: CommandParser.OnGlobalArgumentsCreate [<function on_global_arguments at 0x7f2cb35cf410>, <function on_global_arguments at 0x7f2cb2ca28c0>, <function on_gl
obal_arguments at 0x7f2cb2cb7f50>]
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Event: CommandInvoker.OnPreCommandTableCreate []
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Installed command modules ['acr', 'acs', 'advisor', 'ams', 'appservice', 'backup', 'batch', 'batchai', 'billing', 'botservice', 'cdn', 'cloud', 'cognitiveservice
s', 'configure', 'consumption', 'container', 'cosmosdb', 'dla', 'dls', 'dms', 'eventgrid', 'eventhubs', 'extension', 'feedback', 'find', 'hdinsight', 'interactive', 'iot', 'iotcentral', 'keyvault', 'kusto', 'lab', 'maps', 'monitor', 'network', 'policyins
ights', 'profile', 'rdbms', 'redis', 'relay', 'reservations', 'resource', 'role', 'search', 'security', 'servicebus', 'servicefabric', 'signalr', 'sql', 'sqlvm', 'storage', 'vm']
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'acr' in 0.004 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'acs' in 0.003 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'advisor' in 0.001 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Event: CommandLoader.OnLoadCommandTable []
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'ams' in 0.007 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'appservice' in 0.078 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'backup' in 0.005 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Event: CommandLoader.OnLoadCommandTable []
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'batch' in 0.010 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'batchai' in 0.003 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'billing' in 0.002 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'botservice' in 0.003 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Event: CommandLoader.OnLoadCommandTable []
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'cdn' in 0.002 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'cloud' in 0.001 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'cognitiveservices' in 0.001 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'configure' in 0.001 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'consumption' in 0.002 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'container' in 0.068 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'cosmosdb' in 0.004 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'dla' in 0.004 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'dls' in 0.003 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'dms' in 0.002 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'eventgrid' in 0.002 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'eventhubs' in 0.004 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'extension' in 0.001 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'feedback' in 0.001 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'find' in 0.001 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'hdinsight' in 0.002 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'interactive' in 0.000 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'iot' in 0.077 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'iotcentral' in 0.001 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'keyvault' in 0.006 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'kusto' in 0.002 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'lab' in 0.003 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'maps' in 0.001 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'monitor' in 0.005 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'iot' in 0.077 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'iotcentral' in 0.001 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'keyvault' in 0.006 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'kusto' in 0.002 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'lab' in 0.003 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'maps' in 0.001 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'monitor' in 0.005 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'network' in 0.182 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'policyinsights' in 0.001 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'profile' in 0.001 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'rdbms' in 0.007 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'redis' in 0.002 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'relay' in 0.003 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'reservations' in 0.002 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'resource' in 0.008 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'role' in 0.005 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'search' in 0.066 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'security' in 0.003 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'servicebus' in 0.005 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'servicefabric' in 0.002 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'signalr' in 0.001 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'sql' in 0.007 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'sqlvm' in 0.002 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Event: CommandLoader.OnLoadCommandTable []
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'storage' in 0.111 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded module 'vm' in 0.079 seconds.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Loaded all modules in 0.798 seconds. (note: there's always an overhead with the first module loaded)
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Extensions directory: '/home/jenkins/.azure/cliextensions'
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Event: CommandInvoker.OnPreCommandTableTruncate []
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Event: CommandInvoker.OnPostCommandTableCreate [<function add_subscription_parameter at 0x7f2cb2a09578>, <function add_ids_arguments at 0x7f2cb0feb2a8>]
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Event: CommandInvoker.OnCommandTableLoaded []
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Event: CommandInvoker.OnPreParseArgs [<function _documentdb_deprecate at 0x7f2cb06c8cf8>]
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Event: CommandInvoker.OnPostParseArgs [<function handle_output_argument at 0x7f2cb2ca2938>, <function handle_query_parameter at 0x7f2cb2cbb050>, <function parse_ids_arguments at 0x7f2cb0feb488>, <function handler at 0x7f2cb05c20c8>]
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Getting management service client client_type=RedisManagementClient
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: msrest.universal_http.requests : Configuring retry: max_retries=4, backoff_factor=0.8, max_backoff=90
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: Getting management service client client_type=RedisManagementClient
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: msrest.universal_http.requests : Configuring retry: max_retries=4, backoff_factor=0.8, max_backoff=90
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: attempting to read file /home/jenkins/.azure/accessTokens.json as utf-8-sig
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: adal-python : a00a7d5e-29c0-4f32-9367-a67439f84d7d - Authority:Performing instance discovery: ...
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: adal-python : a00a7d5e-29c0-4f32-9367-a67439f84d7d - Authority:Performing static instance discovery
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: adal-python : a00a7d5e-29c0-4f32-9367-a67439f84d7d - Authority:Authority validated via static instance discovery
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): INFO: adal-python : a00a7d5e-29c0-4f32-9367-a67439f84d7d - TokenRequest:Getting token with client credentials.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: adal-python : a00a7d5e-29c0-4f32-9367-a67439f84d7d - TokenRequest:No user_id passed for cache query
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: adal-python : a00a7d5e-29c0-4f32-9367-a67439f84d7d - CacheDriver:finding with query keys: {'_clientId': '...'}
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: adal-python : a00a7d5e-29c0-4f32-9367-a67439f84d7d - CacheDriver:Looking for potential cache entries: {'_clientId': '...'}
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: adal-python : a00a7d5e-29c0-4f32-9367-a67439f84d7d - CacheDriver:Found 0 potential entries.
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: urllib3.connectionpool : Starting new HTTPS connection (1): login.microsoftonline.com:443
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: urllib3.connectionpool : https://login.microsoftonline.com:443 "POST /fe72a01e-5c4a-48a4-9b75-7a7c7836ae30/oauth2/token HTTP/1.1" 200 1421
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): INFO: adal-python : a00a7d5e-29c0-4f32-9367-a67439f84d7d - OAuth2Client:Get Token Server returned this correlation_id: a00a7d5e-29c0-4f32-9367-a67439f84d7d
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: adal-python : a00a7d5e-29c0-4f32-9367-a67439f84d7d - CacheDriver:Adding entry AccessTokenId: cAjFZvRFhXx4eC/MXVq6L2IUgJHbsZL6YhlgbtF8rVU=
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: adal-python : a00a7d5e-29c0-4f32-9367-a67439f84d7d - CacheDriver:Added entry is MRRT
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: msrest.http_logger : Request URL: 'https://management.azure.com/subscriptions/redacted/resourceGroups/rsg-lol-dev-az1-00001/providers/Microsoft.Cache/Redis/rds-lol-dev-az1-000012?api-version=2018-03-01'
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: msrest.http_logger : Request method: 'GET'
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: msrest.http_logger : Request headers:
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: msrest.http_logger :     'accept-language': 'en-US'
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: msrest.http_logger :     'Accept': 'application/json'
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: msrest.http_logger :     'User-Agent': 'python/2.7.5 (Linux-3.10.0-957.el7.x86_64-x86_64-with-redhat-7.6-Maipo) msrest/0.6.4 msrest_azure/0.6.0 azure-mgmt-redis/6.0.0 Azure-SDK-For-Python AZURECLI/2.0.60'
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: msrest.http_logger : Request body:
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: msrest.http_logger : None
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: msrest.universal_http : Configuring redirects: allow=True, max=30
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: msrest.universal_http : Configuring request: timeout=100, verify=True, cert=None
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: msrest.universal_http : Configuring proxies: ''
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: msrest.universal_http : Evaluate proxies against ENV settings: True
module.rediscache_geo_replication_az1_az2.null_resource.geo_replication (local-exec): DEBUG: urllib3.connectionpool : Starting new HTTPS connection (1): management.azure.com:443
module.rediscache_geo_replication_az1_az2b.null_resource.geo_replication (local-exec): ERROR: The Resource 'Microsoft.Cache/Redis/rds-lol-dev-az2-00001' under resource group 'rsg-lol-dev-az1-00001' was not found.
module.rediscache_geo_replication_az1_az2b.null_resource.geo_replication (local-exec): ----</debug>-----

[archmangler]

Note: Geo-replication using Powershell works smoothly, implementation also appears to be simpler than azure cli, unexpectedly .

Executing: ["/bin/sh" "-c" "/home/jenkins/workspace/module-tests/blueprint/.terraform/modules/83952271c748ac3ccc466b63d1a70b63/scripts/configure-georeplication.ps1 rds-lol-dev-az1-000013 rds-lol-dev-az2-00001 link"]
================================<BEGIN Geo-Replication Configuration>==============================================
Linking primary rds-lol-dev-az1-000013 to secondary rds-lol-dev-az2-00001


PrimaryServerName   : rds-lol-dev-az1-000013
SecondaryServerName : rds-lol-dev-az2-00001
ProvisioningState   : Succeeded

================================<END Geo-Replication Configuration>================================================
Getting status of redis geo-replication link rds-lol-dev-az1-000013 <==> rds-lol-dev-az2-00001
PrimaryServerName   : rds-lol-dev-az1-000013
SecondaryServerName : rds-lol-dev-az2-00001
ProvisioningState   : Succeeded

[alfantp]

Hello @archmangler , when resource ID is mentioned in azure, it generally refers to the ARM resource id, which you posted in the beginning of your post.
When the cache you are trying to link is in a different Resource Group, you can use the resource id as the parameter value for 'server-to-link' as mentioned in the docs. In the variations you mentioned above, i only see resource name being tried and not ID.
I just tried this and it worked:
az redis server-link create --name < primarycachename > --resource-group < primaryRG > --server-to-link /subscriptions/< subscription >/resourceGroups/< secondaryRG >/providers/Microsoft.Cache/Redis/< secondarycachename > --replication-role Secondary

Design principles for CLI mentions using a single parameter to accept either ARM Resource ID or resource name for a supporting resource in an operation:
https://github.com/Azure/azure-cli/blob/dev/doc/authoring_command_modules/authoring_commands.md#supporting-name-or-id-parameters
This is so that when the supporting resource is in the same resource group, you can just type in resource name and if not, you can enter the resource ID.

[tjprescott]

Thanks for looking into the @alfantp. Since you have verified the command is working as intended and that the functionality is supported, I will close this issue.

@archmangler supplying --resource-group twice simply overwrites the first value with the second. As @alfantp mentioned, you need to supply the ARM resource ID for the secondary server.