This repository has been archived by the owner on Sep 13, 2023. It is now read-only.
forked from pivotal-cf/docs-a9s-rabbitmq
-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathusing.html.md.erb
565 lines (429 loc) · 20.4 KB
/
using.html.md.erb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
---
title: Using a9s RabbitMQ for Pivotal Platform
owner: Partners
---
This topic describes how developers use a9s RabbitMQ.
## <a id='use-a9s-rabbimq-with-an-app'></a> Use a9s RabbitMQ with an App
To use a9s RabbitMQ with an app, create a service instance and bind the service
instance to your app. For more information about managing service instances, see
[Managing Service Instances with the cf CLI](https://docs.cloudfoundry.org/devguide/services/managing-services.html).
### <a id='view-the-a9s-rabqitmw-service'></a> View the a9s RabbitMQ Service
After the tile is installed, you can see the `a9s-rabbitmq` and its service plans
appear in your Pivotal Platform marketplace. Run `cf marketplace` to see the service listing:
```bash
$ cf marketplace
Getting services from marketplace in org test / space test as admin...
OK
service plans description
a9s-rabbitmq36 rabbitmq-single-small, rabbitmq-cluster-small, This is the anynines rabbitmq 36 service.
rabbitmq-single-big, rabbitmq-cluster-big
```
### <a id='create-a-service-instance'></a> Create a Service Instance
To provision a RabbitMQ database, run `cf create-service`. For example:
```bash
$ cf create-service a9s-rabbitmq36 rabbitmq-single-small my-rabbitmq-service
```
Depending on your infrastructure and service broker utilization, it might take
several minutes to create the service instance.
Run `cf services` to view the creation status. This command displays a list of all
your service instances. To view the status of a specific service instance, run
`cf service NAME-OF-YOUR-SERVICE`.
### <a id='bind-an-app-to-a-service-instance'></a> Bind an App to a Service Instance
After your database is created, run `cf bind-service` to bind the service to your
app:
```bash
$ cf bind-service a9s-rabbitmq-app my-rabbitmq-service
```
### <a id='restage-or-restart-your-app'></a> Restage or Restart Your App
To enable your appliation to access the service instance, run `cf restage` or
`cf restart` to restage or restart your app.
### <a id='obtain-service-instance-access-credentials'></a> Obtain Service Instance Access Credentials
After a service instance is bound to an app, the credentials of your RabbitMQ
database are stored in the environment variables of the app. Run `cf env APP-NAME`
to display the environment variables.
You can find the credentials in the [VCAP_SERVICES](https://docs.cloudfoundry.org/devguide/deploy-apps/environment-variable.html#VCAP-SERVICES) key.
```bash
$ cf env a9s-rabbitmq-app
Getting env variables for app a9s-rabbitmq-app in org test / space test as admin...
OK
System-Provided:
{
"VCAP_SERVICES": {
"a9s-rabbitmq36": [
{
"credentials": {
"dns_servers": [
"172.28.10.20",
"172.28.11.30",
"172.28.12.23"
],
"host": "d87e464.service.dc1.a9s-rabbitmq-consul",
"host_ip": "172.28.25.12",
"host_ips": [
"172.28.25.12",
"172.28.26.11",
"172.28.27.12"
],
"hosts": [
"d87e464-mq-0.node.dc1.a9s-rabbitmq-consul",
"d87e464-mq-1.node.dc1.a9s-rabbitmq-consul",
"d87e464-mq-2.node.dc1.a9s-rabbitmq-consul"
],
"password": "a9s-pwd",
"port": 5672,
"uri": "amqps://a9s-brk-usr:[email protected]:5672",
"username": "a9s-brk-usr"
},
"label": "a9s-rabbitmq36",
"name": "2452-23360-25011-17366",
"plan": "rabbitmq-replica-small-ssl",
"provider": null,
"syslog_drain_url": null,
"tags": [
"messaging",
"queue"
],
"volume_mounts": []
}
]
}
}
...
```
You can use the host_ips, username, and password values to connect to your database
with a RabbitMQ client.
a9s RabbitMQ for Pivotal Platform comes with its own Consul cluster to provide hostname resolving.
When your DNS is configured to resolve the a9s RabbitMQ for Pivotal Platform service instance
domains you can make use of `hosts` or `uri`.
<p class="note"><strong>Note:</strong> For more information, see
[Name Resolving With Consul](./name-resolving.html#name-resolving-with-consul).</p>
## <a id='delete-an-a9s-rabbitmq-service-instance'></a> Delete an a9s RabbitMQ Service Instance
<p class='note warning'><strong>WARNING:</strong> Before deleting a service instance,
you must back up data stored in your database. This operation cannot be undone and
all the data is lost when the service is deleted.</p>
Before you can delete a service instance, you must unbind it from all apps.
### <a id='list-available-services'></a> List Available Services
Run `cf services` to list your available services.
```bash
$ cf services
Getting services in org test / space test as admin...
OK
name service plan bound apps last operation
my-rabbitmq-service a9s-rabbitmq36 rabbitmq-single-small a9s-rabbitmq-app create succeeded
```
This example shows that `my-rabbitmq-service` is bound to the `a9s-rabbitmq-app` app.
### <a id='unbind-a-service-instance'></a> Unbind a Service Instance
Run `cf unbind-service` to unbind the service instance from your app.
```bash
$ cf unbind-service a9s-rabbitmq-app my-rabbitmq-service
```
### <a id='delete-a-service-instance'></a> Delete a Service Instance
After unbinding the service, it is no longer bound to an app. Run `cf delete-service`
to delete the service:
```bash
$ cf delete-service my-rabbitmq-service
```
It may take several minutes to delete the service. Deleting a service de-provisions
the corresponding infrastructure resources. Run the `cf services` command to view
the deletion status.
## <a id='upgrade-the-service-instance-to-another-service-plan'></a> Upgrade the Service Instance to Another Service Plan
Once created, you can upgrade your service instance to another, larger service plan.
A larger service plan provides more CPU, RAM and storage. For more information, see
the [Update a Service Instance](https://docs.cloudfoundry.org/devguide/services/managing-services.html#update_service)
of the _Managing Service Instances with the cf CLI_ topic.
```bash
$ cf update-service my-rabbitmq-service -p a-bigger-plan
```
Depending on the one you are currently using, the plans you can upgrade to are
as follows:
| Base Plan | Target Plan |
|------|-----------|
| rabbitmq-single-nano | rabbitmq-single-small <br> rabbitmq-single-medium <br> rabbitmq-single-big <br> rabbitmq-replica-small <br> rabbitmq-replica-medium <br> rabbitmq-replica-big |
| rabbitmq-single-small | rabbitmq-single-medium <br> rabbitmq-single-big <br> rabbitmq-replica-small <br> rabbitmq-replica-medium <br> rabbitmq-replica-big |
| rabbitmq-single-medium | rabbitmq-single-big <br> rabbitmq-replica-medium <br> rabbitmq-replica-big |
| rabbitmq-single-big | rabbitmq-replica-big |
| rabbitmq-replica-small | rabbitmq-replica-medium <br> rabbitmq-replica-big |
| rabbitmq-replica-medium | rabbitmq-replica-big |
| rabbitmq-single-nano-ssl | rabbitmq-single-small-ssl <br> rabbitmq-single-medium-ssl <br> rabbitmq-single-big-ssl <br> rabbitmq-replica-small-ssl <br> rabbitmq-replica-medium-ssl <br> rabbitmq-replica-big-ssl |
| rabbitmq-single-small-ssl | rabbitmq-single-medium-ssl <br> rabbitmq-single-big-ssl <br> rabbitmq-replica-small-ssl <br> rabbitmq-replica-medium-ssl <br> rabbitmq-replica-big-ssl |
| rabbitmq-single-medium-ssl | rabbitmq-single-big-ssl <br> rabbitmq-replica-medium-ssl <br> rabbitmq-replica-big-ssl |
| rabbitmq-single-big-ssl | rabbitmq-replica-big-ssl |
| rabbitmq-replica-small-ssl | rabbitmq-replica-medium-ssl <br> rabbitmq-replica-big-ssl |
| rabbitmq-replica-medium-ssl | rabbitmq-replica-big-ssl |
## <a id='add-a-graphite-endpoint'></a> Add a Graphite Endpoint
If you want to monitor your service with Graphite, you can set an endpoint to
where the information is sent with the `cf update-service` command. This
command expects the [-c flag](https://docs.cloudfoundry.org/devguide/services/managing-services.html#arbitrary-params-binding)
and a JSON string containing the `graphite` and `metrics_prefix` keys.
Depending on your Graphite provider, the `metrics_prefix` might require that each
metric must start with an API key in their name. You can also change the interval
within the data is send to the endpoint. To do this, modify `interval`; the default
is 10s.
```bash
$ cf update-service my-rabbitmq-service -c '{ "graphite": ["yourspace.your-graphite-endpoint.com:12345"], "metrics_prefix": "your-api-key.my-cluster-rabbitmq", "interval": "5"}'
```
## <a id='add-a-syslog-endpoint'></a> Add a Syslog Endpoint
The `cf update-service` command used with the [-c flag](https://docs.cloudfoundry.org/devguide/services/managing-services.html#arbitrary-params-binding)
can let you stream your logs to a third-party service. In this case, the command
expects a JSON string containing the `syslog` key. You can also change the interval
for the syslog with the same key than for the Graphite endpoint `interval`.
```bash
$ cf update-service my-rabbitmq-service -c '{ "syslog": ["logs4.your-syslog-endpoint.com:54321"], "interval": "5" }'
```
## <a id='cloud-foundry-app-security-groups'></a> Cloud Foundry App Security Groups
This section describes how to check whether a security group was created.
Each a9s Data Service automatically creates and updates Cloud Foundry security groups
in order to protect service instances to be accessed by apps not running in the
same Cloud Foundry apps space. To understand Security Groups, see
[Application Security Groups](https://docs.cloudfoundry.org/concepts/asg.html).
### <a id='get-service-instance-guid'></a> Get Service Instance GUID
Run `cf service INSTANCE_NAME --guid` to get the GUID of the service instance.
```bash
$ cf service my-rabbitmq --guid
ca16f111-5073-40b7-973a-156c75dd3028
```
### <a id='check-available-security-groups'></a> Check Available Security Groups
To see all available security groups, run `cf security-groups`.
```bash
$cf security-groups
Getting security groups as [email protected]
OK
Name Organization Space
#0 public_networks
#1 dns
#2 tcp_open
#3 guard_432fb752-876d-443b-a311-a075f4df2237 demonstrations demo
#4 guard_ca16f111-5073-40b7-973a-156c75dd3028 demonstrations demo
```
In the above example, you can see a security group with the name
`guard_ca16f111-5073-40b7-973a-156c75dd3028` was successfully created.
<p class='note'><strong>Note:</strong> In some circumstances, the connection
between the app and the service instance is not possible. In this case, check
to see if a security group was created.</p>
## <a id='backup-and-restore-service-instances'></a> Back Up and Restore Service Instances
a9s RabbitMQ provides an easy way to create backups and restore if needed.
For a more detailed description see the [a9s Service Dashboard](a9s-service-dashboard.html)
documentation.
## <a id='use-rabbitmq-plugins'></a> Use RabbitMQ Plugins
a9s RabbitMQ allows you to enable the following RabbitMQ plugins:
* rabbitmq_consistent_hash_exchange
* rabbitmq_federation
* rabbitmq_federation_management
* rabbitmq_mqtt
* rabbitmq_sharding
* rabbitmq_shovel
* rabbitmq_shovel_management
* rabbitmq_stomp
* rabbitmq_tracing
### <a id='installation'></a> Installation
You can install RabbitMQ plugins by running `cf create-service` and `cf update-service`
using additional configuration parameters.
```bash
$ cf create-service a9s-rabbitmq36 rabbitmq-single-small my-rabbitmq-service -c '{ "plugins": ["rabbitmq_shovel", "rabbitmq_shovel_management"] }'
$ cf update-service my-rabbitmq-service -c '{ "plugins": ["rabbitmq_shovel", "rabbitmq_shovel_management"] }'
```
## <a id='create-rabbitmq-users'></a> Create RabbitMQ Users
a9s RabbitMQ allows you to create new users with different roles (permissions) by
creating [service keys](https://docs.cloudfoundry.org/devguide/services/service-keys.html).
The possible roles are:
* management
* policymaker
* monitoring
* administrator
You can read more about RabbitMQ roles and permissions in the [Permissions](https://www.rabbitmq.com/management.html#permissions)
section of the _Management Plugin_ topic of the RabbitMQ documentation.
### <a id='create-a-new-role'></a> Create a New Role
You can create a new user with specific permissions by giving the `role` the
following custom parameter. This parameter receives an array of roles:
```bash
$ cf create-service-key my-rabbitmq-service my-key -c '{"roles": ["administrator", "management"]}'
```
## <a id='make-a-service-instance-locally-available'></a> Make a Service Instance Locally Available
You can access any of the a9s Data Services locally and connect with a local
client to the service for any purpose, such as debugging. Cloud Foundry provides a
smart way to create SSH forward tunnels through a pushed app. For more information
about this feature, see the [Accessing Apps with SSH](https://docs.cloudfoundry.org/devguide/deploy-apps/ssh-apps.html)
section of the Cloud Foundry documentation.
First, you must have an app bound to the service. To do this, see
[Bind an App to a Service Instance](./using.html#bind-an-app-to-a-service-instance).
<p class='note'><strong>Note:</strong> <code>cf ssh</code> support must be enabled in the
platform. Consult your administrator to ensure it is enabled.</p>
### <a id='get-dashboard-url-and-credentials'></a> Get Dashboard URL and Credentials
Use the hostname of the service and the user credentials obtained in
[Obtain Service Instance Access Credentials](./using.html#obtain-service-instance-access-credentials) for this procedure.
```bash
$ cf env a9s-rabbitmq-app
Getting env variables for app a9s-rabbitmq-app in org test / space test as admin...
OK
System-Provided:
{
"VCAP_SERVICES": {
"a9s-rabbitmq36": [
{
"credentials": {
"dns_servers": [
"172.28.10.20",
"172.28.11.30",
"172.28.12.23"
],
"host": "d87e464.service.dc1.a9s-rabbitmq-consul",
"host_ip": "172.28.25.12",
"host_ips": [
"172.28.25.12",
"172.28.26.11",
"172.28.27.12"
],
"hosts": [
"d87e464-mq-0.node.dc1.a9s-rabbitmq-consul",
"d87e464-mq-1.node.dc1.a9s-rabbitmq-consul",
"d87e464-mq-2.node.dc1.a9s-rabbitmq-consul"
],
"password": "a9s-pwd",
"port": 5672,
"uri": "amqps://a9s-brk-usr:[email protected]:5672",
"username": "a9s-brk-usr"
},
"label": "a9s-rabbitmq36",
"name": "2452-23360-25011-17366",
"plan": "rabbitmq-replica-small-ssl",
"provider": null,
"syslog_drain_url": null,
"tags": [
"messaging",
"queue"
],
"volume_mounts": []
}
]
}
}
...
```
The hostname `d67901c.service.dc1.a9svs`, username `a9s-brk-usr` and password
`a9s-password` are required in the next step.
### <a id='create-tunnel-to-the-management-dashboard'></a> Create Tunnel to the
Management Dashboard
With `cf ssh`, you can create am SSH forward tunnel to the management dashboard.
Use port `9200` to connect to the a9s RabbitMQ Instance.
```bash
$ cf ssh a9s-rabbitmq-app -L 9200:d67901c.service.dc1.a9svs:9200
vcap@956aaf4e-6da9-4f69-4b1d-8e631a403312:~$
```
When the SSH tunnel is open, you can access the instance using the address `localhost:9200`.
<p class='note'><strong>Note:</strong> Close the session by running <code>exit</code>.</p>
## <a id='create-a-fork-of-a-service-instance'></a> Create a Fork of a Service Instance
Forking a service instance involves creating a backup of the service instance
and restoring it to a different service instance.
Having two service instances is a prerequisite for the process:
```bash
$ cf s
Getting services in org jebreuer_anynines_com / space test as [email protected]...
name service plan bound apps last operation
rabbit1 a9s-rabbitmq37 rabbitmq-cluster-small bindingo create succeeded
rabbit2 a9s-rabbitmq37 rabbitmq-cluster-small create succeeded
```
Additional prerequisites regarding command line tools:
- BASH (some shell)
- CAT
- OpenSSL
- Python
- Node (tested with v6.11.0)
Do the following:
1. Open the service dashboard of the service instance you want to fork, such as
`rabbit1` in this example. You can find the dashboard URL by running the following command:
```bash
$ cf service rabbit1
Showing info of service rabbit1 in org jebreuer_anynines_com / space test as [email protected]...
name: rabbit1
service: a9s-rabbitmq37
bound apps: bindingo
tags:
plan: rabbitmq-cluster-small
description: This is a service creating and managing dedicated RabbitMQ service instances, powered by the anynines Service Framework
documentation:
dashboard: https://a9s-rabbitmq-dashboard.de.a9s.eu/service-instances/950cb675-3ed9-4613-8bb6-b2d618391d2f
[...]
```
<span>2.</span> Ensure you set a encryption password for the backups using the service instance
dashboard (**Change Backup Settings**).
<span>3.</span> Create a backup using the dashboard and download the backup to your local machine.
The filename is similar to `racsd92baee-1522222422893`.
<span>4.</span> Decrypt the backup and write its contents to a file:
```bash
$ cat racsd92baee-1522222422893 | openssl enc -aes256 -d -pass 'pass:mytopsecretpassword' > backup_settings.json
```
<span>5.</span> Create a tunnel to the admin interface of the RabbitMQ instance which is the
fork of the original instance. You need the tunnel to get the matching version of the
`rabbitmqadmin` script as well as to restore the backed-up data.
```bash
$ cf ssh someapp -L 127.0.0.1:15672:racsd92baee.service.dc1.a9ssvc:15672
```
`racsd92baee.service.dc1.a9ssvc` is the host of the rabbit2 service instance.
<span>6.</span> Go to [http://127.0.0.1:15672/cli/](http://127.0.0.1:15672/cli/) to download the
`rabbitmqadmin` tool. `rabbitmqadmin` is a Python script.
<span>7.</span> Download a copy of the backup script [restore_queues.js](./restore_queues.js).
Make sure to `chmod u+x` the script.
<span>8.</span> Restore the backed-up queues using the restore script. For example:
```bash
./restore_queues.js $(which python) ~/Downloads/rabbitmqadmin 127.0.0.1 15672 a9s-brk-usr-xxxxxxxx xxxxxxxyyyyyyyyyzzzzzzzzz ./backup_settings.json
```
## <a id="service-keys"></a> Service Keys
To gain access to the service manually rather than binding apps to it, you can
use service keys.
### <a id="create-a-service-key"></a> Create a Service Key
To create a key to the service instance `rabbitmq1` called `mykey`, run the
following command:
```bash
cf create-service-key rabbitmq1 mykey
```
### <a id="listing-service-keys"></a> List Service Keys
To list all the keys for the `rabbitmq1` service instance, run the following command:
```bash
cf service-keys rabbitmq1
```
### <a id="accessing-service-keys"></a> Access Service Keys
To obtain the key `mykey` from the `rabbitmq1` service instance, run the following command:
```bash
cf service-key rabbitmq1 mykey
```
### <a id="deleting-service-keys"></a> Delete Service Keys
To delete the key `mykey` from the service instance `rabbitmq1`, run the following command:
```bash
cf delete-service-key rabbitmq1 mykey
```
## <a id='setup-disk-usage_alerts'></a> Set Up Disk Usage Alerts
Each service instance node comes with the **a9s Parachute** component. This component
monitors *ephemeral* and *persistent* disk usage. If one of these disk usages reaches
the configured threshold of 80%, the **a9s Parachute** stops all processes on that
node. Once the processes gets stopped, **a9s Parachute** logs the following messages
to its log directory:
```
Limit reached for: persistent disk
```
or
```
Limit reached for: ephemeral disk
```
### <a id='how-to-restart-stopped-instances'></a> How to Restart Stopped Instances
To restart a stopped instance, the user can use the `cf update-service` command to
set the `max_disk_threshold` custom parameter. All values between 0 and 100 are allowed.
Updating the instance does two things, at first the limit is set to the new value
and the instance restarts.
To set the value to 90% do the following:
```bash
$ cf update-service <SERVICE_NAME> -c '{"max_disk_threshold": "90"}'
```
The update takes some minutes until the service is restarted.
<p class="note"><strong>Note:</strong> This is only a temporary solution. The nodes
are stopped when the threshold is reached again. For a long term solution the
service instance must be updated to a bigger plan.</p>