Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[event-hubs] Update migration guide to follow template #13003

Merged
14 commits merged into from
Dec 23, 2020
Merged
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
173 changes: 105 additions & 68 deletions sdk/eventhub/event-hubs/migrationguide.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,33 @@
# Guide to migrate from @azure/event-hubs v2 to v5

This document is intended for users that are familiar with V2 of the JavaScript SDK for Event Hubs library (`@azure/[email protected]` & `@azure/[email protected]`) and wish
to migrate their application to V5 of the same library.

For users new to the JavaScript SDK for Event Hubs, please see the [readme file for the @azure/event-hubs package](https://github.com/Azure/azure-sdk-for-js/blob/master/sdk/eventhub/event-hubs/README.md).
This guide is intended to assist in the migration from version 2 of the Event Hubs client library `@azure/event-hubs` and version 2 of the Event Processor Host library `@azure/event-processor-host` to version 5 of the `@azure/event-hubs` library.
It will focus on side-by-side comparisons for similar operations between the two packages.

Familiarity with the version 2 of the `@azure/event-hubs` and `@azure/event-processor-host` libraries are assumed.
For those new to the Event Hubs client library, please refer to the [README](https://github.com/Azure/azure-sdk-for-js/blob/master/sdk/eventhub/event-hubs/README.md).
and [Event Hubs samples](https://github.com/Azure/azure-sdk-for-js/tree/master/sdk/eventhub/event-hubs/samples) for the `@azure/event-hubs` library rather than this guide.

## Table of contents

- [Migration benefits](#migration-benefits)
- [Cross Service SDK improvements](#cross-service-sdk-improvements)
ramya-rao-a marked this conversation as resolved.
Show resolved Hide resolved
- [New features](#new-features)
- [Important changes](#important-changes)

- [Client hierarchy](#client-hierarchy)
- [Client constructors](#client-constructors)
- [Sending events](#sending-events)
- [Receiving messages](#receiving-messages)
- [Handling backpressure](#handling-backpressure)
- [Creating EventPosition](#creating-eventposition)
- [Granular control over retries](#granular-control-over-retries)
- [Handling errors](#handling-errors)

- [Migration samples](#migration-samples)
- [Migrating from EventHubClient to EventHubConsumerClient for receiving events](#migrating-from-eventhubclient-to-eventhubconsumerclient-for-receiving-events)
- [Migrating from EventHubClient to EventHubProducerClient for sending events](#migrating-from-eventhubclient-to-eventhubproducerclient-for-sending-events)
- [Migrating from EventProcessorHost to EventHubConsumerClient for receiving events](#migrating-from-eventprocessorhost-to-eventhubconsumerclient-for-receiving-events)
- [Additional samples](#additional-samples)

## Migration benefits

Expand All @@ -13,60 +37,33 @@ There were several areas of consistent feedback expressed across the Azure clien

To try and improve the development experience across Azure services, including Event Hubs, a set of uniform [design guidelines](https://azure.github.io/azure-sdk/general_introduction.html) was created for all languages to drive a consistent experience with established API patterns for all services. A set of [TypeScript & JavaScript specific guidelines](https://azure.github.io/azure-sdk/typescript_introduction.html) was also introduced to ensure that these libraries have a natural and idiomatic feel. Further details are available in the guidelines for those interested.

The new version 5 of the Event Hubs library provides the ability to share in some of the cross-service improvements made to the Azure development experience, such as using the new `@azure/identity` library to share a single authentication between clients and a unified diagnostics pipeline offering a common view of the activities across each of the client libraries.

## General changes

Version 5 of the `@azure/event-hubs` package is a result of our efforts to
create a client library that is user-friendly and idiomatic to the JavaScript
ecosystem.
### Cross Service SDK improvements

Apart from redesigns resulting from the new [Azure SDK Design Guidelines for Typescript](https://azure.github.io/azure-sdk/typescript_introduction.html#design-principles),
the latest version improves on several areas from V2:

### Handling backpressure
The new version of the Event Hubs library also shares some of the cross-service improvements made to the Azure development experience, such as:

Prior to V5, events were delivered as they were received with no ability
for the user to control the pace. This could result in flooding of downstream
dependencies as well as confusion about which events had been consumed in
what order, making checkpointing difficult to do correctly.
- Using the new `@azure/identity` library to share a single authentication between clients.
- A unified diagnostics pipeline that offers a common view of the activities across each of the client libraries.

In V5 the model has been simplified so new events are not delivered until the
previous batch has been consumed by your event handler. You can see a sample
demonstrating this [here](https://github.com/Azure/azure-sdk-for-js/blob/master/sdk/eventhub/event-hubs/samples/typescript/src/receiveEvents.ts)

### Specific clients for sending vs receiving

In V5 we've simplified the API surface, making two distinct clients, rather
than having a single `EventHubClient`:

- [EventHubProducerClient](https://docs.microsoft.com/javascript/api/@azure/event-hubs/eventhubproducerclient)
for sending messages.
- [EventHubConsumerClient](https://docs.microsoft.com/javascript/api/@azure/event-hubs/eventhubconsumerclient)
for receiving messages.

We've also merged the functionality from `EventProcessorHost` from the `@azure/event-processor-host` package into
`EventHubConsumerClient` in the `@azure/event-hubs` package, allowing `EventHubConsumerClient` to be the single
point of entry for receiving of any type (from single partition, all partitions, or with load balancing and checkpointing features) within Event Hubs.

Important note on checkpoints: `EventHubConsumerClient` does not support legacy checkpoint data.
i.e. the checkpoints made using the package `@azure/event-processor-host`.
In order to support interopability between different programming languages, a unifying checkpoint format was needed.
This and the need to support improvements to the algorithm used for managing partition ownership made breaking changes necessary.
### New features

### Granular control over retries
We have a variety of new features in version 5 of the Event Hubs library.

Retry logic and tuning has been externalized, allowing for better configuration
to better suit your network configuration and reliability.
- Ability to create a batch of messages with the `EventHubProducerClient.createBatch()` and `EventDataBatch.tryAdd()` APIs.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about adding the backpressure point here? From what I recall that was one of the big benefits people have to move to Track 2. We can add a link to the detailed section that comes later

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure! I was trying to limit how much I 'repeated' myself, but it is a pretty important callout.

This will help you manage events to be sent in the most optimal way.
- Ability to configure the retry policy used by operations on the clients.
- Ability to cancel async operations on the clients using the abort signal from `@azure/abort-controller`.
- Authentication with AAD credentials using `@azure/identity`.

More information about configuring and tuning retries can be found [here](https://github.com/Azure/azure-sdk-for-js/tree/master/sdk/eventhub/event-hubs#guidance-around-retries).
Refer to the [changelog](https://github.com/Azure/azure-sdk-for-js/blob/master/sdk/eventhub/event-hubs/CHANGELOG.md) for more new features, changes and bug fixes.

### Unified logging and diagnostics
## Important changes

With V5 we've unified the Azure SDK logging to work in a uniform way, making
troubleshooting simpler.
### Client hierarchy

Documentation for enabling logging in EventHubs is [here](https://github.com/Azure/azure-sdk-for-js/tree/master/sdk/eventhub/event-hubs#enable-logs).
In the interest of simplifying the API surface, we have two distinct clients: the `EventHubProducerClient` for sending events and the `EventHubConsumerClient` for receiving events.
This is in contrast the the single `EventHubClient` that was used to create senders and receivers.
We've also merged the functionality from `EventProcessorHost` from the `@azure/event-processor-host` library into `EventHubConsumerClient`,
allowing the `EventHubConsumerClient` to be the single point of entry for receiving of any type (from single partition, all partitions, or with load balancing and checkpointing features) within Event Hubs.

### Client constructors

Expand All @@ -83,6 +80,25 @@ Other noteworthy changes:
- For a checkpoint store implementation using Azure Storage Blobs, use the
[@azure/eventhubs-checkpointstore-blob](https://www.npmjs.com/package/@azure/eventhubs-checkpointstore-blob) package.

Important note on checkpoints: `EventHubConsumerClient` does not support legacy checkpoint data.
i.e. the checkpoints made using the package `@azure/event-processor-host`.
In order to support interopability between different programming languages, a unifying checkpoint format was needed.
This and the need to support improvements to the algorithm used for managing partition ownership made breaking changes necessary.

### Sending events

| In v2 | Equivalent in v5 | Sample |
| ---------------------------------- | ---------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| `EventHubClient.sendBatch(events)` | `EventHubProducerClient.sendBatch(eventBatch)` | [sendEvents](https://github.com/Azure/azure-sdk-for-js/blob/master/sdk/eventhub/event-hubs/samples/typescript/src/sendEvents.ts) |

Other noteworthy changes:

- The `send` method on the client is removed in favor of the `sendBatch` to encourage sending
events in batches for better throughput.
- The `sendBatch` method on the client takes a object of type `EventDataBatch` that should be created
ramya-rao-a marked this conversation as resolved.
Show resolved Hide resolved
using the `createBatch` method on the client.
An array of events can be passed to `sendBatch` instead of an `EventDataBatch` similarly to the removed `send` method.
chradek marked this conversation as resolved.
Show resolved Hide resolved

### Receiving events

| In v2 | Equivalent in v5 | Sample |
Expand All @@ -96,18 +112,16 @@ Other noteworthy changes:
- The user provided `processEvents` function to process events will be invoked only after the previous invocation completes.
This is different from v2 where the function was invoked for each event without waiting for the previous call to complete.

### Sending events

| In v2 | Equivalent in v5 | Sample |
| ---------------------------------- | ---------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| `EventHubClient.sendBatch(events)` | `EventHubProducerClient.sendBatch(eventBatch)` | [sendEvents](https://github.com/Azure/azure-sdk-for-js/blob/master/sdk/eventhub/event-hubs/samples/typescript/src/sendEvents.ts) |
### Handling backpressure

Other noteworthy changes:
Prior to V5, events were delivered as they were received with no ability
for the user to control the pace. This could result in flooding of downstream
dependencies as well as confusion about which events had been consumed in
what order, making checkpointing difficult to do correctly.

- The `send` method on the client is deprecated in favor of the `sendBatch` to encourage sending
events in batches for better throughput.
- The `sendBatch` method on the client takes a object of type `EventDataBatch` that should be created
using the `createBatch` method on the client.
In V5 the model has been simplified so new events are not delivered until the
previous batch has been consumed by your event handler. You can see a sample
demonstrating this [here](https://github.com/Azure/azure-sdk-for-js/blob/master/sdk/eventhub/event-hubs/samples/typescript/src/receiveEvents.ts)

### Creating EventPosition

Expand All @@ -119,6 +133,13 @@ Other noteworthy changes:
| `EventPosition.fromSequenceNumber(value)` | `{ sequenceNumber: value }` |
| `EventPosition.fromEnqueuedTime(value)` | `{ enqueuedOn: value }` |

### Granular control over retries

Retry logic and tuning has been externalized, allowing for better configuration
to better suit your network configuration and reliability.

More information about configuring and tuning retries can be found [here](https://github.com/Azure/azure-sdk-for-js/tree/master/sdk/eventhub/event-hubs#guidance-around-retries).

### Handling errors

- In v2, the `name` property on an error of class `MessagingError` was used to reflect the different
Expand All @@ -138,7 +159,7 @@ Other noteworthy changes:
- [Receiving events with checkpointing](#migrating-code-from-eventprocessorhost-to-eventhubconsumerclient-for-receiving-events)
- [Sending events](#migrating-code-from-eventhubclient-to-eventhubproducerclient-for-sending-events)

### Migrating code from `EventHubClient` to `EventHubConsumerClient` for receiving events
### Migrating from `EventHubClient` to `EventHubConsumerClient` for receiving events
ramya-rao-a marked this conversation as resolved.
Show resolved Hide resolved

In V2, event handlers were passed as positional arguments to `receive`.

Expand All @@ -158,30 +179,42 @@ await rcvHandler.stop();
Becomes this in V5:

```typescript
import { EventHubConsumerClient, earliestEventPosition } from "@azure/event-hubs";

const eventHubConsumerClient = new EventHubConsumerClient(consumerGroupName, connectionString);

const subscription = eventHubConsumerClient.subscribe(partitionId, {
processInitialize: (initContext) => {
initContext.setStartingPosition(earliestEventPosition);
const subscription = eventHubConsumerClient.subscribe(
partitionId,
{
processInitialize: (initContext) => {
initContext.setStartingPosition(earliestEventPosition);
},
processEvents: onMessageHandler,
processError: onErrorHandler
},
processEvents: onMessageHandler,
processError: onErrorHandler
});
{
startPosition: earliestEventPosition
}
);

await subscription.close();
```

See [`receiveEvents.ts`](https://github.com/Azure/azure-sdk-for-js/blob/master/sdk/eventhub/event-hubs/samples/typescript/src/receiveEvents.ts)
for a sample program demonstrating this.

### Migrating code from `EventHubClient` to `EventHubProducerClient` for sending events
### Migrating from `EventHubClient` to `EventHubProducerClient` for sending events
ramya-rao-a marked this conversation as resolved.
Show resolved Hide resolved

In V2, there were multiple options on how to send data.

In V5, this has been consolidated into a more efficient `sendBatch` method.
Batching merges information from multiple messages into a single send, reducing
the amount of network communication needed vs sending messages one at a time.

Note: As of version 5.2.0, `sendBatch` also accepts an array of events to send.
This can be used if you're sure that the events you're sending can fit within a single batch.
Creating a batch using `createBatch` and adding events using `batch.tryAdd()` is the safest way to send events since you can be sure the batch size won't exceed service limits.

So in V2:

```typescript
Expand Down Expand Up @@ -244,7 +277,7 @@ if (batch.count > 0) {
}
```

### Migrating code from `EventProcessorHost` to `EventHubConsumerClient` for receiving events
### Migrating from `EventProcessorHost` to `EventHubConsumerClient` for receiving events
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think of moving this under the Sending events section as an h4? Same for the "migration samples" for receiving?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That seems reasonable, will do.


In V2, `EventProcessorHost` allowed you to balance the load between multiple instances of
your program when receiving events.
Expand Down Expand Up @@ -324,4 +357,8 @@ const subscription = eventHubConsumerClient.subscribe(partitionId, {
await subscription.close();
```

## Additional samples

More examples can be found at [samples for @azure/event-hubs](https://github.com/Azure/azure-sdk-for-js/tree/master/sdk/eventhub/event-hubs/samples).

![Impressions](https://azure-sdk-impressions.azurewebsites.net/api/impressions/azure-sdk-for-js%2Fsdk%2Feventhub%2Fevent-hubs%2FMIGRATIONGUIDE.png)