forked from Azure/azure-sdk-for-python
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Align eventhub package folder names with package name (Azure#8813)
- Loading branch information
1 parent
abc5dfd
commit 2b8a6ed
Showing
121 changed files
with
74 additions
and
85 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3,9 +3,9 @@ | |
| Azure EventHubs Checkpoint Store is used for storing checkpoints while processing events from Azure Event Hubs. | ||
| This Checkpoint Store package works as a plug-in package to `EventHubConsumerClient`. It uses Azure Storage Blob as the persistent store for maintaining checkpoints and partition ownership information. | ||
|
|
||
| Please note that this is an async library, for sync version of the Azure EventHubs Checkpoint Store client library, please refer to [azure-eventhubs-checkpointstoreblob](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/eventhub/azure-eventhubs-checkpointstoreblob). | ||
| Please note that this is an async library, for sync version of the Azure EventHubs Checkpoint Store client library, please refer to [azure-eventhub-checkpointstoreblob](./). | ||
|
|
||
| [Source code](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/eventhub/azure-eventhubs-checkpointstoreblob-aio) | [Package (PyPi)](https://pypi.org/project/azure-eventhub-checkpointstoreblob-aio/) | [API reference documentation](https://azuresdkdocs.blob.core.windows.net/$web/python/azure-eventhub/5.0.0b6/azure.eventhub.aio.html#azure.eventhub.aio.CheckpointStore) | [Azure Eventhubs documentation](https://docs.microsoft.com/en-us/azure/event-hubs/) | [Azure Storage documentation](https://docs.microsoft.com/en-us/azure/storage/) | ||
| [Source code](./) | [Package (PyPi)](https://pypi.org/project/azure-eventhub-checkpointstoreblob-aio/) | [API reference documentation](https://azuresdkdocs.blob.core.windows.net/$web/python/azure-eventhub/5.0.0b6/azure.eventhub.aio.html#azure.eventhub.aio.CheckpointStore) | [Azure Eventhubs documentation](https://docs.microsoft.com/en-us/azure/event-hubs/) | [Azure Storage documentation](https://docs.microsoft.com/en-us/azure/storage/) | ||
|
|
||
| ## Getting started | ||
|
|
||
|
|
@@ -29,23 +29,23 @@ $ pip install --pre azure-eventhub-checkpointstoreblob-aio | |
|
|
||
| ### Checkpointing | ||
|
|
||
| Checkpointing is a process by which readers mark or commit their position within a partition event sequence. | ||
| Checkpointing is the responsibility of the consumer and occurs on a per-partition basis within a consumer group. | ||
| This responsibility means that for each consumer group, each partition reader must keep track of its current position | ||
| Checkpointing is a process by which readers mark or commit their position within a partition event sequence. | ||
| Checkpointing is the responsibility of the consumer and occurs on a per-partition basis within a consumer group. | ||
| This responsibility means that for each consumer group, each partition reader must keep track of its current position | ||
| in the event stream, and can inform the service when it considers the data stream complete. If a reader disconnects from | ||
| a partition, when it reconnects it begins reading at the checkpoint that was previously submitted by the last reader of | ||
| that partition in that consumer group. When the reader connects, it passes the offset to the event hub to specify the | ||
| location at which to start reading. In this way, you can use checkpointing to both mark events as "complete" by | ||
| downstream applications, and to provide resiliency if a failover between readers running on different machines occurs. | ||
| It is possible to return to older data by specifying a lower offset from this checkpointing process. Through this | ||
| that partition in that consumer group. When the reader connects, it passes the offset to the event hub to specify the | ||
| location at which to start reading. In this way, you can use checkpointing to both mark events as "complete" by | ||
| downstream applications, and to provide resiliency if a failover between readers running on different machines occurs. | ||
| It is possible to return to older data by specifying a lower offset from this checkpointing process. Through this | ||
| mechanism, checkpointing enables both failover resiliency and event stream replay. | ||
|
|
||
| ### Offsets & sequence numbers | ||
| Both offset & sequence number refer to the position of an event within a partition. You can think of them as a | ||
| client-side cursor. The offset is a byte numbering of the event. The offset/sequence number enables an event consumer | ||
| (reader) to specify a point in the event stream from which they want to begin reading events. You can specify a | ||
| timestamp such that you receive events enqueued only after the given timestamp. Consumers are responsible for | ||
| storing their own offset values outside of the Event Hubs service. Within a partition, each event includes an offset, | ||
| Both offset & sequence number refer to the position of an event within a partition. You can think of them as a | ||
| client-side cursor. The offset is a byte numbering of the event. The offset/sequence number enables an event consumer | ||
| (reader) to specify a point in the event stream from which they want to begin reading events. You can specify a | ||
| timestamp such that you receive events enqueued only after the given timestamp. Consumers are responsible for | ||
| storing their own offset values outside of the Event Hubs service. Within a partition, each event includes an offset, | ||
| sequence number and the timestamp of when it was enqueued. | ||
|
|
||
| ## Examples | ||
|
|
@@ -58,7 +58,7 @@ The easiest way to create a `EventHubConsumerClient` is to use a connection stri | |
| from azure.eventhub.aio import EventHubConsumerClient | ||
| eventhub_client = EventHubConsumerClient.from_connection_string("my_eventhub_namespace_connection_string", "my_consumer_group", eventhub_name="my_eventhub") | ||
| ``` | ||
| For other ways of creating a `EventHubConsumerClient`, refer to [EventHubs library](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/eventhub/azure-eventhubs) for more details. | ||
| For other ways of creating a `EventHubConsumerClient`, refer to [EventHubs library](../azure-eventhub) for more details. | ||
|
|
||
| ### Consume events using a `BlobCheckpointStore` to do checkpoint | ||
| ```python | ||
|
|
@@ -108,7 +108,7 @@ Refer to [Logging](#logging) to enable loggers for related libraries. | |
| ## Next steps | ||
|
|
||
| ### Examples | ||
| - [./samples/event_processor_blob_storage_example.py](https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/eventhub/azure-eventhubs-checkpointstoreblob-aio/samples/event_processor_blob_storage_example.py) - EventHubConsumerClient with blob checkpoint store example | ||
| - [./samples/event_processor_blob_storage_example.py](./samples/event_processor_blob_storage_example.py) - EventHubConsumerClient with blob checkpoint store example | ||
|
|
||
| ### Documentation | ||
|
|
||
|
|
@@ -135,4 +135,4 @@ When you submit a pull request, a CLA-bot will automatically determine whether y | |
| This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). | ||
| For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [[email protected]](mailto:[email protected]) with any additional questions or comments. | ||
|
|
||
|  | ||
|  | ||
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
2 changes: 1 addition & 1 deletion
2
...ckpointstoreblob-aio/dev_requirements.txt → ...ckpointstoreblob-aio/dev_requirements.txt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,4 @@ | ||
| -e ../../../tools/azure-sdk-tools | ||
| ../../core/azure-core | ||
| -e ../../storage/azure-storage-blob | ||
| ../azure-eventhubs | ||
| ../azure-eventhub |
File renamed without changes.
File renamed without changes.
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -51,7 +51,7 @@ | |
| license='MIT License', | ||
| author='Microsoft Corporation', | ||
| author_email='[email protected]', | ||
| url='https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/eventhub/azure-eventhubs-checkpointstoreblob-aio', | ||
| url='https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/eventhub/azure-eventhub-checkpointstoreblob-aio', | ||
| classifiers=[ | ||
| 'Development Status :: 3 - Alpha', | ||
| 'Programming Language :: Python', | ||
|
|
||
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
File renamed without changes.
Oops, something went wrong.