forked from Azure/azure-sdk-for-python
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
improve docs and samples for glossaries and custom models (Azure#18587)
* update the readme * update readme file * added custom translation samples * fix 'no-locale' thing in links * update glossary docs * update glossaries * link to sample glossaries instead of writing code in readme * update custom model sample linking * remove relative linking in readme * make subheadings in bold text to be more readable * conform with 'Document Translation' naming * disambiguate container sas url * capitaliz Azure name * remove misplaced period * update samples -> custom model * update async sample -> custom model * remove localization from url * update readme with new file types for glossaries * adding sample glossaries -> xlf * white space * use simplified single input method * update 'job' terminology * update azure-core naming * update glossary blob file reference name * link to supported glossaries table * remove locale from url
- Loading branch information
1 parent
08043d1
commit 5702a23
Showing
6 changed files
with
233 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
4 changes: 4 additions & 0 deletions
4
sdk/translation/azure-ai-translation-document/samples/assets/glossary_sample.csv
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| skull,le crâne | ||
| body,corps | ||
| heart,cœur | ||
| lungs,poumons |
4 changes: 4 additions & 0 deletions
4
sdk/translation/azure-ai-translation-document/samples/assets/glossary_sample.tsv
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| skull le crâne | ||
| body corps | ||
| heart cœur | ||
| lungs poumons |
23 changes: 23 additions & 0 deletions
23
sdk/translation/azure-ai-translation-document/samples/assets/glossary_sample.xlf
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| <?xml version="1.0" encoding="UTF-8"?> | ||
| <xliff xmlns="urn:oasis:names:tc:xliff:document:1.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="1.2" xsi:schemaLocation="urn:oasis:names:tc:xliff:document:1.2 http://docs.oasis-open.org/xliff/v1.2/os/xliff-core-1.2-strict.xsd"> | ||
| <file original="EN-TA.tmx" source-language="en-US" target-language="fr-FR" datatype="xml"> | ||
| <body> | ||
| <trans-unit id="1" datatype="plaintext"> | ||
| <source>skull</source> | ||
| <target state="translated">le crâne</target> | ||
| </trans-unit> | ||
| <trans-unit id="2" datatype="plaintext"> | ||
| <source>body</source> | ||
| <target state="translated">corps</target> | ||
| </trans-unit> | ||
| <trans-unit id="3" datatype="plaintext"> | ||
| <source>heart</source> | ||
| <target state="translated">cœur</target> | ||
| </trans-unit> | ||
| <trans-unit id="4" datatype="plaintext"> | ||
| <source>lungs</source> | ||
| <target state="translated">poumons</target> | ||
| </trans-unit> | ||
| </body> | ||
| </file> | ||
| </xliff> |
82 changes: 82 additions & 0 deletions
82
...-translation-document/samples/async_samples/sample_translation_with_custom_model_async.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,82 @@ | ||
| # coding=utf-8 | ||
| # ------------------------------------ | ||
| # Copyright (c) Microsoft Corporation. | ||
| # Licensed under the MIT License. | ||
| # ------------------------------------ | ||
|
|
||
| """ | ||
| FILE: sample_translation_with_custom_model_async.py | ||
| DESCRIPTION: | ||
| This sample demonstrates how to create a translation operation and apply custom azure translation model when doing the translation. | ||
| To set up your containers for translation and generate SAS tokens to your containers (or files) | ||
| with the appropriate permissions, see the README. | ||
| USAGE: | ||
| python sample_translation_with_custom_model_async.py | ||
| Set the environment variables with your own values before running the sample: | ||
| 1) AZURE_DOCUMENT_TRANSLATION_ENDPOINT - the endpoint to your Document Translation resource. | ||
| 2) AZURE_DOCUMENT_TRANSLATION_KEY - your Document Translation API key. | ||
| 3) AZURE_SOURCE_CONTAINER_URL - the container SAS URL to your source container which has the documents | ||
| to be translated. | ||
| 4) AZURE_TARGET_CONTAINER_URL - the container SAS URL to your target container where the translated documents | ||
| will be written. | ||
| 5) AZURE_CUSTOM_MODEL_ID - the URL to your Azure custom translation model. | ||
| """ | ||
|
|
||
| import asyncio | ||
|
|
||
|
|
||
| async def sample_translation_with_custom_model_async(): | ||
| import os | ||
| from azure.core.credentials import AzureKeyCredential | ||
| from azure.ai.translation.document.aio import DocumentTranslationClient | ||
|
|
||
| endpoint = os.environ["AZURE_DOCUMENT_TRANSLATION_ENDPOINT"] | ||
| key = os.environ["AZURE_DOCUMENT_TRANSLATION_KEY"] | ||
| source_container_url = os.environ["AZURE_SOURCE_CONTAINER_URL"] | ||
| target_container_url = os.environ["AZURE_TARGET_CONTAINER_URL"] | ||
| custom_model_id = os.environ["AZURE_CUSTOM_MODEL_ID"] | ||
|
|
||
| client = DocumentTranslationClient(endpoint, AzureKeyCredential(key)) | ||
|
|
||
|
|
||
|
|
||
| async with client: | ||
| poller = await client.begin_translation( | ||
| source_container_url, | ||
| target_container_url, | ||
| "es", | ||
| category_id=custom_model_id | ||
| ) | ||
| result = await poller.result() | ||
|
|
||
| print("Operation status: {}".format(result.status)) | ||
| print("Operation created on: {}".format(result.created_on)) | ||
| print("Operation last updated on: {}".format(result.last_updated_on)) | ||
| print("Total number of translations on documents: {}".format(result.documents_total_count)) | ||
|
|
||
| print("\nOf total documents...") | ||
| print("{} failed".format(result.documents_failed_count)) | ||
| print("{} succeeded".format(result.documents_succeeded_count)) | ||
|
|
||
| doc_results = client.list_all_document_statuses(result.id) | ||
| async for document in doc_results: | ||
| print("Document ID: {}".format(document.id)) | ||
| print("Document status: {}".format(document.status)) | ||
| if document.status == "Succeeded": | ||
| print("Source document location: {}".format(document.source_document_url)) | ||
| print("Translated document location: {}".format(document.translated_document_url)) | ||
| print("Translated to language: {}\n".format(document.translate_to)) | ||
| else: | ||
| print("Error Code: {}, Message: {}\n".format(document.error.code, document.error.message)) | ||
|
|
||
|
|
||
| async def main(): | ||
| await sample_translation_with_custom_model_async() | ||
|
|
||
| if __name__ == '__main__': | ||
| loop = asyncio.get_event_loop() | ||
| loop.run_until_complete(main()) |
75 changes: 75 additions & 0 deletions
75
...translation/azure-ai-translation-document/samples/sample_translation_with_custom_model.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,75 @@ | ||
| # coding=utf-8 | ||
| # ------------------------------------ | ||
| # Copyright (c) Microsoft Corporation. | ||
| # Licensed under the MIT License. | ||
| # ------------------------------------ | ||
|
|
||
| """ | ||
| FILE: sample_translation_with_custom_model.py | ||
| DESCRIPTION: | ||
| This sample demonstrates how to create a translation operation and apply custom azure translation model when doing the translation. | ||
| To set up your containers for translation and generate SAS tokens to your containers (or files) | ||
| with the appropriate permissions, see the README. | ||
| USAGE: | ||
| python sample_translation_with_custom_model.py | ||
| Set the environment variables with your own values before running the sample: | ||
| 1) AZURE_DOCUMENT_TRANSLATION_ENDPOINT - the endpoint to your Document Translation resource. | ||
| 2) AZURE_DOCUMENT_TRANSLATION_KEY - your Document Translation API key. | ||
| 3) AZURE_SOURCE_CONTAINER_URL - the container SAS URL to your source container which has the documents | ||
| to be translated. | ||
| 4) AZURE_TARGET_CONTAINER_URL - the container SAS URL to your target container where the translated documents | ||
| will be written. | ||
| 5) AZURE_CUSTOM_MODEL_ID - the URL to your Azure custom translation model. | ||
| """ | ||
|
|
||
|
|
||
| def sample_translation_with_custom_model(): | ||
| import os | ||
| from azure.core.credentials import AzureKeyCredential | ||
| from azure.ai.translation.document import ( | ||
| DocumentTranslationClient | ||
| ) | ||
|
|
||
| endpoint = os.environ["AZURE_DOCUMENT_TRANSLATION_ENDPOINT"] | ||
| key = os.environ["AZURE_DOCUMENT_TRANSLATION_KEY"] | ||
| source_container_url = os.environ["AZURE_SOURCE_CONTAINER_URL"] | ||
| target_container_url = os.environ["AZURE_TARGET_CONTAINER_URL"] | ||
| custom_model_id = os.environ["AZURE_CUSTOM_MODEL_ID"] | ||
|
|
||
| client = DocumentTranslationClient(endpoint, AzureKeyCredential(key)) | ||
|
|
||
| poller = client.begin_translation( | ||
| source_container_url, | ||
| target_container_url, | ||
| "es", | ||
| category_id=custom_model_id | ||
| ) | ||
| result = poller.result() | ||
|
|
||
| print("Operation status: {}".format(result.status)) | ||
| print("Operation created on: {}".format(result.created_on)) | ||
| print("Operation last updated on: {}".format(result.last_updated_on)) | ||
| print("Total number of translations on documents: {}".format(result.documents_total_count)) | ||
|
|
||
| print("\nOf total documents...") | ||
| print("{} failed".format(result.documents_failed_count)) | ||
| print("{} succeeded".format(result.documents_succeeded_count)) | ||
|
|
||
| doc_results = client.list_all_document_statuses(result.id) | ||
| for document in doc_results: | ||
| print("Document ID: {}".format(document.id)) | ||
| print("Document status: {}".format(document.status)) | ||
| if document.status == "Succeeded": | ||
| print("Source document location: {}".format(document.source_document_url)) | ||
| print("Translated document location: {}".format(document.translated_document_url)) | ||
| print("Translated to language: {}\n".format(document.translate_to)) | ||
| else: | ||
| print("Error Code: {}, Message: {}\n".format(document.error.code, document.error.message)) | ||
|
|
||
|
|
||
| if __name__ == '__main__': | ||
| sample_translation_with_custom_model() |