Skip to content

Commit

Permalink
Added the Bulk Task Create Utility, removed older compute node client…
Browse files Browse the repository at this point in the history
…s and improved documentation (#36204)

* Added bulk task create operation

* Modified error handling of bulk task create

* Added more Task related tests

* Removed renamed and removed ComputeNode and Extension clients and modified documentation
  • Loading branch information
NickKouds authored and skapur12 committed Mar 26, 2024
1 parent 82f57d5 commit 7382355
Show file tree
Hide file tree
Showing 24 changed files with 1,123,488 additions and 5,276 deletions.
128 changes: 124 additions & 4 deletions sdk/batch/azure-compute-batch/README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Azure Batch client library for Java

Azure Batch client library for Java.
This README is based on the latest released version of the Azure Compute Batch SDK, otherwise known as the track 2 Azure Batch Data Plane SDK.

This package contains Microsoft Azure Batch client library.
> The SDK supports features of the Azure Batch service starting from API version **2023-05-01.16.0**. We will be adding support for more new features and tweaking the API associated with Azure Batch service newer release.
## Documentation

Expand Down Expand Up @@ -35,13 +35,78 @@ Various documentation is available to help you get started
[Azure Identity][azure_identity] package provides the default implementation for authenticating the client.

## Key concepts
**Azure Batch Authentication**

## Examples
You need to create a Batch account through the [Azure portal](https://portal.azure.com) or Azure cli.

* Use the account name, key, and URL to create a `BatchSharedKeyCredentials` instance for authentication with the Azure Batch service.
The `BatchServiceClientBuilder` class is the simplest entry point for creating and interacting with Azure Batch objects.

```
BatchServiceClientBuilder batchClientBuilder = new BatchServiceClientBuilder()
.endpoint(batchEndpoint)
.httpClient(HttpClient.createDefault());
BatchSharedKeyCredentials sharedKeyCred = new BatchSharedKeyCredentials(batchEndpoint, accountName, accountKey);
batchClientBuilder.credential(sharedKeyCred);
```

* The other way is using AAD (Azure Active Directory) authentication to create the client. See this [document](https://docs.microsoft.com/azure/batch/batch-aad-auth) for details on authenticating to Batch with AAD and this [document](azure_identity) for understanding how the Azure Identity library supports AAD token authentication in Java.
For example:

```
batchClientBuilder.credential(new DefaultAzureCredentialBuilder().build());
```

**Create a pool using an Azure Marketplace image**

You can create a pool of Azure virtual machines which can be used to execute tasks.

```
System.out.println("Created a pool using an Azure Marketplace image.");
ImageReference imgRef = new ImageReference().setPublisher("Canonical").setOffer("UbuntuServer")
.setSku("18.04-LTS").setVersion("latest");
String poolVmSize = "STANDARD_D1_V2";
VirtualMachineConfiguration configuration = new VirtualMachineConfiguration(imgRef, "batch.node.ubuntu 18.04");
BatchPoolCreateParameters poolCreateParameters = new BatchPoolCreateParameters(poolId, poolVmSize);
poolCreateParameters.setTargetDedicatedNodes(1)
.setVirtualMachineConfiguration(configuration)
.setUserAccounts(userList)
.setNetworkConfiguration(networkConfiguration);
batchClientBuilder.buildPoolClient().create(poolCreateParameters);
System.out.println("Created a Pool: " + poolId);
```

**Create a Job**

You can create a job by using the recently created pool.

```
PoolInformation poolInfo = new PoolInformation();
poolInfo.setPoolId(poolId);
BatchJobCreateParameters jobCreateParameters = new BatchJobCreateParameters(jobId, poolInfo);
batchClientBuilder.buildJobClient().create(jobCreateParameters);
```
# Sample Code
//TODO

You can find sample code that illustrates Batch usage scenarios in https://github.com/azure/azure-batch-samples


## Examples
TODO?
```java com.azure.compute.batch.readme
```

## Troubleshooting
## Help

If you encounter any bugs with these libraries, please file issues via [Issues](https://github.com/Azure/azure-sdk-for-java) or checkout [StackOverflow for Azure Java SDK](https://stackoverflow.com/questions/tagged/azure-java-sdk).

## Next steps

Expand All @@ -55,6 +120,61 @@ For details on contributing to this repository, see the [contributing guide](htt
1. Push to the branch (`git push origin my-new-feature`)
1. Create new Pull Request

# Build Code
To build the code open a console, navigate to the project subdirectory (sdk/batch/azure-compute-batch/, and run
```
maven build
```

# Test Code

All tests are run from the `sdk/batch/azure-compute-batch` directory. They can be run either on the command line or from a Java IDE, such as IntelliJ as Junit (Note that if you wish to run the tests within IntelliJ, you will need to temporarily delete the module-info.java file.

Tests are run in two phases: Record and Playback. During the first Record phase, integration tests create real Batch resources on Azure using the Batch API, and JSON files are created locally to capture the response from Azure. In the second Playback phase, the integrations tests only exercise assertions against the JSON files themselves.

- A valid Azure subscription that can create resources
- A service principal with contributor access to the subscription
If not already available, create an app registration in "Azure Active Directory"
Generate a client secret for this principal
- A clean Batch account
- A storage account
- A virtual network

## Step 1: Run tests in Record mode

1. Deploy test resources in Azure and set the following environment variables:

* AZURE_CLIENT_SECRET
* AZURE_TENANT_ID
* AZURE_BATCH_ACCESS_KEY
* AZURE_BATCH_ACCOUNT
* AZURE_BATCH_ENDPOINT
* AZURE_BATCH_REGION
* AZURE_VNET
* AZURE_VNET_ADDRESS_SPACE
* AZURE_VNET_RESOURCE_GROUP
* AZURE_VNET_SUBNET
* AZURE_VNET_SUBNET_ADDRESS_SPACE
* AZURE_CLIENT_ID
* STORAGE_ACCOUNT_KEY
* STORAGE_ACCOUNT_NAME
* AZURE_SUBSCRIPTION_ID

1. Set `AZURE_TEST_MODE` to `Record`
1. Run the tests in `src/test/java`
1. From the command-line, run `mvn test` (can also supply `-DAZURE_TEST_MODE=Record` instead of setting environment variable)
1. Test recordings will be created/modified in `azure-compute-batch/src/test/resources/session-records`


## Step 2: Run tests in Playback mode

1. Set `AZURE_TEST_MODE` to `Playback`
1. Run the Jetty test server
1. CLI: `mvn jetty:start`
1. Eclipse: Install Jetty plugin for Eclipse from marketplace and create two run configurations (one for 11080 and one for 11081)
1. Run the tests
1. CLI: `mvn test -DAZURE_TEST_MODE=Playback`

<!-- LINKS -->
[product_documentation]: https://azure.microsoft.com/services/
[docs]: https://azure.github.io/azure-sdk-for-java/
Expand Down
4 changes: 2 additions & 2 deletions sdk/batch/azure-compute-batch/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@
<version>1.0.0-beta.1</version> <!-- {x-version-update;com.azure:azure-compute-batch;current} -->
<packaging>jar</packaging>

<name>Microsoft Azure SDK for Batch Management</name>
<description>This package contains Microsoft Azure Batch Track 2 client library.</description>
<name>Microsoft Azure SDK for Batch Client</name>
<description>This package contains Microsoft Azure Batch Track 2 data plane client library.</description>
<url>https://github.com/Azure/azure-sdk-for-java</url>

<licenses>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,7 @@
import com.azure.core.client.traits.HttpTrait;
import com.azure.core.client.traits.TokenCredentialTrait;
import com.azure.core.credential.TokenCredential;
import com.azure.core.http.HttpClient;
import com.azure.core.http.HttpHeaders;
import com.azure.core.http.HttpPipeline;
import com.azure.core.http.HttpPipelineBuilder;
import com.azure.core.http.HttpPipelinePosition;
import com.azure.core.http.*;
import com.azure.core.http.policy.AddDatePolicy;
import com.azure.core.http.policy.AddHeadersFromContextPolicy;
import com.azure.core.http.policy.AddHeadersPolicy;
Expand Down Expand Up @@ -274,7 +270,7 @@ private HttpPipeline createHttpPipeline() {
policies.add(new RequestIdPolicy());
policies.add(new AddHeadersFromContextPolicy());
HttpHeaders headers = new HttpHeaders();
localClientOptions.getHeaders().forEach(header -> headers.set(header.getName(), header.getValue()));
localClientOptions.getHeaders().forEach(header -> headers.set(HttpHeaderName.fromString(header.getName()), header.getValue()));
if (headers.getSize() > 0) {
policies.add(new AddHeadersPolicy(headers));
}
Expand Down
Loading

0 comments on commit 7382355

Please sign in to comment.