Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Draft - Do not merge] [Quantum] Use Post Storage URI and SAS token like Quantum Python SDK (27643) #8242

Draft
wants to merge 29 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
e3c8054
Temporarily append '.dev3' to version number
warren-jones Nov 6, 2024
d818972
Calling upload-blob with return_sas_token = True
warren-jones Nov 22, 2024
b6fa1db
Incorporated azure-quantum-python functions
warren-jones Dec 2, 2024
1ca2709
Adding a SAS token to the storage container URL
warren-jones Dec 4, 2024
2bdf773
Moved an import statement from _storage.py to job.py
warren-jones Dec 4, 2024
1d06fd9
Fixed style check violations
warren-jones Dec 5, 2024
1983073
Deleted an extraneous comment
warren-jones Dec 5, 2024
7bb57e3
Added a unit test
warren-jones Dec 5, 2024
1a105cc
Expanded the unit test
warren-jones Dec 5, 2024
042064b
Made changes to test_submit
warren-jones Dec 6, 2024
703f3e9
Made changes to test_submit
warren-jones Dec 6, 2024
b4ef46d
Updated test_provider_sku_list
warren-jones Dec 6, 2024
950a4e0
Added experimental Python SDK code to run a Qiskit input file
warren-jones Dec 13, 2024
dc940c1
Added test input files and simplified the job submit code
warren-jones Dec 13, 2024
50cee37
Rearranged the submit code to use old param validation
warren-jones Dec 14, 2024
b85a23f
Got it working with imported Python SDK methods
warren-jones Dec 19, 2024
5d71981
Un-did changes to _storage.py and _client_factory.py
warren-jones Dec 19, 2024
941ef50
Fixed Pylint rule violations
warren-jones Dec 19, 2024
9101d4e
Experimenting with ways to suppress the expected azure.identity Crede…
warren-jones Dec 20, 2024
be59ec2
Deleted unused import statements
warren-jones Dec 20, 2024
3733adc
Copied azure-quantum-python files to the CLI repo
warren-jones Dec 21, 2024
a282256
Added missing SDK files and fixed another import
warren-jones Dec 21, 2024
c00aa7c
Added the azure.identity files to vendored_sdks
warren-jones Dec 23, 2024
1ef8c76
Fixed another azure.identity import
warren-jones Dec 23, 2024
40af901
Deleted a commented-out line
warren-jones Dec 23, 2024
3fc5f02
Moved the azure-quantum-python files to the vendored_sdks folder
warren-jones Dec 24, 2024
31cca14
Deleted commented-out lines
warren-jones Dec 26, 2024
5556f1b
Removed 'microsoft.qc' from the default test provider/SKU list and re…
warren-jones Dec 27, 2024
0ad93b0
Made test changes like in branch 29126-remove-resource-estimator-refe…
warren-jones Jan 2, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 0 additions & 128 deletions src/quantum/azext_quantum/_storage.py

This file was deleted.

24 changes: 13 additions & 11 deletions src/quantum/azext_quantum/operations/job.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,15 @@
import uuid
import knack.log

from azure.cli.command_modules.storage.operations.account import show_storage_account_connection_string
from azure.cli.core.azclierror import (FileOperationError, AzureInternalError,
InvalidArgumentValueError, AzureResponseError,
RequiredArgumentMissingError)

from .._storage import create_container, upload_blob

from ..vendored_sdks.azure_quantum_python.workspace import Workspace
from ..vendored_sdks.azure_quantum_python.storage import upload_blob
from ..vendored_sdks.azure_storage_blob import ContainerClient
from .._client_factory import cf_jobs

from .workspace import WorkspaceInfo
from .target import TargetInfo, get_provider

Expand Down Expand Up @@ -234,15 +235,19 @@ def submit(cmd, resource_group_name, workspace_name, location, target_id, job_in
raise RequiredArgumentMissingError("No storage account specified or linked with workspace.")
storage = ws.properties.storage_account.split('/')[-1]
job_id = str(uuid.uuid4())
container_name = "quantum-job-" + job_id
connection_string_dict = show_storage_account_connection_string(cmd, resource_group_name, storage)
connection_string = connection_string_dict["connectionString"]
container_client = create_container(connection_string, container_name)
blob_name = "inputData"

resource_id = "/subscriptions/" + ws_info.subscription + "/resourceGroups/" + ws_info.resource_group + "/providers/Microsoft.Quantum/Workspaces/" + ws_info.name
workspace = Workspace(resource_id=resource_id, location=location)

knack_logger.warning("Getting Azure credential token...")
container_uri = workspace.get_container_uri(job_id=job_id)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does anything here create an actual container from the containerUri? Previously, it looks like create_container would do this.
We need to create a container if it doesn't already exists. The service just returns a URI, it doesn't actually create it today., although we are planning to modify it so that it creates it. The cli should handle both cases.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like upload_blob will create a container if it doesn't already exist, so I think we are good.

container_client = ContainerClient.from_container_url(container_uri)

knack_logger.warning("Uploading input data...")
try:
blob_uri = upload_blob(container_client, blob_name, content_type, content_encoding, blob_data, False)
blob_uri = upload_blob(container_client, blob_name, content_type, content_encoding, blob_data, return_sas_token=False)
logger.debug(" - blob uri: %s", blob_uri)
except Exception as e:
# Unexplained behavior:
# QIR bitcode input and QIO (gzip) input data get UnicodeDecodeError on jobs run in tests using
Expand All @@ -253,9 +258,6 @@ def submit(cmd, resource_group_name, workspace_name, location, target_id, job_in
error_msg += f"\nReason: {e.reason}"
raise AzureResponseError(error_msg) from e

start_of_blob_name = blob_uri.find(blob_name)
container_uri = blob_uri[0:start_of_blob_name - 1]

# Combine separate command-line parameters (like shots, target_capability, and entry_point) with job_params
if job_params is None:
job_params = {}
Expand Down
22 changes: 21 additions & 1 deletion src/quantum/azext_quantum/tests/latest/test_quantum_jobs.py
Original file line number Diff line number Diff line change
Expand Up @@ -198,12 +198,32 @@ def test_submit(self):
test_location = get_test_workspace_location()
test_resource_group = get_test_resource_group()
test_workspace_temp = get_test_workspace_random_name()
test_provider_sku_list = "qci/qci-freepreview,rigetti/azure-quantum-credits,ionq/pay-as-you-go-cred,microsoft-qc/learn-and-develop"
test_provider_sku_list = "qci/qci-freepreview,rigetti/azure-quantum-credits,ionq/aq-internal-testing"
test_storage = get_test_workspace_storage()

self.cmd(f"az quantum workspace create -g {test_resource_group} -w {test_workspace_temp} -l {test_location} -a {test_storage} -r {test_provider_sku_list} --skip-autoadd")
self.cmd(f"az quantum workspace set -g {test_resource_group} -w {test_workspace_temp} -l {test_location}")

# Submit a job to Rigetti and look for SAS tokens in URIs in the output
results = self.cmd("az quantum job submit -t rigetti.sim.qvm --job-input-format rigetti.quil.v1 -t rigetti.sim.qvm --job-input-file src/quantum/azext_quantum/tests/latest/input_data/bell-state.quil --job-output-format rigetti.quil-results.v1 -o json").get_output_in_json()
self.assertIn("?sv=", results["containerUri"])
self.assertIn("&st=", results["containerUri"])
self.assertIn("&se=", results["containerUri"])
self.assertIn("&sp=", results["containerUri"])
self.assertIn("&sig=", results["containerUri"])

self.assertIn("?sv=", results["inputDataUri"])
self.assertIn("&st=", results["inputDataUri"])
self.assertIn("&se=", results["inputDataUri"])
self.assertIn("&sp=", results["inputDataUri"])
self.assertIn("&sig=", results["inputDataUri"])

self.assertIn("?sv=", results["outputDataUri"])
self.assertIn("&st=", results["outputDataUri"])
self.assertIn("&se=", results["outputDataUri"])
self.assertIn("&sp=", results["outputDataUri"])
self.assertIn("&sig=", results["outputDataUri"])

# Run a Quil pass-through job on Rigetti
results = self.cmd("az quantum run -t rigetti.sim.qvm --job-input-format rigetti.quil.v1 -t rigetti.sim.qvm --job-input-file src/quantum/azext_quantum/tests/latest/input_data/bell-state.quil --job-output-format rigetti.quil-results.v1 -o json").get_output_in_json()
self.assertIn("ro", results)
Expand Down
11 changes: 5 additions & 6 deletions src/quantum/azext_quantum/tests/latest/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,11 @@
TEST_WORKSPACE_DEFAULT_LOCATION = "westus2"
TEST_WORKSPACE_DEFAULT_STORAGE = "e2etests"
TEST_WORKSPACE_DEFAULT_STORAGE_GRS = "e2etestsgrs"
TEST_WORKSPACE_DEFAULT_PROVIDER_SKU_LIST = "microsoft-qc/learn-and-develop"
TEST_CAPABILITIES_DEFAULT = "new.microsoft-qc;submit.microsoft-qc"

TEST_TARGET_DEFAULT_PROVIDER_SKU_LIST = "microsoft-qc/learn-and-develop"
TEST_TARGET_DEFAULT_PROVIDER = "microsoft-qc"
TEST_TARGET_DEFAULT_TARGET = "microsoft.estimator"
TEST_WORKSPACE_DEFAULT_PROVIDER_SKU_LIST = "Microsoft.ChemistryHpc/Basic"
TEST_CAPABILITIES_DEFAULT = "new.Microsoft.ChemistryHpc;submit.Microsoft.ChemistryHpc"
TEST_TARGET_DEFAULT_PROVIDER_SKU_LIST = "Microsoft.ChemistryHpc/Basic"
TEST_TARGET_DEFAULT_PROVIDER = "Microsoft.ChemistryHpc"
TEST_TARGET_DEFAULT_TARGET = "microsoft.hpc"


def get_from_os_environment(env_name, default):
Expand Down
66 changes: 66 additions & 0 deletions src/quantum/azext_quantum/vendored_sdks/azure_identity/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# ------------------------------------
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.
# ------------------------------------
"""Credentials for Azure SDK clients."""

from ._auth_record import AuthenticationRecord
from ._exceptions import AuthenticationRequiredError, CredentialUnavailableError
from ._constants import AzureAuthorityHosts, KnownAuthorities
from ._credentials import (
AuthorizationCodeCredential,
AzureDeveloperCliCredential,
AzureCliCredential,
AzurePowerShellCredential,
CertificateCredential,
ChainedTokenCredential,
ClientAssertionCredential,
ClientSecretCredential,
DefaultAzureCredential,
DeviceCodeCredential,
EnvironmentCredential,
InteractiveBrowserCredential,
ManagedIdentityCredential,
OnBehalfOfCredential,
SharedTokenCacheCredential,
UsernamePasswordCredential,
VisualStudioCodeCredential,
WorkloadIdentityCredential,
AzurePipelinesCredential,
)
from ._persistent_cache import TokenCachePersistenceOptions
from ._bearer_token_provider import get_bearer_token_provider


__all__ = [
"AuthenticationRecord",
"AuthenticationRequiredError",
"AuthorizationCodeCredential",
"AzureAuthorityHosts",
"AzureCliCredential",
"AzureDeveloperCliCredential",
"AzurePipelinesCredential",
"AzurePowerShellCredential",
"CertificateCredential",
"ChainedTokenCredential",
"ClientAssertionCredential",
"ClientSecretCredential",
"CredentialUnavailableError",
"DefaultAzureCredential",
"DeviceCodeCredential",
"EnvironmentCredential",
"InteractiveBrowserCredential",
"KnownAuthorities",
"OnBehalfOfCredential",
"ManagedIdentityCredential",
"SharedTokenCacheCredential",
"TokenCachePersistenceOptions",
"UsernamePasswordCredential",
"VisualStudioCodeCredential",
"WorkloadIdentityCredential",
"get_bearer_token_provider",
]

from ._version import VERSION

__version__ = VERSION
Loading
Loading