Add aml deployment packaging (#1090)

## Describe your changes

Add aml deployment packaging. AzureMLDeployment packaging will
[package](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-package-models?view=azureml-api-2&tabs=sdk)
ranked No. 1 model across all output models to Azure ML workspace, and
create an endpoint for it if the endpoint doesn't exist, then deploy the
output model to this endpoint.

## Checklist before requesting a review
- [ ] Add unit tests for this change.
- [ ] Make sure all tests can pass.
- [ ] Update documents if necessary.
- [ ] Lint and apply fixes to your code by running `lintrunner -a`
- [ ] Is this a user-facing change? If yes, give a description of this
change to be included in the release notes.
- [ ] Is this PR including examples changes? If yes, please remember to
update [example
documentation](https://github.com/microsoft/Olive/blob/main/docs/source/examples.md)
in a follow-up PR.

## (Optional) Issue link

docs/source/features/packaging_output_models.md

@@ -1,7 +1,7 @@
 # Packaging Olive artifacts
 
 ## What is Olive Packaging
-Olive will output multiple candidate models based on metrics priorities. It also can package output artifacts when the user requires. Olive packaging can be used in different scenarios. There are 3 packaging types: `Zipfile`, `AzureMLModels` and `AzureMLData`.
+Olive will output multiple candidate models based on metrics priorities. It also can package output artifacts when the user requires. Olive packaging can be used in different scenarios. There are 4 packaging types: `Zipfile`, `AzureMLModels`, `AzureMLData` and `AzureMLDeployment`.
 
 ### Zipfile
 Zipfile packaging will generate a ZIP file which includes 3 folders: `CandidateModels`, `SampleCode` and `ONNXRuntimePackages`, and a `models_rank.json` file in the `output_dir` folder (from Engine Configuration):
@@ -107,6 +107,9 @@ and for CPU, the best execution provider is CPUExecutionProvider, so the first r
 
 Olive will also upload model configuration file, inference config file, metrics file and model info file to the Azure ML.
 
+### AzureMLDeployment
+AzureMLDeployment packaging will [package](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-package-models?view=azureml-api-2&tabs=sdk) ranked No. 1 model across all output models to Azure ML workspace, and create an endpoint for it if the endpoint doesn't exist, then deploy the output model to this endpoint.
+
 ## How to package Olive artifacts
 Olive packaging configuration is configured in `PackagingConfig` in Engine configuration. `PackagingConfig` can be a single packging configuration. Alternatively, if you want to apply multiple packaging types, you can also define a list of packaging configurations.
 
@@ -138,12 +141,52 @@ If not specified, Olive will not package artifacts.
           The version for this data asset. This is `1` by default.
         * `description [str]`
           The description for this data asset. This is `None` by default.
+      * `AzureMLDeployment`
+        * `model_name [str]`:
+          The model name when registering your output model to your Azure ML workspace. `olive-deployment-model` by default
+        * `model_version [int | str]`:
+          The model version when registering your output model to your Azure ML workspace. Please note if there is already a model with the same name and the same version in your workspace, this will override your existing registered model. `1` by default.
+        * `description [str]`
+          The description for this model registration. This is `None` by default.
+        * `model_package [ModelPackageConfig]`:
+          The configurations for model packaging.
+          * `target_environment [str]`:
+            The environment name for the environment created by Olive. `olive-target-environment` by default.
+          * `target_environment_version [str]`
+            The environment version for the environment created by Olive. Please note if there is already an environment with the same name and the same version in your workspace, your existing environment version will plus 1. This `target_environment_version` will not be applied for your environment. `None` by default.
+          * `inferencing_server [InferenceServerConfig]`
+            * `type [str]`
+              The targeted inferencing server type. `AzureMLOnline` or `AzureMLBatch`.
+            * `code_folder [str]`
+              The folder path to your scoring script.
+            * `scoring_script [str]`
+              The scoring script name.
+          * `base_environment_id [str]`
+            The base environment id that will be used for Azure ML packaging. The format is `azureml:<base-environment-name>:<base-environment-version>`.
+          * `environment_variables [dict]`
+            Env vars that are required for the package to run, but not necessarily known at Environment creation time. `None` by default.
+        * `deployment_config [DeploymentConfig]`
+          The deployment configuration.
+          * `endpoint_name [str]`
+            The endpoint name for the deployment. If the endpoint doesn't exist, Olive will create one endpoint with this name. `olive-default-endpoint` by default.
+          * `deployment_name [str]`
+            The name of the deployment. `olive-default-deployment` by default.
+          * `instance_type [str]`
+            Azure compute sku. ManagedOnlineDeployment only. `None` by default.
+          * `compute [str]`
+            Compute target for batch inference operation. BatchDeployment only. `None` by default.
+          * `instance_count [str]`
+            Number of instances the interfering will run on. `1` by default.
+          * `mini_batch_size [str]`
+            Size of the mini-batch passed to each batch invocation. `10` by default.
+          * `extra_config [dict]`
+            Extra configurations for deployment. `None` by default.
     * `include_sample_code [bool]`:
       Whether or not to include sample code in zip file. Defaults to True
     * `include_runtime_packages [bool]`:
       Whether or not to include runtime packages (like onnxruntime) in zip file. Defaults to True
 
-You can add `PackagingConfig` to Engine configurations. e.g.:
+You can add different types `PackagingConfig` as a list to Engine configurations. e.g.:
 
 ```
 "engine": {
@@ -170,9 +213,21 @@ You can add `PackagingConfig` to Engine configurations. e.g.:
         {
             "type": "AzureMLData",
             "name": "OutputModels"
+        },
+        {
+            "type": "AzureMLDeployment",
+            "config": {
+                "model_package": {
+                    "inferencing_server": {
+                        "type": "AzureMLOnline",
+                        "code_folder": "code",
+                        "scoring_script": "score.py"
+                    },
+                    "base_environment_id": "azureml:olive-aml-packaging:1"
+                }
+            }
         }
     ]
-    "clean_cache": true,
     "cache_dir": "cache"
 }
 ```

olive/engine/footprint.py

@@ -101,7 +101,7 @@ def create_pareto_frontier(self, output_model_num: int = None) -> Optional["Foot
             logger.info("Output all %d models", len(self.nodes))
             return self._create_pareto_frontier_from_nodes(self.nodes)
         else:
-            topk_nodes = self._get_top_ranked_nodes(output_model_num)
+            topk_nodes = self.get_top_ranked_nodes(output_model_num)
             logger.info("Output top ranked %d models based on metric priorities", len(topk_nodes))
             return self._create_pareto_frontier_from_nodes(topk_nodes)
 
@@ -298,7 +298,7 @@ def _mark_pareto_frontier(self):
             self.nodes[k].is_pareto_frontier = cmp_flag
         self.is_marked_pareto_frontier = True
 
-    def _get_top_ranked_nodes(self, k: int) -> List[FootprintNode]:
+    def get_top_ranked_nodes(self, k: int) -> List[FootprintNode]:
         footprint_node_list = self.nodes.values()
         sorted_footprint_node_list = sorted(
             footprint_node_list,

olive/engine/packaging/packaging_config.py

@@ -15,6 +15,7 @@ class PackagingType(str, Enum):
     Zipfile = "Zipfile"
     AzureMLModels = "AzureMLModels"
     AzureMLData = "AzureMLData"
+    AzureMLDeployment = "AzureMLDeployment"
 
 
 class CommonPackagingConfig(ConfigBase):
@@ -35,10 +36,59 @@ class AzureMLModelsPackagingConfig(CommonPackagingConfig):
     description: Optional[str] = None
 
 
+class InferencingServerType(str, Enum):
+    AzureMLOnline = "AzureMLOnline"
+    AzureMLBatch = "AzureMLBatch"
+
+
+class InferenceServerConfig(ConfigBase):
+    type: InferencingServerType
+    code_folder: str
+    scoring_script: str
+
+
+class AzureMLModelModeType(str, Enum):
+    download = "download"
+    copy = "copy"
+
+
+class ModelConfigurationConfig(ConfigBase):
+    mode: AzureMLModelModeType = AzureMLModelModeType.download
+    mount_path: Optional[str] = None  # Relative mount path
+
+
+class ModelPackageConfig(ConfigBase):
+    target_environment: str = "olive-target-environment"
+    target_environment_version: Optional[str] = None
+    inferencing_server: InferenceServerConfig
+    base_environment_id: str
+    model_configurations: ModelConfigurationConfig = None
+    environment_variables: Optional[dict] = None
+
+
+class DeploymentConfig(ConfigBase):
+    endpoint_name: str = "olive-default-endpoint"
+    deployment_name: str = "olive-default-deployment"
+    instance_type: Optional[str] = None
+    compute: Optional[str] = None
+    instance_count: int = 1
+    mini_batch_size: int = 10  # AzureMLBatch only
+    extra_config: Optional[dict] = None
+
+
+class AzureMLDeploymentPackagingConfig(CommonPackagingConfig):
+    model_name: str = "olive-deployment-model"
+    model_version: Union[int, str] = "1"
+    model_description: Optional[str] = None
+    model_package: ModelPackageConfig
+    deployment_config: DeploymentConfig = DeploymentConfig()
+
+
 _type_to_config = {
     PackagingType.Zipfile: ZipfilePackagingConfig,
     PackagingType.AzureMLModels: AzureMLModelsPackagingConfig,
     PackagingType.AzureMLData: AzureMLDataPackagingConfig,
+    PackagingType.AzureMLDeployment: AzureMLDeploymentPackagingConfig,
 }