From 223399397133cf2b5b8192674fe45708a1fb6b55 Mon Sep 17 00:00:00 2001 From: Han Wang Date: Sat, 11 May 2024 15:35:16 +0800 Subject: [PATCH 1/7] [SDK] Gen meta before update signatures (#3195) # Description Please add an informative description that covers that changes made by the pull request and link all relevant issues. This pull request includes changes that primarily focus on modifying the handling of flow variants and initial keyword arguments (`init_kwargs`) in the `promptflow-devkit` package. The most significant changes involve removing the usage of `ProxyFactory` and its `prepare_metadata` method, adding `init_kwargs` parameter to various methods and functions, and modifying the `flow_overwrite_context` function to accommodate these changes. Here are the key changes, grouped by their themes: Removal of `ProxyFactory`: * [`src/promptflow-devkit/promptflow/_sdk/_orchestrator/run_submitter.py`](diffhunk://#diff-eb5aea25b8997068780361beb30138aaad86db060b97fe9673585ef258cd8b4bL11): Removed the import of `ProxyFactory` and its usage in `_submit_bulk_run` method. [[1]](diffhunk://#diff-eb5aea25b8997068780361beb30138aaad86db060b97fe9673585ef258cd8b4bL11) [[2]](diffhunk://#diff-eb5aea25b8997068780361beb30138aaad86db060b97fe9673585ef258cd8b4bL125-L130) Changes to `init_kwargs` handling: * [`src/promptflow-devkit/promptflow/_sdk/_orchestrator/run_submitter.py`](diffhunk://#diff-eb5aea25b8997068780361beb30138aaad86db060b97fe9673585ef258cd8b4bL111-R112): Added `init_kwargs` as a parameter to the `flow_overwrite_context` in the `_run_bulk` method. * [`src/promptflow-devkit/promptflow/_sdk/_orchestrator/test_submitter.py`](diffhunk://#diff-d5ded9a013487d37d3519fcb8c5433381c64ecc0d922629e87429b645a2e9d35L138-R138): Added `init_kwargs` as a parameter to the `_resolve_variant` method and used it in the `init` method. Removed the usage of `ProxyFactory` and its `prepare_metadata` method in the `init` method. [[1]](diffhunk://#diff-d5ded9a013487d37d3519fcb8c5433381c64ecc0d922629e87429b645a2e9d35L138-R138) [[2]](diffhunk://#diff-d5ded9a013487d37d3519fcb8c5433381c64ecc0d922629e87429b645a2e9d35R150) [[3]](diffhunk://#diff-d5ded9a013487d37d3519fcb8c5433381c64ecc0d922629e87429b645a2e9d35L225-L236) * [`src/promptflow-devkit/promptflow/_sdk/_orchestrator/utils.py`](diffhunk://#diff-bad1e9183cc8452c12df1c0932a6a10c7c8224382761a4a6702a8737981da5f9R216-R230): Added `init_kwargs` as a parameter to the `override_flow_yaml` and `flow_overwrite_context` functions. Also, moved the `prepare_metadata` method call to the `override_flow_yaml` function. [[1]](diffhunk://#diff-bad1e9183cc8452c12df1c0932a6a10c7c8224382761a4a6702a8737981da5f9R216-R230) [[2]](diffhunk://#diff-bad1e9183cc8452c12df1c0932a6a10c7c8224382761a4a6702a8737981da5f9R248) [[3]](diffhunk://#diff-bad1e9183cc8452c12df1c0932a6a10c7c8224382761a4a6702a8737981da5f9R268) [[4]](diffhunk://#diff-bad1e9183cc8452c12df1c0932a6a10c7c8224382761a4a6702a8737981da5f9R287) These changes seem to simplify the codebase by reducing the reliance on the `ProxyFactory` and its `prepare_metadata` method, and provide a more consistent way of handling `init_kwargs` across different methods and functions. # All Promptflow Contribution checklist: - [ ] **The pull request does not introduce [breaking changes].** - [ ] **CHANGELOG is updated for new features, bug fixes or other significant changes.** - [ ] **I have read the [contribution guidelines](../CONTRIBUTING.md).** - [ ] **Create an issue and link to the pull request to get dedicated review from promptflow team. Learn more: [suggested workflow](../CONTRIBUTING.md#suggested-workflow).** ## General Guidelines and Best Practices - [ ] Title of the pull request is clear and informative. - [ ] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, [see this page](https://github.com/Azure/azure-powershell/blob/master/documentation/development-docs/cleaning-up-commits.md). ### Testing Guidelines - [ ] Pull request includes test coverage for the included changes. --- .../promptflow/azure/_entities/_flow.py | 3 +-- .../_sdk/_orchestrator/run_submitter.py | 11 +++-------- .../_sdk/_orchestrator/test_submitter.py | 17 +++-------------- .../promptflow/_sdk/_orchestrator/utils.py | 15 +++++++++++++++ 4 files changed, 22 insertions(+), 24 deletions(-) diff --git a/src/promptflow-azure/promptflow/azure/_entities/_flow.py b/src/promptflow-azure/promptflow/azure/_entities/_flow.py index 7e471eb6344..150e5e05000 100644 --- a/src/promptflow-azure/promptflow/azure/_entities/_flow.py +++ b/src/promptflow-azure/promptflow/azure/_entities/_flow.py @@ -161,9 +161,8 @@ def _try_build_local_code(self) -> Optional[Code]: # generate .promptflow/flow.json for csharp flow as it's required to infer signature for csharp flow flow_directory, flow_file = resolve_flow_path(code.path) - # TODO: pass in init_kwargs to support csharp class init flex flow ProxyFactory().create_inspector_proxy(self.language).prepare_metadata( - flow_file=flow_directory / flow_file, working_dir=flow_directory + flow_file=flow_directory / flow_file, working_dir=flow_directory, init_kwargs=self._init_kwargs ) dag_updated = update_signatures(code=flow_dir, data=flow_dag) or dag_updated # validate init kwargs with signature diff --git a/src/promptflow-devkit/promptflow/_sdk/_orchestrator/run_submitter.py b/src/promptflow-devkit/promptflow/_sdk/_orchestrator/run_submitter.py index 37a73ced47d..9e83fc66478 100644 --- a/src/promptflow-devkit/promptflow/_sdk/_orchestrator/run_submitter.py +++ b/src/promptflow-devkit/promptflow/_sdk/_orchestrator/run_submitter.py @@ -8,7 +8,6 @@ from typing import Union from promptflow._constants import SystemMetricKeys -from promptflow._proxy import ProxyFactory from promptflow._sdk._constants import REMOTE_URI_PREFIX, ContextAttributeKey, FlowRunProperties from promptflow._sdk.entities._flows import Flow, Prompty from promptflow._sdk.entities._run import Run @@ -108,7 +107,9 @@ def _run_bulk(self, run: Run, stream=False, **kwargs): local_storage = LocalStorageOperations(run, stream=stream, run_mode=RunMode.Batch) with local_storage.logger: flow_obj = load_flow(source=run.flow) - with flow_overwrite_context(flow_obj, tuning_node, variant, connections=run.connections) as flow: + with flow_overwrite_context( + flow_obj, tuning_node, variant, connections=run.connections, init_kwargs=run.init + ) as flow: self._submit_bulk_run(flow=flow, run=run, local_storage=local_storage) @classmethod @@ -122,12 +123,6 @@ def _submit_bulk_run( ) -> dict: logger.info(f"Submitting run {run.name}, log path: {local_storage.logger.file_path}") run_id = run.name - # TODO: unify the logic for prompty and other flows - if not isinstance(flow, Prompty): - # variants are resolved in the context, so we can't move this logic to Operations for now - ProxyFactory().create_inspector_proxy(flow.language).prepare_metadata( - flow_file=Path(flow.path), working_dir=Path(flow.code), init_kwargs=run.init - ) with _change_working_dir(flow.code): # resolve connections with environment variables overrides to avoid getting unused connections diff --git a/src/promptflow-devkit/promptflow/_sdk/_orchestrator/test_submitter.py b/src/promptflow-devkit/promptflow/_sdk/_orchestrator/test_submitter.py index e0cf40858f2..06c5eaf3e50 100644 --- a/src/promptflow-devkit/promptflow/_sdk/_orchestrator/test_submitter.py +++ b/src/promptflow-devkit/promptflow/_sdk/_orchestrator/test_submitter.py @@ -135,10 +135,7 @@ def target_node(self) -> Optional[str]: return self._target_node @contextlib.contextmanager - def _resolve_variant(self): - # TODO(2901096): validate invalid configs like variant & connections - # no variant overwrite for eager flow - # no connection overwrite for eager flow + def _resolve_variant(self, init_kwargs=None): if self.flow_context.variant: tuning_node, node_variant = parse_variant(self.flow_context.variant) else: @@ -150,6 +147,7 @@ def _resolve_variant(self): variant=node_variant, connections=self.flow_context.connections, overrides=self.flow_context.overrides, + init_kwargs=init_kwargs, ) as temp_flow: # TODO execute flow test in a separate process. @@ -222,18 +220,9 @@ def init( :return: TestSubmitter instance. :rtype: TestSubmitter """ - with self._resolve_variant(): + with self._resolve_variant(init_kwargs=init_kwargs): # temp flow is generated, will use self.flow instead of self._origin_flow in the following context self._within_init_context = True - - if not isinstance(self.flow, Prompty): - # variant is resolve in the context, so we can't move this to Operations for now - ProxyFactory().create_inspector_proxy(self.flow.language).prepare_metadata( - flow_file=self.flow.path, - working_dir=self.flow.code, - init_kwargs=init_kwargs, - ) - self._target_node = target_node self._enable_stream_output = stream_output diff --git a/src/promptflow-devkit/promptflow/_sdk/_orchestrator/utils.py b/src/promptflow-devkit/promptflow/_sdk/_orchestrator/utils.py index 4d835cff802..5bbafc1f818 100644 --- a/src/promptflow-devkit/promptflow/_sdk/_orchestrator/utils.py +++ b/src/promptflow-devkit/promptflow/_sdk/_orchestrator/utils.py @@ -213,9 +213,21 @@ def override_flow_yaml( *, overrides: dict = None, drop_node_variants: bool = False, + init_kwargs: dict = None, ): + # generate meta before updating signatures since update signatures requires it. + if not isinstance(flow, Prompty): + ProxyFactory().create_inspector_proxy(flow.language).prepare_metadata( + flow_file=Path(flow.path), working_dir=Path(flow.code), init_kwargs=init_kwargs + ) if isinstance(flow, FlexFlow): # update signatures for flex flow + # no variant overwrite for eager flow + for param in [tuning_node, variant, connections, overrides]: + if param: + logger.warning( + "Eager flow does not support tuning node, variant, connection override. " f"Dropping params {param}" + ) update_signatures(code=flow_dir_path, data=flow_dag) else: # always overwrite variant since we need to overwrite default variant if not specified. @@ -233,6 +245,7 @@ def flow_overwrite_context( *, overrides: dict = None, drop_node_variants: bool = False, + init_kwargs: dict = None, ): """Override variant and connections in the flow.""" flow_dag = flow._data @@ -252,6 +265,7 @@ def flow_overwrite_context( connections=connections, overrides=overrides, drop_node_variants=drop_node_variants, + init_kwargs=init_kwargs, ) flow_dag.pop("additional_includes", None) dump_flow_dag_according_to_content(flow_dag=flow_dag, flow_path=Path(temp_dir)) @@ -270,6 +284,7 @@ def flow_overwrite_context( connections=connections, overrides=overrides, drop_node_variants=drop_node_variants, + init_kwargs=init_kwargs, ) flow_path = dump_flow_dag_according_to_content(flow_dag=flow_dag, flow_path=Path(temp_dir)) if isinstance(flow, FlexFlow): From 9d3a063103f30cf52cdedb7f80b1c2f382737b04 Mon Sep 17 00:00:00 2001 From: Brynn Yin <24237253+brynn-code@users.noreply.github.com> Date: Sat, 11 May 2024 15:36:53 +0800 Subject: [PATCH 2/7] [Connection] Return secrets in connection obj for better flex flow experience (#3216) # Description Please add an informative description that covers that changes made by the pull request and link all relevant issues. - Return the secrets in the connection object by default to improve flex flow experience. - Behaviors not changed: 'pfazure connection'/'pf connection' command will scrub secrets. - New behavior: connection object by `client.connection.get` will have real secrets. `print(connection_obj)` directly will scrub those secrets. `print(connection_obj.api_key)` or `print(connection_obj.secrets)` will print the REAL secrets. - Workspace listsecrets permission is required to get the Azure AI connection secrets. Call `client.connection.get(name, with_secrets=True)` if you want to get without the secrets and listsecrets permission. # All Promptflow Contribution checklist: - [ ] **The pull request does not introduce [breaking changes].** - [ ] **CHANGELOG is updated for new features, bug fixes or other significant changes.** - [ ] **I have read the [contribution guidelines](../CONTRIBUTING.md).** - [ ] **Create an issue and link to the pull request to get dedicated review from promptflow team. Learn more: [suggested workflow](../CONTRIBUTING.md#suggested-workflow).** ## General Guidelines and Best Practices - [ ] Title of the pull request is clear and informative. - [ ] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, [see this page](https://github.com/Azure/azure-powershell/blob/master/documentation/development-docs/cleaning-up-commits.md). ### Testing Guidelines - [ ] Pull request includes test coverage for the included changes. Signed-off-by: Brynn Yin --- src/promptflow-azure/CHANGELOG.md | 4 ++++ .../operations/_arm_connection_operations.py | 5 +++++ .../promptflow/core/_connection.py | 5 ----- src/promptflow-devkit/CHANGELOG.md | 3 +++ .../promptflow/_cli/_pf/_connection.py | 8 +++---- .../promptflow/_sdk/entities/_connection.py | 11 ++++++++++ .../_sdk/operations/_connection_operations.py | 4 ++-- .../_local_azure_connection_operations.py | 2 +- .../sdk_cli_test/e2etests/test_connection.py | 21 +++++++++++-------- .../test_custom_strong_type_connection.py | 18 +++++++++------- .../sdk_cli_test/unittests/test_connection.py | 4 ++-- src/promptflow/CHANGELOG.md | 2 ++ 12 files changed, 56 insertions(+), 31 deletions(-) diff --git a/src/promptflow-azure/CHANGELOG.md b/src/promptflow-azure/CHANGELOG.md index a27a5c9376d..9ab3e264239 100644 --- a/src/promptflow-azure/CHANGELOG.md +++ b/src/promptflow-azure/CHANGELOG.md @@ -4,6 +4,10 @@ ### Improvements - Refine trace Cosmos DB setup process to print setup status during the process, and display error message from service when setup failed. +- Return the secrets in the connection object by default to improve flex flow experience. + - Behaviors not changed: 'pfazure connection' command will scrub secrets. + - New behavior: connection object by `client.connection.get` will have real secrets. `print(connection_obj)` directly will scrub those secrets. `print(connection_obj.api_key)` or `print(connection_obj.secrets)` will print the REAL secrets. + - Workspace listsecrets permission is required to get the secrets. Call `client.connection.get(name, with_secrets=True)` if you want to get without the secrets and listsecrets permission. ## v1.10.0 (2024.04.26) diff --git a/src/promptflow-azure/promptflow/azure/operations/_arm_connection_operations.py b/src/promptflow-azure/promptflow/azure/operations/_arm_connection_operations.py index 7686504c066..8e9364b8fba 100644 --- a/src/promptflow-azure/promptflow/azure/operations/_arm_connection_operations.py +++ b/src/promptflow-azure/promptflow/azure/operations/_arm_connection_operations.py @@ -45,6 +45,11 @@ def __init__( ) def get(self, name, **kwargs): + with_secrets = kwargs.get("with_secrets", True) + if with_secrets: + return self._direct_get( + name, self._subscription_id, self._resource_group_name, self._workspace_name, self._credential + ) return _Connection._from_core_connection(self._provider.get(name)) @classmethod diff --git a/src/promptflow-core/promptflow/core/_connection.py b/src/promptflow-core/promptflow/core/_connection.py index bbf76b4b098..0faec530801 100644 --- a/src/promptflow-core/promptflow/core/_connection.py +++ b/src/promptflow-core/promptflow/core/_connection.py @@ -10,7 +10,6 @@ from promptflow._constants import CONNECTION_SCRUBBED_VALUE_NO_CHANGE, ConnectionType, CustomStrongTypeConnectionConfigs from promptflow._core.token_provider import AzureTokenProvider from promptflow._utils.logger_utils import LoggerFactory -from promptflow._utils.utils import in_jupyter_notebook from promptflow.constants import ConnectionAuthMode, ConnectionDefaultApiVersion from promptflow.contracts.types import Secret from promptflow.core._errors import RequiredEnvironmentVariablesNotSetError @@ -64,10 +63,6 @@ def __init__( self.expiry_time = kwargs.get("expiry_time", None) self.created_date = kwargs.get("created_date", None) self.last_modified_date = kwargs.get("last_modified_date", None) - # Conditional assignment to prevent entity bloat when unused. - print_as_yaml = kwargs.pop("print_as_yaml", in_jupyter_notebook()) - if print_as_yaml: - self.print_as_yaml = True def keys(self) -> List: """Return keys of the connection properties.""" diff --git a/src/promptflow-devkit/CHANGELOG.md b/src/promptflow-devkit/CHANGELOG.md index e70947b1a1b..1fa4efd1e3f 100644 --- a/src/promptflow-devkit/CHANGELOG.md +++ b/src/promptflow-devkit/CHANGELOG.md @@ -5,6 +5,9 @@ ### Improvements - Interactive browser credential is excluded by default when using Azure AI connections, user could set `PF_NO_INTERACTIVE_LOGIN=False` to enable it. - Visualize flex flow run(s) switches to trace UI page. +- Return the secrets in the connection object by default to improve flex flow experience. + - Behaviors not changed: 'pf connection' command will scrub secrets. + - New behavior: connection object by `client.connection.get` will have real secrets. `print(connection_obj)` directly will scrub those secrets. `print(connection_obj.api_key)` or `print(connection_obj.secrets)` will print the REAL secrets. ### Bugs Fixed - Fix the issue that import error will be raised after downgrading promptflow from >=1.10.0 to <1.8.0. diff --git a/src/promptflow-devkit/promptflow/_cli/_pf/_connection.py b/src/promptflow-devkit/promptflow/_cli/_pf/_connection.py index 5cb263207ac..df754c864e6 100644 --- a/src/promptflow-devkit/promptflow/_cli/_pf/_connection.py +++ b/src/promptflow-devkit/promptflow/_cli/_pf/_connection.py @@ -202,12 +202,12 @@ def create_connection(file_path, params_override=None, name=None): logger.warning(f"Connection with name {connection.name} already exists. Updating it.") # Note: We don't set the existing secret back here, let user input the secrets. validate_and_interactive_get_secrets(connection) - connection = _get_pf_client().connections.create_or_update(connection) + connection = _get_pf_client().connections.create_or_update(connection, with_secrets=False) print(json.dumps(connection._to_dict(), indent=4)) def show_connection(name): - connection = _get_pf_client().connections.get(name) + connection = _get_pf_client().connections.get(name, with_secrets=False) print(json.dumps(connection._to_dict(), indent=4)) @@ -229,7 +229,7 @@ def _upsert_connection_from_file(file, params_override=None): connection._secrets = existing_connection._secrets else: validate_and_interactive_get_secrets(connection) - connection = _get_pf_client().connections.create_or_update(connection) + connection = _get_pf_client().connections.create_or_update(connection, with_secrets=False) return connection @@ -240,7 +240,7 @@ def update_connection(name, params_override=None): validate_and_interactive_get_secrets(connection, is_update=True) # Set the secrets not scrubbed, as _to_dict() dump scrubbed connections. connection._secrets = existing_connection._secrets - connection = _get_pf_client().connections.create_or_update(connection) + connection = _get_pf_client().connections.create_or_update(connection, with_secrets=False) print(json.dumps(connection._to_dict(), indent=4)) diff --git a/src/promptflow-devkit/promptflow/_sdk/entities/_connection.py b/src/promptflow-devkit/promptflow/_sdk/entities/_connection.py index 857bf68be19..6289355d05b 100644 --- a/src/promptflow-devkit/promptflow/_sdk/entities/_connection.py +++ b/src/promptflow-devkit/promptflow/_sdk/entities/_connection.py @@ -2,6 +2,7 @@ # Copyright (c) Microsoft Corporation. All rights reserved. # --------------------------------------------------------- import abc +import copy import importlib import json from os import PathLike @@ -69,6 +70,16 @@ class _Connection(_CoreConnection, YAMLTranslatableMixin): SUPPORTED_TYPES = {} + def __str__(self): + """Override this function to scrub secrets in connection when print.""" + obj_for_dump = copy.deepcopy(self) + # Scrub secrets. + obj_for_dump.secrets = {k: SCRUBBED_VALUE for k in obj_for_dump.secrets} + try: + return obj_for_dump._to_yaml() + except BaseException: # pylint: disable=broad-except + return super(YAMLTranslatableMixin, self).__str__() + @classmethod def _casting_type(cls, typ): type_dict = { diff --git a/src/promptflow-devkit/promptflow/_sdk/operations/_connection_operations.py b/src/promptflow-devkit/promptflow/_sdk/operations/_connection_operations.py index ba1e651ccae..d39715e3792 100644 --- a/src/promptflow-devkit/promptflow/_sdk/operations/_connection_operations.py +++ b/src/promptflow-devkit/promptflow/_sdk/operations/_connection_operations.py @@ -55,7 +55,7 @@ def get(self, name: str, **kwargs) -> _Connection: return self._get(name, **kwargs) def _get(self, name: str, **kwargs) -> _Connection: - with_secrets = kwargs.get("with_secrets", False) + with_secrets = kwargs.get("with_secrets", True) raise_error = kwargs.get("raise_error", True) orm_connection = ORMConnection.get(name, raise_error) if orm_connection is None: @@ -90,4 +90,4 @@ def create_or_update(self, connection: Type[_Connection], **kwargs): orm_object.createdDate = now orm_object.lastModifiedDate = now ORMConnection.create_or_update(orm_object) - return self.get(connection.name) + return self.get(connection.name, **kwargs) diff --git a/src/promptflow-devkit/promptflow/_sdk/operations/_local_azure_connection_operations.py b/src/promptflow-devkit/promptflow/_sdk/operations/_local_azure_connection_operations.py index 413de20a223..89b27c0a81f 100644 --- a/src/promptflow-devkit/promptflow/_sdk/operations/_local_azure_connection_operations.py +++ b/src/promptflow-devkit/promptflow/_sdk/operations/_local_azure_connection_operations.py @@ -118,7 +118,7 @@ def _get(self, name: str, **kwargs) -> _Connection: :return: connection object retrieved from Azure. :rtype: ~promptflow.sdk.entities._connection._Connection """ - with_secrets = kwargs.get("with_secrets", False) + with_secrets = kwargs.get("with_secrets", True) if with_secrets: # Do not use pfazure_client here as it requires workspace read permission # Get secrets from arm only requires workspace listsecrets permission diff --git a/src/promptflow-devkit/tests/sdk_cli_test/e2etests/test_connection.py b/src/promptflow-devkit/tests/sdk_cli_test/e2etests/test_connection.py index 3e3b12847a0..30c8aaef729 100644 --- a/src/promptflow-devkit/tests/sdk_cli_test/e2etests/test_connection.py +++ b/src/promptflow-devkit/tests/sdk_cli_test/e2etests/test_connection.py @@ -6,7 +6,6 @@ from _constants import PROMPTFLOW_ROOT from mock import mock -from promptflow._sdk._constants import SCRUBBED_VALUE from promptflow._sdk._errors import ConnectionNameNotSetError from promptflow._sdk._pf_client import PFClient from promptflow._sdk.entities import AzureOpenAIConnection, CustomConnection, OpenAIConnection @@ -30,7 +29,7 @@ def test_connection_operations(self): assert pydash.omit(result._to_dict(), ["created_date", "last_modified_date", "name"]) == { "module": "promptflow.connections", "type": "azure_open_ai", - "api_key": "******", + "api_key": "test", # get return real key now "auth_mode": "key", "api_base": "test", "api_type": "azure", @@ -42,7 +41,7 @@ def test_connection_operations(self): assert pydash.omit(result._to_dict(), ["created_date", "last_modified_date", "name"]) == { "module": "promptflow.connections", "type": "azure_open_ai", - "api_key": "******", + "api_key": "test", # get return real key now "auth_mode": "key", "api_base": "test2", "api_type": "azure", @@ -60,38 +59,42 @@ def test_connection_operations(self): def test_connection_get_and_update(self): # Test api key not updated name = f"Connection_{str(uuid.uuid4())[:4]}" - conn = AzureOpenAIConnection(name=name, api_key="test", api_base="test") + conn = AzureOpenAIConnection(name=name, api_key="test_key", api_base="test") result = _client.connections.create_or_update(conn) - assert result.api_key == SCRUBBED_VALUE + assert result.api_key == "test_key" + assert "test_key" not in str(result) # Assert key scrubbed when print # Update api_base only Assert no exception result.api_base = "test2" result = _client.connections.create_or_update(result) assert result._to_dict()["api_base"] == "test2" # Assert value not scrubbed - assert result._secrets["api_key"] == "test" + assert result._secrets["api_key"] == "test_key" _client.connections.delete(name) # Invalid update with pytest.raises(Exception) as e: result._secrets = {} + result.secrets["api_key"] = "****" _client.connections.create_or_update(result) assert "secrets ['api_key'] value invalid, please fill them" in str(e.value) def test_custom_connection_get_and_update(self): # Test api key not updated name = f"Connection_{str(uuid.uuid4())[:4]}" - conn = CustomConnection(name=name, secrets={"api_key": "test"}, configs={"api_base": "test"}) + conn = CustomConnection(name=name, secrets={"api_key": "test_key"}, configs={"api_base": "test"}) result = _client.connections.create_or_update(conn) - assert result.secrets["api_key"] == SCRUBBED_VALUE + assert "test_key" not in str(result) # Assert key scrubbed when print + assert result.secrets["api_key"] == "test_key" # Update api_base only Assert no exception result.configs["api_base"] = "test2" result = _client.connections.create_or_update(result) assert result._to_dict()["configs"]["api_base"] == "test2" # Assert value not scrubbed - assert result._secrets["api_key"] == "test" + assert result._secrets["api_key"] == "test_key" _client.connections.delete(name) # Invalid update with pytest.raises(Exception) as e: result._secrets = {} + result.secrets["api_key"] = "****" _client.connections.create_or_update(result) assert "secrets ['api_key'] value invalid, please fill them" in str(e.value) diff --git a/src/promptflow-devkit/tests/sdk_cli_test/e2etests/test_custom_strong_type_connection.py b/src/promptflow-devkit/tests/sdk_cli_test/e2etests/test_custom_strong_type_connection.py index ea485d20f4e..22d752fa644 100644 --- a/src/promptflow-devkit/tests/sdk_cli_test/e2etests/test_custom_strong_type_connection.py +++ b/src/promptflow-devkit/tests/sdk_cli_test/e2etests/test_custom_strong_type_connection.py @@ -4,7 +4,7 @@ import pytest from _constants import PROMPTFLOW_ROOT -from promptflow._sdk._constants import SCRUBBED_VALUE, CustomStrongTypeConnectionConfigs +from promptflow._sdk._constants import CustomStrongTypeConnectionConfigs from promptflow._sdk._pf_client import PFClient from promptflow._sdk.entities import CustomStrongTypeConnection from promptflow.contracts.types import Secret @@ -40,7 +40,7 @@ def test_connection_operations(self): "promptflow.connection.custom_type": "MyCustomConnection", "promptflow.connection.module": "sdk_cli_test.e2etests.test_custom_strong_type_connection", }, - "secrets": {"api_key": "******"}, + "secrets": {"api_key": "test"}, } # Update conn.configs["api_base"] = "test2" @@ -53,7 +53,7 @@ def test_connection_operations(self): "promptflow.connection.custom_type": "MyCustomConnection", "promptflow.connection.module": "sdk_cli_test.e2etests.test_custom_strong_type_connection", }, - "secrets": {"api_key": "******"}, + "secrets": {"api_key": "test"}, } # List result = _client.connections.list() @@ -79,7 +79,7 @@ def test_connection_update(self): "promptflow.connection.custom_type": "MyCustomConnection", "promptflow.connection.module": "sdk_cli_test.e2etests.test_custom_strong_type_connection", }, - "secrets": {"api_key": "******"}, + "secrets": {"api_key": "test"}, } # Update custom_conn.configs["api_base"] = "test2" @@ -92,7 +92,7 @@ def test_connection_update(self): "promptflow.connection.custom_type": "MyCustomConnection", "promptflow.connection.module": "sdk_cli_test.e2etests.test_custom_strong_type_connection", }, - "secrets": {"api_key": "******"}, + "secrets": {"api_key": "test"}, } # List result = _client.connections.list() @@ -106,19 +106,21 @@ def test_connection_update(self): def test_connection_get_and_update(self): # Test api key not updated name = f"Connection_{str(uuid.uuid4())[:4]}" - conn = MyCustomConnection(name=name, secrets={"api_key": "test"}, configs={"api_base": "test"}) + conn = MyCustomConnection(name=name, secrets={"api_key": "test_key"}, configs={"api_base": "test"}) result = _client.connections.create_or_update(conn) - assert result.secrets["api_key"] == SCRUBBED_VALUE + assert result.secrets["api_key"] == "test_key" + assert "test_key" not in str(result) # Assert key scrubbed when print # Update api_base only Assert no exception result.configs["api_base"] = "test2" result = _client.connections.create_or_update(result) assert result._to_dict()["configs"]["api_base"] == "test2" # Assert value not scrubbed - assert result._secrets["api_key"] == "test" + assert result._secrets["api_key"] == "test_key" _client.connections.delete(name) # Invalid update with pytest.raises(Exception) as e: result._secrets = {} + result.secrets["api_key"] = "****" _client.connections.create_or_update(result) assert "secrets ['api_key'] value invalid, please fill them" in str(e.value) diff --git a/src/promptflow-devkit/tests/sdk_cli_test/unittests/test_connection.py b/src/promptflow-devkit/tests/sdk_cli_test/unittests/test_connection.py index d8ace1f6052..838663d8cc7 100644 --- a/src/promptflow-devkit/tests/sdk_cli_test/unittests/test_connection.py +++ b/src/promptflow-devkit/tests/sdk_cli_test/unittests/test_connection.py @@ -288,8 +288,8 @@ def test_connection_load_from_env(self): type: custom configs: {} secrets: - aaa: bbb - ccc: ddd + aaa: '******' + ccc: '******' """ ) diff --git a/src/promptflow/CHANGELOG.md b/src/promptflow/CHANGELOG.md index 4c21581beec..61eac63b1a4 100644 --- a/src/promptflow/CHANGELOG.md +++ b/src/promptflow/CHANGELOG.md @@ -5,6 +5,8 @@ ### Improvements - [promptflow-devkit]: Interactive browser credential is excluded by default when using Azure AI connections, user could set `PF_NO_INTERACTIVE_LOGIN=False` to enable it. - [promptflow-azure]: Refine trace Cosmos DB setup process to print setup status during the process, and display error message from service when setup failed. +- [promptflow-devkit][promptflow-azure] - Return the secrets in the connection object by default to improve flex flow experience. + - Reach the sub package docs for more details about this. [promptflow-devkit](https://microsoft.github.io/promptflow/reference/changelog/promptflow-devkit.html) [promptflow-azure](https://microsoft.github.io/promptflow/reference/changelog/promptflow-azure.html) ### Bugs Fixed - Fix the issue that import error will be raised after downgrading promptflow from >=1.10.0 to <1.8.0. From dbd9b42aef1eaca3aa6d9411b630297cd2ccd7f0 Mon Sep 17 00:00:00 2001 From: Xiaopeng Wang Date: Sat, 11 May 2024 16:57:13 +0800 Subject: [PATCH 3/7] support serving engine switch in pf serve cmd (#3161) # Description Add --engine parameter in `pf flow serve` command, customer can use this parameter to switch the python serving engine between `flask` and `fastapi`, it's default to `flask` # All Promptflow Contribution checklist: - [x] **The pull request does not introduce [breaking changes].** - [x] **CHANGELOG is updated for new features, bug fixes or other significant changes.** - [ ] **I have read the [contribution guidelines](../CONTRIBUTING.md).** - [ ] **Create an issue and link to the pull request to get dedicated review from promptflow team. Learn more: [suggested workflow](../CONTRIBUTING.md#suggested-workflow).** ## General Guidelines and Best Practices - [ ] Title of the pull request is clear and informative. - [ ] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, [see this page](https://github.com/Azure/azure-powershell/blob/master/documentation/development-docs/cleaning-up-commits.md). ### Testing Guidelines - [ ] Pull request includes test coverage for the included changes. --------- Co-authored-by: xiaopwan --- docs/reference/pf-command-reference.md | 11 +++++++ src/promptflow-devkit/CHANGELOG.md | 1 + .../promptflow/_cli/_pf/_flow.py | 5 +++ .../promptflow/_sdk/_utilities/serve_utils.py | 21 +++++++++++- .../unittests/test_flow_serve_cli.py | 33 +++++++++++++++++++ src/promptflow/CHANGELOG.md | 1 + 6 files changed, 71 insertions(+), 1 deletion(-) diff --git a/docs/reference/pf-command-reference.md b/docs/reference/pf-command-reference.md index 0b9c623add8..076e5f4882b 100644 --- a/docs/reference/pf-command-reference.md +++ b/docs/reference/pf-command-reference.md @@ -294,6 +294,7 @@ pf flow serve --source [--verbose] [--debug] [--skip-open-browser] + [--engine] ``` #### Examples @@ -310,6 +311,12 @@ Serve flow as an endpoint with specific port and host. pf flow serve --source --port --host --environment-variables key1="`${my_connection.api_key}`" key2="value2" ``` +Serve flow as an endpoint with specific port, host, environment-variables and fastapi serving engine. + +```bash +pf flow serve --source --port --host --environment-variables key1="`${my_connection.api_key}`" key2="value2" --engine fastapi +``` + #### Required Parameter `--source` @@ -342,6 +349,10 @@ Show debug information during serve. Skip opening browser after serve. Store true parameter. +`--engine` + +Switch python serving engine between `flask` amd `fastapi`, default to `flask`. + ## pf connection Manage prompt flow connections. diff --git a/src/promptflow-devkit/CHANGELOG.md b/src/promptflow-devkit/CHANGELOG.md index 1fa4efd1e3f..7f0401a9889 100644 --- a/src/promptflow-devkit/CHANGELOG.md +++ b/src/promptflow-devkit/CHANGELOG.md @@ -5,6 +5,7 @@ ### Improvements - Interactive browser credential is excluded by default when using Azure AI connections, user could set `PF_NO_INTERACTIVE_LOGIN=False` to enable it. - Visualize flex flow run(s) switches to trace UI page. +- Add new `--engine` parameter for `pf flow serve`. This parameter can be used to switch python serving engine between `flask` and `fastapi`, currently it defaults to `flask`. - Return the secrets in the connection object by default to improve flex flow experience. - Behaviors not changed: 'pf connection' command will scrub secrets. - New behavior: connection object by `client.connection.get` will have real secrets. `print(connection_obj)` directly will scrub those secrets. `print(connection_obj.api_key)` or `print(connection_obj.secrets)` will print the REAL secrets. diff --git a/src/promptflow-devkit/promptflow/_cli/_pf/_flow.py b/src/promptflow-devkit/promptflow/_cli/_pf/_flow.py index 29e316ed73e..d0e1b348585 100644 --- a/src/promptflow-devkit/promptflow/_cli/_pf/_flow.py +++ b/src/promptflow-devkit/promptflow/_cli/_pf/_flow.py @@ -199,6 +199,9 @@ def add_parser_serve_flow(subparsers): add_param_skip_browser = lambda parser: parser.add_argument( # noqa: E731 "--skip-open-browser", action="store_true", default=False, help="Skip open browser for flow serving." ) + add_param_engine = lambda parser: parser.add_argument( # noqa: E731 + "--engine", type=str, default="flask", help="The engine to serve the flow, can be flask or fastapi." + ) activate_action( name="serve", description="Serving a flow as an endpoint.", @@ -207,6 +210,7 @@ def add_parser_serve_flow(subparsers): add_param_source, add_param_port, add_param_host, + add_param_engine, add_param_static_folder, add_param_environment_variables, add_param_config, @@ -595,6 +599,7 @@ def serve_flow(args): host=args.host, port=args.port, skip_open_browser=args.skip_open_browser, + engine=args.engine, ) logger.info("Promptflow app ended") diff --git a/src/promptflow-devkit/promptflow/_sdk/_utilities/serve_utils.py b/src/promptflow-devkit/promptflow/_sdk/_utilities/serve_utils.py index cb6414900d4..7d41324a708 100644 --- a/src/promptflow-devkit/promptflow/_sdk/_utilities/serve_utils.py +++ b/src/promptflow-devkit/promptflow/_sdk/_utilities/serve_utils.py @@ -58,6 +58,7 @@ def start_flow_service( environment_variables: Dict[str, str] = None, init: Dict[str, Any] = None, skip_open_browser: bool = True, + engine: str = "flask", ): logger.info( "Start promptflow server with port %s", @@ -72,6 +73,11 @@ def start_flow_service( message_format="Support directory `source` for Python flow only for now, but got {source}.", source=source, ) + if engine not in ["flask", "fastapi"]: + raise UserErrorException( + message_format="Unsupported engine {engine} for Python flow, only support 'flask' and 'fastapi'.", + engine=engine, + ) serve_python_flow( flow_file_name=flow_file_name, flow_dir=flow_dir, @@ -82,6 +88,7 @@ def start_flow_service( config=config or {}, environment_variables=environment_variables or {}, skip_open_browser=skip_open_browser, + engine=engine, ) else: serve_csharp_flow( @@ -103,6 +110,7 @@ def serve_python_flow( environment_variables, init, skip_open_browser: bool, + engine, ): from promptflow._sdk._configuration import Configuration from promptflow.core._serving.app import create_app @@ -121,13 +129,24 @@ def serve_python_flow( environment_variables=environment_variables, connection_provider=connection_provider, init=init, + engine=engine, ) if not skip_open_browser: target = f"http://{host}:{port}" logger.info(f"Opening browser {target}...") webbrowser.open(target) # Debug is not supported for now as debug will rerun command, and we changed working directory. - app.run(port=port, host=host) + if engine == "flask": + app.run(port=port, host=host) + else: + try: + import uvicorn + + uvicorn.run(app, host=host, port=port, access_log=False, log_config=None) + except ImportError: + raise UserErrorException( + message_format="FastAPI engine requires uvicorn, please install uvicorn by `pip install uvicorn`." + ) @contextlib.contextmanager diff --git a/src/promptflow-devkit/tests/sdk_cli_test/unittests/test_flow_serve_cli.py b/src/promptflow-devkit/tests/sdk_cli_test/unittests/test_flow_serve_cli.py index 3a2897e06dc..fc7aa6bdd1b 100644 --- a/src/promptflow-devkit/tests/sdk_cli_test/unittests/test_flow_serve_cli.py +++ b/src/promptflow-devkit/tests/sdk_cli_test/unittests/test_flow_serve_cli.py @@ -49,6 +49,17 @@ def test_flow_serve(self, source: Path): "--skip-open-browser", ) mock_run.assert_called_once_with(port=8080, host="localhost") + with mock.patch("uvicorn.run") as mock_run: + run_pf_command( + "flow", + "serve", + "--source", + source.as_posix(), + "--skip-open-browser", + "--engine", + "fastapi", + ) + mock_run.assert_called_once() @pytest.mark.parametrize( "source", @@ -71,3 +82,25 @@ def test_flow_serve_failed(self, source: Path, capsys): "pf.flow.serve failed with UserErrorException: Support directory `source` for Python flow only for now" in out ) + + @pytest.mark.parametrize( + "source", + [ + pytest.param(EAGER_FLOWS_DIR / "simple_with_yaml", id="simple_with_yaml_file"), + pytest.param(FLOWS_DIR / "simple_hello_world", id="simple_hello_world_file"), + ], + ) + def test_flow_serve_invalid_engine(self, source: Path, capsys): + invalid_engine = "invalid_engine" + with pytest.raises(SystemExit): + run_pf_command( + "flow", + "serve", + "--source", + source.as_posix(), + "--skip-open-browser", + "--engine", + invalid_engine, + ) + out, err = capsys.readouterr() + assert f"Unsupported engine {invalid_engine} for Python flow, only support 'flask' and 'fastapi'." in out diff --git a/src/promptflow/CHANGELOG.md b/src/promptflow/CHANGELOG.md index 61eac63b1a4..b4c261eb1a7 100644 --- a/src/promptflow/CHANGELOG.md +++ b/src/promptflow/CHANGELOG.md @@ -4,6 +4,7 @@ ### Improvements - [promptflow-devkit]: Interactive browser credential is excluded by default when using Azure AI connections, user could set `PF_NO_INTERACTIVE_LOGIN=False` to enable it. +- [promptflow-devkit]: Add new `--engine` parameter for `pf flow serve`. This parameter can be used to switch python serving engine between `flask` and `fastapi`, currently it defaults to `flask`. - [promptflow-azure]: Refine trace Cosmos DB setup process to print setup status during the process, and display error message from service when setup failed. - [promptflow-devkit][promptflow-azure] - Return the secrets in the connection object by default to improve flex flow experience. - Reach the sub package docs for more details about this. [promptflow-devkit](https://microsoft.github.io/promptflow/reference/changelog/promptflow-devkit.html) [promptflow-azure](https://microsoft.github.io/promptflow/reference/changelog/promptflow-azure.html) From 5c3a5c2385d42908a38954182d53001a246a12bb Mon Sep 17 00:00:00 2001 From: Heyi Tang Date: Sat, 11 May 2024 17:53:21 +0800 Subject: [PATCH 4/7] [Internal][tracing] Refactor span enrich logic to make it more clear. (#3056) # Description Refactor span enrich logic to make it more clear. This pull request introduces a new structure for enriching spans in the tracing module of the `promptflow-tracing` package. The changes primarily involve the creation of a `SpanEnricher` class and a `SpanEnricherManager` singleton class in the `src/promptflow-tracing/promptflow/tracing/_span_enricher.py` file. These classes are used to enrich spans with inputs and outputs of traced functions. Two specific enrichers, `LLMSpanEnricher` and `EmbeddingSpanEnricher`, are created and registered with the manager. The `TraceType` enum is also updated to include a new `RETRIEVAL` type. Key changes include: * [`src/promptflow-tracing/promptflow/tracing/_span_enricher.py`](diffhunk://#diff-695282eb7072d912e1bbe30979ce4b19b7c4ffe45a76fd64d48d1c392d4f5c25R1-R49): Introduced `SpanEnricher` and `SpanEnricherManager` classes. The manager is a singleton that maintains a dictionary mapping trace types to their respective span enrichers. The base `SpanEnricher` class provides a method to enrich a span with the output of a traced function. * [`src/promptflow-tracing/promptflow/tracing/_trace.py`](diffhunk://#diff-580941184737186a94e3e9c06e467e86c06ce846d30ffa42c1c43b45812ddcdcR23): Imported the `SpanEnricher` and `SpanEnricherManager` classes. The `enrich_span_with_trace_type` function now uses the `SpanEnricherManager` to enrich spans based on their trace type. Two specific enrichers, `LLMSpanEnricher` and `EmbeddingSpanEnricher`, are implemented and registered with the `SpanEnricherManager`. [[1]](diffhunk://#diff-580941184737186a94e3e9c06e467e86c06ce846d30ffa42c1c43b45812ddcdcR23) [[2]](diffhunk://#diff-580941184737186a94e3e9c06e467e86c06ce846d30ffa42c1c43b45812ddcdcL150-R154) [[3]](diffhunk://#diff-580941184737186a94e3e9c06e467e86c06ce846d30ffa42c1c43b45812ddcdcR515-R532) * [`src/promptflow-tracing/promptflow/tracing/contracts/trace.py`](diffhunk://#diff-064c539f55a396428c9d28af1cbcf4fe39beb69daac7d9891cd6f713897fa704R18): Added `RETRIEVAL` as a new trace type to the `TraceType` enum. These changes aim to make the process of enriching spans more flexible and extensible, allowing for different enrichment logic based on the trace type. # All Promptflow Contribution checklist: - [ ] **The pull request does not introduce [breaking changes].** - [ ] **CHANGELOG is updated for new features, bug fixes or other significant changes.** - [ ] **I have read the [contribution guidelines](../CONTRIBUTING.md).** - [ ] **Create an issue and link to the pull request to get dedicated review from promptflow team. Learn more: [suggested workflow](../CONTRIBUTING.md#suggested-workflow).** ## General Guidelines and Best Practices - [ ] Title of the pull request is clear and informative. - [ ] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, [see this page](https://github.com/Azure/azure-powershell/blob/master/documentation/development-docs/cleaning-up-commits.md). ### Testing Guidelines - [ ] Pull request includes test coverage for the included changes. Co-authored-by: Heyi --- .../promptflow/tracing/_span_enricher.py | 49 +++++++++++++++++++ .../promptflow/tracing/_trace.py | 32 ++++++++---- .../promptflow/tracing/contracts/trace.py | 1 + 3 files changed, 72 insertions(+), 10 deletions(-) create mode 100644 src/promptflow-tracing/promptflow/tracing/_span_enricher.py diff --git a/src/promptflow-tracing/promptflow/tracing/_span_enricher.py b/src/promptflow-tracing/promptflow/tracing/_span_enricher.py new file mode 100644 index 00000000000..da4b4f14275 --- /dev/null +++ b/src/promptflow-tracing/promptflow/tracing/_span_enricher.py @@ -0,0 +1,49 @@ +from typing import Dict + +from .contracts.trace import TraceType + + +class SpanEnricher: + def __init__(self): + pass + + def enrich(self, span, inputs, output): + """This method is used to enrich the span with the inputs and output of the traced function. + Note that this method is called after the function is called, so some inputs related logic is not here. + """ + # TODO: Also move input related logic here. + from ._trace import enrich_span_with_output + + enrich_span_with_output(span, output) + + +class SpanEnricherManager: + _instance = None + + def __init__(self): + self._type2enricher: Dict[str, SpanEnricher] = {} + self._base_enricher = SpanEnricher() + + @classmethod + def get_instance(cls) -> "SpanEnricherManager": + if cls._instance is None: + cls._instance = SpanEnricherManager() + return cls._instance + + @classmethod + def register(cls, trace_type, enricher: SpanEnricher): + cls.get_instance()._register(trace_type, enricher) + + @classmethod + def enrich(cls, span, inputs, output, trace_type): + cls.get_instance()._enrich(span, inputs, output, trace_type) + + def _register(self, trace_type, enricher: SpanEnricher): + self._type2enricher[trace_type] = enricher + + def _enrich(self, span, inputs, output, trace_type): + enricher = self._type2enricher.get(trace_type, self._base_enricher) + enricher.enrich(span, inputs, output) + + +SpanEnricherManager.register(TraceType.FUNCTION, SpanEnricher()) diff --git a/src/promptflow-tracing/promptflow/tracing/_trace.py b/src/promptflow-tracing/promptflow/tracing/_trace.py index ba9d56ab15d..9d93710f79c 100644 --- a/src/promptflow-tracing/promptflow/tracing/_trace.py +++ b/src/promptflow-tracing/promptflow/tracing/_trace.py @@ -21,6 +21,7 @@ from ._openai_utils import OpenAIMetricsCalculator, OpenAIResponseParser from ._operation_context import OperationContext +from ._span_enricher import SpanEnricher, SpanEnricherManager from ._tracer import Tracer, _create_trace_from_function_call, get_node_name_from_context from ._utils import get_input_names_for_prompt_template, get_prompt_param_name_from_func, serialize from .contracts.generator_proxy import AsyncGeneratorProxy, GeneratorProxy @@ -148,17 +149,10 @@ def enrich_span_with_input(span, input): def enrich_span_with_trace_type(span, inputs, output, trace_type): - if trace_type == TraceType.LLM: - # Handle the non-streaming output of LLM, the streaming output will be handled in traced_generator. - token_collector.collect_openai_tokens(span, output) - enrich_span_with_llm_output(span, output) - elif trace_type == TraceType.EMBEDDING: - token_collector.collect_openai_tokens(span, output) - enrich_span_with_embedding(span, inputs, output) + SpanEnricherManager.enrich(span, inputs, output, trace_type) + # TODO: Move the following logic to SpanEnricher enrich_span_with_openai_tokens(span, trace_type) - enrich_span_with_output(span, output) - output = trace_iterator_if_needed(span, inputs, output) - return output + return trace_iterator_if_needed(span, inputs, output) def trace_iterator_if_needed(span, inputs, output): @@ -519,3 +513,21 @@ async def greetings_async(user_id): """ return _traced(func, trace_type=TraceType.FUNCTION) + + +class LLMSpanEnricher(SpanEnricher): + def enrich(self, span, inputs, output): + token_collector.collect_openai_tokens(span, output) + enrich_span_with_llm_output(span, output) + super().enrich(span, inputs, output) + + +class EmbeddingSpanEnricher(SpanEnricher): + def enrich(self, span, inputs, output): + token_collector.collect_openai_tokens(span, output) + enrich_span_with_embedding(span, inputs, output) + super().enrich(span, inputs, output) + + +SpanEnricherManager.register(TraceType.LLM, LLMSpanEnricher()) +SpanEnricherManager.register(TraceType.EMBEDDING, EmbeddingSpanEnricher()) diff --git a/src/promptflow-tracing/promptflow/tracing/contracts/trace.py b/src/promptflow-tracing/promptflow/tracing/contracts/trace.py index 3003967314a..980e3f7d9c2 100644 --- a/src/promptflow-tracing/promptflow/tracing/contracts/trace.py +++ b/src/promptflow-tracing/promptflow/tracing/contracts/trace.py @@ -15,6 +15,7 @@ class TraceType(str, Enum): LANGCHAIN = "LangChain" FLOW = "Flow" EMBEDDING = "Embedding" + RETRIEVAL = "Retrieval" @dataclass From 2cf0fbef18ba46aaa1bc81e875cb5a672de4a0f2 Mon Sep 17 00:00:00 2001 From: Peiwen Gao <111329184+PeiwenGaoMS@users.noreply.github.com> Date: Sat, 11 May 2024 17:54:34 +0800 Subject: [PATCH 5/7] [Internal][Executor] Return inputs definition and has aggregation by initialize api in execution server (#3212) # Description Return inputs definition and has aggregation by initialize api in execution server to avoid creating flow executor in runtime container. # All Promptflow Contribution checklist: - [x] **The pull request does not introduce [breaking changes].** - [ ] **CHANGELOG is updated for new features, bug fixes or other significant changes.** - [x] **I have read the [contribution guidelines](../CONTRIBUTING.md).** - [ ] **Create an issue and link to the pull request to get dedicated review from promptflow team. Learn more: [suggested workflow](../CONTRIBUTING.md#suggested-workflow).** ## General Guidelines and Best Practices - [x] Title of the pull request is clear and informative. - [x] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, [see this page](https://github.com/Azure/azure-powershell/blob/master/documentation/development-docs/cleaning-up-commits.md). ### Testing Guidelines - [ ] Pull request includes test coverage for the included changes. --- .../promptflow/executor/_prompty_executor.py | 3 + .../executor/_service/apis/batch.py | 10 +- .../_service/utils/batch_coordinator.py | 6 + .../promptflow/executor/flow_executor.py | 3 + .../executor/_service/apis/test_batch.py | 108 ++++++++++++++++++ src/promptflow/tests/executor/utils.py | 34 ++++-- 6 files changed, 151 insertions(+), 13 deletions(-) create mode 100644 src/promptflow/tests/executor/unittests/executor/_service/apis/test_batch.py diff --git a/src/promptflow-core/promptflow/executor/_prompty_executor.py b/src/promptflow-core/promptflow/executor/_prompty_executor.py index d8a7abed15b..a38e4ae9699 100644 --- a/src/promptflow-core/promptflow/executor/_prompty_executor.py +++ b/src/promptflow-core/promptflow/executor/_prompty_executor.py @@ -66,3 +66,6 @@ def _init_input_sign(self): self._inputs_sign = flow.inputs # The init signature only used for flex flow, so we set the _init_sign to empty dict for prompty flow. self._init_sign = {} + + def get_inputs_definition(self): + return self._inputs diff --git a/src/promptflow-core/promptflow/executor/_service/apis/batch.py b/src/promptflow-core/promptflow/executor/_service/apis/batch.py index 22d237a1347..ace583e86b5 100644 --- a/src/promptflow-core/promptflow/executor/_service/apis/batch.py +++ b/src/promptflow-core/promptflow/executor/_service/apis/batch.py @@ -24,13 +24,13 @@ @router.post("/initialize") def initialize(request: InitializationRequest): with get_log_context(request, enable_service_logger=True): - # validate request and get operation context + # Validate request and get operation context. request.validate_request() operation_context = update_and_get_operation_context(request.operation_context) service_logger.info(f"Received batch init request, executor version: {operation_context.get_user_agent()}.") - # resolve environment variables + # Resolve environment variables. set_environment_variables(request.environment_variables) - # init batch coordinator to validate flow and create process pool + # Init batch coordinator to validate flow and create process pool. batch_coordinator = BatchCoordinator( working_dir=request.working_dir, flow_file=request.flow_file, @@ -42,8 +42,8 @@ def initialize(request: InitializationRequest): init_kwargs=request.init_kwargs, ) batch_coordinator.start() - # return json response - return {"status": "initialized"} + # Return some flow infos including the flow inputs definition and whether it has aggregation nodes. + return batch_coordinator.get_flow_infos() @router.post("/execution") diff --git a/src/promptflow-core/promptflow/executor/_service/utils/batch_coordinator.py b/src/promptflow-core/promptflow/executor/_service/utils/batch_coordinator.py index fe6d0467452..a586445b451 100644 --- a/src/promptflow-core/promptflow/executor/_service/utils/batch_coordinator.py +++ b/src/promptflow-core/promptflow/executor/_service/utils/batch_coordinator.py @@ -79,6 +79,12 @@ def get_instance(cls): def get_log_context(self): return self._log_context + def get_flow_infos(self): + return { + "inputs_definition": self._flow_executor.get_inputs_definition(), + "has_aggregation": self._flow_executor.has_aggregation_node, + } + def start(self): """Start the process pool.""" self._process_pool.start() diff --git a/src/promptflow-core/promptflow/executor/flow_executor.py b/src/promptflow-core/promptflow/executor/flow_executor.py index df3099404b4..4ffede414c4 100644 --- a/src/promptflow-core/promptflow/executor/flow_executor.py +++ b/src/promptflow-core/promptflow/executor/flow_executor.py @@ -681,6 +681,9 @@ def _exec_in_thread(self, args) -> LineResult: self._completed_idx[line_number] = thread_name return results + def get_inputs_definition(self): + return self._flow.inputs + def exec_line( self, inputs: Mapping[str, Any], diff --git a/src/promptflow/tests/executor/unittests/executor/_service/apis/test_batch.py b/src/promptflow/tests/executor/unittests/executor/_service/apis/test_batch.py new file mode 100644 index 00000000000..ab0ca967992 --- /dev/null +++ b/src/promptflow/tests/executor/unittests/executor/_service/apis/test_batch.py @@ -0,0 +1,108 @@ +import pytest +from fastapi.testclient import TestClient + +from .....utils import construct_initialization_request_json + + +def construct_initialize_request_json(): + return {} + + +@pytest.mark.unittest +class TestBatchApis: + @pytest.mark.parametrize( + "flow_folder, flow_file, init_kwargs, expected_inputs_definition, expected_has_aggregation", + [ + # dag flow without aggregation nodes + ( + "print_input_flow", + "flow.dag.yaml", + None, + { + "text": { + "type": "string", + "default": None, + "description": "", + "enum": [], + "is_chat_input": False, + "is_chat_history": None, + } + }, + False, + ), + # dag flow with aggregation nodes + ( + "simple_aggregation", + "flow.dag.yaml", + None, + { + "text": { + "type": "string", + "default": "play", + "description": "", + "enum": [], + "is_chat_input": False, + "is_chat_history": None, + } + }, + True, + ), + # flex flow without aggregation + ( + "simple_with_yaml", + "flow.flex.yaml", + None, + { + "input_val": { + "type": "string", + "default": "gpt", + "description": None, + "enum": None, + "is_chat_input": False, + "is_chat_history": None, + } + }, + False, + ), + # flex flow with aggregation + ( + "basic_callable_class_async", + "flow.flex.yaml", + {"obj_input": "obj_input"}, + { + "func_input": { + "type": "string", + "default": None, + "description": None, + "enum": None, + "is_chat_input": False, + "is_chat_history": None, + } + }, + True, + ), + ], + ) + def test_initialize( + self, + executor_client: TestClient, + flow_folder, + flow_file, + init_kwargs, + expected_inputs_definition, + expected_has_aggregation, + ): + initialization_request = construct_initialization_request_json( + flow_folder=flow_folder, + flow_file=flow_file, + init_kwargs=init_kwargs, + ) + response = executor_client.post(url="/initialize", json=initialization_request) + # assert response + assert response.status_code == 200 + assert response.json() == { + "inputs_definition": expected_inputs_definition, + "has_aggregation": expected_has_aggregation, + } + executor_client.post(url="/finalize") + assert response.status_code == 200 diff --git a/src/promptflow/tests/executor/utils.py b/src/promptflow/tests/executor/utils.py index fd0edf82dbb..9ccab8bf4d0 100644 --- a/src/promptflow/tests/executor/utils.py +++ b/src/promptflow/tests/executor/utils.py @@ -132,21 +132,39 @@ def is_image_file(file_path: Path): def construct_flow_execution_request_json(flow_folder, root=FLOW_ROOT, inputs=None, connections=None): + base_execution_request = construct_base_execution_request_json(flow_folder, root=root, connections=connections) + flow_execution_request = { + "run_id": str(uuid.uuid4()), + "inputs": inputs, + "operation_context": { + "request_id": "test-request-id", + "user_agent": "test-user-agent", + }, + } + return {**base_execution_request, **flow_execution_request} + + +def construct_initialization_request_json( + flow_folder, root=FLOW_ROOT, flow_file="flow.dag.yaml", connections=None, init_kwargs=None +): + if flow_file == "flow.flex.yaml": + root = EAGER_FLOW_ROOT + base_execution_request = construct_base_execution_request_json( + flow_folder, root=root, connections=connections, flow_file=flow_file + ) + return {**base_execution_request, "init_kwargs": init_kwargs} if init_kwargs is not None else base_execution_request + + +def construct_base_execution_request_json(flow_folder, root=FLOW_ROOT, connections=None, flow_file="flow.dag.yaml"): working_dir = get_flow_folder(flow_folder, root=root) tmp_dir = Path(mkdtemp()) log_path = tmp_dir / "log.txt" return { - "run_id": str(uuid.uuid4()), "working_dir": working_dir.as_posix(), - "flow_file": "flow.dag.yaml", + "flow_file": flow_file, "output_dir": tmp_dir.as_posix(), - "connections": connections, "log_path": log_path.as_posix(), - "inputs": inputs, - "operation_context": { - "request_id": "test-request-id", - "user_agent": "test-user-agent", - }, + "connections": connections, } From eb7093ad58166cd0f0cc14847fb9822d931dff52 Mon Sep 17 00:00:00 2001 From: chenyang Date: Sat, 11 May 2024 19:39:25 +0800 Subject: [PATCH 6/7] build support flex flow (#2956) # Description Please add an informative description that covers that changes made by the pull request and link all relevant issues. Build flex flow command: `pf flow build --source flex_flow_yaml_file --output output_folder --format docker` **Note: Build will cannot generate connection.** Docker run for flex flow: `docker run -p 8080:8080 -e PROMPTFLOW_WORKER_NUM=1 -e PROMPTFLOW_WORKER_THREADS=1 -e PF_FLOW_INIT_CONFIG='{"model_config": {"api_key": "111", "azure_endpoint": "https://test.openai.azure.com/", "azure_deployment": "gpt-35-turbo"}}' flex_flow_image` **Note: Need pass connection detail info by PF_FLOW_INIT_CONFIG** # All Promptflow Contribution checklist: - [x] **The pull request does not introduce [breaking changes].** - [ ] **CHANGELOG is updated for new features, bug fixes or other significant changes.** - [ ] **I have read the [contribution guidelines](../CONTRIBUTING.md).** - [ ] **Create an issue and link to the pull request to get dedicated review from promptflow team. Learn more: [suggested workflow](../CONTRIBUTING.md#suggested-workflow).** ## General Guidelines and Best Practices - [x] Title of the pull request is clear and informative. - [ ] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, [see this page](https://github.com/Azure/azure-powershell/blob/master/documentation/development-docs/cleaning-up-commits.md). ### Testing Guidelines - [x] Pull request includes test coverage for the included changes. --------- Co-authored-by: Philip Gao --- src/promptflow-core/promptflow/_constants.py | 1 + .../promptflow/_utils/flow_utils.py | 7 ++- .../promptflow/contracts/flow.py | 58 ++++++++++++++++--- .../promptflow/core/_serving/app_base.py | 10 +++- src/promptflow-core/promptflow/core/_utils.py | 36 +++++------- .../promptflow/_cli/_pf/_flow.py | 2 +- .../promptflow/_sdk/_orchestrator/utils.py | 2 +- .../_sdk/_utilities/general_utils.py | 4 +- .../_sdk/data/docker/Dockerfile.jinja2 | 2 +- .../promptflow/_sdk/data/docker/README.md | 20 +++++++ .../_sdk/data/docker_csharp/README.md | 20 +++++++ .../promptflow/_sdk/entities/_flows/base.py | 2 +- .../promptflow/_sdk/entities/_flows/dag.py | 7 ++- .../promptflow/_sdk/entities/_flows/flex.py | 10 +++- .../_sdk/entities/_flows/prompty.py | 5 ++ .../_sdk/operations/_flow_operations.py | 39 +++++++------ .../tests/sdk_cli_test/e2etests/test_cli.py | 31 ++++++++++ .../eager_flows/chat-basic/chat.jinja2 | 12 ++++ .../eager_flows/chat-basic/chat.prompty | 28 +++++++++ .../eager_flows/chat-basic/data.jsonl | 3 + .../eager_flows/chat-basic/flow.flex.yaml | 5 ++ .../eager_flows/chat-basic/flow.py | 47 +++++++++++++++ .../eager_flows/chat-basic/init.json | 6 ++ .../eager_flows/chat-basic/paths.py | 6 ++ .../chat-basic/requirements-azure.txt | 1 + .../eager_flows/chat-basic/requirements.txt | 1 + .../eager_flows/chat-basic/run.yml | 9 +++ .../eager_flows/chat-basic/sample.json | 1 + 28 files changed, 316 insertions(+), 59 deletions(-) create mode 100644 src/promptflow/tests/test_configs/eager_flows/chat-basic/chat.jinja2 create mode 100644 src/promptflow/tests/test_configs/eager_flows/chat-basic/chat.prompty create mode 100644 src/promptflow/tests/test_configs/eager_flows/chat-basic/data.jsonl create mode 100644 src/promptflow/tests/test_configs/eager_flows/chat-basic/flow.flex.yaml create mode 100644 src/promptflow/tests/test_configs/eager_flows/chat-basic/flow.py create mode 100644 src/promptflow/tests/test_configs/eager_flows/chat-basic/init.json create mode 100644 src/promptflow/tests/test_configs/eager_flows/chat-basic/paths.py create mode 100644 src/promptflow/tests/test_configs/eager_flows/chat-basic/requirements-azure.txt create mode 100644 src/promptflow/tests/test_configs/eager_flows/chat-basic/requirements.txt create mode 100644 src/promptflow/tests/test_configs/eager_flows/chat-basic/run.yml create mode 100644 src/promptflow/tests/test_configs/eager_flows/chat-basic/sample.json diff --git a/src/promptflow-core/promptflow/_constants.py b/src/promptflow-core/promptflow/_constants.py index 2caa3ad32a3..c9a40fff9d3 100644 --- a/src/promptflow-core/promptflow/_constants.py +++ b/src/promptflow-core/promptflow/_constants.py @@ -12,6 +12,7 @@ PROMPTFLOW_SECRETS_FILE = "PROMPTFLOW_SECRETS_FILE" PF_NO_INTERACTIVE_LOGIN = "PF_NO_INTERACTIVE_LOGIN" PF_RUN_AS_BUILT_BINARY = "PF_RUN_AS_BUILT_BINARY" +PF_FLOW_INIT_CONFIG = "PF_FLOW_INIT_CONFIG" ENABLE_MULTI_CONTAINER_KEY = "PF_ENABLE_MULTI_CONTAINER" PF_LOGGING_LEVEL = "PF_LOGGING_LEVEL" OPENAI_API_KEY = "openai-api-key" diff --git a/src/promptflow-core/promptflow/_utils/flow_utils.py b/src/promptflow-core/promptflow/_utils/flow_utils.py index b2f8540659c..df8115f067e 100644 --- a/src/promptflow-core/promptflow/_utils/flow_utils.py +++ b/src/promptflow-core/promptflow/_utils/flow_utils.py @@ -67,6 +67,7 @@ def resolve_flow_path( flow_path: Union[str, Path, PathLike], base_path: Union[str, Path, PathLike, None] = None, check_flow_exist: bool = True, + default_flow_file: str = FLOW_DAG_YAML, ) -> Tuple[Path, str]: """Resolve flow path and return the flow directory path and the file name of the target yaml. @@ -79,6 +80,8 @@ def resolve_flow_path( :param check_flow_exist: If True, the function will try to check the target yaml and raise FileNotFoundError if not found. If False, the function will return the flow directory path and the file name of the target yaml. + :param default_flow_file: Default file name used when flow file is not found. + :type default_flow_file: str :return: The flow directory path and the file name of the target yaml. :rtype: Tuple[Path, str] """ @@ -89,7 +92,7 @@ def resolve_flow_path( if flow_path.is_dir(): flow_folder = flow_path - flow_file = FLOW_DAG_YAML + flow_file = default_flow_file flow_file_list = [] for flow_name, suffix in itertools.product([FLOW_DAG_YAML, FLOW_FLEX_YAML], [".yaml", ".yml"]): flow_file_name = flow_name.replace(".yaml", suffix) @@ -109,7 +112,7 @@ def resolve_flow_path( flow_file = flow_path.name else: # flow_path doesn't exist flow_folder = flow_path - flow_file = FLOW_DAG_YAML + flow_file = default_flow_file file_path = flow_folder / flow_file if file_path.suffix.lower() not in FLOW_FILE_SUFFIX: diff --git a/src/promptflow-core/promptflow/contracts/flow.py b/src/promptflow-core/promptflow/contracts/flow.py index 3ead2481d89..6d352764308 100644 --- a/src/promptflow-core/promptflow/contracts/flow.py +++ b/src/promptflow-core/promptflow/contracts/flow.py @@ -751,19 +751,19 @@ def from_yaml(cls, flow_file: Path, working_dir=None, name=None) -> "Flow": """Load flow from yaml file.""" working_dir = cls._parse_working_dir(flow_file, working_dir) with open(working_dir / flow_file, "r", encoding=DEFAULT_ENCODING) as fin: - flow_dag = load_yaml(fin) + flow_data = load_yaml(fin) # Name priority: name from payload > name from yaml content > working_dir.stem # For portal created flow, there is a meaningless predefined name in yaml, use name from payload to override it. - if name is None: - name = flow_dag.get("name", _sanitize_python_variable_name(working_dir.stem)) - flow_dag["name"] = name - return Flow._from_dict(flow_dag=flow_dag, working_dir=working_dir) + return Flow._from_dict(flow_data=flow_data, working_dir=working_dir, name=name) @classmethod - def _from_dict(cls, flow_dag: dict, working_dir: Path) -> "Flow": + def _from_dict(cls, flow_data: dict, working_dir: Path, name=None) -> "Flow": """Load flow from dict.""" cls._update_working_dir(working_dir) - flow = Flow.deserialize(flow_dag) + if name is None: + name = flow_data.get("name", _sanitize_python_variable_name(working_dir.stem)) + flow_data["name"] = name + flow = Flow.deserialize(flow_data) flow._set_tool_loader(working_dir) return flow @@ -1019,6 +1019,34 @@ def deserialize(data: dict) -> "FlexFlow": environment_variables=data.get("environment_variables") or {}, ) + @classmethod + def _from_dict(cls, flow_data: dict, working_dir: Path, name=None) -> "FlexFlow": + """Load flow from dict.""" + from promptflow._core.entry_meta_generator import generate_flow_meta + + from .._utils.flow_utils import resolve_python_entry_file + + Flow._update_working_dir(working_dir) + if name is None: + name = flow_data.get("name", _sanitize_python_variable_name(working_dir.stem)) + flow_data["name"] = name + + entry = flow_data.get("entry") + entry_file = resolve_python_entry_file(entry=entry, working_dir=working_dir) + + meta_dict = generate_flow_meta( + flow_directory=working_dir, + source_path=entry_file, + data=flow_data, + ) + return cls.deserialize(meta_dict) + + def get_connection_names(self, environment_variables_overrides: Dict[str, str] = None): + """Return connection names.""" + connection_names = super().get_connection_names(environment_variables_overrides=environment_variables_overrides) + + return set({item for item in connection_names if item}) + @dataclass class PromptyFlow(FlowBase): @@ -1064,3 +1092,19 @@ def deserialize(cls, data: dict) -> "PromptyFlow": environment_variables=data.get("environment_variables") or {}, message_format=data.get("message_format", MessageFormatType.BASIC), ) + + @classmethod + def _from_dict(cls, flow_data: dict, working_dir: Path, name=None) -> "PromptyFlow": + """Load flow from dict.""" + Flow._update_working_dir(working_dir) + if name is None: + name = flow_data.get("name", _sanitize_python_variable_name(working_dir.stem)) + flow_data["name"] = name + + return cls.deserialize(flow_data) + + def get_connection_names(self, environment_variables_overrides: Dict[str, str] = None): + """Return connection names.""" + connection_names = super().get_connection_names(environment_variables_overrides=environment_variables_overrides) + + return set({item for item in connection_names if item}) diff --git a/src/promptflow-core/promptflow/core/_serving/app_base.py b/src/promptflow-core/promptflow/core/_serving/app_base.py index c7d22c36c1b..d7165bfd9a1 100644 --- a/src/promptflow-core/promptflow/core/_serving/app_base.py +++ b/src/promptflow-core/promptflow/core/_serving/app_base.py @@ -1,7 +1,7 @@ # --------------------------------------------------------- # Copyright (c) Microsoft Corporation. All rights reserved. # --------------------------------------------------------- - +import json import mimetypes import os from abc import ABC, abstractmethod @@ -17,6 +17,7 @@ from promptflow.core._utils import init_executable from promptflow.storage._run_storage import DummyRunStorage +from ..._constants import PF_FLOW_INIT_CONFIG from .swagger import generate_swagger @@ -55,7 +56,12 @@ def init_app(self, **kwargs): self.sample = get_sample_json(self.project_path, logger) self.init = kwargs.get("init", {}) - logger.info("Init params: " + str(self.init)) + if not self.init: + init_params = os.environ.get(PF_FLOW_INIT_CONFIG, "{}") + init_dict: dict = json.loads(init_params) + self.init = init_dict + + logger.debug("Init params: " + str(self.init)) self.init_swagger() # try to initialize the flow invoker diff --git a/src/promptflow-core/promptflow/core/_utils.py b/src/promptflow-core/promptflow/core/_utils.py index e619b404037..93900b251d2 100644 --- a/src/promptflow-core/promptflow/core/_utils.py +++ b/src/promptflow-core/promptflow/core/_utils.py @@ -12,7 +12,7 @@ from jinja2 import Template from promptflow._constants import AZURE_WORKSPACE_REGEX_FORMAT -from promptflow._utils.flow_utils import is_flex_flow, resolve_flow_path, resolve_python_entry_file +from promptflow._utils.flow_utils import is_flex_flow, is_prompty_flow, resolve_flow_path from promptflow._utils.logger_utils import LoggerFactory from promptflow._utils.utils import _match_reference from promptflow._utils.yaml_utils import load_yaml @@ -27,39 +27,35 @@ def render_jinja_template_content(template_content, *, trim_blocks=True, keep_tr return template.render(**kwargs) -def init_executable(*, flow_dag: dict = None, flow_path: Path = None, working_dir: Path = None): - if flow_dag and flow_path: +def init_executable(*, flow_data: dict = None, flow_path: Path = None, working_dir: Path = None): + if flow_data and flow_path: raise ValueError("flow_dag and flow_path cannot be both provided.") - if not flow_dag and not flow_path: + if not flow_data and not flow_path: raise ValueError("flow_dag or flow_path must be provided.") - if flow_dag and not working_dir: + if flow_data and not working_dir: raise ValueError("working_dir must be provided when flow_dag is provided.") if flow_path: + if is_prompty_flow(file_path=flow_path): + from promptflow.contracts.flow import PromptyFlow as ExecutablePromptyFlow + from promptflow.core._flow import Prompty + + configs, _ = Prompty._parse_prompty(flow_path) + return ExecutablePromptyFlow._from_dict(flow_data=configs, working_dir=working_dir or flow_path.parent) + flow_dir, flow_filename = resolve_flow_path(flow_path) - flow_dag = load_yaml(flow_dir / flow_filename) + flow_data = load_yaml(flow_dir / flow_filename) if not working_dir: working_dir = flow_dir from promptflow.contracts.flow import FlexFlow as ExecutableEagerFlow from promptflow.contracts.flow import Flow as ExecutableFlow - if is_flex_flow(yaml_dict=flow_dag): - - entry = flow_dag.get("entry") - entry_file = resolve_python_entry_file(entry=entry, working_dir=working_dir) - - from promptflow._core.entry_meta_generator import generate_flow_meta - - meta_dict = generate_flow_meta( - flow_directory=working_dir, - source_path=entry_file, - data=flow_dag, - ) - return ExecutableEagerFlow.deserialize(meta_dict) + if is_flex_flow(yaml_dict=flow_data): + return ExecutableEagerFlow._from_dict(flow_data=flow_data, working_dir=working_dir) # for DAG flow, use data to init executable to improve performance - return ExecutableFlow._from_dict(flow_dag=flow_dag, working_dir=working_dir) + return ExecutableFlow._from_dict(flow_data=flow_data, working_dir=working_dir) # !!! Attention!!!: Please make sure you have contact with PRS team before changing the interface. diff --git a/src/promptflow-devkit/promptflow/_cli/_pf/_flow.py b/src/promptflow-devkit/promptflow/_cli/_pf/_flow.py index d0e1b348585..ef15d264e49 100644 --- a/src/promptflow-devkit/promptflow/_cli/_pf/_flow.py +++ b/src/promptflow-devkit/promptflow/_cli/_pf/_flow.py @@ -510,7 +510,7 @@ def _test_flow_multi_modal(args, pf_client): for script in script_path: StreamlitFileReplicator( flow_name=flow.display_name if flow.display_name else flow.name, - flow_dag_path=flow.flow_dag_path, + flow_dag_path=flow._flow_file_path, ).generate_to_file(script) main_script_path = os.path.join(temp_dir, "main.py") logger.info("Start streamlit with main script generated at: %s", main_script_path) diff --git a/src/promptflow-devkit/promptflow/_sdk/_orchestrator/utils.py b/src/promptflow-devkit/promptflow/_sdk/_orchestrator/utils.py index 5bbafc1f818..9a5e6a59078 100644 --- a/src/promptflow-devkit/promptflow/_sdk/_orchestrator/utils.py +++ b/src/promptflow-devkit/promptflow/_sdk/_orchestrator/utils.py @@ -101,7 +101,7 @@ def overwrite_connections(flow_dag: dict, connections: dict, working_dir: PathLi raise InvalidFlowError(f"Invalid connections overwrite format: {connections}, only list is supported.") # Load executable flow to check if connection is LLM connection - executable_flow = ExecutableFlow._from_dict(flow_dag=flow_dag, working_dir=Path(working_dir)) + executable_flow = ExecutableFlow._from_dict(flow_data=flow_dag, working_dir=Path(working_dir)) # generate tool meta for deployment name, model override # tools_meta = generate_flow_tools_json(flow_directory=working_dir, dump=False, used_packages_only=True) diff --git a/src/promptflow-devkit/promptflow/_sdk/_utilities/general_utils.py b/src/promptflow-devkit/promptflow/_sdk/_utilities/general_utils.py index d1aa7332bea..17eab02b979 100644 --- a/src/promptflow-devkit/promptflow/_sdk/_utilities/general_utils.py +++ b/src/promptflow-devkit/promptflow/_sdk/_utilities/general_utils.py @@ -277,7 +277,7 @@ def _resolve_folder_to_compress(base_path: Path, include: str, dst_path: Path) - @contextmanager def _merge_local_code_and_additional_includes(code_path: Path): - # TODO: unify variable names: flow_dir_path, flow_dag_path, flow_path + # TODO: unify variable names: flow_dir_path, flow_file_path, flow_path def additional_includes_copy(src, relative_path, target_dir): if src.is_file(): @@ -1103,7 +1103,7 @@ def get_flow_path(flow) -> Path: from promptflow._sdk.entities._flows.prompty import Prompty if isinstance(flow, DAGFlow): - return flow.flow_dag_path.parent.resolve() + return flow._flow_file_path.parent.resolve() if isinstance(flow, (FlexFlow, Prompty)): # Use code path to return as flow path, since code path is the same as flow directory for yaml case and code # path points to original code path in non-yaml case diff --git a/src/promptflow-devkit/promptflow/_sdk/data/docker/Dockerfile.jinja2 b/src/promptflow-devkit/promptflow/_sdk/data/docker/Dockerfile.jinja2 index c321330ec51..7e6b25c6189 100644 --- a/src/promptflow-devkit/promptflow/_sdk/data/docker/Dockerfile.jinja2 +++ b/src/promptflow-devkit/promptflow/_sdk/data/docker/Dockerfile.jinja2 @@ -50,7 +50,7 @@ RUN conda run -n {{env.conda_env_name}} sh /flow/{{ env.setup_sh }} EXPOSE 8080 -COPY ./connections/* /connections/ +COPY ./connections /connections # reset runsvdir RUN rm -rf /var/runit diff --git a/src/promptflow-devkit/promptflow/_sdk/data/docker/README.md b/src/promptflow-devkit/promptflow/_sdk/data/docker/README.md index 5ae3319d529..437374f4f04 100644 --- a/src/promptflow-devkit/promptflow/_sdk/data/docker/README.md +++ b/src/promptflow-devkit/promptflow/_sdk/data/docker/README.md @@ -10,5 +10,25 @@ Exported Dockerfile & its dependencies are located in the same folder. The struc - settings.json: a json file to store the settings of the docker image - README.md: the readme file to describe how to use the dockerfile + +Build Docker image: +`docker build -t ` + + +**Run dag flow** with docker run by flask serving engine: +`docker run -p 8080:8080 -e OPEN_AI_CONNECTION_API_KEY= -e PROMPTFLOW_WORKER_NUM= -e PROMPTFLOW_WORKER_THREADS= ` +example: +`docker run -p 8080:8080 -e OPEN_AI_CONNECTION_API_KEY=111 -e PROMPTFLOW_WORKER_NUM=1 -e PROMPTFLOW_WORKER_THREADS=1 dag_flow_image` + +**Run flex flow** with docker run by flask serving engine: +`docker run -p 8080:8080 -e PROMPTFLOW_WORKER_NUM= -e PROMPTFLOW_WORKER_THREADS= -e PF_FLOW_INIT_CONFIG= ` +example: +`docker run -p 8080:8080 -e PROMPTFLOW_WORKER_NUM=1 -e PROMPTFLOW_WORKER_THREADS=1 -e PF_FLOW_INIT_CONFIG='{"model_config": {"api_key": "111", "azure_endpoint": "https://test.openai.azure.com/", "azure_deployment": "gpt-35-turbo"}}' flex_flow_image` + + +Test the endpoint: +After start the service, you can use curl to test it: `curl http://localhost:8080/score --data ` + + Please refer to [official doc](https://microsoft.github.io/promptflow/how-to-guides/deploy-a-flow/deploy-using-docker.html) for more details about how to use the exported dockerfile and scripts. diff --git a/src/promptflow-devkit/promptflow/_sdk/data/docker_csharp/README.md b/src/promptflow-devkit/promptflow/_sdk/data/docker_csharp/README.md index 4e05d8ee2b7..a4494404124 100644 --- a/src/promptflow-devkit/promptflow/_sdk/data/docker_csharp/README.md +++ b/src/promptflow-devkit/promptflow/_sdk/data/docker_csharp/README.md @@ -7,5 +7,25 @@ Exported Dockerfile & its dependencies are located in the same folder. The struc - settings.json: a json file to store the settings of the docker image - README.md: the readme file to describe how to use the dockerfile + +Build Docker image: +`docker build -t ` + + +**Run dag flow** with docker run by flask serving engine: +`docker run -p 8080:8080 -e OPEN_AI_CONNECTION_API_KEY= -e PROMPTFLOW_WORKER_NUM= -e PROMPTFLOW_WORKER_THREADS= ` +example: +`docker run -p 8080:8080 -e OPEN_AI_CONNECTION_API_KEY=111 -e PROMPTFLOW_WORKER_NUM=1 -e PROMPTFLOW_WORKER_THREADS=1 dag_flow_image` + +**Run flex flow** with docker run by flask serving engine: +`docker run -p 8080:8080 -e PROMPTFLOW_WORKER_NUM= -e PROMPTFLOW_WORKER_THREADS= -e PF_FLOW_INIT_CONFIG= ` +example: +`docker run -p 8080:8080 -e PROMPTFLOW_WORKER_NUM=1 -e PROMPTFLOW_WORKER_THREADS=1 -e PF_FLOW_INIT_CONFIG='{"model_config": {"api_key": "111", "azure_endpoint": "https://test.openai.azure.com/", "azure_deployment": "gpt-35-turbo"}}' flex_flow_image` + + +Test the endpoint: +After start the service, you can use curl to test it: `curl http://localhost:8080/score --data ` + + Please refer to [official doc](https://microsoft.github.io/promptflow/how-to-guides/deploy-a-flow/deploy-using-docker.html) for more details about how to use the exported dockerfile and scripts. diff --git a/src/promptflow-devkit/promptflow/_sdk/entities/_flows/base.py b/src/promptflow-devkit/promptflow/_sdk/entities/_flows/base.py index 900b4ea7497..9482888b0cc 100644 --- a/src/promptflow-devkit/promptflow/_sdk/entities/_flows/base.py +++ b/src/promptflow-devkit/promptflow/_sdk/entities/_flows/base.py @@ -212,7 +212,7 @@ def _init_executable(self): from promptflow.contracts.flow import Flow as ExecutableFlow # for DAG flow, use data to init executable to improve performance - return ExecutableFlow._from_dict(flow_dag=self._data, working_dir=self.code) + return ExecutableFlow._from_dict(flow_data=self._data, working_dir=self.code) def __eq__(self, other): if isinstance(other, Flow): diff --git a/src/promptflow-devkit/promptflow/_sdk/entities/_flows/dag.py b/src/promptflow-devkit/promptflow/_sdk/entities/_flows/dag.py index e410d452a9f..28b70ee0189 100644 --- a/src/promptflow-devkit/promptflow/_sdk/entities/_flows/dag.py +++ b/src/promptflow-devkit/promptflow/_sdk/entities/_flows/dag.py @@ -36,7 +36,8 @@ def __init__( # TODO: this can be dangerous. path always point to the flow yaml file; code always point to the flow directory; # but path may not under code (like a temp generated flow yaml file). - self._flow_dir, self._dag_file_name = resolve_flow_path(self.path) + self._flow_dir, self._flow_file_name = resolve_flow_path(self.path) + self._flow_file_path = self._flow_dir / self._flow_file_name self._executable = None self._params_override = params_override @@ -47,7 +48,7 @@ def name(self) -> str: @property def flow_dag_path(self) -> Path: - return self._flow_dir / self._dag_file_name + return self._flow_file_path @property def tools_meta_path(self) -> Path: @@ -93,7 +94,7 @@ def _create_validation_error(self, message, no_personal_data_message=None): def _dump_for_validation(self) -> Dict: # Flow is read-only in control plane, so we always dump the flow from file - data = load_yaml(self.flow_dag_path) + data = load_yaml(self._flow_file_path) if isinstance(self._params_override, dict): data.update(self._params_override) return data diff --git a/src/promptflow-devkit/promptflow/_sdk/entities/_flows/flex.py b/src/promptflow-devkit/promptflow/_sdk/entities/_flows/flex.py index 2880119ca1e..92704b7d737 100644 --- a/src/promptflow-devkit/promptflow/_sdk/entities/_flows/flex.py +++ b/src/promptflow-devkit/promptflow/_sdk/entities/_flows/flex.py @@ -5,8 +5,9 @@ from pathlib import Path from typing import Dict, Union -from promptflow._constants import LANGUAGE_KEY, FlowLanguage +from promptflow._constants import LANGUAGE_KEY, FlowLanguage, FLOW_FLEX_YAML from promptflow._sdk._constants import BASE_PATH_CONTEXT_KEY +from promptflow._utils.flow_utils import resolve_flow_path from promptflow.exceptions import ErrorTarget, UserErrorException from .base import Flow as FlowBase @@ -33,9 +34,16 @@ def __init__( code = Path(code) # entry function name self.entry = entry + self._flow_dir, self._flow_file_name = resolve_flow_path(path, + check_flow_exist=False, + default_flow_file=FLOW_FLEX_YAML) # TODO(2910062): support non-dag flow execution cache super().__init__(code=code, path=path, dag=data, content_hash=None, **kwargs) + @property + def name(self) -> str: + return self._flow_dir.name + # region properties @property def language(self) -> str: diff --git a/src/promptflow-devkit/promptflow/_sdk/entities/_flows/prompty.py b/src/promptflow-devkit/promptflow/_sdk/entities/_flows/prompty.py index 29bac147694..e2e5eb8d712 100644 --- a/src/promptflow-devkit/promptflow/_sdk/entities/_flows/prompty.py +++ b/src/promptflow-devkit/promptflow/_sdk/entities/_flows/prompty.py @@ -27,9 +27,14 @@ def __init__( path = Path(path) # prompty folder path code = Path(code) + self._flow_file_path = path self._core_prompty = CorePrompty(path=path, **kwargs) super().__init__(code=code, path=path, data=data, content_hash=None, **kwargs) + @property + def name(self) -> str: + return self.code.name + @property def language(self) -> str: return self._data.get(LANGUAGE_KEY, FlowLanguage.Python) diff --git a/src/promptflow-devkit/promptflow/_sdk/operations/_flow_operations.py b/src/promptflow-devkit/promptflow/_sdk/operations/_flow_operations.py index 2c9e4d10feb..e6280ad6425 100644 --- a/src/promptflow-devkit/promptflow/_sdk/operations/_flow_operations.py +++ b/src/promptflow-devkit/promptflow/_sdk/operations/_flow_operations.py @@ -64,7 +64,7 @@ parse_variant, ) from promptflow._utils.yaml_utils import dump_yaml, load_yaml -from promptflow.core._utils import load_inputs_from_sample +from promptflow.core._utils import init_executable, load_inputs_from_sample from promptflow.exceptions import ErrorTarget, UserErrorException @@ -387,15 +387,18 @@ def _chat_with_ui(self, script, skip_open_browser: bool = False): st_cli.main() def _build_environment_config(self, flow_dag_path: Path): - flow_info = load_yaml(flow_dag_path) - # standard env object: - # environment: - # image: xxx - # conda_file: xxx - # python_requirements_txt: xxx - # setup_sh: xxx - # TODO: deserialize dag with structured class here to avoid using so many magic strings - env_obj = flow_info.get("environment", {}) + if is_prompty_flow(file_path=flow_dag_path): + env_obj = {} + else: + flow_info = load_yaml(flow_dag_path) + # standard env object: + # environment: + # image: xxx + # conda_file: xxx + # python_requirements_txt: xxx + # setup_sh: xxx + # TODO: deserialize dag with structured class here to avoid using so many magic strings + env_obj = flow_info.get("environment", {}) env_obj["sdk_version"] = version("promptflow") # version 0.0.1 is the dev version of promptflow @@ -499,7 +502,7 @@ def _export_flow_connections( return self._migrate_connections( connection_names=SubmitterHelper.get_used_connection_names( tools_meta=CSharpExecutorProxy.generate_flow_tools_json( - flow_file=flow.flow_dag_path, + flow_file=flow._flow_file_path, working_dir=flow.code, ), flow_dag=flow._data, @@ -508,9 +511,7 @@ def _export_flow_connections( ) else: # TODO: avoid using executable here - from promptflow.contracts.flow import Flow as ExecutableFlow - - executable = ExecutableFlow.from_yaml(flow_file=flow.path, working_dir=flow.code) + executable = init_executable(flow_path=flow.path, working_dir=flow.code) return self._migrate_connections( connection_names=executable.get_connection_names(), output_dir=output_dir, @@ -729,6 +730,8 @@ def build( flow = load_flow(flow) is_csharp_flow = flow.language == FlowLanguage.CSharp + is_flex_flow = isinstance(flow, FlexFlow) + is_prompty_flow = isinstance(flow, Prompty) if format not in ["docker", "executable"]: raise ValueError(f"Unsupported export format: {format}") @@ -749,7 +752,7 @@ def build( output=output_flow_dir, tuning_node=tuning_node, node_variant=node_variant, - update_flow_tools_json=False if is_csharp_flow else True, + update_flow_tools_json=False if is_csharp_flow or is_flex_flow or is_prompty_flow else True, ) if flow_only: @@ -791,7 +794,7 @@ def is_yaml_file(file_path): if is_yaml_file(flow_dag_path) and _get_additional_includes(flow_dag_path): # Merge the flow folder and additional includes to temp folder. - # TODO: support a flow_dag_path with a name different from flow.dag.yaml + # TODO: support a flow_file_path with a name different from flow.dag.yaml with _merge_local_code_and_additional_includes(code_path=flow_dag_path.parent) as temp_dir: remove_additional_includes(Path(temp_dir)) yield Path(temp_dir) / flow_dag_path.name @@ -891,7 +894,7 @@ def _generate_tools_meta( }, }, {} - with self._resolve_additional_includes(flow.flow_dag_path) as new_flow_dag_path: + with self._resolve_additional_includes(flow._flow_file_path) as new_flow_dag_path: flow_tools = generate_flow_tools_json( flow_directory=new_flow_dag_path.parent, dump=False, @@ -910,7 +913,7 @@ def _generate_tools_meta( for node_name in nodes_with_error: tools_errors[node_name] = flow_tools_meta.pop(node_name) - additional_includes = _get_additional_includes(flow.flow_dag_path) + additional_includes = _get_additional_includes(flow._flow_file_path) if additional_includes: additional_files = {} for include in additional_includes: diff --git a/src/promptflow-devkit/tests/sdk_cli_test/e2etests/test_cli.py b/src/promptflow-devkit/tests/sdk_cli_test/e2etests/test_cli.py index dad2b6fce42..a2ea0e9b2d2 100644 --- a/src/promptflow-devkit/tests/sdk_cli_test/e2etests/test_cli.py +++ b/src/promptflow-devkit/tests/sdk_cli_test/e2etests/test_cli.py @@ -1318,6 +1318,37 @@ def get_node_settings(_flow_dag_path: Path): finally: shutil.rmtree(output_path, ignore_errors=True) + def test_flex_flow_build(self): + from promptflow._cli._pf.entry import main + + with tempfile.TemporaryDirectory() as temp: + temp = Path(temp) + cmd = ( + "pf", + "flow", + "build", + "--source", + f"{EAGER_FLOWS_DIR}/chat-basic/flow.flex.yaml", + "--output", + temp.as_posix(), + "--format", + "docker", + ) + sys.argv = list(cmd) + main() + assert (temp / "connections").is_dir() + assert (temp / "flow").is_dir() + assert (temp / "runit").is_dir() + assert (temp / "Dockerfile").is_file() + with open(temp / "Dockerfile", "r") as f: + assert r"/connections" in f.read() + + origin_flow = Path(f"{EAGER_FLOWS_DIR}/chat-basic") + temp_flow = temp / "flow" + for file_path in origin_flow.rglob("*"): + relative_path = file_path.relative_to(origin_flow) + assert (temp_flow / relative_path).exists() + def test_flow_build_with_ua(self, capsys): with pytest.raises(SystemExit): run_pf_command( diff --git a/src/promptflow/tests/test_configs/eager_flows/chat-basic/chat.jinja2 b/src/promptflow/tests/test_configs/eager_flows/chat-basic/chat.jinja2 new file mode 100644 index 00000000000..c5e811e1969 --- /dev/null +++ b/src/promptflow/tests/test_configs/eager_flows/chat-basic/chat.jinja2 @@ -0,0 +1,12 @@ +system: +You are a helpful assistant. + +{% for item in chat_history %} +user: +{{item.inputs.question}} +assistant: +{{item.outputs.answer}} +{% endfor %} + +user: +{{question}} \ No newline at end of file diff --git a/src/promptflow/tests/test_configs/eager_flows/chat-basic/chat.prompty b/src/promptflow/tests/test_configs/eager_flows/chat-basic/chat.prompty new file mode 100644 index 00000000000..90cd0054c2b --- /dev/null +++ b/src/promptflow/tests/test_configs/eager_flows/chat-basic/chat.prompty @@ -0,0 +1,28 @@ +--- +name: Stream Chat +description: Chat with stream enabled. +model: + api: chat + configuration: + type: azure_openai + azure_deployment: gpt-35-turbo + parameters: + temperature: 0.2 +inputs: + question: + type: string + chat_history: + type: list +sample: sample.json +--- + +system: +You are a helpful assistant. + +{% for item in chat_history %} +{{item.role}}: +{{item.content}} +{% endfor %} + +user: +{{question}} diff --git a/src/promptflow/tests/test_configs/eager_flows/chat-basic/data.jsonl b/src/promptflow/tests/test_configs/eager_flows/chat-basic/data.jsonl new file mode 100644 index 00000000000..5dd60d3cc11 --- /dev/null +++ b/src/promptflow/tests/test_configs/eager_flows/chat-basic/data.jsonl @@ -0,0 +1,3 @@ +{"question": "What is Prompt flow?", "chat_history":[], "statements": {"correctness": "should explain what's 'Prompt flow'", "consise": "It is a consise statement."}} +{"question": "What is ChatGPT? Please explain with consise statement", "chat_history":[], "statements": { "correctness": "should explain what's ChatGPT", "consise": "It is a consise statement."}} +{"question": "How many questions did user ask?", "chat_history": [{"role": "user","content": "where is the nearest coffee shop?"},{"role": "system","content": "I'm sorry, I don't know that. Would you like me to look it up for you?"}], "statements": { "correctness": "result should be 2", "consise": "It is a consise statement."}} \ No newline at end of file diff --git a/src/promptflow/tests/test_configs/eager_flows/chat-basic/flow.flex.yaml b/src/promptflow/tests/test_configs/eager_flows/chat-basic/flow.flex.yaml new file mode 100644 index 00000000000..ea0410185ec --- /dev/null +++ b/src/promptflow/tests/test_configs/eager_flows/chat-basic/flow.flex.yaml @@ -0,0 +1,5 @@ +$schema: https://azuremlschemas.azureedge.net/promptflow/latest/Flow.schema.json +entry: flow:ChatFlow +environment: + # image: mcr.microsoft.com/azureml/promptflow/promptflow-python + python_requirements_txt: requirements.txt diff --git a/src/promptflow/tests/test_configs/eager_flows/chat-basic/flow.py b/src/promptflow/tests/test_configs/eager_flows/chat-basic/flow.py new file mode 100644 index 00000000000..78c78a23ebf --- /dev/null +++ b/src/promptflow/tests/test_configs/eager_flows/chat-basic/flow.py @@ -0,0 +1,47 @@ +from dataclasses import dataclass +from pathlib import Path + +from promptflow.tracing import trace +from promptflow.core import AzureOpenAIModelConfiguration, Prompty + +BASE_DIR = Path(__file__).absolute().parent + + +@dataclass +class Result: + answer: str + + +class ChatFlow: + def __init__(self, model_config: AzureOpenAIModelConfiguration): + self.model_config = model_config + + @trace + def __call__( + self, question: str = "What is ChatGPT?", chat_history: list = None + ) -> Result: + """Flow entry function.""" + + chat_history = chat_history or [] + + prompty = Prompty.load( + source=BASE_DIR / "chat.prompty", + model={"configuration": self.model_config}, + ) + + # output is a string + output = prompty(question=question, chat_history=chat_history) + + return Result(answer=output) + + +if __name__ == "__main__": + from promptflow.tracing import start_trace + + start_trace() + config = AzureOpenAIModelConfiguration( + connection="open_ai_connection", azure_deployment="gpt-35-turbo" + ) + flow = ChatFlow(config) + result = flow("What's Azure Machine Learning?", []) + print(result) diff --git a/src/promptflow/tests/test_configs/eager_flows/chat-basic/init.json b/src/promptflow/tests/test_configs/eager_flows/chat-basic/init.json new file mode 100644 index 00000000000..07cf7a1845f --- /dev/null +++ b/src/promptflow/tests/test_configs/eager_flows/chat-basic/init.json @@ -0,0 +1,6 @@ +{ + "model_config": { + "connection": "open_ai_connection", + "azure_deployment": "gpt-35-turbo" + } +} \ No newline at end of file diff --git a/src/promptflow/tests/test_configs/eager_flows/chat-basic/paths.py b/src/promptflow/tests/test_configs/eager_flows/chat-basic/paths.py new file mode 100644 index 00000000000..b43e12469e8 --- /dev/null +++ b/src/promptflow/tests/test_configs/eager_flows/chat-basic/paths.py @@ -0,0 +1,6 @@ +import sys +import pathlib + +# Add the path to the evaluation module +code_path = str(pathlib.Path(__file__).parent / "../eval-checklist") +sys.path.insert(0, code_path) diff --git a/src/promptflow/tests/test_configs/eager_flows/chat-basic/requirements-azure.txt b/src/promptflow/tests/test_configs/eager_flows/chat-basic/requirements-azure.txt new file mode 100644 index 00000000000..f72e46bfbb6 --- /dev/null +++ b/src/promptflow/tests/test_configs/eager_flows/chat-basic/requirements-azure.txt @@ -0,0 +1 @@ +promptflow-azure diff --git a/src/promptflow/tests/test_configs/eager_flows/chat-basic/requirements.txt b/src/promptflow/tests/test_configs/eager_flows/chat-basic/requirements.txt new file mode 100644 index 00000000000..8178d51a677 --- /dev/null +++ b/src/promptflow/tests/test_configs/eager_flows/chat-basic/requirements.txt @@ -0,0 +1 @@ +promptflow[azure]>=1.10.0 \ No newline at end of file diff --git a/src/promptflow/tests/test_configs/eager_flows/chat-basic/run.yml b/src/promptflow/tests/test_configs/eager_flows/chat-basic/run.yml new file mode 100644 index 00000000000..5aba67e82e2 --- /dev/null +++ b/src/promptflow/tests/test_configs/eager_flows/chat-basic/run.yml @@ -0,0 +1,9 @@ +$schema: https://azuremlschemas.azureedge.net/promptflow/latest/Run.schema.json +flow: . +data: data.jsonl +init: + model_config: + connection: open_ai_connection + azure_deployment: gpt-35-turbo +column_mapping: + question: ${data.question} diff --git a/src/promptflow/tests/test_configs/eager_flows/chat-basic/sample.json b/src/promptflow/tests/test_configs/eager_flows/chat-basic/sample.json new file mode 100644 index 00000000000..b1af8226725 --- /dev/null +++ b/src/promptflow/tests/test_configs/eager_flows/chat-basic/sample.json @@ -0,0 +1 @@ +{"question": "What is Prompt flow?"} \ No newline at end of file From 6d49c8794f3c2c41acdd76c1bf06be350945d5d7 Mon Sep 17 00:00:00 2001 From: Honglin Date: Sat, 11 May 2024 19:40:03 +0800 Subject: [PATCH 7/7] [SDK/CLI] Use new trace link instead of original run portal link (#3193) # Description ![image](https://github.com/microsoft/promptflow/assets/2572521/445273b1-edab-461f-934f-4e3eb7299b19) # All Promptflow Contribution checklist: - [ ] **The pull request does not introduce [breaking changes].** - [ ] **CHANGELOG is updated for new features, bug fixes or other significant changes.** - [ ] **I have read the [contribution guidelines](../CONTRIBUTING.md).** - [ ] **Create an issue and link to the pull request to get dedicated review from promptflow team. Learn more: [suggested workflow](../CONTRIBUTING.md#suggested-workflow).** ## General Guidelines and Best Practices - [ ] Title of the pull request is clear and informative. - [ ] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, [see this page](https://github.com/Azure/azure-powershell/blob/master/documentation/development-docs/cleaning-up-commits.md). ### Testing Guidelines - [ ] Pull request includes test coverage for the included changes. --- .../promptflow/azure/_restclient/README.md | 1 + .../azure/_restclient/flow/models/_models.py | 6 + .../_restclient/flow/models/_models_py3.py | 7 + .../promptflow/azure/_restclient/swagger.json | 4 + .../azure/operations/_run_operations.py | 15 +- src/promptflow-devkit/CHANGELOG.md | 3 + .../promptflow/_sdk/_configuration.py | 5 + .../_sdk/_orchestrator/run_submitter.py | 13 +- .../promptflow/_sdk/_orm/run_info.py | 7 +- .../promptflow/_sdk/_tracing.py | 17 +- .../promptflow/_sdk/entities/_run.py | 4 + ..._operations_TestFlowRun_test_show_run.yaml | 6608 ++++++++++++----- src/promptflow/CHANGELOG.md | 3 + 13 files changed, 4984 insertions(+), 1709 deletions(-) diff --git a/src/promptflow-azure/promptflow/azure/_restclient/README.md b/src/promptflow-azure/promptflow/azure/_restclient/README.md index fae723d25f7..b019356ba8c 100644 --- a/src/promptflow-azure/promptflow/azure/_restclient/README.md +++ b/src/promptflow-azure/promptflow/azure/_restclient/README.md @@ -25,6 +25,7 @@ Download swagger.json from [here](https://int.api.azureml-test.ms/flow/swagger/v - 2024.3.14 - [Add enable_multi_container](https://github.com/microsoft/promptflow/pull/2313) - 2024.4.7 - [Update SDK restclient](https://github.com/microsoft/promptflow/pull/2670) - 2024.5.9 - [Support init Cosmos DB with setup API](https://github.com/microsoft/promptflow/pull/3167) +- 2024.5.10 - [Use new trace link instead of original run portal link](https://github.com/microsoft/promptflow/pull/3193) ## Troubleshooting diff --git a/src/promptflow-azure/promptflow/azure/_restclient/flow/models/_models.py b/src/promptflow-azure/promptflow/azure/_restclient/flow/models/_models.py index ec6232d0867..3660a3df07b 100644 --- a/src/promptflow-azure/promptflow/azure/_restclient/flow/models/_models.py +++ b/src/promptflow-azure/promptflow/azure/_restclient/flow/models/_models.py @@ -19451,6 +19451,8 @@ class FlowRunInfo(msrest.serialization.Model): :vartype session_id: str :ivar studio_portal_endpoint: :vartype studio_portal_endpoint: str + :ivar studio_portal_trace_endpoint: + :vartype studio_portal_trace_endpoint: str """ _attribute_map = { @@ -19476,6 +19478,7 @@ class FlowRunInfo(msrest.serialization.Model): 'flow_snapshot_id': {'key': 'flowSnapshotId', 'type': 'str'}, 'session_id': {'key': 'sessionId', 'type': 'str'}, 'studio_portal_endpoint': {'key': 'studioPortalEndpoint', 'type': 'str'}, + 'studio_portal_trace_endpoint': {'key': 'studioPortalTraceEndpoint', 'type': 'str'}, } def __init__( @@ -19528,6 +19531,8 @@ def __init__( :paramtype session_id: str :keyword studio_portal_endpoint: :paramtype studio_portal_endpoint: str + :keyword studio_portal_trace_endpoint: + :paramtype studio_portal_trace_endpoint: str """ super(FlowRunInfo, self).__init__(**kwargs) self.flow_graph = kwargs.get('flow_graph', None) @@ -19552,6 +19557,7 @@ def __init__( self.flow_snapshot_id = kwargs.get('flow_snapshot_id', None) self.session_id = kwargs.get('session_id', None) self.studio_portal_endpoint = kwargs.get('studio_portal_endpoint', None) + self.studio_portal_trace_endpoint = kwargs.get('studio_portal_trace_endpoint', None) class FlowRunResult(msrest.serialization.Model): diff --git a/src/promptflow-azure/promptflow/azure/_restclient/flow/models/_models_py3.py b/src/promptflow-azure/promptflow/azure/_restclient/flow/models/_models_py3.py index 8070a2313cd..d2a7f94551b 100644 --- a/src/promptflow-azure/promptflow/azure/_restclient/flow/models/_models_py3.py +++ b/src/promptflow-azure/promptflow/azure/_restclient/flow/models/_models_py3.py @@ -21962,6 +21962,8 @@ class FlowRunInfo(msrest.serialization.Model): :vartype session_id: str :ivar studio_portal_endpoint: :vartype studio_portal_endpoint: str + :ivar studio_portal_trace_endpoint: + :vartype studio_portal_trace_endpoint: str """ _attribute_map = { @@ -21987,6 +21989,7 @@ class FlowRunInfo(msrest.serialization.Model): 'flow_snapshot_id': {'key': 'flowSnapshotId', 'type': 'str'}, 'session_id': {'key': 'sessionId', 'type': 'str'}, 'studio_portal_endpoint': {'key': 'studioPortalEndpoint', 'type': 'str'}, + 'studio_portal_trace_endpoint': {'key': 'studioPortalTraceEndpoint', 'type': 'str'}, } def __init__( @@ -22014,6 +22017,7 @@ def __init__( flow_snapshot_id: Optional[str] = None, session_id: Optional[str] = None, studio_portal_endpoint: Optional[str] = None, + studio_portal_trace_endpoint: Optional[str] = None, **kwargs ): """ @@ -22062,6 +22066,8 @@ def __init__( :paramtype session_id: str :keyword studio_portal_endpoint: :paramtype studio_portal_endpoint: str + :keyword studio_portal_trace_endpoint: + :paramtype studio_portal_trace_endpoint: str """ super(FlowRunInfo, self).__init__(**kwargs) self.flow_graph = flow_graph @@ -22086,6 +22092,7 @@ def __init__( self.flow_snapshot_id = flow_snapshot_id self.session_id = session_id self.studio_portal_endpoint = studio_portal_endpoint + self.studio_portal_trace_endpoint = studio_portal_trace_endpoint class FlowRunResult(msrest.serialization.Model): diff --git a/src/promptflow-azure/promptflow/azure/_restclient/swagger.json b/src/promptflow-azure/promptflow/azure/_restclient/swagger.json index e7b0b090a5f..c0868136fc8 100644 --- a/src/promptflow-azure/promptflow/azure/_restclient/swagger.json +++ b/src/promptflow-azure/promptflow/azure/_restclient/swagger.json @@ -17879,6 +17879,10 @@ "studioPortalEndpoint": { "type": "string", "nullable": true + }, + "studioPortalTraceEndpoint": { + "type": "string", + "nullable": true } }, "additionalProperties": false diff --git a/src/promptflow-azure/promptflow/azure/operations/_run_operations.py b/src/promptflow-azure/promptflow/azure/operations/_run_operations.py index 789b29b916f..89179c1bddc 100644 --- a/src/promptflow-azure/promptflow/azure/operations/_run_operations.py +++ b/src/promptflow-azure/promptflow/azure/operations/_run_operations.py @@ -142,8 +142,8 @@ def _get_run_portal_url(self, run_id: str): except Exception as e: logger.warning(f"Failed to get run portal url from pfs for run {run_id!r}: {str(e)}") - if run_info and hasattr(run_info, "studio_portal_endpoint"): - portal_url = run_info.studio_portal_endpoint + if run_info and hasattr(run_info, "studio_portal_trace_endpoint"): + portal_url = run_info.studio_portal_trace_endpoint return portal_url @@ -936,7 +936,7 @@ def download( logger.info(f"Successfully downloaded run {run!r} to {result_path!r}.") return result_path - def _upload(self, run: Union[str, Run]): + def _upload(self, run: Union[str, Run]) -> str: from promptflow._sdk._pf_client import PFClient from promptflow.azure.operations._async_run_uploader import AsyncRunUploader @@ -973,16 +973,19 @@ def _upload(self, run: Union[str, Run]): logger.debug(f"Successfully uploaded run details of {run!r} to cloud.") # registry the run in the cloud - self._registry_existing_bulk_run(run=run) + self._register_existing_bulk_run(run=run) # post process after run upload, it can only be done after the run history record is created async_run_allowing_running_loop(run_uploader.post_process) + portal_url = self._get_run_portal_url(run_id=run.name) # print portal url when executing in jupyter notebook if in_jupyter_notebook(): - print(f"Portal url: {self._get_run_portal_url(run_id=run.name)}") + print(f"Portal url: {portal_url}") + + return portal_url - def _registry_existing_bulk_run(self, run: Run): + def _register_existing_bulk_run(self, run: Run): """Register the run in the cloud""" rest_obj = run._to_rest_object() self._service_caller.create_existing_bulk_run( diff --git a/src/promptflow-devkit/CHANGELOG.md b/src/promptflow-devkit/CHANGELOG.md index 7f0401a9889..b0ab474ef9f 100644 --- a/src/promptflow-devkit/CHANGELOG.md +++ b/src/promptflow-devkit/CHANGELOG.md @@ -2,6 +2,9 @@ ## v1.11.0 (Upcoming) +### Features Added +- Upload local run details to cloud when trace destination is configured to cloud. + ### Improvements - Interactive browser credential is excluded by default when using Azure AI connections, user could set `PF_NO_INTERACTIVE_LOGIN=False` to enable it. - Visualize flex flow run(s) switches to trace UI page. diff --git a/src/promptflow-devkit/promptflow/_sdk/_configuration.py b/src/promptflow-devkit/promptflow/_sdk/_configuration.py index 924b49f5e91..3611adfcc49 100644 --- a/src/promptflow-devkit/promptflow/_sdk/_configuration.py +++ b/src/promptflow-devkit/promptflow/_sdk/_configuration.py @@ -16,6 +16,7 @@ DEFAULT_ENCODING, FLOW_DIRECTORY_MACRO_IN_CONFIG, HOME_PROMPT_FLOW_DIR, + REMOTE_URI_PREFIX, SERVICE_CONFIG_FILE, ) from promptflow._sdk._utilities.general_utils import call_from_extension, gen_uuid_by_compute_info, read_write_by_user @@ -215,6 +216,10 @@ def get_trace_destination(self, path: Optional[Path] = None) -> Optional[str]: logger.debug("trace destination does not need to be resolved, directly return...") return value + def _is_cloud_trace_destination(self, path: Optional[Path] = None) -> bool: + trace_destination = self.get_trace_destination(path=path) + return trace_destination and trace_destination.startswith(REMOTE_URI_PREFIX) + def _resolve_trace_destination(self, path: Optional[Path] = None) -> str: return "azureml:/" + self._get_workspace_from_config(path=path) diff --git a/src/promptflow-devkit/promptflow/_sdk/_orchestrator/run_submitter.py b/src/promptflow-devkit/promptflow/_sdk/_orchestrator/run_submitter.py index 9e83fc66478..15df829a15a 100644 --- a/src/promptflow-devkit/promptflow/_sdk/_orchestrator/run_submitter.py +++ b/src/promptflow-devkit/promptflow/_sdk/_orchestrator/run_submitter.py @@ -8,7 +8,7 @@ from typing import Union from promptflow._constants import SystemMetricKeys -from promptflow._sdk._constants import REMOTE_URI_PREFIX, ContextAttributeKey, FlowRunProperties +from promptflow._sdk._constants import ContextAttributeKey, FlowRunProperties from promptflow._sdk.entities._flows import Flow, Prompty from promptflow._sdk.entities._run import Run from promptflow._sdk.operations._local_storage_operations import LocalStorageOperations @@ -229,10 +229,9 @@ def _submit_bulk_run( ) # upload run to cloud if the trace destination is set to cloud - trace_destination = self._config.get_trace_destination(path=run._get_flow_dir().resolve()) - if trace_destination and trace_destination.startswith(REMOTE_URI_PREFIX): - logger.debug(f"Trace destination set to {trace_destination!r}, uploading run to cloud...") - self._upload_run_to_cloud(run=run, config=self._config) + if self._config._is_cloud_trace_destination(path=run._get_flow_dir().resolve()): + portal_url = self._upload_run_to_cloud(run=run, config=self._config) + self.run_operations.update(name=run.name, portal_url=portal_url) def _resolve_input_dirs(self, run: Run): result = {"data": run.data if run.data else None} @@ -264,7 +263,7 @@ def _validate_column_mapping(cls, column_mapping: dict): ) @classmethod - def _upload_run_to_cloud(cls, run: Run, config=None): + def _upload_run_to_cloud(cls, run: Run, config=None) -> str: error_msg_prefix = f"Failed to upload run {run.name!r} to cloud." try: from promptflow._sdk._tracing import _get_ws_triad_from_pf_config @@ -276,7 +275,7 @@ def _upload_run_to_cloud(cls, run: Run, config=None): resource_group=ws_triad.resource_group_name, workspace_name=ws_triad.workspace_name, ) - pf.runs._upload(run=run) + return pf.runs._upload(run=run) except ImportError as e: error_message = ( f'{error_msg_prefix}. "promptflow[azure]" is required for local to cloud tracing experience, ' diff --git a/src/promptflow-devkit/promptflow/_sdk/_orm/run_info.py b/src/promptflow-devkit/promptflow/_sdk/_orm/run_info.py index ec7e02781e1..c2e2fa009fa 100644 --- a/src/promptflow-devkit/promptflow/_sdk/_orm/run_info.py +++ b/src/promptflow-devkit/promptflow/_sdk/_orm/run_info.py @@ -49,10 +49,11 @@ class RunInfo(Base): end_time = Column(TEXT) # ISO8601("YYYY-MM-DD HH:MM:SS.SSS"), string data = Column(TEXT) # local path of original run data, string run_source = Column(TEXT) # run source, string + portal_url = Column(TEXT) # portal url when trace destination is set to cloud, string __table_args__ = (Index(RUN_INFO_CREATED_ON_INDEX_NAME, "created_on"),) # schema version, increase the version number when you change the schema - __pf_schema_version__ = "3" + __pf_schema_version__ = "4" @sqlite_retry def dump(self) -> None: @@ -96,6 +97,7 @@ def update( start_time: Optional[Union[str, datetime.datetime]] = None, end_time: Optional[Union[str, datetime.datetime]] = None, system_metrics: Optional[Dict[str, int]] = None, + portal_url: Optional[str] = None, ) -> None: update_dict = {} if status is not None: @@ -116,6 +118,9 @@ def update( if end_time is not None: self.end_time = end_time if isinstance(end_time, str) else end_time.isoformat() update_dict["end_time"] = self.end_time + if portal_url is not None: + self.portal_url = portal_url + update_dict["portal_url"] = self.portal_url with mgmt_db_session() as session: # if not update system metrics, we can directly update the row; # otherwise, we need to get properties first, update the dict and finally update the row diff --git a/src/promptflow-devkit/promptflow/_sdk/_tracing.py b/src/promptflow-devkit/promptflow/_sdk/_tracing.py index fbe218b5ad1..d3ab65c0b7c 100644 --- a/src/promptflow-devkit/promptflow/_sdk/_tracing.py +++ b/src/promptflow-devkit/promptflow/_sdk/_tracing.py @@ -310,7 +310,10 @@ def _print_tracing_url_from_azure_portal( return url = url.format(query=query) - print(f"You can view the traces in cloud from Azure portal: {url}") + print( + f"You can view the traces in the cloud from the Azure portal. " + f"Please note that the page may remain empty for a while until the traces are successfully uploaded:\n{url}." + ) def _inject_res_attrs_to_environ( @@ -371,7 +374,9 @@ def start_trace_with_devkit(collection: str, **kwargs: typing.Any) -> None: _logger.debug("kwargs: %s", kwargs) attrs = kwargs.get("attributes", None) run = kwargs.get("run", None) + flow_path = None if isinstance(run, Run): + flow_path = run._get_flow_dir().resolve() run_config = run._config run = run.name else: @@ -440,7 +445,15 @@ def start_trace_with_devkit(collection: str, **kwargs: typing.Any) -> None: return # print tracing url(s) when run is specified _print_tracing_url_from_local(pfs_port=pfs_port, collection=collection, exp=exp, run=run) - if ws_triad is not None and is_azure_ext_installed: + + if run is not None and run_config._is_cloud_trace_destination(path=flow_path): + trace_destination = run_config.get_trace_destination(path=flow_path) + print( + f"You can view the traces in azure portal since trace destination is set to: {trace_destination}. " + f"The link will be printed once the run is finished." + ) + elif ws_triad is not None and is_azure_ext_installed: + # if run does not exist, print collection trace link _print_tracing_url_from_azure_portal(ws_triad=ws_triad, collection=collection, exp=exp, run=run) diff --git a/src/promptflow-devkit/promptflow/_sdk/entities/_run.py b/src/promptflow-devkit/promptflow/_sdk/entities/_run.py index ce4479ee617..a2582f0bdec 100644 --- a/src/promptflow-devkit/promptflow/_sdk/entities/_run.py +++ b/src/promptflow-devkit/promptflow/_sdk/entities/_run.py @@ -335,6 +335,7 @@ def _from_orm_object(cls, obj: ORMRun) -> "Run": command=properties_json.get(FlowRunProperties.COMMAND, None), outputs=properties_json.get(FlowRunProperties.OUTPUTS, None), column_mapping=properties_json.get(FlowRunProperties.COLUMN_MAPPING, None), + portal_url=obj.portal_url, ) @classmethod @@ -428,6 +429,7 @@ def _to_orm_object(self) -> ORMRun: properties=json.dumps(self.properties, default=asdict), data=Path(self.data).resolve().absolute().as_posix() if self.data else None, run_source=self._run_source, + portal_url=self._portal_url, ) def _dump(self) -> None: @@ -474,6 +476,8 @@ def _to_dict( if exclude_debug_info: exception_dict.pop("debugInfo", None) result["error"] = exception_dict + if self._portal_url: + result[RunDataKeys.PORTAL_URL] = self._portal_url elif self._run_source == RunInfoSources.INDEX_SERVICE: result["creation_context"] = self._creation_context result["flow_name"] = self._experiment_name diff --git a/src/promptflow-recording/recordings/azure/test_run_operations_TestFlowRun_test_show_run.yaml b/src/promptflow-recording/recordings/azure/test_run_operations_TestFlowRun_test_show_run.yaml index 2d76aa99ff2..1c771d8cb63 100644 --- a/src/promptflow-recording/recordings/azure/test_run_operations_TestFlowRun_test_show_run.yaml +++ b/src/promptflow-recording/recordings/azure/test_run_operations_TestFlowRun_test_show_run.yaml @@ -5,12 +5,12 @@ interactions: Accept: - application/json Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive User-Agent: - - promptflow-sdk/0.0.1 azure-ai-ml/1.12.1 azsdk-python-mgmt-machinelearningservices/0.1.0 - Python/3.10.13 (Windows-10-10.0.22631-SP0) + - promptflow-azure-sdk/1.9.0.dev0 azure-ai-ml/1.15.0 azsdk-python-mgmt-machinelearningservices/0.1.0 + Python/3.9.19 (Windows-10-10.0.22631-SP0) method: GET uri: https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000 response: @@ -23,7 +23,7 @@ interactions: cache-control: - no-cache content-length: - - '3630' + - '3761' content-type: - application/json; charset=utf-8 expires: @@ -32,14 +32,14 @@ interactions: - no-cache strict-transport-security: - max-age=31536000; includeSubDomains - transfer-encoding: - - chunked vary: - - Accept-Encoding,Accept-Encoding + - Accept-Encoding + x-cache: + - CONFIG_NOCACHE x-content-type-options: - nosniff x-request-time: - - '0.026' + - '0.041' status: code: 200 message: OK @@ -49,12 +49,12 @@ interactions: Accept: - application/json Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive User-Agent: - - promptflow-sdk/0.0.1 azure-ai-ml/1.12.1 azsdk-python-mgmt-machinelearningservices/0.1.0 - Python/3.10.13 (Windows-10-10.0.22631-SP0) + - promptflow-azure-sdk/1.9.0.dev0 azure-ai-ml/1.15.0 azsdk-python-mgmt-machinelearningservices/0.1.0 + Python/3.9.19 (Windows-10-10.0.22631-SP0) method: GET uri: https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/datastores?count=30&isDefault=true&orderByAsc=false response: @@ -84,14 +84,14 @@ interactions: - no-cache strict-transport-security: - max-age=31536000; includeSubDomains - transfer-encoding: - - chunked vary: - - Accept-Encoding,Accept-Encoding + - Accept-Encoding + x-cache: + - CONFIG_NOCACHE x-content-type-options: - nosniff x-request-time: - - '0.151' + - '0.073' status: code: 200 message: OK @@ -101,12 +101,12 @@ interactions: Accept: - application/json Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive User-Agent: - - promptflow-sdk/0.0.1 azure-ai-ml/1.12.1 azsdk-python-mgmt-machinelearningservices/0.1.0 - Python/3.10.13 (Windows-10-10.0.22631-SP0) + - promptflow-azure-sdk/1.9.0.dev0 azure-ai-ml/1.15.0 azsdk-python-mgmt-machinelearningservices/0.1.0 + Python/3.9.19 (Windows-10-10.0.22631-SP0) method: GET uri: https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/datastores/workspaceblobstore response: @@ -136,14 +136,14 @@ interactions: - no-cache strict-transport-security: - max-age=31536000; includeSubDomains - transfer-encoding: - - chunked vary: - - Accept-Encoding,Accept-Encoding + - Accept-Encoding + x-cache: + - CONFIG_NOCACHE x-content-type-options: - nosniff x-request-time: - - '0.095' + - '0.086' status: code: 200 message: OK @@ -153,14 +153,14 @@ interactions: Accept: - application/json Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive Content-Length: - '0' User-Agent: - - promptflow-sdk/0.0.1 azure-ai-ml/1.12.1 azsdk-python-mgmt-machinelearningservices/0.1.0 - Python/3.10.13 (Windows-10-10.0.22631-SP0) + - promptflow-azure-sdk/1.9.0.dev0 azure-ai-ml/1.15.0 azsdk-python-mgmt-machinelearningservices/0.1.0 + Python/3.9.19 (Windows-10-10.0.22631-SP0) method: POST uri: https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/datastores/workspaceblobstore/listSecrets response: @@ -179,14 +179,12 @@ interactions: - no-cache strict-transport-security: - max-age=31536000; includeSubDomains - transfer-encoding: - - chunked - vary: - - Accept-Encoding + x-cache: + - CONFIG_NOCACHE x-content-type-options: - nosniff x-request-time: - - '0.224' + - '0.212' status: code: 200 message: OK @@ -196,13 +194,13 @@ interactions: Accept: - application/xml Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive User-Agent: - - azsdk-python-storage-blob/12.19.0 Python/3.10.13 (Windows-10-10.0.22631-SP0) + - azsdk-python-storage-blob/12.19.1 Python/3.9.19 (Windows-10-10.0.22631-SP0) x-ms-date: - - Fri, 12 Jan 2024 07:51:37 GMT + - Fri, 10 May 2024 08:12:59 GMT x-ms-version: - '2023-11-03' method: HEAD @@ -246,13 +244,13 @@ interactions: Accept: - application/xml Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive User-Agent: - - azsdk-python-storage-blob/12.19.0 Python/3.10.13 (Windows-10-10.0.22631-SP0) + - azsdk-python-storage-blob/12.19.1 Python/3.9.19 (Windows-10-10.0.22631-SP0) x-ms-date: - - Fri, 12 Jan 2024 07:51:38 GMT + - Fri, 10 May 2024 08:13:00 GMT x-ms-version: - '2023-11-03' method: HEAD @@ -280,12 +278,12 @@ interactions: Accept: - application/json Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive User-Agent: - - promptflow-sdk/0.0.1 azure-ai-ml/1.12.1 azsdk-python-mgmt-machinelearningservices/0.1.0 - Python/3.10.13 (Windows-10-10.0.22631-SP0) + - promptflow-azure-sdk/1.9.0.dev0 azure-ai-ml/1.15.0 azsdk-python-mgmt-machinelearningservices/0.1.0 + Python/3.9.19 (Windows-10-10.0.22631-SP0) method: GET uri: https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/datastores/workspaceblobstore response: @@ -315,14 +313,14 @@ interactions: - no-cache strict-transport-security: - max-age=31536000; includeSubDomains - transfer-encoding: - - chunked vary: - - Accept-Encoding,Accept-Encoding + - Accept-Encoding + x-cache: + - CONFIG_NOCACHE x-content-type-options: - nosniff x-request-time: - - '0.070' + - '0.099' status: code: 200 message: OK @@ -332,14 +330,14 @@ interactions: Accept: - application/json Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive Content-Length: - '0' User-Agent: - - promptflow-sdk/0.0.1 azure-ai-ml/1.12.1 azsdk-python-mgmt-machinelearningservices/0.1.0 - Python/3.10.13 (Windows-10-10.0.22631-SP0) + - promptflow-azure-sdk/1.9.0.dev0 azure-ai-ml/1.15.0 azsdk-python-mgmt-machinelearningservices/0.1.0 + Python/3.9.19 (Windows-10-10.0.22631-SP0) method: POST uri: https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/datastores/workspaceblobstore/listSecrets response: @@ -358,14 +356,12 @@ interactions: - no-cache strict-transport-security: - max-age=31536000; includeSubDomains - transfer-encoding: - - chunked - vary: - - Accept-Encoding + x-cache: + - CONFIG_NOCACHE x-content-type-options: - nosniff x-request-time: - - '0.136' + - '0.104' status: code: 200 message: OK @@ -375,13 +371,13 @@ interactions: Accept: - application/xml Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive User-Agent: - - azsdk-python-storage-blob/12.19.0 Python/3.10.13 (Windows-10-10.0.22631-SP0) + - azsdk-python-storage-blob/12.19.1 Python/3.9.19 (Windows-10-10.0.22631-SP0) x-ms-date: - - Fri, 12 Jan 2024 07:51:41 GMT + - Fri, 10 May 2024 08:13:03 GMT x-ms-version: - '2023-11-03' method: HEAD @@ -393,13 +389,13 @@ interactions: accept-ranges: - bytes content-length: - - '266' + - '250' content-md5: - - UZm3TyOoKWjSR23+Up6qUA== + - CT1FTZp5JScB8fq+HjnINw== content-type: - application/octet-stream last-modified: - - Tue, 19 Dec 2023 06:05:25 GMT + - Fri, 12 Apr 2024 16:00:21 GMT server: - Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0 vary: @@ -407,9 +403,9 @@ interactions: x-ms-blob-type: - BlockBlob x-ms-creation-time: - - Tue, 19 Dec 2023 06:05:25 GMT + - Fri, 12 Apr 2024 16:00:20 GMT x-ms-meta-name: - - 7b68bf5e-6ef4-4eb3-9f49-28f9a5baad87 + - a8efcf33-8cda-4851-b4aa-2a5fc6271890 x-ms-meta-upload_status: - completed x-ms-meta-version: @@ -425,13 +421,13 @@ interactions: Accept: - application/xml Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive User-Agent: - - azsdk-python-storage-blob/12.19.0 Python/3.10.13 (Windows-10-10.0.22631-SP0) + - azsdk-python-storage-blob/12.19.1 Python/3.9.19 (Windows-10-10.0.22631-SP0) x-ms-date: - - Fri, 12 Jan 2024 07:51:43 GMT + - Fri, 10 May 2024 08:13:04 GMT x-ms-version: - '2023-11-03' method: HEAD @@ -457,25 +453,26 @@ interactions: body: '{"flowDefinitionDataStoreName": "workspaceblobstore", "flowDefinitionBlobPath": "LocalUpload/000000000000000000000000000000000000/hello-world/flow.dag.yaml", "runId": "batch_run_name", "runDisplayName": "sdk-cli-test-fixture-batch-run-without-llm", - "runExperimentName": "", "batchDataInput": {"dataUri": "azureml://datastores/workspaceblobstore/paths/LocalUpload/000000000000000000000000000000000000/webClassification3.jsonl"}, - "inputsMapping": {"name": "${data.url}"}, "connections": {}, "environmentVariables": - {}, "runtimeName": "fake-runtime-name", "sessionId": "000000000000000000000000000000000000000000000000", + "runExperimentName": "", "sessionId": "000000000000000000000000000000000000000000000000", "sessionSetupMode": "SystemWait", "flowLineageId": "0000000000000000000000000000000000000000000000000000000000000000", - "runDisplayNameGenerationType": "UserProvidedMacro"}' + "runDisplayNameGenerationType": "UserProvidedMacro", "batchDataInput": {"dataUri": + "azureml://datastores/workspaceblobstore/paths/LocalUpload/000000000000000000000000000000000000/webClassification3.jsonl"}, + "inputsMapping": {"name": "${data.url}"}, "environmentVariables": {}, "connections": + {}, "runtimeName": "fake-runtime-name"}' headers: Accept: - application/json Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive Content-Length: - - '812' + - '806' Content-Type: - application/json User-Agent: - - promptflow-sdk/0.0.1 azsdk-python-azuremachinelearningdesignerserviceclient/unknown - Python/3.10.13 (Windows-10-10.0.22631-SP0) + - promptflow-azure-sdk/1.9.0.dev0 azsdk-python-azuremachinelearningdesignerserviceclient/unknown + Python/3.9.19 (Windows-10-10.0.22631-SP0) method: POST uri: https://eastus.api.azureml.ms/flow/api/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/BulkRuns/submit response: @@ -489,11 +486,11 @@ interactions: content-type: - application/json; charset=utf-8 strict-transport-security: - - max-age=15724800; includeSubDomains; preload + - max-age=31536000; includeSubDomains; preload x-content-type-options: - nosniff x-request-time: - - '7.941' + - '6.673' status: code: 200 message: OK @@ -503,178 +500,34 @@ interactions: Accept: - application/json Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive User-Agent: - - promptflow-sdk/0.0.1 azsdk-python-azuremachinelearningdesignerserviceclient/unknown - Python/3.10.13 (Windows-10-10.0.22631-SP0) + - promptflow-azure-sdk/1.9.0.dev0 azsdk-python-azuremachinelearningdesignerserviceclient/unknown + Python/3.9.19 (Windows-10-10.0.22631-SP0) method: GET uri: https://eastus.api.azureml.ms/flow/api/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/BulkRuns/batch_run_name response: body: - string: '{"flowGraph": {"nodes": [{"name": "hello_world", "type": "python", - "source": {"type": "code", "path": "hello_world.py"}, "inputs": {"name": "${inputs.name}"}, - "tool": "hello_world.py", "reduce": false}], "tools": [{"name": "Content Safety - (Text Analyze)", "type": "python", "inputs": {"connection": {"type": ["AzureContentSafetyConnection"], - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "hate_category": {"type": ["string"], "default": "medium_sensitivity", "enum": - ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "self_harm_category": {"type": ["string"], "default": "medium_sensitivity", - "enum": ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "sexual_category": {"type": ["string"], "default": "medium_sensitivity", "enum": - ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "text": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "violence_category": {"type": ["string"], - "default": "medium_sensitivity", "enum": ["disable", "low_sensitivity", "medium_sensitivity", - "high_sensitivity"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}}, "description": "Use Azure Content Safety to detect - harmful content.", "module": "promptflow.tools.azure_content_safety", "function": - "analyze_text", "is_builtin": true, "package": "promptflow-tools", "package_version": - "0.0.216", "enable_kwargs": false, "deprecated_tools": ["content_safety_text.tools.content_safety_text_tool.analyze_text"], - "tool_state": "stable"}, {"name": "Embedding", "type": "python", "inputs": - {"connection": {"type": ["AzureOpenAIConnection", "OpenAIConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "deployment_name": - {"type": ["string"], "enabled_by": "connection", "enabled_by_type": ["AzureOpenAIConnection"], - "model_list": ["text-embedding-ada-002", "text-search-ada-doc-001", "text-search-ada-query-001"], - "capabilities": {"completion": false, "chat_completion": false, "embeddings": - true}, "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default"}, "input": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "model": {"type": ["string"], "enum": ["text-embedding-ada-002", - "text-search-ada-doc-001", "text-search-ada-query-001"], "enabled_by": "connection", - "enabled_by_type": ["OpenAIConnection"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Use Open AI''s embedding - model to create an embedding vector representing the input text.", "module": - "promptflow.tools.embedding", "function": "embedding", "is_builtin": true, - "package": "promptflow-tools", "package_version": "0.0.216", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "Open Source LLM", "type": "custom_llm", - "inputs": {"api": {"type": ["string"], "enum": ["chat", "completion"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "connection": {"type": - ["CustomConnection"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "deployment_name": {"type": ["string"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "endpoint_name": - {"type": ["string"], "default": "-- please enter an endpoint name --", "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "max_new_tokens": - {"type": ["int"], "default": 500, "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "model_kwargs": {"type": ["object"], "default": - "{}", "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default", "advanced": true}, "temperature": {"type": ["double"], "default": - 1.0, "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default"}, "top_p": {"type": ["double"], "default": 1.0, "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default", "advanced": true}}, - "description": "Use an Open Source model from the Azure Model catalog, deployed - to an AzureML Online Endpoint for LLM Chat or Completion API calls.", "module": - "promptflow.tools.open_source_llm", "class_name": "OpenSourceLLM", "function": - "call", "icon": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAACgElEQVR4nGWSz2vcVRTFP/e9NzOZ1KDGohASslLEH6VLV0ak4l/QpeDCrfQPcNGliODKnVm4EBdBsIjQIlhciKW0ycKFVCSNbYnjdDLtmPnmO/nO9917XcxMkjYX3uLx7nnn3HOuMK2Nix4fP78ZdrYXVkLVWjf3l3B1B+HpcjzGFtmqa6cePz7/x0dnn1n5qhj3iBJPYREIURAJuCtpY8PjReDbrf9WG7H1fuefwQU9qKztTcMJT+PNnEFvjGVDBDlSsH6p/9MLzy6+NxwVqI8RAg4IPmWedMckdLYP6O6UpIaQfvyyXG012+e79/ZfHukoS1ISMT2hGTB1RkUmNgQ5QZ0w+a2VWDq73MbdEWmfnnv6UWe7oNzPaLapl5CwuLTXK9WUGBuCjqekzhP+z52ZXOrKMD3OJg0Hh778aiOuvpnYvp05d6GJO4iAO4QAe/eV36/X5LFRV4Zmn+AdkqlL8Vjp3oVioOz+WTPzzYEgsN+fgPLYyJVheSbPPVl2ikeGZRjtG52/8rHuaV9VOlpP2OtKyVndcRVCSqOhsvxa4vW359i6OuKdD+aP8Q4SYPdOzS/flGjt1JUSaMqZ5nwa1Y8qWb/Ud/eZZkHisYezEM0m+fcelDr8F1SqW2LNK6r1jXQwyLzy1hxvrLXZulry7ocL+FS6G4QIu3fG/Px1gdYeW7LIgXU2P/115TOA5G7e3Rmj2aS/m7l5pThiZzrCcE/d1XHzbln373nw7y6veeoUm5KCNKT/IPPwbiY1hYd/l5MIT65BMFt87sU4v9D7/JMflr44uV6hGh1+L4RCkg6z5iK2tAhNLeLsNGwYA4fDYnC/drvuuFxe86NV/x+Ut27g0FvykgAAAABJRU5ErkJggg==", - "is_builtin": true, "package": "promptflow-tools", "package_version": "0.0.216", - "enable_kwargs": false, "tool_state": "stable"}, {"name": "OpenAI GPT-4V", - "type": "custom_llm", "inputs": {"connection": {"type": ["OpenAIConnection"], - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "frequency_penalty": {"type": ["double"], "default": 0, "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "max_tokens": {"type": - ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "model": {"type": ["string"], "enum": ["gpt-4-vision-preview"], - "allow_manual_entry": true, "is_multi_select": false, "input_type": "default"}, - "presence_penalty": {"type": ["double"], "default": 0, "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "stop": {"type": - ["list"], "default": "", "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "temperature": {"type": ["double"], "default": 1, - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "top_p": {"type": ["double"], "default": 1, "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Use OpenAI GPT-4V to leverage - vision ability.", "module": "promptflow.tools.openai_gpt4v", "class_name": - "OpenAI", "function": "chat", "is_builtin": true, "package": "promptflow-tools", - "package_version": "0.0.216", "default_prompt": "# system:\nAs an AI assistant, - your task involves interpreting images and responding to questions about the - image.\nRemember to provide accurate answers based on the information present - in the image.\n\n# user:\nCan you tell me what the image depicts?\n![image]({{image_input}})\n", - "enable_kwargs": false, "tool_state": "stable"}, {"name": "Serp API", "type": - "python", "inputs": {"connection": {"type": ["SerpConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "engine": {"type": - ["string"], "default": "google", "enum": ["google", "bing"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "location": {"type": - ["string"], "default": "", "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "num": {"type": ["int"], "default": "10", - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "query": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "safe": {"type": ["string"], "default": "off", - "enum": ["active", "off"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Use Serp API to obtain search - results from a specific search engine.", "module": "promptflow.tools.serpapi", - "class_name": "SerpAPI", "function": "search", "is_builtin": true, "package": - "promptflow-tools", "package_version": "0.0.216", "enable_kwargs": false, - "tool_state": "stable"}, {"name": "Faiss Index Lookup", "type": "python", - "inputs": {"path": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "vector": {"type": ["list"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Search vector based query - from the FAISS index file.", "module": "promptflow_vectordb.tool.faiss_index_lookup", - "class_name": "FaissIndexLookup", "function": "search", "is_builtin": true, - "package": "promptflow-vectordb", "package_version": "0.0.1", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "Vector DB Lookup", "type": "python", - "inputs": {"class_name": {"type": ["string"], "enabled_by": "connection", - "enabled_by_type": ["WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "collection_name": {"type": ["string"], "enabled_by": - "connection", "enabled_by_type": ["QdrantConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "connection": {"type": - ["CognitiveSearchConnection", "QdrantConnection", "WeaviateConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "index_name": {"type": - ["string"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection"], - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "search_filters": {"type": ["object"], "enabled_by": "connection", "enabled_by_type": - ["CognitiveSearchConnection", "QdrantConnection"], "allow_manual_entry": false, - "is_multi_select": false, "input_type": "default"}, "search_params": {"type": - ["object"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection", - "QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "text_field": {"type": ["string"], "enabled_by": - "connection", "enabled_by_type": ["CognitiveSearchConnection", "QdrantConnection", - "WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "vector": {"type": - ["list"], "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default"}, "vector_field": {"type": ["string"], "enabled_by": "connection", - "enabled_by_type": ["CognitiveSearchConnection"], "allow_manual_entry": false, - "is_multi_select": false, "input_type": "default"}}, "description": "Search - vector based query from existing Vector Database.", "module": "promptflow_vectordb.tool.vector_db_lookup", - "class_name": "VectorDBLookup", "function": "search", "is_builtin": true, - "package": "promptflow-vectordb", "package_version": "0.0.1", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "Vector Index Lookup", "type": "python", - "inputs": {"path": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "query": {"type": ["object"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "top_k": {"type": - ["int"], "default": "3", "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}}, "description": "Search text or vector based query - from AzureML Vector Index.", "module": "promptflow_vectordb.tool.vector_index_lookup", - "class_name": "VectorIndexLookup", "function": "search", "is_builtin": true, - "package": "promptflow-vectordb", "package_version": "0.0.1", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "hello_world.py", "type": "python", - "inputs": {"name": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "source": "hello_world.py", "function": - "hello_world", "is_builtin": false, "enable_kwargs": false, "tool_state": - "stable"}], "inputs": {"name": {"type": "string", "default": "hod", "is_chat_input": - false}}, "outputs": {"result": {"type": "string", "reference": "${hello_world.output}", - "evaluation_only": false, "is_chat_output": false}}}, "flowRunResourceId": - "azureml://locations/eastus/workspaces/00000/flows/batch_run_name/flowRuns/batch_run_name", + string: '{"flowRunResourceId": "azureml://locations/eastus/workspaces/00000/flows/batch_run_name/flowRuns/batch_run_name", "flowRunId": "batch_run_name", "flowRunDisplayName": "sdk-cli-test-fixture-batch-run-without-llm", "batchDataInput": {"dataUri": "azureml://datastores/workspaceblobstore/paths/LocalUpload/74c11bba717480b2d6b04b8e746d09d7/webClassification3.jsonl"}, - "flowRunType": "FlowRun", "flowType": "Default", "runtimeName": "test-runtime-ci", + "flowRunType": "FlowRun", "flowType": "Default", "runtimeName": "automatic", "inputsMapping": {"name": "${data.url}"}, "outputDatastoreName": "workspaceblobstore", "childRunBasePath": "promptflow/PromptFlowArtifacts/batch_run_name/flow_artifacts", - "flowDagFileRelativePath": "flow.dag.yaml", "flowSnapshotId": "38053316-7591-4ac8-b718-d76b16b48bbe", - "studioPortalEndpoint": "https://ml.azure.com/runs/batch_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000"}' + "sessionId": "4a6867a952826a2a51729e6bb4bf21ac65b46eec3ae5ed10", "studioPortalEndpoint": + "https://ml.azure.com/runs/batch_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000", + "studioPortalTraceEndpoint": "https://ml.azure.com/prompts/trace/run/batch_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/promptflow-eastus"}' headers: connection: - keep-alive content-length: - - '12912' + - '1277' content-type: - application/json; charset=utf-8 strict-transport-security: - - max-age=15724800; includeSubDomains; preload + - max-age=31536000; includeSubDomains; preload transfer-encoding: - chunked vary: @@ -682,7 +535,7 @@ interactions: x-content-type-options: - nosniff x-request-time: - - '0.407' + - '0.354' status: code: 200 message: OK @@ -692,20 +545,143 @@ interactions: Accept: - application/json Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive User-Agent: - - promptflow-sdk/0.0.1 azsdk-python-azuremachinelearningdesignerserviceclient/unknown - Python/3.10.13 (Windows-10-10.0.22631-SP0) + - promptflow-azure-sdk/1.9.0.dev0 azsdk-python-azuremachinelearningdesignerserviceclient/unknown + Python/3.9.19 (Windows-10-10.0.22631-SP0) + method: GET + uri: https://eastus.api.azureml.ms/flow/api/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/BulkRuns/batch_run_name + response: + body: + string: '{"flowRunResourceId": "azureml://locations/eastus/workspaces/00000/flows/batch_run_name/flowRuns/batch_run_name", + "flowRunId": "batch_run_name", "flowRunDisplayName": "sdk-cli-test-fixture-batch-run-without-llm", + "batchDataInput": {"dataUri": "azureml://datastores/workspaceblobstore/paths/LocalUpload/74c11bba717480b2d6b04b8e746d09d7/webClassification3.jsonl"}, + "flowRunType": "FlowRun", "flowType": "Default", "runtimeName": "automatic", + "inputsMapping": {"name": "${data.url}"}, "outputDatastoreName": "workspaceblobstore", + "childRunBasePath": "promptflow/PromptFlowArtifacts/batch_run_name/flow_artifacts", + "sessionId": "4a6867a952826a2a51729e6bb4bf21ac65b46eec3ae5ed10", "studioPortalEndpoint": + "https://ml.azure.com/runs/batch_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000", + "studioPortalTraceEndpoint": "https://ml.azure.com/prompts/trace/run/batch_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/promptflow-eastus"}' + headers: + connection: + - keep-alive + content-length: + - '1277' + content-type: + - application/json; charset=utf-8 + strict-transport-security: + - max-age=31536000; includeSubDomains; preload + transfer-encoding: + - chunked + vary: + - Accept-Encoding + x-content-type-options: + - nosniff + x-request-time: + - '0.319' + status: + code: 200 + message: OK +- request: + body: null + headers: + Accept: + - application/json + Accept-Encoding: + - gzip, deflate, br + Connection: + - keep-alive + User-Agent: + - promptflow-azure-sdk/1.9.0.dev0 azsdk-python-azuremachinelearningdesignerserviceclient/unknown + Python/3.9.19 (Windows-10-10.0.22631-SP0) + method: GET + uri: https://eastus.api.azureml.ms/flow/api/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/BulkRuns/batch_run_name + response: + body: + string: '{"flowRunResourceId": "azureml://locations/eastus/workspaces/00000/flows/batch_run_name/flowRuns/batch_run_name", + "flowRunId": "batch_run_name", "flowRunDisplayName": "sdk-cli-test-fixture-batch-run-without-llm", + "batchDataInput": {"dataUri": "azureml://datastores/workspaceblobstore/paths/LocalUpload/74c11bba717480b2d6b04b8e746d09d7/webClassification3.jsonl"}, + "flowRunType": "FlowRun", "flowType": "Default", "runtimeName": "automatic", + "inputsMapping": {"name": "${data.url}"}, "outputDatastoreName": "workspaceblobstore", + "childRunBasePath": "promptflow/PromptFlowArtifacts/batch_run_name/flow_artifacts", + "sessionId": "4a6867a952826a2a51729e6bb4bf21ac65b46eec3ae5ed10", "studioPortalEndpoint": + "https://ml.azure.com/runs/batch_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000", + "studioPortalTraceEndpoint": "https://ml.azure.com/prompts/trace/run/batch_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/promptflow-eastus"}' + headers: + connection: + - keep-alive + content-length: + - '1277' + content-type: + - application/json; charset=utf-8 + strict-transport-security: + - max-age=31536000; includeSubDomains; preload + transfer-encoding: + - chunked + vary: + - Accept-Encoding + x-content-type-options: + - nosniff + x-request-time: + - '0.187' + status: + code: 200 + message: OK +- request: + body: null + headers: + Accept: + - application/json + Accept-Encoding: + - gzip, deflate, br + Connection: + - keep-alive + User-Agent: + - promptflow-azure-sdk/1.9.0.dev0 azsdk-python-azuremachinelearningdesignerserviceclient/unknown + Python/3.9.19 (Windows-10-10.0.22631-SP0) method: GET uri: https://eastus.api.azureml.ms/flow/api/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/BulkRuns/batch_run_name response: body: string: '{"flowGraph": {"nodes": [{"name": "hello_world", "type": "python", "source": {"type": "code", "path": "hello_world.py"}, "inputs": {"name": "${inputs.name}"}, - "tool": "hello_world.py", "reduce": false}], "tools": [{"name": "Content Safety - (Text Analyze)", "type": "python", "inputs": {"connection": {"type": ["AzureContentSafetyConnection"], + "tool": "hello_world.py", "reduce": false}], "tools": [{"name": "Azure OpenAI + GPT-4 Turbo with Vision", "type": "custom_llm", "inputs": {"connection": {"type": + ["AzureOpenAIConnection"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 0}}, "deployment_name": + {"type": ["string"], "enabled_by": "connection", "dynamic_list": {"func_path": + "promptflow.tools.aoai_gpt4v.list_deployment_names", "func_kwargs": [{"name": + "connection", "reference": "${inputs.connection}", "type": ["AzureOpenAIConnection"]}]}, + "allow_manual_entry": true, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 1}}, "detail": {"type": ["string"], "default": "auto", + "enum": ["low", "high", "auto"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 9}}, "frequency_penalty": + {"type": ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 7}}, "max_tokens": {"type": + ["int"], "default": 512, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 4}}, "presence_penalty": {"type": + ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 6}}, "seed": {"type": + ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 8}}, "stop": {"type": ["list"], + "default": "", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 5}}, "temperature": {"type": ["double"], + "default": 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 2}}, "top_p": {"type": ["double"], "default": + 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 3}}}, "description": "Use Azure OpenAI GPT-4 Turbo with + Vision to leverage AOAI vision ability.", "module": "promptflow.tools.aoai_gpt4v", + "class_name": "AzureOpenAI", "function": "chat", "icon": {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "default_prompt": "# system:\nAs an AI assistant, your task involves interpreting + images and responding to questions about the image.\nRemember to provide accurate + answers based on the information present in the image.\n\n# user:\nCan you + tell me what the image depicts?\n![image]({{image_input}})\n", "enable_kwargs": + false, "tool_state": "preview"}, {"name": "Content Safety (Text Analyze)", + "type": "python", "inputs": {"connection": {"type": ["AzureContentSafetyConnection"], "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, "hate_category": {"type": ["string"], "default": "medium_sensitivity", "enum": ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], @@ -723,147 +699,457 @@ interactions: "input_type": "default"}}, "description": "Use Azure Content Safety to detect harmful content.", "module": "promptflow.tools.azure_content_safety", "function": "analyze_text", "is_builtin": true, "package": "promptflow-tools", "package_version": - "0.0.216", "enable_kwargs": false, "deprecated_tools": ["content_safety_text.tools.content_safety_text_tool.analyze_text"], + "1.5.0rc8", "enable_kwargs": false, "deprecated_tools": ["content_safety_text.tools.content_safety_text_tool.analyze_text"], "tool_state": "stable"}, {"name": "Embedding", "type": "python", "inputs": - {"connection": {"type": ["AzureOpenAIConnection", "OpenAIConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "deployment_name": - {"type": ["string"], "enabled_by": "connection", "enabled_by_type": ["AzureOpenAIConnection"], - "model_list": ["text-embedding-ada-002", "text-search-ada-doc-001", "text-search-ada-query-001"], - "capabilities": {"completion": false, "chat_completion": false, "embeddings": - true}, "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default"}, "input": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "model": {"type": ["string"], "enum": ["text-embedding-ada-002", - "text-search-ada-doc-001", "text-search-ada-query-001"], "enabled_by": "connection", - "enabled_by_type": ["OpenAIConnection"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Use Open AI''s embedding - model to create an embedding vector representing the input text.", "module": - "promptflow.tools.embedding", "function": "embedding", "is_builtin": true, - "package": "promptflow-tools", "package_version": "0.0.216", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "Open Source LLM", "type": "custom_llm", - "inputs": {"api": {"type": ["string"], "enum": ["chat", "completion"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "connection": {"type": - ["CustomConnection"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "deployment_name": {"type": ["string"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "endpoint_name": - {"type": ["string"], "default": "-- please enter an endpoint name --", "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "max_new_tokens": - {"type": ["int"], "default": 500, "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "model_kwargs": {"type": ["object"], "default": - "{}", "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default", "advanced": true}, "temperature": {"type": ["double"], "default": - 1.0, "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default"}, "top_p": {"type": ["double"], "default": 1.0, "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default", "advanced": true}}, - "description": "Use an Open Source model from the Azure Model catalog, deployed - to an AzureML Online Endpoint for LLM Chat or Completion API calls.", "module": - "promptflow.tools.open_source_llm", "class_name": "OpenSourceLLM", "function": + {"connection": {"type": ["AzureOpenAIConnection", "OpenAIConnection", "ServerlessConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "deployment_name": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["AzureOpenAIConnection"], "model_list": ["text-embedding-ada-002", "text-search-ada-doc-001", + "text-search-ada-query-001"], "capabilities": {"completion": false, "chat_completion": + false, "embeddings": true}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "input": {"type": ["string"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "model": {"type": + ["string"], "enum": ["text-embedding-ada-002", "text-search-ada-doc-001", + "text-search-ada-query-001"], "enabled_by": "connection", "enabled_by_type": + ["OpenAIConnection"], "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default"}}, "description": "Use Open AI''s embedding model + to create an embedding vector representing the input text.", "module": "promptflow.tools.embedding", + "function": "embedding", "is_builtin": true, "package": "promptflow-tools", + "package_version": "1.5.0rc8", "enable_kwargs": false, "tool_state": "stable"}, + {"name": "LLM", "type": "custom_llm", "inputs": {"api": {"type": ["string"], + "default": "chat", "allow_manual_entry": true, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 1, "text_box_size": "sm"}, "filter_by": {"input_name": + "connection", "values": {"AzureOpenAIConnection": {"enum": ["chat", "completion"]}, + "OpenAIConnection": {"enum": ["chat", "completion"]}, "ServerlessConnection": + {"enum": ["chat"]}}, "filter_attribute": "type"}}, "best_of": {"type": ["int"], + "default": 1, "enabled_by": "api", "enabled_by_value": ["completion"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 16, "text_box_size": "xs"}}, "connection": {"type": + ["AzureOpenAIConnection", "OpenAIConnection", "ServerlessConnection"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 0}}, "deployment_name": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["AzureOpenAIConnection"], "allow_manual_entry": true, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 2, "text_box_size": + "lg"}, "filter_by": {"input_name": "api", "values": {"chat": {"capabilities": + {"chat_completion": true, "exclude": {"model_version": ["vision-preview"]}}}, + "completion": {"capabilities": {"completion": true, "include": {"model_name": + ["gpt-35-turbo-instruct"]}}}}}}, "echo": {"type": ["bool"], "default": false, + "enabled_by": "api", "enabled_by_value": ["completion"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 15, "text_box_size": "xs"}}, "frequency_penalty": {"type": + ["int"], "default": 0, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "advanced": true, "ui_hints": {"index": 10, "text_box_size": + "xs"}}, "logit_bias": {"type": ["object"], "default": "", "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 11, "text_box_size": "lg"}}, "logprobs": {"type": ["int"], + "default": "", "enabled_by": "api", "enabled_by_value": ["completion"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 14, "text_box_size": "xs"}}, "max_tokens": {"type": + ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 7, "text_box_size": "xs"}}, + "model": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["OpenAIConnection"], "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 3, "text_box_size": "lg"}, + "filter_by": {"input_name": "api", "values": {"chat": {"enum": ["gpt-3.5-turbo", + "gpt-3.5-turbo-0301", "gpt-3.5-turbo-16k", "gpt-3.5-turbo-1106", "gpt-4", + "gpt-4-1106-preview", "gpt-4-0314", "gpt-4-32k", "gpt-4-32k-0314"]}, "completion": + {"enum": ["gpt-3.5-turbo-instruct"]}}}}, "presence_penalty": {"type": ["double"], + "default": 0, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "advanced": true, "ui_hints": {"index": 8}}, "response_format": + {"type": ["object"], "default": "", "enum": [{"type": "json_object"}, {"type": + "text"}], "enabled_by": "api", "enabled_by_value": ["chat"], "allow_manual_entry": + true, "is_multi_select": false, "input_type": "default", "advanced": false, + "ui_hints": {"index": 9, "text_box_size": "lg"}}, "seed": {"type": ["int"], + "default": "", "enabled_by": "api", "enabled_by_value": ["chat"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 12, "text_box_size": "xs"}}, "stop": {"type": ["list"], + "default": "", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 6, "text_box_size": "md"}}, "suffix": {"type": + ["string"], "default": "", "enabled_by": "api", "enabled_by_value": ["completion"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "advanced": true, "ui_hints": {"index": 13, "text_box_size": "xs"}}, "temperature": + {"type": ["double"], "default": 1, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 4, "text_box_size": + "xs"}}, "tool_choice": {"type": ["object", "string"], "default": "", "enabled_by": + "api", "enabled_by_value": ["chat"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 18}}, "tools": {"type": + ["list"], "default": [], "enabled_by": "api", "enabled_by_value": ["chat"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"display_style": "reusable_tool", "index": 17}}, "top_p": {"type": + ["double"], "default": 1, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "advanced": true, "ui_hints": {"index": 5, + "text_box_size": "xs"}}}, "description": {"ai_studio": "Use OpenAI-like Large + Language Model for text completion or chat. For more details, refer to this + [document](https://aka.ms/aistudio/pf/llm-tool).", "aml": "Use OpenAI-like + Large Language Model for text completion or chat. For more details, refer + to this [document](https://aka.ms/promptflow/llm-tool).", "default": "Use + OpenAI-like Large Language Model for text completion or chat."}, "module": + "promptflow.tools.llm", "function": "llm", "icon": {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "category": "llm", "is_builtin": true, "package": "promptflow-tools", "package_version": + "1.5.0rc8", "default_prompt": "# system:\nYou are a helpful assistant.\n\n# + user:\n{{question}}\n", "enable_kwargs": false, "tool_state": "stable", "groups": + [{"name": "Tools", "description": "Configure interactive capabilities by selecting + tools for the model to use and guiding its tool call decisions.", "inputs": + ["tools", "tool_choice"], "ui_hints": {"display_style": "table"}}]}, {"name": + "LLM-Vision", "type": "custom_llm", "inputs": {"connection": {"type": ["AzureOpenAIConnection", + "OpenAIConnection"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 0}}, "deployment_name": {"type": + ["string"], "enabled_by": "connection", "enabled_by_type": ["AzureOpenAIConnection"], + "capabilities": {"chat_completion": true, "include": {"model_version": ["vision-preview"]}}, + "allow_manual_entry": true, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 1, "text_box_size": "lg"}}, "detail": {"type": ["string"], + "default": "auto", "enum": ["low", "high", "auto"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 10}}, "frequency_penalty": {"type": ["int"], "default": 0, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 8, "text_box_size": "xs"}}, "max_tokens": {"type": ["int"], "default": "", + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 6, "text_box_size": "xs"}}, "model": {"type": ["string"], + "enum": ["gpt-4-vision-preview"], "enabled_by": "connection", "enabled_by_type": + ["OpenAIConnection"], "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 2, "text_box_size": "lg"}}, + "presence_penalty": {"type": ["double"], "default": 0, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 7}}, "seed": {"type": ["int"], "default": "", "allow_manual_entry": false, + "is_multi_select": false, "input_type": "default", "ui_hints": {"index": 9, + "text_box_size": "xs"}}, "stop": {"type": ["list"], "default": "", "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 5, "text_box_size": "md"}}, "temperature": {"type": ["double"], "default": + 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 3, "text_box_size": "xs"}}, "top_p": {"type": ["double"], + "default": 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 4, "text_box_size": "xs"}}}, "description": + {"ai_studio": "Use OpenAI-like Large Language Model to leverage vision ability. + For more details, refer to this [document](https://aka.ms/aistudio/pf/llm-vision-tool).", + "aml": "Use OpenAI-like Large Language Model to leverage vision ability. For + more details, refer to this [document](https://aka.ms/promptflow/llm-vision-tool).", + "default": "Use OpenAI-like Large Language Model to leverage vision ability."}, + "module": "promptflow.tools.llm_vision", "function": "llm_vision", "icon": + {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "category": "llm", "is_builtin": true, "package": "promptflow-tools", "package_version": + "1.5.0rc8", "default_prompt": "# system:\nAs an AI assistant, your task involves + interpreting images and responding to questions about the image.\nRemember + to provide accurate answers based on the information present in the image.\n\n# + user:\nCan you tell me what the image depicts?\n![image]({{image_input}})\n", + "enable_kwargs": false, "tool_state": "preview"}, {"name": "Open Model LLM", + "type": "custom_llm", "inputs": {"api": {"type": ["string"], "enum": ["chat", + "completion"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 2}}, "deployment_name": {"type": ["string"], + "default": "", "dynamic_list": {"func_path": "promptflow.tools.open_model_llm.list_deployment_names", + "func_kwargs": [{"name": "endpoint", "optional": true, "reference": "${inputs.endpoint}", + "type": ["string"]}]}, "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 1}}, "endpoint_name": {"type": + ["string"], "dynamic_list": {"func_path": "promptflow.tools.open_model_llm.list_endpoint_names"}, + "allow_manual_entry": true, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 0}}, "max_new_tokens": {"type": ["int"], "default": + 500, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 4}}, "model_kwargs": {"type": ["object"], + "default": "{}", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "advanced": true, "ui_hints": {"index": 6}}, "temperature": {"type": + ["double"], "default": 1.0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 3}}, "top_p": {"type": + ["double"], "default": 1.0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "advanced": true, "ui_hints": {"index": 5}}}, + "description": "Use an open model from the Azure Model catalog, deployed to + an AzureML Online Endpoint for LLM Chat or Completion API calls.", "module": + "promptflow.tools.open_model_llm", "class_name": "OpenModelLLM", "function": "call", "icon": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAACgElEQVR4nGWSz2vcVRTFP/e9NzOZ1KDGohASslLEH6VLV0ak4l/QpeDCrfQPcNGliODKnVm4EBdBsIjQIlhciKW0ycKFVCSNbYnjdDLtmPnmO/nO9917XcxMkjYX3uLx7nnn3HOuMK2Nix4fP78ZdrYXVkLVWjf3l3B1B+HpcjzGFtmqa6cePz7/x0dnn1n5qhj3iBJPYREIURAJuCtpY8PjReDbrf9WG7H1fuefwQU9qKztTcMJT+PNnEFvjGVDBDlSsH6p/9MLzy6+NxwVqI8RAg4IPmWedMckdLYP6O6UpIaQfvyyXG012+e79/ZfHukoS1ISMT2hGTB1RkUmNgQ5QZ0w+a2VWDq73MbdEWmfnnv6UWe7oNzPaLapl5CwuLTXK9WUGBuCjqekzhP+z52ZXOrKMD3OJg0Hh778aiOuvpnYvp05d6GJO4iAO4QAe/eV36/X5LFRV4Zmn+AdkqlL8Vjp3oVioOz+WTPzzYEgsN+fgPLYyJVheSbPPVl2ikeGZRjtG52/8rHuaV9VOlpP2OtKyVndcRVCSqOhsvxa4vW359i6OuKdD+aP8Q4SYPdOzS/flGjt1JUSaMqZ5nwa1Y8qWb/Ud/eZZkHisYezEM0m+fcelDr8F1SqW2LNK6r1jXQwyLzy1hxvrLXZulry7ocL+FS6G4QIu3fG/Px1gdYeW7LIgXU2P/115TOA5G7e3Rmj2aS/m7l5pThiZzrCcE/d1XHzbln373nw7y6veeoUm5KCNKT/IPPwbiY1hYd/l5MIT65BMFt87sU4v9D7/JMflr44uV6hGh1+L4RCkg6z5iK2tAhNLeLsNGwYA4fDYnC/drvuuFxe86NV/x+Ut27g0FvykgAAAABJRU5ErkJggg==", - "is_builtin": true, "package": "promptflow-tools", "package_version": "0.0.216", + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", "enable_kwargs": false, "tool_state": "stable"}, {"name": "OpenAI GPT-4V", "type": "custom_llm", "inputs": {"connection": {"type": ["OpenAIConnection"], - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "frequency_penalty": {"type": ["double"], "default": 0, "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "max_tokens": {"type": + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 0}}, "detail": {"type": ["string"], "default": "auto", + "enum": ["low", "high", "auto"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 9}}, "frequency_penalty": + {"type": ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 7}}, "max_tokens": {"type": + ["int"], "default": 512, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 4}}, "model": {"type": ["string"], + "enum": ["gpt-4-vision-preview"], "allow_manual_entry": true, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 1}}, "presence_penalty": + {"type": ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 6}}, "seed": {"type": ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "model": {"type": ["string"], "enum": ["gpt-4-vision-preview"], - "allow_manual_entry": true, "is_multi_select": false, "input_type": "default"}, - "presence_penalty": {"type": ["double"], "default": 0, "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "stop": {"type": - ["list"], "default": "", "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "temperature": {"type": ["double"], "default": 1, - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "top_p": {"type": ["double"], "default": 1, "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Use OpenAI GPT-4V to leverage + "input_type": "default", "ui_hints": {"index": 8}}, "stop": {"type": ["list"], + "default": "", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 5}}, "temperature": {"type": ["double"], + "default": 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 2}}, "top_p": {"type": ["double"], "default": + 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 3}}}, "description": "Use OpenAI GPT-4V to leverage vision ability.", "module": "promptflow.tools.openai_gpt4v", "class_name": - "OpenAI", "function": "chat", "is_builtin": true, "package": "promptflow-tools", - "package_version": "0.0.216", "default_prompt": "# system:\nAs an AI assistant, - your task involves interpreting images and responding to questions about the - image.\nRemember to provide accurate answers based on the information present - in the image.\n\n# user:\nCan you tell me what the image depicts?\n![image]({{image_input}})\n", - "enable_kwargs": false, "tool_state": "stable"}, {"name": "Serp API", "type": - "python", "inputs": {"connection": {"type": ["SerpConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "engine": {"type": - ["string"], "default": "google", "enum": ["google", "bing"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "location": {"type": - ["string"], "default": "", "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "num": {"type": ["int"], "default": "10", - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "query": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "safe": {"type": ["string"], "default": "off", - "enum": ["active", "off"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Use Serp API to obtain search - results from a specific search engine.", "module": "promptflow.tools.serpapi", - "class_name": "SerpAPI", "function": "search", "is_builtin": true, "package": - "promptflow-tools", "package_version": "0.0.216", "enable_kwargs": false, - "tool_state": "stable"}, {"name": "Faiss Index Lookup", "type": "python", - "inputs": {"path": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "vector": {"type": ["list"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Search vector based query - from the FAISS index file.", "module": "promptflow_vectordb.tool.faiss_index_lookup", - "class_name": "FaissIndexLookup", "function": "search", "is_builtin": true, - "package": "promptflow-vectordb", "package_version": "0.0.1", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "Vector DB Lookup", "type": "python", - "inputs": {"class_name": {"type": ["string"], "enabled_by": "connection", - "enabled_by_type": ["WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "collection_name": {"type": ["string"], "enabled_by": - "connection", "enabled_by_type": ["QdrantConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "connection": {"type": - ["CognitiveSearchConnection", "QdrantConnection", "WeaviateConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "index_name": {"type": - ["string"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection"], - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "search_filters": {"type": ["object"], "enabled_by": "connection", "enabled_by_type": - ["CognitiveSearchConnection", "QdrantConnection"], "allow_manual_entry": false, - "is_multi_select": false, "input_type": "default"}, "search_params": {"type": - ["object"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection", - "QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "text_field": {"type": ["string"], "enabled_by": - "connection", "enabled_by_type": ["CognitiveSearchConnection", "QdrantConnection", - "WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", "allow_manual_entry": + "OpenAI", "function": "chat", "icon": {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "default_prompt": "# system:\nAs an AI assistant, your task involves interpreting + images and responding to questions about the image.\nRemember to provide accurate + answers based on the information present in the image.\n\n# user:\nCan you + tell me what the image depicts?\n![image]({{image_input}})\n", "enable_kwargs": + false, "tool_state": "preview"}, {"name": "Serp API", "type": "python", "inputs": + {"connection": {"type": ["SerpConnection"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "engine": {"type": ["string"], "default": + "google", "enum": ["google", "bing"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "location": {"type": ["string"], "default": + "", "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "num": {"type": ["int"], "default": "10", "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "query": {"type": ["string"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "safe": {"type": + ["string"], "default": "off", "enum": ["active", "off"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}}, "description": + "Use Serp API to obtain search results from a specific search engine.", "module": + "promptflow.tools.serpapi", "class_name": "SerpAPI", "function": "search", + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "enable_kwargs": false, "tool_state": "stable"}, {"name": "Index Lookup", + "type": "python", "inputs": {"acs_content_field": {"type": ["string"], "enabled_by": + "index_type", "enabled_by_value": ["Azure AI Search"], "dynamic_list": {"func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_fields", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}, {"default": "Edm.String", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "acs_embedding_field": {"type": ["string"], + "enabled_by": "index_type", "enabled_by_value": ["Azure AI Search"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_fields", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}, {"default": "Collection(Edm.Single)", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "acs_index_connection": {"type": ["CognitiveSearchConnection"], + "enabled_by": "index_type", "enabled_by_value": ["Azure AI Search"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "acs_index_name": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + AI Search"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_indices", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}]}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "acs_metadata_field": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Azure AI Search"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_fields", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}, {"default": "Edm.String", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "aoai_embedding_connection": {"type": + ["AzureOpenAIConnection"], "enabled_by": "embedding_type", "enabled_by_value": + ["Azure OpenAI"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "uionly_hidden"}, "embedding_deployment": {"type": ["string"], "enabled_by": + "embedding_type", "enabled_by_value": ["Azure OpenAI"], "dynamic_list": {"func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.list_aoai_embedding_deployments", + "func_kwargs": [{"name": "aoai_connection", "optional": false, "reference": + "${inputs.aoai_embedding_connection}", "type": ["AzurOpenAIConnection"]}]}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "embedding_model": {"type": ["string"], "enabled_by": "embedding_type", "enabled_by_value": + ["OpenAI", "Hugging Face"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_embedding_models", + "func_kwargs": [{"name": "embedding_type", "optional": false, "reference": + "${inputs.embedding_type}", "type": ["string"]}]}, "allow_manual_entry": false, + "is_multi_select": false, "input_type": "uionly_hidden"}, "embedding_type": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + AI Search", "Azure CosmosDB for MongoDB vCore", "FAISS", "Pinecone"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_available_embedding_types", + "func_kwargs": [{"name": "index_type", "optional": false, "reference": "${inputs.index_type}", + "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "uionly_hidden"}, "faiss_index_path": {"type": ["string"], "enabled_by": + "index_type", "enabled_by_value": ["FAISS"], "allow_manual_entry": false, + "is_multi_select": false, "input_type": "uionly_hidden"}, "index_type": {"type": + ["string"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_available_index_types"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mlindex_asset_id": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Registered Index"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_registered_mlindices"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mlindex_content": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": + false, "generated_by": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.forward_mapping", + "func_kwargs": [{"name": "index_type", "reference": "${inputs.index_type}", + "type": ["string"]}, {"name": "mlindex_asset_id", "optional": true, "reference": + "${inputs.mlindex_asset_id}", "type": ["string"]}, {"name": "mlindex_path", + "optional": true, "reference": "${inputs.mlindex_path}", "type": ["string"]}, + {"name": "acs_index_connection", "optional": true, "reference": "${inputs.acs_index_connection}", + "type": ["CognitiveSearchConnection"]}, {"name": "acs_index_name", "optional": + true, "reference": "${inputs.acs_index_name}", "type": ["string"]}, {"name": + "acs_content_field", "optional": true, "reference": "${inputs.acs_content_field}", + "type": ["string"]}, {"name": "acs_embedding_field", "optional": true, "reference": + "${inputs.acs_embedding_field}", "type": ["string"]}, {"name": "acs_metadata_field", + "optional": true, "reference": "${inputs.acs_metadata_field}", "type": ["string"]}, + {"name": "semantic_configuration", "optional": true, "reference": "${inputs.semantic_configuration}", + "type": ["string"]}, {"name": "faiss_index_path", "optional": true, "reference": + "${inputs.faiss_index_path}", "type": ["string"]}, {"name": "pinecone_index_connection", + "optional": true, "reference": "${inputs.pinecone_index_connection}", "type": + ["string"]}, {"name": "pinecone_index_name", "optional": true, "reference": + "${inputs.pinecone_index_name}", "type": ["string"]}, {"name": "pinecone_content_field", + "optional": true, "reference": "${inputs.pinecone_content_field}", "type": + ["string"]}, {"name": "pinecone_metadata_field", "optional": true, "reference": + "${inputs.pinecone_metadata_field}", "type": ["string"]}, {"name": "mongodb_connection", + "optional": true, "reference": "${inputs.mongodb_connection}", "type": ["string"]}, + {"name": "mongodb_database", "optional": true, "reference": "${inputs.mongodb_database}", + "type": ["string"]}, {"name": "mongodb_collection", "optional": true, "reference": + "${inputs.mongodb_collection}", "type": ["string"]}, {"name": "mongodb_index_name", + "optional": true, "reference": "${inputs.mongodb_index_name}", "type": ["string"]}, + {"name": "mongodb_content_field", "optional": true, "reference": "${inputs.mongodb_content_field}", + "type": ["string"]}, {"name": "mongodb_embedding_field", "optional": true, + "reference": "${inputs.mongodb_embedding_field}", "type": ["string"]}, {"name": + "embedding_type", "optional": true, "reference": "${inputs.embedding_type}", + "type": ["string"]}, {"name": "aoai_embedding_connection", "optional": true, + "reference": "${inputs.aoai_embedding_connection}", "type": ["AzureOpenAIConnection"]}, + {"name": "oai_embedding_connection", "optional": true, "reference": "${inputs.oai_embedding_connection}", + "type": ["string"]}, {"name": "embedding_model", "optional": true, "reference": + "${inputs.embedding_model}", "type": ["string"]}, {"name": "embedding_deployment", + "optional": true, "reference": "${inputs.embedding_deployment}", "type": ["string"]}, + {"name": "serverless_embedding_connection", "optional": true, "reference": + "${inputs.serverless_embedding_connection}", "type": ["string"]}], "reverse_func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.reverse_mapping"}, "input_type": + "default"}, "mlindex_path": {"type": ["string"], "enabled_by": "index_type", + "enabled_by_value": ["MLIndex file from path"], "allow_manual_entry": false, + "is_multi_select": false, "input_type": "uionly_hidden"}, "mongodb_collection": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + CosmosDB for MongoDB vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_collections", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}]}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mongodb_connection": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Azure CosmosDB for MongoDB vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_connections"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mongodb_content_field": {"type": ["string"], "enabled_by": "index_type", + "enabled_by_value": ["Azure CosmosDB for MongoDB vCore"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_index_fields", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}, + {"name": "mongodb_collection", "optional": false, "reference": "${inputs.mongodb_collection}", + "type": ["string"]}, {"default": "string", "name": "field_data_type", "optional": + false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "mongodb_database": {"type": ["string"], + "enabled_by": "index_type", "enabled_by_value": ["Azure CosmosDB for MongoDB + vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_databases", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}]}, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "mongodb_embedding_field": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + CosmosDB for MongoDB vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_index_fields", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}, + {"name": "mongodb_collection", "optional": false, "reference": "${inputs.mongodb_collection}", + "type": ["string"]}, {"default": "ARRAY(float)", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "mongodb_index_name": {"type": ["string"], + "enabled_by": "index_type", "enabled_by_value": ["Azure CosmosDB for MongoDB + vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_indexes", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}, + {"name": "mongodb_collection", "optional": false, "reference": "${inputs.mongodb_collection}", + "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "uionly_hidden"}, "oai_embedding_connection": {"type": ["OpenAIConnection"], + "enabled_by": "embedding_type", "enabled_by_value": ["OpenAI"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "pinecone_content_field": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Pinecone"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "pinecone_index_connection": {"type": ["PineconeConnection"], "enabled_by": + "index_type", "enabled_by_value": ["Pinecone"], "dynamic_list": {"func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.list_pinecone_connections"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "pinecone_index_name": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Pinecone"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_pinecone_indices", + "func_kwargs": [{"name": "pinecone_connection_name", "optional": false, "reference": + "${inputs.pinecone_index_connection}", "type": ["string"]}]}, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "pinecone_metadata_field": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Pinecone"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "queries": {"type": ["object"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "query_type": {"type": ["string"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_available_query_types", + "func_kwargs": [{"name": "mlindex_content", "optional": false, "reference": + "${inputs.mlindex_content}", "type": ["string"]}]}, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "semantic_configuration": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + AI Search"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_semantic_configurations", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "uionly_hidden"}, "serverless_embedding_connection": {"type": + ["string"], "enabled_by": "embedding_type", "enabled_by_value": ["Serverless + Endpoint"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_serverless_embedding_connections"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "top_k": {"type": ["int"], "default": 3, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}}, "description": "Search an AzureML Vector + Index for relevant results using one or more text queries.", "module": "promptflow_vectordb.tool.common_index_lookup", + "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "preview"}, + {"name": "Faiss Index Lookup", "type": "python", "inputs": {"path": {"type": + ["string"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default"}, "top_k": {"type": ["int"], "default": "3", "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, "vector": {"type": ["list"], "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default"}, "vector_field": {"type": ["string"], "enabled_by": "connection", - "enabled_by_type": ["CognitiveSearchConnection"], "allow_manual_entry": false, - "is_multi_select": false, "input_type": "default"}}, "description": "Search - vector based query from existing Vector Database.", "module": "promptflow_vectordb.tool.vector_db_lookup", - "class_name": "VectorDBLookup", "function": "search", "is_builtin": true, - "package": "promptflow-vectordb", "package_version": "0.0.1", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "Vector Index Lookup", "type": "python", - "inputs": {"path": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "query": {"type": ["object"], "allow_manual_entry": + "default"}}, "description": "Search vector based query from the FAISS index + file.", "module": "promptflow_vectordb.tool.faiss_index_lookup", "class_name": + "FaissIndexLookup", "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "archived"}, + {"name": "Vector DB Lookup", "type": "python", "inputs": {"class_name": {"type": + ["string"], "enabled_by": "connection", "enabled_by_type": ["WeaviateConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "collection_name": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default"}, "connection": {"type": ["CognitiveSearchConnection", + "QdrantConnection", "WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "index_name": {"type": ["string"], "enabled_by": + "connection", "enabled_by_type": ["CognitiveSearchConnection"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "search_filters": + {"type": ["object"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection", + "QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default"}, "search_params": {"type": ["object"], "enabled_by": + "connection", "enabled_by_type": ["CognitiveSearchConnection", "QdrantConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "text_field": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["CognitiveSearchConnection", "QdrantConnection", "WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}}, "description": "Search text or vector based query - from AzureML Vector Index.", "module": "promptflow_vectordb.tool.vector_index_lookup", - "class_name": "VectorIndexLookup", "function": "search", "is_builtin": true, - "package": "promptflow-vectordb", "package_version": "0.0.1", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "hello_world.py", "type": "python", - "inputs": {"name": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "source": "hello_world.py", "function": - "hello_world", "is_builtin": false, "enable_kwargs": false, "tool_state": - "stable"}], "inputs": {"name": {"type": "string", "default": "hod", "is_chat_input": - false}}, "outputs": {"result": {"type": "string", "reference": "${hello_world.output}", - "evaluation_only": false, "is_chat_output": false}}}, "flowRunResourceId": - "azureml://locations/eastus/workspaces/00000/flows/batch_run_name/flowRuns/batch_run_name", + "input_type": "default"}, "vector": {"type": ["list"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "vector_field": + {"type": ["string"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}}, + "description": "Search vector based query from existing Vector Database.", + "module": "promptflow_vectordb.tool.vector_db_lookup", "class_name": "VectorDBLookup", + "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "archived"}, + {"name": "Vector Index Lookup", "type": "python", "inputs": {"path": {"type": + ["string"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default"}, "query": {"type": ["object"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}}, + "description": "Search text or vector based query from AzureML Vector Index.", + "module": "promptflow_vectordb.tool.vector_index_lookup", "class_name": "VectorIndexLookup", + "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "archived"}, + {"name": "hello_world.py", "type": "python", "inputs": {"name": {"type": ["string"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}}, + "source": "hello_world.py", "function": "hello_world", "is_builtin": false, + "enable_kwargs": false, "tool_state": "stable"}], "inputs": {"name": {"type": + "string", "default": "hod", "is_chat_input": false}}, "outputs": {"result": + {"type": "string", "reference": "${hello_world.output}", "evaluation_only": + false, "is_chat_output": false}}}, "flowRunResourceId": "azureml://locations/eastus/workspaces/00000/flows/batch_run_name/flowRuns/batch_run_name", "flowRunId": "batch_run_name", "flowRunDisplayName": "sdk-cli-test-fixture-batch-run-without-llm", "batchDataInput": {"dataUri": "azureml://datastores/workspaceblobstore/paths/LocalUpload/74c11bba717480b2d6b04b8e746d09d7/webClassification3.jsonl"}, - "flowRunType": "FlowRun", "flowType": "Default", "runtimeName": "test-runtime-ci", + "flowRunType": "FlowRun", "flowType": "Default", "runtimeName": "automatic", "inputsMapping": {"name": "${data.url}"}, "outputDatastoreName": "workspaceblobstore", "childRunBasePath": "promptflow/PromptFlowArtifacts/batch_run_name/flow_artifacts", - "flowDagFileRelativePath": "flow.dag.yaml", "flowSnapshotId": "38053316-7591-4ac8-b718-d76b16b48bbe", - "studioPortalEndpoint": "https://ml.azure.com/runs/batch_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000"}' + "flowDagFileRelativePath": "flow.dag.yaml", "flowSnapshotId": "b98f6388-29a7-47f6-93a9-09b2e2b55cb9", + "sessionId": "4a6867a952826a2a51729e6bb4bf21ac65b46eec3ae5ed10", "studioPortalEndpoint": + "https://ml.azure.com/runs/batch_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000", + "studioPortalTraceEndpoint": "https://ml.azure.com/prompts/trace/run/batch_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/promptflow-eastus"}' headers: connection: - keep-alive content-length: - - '12912' + - '42442' content-type: - application/json; charset=utf-8 strict-transport-security: - - max-age=15724800; includeSubDomains; preload + - max-age=31536000; includeSubDomains; preload transfer-encoding: - chunked vary: @@ -871,7 +1157,7 @@ interactions: x-content-type-options: - nosniff x-request-time: - - '0.393' + - '0.403' status: code: 200 message: OK @@ -881,20 +1167,53 @@ interactions: Accept: - application/json Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive User-Agent: - - promptflow-sdk/0.0.1 azsdk-python-azuremachinelearningdesignerserviceclient/unknown - Python/3.10.13 (Windows-10-10.0.22631-SP0) + - promptflow-azure-sdk/1.9.0.dev0 azsdk-python-azuremachinelearningdesignerserviceclient/unknown + Python/3.9.19 (Windows-10-10.0.22631-SP0) method: GET uri: https://eastus.api.azureml.ms/flow/api/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/BulkRuns/batch_run_name response: body: string: '{"flowGraph": {"nodes": [{"name": "hello_world", "type": "python", "source": {"type": "code", "path": "hello_world.py"}, "inputs": {"name": "${inputs.name}"}, - "tool": "hello_world.py", "reduce": false}], "tools": [{"name": "Content Safety - (Text Analyze)", "type": "python", "inputs": {"connection": {"type": ["AzureContentSafetyConnection"], + "tool": "hello_world.py", "reduce": false}], "tools": [{"name": "Azure OpenAI + GPT-4 Turbo with Vision", "type": "custom_llm", "inputs": {"connection": {"type": + ["AzureOpenAIConnection"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 0}}, "deployment_name": + {"type": ["string"], "enabled_by": "connection", "dynamic_list": {"func_path": + "promptflow.tools.aoai_gpt4v.list_deployment_names", "func_kwargs": [{"name": + "connection", "reference": "${inputs.connection}", "type": ["AzureOpenAIConnection"]}]}, + "allow_manual_entry": true, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 1}}, "detail": {"type": ["string"], "default": "auto", + "enum": ["low", "high", "auto"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 9}}, "frequency_penalty": + {"type": ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 7}}, "max_tokens": {"type": + ["int"], "default": 512, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 4}}, "presence_penalty": {"type": + ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 6}}, "seed": {"type": + ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 8}}, "stop": {"type": ["list"], + "default": "", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 5}}, "temperature": {"type": ["double"], + "default": 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 2}}, "top_p": {"type": ["double"], "default": + 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 3}}}, "description": "Use Azure OpenAI GPT-4 Turbo with + Vision to leverage AOAI vision ability.", "module": "promptflow.tools.aoai_gpt4v", + "class_name": "AzureOpenAI", "function": "chat", "icon": {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "default_prompt": "# system:\nAs an AI assistant, your task involves interpreting + images and responding to questions about the image.\nRemember to provide accurate + answers based on the information present in the image.\n\n# user:\nCan you + tell me what the image depicts?\n![image]({{image_input}})\n", "enable_kwargs": + false, "tool_state": "preview"}, {"name": "Content Safety (Text Analyze)", + "type": "python", "inputs": {"connection": {"type": ["AzureContentSafetyConnection"], "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, "hate_category": {"type": ["string"], "default": "medium_sensitivity", "enum": ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], @@ -912,147 +1231,457 @@ interactions: "input_type": "default"}}, "description": "Use Azure Content Safety to detect harmful content.", "module": "promptflow.tools.azure_content_safety", "function": "analyze_text", "is_builtin": true, "package": "promptflow-tools", "package_version": - "0.0.216", "enable_kwargs": false, "deprecated_tools": ["content_safety_text.tools.content_safety_text_tool.analyze_text"], + "1.5.0rc8", "enable_kwargs": false, "deprecated_tools": ["content_safety_text.tools.content_safety_text_tool.analyze_text"], "tool_state": "stable"}, {"name": "Embedding", "type": "python", "inputs": - {"connection": {"type": ["AzureOpenAIConnection", "OpenAIConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "deployment_name": - {"type": ["string"], "enabled_by": "connection", "enabled_by_type": ["AzureOpenAIConnection"], - "model_list": ["text-embedding-ada-002", "text-search-ada-doc-001", "text-search-ada-query-001"], - "capabilities": {"completion": false, "chat_completion": false, "embeddings": - true}, "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default"}, "input": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "model": {"type": ["string"], "enum": ["text-embedding-ada-002", - "text-search-ada-doc-001", "text-search-ada-query-001"], "enabled_by": "connection", - "enabled_by_type": ["OpenAIConnection"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Use Open AI''s embedding - model to create an embedding vector representing the input text.", "module": - "promptflow.tools.embedding", "function": "embedding", "is_builtin": true, - "package": "promptflow-tools", "package_version": "0.0.216", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "Open Source LLM", "type": "custom_llm", - "inputs": {"api": {"type": ["string"], "enum": ["chat", "completion"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "connection": {"type": - ["CustomConnection"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "deployment_name": {"type": ["string"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "endpoint_name": - {"type": ["string"], "default": "-- please enter an endpoint name --", "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "max_new_tokens": - {"type": ["int"], "default": 500, "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "model_kwargs": {"type": ["object"], "default": - "{}", "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default", "advanced": true}, "temperature": {"type": ["double"], "default": - 1.0, "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default"}, "top_p": {"type": ["double"], "default": 1.0, "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default", "advanced": true}}, - "description": "Use an Open Source model from the Azure Model catalog, deployed - to an AzureML Online Endpoint for LLM Chat or Completion API calls.", "module": - "promptflow.tools.open_source_llm", "class_name": "OpenSourceLLM", "function": + {"connection": {"type": ["AzureOpenAIConnection", "OpenAIConnection", "ServerlessConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "deployment_name": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["AzureOpenAIConnection"], "model_list": ["text-embedding-ada-002", "text-search-ada-doc-001", + "text-search-ada-query-001"], "capabilities": {"completion": false, "chat_completion": + false, "embeddings": true}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "input": {"type": ["string"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "model": {"type": + ["string"], "enum": ["text-embedding-ada-002", "text-search-ada-doc-001", + "text-search-ada-query-001"], "enabled_by": "connection", "enabled_by_type": + ["OpenAIConnection"], "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default"}}, "description": "Use Open AI''s embedding model + to create an embedding vector representing the input text.", "module": "promptflow.tools.embedding", + "function": "embedding", "is_builtin": true, "package": "promptflow-tools", + "package_version": "1.5.0rc8", "enable_kwargs": false, "tool_state": "stable"}, + {"name": "LLM", "type": "custom_llm", "inputs": {"api": {"type": ["string"], + "default": "chat", "allow_manual_entry": true, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 1, "text_box_size": "sm"}, "filter_by": {"input_name": + "connection", "values": {"AzureOpenAIConnection": {"enum": ["chat", "completion"]}, + "OpenAIConnection": {"enum": ["chat", "completion"]}, "ServerlessConnection": + {"enum": ["chat"]}}, "filter_attribute": "type"}}, "best_of": {"type": ["int"], + "default": 1, "enabled_by": "api", "enabled_by_value": ["completion"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 16, "text_box_size": "xs"}}, "connection": {"type": + ["AzureOpenAIConnection", "OpenAIConnection", "ServerlessConnection"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 0}}, "deployment_name": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["AzureOpenAIConnection"], "allow_manual_entry": true, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 2, "text_box_size": + "lg"}, "filter_by": {"input_name": "api", "values": {"chat": {"capabilities": + {"chat_completion": true, "exclude": {"model_version": ["vision-preview"]}}}, + "completion": {"capabilities": {"completion": true, "include": {"model_name": + ["gpt-35-turbo-instruct"]}}}}}}, "echo": {"type": ["bool"], "default": false, + "enabled_by": "api", "enabled_by_value": ["completion"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 15, "text_box_size": "xs"}}, "frequency_penalty": {"type": + ["int"], "default": 0, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "advanced": true, "ui_hints": {"index": 10, "text_box_size": + "xs"}}, "logit_bias": {"type": ["object"], "default": "", "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 11, "text_box_size": "lg"}}, "logprobs": {"type": ["int"], + "default": "", "enabled_by": "api", "enabled_by_value": ["completion"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 14, "text_box_size": "xs"}}, "max_tokens": {"type": + ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 7, "text_box_size": "xs"}}, + "model": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["OpenAIConnection"], "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 3, "text_box_size": "lg"}, + "filter_by": {"input_name": "api", "values": {"chat": {"enum": ["gpt-3.5-turbo", + "gpt-3.5-turbo-0301", "gpt-3.5-turbo-16k", "gpt-3.5-turbo-1106", "gpt-4", + "gpt-4-1106-preview", "gpt-4-0314", "gpt-4-32k", "gpt-4-32k-0314"]}, "completion": + {"enum": ["gpt-3.5-turbo-instruct"]}}}}, "presence_penalty": {"type": ["double"], + "default": 0, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "advanced": true, "ui_hints": {"index": 8}}, "response_format": + {"type": ["object"], "default": "", "enum": [{"type": "json_object"}, {"type": + "text"}], "enabled_by": "api", "enabled_by_value": ["chat"], "allow_manual_entry": + true, "is_multi_select": false, "input_type": "default", "advanced": false, + "ui_hints": {"index": 9, "text_box_size": "lg"}}, "seed": {"type": ["int"], + "default": "", "enabled_by": "api", "enabled_by_value": ["chat"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 12, "text_box_size": "xs"}}, "stop": {"type": ["list"], + "default": "", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 6, "text_box_size": "md"}}, "suffix": {"type": + ["string"], "default": "", "enabled_by": "api", "enabled_by_value": ["completion"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "advanced": true, "ui_hints": {"index": 13, "text_box_size": "xs"}}, "temperature": + {"type": ["double"], "default": 1, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 4, "text_box_size": + "xs"}}, "tool_choice": {"type": ["object", "string"], "default": "", "enabled_by": + "api", "enabled_by_value": ["chat"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 18}}, "tools": {"type": + ["list"], "default": [], "enabled_by": "api", "enabled_by_value": ["chat"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"display_style": "reusable_tool", "index": 17}}, "top_p": {"type": + ["double"], "default": 1, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "advanced": true, "ui_hints": {"index": 5, + "text_box_size": "xs"}}}, "description": {"ai_studio": "Use OpenAI-like Large + Language Model for text completion or chat. For more details, refer to this + [document](https://aka.ms/aistudio/pf/llm-tool).", "aml": "Use OpenAI-like + Large Language Model for text completion or chat. For more details, refer + to this [document](https://aka.ms/promptflow/llm-tool).", "default": "Use + OpenAI-like Large Language Model for text completion or chat."}, "module": + "promptflow.tools.llm", "function": "llm", "icon": {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "category": "llm", "is_builtin": true, "package": "promptflow-tools", "package_version": + "1.5.0rc8", "default_prompt": "# system:\nYou are a helpful assistant.\n\n# + user:\n{{question}}\n", "enable_kwargs": false, "tool_state": "stable", "groups": + [{"name": "Tools", "description": "Configure interactive capabilities by selecting + tools for the model to use and guiding its tool call decisions.", "inputs": + ["tools", "tool_choice"], "ui_hints": {"display_style": "table"}}]}, {"name": + "LLM-Vision", "type": "custom_llm", "inputs": {"connection": {"type": ["AzureOpenAIConnection", + "OpenAIConnection"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 0}}, "deployment_name": {"type": + ["string"], "enabled_by": "connection", "enabled_by_type": ["AzureOpenAIConnection"], + "capabilities": {"chat_completion": true, "include": {"model_version": ["vision-preview"]}}, + "allow_manual_entry": true, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 1, "text_box_size": "lg"}}, "detail": {"type": ["string"], + "default": "auto", "enum": ["low", "high", "auto"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 10}}, "frequency_penalty": {"type": ["int"], "default": 0, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 8, "text_box_size": "xs"}}, "max_tokens": {"type": ["int"], "default": "", + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 6, "text_box_size": "xs"}}, "model": {"type": ["string"], + "enum": ["gpt-4-vision-preview"], "enabled_by": "connection", "enabled_by_type": + ["OpenAIConnection"], "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 2, "text_box_size": "lg"}}, + "presence_penalty": {"type": ["double"], "default": 0, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 7}}, "seed": {"type": ["int"], "default": "", "allow_manual_entry": false, + "is_multi_select": false, "input_type": "default", "ui_hints": {"index": 9, + "text_box_size": "xs"}}, "stop": {"type": ["list"], "default": "", "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 5, "text_box_size": "md"}}, "temperature": {"type": ["double"], "default": + 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 3, "text_box_size": "xs"}}, "top_p": {"type": ["double"], + "default": 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 4, "text_box_size": "xs"}}}, "description": + {"ai_studio": "Use OpenAI-like Large Language Model to leverage vision ability. + For more details, refer to this [document](https://aka.ms/aistudio/pf/llm-vision-tool).", + "aml": "Use OpenAI-like Large Language Model to leverage vision ability. For + more details, refer to this [document](https://aka.ms/promptflow/llm-vision-tool).", + "default": "Use OpenAI-like Large Language Model to leverage vision ability."}, + "module": "promptflow.tools.llm_vision", "function": "llm_vision", "icon": + {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "category": "llm", "is_builtin": true, "package": "promptflow-tools", "package_version": + "1.5.0rc8", "default_prompt": "# system:\nAs an AI assistant, your task involves + interpreting images and responding to questions about the image.\nRemember + to provide accurate answers based on the information present in the image.\n\n# + user:\nCan you tell me what the image depicts?\n![image]({{image_input}})\n", + "enable_kwargs": false, "tool_state": "preview"}, {"name": "Open Model LLM", + "type": "custom_llm", "inputs": {"api": {"type": ["string"], "enum": ["chat", + "completion"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 2}}, "deployment_name": {"type": ["string"], + "default": "", "dynamic_list": {"func_path": "promptflow.tools.open_model_llm.list_deployment_names", + "func_kwargs": [{"name": "endpoint", "optional": true, "reference": "${inputs.endpoint}", + "type": ["string"]}]}, "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 1}}, "endpoint_name": {"type": + ["string"], "dynamic_list": {"func_path": "promptflow.tools.open_model_llm.list_endpoint_names"}, + "allow_manual_entry": true, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 0}}, "max_new_tokens": {"type": ["int"], "default": + 500, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 4}}, "model_kwargs": {"type": ["object"], + "default": "{}", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "advanced": true, "ui_hints": {"index": 6}}, "temperature": {"type": + ["double"], "default": 1.0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 3}}, "top_p": {"type": + ["double"], "default": 1.0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "advanced": true, "ui_hints": {"index": 5}}}, + "description": "Use an open model from the Azure Model catalog, deployed to + an AzureML Online Endpoint for LLM Chat or Completion API calls.", "module": + "promptflow.tools.open_model_llm", "class_name": "OpenModelLLM", "function": "call", "icon": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAACgElEQVR4nGWSz2vcVRTFP/e9NzOZ1KDGohASslLEH6VLV0ak4l/QpeDCrfQPcNGliODKnVm4EBdBsIjQIlhciKW0ycKFVCSNbYnjdDLtmPnmO/nO9917XcxMkjYX3uLx7nnn3HOuMK2Nix4fP78ZdrYXVkLVWjf3l3B1B+HpcjzGFtmqa6cePz7/x0dnn1n5qhj3iBJPYREIURAJuCtpY8PjReDbrf9WG7H1fuefwQU9qKztTcMJT+PNnEFvjGVDBDlSsH6p/9MLzy6+NxwVqI8RAg4IPmWedMckdLYP6O6UpIaQfvyyXG012+e79/ZfHukoS1ISMT2hGTB1RkUmNgQ5QZ0w+a2VWDq73MbdEWmfnnv6UWe7oNzPaLapl5CwuLTXK9WUGBuCjqekzhP+z52ZXOrKMD3OJg0Hh778aiOuvpnYvp05d6GJO4iAO4QAe/eV36/X5LFRV4Zmn+AdkqlL8Vjp3oVioOz+WTPzzYEgsN+fgPLYyJVheSbPPVl2ikeGZRjtG52/8rHuaV9VOlpP2OtKyVndcRVCSqOhsvxa4vW359i6OuKdD+aP8Q4SYPdOzS/flGjt1JUSaMqZ5nwa1Y8qWb/Ud/eZZkHisYezEM0m+fcelDr8F1SqW2LNK6r1jXQwyLzy1hxvrLXZulry7ocL+FS6G4QIu3fG/Px1gdYeW7LIgXU2P/115TOA5G7e3Rmj2aS/m7l5pThiZzrCcE/d1XHzbln373nw7y6veeoUm5KCNKT/IPPwbiY1hYd/l5MIT65BMFt87sU4v9D7/JMflr44uV6hGh1+L4RCkg6z5iK2tAhNLeLsNGwYA4fDYnC/drvuuFxe86NV/x+Ut27g0FvykgAAAABJRU5ErkJggg==", - "is_builtin": true, "package": "promptflow-tools", "package_version": "0.0.216", + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", "enable_kwargs": false, "tool_state": "stable"}, {"name": "OpenAI GPT-4V", "type": "custom_llm", "inputs": {"connection": {"type": ["OpenAIConnection"], - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "frequency_penalty": {"type": ["double"], "default": 0, "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "max_tokens": {"type": + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 0}}, "detail": {"type": ["string"], "default": "auto", + "enum": ["low", "high", "auto"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 9}}, "frequency_penalty": + {"type": ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 7}}, "max_tokens": {"type": + ["int"], "default": 512, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 4}}, "model": {"type": ["string"], + "enum": ["gpt-4-vision-preview"], "allow_manual_entry": true, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 1}}, "presence_penalty": + {"type": ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 6}}, "seed": {"type": ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "model": {"type": ["string"], "enum": ["gpt-4-vision-preview"], - "allow_manual_entry": true, "is_multi_select": false, "input_type": "default"}, - "presence_penalty": {"type": ["double"], "default": 0, "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "stop": {"type": - ["list"], "default": "", "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "temperature": {"type": ["double"], "default": 1, - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "top_p": {"type": ["double"], "default": 1, "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Use OpenAI GPT-4V to leverage + "input_type": "default", "ui_hints": {"index": 8}}, "stop": {"type": ["list"], + "default": "", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 5}}, "temperature": {"type": ["double"], + "default": 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 2}}, "top_p": {"type": ["double"], "default": + 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 3}}}, "description": "Use OpenAI GPT-4V to leverage vision ability.", "module": "promptflow.tools.openai_gpt4v", "class_name": - "OpenAI", "function": "chat", "is_builtin": true, "package": "promptflow-tools", - "package_version": "0.0.216", "default_prompt": "# system:\nAs an AI assistant, - your task involves interpreting images and responding to questions about the - image.\nRemember to provide accurate answers based on the information present - in the image.\n\n# user:\nCan you tell me what the image depicts?\n![image]({{image_input}})\n", - "enable_kwargs": false, "tool_state": "stable"}, {"name": "Serp API", "type": - "python", "inputs": {"connection": {"type": ["SerpConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "engine": {"type": - ["string"], "default": "google", "enum": ["google", "bing"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "location": {"type": - ["string"], "default": "", "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "num": {"type": ["int"], "default": "10", - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "query": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "safe": {"type": ["string"], "default": "off", - "enum": ["active", "off"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Use Serp API to obtain search - results from a specific search engine.", "module": "promptflow.tools.serpapi", - "class_name": "SerpAPI", "function": "search", "is_builtin": true, "package": - "promptflow-tools", "package_version": "0.0.216", "enable_kwargs": false, - "tool_state": "stable"}, {"name": "Faiss Index Lookup", "type": "python", - "inputs": {"path": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "vector": {"type": ["list"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Search vector based query - from the FAISS index file.", "module": "promptflow_vectordb.tool.faiss_index_lookup", - "class_name": "FaissIndexLookup", "function": "search", "is_builtin": true, - "package": "promptflow-vectordb", "package_version": "0.0.1", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "Vector DB Lookup", "type": "python", - "inputs": {"class_name": {"type": ["string"], "enabled_by": "connection", - "enabled_by_type": ["WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "collection_name": {"type": ["string"], "enabled_by": - "connection", "enabled_by_type": ["QdrantConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "connection": {"type": - ["CognitiveSearchConnection", "QdrantConnection", "WeaviateConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "index_name": {"type": - ["string"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection"], - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "search_filters": {"type": ["object"], "enabled_by": "connection", "enabled_by_type": - ["CognitiveSearchConnection", "QdrantConnection"], "allow_manual_entry": false, - "is_multi_select": false, "input_type": "default"}, "search_params": {"type": - ["object"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection", - "QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "text_field": {"type": ["string"], "enabled_by": - "connection", "enabled_by_type": ["CognitiveSearchConnection", "QdrantConnection", - "WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", "allow_manual_entry": + "OpenAI", "function": "chat", "icon": {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "default_prompt": "# system:\nAs an AI assistant, your task involves interpreting + images and responding to questions about the image.\nRemember to provide accurate + answers based on the information present in the image.\n\n# user:\nCan you + tell me what the image depicts?\n![image]({{image_input}})\n", "enable_kwargs": + false, "tool_state": "preview"}, {"name": "Serp API", "type": "python", "inputs": + {"connection": {"type": ["SerpConnection"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "engine": {"type": ["string"], "default": + "google", "enum": ["google", "bing"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "location": {"type": ["string"], "default": + "", "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "num": {"type": ["int"], "default": "10", "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "query": {"type": ["string"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "safe": {"type": + ["string"], "default": "off", "enum": ["active", "off"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}}, "description": + "Use Serp API to obtain search results from a specific search engine.", "module": + "promptflow.tools.serpapi", "class_name": "SerpAPI", "function": "search", + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "enable_kwargs": false, "tool_state": "stable"}, {"name": "Index Lookup", + "type": "python", "inputs": {"acs_content_field": {"type": ["string"], "enabled_by": + "index_type", "enabled_by_value": ["Azure AI Search"], "dynamic_list": {"func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_fields", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}, {"default": "Edm.String", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "acs_embedding_field": {"type": ["string"], + "enabled_by": "index_type", "enabled_by_value": ["Azure AI Search"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_fields", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}, {"default": "Collection(Edm.Single)", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "acs_index_connection": {"type": ["CognitiveSearchConnection"], + "enabled_by": "index_type", "enabled_by_value": ["Azure AI Search"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "acs_index_name": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + AI Search"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_indices", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}]}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "acs_metadata_field": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Azure AI Search"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_fields", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}, {"default": "Edm.String", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "aoai_embedding_connection": {"type": + ["AzureOpenAIConnection"], "enabled_by": "embedding_type", "enabled_by_value": + ["Azure OpenAI"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "uionly_hidden"}, "embedding_deployment": {"type": ["string"], "enabled_by": + "embedding_type", "enabled_by_value": ["Azure OpenAI"], "dynamic_list": {"func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.list_aoai_embedding_deployments", + "func_kwargs": [{"name": "aoai_connection", "optional": false, "reference": + "${inputs.aoai_embedding_connection}", "type": ["AzurOpenAIConnection"]}]}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "embedding_model": {"type": ["string"], "enabled_by": "embedding_type", "enabled_by_value": + ["OpenAI", "Hugging Face"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_embedding_models", + "func_kwargs": [{"name": "embedding_type", "optional": false, "reference": + "${inputs.embedding_type}", "type": ["string"]}]}, "allow_manual_entry": false, + "is_multi_select": false, "input_type": "uionly_hidden"}, "embedding_type": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + AI Search", "Azure CosmosDB for MongoDB vCore", "FAISS", "Pinecone"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_available_embedding_types", + "func_kwargs": [{"name": "index_type", "optional": false, "reference": "${inputs.index_type}", + "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "uionly_hidden"}, "faiss_index_path": {"type": ["string"], "enabled_by": + "index_type", "enabled_by_value": ["FAISS"], "allow_manual_entry": false, + "is_multi_select": false, "input_type": "uionly_hidden"}, "index_type": {"type": + ["string"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_available_index_types"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mlindex_asset_id": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Registered Index"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_registered_mlindices"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mlindex_content": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": + false, "generated_by": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.forward_mapping", + "func_kwargs": [{"name": "index_type", "reference": "${inputs.index_type}", + "type": ["string"]}, {"name": "mlindex_asset_id", "optional": true, "reference": + "${inputs.mlindex_asset_id}", "type": ["string"]}, {"name": "mlindex_path", + "optional": true, "reference": "${inputs.mlindex_path}", "type": ["string"]}, + {"name": "acs_index_connection", "optional": true, "reference": "${inputs.acs_index_connection}", + "type": ["CognitiveSearchConnection"]}, {"name": "acs_index_name", "optional": + true, "reference": "${inputs.acs_index_name}", "type": ["string"]}, {"name": + "acs_content_field", "optional": true, "reference": "${inputs.acs_content_field}", + "type": ["string"]}, {"name": "acs_embedding_field", "optional": true, "reference": + "${inputs.acs_embedding_field}", "type": ["string"]}, {"name": "acs_metadata_field", + "optional": true, "reference": "${inputs.acs_metadata_field}", "type": ["string"]}, + {"name": "semantic_configuration", "optional": true, "reference": "${inputs.semantic_configuration}", + "type": ["string"]}, {"name": "faiss_index_path", "optional": true, "reference": + "${inputs.faiss_index_path}", "type": ["string"]}, {"name": "pinecone_index_connection", + "optional": true, "reference": "${inputs.pinecone_index_connection}", "type": + ["string"]}, {"name": "pinecone_index_name", "optional": true, "reference": + "${inputs.pinecone_index_name}", "type": ["string"]}, {"name": "pinecone_content_field", + "optional": true, "reference": "${inputs.pinecone_content_field}", "type": + ["string"]}, {"name": "pinecone_metadata_field", "optional": true, "reference": + "${inputs.pinecone_metadata_field}", "type": ["string"]}, {"name": "mongodb_connection", + "optional": true, "reference": "${inputs.mongodb_connection}", "type": ["string"]}, + {"name": "mongodb_database", "optional": true, "reference": "${inputs.mongodb_database}", + "type": ["string"]}, {"name": "mongodb_collection", "optional": true, "reference": + "${inputs.mongodb_collection}", "type": ["string"]}, {"name": "mongodb_index_name", + "optional": true, "reference": "${inputs.mongodb_index_name}", "type": ["string"]}, + {"name": "mongodb_content_field", "optional": true, "reference": "${inputs.mongodb_content_field}", + "type": ["string"]}, {"name": "mongodb_embedding_field", "optional": true, + "reference": "${inputs.mongodb_embedding_field}", "type": ["string"]}, {"name": + "embedding_type", "optional": true, "reference": "${inputs.embedding_type}", + "type": ["string"]}, {"name": "aoai_embedding_connection", "optional": true, + "reference": "${inputs.aoai_embedding_connection}", "type": ["AzureOpenAIConnection"]}, + {"name": "oai_embedding_connection", "optional": true, "reference": "${inputs.oai_embedding_connection}", + "type": ["string"]}, {"name": "embedding_model", "optional": true, "reference": + "${inputs.embedding_model}", "type": ["string"]}, {"name": "embedding_deployment", + "optional": true, "reference": "${inputs.embedding_deployment}", "type": ["string"]}, + {"name": "serverless_embedding_connection", "optional": true, "reference": + "${inputs.serverless_embedding_connection}", "type": ["string"]}], "reverse_func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.reverse_mapping"}, "input_type": + "default"}, "mlindex_path": {"type": ["string"], "enabled_by": "index_type", + "enabled_by_value": ["MLIndex file from path"], "allow_manual_entry": false, + "is_multi_select": false, "input_type": "uionly_hidden"}, "mongodb_collection": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + CosmosDB for MongoDB vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_collections", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}]}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mongodb_connection": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Azure CosmosDB for MongoDB vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_connections"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mongodb_content_field": {"type": ["string"], "enabled_by": "index_type", + "enabled_by_value": ["Azure CosmosDB for MongoDB vCore"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_index_fields", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}, + {"name": "mongodb_collection", "optional": false, "reference": "${inputs.mongodb_collection}", + "type": ["string"]}, {"default": "string", "name": "field_data_type", "optional": + false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "mongodb_database": {"type": ["string"], + "enabled_by": "index_type", "enabled_by_value": ["Azure CosmosDB for MongoDB + vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_databases", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}]}, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "mongodb_embedding_field": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + CosmosDB for MongoDB vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_index_fields", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}, + {"name": "mongodb_collection", "optional": false, "reference": "${inputs.mongodb_collection}", + "type": ["string"]}, {"default": "ARRAY(float)", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "mongodb_index_name": {"type": ["string"], + "enabled_by": "index_type", "enabled_by_value": ["Azure CosmosDB for MongoDB + vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_indexes", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}, + {"name": "mongodb_collection", "optional": false, "reference": "${inputs.mongodb_collection}", + "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "uionly_hidden"}, "oai_embedding_connection": {"type": ["OpenAIConnection"], + "enabled_by": "embedding_type", "enabled_by_value": ["OpenAI"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "pinecone_content_field": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Pinecone"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "pinecone_index_connection": {"type": ["PineconeConnection"], "enabled_by": + "index_type", "enabled_by_value": ["Pinecone"], "dynamic_list": {"func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.list_pinecone_connections"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "pinecone_index_name": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Pinecone"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_pinecone_indices", + "func_kwargs": [{"name": "pinecone_connection_name", "optional": false, "reference": + "${inputs.pinecone_index_connection}", "type": ["string"]}]}, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "pinecone_metadata_field": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Pinecone"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "queries": {"type": ["object"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "query_type": {"type": ["string"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_available_query_types", + "func_kwargs": [{"name": "mlindex_content", "optional": false, "reference": + "${inputs.mlindex_content}", "type": ["string"]}]}, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "semantic_configuration": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + AI Search"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_semantic_configurations", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "uionly_hidden"}, "serverless_embedding_connection": {"type": + ["string"], "enabled_by": "embedding_type", "enabled_by_value": ["Serverless + Endpoint"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_serverless_embedding_connections"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "top_k": {"type": ["int"], "default": 3, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}}, "description": "Search an AzureML Vector + Index for relevant results using one or more text queries.", "module": "promptflow_vectordb.tool.common_index_lookup", + "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "preview"}, + {"name": "Faiss Index Lookup", "type": "python", "inputs": {"path": {"type": + ["string"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default"}, "top_k": {"type": ["int"], "default": "3", "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, "vector": {"type": ["list"], "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default"}, "vector_field": {"type": ["string"], "enabled_by": "connection", - "enabled_by_type": ["CognitiveSearchConnection"], "allow_manual_entry": false, - "is_multi_select": false, "input_type": "default"}}, "description": "Search - vector based query from existing Vector Database.", "module": "promptflow_vectordb.tool.vector_db_lookup", - "class_name": "VectorDBLookup", "function": "search", "is_builtin": true, - "package": "promptflow-vectordb", "package_version": "0.0.1", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "Vector Index Lookup", "type": "python", - "inputs": {"path": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "query": {"type": ["object"], "allow_manual_entry": + "default"}}, "description": "Search vector based query from the FAISS index + file.", "module": "promptflow_vectordb.tool.faiss_index_lookup", "class_name": + "FaissIndexLookup", "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "archived"}, + {"name": "Vector DB Lookup", "type": "python", "inputs": {"class_name": {"type": + ["string"], "enabled_by": "connection", "enabled_by_type": ["WeaviateConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "collection_name": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default"}, "connection": {"type": ["CognitiveSearchConnection", + "QdrantConnection", "WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "index_name": {"type": ["string"], "enabled_by": + "connection", "enabled_by_type": ["CognitiveSearchConnection"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "search_filters": + {"type": ["object"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection", + "QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default"}, "search_params": {"type": ["object"], "enabled_by": + "connection", "enabled_by_type": ["CognitiveSearchConnection", "QdrantConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "text_field": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["CognitiveSearchConnection", "QdrantConnection", "WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}}, "description": "Search text or vector based query - from AzureML Vector Index.", "module": "promptflow_vectordb.tool.vector_index_lookup", - "class_name": "VectorIndexLookup", "function": "search", "is_builtin": true, - "package": "promptflow-vectordb", "package_version": "0.0.1", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "hello_world.py", "type": "python", - "inputs": {"name": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "source": "hello_world.py", "function": - "hello_world", "is_builtin": false, "enable_kwargs": false, "tool_state": - "stable"}], "inputs": {"name": {"type": "string", "default": "hod", "is_chat_input": - false}}, "outputs": {"result": {"type": "string", "reference": "${hello_world.output}", - "evaluation_only": false, "is_chat_output": false}}}, "flowRunResourceId": - "azureml://locations/eastus/workspaces/00000/flows/batch_run_name/flowRuns/batch_run_name", + "input_type": "default"}, "vector": {"type": ["list"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "vector_field": + {"type": ["string"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}}, + "description": "Search vector based query from existing Vector Database.", + "module": "promptflow_vectordb.tool.vector_db_lookup", "class_name": "VectorDBLookup", + "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "archived"}, + {"name": "Vector Index Lookup", "type": "python", "inputs": {"path": {"type": + ["string"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default"}, "query": {"type": ["object"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}}, + "description": "Search text or vector based query from AzureML Vector Index.", + "module": "promptflow_vectordb.tool.vector_index_lookup", "class_name": "VectorIndexLookup", + "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "archived"}, + {"name": "hello_world.py", "type": "python", "inputs": {"name": {"type": ["string"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}}, + "source": "hello_world.py", "function": "hello_world", "is_builtin": false, + "enable_kwargs": false, "tool_state": "stable"}], "inputs": {"name": {"type": + "string", "default": "hod", "is_chat_input": false}}, "outputs": {"result": + {"type": "string", "reference": "${hello_world.output}", "evaluation_only": + false, "is_chat_output": false}}}, "flowRunResourceId": "azureml://locations/eastus/workspaces/00000/flows/batch_run_name/flowRuns/batch_run_name", "flowRunId": "batch_run_name", "flowRunDisplayName": "sdk-cli-test-fixture-batch-run-without-llm", "batchDataInput": {"dataUri": "azureml://datastores/workspaceblobstore/paths/LocalUpload/74c11bba717480b2d6b04b8e746d09d7/webClassification3.jsonl"}, - "flowRunType": "FlowRun", "flowType": "Default", "runtimeName": "test-runtime-ci", + "flowRunType": "FlowRun", "flowType": "Default", "runtimeName": "automatic", "inputsMapping": {"name": "${data.url}"}, "outputDatastoreName": "workspaceblobstore", "childRunBasePath": "promptflow/PromptFlowArtifacts/batch_run_name/flow_artifacts", - "flowDagFileRelativePath": "flow.dag.yaml", "flowSnapshotId": "38053316-7591-4ac8-b718-d76b16b48bbe", - "studioPortalEndpoint": "https://ml.azure.com/runs/batch_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000"}' + "flowDagFileRelativePath": "flow.dag.yaml", "flowSnapshotId": "b98f6388-29a7-47f6-93a9-09b2e2b55cb9", + "sessionId": "4a6867a952826a2a51729e6bb4bf21ac65b46eec3ae5ed10", "studioPortalEndpoint": + "https://ml.azure.com/runs/batch_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000", + "studioPortalTraceEndpoint": "https://ml.azure.com/prompts/trace/run/batch_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/promptflow-eastus"}' headers: connection: - keep-alive content-length: - - '12912' + - '42442' content-type: - application/json; charset=utf-8 strict-transport-security: - - max-age=15724800; includeSubDomains; preload + - max-age=31536000; includeSubDomains; preload transfer-encoding: - chunked vary: @@ -1060,7 +1689,7 @@ interactions: x-content-type-options: - nosniff x-request-time: - - '0.373' + - '0.482' status: code: 200 message: OK @@ -1070,20 +1699,53 @@ interactions: Accept: - application/json Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive User-Agent: - - promptflow-sdk/0.0.1 azsdk-python-azuremachinelearningdesignerserviceclient/unknown - Python/3.10.13 (Windows-10-10.0.22631-SP0) + - promptflow-azure-sdk/1.9.0.dev0 azsdk-python-azuremachinelearningdesignerserviceclient/unknown + Python/3.9.19 (Windows-10-10.0.22631-SP0) method: GET uri: https://eastus.api.azureml.ms/flow/api/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/BulkRuns/batch_run_name response: body: string: '{"flowGraph": {"nodes": [{"name": "hello_world", "type": "python", "source": {"type": "code", "path": "hello_world.py"}, "inputs": {"name": "${inputs.name}"}, - "tool": "hello_world.py", "reduce": false}], "tools": [{"name": "Content Safety - (Text Analyze)", "type": "python", "inputs": {"connection": {"type": ["AzureContentSafetyConnection"], + "tool": "hello_world.py", "reduce": false}], "tools": [{"name": "Azure OpenAI + GPT-4 Turbo with Vision", "type": "custom_llm", "inputs": {"connection": {"type": + ["AzureOpenAIConnection"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 0}}, "deployment_name": + {"type": ["string"], "enabled_by": "connection", "dynamic_list": {"func_path": + "promptflow.tools.aoai_gpt4v.list_deployment_names", "func_kwargs": [{"name": + "connection", "reference": "${inputs.connection}", "type": ["AzureOpenAIConnection"]}]}, + "allow_manual_entry": true, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 1}}, "detail": {"type": ["string"], "default": "auto", + "enum": ["low", "high", "auto"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 9}}, "frequency_penalty": + {"type": ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 7}}, "max_tokens": {"type": + ["int"], "default": 512, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 4}}, "presence_penalty": {"type": + ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 6}}, "seed": {"type": + ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 8}}, "stop": {"type": ["list"], + "default": "", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 5}}, "temperature": {"type": ["double"], + "default": 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 2}}, "top_p": {"type": ["double"], "default": + 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 3}}}, "description": "Use Azure OpenAI GPT-4 Turbo with + Vision to leverage AOAI vision ability.", "module": "promptflow.tools.aoai_gpt4v", + "class_name": "AzureOpenAI", "function": "chat", "icon": {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "default_prompt": "# system:\nAs an AI assistant, your task involves interpreting + images and responding to questions about the image.\nRemember to provide accurate + answers based on the information present in the image.\n\n# user:\nCan you + tell me what the image depicts?\n![image]({{image_input}})\n", "enable_kwargs": + false, "tool_state": "preview"}, {"name": "Content Safety (Text Analyze)", + "type": "python", "inputs": {"connection": {"type": ["AzureContentSafetyConnection"], "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, "hate_category": {"type": ["string"], "default": "medium_sensitivity", "enum": ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], @@ -1101,147 +1763,457 @@ interactions: "input_type": "default"}}, "description": "Use Azure Content Safety to detect harmful content.", "module": "promptflow.tools.azure_content_safety", "function": "analyze_text", "is_builtin": true, "package": "promptflow-tools", "package_version": - "0.0.216", "enable_kwargs": false, "deprecated_tools": ["content_safety_text.tools.content_safety_text_tool.analyze_text"], + "1.5.0rc8", "enable_kwargs": false, "deprecated_tools": ["content_safety_text.tools.content_safety_text_tool.analyze_text"], "tool_state": "stable"}, {"name": "Embedding", "type": "python", "inputs": - {"connection": {"type": ["AzureOpenAIConnection", "OpenAIConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "deployment_name": - {"type": ["string"], "enabled_by": "connection", "enabled_by_type": ["AzureOpenAIConnection"], - "model_list": ["text-embedding-ada-002", "text-search-ada-doc-001", "text-search-ada-query-001"], - "capabilities": {"completion": false, "chat_completion": false, "embeddings": - true}, "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default"}, "input": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "model": {"type": ["string"], "enum": ["text-embedding-ada-002", - "text-search-ada-doc-001", "text-search-ada-query-001"], "enabled_by": "connection", - "enabled_by_type": ["OpenAIConnection"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Use Open AI''s embedding - model to create an embedding vector representing the input text.", "module": - "promptflow.tools.embedding", "function": "embedding", "is_builtin": true, - "package": "promptflow-tools", "package_version": "0.0.216", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "Open Source LLM", "type": "custom_llm", - "inputs": {"api": {"type": ["string"], "enum": ["chat", "completion"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "connection": {"type": - ["CustomConnection"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "deployment_name": {"type": ["string"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "endpoint_name": - {"type": ["string"], "default": "-- please enter an endpoint name --", "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "max_new_tokens": - {"type": ["int"], "default": 500, "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "model_kwargs": {"type": ["object"], "default": - "{}", "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default", "advanced": true}, "temperature": {"type": ["double"], "default": - 1.0, "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default"}, "top_p": {"type": ["double"], "default": 1.0, "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default", "advanced": true}}, - "description": "Use an Open Source model from the Azure Model catalog, deployed - to an AzureML Online Endpoint for LLM Chat or Completion API calls.", "module": - "promptflow.tools.open_source_llm", "class_name": "OpenSourceLLM", "function": + {"connection": {"type": ["AzureOpenAIConnection", "OpenAIConnection", "ServerlessConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "deployment_name": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["AzureOpenAIConnection"], "model_list": ["text-embedding-ada-002", "text-search-ada-doc-001", + "text-search-ada-query-001"], "capabilities": {"completion": false, "chat_completion": + false, "embeddings": true}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "input": {"type": ["string"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "model": {"type": + ["string"], "enum": ["text-embedding-ada-002", "text-search-ada-doc-001", + "text-search-ada-query-001"], "enabled_by": "connection", "enabled_by_type": + ["OpenAIConnection"], "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default"}}, "description": "Use Open AI''s embedding model + to create an embedding vector representing the input text.", "module": "promptflow.tools.embedding", + "function": "embedding", "is_builtin": true, "package": "promptflow-tools", + "package_version": "1.5.0rc8", "enable_kwargs": false, "tool_state": "stable"}, + {"name": "LLM", "type": "custom_llm", "inputs": {"api": {"type": ["string"], + "default": "chat", "allow_manual_entry": true, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 1, "text_box_size": "sm"}, "filter_by": {"input_name": + "connection", "values": {"AzureOpenAIConnection": {"enum": ["chat", "completion"]}, + "OpenAIConnection": {"enum": ["chat", "completion"]}, "ServerlessConnection": + {"enum": ["chat"]}}, "filter_attribute": "type"}}, "best_of": {"type": ["int"], + "default": 1, "enabled_by": "api", "enabled_by_value": ["completion"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 16, "text_box_size": "xs"}}, "connection": {"type": + ["AzureOpenAIConnection", "OpenAIConnection", "ServerlessConnection"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 0}}, "deployment_name": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["AzureOpenAIConnection"], "allow_manual_entry": true, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 2, "text_box_size": + "lg"}, "filter_by": {"input_name": "api", "values": {"chat": {"capabilities": + {"chat_completion": true, "exclude": {"model_version": ["vision-preview"]}}}, + "completion": {"capabilities": {"completion": true, "include": {"model_name": + ["gpt-35-turbo-instruct"]}}}}}}, "echo": {"type": ["bool"], "default": false, + "enabled_by": "api", "enabled_by_value": ["completion"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 15, "text_box_size": "xs"}}, "frequency_penalty": {"type": + ["int"], "default": 0, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "advanced": true, "ui_hints": {"index": 10, "text_box_size": + "xs"}}, "logit_bias": {"type": ["object"], "default": "", "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 11, "text_box_size": "lg"}}, "logprobs": {"type": ["int"], + "default": "", "enabled_by": "api", "enabled_by_value": ["completion"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 14, "text_box_size": "xs"}}, "max_tokens": {"type": + ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 7, "text_box_size": "xs"}}, + "model": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["OpenAIConnection"], "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 3, "text_box_size": "lg"}, + "filter_by": {"input_name": "api", "values": {"chat": {"enum": ["gpt-3.5-turbo", + "gpt-3.5-turbo-0301", "gpt-3.5-turbo-16k", "gpt-3.5-turbo-1106", "gpt-4", + "gpt-4-1106-preview", "gpt-4-0314", "gpt-4-32k", "gpt-4-32k-0314"]}, "completion": + {"enum": ["gpt-3.5-turbo-instruct"]}}}}, "presence_penalty": {"type": ["double"], + "default": 0, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "advanced": true, "ui_hints": {"index": 8}}, "response_format": + {"type": ["object"], "default": "", "enum": [{"type": "json_object"}, {"type": + "text"}], "enabled_by": "api", "enabled_by_value": ["chat"], "allow_manual_entry": + true, "is_multi_select": false, "input_type": "default", "advanced": false, + "ui_hints": {"index": 9, "text_box_size": "lg"}}, "seed": {"type": ["int"], + "default": "", "enabled_by": "api", "enabled_by_value": ["chat"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 12, "text_box_size": "xs"}}, "stop": {"type": ["list"], + "default": "", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 6, "text_box_size": "md"}}, "suffix": {"type": + ["string"], "default": "", "enabled_by": "api", "enabled_by_value": ["completion"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "advanced": true, "ui_hints": {"index": 13, "text_box_size": "xs"}}, "temperature": + {"type": ["double"], "default": 1, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 4, "text_box_size": + "xs"}}, "tool_choice": {"type": ["object", "string"], "default": "", "enabled_by": + "api", "enabled_by_value": ["chat"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 18}}, "tools": {"type": + ["list"], "default": [], "enabled_by": "api", "enabled_by_value": ["chat"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"display_style": "reusable_tool", "index": 17}}, "top_p": {"type": + ["double"], "default": 1, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "advanced": true, "ui_hints": {"index": 5, + "text_box_size": "xs"}}}, "description": {"ai_studio": "Use OpenAI-like Large + Language Model for text completion or chat. For more details, refer to this + [document](https://aka.ms/aistudio/pf/llm-tool).", "aml": "Use OpenAI-like + Large Language Model for text completion or chat. For more details, refer + to this [document](https://aka.ms/promptflow/llm-tool).", "default": "Use + OpenAI-like Large Language Model for text completion or chat."}, "module": + "promptflow.tools.llm", "function": "llm", "icon": {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "category": "llm", "is_builtin": true, "package": "promptflow-tools", "package_version": + "1.5.0rc8", "default_prompt": "# system:\nYou are a helpful assistant.\n\n# + user:\n{{question}}\n", "enable_kwargs": false, "tool_state": "stable", "groups": + [{"name": "Tools", "description": "Configure interactive capabilities by selecting + tools for the model to use and guiding its tool call decisions.", "inputs": + ["tools", "tool_choice"], "ui_hints": {"display_style": "table"}}]}, {"name": + "LLM-Vision", "type": "custom_llm", "inputs": {"connection": {"type": ["AzureOpenAIConnection", + "OpenAIConnection"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 0}}, "deployment_name": {"type": + ["string"], "enabled_by": "connection", "enabled_by_type": ["AzureOpenAIConnection"], + "capabilities": {"chat_completion": true, "include": {"model_version": ["vision-preview"]}}, + "allow_manual_entry": true, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 1, "text_box_size": "lg"}}, "detail": {"type": ["string"], + "default": "auto", "enum": ["low", "high", "auto"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 10}}, "frequency_penalty": {"type": ["int"], "default": 0, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 8, "text_box_size": "xs"}}, "max_tokens": {"type": ["int"], "default": "", + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 6, "text_box_size": "xs"}}, "model": {"type": ["string"], + "enum": ["gpt-4-vision-preview"], "enabled_by": "connection", "enabled_by_type": + ["OpenAIConnection"], "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 2, "text_box_size": "lg"}}, + "presence_penalty": {"type": ["double"], "default": 0, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 7}}, "seed": {"type": ["int"], "default": "", "allow_manual_entry": false, + "is_multi_select": false, "input_type": "default", "ui_hints": {"index": 9, + "text_box_size": "xs"}}, "stop": {"type": ["list"], "default": "", "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 5, "text_box_size": "md"}}, "temperature": {"type": ["double"], "default": + 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 3, "text_box_size": "xs"}}, "top_p": {"type": ["double"], + "default": 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 4, "text_box_size": "xs"}}}, "description": + {"ai_studio": "Use OpenAI-like Large Language Model to leverage vision ability. + For more details, refer to this [document](https://aka.ms/aistudio/pf/llm-vision-tool).", + "aml": "Use OpenAI-like Large Language Model to leverage vision ability. For + more details, refer to this [document](https://aka.ms/promptflow/llm-vision-tool).", + "default": "Use OpenAI-like Large Language Model to leverage vision ability."}, + "module": "promptflow.tools.llm_vision", "function": "llm_vision", "icon": + {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "category": "llm", "is_builtin": true, "package": "promptflow-tools", "package_version": + "1.5.0rc8", "default_prompt": "# system:\nAs an AI assistant, your task involves + interpreting images and responding to questions about the image.\nRemember + to provide accurate answers based on the information present in the image.\n\n# + user:\nCan you tell me what the image depicts?\n![image]({{image_input}})\n", + "enable_kwargs": false, "tool_state": "preview"}, {"name": "Open Model LLM", + "type": "custom_llm", "inputs": {"api": {"type": ["string"], "enum": ["chat", + "completion"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 2}}, "deployment_name": {"type": ["string"], + "default": "", "dynamic_list": {"func_path": "promptflow.tools.open_model_llm.list_deployment_names", + "func_kwargs": [{"name": "endpoint", "optional": true, "reference": "${inputs.endpoint}", + "type": ["string"]}]}, "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 1}}, "endpoint_name": {"type": + ["string"], "dynamic_list": {"func_path": "promptflow.tools.open_model_llm.list_endpoint_names"}, + "allow_manual_entry": true, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 0}}, "max_new_tokens": {"type": ["int"], "default": + 500, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 4}}, "model_kwargs": {"type": ["object"], + "default": "{}", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "advanced": true, "ui_hints": {"index": 6}}, "temperature": {"type": + ["double"], "default": 1.0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 3}}, "top_p": {"type": + ["double"], "default": 1.0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "advanced": true, "ui_hints": {"index": 5}}}, + "description": "Use an open model from the Azure Model catalog, deployed to + an AzureML Online Endpoint for LLM Chat or Completion API calls.", "module": + "promptflow.tools.open_model_llm", "class_name": "OpenModelLLM", "function": "call", "icon": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAACgElEQVR4nGWSz2vcVRTFP/e9NzOZ1KDGohASslLEH6VLV0ak4l/QpeDCrfQPcNGliODKnVm4EBdBsIjQIlhciKW0ycKFVCSNbYnjdDLtmPnmO/nO9917XcxMkjYX3uLx7nnn3HOuMK2Nix4fP78ZdrYXVkLVWjf3l3B1B+HpcjzGFtmqa6cePz7/x0dnn1n5qhj3iBJPYREIURAJuCtpY8PjReDbrf9WG7H1fuefwQU9qKztTcMJT+PNnEFvjGVDBDlSsH6p/9MLzy6+NxwVqI8RAg4IPmWedMckdLYP6O6UpIaQfvyyXG012+e79/ZfHukoS1ISMT2hGTB1RkUmNgQ5QZ0w+a2VWDq73MbdEWmfnnv6UWe7oNzPaLapl5CwuLTXK9WUGBuCjqekzhP+z52ZXOrKMD3OJg0Hh778aiOuvpnYvp05d6GJO4iAO4QAe/eV36/X5LFRV4Zmn+AdkqlL8Vjp3oVioOz+WTPzzYEgsN+fgPLYyJVheSbPPVl2ikeGZRjtG52/8rHuaV9VOlpP2OtKyVndcRVCSqOhsvxa4vW359i6OuKdD+aP8Q4SYPdOzS/flGjt1JUSaMqZ5nwa1Y8qWb/Ud/eZZkHisYezEM0m+fcelDr8F1SqW2LNK6r1jXQwyLzy1hxvrLXZulry7ocL+FS6G4QIu3fG/Px1gdYeW7LIgXU2P/115TOA5G7e3Rmj2aS/m7l5pThiZzrCcE/d1XHzbln373nw7y6veeoUm5KCNKT/IPPwbiY1hYd/l5MIT65BMFt87sU4v9D7/JMflr44uV6hGh1+L4RCkg6z5iK2tAhNLeLsNGwYA4fDYnC/drvuuFxe86NV/x+Ut27g0FvykgAAAABJRU5ErkJggg==", - "is_builtin": true, "package": "promptflow-tools", "package_version": "0.0.216", + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", "enable_kwargs": false, "tool_state": "stable"}, {"name": "OpenAI GPT-4V", "type": "custom_llm", "inputs": {"connection": {"type": ["OpenAIConnection"], - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "frequency_penalty": {"type": ["double"], "default": 0, "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "max_tokens": {"type": + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 0}}, "detail": {"type": ["string"], "default": "auto", + "enum": ["low", "high", "auto"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 9}}, "frequency_penalty": + {"type": ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 7}}, "max_tokens": {"type": + ["int"], "default": 512, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 4}}, "model": {"type": ["string"], + "enum": ["gpt-4-vision-preview"], "allow_manual_entry": true, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 1}}, "presence_penalty": + {"type": ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 6}}, "seed": {"type": ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "model": {"type": ["string"], "enum": ["gpt-4-vision-preview"], - "allow_manual_entry": true, "is_multi_select": false, "input_type": "default"}, - "presence_penalty": {"type": ["double"], "default": 0, "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "stop": {"type": - ["list"], "default": "", "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "temperature": {"type": ["double"], "default": 1, - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "top_p": {"type": ["double"], "default": 1, "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Use OpenAI GPT-4V to leverage + "input_type": "default", "ui_hints": {"index": 8}}, "stop": {"type": ["list"], + "default": "", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 5}}, "temperature": {"type": ["double"], + "default": 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 2}}, "top_p": {"type": ["double"], "default": + 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 3}}}, "description": "Use OpenAI GPT-4V to leverage vision ability.", "module": "promptflow.tools.openai_gpt4v", "class_name": - "OpenAI", "function": "chat", "is_builtin": true, "package": "promptflow-tools", - "package_version": "0.0.216", "default_prompt": "# system:\nAs an AI assistant, - your task involves interpreting images and responding to questions about the - image.\nRemember to provide accurate answers based on the information present - in the image.\n\n# user:\nCan you tell me what the image depicts?\n![image]({{image_input}})\n", - "enable_kwargs": false, "tool_state": "stable"}, {"name": "Serp API", "type": - "python", "inputs": {"connection": {"type": ["SerpConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "engine": {"type": - ["string"], "default": "google", "enum": ["google", "bing"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "location": {"type": - ["string"], "default": "", "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "num": {"type": ["int"], "default": "10", - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "query": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "safe": {"type": ["string"], "default": "off", - "enum": ["active", "off"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Use Serp API to obtain search - results from a specific search engine.", "module": "promptflow.tools.serpapi", - "class_name": "SerpAPI", "function": "search", "is_builtin": true, "package": - "promptflow-tools", "package_version": "0.0.216", "enable_kwargs": false, - "tool_state": "stable"}, {"name": "Faiss Index Lookup", "type": "python", - "inputs": {"path": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "vector": {"type": ["list"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Search vector based query - from the FAISS index file.", "module": "promptflow_vectordb.tool.faiss_index_lookup", - "class_name": "FaissIndexLookup", "function": "search", "is_builtin": true, - "package": "promptflow-vectordb", "package_version": "0.0.1", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "Vector DB Lookup", "type": "python", - "inputs": {"class_name": {"type": ["string"], "enabled_by": "connection", - "enabled_by_type": ["WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "collection_name": {"type": ["string"], "enabled_by": - "connection", "enabled_by_type": ["QdrantConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "connection": {"type": - ["CognitiveSearchConnection", "QdrantConnection", "WeaviateConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "index_name": {"type": - ["string"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection"], - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "search_filters": {"type": ["object"], "enabled_by": "connection", "enabled_by_type": - ["CognitiveSearchConnection", "QdrantConnection"], "allow_manual_entry": false, - "is_multi_select": false, "input_type": "default"}, "search_params": {"type": - ["object"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection", - "QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "text_field": {"type": ["string"], "enabled_by": - "connection", "enabled_by_type": ["CognitiveSearchConnection", "QdrantConnection", - "WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", "allow_manual_entry": + "OpenAI", "function": "chat", "icon": {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "default_prompt": "# system:\nAs an AI assistant, your task involves interpreting + images and responding to questions about the image.\nRemember to provide accurate + answers based on the information present in the image.\n\n# user:\nCan you + tell me what the image depicts?\n![image]({{image_input}})\n", "enable_kwargs": + false, "tool_state": "preview"}, {"name": "Serp API", "type": "python", "inputs": + {"connection": {"type": ["SerpConnection"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "engine": {"type": ["string"], "default": + "google", "enum": ["google", "bing"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "location": {"type": ["string"], "default": + "", "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "num": {"type": ["int"], "default": "10", "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "query": {"type": ["string"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "safe": {"type": + ["string"], "default": "off", "enum": ["active", "off"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}}, "description": + "Use Serp API to obtain search results from a specific search engine.", "module": + "promptflow.tools.serpapi", "class_name": "SerpAPI", "function": "search", + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "enable_kwargs": false, "tool_state": "stable"}, {"name": "Index Lookup", + "type": "python", "inputs": {"acs_content_field": {"type": ["string"], "enabled_by": + "index_type", "enabled_by_value": ["Azure AI Search"], "dynamic_list": {"func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_fields", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}, {"default": "Edm.String", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "acs_embedding_field": {"type": ["string"], + "enabled_by": "index_type", "enabled_by_value": ["Azure AI Search"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_fields", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}, {"default": "Collection(Edm.Single)", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "acs_index_connection": {"type": ["CognitiveSearchConnection"], + "enabled_by": "index_type", "enabled_by_value": ["Azure AI Search"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "acs_index_name": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + AI Search"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_indices", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}]}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "acs_metadata_field": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Azure AI Search"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_fields", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}, {"default": "Edm.String", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "aoai_embedding_connection": {"type": + ["AzureOpenAIConnection"], "enabled_by": "embedding_type", "enabled_by_value": + ["Azure OpenAI"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "uionly_hidden"}, "embedding_deployment": {"type": ["string"], "enabled_by": + "embedding_type", "enabled_by_value": ["Azure OpenAI"], "dynamic_list": {"func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.list_aoai_embedding_deployments", + "func_kwargs": [{"name": "aoai_connection", "optional": false, "reference": + "${inputs.aoai_embedding_connection}", "type": ["AzurOpenAIConnection"]}]}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "embedding_model": {"type": ["string"], "enabled_by": "embedding_type", "enabled_by_value": + ["OpenAI", "Hugging Face"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_embedding_models", + "func_kwargs": [{"name": "embedding_type", "optional": false, "reference": + "${inputs.embedding_type}", "type": ["string"]}]}, "allow_manual_entry": false, + "is_multi_select": false, "input_type": "uionly_hidden"}, "embedding_type": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + AI Search", "Azure CosmosDB for MongoDB vCore", "FAISS", "Pinecone"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_available_embedding_types", + "func_kwargs": [{"name": "index_type", "optional": false, "reference": "${inputs.index_type}", + "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "uionly_hidden"}, "faiss_index_path": {"type": ["string"], "enabled_by": + "index_type", "enabled_by_value": ["FAISS"], "allow_manual_entry": false, + "is_multi_select": false, "input_type": "uionly_hidden"}, "index_type": {"type": + ["string"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_available_index_types"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mlindex_asset_id": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Registered Index"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_registered_mlindices"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mlindex_content": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": + false, "generated_by": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.forward_mapping", + "func_kwargs": [{"name": "index_type", "reference": "${inputs.index_type}", + "type": ["string"]}, {"name": "mlindex_asset_id", "optional": true, "reference": + "${inputs.mlindex_asset_id}", "type": ["string"]}, {"name": "mlindex_path", + "optional": true, "reference": "${inputs.mlindex_path}", "type": ["string"]}, + {"name": "acs_index_connection", "optional": true, "reference": "${inputs.acs_index_connection}", + "type": ["CognitiveSearchConnection"]}, {"name": "acs_index_name", "optional": + true, "reference": "${inputs.acs_index_name}", "type": ["string"]}, {"name": + "acs_content_field", "optional": true, "reference": "${inputs.acs_content_field}", + "type": ["string"]}, {"name": "acs_embedding_field", "optional": true, "reference": + "${inputs.acs_embedding_field}", "type": ["string"]}, {"name": "acs_metadata_field", + "optional": true, "reference": "${inputs.acs_metadata_field}", "type": ["string"]}, + {"name": "semantic_configuration", "optional": true, "reference": "${inputs.semantic_configuration}", + "type": ["string"]}, {"name": "faiss_index_path", "optional": true, "reference": + "${inputs.faiss_index_path}", "type": ["string"]}, {"name": "pinecone_index_connection", + "optional": true, "reference": "${inputs.pinecone_index_connection}", "type": + ["string"]}, {"name": "pinecone_index_name", "optional": true, "reference": + "${inputs.pinecone_index_name}", "type": ["string"]}, {"name": "pinecone_content_field", + "optional": true, "reference": "${inputs.pinecone_content_field}", "type": + ["string"]}, {"name": "pinecone_metadata_field", "optional": true, "reference": + "${inputs.pinecone_metadata_field}", "type": ["string"]}, {"name": "mongodb_connection", + "optional": true, "reference": "${inputs.mongodb_connection}", "type": ["string"]}, + {"name": "mongodb_database", "optional": true, "reference": "${inputs.mongodb_database}", + "type": ["string"]}, {"name": "mongodb_collection", "optional": true, "reference": + "${inputs.mongodb_collection}", "type": ["string"]}, {"name": "mongodb_index_name", + "optional": true, "reference": "${inputs.mongodb_index_name}", "type": ["string"]}, + {"name": "mongodb_content_field", "optional": true, "reference": "${inputs.mongodb_content_field}", + "type": ["string"]}, {"name": "mongodb_embedding_field", "optional": true, + "reference": "${inputs.mongodb_embedding_field}", "type": ["string"]}, {"name": + "embedding_type", "optional": true, "reference": "${inputs.embedding_type}", + "type": ["string"]}, {"name": "aoai_embedding_connection", "optional": true, + "reference": "${inputs.aoai_embedding_connection}", "type": ["AzureOpenAIConnection"]}, + {"name": "oai_embedding_connection", "optional": true, "reference": "${inputs.oai_embedding_connection}", + "type": ["string"]}, {"name": "embedding_model", "optional": true, "reference": + "${inputs.embedding_model}", "type": ["string"]}, {"name": "embedding_deployment", + "optional": true, "reference": "${inputs.embedding_deployment}", "type": ["string"]}, + {"name": "serverless_embedding_connection", "optional": true, "reference": + "${inputs.serverless_embedding_connection}", "type": ["string"]}], "reverse_func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.reverse_mapping"}, "input_type": + "default"}, "mlindex_path": {"type": ["string"], "enabled_by": "index_type", + "enabled_by_value": ["MLIndex file from path"], "allow_manual_entry": false, + "is_multi_select": false, "input_type": "uionly_hidden"}, "mongodb_collection": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + CosmosDB for MongoDB vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_collections", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}]}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mongodb_connection": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Azure CosmosDB for MongoDB vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_connections"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mongodb_content_field": {"type": ["string"], "enabled_by": "index_type", + "enabled_by_value": ["Azure CosmosDB for MongoDB vCore"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_index_fields", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}, + {"name": "mongodb_collection", "optional": false, "reference": "${inputs.mongodb_collection}", + "type": ["string"]}, {"default": "string", "name": "field_data_type", "optional": + false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "mongodb_database": {"type": ["string"], + "enabled_by": "index_type", "enabled_by_value": ["Azure CosmosDB for MongoDB + vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_databases", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}]}, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "mongodb_embedding_field": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + CosmosDB for MongoDB vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_index_fields", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}, + {"name": "mongodb_collection", "optional": false, "reference": "${inputs.mongodb_collection}", + "type": ["string"]}, {"default": "ARRAY(float)", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "mongodb_index_name": {"type": ["string"], + "enabled_by": "index_type", "enabled_by_value": ["Azure CosmosDB for MongoDB + vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_indexes", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}, + {"name": "mongodb_collection", "optional": false, "reference": "${inputs.mongodb_collection}", + "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "uionly_hidden"}, "oai_embedding_connection": {"type": ["OpenAIConnection"], + "enabled_by": "embedding_type", "enabled_by_value": ["OpenAI"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "pinecone_content_field": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Pinecone"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "pinecone_index_connection": {"type": ["PineconeConnection"], "enabled_by": + "index_type", "enabled_by_value": ["Pinecone"], "dynamic_list": {"func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.list_pinecone_connections"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "pinecone_index_name": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Pinecone"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_pinecone_indices", + "func_kwargs": [{"name": "pinecone_connection_name", "optional": false, "reference": + "${inputs.pinecone_index_connection}", "type": ["string"]}]}, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "pinecone_metadata_field": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Pinecone"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "queries": {"type": ["object"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "query_type": {"type": ["string"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_available_query_types", + "func_kwargs": [{"name": "mlindex_content", "optional": false, "reference": + "${inputs.mlindex_content}", "type": ["string"]}]}, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "semantic_configuration": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + AI Search"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_semantic_configurations", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "uionly_hidden"}, "serverless_embedding_connection": {"type": + ["string"], "enabled_by": "embedding_type", "enabled_by_value": ["Serverless + Endpoint"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_serverless_embedding_connections"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "top_k": {"type": ["int"], "default": 3, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}}, "description": "Search an AzureML Vector + Index for relevant results using one or more text queries.", "module": "promptflow_vectordb.tool.common_index_lookup", + "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "preview"}, + {"name": "Faiss Index Lookup", "type": "python", "inputs": {"path": {"type": + ["string"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default"}, "top_k": {"type": ["int"], "default": "3", "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, "vector": {"type": ["list"], "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default"}, "vector_field": {"type": ["string"], "enabled_by": "connection", - "enabled_by_type": ["CognitiveSearchConnection"], "allow_manual_entry": false, - "is_multi_select": false, "input_type": "default"}}, "description": "Search - vector based query from existing Vector Database.", "module": "promptflow_vectordb.tool.vector_db_lookup", - "class_name": "VectorDBLookup", "function": "search", "is_builtin": true, - "package": "promptflow-vectordb", "package_version": "0.0.1", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "Vector Index Lookup", "type": "python", - "inputs": {"path": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "query": {"type": ["object"], "allow_manual_entry": + "default"}}, "description": "Search vector based query from the FAISS index + file.", "module": "promptflow_vectordb.tool.faiss_index_lookup", "class_name": + "FaissIndexLookup", "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "archived"}, + {"name": "Vector DB Lookup", "type": "python", "inputs": {"class_name": {"type": + ["string"], "enabled_by": "connection", "enabled_by_type": ["WeaviateConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "collection_name": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default"}, "connection": {"type": ["CognitiveSearchConnection", + "QdrantConnection", "WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "index_name": {"type": ["string"], "enabled_by": + "connection", "enabled_by_type": ["CognitiveSearchConnection"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "search_filters": + {"type": ["object"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection", + "QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default"}, "search_params": {"type": ["object"], "enabled_by": + "connection", "enabled_by_type": ["CognitiveSearchConnection", "QdrantConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "text_field": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["CognitiveSearchConnection", "QdrantConnection", "WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}}, "description": "Search text or vector based query - from AzureML Vector Index.", "module": "promptflow_vectordb.tool.vector_index_lookup", - "class_name": "VectorIndexLookup", "function": "search", "is_builtin": true, - "package": "promptflow-vectordb", "package_version": "0.0.1", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "hello_world.py", "type": "python", - "inputs": {"name": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "source": "hello_world.py", "function": - "hello_world", "is_builtin": false, "enable_kwargs": false, "tool_state": - "stable"}], "inputs": {"name": {"type": "string", "default": "hod", "is_chat_input": - false}}, "outputs": {"result": {"type": "string", "reference": "${hello_world.output}", - "evaluation_only": false, "is_chat_output": false}}}, "flowRunResourceId": - "azureml://locations/eastus/workspaces/00000/flows/batch_run_name/flowRuns/batch_run_name", + "input_type": "default"}, "vector": {"type": ["list"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "vector_field": + {"type": ["string"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}}, + "description": "Search vector based query from existing Vector Database.", + "module": "promptflow_vectordb.tool.vector_db_lookup", "class_name": "VectorDBLookup", + "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "archived"}, + {"name": "Vector Index Lookup", "type": "python", "inputs": {"path": {"type": + ["string"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default"}, "query": {"type": ["object"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}}, + "description": "Search text or vector based query from AzureML Vector Index.", + "module": "promptflow_vectordb.tool.vector_index_lookup", "class_name": "VectorIndexLookup", + "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "archived"}, + {"name": "hello_world.py", "type": "python", "inputs": {"name": {"type": ["string"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}}, + "source": "hello_world.py", "function": "hello_world", "is_builtin": false, + "enable_kwargs": false, "tool_state": "stable"}], "inputs": {"name": {"type": + "string", "default": "hod", "is_chat_input": false}}, "outputs": {"result": + {"type": "string", "reference": "${hello_world.output}", "evaluation_only": + false, "is_chat_output": false}}}, "flowRunResourceId": "azureml://locations/eastus/workspaces/00000/flows/batch_run_name/flowRuns/batch_run_name", "flowRunId": "batch_run_name", "flowRunDisplayName": "sdk-cli-test-fixture-batch-run-without-llm", "batchDataInput": {"dataUri": "azureml://datastores/workspaceblobstore/paths/LocalUpload/74c11bba717480b2d6b04b8e746d09d7/webClassification3.jsonl"}, - "flowRunType": "FlowRun", "flowType": "Default", "runtimeName": "test-runtime-ci", + "flowRunType": "FlowRun", "flowType": "Default", "runtimeName": "automatic", "inputsMapping": {"name": "${data.url}"}, "outputDatastoreName": "workspaceblobstore", "childRunBasePath": "promptflow/PromptFlowArtifacts/batch_run_name/flow_artifacts", - "flowDagFileRelativePath": "flow.dag.yaml", "flowSnapshotId": "38053316-7591-4ac8-b718-d76b16b48bbe", - "studioPortalEndpoint": "https://ml.azure.com/runs/batch_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000"}' + "flowDagFileRelativePath": "flow.dag.yaml", "flowSnapshotId": "b98f6388-29a7-47f6-93a9-09b2e2b55cb9", + "sessionId": "4a6867a952826a2a51729e6bb4bf21ac65b46eec3ae5ed10", "studioPortalEndpoint": + "https://ml.azure.com/runs/batch_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000", + "studioPortalTraceEndpoint": "https://ml.azure.com/prompts/trace/run/batch_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/promptflow-eastus"}' headers: connection: - keep-alive content-length: - - '12912' + - '42442' content-type: - application/json; charset=utf-8 strict-transport-security: - - max-age=15724800; includeSubDomains; preload + - max-age=31536000; includeSubDomains; preload transfer-encoding: - chunked vary: @@ -1249,7 +2221,7 @@ interactions: x-content-type-options: - nosniff x-request-time: - - '0.523' + - '0.438' status: code: 200 message: OK @@ -1259,12 +2231,12 @@ interactions: Accept: - application/json Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive User-Agent: - - promptflow-sdk/0.0.1 azure-ai-ml/1.12.1 azsdk-python-mgmt-machinelearningservices/0.1.0 - Python/3.10.13 (Windows-10-10.0.22631-SP0) + - promptflow-azure-sdk/1.9.0.dev0 azure-ai-ml/1.15.0 azsdk-python-mgmt-machinelearningservices/0.1.0 + Python/3.9.19 (Windows-10-10.0.22631-SP0) method: GET uri: https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/datastores/workspaceblobstore response: @@ -1294,14 +2266,14 @@ interactions: - no-cache strict-transport-security: - max-age=31536000; includeSubDomains - transfer-encoding: - - chunked vary: - - Accept-Encoding,Accept-Encoding + - Accept-Encoding + x-cache: + - CONFIG_NOCACHE x-content-type-options: - nosniff x-request-time: - - '0.066' + - '0.103' status: code: 200 message: OK @@ -1311,14 +2283,14 @@ interactions: Accept: - application/json Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive Content-Length: - '0' User-Agent: - - promptflow-sdk/0.0.1 azure-ai-ml/1.12.1 azsdk-python-mgmt-machinelearningservices/0.1.0 - Python/3.10.13 (Windows-10-10.0.22631-SP0) + - promptflow-azure-sdk/1.9.0.dev0 azure-ai-ml/1.15.0 azsdk-python-mgmt-machinelearningservices/0.1.0 + Python/3.9.19 (Windows-10-10.0.22631-SP0) method: POST uri: https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/datastores/workspaceblobstore/listSecrets response: @@ -1337,14 +2309,12 @@ interactions: - no-cache strict-transport-security: - max-age=31536000; includeSubDomains - transfer-encoding: - - chunked - vary: - - Accept-Encoding + x-cache: + - CONFIG_NOCACHE x-content-type-options: - nosniff x-request-time: - - '0.128' + - '0.095' status: code: 200 message: OK @@ -1354,13 +2324,13 @@ interactions: Accept: - application/xml Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive User-Agent: - - azsdk-python-storage-blob/12.19.0 Python/3.10.13 (Windows-10-10.0.22631-SP0) + - azsdk-python-storage-blob/12.19.1 Python/3.9.19 (Windows-10-10.0.22631-SP0) x-ms-date: - - Fri, 12 Jan 2024 07:52:31 GMT + - Fri, 10 May 2024 08:14:29 GMT x-ms-version: - '2023-11-03' method: HEAD @@ -1404,13 +2374,13 @@ interactions: Accept: - application/xml Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive User-Agent: - - azsdk-python-storage-blob/12.19.0 Python/3.10.13 (Windows-10-10.0.22631-SP0) + - azsdk-python-storage-blob/12.19.1 Python/3.9.19 (Windows-10-10.0.22631-SP0) x-ms-date: - - Fri, 12 Jan 2024 07:52:32 GMT + - Fri, 10 May 2024 08:14:30 GMT x-ms-version: - '2023-11-03' method: HEAD @@ -1438,12 +2408,12 @@ interactions: Accept: - application/json Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive User-Agent: - - promptflow-sdk/0.0.1 azure-ai-ml/1.12.1 azsdk-python-mgmt-machinelearningservices/0.1.0 - Python/3.10.13 (Windows-10-10.0.22631-SP0) + - promptflow-azure-sdk/1.9.0.dev0 azure-ai-ml/1.15.0 azsdk-python-mgmt-machinelearningservices/0.1.0 + Python/3.9.19 (Windows-10-10.0.22631-SP0) method: GET uri: https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/datastores/workspaceblobstore response: @@ -1473,14 +2443,14 @@ interactions: - no-cache strict-transport-security: - max-age=31536000; includeSubDomains - transfer-encoding: - - chunked vary: - - Accept-Encoding,Accept-Encoding + - Accept-Encoding + x-cache: + - CONFIG_NOCACHE x-content-type-options: - nosniff x-request-time: - - '0.065' + - '0.081' status: code: 200 message: OK @@ -1490,14 +2460,14 @@ interactions: Accept: - application/json Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive Content-Length: - '0' User-Agent: - - promptflow-sdk/0.0.1 azure-ai-ml/1.12.1 azsdk-python-mgmt-machinelearningservices/0.1.0 - Python/3.10.13 (Windows-10-10.0.22631-SP0) + - promptflow-azure-sdk/1.9.0.dev0 azure-ai-ml/1.15.0 azsdk-python-mgmt-machinelearningservices/0.1.0 + Python/3.9.19 (Windows-10-10.0.22631-SP0) method: POST uri: https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/datastores/workspaceblobstore/listSecrets response: @@ -1516,14 +2486,12 @@ interactions: - no-cache strict-transport-security: - max-age=31536000; includeSubDomains - transfer-encoding: - - chunked - vary: - - Accept-Encoding + x-cache: + - CONFIG_NOCACHE x-content-type-options: - nosniff x-request-time: - - '0.113' + - '0.090' status: code: 200 message: OK @@ -1533,13 +2501,13 @@ interactions: Accept: - application/xml Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive User-Agent: - - azsdk-python-storage-blob/12.19.0 Python/3.10.13 (Windows-10-10.0.22631-SP0) + - azsdk-python-storage-blob/12.19.1 Python/3.9.19 (Windows-10-10.0.22631-SP0) x-ms-date: - - Fri, 12 Jan 2024 07:52:35 GMT + - Fri, 10 May 2024 08:14:34 GMT x-ms-version: - '2023-11-03' method: HEAD @@ -1551,13 +2519,13 @@ interactions: accept-ranges: - bytes content-length: - - '409' + - '414' content-md5: - - OyENtlqGVUTrY5zKuzo8XA== + - VW9g++E+ajc1rRZjwT916A== content-type: - application/octet-stream last-modified: - - Tue, 21 Nov 2023 08:03:40 GMT + - Fri, 12 Apr 2024 15:53:20 GMT server: - Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0 vary: @@ -1565,9 +2533,9 @@ interactions: x-ms-blob-type: - BlockBlob x-ms-creation-time: - - Tue, 21 Nov 2023 08:03:39 GMT + - Fri, 12 Apr 2024 15:53:19 GMT x-ms-meta-name: - - fd932777-4f3a-4c1d-9c3a-24d45835d7e1 + - c360fbde-c4bd-4003-9da3-42aa46b541b4 x-ms-meta-upload_status: - completed x-ms-meta-version: @@ -1583,13 +2551,13 @@ interactions: Accept: - application/xml Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive User-Agent: - - azsdk-python-storage-blob/12.19.0 Python/3.10.13 (Windows-10-10.0.22631-SP0) + - azsdk-python-storage-blob/12.19.1 Python/3.9.19 (Windows-10-10.0.22631-SP0) x-ms-date: - - Fri, 12 Jan 2024 07:52:36 GMT + - Fri, 10 May 2024 08:14:35 GMT x-ms-version: - '2023-11-03' method: HEAD @@ -1615,27 +2583,26 @@ interactions: body: '{"flowDefinitionDataStoreName": "workspaceblobstore", "flowDefinitionBlobPath": "LocalUpload/000000000000000000000000000000000000/eval-classification-accuracy/flow.dag.yaml", "runId": "eval_run_name", "runDisplayName": "sdk-cli-test-fixture-eval-run-without-llm", - "runExperimentName": "", "variantRunId": "batch_run_name", "batchDataInput": - {"dataUri": "azureml://datastores/workspaceblobstore/paths/LocalUpload/000000000000000000000000000000000000/webClassification3.jsonl"}, + "runExperimentName": "", "variantRunId": "batch_run_name", "sessionId": "000000000000000000000000000000000000000000000000", + "sessionSetupMode": "SystemWait", "flowLineageId": "0000000000000000000000000000000000000000000000000000000000000000", + "runDisplayNameGenerationType": "UserProvidedMacro", "batchDataInput": {"dataUri": + "azureml://datastores/workspaceblobstore/paths/LocalUpload/000000000000000000000000000000000000/webClassification3.jsonl"}, "inputsMapping": {"groundtruth": "${data.answer}", "prediction": "${run.outputs.result}"}, - "connections": {}, "environmentVariables": {}, "runtimeName": "fake-runtime-name", - "sessionId": "000000000000000000000000000000000000000000000000", "sessionSetupMode": - "SystemWait", "flowLineageId": "0000000000000000000000000000000000000000000000000000000000000000", - "runDisplayNameGenerationType": "UserProvidedMacro"}' + "environmentVariables": {}, "connections": {}, "runtimeName": "fake-runtime-name"}' headers: Accept: - application/json Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive Content-Length: - - '950' + - '941' Content-Type: - application/json User-Agent: - - promptflow-sdk/0.0.1 azsdk-python-azuremachinelearningdesignerserviceclient/unknown - Python/3.10.13 (Windows-10-10.0.22631-SP0) + - promptflow-azure-sdk/1.9.0.dev0 azsdk-python-azuremachinelearningdesignerserviceclient/unknown + Python/3.9.19 (Windows-10-10.0.22631-SP0) method: POST uri: https://eastus.api.azureml.ms/flow/api/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/BulkRuns/submit response: @@ -1649,11 +2616,11 @@ interactions: content-type: - application/json; charset=utf-8 strict-transport-security: - - max-age=15724800; includeSubDomains; preload + - max-age=31536000; includeSubDomains; preload x-content-type-options: - nosniff x-request-time: - - '5.293' + - '6.502' status: code: 200 message: OK @@ -1663,12 +2630,12 @@ interactions: Accept: - application/json Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive User-Agent: - - promptflow-sdk/0.0.1 azsdk-python-azuremachinelearningdesignerserviceclient/unknown - Python/3.10.13 (Windows-10-10.0.22631-SP0) + - promptflow-azure-sdk/1.9.0.dev0 azsdk-python-azuremachinelearningdesignerserviceclient/unknown + Python/3.9.19 (Windows-10-10.0.22631-SP0) method: GET uri: https://eastus.api.azureml.ms/flow/api/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/BulkRuns/eval_run_name response: @@ -1678,176 +2645,519 @@ interactions: "prediction": "${inputs.prediction}"}, "tool": "grade.py", "reduce": false}, {"name": "calculate_accuracy", "type": "python", "source": {"type": "code", "path": "calculate_accuracy.py"}, "inputs": {"grades": "${grade.output}"}, - "tool": "calculate_accuracy.py", "reduce": true}], "tools": [{"name": "Content - Safety (Text Analyze)", "type": "python", "inputs": {"connection": {"type": - ["AzureContentSafetyConnection"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "hate_category": {"type": ["string"], "default": - "medium_sensitivity", "enum": ["disable", "low_sensitivity", "medium_sensitivity", - "high_sensitivity"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "self_harm_category": {"type": ["string"], "default": - "medium_sensitivity", "enum": ["disable", "low_sensitivity", "medium_sensitivity", - "high_sensitivity"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "sexual_category": {"type": ["string"], "default": - "medium_sensitivity", "enum": ["disable", "low_sensitivity", "medium_sensitivity", + "tool": "calculate_accuracy.py", "reduce": true}], "tools": [{"name": "Azure + OpenAI GPT-4 Turbo with Vision", "type": "custom_llm", "inputs": {"connection": + {"type": ["AzureOpenAIConnection"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 0}}, "deployment_name": + {"type": ["string"], "enabled_by": "connection", "dynamic_list": {"func_path": + "promptflow.tools.aoai_gpt4v.list_deployment_names", "func_kwargs": [{"name": + "connection", "reference": "${inputs.connection}", "type": ["AzureOpenAIConnection"]}]}, + "allow_manual_entry": true, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 1}}, "detail": {"type": ["string"], "default": "auto", + "enum": ["low", "high", "auto"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 9}}, "frequency_penalty": + {"type": ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 7}}, "max_tokens": {"type": + ["int"], "default": 512, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 4}}, "presence_penalty": {"type": + ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 6}}, "seed": {"type": + ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 8}}, "stop": {"type": ["list"], + "default": "", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 5}}, "temperature": {"type": ["double"], + "default": 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 2}}, "top_p": {"type": ["double"], "default": + 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 3}}}, "description": "Use Azure OpenAI GPT-4 Turbo with + Vision to leverage AOAI vision ability.", "module": "promptflow.tools.aoai_gpt4v", + "class_name": "AzureOpenAI", "function": "chat", "icon": {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "default_prompt": "# system:\nAs an AI assistant, your task involves interpreting + images and responding to questions about the image.\nRemember to provide accurate + answers based on the information present in the image.\n\n# user:\nCan you + tell me what the image depicts?\n![image]({{image_input}})\n", "enable_kwargs": + false, "tool_state": "preview"}, {"name": "Content Safety (Text Analyze)", + "type": "python", "inputs": {"connection": {"type": ["AzureContentSafetyConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "hate_category": {"type": ["string"], "default": "medium_sensitivity", "enum": + ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "self_harm_category": {"type": ["string"], "default": "medium_sensitivity", + "enum": ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "sexual_category": {"type": ["string"], "default": "medium_sensitivity", "enum": + ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "text": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "violence_category": {"type": ["string"], + "default": "medium_sensitivity", "enum": ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "text": {"type": ["string"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "violence_category": - {"type": ["string"], "default": "medium_sensitivity", "enum": ["disable", - "low_sensitivity", "medium_sensitivity", "high_sensitivity"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}}, "description": - "Use Azure Content Safety to detect harmful content.", "module": "promptflow.tools.azure_content_safety", - "function": "analyze_text", "is_builtin": true, "package": "promptflow-tools", - "package_version": "0.0.216", "enable_kwargs": false, "deprecated_tools": - ["content_safety_text.tools.content_safety_text_tool.analyze_text"], "tool_state": - "stable"}, {"name": "Embedding", "type": "python", "inputs": {"connection": - {"type": ["AzureOpenAIConnection", "OpenAIConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "deployment_name": - {"type": ["string"], "enabled_by": "connection", "enabled_by_type": ["AzureOpenAIConnection"], - "model_list": ["text-embedding-ada-002", "text-search-ada-doc-001", "text-search-ada-query-001"], - "capabilities": {"completion": false, "chat_completion": false, "embeddings": - true}, "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default"}, "input": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "model": {"type": ["string"], "enum": ["text-embedding-ada-002", - "text-search-ada-doc-001", "text-search-ada-query-001"], "enabled_by": "connection", - "enabled_by_type": ["OpenAIConnection"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Use Open AI''s embedding - model to create an embedding vector representing the input text.", "module": - "promptflow.tools.embedding", "function": "embedding", "is_builtin": true, - "package": "promptflow-tools", "package_version": "0.0.216", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "Open Source LLM", "type": "custom_llm", - "inputs": {"api": {"type": ["string"], "enum": ["chat", "completion"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "connection": {"type": - ["CustomConnection"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "deployment_name": {"type": ["string"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "endpoint_name": - {"type": ["string"], "default": "-- please enter an endpoint name --", "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "max_new_tokens": - {"type": ["int"], "default": 500, "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "model_kwargs": {"type": ["object"], "default": - "{}", "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default", "advanced": true}, "temperature": {"type": ["double"], "default": - 1.0, "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default"}, "top_p": {"type": ["double"], "default": 1.0, "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default", "advanced": true}}, - "description": "Use an Open Source model from the Azure Model catalog, deployed - to an AzureML Online Endpoint for LLM Chat or Completion API calls.", "module": - "promptflow.tools.open_source_llm", "class_name": "OpenSourceLLM", "function": + "input_type": "default"}}, "description": "Use Azure Content Safety to detect + harmful content.", "module": "promptflow.tools.azure_content_safety", "function": + "analyze_text", "is_builtin": true, "package": "promptflow-tools", "package_version": + "1.5.0rc8", "enable_kwargs": false, "deprecated_tools": ["content_safety_text.tools.content_safety_text_tool.analyze_text"], + "tool_state": "stable"}, {"name": "Embedding", "type": "python", "inputs": + {"connection": {"type": ["AzureOpenAIConnection", "OpenAIConnection", "ServerlessConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "deployment_name": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["AzureOpenAIConnection"], "model_list": ["text-embedding-ada-002", "text-search-ada-doc-001", + "text-search-ada-query-001"], "capabilities": {"completion": false, "chat_completion": + false, "embeddings": true}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "input": {"type": ["string"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "model": {"type": + ["string"], "enum": ["text-embedding-ada-002", "text-search-ada-doc-001", + "text-search-ada-query-001"], "enabled_by": "connection", "enabled_by_type": + ["OpenAIConnection"], "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default"}}, "description": "Use Open AI''s embedding model + to create an embedding vector representing the input text.", "module": "promptflow.tools.embedding", + "function": "embedding", "is_builtin": true, "package": "promptflow-tools", + "package_version": "1.5.0rc8", "enable_kwargs": false, "tool_state": "stable"}, + {"name": "LLM", "type": "custom_llm", "inputs": {"api": {"type": ["string"], + "default": "chat", "allow_manual_entry": true, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 1, "text_box_size": "sm"}, "filter_by": {"input_name": + "connection", "values": {"AzureOpenAIConnection": {"enum": ["chat", "completion"]}, + "OpenAIConnection": {"enum": ["chat", "completion"]}, "ServerlessConnection": + {"enum": ["chat"]}}, "filter_attribute": "type"}}, "best_of": {"type": ["int"], + "default": 1, "enabled_by": "api", "enabled_by_value": ["completion"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 16, "text_box_size": "xs"}}, "connection": {"type": + ["AzureOpenAIConnection", "OpenAIConnection", "ServerlessConnection"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 0}}, "deployment_name": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["AzureOpenAIConnection"], "allow_manual_entry": true, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 2, "text_box_size": + "lg"}, "filter_by": {"input_name": "api", "values": {"chat": {"capabilities": + {"chat_completion": true, "exclude": {"model_version": ["vision-preview"]}}}, + "completion": {"capabilities": {"completion": true, "include": {"model_name": + ["gpt-35-turbo-instruct"]}}}}}}, "echo": {"type": ["bool"], "default": false, + "enabled_by": "api", "enabled_by_value": ["completion"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 15, "text_box_size": "xs"}}, "frequency_penalty": {"type": + ["int"], "default": 0, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "advanced": true, "ui_hints": {"index": 10, "text_box_size": + "xs"}}, "logit_bias": {"type": ["object"], "default": "", "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 11, "text_box_size": "lg"}}, "logprobs": {"type": ["int"], + "default": "", "enabled_by": "api", "enabled_by_value": ["completion"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 14, "text_box_size": "xs"}}, "max_tokens": {"type": + ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 7, "text_box_size": "xs"}}, + "model": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["OpenAIConnection"], "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 3, "text_box_size": "lg"}, + "filter_by": {"input_name": "api", "values": {"chat": {"enum": ["gpt-3.5-turbo", + "gpt-3.5-turbo-0301", "gpt-3.5-turbo-16k", "gpt-3.5-turbo-1106", "gpt-4", + "gpt-4-1106-preview", "gpt-4-0314", "gpt-4-32k", "gpt-4-32k-0314"]}, "completion": + {"enum": ["gpt-3.5-turbo-instruct"]}}}}, "presence_penalty": {"type": ["double"], + "default": 0, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "advanced": true, "ui_hints": {"index": 8}}, "response_format": + {"type": ["object"], "default": "", "enum": [{"type": "json_object"}, {"type": + "text"}], "enabled_by": "api", "enabled_by_value": ["chat"], "allow_manual_entry": + true, "is_multi_select": false, "input_type": "default", "advanced": false, + "ui_hints": {"index": 9, "text_box_size": "lg"}}, "seed": {"type": ["int"], + "default": "", "enabled_by": "api", "enabled_by_value": ["chat"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 12, "text_box_size": "xs"}}, "stop": {"type": ["list"], + "default": "", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 6, "text_box_size": "md"}}, "suffix": {"type": + ["string"], "default": "", "enabled_by": "api", "enabled_by_value": ["completion"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "advanced": true, "ui_hints": {"index": 13, "text_box_size": "xs"}}, "temperature": + {"type": ["double"], "default": 1, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 4, "text_box_size": + "xs"}}, "tool_choice": {"type": ["object", "string"], "default": "", "enabled_by": + "api", "enabled_by_value": ["chat"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 18}}, "tools": {"type": + ["list"], "default": [], "enabled_by": "api", "enabled_by_value": ["chat"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"display_style": "reusable_tool", "index": 17}}, "top_p": {"type": + ["double"], "default": 1, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "advanced": true, "ui_hints": {"index": 5, + "text_box_size": "xs"}}}, "description": {"ai_studio": "Use OpenAI-like Large + Language Model for text completion or chat. For more details, refer to this + [document](https://aka.ms/aistudio/pf/llm-tool).", "aml": "Use OpenAI-like + Large Language Model for text completion or chat. For more details, refer + to this [document](https://aka.ms/promptflow/llm-tool).", "default": "Use + OpenAI-like Large Language Model for text completion or chat."}, "module": + "promptflow.tools.llm", "function": "llm", "icon": {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "category": "llm", "is_builtin": true, "package": "promptflow-tools", "package_version": + "1.5.0rc8", "default_prompt": "# system:\nYou are a helpful assistant.\n\n# + user:\n{{question}}\n", "enable_kwargs": false, "tool_state": "stable", "groups": + [{"name": "Tools", "description": "Configure interactive capabilities by selecting + tools for the model to use and guiding its tool call decisions.", "inputs": + ["tools", "tool_choice"], "ui_hints": {"display_style": "table"}}]}, {"name": + "LLM-Vision", "type": "custom_llm", "inputs": {"connection": {"type": ["AzureOpenAIConnection", + "OpenAIConnection"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 0}}, "deployment_name": {"type": + ["string"], "enabled_by": "connection", "enabled_by_type": ["AzureOpenAIConnection"], + "capabilities": {"chat_completion": true, "include": {"model_version": ["vision-preview"]}}, + "allow_manual_entry": true, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 1, "text_box_size": "lg"}}, "detail": {"type": ["string"], + "default": "auto", "enum": ["low", "high", "auto"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 10}}, "frequency_penalty": {"type": ["int"], "default": 0, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 8, "text_box_size": "xs"}}, "max_tokens": {"type": ["int"], "default": "", + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 6, "text_box_size": "xs"}}, "model": {"type": ["string"], + "enum": ["gpt-4-vision-preview"], "enabled_by": "connection", "enabled_by_type": + ["OpenAIConnection"], "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 2, "text_box_size": "lg"}}, + "presence_penalty": {"type": ["double"], "default": 0, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 7}}, "seed": {"type": ["int"], "default": "", "allow_manual_entry": false, + "is_multi_select": false, "input_type": "default", "ui_hints": {"index": 9, + "text_box_size": "xs"}}, "stop": {"type": ["list"], "default": "", "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 5, "text_box_size": "md"}}, "temperature": {"type": ["double"], "default": + 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 3, "text_box_size": "xs"}}, "top_p": {"type": ["double"], + "default": 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 4, "text_box_size": "xs"}}}, "description": + {"ai_studio": "Use OpenAI-like Large Language Model to leverage vision ability. + For more details, refer to this [document](https://aka.ms/aistudio/pf/llm-vision-tool).", + "aml": "Use OpenAI-like Large Language Model to leverage vision ability. For + more details, refer to this [document](https://aka.ms/promptflow/llm-vision-tool).", + "default": "Use OpenAI-like Large Language Model to leverage vision ability."}, + "module": "promptflow.tools.llm_vision", "function": "llm_vision", "icon": + {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "category": "llm", "is_builtin": true, "package": "promptflow-tools", "package_version": + "1.5.0rc8", "default_prompt": "# system:\nAs an AI assistant, your task involves + interpreting images and responding to questions about the image.\nRemember + to provide accurate answers based on the information present in the image.\n\n# + user:\nCan you tell me what the image depicts?\n![image]({{image_input}})\n", + "enable_kwargs": false, "tool_state": "preview"}, {"name": "Open Model LLM", + "type": "custom_llm", "inputs": {"api": {"type": ["string"], "enum": ["chat", + "completion"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 2}}, "deployment_name": {"type": ["string"], + "default": "", "dynamic_list": {"func_path": "promptflow.tools.open_model_llm.list_deployment_names", + "func_kwargs": [{"name": "endpoint", "optional": true, "reference": "${inputs.endpoint}", + "type": ["string"]}]}, "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 1}}, "endpoint_name": {"type": + ["string"], "dynamic_list": {"func_path": "promptflow.tools.open_model_llm.list_endpoint_names"}, + "allow_manual_entry": true, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 0}}, "max_new_tokens": {"type": ["int"], "default": + 500, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 4}}, "model_kwargs": {"type": ["object"], + "default": "{}", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "advanced": true, "ui_hints": {"index": 6}}, "temperature": {"type": + ["double"], "default": 1.0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 3}}, "top_p": {"type": + ["double"], "default": 1.0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "advanced": true, "ui_hints": {"index": 5}}}, + "description": "Use an open model from the Azure Model catalog, deployed to + an AzureML Online Endpoint for LLM Chat or Completion API calls.", "module": + "promptflow.tools.open_model_llm", "class_name": "OpenModelLLM", "function": "call", "icon": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAACgElEQVR4nGWSz2vcVRTFP/e9NzOZ1KDGohASslLEH6VLV0ak4l/QpeDCrfQPcNGliODKnVm4EBdBsIjQIlhciKW0ycKFVCSNbYnjdDLtmPnmO/nO9917XcxMkjYX3uLx7nnn3HOuMK2Nix4fP78ZdrYXVkLVWjf3l3B1B+HpcjzGFtmqa6cePz7/x0dnn1n5qhj3iBJPYREIURAJuCtpY8PjReDbrf9WG7H1fuefwQU9qKztTcMJT+PNnEFvjGVDBDlSsH6p/9MLzy6+NxwVqI8RAg4IPmWedMckdLYP6O6UpIaQfvyyXG012+e79/ZfHukoS1ISMT2hGTB1RkUmNgQ5QZ0w+a2VWDq73MbdEWmfnnv6UWe7oNzPaLapl5CwuLTXK9WUGBuCjqekzhP+z52ZXOrKMD3OJg0Hh778aiOuvpnYvp05d6GJO4iAO4QAe/eV36/X5LFRV4Zmn+AdkqlL8Vjp3oVioOz+WTPzzYEgsN+fgPLYyJVheSbPPVl2ikeGZRjtG52/8rHuaV9VOlpP2OtKyVndcRVCSqOhsvxa4vW359i6OuKdD+aP8Q4SYPdOzS/flGjt1JUSaMqZ5nwa1Y8qWb/Ud/eZZkHisYezEM0m+fcelDr8F1SqW2LNK6r1jXQwyLzy1hxvrLXZulry7ocL+FS6G4QIu3fG/Px1gdYeW7LIgXU2P/115TOA5G7e3Rmj2aS/m7l5pThiZzrCcE/d1XHzbln373nw7y6veeoUm5KCNKT/IPPwbiY1hYd/l5MIT65BMFt87sU4v9D7/JMflr44uV6hGh1+L4RCkg6z5iK2tAhNLeLsNGwYA4fDYnC/drvuuFxe86NV/x+Ut27g0FvykgAAAABJRU5ErkJggg==", - "is_builtin": true, "package": "promptflow-tools", "package_version": "0.0.216", + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", "enable_kwargs": false, "tool_state": "stable"}, {"name": "OpenAI GPT-4V", "type": "custom_llm", "inputs": {"connection": {"type": ["OpenAIConnection"], - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "frequency_penalty": {"type": ["double"], "default": 0, "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "max_tokens": {"type": + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 0}}, "detail": {"type": ["string"], "default": "auto", + "enum": ["low", "high", "auto"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 9}}, "frequency_penalty": + {"type": ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 7}}, "max_tokens": {"type": + ["int"], "default": 512, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 4}}, "model": {"type": ["string"], + "enum": ["gpt-4-vision-preview"], "allow_manual_entry": true, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 1}}, "presence_penalty": + {"type": ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 6}}, "seed": {"type": ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "model": {"type": ["string"], "enum": ["gpt-4-vision-preview"], - "allow_manual_entry": true, "is_multi_select": false, "input_type": "default"}, - "presence_penalty": {"type": ["double"], "default": 0, "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "stop": {"type": - ["list"], "default": "", "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "temperature": {"type": ["double"], "default": 1, - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "top_p": {"type": ["double"], "default": 1, "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Use OpenAI GPT-4V to leverage + "input_type": "default", "ui_hints": {"index": 8}}, "stop": {"type": ["list"], + "default": "", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 5}}, "temperature": {"type": ["double"], + "default": 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 2}}, "top_p": {"type": ["double"], "default": + 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 3}}}, "description": "Use OpenAI GPT-4V to leverage vision ability.", "module": "promptflow.tools.openai_gpt4v", "class_name": - "OpenAI", "function": "chat", "is_builtin": true, "package": "promptflow-tools", - "package_version": "0.0.216", "default_prompt": "# system:\nAs an AI assistant, - your task involves interpreting images and responding to questions about the - image.\nRemember to provide accurate answers based on the information present - in the image.\n\n# user:\nCan you tell me what the image depicts?\n![image]({{image_input}})\n", - "enable_kwargs": false, "tool_state": "stable"}, {"name": "Serp API", "type": - "python", "inputs": {"connection": {"type": ["SerpConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "engine": {"type": - ["string"], "default": "google", "enum": ["google", "bing"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "location": {"type": - ["string"], "default": "", "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "num": {"type": ["int"], "default": "10", - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "query": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "safe": {"type": ["string"], "default": "off", - "enum": ["active", "off"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Use Serp API to obtain search - results from a specific search engine.", "module": "promptflow.tools.serpapi", - "class_name": "SerpAPI", "function": "search", "is_builtin": true, "package": - "promptflow-tools", "package_version": "0.0.216", "enable_kwargs": false, - "tool_state": "stable"}, {"name": "Faiss Index Lookup", "type": "python", - "inputs": {"path": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "vector": {"type": ["list"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Search vector based query - from the FAISS index file.", "module": "promptflow_vectordb.tool.faiss_index_lookup", - "class_name": "FaissIndexLookup", "function": "search", "is_builtin": true, - "package": "promptflow-vectordb", "package_version": "0.0.1", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "Vector DB Lookup", "type": "python", - "inputs": {"class_name": {"type": ["string"], "enabled_by": "connection", - "enabled_by_type": ["WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "collection_name": {"type": ["string"], "enabled_by": - "connection", "enabled_by_type": ["QdrantConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "connection": {"type": - ["CognitiveSearchConnection", "QdrantConnection", "WeaviateConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "index_name": {"type": - ["string"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection"], - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "search_filters": {"type": ["object"], "enabled_by": "connection", "enabled_by_type": - ["CognitiveSearchConnection", "QdrantConnection"], "allow_manual_entry": false, - "is_multi_select": false, "input_type": "default"}, "search_params": {"type": - ["object"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection", - "QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "text_field": {"type": ["string"], "enabled_by": - "connection", "enabled_by_type": ["CognitiveSearchConnection", "QdrantConnection", - "WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", "allow_manual_entry": + "OpenAI", "function": "chat", "icon": {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "default_prompt": "# system:\nAs an AI assistant, your task involves interpreting + images and responding to questions about the image.\nRemember to provide accurate + answers based on the information present in the image.\n\n# user:\nCan you + tell me what the image depicts?\n![image]({{image_input}})\n", "enable_kwargs": + false, "tool_state": "preview"}, {"name": "Serp API", "type": "python", "inputs": + {"connection": {"type": ["SerpConnection"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "engine": {"type": ["string"], "default": + "google", "enum": ["google", "bing"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "location": {"type": ["string"], "default": + "", "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "num": {"type": ["int"], "default": "10", "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "query": {"type": ["string"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "safe": {"type": + ["string"], "default": "off", "enum": ["active", "off"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}}, "description": + "Use Serp API to obtain search results from a specific search engine.", "module": + "promptflow.tools.serpapi", "class_name": "SerpAPI", "function": "search", + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "enable_kwargs": false, "tool_state": "stable"}, {"name": "Index Lookup", + "type": "python", "inputs": {"acs_content_field": {"type": ["string"], "enabled_by": + "index_type", "enabled_by_value": ["Azure AI Search"], "dynamic_list": {"func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_fields", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}, {"default": "Edm.String", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "acs_embedding_field": {"type": ["string"], + "enabled_by": "index_type", "enabled_by_value": ["Azure AI Search"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_fields", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}, {"default": "Collection(Edm.Single)", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "acs_index_connection": {"type": ["CognitiveSearchConnection"], + "enabled_by": "index_type", "enabled_by_value": ["Azure AI Search"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "acs_index_name": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + AI Search"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_indices", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}]}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "acs_metadata_field": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Azure AI Search"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_fields", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}, {"default": "Edm.String", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "aoai_embedding_connection": {"type": + ["AzureOpenAIConnection"], "enabled_by": "embedding_type", "enabled_by_value": + ["Azure OpenAI"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "uionly_hidden"}, "embedding_deployment": {"type": ["string"], "enabled_by": + "embedding_type", "enabled_by_value": ["Azure OpenAI"], "dynamic_list": {"func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.list_aoai_embedding_deployments", + "func_kwargs": [{"name": "aoai_connection", "optional": false, "reference": + "${inputs.aoai_embedding_connection}", "type": ["AzurOpenAIConnection"]}]}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "embedding_model": {"type": ["string"], "enabled_by": "embedding_type", "enabled_by_value": + ["OpenAI", "Hugging Face"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_embedding_models", + "func_kwargs": [{"name": "embedding_type", "optional": false, "reference": + "${inputs.embedding_type}", "type": ["string"]}]}, "allow_manual_entry": false, + "is_multi_select": false, "input_type": "uionly_hidden"}, "embedding_type": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + AI Search", "Azure CosmosDB for MongoDB vCore", "FAISS", "Pinecone"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_available_embedding_types", + "func_kwargs": [{"name": "index_type", "optional": false, "reference": "${inputs.index_type}", + "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "uionly_hidden"}, "faiss_index_path": {"type": ["string"], "enabled_by": + "index_type", "enabled_by_value": ["FAISS"], "allow_manual_entry": false, + "is_multi_select": false, "input_type": "uionly_hidden"}, "index_type": {"type": + ["string"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_available_index_types"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mlindex_asset_id": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Registered Index"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_registered_mlindices"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mlindex_content": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": + false, "generated_by": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.forward_mapping", + "func_kwargs": [{"name": "index_type", "reference": "${inputs.index_type}", + "type": ["string"]}, {"name": "mlindex_asset_id", "optional": true, "reference": + "${inputs.mlindex_asset_id}", "type": ["string"]}, {"name": "mlindex_path", + "optional": true, "reference": "${inputs.mlindex_path}", "type": ["string"]}, + {"name": "acs_index_connection", "optional": true, "reference": "${inputs.acs_index_connection}", + "type": ["CognitiveSearchConnection"]}, {"name": "acs_index_name", "optional": + true, "reference": "${inputs.acs_index_name}", "type": ["string"]}, {"name": + "acs_content_field", "optional": true, "reference": "${inputs.acs_content_field}", + "type": ["string"]}, {"name": "acs_embedding_field", "optional": true, "reference": + "${inputs.acs_embedding_field}", "type": ["string"]}, {"name": "acs_metadata_field", + "optional": true, "reference": "${inputs.acs_metadata_field}", "type": ["string"]}, + {"name": "semantic_configuration", "optional": true, "reference": "${inputs.semantic_configuration}", + "type": ["string"]}, {"name": "faiss_index_path", "optional": true, "reference": + "${inputs.faiss_index_path}", "type": ["string"]}, {"name": "pinecone_index_connection", + "optional": true, "reference": "${inputs.pinecone_index_connection}", "type": + ["string"]}, {"name": "pinecone_index_name", "optional": true, "reference": + "${inputs.pinecone_index_name}", "type": ["string"]}, {"name": "pinecone_content_field", + "optional": true, "reference": "${inputs.pinecone_content_field}", "type": + ["string"]}, {"name": "pinecone_metadata_field", "optional": true, "reference": + "${inputs.pinecone_metadata_field}", "type": ["string"]}, {"name": "mongodb_connection", + "optional": true, "reference": "${inputs.mongodb_connection}", "type": ["string"]}, + {"name": "mongodb_database", "optional": true, "reference": "${inputs.mongodb_database}", + "type": ["string"]}, {"name": "mongodb_collection", "optional": true, "reference": + "${inputs.mongodb_collection}", "type": ["string"]}, {"name": "mongodb_index_name", + "optional": true, "reference": "${inputs.mongodb_index_name}", "type": ["string"]}, + {"name": "mongodb_content_field", "optional": true, "reference": "${inputs.mongodb_content_field}", + "type": ["string"]}, {"name": "mongodb_embedding_field", "optional": true, + "reference": "${inputs.mongodb_embedding_field}", "type": ["string"]}, {"name": + "embedding_type", "optional": true, "reference": "${inputs.embedding_type}", + "type": ["string"]}, {"name": "aoai_embedding_connection", "optional": true, + "reference": "${inputs.aoai_embedding_connection}", "type": ["AzureOpenAIConnection"]}, + {"name": "oai_embedding_connection", "optional": true, "reference": "${inputs.oai_embedding_connection}", + "type": ["string"]}, {"name": "embedding_model", "optional": true, "reference": + "${inputs.embedding_model}", "type": ["string"]}, {"name": "embedding_deployment", + "optional": true, "reference": "${inputs.embedding_deployment}", "type": ["string"]}, + {"name": "serverless_embedding_connection", "optional": true, "reference": + "${inputs.serverless_embedding_connection}", "type": ["string"]}], "reverse_func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.reverse_mapping"}, "input_type": + "default"}, "mlindex_path": {"type": ["string"], "enabled_by": "index_type", + "enabled_by_value": ["MLIndex file from path"], "allow_manual_entry": false, + "is_multi_select": false, "input_type": "uionly_hidden"}, "mongodb_collection": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + CosmosDB for MongoDB vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_collections", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}]}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mongodb_connection": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Azure CosmosDB for MongoDB vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_connections"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mongodb_content_field": {"type": ["string"], "enabled_by": "index_type", + "enabled_by_value": ["Azure CosmosDB for MongoDB vCore"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_index_fields", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}, + {"name": "mongodb_collection", "optional": false, "reference": "${inputs.mongodb_collection}", + "type": ["string"]}, {"default": "string", "name": "field_data_type", "optional": + false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "mongodb_database": {"type": ["string"], + "enabled_by": "index_type", "enabled_by_value": ["Azure CosmosDB for MongoDB + vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_databases", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}]}, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "mongodb_embedding_field": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + CosmosDB for MongoDB vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_index_fields", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}, + {"name": "mongodb_collection", "optional": false, "reference": "${inputs.mongodb_collection}", + "type": ["string"]}, {"default": "ARRAY(float)", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "mongodb_index_name": {"type": ["string"], + "enabled_by": "index_type", "enabled_by_value": ["Azure CosmosDB for MongoDB + vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_indexes", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}, + {"name": "mongodb_collection", "optional": false, "reference": "${inputs.mongodb_collection}", + "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "uionly_hidden"}, "oai_embedding_connection": {"type": ["OpenAIConnection"], + "enabled_by": "embedding_type", "enabled_by_value": ["OpenAI"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "pinecone_content_field": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Pinecone"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "pinecone_index_connection": {"type": ["PineconeConnection"], "enabled_by": + "index_type", "enabled_by_value": ["Pinecone"], "dynamic_list": {"func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.list_pinecone_connections"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "pinecone_index_name": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Pinecone"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_pinecone_indices", + "func_kwargs": [{"name": "pinecone_connection_name", "optional": false, "reference": + "${inputs.pinecone_index_connection}", "type": ["string"]}]}, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "pinecone_metadata_field": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Pinecone"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "queries": {"type": ["object"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "query_type": {"type": ["string"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_available_query_types", + "func_kwargs": [{"name": "mlindex_content", "optional": false, "reference": + "${inputs.mlindex_content}", "type": ["string"]}]}, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "semantic_configuration": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + AI Search"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_semantic_configurations", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "uionly_hidden"}, "serverless_embedding_connection": {"type": + ["string"], "enabled_by": "embedding_type", "enabled_by_value": ["Serverless + Endpoint"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_serverless_embedding_connections"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "top_k": {"type": ["int"], "default": 3, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}}, "description": "Search an AzureML Vector + Index for relevant results using one or more text queries.", "module": "promptflow_vectordb.tool.common_index_lookup", + "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "preview"}, + {"name": "Faiss Index Lookup", "type": "python", "inputs": {"path": {"type": + ["string"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default"}, "top_k": {"type": ["int"], "default": "3", "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, "vector": {"type": ["list"], "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default"}, "vector_field": {"type": ["string"], "enabled_by": "connection", - "enabled_by_type": ["CognitiveSearchConnection"], "allow_manual_entry": false, - "is_multi_select": false, "input_type": "default"}}, "description": "Search - vector based query from existing Vector Database.", "module": "promptflow_vectordb.tool.vector_db_lookup", - "class_name": "VectorDBLookup", "function": "search", "is_builtin": true, - "package": "promptflow-vectordb", "package_version": "0.0.1", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "Vector Index Lookup", "type": "python", - "inputs": {"path": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "query": {"type": ["object"], "allow_manual_entry": + "default"}}, "description": "Search vector based query from the FAISS index + file.", "module": "promptflow_vectordb.tool.faiss_index_lookup", "class_name": + "FaissIndexLookup", "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "archived"}, + {"name": "Vector DB Lookup", "type": "python", "inputs": {"class_name": {"type": + ["string"], "enabled_by": "connection", "enabled_by_type": ["WeaviateConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "collection_name": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default"}, "connection": {"type": ["CognitiveSearchConnection", + "QdrantConnection", "WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "index_name": {"type": ["string"], "enabled_by": + "connection", "enabled_by_type": ["CognitiveSearchConnection"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "search_filters": + {"type": ["object"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection", + "QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default"}, "search_params": {"type": ["object"], "enabled_by": + "connection", "enabled_by_type": ["CognitiveSearchConnection", "QdrantConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "text_field": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["CognitiveSearchConnection", "QdrantConnection", "WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}}, "description": "Search text or vector based query - from AzureML Vector Index.", "module": "promptflow_vectordb.tool.vector_index_lookup", - "class_name": "VectorIndexLookup", "function": "search", "is_builtin": true, - "package": "promptflow-vectordb", "package_version": "0.0.1", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "calculate_accuracy.py", "type": - "python", "inputs": {"grades": {"type": ["object"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}}, "source": "calculate_accuracy.py", - "function": "calculate_accuracy", "is_builtin": false, "enable_kwargs": false, - "tool_state": "stable"}, {"name": "grade.py", "type": "python", "inputs": - {"groundtruth": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "prediction": {"type": ["string"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}}, "source": "grade.py", - "function": "grade", "is_builtin": false, "enable_kwargs": false, "tool_state": - "stable"}], "inputs": {"groundtruth": {"type": "string", "default": "APP", - "description": "Please specify the groundtruth column, which contains the - true label to the outputs that your flow produces.", "is_chat_input": false}, - "prediction": {"type": "string", "default": "APP", "description": "Please - specify the prediction column, which contains the predicted outputs that your - flow produces.", "is_chat_input": false}}, "outputs": {"grade": {"type": "string", - "reference": "${grade.output}", "evaluation_only": false, "is_chat_output": - false}}}, "flowRunResourceId": "azureml://locations/eastus/workspaces/00000/flows/eval_run_name/flowRuns/eval_run_name", + "input_type": "default"}, "vector": {"type": ["list"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "vector_field": + {"type": ["string"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}}, + "description": "Search vector based query from existing Vector Database.", + "module": "promptflow_vectordb.tool.vector_db_lookup", "class_name": "VectorDBLookup", + "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "archived"}, + {"name": "Vector Index Lookup", "type": "python", "inputs": {"path": {"type": + ["string"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default"}, "query": {"type": ["object"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}}, + "description": "Search text or vector based query from AzureML Vector Index.", + "module": "promptflow_vectordb.tool.vector_index_lookup", "class_name": "VectorIndexLookup", + "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "archived"}, + {"name": "calculate_accuracy.py", "type": "python", "inputs": {"grades": {"type": + ["object"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default"}}, "source": "calculate_accuracy.py", "function": "calculate_accuracy", + "is_builtin": false, "enable_kwargs": false, "tool_state": "stable"}, {"name": + "grade.py", "type": "python", "inputs": {"groundtruth": {"type": ["string"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "prediction": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}}, "source": "grade.py", "function": "grade", + "is_builtin": false, "enable_kwargs": false, "tool_state": "stable"}], "inputs": + {"groundtruth": {"type": "string", "default": "APP", "description": "Please + specify the groundtruth column, which contains the true label to the outputs + that your flow produces.", "is_chat_input": false}, "prediction": {"type": + "string", "default": "APP", "description": "Please specify the prediction + column, which contains the predicted outputs that your flow produces.", "is_chat_input": + false}}, "outputs": {"grade": {"type": "string", "reference": "${grade.output}", + "evaluation_only": false, "is_chat_output": false}}}, "flowRunResourceId": + "azureml://locations/eastus/workspaces/00000/flows/eval_run_name/flowRuns/eval_run_name", "flowRunId": "eval_run_name", "flowRunDisplayName": "sdk-cli-test-fixture-eval-run-without-llm", "batchDataInput": {"dataUri": "azureml://datastores/workspaceblobstore/paths/LocalUpload/74c11bba717480b2d6b04b8e746d09d7/webClassification3.jsonl"}, - "flowRunType": "FlowRun", "flowType": "Default", "runtimeName": "test-runtime-ci", + "flowRunType": "FlowRun", "flowType": "Default", "runtimeName": "hod-ci", "inputsMapping": {"groundtruth": "${data.answer}", "prediction": "${run.outputs.result}"}, "outputDatastoreName": "workspaceblobstore", "childRunBasePath": "promptflow/PromptFlowArtifacts/eval_run_name/flow_artifacts", - "flowDagFileRelativePath": "flow.dag.yaml", "flowSnapshotId": "1885cd85-1969-4fca-8c21-d8826ed5d886", - "studioPortalEndpoint": "https://ml.azure.com/runs/eval_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000"}' + "flowDagFileRelativePath": "flow.dag.yaml", "flowSnapshotId": "d1dc26d0-4657-4c78-a208-e48207b3a47c", + "sessionId": "766232492b56bb90a427434c7071a277e4eed302ecb37ddb", "studioPortalEndpoint": + "https://ml.azure.com/runs/eval_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000", + "studioPortalTraceEndpoint": "https://ml.azure.com/prompts/trace/run/eval_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/promptflow-eastus"}' headers: connection: - keep-alive content-length: - - '13872' + - '43399' content-type: - application/json; charset=utf-8 strict-transport-security: - - max-age=15724800; includeSubDomains; preload + - max-age=31536000; includeSubDomains; preload transfer-encoding: - chunked vary: @@ -1855,7 +3165,7 @@ interactions: x-content-type-options: - nosniff x-request-time: - - '0.547' + - '0.220' status: code: 200 message: OK @@ -1865,12 +3175,12 @@ interactions: Accept: - application/json Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive User-Agent: - - promptflow-sdk/0.0.1 azsdk-python-azuremachinelearningdesignerserviceclient/unknown - Python/3.10.13 (Windows-10-10.0.22631-SP0) + - promptflow-azure-sdk/1.9.0.dev0 azsdk-python-azuremachinelearningdesignerserviceclient/unknown + Python/3.9.19 (Windows-10-10.0.22631-SP0) method: GET uri: https://eastus.api.azureml.ms/flow/api/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/BulkRuns/eval_run_name response: @@ -1880,176 +3190,519 @@ interactions: "prediction": "${inputs.prediction}"}, "tool": "grade.py", "reduce": false}, {"name": "calculate_accuracy", "type": "python", "source": {"type": "code", "path": "calculate_accuracy.py"}, "inputs": {"grades": "${grade.output}"}, - "tool": "calculate_accuracy.py", "reduce": true}], "tools": [{"name": "Content - Safety (Text Analyze)", "type": "python", "inputs": {"connection": {"type": - ["AzureContentSafetyConnection"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "hate_category": {"type": ["string"], "default": - "medium_sensitivity", "enum": ["disable", "low_sensitivity", "medium_sensitivity", - "high_sensitivity"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "self_harm_category": {"type": ["string"], "default": - "medium_sensitivity", "enum": ["disable", "low_sensitivity", "medium_sensitivity", - "high_sensitivity"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "sexual_category": {"type": ["string"], "default": - "medium_sensitivity", "enum": ["disable", "low_sensitivity", "medium_sensitivity", + "tool": "calculate_accuracy.py", "reduce": true}], "tools": [{"name": "Azure + OpenAI GPT-4 Turbo with Vision", "type": "custom_llm", "inputs": {"connection": + {"type": ["AzureOpenAIConnection"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 0}}, "deployment_name": + {"type": ["string"], "enabled_by": "connection", "dynamic_list": {"func_path": + "promptflow.tools.aoai_gpt4v.list_deployment_names", "func_kwargs": [{"name": + "connection", "reference": "${inputs.connection}", "type": ["AzureOpenAIConnection"]}]}, + "allow_manual_entry": true, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 1}}, "detail": {"type": ["string"], "default": "auto", + "enum": ["low", "high", "auto"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 9}}, "frequency_penalty": + {"type": ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 7}}, "max_tokens": {"type": + ["int"], "default": 512, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 4}}, "presence_penalty": {"type": + ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 6}}, "seed": {"type": + ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 8}}, "stop": {"type": ["list"], + "default": "", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 5}}, "temperature": {"type": ["double"], + "default": 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 2}}, "top_p": {"type": ["double"], "default": + 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 3}}}, "description": "Use Azure OpenAI GPT-4 Turbo with + Vision to leverage AOAI vision ability.", "module": "promptflow.tools.aoai_gpt4v", + "class_name": "AzureOpenAI", "function": "chat", "icon": {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "default_prompt": "# system:\nAs an AI assistant, your task involves interpreting + images and responding to questions about the image.\nRemember to provide accurate + answers based on the information present in the image.\n\n# user:\nCan you + tell me what the image depicts?\n![image]({{image_input}})\n", "enable_kwargs": + false, "tool_state": "preview"}, {"name": "Content Safety (Text Analyze)", + "type": "python", "inputs": {"connection": {"type": ["AzureContentSafetyConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "hate_category": {"type": ["string"], "default": "medium_sensitivity", "enum": + ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "self_harm_category": {"type": ["string"], "default": "medium_sensitivity", + "enum": ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "sexual_category": {"type": ["string"], "default": "medium_sensitivity", "enum": + ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "text": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "violence_category": {"type": ["string"], + "default": "medium_sensitivity", "enum": ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "text": {"type": ["string"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "violence_category": - {"type": ["string"], "default": "medium_sensitivity", "enum": ["disable", - "low_sensitivity", "medium_sensitivity", "high_sensitivity"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}}, "description": - "Use Azure Content Safety to detect harmful content.", "module": "promptflow.tools.azure_content_safety", - "function": "analyze_text", "is_builtin": true, "package": "promptflow-tools", - "package_version": "0.0.216", "enable_kwargs": false, "deprecated_tools": - ["content_safety_text.tools.content_safety_text_tool.analyze_text"], "tool_state": - "stable"}, {"name": "Embedding", "type": "python", "inputs": {"connection": - {"type": ["AzureOpenAIConnection", "OpenAIConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "deployment_name": - {"type": ["string"], "enabled_by": "connection", "enabled_by_type": ["AzureOpenAIConnection"], - "model_list": ["text-embedding-ada-002", "text-search-ada-doc-001", "text-search-ada-query-001"], - "capabilities": {"completion": false, "chat_completion": false, "embeddings": - true}, "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default"}, "input": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "model": {"type": ["string"], "enum": ["text-embedding-ada-002", - "text-search-ada-doc-001", "text-search-ada-query-001"], "enabled_by": "connection", - "enabled_by_type": ["OpenAIConnection"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Use Open AI''s embedding - model to create an embedding vector representing the input text.", "module": - "promptflow.tools.embedding", "function": "embedding", "is_builtin": true, - "package": "promptflow-tools", "package_version": "0.0.216", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "Open Source LLM", "type": "custom_llm", - "inputs": {"api": {"type": ["string"], "enum": ["chat", "completion"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "connection": {"type": - ["CustomConnection"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "deployment_name": {"type": ["string"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "endpoint_name": - {"type": ["string"], "default": "-- please enter an endpoint name --", "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "max_new_tokens": - {"type": ["int"], "default": 500, "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "model_kwargs": {"type": ["object"], "default": - "{}", "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default", "advanced": true}, "temperature": {"type": ["double"], "default": - 1.0, "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default"}, "top_p": {"type": ["double"], "default": 1.0, "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default", "advanced": true}}, - "description": "Use an Open Source model from the Azure Model catalog, deployed - to an AzureML Online Endpoint for LLM Chat or Completion API calls.", "module": - "promptflow.tools.open_source_llm", "class_name": "OpenSourceLLM", "function": + "input_type": "default"}}, "description": "Use Azure Content Safety to detect + harmful content.", "module": "promptflow.tools.azure_content_safety", "function": + "analyze_text", "is_builtin": true, "package": "promptflow-tools", "package_version": + "1.5.0rc8", "enable_kwargs": false, "deprecated_tools": ["content_safety_text.tools.content_safety_text_tool.analyze_text"], + "tool_state": "stable"}, {"name": "Embedding", "type": "python", "inputs": + {"connection": {"type": ["AzureOpenAIConnection", "OpenAIConnection", "ServerlessConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "deployment_name": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["AzureOpenAIConnection"], "model_list": ["text-embedding-ada-002", "text-search-ada-doc-001", + "text-search-ada-query-001"], "capabilities": {"completion": false, "chat_completion": + false, "embeddings": true}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "input": {"type": ["string"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "model": {"type": + ["string"], "enum": ["text-embedding-ada-002", "text-search-ada-doc-001", + "text-search-ada-query-001"], "enabled_by": "connection", "enabled_by_type": + ["OpenAIConnection"], "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default"}}, "description": "Use Open AI''s embedding model + to create an embedding vector representing the input text.", "module": "promptflow.tools.embedding", + "function": "embedding", "is_builtin": true, "package": "promptflow-tools", + "package_version": "1.5.0rc8", "enable_kwargs": false, "tool_state": "stable"}, + {"name": "LLM", "type": "custom_llm", "inputs": {"api": {"type": ["string"], + "default": "chat", "allow_manual_entry": true, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 1, "text_box_size": "sm"}, "filter_by": {"input_name": + "connection", "values": {"AzureOpenAIConnection": {"enum": ["chat", "completion"]}, + "OpenAIConnection": {"enum": ["chat", "completion"]}, "ServerlessConnection": + {"enum": ["chat"]}}, "filter_attribute": "type"}}, "best_of": {"type": ["int"], + "default": 1, "enabled_by": "api", "enabled_by_value": ["completion"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 16, "text_box_size": "xs"}}, "connection": {"type": + ["AzureOpenAIConnection", "OpenAIConnection", "ServerlessConnection"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 0}}, "deployment_name": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["AzureOpenAIConnection"], "allow_manual_entry": true, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 2, "text_box_size": + "lg"}, "filter_by": {"input_name": "api", "values": {"chat": {"capabilities": + {"chat_completion": true, "exclude": {"model_version": ["vision-preview"]}}}, + "completion": {"capabilities": {"completion": true, "include": {"model_name": + ["gpt-35-turbo-instruct"]}}}}}}, "echo": {"type": ["bool"], "default": false, + "enabled_by": "api", "enabled_by_value": ["completion"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 15, "text_box_size": "xs"}}, "frequency_penalty": {"type": + ["int"], "default": 0, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "advanced": true, "ui_hints": {"index": 10, "text_box_size": + "xs"}}, "logit_bias": {"type": ["object"], "default": "", "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 11, "text_box_size": "lg"}}, "logprobs": {"type": ["int"], + "default": "", "enabled_by": "api", "enabled_by_value": ["completion"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 14, "text_box_size": "xs"}}, "max_tokens": {"type": + ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 7, "text_box_size": "xs"}}, + "model": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["OpenAIConnection"], "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 3, "text_box_size": "lg"}, + "filter_by": {"input_name": "api", "values": {"chat": {"enum": ["gpt-3.5-turbo", + "gpt-3.5-turbo-0301", "gpt-3.5-turbo-16k", "gpt-3.5-turbo-1106", "gpt-4", + "gpt-4-1106-preview", "gpt-4-0314", "gpt-4-32k", "gpt-4-32k-0314"]}, "completion": + {"enum": ["gpt-3.5-turbo-instruct"]}}}}, "presence_penalty": {"type": ["double"], + "default": 0, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "advanced": true, "ui_hints": {"index": 8}}, "response_format": + {"type": ["object"], "default": "", "enum": [{"type": "json_object"}, {"type": + "text"}], "enabled_by": "api", "enabled_by_value": ["chat"], "allow_manual_entry": + true, "is_multi_select": false, "input_type": "default", "advanced": false, + "ui_hints": {"index": 9, "text_box_size": "lg"}}, "seed": {"type": ["int"], + "default": "", "enabled_by": "api", "enabled_by_value": ["chat"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 12, "text_box_size": "xs"}}, "stop": {"type": ["list"], + "default": "", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 6, "text_box_size": "md"}}, "suffix": {"type": + ["string"], "default": "", "enabled_by": "api", "enabled_by_value": ["completion"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "advanced": true, "ui_hints": {"index": 13, "text_box_size": "xs"}}, "temperature": + {"type": ["double"], "default": 1, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 4, "text_box_size": + "xs"}}, "tool_choice": {"type": ["object", "string"], "default": "", "enabled_by": + "api", "enabled_by_value": ["chat"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 18}}, "tools": {"type": + ["list"], "default": [], "enabled_by": "api", "enabled_by_value": ["chat"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"display_style": "reusable_tool", "index": 17}}, "top_p": {"type": + ["double"], "default": 1, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "advanced": true, "ui_hints": {"index": 5, + "text_box_size": "xs"}}}, "description": {"ai_studio": "Use OpenAI-like Large + Language Model for text completion or chat. For more details, refer to this + [document](https://aka.ms/aistudio/pf/llm-tool).", "aml": "Use OpenAI-like + Large Language Model for text completion or chat. For more details, refer + to this [document](https://aka.ms/promptflow/llm-tool).", "default": "Use + OpenAI-like Large Language Model for text completion or chat."}, "module": + "promptflow.tools.llm", "function": "llm", "icon": {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "category": "llm", "is_builtin": true, "package": "promptflow-tools", "package_version": + "1.5.0rc8", "default_prompt": "# system:\nYou are a helpful assistant.\n\n# + user:\n{{question}}\n", "enable_kwargs": false, "tool_state": "stable", "groups": + [{"name": "Tools", "description": "Configure interactive capabilities by selecting + tools for the model to use and guiding its tool call decisions.", "inputs": + ["tools", "tool_choice"], "ui_hints": {"display_style": "table"}}]}, {"name": + "LLM-Vision", "type": "custom_llm", "inputs": {"connection": {"type": ["AzureOpenAIConnection", + "OpenAIConnection"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 0}}, "deployment_name": {"type": + ["string"], "enabled_by": "connection", "enabled_by_type": ["AzureOpenAIConnection"], + "capabilities": {"chat_completion": true, "include": {"model_version": ["vision-preview"]}}, + "allow_manual_entry": true, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 1, "text_box_size": "lg"}}, "detail": {"type": ["string"], + "default": "auto", "enum": ["low", "high", "auto"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 10}}, "frequency_penalty": {"type": ["int"], "default": 0, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 8, "text_box_size": "xs"}}, "max_tokens": {"type": ["int"], "default": "", + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 6, "text_box_size": "xs"}}, "model": {"type": ["string"], + "enum": ["gpt-4-vision-preview"], "enabled_by": "connection", "enabled_by_type": + ["OpenAIConnection"], "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 2, "text_box_size": "lg"}}, + "presence_penalty": {"type": ["double"], "default": 0, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 7}}, "seed": {"type": ["int"], "default": "", "allow_manual_entry": false, + "is_multi_select": false, "input_type": "default", "ui_hints": {"index": 9, + "text_box_size": "xs"}}, "stop": {"type": ["list"], "default": "", "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 5, "text_box_size": "md"}}, "temperature": {"type": ["double"], "default": + 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 3, "text_box_size": "xs"}}, "top_p": {"type": ["double"], + "default": 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 4, "text_box_size": "xs"}}}, "description": + {"ai_studio": "Use OpenAI-like Large Language Model to leverage vision ability. + For more details, refer to this [document](https://aka.ms/aistudio/pf/llm-vision-tool).", + "aml": "Use OpenAI-like Large Language Model to leverage vision ability. For + more details, refer to this [document](https://aka.ms/promptflow/llm-vision-tool).", + "default": "Use OpenAI-like Large Language Model to leverage vision ability."}, + "module": "promptflow.tools.llm_vision", "function": "llm_vision", "icon": + {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "category": "llm", "is_builtin": true, "package": "promptflow-tools", "package_version": + "1.5.0rc8", "default_prompt": "# system:\nAs an AI assistant, your task involves + interpreting images and responding to questions about the image.\nRemember + to provide accurate answers based on the information present in the image.\n\n# + user:\nCan you tell me what the image depicts?\n![image]({{image_input}})\n", + "enable_kwargs": false, "tool_state": "preview"}, {"name": "Open Model LLM", + "type": "custom_llm", "inputs": {"api": {"type": ["string"], "enum": ["chat", + "completion"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 2}}, "deployment_name": {"type": ["string"], + "default": "", "dynamic_list": {"func_path": "promptflow.tools.open_model_llm.list_deployment_names", + "func_kwargs": [{"name": "endpoint", "optional": true, "reference": "${inputs.endpoint}", + "type": ["string"]}]}, "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 1}}, "endpoint_name": {"type": + ["string"], "dynamic_list": {"func_path": "promptflow.tools.open_model_llm.list_endpoint_names"}, + "allow_manual_entry": true, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 0}}, "max_new_tokens": {"type": ["int"], "default": + 500, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 4}}, "model_kwargs": {"type": ["object"], + "default": "{}", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "advanced": true, "ui_hints": {"index": 6}}, "temperature": {"type": + ["double"], "default": 1.0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 3}}, "top_p": {"type": + ["double"], "default": 1.0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "advanced": true, "ui_hints": {"index": 5}}}, + "description": "Use an open model from the Azure Model catalog, deployed to + an AzureML Online Endpoint for LLM Chat or Completion API calls.", "module": + "promptflow.tools.open_model_llm", "class_name": "OpenModelLLM", "function": "call", "icon": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAACgElEQVR4nGWSz2vcVRTFP/e9NzOZ1KDGohASslLEH6VLV0ak4l/QpeDCrfQPcNGliODKnVm4EBdBsIjQIlhciKW0ycKFVCSNbYnjdDLtmPnmO/nO9917XcxMkjYX3uLx7nnn3HOuMK2Nix4fP78ZdrYXVkLVWjf3l3B1B+HpcjzGFtmqa6cePz7/x0dnn1n5qhj3iBJPYREIURAJuCtpY8PjReDbrf9WG7H1fuefwQU9qKztTcMJT+PNnEFvjGVDBDlSsH6p/9MLzy6+NxwVqI8RAg4IPmWedMckdLYP6O6UpIaQfvyyXG012+e79/ZfHukoS1ISMT2hGTB1RkUmNgQ5QZ0w+a2VWDq73MbdEWmfnnv6UWe7oNzPaLapl5CwuLTXK9WUGBuCjqekzhP+z52ZXOrKMD3OJg0Hh778aiOuvpnYvp05d6GJO4iAO4QAe/eV36/X5LFRV4Zmn+AdkqlL8Vjp3oVioOz+WTPzzYEgsN+fgPLYyJVheSbPPVl2ikeGZRjtG52/8rHuaV9VOlpP2OtKyVndcRVCSqOhsvxa4vW359i6OuKdD+aP8Q4SYPdOzS/flGjt1JUSaMqZ5nwa1Y8qWb/Ud/eZZkHisYezEM0m+fcelDr8F1SqW2LNK6r1jXQwyLzy1hxvrLXZulry7ocL+FS6G4QIu3fG/Px1gdYeW7LIgXU2P/115TOA5G7e3Rmj2aS/m7l5pThiZzrCcE/d1XHzbln373nw7y6veeoUm5KCNKT/IPPwbiY1hYd/l5MIT65BMFt87sU4v9D7/JMflr44uV6hGh1+L4RCkg6z5iK2tAhNLeLsNGwYA4fDYnC/drvuuFxe86NV/x+Ut27g0FvykgAAAABJRU5ErkJggg==", - "is_builtin": true, "package": "promptflow-tools", "package_version": "0.0.216", + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", "enable_kwargs": false, "tool_state": "stable"}, {"name": "OpenAI GPT-4V", "type": "custom_llm", "inputs": {"connection": {"type": ["OpenAIConnection"], - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "frequency_penalty": {"type": ["double"], "default": 0, "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "max_tokens": {"type": + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 0}}, "detail": {"type": ["string"], "default": "auto", + "enum": ["low", "high", "auto"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 9}}, "frequency_penalty": + {"type": ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 7}}, "max_tokens": {"type": + ["int"], "default": 512, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 4}}, "model": {"type": ["string"], + "enum": ["gpt-4-vision-preview"], "allow_manual_entry": true, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 1}}, "presence_penalty": + {"type": ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 6}}, "seed": {"type": ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "model": {"type": ["string"], "enum": ["gpt-4-vision-preview"], - "allow_manual_entry": true, "is_multi_select": false, "input_type": "default"}, - "presence_penalty": {"type": ["double"], "default": 0, "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "stop": {"type": - ["list"], "default": "", "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "temperature": {"type": ["double"], "default": 1, - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "top_p": {"type": ["double"], "default": 1, "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Use OpenAI GPT-4V to leverage + "input_type": "default", "ui_hints": {"index": 8}}, "stop": {"type": ["list"], + "default": "", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 5}}, "temperature": {"type": ["double"], + "default": 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 2}}, "top_p": {"type": ["double"], "default": + 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 3}}}, "description": "Use OpenAI GPT-4V to leverage vision ability.", "module": "promptflow.tools.openai_gpt4v", "class_name": - "OpenAI", "function": "chat", "is_builtin": true, "package": "promptflow-tools", - "package_version": "0.0.216", "default_prompt": "# system:\nAs an AI assistant, - your task involves interpreting images and responding to questions about the - image.\nRemember to provide accurate answers based on the information present - in the image.\n\n# user:\nCan you tell me what the image depicts?\n![image]({{image_input}})\n", - "enable_kwargs": false, "tool_state": "stable"}, {"name": "Serp API", "type": - "python", "inputs": {"connection": {"type": ["SerpConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "engine": {"type": - ["string"], "default": "google", "enum": ["google", "bing"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "location": {"type": - ["string"], "default": "", "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "num": {"type": ["int"], "default": "10", - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "query": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "safe": {"type": ["string"], "default": "off", - "enum": ["active", "off"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Use Serp API to obtain search - results from a specific search engine.", "module": "promptflow.tools.serpapi", - "class_name": "SerpAPI", "function": "search", "is_builtin": true, "package": - "promptflow-tools", "package_version": "0.0.216", "enable_kwargs": false, - "tool_state": "stable"}, {"name": "Faiss Index Lookup", "type": "python", - "inputs": {"path": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "vector": {"type": ["list"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Search vector based query - from the FAISS index file.", "module": "promptflow_vectordb.tool.faiss_index_lookup", - "class_name": "FaissIndexLookup", "function": "search", "is_builtin": true, - "package": "promptflow-vectordb", "package_version": "0.0.1", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "Vector DB Lookup", "type": "python", - "inputs": {"class_name": {"type": ["string"], "enabled_by": "connection", - "enabled_by_type": ["WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "collection_name": {"type": ["string"], "enabled_by": - "connection", "enabled_by_type": ["QdrantConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "connection": {"type": - ["CognitiveSearchConnection", "QdrantConnection", "WeaviateConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "index_name": {"type": - ["string"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection"], - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "search_filters": {"type": ["object"], "enabled_by": "connection", "enabled_by_type": - ["CognitiveSearchConnection", "QdrantConnection"], "allow_manual_entry": false, - "is_multi_select": false, "input_type": "default"}, "search_params": {"type": - ["object"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection", - "QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "text_field": {"type": ["string"], "enabled_by": - "connection", "enabled_by_type": ["CognitiveSearchConnection", "QdrantConnection", - "WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", "allow_manual_entry": + "OpenAI", "function": "chat", "icon": {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "default_prompt": "# system:\nAs an AI assistant, your task involves interpreting + images and responding to questions about the image.\nRemember to provide accurate + answers based on the information present in the image.\n\n# user:\nCan you + tell me what the image depicts?\n![image]({{image_input}})\n", "enable_kwargs": + false, "tool_state": "preview"}, {"name": "Serp API", "type": "python", "inputs": + {"connection": {"type": ["SerpConnection"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "engine": {"type": ["string"], "default": + "google", "enum": ["google", "bing"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "location": {"type": ["string"], "default": + "", "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "num": {"type": ["int"], "default": "10", "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "query": {"type": ["string"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "safe": {"type": + ["string"], "default": "off", "enum": ["active", "off"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}}, "description": + "Use Serp API to obtain search results from a specific search engine.", "module": + "promptflow.tools.serpapi", "class_name": "SerpAPI", "function": "search", + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "enable_kwargs": false, "tool_state": "stable"}, {"name": "Index Lookup", + "type": "python", "inputs": {"acs_content_field": {"type": ["string"], "enabled_by": + "index_type", "enabled_by_value": ["Azure AI Search"], "dynamic_list": {"func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_fields", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}, {"default": "Edm.String", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "acs_embedding_field": {"type": ["string"], + "enabled_by": "index_type", "enabled_by_value": ["Azure AI Search"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_fields", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}, {"default": "Collection(Edm.Single)", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "acs_index_connection": {"type": ["CognitiveSearchConnection"], + "enabled_by": "index_type", "enabled_by_value": ["Azure AI Search"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "acs_index_name": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + AI Search"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_indices", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}]}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "acs_metadata_field": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Azure AI Search"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_fields", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}, {"default": "Edm.String", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "aoai_embedding_connection": {"type": + ["AzureOpenAIConnection"], "enabled_by": "embedding_type", "enabled_by_value": + ["Azure OpenAI"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "uionly_hidden"}, "embedding_deployment": {"type": ["string"], "enabled_by": + "embedding_type", "enabled_by_value": ["Azure OpenAI"], "dynamic_list": {"func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.list_aoai_embedding_deployments", + "func_kwargs": [{"name": "aoai_connection", "optional": false, "reference": + "${inputs.aoai_embedding_connection}", "type": ["AzurOpenAIConnection"]}]}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "embedding_model": {"type": ["string"], "enabled_by": "embedding_type", "enabled_by_value": + ["OpenAI", "Hugging Face"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_embedding_models", + "func_kwargs": [{"name": "embedding_type", "optional": false, "reference": + "${inputs.embedding_type}", "type": ["string"]}]}, "allow_manual_entry": false, + "is_multi_select": false, "input_type": "uionly_hidden"}, "embedding_type": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + AI Search", "Azure CosmosDB for MongoDB vCore", "FAISS", "Pinecone"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_available_embedding_types", + "func_kwargs": [{"name": "index_type", "optional": false, "reference": "${inputs.index_type}", + "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "uionly_hidden"}, "faiss_index_path": {"type": ["string"], "enabled_by": + "index_type", "enabled_by_value": ["FAISS"], "allow_manual_entry": false, + "is_multi_select": false, "input_type": "uionly_hidden"}, "index_type": {"type": + ["string"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_available_index_types"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mlindex_asset_id": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Registered Index"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_registered_mlindices"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mlindex_content": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": + false, "generated_by": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.forward_mapping", + "func_kwargs": [{"name": "index_type", "reference": "${inputs.index_type}", + "type": ["string"]}, {"name": "mlindex_asset_id", "optional": true, "reference": + "${inputs.mlindex_asset_id}", "type": ["string"]}, {"name": "mlindex_path", + "optional": true, "reference": "${inputs.mlindex_path}", "type": ["string"]}, + {"name": "acs_index_connection", "optional": true, "reference": "${inputs.acs_index_connection}", + "type": ["CognitiveSearchConnection"]}, {"name": "acs_index_name", "optional": + true, "reference": "${inputs.acs_index_name}", "type": ["string"]}, {"name": + "acs_content_field", "optional": true, "reference": "${inputs.acs_content_field}", + "type": ["string"]}, {"name": "acs_embedding_field", "optional": true, "reference": + "${inputs.acs_embedding_field}", "type": ["string"]}, {"name": "acs_metadata_field", + "optional": true, "reference": "${inputs.acs_metadata_field}", "type": ["string"]}, + {"name": "semantic_configuration", "optional": true, "reference": "${inputs.semantic_configuration}", + "type": ["string"]}, {"name": "faiss_index_path", "optional": true, "reference": + "${inputs.faiss_index_path}", "type": ["string"]}, {"name": "pinecone_index_connection", + "optional": true, "reference": "${inputs.pinecone_index_connection}", "type": + ["string"]}, {"name": "pinecone_index_name", "optional": true, "reference": + "${inputs.pinecone_index_name}", "type": ["string"]}, {"name": "pinecone_content_field", + "optional": true, "reference": "${inputs.pinecone_content_field}", "type": + ["string"]}, {"name": "pinecone_metadata_field", "optional": true, "reference": + "${inputs.pinecone_metadata_field}", "type": ["string"]}, {"name": "mongodb_connection", + "optional": true, "reference": "${inputs.mongodb_connection}", "type": ["string"]}, + {"name": "mongodb_database", "optional": true, "reference": "${inputs.mongodb_database}", + "type": ["string"]}, {"name": "mongodb_collection", "optional": true, "reference": + "${inputs.mongodb_collection}", "type": ["string"]}, {"name": "mongodb_index_name", + "optional": true, "reference": "${inputs.mongodb_index_name}", "type": ["string"]}, + {"name": "mongodb_content_field", "optional": true, "reference": "${inputs.mongodb_content_field}", + "type": ["string"]}, {"name": "mongodb_embedding_field", "optional": true, + "reference": "${inputs.mongodb_embedding_field}", "type": ["string"]}, {"name": + "embedding_type", "optional": true, "reference": "${inputs.embedding_type}", + "type": ["string"]}, {"name": "aoai_embedding_connection", "optional": true, + "reference": "${inputs.aoai_embedding_connection}", "type": ["AzureOpenAIConnection"]}, + {"name": "oai_embedding_connection", "optional": true, "reference": "${inputs.oai_embedding_connection}", + "type": ["string"]}, {"name": "embedding_model", "optional": true, "reference": + "${inputs.embedding_model}", "type": ["string"]}, {"name": "embedding_deployment", + "optional": true, "reference": "${inputs.embedding_deployment}", "type": ["string"]}, + {"name": "serverless_embedding_connection", "optional": true, "reference": + "${inputs.serverless_embedding_connection}", "type": ["string"]}], "reverse_func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.reverse_mapping"}, "input_type": + "default"}, "mlindex_path": {"type": ["string"], "enabled_by": "index_type", + "enabled_by_value": ["MLIndex file from path"], "allow_manual_entry": false, + "is_multi_select": false, "input_type": "uionly_hidden"}, "mongodb_collection": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + CosmosDB for MongoDB vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_collections", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}]}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mongodb_connection": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Azure CosmosDB for MongoDB vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_connections"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mongodb_content_field": {"type": ["string"], "enabled_by": "index_type", + "enabled_by_value": ["Azure CosmosDB for MongoDB vCore"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_index_fields", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}, + {"name": "mongodb_collection", "optional": false, "reference": "${inputs.mongodb_collection}", + "type": ["string"]}, {"default": "string", "name": "field_data_type", "optional": + false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "mongodb_database": {"type": ["string"], + "enabled_by": "index_type", "enabled_by_value": ["Azure CosmosDB for MongoDB + vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_databases", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}]}, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "mongodb_embedding_field": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + CosmosDB for MongoDB vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_index_fields", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}, + {"name": "mongodb_collection", "optional": false, "reference": "${inputs.mongodb_collection}", + "type": ["string"]}, {"default": "ARRAY(float)", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "mongodb_index_name": {"type": ["string"], + "enabled_by": "index_type", "enabled_by_value": ["Azure CosmosDB for MongoDB + vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_indexes", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}, + {"name": "mongodb_collection", "optional": false, "reference": "${inputs.mongodb_collection}", + "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "uionly_hidden"}, "oai_embedding_connection": {"type": ["OpenAIConnection"], + "enabled_by": "embedding_type", "enabled_by_value": ["OpenAI"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "pinecone_content_field": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Pinecone"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "pinecone_index_connection": {"type": ["PineconeConnection"], "enabled_by": + "index_type", "enabled_by_value": ["Pinecone"], "dynamic_list": {"func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.list_pinecone_connections"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "pinecone_index_name": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Pinecone"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_pinecone_indices", + "func_kwargs": [{"name": "pinecone_connection_name", "optional": false, "reference": + "${inputs.pinecone_index_connection}", "type": ["string"]}]}, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "pinecone_metadata_field": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Pinecone"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "queries": {"type": ["object"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "query_type": {"type": ["string"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_available_query_types", + "func_kwargs": [{"name": "mlindex_content", "optional": false, "reference": + "${inputs.mlindex_content}", "type": ["string"]}]}, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "semantic_configuration": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + AI Search"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_semantic_configurations", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "uionly_hidden"}, "serverless_embedding_connection": {"type": + ["string"], "enabled_by": "embedding_type", "enabled_by_value": ["Serverless + Endpoint"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_serverless_embedding_connections"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "top_k": {"type": ["int"], "default": 3, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}}, "description": "Search an AzureML Vector + Index for relevant results using one or more text queries.", "module": "promptflow_vectordb.tool.common_index_lookup", + "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "preview"}, + {"name": "Faiss Index Lookup", "type": "python", "inputs": {"path": {"type": + ["string"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default"}, "top_k": {"type": ["int"], "default": "3", "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, "vector": {"type": ["list"], "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default"}, "vector_field": {"type": ["string"], "enabled_by": "connection", - "enabled_by_type": ["CognitiveSearchConnection"], "allow_manual_entry": false, - "is_multi_select": false, "input_type": "default"}}, "description": "Search - vector based query from existing Vector Database.", "module": "promptflow_vectordb.tool.vector_db_lookup", - "class_name": "VectorDBLookup", "function": "search", "is_builtin": true, - "package": "promptflow-vectordb", "package_version": "0.0.1", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "Vector Index Lookup", "type": "python", - "inputs": {"path": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "query": {"type": ["object"], "allow_manual_entry": + "default"}}, "description": "Search vector based query from the FAISS index + file.", "module": "promptflow_vectordb.tool.faiss_index_lookup", "class_name": + "FaissIndexLookup", "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "archived"}, + {"name": "Vector DB Lookup", "type": "python", "inputs": {"class_name": {"type": + ["string"], "enabled_by": "connection", "enabled_by_type": ["WeaviateConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "collection_name": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default"}, "connection": {"type": ["CognitiveSearchConnection", + "QdrantConnection", "WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "index_name": {"type": ["string"], "enabled_by": + "connection", "enabled_by_type": ["CognitiveSearchConnection"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "search_filters": + {"type": ["object"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection", + "QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default"}, "search_params": {"type": ["object"], "enabled_by": + "connection", "enabled_by_type": ["CognitiveSearchConnection", "QdrantConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "text_field": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["CognitiveSearchConnection", "QdrantConnection", "WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}}, "description": "Search text or vector based query - from AzureML Vector Index.", "module": "promptflow_vectordb.tool.vector_index_lookup", - "class_name": "VectorIndexLookup", "function": "search", "is_builtin": true, - "package": "promptflow-vectordb", "package_version": "0.0.1", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "calculate_accuracy.py", "type": - "python", "inputs": {"grades": {"type": ["object"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}}, "source": "calculate_accuracy.py", - "function": "calculate_accuracy", "is_builtin": false, "enable_kwargs": false, - "tool_state": "stable"}, {"name": "grade.py", "type": "python", "inputs": - {"groundtruth": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "prediction": {"type": ["string"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}}, "source": "grade.py", - "function": "grade", "is_builtin": false, "enable_kwargs": false, "tool_state": - "stable"}], "inputs": {"groundtruth": {"type": "string", "default": "APP", - "description": "Please specify the groundtruth column, which contains the - true label to the outputs that your flow produces.", "is_chat_input": false}, - "prediction": {"type": "string", "default": "APP", "description": "Please - specify the prediction column, which contains the predicted outputs that your - flow produces.", "is_chat_input": false}}, "outputs": {"grade": {"type": "string", - "reference": "${grade.output}", "evaluation_only": false, "is_chat_output": - false}}}, "flowRunResourceId": "azureml://locations/eastus/workspaces/00000/flows/eval_run_name/flowRuns/eval_run_name", + "input_type": "default"}, "vector": {"type": ["list"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "vector_field": + {"type": ["string"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}}, + "description": "Search vector based query from existing Vector Database.", + "module": "promptflow_vectordb.tool.vector_db_lookup", "class_name": "VectorDBLookup", + "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "archived"}, + {"name": "Vector Index Lookup", "type": "python", "inputs": {"path": {"type": + ["string"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default"}, "query": {"type": ["object"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}}, + "description": "Search text or vector based query from AzureML Vector Index.", + "module": "promptflow_vectordb.tool.vector_index_lookup", "class_name": "VectorIndexLookup", + "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "archived"}, + {"name": "calculate_accuracy.py", "type": "python", "inputs": {"grades": {"type": + ["object"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default"}}, "source": "calculate_accuracy.py", "function": "calculate_accuracy", + "is_builtin": false, "enable_kwargs": false, "tool_state": "stable"}, {"name": + "grade.py", "type": "python", "inputs": {"groundtruth": {"type": ["string"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "prediction": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}}, "source": "grade.py", "function": "grade", + "is_builtin": false, "enable_kwargs": false, "tool_state": "stable"}], "inputs": + {"groundtruth": {"type": "string", "default": "APP", "description": "Please + specify the groundtruth column, which contains the true label to the outputs + that your flow produces.", "is_chat_input": false}, "prediction": {"type": + "string", "default": "APP", "description": "Please specify the prediction + column, which contains the predicted outputs that your flow produces.", "is_chat_input": + false}}, "outputs": {"grade": {"type": "string", "reference": "${grade.output}", + "evaluation_only": false, "is_chat_output": false}}}, "flowRunResourceId": + "azureml://locations/eastus/workspaces/00000/flows/eval_run_name/flowRuns/eval_run_name", "flowRunId": "eval_run_name", "flowRunDisplayName": "sdk-cli-test-fixture-eval-run-without-llm", "batchDataInput": {"dataUri": "azureml://datastores/workspaceblobstore/paths/LocalUpload/74c11bba717480b2d6b04b8e746d09d7/webClassification3.jsonl"}, - "flowRunType": "FlowRun", "flowType": "Default", "runtimeName": "test-runtime-ci", + "flowRunType": "FlowRun", "flowType": "Default", "runtimeName": "hod-ci", "inputsMapping": {"groundtruth": "${data.answer}", "prediction": "${run.outputs.result}"}, "outputDatastoreName": "workspaceblobstore", "childRunBasePath": "promptflow/PromptFlowArtifacts/eval_run_name/flow_artifacts", - "flowDagFileRelativePath": "flow.dag.yaml", "flowSnapshotId": "1885cd85-1969-4fca-8c21-d8826ed5d886", - "studioPortalEndpoint": "https://ml.azure.com/runs/eval_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000"}' + "flowDagFileRelativePath": "flow.dag.yaml", "flowSnapshotId": "d1dc26d0-4657-4c78-a208-e48207b3a47c", + "sessionId": "766232492b56bb90a427434c7071a277e4eed302ecb37ddb", "studioPortalEndpoint": + "https://ml.azure.com/runs/eval_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000", + "studioPortalTraceEndpoint": "https://ml.azure.com/prompts/trace/run/eval_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/promptflow-eastus"}' headers: connection: - keep-alive content-length: - - '13872' + - '43399' content-type: - application/json; charset=utf-8 strict-transport-security: - - max-age=15724800; includeSubDomains; preload + - max-age=31536000; includeSubDomains; preload transfer-encoding: - chunked vary: @@ -2057,7 +3710,7 @@ interactions: x-content-type-options: - nosniff x-request-time: - - '0.526' + - '0.412' status: code: 200 message: OK @@ -2067,12 +3720,12 @@ interactions: Accept: - application/json Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive User-Agent: - - promptflow-sdk/0.0.1 azsdk-python-azuremachinelearningdesignerserviceclient/unknown - Python/3.10.13 (Windows-10-10.0.22631-SP0) + - promptflow-azure-sdk/1.9.0.dev0 azsdk-python-azuremachinelearningdesignerserviceclient/unknown + Python/3.9.19 (Windows-10-10.0.22631-SP0) method: GET uri: https://eastus.api.azureml.ms/flow/api/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/BulkRuns/eval_run_name response: @@ -2082,176 +3735,519 @@ interactions: "prediction": "${inputs.prediction}"}, "tool": "grade.py", "reduce": false}, {"name": "calculate_accuracy", "type": "python", "source": {"type": "code", "path": "calculate_accuracy.py"}, "inputs": {"grades": "${grade.output}"}, - "tool": "calculate_accuracy.py", "reduce": true}], "tools": [{"name": "Content - Safety (Text Analyze)", "type": "python", "inputs": {"connection": {"type": - ["AzureContentSafetyConnection"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "hate_category": {"type": ["string"], "default": - "medium_sensitivity", "enum": ["disable", "low_sensitivity", "medium_sensitivity", - "high_sensitivity"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "self_harm_category": {"type": ["string"], "default": - "medium_sensitivity", "enum": ["disable", "low_sensitivity", "medium_sensitivity", - "high_sensitivity"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "sexual_category": {"type": ["string"], "default": - "medium_sensitivity", "enum": ["disable", "low_sensitivity", "medium_sensitivity", + "tool": "calculate_accuracy.py", "reduce": true}], "tools": [{"name": "Azure + OpenAI GPT-4 Turbo with Vision", "type": "custom_llm", "inputs": {"connection": + {"type": ["AzureOpenAIConnection"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 0}}, "deployment_name": + {"type": ["string"], "enabled_by": "connection", "dynamic_list": {"func_path": + "promptflow.tools.aoai_gpt4v.list_deployment_names", "func_kwargs": [{"name": + "connection", "reference": "${inputs.connection}", "type": ["AzureOpenAIConnection"]}]}, + "allow_manual_entry": true, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 1}}, "detail": {"type": ["string"], "default": "auto", + "enum": ["low", "high", "auto"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 9}}, "frequency_penalty": + {"type": ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 7}}, "max_tokens": {"type": + ["int"], "default": 512, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 4}}, "presence_penalty": {"type": + ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 6}}, "seed": {"type": + ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 8}}, "stop": {"type": ["list"], + "default": "", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 5}}, "temperature": {"type": ["double"], + "default": 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 2}}, "top_p": {"type": ["double"], "default": + 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 3}}}, "description": "Use Azure OpenAI GPT-4 Turbo with + Vision to leverage AOAI vision ability.", "module": "promptflow.tools.aoai_gpt4v", + "class_name": "AzureOpenAI", "function": "chat", "icon": {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "default_prompt": "# system:\nAs an AI assistant, your task involves interpreting + images and responding to questions about the image.\nRemember to provide accurate + answers based on the information present in the image.\n\n# user:\nCan you + tell me what the image depicts?\n![image]({{image_input}})\n", "enable_kwargs": + false, "tool_state": "preview"}, {"name": "Content Safety (Text Analyze)", + "type": "python", "inputs": {"connection": {"type": ["AzureContentSafetyConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "hate_category": {"type": ["string"], "default": "medium_sensitivity", "enum": + ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "self_harm_category": {"type": ["string"], "default": "medium_sensitivity", + "enum": ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "sexual_category": {"type": ["string"], "default": "medium_sensitivity", "enum": + ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "text": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "violence_category": {"type": ["string"], + "default": "medium_sensitivity", "enum": ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "text": {"type": ["string"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "violence_category": - {"type": ["string"], "default": "medium_sensitivity", "enum": ["disable", - "low_sensitivity", "medium_sensitivity", "high_sensitivity"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}}, "description": - "Use Azure Content Safety to detect harmful content.", "module": "promptflow.tools.azure_content_safety", - "function": "analyze_text", "is_builtin": true, "package": "promptflow-tools", - "package_version": "0.0.216", "enable_kwargs": false, "deprecated_tools": - ["content_safety_text.tools.content_safety_text_tool.analyze_text"], "tool_state": - "stable"}, {"name": "Embedding", "type": "python", "inputs": {"connection": - {"type": ["AzureOpenAIConnection", "OpenAIConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "deployment_name": - {"type": ["string"], "enabled_by": "connection", "enabled_by_type": ["AzureOpenAIConnection"], - "model_list": ["text-embedding-ada-002", "text-search-ada-doc-001", "text-search-ada-query-001"], - "capabilities": {"completion": false, "chat_completion": false, "embeddings": - true}, "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default"}, "input": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "model": {"type": ["string"], "enum": ["text-embedding-ada-002", - "text-search-ada-doc-001", "text-search-ada-query-001"], "enabled_by": "connection", - "enabled_by_type": ["OpenAIConnection"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Use Open AI''s embedding - model to create an embedding vector representing the input text.", "module": - "promptflow.tools.embedding", "function": "embedding", "is_builtin": true, - "package": "promptflow-tools", "package_version": "0.0.216", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "Open Source LLM", "type": "custom_llm", - "inputs": {"api": {"type": ["string"], "enum": ["chat", "completion"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "connection": {"type": - ["CustomConnection"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "deployment_name": {"type": ["string"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "endpoint_name": - {"type": ["string"], "default": "-- please enter an endpoint name --", "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "max_new_tokens": - {"type": ["int"], "default": 500, "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "model_kwargs": {"type": ["object"], "default": - "{}", "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default", "advanced": true}, "temperature": {"type": ["double"], "default": - 1.0, "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default"}, "top_p": {"type": ["double"], "default": 1.0, "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default", "advanced": true}}, - "description": "Use an Open Source model from the Azure Model catalog, deployed - to an AzureML Online Endpoint for LLM Chat or Completion API calls.", "module": - "promptflow.tools.open_source_llm", "class_name": "OpenSourceLLM", "function": + "input_type": "default"}}, "description": "Use Azure Content Safety to detect + harmful content.", "module": "promptflow.tools.azure_content_safety", "function": + "analyze_text", "is_builtin": true, "package": "promptflow-tools", "package_version": + "1.5.0rc8", "enable_kwargs": false, "deprecated_tools": ["content_safety_text.tools.content_safety_text_tool.analyze_text"], + "tool_state": "stable"}, {"name": "Embedding", "type": "python", "inputs": + {"connection": {"type": ["AzureOpenAIConnection", "OpenAIConnection", "ServerlessConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "deployment_name": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["AzureOpenAIConnection"], "model_list": ["text-embedding-ada-002", "text-search-ada-doc-001", + "text-search-ada-query-001"], "capabilities": {"completion": false, "chat_completion": + false, "embeddings": true}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "input": {"type": ["string"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "model": {"type": + ["string"], "enum": ["text-embedding-ada-002", "text-search-ada-doc-001", + "text-search-ada-query-001"], "enabled_by": "connection", "enabled_by_type": + ["OpenAIConnection"], "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default"}}, "description": "Use Open AI''s embedding model + to create an embedding vector representing the input text.", "module": "promptflow.tools.embedding", + "function": "embedding", "is_builtin": true, "package": "promptflow-tools", + "package_version": "1.5.0rc8", "enable_kwargs": false, "tool_state": "stable"}, + {"name": "LLM", "type": "custom_llm", "inputs": {"api": {"type": ["string"], + "default": "chat", "allow_manual_entry": true, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 1, "text_box_size": "sm"}, "filter_by": {"input_name": + "connection", "values": {"AzureOpenAIConnection": {"enum": ["chat", "completion"]}, + "OpenAIConnection": {"enum": ["chat", "completion"]}, "ServerlessConnection": + {"enum": ["chat"]}}, "filter_attribute": "type"}}, "best_of": {"type": ["int"], + "default": 1, "enabled_by": "api", "enabled_by_value": ["completion"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 16, "text_box_size": "xs"}}, "connection": {"type": + ["AzureOpenAIConnection", "OpenAIConnection", "ServerlessConnection"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 0}}, "deployment_name": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["AzureOpenAIConnection"], "allow_manual_entry": true, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 2, "text_box_size": + "lg"}, "filter_by": {"input_name": "api", "values": {"chat": {"capabilities": + {"chat_completion": true, "exclude": {"model_version": ["vision-preview"]}}}, + "completion": {"capabilities": {"completion": true, "include": {"model_name": + ["gpt-35-turbo-instruct"]}}}}}}, "echo": {"type": ["bool"], "default": false, + "enabled_by": "api", "enabled_by_value": ["completion"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 15, "text_box_size": "xs"}}, "frequency_penalty": {"type": + ["int"], "default": 0, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "advanced": true, "ui_hints": {"index": 10, "text_box_size": + "xs"}}, "logit_bias": {"type": ["object"], "default": "", "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 11, "text_box_size": "lg"}}, "logprobs": {"type": ["int"], + "default": "", "enabled_by": "api", "enabled_by_value": ["completion"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 14, "text_box_size": "xs"}}, "max_tokens": {"type": + ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 7, "text_box_size": "xs"}}, + "model": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["OpenAIConnection"], "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 3, "text_box_size": "lg"}, + "filter_by": {"input_name": "api", "values": {"chat": {"enum": ["gpt-3.5-turbo", + "gpt-3.5-turbo-0301", "gpt-3.5-turbo-16k", "gpt-3.5-turbo-1106", "gpt-4", + "gpt-4-1106-preview", "gpt-4-0314", "gpt-4-32k", "gpt-4-32k-0314"]}, "completion": + {"enum": ["gpt-3.5-turbo-instruct"]}}}}, "presence_penalty": {"type": ["double"], + "default": 0, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "advanced": true, "ui_hints": {"index": 8}}, "response_format": + {"type": ["object"], "default": "", "enum": [{"type": "json_object"}, {"type": + "text"}], "enabled_by": "api", "enabled_by_value": ["chat"], "allow_manual_entry": + true, "is_multi_select": false, "input_type": "default", "advanced": false, + "ui_hints": {"index": 9, "text_box_size": "lg"}}, "seed": {"type": ["int"], + "default": "", "enabled_by": "api", "enabled_by_value": ["chat"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 12, "text_box_size": "xs"}}, "stop": {"type": ["list"], + "default": "", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 6, "text_box_size": "md"}}, "suffix": {"type": + ["string"], "default": "", "enabled_by": "api", "enabled_by_value": ["completion"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "advanced": true, "ui_hints": {"index": 13, "text_box_size": "xs"}}, "temperature": + {"type": ["double"], "default": 1, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 4, "text_box_size": + "xs"}}, "tool_choice": {"type": ["object", "string"], "default": "", "enabled_by": + "api", "enabled_by_value": ["chat"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 18}}, "tools": {"type": + ["list"], "default": [], "enabled_by": "api", "enabled_by_value": ["chat"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"display_style": "reusable_tool", "index": 17}}, "top_p": {"type": + ["double"], "default": 1, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "advanced": true, "ui_hints": {"index": 5, + "text_box_size": "xs"}}}, "description": {"ai_studio": "Use OpenAI-like Large + Language Model for text completion or chat. For more details, refer to this + [document](https://aka.ms/aistudio/pf/llm-tool).", "aml": "Use OpenAI-like + Large Language Model for text completion or chat. For more details, refer + to this [document](https://aka.ms/promptflow/llm-tool).", "default": "Use + OpenAI-like Large Language Model for text completion or chat."}, "module": + "promptflow.tools.llm", "function": "llm", "icon": {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "category": "llm", "is_builtin": true, "package": "promptflow-tools", "package_version": + "1.5.0rc8", "default_prompt": "# system:\nYou are a helpful assistant.\n\n# + user:\n{{question}}\n", "enable_kwargs": false, "tool_state": "stable", "groups": + [{"name": "Tools", "description": "Configure interactive capabilities by selecting + tools for the model to use and guiding its tool call decisions.", "inputs": + ["tools", "tool_choice"], "ui_hints": {"display_style": "table"}}]}, {"name": + "LLM-Vision", "type": "custom_llm", "inputs": {"connection": {"type": ["AzureOpenAIConnection", + "OpenAIConnection"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 0}}, "deployment_name": {"type": + ["string"], "enabled_by": "connection", "enabled_by_type": ["AzureOpenAIConnection"], + "capabilities": {"chat_completion": true, "include": {"model_version": ["vision-preview"]}}, + "allow_manual_entry": true, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 1, "text_box_size": "lg"}}, "detail": {"type": ["string"], + "default": "auto", "enum": ["low", "high", "auto"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 10}}, "frequency_penalty": {"type": ["int"], "default": 0, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 8, "text_box_size": "xs"}}, "max_tokens": {"type": ["int"], "default": "", + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 6, "text_box_size": "xs"}}, "model": {"type": ["string"], + "enum": ["gpt-4-vision-preview"], "enabled_by": "connection", "enabled_by_type": + ["OpenAIConnection"], "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 2, "text_box_size": "lg"}}, + "presence_penalty": {"type": ["double"], "default": 0, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 7}}, "seed": {"type": ["int"], "default": "", "allow_manual_entry": false, + "is_multi_select": false, "input_type": "default", "ui_hints": {"index": 9, + "text_box_size": "xs"}}, "stop": {"type": ["list"], "default": "", "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 5, "text_box_size": "md"}}, "temperature": {"type": ["double"], "default": + 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 3, "text_box_size": "xs"}}, "top_p": {"type": ["double"], + "default": 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 4, "text_box_size": "xs"}}}, "description": + {"ai_studio": "Use OpenAI-like Large Language Model to leverage vision ability. + For more details, refer to this [document](https://aka.ms/aistudio/pf/llm-vision-tool).", + "aml": "Use OpenAI-like Large Language Model to leverage vision ability. For + more details, refer to this [document](https://aka.ms/promptflow/llm-vision-tool).", + "default": "Use OpenAI-like Large Language Model to leverage vision ability."}, + "module": "promptflow.tools.llm_vision", "function": "llm_vision", "icon": + {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "category": "llm", "is_builtin": true, "package": "promptflow-tools", "package_version": + "1.5.0rc8", "default_prompt": "# system:\nAs an AI assistant, your task involves + interpreting images and responding to questions about the image.\nRemember + to provide accurate answers based on the information present in the image.\n\n# + user:\nCan you tell me what the image depicts?\n![image]({{image_input}})\n", + "enable_kwargs": false, "tool_state": "preview"}, {"name": "Open Model LLM", + "type": "custom_llm", "inputs": {"api": {"type": ["string"], "enum": ["chat", + "completion"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 2}}, "deployment_name": {"type": ["string"], + "default": "", "dynamic_list": {"func_path": "promptflow.tools.open_model_llm.list_deployment_names", + "func_kwargs": [{"name": "endpoint", "optional": true, "reference": "${inputs.endpoint}", + "type": ["string"]}]}, "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 1}}, "endpoint_name": {"type": + ["string"], "dynamic_list": {"func_path": "promptflow.tools.open_model_llm.list_endpoint_names"}, + "allow_manual_entry": true, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 0}}, "max_new_tokens": {"type": ["int"], "default": + 500, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 4}}, "model_kwargs": {"type": ["object"], + "default": "{}", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "advanced": true, "ui_hints": {"index": 6}}, "temperature": {"type": + ["double"], "default": 1.0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 3}}, "top_p": {"type": + ["double"], "default": 1.0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "advanced": true, "ui_hints": {"index": 5}}}, + "description": "Use an open model from the Azure Model catalog, deployed to + an AzureML Online Endpoint for LLM Chat or Completion API calls.", "module": + "promptflow.tools.open_model_llm", "class_name": "OpenModelLLM", "function": "call", "icon": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAACgElEQVR4nGWSz2vcVRTFP/e9NzOZ1KDGohASslLEH6VLV0ak4l/QpeDCrfQPcNGliODKnVm4EBdBsIjQIlhciKW0ycKFVCSNbYnjdDLtmPnmO/nO9917XcxMkjYX3uLx7nnn3HOuMK2Nix4fP78ZdrYXVkLVWjf3l3B1B+HpcjzGFtmqa6cePz7/x0dnn1n5qhj3iBJPYREIURAJuCtpY8PjReDbrf9WG7H1fuefwQU9qKztTcMJT+PNnEFvjGVDBDlSsH6p/9MLzy6+NxwVqI8RAg4IPmWedMckdLYP6O6UpIaQfvyyXG012+e79/ZfHukoS1ISMT2hGTB1RkUmNgQ5QZ0w+a2VWDq73MbdEWmfnnv6UWe7oNzPaLapl5CwuLTXK9WUGBuCjqekzhP+z52ZXOrKMD3OJg0Hh778aiOuvpnYvp05d6GJO4iAO4QAe/eV36/X5LFRV4Zmn+AdkqlL8Vjp3oVioOz+WTPzzYEgsN+fgPLYyJVheSbPPVl2ikeGZRjtG52/8rHuaV9VOlpP2OtKyVndcRVCSqOhsvxa4vW359i6OuKdD+aP8Q4SYPdOzS/flGjt1JUSaMqZ5nwa1Y8qWb/Ud/eZZkHisYezEM0m+fcelDr8F1SqW2LNK6r1jXQwyLzy1hxvrLXZulry7ocL+FS6G4QIu3fG/Px1gdYeW7LIgXU2P/115TOA5G7e3Rmj2aS/m7l5pThiZzrCcE/d1XHzbln373nw7y6veeoUm5KCNKT/IPPwbiY1hYd/l5MIT65BMFt87sU4v9D7/JMflr44uV6hGh1+L4RCkg6z5iK2tAhNLeLsNGwYA4fDYnC/drvuuFxe86NV/x+Ut27g0FvykgAAAABJRU5ErkJggg==", - "is_builtin": true, "package": "promptflow-tools", "package_version": "0.0.216", + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", "enable_kwargs": false, "tool_state": "stable"}, {"name": "OpenAI GPT-4V", "type": "custom_llm", "inputs": {"connection": {"type": ["OpenAIConnection"], - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "frequency_penalty": {"type": ["double"], "default": 0, "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "max_tokens": {"type": + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 0}}, "detail": {"type": ["string"], "default": "auto", + "enum": ["low", "high", "auto"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 9}}, "frequency_penalty": + {"type": ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 7}}, "max_tokens": {"type": + ["int"], "default": 512, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 4}}, "model": {"type": ["string"], + "enum": ["gpt-4-vision-preview"], "allow_manual_entry": true, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 1}}, "presence_penalty": + {"type": ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 6}}, "seed": {"type": ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "model": {"type": ["string"], "enum": ["gpt-4-vision-preview"], - "allow_manual_entry": true, "is_multi_select": false, "input_type": "default"}, - "presence_penalty": {"type": ["double"], "default": 0, "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "stop": {"type": - ["list"], "default": "", "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "temperature": {"type": ["double"], "default": 1, - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "top_p": {"type": ["double"], "default": 1, "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Use OpenAI GPT-4V to leverage + "input_type": "default", "ui_hints": {"index": 8}}, "stop": {"type": ["list"], + "default": "", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 5}}, "temperature": {"type": ["double"], + "default": 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 2}}, "top_p": {"type": ["double"], "default": + 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 3}}}, "description": "Use OpenAI GPT-4V to leverage vision ability.", "module": "promptflow.tools.openai_gpt4v", "class_name": - "OpenAI", "function": "chat", "is_builtin": true, "package": "promptflow-tools", - "package_version": "0.0.216", "default_prompt": "# system:\nAs an AI assistant, - your task involves interpreting images and responding to questions about the - image.\nRemember to provide accurate answers based on the information present - in the image.\n\n# user:\nCan you tell me what the image depicts?\n![image]({{image_input}})\n", - "enable_kwargs": false, "tool_state": "stable"}, {"name": "Serp API", "type": - "python", "inputs": {"connection": {"type": ["SerpConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "engine": {"type": - ["string"], "default": "google", "enum": ["google", "bing"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "location": {"type": - ["string"], "default": "", "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "num": {"type": ["int"], "default": "10", - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "query": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "safe": {"type": ["string"], "default": "off", - "enum": ["active", "off"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Use Serp API to obtain search - results from a specific search engine.", "module": "promptflow.tools.serpapi", - "class_name": "SerpAPI", "function": "search", "is_builtin": true, "package": - "promptflow-tools", "package_version": "0.0.216", "enable_kwargs": false, - "tool_state": "stable"}, {"name": "Faiss Index Lookup", "type": "python", - "inputs": {"path": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "vector": {"type": ["list"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Search vector based query - from the FAISS index file.", "module": "promptflow_vectordb.tool.faiss_index_lookup", - "class_name": "FaissIndexLookup", "function": "search", "is_builtin": true, - "package": "promptflow-vectordb", "package_version": "0.0.1", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "Vector DB Lookup", "type": "python", - "inputs": {"class_name": {"type": ["string"], "enabled_by": "connection", - "enabled_by_type": ["WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "collection_name": {"type": ["string"], "enabled_by": - "connection", "enabled_by_type": ["QdrantConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "connection": {"type": - ["CognitiveSearchConnection", "QdrantConnection", "WeaviateConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "index_name": {"type": - ["string"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection"], - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "search_filters": {"type": ["object"], "enabled_by": "connection", "enabled_by_type": - ["CognitiveSearchConnection", "QdrantConnection"], "allow_manual_entry": false, - "is_multi_select": false, "input_type": "default"}, "search_params": {"type": - ["object"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection", - "QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "text_field": {"type": ["string"], "enabled_by": - "connection", "enabled_by_type": ["CognitiveSearchConnection", "QdrantConnection", - "WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", "allow_manual_entry": + "OpenAI", "function": "chat", "icon": {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "default_prompt": "# system:\nAs an AI assistant, your task involves interpreting + images and responding to questions about the image.\nRemember to provide accurate + answers based on the information present in the image.\n\n# user:\nCan you + tell me what the image depicts?\n![image]({{image_input}})\n", "enable_kwargs": + false, "tool_state": "preview"}, {"name": "Serp API", "type": "python", "inputs": + {"connection": {"type": ["SerpConnection"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "engine": {"type": ["string"], "default": + "google", "enum": ["google", "bing"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "location": {"type": ["string"], "default": + "", "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "num": {"type": ["int"], "default": "10", "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "query": {"type": ["string"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "safe": {"type": + ["string"], "default": "off", "enum": ["active", "off"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}}, "description": + "Use Serp API to obtain search results from a specific search engine.", "module": + "promptflow.tools.serpapi", "class_name": "SerpAPI", "function": "search", + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "enable_kwargs": false, "tool_state": "stable"}, {"name": "Index Lookup", + "type": "python", "inputs": {"acs_content_field": {"type": ["string"], "enabled_by": + "index_type", "enabled_by_value": ["Azure AI Search"], "dynamic_list": {"func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_fields", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}, {"default": "Edm.String", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "acs_embedding_field": {"type": ["string"], + "enabled_by": "index_type", "enabled_by_value": ["Azure AI Search"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_fields", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}, {"default": "Collection(Edm.Single)", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "acs_index_connection": {"type": ["CognitiveSearchConnection"], + "enabled_by": "index_type", "enabled_by_value": ["Azure AI Search"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "acs_index_name": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + AI Search"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_indices", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}]}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "acs_metadata_field": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Azure AI Search"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_fields", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}, {"default": "Edm.String", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "aoai_embedding_connection": {"type": + ["AzureOpenAIConnection"], "enabled_by": "embedding_type", "enabled_by_value": + ["Azure OpenAI"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "uionly_hidden"}, "embedding_deployment": {"type": ["string"], "enabled_by": + "embedding_type", "enabled_by_value": ["Azure OpenAI"], "dynamic_list": {"func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.list_aoai_embedding_deployments", + "func_kwargs": [{"name": "aoai_connection", "optional": false, "reference": + "${inputs.aoai_embedding_connection}", "type": ["AzurOpenAIConnection"]}]}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "embedding_model": {"type": ["string"], "enabled_by": "embedding_type", "enabled_by_value": + ["OpenAI", "Hugging Face"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_embedding_models", + "func_kwargs": [{"name": "embedding_type", "optional": false, "reference": + "${inputs.embedding_type}", "type": ["string"]}]}, "allow_manual_entry": false, + "is_multi_select": false, "input_type": "uionly_hidden"}, "embedding_type": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + AI Search", "Azure CosmosDB for MongoDB vCore", "FAISS", "Pinecone"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_available_embedding_types", + "func_kwargs": [{"name": "index_type", "optional": false, "reference": "${inputs.index_type}", + "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "uionly_hidden"}, "faiss_index_path": {"type": ["string"], "enabled_by": + "index_type", "enabled_by_value": ["FAISS"], "allow_manual_entry": false, + "is_multi_select": false, "input_type": "uionly_hidden"}, "index_type": {"type": + ["string"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_available_index_types"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mlindex_asset_id": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Registered Index"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_registered_mlindices"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mlindex_content": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": + false, "generated_by": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.forward_mapping", + "func_kwargs": [{"name": "index_type", "reference": "${inputs.index_type}", + "type": ["string"]}, {"name": "mlindex_asset_id", "optional": true, "reference": + "${inputs.mlindex_asset_id}", "type": ["string"]}, {"name": "mlindex_path", + "optional": true, "reference": "${inputs.mlindex_path}", "type": ["string"]}, + {"name": "acs_index_connection", "optional": true, "reference": "${inputs.acs_index_connection}", + "type": ["CognitiveSearchConnection"]}, {"name": "acs_index_name", "optional": + true, "reference": "${inputs.acs_index_name}", "type": ["string"]}, {"name": + "acs_content_field", "optional": true, "reference": "${inputs.acs_content_field}", + "type": ["string"]}, {"name": "acs_embedding_field", "optional": true, "reference": + "${inputs.acs_embedding_field}", "type": ["string"]}, {"name": "acs_metadata_field", + "optional": true, "reference": "${inputs.acs_metadata_field}", "type": ["string"]}, + {"name": "semantic_configuration", "optional": true, "reference": "${inputs.semantic_configuration}", + "type": ["string"]}, {"name": "faiss_index_path", "optional": true, "reference": + "${inputs.faiss_index_path}", "type": ["string"]}, {"name": "pinecone_index_connection", + "optional": true, "reference": "${inputs.pinecone_index_connection}", "type": + ["string"]}, {"name": "pinecone_index_name", "optional": true, "reference": + "${inputs.pinecone_index_name}", "type": ["string"]}, {"name": "pinecone_content_field", + "optional": true, "reference": "${inputs.pinecone_content_field}", "type": + ["string"]}, {"name": "pinecone_metadata_field", "optional": true, "reference": + "${inputs.pinecone_metadata_field}", "type": ["string"]}, {"name": "mongodb_connection", + "optional": true, "reference": "${inputs.mongodb_connection}", "type": ["string"]}, + {"name": "mongodb_database", "optional": true, "reference": "${inputs.mongodb_database}", + "type": ["string"]}, {"name": "mongodb_collection", "optional": true, "reference": + "${inputs.mongodb_collection}", "type": ["string"]}, {"name": "mongodb_index_name", + "optional": true, "reference": "${inputs.mongodb_index_name}", "type": ["string"]}, + {"name": "mongodb_content_field", "optional": true, "reference": "${inputs.mongodb_content_field}", + "type": ["string"]}, {"name": "mongodb_embedding_field", "optional": true, + "reference": "${inputs.mongodb_embedding_field}", "type": ["string"]}, {"name": + "embedding_type", "optional": true, "reference": "${inputs.embedding_type}", + "type": ["string"]}, {"name": "aoai_embedding_connection", "optional": true, + "reference": "${inputs.aoai_embedding_connection}", "type": ["AzureOpenAIConnection"]}, + {"name": "oai_embedding_connection", "optional": true, "reference": "${inputs.oai_embedding_connection}", + "type": ["string"]}, {"name": "embedding_model", "optional": true, "reference": + "${inputs.embedding_model}", "type": ["string"]}, {"name": "embedding_deployment", + "optional": true, "reference": "${inputs.embedding_deployment}", "type": ["string"]}, + {"name": "serverless_embedding_connection", "optional": true, "reference": + "${inputs.serverless_embedding_connection}", "type": ["string"]}], "reverse_func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.reverse_mapping"}, "input_type": + "default"}, "mlindex_path": {"type": ["string"], "enabled_by": "index_type", + "enabled_by_value": ["MLIndex file from path"], "allow_manual_entry": false, + "is_multi_select": false, "input_type": "uionly_hidden"}, "mongodb_collection": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + CosmosDB for MongoDB vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_collections", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}]}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mongodb_connection": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Azure CosmosDB for MongoDB vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_connections"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mongodb_content_field": {"type": ["string"], "enabled_by": "index_type", + "enabled_by_value": ["Azure CosmosDB for MongoDB vCore"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_index_fields", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}, + {"name": "mongodb_collection", "optional": false, "reference": "${inputs.mongodb_collection}", + "type": ["string"]}, {"default": "string", "name": "field_data_type", "optional": + false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "mongodb_database": {"type": ["string"], + "enabled_by": "index_type", "enabled_by_value": ["Azure CosmosDB for MongoDB + vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_databases", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}]}, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "mongodb_embedding_field": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + CosmosDB for MongoDB vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_index_fields", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}, + {"name": "mongodb_collection", "optional": false, "reference": "${inputs.mongodb_collection}", + "type": ["string"]}, {"default": "ARRAY(float)", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "mongodb_index_name": {"type": ["string"], + "enabled_by": "index_type", "enabled_by_value": ["Azure CosmosDB for MongoDB + vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_indexes", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}, + {"name": "mongodb_collection", "optional": false, "reference": "${inputs.mongodb_collection}", + "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "uionly_hidden"}, "oai_embedding_connection": {"type": ["OpenAIConnection"], + "enabled_by": "embedding_type", "enabled_by_value": ["OpenAI"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "pinecone_content_field": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Pinecone"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "pinecone_index_connection": {"type": ["PineconeConnection"], "enabled_by": + "index_type", "enabled_by_value": ["Pinecone"], "dynamic_list": {"func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.list_pinecone_connections"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "pinecone_index_name": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Pinecone"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_pinecone_indices", + "func_kwargs": [{"name": "pinecone_connection_name", "optional": false, "reference": + "${inputs.pinecone_index_connection}", "type": ["string"]}]}, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "pinecone_metadata_field": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Pinecone"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "queries": {"type": ["object"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "query_type": {"type": ["string"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_available_query_types", + "func_kwargs": [{"name": "mlindex_content", "optional": false, "reference": + "${inputs.mlindex_content}", "type": ["string"]}]}, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "semantic_configuration": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + AI Search"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_semantic_configurations", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "uionly_hidden"}, "serverless_embedding_connection": {"type": + ["string"], "enabled_by": "embedding_type", "enabled_by_value": ["Serverless + Endpoint"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_serverless_embedding_connections"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "top_k": {"type": ["int"], "default": 3, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}}, "description": "Search an AzureML Vector + Index for relevant results using one or more text queries.", "module": "promptflow_vectordb.tool.common_index_lookup", + "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "preview"}, + {"name": "Faiss Index Lookup", "type": "python", "inputs": {"path": {"type": + ["string"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default"}, "top_k": {"type": ["int"], "default": "3", "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, "vector": {"type": ["list"], "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default"}, "vector_field": {"type": ["string"], "enabled_by": "connection", - "enabled_by_type": ["CognitiveSearchConnection"], "allow_manual_entry": false, - "is_multi_select": false, "input_type": "default"}}, "description": "Search - vector based query from existing Vector Database.", "module": "promptflow_vectordb.tool.vector_db_lookup", - "class_name": "VectorDBLookup", "function": "search", "is_builtin": true, - "package": "promptflow-vectordb", "package_version": "0.0.1", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "Vector Index Lookup", "type": "python", - "inputs": {"path": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "query": {"type": ["object"], "allow_manual_entry": + "default"}}, "description": "Search vector based query from the FAISS index + file.", "module": "promptflow_vectordb.tool.faiss_index_lookup", "class_name": + "FaissIndexLookup", "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "archived"}, + {"name": "Vector DB Lookup", "type": "python", "inputs": {"class_name": {"type": + ["string"], "enabled_by": "connection", "enabled_by_type": ["WeaviateConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "collection_name": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default"}, "connection": {"type": ["CognitiveSearchConnection", + "QdrantConnection", "WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "index_name": {"type": ["string"], "enabled_by": + "connection", "enabled_by_type": ["CognitiveSearchConnection"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "search_filters": + {"type": ["object"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection", + "QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default"}, "search_params": {"type": ["object"], "enabled_by": + "connection", "enabled_by_type": ["CognitiveSearchConnection", "QdrantConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "text_field": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["CognitiveSearchConnection", "QdrantConnection", "WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}}, "description": "Search text or vector based query - from AzureML Vector Index.", "module": "promptflow_vectordb.tool.vector_index_lookup", - "class_name": "VectorIndexLookup", "function": "search", "is_builtin": true, - "package": "promptflow-vectordb", "package_version": "0.0.1", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "calculate_accuracy.py", "type": - "python", "inputs": {"grades": {"type": ["object"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}}, "source": "calculate_accuracy.py", - "function": "calculate_accuracy", "is_builtin": false, "enable_kwargs": false, - "tool_state": "stable"}, {"name": "grade.py", "type": "python", "inputs": - {"groundtruth": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "prediction": {"type": ["string"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}}, "source": "grade.py", - "function": "grade", "is_builtin": false, "enable_kwargs": false, "tool_state": - "stable"}], "inputs": {"groundtruth": {"type": "string", "default": "APP", - "description": "Please specify the groundtruth column, which contains the - true label to the outputs that your flow produces.", "is_chat_input": false}, - "prediction": {"type": "string", "default": "APP", "description": "Please - specify the prediction column, which contains the predicted outputs that your - flow produces.", "is_chat_input": false}}, "outputs": {"grade": {"type": "string", - "reference": "${grade.output}", "evaluation_only": false, "is_chat_output": - false}}}, "flowRunResourceId": "azureml://locations/eastus/workspaces/00000/flows/eval_run_name/flowRuns/eval_run_name", + "input_type": "default"}, "vector": {"type": ["list"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "vector_field": + {"type": ["string"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}}, + "description": "Search vector based query from existing Vector Database.", + "module": "promptflow_vectordb.tool.vector_db_lookup", "class_name": "VectorDBLookup", + "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "archived"}, + {"name": "Vector Index Lookup", "type": "python", "inputs": {"path": {"type": + ["string"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default"}, "query": {"type": ["object"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}}, + "description": "Search text or vector based query from AzureML Vector Index.", + "module": "promptflow_vectordb.tool.vector_index_lookup", "class_name": "VectorIndexLookup", + "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "archived"}, + {"name": "calculate_accuracy.py", "type": "python", "inputs": {"grades": {"type": + ["object"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default"}}, "source": "calculate_accuracy.py", "function": "calculate_accuracy", + "is_builtin": false, "enable_kwargs": false, "tool_state": "stable"}, {"name": + "grade.py", "type": "python", "inputs": {"groundtruth": {"type": ["string"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "prediction": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}}, "source": "grade.py", "function": "grade", + "is_builtin": false, "enable_kwargs": false, "tool_state": "stable"}], "inputs": + {"groundtruth": {"type": "string", "default": "APP", "description": "Please + specify the groundtruth column, which contains the true label to the outputs + that your flow produces.", "is_chat_input": false}, "prediction": {"type": + "string", "default": "APP", "description": "Please specify the prediction + column, which contains the predicted outputs that your flow produces.", "is_chat_input": + false}}, "outputs": {"grade": {"type": "string", "reference": "${grade.output}", + "evaluation_only": false, "is_chat_output": false}}}, "flowRunResourceId": + "azureml://locations/eastus/workspaces/00000/flows/eval_run_name/flowRuns/eval_run_name", "flowRunId": "eval_run_name", "flowRunDisplayName": "sdk-cli-test-fixture-eval-run-without-llm", "batchDataInput": {"dataUri": "azureml://datastores/workspaceblobstore/paths/LocalUpload/74c11bba717480b2d6b04b8e746d09d7/webClassification3.jsonl"}, - "flowRunType": "FlowRun", "flowType": "Default", "runtimeName": "test-runtime-ci", + "flowRunType": "FlowRun", "flowType": "Default", "runtimeName": "hod-ci", "inputsMapping": {"groundtruth": "${data.answer}", "prediction": "${run.outputs.result}"}, "outputDatastoreName": "workspaceblobstore", "childRunBasePath": "promptflow/PromptFlowArtifacts/eval_run_name/flow_artifacts", - "flowDagFileRelativePath": "flow.dag.yaml", "flowSnapshotId": "1885cd85-1969-4fca-8c21-d8826ed5d886", - "studioPortalEndpoint": "https://ml.azure.com/runs/eval_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000"}' + "flowDagFileRelativePath": "flow.dag.yaml", "flowSnapshotId": "d1dc26d0-4657-4c78-a208-e48207b3a47c", + "sessionId": "766232492b56bb90a427434c7071a277e4eed302ecb37ddb", "studioPortalEndpoint": + "https://ml.azure.com/runs/eval_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000", + "studioPortalTraceEndpoint": "https://ml.azure.com/prompts/trace/run/eval_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/promptflow-eastus"}' headers: connection: - keep-alive content-length: - - '13872' + - '43399' content-type: - application/json; charset=utf-8 strict-transport-security: - - max-age=15724800; includeSubDomains; preload + - max-age=31536000; includeSubDomains; preload transfer-encoding: - chunked vary: @@ -2259,7 +4255,7 @@ interactions: x-content-type-options: - nosniff x-request-time: - - '0.270' + - '0.282' status: code: 200 message: OK @@ -2269,12 +4265,12 @@ interactions: Accept: - application/json Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive User-Agent: - - promptflow-sdk/0.0.1 azsdk-python-azuremachinelearningdesignerserviceclient/unknown - Python/3.10.13 (Windows-10-10.0.22631-SP0) + - promptflow-azure-sdk/1.9.0.dev0 azsdk-python-azuremachinelearningdesignerserviceclient/unknown + Python/3.9.19 (Windows-10-10.0.22631-SP0) method: GET uri: https://eastus.api.azureml.ms/flow/api/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/BulkRuns/eval_run_name response: @@ -2284,176 +4280,519 @@ interactions: "prediction": "${inputs.prediction}"}, "tool": "grade.py", "reduce": false}, {"name": "calculate_accuracy", "type": "python", "source": {"type": "code", "path": "calculate_accuracy.py"}, "inputs": {"grades": "${grade.output}"}, - "tool": "calculate_accuracy.py", "reduce": true}], "tools": [{"name": "Content - Safety (Text Analyze)", "type": "python", "inputs": {"connection": {"type": - ["AzureContentSafetyConnection"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "hate_category": {"type": ["string"], "default": - "medium_sensitivity", "enum": ["disable", "low_sensitivity", "medium_sensitivity", - "high_sensitivity"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "self_harm_category": {"type": ["string"], "default": - "medium_sensitivity", "enum": ["disable", "low_sensitivity", "medium_sensitivity", - "high_sensitivity"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "sexual_category": {"type": ["string"], "default": - "medium_sensitivity", "enum": ["disable", "low_sensitivity", "medium_sensitivity", + "tool": "calculate_accuracy.py", "reduce": true}], "tools": [{"name": "Azure + OpenAI GPT-4 Turbo with Vision", "type": "custom_llm", "inputs": {"connection": + {"type": ["AzureOpenAIConnection"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 0}}, "deployment_name": + {"type": ["string"], "enabled_by": "connection", "dynamic_list": {"func_path": + "promptflow.tools.aoai_gpt4v.list_deployment_names", "func_kwargs": [{"name": + "connection", "reference": "${inputs.connection}", "type": ["AzureOpenAIConnection"]}]}, + "allow_manual_entry": true, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 1}}, "detail": {"type": ["string"], "default": "auto", + "enum": ["low", "high", "auto"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 9}}, "frequency_penalty": + {"type": ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 7}}, "max_tokens": {"type": + ["int"], "default": 512, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 4}}, "presence_penalty": {"type": + ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 6}}, "seed": {"type": + ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 8}}, "stop": {"type": ["list"], + "default": "", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 5}}, "temperature": {"type": ["double"], + "default": 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 2}}, "top_p": {"type": ["double"], "default": + 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 3}}}, "description": "Use Azure OpenAI GPT-4 Turbo with + Vision to leverage AOAI vision ability.", "module": "promptflow.tools.aoai_gpt4v", + "class_name": "AzureOpenAI", "function": "chat", "icon": {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "default_prompt": "# system:\nAs an AI assistant, your task involves interpreting + images and responding to questions about the image.\nRemember to provide accurate + answers based on the information present in the image.\n\n# user:\nCan you + tell me what the image depicts?\n![image]({{image_input}})\n", "enable_kwargs": + false, "tool_state": "preview"}, {"name": "Content Safety (Text Analyze)", + "type": "python", "inputs": {"connection": {"type": ["AzureContentSafetyConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "hate_category": {"type": ["string"], "default": "medium_sensitivity", "enum": + ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "self_harm_category": {"type": ["string"], "default": "medium_sensitivity", + "enum": ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "sexual_category": {"type": ["string"], "default": "medium_sensitivity", "enum": + ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "text": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "violence_category": {"type": ["string"], + "default": "medium_sensitivity", "enum": ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "text": {"type": ["string"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "violence_category": - {"type": ["string"], "default": "medium_sensitivity", "enum": ["disable", - "low_sensitivity", "medium_sensitivity", "high_sensitivity"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}}, "description": - "Use Azure Content Safety to detect harmful content.", "module": "promptflow.tools.azure_content_safety", - "function": "analyze_text", "is_builtin": true, "package": "promptflow-tools", - "package_version": "0.0.216", "enable_kwargs": false, "deprecated_tools": - ["content_safety_text.tools.content_safety_text_tool.analyze_text"], "tool_state": - "stable"}, {"name": "Embedding", "type": "python", "inputs": {"connection": - {"type": ["AzureOpenAIConnection", "OpenAIConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "deployment_name": - {"type": ["string"], "enabled_by": "connection", "enabled_by_type": ["AzureOpenAIConnection"], - "model_list": ["text-embedding-ada-002", "text-search-ada-doc-001", "text-search-ada-query-001"], - "capabilities": {"completion": false, "chat_completion": false, "embeddings": - true}, "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default"}, "input": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "model": {"type": ["string"], "enum": ["text-embedding-ada-002", - "text-search-ada-doc-001", "text-search-ada-query-001"], "enabled_by": "connection", - "enabled_by_type": ["OpenAIConnection"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Use Open AI''s embedding - model to create an embedding vector representing the input text.", "module": - "promptflow.tools.embedding", "function": "embedding", "is_builtin": true, - "package": "promptflow-tools", "package_version": "0.0.216", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "Open Source LLM", "type": "custom_llm", - "inputs": {"api": {"type": ["string"], "enum": ["chat", "completion"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "connection": {"type": - ["CustomConnection"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "deployment_name": {"type": ["string"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "endpoint_name": - {"type": ["string"], "default": "-- please enter an endpoint name --", "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "max_new_tokens": - {"type": ["int"], "default": 500, "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "model_kwargs": {"type": ["object"], "default": - "{}", "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default", "advanced": true}, "temperature": {"type": ["double"], "default": - 1.0, "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default"}, "top_p": {"type": ["double"], "default": 1.0, "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default", "advanced": true}}, - "description": "Use an Open Source model from the Azure Model catalog, deployed - to an AzureML Online Endpoint for LLM Chat or Completion API calls.", "module": - "promptflow.tools.open_source_llm", "class_name": "OpenSourceLLM", "function": + "input_type": "default"}}, "description": "Use Azure Content Safety to detect + harmful content.", "module": "promptflow.tools.azure_content_safety", "function": + "analyze_text", "is_builtin": true, "package": "promptflow-tools", "package_version": + "1.5.0rc8", "enable_kwargs": false, "deprecated_tools": ["content_safety_text.tools.content_safety_text_tool.analyze_text"], + "tool_state": "stable"}, {"name": "Embedding", "type": "python", "inputs": + {"connection": {"type": ["AzureOpenAIConnection", "OpenAIConnection", "ServerlessConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "deployment_name": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["AzureOpenAIConnection"], "model_list": ["text-embedding-ada-002", "text-search-ada-doc-001", + "text-search-ada-query-001"], "capabilities": {"completion": false, "chat_completion": + false, "embeddings": true}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "input": {"type": ["string"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "model": {"type": + ["string"], "enum": ["text-embedding-ada-002", "text-search-ada-doc-001", + "text-search-ada-query-001"], "enabled_by": "connection", "enabled_by_type": + ["OpenAIConnection"], "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default"}}, "description": "Use Open AI''s embedding model + to create an embedding vector representing the input text.", "module": "promptflow.tools.embedding", + "function": "embedding", "is_builtin": true, "package": "promptflow-tools", + "package_version": "1.5.0rc8", "enable_kwargs": false, "tool_state": "stable"}, + {"name": "LLM", "type": "custom_llm", "inputs": {"api": {"type": ["string"], + "default": "chat", "allow_manual_entry": true, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 1, "text_box_size": "sm"}, "filter_by": {"input_name": + "connection", "values": {"AzureOpenAIConnection": {"enum": ["chat", "completion"]}, + "OpenAIConnection": {"enum": ["chat", "completion"]}, "ServerlessConnection": + {"enum": ["chat"]}}, "filter_attribute": "type"}}, "best_of": {"type": ["int"], + "default": 1, "enabled_by": "api", "enabled_by_value": ["completion"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 16, "text_box_size": "xs"}}, "connection": {"type": + ["AzureOpenAIConnection", "OpenAIConnection", "ServerlessConnection"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 0}}, "deployment_name": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["AzureOpenAIConnection"], "allow_manual_entry": true, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 2, "text_box_size": + "lg"}, "filter_by": {"input_name": "api", "values": {"chat": {"capabilities": + {"chat_completion": true, "exclude": {"model_version": ["vision-preview"]}}}, + "completion": {"capabilities": {"completion": true, "include": {"model_name": + ["gpt-35-turbo-instruct"]}}}}}}, "echo": {"type": ["bool"], "default": false, + "enabled_by": "api", "enabled_by_value": ["completion"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 15, "text_box_size": "xs"}}, "frequency_penalty": {"type": + ["int"], "default": 0, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "advanced": true, "ui_hints": {"index": 10, "text_box_size": + "xs"}}, "logit_bias": {"type": ["object"], "default": "", "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 11, "text_box_size": "lg"}}, "logprobs": {"type": ["int"], + "default": "", "enabled_by": "api", "enabled_by_value": ["completion"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 14, "text_box_size": "xs"}}, "max_tokens": {"type": + ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 7, "text_box_size": "xs"}}, + "model": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["OpenAIConnection"], "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 3, "text_box_size": "lg"}, + "filter_by": {"input_name": "api", "values": {"chat": {"enum": ["gpt-3.5-turbo", + "gpt-3.5-turbo-0301", "gpt-3.5-turbo-16k", "gpt-3.5-turbo-1106", "gpt-4", + "gpt-4-1106-preview", "gpt-4-0314", "gpt-4-32k", "gpt-4-32k-0314"]}, "completion": + {"enum": ["gpt-3.5-turbo-instruct"]}}}}, "presence_penalty": {"type": ["double"], + "default": 0, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "advanced": true, "ui_hints": {"index": 8}}, "response_format": + {"type": ["object"], "default": "", "enum": [{"type": "json_object"}, {"type": + "text"}], "enabled_by": "api", "enabled_by_value": ["chat"], "allow_manual_entry": + true, "is_multi_select": false, "input_type": "default", "advanced": false, + "ui_hints": {"index": 9, "text_box_size": "lg"}}, "seed": {"type": ["int"], + "default": "", "enabled_by": "api", "enabled_by_value": ["chat"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 12, "text_box_size": "xs"}}, "stop": {"type": ["list"], + "default": "", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 6, "text_box_size": "md"}}, "suffix": {"type": + ["string"], "default": "", "enabled_by": "api", "enabled_by_value": ["completion"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "advanced": true, "ui_hints": {"index": 13, "text_box_size": "xs"}}, "temperature": + {"type": ["double"], "default": 1, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 4, "text_box_size": + "xs"}}, "tool_choice": {"type": ["object", "string"], "default": "", "enabled_by": + "api", "enabled_by_value": ["chat"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 18}}, "tools": {"type": + ["list"], "default": [], "enabled_by": "api", "enabled_by_value": ["chat"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"display_style": "reusable_tool", "index": 17}}, "top_p": {"type": + ["double"], "default": 1, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "advanced": true, "ui_hints": {"index": 5, + "text_box_size": "xs"}}}, "description": {"ai_studio": "Use OpenAI-like Large + Language Model for text completion or chat. For more details, refer to this + [document](https://aka.ms/aistudio/pf/llm-tool).", "aml": "Use OpenAI-like + Large Language Model for text completion or chat. For more details, refer + to this [document](https://aka.ms/promptflow/llm-tool).", "default": "Use + OpenAI-like Large Language Model for text completion or chat."}, "module": + "promptflow.tools.llm", "function": "llm", "icon": {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "category": "llm", "is_builtin": true, "package": "promptflow-tools", "package_version": + "1.5.0rc8", "default_prompt": "# system:\nYou are a helpful assistant.\n\n# + user:\n{{question}}\n", "enable_kwargs": false, "tool_state": "stable", "groups": + [{"name": "Tools", "description": "Configure interactive capabilities by selecting + tools for the model to use and guiding its tool call decisions.", "inputs": + ["tools", "tool_choice"], "ui_hints": {"display_style": "table"}}]}, {"name": + "LLM-Vision", "type": "custom_llm", "inputs": {"connection": {"type": ["AzureOpenAIConnection", + "OpenAIConnection"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 0}}, "deployment_name": {"type": + ["string"], "enabled_by": "connection", "enabled_by_type": ["AzureOpenAIConnection"], + "capabilities": {"chat_completion": true, "include": {"model_version": ["vision-preview"]}}, + "allow_manual_entry": true, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 1, "text_box_size": "lg"}}, "detail": {"type": ["string"], + "default": "auto", "enum": ["low", "high", "auto"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 10}}, "frequency_penalty": {"type": ["int"], "default": 0, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 8, "text_box_size": "xs"}}, "max_tokens": {"type": ["int"], "default": "", + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 6, "text_box_size": "xs"}}, "model": {"type": ["string"], + "enum": ["gpt-4-vision-preview"], "enabled_by": "connection", "enabled_by_type": + ["OpenAIConnection"], "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 2, "text_box_size": "lg"}}, + "presence_penalty": {"type": ["double"], "default": 0, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 7}}, "seed": {"type": ["int"], "default": "", "allow_manual_entry": false, + "is_multi_select": false, "input_type": "default", "ui_hints": {"index": 9, + "text_box_size": "xs"}}, "stop": {"type": ["list"], "default": "", "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 5, "text_box_size": "md"}}, "temperature": {"type": ["double"], "default": + 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 3, "text_box_size": "xs"}}, "top_p": {"type": ["double"], + "default": 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 4, "text_box_size": "xs"}}}, "description": + {"ai_studio": "Use OpenAI-like Large Language Model to leverage vision ability. + For more details, refer to this [document](https://aka.ms/aistudio/pf/llm-vision-tool).", + "aml": "Use OpenAI-like Large Language Model to leverage vision ability. For + more details, refer to this [document](https://aka.ms/promptflow/llm-vision-tool).", + "default": "Use OpenAI-like Large Language Model to leverage vision ability."}, + "module": "promptflow.tools.llm_vision", "function": "llm_vision", "icon": + {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "category": "llm", "is_builtin": true, "package": "promptflow-tools", "package_version": + "1.5.0rc8", "default_prompt": "# system:\nAs an AI assistant, your task involves + interpreting images and responding to questions about the image.\nRemember + to provide accurate answers based on the information present in the image.\n\n# + user:\nCan you tell me what the image depicts?\n![image]({{image_input}})\n", + "enable_kwargs": false, "tool_state": "preview"}, {"name": "Open Model LLM", + "type": "custom_llm", "inputs": {"api": {"type": ["string"], "enum": ["chat", + "completion"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 2}}, "deployment_name": {"type": ["string"], + "default": "", "dynamic_list": {"func_path": "promptflow.tools.open_model_llm.list_deployment_names", + "func_kwargs": [{"name": "endpoint", "optional": true, "reference": "${inputs.endpoint}", + "type": ["string"]}]}, "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 1}}, "endpoint_name": {"type": + ["string"], "dynamic_list": {"func_path": "promptflow.tools.open_model_llm.list_endpoint_names"}, + "allow_manual_entry": true, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 0}}, "max_new_tokens": {"type": ["int"], "default": + 500, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 4}}, "model_kwargs": {"type": ["object"], + "default": "{}", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "advanced": true, "ui_hints": {"index": 6}}, "temperature": {"type": + ["double"], "default": 1.0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 3}}, "top_p": {"type": + ["double"], "default": 1.0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "advanced": true, "ui_hints": {"index": 5}}}, + "description": "Use an open model from the Azure Model catalog, deployed to + an AzureML Online Endpoint for LLM Chat or Completion API calls.", "module": + "promptflow.tools.open_model_llm", "class_name": "OpenModelLLM", "function": "call", "icon": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAACgElEQVR4nGWSz2vcVRTFP/e9NzOZ1KDGohASslLEH6VLV0ak4l/QpeDCrfQPcNGliODKnVm4EBdBsIjQIlhciKW0ycKFVCSNbYnjdDLtmPnmO/nO9917XcxMkjYX3uLx7nnn3HOuMK2Nix4fP78ZdrYXVkLVWjf3l3B1B+HpcjzGFtmqa6cePz7/x0dnn1n5qhj3iBJPYREIURAJuCtpY8PjReDbrf9WG7H1fuefwQU9qKztTcMJT+PNnEFvjGVDBDlSsH6p/9MLzy6+NxwVqI8RAg4IPmWedMckdLYP6O6UpIaQfvyyXG012+e79/ZfHukoS1ISMT2hGTB1RkUmNgQ5QZ0w+a2VWDq73MbdEWmfnnv6UWe7oNzPaLapl5CwuLTXK9WUGBuCjqekzhP+z52ZXOrKMD3OJg0Hh778aiOuvpnYvp05d6GJO4iAO4QAe/eV36/X5LFRV4Zmn+AdkqlL8Vjp3oVioOz+WTPzzYEgsN+fgPLYyJVheSbPPVl2ikeGZRjtG52/8rHuaV9VOlpP2OtKyVndcRVCSqOhsvxa4vW359i6OuKdD+aP8Q4SYPdOzS/flGjt1JUSaMqZ5nwa1Y8qWb/Ud/eZZkHisYezEM0m+fcelDr8F1SqW2LNK6r1jXQwyLzy1hxvrLXZulry7ocL+FS6G4QIu3fG/Px1gdYeW7LIgXU2P/115TOA5G7e3Rmj2aS/m7l5pThiZzrCcE/d1XHzbln373nw7y6veeoUm5KCNKT/IPPwbiY1hYd/l5MIT65BMFt87sU4v9D7/JMflr44uV6hGh1+L4RCkg6z5iK2tAhNLeLsNGwYA4fDYnC/drvuuFxe86NV/x+Ut27g0FvykgAAAABJRU5ErkJggg==", - "is_builtin": true, "package": "promptflow-tools", "package_version": "0.0.216", + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", "enable_kwargs": false, "tool_state": "stable"}, {"name": "OpenAI GPT-4V", "type": "custom_llm", "inputs": {"connection": {"type": ["OpenAIConnection"], - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "frequency_penalty": {"type": ["double"], "default": 0, "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "max_tokens": {"type": + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 0}}, "detail": {"type": ["string"], "default": "auto", + "enum": ["low", "high", "auto"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 9}}, "frequency_penalty": + {"type": ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 7}}, "max_tokens": {"type": + ["int"], "default": 512, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 4}}, "model": {"type": ["string"], + "enum": ["gpt-4-vision-preview"], "allow_manual_entry": true, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 1}}, "presence_penalty": + {"type": ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 6}}, "seed": {"type": ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "model": {"type": ["string"], "enum": ["gpt-4-vision-preview"], - "allow_manual_entry": true, "is_multi_select": false, "input_type": "default"}, - "presence_penalty": {"type": ["double"], "default": 0, "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "stop": {"type": - ["list"], "default": "", "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "temperature": {"type": ["double"], "default": 1, - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "top_p": {"type": ["double"], "default": 1, "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Use OpenAI GPT-4V to leverage + "input_type": "default", "ui_hints": {"index": 8}}, "stop": {"type": ["list"], + "default": "", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 5}}, "temperature": {"type": ["double"], + "default": 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 2}}, "top_p": {"type": ["double"], "default": + 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 3}}}, "description": "Use OpenAI GPT-4V to leverage vision ability.", "module": "promptflow.tools.openai_gpt4v", "class_name": - "OpenAI", "function": "chat", "is_builtin": true, "package": "promptflow-tools", - "package_version": "0.0.216", "default_prompt": "# system:\nAs an AI assistant, - your task involves interpreting images and responding to questions about the - image.\nRemember to provide accurate answers based on the information present - in the image.\n\n# user:\nCan you tell me what the image depicts?\n![image]({{image_input}})\n", - "enable_kwargs": false, "tool_state": "stable"}, {"name": "Serp API", "type": - "python", "inputs": {"connection": {"type": ["SerpConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "engine": {"type": - ["string"], "default": "google", "enum": ["google", "bing"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "location": {"type": - ["string"], "default": "", "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "num": {"type": ["int"], "default": "10", - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "query": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "safe": {"type": ["string"], "default": "off", - "enum": ["active", "off"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Use Serp API to obtain search - results from a specific search engine.", "module": "promptflow.tools.serpapi", - "class_name": "SerpAPI", "function": "search", "is_builtin": true, "package": - "promptflow-tools", "package_version": "0.0.216", "enable_kwargs": false, - "tool_state": "stable"}, {"name": "Faiss Index Lookup", "type": "python", - "inputs": {"path": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "vector": {"type": ["list"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Search vector based query - from the FAISS index file.", "module": "promptflow_vectordb.tool.faiss_index_lookup", - "class_name": "FaissIndexLookup", "function": "search", "is_builtin": true, - "package": "promptflow-vectordb", "package_version": "0.0.1", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "Vector DB Lookup", "type": "python", - "inputs": {"class_name": {"type": ["string"], "enabled_by": "connection", - "enabled_by_type": ["WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "collection_name": {"type": ["string"], "enabled_by": - "connection", "enabled_by_type": ["QdrantConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "connection": {"type": - ["CognitiveSearchConnection", "QdrantConnection", "WeaviateConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "index_name": {"type": - ["string"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection"], - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "search_filters": {"type": ["object"], "enabled_by": "connection", "enabled_by_type": - ["CognitiveSearchConnection", "QdrantConnection"], "allow_manual_entry": false, - "is_multi_select": false, "input_type": "default"}, "search_params": {"type": - ["object"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection", - "QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "text_field": {"type": ["string"], "enabled_by": - "connection", "enabled_by_type": ["CognitiveSearchConnection", "QdrantConnection", - "WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", "allow_manual_entry": + "OpenAI", "function": "chat", "icon": {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "default_prompt": "# system:\nAs an AI assistant, your task involves interpreting + images and responding to questions about the image.\nRemember to provide accurate + answers based on the information present in the image.\n\n# user:\nCan you + tell me what the image depicts?\n![image]({{image_input}})\n", "enable_kwargs": + false, "tool_state": "preview"}, {"name": "Serp API", "type": "python", "inputs": + {"connection": {"type": ["SerpConnection"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "engine": {"type": ["string"], "default": + "google", "enum": ["google", "bing"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "location": {"type": ["string"], "default": + "", "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "num": {"type": ["int"], "default": "10", "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "query": {"type": ["string"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "safe": {"type": + ["string"], "default": "off", "enum": ["active", "off"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}}, "description": + "Use Serp API to obtain search results from a specific search engine.", "module": + "promptflow.tools.serpapi", "class_name": "SerpAPI", "function": "search", + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "enable_kwargs": false, "tool_state": "stable"}, {"name": "Index Lookup", + "type": "python", "inputs": {"acs_content_field": {"type": ["string"], "enabled_by": + "index_type", "enabled_by_value": ["Azure AI Search"], "dynamic_list": {"func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_fields", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}, {"default": "Edm.String", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "acs_embedding_field": {"type": ["string"], + "enabled_by": "index_type", "enabled_by_value": ["Azure AI Search"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_fields", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}, {"default": "Collection(Edm.Single)", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "acs_index_connection": {"type": ["CognitiveSearchConnection"], + "enabled_by": "index_type", "enabled_by_value": ["Azure AI Search"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "acs_index_name": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + AI Search"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_indices", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}]}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "acs_metadata_field": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Azure AI Search"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_fields", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}, {"default": "Edm.String", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "aoai_embedding_connection": {"type": + ["AzureOpenAIConnection"], "enabled_by": "embedding_type", "enabled_by_value": + ["Azure OpenAI"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "uionly_hidden"}, "embedding_deployment": {"type": ["string"], "enabled_by": + "embedding_type", "enabled_by_value": ["Azure OpenAI"], "dynamic_list": {"func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.list_aoai_embedding_deployments", + "func_kwargs": [{"name": "aoai_connection", "optional": false, "reference": + "${inputs.aoai_embedding_connection}", "type": ["AzurOpenAIConnection"]}]}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "embedding_model": {"type": ["string"], "enabled_by": "embedding_type", "enabled_by_value": + ["OpenAI", "Hugging Face"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_embedding_models", + "func_kwargs": [{"name": "embedding_type", "optional": false, "reference": + "${inputs.embedding_type}", "type": ["string"]}]}, "allow_manual_entry": false, + "is_multi_select": false, "input_type": "uionly_hidden"}, "embedding_type": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + AI Search", "Azure CosmosDB for MongoDB vCore", "FAISS", "Pinecone"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_available_embedding_types", + "func_kwargs": [{"name": "index_type", "optional": false, "reference": "${inputs.index_type}", + "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "uionly_hidden"}, "faiss_index_path": {"type": ["string"], "enabled_by": + "index_type", "enabled_by_value": ["FAISS"], "allow_manual_entry": false, + "is_multi_select": false, "input_type": "uionly_hidden"}, "index_type": {"type": + ["string"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_available_index_types"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mlindex_asset_id": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Registered Index"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_registered_mlindices"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mlindex_content": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": + false, "generated_by": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.forward_mapping", + "func_kwargs": [{"name": "index_type", "reference": "${inputs.index_type}", + "type": ["string"]}, {"name": "mlindex_asset_id", "optional": true, "reference": + "${inputs.mlindex_asset_id}", "type": ["string"]}, {"name": "mlindex_path", + "optional": true, "reference": "${inputs.mlindex_path}", "type": ["string"]}, + {"name": "acs_index_connection", "optional": true, "reference": "${inputs.acs_index_connection}", + "type": ["CognitiveSearchConnection"]}, {"name": "acs_index_name", "optional": + true, "reference": "${inputs.acs_index_name}", "type": ["string"]}, {"name": + "acs_content_field", "optional": true, "reference": "${inputs.acs_content_field}", + "type": ["string"]}, {"name": "acs_embedding_field", "optional": true, "reference": + "${inputs.acs_embedding_field}", "type": ["string"]}, {"name": "acs_metadata_field", + "optional": true, "reference": "${inputs.acs_metadata_field}", "type": ["string"]}, + {"name": "semantic_configuration", "optional": true, "reference": "${inputs.semantic_configuration}", + "type": ["string"]}, {"name": "faiss_index_path", "optional": true, "reference": + "${inputs.faiss_index_path}", "type": ["string"]}, {"name": "pinecone_index_connection", + "optional": true, "reference": "${inputs.pinecone_index_connection}", "type": + ["string"]}, {"name": "pinecone_index_name", "optional": true, "reference": + "${inputs.pinecone_index_name}", "type": ["string"]}, {"name": "pinecone_content_field", + "optional": true, "reference": "${inputs.pinecone_content_field}", "type": + ["string"]}, {"name": "pinecone_metadata_field", "optional": true, "reference": + "${inputs.pinecone_metadata_field}", "type": ["string"]}, {"name": "mongodb_connection", + "optional": true, "reference": "${inputs.mongodb_connection}", "type": ["string"]}, + {"name": "mongodb_database", "optional": true, "reference": "${inputs.mongodb_database}", + "type": ["string"]}, {"name": "mongodb_collection", "optional": true, "reference": + "${inputs.mongodb_collection}", "type": ["string"]}, {"name": "mongodb_index_name", + "optional": true, "reference": "${inputs.mongodb_index_name}", "type": ["string"]}, + {"name": "mongodb_content_field", "optional": true, "reference": "${inputs.mongodb_content_field}", + "type": ["string"]}, {"name": "mongodb_embedding_field", "optional": true, + "reference": "${inputs.mongodb_embedding_field}", "type": ["string"]}, {"name": + "embedding_type", "optional": true, "reference": "${inputs.embedding_type}", + "type": ["string"]}, {"name": "aoai_embedding_connection", "optional": true, + "reference": "${inputs.aoai_embedding_connection}", "type": ["AzureOpenAIConnection"]}, + {"name": "oai_embedding_connection", "optional": true, "reference": "${inputs.oai_embedding_connection}", + "type": ["string"]}, {"name": "embedding_model", "optional": true, "reference": + "${inputs.embedding_model}", "type": ["string"]}, {"name": "embedding_deployment", + "optional": true, "reference": "${inputs.embedding_deployment}", "type": ["string"]}, + {"name": "serverless_embedding_connection", "optional": true, "reference": + "${inputs.serverless_embedding_connection}", "type": ["string"]}], "reverse_func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.reverse_mapping"}, "input_type": + "default"}, "mlindex_path": {"type": ["string"], "enabled_by": "index_type", + "enabled_by_value": ["MLIndex file from path"], "allow_manual_entry": false, + "is_multi_select": false, "input_type": "uionly_hidden"}, "mongodb_collection": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + CosmosDB for MongoDB vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_collections", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}]}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mongodb_connection": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Azure CosmosDB for MongoDB vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_connections"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mongodb_content_field": {"type": ["string"], "enabled_by": "index_type", + "enabled_by_value": ["Azure CosmosDB for MongoDB vCore"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_index_fields", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}, + {"name": "mongodb_collection", "optional": false, "reference": "${inputs.mongodb_collection}", + "type": ["string"]}, {"default": "string", "name": "field_data_type", "optional": + false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "mongodb_database": {"type": ["string"], + "enabled_by": "index_type", "enabled_by_value": ["Azure CosmosDB for MongoDB + vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_databases", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}]}, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "mongodb_embedding_field": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + CosmosDB for MongoDB vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_index_fields", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}, + {"name": "mongodb_collection", "optional": false, "reference": "${inputs.mongodb_collection}", + "type": ["string"]}, {"default": "ARRAY(float)", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "mongodb_index_name": {"type": ["string"], + "enabled_by": "index_type", "enabled_by_value": ["Azure CosmosDB for MongoDB + vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_indexes", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}, + {"name": "mongodb_collection", "optional": false, "reference": "${inputs.mongodb_collection}", + "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "uionly_hidden"}, "oai_embedding_connection": {"type": ["OpenAIConnection"], + "enabled_by": "embedding_type", "enabled_by_value": ["OpenAI"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "pinecone_content_field": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Pinecone"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "pinecone_index_connection": {"type": ["PineconeConnection"], "enabled_by": + "index_type", "enabled_by_value": ["Pinecone"], "dynamic_list": {"func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.list_pinecone_connections"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "pinecone_index_name": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Pinecone"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_pinecone_indices", + "func_kwargs": [{"name": "pinecone_connection_name", "optional": false, "reference": + "${inputs.pinecone_index_connection}", "type": ["string"]}]}, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "pinecone_metadata_field": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Pinecone"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "queries": {"type": ["object"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "query_type": {"type": ["string"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_available_query_types", + "func_kwargs": [{"name": "mlindex_content", "optional": false, "reference": + "${inputs.mlindex_content}", "type": ["string"]}]}, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "semantic_configuration": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + AI Search"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_semantic_configurations", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "uionly_hidden"}, "serverless_embedding_connection": {"type": + ["string"], "enabled_by": "embedding_type", "enabled_by_value": ["Serverless + Endpoint"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_serverless_embedding_connections"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "top_k": {"type": ["int"], "default": 3, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}}, "description": "Search an AzureML Vector + Index for relevant results using one or more text queries.", "module": "promptflow_vectordb.tool.common_index_lookup", + "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "preview"}, + {"name": "Faiss Index Lookup", "type": "python", "inputs": {"path": {"type": + ["string"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default"}, "top_k": {"type": ["int"], "default": "3", "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, "vector": {"type": ["list"], "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default"}, "vector_field": {"type": ["string"], "enabled_by": "connection", - "enabled_by_type": ["CognitiveSearchConnection"], "allow_manual_entry": false, - "is_multi_select": false, "input_type": "default"}}, "description": "Search - vector based query from existing Vector Database.", "module": "promptflow_vectordb.tool.vector_db_lookup", - "class_name": "VectorDBLookup", "function": "search", "is_builtin": true, - "package": "promptflow-vectordb", "package_version": "0.0.1", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "Vector Index Lookup", "type": "python", - "inputs": {"path": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "query": {"type": ["object"], "allow_manual_entry": + "default"}}, "description": "Search vector based query from the FAISS index + file.", "module": "promptflow_vectordb.tool.faiss_index_lookup", "class_name": + "FaissIndexLookup", "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "archived"}, + {"name": "Vector DB Lookup", "type": "python", "inputs": {"class_name": {"type": + ["string"], "enabled_by": "connection", "enabled_by_type": ["WeaviateConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "collection_name": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default"}, "connection": {"type": ["CognitiveSearchConnection", + "QdrantConnection", "WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "index_name": {"type": ["string"], "enabled_by": + "connection", "enabled_by_type": ["CognitiveSearchConnection"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "search_filters": + {"type": ["object"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection", + "QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default"}, "search_params": {"type": ["object"], "enabled_by": + "connection", "enabled_by_type": ["CognitiveSearchConnection", "QdrantConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "text_field": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["CognitiveSearchConnection", "QdrantConnection", "WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}}, "description": "Search text or vector based query - from AzureML Vector Index.", "module": "promptflow_vectordb.tool.vector_index_lookup", - "class_name": "VectorIndexLookup", "function": "search", "is_builtin": true, - "package": "promptflow-vectordb", "package_version": "0.0.1", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "calculate_accuracy.py", "type": - "python", "inputs": {"grades": {"type": ["object"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}}, "source": "calculate_accuracy.py", - "function": "calculate_accuracy", "is_builtin": false, "enable_kwargs": false, - "tool_state": "stable"}, {"name": "grade.py", "type": "python", "inputs": - {"groundtruth": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "prediction": {"type": ["string"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}}, "source": "grade.py", - "function": "grade", "is_builtin": false, "enable_kwargs": false, "tool_state": - "stable"}], "inputs": {"groundtruth": {"type": "string", "default": "APP", - "description": "Please specify the groundtruth column, which contains the - true label to the outputs that your flow produces.", "is_chat_input": false}, - "prediction": {"type": "string", "default": "APP", "description": "Please - specify the prediction column, which contains the predicted outputs that your - flow produces.", "is_chat_input": false}}, "outputs": {"grade": {"type": "string", - "reference": "${grade.output}", "evaluation_only": false, "is_chat_output": - false}}}, "flowRunResourceId": "azureml://locations/eastus/workspaces/00000/flows/eval_run_name/flowRuns/eval_run_name", + "input_type": "default"}, "vector": {"type": ["list"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "vector_field": + {"type": ["string"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}}, + "description": "Search vector based query from existing Vector Database.", + "module": "promptflow_vectordb.tool.vector_db_lookup", "class_name": "VectorDBLookup", + "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "archived"}, + {"name": "Vector Index Lookup", "type": "python", "inputs": {"path": {"type": + ["string"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default"}, "query": {"type": ["object"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}}, + "description": "Search text or vector based query from AzureML Vector Index.", + "module": "promptflow_vectordb.tool.vector_index_lookup", "class_name": "VectorIndexLookup", + "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "archived"}, + {"name": "calculate_accuracy.py", "type": "python", "inputs": {"grades": {"type": + ["object"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default"}}, "source": "calculate_accuracy.py", "function": "calculate_accuracy", + "is_builtin": false, "enable_kwargs": false, "tool_state": "stable"}, {"name": + "grade.py", "type": "python", "inputs": {"groundtruth": {"type": ["string"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "prediction": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}}, "source": "grade.py", "function": "grade", + "is_builtin": false, "enable_kwargs": false, "tool_state": "stable"}], "inputs": + {"groundtruth": {"type": "string", "default": "APP", "description": "Please + specify the groundtruth column, which contains the true label to the outputs + that your flow produces.", "is_chat_input": false}, "prediction": {"type": + "string", "default": "APP", "description": "Please specify the prediction + column, which contains the predicted outputs that your flow produces.", "is_chat_input": + false}}, "outputs": {"grade": {"type": "string", "reference": "${grade.output}", + "evaluation_only": false, "is_chat_output": false}}}, "flowRunResourceId": + "azureml://locations/eastus/workspaces/00000/flows/eval_run_name/flowRuns/eval_run_name", "flowRunId": "eval_run_name", "flowRunDisplayName": "sdk-cli-test-fixture-eval-run-without-llm", "batchDataInput": {"dataUri": "azureml://datastores/workspaceblobstore/paths/LocalUpload/74c11bba717480b2d6b04b8e746d09d7/webClassification3.jsonl"}, - "flowRunType": "FlowRun", "flowType": "Default", "runtimeName": "test-runtime-ci", + "flowRunType": "FlowRun", "flowType": "Default", "runtimeName": "hod-ci", "inputsMapping": {"groundtruth": "${data.answer}", "prediction": "${run.outputs.result}"}, "outputDatastoreName": "workspaceblobstore", "childRunBasePath": "promptflow/PromptFlowArtifacts/eval_run_name/flow_artifacts", - "flowDagFileRelativePath": "flow.dag.yaml", "flowSnapshotId": "1885cd85-1969-4fca-8c21-d8826ed5d886", - "studioPortalEndpoint": "https://ml.azure.com/runs/eval_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000"}' + "flowDagFileRelativePath": "flow.dag.yaml", "flowSnapshotId": "d1dc26d0-4657-4c78-a208-e48207b3a47c", + "sessionId": "766232492b56bb90a427434c7071a277e4eed302ecb37ddb", "studioPortalEndpoint": + "https://ml.azure.com/runs/eval_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000", + "studioPortalTraceEndpoint": "https://ml.azure.com/prompts/trace/run/eval_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/promptflow-eastus"}' headers: connection: - keep-alive content-length: - - '13872' + - '43399' content-type: - application/json; charset=utf-8 strict-transport-security: - - max-age=15724800; includeSubDomains; preload + - max-age=31536000; includeSubDomains; preload transfer-encoding: - chunked vary: @@ -2461,7 +4800,7 @@ interactions: x-content-type-options: - nosniff x-request-time: - - '0.231' + - '0.573' status: code: 200 message: OK @@ -2471,12 +4810,12 @@ interactions: Accept: - application/json Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive User-Agent: - - promptflow-sdk/0.0.1 azsdk-python-azuremachinelearningdesignerserviceclient/unknown - Python/3.10.13 (Windows-10-10.0.22631-SP0) + - promptflow-azure-sdk/1.9.0.dev0 azsdk-python-azuremachinelearningdesignerserviceclient/unknown + Python/3.9.19 (Windows-10-10.0.22631-SP0) method: GET uri: https://eastus.api.azureml.ms/flow/api/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/BulkRuns/eval_run_name response: @@ -2486,176 +4825,1064 @@ interactions: "prediction": "${inputs.prediction}"}, "tool": "grade.py", "reduce": false}, {"name": "calculate_accuracy", "type": "python", "source": {"type": "code", "path": "calculate_accuracy.py"}, "inputs": {"grades": "${grade.output}"}, - "tool": "calculate_accuracy.py", "reduce": true}], "tools": [{"name": "Content - Safety (Text Analyze)", "type": "python", "inputs": {"connection": {"type": - ["AzureContentSafetyConnection"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "hate_category": {"type": ["string"], "default": - "medium_sensitivity", "enum": ["disable", "low_sensitivity", "medium_sensitivity", - "high_sensitivity"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "self_harm_category": {"type": ["string"], "default": - "medium_sensitivity", "enum": ["disable", "low_sensitivity", "medium_sensitivity", - "high_sensitivity"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "sexual_category": {"type": ["string"], "default": - "medium_sensitivity", "enum": ["disable", "low_sensitivity", "medium_sensitivity", + "tool": "calculate_accuracy.py", "reduce": true}], "tools": [{"name": "Azure + OpenAI GPT-4 Turbo with Vision", "type": "custom_llm", "inputs": {"connection": + {"type": ["AzureOpenAIConnection"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 0}}, "deployment_name": + {"type": ["string"], "enabled_by": "connection", "dynamic_list": {"func_path": + "promptflow.tools.aoai_gpt4v.list_deployment_names", "func_kwargs": [{"name": + "connection", "reference": "${inputs.connection}", "type": ["AzureOpenAIConnection"]}]}, + "allow_manual_entry": true, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 1}}, "detail": {"type": ["string"], "default": "auto", + "enum": ["low", "high", "auto"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 9}}, "frequency_penalty": + {"type": ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 7}}, "max_tokens": {"type": + ["int"], "default": 512, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 4}}, "presence_penalty": {"type": + ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 6}}, "seed": {"type": + ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 8}}, "stop": {"type": ["list"], + "default": "", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 5}}, "temperature": {"type": ["double"], + "default": 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 2}}, "top_p": {"type": ["double"], "default": + 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 3}}}, "description": "Use Azure OpenAI GPT-4 Turbo with + Vision to leverage AOAI vision ability.", "module": "promptflow.tools.aoai_gpt4v", + "class_name": "AzureOpenAI", "function": "chat", "icon": {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "default_prompt": "# system:\nAs an AI assistant, your task involves interpreting + images and responding to questions about the image.\nRemember to provide accurate + answers based on the information present in the image.\n\n# user:\nCan you + tell me what the image depicts?\n![image]({{image_input}})\n", "enable_kwargs": + false, "tool_state": "preview"}, {"name": "Content Safety (Text Analyze)", + "type": "python", "inputs": {"connection": {"type": ["AzureContentSafetyConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "hate_category": {"type": ["string"], "default": "medium_sensitivity", "enum": + ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "self_harm_category": {"type": ["string"], "default": "medium_sensitivity", + "enum": ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "sexual_category": {"type": ["string"], "default": "medium_sensitivity", "enum": + ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "text": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "violence_category": {"type": ["string"], + "default": "medium_sensitivity", "enum": ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "text": {"type": ["string"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "violence_category": - {"type": ["string"], "default": "medium_sensitivity", "enum": ["disable", - "low_sensitivity", "medium_sensitivity", "high_sensitivity"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}}, "description": - "Use Azure Content Safety to detect harmful content.", "module": "promptflow.tools.azure_content_safety", - "function": "analyze_text", "is_builtin": true, "package": "promptflow-tools", - "package_version": "0.0.216", "enable_kwargs": false, "deprecated_tools": - ["content_safety_text.tools.content_safety_text_tool.analyze_text"], "tool_state": - "stable"}, {"name": "Embedding", "type": "python", "inputs": {"connection": - {"type": ["AzureOpenAIConnection", "OpenAIConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "deployment_name": - {"type": ["string"], "enabled_by": "connection", "enabled_by_type": ["AzureOpenAIConnection"], - "model_list": ["text-embedding-ada-002", "text-search-ada-doc-001", "text-search-ada-query-001"], - "capabilities": {"completion": false, "chat_completion": false, "embeddings": - true}, "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default"}, "input": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "model": {"type": ["string"], "enum": ["text-embedding-ada-002", - "text-search-ada-doc-001", "text-search-ada-query-001"], "enabled_by": "connection", - "enabled_by_type": ["OpenAIConnection"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Use Open AI''s embedding - model to create an embedding vector representing the input text.", "module": - "promptflow.tools.embedding", "function": "embedding", "is_builtin": true, - "package": "promptflow-tools", "package_version": "0.0.216", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "Open Source LLM", "type": "custom_llm", - "inputs": {"api": {"type": ["string"], "enum": ["chat", "completion"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "connection": {"type": - ["CustomConnection"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "deployment_name": {"type": ["string"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "endpoint_name": - {"type": ["string"], "default": "-- please enter an endpoint name --", "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "max_new_tokens": - {"type": ["int"], "default": 500, "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "model_kwargs": {"type": ["object"], "default": - "{}", "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default", "advanced": true}, "temperature": {"type": ["double"], "default": - 1.0, "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default"}, "top_p": {"type": ["double"], "default": 1.0, "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default", "advanced": true}}, - "description": "Use an Open Source model from the Azure Model catalog, deployed - to an AzureML Online Endpoint for LLM Chat or Completion API calls.", "module": - "promptflow.tools.open_source_llm", "class_name": "OpenSourceLLM", "function": + "input_type": "default"}}, "description": "Use Azure Content Safety to detect + harmful content.", "module": "promptflow.tools.azure_content_safety", "function": + "analyze_text", "is_builtin": true, "package": "promptflow-tools", "package_version": + "1.5.0rc8", "enable_kwargs": false, "deprecated_tools": ["content_safety_text.tools.content_safety_text_tool.analyze_text"], + "tool_state": "stable"}, {"name": "Embedding", "type": "python", "inputs": + {"connection": {"type": ["AzureOpenAIConnection", "OpenAIConnection", "ServerlessConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "deployment_name": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["AzureOpenAIConnection"], "model_list": ["text-embedding-ada-002", "text-search-ada-doc-001", + "text-search-ada-query-001"], "capabilities": {"completion": false, "chat_completion": + false, "embeddings": true}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "input": {"type": ["string"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "model": {"type": + ["string"], "enum": ["text-embedding-ada-002", "text-search-ada-doc-001", + "text-search-ada-query-001"], "enabled_by": "connection", "enabled_by_type": + ["OpenAIConnection"], "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default"}}, "description": "Use Open AI''s embedding model + to create an embedding vector representing the input text.", "module": "promptflow.tools.embedding", + "function": "embedding", "is_builtin": true, "package": "promptflow-tools", + "package_version": "1.5.0rc8", "enable_kwargs": false, "tool_state": "stable"}, + {"name": "LLM", "type": "custom_llm", "inputs": {"api": {"type": ["string"], + "default": "chat", "allow_manual_entry": true, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 1, "text_box_size": "sm"}, "filter_by": {"input_name": + "connection", "values": {"AzureOpenAIConnection": {"enum": ["chat", "completion"]}, + "OpenAIConnection": {"enum": ["chat", "completion"]}, "ServerlessConnection": + {"enum": ["chat"]}}, "filter_attribute": "type"}}, "best_of": {"type": ["int"], + "default": 1, "enabled_by": "api", "enabled_by_value": ["completion"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 16, "text_box_size": "xs"}}, "connection": {"type": + ["AzureOpenAIConnection", "OpenAIConnection", "ServerlessConnection"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 0}}, "deployment_name": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["AzureOpenAIConnection"], "allow_manual_entry": true, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 2, "text_box_size": + "lg"}, "filter_by": {"input_name": "api", "values": {"chat": {"capabilities": + {"chat_completion": true, "exclude": {"model_version": ["vision-preview"]}}}, + "completion": {"capabilities": {"completion": true, "include": {"model_name": + ["gpt-35-turbo-instruct"]}}}}}}, "echo": {"type": ["bool"], "default": false, + "enabled_by": "api", "enabled_by_value": ["completion"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 15, "text_box_size": "xs"}}, "frequency_penalty": {"type": + ["int"], "default": 0, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "advanced": true, "ui_hints": {"index": 10, "text_box_size": + "xs"}}, "logit_bias": {"type": ["object"], "default": "", "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 11, "text_box_size": "lg"}}, "logprobs": {"type": ["int"], + "default": "", "enabled_by": "api", "enabled_by_value": ["completion"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 14, "text_box_size": "xs"}}, "max_tokens": {"type": + ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 7, "text_box_size": "xs"}}, + "model": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["OpenAIConnection"], "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 3, "text_box_size": "lg"}, + "filter_by": {"input_name": "api", "values": {"chat": {"enum": ["gpt-3.5-turbo", + "gpt-3.5-turbo-0301", "gpt-3.5-turbo-16k", "gpt-3.5-turbo-1106", "gpt-4", + "gpt-4-1106-preview", "gpt-4-0314", "gpt-4-32k", "gpt-4-32k-0314"]}, "completion": + {"enum": ["gpt-3.5-turbo-instruct"]}}}}, "presence_penalty": {"type": ["double"], + "default": 0, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "advanced": true, "ui_hints": {"index": 8}}, "response_format": + {"type": ["object"], "default": "", "enum": [{"type": "json_object"}, {"type": + "text"}], "enabled_by": "api", "enabled_by_value": ["chat"], "allow_manual_entry": + true, "is_multi_select": false, "input_type": "default", "advanced": false, + "ui_hints": {"index": 9, "text_box_size": "lg"}}, "seed": {"type": ["int"], + "default": "", "enabled_by": "api", "enabled_by_value": ["chat"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 12, "text_box_size": "xs"}}, "stop": {"type": ["list"], + "default": "", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 6, "text_box_size": "md"}}, "suffix": {"type": + ["string"], "default": "", "enabled_by": "api", "enabled_by_value": ["completion"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "advanced": true, "ui_hints": {"index": 13, "text_box_size": "xs"}}, "temperature": + {"type": ["double"], "default": 1, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 4, "text_box_size": + "xs"}}, "tool_choice": {"type": ["object", "string"], "default": "", "enabled_by": + "api", "enabled_by_value": ["chat"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 18}}, "tools": {"type": + ["list"], "default": [], "enabled_by": "api", "enabled_by_value": ["chat"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"display_style": "reusable_tool", "index": 17}}, "top_p": {"type": + ["double"], "default": 1, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "advanced": true, "ui_hints": {"index": 5, + "text_box_size": "xs"}}}, "description": {"ai_studio": "Use OpenAI-like Large + Language Model for text completion or chat. For more details, refer to this + [document](https://aka.ms/aistudio/pf/llm-tool).", "aml": "Use OpenAI-like + Large Language Model for text completion or chat. For more details, refer + to this [document](https://aka.ms/promptflow/llm-tool).", "default": "Use + OpenAI-like Large Language Model for text completion or chat."}, "module": + "promptflow.tools.llm", "function": "llm", "icon": {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "category": "llm", "is_builtin": true, "package": "promptflow-tools", "package_version": + "1.5.0rc8", "default_prompt": "# system:\nYou are a helpful assistant.\n\n# + user:\n{{question}}\n", "enable_kwargs": false, "tool_state": "stable", "groups": + [{"name": "Tools", "description": "Configure interactive capabilities by selecting + tools for the model to use and guiding its tool call decisions.", "inputs": + ["tools", "tool_choice"], "ui_hints": {"display_style": "table"}}]}, {"name": + "LLM-Vision", "type": "custom_llm", "inputs": {"connection": {"type": ["AzureOpenAIConnection", + "OpenAIConnection"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 0}}, "deployment_name": {"type": + ["string"], "enabled_by": "connection", "enabled_by_type": ["AzureOpenAIConnection"], + "capabilities": {"chat_completion": true, "include": {"model_version": ["vision-preview"]}}, + "allow_manual_entry": true, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 1, "text_box_size": "lg"}}, "detail": {"type": ["string"], + "default": "auto", "enum": ["low", "high", "auto"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 10}}, "frequency_penalty": {"type": ["int"], "default": 0, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 8, "text_box_size": "xs"}}, "max_tokens": {"type": ["int"], "default": "", + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 6, "text_box_size": "xs"}}, "model": {"type": ["string"], + "enum": ["gpt-4-vision-preview"], "enabled_by": "connection", "enabled_by_type": + ["OpenAIConnection"], "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 2, "text_box_size": "lg"}}, + "presence_penalty": {"type": ["double"], "default": 0, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 7}}, "seed": {"type": ["int"], "default": "", "allow_manual_entry": false, + "is_multi_select": false, "input_type": "default", "ui_hints": {"index": 9, + "text_box_size": "xs"}}, "stop": {"type": ["list"], "default": "", "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 5, "text_box_size": "md"}}, "temperature": {"type": ["double"], "default": + 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 3, "text_box_size": "xs"}}, "top_p": {"type": ["double"], + "default": 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 4, "text_box_size": "xs"}}}, "description": + {"ai_studio": "Use OpenAI-like Large Language Model to leverage vision ability. + For more details, refer to this [document](https://aka.ms/aistudio/pf/llm-vision-tool).", + "aml": "Use OpenAI-like Large Language Model to leverage vision ability. For + more details, refer to this [document](https://aka.ms/promptflow/llm-vision-tool).", + "default": "Use OpenAI-like Large Language Model to leverage vision ability."}, + "module": "promptflow.tools.llm_vision", "function": "llm_vision", "icon": + {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "category": "llm", "is_builtin": true, "package": "promptflow-tools", "package_version": + "1.5.0rc8", "default_prompt": "# system:\nAs an AI assistant, your task involves + interpreting images and responding to questions about the image.\nRemember + to provide accurate answers based on the information present in the image.\n\n# + user:\nCan you tell me what the image depicts?\n![image]({{image_input}})\n", + "enable_kwargs": false, "tool_state": "preview"}, {"name": "Open Model LLM", + "type": "custom_llm", "inputs": {"api": {"type": ["string"], "enum": ["chat", + "completion"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 2}}, "deployment_name": {"type": ["string"], + "default": "", "dynamic_list": {"func_path": "promptflow.tools.open_model_llm.list_deployment_names", + "func_kwargs": [{"name": "endpoint", "optional": true, "reference": "${inputs.endpoint}", + "type": ["string"]}]}, "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 1}}, "endpoint_name": {"type": + ["string"], "dynamic_list": {"func_path": "promptflow.tools.open_model_llm.list_endpoint_names"}, + "allow_manual_entry": true, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 0}}, "max_new_tokens": {"type": ["int"], "default": + 500, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 4}}, "model_kwargs": {"type": ["object"], + "default": "{}", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "advanced": true, "ui_hints": {"index": 6}}, "temperature": {"type": + ["double"], "default": 1.0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 3}}, "top_p": {"type": + ["double"], "default": 1.0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "advanced": true, "ui_hints": {"index": 5}}}, + "description": "Use an open model from the Azure Model catalog, deployed to + an AzureML Online Endpoint for LLM Chat or Completion API calls.", "module": + "promptflow.tools.open_model_llm", "class_name": "OpenModelLLM", "function": "call", "icon": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAACgElEQVR4nGWSz2vcVRTFP/e9NzOZ1KDGohASslLEH6VLV0ak4l/QpeDCrfQPcNGliODKnVm4EBdBsIjQIlhciKW0ycKFVCSNbYnjdDLtmPnmO/nO9917XcxMkjYX3uLx7nnn3HOuMK2Nix4fP78ZdrYXVkLVWjf3l3B1B+HpcjzGFtmqa6cePz7/x0dnn1n5qhj3iBJPYREIURAJuCtpY8PjReDbrf9WG7H1fuefwQU9qKztTcMJT+PNnEFvjGVDBDlSsH6p/9MLzy6+NxwVqI8RAg4IPmWedMckdLYP6O6UpIaQfvyyXG012+e79/ZfHukoS1ISMT2hGTB1RkUmNgQ5QZ0w+a2VWDq73MbdEWmfnnv6UWe7oNzPaLapl5CwuLTXK9WUGBuCjqekzhP+z52ZXOrKMD3OJg0Hh778aiOuvpnYvp05d6GJO4iAO4QAe/eV36/X5LFRV4Zmn+AdkqlL8Vjp3oVioOz+WTPzzYEgsN+fgPLYyJVheSbPPVl2ikeGZRjtG52/8rHuaV9VOlpP2OtKyVndcRVCSqOhsvxa4vW359i6OuKdD+aP8Q4SYPdOzS/flGjt1JUSaMqZ5nwa1Y8qWb/Ud/eZZkHisYezEM0m+fcelDr8F1SqW2LNK6r1jXQwyLzy1hxvrLXZulry7ocL+FS6G4QIu3fG/Px1gdYeW7LIgXU2P/115TOA5G7e3Rmj2aS/m7l5pThiZzrCcE/d1XHzbln373nw7y6veeoUm5KCNKT/IPPwbiY1hYd/l5MIT65BMFt87sU4v9D7/JMflr44uV6hGh1+L4RCkg6z5iK2tAhNLeLsNGwYA4fDYnC/drvuuFxe86NV/x+Ut27g0FvykgAAAABJRU5ErkJggg==", - "is_builtin": true, "package": "promptflow-tools", "package_version": "0.0.216", + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", "enable_kwargs": false, "tool_state": "stable"}, {"name": "OpenAI GPT-4V", "type": "custom_llm", "inputs": {"connection": {"type": ["OpenAIConnection"], - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "frequency_penalty": {"type": ["double"], "default": 0, "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "max_tokens": {"type": + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 0}}, "detail": {"type": ["string"], "default": "auto", + "enum": ["low", "high", "auto"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 9}}, "frequency_penalty": + {"type": ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 7}}, "max_tokens": {"type": + ["int"], "default": 512, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 4}}, "model": {"type": ["string"], + "enum": ["gpt-4-vision-preview"], "allow_manual_entry": true, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 1}}, "presence_penalty": + {"type": ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 6}}, "seed": {"type": ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "model": {"type": ["string"], "enum": ["gpt-4-vision-preview"], - "allow_manual_entry": true, "is_multi_select": false, "input_type": "default"}, - "presence_penalty": {"type": ["double"], "default": 0, "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "stop": {"type": - ["list"], "default": "", "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "temperature": {"type": ["double"], "default": 1, - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "top_p": {"type": ["double"], "default": 1, "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Use OpenAI GPT-4V to leverage + "input_type": "default", "ui_hints": {"index": 8}}, "stop": {"type": ["list"], + "default": "", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 5}}, "temperature": {"type": ["double"], + "default": 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 2}}, "top_p": {"type": ["double"], "default": + 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 3}}}, "description": "Use OpenAI GPT-4V to leverage vision ability.", "module": "promptflow.tools.openai_gpt4v", "class_name": - "OpenAI", "function": "chat", "is_builtin": true, "package": "promptflow-tools", - "package_version": "0.0.216", "default_prompt": "# system:\nAs an AI assistant, - your task involves interpreting images and responding to questions about the - image.\nRemember to provide accurate answers based on the information present - in the image.\n\n# user:\nCan you tell me what the image depicts?\n![image]({{image_input}})\n", - "enable_kwargs": false, "tool_state": "stable"}, {"name": "Serp API", "type": - "python", "inputs": {"connection": {"type": ["SerpConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "engine": {"type": - ["string"], "default": "google", "enum": ["google", "bing"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "location": {"type": - ["string"], "default": "", "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "num": {"type": ["int"], "default": "10", - "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "query": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "safe": {"type": ["string"], "default": "off", - "enum": ["active", "off"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Use Serp API to obtain search - results from a specific search engine.", "module": "promptflow.tools.serpapi", - "class_name": "SerpAPI", "function": "search", "is_builtin": true, "package": - "promptflow-tools", "package_version": "0.0.216", "enable_kwargs": false, - "tool_state": "stable"}, {"name": "Faiss Index Lookup", "type": "python", - "inputs": {"path": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", + "OpenAI", "function": "chat", "icon": {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "default_prompt": "# system:\nAs an AI assistant, your task involves interpreting + images and responding to questions about the image.\nRemember to provide accurate + answers based on the information present in the image.\n\n# user:\nCan you + tell me what the image depicts?\n![image]({{image_input}})\n", "enable_kwargs": + false, "tool_state": "preview"}, {"name": "Serp API", "type": "python", "inputs": + {"connection": {"type": ["SerpConnection"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "engine": {"type": ["string"], "default": + "google", "enum": ["google", "bing"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "location": {"type": ["string"], "default": + "", "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "num": {"type": ["int"], "default": "10", "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "query": {"type": ["string"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "safe": {"type": + ["string"], "default": "off", "enum": ["active", "off"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}}, "description": + "Use Serp API to obtain search results from a specific search engine.", "module": + "promptflow.tools.serpapi", "class_name": "SerpAPI", "function": "search", + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "enable_kwargs": false, "tool_state": "stable"}, {"name": "Index Lookup", + "type": "python", "inputs": {"acs_content_field": {"type": ["string"], "enabled_by": + "index_type", "enabled_by_value": ["Azure AI Search"], "dynamic_list": {"func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_fields", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}, {"default": "Edm.String", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "acs_embedding_field": {"type": ["string"], + "enabled_by": "index_type", "enabled_by_value": ["Azure AI Search"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_fields", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}, {"default": "Collection(Edm.Single)", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "acs_index_connection": {"type": ["CognitiveSearchConnection"], + "enabled_by": "index_type", "enabled_by_value": ["Azure AI Search"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "acs_index_name": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + AI Search"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_indices", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}]}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "acs_metadata_field": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Azure AI Search"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_fields", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}, {"default": "Edm.String", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "aoai_embedding_connection": {"type": + ["AzureOpenAIConnection"], "enabled_by": "embedding_type", "enabled_by_value": + ["Azure OpenAI"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "uionly_hidden"}, "embedding_deployment": {"type": ["string"], "enabled_by": + "embedding_type", "enabled_by_value": ["Azure OpenAI"], "dynamic_list": {"func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.list_aoai_embedding_deployments", + "func_kwargs": [{"name": "aoai_connection", "optional": false, "reference": + "${inputs.aoai_embedding_connection}", "type": ["AzurOpenAIConnection"]}]}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "embedding_model": {"type": ["string"], "enabled_by": "embedding_type", "enabled_by_value": + ["OpenAI", "Hugging Face"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_embedding_models", + "func_kwargs": [{"name": "embedding_type", "optional": false, "reference": + "${inputs.embedding_type}", "type": ["string"]}]}, "allow_manual_entry": false, + "is_multi_select": false, "input_type": "uionly_hidden"}, "embedding_type": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + AI Search", "Azure CosmosDB for MongoDB vCore", "FAISS", "Pinecone"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_available_embedding_types", + "func_kwargs": [{"name": "index_type", "optional": false, "reference": "${inputs.index_type}", + "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "uionly_hidden"}, "faiss_index_path": {"type": ["string"], "enabled_by": + "index_type", "enabled_by_value": ["FAISS"], "allow_manual_entry": false, + "is_multi_select": false, "input_type": "uionly_hidden"}, "index_type": {"type": + ["string"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_available_index_types"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mlindex_asset_id": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Registered Index"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_registered_mlindices"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mlindex_content": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": + false, "generated_by": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.forward_mapping", + "func_kwargs": [{"name": "index_type", "reference": "${inputs.index_type}", + "type": ["string"]}, {"name": "mlindex_asset_id", "optional": true, "reference": + "${inputs.mlindex_asset_id}", "type": ["string"]}, {"name": "mlindex_path", + "optional": true, "reference": "${inputs.mlindex_path}", "type": ["string"]}, + {"name": "acs_index_connection", "optional": true, "reference": "${inputs.acs_index_connection}", + "type": ["CognitiveSearchConnection"]}, {"name": "acs_index_name", "optional": + true, "reference": "${inputs.acs_index_name}", "type": ["string"]}, {"name": + "acs_content_field", "optional": true, "reference": "${inputs.acs_content_field}", + "type": ["string"]}, {"name": "acs_embedding_field", "optional": true, "reference": + "${inputs.acs_embedding_field}", "type": ["string"]}, {"name": "acs_metadata_field", + "optional": true, "reference": "${inputs.acs_metadata_field}", "type": ["string"]}, + {"name": "semantic_configuration", "optional": true, "reference": "${inputs.semantic_configuration}", + "type": ["string"]}, {"name": "faiss_index_path", "optional": true, "reference": + "${inputs.faiss_index_path}", "type": ["string"]}, {"name": "pinecone_index_connection", + "optional": true, "reference": "${inputs.pinecone_index_connection}", "type": + ["string"]}, {"name": "pinecone_index_name", "optional": true, "reference": + "${inputs.pinecone_index_name}", "type": ["string"]}, {"name": "pinecone_content_field", + "optional": true, "reference": "${inputs.pinecone_content_field}", "type": + ["string"]}, {"name": "pinecone_metadata_field", "optional": true, "reference": + "${inputs.pinecone_metadata_field}", "type": ["string"]}, {"name": "mongodb_connection", + "optional": true, "reference": "${inputs.mongodb_connection}", "type": ["string"]}, + {"name": "mongodb_database", "optional": true, "reference": "${inputs.mongodb_database}", + "type": ["string"]}, {"name": "mongodb_collection", "optional": true, "reference": + "${inputs.mongodb_collection}", "type": ["string"]}, {"name": "mongodb_index_name", + "optional": true, "reference": "${inputs.mongodb_index_name}", "type": ["string"]}, + {"name": "mongodb_content_field", "optional": true, "reference": "${inputs.mongodb_content_field}", + "type": ["string"]}, {"name": "mongodb_embedding_field", "optional": true, + "reference": "${inputs.mongodb_embedding_field}", "type": ["string"]}, {"name": + "embedding_type", "optional": true, "reference": "${inputs.embedding_type}", + "type": ["string"]}, {"name": "aoai_embedding_connection", "optional": true, + "reference": "${inputs.aoai_embedding_connection}", "type": ["AzureOpenAIConnection"]}, + {"name": "oai_embedding_connection", "optional": true, "reference": "${inputs.oai_embedding_connection}", + "type": ["string"]}, {"name": "embedding_model", "optional": true, "reference": + "${inputs.embedding_model}", "type": ["string"]}, {"name": "embedding_deployment", + "optional": true, "reference": "${inputs.embedding_deployment}", "type": ["string"]}, + {"name": "serverless_embedding_connection", "optional": true, "reference": + "${inputs.serverless_embedding_connection}", "type": ["string"]}], "reverse_func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.reverse_mapping"}, "input_type": + "default"}, "mlindex_path": {"type": ["string"], "enabled_by": "index_type", + "enabled_by_value": ["MLIndex file from path"], "allow_manual_entry": false, + "is_multi_select": false, "input_type": "uionly_hidden"}, "mongodb_collection": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + CosmosDB for MongoDB vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_collections", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}]}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mongodb_connection": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Azure CosmosDB for MongoDB vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_connections"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mongodb_content_field": {"type": ["string"], "enabled_by": "index_type", + "enabled_by_value": ["Azure CosmosDB for MongoDB vCore"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_index_fields", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}, + {"name": "mongodb_collection", "optional": false, "reference": "${inputs.mongodb_collection}", + "type": ["string"]}, {"default": "string", "name": "field_data_type", "optional": + false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "mongodb_database": {"type": ["string"], + "enabled_by": "index_type", "enabled_by_value": ["Azure CosmosDB for MongoDB + vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_databases", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}]}, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "mongodb_embedding_field": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + CosmosDB for MongoDB vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_index_fields", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}, + {"name": "mongodb_collection", "optional": false, "reference": "${inputs.mongodb_collection}", + "type": ["string"]}, {"default": "ARRAY(float)", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "mongodb_index_name": {"type": ["string"], + "enabled_by": "index_type", "enabled_by_value": ["Azure CosmosDB for MongoDB + vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_indexes", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}, + {"name": "mongodb_collection", "optional": false, "reference": "${inputs.mongodb_collection}", + "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "uionly_hidden"}, "oai_embedding_connection": {"type": ["OpenAIConnection"], + "enabled_by": "embedding_type", "enabled_by_value": ["OpenAI"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "pinecone_content_field": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Pinecone"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "pinecone_index_connection": {"type": ["PineconeConnection"], "enabled_by": + "index_type", "enabled_by_value": ["Pinecone"], "dynamic_list": {"func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.list_pinecone_connections"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "pinecone_index_name": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Pinecone"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_pinecone_indices", + "func_kwargs": [{"name": "pinecone_connection_name", "optional": false, "reference": + "${inputs.pinecone_index_connection}", "type": ["string"]}]}, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "pinecone_metadata_field": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Pinecone"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "queries": {"type": ["object"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "query_type": {"type": ["string"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_available_query_types", + "func_kwargs": [{"name": "mlindex_content", "optional": false, "reference": + "${inputs.mlindex_content}", "type": ["string"]}]}, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "semantic_configuration": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + AI Search"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_semantic_configurations", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "uionly_hidden"}, "serverless_embedding_connection": {"type": + ["string"], "enabled_by": "embedding_type", "enabled_by_value": ["Serverless + Endpoint"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_serverless_embedding_connections"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "top_k": {"type": ["int"], "default": 3, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}}, "description": "Search an AzureML Vector + Index for relevant results using one or more text queries.", "module": "promptflow_vectordb.tool.common_index_lookup", + "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "preview"}, + {"name": "Faiss Index Lookup", "type": "python", "inputs": {"path": {"type": + ["string"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default"}, "top_k": {"type": ["int"], "default": "3", "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "vector": {"type": + ["list"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default"}}, "description": "Search vector based query from the FAISS index + file.", "module": "promptflow_vectordb.tool.faiss_index_lookup", "class_name": + "FaissIndexLookup", "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "archived"}, + {"name": "Vector DB Lookup", "type": "python", "inputs": {"class_name": {"type": + ["string"], "enabled_by": "connection", "enabled_by_type": ["WeaviateConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "collection_name": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default"}, "connection": {"type": ["CognitiveSearchConnection", + "QdrantConnection", "WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "index_name": {"type": ["string"], "enabled_by": + "connection", "enabled_by_type": ["CognitiveSearchConnection"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "search_filters": + {"type": ["object"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection", + "QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default"}, "search_params": {"type": ["object"], "enabled_by": + "connection", "enabled_by_type": ["CognitiveSearchConnection", "QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "vector": {"type": ["list"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}}, "description": "Search vector based query - from the FAISS index file.", "module": "promptflow_vectordb.tool.faiss_index_lookup", - "class_name": "FaissIndexLookup", "function": "search", "is_builtin": true, - "package": "promptflow-vectordb", "package_version": "0.0.1", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "Vector DB Lookup", "type": "python", - "inputs": {"class_name": {"type": ["string"], "enabled_by": "connection", - "enabled_by_type": ["WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "collection_name": {"type": ["string"], "enabled_by": - "connection", "enabled_by_type": ["QdrantConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "connection": {"type": + "text_field": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection", "QdrantConnection", "WeaviateConnection"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}, "index_name": {"type": - ["string"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection"], + false, "is_multi_select": false, "input_type": "default"}, "top_k": {"type": + ["int"], "default": "3", "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default"}, "vector": {"type": ["list"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "vector_field": + {"type": ["string"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}}, + "description": "Search vector based query from existing Vector Database.", + "module": "promptflow_vectordb.tool.vector_db_lookup", "class_name": "VectorDBLookup", + "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "archived"}, + {"name": "Vector Index Lookup", "type": "python", "inputs": {"path": {"type": + ["string"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default"}, "query": {"type": ["object"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}}, + "description": "Search text or vector based query from AzureML Vector Index.", + "module": "promptflow_vectordb.tool.vector_index_lookup", "class_name": "VectorIndexLookup", + "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "archived"}, + {"name": "calculate_accuracy.py", "type": "python", "inputs": {"grades": {"type": + ["object"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default"}}, "source": "calculate_accuracy.py", "function": "calculate_accuracy", + "is_builtin": false, "enable_kwargs": false, "tool_state": "stable"}, {"name": + "grade.py", "type": "python", "inputs": {"groundtruth": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, - "search_filters": {"type": ["object"], "enabled_by": "connection", "enabled_by_type": - ["CognitiveSearchConnection", "QdrantConnection"], "allow_manual_entry": false, - "is_multi_select": false, "input_type": "default"}, "search_params": {"type": - ["object"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection", - "QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "text_field": {"type": ["string"], "enabled_by": - "connection", "enabled_by_type": ["CognitiveSearchConnection", "QdrantConnection", - "WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", "allow_manual_entry": + "prediction": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}}, "source": "grade.py", "function": "grade", + "is_builtin": false, "enable_kwargs": false, "tool_state": "stable"}], "inputs": + {"groundtruth": {"type": "string", "default": "APP", "description": "Please + specify the groundtruth column, which contains the true label to the outputs + that your flow produces.", "is_chat_input": false}, "prediction": {"type": + "string", "default": "APP", "description": "Please specify the prediction + column, which contains the predicted outputs that your flow produces.", "is_chat_input": + false}}, "outputs": {"grade": {"type": "string", "reference": "${grade.output}", + "evaluation_only": false, "is_chat_output": false}}}, "flowRunResourceId": + "azureml://locations/eastus/workspaces/00000/flows/eval_run_name/flowRuns/eval_run_name", + "flowRunId": "eval_run_name", "flowRunDisplayName": "sdk-cli-test-fixture-eval-run-without-llm", + "batchDataInput": {"dataUri": "azureml://datastores/workspaceblobstore/paths/LocalUpload/74c11bba717480b2d6b04b8e746d09d7/webClassification3.jsonl"}, + "flowRunType": "FlowRun", "flowType": "Default", "runtimeName": "hod-ci", + "inputsMapping": {"groundtruth": "${data.answer}", "prediction": "${run.outputs.result}"}, + "outputDatastoreName": "workspaceblobstore", "childRunBasePath": "promptflow/PromptFlowArtifacts/eval_run_name/flow_artifacts", + "flowDagFileRelativePath": "flow.dag.yaml", "flowSnapshotId": "d1dc26d0-4657-4c78-a208-e48207b3a47c", + "sessionId": "766232492b56bb90a427434c7071a277e4eed302ecb37ddb", "studioPortalEndpoint": + "https://ml.azure.com/runs/eval_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000", + "studioPortalTraceEndpoint": "https://ml.azure.com/prompts/trace/run/eval_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/promptflow-eastus"}' + headers: + connection: + - keep-alive + content-length: + - '43399' + content-type: + - application/json; charset=utf-8 + strict-transport-security: + - max-age=31536000; includeSubDomains; preload + transfer-encoding: + - chunked + vary: + - Accept-Encoding + x-content-type-options: + - nosniff + x-request-time: + - '0.372' + status: + code: 200 + message: OK +- request: + body: null + headers: + Accept: + - application/json + Accept-Encoding: + - gzip, deflate, br + Connection: + - keep-alive + User-Agent: + - promptflow-azure-sdk/1.9.0.dev0 azsdk-python-azuremachinelearningdesignerserviceclient/unknown + Python/3.9.19 (Windows-10-10.0.22631-SP0) + method: GET + uri: https://eastus.api.azureml.ms/flow/api/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/BulkRuns/eval_run_name + response: + body: + string: '{"flowGraph": {"nodes": [{"name": "grade", "type": "python", "source": + {"type": "code", "path": "grade.py"}, "inputs": {"groundtruth": "${inputs.groundtruth}", + "prediction": "${inputs.prediction}"}, "tool": "grade.py", "reduce": false}, + {"name": "calculate_accuracy", "type": "python", "source": {"type": "code", + "path": "calculate_accuracy.py"}, "inputs": {"grades": "${grade.output}"}, + "tool": "calculate_accuracy.py", "reduce": true}], "tools": [{"name": "Azure + OpenAI GPT-4 Turbo with Vision", "type": "custom_llm", "inputs": {"connection": + {"type": ["AzureOpenAIConnection"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 0}}, "deployment_name": + {"type": ["string"], "enabled_by": "connection", "dynamic_list": {"func_path": + "promptflow.tools.aoai_gpt4v.list_deployment_names", "func_kwargs": [{"name": + "connection", "reference": "${inputs.connection}", "type": ["AzureOpenAIConnection"]}]}, + "allow_manual_entry": true, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 1}}, "detail": {"type": ["string"], "default": "auto", + "enum": ["low", "high", "auto"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 9}}, "frequency_penalty": + {"type": ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 7}}, "max_tokens": {"type": + ["int"], "default": 512, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 4}}, "presence_penalty": {"type": + ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 6}}, "seed": {"type": + ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 8}}, "stop": {"type": ["list"], + "default": "", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 5}}, "temperature": {"type": ["double"], + "default": 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 2}}, "top_p": {"type": ["double"], "default": + 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 3}}}, "description": "Use Azure OpenAI GPT-4 Turbo with + Vision to leverage AOAI vision ability.", "module": "promptflow.tools.aoai_gpt4v", + "class_name": "AzureOpenAI", "function": "chat", "icon": {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "default_prompt": "# system:\nAs an AI assistant, your task involves interpreting + images and responding to questions about the image.\nRemember to provide accurate + answers based on the information present in the image.\n\n# user:\nCan you + tell me what the image depicts?\n![image]({{image_input}})\n", "enable_kwargs": + false, "tool_state": "preview"}, {"name": "Content Safety (Text Analyze)", + "type": "python", "inputs": {"connection": {"type": ["AzureContentSafetyConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "hate_category": {"type": ["string"], "default": "medium_sensitivity", "enum": + ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "self_harm_category": {"type": ["string"], "default": "medium_sensitivity", + "enum": ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "sexual_category": {"type": ["string"], "default": "medium_sensitivity", "enum": + ["disable", "low_sensitivity", "medium_sensitivity", "high_sensitivity"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "text": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "violence_category": {"type": ["string"], + "default": "medium_sensitivity", "enum": ["disable", "low_sensitivity", "medium_sensitivity", + "high_sensitivity"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default"}}, "description": "Use Azure Content Safety to detect + harmful content.", "module": "promptflow.tools.azure_content_safety", "function": + "analyze_text", "is_builtin": true, "package": "promptflow-tools", "package_version": + "1.5.0rc8", "enable_kwargs": false, "deprecated_tools": ["content_safety_text.tools.content_safety_text_tool.analyze_text"], + "tool_state": "stable"}, {"name": "Embedding", "type": "python", "inputs": + {"connection": {"type": ["AzureOpenAIConnection", "OpenAIConnection", "ServerlessConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "deployment_name": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["AzureOpenAIConnection"], "model_list": ["text-embedding-ada-002", "text-search-ada-doc-001", + "text-search-ada-query-001"], "capabilities": {"completion": false, "chat_completion": + false, "embeddings": true}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "input": {"type": ["string"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "model": {"type": + ["string"], "enum": ["text-embedding-ada-002", "text-search-ada-doc-001", + "text-search-ada-query-001"], "enabled_by": "connection", "enabled_by_type": + ["OpenAIConnection"], "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default"}}, "description": "Use Open AI''s embedding model + to create an embedding vector representing the input text.", "module": "promptflow.tools.embedding", + "function": "embedding", "is_builtin": true, "package": "promptflow-tools", + "package_version": "1.5.0rc8", "enable_kwargs": false, "tool_state": "stable"}, + {"name": "LLM", "type": "custom_llm", "inputs": {"api": {"type": ["string"], + "default": "chat", "allow_manual_entry": true, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 1, "text_box_size": "sm"}, "filter_by": {"input_name": + "connection", "values": {"AzureOpenAIConnection": {"enum": ["chat", "completion"]}, + "OpenAIConnection": {"enum": ["chat", "completion"]}, "ServerlessConnection": + {"enum": ["chat"]}}, "filter_attribute": "type"}}, "best_of": {"type": ["int"], + "default": 1, "enabled_by": "api", "enabled_by_value": ["completion"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 16, "text_box_size": "xs"}}, "connection": {"type": + ["AzureOpenAIConnection", "OpenAIConnection", "ServerlessConnection"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 0}}, "deployment_name": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["AzureOpenAIConnection"], "allow_manual_entry": true, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 2, "text_box_size": + "lg"}, "filter_by": {"input_name": "api", "values": {"chat": {"capabilities": + {"chat_completion": true, "exclude": {"model_version": ["vision-preview"]}}}, + "completion": {"capabilities": {"completion": true, "include": {"model_name": + ["gpt-35-turbo-instruct"]}}}}}}, "echo": {"type": ["bool"], "default": false, + "enabled_by": "api", "enabled_by_value": ["completion"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 15, "text_box_size": "xs"}}, "frequency_penalty": {"type": + ["int"], "default": 0, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "advanced": true, "ui_hints": {"index": 10, "text_box_size": + "xs"}}, "logit_bias": {"type": ["object"], "default": "", "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 11, "text_box_size": "lg"}}, "logprobs": {"type": ["int"], + "default": "", "enabled_by": "api", "enabled_by_value": ["completion"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 14, "text_box_size": "xs"}}, "max_tokens": {"type": + ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 7, "text_box_size": "xs"}}, + "model": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["OpenAIConnection"], "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 3, "text_box_size": "lg"}, + "filter_by": {"input_name": "api", "values": {"chat": {"enum": ["gpt-3.5-turbo", + "gpt-3.5-turbo-0301", "gpt-3.5-turbo-16k", "gpt-3.5-turbo-1106", "gpt-4", + "gpt-4-1106-preview", "gpt-4-0314", "gpt-4-32k", "gpt-4-32k-0314"]}, "completion": + {"enum": ["gpt-3.5-turbo-instruct"]}}}}, "presence_penalty": {"type": ["double"], + "default": 0, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "advanced": true, "ui_hints": {"index": 8}}, "response_format": + {"type": ["object"], "default": "", "enum": [{"type": "json_object"}, {"type": + "text"}], "enabled_by": "api", "enabled_by_value": ["chat"], "allow_manual_entry": + true, "is_multi_select": false, "input_type": "default", "advanced": false, + "ui_hints": {"index": 9, "text_box_size": "lg"}}, "seed": {"type": ["int"], + "default": "", "enabled_by": "api", "enabled_by_value": ["chat"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "advanced": true, + "ui_hints": {"index": 12, "text_box_size": "xs"}}, "stop": {"type": ["list"], + "default": "", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 6, "text_box_size": "md"}}, "suffix": {"type": + ["string"], "default": "", "enabled_by": "api", "enabled_by_value": ["completion"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "advanced": true, "ui_hints": {"index": 13, "text_box_size": "xs"}}, "temperature": + {"type": ["double"], "default": 1, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 4, "text_box_size": + "xs"}}, "tool_choice": {"type": ["object", "string"], "default": "", "enabled_by": + "api", "enabled_by_value": ["chat"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 18}}, "tools": {"type": + ["list"], "default": [], "enabled_by": "api", "enabled_by_value": ["chat"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"display_style": "reusable_tool", "index": 17}}, "top_p": {"type": + ["double"], "default": 1, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "advanced": true, "ui_hints": {"index": 5, + "text_box_size": "xs"}}}, "description": {"ai_studio": "Use OpenAI-like Large + Language Model for text completion or chat. For more details, refer to this + [document](https://aka.ms/aistudio/pf/llm-tool).", "aml": "Use OpenAI-like + Large Language Model for text completion or chat. For more details, refer + to this [document](https://aka.ms/promptflow/llm-tool).", "default": "Use + OpenAI-like Large Language Model for text completion or chat."}, "module": + "promptflow.tools.llm", "function": "llm", "icon": {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "category": "llm", "is_builtin": true, "package": "promptflow-tools", "package_version": + "1.5.0rc8", "default_prompt": "# system:\nYou are a helpful assistant.\n\n# + user:\n{{question}}\n", "enable_kwargs": false, "tool_state": "stable", "groups": + [{"name": "Tools", "description": "Configure interactive capabilities by selecting + tools for the model to use and guiding its tool call decisions.", "inputs": + ["tools", "tool_choice"], "ui_hints": {"display_style": "table"}}]}, {"name": + "LLM-Vision", "type": "custom_llm", "inputs": {"connection": {"type": ["AzureOpenAIConnection", + "OpenAIConnection"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 0}}, "deployment_name": {"type": + ["string"], "enabled_by": "connection", "enabled_by_type": ["AzureOpenAIConnection"], + "capabilities": {"chat_completion": true, "include": {"model_version": ["vision-preview"]}}, + "allow_manual_entry": true, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 1, "text_box_size": "lg"}}, "detail": {"type": ["string"], + "default": "auto", "enum": ["low", "high", "auto"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 10}}, "frequency_penalty": {"type": ["int"], "default": 0, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 8, "text_box_size": "xs"}}, "max_tokens": {"type": ["int"], "default": "", + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 6, "text_box_size": "xs"}}, "model": {"type": ["string"], + "enum": ["gpt-4-vision-preview"], "enabled_by": "connection", "enabled_by_type": + ["OpenAIConnection"], "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 2, "text_box_size": "lg"}}, + "presence_penalty": {"type": ["double"], "default": 0, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 7}}, "seed": {"type": ["int"], "default": "", "allow_manual_entry": false, + "is_multi_select": false, "input_type": "default", "ui_hints": {"index": 9, + "text_box_size": "xs"}}, "stop": {"type": ["list"], "default": "", "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default", "ui_hints": {"index": + 5, "text_box_size": "md"}}, "temperature": {"type": ["double"], "default": + 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 3, "text_box_size": "xs"}}, "top_p": {"type": ["double"], + "default": 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 4, "text_box_size": "xs"}}}, "description": + {"ai_studio": "Use OpenAI-like Large Language Model to leverage vision ability. + For more details, refer to this [document](https://aka.ms/aistudio/pf/llm-vision-tool).", + "aml": "Use OpenAI-like Large Language Model to leverage vision ability. For + more details, refer to this [document](https://aka.ms/promptflow/llm-vision-tool).", + "default": "Use OpenAI-like Large Language Model to leverage vision ability."}, + "module": "promptflow.tools.llm_vision", "function": "llm_vision", "icon": + {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "category": "llm", "is_builtin": true, "package": "promptflow-tools", "package_version": + "1.5.0rc8", "default_prompt": "# system:\nAs an AI assistant, your task involves + interpreting images and responding to questions about the image.\nRemember + to provide accurate answers based on the information present in the image.\n\n# + user:\nCan you tell me what the image depicts?\n![image]({{image_input}})\n", + "enable_kwargs": false, "tool_state": "preview"}, {"name": "Open Model LLM", + "type": "custom_llm", "inputs": {"api": {"type": ["string"], "enum": ["chat", + "completion"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 2}}, "deployment_name": {"type": ["string"], + "default": "", "dynamic_list": {"func_path": "promptflow.tools.open_model_llm.list_deployment_names", + "func_kwargs": [{"name": "endpoint", "optional": true, "reference": "${inputs.endpoint}", + "type": ["string"]}]}, "allow_manual_entry": true, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 1}}, "endpoint_name": {"type": + ["string"], "dynamic_list": {"func_path": "promptflow.tools.open_model_llm.list_endpoint_names"}, + "allow_manual_entry": true, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 0}}, "max_new_tokens": {"type": ["int"], "default": + 500, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 4}}, "model_kwargs": {"type": ["object"], + "default": "{}", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "advanced": true, "ui_hints": {"index": 6}}, "temperature": {"type": + ["double"], "default": 1.0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 3}}, "top_p": {"type": + ["double"], "default": 1.0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "advanced": true, "ui_hints": {"index": 5}}}, + "description": "Use an open model from the Azure Model catalog, deployed to + an AzureML Online Endpoint for LLM Chat or Completion API calls.", "module": + "promptflow.tools.open_model_llm", "class_name": "OpenModelLLM", "function": + "call", "icon": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAACgElEQVR4nGWSz2vcVRTFP/e9NzOZ1KDGohASslLEH6VLV0ak4l/QpeDCrfQPcNGliODKnVm4EBdBsIjQIlhciKW0ycKFVCSNbYnjdDLtmPnmO/nO9917XcxMkjYX3uLx7nnn3HOuMK2Nix4fP78ZdrYXVkLVWjf3l3B1B+HpcjzGFtmqa6cePz7/x0dnn1n5qhj3iBJPYREIURAJuCtpY8PjReDbrf9WG7H1fuefwQU9qKztTcMJT+PNnEFvjGVDBDlSsH6p/9MLzy6+NxwVqI8RAg4IPmWedMckdLYP6O6UpIaQfvyyXG012+e79/ZfHukoS1ISMT2hGTB1RkUmNgQ5QZ0w+a2VWDq73MbdEWmfnnv6UWe7oNzPaLapl5CwuLTXK9WUGBuCjqekzhP+z52ZXOrKMD3OJg0Hh778aiOuvpnYvp05d6GJO4iAO4QAe/eV36/X5LFRV4Zmn+AdkqlL8Vjp3oVioOz+WTPzzYEgsN+fgPLYyJVheSbPPVl2ikeGZRjtG52/8rHuaV9VOlpP2OtKyVndcRVCSqOhsvxa4vW359i6OuKdD+aP8Q4SYPdOzS/flGjt1JUSaMqZ5nwa1Y8qWb/Ud/eZZkHisYezEM0m+fcelDr8F1SqW2LNK6r1jXQwyLzy1hxvrLXZulry7ocL+FS6G4QIu3fG/Px1gdYeW7LIgXU2P/115TOA5G7e3Rmj2aS/m7l5pThiZzrCcE/d1XHzbln373nw7y6veeoUm5KCNKT/IPPwbiY1hYd/l5MIT65BMFt87sU4v9D7/JMflr44uV6hGh1+L4RCkg6z5iK2tAhNLeLsNGwYA4fDYnC/drvuuFxe86NV/x+Ut27g0FvykgAAAABJRU5ErkJggg==", + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "enable_kwargs": false, "tool_state": "stable"}, {"name": "OpenAI GPT-4V", + "type": "custom_llm", "inputs": {"connection": {"type": ["OpenAIConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 0}}, "detail": {"type": ["string"], "default": "auto", + "enum": ["low", "high", "auto"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 9}}, "frequency_penalty": + {"type": ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 7}}, "max_tokens": {"type": + ["int"], "default": 512, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 4}}, "model": {"type": ["string"], + "enum": ["gpt-4-vision-preview"], "allow_manual_entry": true, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 1}}, "presence_penalty": + {"type": ["double"], "default": 0, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default", "ui_hints": {"index": 6}}, "seed": {"type": + ["int"], "default": "", "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default", "ui_hints": {"index": 8}}, "stop": {"type": ["list"], + "default": "", "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 5}}, "temperature": {"type": ["double"], + "default": 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default", "ui_hints": {"index": 2}}, "top_p": {"type": ["double"], "default": + 1, "allow_manual_entry": false, "is_multi_select": false, "input_type": "default", + "ui_hints": {"index": 3}}}, "description": "Use OpenAI GPT-4V to leverage + vision ability.", "module": "promptflow.tools.openai_gpt4v", "class_name": + "OpenAI", "function": "chat", "icon": {"dark": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAA2ElEQVR4nJXSzW3CQBAF4DUSTjk+Al1AD0ikESslpBIEheRALhEpgAYSWV8OGUublf/yLuP3PPNmdndS+gdwXZrYDmh7fGE/W+wXbaYd8IYm4rxJPnZ0boI3wZcdJxs/n+AwV7DFK7aFyfQdYIMLPvES8YJNf5yp4jMeeEYdWh38gXOR35YGHe5xabvQdsHv6PLi8qV6gycc8YH3iMfQu6Lh4ASr+F5Hh3XwVWnQYzUkVlX1nccplAb1SN6Y/sfgmlK64VS8wimldIv/0yj2QLkHizG0iWP4AVAfQ34DVQONAAAAAElFTkSuQmCC", + "light": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAx0lEQVR4nJWSwQ2CQBBFX0jAcjgqXUgPJNiIsQQrIVCIFy8GC6ABDcGDX7Mus9n1Xz7zZ+fPsLPwH4bUg0dD2wMPcbR48Uxq4AKU4iSTDwZ1LhWXipN/B3V0J6hjBTvgLHZNonewBXrgDpzEvXSIjN0BE3AACmmF4kl5F6tNzcCoLpW0SvGovFvsb4oZ2AANcAOu4ka6axCcINN3rg654sww+CYsPD0OwjcozFNh/Qcd78tqVbCIW+n+Fky472Bh/Q6SYb1EEy8tDzd+9IsVPAAAAABJRU5ErkJggg=="}, + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "default_prompt": "# system:\nAs an AI assistant, your task involves interpreting + images and responding to questions about the image.\nRemember to provide accurate + answers based on the information present in the image.\n\n# user:\nCan you + tell me what the image depicts?\n![image]({{image_input}})\n", "enable_kwargs": + false, "tool_state": "preview"}, {"name": "Serp API", "type": "python", "inputs": + {"connection": {"type": ["SerpConnection"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "engine": {"type": ["string"], "default": + "google", "enum": ["google", "bing"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "location": {"type": ["string"], "default": + "", "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "num": {"type": ["int"], "default": "10", "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "query": {"type": ["string"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "safe": {"type": + ["string"], "default": "off", "enum": ["active", "off"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}}, "description": + "Use Serp API to obtain search results from a specific search engine.", "module": + "promptflow.tools.serpapi", "class_name": "SerpAPI", "function": "search", + "is_builtin": true, "package": "promptflow-tools", "package_version": "1.5.0rc8", + "enable_kwargs": false, "tool_state": "stable"}, {"name": "Index Lookup", + "type": "python", "inputs": {"acs_content_field": {"type": ["string"], "enabled_by": + "index_type", "enabled_by_value": ["Azure AI Search"], "dynamic_list": {"func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_fields", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}, {"default": "Edm.String", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "acs_embedding_field": {"type": ["string"], + "enabled_by": "index_type", "enabled_by_value": ["Azure AI Search"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_fields", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}, {"default": "Collection(Edm.Single)", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "acs_index_connection": {"type": ["CognitiveSearchConnection"], + "enabled_by": "index_type", "enabled_by_value": ["Azure AI Search"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "acs_index_name": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + AI Search"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_indices", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}]}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "acs_metadata_field": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Azure AI Search"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_fields", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}, {"default": "Edm.String", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "aoai_embedding_connection": {"type": + ["AzureOpenAIConnection"], "enabled_by": "embedding_type", "enabled_by_value": + ["Azure OpenAI"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "uionly_hidden"}, "embedding_deployment": {"type": ["string"], "enabled_by": + "embedding_type", "enabled_by_value": ["Azure OpenAI"], "dynamic_list": {"func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.list_aoai_embedding_deployments", + "func_kwargs": [{"name": "aoai_connection", "optional": false, "reference": + "${inputs.aoai_embedding_connection}", "type": ["AzurOpenAIConnection"]}]}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "embedding_model": {"type": ["string"], "enabled_by": "embedding_type", "enabled_by_value": + ["OpenAI", "Hugging Face"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_embedding_models", + "func_kwargs": [{"name": "embedding_type", "optional": false, "reference": + "${inputs.embedding_type}", "type": ["string"]}]}, "allow_manual_entry": false, + "is_multi_select": false, "input_type": "uionly_hidden"}, "embedding_type": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + AI Search", "Azure CosmosDB for MongoDB vCore", "FAISS", "Pinecone"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_available_embedding_types", + "func_kwargs": [{"name": "index_type", "optional": false, "reference": "${inputs.index_type}", + "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "uionly_hidden"}, "faiss_index_path": {"type": ["string"], "enabled_by": + "index_type", "enabled_by_value": ["FAISS"], "allow_manual_entry": false, + "is_multi_select": false, "input_type": "uionly_hidden"}, "index_type": {"type": + ["string"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_available_index_types"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mlindex_asset_id": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Registered Index"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_registered_mlindices"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mlindex_content": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": + false, "generated_by": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.forward_mapping", + "func_kwargs": [{"name": "index_type", "reference": "${inputs.index_type}", + "type": ["string"]}, {"name": "mlindex_asset_id", "optional": true, "reference": + "${inputs.mlindex_asset_id}", "type": ["string"]}, {"name": "mlindex_path", + "optional": true, "reference": "${inputs.mlindex_path}", "type": ["string"]}, + {"name": "acs_index_connection", "optional": true, "reference": "${inputs.acs_index_connection}", + "type": ["CognitiveSearchConnection"]}, {"name": "acs_index_name", "optional": + true, "reference": "${inputs.acs_index_name}", "type": ["string"]}, {"name": + "acs_content_field", "optional": true, "reference": "${inputs.acs_content_field}", + "type": ["string"]}, {"name": "acs_embedding_field", "optional": true, "reference": + "${inputs.acs_embedding_field}", "type": ["string"]}, {"name": "acs_metadata_field", + "optional": true, "reference": "${inputs.acs_metadata_field}", "type": ["string"]}, + {"name": "semantic_configuration", "optional": true, "reference": "${inputs.semantic_configuration}", + "type": ["string"]}, {"name": "faiss_index_path", "optional": true, "reference": + "${inputs.faiss_index_path}", "type": ["string"]}, {"name": "pinecone_index_connection", + "optional": true, "reference": "${inputs.pinecone_index_connection}", "type": + ["string"]}, {"name": "pinecone_index_name", "optional": true, "reference": + "${inputs.pinecone_index_name}", "type": ["string"]}, {"name": "pinecone_content_field", + "optional": true, "reference": "${inputs.pinecone_content_field}", "type": + ["string"]}, {"name": "pinecone_metadata_field", "optional": true, "reference": + "${inputs.pinecone_metadata_field}", "type": ["string"]}, {"name": "mongodb_connection", + "optional": true, "reference": "${inputs.mongodb_connection}", "type": ["string"]}, + {"name": "mongodb_database", "optional": true, "reference": "${inputs.mongodb_database}", + "type": ["string"]}, {"name": "mongodb_collection", "optional": true, "reference": + "${inputs.mongodb_collection}", "type": ["string"]}, {"name": "mongodb_index_name", + "optional": true, "reference": "${inputs.mongodb_index_name}", "type": ["string"]}, + {"name": "mongodb_content_field", "optional": true, "reference": "${inputs.mongodb_content_field}", + "type": ["string"]}, {"name": "mongodb_embedding_field", "optional": true, + "reference": "${inputs.mongodb_embedding_field}", "type": ["string"]}, {"name": + "embedding_type", "optional": true, "reference": "${inputs.embedding_type}", + "type": ["string"]}, {"name": "aoai_embedding_connection", "optional": true, + "reference": "${inputs.aoai_embedding_connection}", "type": ["AzureOpenAIConnection"]}, + {"name": "oai_embedding_connection", "optional": true, "reference": "${inputs.oai_embedding_connection}", + "type": ["string"]}, {"name": "embedding_model", "optional": true, "reference": + "${inputs.embedding_model}", "type": ["string"]}, {"name": "embedding_deployment", + "optional": true, "reference": "${inputs.embedding_deployment}", "type": ["string"]}, + {"name": "serverless_embedding_connection", "optional": true, "reference": + "${inputs.serverless_embedding_connection}", "type": ["string"]}], "reverse_func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.reverse_mapping"}, "input_type": + "default"}, "mlindex_path": {"type": ["string"], "enabled_by": "index_type", + "enabled_by_value": ["MLIndex file from path"], "allow_manual_entry": false, + "is_multi_select": false, "input_type": "uionly_hidden"}, "mongodb_collection": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + CosmosDB for MongoDB vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_collections", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}]}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mongodb_connection": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Azure CosmosDB for MongoDB vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_connections"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "mongodb_content_field": {"type": ["string"], "enabled_by": "index_type", + "enabled_by_value": ["Azure CosmosDB for MongoDB vCore"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_index_fields", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}, + {"name": "mongodb_collection", "optional": false, "reference": "${inputs.mongodb_collection}", + "type": ["string"]}, {"default": "string", "name": "field_data_type", "optional": + false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "mongodb_database": {"type": ["string"], + "enabled_by": "index_type", "enabled_by_value": ["Azure CosmosDB for MongoDB + vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_databases", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}]}, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "mongodb_embedding_field": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + CosmosDB for MongoDB vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_index_fields", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}, + {"name": "mongodb_collection", "optional": false, "reference": "${inputs.mongodb_collection}", + "type": ["string"]}, {"default": "ARRAY(float)", "name": "field_data_type", + "optional": false, "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "uionly_hidden"}, "mongodb_index_name": {"type": ["string"], + "enabled_by": "index_type", "enabled_by_value": ["Azure CosmosDB for MongoDB + vCore"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_mongodb_indexes", + "func_kwargs": [{"name": "mongodb_connection", "optional": false, "reference": + "${inputs.mongodb_connection}", "type": ["string"]}, {"name": "mongodb_database", + "optional": false, "reference": "${inputs.mongodb_database}", "type": ["string"]}, + {"name": "mongodb_collection", "optional": false, "reference": "${inputs.mongodb_collection}", + "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "uionly_hidden"}, "oai_embedding_connection": {"type": ["OpenAIConnection"], + "enabled_by": "embedding_type", "enabled_by_value": ["OpenAI"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "pinecone_content_field": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Pinecone"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "pinecone_index_connection": {"type": ["PineconeConnection"], "enabled_by": + "index_type", "enabled_by_value": ["Pinecone"], "dynamic_list": {"func_path": + "promptflow_vectordb.tool.common_index_lookup_utils.list_pinecone_connections"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "pinecone_index_name": {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": + ["Pinecone"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_pinecone_indices", + "func_kwargs": [{"name": "pinecone_connection_name", "optional": false, "reference": + "${inputs.pinecone_index_connection}", "type": ["string"]}]}, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "uionly_hidden"}, "pinecone_metadata_field": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Pinecone"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "queries": {"type": ["object"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "query_type": {"type": ["string"], "dynamic_list": + {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_available_query_types", + "func_kwargs": [{"name": "mlindex_content", "optional": false, "reference": + "${inputs.mlindex_content}", "type": ["string"]}]}, "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "semantic_configuration": + {"type": ["string"], "enabled_by": "index_type", "enabled_by_value": ["Azure + AI Search"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_acs_index_semantic_configurations", + "func_kwargs": [{"name": "acs_connection", "optional": false, "reference": + "${inputs.acs_index_connection}", "type": ["CognitiveSearchConnection"]}, + {"name": "acs_index_name", "optional": false, "reference": "${inputs.acs_index_name}", + "type": ["string"]}]}, "allow_manual_entry": false, "is_multi_select": false, + "input_type": "uionly_hidden"}, "serverless_embedding_connection": {"type": + ["string"], "enabled_by": "embedding_type", "enabled_by_value": ["Serverless + Endpoint"], "dynamic_list": {"func_path": "promptflow_vectordb.tool.common_index_lookup_utils.list_serverless_embedding_connections"}, + "allow_manual_entry": false, "is_multi_select": false, "input_type": "uionly_hidden"}, + "top_k": {"type": ["int"], "default": 3, "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}}, "description": "Search an AzureML Vector + Index for relevant results using one or more text queries.", "module": "promptflow_vectordb.tool.common_index_lookup", + "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "preview"}, + {"name": "Faiss Index Lookup", "type": "python", "inputs": {"path": {"type": + ["string"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default"}, "top_k": {"type": ["int"], "default": "3", "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, "vector": {"type": ["list"], "allow_manual_entry": false, "is_multi_select": false, "input_type": - "default"}, "vector_field": {"type": ["string"], "enabled_by": "connection", - "enabled_by_type": ["CognitiveSearchConnection"], "allow_manual_entry": false, - "is_multi_select": false, "input_type": "default"}}, "description": "Search - vector based query from existing Vector Database.", "module": "promptflow_vectordb.tool.vector_db_lookup", - "class_name": "VectorDBLookup", "function": "search", "is_builtin": true, - "package": "promptflow-vectordb", "package_version": "0.0.1", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "Vector Index Lookup", "type": "python", - "inputs": {"path": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "query": {"type": ["object"], "allow_manual_entry": + "default"}}, "description": "Search vector based query from the FAISS index + file.", "module": "promptflow_vectordb.tool.faiss_index_lookup", "class_name": + "FaissIndexLookup", "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "archived"}, + {"name": "Vector DB Lookup", "type": "python", "inputs": {"class_name": {"type": + ["string"], "enabled_by": "connection", "enabled_by_type": ["WeaviateConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "collection_name": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default"}, "connection": {"type": ["CognitiveSearchConnection", + "QdrantConnection", "WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "index_name": {"type": ["string"], "enabled_by": + "connection", "enabled_by_type": ["CognitiveSearchConnection"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "search_filters": + {"type": ["object"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection", + "QdrantConnection"], "allow_manual_entry": false, "is_multi_select": false, + "input_type": "default"}, "search_params": {"type": ["object"], "enabled_by": + "connection", "enabled_by_type": ["CognitiveSearchConnection", "QdrantConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "text_field": {"type": ["string"], "enabled_by": "connection", "enabled_by_type": + ["CognitiveSearchConnection", "QdrantConnection", "WeaviateConnection"], "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", "allow_manual_entry": false, "is_multi_select": false, - "input_type": "default"}}, "description": "Search text or vector based query - from AzureML Vector Index.", "module": "promptflow_vectordb.tool.vector_index_lookup", - "class_name": "VectorIndexLookup", "function": "search", "is_builtin": true, - "package": "promptflow-vectordb", "package_version": "0.0.1", "enable_kwargs": - false, "tool_state": "stable"}, {"name": "calculate_accuracy.py", "type": - "python", "inputs": {"grades": {"type": ["object"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}}, "source": "calculate_accuracy.py", - "function": "calculate_accuracy", "is_builtin": false, "enable_kwargs": false, - "tool_state": "stable"}, {"name": "grade.py", "type": "python", "inputs": - {"groundtruth": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": - false, "input_type": "default"}, "prediction": {"type": ["string"], "allow_manual_entry": - false, "is_multi_select": false, "input_type": "default"}}, "source": "grade.py", - "function": "grade", "is_builtin": false, "enable_kwargs": false, "tool_state": - "stable"}], "inputs": {"groundtruth": {"type": "string", "default": "APP", - "description": "Please specify the groundtruth column, which contains the - true label to the outputs that your flow produces.", "is_chat_input": false}, - "prediction": {"type": "string", "default": "APP", "description": "Please - specify the prediction column, which contains the predicted outputs that your - flow produces.", "is_chat_input": false}}, "outputs": {"grade": {"type": "string", - "reference": "${grade.output}", "evaluation_only": false, "is_chat_output": - false}}}, "flowRunResourceId": "azureml://locations/eastus/workspaces/00000/flows/eval_run_name/flowRuns/eval_run_name", + "input_type": "default"}, "vector": {"type": ["list"], "allow_manual_entry": + false, "is_multi_select": false, "input_type": "default"}, "vector_field": + {"type": ["string"], "enabled_by": "connection", "enabled_by_type": ["CognitiveSearchConnection"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}}, + "description": "Search vector based query from existing Vector Database.", + "module": "promptflow_vectordb.tool.vector_db_lookup", "class_name": "VectorDBLookup", + "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "archived"}, + {"name": "Vector Index Lookup", "type": "python", "inputs": {"path": {"type": + ["string"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default"}, "query": {"type": ["object"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}, "top_k": {"type": ["int"], "default": "3", + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}}, + "description": "Search text or vector based query from AzureML Vector Index.", + "module": "promptflow_vectordb.tool.vector_index_lookup", "class_name": "VectorIndexLookup", + "function": "search", "is_builtin": true, "package": "promptflow_vectordb", + "package_version": "0.2.9", "enable_kwargs": false, "tool_state": "archived"}, + {"name": "calculate_accuracy.py", "type": "python", "inputs": {"grades": {"type": + ["object"], "allow_manual_entry": false, "is_multi_select": false, "input_type": + "default"}}, "source": "calculate_accuracy.py", "function": "calculate_accuracy", + "is_builtin": false, "enable_kwargs": false, "tool_state": "stable"}, {"name": + "grade.py", "type": "python", "inputs": {"groundtruth": {"type": ["string"], + "allow_manual_entry": false, "is_multi_select": false, "input_type": "default"}, + "prediction": {"type": ["string"], "allow_manual_entry": false, "is_multi_select": + false, "input_type": "default"}}, "source": "grade.py", "function": "grade", + "is_builtin": false, "enable_kwargs": false, "tool_state": "stable"}], "inputs": + {"groundtruth": {"type": "string", "default": "APP", "description": "Please + specify the groundtruth column, which contains the true label to the outputs + that your flow produces.", "is_chat_input": false}, "prediction": {"type": + "string", "default": "APP", "description": "Please specify the prediction + column, which contains the predicted outputs that your flow produces.", "is_chat_input": + false}}, "outputs": {"grade": {"type": "string", "reference": "${grade.output}", + "evaluation_only": false, "is_chat_output": false}}}, "flowRunResourceId": + "azureml://locations/eastus/workspaces/00000/flows/eval_run_name/flowRuns/eval_run_name", "flowRunId": "eval_run_name", "flowRunDisplayName": "sdk-cli-test-fixture-eval-run-without-llm", "batchDataInput": {"dataUri": "azureml://datastores/workspaceblobstore/paths/LocalUpload/74c11bba717480b2d6b04b8e746d09d7/webClassification3.jsonl"}, - "flowRunType": "FlowRun", "flowType": "Default", "runtimeName": "test-runtime-ci", + "flowRunType": "FlowRun", "flowType": "Default", "runtimeName": "hod-ci", "inputsMapping": {"groundtruth": "${data.answer}", "prediction": "${run.outputs.result}"}, "outputDatastoreName": "workspaceblobstore", "childRunBasePath": "promptflow/PromptFlowArtifacts/eval_run_name/flow_artifacts", - "flowDagFileRelativePath": "flow.dag.yaml", "flowSnapshotId": "1885cd85-1969-4fca-8c21-d8826ed5d886", - "studioPortalEndpoint": "https://ml.azure.com/runs/eval_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000"}' + "flowDagFileRelativePath": "flow.dag.yaml", "flowSnapshotId": "d1dc26d0-4657-4c78-a208-e48207b3a47c", + "sessionId": "766232492b56bb90a427434c7071a277e4eed302ecb37ddb", "studioPortalEndpoint": + "https://ml.azure.com/runs/eval_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000", + "studioPortalTraceEndpoint": "https://ml.azure.com/prompts/trace/run/eval_run_name?wsid=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/00000/providers/Microsoft.MachineLearningServices/workspaces/promptflow-eastus"}' headers: connection: - keep-alive content-length: - - '13872' + - '43399' content-type: - application/json; charset=utf-8 strict-transport-security: - - max-age=15724800; includeSubDomains; preload + - max-age=31536000; includeSubDomains; preload transfer-encoding: - chunked vary: @@ -2663,7 +5890,7 @@ interactions: x-content-type-options: - nosniff x-request-time: - - '0.534' + - '0.349' status: code: 200 message: OK @@ -2674,7 +5901,7 @@ interactions: Accept: - '*/*' Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive Content-Length: @@ -2687,39 +5914,41 @@ interactions: uri: https://eastus.api.azureml.ms/history/v1.0/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/rundata response: body: - string: '{"runMetadata": {"runNumber": 1705045909, "rootRunId": "batch_run_name", - "createdUtc": "2024-01-12T07:51:49.7910367+00:00", "createdBy": {"userObjectId": - "00000000-0000-0000-0000-000000000000", "userPuId": null, "userIdp": "https://sts.windows.net/00000000-0000-0000-0000-000000000000/", - "userAltSecId": null, "userIss": "https://sts.windows.net/00000000-0000-0000-0000-000000000000/", - "userTenantId": "00000000-0000-0000-0000-000000000000", "userName": "4cbd0e2e-aae4-4099-b4ba-94d3a4910587", - "upn": null}, "userId": "00000000-0000-0000-0000-000000000000", "token": null, - "tokenExpiryTimeUtc": null, "error": null, "warnings": null, "revision": 6, - "statusRevision": 3, "runUuid": "de4d3748-43e9-499d-bacb-28bacd51d7dd", "parentRunUuid": - null, "rootRunUuid": "de4d3748-43e9-499d-bacb-28bacd51d7dd", "lastStartTimeUtc": - null, "currentComputeTime": null, "computeDuration": "00:00:04.0496282", "effectiveStartTimeUtc": - null, "lastModifiedBy": {"userObjectId": "00000000-0000-0000-0000-000000000000", - "userPuId": null, "userIdp": "https://sts.windows.net/00000000-0000-0000-0000-000000000000/", - "userAltSecId": null, "userIss": "https://sts.windows.net/00000000-0000-0000-0000-000000000000/", - "userTenantId": "00000000-0000-0000-0000-000000000000", "userName": "18a66f5f-dbdf-4c17-9dd7-1634712a9cbe", - "upn": null}, "lastModifiedUtc": "2024-01-12T07:52:10.1905954+00:00", "duration": - "00:00:04.0496282", "cancelationReason": null, "currentAttemptId": 1, "runId": - "batch_run_name", "parentRunId": null, "experimentId": "b1e733a1-2a5f-4c17-bc34-4d66d2858228", - "status": "Completed", "startTimeUtc": "2024-01-12T07:52:07.2312941+00:00", - "endTimeUtc": "2024-01-12T07:52:11.2809223+00:00", "scheduleId": null, "displayName": + string: '{"runMetadata": {"runNumber": 1715328791, "rootRunId": "batch_run_name", + "createdUtc": "2024-05-10T08:13:11.283085+00:00", "createdBy": {"userObjectId": + "00000000-0000-0000-0000-000000000000", "userPuId": "10032000324F7449", "userIdp": + null, "userAltSecId": null, "userIss": "https://sts.windows.net/00000000-0000-0000-0000-000000000000/", + "userTenantId": "00000000-0000-0000-0000-000000000000", "userName": "Honglin + Du", "upn": null}, "userId": "00000000-0000-0000-0000-000000000000", "token": + null, "tokenExpiryTimeUtc": null, "error": null, "warnings": null, "revision": + 7, "statusRevision": 3, "runUuid": "4f4ecb17-5b95-4140-9cd6-159027d24856", + "parentRunUuid": null, "rootRunUuid": "4f4ecb17-5b95-4140-9cd6-159027d24856", + "lastStartTimeUtc": null, "currentComputeTime": null, "computeDuration": "00:00:11.1571743", + "effectiveStartTimeUtc": null, "lastModifiedBy": {"userObjectId": "00000000-0000-0000-0000-000000000000", + "userPuId": "10032000324F7449", "userIdp": null, "userAltSecId": null, "userIss": + "https://sts.windows.net/00000000-0000-0000-0000-000000000000/", "userTenantId": + "00000000-0000-0000-0000-000000000000", "userName": "Honglin Du", "upn": "username@microsoft.com"}, + "lastModifiedUtc": "2024-05-10T08:14:14.0032237+00:00", "duration": "00:00:11.1571743", + "cancelationReason": null, "currentAttemptId": 1, "runId": "batch_run_name", + "parentRunId": null, "experimentId": "b1e733a1-2a5f-4c17-bc34-4d66d2858228", + "status": "Completed", "startTimeUtc": "2024-05-10T08:14:03.1715912+00:00", + "endTimeUtc": "2024-05-10T08:14:14.3287655+00:00", "scheduleId": null, "displayName": "sdk-cli-test-fixture-batch-run-without-llm", "name": null, "dataContainerId": "dcid.batch_run_name", "description": null, "hidden": false, "runType": "azureml.promptflow.FlowRun", "runTypeV2": {"orchestrator": null, "traits": [], "attribution": "PromptFlow", - "computeType": "AmlcDsi"}, "properties": {"azureml.promptflow.runtime_name": - "test-runtime-ci", "azureml.promptflow.runtime_version": "20231204.v4", "azureml.promptflow.definition_file_name": - "flow.dag.yaml", "azureml.promptflow.session_id": "bee356189f7e7f18671a79369c78df4cfb1bbd0c99069074", - "azureml.promptflow.flow_lineage_id": "f7ee724d91e4f4a7501bdc0b66995bc8b57f86b3a526fa2a81c34ebcccbbd912", - "azureml.promptflow.flow_definition_datastore_name": "workspaceblobstore", - "azureml.promptflow.flow_definition_blob_path": "LocalUpload/36774154bc3ecde4aa21054b3052221f/hello-world/flow.dag.yaml", - "azureml.promptflow.input_data": "azureml://datastores/workspaceblobstore/paths/LocalUpload/74c11bba717480b2d6b04b8e746d09d7/webClassification3.jsonl", - "azureml.promptflow.inputs_mapping": "{\"name\":\"${data.url}\"}", "_azureml.evaluation_run": - "promptflow.BatchRun", "azureml.promptflow.snapshot_id": "38053316-7591-4ac8-b718-d76b16b48bbe", - "azureml.promptflow.total_tokens": "0", "_azureml.evaluate_artifacts": "[{\"path\": - \"instance_results.jsonl\", \"type\": \"table\"}]"}, "parameters": {}, "actionUris": + "computeType": null}, "properties": {"azureml.promptflow.inputs_mapping": + "{\"name\":\"${data.url}\"}", "azureml.promptflow.runtime_name": "automatic", + "azureml.promptflow.disable_trace": "false", "azureml.promptflow.input_data": + "azureml://datastores/workspaceblobstore/paths/LocalUpload/74c11bba717480b2d6b04b8e746d09d7/webClassification3.jsonl", + "azureml.promptflow.session_id": "4a6867a952826a2a51729e6bb4bf21ac65b46eec3ae5ed10", + "azureml.promptflow.definition_file_name": "flow.dag.yaml", "azureml.promptflow.flow_lineage_id": + "f19900c5ab3df0dabfcf927fec52c0305ae6271bf36dc3d4935bb9137e86252e", "azureml.promptflow.flow_definition_datastore_name": + "workspaceblobstore", "azureml.promptflow.flow_definition_blob_path": "LocalUpload/cb521f3f2818474c08a3235dc45d5d4e/hello-world/flow.dag.yaml", + "_azureml.evaluation_run": "promptflow.BatchRun", "azureml.promptflow.snapshot_id": + "b98f6388-29a7-47f6-93a9-09b2e2b55cb9", "azureml.promptflow.runtime_version": + "20240429.v8", "_azureml.evaluate_artifacts": "[{\"path\": \"instance_results.jsonl\", + \"type\": \"table\"}]", "azureml.promptflow.total_tokens": "0", "azureml.promptflow.completion_tokens": + "0", "azureml.promptflow.prompt_tokens": "0"}, "parameters": {}, "actionUris": {}, "scriptName": null, "target": null, "uniqueChildRunComputeTargets": [], "tags": {}, "settings": {}, "services": {}, "inputDatasets": [], "outputDatasets": [], "runDefinition": null, "jobSpecification": null, "primaryMetricName": @@ -2734,11 +5963,11 @@ interactions: connection: - keep-alive content-length: - - '4649' + - '4659' content-type: - application/json; charset=utf-8 strict-transport-security: - - max-age=15724800; includeSubDomains; preload + - max-age=31536000; includeSubDomains; preload transfer-encoding: - chunked vary: @@ -2746,7 +5975,7 @@ interactions: x-content-type-options: - nosniff x-request-time: - - '0.052' + - '0.084' status: code: 200 message: OK @@ -2757,7 +5986,7 @@ interactions: Accept: - '*/*' Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive Content-Length: @@ -2770,47 +5999,49 @@ interactions: uri: https://eastus.api.azureml.ms/history/v1.0/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/rundata response: body: - string: '{"runMetadata": {"runNumber": 1705045961, "rootRunId": "eval_run_name", - "createdUtc": "2024-01-12T07:52:41.9028361+00:00", "createdBy": {"userObjectId": - "00000000-0000-0000-0000-000000000000", "userPuId": null, "userIdp": "https://sts.windows.net/00000000-0000-0000-0000-000000000000/", - "userAltSecId": null, "userIss": "https://sts.windows.net/00000000-0000-0000-0000-000000000000/", - "userTenantId": "00000000-0000-0000-0000-000000000000", "userName": "4cbd0e2e-aae4-4099-b4ba-94d3a4910587", - "upn": null}, "userId": "00000000-0000-0000-0000-000000000000", "token": null, - "tokenExpiryTimeUtc": null, "error": null, "warnings": null, "revision": 6, - "statusRevision": 3, "runUuid": "b588472b-0935-433f-9bcb-b728c4f01254", "parentRunUuid": - null, "rootRunUuid": "b588472b-0935-433f-9bcb-b728c4f01254", "lastStartTimeUtc": - null, "currentComputeTime": null, "computeDuration": "00:00:04.3642942", "effectiveStartTimeUtc": - null, "lastModifiedBy": {"userObjectId": "00000000-0000-0000-0000-000000000000", - "userPuId": null, "userIdp": "https://sts.windows.net/00000000-0000-0000-0000-000000000000/", - "userAltSecId": null, "userIss": "https://sts.windows.net/00000000-0000-0000-0000-000000000000/", - "userTenantId": "00000000-0000-0000-0000-000000000000", "userName": "18a66f5f-dbdf-4c17-9dd7-1634712a9cbe", - "upn": null}, "lastModifiedUtc": "2024-01-12T07:53:02.5591186+00:00", "duration": - "00:00:04.3642942", "cancelationReason": null, "currentAttemptId": 1, "runId": - "eval_run_name", "parentRunId": null, "experimentId": "7bdec279-f99c-4ed3-b0b8-dd75698b8fd0", - "status": "Completed", "startTimeUtc": "2024-01-12T07:52:58.9604157+00:00", - "endTimeUtc": "2024-01-12T07:53:03.3247099+00:00", "scheduleId": null, "displayName": + string: '{"runMetadata": {"runNumber": 1715328880, "rootRunId": "eval_run_name", + "createdUtc": "2024-05-10T08:14:40.1870209+00:00", "createdBy": {"userObjectId": + "00000000-0000-0000-0000-000000000000", "userPuId": "10032000324F7449", "userIdp": + null, "userAltSecId": null, "userIss": "https://sts.windows.net/00000000-0000-0000-0000-000000000000/", + "userTenantId": "00000000-0000-0000-0000-000000000000", "userName": "Honglin + Du", "upn": null}, "userId": "00000000-0000-0000-0000-000000000000", "token": + null, "tokenExpiryTimeUtc": null, "error": null, "warnings": null, "revision": + 7, "statusRevision": 3, "runUuid": "8d69f704-ea7c-4b50-bfea-71cb2e560e4c", + "parentRunUuid": null, "rootRunUuid": "8d69f704-ea7c-4b50-bfea-71cb2e560e4c", + "lastStartTimeUtc": null, "currentComputeTime": null, "computeDuration": "00:00:12.8313797", + "effectiveStartTimeUtc": null, "lastModifiedBy": {"userObjectId": "00000000-0000-0000-0000-000000000000", + "userPuId": "10032000324F7449", "userIdp": null, "userAltSecId": null, "userIss": + "https://sts.windows.net/00000000-0000-0000-0000-000000000000/", "userTenantId": + "00000000-0000-0000-0000-000000000000", "userName": "Honglin Du", "upn": "username@microsoft.com"}, + "lastModifiedUtc": "2024-05-10T08:15:11.4462467+00:00", "duration": "00:00:12.8313797", + "cancelationReason": null, "currentAttemptId": 1, "runId": "eval_run_name", + "parentRunId": null, "experimentId": "7bdec279-f99c-4ed3-b0b8-dd75698b8fd0", + "status": "Completed", "startTimeUtc": "2024-05-10T08:14:58.915785+00:00", + "endTimeUtc": "2024-05-10T08:15:11.7471647+00:00", "scheduleId": null, "displayName": "sdk-cli-test-fixture-eval-run-without-llm", "name": null, "dataContainerId": "dcid.eval_run_name", "description": null, "hidden": false, "runType": "azureml.promptflow.FlowRun", "runTypeV2": {"orchestrator": null, "traits": [], "attribution": "PromptFlow", - "computeType": "AmlcDsi"}, "properties": {"azureml.promptflow.runtime_name": - "test-runtime-ci", "azureml.promptflow.runtime_version": "20231204.v4", "azureml.promptflow.definition_file_name": - "flow.dag.yaml", "azureml.promptflow.session_id": "f8e4236a4e78e7f7125bbd811ec7976cb330412723a530f8", - "azureml.promptflow.flow_lineage_id": "26c575d863a85371ef937096728441d8c68c3e737b5a1bfeae5ac8f3b9ccb048", - "azureml.promptflow.flow_definition_datastore_name": "workspaceblobstore", - "azureml.promptflow.flow_definition_blob_path": "LocalUpload/1aa3064d06f6170abbc488cc35c713b9/eval-classification-accuracy/flow.dag.yaml", - "azureml.promptflow.input_data": "azureml://datastores/workspaceblobstore/paths/LocalUpload/74c11bba717480b2d6b04b8e746d09d7/webClassification3.jsonl", - "azureml.promptflow.input_run_id": "batch_run_name", "azureml.promptflow.inputs_mapping": + "computeType": "AmlcDsi"}, "properties": {"azureml.promptflow.inputs_mapping": "{\"groundtruth\":\"${data.answer}\",\"prediction\":\"${run.outputs.result}\"}", + "azureml.promptflow.runtime_name": "hod-ci", "azureml.promptflow.disable_trace": + "false", "azureml.promptflow.input_data": "azureml://datastores/workspaceblobstore/paths/LocalUpload/74c11bba717480b2d6b04b8e746d09d7/webClassification3.jsonl", + "azureml.promptflow.input_run_id": "batch_run_name", "azureml.promptflow.variant_run_id": + "batch_run_name", "azureml.promptflow.definition_file_name": "flow.dag.yaml", + "azureml.promptflow.flow_lineage_id": "5b3e40bae2d04de70bc4932d0d088686baf0701ee5bb9914ee5fdd6e74816523", + "azureml.promptflow.flow_definition_datastore_name": "workspaceblobstore", + "azureml.promptflow.flow_definition_blob_path": "LocalUpload/60d5ced709bd5d8996cc46086ec16f49/eval-classification-accuracy/flow.dag.yaml", "_azureml.evaluation_run": "promptflow.BatchRun", "azureml.promptflow.snapshot_id": - "1885cd85-1969-4fca-8c21-d8826ed5d886", "azureml.promptflow.total_tokens": - "0", "_azureml.evaluate_artifacts": "[{\"path\": \"instance_results.jsonl\", - \"type\": \"table\"}]"}, "parameters": {}, "actionUris": {}, "scriptName": - null, "target": null, "uniqueChildRunComputeTargets": [], "tags": {}, "settings": - {}, "services": {}, "inputDatasets": [], "outputDatasets": [], "runDefinition": - null, "jobSpecification": null, "primaryMetricName": null, "createdFrom": - null, "cancelUri": null, "completeUri": null, "diagnosticsUri": null, "computeRequest": - null, "compute": null, "retainForLifetimeOfWorkspace": false, "queueingInfo": - null, "inputs": null, "outputs": {"debug_info": {"assetId": "azureml://locations/eastus/workspaces/00000/data/azureml_eval_run_name_output_data_debug_info/versions/1", + "d1dc26d0-4657-4c78-a208-e48207b3a47c", "azureml.promptflow.runtime_version": + "20240429.v8", "_azureml.evaluate_artifacts": "[{\"path\": \"instance_results.jsonl\", + \"type\": \"table\"}]", "azureml.promptflow.total_tokens": "0", "azureml.promptflow.completion_tokens": + "0", "azureml.promptflow.prompt_tokens": "0"}, "parameters": {}, "actionUris": + {}, "scriptName": null, "target": null, "uniqueChildRunComputeTargets": [], + "tags": {}, "settings": {}, "services": {}, "inputDatasets": [], "outputDatasets": + [], "runDefinition": null, "jobSpecification": null, "primaryMetricName": + null, "createdFrom": null, "cancelUri": null, "completeUri": null, "diagnosticsUri": + null, "computeRequest": null, "compute": null, "retainForLifetimeOfWorkspace": + false, "queueingInfo": null, "inputs": null, "outputs": {"debug_info": {"assetId": + "azureml://locations/eastus/workspaces/00000/data/azureml_eval_run_name_output_data_debug_info/versions/1", "type": "UriFolder"}, "flow_outputs": {"assetId": "azureml://locations/eastus/workspaces/00000/data/azureml_eval_run_name_output_data_flow_outputs/versions/1", "type": "UriFolder"}}}, "runDefinition": null, "jobSpecification": null, "systemSettings": null}' @@ -2818,11 +6049,11 @@ interactions: connection: - keep-alive content-length: - - '4797' + - '4801' content-type: - application/json; charset=utf-8 strict-transport-security: - - max-age=15724800; includeSubDomains; preload + - max-age=31536000; includeSubDomains; preload transfer-encoding: - chunked vary: @@ -2830,7 +6061,7 @@ interactions: x-content-type-options: - nosniff x-request-time: - - '0.045' + - '0.041' status: code: 200 message: OK @@ -2840,121 +6071,117 @@ interactions: Accept: - application/json Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive Content-Type: - application/json User-Agent: - - promptflow-sdk/0.0.1 azsdk-python-azuremachinelearningdesignerserviceclient/unknown - Python/3.10.13 (Windows-10-10.0.22631-SP0) + - promptflow-azure-sdk/1.9.0.dev0 azsdk-python-azuremachinelearningdesignerserviceclient/unknown + Python/3.9.19 (Windows-10-10.0.22631-SP0) method: GET uri: https://eastus.api.azureml.ms/flow/api/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/BulkRuns/batch_run_name/logContent response: body: - string: '"2024-01-12 07:51:54 +0000 49 promptflow-runtime INFO [batch_run_name] - Receiving v2 bulk run request f3011800-5f03-4911-b5a1-b1f3cf942c03: {\"flow_id\": + string: '"2024-05-10 08:13:51 +0000 24 promptflow-runtime INFO [batch_run_name] + Receiving v2 bulk run request 7c17e14f-087a-4ec9-9cc1-778e075880d4: {\"flow_id\": \"batch_run_name\", \"flow_run_id\": \"batch_run_name\", \"flow_source\": - {\"flow_source_type\": 1, \"flow_source_info\": {\"snapshot_id\": \"38053316-7591-4ac8-b718-d76b16b48bbe\"}, - \"flow_dag_file\": \"flow.dag.yaml\"}, \"log_path\": \"https://promptfloweast4063704120.blob.core.windows.net/azureml/ExperimentRun/dcid.batch_run_name/logs/azureml/executionlogs.txt?sv=2019-07-07&sr=b&sig=**data_scrubbed**&skoid=55b92eba-d7c7-4afd-ab76-7bb1cd345283&sktid=00000000-0000-0000-0000-000000000000&skt=2024-01-12T07%3A41%3A49Z&ske=2024-01-13T15%3A51%3A49Z&sks=b&skv=2019-07-07&st=2024-01-12T07%3A41%3A53Z&se=2024-01-12T15%3A51%3A53Z&sp=rcw\", + {\"flow_source_type\": 1, \"flow_source_info\": {\"snapshot_id\": \"b98f6388-29a7-47f6-93a9-09b2e2b55cb9\"}, + \"flow_dag_file\": \"flow.dag.yaml\"}, \"log_path\": \"https://promptfloweast4063704120.blob.core.windows.net/azureml/ExperimentRun/dcid.batch_run_name/logs/azureml/executionlogs.txt?sv=2019-07-07&sr=b&sig=**data_scrubbed**&skoid=55b92eba-d7c7-4afd-ab76-7bb1cd345283&sktid=00000000-0000-0000-0000-000000000000&skt=2024-05-10T06%3A08%3A20Z&ske=2024-05-11T14%3A18%3A20Z&sks=b&skv=2019-07-07&st=2024-05-10T08%3A03%3A49Z&se=2024-05-10T16%3A13%3A49Z&sp=rcw\", \"app_insights_instrumentation_key\": \"InstrumentationKey=**data_scrubbed**;IngestionEndpoint=https://eastus-6.in.applicationinsights.azure.com/;LiveEndpoint=https://eastus.livediagnostics.monitor.azure.com/\", - \"data_inputs\": {\"data\": \"azureml://datastores/workspaceblobstore/paths/LocalUpload/74c11bba717480b2d6b04b8e746d09d7/webClassification3.jsonl\"}, + \"flow_name\": \"sdk-cli-test-fixture-batch-run-without-llm\", \"batch_timeout_sec\": + 36000, \"data_inputs\": {\"data\": \"azureml://datastores/workspaceblobstore/paths/LocalUpload/74c11bba717480b2d6b04b8e746d09d7/webClassification3.jsonl\"}, \"inputs_mapping\": {\"name\": \"${data.url}\"}, \"azure_storage_setting\": {\"azure_storage_mode\": 1, \"storage_account_name\": \"promptfloweast4063704120\", \"blob_container_name\": \"azureml-blobstore-3e123da1-f9a5-4c91-9234-8d9ffbb39ff5\", \"flow_artifacts_root_path\": \"promptflow/PromptFlowArtifacts/batch_run_name\", - \"blob_container_sas_token\": \"?sv=2019-07-07&sr=c&sig=**data_scrubbed**&skoid=55b92eba-d7c7-4afd-ab76-7bb1cd345283&sktid=00000000-0000-0000-0000-000000000000&skt=2024-01-12T07%3A51%3A54Z&ske=2024-01-19T07%3A51%3A54Z&sks=b&skv=2019-07-07&se=2024-01-19T07%3A51%3A54Z&sp=racwl\", - \"output_datastore_name\": \"workspaceblobstore\"}}\n2024-01-12 07:51:54 +0000 49 - promptflow-runtime INFO Runtime version: 20231204.v4. PromptFlow version: - 1.2.0rc1\n2024-01-12 07:51:54 +0000 49 promptflow-runtime INFO Updating - batch_run_name to Status.Preparing...\n2024-01-12 07:51:55 +0000 49 promptflow-runtime - INFO Downloading snapshot to /mnt/host/service/app/39649/requests/batch_run_name\n2024-01-12 - 07:51:55 +0000 49 promptflow-runtime INFO Get snapshot sas url for - 38053316-7591-4ac8-b718-d76b16b48bbe...\n2024-01-12 07:52:01 +0000 49 - promptflow-runtime INFO Downloading snapshot 38053316-7591-4ac8-b718-d76b16b48bbe - from uri https://promptfloweast4063704120.blob.core.windows.net/snapshotzips/promptflow-eastus:3e123da1-f9a5-4c91-9234-8d9ffbb39ff5:snapshotzip/38053316-7591-4ac8-b718-d76b16b48bbe.zip...\n2024-01-12 - 07:52:01 +0000 49 promptflow-runtime INFO Downloaded file /mnt/host/service/app/39649/requests/batch_run_name/38053316-7591-4ac8-b718-d76b16b48bbe.zip - with size 495 for snapshot 38053316-7591-4ac8-b718-d76b16b48bbe.\n2024-01-12 - 07:52:01 +0000 49 promptflow-runtime INFO Download snapshot 38053316-7591-4ac8-b718-d76b16b48bbe - completed.\n2024-01-12 07:52:01 +0000 49 promptflow-runtime INFO Successfully - download snapshot to /mnt/host/service/app/39649/requests/batch_run_name\n2024-01-12 - 07:52:01 +0000 49 promptflow-runtime INFO About to execute a python - flow.\n2024-01-12 07:52:01 +0000 49 promptflow-runtime INFO Use spawn - method to start child process.\n2024-01-12 07:52:01 +0000 49 promptflow-runtime - INFO Starting to check process 2809 status for run batch_run_name\n2024-01-12 - 07:52:01 +0000 49 promptflow-runtime INFO Start checking run status - for run batch_run_name\n2024-01-12 07:52:05 +0000 2809 promptflow-runtime - INFO [49--2809] Start processing flowV2......\n2024-01-12 07:52:05 +0000 2809 - promptflow-runtime INFO Runtime version: 20231204.v4. PromptFlow version: - 1.2.0rc1\n2024-01-12 07:52:05 +0000 2809 promptflow-runtime INFO Setting - mlflow tracking uri...\n2024-01-12 07:52:05 +0000 2809 promptflow-runtime - INFO Validating ''AzureML Data Scientist'' user authentication...\n2024-01-12 - 07:52:06 +0000 2809 promptflow-runtime INFO Successfully validated - ''AzureML Data Scientist'' user authentication.\n2024-01-12 07:52:06 +0000 2809 - promptflow-runtime INFO Using AzureMLRunStorageV2\n2024-01-12 07:52:06 - +0000 2809 promptflow-runtime INFO Setting mlflow tracking uri to ''azureml://eastus.api.azureml.ms/mlflow/v1.0/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/promptflow-eastus''\n2024-01-12 - 07:52:06 +0000 2809 promptflow-runtime INFO Initialized blob service - client for AzureMLRunTracker.\n2024-01-12 07:52:06 +0000 2809 promptflow-runtime - INFO Setting mlflow tracking uri to ''azureml://eastus.api.azureml.ms/mlflow/v1.0/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/promptflow-eastus''\n2024-01-12 - 07:52:06 +0000 2809 promptflow-runtime INFO Resolve data from url finished - in 0.4533178601413965 seconds\n2024-01-12 07:52:06 +0000 2809 promptflow-runtime - INFO Starting the aml run ''batch_run_name''...\n2024-01-12 07:52:07 +0000 2809 - execution.bulk INFO Using fork, process count: 3\n2024-01-12 07:52:07 - +0000 2850 execution.bulk INFO Process 2850 started.\n2024-01-12 - 07:52:07 +0000 2856 execution.bulk INFO Process 2856 started.\n2024-01-12 - 07:52:07 +0000 2809 execution.bulk INFO Process name: ForkProcess-32:2, - Process id: 2850, Line number: 0 start execution.\n2024-01-12 07:52:07 +0000 2809 - execution.bulk INFO Process name: ForkProcess-32:3, Process id: 2856, - Line number: 1 start execution.\n2024-01-12 07:52:07 +0000 2809 execution.bulk INFO Process - name: ForkProcess-32:2, Process id: 2850, Line number: 0 completed.\n2024-01-12 - 07:52:07 +0000 2809 execution.bulk INFO Finished 1 / 3 lines.\n2024-01-12 - 07:52:07 +0000 2809 execution.bulk INFO Average execution time - for completed lines: 0.2 seconds. Estimated time for incomplete lines: 0.4 - seconds.\n2024-01-12 07:52:07 +0000 2861 execution.bulk INFO Process - 2861 started.\n2024-01-12 07:52:07 +0000 2809 execution.bulk INFO Process - name: ForkProcess-32:2, Process id: 2850, Line number: 2 start execution.\n2024-01-12 - 07:52:07 +0000 2809 execution.bulk INFO Process name: ForkProcess-32:3, - Process id: 2856, Line number: 1 completed.\n2024-01-12 07:52:07 +0000 2809 - execution.bulk INFO Finished 2 / 3 lines.\n2024-01-12 07:52:07 +0000 2809 - execution.bulk INFO Average execution time for completed lines: 0.14 - seconds. Estimated time for incomplete lines: 0.14 seconds.\n2024-01-12 07:52:07 - +0000 2809 execution.bulk INFO Process name: ForkProcess-32:2, - Process id: 2850, Line number: 2 completed.\n2024-01-12 07:52:07 +0000 2809 - execution.bulk INFO Finished 3 / 3 lines.\n2024-01-12 07:52:07 +0000 2809 - execution.bulk INFO Average execution time for completed lines: 0.11 - seconds. Estimated time for incomplete lines: 0.0 seconds.\n2024-01-12 07:52:10 - +0000 2809 execution.bulk INFO Upload status summary metrics for - run batch_run_name finished in 1.258488142862916 seconds\n2024-01-12 07:52:10 - +0000 2809 promptflow-runtime INFO Successfully write run properties - {\"azureml.promptflow.total_tokens\": 0, \"_azureml.evaluate_artifacts\": - \"[{\\\"path\\\": \\\"instance_results.jsonl\\\", \\\"type\\\": \\\"table\\\"}]\"} - with run id ''batch_run_name''\n2024-01-12 07:52:10 +0000 2809 execution.bulk INFO Upload - RH properties for run batch_run_name finished in 0.0779735017567873 seconds\n2024-01-12 - 07:52:10 +0000 2809 promptflow-runtime INFO Creating unregistered output - Asset for Run batch_run_name...\n2024-01-12 07:52:10 +0000 2809 promptflow-runtime - INFO Created debug_info Asset: azureml://locations/eastus/workspaces/00000/data/azureml_batch_run_name_output_data_debug_info/versions/1\n2024-01-12 - 07:52:10 +0000 2809 promptflow-runtime INFO Creating unregistered output - Asset for Run batch_run_name...\n2024-01-12 07:52:10 +0000 2809 promptflow-runtime - INFO Created flow_outputs output Asset: azureml://locations/eastus/workspaces/00000/data/azureml_batch_run_name_output_data_flow_outputs/versions/1\n2024-01-12 - 07:52:10 +0000 2809 promptflow-runtime INFO Creating Artifact for Run - batch_run_name...\n2024-01-12 07:52:11 +0000 2809 promptflow-runtime INFO Created - instance_results.jsonl Artifact.\n2024-01-12 07:52:11 +0000 2809 promptflow-runtime - INFO Patching batch_run_name...\n2024-01-12 07:52:11 +0000 2809 promptflow-runtime - INFO Ending the aml run ''batch_run_name'' with status ''Completed''...\n2024-01-12 - 07:52:12 +0000 49 promptflow-runtime INFO Process 2809 finished\n2024-01-12 - 07:52:12 +0000 49 promptflow-runtime INFO [49] Child process finished!\n2024-01-12 - 07:52:12 +0000 49 promptflow-runtime INFO [batch_run_name] End processing - bulk run\n2024-01-12 07:52:12 +0000 49 promptflow-runtime INFO Cleanup - working dir /mnt/host/service/app/39649/requests/batch_run_name for bulk run\n"' + \"blob_container_sas_token\": \"?sv=2019-07-07&sr=c&sig=**data_scrubbed**&skoid=55b92eba-d7c7-4afd-ab76-7bb1cd345283&sktid=00000000-0000-0000-0000-000000000000&skt=2024-05-10T08%3A13%3A50Z&ske=2024-05-17T08%3A13%3A50Z&sks=b&skv=2019-07-07&se=2024-05-17T08%3A13%3A50Z&sp=racwl\", + \"output_datastore_name\": \"workspaceblobstore\"}}\n2024-05-10 08:13:51 +0000 24 + promptflow-runtime INFO Runtime version: 20240429.v8. PromptFlow version: + 1.10.0rc11\n2024-05-10 08:13:51 +0000 24 promptflow-runtime INFO Updating + batch_run_name to Status.Preparing...\n2024-05-10 08:13:51 +0000 24 promptflow-runtime + INFO Downloading snapshot to /mnt/host/service/app/33681/requests/batch_run_name\n2024-05-10 + 08:13:51 +0000 24 promptflow-runtime INFO Get snapshot sas url for + b98f6388-29a7-47f6-93a9-09b2e2b55cb9.\n2024-05-10 08:13:52 +0000 24 promptflow-runtime + INFO Snapshot b98f6388-29a7-47f6-93a9-09b2e2b55cb9 contains 2 files.\n2024-05-10 + 08:13:52 +0000 24 promptflow-runtime INFO Download snapshot b98f6388-29a7-47f6-93a9-09b2e2b55cb9 + completed.\n2024-05-10 08:13:52 +0000 24 promptflow-runtime INFO Successfully + download snapshot to /mnt/host/service/app/33681/requests/batch_run_name\n2024-05-10 + 08:13:52 +0000 24 promptflow-runtime INFO About to execute a python + flow.\n2024-05-10 08:13:53 +0000 24 promptflow-runtime INFO Use spawn + method to start child process.\n2024-05-10 08:13:53 +0000 24 promptflow-runtime + INFO Starting to check process 138 status for run batch_run_name\n2024-05-10 + 08:13:53 +0000 24 promptflow-runtime INFO Start checking run status + for run batch_run_name\n2024-05-10 08:13:58 +0000 138 promptflow-runtime + INFO [24--138] Start processing flowV2......\n2024-05-10 08:13:58 +0000 138 + promptflow-runtime INFO Runtime version: 20240429.v8. PromptFlow version: + 1.10.0rc11\n2024-05-10 08:13:58 +0000 138 promptflow-runtime INFO Setting + mlflow tracking uri...\n2024-05-10 08:13:58 +0000 138 promptflow-runtime + INFO Validating ''AzureML Data Scientist'' user authentication...\n2024-05-10 + 08:13:58 +0000 138 promptflow-runtime INFO Successfully validated + ''AzureML Data Scientist'' user authentication.\n2024-05-10 08:13:58 +0000 138 + promptflow-runtime INFO Using AzureMLRunStorageV2\n2024-05-10 08:13:58 + +0000 138 promptflow-runtime INFO Setting mlflow tracking uri to ''azureml://eastus.api.azureml.ms/mlflow/v1.0/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/promptflow-eastus''\n2024-05-10 + 08:13:59 +0000 138 promptflow-runtime INFO Setting mlflow tracking + uri to ''azureml://eastus.api.azureml.ms/mlflow/v1.0/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/promptflow-eastus''\n2024-05-10 + 08:13:59 +0000 138 promptflow-runtime INFO Creating unregistered output + Asset for Run batch_run_name...\n2024-05-10 08:13:59 +0000 138 promptflow-runtime + INFO Created debug_info Asset: azureml://locations/eastus/workspaces/00000/data/azureml_batch_run_name_output_data_debug_info/versions/1\n2024-05-10 + 08:13:59 +0000 138 promptflow-runtime INFO Creating unregistered output + Asset for Run batch_run_name...\n2024-05-10 08:13:59 +0000 138 promptflow-runtime + INFO Created flow_outputs output Asset: azureml://locations/eastus/workspaces/00000/data/azureml_batch_run_name_output_data_flow_outputs/versions/1\n2024-05-10 + 08:13:59 +0000 138 promptflow-runtime INFO Patching batch_run_name...\n2024-05-10 + 08:14:02 +0000 138 promptflow-runtime INFO Resolve data from url finished + in 1.2249737120000646 seconds\n2024-05-10 08:14:02 +0000 138 promptflow-runtime + INFO Starting the aml run ''batch_run_name''...\n2024-05-10 08:14:03 +0000 138 + execution.bulk INFO The timeout for the batch run is 36000 seconds.\n2024-05-10 + 08:14:03 +0000 138 execution.bulk INFO Set process count to 3 + by taking the minimum value among the factors of {''default_worker_count'': + 4, ''row_count'': 3}.\n2024-05-10 08:14:08 +0000 138 execution.bulk INFO Process + name(ForkProcess-2:2:2)-Process id(287)-Line number(0) start execution.\n2024-05-10 + 08:14:08 +0000 138 execution.bulk INFO Process name(ForkProcess-2:2:3)-Process + id(296)-Line number(1) start execution.\n2024-05-10 08:14:08 +0000 138 + execution.bulk INFO Process name(ForkProcess-2:2:1)-Process id(280)-Line + number(2) start execution.\n2024-05-10 08:14:10 +0000 138 execution.bulk INFO Process + name(ForkProcess-2:2:3)-Process id(296)-Line number(1) completed.\n2024-05-10 + 08:14:10 +0000 138 execution.bulk INFO Process name(ForkProcess-2:2:1)-Process + id(280)-Line number(2) completed.\n2024-05-10 08:14:10 +0000 138 execution.bulk INFO Process + name(ForkProcess-2:2:2)-Process id(287)-Line number(0) completed.\n2024-05-10 + 08:14:10 +0000 138 execution.bulk INFO Finished 3 / 3 lines.\n2024-05-10 + 08:14:10 +0000 138 execution.bulk INFO Average execution time + for completed lines: 2.34 seconds. Estimated time for incomplete lines: 0.0 + seconds.\n2024-05-10 08:14:10 +0000 138 execution.bulk INFO The + thread monitoring the process [296-ForkProcess-2:2:3] will be terminated.\n2024-05-10 + 08:14:10 +0000 138 execution.bulk INFO The thread monitoring the + process [280-ForkProcess-2:2:1] will be terminated.\n2024-05-10 08:14:10 +0000 138 + execution.bulk INFO The thread monitoring the process [287-ForkProcess-2:2:2] + will be terminated.\n2024-05-10 08:14:10 +0000 280 execution.bulk INFO The + process [280] has received a terminate signal.\n2024-05-10 08:14:10 +0000 287 + execution.bulk INFO The process [287] has received a terminate signal.\n2024-05-10 + 08:14:10 +0000 296 execution.bulk INFO The process [296] has received + a terminate signal.\n2024-05-10 08:14:12 +0000 138 promptflow-runtime + INFO Post processing batch result...\n2024-05-10 08:14:13 +0000 138 + execution.bulk INFO Upload status summary metrics for run batch_run_name + finished in 1.2457163140002194 seconds\n2024-05-10 08:14:14 +0000 138 + promptflow-runtime INFO Successfully write run properties {\"_azureml.evaluate_artifacts\": + \"[{\\\"path\\\": \\\"instance_results.jsonl\\\", \\\"type\\\": \\\"table\\\"}]\", + \"azureml.promptflow.total_tokens\": 0, \"azureml.promptflow.completion_tokens\": + 0, \"azureml.promptflow.prompt_tokens\": 0} with run id ''batch_run_name''\n2024-05-10 + 08:14:14 +0000 138 execution.bulk INFO Upload RH properties for + run batch_run_name finished in 0.07404946799988466 seconds\n2024-05-10 08:14:14 + +0000 138 promptflow-runtime INFO Creating Artifact for Run batch_run_name...\n2024-05-10 + 08:14:14 +0000 138 promptflow-runtime INFO Created instance_results.jsonl + Artifact.\n2024-05-10 08:14:14 +0000 138 promptflow-runtime INFO Ending + the aml run ''batch_run_name'' with status ''Completed''...\n"' headers: connection: - keep-alive content-length: - - '9813' + - '9242' content-type: - application/json; charset=utf-8 strict-transport-security: - - max-age=15724800; includeSubDomains; preload + - max-age=31536000; includeSubDomains; preload transfer-encoding: - chunked vary: @@ -2962,7 +6189,7 @@ interactions: x-content-type-options: - nosniff x-request-time: - - '0.829' + - '0.514' status: code: 200 message: OK @@ -2972,128 +6199,123 @@ interactions: Accept: - application/json Accept-Encoding: - - gzip, deflate + - gzip, deflate, br Connection: - keep-alive Content-Type: - application/json User-Agent: - - promptflow-sdk/0.0.1 azsdk-python-azuremachinelearningdesignerserviceclient/unknown - Python/3.10.13 (Windows-10-10.0.22631-SP0) + - promptflow-azure-sdk/1.9.0.dev0 azsdk-python-azuremachinelearningdesignerserviceclient/unknown + Python/3.9.19 (Windows-10-10.0.22631-SP0) method: GET uri: https://eastus.api.azureml.ms/flow/api/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/BulkRuns/eval_run_name/logContent response: body: - string: '"2024-01-12 07:52:45 +0000 49 promptflow-runtime INFO [eval_run_name] - Receiving v2 bulk run request 9ada5207-1c07-4048-8721-8a87560f52bf: {\"flow_id\": + string: '"2024-05-10 08:14:44 +0000 71 promptflow-runtime INFO [eval_run_name] + Receiving v2 bulk run request eaa558a2-ce80-4f4e-9fdc-801469539b53: {\"flow_id\": \"eval_run_name\", \"flow_run_id\": \"eval_run_name\", \"flow_source\": {\"flow_source_type\": - 1, \"flow_source_info\": {\"snapshot_id\": \"1885cd85-1969-4fca-8c21-d8826ed5d886\"}, - \"flow_dag_file\": \"flow.dag.yaml\"}, \"log_path\": \"https://promptfloweast4063704120.blob.core.windows.net/azureml/ExperimentRun/dcid.eval_run_name/logs/azureml/executionlogs.txt?sv=2019-07-07&sr=b&sig=**data_scrubbed**&skoid=55b92eba-d7c7-4afd-ab76-7bb1cd345283&sktid=00000000-0000-0000-0000-000000000000&skt=2024-01-12T07%3A41%3A49Z&ske=2024-01-13T15%3A51%3A49Z&sks=b&skv=2019-07-07&st=2024-01-12T07%3A42%3A44Z&se=2024-01-12T15%3A52%3A44Z&sp=rcw\", + 1, \"flow_source_info\": {\"snapshot_id\": \"d1dc26d0-4657-4c78-a208-e48207b3a47c\"}, + \"flow_dag_file\": \"flow.dag.yaml\"}, \"log_path\": \"https://promptfloweast4063704120.blob.core.windows.net/azureml/ExperimentRun/dcid.eval_run_name/logs/azureml/executionlogs.txt?sv=2019-07-07&sr=b&sig=**data_scrubbed**&skoid=55b92eba-d7c7-4afd-ab76-7bb1cd345283&sktid=00000000-0000-0000-0000-000000000000&skt=2024-05-10T05%3A25%3A29Z&ske=2024-05-11T13%3A35%3A29Z&sks=b&skv=2019-07-07&st=2024-05-10T08%3A04%3A43Z&se=2024-05-10T16%3A14%3A43Z&sp=rcw\", \"app_insights_instrumentation_key\": \"InstrumentationKey=**data_scrubbed**;IngestionEndpoint=https://eastus-6.in.applicationinsights.azure.com/;LiveEndpoint=https://eastus.livediagnostics.monitor.azure.com/\", - \"data_inputs\": {\"data\": \"azureml://datastores/workspaceblobstore/paths/LocalUpload/74c11bba717480b2d6b04b8e746d09d7/webClassification3.jsonl\", + \"flow_name\": \"sdk-cli-test-fixture-eval-run-without-llm\", \"batch_timeout_sec\": + 36000, \"data_inputs\": {\"data\": \"azureml://datastores/workspaceblobstore/paths/LocalUpload/74c11bba717480b2d6b04b8e746d09d7/webClassification3.jsonl\", \"run.outputs\": \"azureml:/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/00000/data/azureml_batch_run_name_output_data_flow_outputs/versions/1\"}, \"inputs_mapping\": {\"groundtruth\": \"${data.answer}\", \"prediction\": \"${run.outputs.result}\"}, \"azure_storage_setting\": {\"azure_storage_mode\": 1, \"storage_account_name\": \"promptfloweast4063704120\", \"blob_container_name\": \"azureml-blobstore-3e123da1-f9a5-4c91-9234-8d9ffbb39ff5\", \"flow_artifacts_root_path\": \"promptflow/PromptFlowArtifacts/eval_run_name\", \"blob_container_sas_token\": - \"?sv=2019-07-07&sr=c&sig=**data_scrubbed**&skoid=55b92eba-d7c7-4afd-ab76-7bb1cd345283&sktid=00000000-0000-0000-0000-000000000000&skt=2024-01-12T07%3A52%3A45Z&ske=2024-01-19T07%3A52%3A45Z&sks=b&skv=2019-07-07&se=2024-01-19T07%3A52%3A45Z&sp=racwl\", - \"output_datastore_name\": \"workspaceblobstore\"}}\n2024-01-12 07:52:45 +0000 49 - promptflow-runtime INFO Runtime version: 20231204.v4. PromptFlow version: - 1.2.0rc1\n2024-01-12 07:52:45 +0000 49 promptflow-runtime INFO Updating - eval_run_name to Status.Preparing...\n2024-01-12 07:52:45 +0000 49 promptflow-runtime - INFO Downloading snapshot to /mnt/host/service/app/39649/requests/eval_run_name\n2024-01-12 - 07:52:45 +0000 49 promptflow-runtime INFO Get snapshot sas url for - 1885cd85-1969-4fca-8c21-d8826ed5d886...\n2024-01-12 07:52:52 +0000 49 - promptflow-runtime INFO Downloading snapshot 1885cd85-1969-4fca-8c21-d8826ed5d886 - from uri https://promptfloweast4063704120.blob.core.windows.net/snapshotzips/promptflow-eastus:3e123da1-f9a5-4c91-9234-8d9ffbb39ff5:snapshotzip/1885cd85-1969-4fca-8c21-d8826ed5d886.zip...\n2024-01-12 - 07:52:52 +0000 49 promptflow-runtime INFO Downloaded file /mnt/host/service/app/39649/requests/eval_run_name/1885cd85-1969-4fca-8c21-d8826ed5d886.zip - with size 1243 for snapshot 1885cd85-1969-4fca-8c21-d8826ed5d886.\n2024-01-12 - 07:52:52 +0000 49 promptflow-runtime INFO Download snapshot 1885cd85-1969-4fca-8c21-d8826ed5d886 - completed.\n2024-01-12 07:52:52 +0000 49 promptflow-runtime INFO Successfully - download snapshot to /mnt/host/service/app/39649/requests/eval_run_name\n2024-01-12 - 07:52:52 +0000 49 promptflow-runtime INFO About to execute a python - flow.\n2024-01-12 07:52:52 +0000 49 promptflow-runtime INFO Use spawn - method to start child process.\n2024-01-12 07:52:52 +0000 49 promptflow-runtime - INFO Starting to check process 2911 status for run eval_run_name\n2024-01-12 - 07:52:52 +0000 49 promptflow-runtime INFO Start checking run status - for run eval_run_name\n2024-01-12 07:52:56 +0000 2911 promptflow-runtime - INFO [49--2911] Start processing flowV2......\n2024-01-12 07:52:56 +0000 2911 - promptflow-runtime INFO Runtime version: 20231204.v4. PromptFlow version: - 1.2.0rc1\n2024-01-12 07:52:56 +0000 2911 promptflow-runtime INFO Setting - mlflow tracking uri...\n2024-01-12 07:52:56 +0000 2911 promptflow-runtime - INFO Validating ''AzureML Data Scientist'' user authentication...\n2024-01-12 - 07:52:56 +0000 2911 promptflow-runtime INFO Successfully validated - ''AzureML Data Scientist'' user authentication.\n2024-01-12 07:52:56 +0000 2911 - promptflow-runtime INFO Using AzureMLRunStorageV2\n2024-01-12 07:52:56 - +0000 2911 promptflow-runtime INFO Setting mlflow tracking uri to ''azureml://eastus.api.azureml.ms/mlflow/v1.0/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/promptflow-eastus''\n2024-01-12 - 07:52:56 +0000 2911 promptflow-runtime INFO Initialized blob service - client for AzureMLRunTracker.\n2024-01-12 07:52:57 +0000 2911 promptflow-runtime - INFO Setting mlflow tracking uri to ''azureml://eastus.api.azureml.ms/mlflow/v1.0/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/promptflow-eastus''\n2024-01-12 - 07:52:57 +0000 2911 promptflow-runtime INFO Resolve data from url finished - in 0.5648300694301724 seconds\n2024-01-12 07:52:58 +0000 2911 promptflow-runtime - INFO Resolve data from url finished in 0.6205331217497587 seconds\n2024-01-12 - 07:52:58 +0000 2911 promptflow-runtime INFO Starting the aml run ''eval_run_name''...\n2024-01-12 - 07:52:59 +0000 2911 execution.bulk INFO Using fork, process count: - 3\n2024-01-12 07:52:59 +0000 2954 execution.bulk INFO Process 2954 - started.\n2024-01-12 07:52:59 +0000 2958 execution.bulk INFO Process - 2958 started.\n2024-01-12 07:52:59 +0000 2911 execution.bulk INFO Process - name: ForkProcess-34:2, Process id: 2954, Line number: 0 start execution.\n2024-01-12 - 07:52:59 +0000 2911 execution.bulk INFO Process name: ForkProcess-34:3, - Process id: 2958, Line number: 1 start execution.\n2024-01-12 07:52:59 +0000 2911 - execution.bulk INFO Process name: ForkProcess-34:2, Process id: 2954, - Line number: 0 completed.\n2024-01-12 07:52:59 +0000 2911 execution.bulk INFO Finished - 1 / 3 lines.\n2024-01-12 07:52:59 +0000 2911 execution.bulk INFO Average - execution time for completed lines: 0.21 seconds. Estimated time for incomplete - lines: 0.42 seconds.\n2024-01-12 07:52:59 +0000 2911 execution.bulk INFO Process - name: ForkProcess-34:2, Process id: 2954, Line number: 2 start execution.\n2024-01-12 - 07:52:59 +0000 2911 execution.bulk INFO Process name: ForkProcess-34:3, - Process id: 2958, Line number: 1 completed.\n2024-01-12 07:52:59 +0000 2911 - execution.bulk INFO Finished 2 / 3 lines.\n2024-01-12 07:52:59 +0000 2961 - execution.bulk INFO Process 2961 started.\n2024-01-12 07:52:59 +0000 2911 - execution.bulk INFO Average execution time for completed lines: 0.13 - seconds. Estimated time for incomplete lines: 0.13 seconds.\n2024-01-12 07:52:59 - +0000 2911 execution.bulk INFO Process name: ForkProcess-34:2, - Process id: 2954, Line number: 2 completed.\n2024-01-12 07:52:59 +0000 2911 - execution.bulk INFO Finished 3 / 3 lines.\n2024-01-12 07:52:59 +0000 2911 - execution.bulk INFO Average execution time for completed lines: 0.12 - seconds. Estimated time for incomplete lines: 0.0 seconds.\n2024-01-12 07:53:00 - +0000 2911 execution.bulk INFO Executing aggregation nodes...\n2024-01-12 - 07:53:00 +0000 2911 execution.bulk INFO Finish executing aggregation - nodes.\n2024-01-12 07:53:02 +0000 2911 execution.bulk INFO Upload - status summary metrics for run eval_run_name finished in 1.4952670317143202 - seconds\n2024-01-12 07:53:02 +0000 2911 execution.bulk INFO Upload - metrics for run eval_run_name finished in 0.3417100487276912 seconds\n2024-01-12 - 07:53:02 +0000 2911 promptflow-runtime INFO Successfully write run - properties {\"azureml.promptflow.total_tokens\": 0, \"_azureml.evaluate_artifacts\": - \"[{\\\"path\\\": \\\"instance_results.jsonl\\\", \\\"type\\\": \\\"table\\\"}]\"} - with run id ''eval_run_name''\n2024-01-12 07:53:02 +0000 2911 execution.bulk INFO Upload - RH properties for run eval_run_name finished in 0.09185165446251631 seconds\n2024-01-12 - 07:53:02 +0000 2911 promptflow-runtime INFO Creating unregistered output - Asset for Run eval_run_name...\n2024-01-12 07:53:02 +0000 2911 promptflow-runtime - INFO Created debug_info Asset: azureml://locations/eastus/workspaces/00000/data/azureml_eval_run_name_output_data_debug_info/versions/1\n2024-01-12 - 07:53:02 +0000 2911 promptflow-runtime INFO Creating unregistered output - Asset for Run eval_run_name...\n2024-01-12 07:53:03 +0000 2911 promptflow-runtime - INFO Created flow_outputs output Asset: azureml://locations/eastus/workspaces/00000/data/azureml_eval_run_name_output_data_flow_outputs/versions/1\n2024-01-12 - 07:53:03 +0000 2911 promptflow-runtime INFO Creating Artifact for Run - eval_run_name...\n2024-01-12 07:53:03 +0000 2911 promptflow-runtime INFO Created - instance_results.jsonl Artifact.\n2024-01-12 07:53:03 +0000 2911 promptflow-runtime - INFO Patching eval_run_name...\n2024-01-12 07:53:03 +0000 2911 promptflow-runtime - INFO Ending the aml run ''eval_run_name'' with status ''Completed''...\n2024-01-12 - 07:53:04 +0000 49 promptflow-runtime INFO Process 2911 finished\n2024-01-12 - 07:53:04 +0000 49 promptflow-runtime INFO [49] Child process finished!\n2024-01-12 - 07:53:04 +0000 49 promptflow-runtime INFO [eval_run_name] End processing - bulk run\n2024-01-12 07:53:04 +0000 49 promptflow-runtime INFO Cleanup - working dir /mnt/host/service/app/39649/requests/eval_run_name for bulk run\n"' + \"?sv=2019-07-07&sr=c&sig=**data_scrubbed**&skoid=55b92eba-d7c7-4afd-ab76-7bb1cd345283&sktid=00000000-0000-0000-0000-000000000000&skt=2024-05-10T08%3A14%3A44Z&ske=2024-05-17T08%3A14%3A44Z&sks=b&skv=2019-07-07&se=2024-05-17T08%3A14%3A44Z&sp=racwl\", + \"output_datastore_name\": \"workspaceblobstore\"}, \"variant_run_id\": \"batch_run_name\"}\n2024-05-10 + 08:14:45 +0000 71 promptflow-runtime INFO Runtime version: 20240429.v8. + PromptFlow version: 1.10.0rc11\n2024-05-10 08:14:45 +0000 71 promptflow-runtime + INFO Updating eval_run_name to Status.Preparing...\n2024-05-10 08:14:45 + +0000 71 promptflow-runtime INFO Downloading snapshot to /mnt/host/service/app/45521/requests/eval_run_name\n2024-05-10 + 08:14:45 +0000 71 promptflow-runtime INFO Get snapshot sas url for + d1dc26d0-4657-4c78-a208-e48207b3a47c.\n2024-05-10 08:14:45 +0000 71 promptflow-runtime + INFO Snapshot d1dc26d0-4657-4c78-a208-e48207b3a47c contains 4 files.\n2024-05-10 + 08:14:45 +0000 71 promptflow-runtime INFO Download snapshot d1dc26d0-4657-4c78-a208-e48207b3a47c + completed.\n2024-05-10 08:14:45 +0000 71 promptflow-runtime INFO Successfully + download snapshot to /mnt/host/service/app/45521/requests/eval_run_name\n2024-05-10 + 08:14:45 +0000 71 promptflow-runtime INFO About to execute a python + flow.\n2024-05-10 08:14:45 +0000 71 promptflow-runtime INFO Use spawn + method to start child process.\n2024-05-10 08:14:45 +0000 71 promptflow-runtime + INFO Starting to check process 576 status for run eval_run_name\n2024-05-10 + 08:14:46 +0000 71 promptflow-runtime INFO Start checking run status + for run eval_run_name\n2024-05-10 08:14:52 +0000 576 promptflow-runtime + INFO [71--576] Start processing flowV2......\n2024-05-10 08:14:52 +0000 576 + promptflow-runtime INFO Runtime version: 20240429.v8. PromptFlow version: + 1.10.0rc11\n2024-05-10 08:14:52 +0000 576 promptflow-runtime INFO Setting + mlflow tracking uri...\n2024-05-10 08:14:52 +0000 576 promptflow-runtime + INFO Validating ''AzureML Data Scientist'' user authentication...\n2024-05-10 + 08:14:52 +0000 576 promptflow-runtime INFO Successfully validated + ''AzureML Data Scientist'' user authentication.\n2024-05-10 08:14:52 +0000 576 + promptflow-runtime INFO Using AzureMLRunStorageV2\n2024-05-10 08:14:52 + +0000 576 promptflow-runtime INFO Setting mlflow tracking uri to ''azureml://eastus.api.azureml.ms/mlflow/v1.0/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/promptflow-eastus''\n2024-05-10 + 08:14:53 +0000 576 promptflow-runtime INFO Setting mlflow tracking + uri to ''azureml://eastus.api.azureml.ms/mlflow/v1.0/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/00000/providers/Microsoft.MachineLearningServices/workspaces/promptflow-eastus''\n2024-05-10 + 08:14:53 +0000 576 promptflow-runtime INFO Creating unregistered output + Asset for Run eval_run_name...\n2024-05-10 08:14:53 +0000 576 promptflow-runtime + INFO Created debug_info Asset: azureml://locations/eastus/workspaces/00000/data/azureml_eval_run_name_output_data_debug_info/versions/1\n2024-05-10 + 08:14:53 +0000 576 promptflow-runtime INFO Creating unregistered output + Asset for Run eval_run_name...\n2024-05-10 08:14:53 +0000 576 promptflow-runtime + INFO Created flow_outputs output Asset: azureml://locations/eastus/workspaces/00000/data/azureml_eval_run_name_output_data_flow_outputs/versions/1\n2024-05-10 + 08:14:53 +0000 576 promptflow-runtime INFO Patching eval_run_name...\n2024-05-10 + 08:14:57 +0000 576 promptflow-runtime INFO Resolve data from url finished + in 1.1214789539999401 seconds\n2024-05-10 08:14:58 +0000 576 promptflow-runtime + INFO Resolve data from url finished in 1.5129429730000084 seconds\n2024-05-10 + 08:14:58 +0000 576 promptflow-runtime INFO Starting the aml run ''eval_run_name''...\n2024-05-10 + 08:14:59 +0000 576 execution.bulk INFO The timeout for the batch + run is 36000 seconds.\n2024-05-10 08:14:59 +0000 576 execution.bulk INFO Set + process count to 3 by taking the minimum value among the factors of {''default_worker_count'': + 4, ''row_count'': 3}.\n2024-05-10 08:15:06 +0000 576 execution.bulk INFO Process + name(ForkProcess-4:2:2)-Process id(728)-Line number(0) start execution.\n2024-05-10 + 08:15:06 +0000 576 execution.bulk INFO Process name(ForkProcess-4:2:1)-Process + id(718)-Line number(1) start execution.\n2024-05-10 08:15:06 +0000 576 + execution.bulk INFO Process name(ForkProcess-4:2:3)-Process id(736)-Line + number(2) start execution.\n2024-05-10 08:15:06 +0000 576 execution.bulk INFO Process + name(ForkProcess-4:2:1)-Process id(718)-Line number(1) completed.\n2024-05-10 + 08:15:07 +0000 576 execution.bulk INFO Process name(ForkProcess-4:2:3)-Process + id(736)-Line number(2) completed.\n2024-05-10 08:15:07 +0000 576 execution.bulk INFO Process + name(ForkProcess-4:2:2)-Process id(728)-Line number(0) completed.\n2024-05-10 + 08:15:07 +0000 576 execution.bulk INFO Finished 3 / 3 lines.\n2024-05-10 + 08:15:07 +0000 576 execution.bulk INFO Average execution time + for completed lines: 2.67 seconds. Estimated time for incomplete lines: 0.0 + seconds.\n2024-05-10 08:15:07 +0000 576 execution.bulk INFO The + thread monitoring the process [718-ForkProcess-4:2:1] will be terminated.\n2024-05-10 + 08:15:07 +0000 576 execution.bulk INFO The thread monitoring the + process [736-ForkProcess-4:2:3] will be terminated.\n2024-05-10 08:15:07 +0000 576 + execution.bulk INFO The thread monitoring the process [728-ForkProcess-4:2:2] + will be terminated.\n2024-05-10 08:15:07 +0000 718 execution.bulk INFO The + process [718] has received a terminate signal.\n2024-05-10 08:15:07 +0000 728 + execution.bulk INFO The process [728] has received a terminate signal.\n2024-05-10 + 08:15:07 +0000 736 execution.bulk INFO The process [736] has received + a terminate signal.\n2024-05-10 08:15:09 +0000 576 execution.bulk INFO Executing + aggregation node...\n2024-05-10 08:15:09 +0000 576 execution.bulk INFO Finish + executing aggregation node.\n2024-05-10 08:15:09 +0000 576 promptflow-runtime + INFO Post processing batch result...\n2024-05-10 08:15:11 +0000 576 + execution.bulk INFO Upload status summary metrics for run eval_run_name + finished in 1.462249165999765 seconds\n2024-05-10 08:15:11 +0000 576 execution.bulk INFO Upload + metrics for run eval_run_name finished in 0.36621387000013783 seconds\n2024-05-10 + 08:15:11 +0000 576 promptflow-runtime INFO Successfully write run + properties {\"_azureml.evaluate_artifacts\": \"[{\\\"path\\\": \\\"instance_results.jsonl\\\", + \\\"type\\\": \\\"table\\\"}]\", \"azureml.promptflow.total_tokens\": 0, \"azureml.promptflow.completion_tokens\": + 0, \"azureml.promptflow.prompt_tokens\": 0} with run id ''eval_run_name''\n2024-05-10 + 08:15:11 +0000 576 execution.bulk INFO Upload RH properties for + run eval_run_name finished in 0.09243804700008695 seconds\n2024-05-10 08:15:11 + +0000 576 promptflow-runtime INFO Creating Artifact for Run eval_run_name...\n2024-05-10 + 08:15:11 +0000 576 promptflow-runtime INFO Created instance_results.jsonl + Artifact.\n2024-05-10 08:15:11 +0000 576 promptflow-runtime INFO Ending + the aml run ''eval_run_name'' with status ''Completed''...\n"' headers: connection: - keep-alive content-length: - - '10617' + - '10100' content-type: - application/json; charset=utf-8 strict-transport-security: - - max-age=15724800; includeSubDomains; preload + - max-age=31536000; includeSubDomains; preload transfer-encoding: - chunked vary: @@ -3101,7 +6323,7 @@ interactions: x-content-type-options: - nosniff x-request-time: - - '0.503' + - '0.484' status: code: 200 message: OK diff --git a/src/promptflow/CHANGELOG.md b/src/promptflow/CHANGELOG.md index b4c261eb1a7..7f0ddc01981 100644 --- a/src/promptflow/CHANGELOG.md +++ b/src/promptflow/CHANGELOG.md @@ -2,6 +2,9 @@ ## v1.11.0 (Upcoming) +### Features Added +- [promptflow-devkit]: Upload local run details to cloud when trace destination is configured to cloud. + ### Improvements - [promptflow-devkit]: Interactive browser credential is excluded by default when using Azure AI connections, user could set `PF_NO_INTERACTIVE_LOGIN=False` to enable it. - [promptflow-devkit]: Add new `--engine` parameter for `pf flow serve`. This parameter can be used to switch python serving engine between `flask` and `fastapi`, currently it defaults to `flask`.