feat(integrations-service): Add new integrations & refactor integrations service #540
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Diwank Singh Tomer <[email protected]>
<!-- ELLIPSIS_HIDDEN --> > [!IMPORTANT] > Implement `ToolCallStep` and fix transition logic after `PromptStep` in workflow execution. > > - **ToolCallStep Implementation**: > - Implements `tool_call_step()` in `tool_call_step.py` to handle tool calls, including generating a call ID and validating tool names. > - Updates `STEP_TO_ACTIVITY` in `task_execution/__init__.py` to map `ToolCallStep` to `tool_call_step()`. > - **PromptStep Transition Fix**: > - Updates transition logic in `task_execution/__init__.py` to handle tool calls after a `PromptStep`. > - Removes unused code related to tool calls in `PromptStep` handling. > - **State Machine Update**: > - Updates `valid_transitions` in `tasks.py` to allow 'wait' to 'step' transition. > > <sup>This description was created by </sup>[<img alt="Ellipsis" src="https://img.shields.io/badge/Ellipsis-blue?color=175173">](https://www.ellipsis.dev?ref=julep-ai%2Fjulep&utm_source=github&utm_medium=referral)<sup> for 5ab9e3a. It will automatically update as commits are pushed.</sup> <!-- ELLIPSIS_HIDDEN -->
Co-authored-by: Julep Developers <[email protected]>
Signed-off-by: Diwank Singh Tomer <[email protected]> <!-- ELLIPSIS_HIDDEN --> ---- > [!IMPORTANT] > This PR updates the handling of integrations and systems by adding new models, updating workflows, and modifying session options, along with dependency updates and a migration script. > > - **Behavior**: > - Adds `execute_integration` function in `execute_integration.py` to handle integration tool calls. > - Updates `prompt_step.py` to handle unwrapping of prompt responses and tool call results. > - Modifies `tool_call_step.py` to handle tool calls using `Tool` model. > - **Models**: > - Adds `IntegrationDef` and `SystemDef` models in `Tools.py`. > - Updates `CreateToolRequest`, `PatchToolRequest`, `UpdateToolRequest`, and `Tool` to use `IntegrationDef` and `SystemDef`. > - Adds `forward_tool_results` option to session models in `Sessions.py`. > - **Workflow**: > - Updates `TaskExecutionWorkflow` in `task_execution/__init__.py` to handle integration tool calls. > - **Dependencies**: > - Updates `@typespec/*` dependencies in `package.json` to version `0.60.x`. > - **Migration**: > - Adds migration script `migrate_1727235852_add_forward_tool_calls_option.py` to add `forward_tool_calls` option to sessions. > > <sup>This description was created by </sup>[<img alt="Ellipsis" src="https://img.shields.io/badge/Ellipsis-blue?color=175173">](https://www.ellipsis.dev?ref=julep-ai%2Fjulep&utm_source=github&utm_medium=referral)<sup> for a49aa12. It will automatically update as commits are pushed.</sup> <!-- ELLIPSIS_HIDDEN --> --------- Signed-off-by: Diwank Singh Tomer <[email protected]>
Introduces a new integration service with FastAPI, Docker setup, and support for multiple integrations like DALL-E and Wikipedia. - Integration Service: + Adds integrations-service with Docker setup in docker-compose.yml and Dockerfile. + FastAPI application in web.py with routers for execution and integration management. - Models: + Defines models for DalleImageGenerator, DuckDuckGoSearch, HackerNews, Weather, and Wikipedia in models directory. + IntegrationExecutionRequest and IntegrationExecutionResponse for handling execution requests. - Utilities: + Implements execute_integration in execute_integration.py to handle integration execution logic. + Integration utilities for DALL-E, DuckDuckGo, Hacker News, Weather, and Wikipedia in utils/integrations. - Configuration: + Adds pyproject.toml for dependency management with Poetry. + Adds pytype.toml for type checking configuration.
#521) - **fix(typespec,agents-api): Rename integration providers** - **feat(agents-api,integrations): Working integrations for tool-call step** <!-- ELLIPSIS_HIDDEN --> ---- > [!IMPORTANT] > Add integration service handling for tool-call steps in agents API, including provider renames and workflow updates. > > - **Integrations**: > - Implement `run_integration_service` in `clients/integrations.py` to handle integration service calls. > - Update `execute_integration` in `activities/execute_integration.py` to use `run_integration_service` for non-dummy providers. > - Add `INTEGRATION_SERVICE_URL` to `.env.example`, `env.py`, and `docker-compose.yml`. > - **Renames**: > - Rename integration providers in `Tools.py` and `scalars.tsp` (e.g., `dall-e` to `dalle_image_generator`). > - **Workflows**: > - Update `task_execution/__init__.py` to handle integration tool calls using `execute_integration`. > - **Tests**: > - Add `integration_example.yaml` for sample tasks. > - Add tests for integration tool calls in `test_execution_workflow.py`. > - Add `patch_integration_service` in `utils.py` for mocking integration service calls. > > <sup>This description was created by </sup>[<img alt="Ellipsis" src="https://img.shields.io/badge/Ellipsis-blue?color=175173">](https://www.ellipsis.dev?ref=julep-ai%2Fjulep&utm_source=github&utm_medium=referral)<sup> for f13f8dd. It will automatically update as commits are pushed.</sup> <!-- ELLIPSIS_HIDDEN --> --------- Signed-off-by: Diwank Singh Tomer <[email protected]>
<!-- ELLIPSIS_HIDDEN --> > [!IMPORTANT] > Removes unnecessary transition call in `WaitForInputStep` case in `TaskExecutionWorkflow.run()` and updates tests accordingly. > > - **Behavior**: > - Removes `await transition(context, type="wait", output=output)` from `WaitForInputStep` case in `run()` in `__init__.py`. > - **Tests**: > - Updates `test_execution_workflow.py` to ensure workflow behaves correctly without the removed transition call. > > <sup>This description was created by </sup>[<img alt="Ellipsis" src="https://img.shields.io/badge/Ellipsis-blue?color=175173">](https://www.ellipsis.dev?ref=julep-ai%2Fjulep&utm_source=github&utm_medium=referral)<sup> for 6600cee. It will automatically update as commits are pushed.</sup> <!-- ELLIPSIS_HIDDEN -->
…d Upgrade to python 3.12 (#525) Signed-off-by: Diwank Singh Tomer <[email protected]> <!-- ELLIPSIS_HIDDEN --> ---- > [!IMPORTANT] > Upgrade to Python 3.12 and add support for reading setup arguments from metadata in `agents-api`. > > - **Python Version Upgrade**: > - Upgrade Python to 3.12 in `.tool-versions`, `Dockerfile`, `Dockerfile.worker`, `pyproject.toml`, and `pytype.toml`. > - **Setup Arguments from Metadata**: > - `execute_integration()` in `execute_integration.py` now reads setup arguments from metadata. > - `get_tool_args_from_metadata()` in `get_tool_args_from_metadata.py` updated to handle `arg_type` for "args" and "setup". > - **CI and Documentation**: > - Delete `generate-docs.yml` workflow. > - Update `lint-and-format.yml` to focus on `agents-api` and use Python 3.12. > - Update `pytype` version in `pyproject.toml`. > - Change `x-tool-parameters` to `x-integration-args` in `julep-concepts.md`. > - Adjust backup schedule in `docker-compose.yml` from 1h to 3h. > > <sup>This description was created by </sup>[<img alt="Ellipsis" src="https://img.shields.io/badge/Ellipsis-blue?color=175173">](https://www.ellipsis.dev?ref=julep-ai%2Fjulep&utm_source=github&utm_medium=referral)<sup> for b1c0615. It will automatically update as commits are pushed.</sup> <!-- ELLIPSIS_HIDDEN --> --------- Signed-off-by: Diwank Singh Tomer <[email protected]>
Introduce Docker Bake for building and pushing images with updated GitHub Actions workflows and Dockerfiles.
- **GitHub Actions Workflows**:
- Add `.github/workflows/dev-push-to-hub.yml` and `.github/workflows/main-push-to-hub.yml` for building and pushing images on `dev` and `main` branch merges.
- Remove `push-to-hub.yml` workflow.
- **Docker Build System**:
- Introduce `docker-bake.hcl` for defining Docker build targets and configurations.
- Update Dockerfiles in `agents-api`, `agents-api.worker`, `agents-api.migration` to use Docker Bake.
- **Documentation**:
- Update `CONTRIBUTING.md` with instructions for building Docker images using Docker Bake.
- **Miscellaneous**:
- Add submodules for `sdks/node-sdk` and `sdks/python-sdk`.
- **GitHub Actions**:
- Add `docker-bake-on-pr.yml` for baking images on pull requests.
- Rename `dev-push-to-hub.yml` to `bake-push-to-hub.yml` and `main-push-to-hub.yml` to `latest-push-to-hub.yml`.
- Update `bake-push-to-hub.yml` and `latest-push-to-hub.yml` to use `docker/bake-action@v5` and remove branch name and git sha setting steps.
Update GitHub Actions workflow to generate changelog from release notes on release event.
- **Workflow Changes**:
- Renames workflow from `Changelog CI` to `Changelog on release` in `.github/workflows/changelog-ci.yml`.
- Changes trigger from `pull_request` to `release` with type `published`.
- **Actions**:
- Updates `actions/checkout` from `v2` to `v4` with `fetch-depth: 0` and `ref: dev`.
- Replaces `saadmk11/[email protected]` with `rhysd/changelog-from-release/action@v3` to generate changelog from release notes.
Co-authored-by: creatorrr <[email protected]>
Enhance workflows to bake images on release and generate changelogs from release notes.
- **Workflows**:
- Update `.github/workflows/bake-push-to-hub.yml` to trigger on `release` events in addition to `push` to `dev` and tags `v*`.
- Update `.github/workflows/changelog-ci.yml` to trigger on `release` events and generate changelog from release notes using `rhysd/changelog-from-release/action@v3`.
…itch to synchronous methods
![ellipsis-dev[bot]](https://avatars.githubusercontent.com/in/64358?s=60&v=4){kind=small}
There was a problem hiding this comment.
❌ Changes requested. Reviewed everything up to c1828e6 in 54 seconds
More details
- Looked at
1247lines of code in30files - Skipped
1files when reviewing. - Skipped posting
2drafted comments based on config settings.
1. agents-api/agents_api/autogen/Tools.py:308
- Draft comment:
Duplicate fields forfunction,integration, andsystem. Remove the duplicates to avoid confusion and potential bugs. - Reason this comment was not posted:
Marked as duplicate.
2. agents-api/agents_api/autogen/Tools.py:355
- Draft comment:
Duplicate fields forfunction,integration, andsystem. Remove the duplicates to avoid confusion and potential bugs. - Reason this comment was not posted:
Marked as duplicate.
Workflow ID: wflow_cVJlQQML7p1fag7N
Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.
| @@ -19,7 +19,7 @@ async def run_integration_service( | |||
| slug = f"{provider}/{method}" if method else provider | |||
| url = f"{integration_service_url}/execute/{slug}" | |||
|
|
|||
| setup = setup or {} | |||
| setup = setup or None | |||
There was a problem hiding this comment.
Redundant assignment: setup = setup or None is unnecessary since setup is already None by default. Remove this line to simplify the code.
There was a problem hiding this comment.
👍 Looks good to me! Incremental review on a6f7e6a in 17 seconds
More details
- Looked at
110lines of code in1files - Skipped
0files when reviewing. - Skipped posting
1drafted comments based on config settings.
1. agents-api/agents_api/autogen/Tools.py:44
- Draft comment:
Duplicate field definitions found inCreateToolRequest,Tool, andUpdateToolRequestclasses. Remove the duplicates to avoid confusion and redundancy. - Reason this comment was not posted:
Comment looked like it was already resolved.
Workflow ID: wflow_qoKWf2cycoRqb2yS
You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Important
Add new integrations for Brave, BrowserBase, and Spider, and refactor the integration service to support these changes.
brave,browserbase, andspiderinproviders.py.wikipedia,weather,hacker_news.BraveSearchSetup,BraveSearchArguments,BraveSearchOutputinbrave.py.BrowserBaseSetup,BrowserBaseLoadArguments,BrowserBaseLoadOutputinbrowserbase.py.SpiderSetup,SpiderFetchArguments,SpiderFetchOutputinspider.py.ExecutionRequest,ExecutionResponseinexecution.pyto include new providers.get_integrationinget_integration.py.get_integration_toolinget_integration_tool.py.executeinexecute.pyto handle new providers and methods.execute_integrationinexecute_integration.pyto dynamically load and execute provider methods.integrations/utils/integrations/.pyproject.tomlto include new dependencies:spider-client,browserbase.scalars.tspandmodels.tspto include new integration providers.This description was created by
for a6f7e6a. It will automatically update as commits are pushed.