[Core feature] Flyte Agent Ecosystem

[pingsutw]

Motivation: Why do you think this is important?

Create this issue to track all the PRs and issues related to the Flyte agent.

Goal: What should the final outcome look like, ideally?

More agents, more reliable, easy to use and deploy.

Describe alternatives you've considered

NA

Propose: Link/Inline OR Additional context

Todo:

• Databricks Agent
• Snowflake Agent
• Memverge Agent
• File sensor
• MMCloud Agent
• Databricks Agent
• Sagemaker Agent
• ChatGPT Agent
• LainChain Agent (WIP)

PRs:

• Flyte Agent Helm Chart #3935
• Async file sensor flytekit#1790
• Render task template in the agent client flyteplugins#384
• Memverge plugin flytekit#1781
• Add agent ctrl-c handler to call the delete function. flytekit#1778
• Add agent ctrl-c handler to call the delete function. (Reupload) flytekit#1782
• Add base sensor flytekit#1773
• Airflow agent flytekit#1725
• Async Agent flytekit#1712
• [RFC] Flyte Agent #3558
• Use agent as name where it fits flyteplugins#381
• Support gRPC config for agent-service plugin flyteplugins#368
• Lazy load gRPC plugin flyteplugins#353
• External Plugin Service (grpc)  flyteplugins#330
• Register gRPC plugin after reading configmap flytepropeller#564
• Add service account for agent #3788
• Add agent to helm chart and single binary #3454
• Update agent endpoint in the single binary #3779
• Doc for flyte agent flytesnacks#971
• Prometheus metrics flytekit#1815
• Mark webapi task failure as retry limit exceeded flyteplugins#392
• Add supportTaskTypes for agentservice without write it in config twice. flyteplugins#398
• Create a FileOutput reader if the agent produce file output flyteplugins#391
• Sync Plugin Interface flyteplugins#402
• Only enqueue non-terminal tasks flytestdlib#164
• Update agent metric name flytekit#1835
• Bump flytestdlib version flyteplugins#404
• Deployment documentation for Bigquery agent #4034

Are you sure this issue hasn't been raised already?

• Yes

Have you read the Code of Conduct?

• Yes

[Future-Outlier]

Hi, just noticed that AWS Athena wasn't listed.
Can I implement it?