[Snowpark] Dedicated resource for files to support PUT/LIST/REMOVE on stage to allow deploy jar/zip/py & etc.

[DmitryMaletin]

Is your feature request related to a problem? Please describe.

Complex functions & stored procedures allow to list packages they relay on and those files should be available in Snowflake before function or sp can be created.

Describe the solution you'd like

Dedicated resource to move files.

Describe alternatives you've considered

as for now I can achieve desired result with snowsql provider.

[sfc-gh-swinkler]

Hello @DmitryMaletin . I am sorry, but I don't fully understand the request. Can you please provide an example of what SQL commands you are executing?

[DmitryMaletin]

Hi @sfc-gh-swinkler ,

The purpose is deploy python/scala based functions/procedures.
An example blow uses snowsql provider.

resource "terraform_data" "replacement" {
  input = "${sha1(filebase64("./snowpark/python_bundle.zip"))}"
}

resource "snowsql_exec" "python" {
  name = "./snowpark/python_bundle.zip"

  create {
    statements =  "PUT file:///snowpark/python_bundle.zip @MYDEPLOYMENT auto_compress = false overwrite = true;"
  }

  read {
    statements = "LIST @DMYDEPLOYMENT/python_bundle.zip"
  }

  update {
    statements =  "PUT file:///snowpark/python_bundle.zip @MYDEPLOYMENT auto_compress = false overwrite = true;"
  }

  delete {
    statements = "REMOVE @MYDEPLOYMENT/python_bundle.zip"
  }

  lifecycle {
        replace_triggered_by = [terraform_data.replacement]
  }
}


resource "snowflake_function" "python_test" {
  name     = "python_test"
  database = "MY_DB"
  schema   = "PUBLIC"
  arguments {
    name = "arg"
    type = "int"
  }
  comment             = "Example for Python language"
  return_type         = "NUMBER(38,0)"
  null_input_behavior = "CALLED ON NULL INPUT"
  return_behavior     = "VOLATILE"
  language            = "python"
  runtime_version     = "3.8"
  handler             = "python_bundle.hello"
  imports             = ["@MYDEPLOYMENT/python_bundle.zip"]
  statement           = ""
}

[imre-kerr-sb1]

We also need this. We are using Dmitry's workaround for now, but it's not ideal. Often, the snowflake_function resource will fail since the file isn't visible in the stage yet. This is despite adding a depends_on to wait until the PUT is complete.

[DmitryMaletin]

@imre-kerr-sb1

Sadly as for now it's not possible to set timeout directly on resource and retry is also not implemented although it's possible
https://developer.hashicorp.com/terraform/plugin/sdkv2/resources/retries-and-customizable-timeouts

One of the option (far to ideal) is to use https://registry.terraform.io/providers/hashicorp/time/latest/docs/resources/sleep