Push data to SFTP

[visch]

Not sure where the right place for this conversation is.

https://meltano.slack.com/archives/C013EKWA2Q1/p1619166461259200 was a conversation a while back in the Meltano Slack about this.

Use case: Push CSVs to SFTP sites.

There's other potential targets here:

1. FTP
2. FTPS
3. SFTP
4. CURL Posting of a file to some endpoint

After the file is created we want to do something things with it potentially.

Few issues with this today:

1. File name has a timestamp so scripting this is a bit painful
2. Have to write a separate script with separate binaries which pulls me out of the singer ecosystem to get this done, not ideal

[hsyyid]

Not sure pushing to SFTP should be part of the target-csv, should it? I think it's worth getting more opinions from the larger Meltano community on this

[visch]

Not sure pushing to SFTP should be part of the target-csv, should it? I think it's worth getting more opinions from the larger Meltano community on this

Definitely agree with you. I'll start a thread

[visch]

https://meltano.slack.com/archives/C01V8L0NZC1/p1626475132026300

[visch]

I think FS support https://docs.pyfilesystem.org/en/latest/ with a standard config interface (shared module for other file related taps/targets) would be a great way to go here.

Biggest question to me is what do you do about dependencies.

Do you pull in all possible fs target dependencies that are supported or not. Pulling them all in at first seems like an ok approach
Does dynamically pulling down the dependencies make any sense (yuck and scary)?