Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Data.post for sending a payload to a Web API #7700

Merged
merged 104 commits into from
Sep 19, 2023
Merged
Show file tree
Hide file tree
Changes from 67 commits
Commits
Show all changes
104 commits
Select commit Hold shift + click to select a range
4c9148a
wip
GregoryTravis Aug 23, 2023
3317801
some fetches
GregoryTravis Aug 24, 2023
371c6ff
two tests
GregoryTravis Aug 24, 2023
54aa399
Merge branch 'develop' into wip/gmt/5105-post
GregoryTravis Aug 25, 2023
f69120e
file upload
GregoryTravis Aug 25, 2023
bfaa0de
Merge branch 'develop' into wip/gmt/5105-post
GregoryTravis Aug 29, 2023
2a8ad25
Upload file with proper file publisher
GregoryTravis Aug 29, 2023
483d01a
Merge branch 'develop' into wip/gmt/5105-post
GregoryTravis Aug 30, 2023
5c9f6ce
post_file_as_form
GregoryTravis Aug 30, 2023
34dadb7
bad to_json
GregoryTravis Aug 30, 2023
b704c98
other request types
GregoryTravis Aug 30, 2023
5221bbb
cleanup
GregoryTravis Aug 30, 2023
588d36c
binary file post
GregoryTravis Aug 31, 2023
a831cde
Merge branch 'develop' into wip/gmt/5105-post
GregoryTravis Aug 31, 2023
3450018
old_
GregoryTravis Aug 31, 2023
5981d83
1
GregoryTravis Aug 31, 2023
bf6a807
cleanup
GregoryTravis Aug 31, 2023
2879c0f
json, error prop
GregoryTravis Sep 5, 2023
6c0d351
encoding
GregoryTravis Sep 5, 2023
78dceee
content type
GregoryTravis Sep 5, 2023
c63fd3a
Merge branch 'develop' into wip/gmt/5105-post
GregoryTravis Sep 5, 2023
ea30c88
binary file
GregoryTravis Sep 5, 2023
2929c4d
urlencoded and multipart forms
GregoryTravis Sep 5, 2023
661bd27
Merge branch 'develop' into wip/gmt/5105-post
GregoryTravis Sep 6, 2023
bd47c9f
custom headers
GregoryTravis Sep 6, 2023
73bd38a
skip auto parse
GregoryTravis Sep 6, 2023
597ddc9
auto conversion
GregoryTravis Sep 6, 2023
254665e
Merge branch 'develop' into wip/gmt/5105-post
GregoryTravis Sep 7, 2023
864f9d8
output context
GregoryTravis Sep 7, 2023
2495fd0
put, patch, unsupported method
GregoryTravis Sep 7, 2023
1f98836
format
GregoryTravis Sep 7, 2023
cf3df00
post_spec passes
GregoryTravis Sep 7, 2023
a1b9550
http_spec one test
GregoryTravis Sep 7, 2023
dacaf18
Merge branch 'develop' into wip/gmt/5105-post
GregoryTravis Sep 8, 2023
336bb17
rearranged
GregoryTravis Sep 8, 2023
6a2901e
from issue
GregoryTravis Sep 8, 2023
010412a
use self
GregoryTravis Sep 8, 2023
c02c888
fetch .body
GregoryTravis Sep 8, 2023
02a805b
Searcher
GregoryTravis Sep 8, 2023
f4608fb
get tests
GregoryTravis Sep 8, 2023
5ce7b4f
unknown host
GregoryTravis Sep 8, 2023
843d937
sd
GregoryTravis Sep 8, 2023
30e7752
wip
GregoryTravis Sep 8, 2023
8bad8e6
added id from clause
GregoryTravis Sep 8, 2023
33892fa
wip
GregoryTravis Sep 8, 2023
22c1022
type request ctor
GregoryTravis Sep 8, 2023
63431e2
wip
GregoryTravis Sep 8, 2023
5bab745
Merge branch 'develop' into wip/gmt/5105-post
GregoryTravis Sep 8, 2023
eba898c
cleanup
GregoryTravis Sep 11, 2023
df0a43f
Merge branch 'develop' into wip/gmt/5105-post
GregoryTravis Sep 11, 2023
b704dca
Searcher
GregoryTravis Sep 11, 2023
fe983b1
rm Form
GregoryTravis Sep 11, 2023
bb3f16d
bad scheme, nonexistent file
GregoryTravis Sep 11, 2023
96dc7aa
cleanup
GregoryTravis Sep 11, 2023
5de387e
cleanup
GregoryTravis Sep 11, 2023
9bb77ac
request doc
GregoryTravis Sep 11, 2023
8ce7624
post docs
GregoryTravis Sep 11, 2023
33afa40
direct json tests
GregoryTravis Sep 11, 2023
0af3e92
post docs
GregoryTravis Sep 11, 2023
3ba1050
docs
GregoryTravis Sep 11, 2023
e91f0db
unused
GregoryTravis Sep 11, 2023
517e0ca
changelog
GregoryTravis Sep 11, 2023
9ca5c2f
review
GregoryTravis Sep 12, 2023
b1cffbe
review
GregoryTravis Sep 12, 2023
695d2e4
send content encoding
GregoryTravis Sep 12, 2023
fbe2d3e
fix post default
GregoryTravis Sep 12, 2023
8604661
back to illegal_argument
GregoryTravis Sep 12, 2023
5fc2976
tests through Data
GregoryTravis Sep 12, 2023
16ff67d
output context only for writelike, do not remove response wrapper for…
GregoryTravis Sep 12, 2023
2ae1e06
move run_with_and_without_output to standard test lib
GregoryTravis Sep 12, 2023
2bc9706
unsupported method error contents
GregoryTravis Sep 12, 2023
45f1771
delete does not require body
GregoryTravis Sep 12, 2023
74d8ebf
review
GregoryTravis Sep 12, 2023
375f69a
cleanup
GregoryTravis Sep 12, 2023
2d899d8
clean up header handling
GregoryTravis Sep 12, 2023
272f334
Merge branch 'develop' into wip/gmt/5105-post
GregoryTravis Sep 13, 2023
2363fd1
to_json error
GregoryTravis Sep 13, 2023
0f79d55
delete fetch test
GregoryTravis Sep 13, 2023
e332bbf
content-type resolution
GregoryTravis Sep 13, 2023
d1bd938
Update distribution/lib/Standard/Base/0.0.0-dev/src/Data.enso
GregoryTravis Sep 13, 2023
7b7af1e
review
GregoryTravis Sep 13, 2023
a7ce3b9
review
GregoryTravis Sep 13, 2023
7afd717
review
GregoryTravis Sep 13, 2023
e827a41
Merge branch 'develop' into wip/gmt/5105-post
GregoryTravis Sep 14, 2023
186c766
body.to_file
GregoryTravis Sep 14, 2023
2073bf2
merge
GregoryTravis Sep 14, 2023
f705c90
fix File_Spec test failure
GregoryTravis Sep 14, 2023
6a4af63
Merge branch 'develop' into wip/gmt/5105-post
GregoryTravis Sep 15, 2023
8270b39
logging for ci failure
GregoryTravis Sep 15, 2023
d8641c4
more logging
GregoryTravis Sep 15, 2023
ae64134
head and options also pending
GregoryTravis Sep 18, 2023
005fcbd
wip
GregoryTravis Sep 18, 2023
bd51a44
custom header pending
GregoryTravis Sep 18, 2023
ef3bf36
typo
GregoryTravis Sep 18, 2023
aed211c
wip
GregoryTravis Sep 18, 2023
29223fd
custom header
GregoryTravis Sep 18, 2023
204dc76
all passing
GregoryTravis Sep 18, 2023
44ecb4d
cleanup
GregoryTravis Sep 18, 2023
25cf3ef
check method too
GregoryTravis Sep 18, 2023
6620fe0
remove go-httpbin docs
GregoryTravis Sep 18, 2023
243a66d
local running docs
GregoryTravis Sep 18, 2023
bb0e223
merge
GregoryTravis Sep 18, 2023
aa8750a
ci failures
GregoryTravis Sep 18, 2023
395e661
Fix failing tests.
jdunkerley Sep 19, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -569,6 +569,7 @@
`Column.map` and `Column.zip`.][7637]
- [Added `delete_rows` method to Database Table, changed the
`update_database_table` API into `update_rows`.][7709]
- [Added `Data.post` method to write to HTTP endpoints.][7700]

[debug-shortcuts]:
https://github.com/enso-org/enso/blob/develop/app/gui/docs/product/shortcuts.md#debug
Expand Down Expand Up @@ -806,6 +807,7 @@
[7704]: https://github.com/enso-org/enso/pull/7704
[7615]: https://github.com/enso-org/enso/pull/7615
[7637]: https://github.com/enso-org/enso/pull/7637
[7700]: https://github.com/enso-org/enso/pull/7700
[7709]: https://github.com/enso-org/enso/pull/7709

#### Enso Compiler
Expand Down
124 changes: 110 additions & 14 deletions distribution/lib/Standard/Base/0.0.0-dev/src/Data.enso
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import project.Network.URI.URI
import project.Nothing.Nothing
import project.System.File.File
from project.Data.Boolean import Boolean, False, True
from project.Network.HTTP.Request_Body import all
from project.System.File_Format import Auto_Detect, File_Format

## ALIAS load, open
Expand Down Expand Up @@ -150,24 +151,119 @@ list_directory directory name_filter=Nothing recursive=False =

Arguments:
- uri: The URI to fetch.
- method: The HTTP method to use. Defaults to `GET`.
- method: The HTTP method to use. Must be one of `HTTP_Method.Get`,
`HTTP_Method.Head`, `HTTP_Method.Delete`, `HTTP_Method.Options`.
Defaults to `HTTP_Method.Get`.

- headers: The headers to send with the request. Defaults to an empty vector.
- try_auto_parse: If successful should the body be attempted to be parsed to
an Enso native object.

> Example
Read from an HTTP endpoint.

import Standard.Base.Data
response = Data.fetch URL

> Example
Read from an HTTP endpoint and write the results to a file.

import Standard.Base.Data
file = enso_project.data / "spreadsheet.xls"
Data.fetch URL . to_file file
fetch : (URI | Text) -> HTTP_Method -> Vector (Header | Pair Text Text) -> Boolean -> Any
fetch uri method=HTTP_Method.Get headers=[] try_auto_parse=True =
parsed_headers = headers . map h-> case h of
_ : Vector -> Header.new (h.at 0) (h.at 1)
_ : Pair -> Header.new (h.at 0) (h.at 1)
_ : Header -> h
_ -> Error.throw (Illegal_Argument.Error "Invalid header type - all values must be Vector, Pair or Header (got "+(Meta.get_simple_type_name h)+").")

request = Request.new method uri parsed_headers
response = HTTP.new.request request

if response.code.is_success.not then Error.throw (Request_Error.Error "Status Code" ("Request failed with status code: " + response.code.to_text + ". " + response.body.decode_as_text)) else
if try_auto_parse.not then response else
response.decode if_unsupported=response . catch handler=(_->response)
fetch (uri:(URI | Text)) (method:HTTP_Method=HTTP_Method.Post) (headers:(Vector (Header | Pair Text Text))=[]) (try_auto_parse_response:Boolean=True) =
HTTP.fetch uri method headers try_auto_parse_response

## ALIAS upload, http post
GROUP Input
Writes the provided data to the provided URI. Returns the response,
parsing the body if the content-type is recognised. Returns an error if the
status code does not represent a successful response.

Arguments:
- uri: The URI to fetch.
- body: The data to write. See `Supported Body Types` below.
- method: The HTTP method to use. Must be one of `HTTP_Method.Post`,
`HTTP_Method.Put`, `HTTP_Method.Patch`. Defaults to `HTTP_Method.Post`.
- headers: The headers to send with the request. Defaults to an empty vector.
- try_auto_parse: If successful should the body be attempted to be parsed to
an Enso native object.

! Supported Body Types

- Request_Body.Text: Sends a text string, with optional encoding and content
type.
- Request_Body.Json: Sends an Enso object, after converting it to JSON.
- Request_Body.Binary: Sends a file.
- Request_Body.Form_Data: Sends a form encoded as key/value pairs. The keys
must be `Text`, and the values must be `Text` or `File`.
- Request_Body.Empty: Sends an empty body.

Additionally, the following types are allowed as the `body` parameter:

- Text: shorthand for `Request_Body.Text`.
- File: shorthand for `Request_Body.Binary`.
- Any Enso object: shorthand for `Request_Body.Json`.
GregoryTravis marked this conversation as resolved.
Show resolved Hide resolved

> Example
Write a text string to an HTTP endpoint.

import Standard.Base.Data
response = Data.post url_post (Request_Body.Text "hello world")

> Example
Write JSON to an HTTP endpoint.

import Standard.Base.Data
json = Json.parse '{"a": "asdf", "b": 123}'
response = Data.post url_post json

> Example
Write an Enso object to an HTTP endpoint.

import Standard.Base.Data
response = Data.post url_post (My_Type.Value 12)

> Example
Write a text string to an HTTP endpoint, with a specific text encoding.

import Standard.Base.Data
body = Request_Body.Text 'Hello World!' encoding=Encoding.utf_16_le
response = Data.post url_post body

> Example
Write a text string to an HTTP endpoint, with a specific content type.

import Standard.Base.Data
body = Request_Body.Text 'a,b,c\n' content_type="text/csv"
response = Data.post url_post body

> Example
Write the contents of a file to an HTTP endpoint.

import Standard.Base.Data
test_file = enso_project.data / "sample.png"
response = Data.post url_post (Request_Body.Binary test_file)

> Example
Write a multipart form to an HTTP endpoint.

import Standard.Base.Data
test_file = enso_project.data / "sample.png"
form_data = Map.from_vector [["key", "val"], ["a_file", test_file]]
response = Data.post url_post (Request_Body.Form_Data form_data)

> Example
Write a URL-encoded form to an HTTP endpoint.

import Standard.Base.Data
test_file = enso_project.data / "sample.txt"
form_data = Map.from_vector [["key", "val"], ["a_file", test_file]]
response = Data.post url_post (Request_Body.Form_Data form_data url_encoded=True)
post : (URI | Text) -> Request_Body -> HTTP_Method -> Vector (Header | Pair Text Text) -> Boolean -> Any
post (uri:(URI | Text)) (body:Request_Body) (method:HTTP_Method=HTTP_Method.Post) (headers:(Vector (Header | Pair Text Text))=[]) (try_auto_parse_response:Boolean=True) =
HTTP.fetch_or_post uri body method headers try_auto_parse_response

## ALIAS download, http get
GROUP Input
Expand Down
Loading