-
Notifications
You must be signed in to change notification settings - Fork 180
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
update docs with an example showing use with Revise.jl #587
Comments
Here is an alternative approach using
|
Suggestion: put |
Wow I was just dreading having to sit down and figure this out. Looking forwards to taking a look later. |
Just a word of warning that I couldn't get the examples here to work properly yet. They're close but AFAICT there's still a couple of problems:
|
Another thing I noticed is that serving with |
Ok, I have a working example (This needs the This incorporates Tim's suggestion to put The example page server. (Note that this may be pretty ugly, as I don't really know the HTTP API yet.) respond_revise.jl: using HTTP
using Base64
function respond_index(req)
headers = Dict("Content-Type" => "text/html")
# Enable this if you want the browser to poll the page.
# (This is pretty annoying, though; should probably use an event-based
# option like websockets instead.)
refresh_meta = "" # "<meta http-equiv=\"refresh\" content=\"1\">"
body = """
<html>
<head>
$refresh_meta
</head>
<body>
<h1>Hi</h1>
<p>
Some info
</body>
</html>
"""
return HTTP.Response(200, headers, body=body, request=req)
end
function redirect_index(req)
# Permanent redirect
return HTTP.Response(301, Dict("Location" => "index.html"))
end
function respond_info(req)
headers = Dict("Content-Type" => "text/json")
body = """
{"info": 1}
"""
return HTTP.Response(200, headers, body=body, request=req)
end
function respond_404(req)
headers = Dict("Content-Type" => "text/html")
return HTTP.Response(404, body="<html><body><h1>404: Not found</body></html>", request=req)
end
function respond_favicon(req)
headers = Dict("Content-Type" => "image/png")
favicon_data = base64decode("iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAIAAACQkWg2AAAABmJLR0QA/wD/AP+gvaeTAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAB3RJTUUH5AsSBxEnP13dzwAAAB1pVFh0Q29tbWVudAAAAAAAQ3JlYXRlZCB3aXRoIEdJTVBkLmUHAAACh0lEQVQoz2WST0sbURTF73szMXES0xiSClGSKEIE/wVXTRFBhG5t9wUh+RTdSLvJd7BgN+ouC4MWalOwaBdWrIqFIE6UqE2CMQ2a0WTmvTfvdmHqxrM+B+7vnkMQUUrZbDYppZqmwRMxxmq1mqZpPp8PACgAFAqFhYWFTCZTr9cBAKREIVAIQLRte3V1NZlMptPpWq0GAKqUslgsLi4uxmKxl4lEl6qa5bJ1fY223eH3217vj+3tjY2Ni4uLubm5QCCgUkqj0WgqlQoGg24hqrlc4+iI394ioqJprkgkHgq9mpkZn5jo6ekBAPLIYDcajc3N+u6ubLUoIYgoAaiiKKEQHR9/Ho8Hw+E2A6XUo2miWLw5PBTN5n6l8nF/P3tycmdZUghRKnVeXnZaluQcANSHV9imeafrwjC+FAoftra4lADwPRJJT087CWmVStbVVWdvLzgctB1otVi9zoX4enb24AaAvUrlr2kCgGRMWJaUsn2SlJJbFiWEEBJ0ux8b6Ha5uhwOAKgytp3P54+PhRDq/f392trawc7OIGNxQt6OjJhC/K5Wu12uVDze5XIVDWOrUvm8vv76/PzdwICq6/r8/Lyu6wGfLzk29mZw8P3UFLdtQohp2+u6ntH1P4bh8Xqf+XyqqqpOp9PpdAJA7ebm08FB1TBCbjcQ0jDNn+Vy/vq6yTkiDg0Pz87Oer1e4JwvLy+Hw2EAIISoiqIQolCqUEoIeYAZisVWVlYYY4gIiMgYW1pampycDPj99L8JANya5vF4RkdHs9msEAIREZEgIgBwzkul0q+9vW+53OnpqaOjY6C//0Ui0dvXFw6Ho9Eope0C2oFHCc4Z55RSRVEcDsfTtf8DHcNdN5N736YAAAAASUVORK5CYII=")
return HTTP.Response(200, headers, body=favicon_data, request=req)
end
function setup_router()
router = HTTP.Router(respond_404)
HTTP.@register(router, "GET", "/", respond_index)
#HTTP.@register(router, "GET", "/", redirect_index) # Another way of
#handling the root, with a redirect
HTTP.@register(router, "GET", "/favicon.ico", respond_favicon)
HTTP.@register(router, "GET", "/index.html", respond_index)
HTTP.@register(router, "GET", "/api/info", respond_info)
router
end
function serve(host, port, server_socket)
router = setup_router()
HTTP.serve(router, host, port; server=server_socket)
end The using Revise
using Sockets
includet("respond_revise.jl")
function launch(host=ip"127.0.0.1", port=8081)
addr = Sockets.InetAddr(host, port)
server_sockets = Channel(1)
@sync begin
@async try
while true
@info "Starting server"
local server_socket
lock(server_sockets)
try
if isopen(server_sockets)
server_socket = Sockets.listen(addr)
put!(server_sockets, server_socket)
else
break
end
finally
unlock(server_sockets)
end
try
Base.invokelatest(serve, host, port, server_socket)
catch
if isopen(server_socket)
rethrow()
end
end
end
catch exc
@error "Server fail" exception=(exc,catch_backtrace())
rethrow()
end
try
entr(["respond_revise.jl"], postpone=true) do
server = take!(server_sockets)
close(server)
@info "Closed server for restart"
# (Press Ctrl-C to exit entr())
end
catch exc
@error "entr failure" exception=(exc,catch_backtrace())
rethrow()
finally
close(take!(server_sockets))
close(server_sockets)
end
end
end |
Thanks. Perhaps this deserves it's own package & documentation, er, ReviseHTTP.jl perhaps? Even so, let's leave this ticket open till such a project is created so others can find this working code? |
I was thinking that the right place to put this stuff might possibly be LiveServer.jl? But anyway I think we should keep this open for the moment until we've settled on something which is reliable. Using it a bit more, my code above still seems to have a few problems. In particular, the somewhat ad-hoc way Julia delivers |
Ok, I've been struggling with Updated live server utility:using Revise
using Sockets
# Some async helper utils
macro async_logged(exs...)
if length(exs) == 2
taskname, body = exs
elseif length(exs) == 1
taskname = "Task"
body = only(exs)
end
quote
@async try
$(esc(body))
catch exc
@error string($(esc(taskname)), " failed") exception=(exc,catch_backtrace())
rethrow()
end
end
end
struct CancelToken
cancelled::Ref{Bool}
cond::Threads.Condition
end
CancelToken() = CancelToken(Ref(false), Threads.Condition())
function Base.close(token::CancelToken)
lock(token.cond) do
token.cancelled[] = true
notify(token.cond)
end
end
Base.isopen(token::CancelToken) = lock(()->!token.cancelled[], token.cond)
Base.wait(token::CancelToken) = lock(()->wait(token.cond), token.cond)
#-------------------------------------------------------------------------------
# The server function
function run_server(serve, token::CancelToken, host=ip"127.0.0.1", port=8081)
addr = Sockets.InetAddr(host, port)
server_sockets = Channel(1)
@sync begin
@async_logged "Server" begin
while isopen(token)
@info "Starting server"
socket = Sockets.listen(addr)
try
put!(server_sockets, socket)
Base.invokelatest(serve, socket)
catch exc
if exc isa Base.IOError && !isopen(socket)
# Ok - server restarted
continue
end
close(socket)
rethrow()
end
end
@info "Exited server loop"
end
@async_logged "Revision loop" begin
# This is like Revise.entr but we control the event loop. This is
# necessary because we need to exit this loop cleanly when the user
# cancels the server, regardless of any revision event.
while isopen(token)
@info "Revision event"
wait(Revise.revision_event)
Revise.revise(throw=true)
# Restart the server's listen loop.
close(take!(server_sockets))
end
@info "Exited revise loop"
end
wait(token)
@assert !isopen(token)
notify(Revise.revision_event) # Trigger revise loop one last time.
@info "Server done"
end
end
Example Usageincludet("respond_revise.jl")
@info """ # Launching server
The server will be automatically restarted when the source code is edited.
Press Return to exit.
"""
@sync begin
token = CancelToken()
@async run_server(serve, token)
readline()
close(token)
end With, for example, respond_revise.jl much the same as before: using HTTP
using Base64
function respond_index(req)
headers = Dict("Content-Type" => "text/html")
# Enable this if you want the browser to poll the page.
# (This is pretty annoying, though; should probably use an event-based
# option like websockets instead.)
refresh_meta = "<meta http-equiv=\"refresh\" content=\"1\">"
body = """
<html>
<head>
$refresh_meta
</head>
<body>
<h1>Hi</h1>
<p>
Some info blah
</body>
</html>
"""
return HTTP.Response(200, headers, body=body, request=req)
end
function redirect_index(req)
# Permanent redirect
return HTTP.Response(301, Dict("Location" => "index.html"))
end
function respond_info(req)
headers = Dict("Content-Type" => "text/json")
body = """
{"info": 1}
"""
return HTTP.Response(200, headers, body=body, request=req)
end
function respond_404(req)
headers = Dict("Content-Type" => "text/html")
return HTTP.Response(404, body="<html><body><h1>404: Not found</body></html>", request=req)
end
function respond_favicon(req)
headers = Dict("Content-Type" => "image/png")
favicon_data = base64decode("iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAIAAACQkWg2AAAABmJLR0QA/wD/AP+gvaeTAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAB3RJTUUH5AsSBxEnP13dzwAAAB1pVFh0Q29tbWVudAAAAAAAQ3JlYXRlZCB3aXRoIEdJTVBkLmUHAAACh0lEQVQoz2WST0sbURTF73szMXES0xiSClGSKEIE/wVXTRFBhG5t9wUh+RTdSLvJd7BgN+ouC4MWalOwaBdWrIqFIE6UqE2CMQ2a0WTmvTfvdmHqxrM+B+7vnkMQUUrZbDYppZqmwRMxxmq1mqZpPp8PACgAFAqFhYWFTCZTr9cBAKREIVAIQLRte3V1NZlMptPpWq0GAKqUslgsLi4uxmKxl4lEl6qa5bJ1fY223eH3217vj+3tjY2Ni4uLubm5QCCgUkqj0WgqlQoGg24hqrlc4+iI394ioqJprkgkHgq9mpkZn5jo6ekBAPLIYDcajc3N+u6ubLUoIYgoAaiiKKEQHR9/Ho8Hw+E2A6XUo2miWLw5PBTN5n6l8nF/P3tycmdZUghRKnVeXnZaluQcANSHV9imeafrwjC+FAoftra4lADwPRJJT087CWmVStbVVWdvLzgctB1otVi9zoX4enb24AaAvUrlr2kCgGRMWJaUsn2SlJJbFiWEEBJ0ux8b6Ha5uhwOAKgytp3P54+PhRDq/f392trawc7OIGNxQt6OjJhC/K5Wu12uVDze5XIVDWOrUvm8vv76/PzdwICq6/r8/Lyu6wGfLzk29mZw8P3UFLdtQohp2+u6ntH1P4bh8Xqf+XyqqqpOp9PpdAJA7ebm08FB1TBCbjcQ0jDNn+Vy/vq6yTkiDg0Pz87Oer1e4JwvLy+Hw2EAIISoiqIQolCqUEoIeYAZisVWVlYYY4gIiMgYW1pampycDPj99L8JANya5vF4RkdHs9msEAIREZEgIgBwzkul0q+9vW+53OnpqaOjY6C//0Ui0dvXFw6Ho9Eope0C2oFHCc4Z55RSRVEcDsfTtf8DHcNdN5N736YAAAAASUVORK5CYII=")
return HTTP.Response(200, headers, body=favicon_data, request=req)
end
function setup_router()
router = HTTP.Router(respond_404)
HTTP.@register(router, "GET", "/", respond_index)
#HTTP.@register(router, "GET", "/", redirect_index) # Another way of
#handling the root, with a redirect
HTTP.@register(router, "GET", "/favicon.ico", respond_favicon)
HTTP.@register(router, "GET", "/index.html", respond_index)
HTTP.@register(router, "GET", "/api/info", respond_info)
router
end
function serve(server_socket)
router = setup_router()
HTTP.serve(router; server=server_socket)
end |
Revise has had to deal with the same issue. In case it's useful, see the fix in timholy/Revise.jl#467. |
Oh.. you're talking about submitting this as a pull request to |
I'm snowed under at the moment so it may be a while before I could get to packaging this code up in some way which could be reused. In the meantime, treating it as a somewhat hacky but useful script to be copied seems fine to me. I think the slightly tricky part in making this reusable for real server code is that you don't want a production server to load any dev tools — including Revise / LiveServer.jl etc etc. But on the other hand, the server loop is almost at the top level of the application where you want to carefully manage tasks and cancellation, etc, and Base doesn't really provide nice tools for managing cancellation. So integrating all the pieces without forcing dependencies on people's production servers will just require a little thought. |
Having said that, if anyone wants to take the code above and run with it, that's great. Ping me for a review if you like. |
@quinnj What do you think about adding something like this as a file in |
It may be interesting to observe that achieving clean cancellation in async is one of the big benefits of structured concurrency, as discussed in Nathaniel Smith's "Timeouts and Cancellations for Humans". @c42f would your work on JuliaLang/julia#33248 solve the problems you've observed in this case? |
A proper structured concurrency and cancellation system would definitely need to cover this use case which is quite simple as far as these things go. For now, the |
There have been a few issues (#587, #563) around the overall ergonomics of using `HTTP.listen` and the code hasn't had a good comb-through in a while. I used the core golang [server code](https://github.com/golang/go/blob/master/src/net/http/server.go) as a reference to see what kinds of things they allowed in terms of configuration, functionality, and overall ergonomics. The changes proposed here include: * New `HTTP.listen!` non-blocking version of `HTTP.listen` that returns a `Server` object * `Server` object supports `wait`, `close`, and `forceclose`; `close` initiates a "graceful" shutdown where active connections are waited for until they are finished being processed; `forceclose` just force closes all open connections. * The combination of `HTTP.listen!` + `Server` mean we have a much more convenient and simple way to interact with a running server. * In addition, when calling the provided handler function `f`, we use `Base.invokelatest(f, stream)` which gives the nice property of allowing Revise to "update" a live server by reflecting edits made to the handler/middleware stack. * Removed `reuse_limit` and `rate_limit` features since they were either problematic in their implementation or not really that useful * Tried to clean up the verbose logging story, though there's more to do here * Tried to clean up some core listenloop logic, though there's more to do here Otherwise, the core `HTTP.listen` function remains mostly unchanged, except for the changes to supported keyword arguments. Still need to update some tests, docs, and comb-through the `handle_transaction` function, but wanted to put this up now in case others are interested in taking a look.
Sorry I missed this fascinating conversation originally! I think @c42f's code is a pretty good way to approach the original issue here. In #854, I'm proposing some improvements to overall Server "ergonomics", including a new These are pretty simple changes code-wise that I think provide "most" of the desired convenience originally requested here without needing to go for a full server shutdown + restart (though hopefully even that workflow will be much easier now!). Happy to hear any feedback on the proposal and if there are other simple things we can do to improve server ergonomics. |
Otherwise, I generally agree with some of the other comments here: if there's a "fully integrated Revise" solution that can find a home somewhere, that'd be great. And as mentioned, happy to take other suggestions of things we can do in HTTP.jl specifically to make things easier to work with. |
…#854) * Modernize core server code and improve overall ergonomics There have been a few issues (#587, #563) around the overall ergonomics of using `HTTP.listen` and the code hasn't had a good comb-through in a while. I used the core golang [server code](https://github.com/golang/go/blob/master/src/net/http/server.go) as a reference to see what kinds of things they allowed in terms of configuration, functionality, and overall ergonomics. The changes proposed here include: * New `HTTP.listen!` non-blocking version of `HTTP.listen` that returns a `Server` object * `Server` object supports `wait`, `close`, and `forceclose`; `close` initiates a "graceful" shutdown where active connections are waited for until they are finished being processed; `forceclose` just force closes all open connections. * The combination of `HTTP.listen!` + `Server` mean we have a much more convenient and simple way to interact with a running server. * In addition, when calling the provided handler function `f`, we use `Base.invokelatest(f, stream)` which gives the nice property of allowing Revise to "update" a live server by reflecting edits made to the handler/middleware stack. * Removed `reuse_limit` and `rate_limit` features since they were either problematic in their implementation or not really that useful * Tried to clean up the verbose logging story, though there's more to do here * Tried to clean up some core listenloop logic, though there's more to do here Otherwise, the core `HTTP.listen` function remains mostly unchanged, except for the changes to supported keyword arguments. Still need to update some tests, docs, and comb-through the `handle_transaction` function, but wanted to put this up now in case others are interested in taking a look. * fix tests * fix cookie test * update docs * fix reuseaddr test * fix * more cleanup * fix test
Web developers need a way to automatically update server behavior with changes to local development files. This is unobvious for those unfamiliar with
HTTP.jl
,Revise.jl
, and their interaction. Below is a proposed example that works, at least for local development.The text was updated successfully, but these errors were encountered: