Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add a network benchmark using an HTTP-like protocol #478

Merged
merged 1 commit into from
Apr 4, 2023
Merged

Add a network benchmark using an HTTP-like protocol #478

merged 1 commit into from
Apr 4, 2023

Conversation

talex5
Copy link
Collaborator

@talex5 talex5 commented Mar 30, 2023 •
edited
Loading

This runs 4 client domains and 4 server domains. Each client opens 25 connections and makes 1000 HTTP POSTs on each one. For each request it uses Buf_write to write the headers and body, then Buf_read to parse the response. Likewise, the server uses Buf_read and Buf_write to handle the each request.

On my system, I get this using eio_linux (eio_posix is about the same):

clients, servers, requests, requests/s
      4,       4,   100000, 367051.7

Part of #450.

@talex5 talex5 force-pushed the bench-server branch 2 times, most recently from ea32900 to 93d5199 Compare March 30, 2023 13:43
@avsm avsm mentioned this pull request Mar 31, 2023
Open
avsm
avsm reviewed Mar 31, 2023
View reviewed changes
bench/bench_server.ml Outdated
let n_client_domains = 4
let n_server_domains = 4
let n_connections_per_domain = 25
let n_requests_per_connection = 1000
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps get these from environment variables? I can crank these up on some of my 128 core machines :-)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I've updated it so you can pass these as arguments.

@patricoferris
Copy link
Collaborator

Looks good to me! :))

I took this for a quick spin on a Mac (so the posix backend) and was just playing around with some of the numbers. If the backlog is set too low I get a bunch of exceptions. For example with ~backlog:50 on a Mac:

Fatal error: exception Eio.Io Multiple_io
- Multiple_io
  - Multiple_io
    - Net Connection_reset Unix_error (Connection reset by peer, "readv", "")
    - Net Connection_reset Unix_error (Connection reset by peer, "readv", "")
  - Multiple_io
    - Multiple_io
      - Multiple_io
        - Multiple_io
          - Multiple_io
            - Multiple_io
              - Net Connection_reset Unix_error (Connection reset by peer, "readv", "")
              - Net Connection_reset Unix_error (Connection reset by peer, "readv", "")
            - Net Connection_reset Unix_error (Connection reset by peer, "readv", "")
          - Net Connection_reset Unix_error (Connection reset by peer, "readv", "")
        - Net Connection_reset Unix_error (Connection reset by peer, "readv", "")
      - Net Connection_reset Unix_error (Connection reset by peer, "readv", "")
    - Net Connection_reset Unix_error (Connection reset by peer, "readv", "")
- Multiple_io
  - Multiple_io
    - Net Connection_reset Unix_error (Connection reset by peer, "readv", "")
    - Net Connection_reset Unix_error (Connection reset by peer, "readv", "")
  - Multiple_io
    - Multiple_io
      - Multiple_io
        - Multiple_io
          - Net Connection_reset Unix_error (Connection reset by peer, "readv", ""),
              handling connection from tcp:127.0.0.1:53920
          - Net Connection_reset Unix_error (Connection reset by peer, "readv", ""),
              handling connection from tcp:127.0.0.1:53923
        - Net Connection_reset Unix_error (Connection reset by peer, "readv", ""),
            handling connection from tcp:127.0.0.1:53925
      - Net Connection_reset Unix_error (Connection reset by peer, "readv", ""),
          handling connection from tcp:127.0.0.1:53927
    - Net Connection_reset Unix_error (Connection reset by peer, "readv", ""),
        handling connection from tcp:127.0.0.1:53907

I think this is because of low ulimits, does that sound about right ?

@talex5
Add a network benchmark using an HTTP-like protocol
3c7277f 
This runs 4 client domains and 4 server domains. Each client opens 25
connections and makes 1000 HTTP POSTs on each one. For each request it
uses Buf_write to write the headers and body, then Buf_read to parse the
response. Likewise, the server uses Buf_read and Buf_write to handle the
each request.

On my system, I get this using eio_linux (eio_posix is about the same):

    clients, servers, requests, requests/s
          4,       4,   100000, 367051.7
@talex5
Copy link
Collaborator Author

talex5 commented Apr 4, 2023

If the backlog is set too low I get a bunch of exceptions.

I've updated it to calculate the backlog automatically now. On Linux, it seems to retry automatically rather than failing.

I would expect something like this happened:

  1. Some clients connected.
  2. One failed to connect. This caused all the fibers to be cancelled.
  3. The other clients disconnected as part of the cancellation.
  4. The servers complained that their clients had disappeared.

Is there more of the error that you didn't include? I would expect to see a connection-refused error in there somewhere.

@talex5 talex5 merged commit 83b764c into ocaml-multicore:main Apr 4, 2023
@talex5 talex5 deleted the bench-server branch April 4, 2023 18:29
@talex5 talex5 mentioned this pull request Apr 11, 2023
Merged
talex5 added a commit to talex5/opam-repository that referenced this pull request Apr 11, 2023
@talex5
[new release] eio_posix, eio_main, eio_luv, eio_linux and eio (0.9)
ac7a2dd 
CHANGES:

New features:

- Add eio_posix backend (@talex5 @haesbaert ocaml-multicore/eio#448 ocaml-multicore/eio#477, reviewed by @avsm @patricoferris @polytypic).
  This replaces eio_luv on all platforms except Windows (which will later switch to its own backend). It is a lot faster, provides access to more modern features (such as `openat`), and can safely share OS resources between domains.

- Add subprocess support (@patricoferris @talex5 ocaml-multicore/eio#461 ocaml-multicore/eio#464 ocaml-multicore/eio#472, reviewed by @haesbaert @avsm).
  This is the low-level API support for eio_linux and eio_posix. A high-level cross-platform API will be added in the next release.

- Add `Fiber.fork_seq` (@talex5 ocaml-multicore/eio#460, reviewed by @avsm).
  This is a light-weight alternative to using a single-producer, single-consumer, 0-capacity stream, similar to a Python generator function.

Bug fixes:

- eio_linux: make it safe to share FDs across domains (@talex5 ocaml-multicore/eio#440, reviewed by @haesbaert).
  It was previously not safe to share file descriptors between domains because if one domain used an FD just as another was closing it, and the FD got reused, then the original operation could act on the wrong file.

- eio_linux: release uring if Linux is too old (@talex5 ocaml-multicore/eio#476).
  Avoids a small resource leak.

- eio_linux: improve error handling creating pipes and sockets (@talex5 ocaml-multicore/eio#474, spotted by @avsm).
  If we get an error (e.g. too many FDs) then report it to the calling fiber, instead of exiting the event loop.

- eio_linux: wait for uring to finish before exiting (@talex5 ocaml-multicore/eio#470, reviewed by @avsm).
  If the main fiber raised an exception then it was possible to exit while a cancellation operation was still in progress.

- eio_main: make `EIO_BACKEND` handling more uniform (@talex5 ocaml-multicore/eio#447).
  Previously this environment variable was only used on Linux. Now all platforms check it.

- Tell dune about `EIO_BACKEND` (@talex5 ocaml-multicore/eio#442).
  If this changes, dune needs to re-run the tests.

- eio_linux: add some missing close-on-execs (@talex5 ocaml-multicore/eio#441).

- eio_linux: `read_exactly` fails to update file offset (@talex5 ocaml-multicore/eio#438).

- Work around dune `enabled_if` bug on non-Linux systems (@polytypic ocaml-multicore/eio#475, reviewed by @talex5).

- Use raw system call of `getrandom` for glibc versions before 2.25 (@zenfey ocaml-multicore/eio#482).

Documentation:

- Add `HACKING.md` with hints for working on Eio (@talex5 ocaml-multicore/eio#443, reviewed by @avsm @polytypic).

- Improve worker pool example (@talex5 ocaml-multicore/eio#454).

- Add more Conditions documentation (@talex5 ocaml-multicore/eio#436, reviewed by @haesbaert).
  This adds a discussion of conditions to the README and provides examples using them to handle signals.

- Condition: fix the example in the docstring (@avsm ocaml-multicore/eio#468).

Performance:

- Add a network benchmark using an HTTP-like protocol (@talex5 ocaml-multicore/eio#478, reviewed by @avsm @patricoferris).

- Add a benchmark for reading from `/dev/zero` (@talex5 ocaml-multicore/eio#439).

Other changes:

- Add CI for macOS (@talex5 ocaml-multicore/eio#452).

- Add tests for `pread`, `pwrite` and `readdir` (@talex5 ocaml-multicore/eio#451).

- eio_linux: split into multiple files (@talex5 ocaml-multicore/eio#465 ocaml-multicore/eio#466, reviewed by @avsm).

- Update Dockerfile (@talex5 ocaml-multicore/eio#471).

- Use dune.3.7.0 (@patricoferris ocaml-multicore/eio#457).

- Mint exclusive IDs across domains (@TheLortex ocaml-multicore/eio#480, reported by @haesbaert, reviewed by @talex5).
  The tracing currently only works with a single domain anyway, but this will change when OCaml 5.1 is released.
@talex5 talex5 mentioned this pull request Jun 2, 2023
Merged
talex5 added a commit to talex5/opam-repository that referenced this pull request Jun 2, 2023
@talex5
[new release] eio_windows, eio_posix, eio_main, eio_linux and eio (0.10)
ede1661 
CHANGES:

New features:

- Add eio_posix backend (@talex5 @haesbaert ocaml-multicore/eio#448 ocaml-multicore/eio#477, reviewed by @avsm @patricoferris @polytypic).
  This replaces eio_luv on all platforms except Windows (which will later switch to its own backend). It is a lot faster, provides access to more modern features (such as `openat`), and can safely share OS resources between domains.

- Add subprocess support (@patricoferris @talex5 ocaml-multicore/eio#461 ocaml-multicore/eio#464 ocaml-multicore/eio#472, reviewed by @haesbaert @avsm).
  This is the low-level API support for eio_linux and eio_posix. A high-level cross-platform API will be added in the next release.

- Add `Fiber.fork_seq` (@talex5 ocaml-multicore/eio#460, reviewed by @avsm).
  This is a light-weight alternative to using a single-producer, single-consumer, 0-capacity stream, similar to a Python generator function.

Bug fixes:

- eio_linux: make it safe to share FDs across domains (@talex5 ocaml-multicore/eio#440, reviewed by @haesbaert).
  It was previously not safe to share file descriptors between domains because if one domain used an FD just as another was closing it, and the FD got reused, then the original operation could act on the wrong file.

- eio_linux: release uring if Linux is too old (@talex5 ocaml-multicore/eio#476).
  Avoids a small resource leak.

- eio_linux: improve error handling creating pipes and sockets (@talex5 ocaml-multicore/eio#474, spotted by @avsm).
  If we get an error (e.g. too many FDs) then report it to the calling fiber, instead of exiting the event loop.

- eio_linux: wait for uring to finish before exiting (@talex5 ocaml-multicore/eio#470, reviewed by @avsm).
  If the main fiber raised an exception then it was possible to exit while a cancellation operation was still in progress.

- eio_main: make `EIO_BACKEND` handling more uniform (@talex5 ocaml-multicore/eio#447).
  Previously this environment variable was only used on Linux. Now all platforms check it.

- Tell dune about `EIO_BACKEND` (@talex5 ocaml-multicore/eio#442).
  If this changes, dune needs to re-run the tests.

- eio_linux: add some missing close-on-execs (@talex5 ocaml-multicore/eio#441).

- eio_linux: `read_exactly` fails to update file offset (@talex5 ocaml-multicore/eio#438).

- Work around dune `enabled_if` bug on non-Linux systems (@polytypic ocaml-multicore/eio#475, reviewed by @talex5).

- Use raw system call of `getrandom` for glibc versions before 2.25 (@zenfey ocaml-multicore/eio#482).

Documentation:

- Add `HACKING.md` with hints for working on Eio (@talex5 ocaml-multicore/eio#443, reviewed by @avsm @polytypic).

- Improve worker pool example (@talex5 ocaml-multicore/eio#454).

- Add more Conditions documentation (@talex5 ocaml-multicore/eio#436, reviewed by @haesbaert).
  This adds a discussion of conditions to the README and provides examples using them to handle signals.

- Condition: fix the example in the docstring (@avsm ocaml-multicore/eio#468).

Performance:

- Add a network benchmark using an HTTP-like protocol (@talex5 ocaml-multicore/eio#478, reviewed by @avsm @patricoferris).

- Add a benchmark for reading from `/dev/zero` (@talex5 ocaml-multicore/eio#439).

Other changes:

- Add CI for macOS (@talex5 ocaml-multicore/eio#452).

- Add tests for `pread`, `pwrite` and `readdir` (@talex5 ocaml-multicore/eio#451).

- eio_linux: split into multiple files (@talex5 ocaml-multicore/eio#465 ocaml-multicore/eio#466, reviewed by @avsm).

- Update Dockerfile (@talex5 ocaml-multicore/eio#471).

- Use dune.3.7.0 (@patricoferris ocaml-multicore/eio#457).

- Mint exclusive IDs across domains (@TheLortex ocaml-multicore/eio#480, reported by @haesbaert, reviewed by @talex5).
  The tracing currently only works with a single domain anyway, but this will change when OCaml 5.1 is released.
talex5 added a commit to talex5/opam-repository that referenced this pull request Jun 2, 2023
@talex5
[new release] eio_windows, eio_posix, eio_main, eio_linux and eio (0.10)
5bb1ac4 
CHANGES:

New features:

- Add eio_posix backend (@talex5 @haesbaert ocaml-multicore/eio#448 ocaml-multicore/eio#477, reviewed by @avsm @patricoferris @polytypic).
  This replaces eio_luv on all platforms except Windows (which will later switch to its own backend). It is a lot faster, provides access to more modern features (such as `openat`), and can safely share OS resources between domains.

- Add subprocess support (@patricoferris @talex5 ocaml-multicore/eio#461 ocaml-multicore/eio#464 ocaml-multicore/eio#472, reviewed by @haesbaert @avsm).
  This is the low-level API support for eio_linux and eio_posix. A high-level cross-platform API will be added in the next release.

- Add `Fiber.fork_seq` (@talex5 ocaml-multicore/eio#460, reviewed by @avsm).
  This is a light-weight alternative to using a single-producer, single-consumer, 0-capacity stream, similar to a Python generator function.

Bug fixes:

- eio_linux: make it safe to share FDs across domains (@talex5 ocaml-multicore/eio#440, reviewed by @haesbaert).
  It was previously not safe to share file descriptors between domains because if one domain used an FD just as another was closing it, and the FD got reused, then the original operation could act on the wrong file.

- eio_linux: release uring if Linux is too old (@talex5 ocaml-multicore/eio#476).
  Avoids a small resource leak.

- eio_linux: improve error handling creating pipes and sockets (@talex5 ocaml-multicore/eio#474, spotted by @avsm).
  If we get an error (e.g. too many FDs) then report it to the calling fiber, instead of exiting the event loop.

- eio_linux: wait for uring to finish before exiting (@talex5 ocaml-multicore/eio#470, reviewed by @avsm).
  If the main fiber raised an exception then it was possible to exit while a cancellation operation was still in progress.

- eio_main: make `EIO_BACKEND` handling more uniform (@talex5 ocaml-multicore/eio#447).
  Previously this environment variable was only used on Linux. Now all platforms check it.

- Tell dune about `EIO_BACKEND` (@talex5 ocaml-multicore/eio#442).
  If this changes, dune needs to re-run the tests.

- eio_linux: add some missing close-on-execs (@talex5 ocaml-multicore/eio#441).

- eio_linux: `read_exactly` fails to update file offset (@talex5 ocaml-multicore/eio#438).

- Work around dune `enabled_if` bug on non-Linux systems (@polytypic ocaml-multicore/eio#475, reviewed by @talex5).

- Use raw system call of `getrandom` for glibc versions before 2.25 (@zenfey ocaml-multicore/eio#482).

Documentation:

- Add `HACKING.md` with hints for working on Eio (@talex5 ocaml-multicore/eio#443, reviewed by @avsm @polytypic).

- Improve worker pool example (@talex5 ocaml-multicore/eio#454).

- Add more Conditions documentation (@talex5 ocaml-multicore/eio#436, reviewed by @haesbaert).
  This adds a discussion of conditions to the README and provides examples using them to handle signals.

- Condition: fix the example in the docstring (@avsm ocaml-multicore/eio#468).

Performance:

- Add a network benchmark using an HTTP-like protocol (@talex5 ocaml-multicore/eio#478, reviewed by @avsm @patricoferris).

- Add a benchmark for reading from `/dev/zero` (@talex5 ocaml-multicore/eio#439).

Other changes:

- Add CI for macOS (@talex5 ocaml-multicore/eio#452).

- Add tests for `pread`, `pwrite` and `readdir` (@talex5 ocaml-multicore/eio#451).

- eio_linux: split into multiple files (@talex5 ocaml-multicore/eio#465 ocaml-multicore/eio#466, reviewed by @avsm).

- Update Dockerfile (@talex5 ocaml-multicore/eio#471).

- Use dune.3.7.0 (@patricoferris ocaml-multicore/eio#457).

- Mint exclusive IDs across domains (@TheLortex ocaml-multicore/eio#480, reported by @haesbaert, reviewed by @talex5).
  The tracing currently only works with a single domain anyway, but this will change when OCaml 5.1 is released.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@avsm avsm avsm left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@talex5 @patricoferris @avsm