Standard IO Interfaces #728

vyzo · 2023-07-17T17:09:36Z

Introduces the standard IO interfaces; see #724.

Follow up:

remove old-style bio/socket modules and adapt users to use stdio
update tutorial

The interfaces look like this:

;; generic binary IO
(interface Reader
  ;; read into a buffer; it _must_ be a u8vector.
  ;; - start denotes the start of the read region; it must be a fixnum within the buffer range.
  ;; - end denotes the read region end
  ;; - need denotes the minimum required input; it must be a fixnum
  ;; Returns the number of bytes read; 0 denotes the end of input.
  ;; If less than the needed bytes are read, an io-error is raised.
  (read u8v (start 0) (end (u8vector-length u8v)) (need 0))

  ;; closes the input source
  (close))

(interface Writer
  ;; write from a buffer; it _must_ be a u8vector
  ;; - start denotes the start of the write region; it must be a fixnum within the buffer range.
  ;; - end denotes the write region end; #f means the end of the buffer
  ;; Returns the number of bytes written.
  (write u8v (start 0) (end (u8vector-length u8v)))

  ;; closes the output sink
  (close))

;; buffered IO
(interface (BufferedReader Reader)
  ;; reads a single byte
  (read-u8)
  ;; peeks the next byte
  (peek-u8)

  ;; skips the next count bytes of input
  (skip count)

  ;; returns a new BufferedReader instance delimiting the input length that shares the underlying
  ;; buffer; the limit must be a fixnum.
  (delimit limit)

  ;; resets the underlying reader and buffer state, allowing reuse of buffers.
  (reset! reader))

(interface (BufferedWriter Writer)
  ;; writes a single byte
  (write-u8 u8)

  ;; flushes the buffer to the underlyin output instance
  (flush)

  ;; resets the underlying output and buffer state, allowing reuse of buffers.
  (reset! output))

;; socket interfaces
(interface Socket
  (domain)
  (address)
  (peer-address)
  (getsockopt level option)
  (setsockopt level option value)
  (set-input-timeout! timeo)
  (set-output-timeout! timeo)
  (close))

(interface (StreamSocket Socket)
  ;; receives data into a buffer; it _must_ be a u8vecotr
  ;; - start denotes the start of the read region; it must be a fixnum within the buffer range.
  ;; - end denotes the read region end.
  ;; Returns the number of bytes read.
  (recv u8v (start 0) (end (u8vector-length u8v)) (flags 0))

  ;; sends data from a buffer; it _must_ be a u8vector
  ;; - start denotes the start of the write region; it must be a fixnum within the buffer range.
  ;; - end denotes the write region end.
  ;; Returns the number of bytes written.
  (send u8v (start 0) (end (u8vector-length u8v)) (flags 0))

  ;; returns a Reader instance reading from the socket
  (reader)

  ;; returns a Writer instance writing to the socket
  (writer)

  ;; shuts down the socket in one direction which must be 'in, 'out or 'inout
  ;; if both directions are closed the socket is also closed.
  (shutdown direction))

(interface (ServerSocket Socket)
  ;; accept waits for an incoming connection and returns a StreamSocket
  (accept))

(interface (DatagramSocket Socket)
  ;; receives data into a buffer; it _must_ be a u8vecotr
  ;; - peer is a _box_ to place the peer's address.
  ;; - start denotes the start of the read region; it must be a fixnum within the buffer range.
  ;; - end denotes the read region end
  ;; Returns the number of bytes read.
  (recvfrom peer u8v (start 0) (end (u8vector-length u8v)) (flags 0))

  ;; sends data from a buffer; it _must_ be a u8vector
  ;; - peer is the address of the peer
  ;; - start denotes the start of the write region; it must be a fixnum within the buffer range.
  ;; - end denotes the write region end; #f means the end of the buffer
  ;; Returns the number of bytes written.
  (sendto peer u8v (start 0) (end (u8vector-length u8v)) (flags 0))

  ;; connect the datagram socket to a peer
  (connect peer)
  ;; recv data from the connected peer
  (recv u8v (start 0) (end (u8vector-length u8v)) (flags 0))
  ;; send data to the connected peer
  (send u8v (start 0) (end (u8vector-length u8v)) (flags 0)))

vyzo · 2023-07-18T07:09:25Z

rebased on master.

vyzo · 2023-07-19T13:04:24Z

Running some benchmarks (see the file-benchmark program), readers/writers blow ports out of the water in terms of performance (and the flexibility/composability is so much nicer!)

>>> benchmark reader [chunk size: 1024]
(time (std/io/file-benchmark#test-reader _reader499_ _buffer497_))
    0.006248 secs real time
    0.006102 secs cpu time (0.001867 user, 0.004235 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered reader [chunk size: 1024]
>>> benchmark buffered reader [buffer size: 4096]
(time (std/io/file-benchmark#test-reader _reader517_ _buffer497_))
    0.007019 secs real time
    0.006950 secs cpu time (0.003009 user, 0.003941 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered reader [buffer size: 32768]
(time (std/io/file-benchmark#test-reader _reader517_ _buffer497_))
    0.006605 secs real time
    0.006534 secs cpu time (0.002659 user, 0.003875 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered reader [buffer size: 65536]
(time (std/io/file-benchmark#test-reader _reader517_ _buffer497_))
    0.007238 secs real time
    0.007098 secs cpu time (0.003179 user, 0.003919 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark unbuffered port [chunk size: 1024]
(time (std/io/file-benchmark#test-port-read _port550_ _buffer497_))
    0.006449 secs real time
    0.006310 secs cpu time (0.002507 user, 0.003803 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered port [chunk size: 1024]
(time (std/io/file-benchmark#test-port-read _port552_ _buffer497_))
    0.006227 secs real time
    0.006153 secs cpu time (0.001831 user, 0.004322 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark reader [chunk size: 4096]
(time (std/io/file-benchmark#test-reader _reader499_ _buffer497_))
    0.002923 secs real time
    0.002922 secs cpu time (0.000000 user, 0.002922 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered reader [chunk size: 4096]
>>> benchmark buffered reader [buffer size: 4096]
(time (std/io/file-benchmark#test-reader _reader517_ _buffer497_))
    0.002793 secs real time
    0.002737 secs cpu time (0.000000 user, 0.002737 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered reader [buffer size: 32768]
(time (std/io/file-benchmark#test-reader _reader517_ _buffer497_))
    0.002595 secs real time
    0.002595 secs cpu time (0.000000 user, 0.002595 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered reader [buffer size: 65536]
(time (std/io/file-benchmark#test-reader _reader517_ _buffer497_))
    0.002770 secs real time
    0.002696 secs cpu time (0.000000 user, 0.002696 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark unbuffered port [chunk size: 4096]
(time (std/io/file-benchmark#test-port-read _port550_ _buffer497_))
    0.006171 secs real time
    0.006092 secs cpu time (0.000001 user, 0.006091 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered port [chunk size: 4096]
(time (std/io/file-benchmark#test-port-read _port552_ _buffer497_))
    0.005882 secs real time
    0.005869 secs cpu time (0.000000 user, 0.005869 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark reader [chunk size: 32768]
(time (std/io/file-benchmark#test-reader _reader499_ _buffer497_))
    0.002160 secs real time
    0.002103 secs cpu time (0.000000 user, 0.002103 system)
    no collections
    no bytes allocated
    7 minor faults
    no major faults
>>> benchmark buffered reader [chunk size: 32768]
>>> benchmark buffered reader [buffer size: 4096]
(time (std/io/file-benchmark#test-reader _reader517_ _buffer497_))
    0.001926 secs real time
    0.001926 secs cpu time (0.000000 user, 0.001926 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered reader [buffer size: 32768]
(time (std/io/file-benchmark#test-reader _reader517_ _buffer497_))
    0.001776 secs real time
    0.001764 secs cpu time (0.000000 user, 0.001764 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered reader [buffer size: 65536]
(time (std/io/file-benchmark#test-reader _reader517_ _buffer497_))
    0.001802 secs real time
    0.001744 secs cpu time (0.000000 user, 0.001744 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark unbuffered port [chunk size: 32768]
(time (std/io/file-benchmark#test-port-read _port550_ _buffer497_))
    0.006092 secs real time
    0.005960 secs cpu time (0.000000 user, 0.005960 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered port [chunk size: 32768]
(time (std/io/file-benchmark#test-port-read _port552_ _buffer497_))
    0.005891 secs real time
    0.005762 secs cpu time (0.001826 user, 0.003936 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark reader [chunk size: 65536]
(time (std/io/file-benchmark#test-reader _reader499_ _buffer497_))
    0.002441 secs real time
    0.002440 secs cpu time (0.000000 user, 0.002440 system)
    no collections
    no bytes allocated
    14 minor faults
    no major faults
>>> benchmark buffered reader [chunk size: 65536]
>>> benchmark buffered reader [buffer size: 4096]
(time (std/io/file-benchmark#test-reader _reader517_ _buffer497_))
    0.002069 secs real time
    0.002056 secs cpu time (0.000672 user, 0.001384 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered reader [buffer size: 32768]
(time (std/io/file-benchmark#test-reader _reader517_ _buffer497_))
    0.001646 secs real time
    0.001587 secs cpu time (0.000000 user, 0.001587 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered reader [buffer size: 65536]
(time (std/io/file-benchmark#test-reader _reader517_ _buffer497_))
    0.001686 secs real time
    0.001686 secs cpu time (0.000000 user, 0.001686 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark unbuffered port [chunk size: 65536]
(time (std/io/file-benchmark#test-port-read _port550_ _buffer497_))
    0.005740 secs real time
    0.005660 secs cpu time (0.002162 user, 0.003498 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered port [chunk size: 65536]
(time (std/io/file-benchmark#test-port-read _port552_ _buffer497_))
    0.005829 secs real time
    0.005702 secs cpu time (0.001248 user, 0.004454 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark writer [chunk size: 1024]
(time (std/io/file-benchmark#test-writer _writer437_ _junk419_ _chunk-size435_))
    0.021387 secs real time
    0.021162 secs cpu time (0.012945 user, 0.008217 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark unbuffered port [chunk size: 1024]
(time (std/io/file-benchmark#test-port-write _port439_ _junk419_ _chunk-size435_))
    0.044553 secs real time
    0.044123 secs cpu time (0.032740 user, 0.011383 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark port [chunk size: 1024]
(time (std/io/file-benchmark#test-port-write _port441_ _junk419_ _chunk-size435_))
    0.043222 secs real time
    0.042785 secs cpu time (0.022956 user, 0.019829 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark writer [chunk size: 4096]
(time (std/io/file-benchmark#test-writer _writer437_ _junk419_ _chunk-size435_))
    0.009779 secs real time
    0.009678 secs cpu time (0.001310 user, 0.008368 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark unbuffered port [chunk size: 4096]
(time (std/io/file-benchmark#test-port-write _port439_ _junk419_ _chunk-size435_))
    0.039919 secs real time
    0.039449 secs cpu time (0.019431 user, 0.020018 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark port [chunk size: 4096]
(time (std/io/file-benchmark#test-port-write _port441_ _junk419_ _chunk-size435_))
    0.038510 secs real time
    0.038068 secs cpu time (0.014904 user, 0.023164 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark writer [chunk size: 32768]
(time (std/io/file-benchmark#test-writer _writer437_ _junk419_ _chunk-size435_))
    0.007181 secs real time
    0.007112 secs cpu time (0.000000 user, 0.007112 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark unbuffered port [chunk size: 32768]
(time (std/io/file-benchmark#test-port-write _port439_ _junk419_ _chunk-size435_))
    0.034732 secs real time
    0.034397 secs cpu time (0.017721 user, 0.016676 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark port [chunk size: 32768]
(time (std/io/file-benchmark#test-port-write _port441_ _junk419_ _chunk-size435_))
    0.036067 secs real time
    0.035694 secs cpu time (0.014128 user, 0.021566 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark writer [chunk size: 65536]
(time (std/io/file-benchmark#test-writer _writer437_ _junk419_ _chunk-size435_))
    0.006370 secs real time
    0.006357 secs cpu time (0.000000 user, 0.006357 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark unbuffered port [chunk size: 65536]
(time (std/io/file-benchmark#test-port-write _port439_ _junk419_ _chunk-size435_))
    0.035275 secs real time
    0.034889 secs cpu time (0.021340 user, 0.013549 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark port [chunk size: 65536]
(time (std/io/file-benchmark#test-port-write _port441_ _junk419_ _chunk-size435_))
    0.033967 secs real time
    0.033567 secs cpu time (0.013022 user, 0.020545 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults

belmarca

Here are a few comments, I'll give more later.

src/std/io/bio/api.ss

src/std/io/bio/util.ss

src/std/io/bio/bio-test.ss

src/std/io/bio/util.ss

belmarca · 2023-07-20T00:41:15Z

src/std/io/bio/api.ss

@@ -39,7 +40,7 @@
 (def (open-buffered-writer writer (buffer-size default-buffer-size))
  (unless (Writer? writer)
    (error "Expected Writer instance" writer))
-  (BufferedWriter (make-output-buffer write (make-u8vector buffer-size) 0)))
+  (BufferedWriter (make-output-buffer writer (make-u8vector buffer-size) 0)))


Btw this should be the kind of error spotted by the new gsc -warnings flag.

interesting

with contracts (also planned for v18), maybe the gerbil compiler will also be able to catch it.

vyzo · 2023-07-20T06:10:28Z

Improved benchmark program so that it is easier to reason about timings.
Here are some timings:

Input

Input Port

>>> benchmark input port [chunk size: 1024]
(time (std/io/file-benchmark#test-port-read _port655_ _buffer653_))
    0.005644 secs real time
    0.005562 secs cpu time (0.000001 user, 0.005561 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark input port [chunk size: 4096]
(time (std/io/file-benchmark#test-port-read _port655_ _buffer653_))
    0.005613 secs real time
    0.005597 secs cpu time (0.000000 user, 0.005597 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark input port [chunk size: 32768]
(time (std/io/file-benchmark#test-port-read _port655_ _buffer653_))
    0.005116 secs real time
    0.005057 secs cpu time (0.001080 user, 0.003977 system)
    no collections
    no bytes allocated
    7 minor faults
    no major faults
>>> benchmark input port [chunk size: 65536]
(time (std/io/file-benchmark#test-port-read _port655_ _buffer653_))
    0.004682 secs real time
    0.004644 secs cpu time (0.000533 user, 0.004111 system)
    no collections
    no bytes allocated
    15 minor faults
    no major faults

Reader

>>> benchmark reader [chunk size: 1024]
(time (std/io/file-benchmark#test-reader _reader810_ _buffer808_))
    0.005391 secs real time
    0.005332 secs cpu time (0.000001 user, 0.005331 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark reader [chunk size: 4096]
(time (std/io/file-benchmark#test-reader _reader810_ _buffer808_))
    0.002976 secs real time
    0.002976 secs cpu time (0.000000 user, 0.002976 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark reader [chunk size: 32768]
(time (std/io/file-benchmark#test-reader _reader810_ _buffer808_))
    0.001997 secs real time
    0.001985 secs cpu time (0.000000 user, 0.001985 system)
    no collections
    no bytes allocated
    7 minor faults
    no major faults
>>> benchmark reader [chunk size: 65536]
(time (std/io/file-benchmark#test-reader _reader810_ _buffer808_))
    0.001596 secs real time
    0.001535 secs cpu time (0.000000 user, 0.001535 system)
    no collections
    no bytes allocated
    15 minor faults
    no major faults

Buffered Reader

>>> benchmark buffered reader [buffer size: 4096 chunk size: 1024]
(time (std/io/file-benchmark#test-reader _reader725_ _buffer707_))
    0.005589 secs real time
    0.005588 secs cpu time (0.000001 user, 0.005587 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered reader [buffer size: 32768 chunk size: 1024]
(time (std/io/file-benchmark#test-reader _reader725_ _buffer707_))
    0.005900 secs real time
    0.005900 secs cpu time (0.000000 user, 0.005900 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered reader [buffer size: 65536 chunk size: 1024]
(time (std/io/file-benchmark#test-reader _reader725_ _buffer707_))
    0.005660 secs real time
    0.005660 secs cpu time (0.000001 user, 0.005659 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered reader [buffer size: 32768 chunk size: 4096]
(time (std/io/file-benchmark#test-reader _reader725_ _buffer707_))
    0.002527 secs real time
    0.002527 secs cpu time (0.000000 user, 0.002527 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered reader [buffer size: 65536 chunk size: 4096]
(time (std/io/file-benchmark#test-reader _reader725_ _buffer707_))
    0.002198 secs real time
    0.002198 secs cpu time (0.000000 user, 0.002198 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered reader [buffer size: 65536 chunk size: 32768]
(time (std/io/file-benchmark#test-reader _reader725_ _buffer707_))
    0.001388 secs real time
    0.001388 secs cpu time (0.000000 user, 0.001388 system)
    no collections
    no bytes allocated
    7 minor faults
    no major faults

Output

Output Port

>>> benchmark output port [chunk size: 1024]
(time (std/io/file-benchmark#test-port-write _port442_ _junk424_ _chunk-size440_))
    0.040098 secs real time
    0.039694 secs cpu time (0.023810 user, 0.015884 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark output port [chunk size: 4096]
(time (std/io/file-benchmark#test-port-write _port442_ _junk424_ _chunk-size440_))
    0.037344 secs real time
    0.036888 secs cpu time (0.014932 user, 0.021956 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark output port [chunk size: 32768]
(time (std/io/file-benchmark#test-port-write _port442_ _junk424_ _chunk-size440_))
    0.035941 secs real time
    0.035527 secs cpu time (0.028227 user, 0.007300 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark output port [chunk size: 65536]
(time (std/io/file-benchmark#test-port-write _port442_ _junk424_ _chunk-size440_))
    0.038558 secs real time
    0.038082 secs cpu time (0.026482 user, 0.011600 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults

Writer

>>> benchmark writer [chunk size: 1024]
(time (std/io/file-benchmark#test-writer _writer601_ _junk583_ _chunk-size599_ void))
    0.019032 secs real time
    0.018750 secs cpu time (0.000002 user, 0.018748 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark writer [chunk size: 4096]
(time (std/io/file-benchmark#test-writer _writer601_ _junk583_ _chunk-size599_ void))
    0.009750 secs real time
    0.009615 secs cpu time (0.000000 user, 0.009615 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark writer [chunk size: 32768]
(time (std/io/file-benchmark#test-writer _writer601_ _junk583_ _chunk-size599_ void))
    0.007204 secs real time
    0.007133 secs cpu time (0.000000 user, 0.007133 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark writer [chunk size: 65536]
(time (std/io/file-benchmark#test-writer _writer601_ _junk583_ _chunk-size599_ void))
    0.007091 secs real time
    0.007012 secs cpu time (0.000000 user, 0.007012 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults

Buffered Writer

>>> benchmark buffered writer [buffer size: 4096 chunk size: 1024]
(time (std/io/file-benchmark#test-writer _writer514_ _junk478_ _chunk-size494_ (lambda () (std/io/interface#BufferedWriter-flush _buffered-writer512_))))
    0.010103 secs real time
    0.010103 secs cpu time (0.002011 user, 0.008092 system)
    no collections
    48 bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered writer [buffer size: 32768 chunk size: 1024]
(time (std/io/file-benchmark#test-writer _writer514_ _junk478_ _chunk-size494_ (lambda () (std/io/interface#BufferedWriter-flush _buffered-writer512_))))
    0.006864 secs real time
    0.006863 secs cpu time (0.000000 user, 0.006863 system)
    no collections
    48 bytes allocated
    7 minor faults
    no major faults
>>> benchmark buffered writer [buffer size: 65536 chunk size: 1024]
(time (std/io/file-benchmark#test-writer _writer514_ _junk478_ _chunk-size494_ (lambda () (std/io/interface#BufferedWriter-flush _buffered-writer512_))))
    0.006549 secs real time
    0.006548 secs cpu time (0.000000 user, 0.006548 system)
    no collections
    48 bytes allocated
    15 minor faults
    no major faults
>>> benchmark buffered writer [buffer size: 32768 chunk size: 4096]
(time (std/io/file-benchmark#test-writer _writer514_ _junk478_ _chunk-size494_ (lambda () (std/io/interface#BufferedWriter-flush _buffered-writer512_))))
    0.007041 secs real time
    0.007040 secs cpu time (0.000000 user, 0.007040 system)
    no collections
    48 bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered writer [buffer size: 65536 chunk size: 4096]
(time (std/io/file-benchmark#test-writer _writer514_ _junk478_ _chunk-size494_ (lambda () (std/io/interface#BufferedWriter-flush _buffered-writer512_))))
    0.006258 secs real time
    0.006258 secs cpu time (0.000001 user, 0.006257 system)
    no collections
    48 bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered writer [buffer size: 65536 chunk size: 32768]
(time (std/io/file-benchmark#test-writer _writer514_ _junk478_ _chunk-size494_ (lambda () (std/io/interface#BufferedWriter-flush _buffered-writer512_))))
    0.006480 secs real time
    0.006479 secs cpu time (0.000000 user, 0.006479 system)
    no collections
    48 bytes allocated
    15 minor faults
    no major faults

vyzo · 2023-07-20T06:17:25Z

Added 16K buffers to the benchmark; timings:

buffered reader

>>> benchmark buffered reader [buffer size: 4096 chunk size: 1024]
(time (std/io/file-benchmark#test-reader _reader725_ _buffer707_))
    0.005196 secs real time
    0.005196 secs cpu time (0.000001 user, 0.005195 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered reader [buffer size: 16384 chunk size: 1024]
(time (std/io/file-benchmark#test-reader _reader725_ _buffer707_))
    0.005642 secs real time
    0.005641 secs cpu time (0.003366 user, 0.002275 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered reader [buffer size: 32768 chunk size: 1024]
(time (std/io/file-benchmark#test-reader _reader725_ _buffer707_))
    0.005224 secs real time
    0.005224 secs cpu time (0.001126 user, 0.004098 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered reader [buffer size: 65536 chunk size: 1024]
(time (std/io/file-benchmark#test-reader _reader725_ _buffer707_))
    0.005052 secs real time
    0.005052 secs cpu time (0.001320 user, 0.003732 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered reader [buffer size: 16384 chunk size: 4096]
(time (std/io/file-benchmark#test-reader _reader725_ _buffer707_))
    0.002143 secs real time
    0.002143 secs cpu time (0.001923 user, 0.000220 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered reader [buffer size: 32768 chunk size: 4096]
(time (std/io/file-benchmark#test-reader _reader725_ _buffer707_))
    0.002125 secs real time
    0.002116 secs cpu time (0.002116 user, 0.000000 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered reader [buffer size: 65536 chunk size: 4096]
(time (std/io/file-benchmark#test-reader _reader725_ _buffer707_))
    0.002714 secs real time
    0.002705 secs cpu time (0.000000 user, 0.002705 system)
    no collections
    no bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered reader [buffer size: 65536 chunk size: 32768]
(time (std/io/file-benchmark#test-reader _reader725_ _buffer707_))
    0.002046 secs real time
    0.002046 secs cpu time (0.001246 user, 0.000800 system)
    no collections
    no bytes allocated
    6 minor faults
    no major faults

buffered writer

>>> benchmark buffered writer [buffer size: 4096 chunk size: 1024]
(time (std/io/file-benchmark#test-writer _writer514_ _junk478_ _chunk-size494_ (lambda () (std/io/interface#BufferedWriter-flush _buffered-writer512_))))
    0.009792 secs real time
    0.009790 secs cpu time (0.005768 user, 0.004022 system)
    no collections
    48 bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered writer [buffer size: 16384 chunk size: 1024]
(time (std/io/file-benchmark#test-writer _writer514_ _junk478_ _chunk-size494_ (lambda () (std/io/interface#BufferedWriter-flush _buffered-writer512_))))
    0.006996 secs real time
    0.006996 secs cpu time (0.000941 user, 0.006055 system)
    no collections
    48 bytes allocated
    3 minor faults
    no major faults
>>> benchmark buffered writer [buffer size: 32768 chunk size: 1024]
(time (std/io/file-benchmark#test-writer _writer514_ _junk478_ _chunk-size494_ (lambda () (std/io/interface#BufferedWriter-flush _buffered-writer512_))))
    0.007136 secs real time
    0.007136 secs cpu time (0.002430 user, 0.004706 system)
    no collections
    48 bytes allocated
    7 minor faults
    no major faults
>>> benchmark buffered writer [buffer size: 65536 chunk size: 1024]
(time (std/io/file-benchmark#test-writer _writer514_ _junk478_ _chunk-size494_ (lambda () (std/io/interface#BufferedWriter-flush _buffered-writer512_))))
    0.006735 secs real time
    0.006734 secs cpu time (0.000000 user, 0.006734 system)
    no collections
    48 bytes allocated
    15 minor faults
    no major faults
>>> benchmark buffered writer [buffer size: 16384 chunk size: 4096]
(time (std/io/file-benchmark#test-writer _writer514_ _junk478_ _chunk-size494_ (lambda () (std/io/interface#BufferedWriter-flush _buffered-writer512_))))
    0.006985 secs real time
    0.006984 secs cpu time (0.003099 user, 0.003885 system)
    no collections
    48 bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered writer [buffer size: 32768 chunk size: 4096]
(time (std/io/file-benchmark#test-writer _writer514_ _junk478_ _chunk-size494_ (lambda () (std/io/interface#BufferedWriter-flush _buffered-writer512_))))
    0.006655 secs real time
    0.006654 secs cpu time (0.000000 user, 0.006654 system)
    no collections
    48 bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered writer [buffer size: 65536 chunk size: 4096]
(time (std/io/file-benchmark#test-writer _writer514_ _junk478_ _chunk-size494_ (lambda () (std/io/interface#BufferedWriter-flush _buffered-writer512_))))
    0.006161 secs real time
    0.006161 secs cpu time (0.000000 user, 0.006161 system)
    no collections
    48 bytes allocated
    no minor faults
    no major faults
>>> benchmark buffered writer [buffer size: 65536 chunk size: 32768]
(time (std/io/file-benchmark#test-writer _writer514_ _junk478_ _chunk-size494_ (lambda () (std/io/interface#BufferedWriter-flush _buffered-writer512_))))
    0.006915 secs real time
    0.006911 secs cpu time (0.000000 user, 0.006911 system)
    no collections
    48 bytes allocated
    15 minor faults
    no major faults

vyzo · 2023-07-22T13:55:37Z

Rebased on master and marked as ready for review; all that's missing is documentation -- Soon[TM]

fare

Still reading the code, but already have questions on the spec.

doc/reference/interface.md

doc/reference/stdio.md

src/std/io/bio/api.ss

src/std/io/socket/api.ss

src/std/io/file.ss

src/std/io/file-test.ss

src/std/io/file-benchmark.ss

src/std/io/bio/bio-test.ss

fare

Overall nice

can use the unchecked methods

no need to recurse

- use inline methods - return smob buf to the cache - raise errors on bad continuation bytes

vyzo · 2023-08-03T15:48:51Z

Rebased on master

vyzo · 2023-08-03T16:08:34Z

Will squash merge once CI is happy.

vyzo requested review from fare, ober and drewc July 17, 2023 17:10

vyzo mentioned this pull request Jul 18, 2023

Miscellaneous fixes #729

Merged

vyzo force-pushed the stdio branch from b1f7555 to 84d5b83 Compare July 18, 2023 07:07

vyzo added this to the Gerbil18 milestone Jul 18, 2023

vyzo force-pushed the stdio branch from d2c6bf1 to dfa2fc0 Compare July 18, 2023 19:43

belmarca reviewed Jul 19, 2023

View reviewed changes

src/std/io/bio/api.ss Outdated Show resolved Hide resolved

src/std/io/bio/util.ss Outdated Show resolved Hide resolved

src/std/io/bio/bio-test.ss Show resolved Hide resolved

src/std/io/bio/util.ss Show resolved Hide resolved

belmarca reviewed Jul 20, 2023

View reviewed changes

This was referenced Jul 16, 2023

Rework the IO system to utilize Interfaces #724

Closed

Miscellaneous fixes #734

Merged

Read-Write Locks #735

Merged

vyzo force-pushed the stdio branch from 0db8c0d to 970cbb8 Compare July 22, 2023 13:39

vyzo changed the title ~~[WIP] Standard IO Interfaces~~ Standard IO Interfaces Jul 22, 2023

vyzo marked this pull request as ready for review July 22, 2023 13:55

This was referenced Jul 24, 2023

Support stdio in std/misc/json #739

Merged

(Buffered) String IO #740

Merged

Use stdio in httpd #743

Merged

fare requested changes Jul 30, 2023

View reviewed changes

fare reviewed Jul 31, 2023

View reviewed changes

src/std/io/socket/api.ss Outdated Show resolved Hide resolved

src/std/io/file.ss Show resolved Hide resolved

src/std/io/file-test.ss Show resolved Hide resolved

src/std/io/file-benchmark.ss Show resolved Hide resolved

src/std/io/bio/bio-test.ss Outdated Show resolved Hide resolved

fare approved these changes Jul 31, 2023

View reviewed changes

vyzo added 4 commits August 3, 2023 18:45

stdio: interfaces and input buffers

8feb246

doc: list interface and stdio in hamburger

2299b6d

add some bio input tests, fix some bugz

b3bfbd3

add set-input-timeout! to DatagramSocket

29d5f52

vyzo added 21 commits August 3, 2023 18:46

cast the Reader/Writer in reset!

c69ade3

ergonomics for extension methods

6bcc942

use SO_REUSEADDR for bound udp sockets

1e081b7

document stdio

d917bca

interface documentation: add tips

8cb149b

smal benchmark tweak

c1502d3

can use the unchecked methods

bio: fix bug in read-u8!

5cf68ed

bio: slightly improve bio-read-u8

434c308

no need to recurse

io: buffers were a size too big

c36dfcf

bio: remove open-string-buffered-reader from the zoo

7119a63

fix test

859b5ed

improve buffered io api ergonomics

56f62d5

add inline bio primitive ops

5ac04a3

bio/util: fix some stuff

4da4da7

- use inline methods - return smob buf to the cache - raise errors on bad continuation bytes

build-deps

293aa0c

add inline macros for read-char and write-char

5c6a244

build-deps

483d14c

improved implementation of bio read-char/peek-char/write-char

e899012

fix bugz

5abf88f

disable interrupts in bio inline methods

df082f0

rip smob cache

741ddb7

vyzo force-pushed the stdio branch from 274c142 to 741ddb7 Compare August 3, 2023 15:48

vyzo added 4 commits August 3, 2023 18:50

make our foxes jump over lazy dogs

65544e8

build-deps

98fe856

newline at EOF

436d35c

documentation fixes

49389dc

vyzo merged commit c5dc2d2 into master Aug 3, 2023

vyzo deleted the stdio branch August 3, 2023 16:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Standard IO Interfaces #728

Standard IO Interfaces #728

vyzo commented Jul 17, 2023 •

edited

Loading

vyzo commented Jul 18, 2023

vyzo commented Jul 19, 2023 •

edited

Loading

belmarca left a comment

belmarca Jul 20, 2023

vyzo Jul 20, 2023

vyzo Jul 20, 2023

vyzo commented Jul 20, 2023

vyzo commented Jul 20, 2023

vyzo commented Jul 22, 2023

fare left a comment

fare left a comment

vyzo commented Aug 3, 2023

vyzo commented Aug 3, 2023

Standard IO Interfaces #728

Standard IO Interfaces #728

Conversation

vyzo commented Jul 17, 2023 • edited Loading

vyzo commented Jul 18, 2023

vyzo commented Jul 19, 2023 • edited Loading

belmarca left a comment

Choose a reason for hiding this comment

belmarca Jul 20, 2023

Choose a reason for hiding this comment

vyzo Jul 20, 2023

Choose a reason for hiding this comment

vyzo Jul 20, 2023

Choose a reason for hiding this comment

vyzo commented Jul 20, 2023

Input

Input Port

Reader

Buffered Reader

Output

Output Port

Writer

Buffered Writer

vyzo commented Jul 20, 2023

buffered reader

buffered writer

vyzo commented Jul 22, 2023

fare left a comment

Choose a reason for hiding this comment

fare left a comment

Choose a reason for hiding this comment

vyzo commented Aug 3, 2023

vyzo commented Aug 3, 2023

vyzo commented Jul 17, 2023 •

edited

Loading

vyzo commented Jul 19, 2023 •

edited

Loading