Skip to content

Commit

Permalink
Update benchmarking user guide for reading private datasets
Browse files Browse the repository at this point in the history
  • Loading branch information
katxiao committed May 12, 2021
1 parent f4bfb07 commit 9329133
Showing 1 changed file with 22 additions and 1 deletion.
23 changes: 22 additions & 1 deletion docs/user_guides/benchmarking/run.rst
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,10 @@ The ``sdgym.run`` function accepts the following arguments:
- ``output_path (str)``:: If an ``output_path`` is given, the generated
leaderboard will be stored in the indicated path as a CSV file. The
given path must be a complete path including the ``.csv`` filename.
- ``aws_key (str)``:: If an ``aws_key`` is provided, the given access
key id will be used to read from the specified bucket.
- ``aws_secret (str)``:: If an ``aws_secret`` is provided, the given
secret access key will be used to read from the specified bucket.

Synthesizers
~~~~~~~~~~~~
Expand Down Expand Up @@ -167,6 +171,23 @@ For example, if we want to evaluate our synthesizer function on only the
In [15]: scores = sdgym.run(my_synthesizer_function, datasets=datasets)
Running on Private Datasets
~~~~~~~~

If we want to run sdgym on datasets in a private s3 bucket, we can pass in
the s3 bucket name, along with the aws access key ID and the aws secret
access key. For example:

.. code:: python3
In [16]: scores = sdgym.run(
...: my_synthesizer_function,
...: datasets=datasets,
...: bucket='my-bucket',
...: aws_key='my-key',
...: aws_secret='my-secret'
...: )
Iterations
~~~~~~~~~~

Expand All @@ -181,4 +202,4 @@ combination can be altered by passing a different value to the

.. code:: python3
In [16]: scores = sdgym.run(my_synthesizer_function, iterations=10)
In [17]: scores = sdgym.run(my_synthesizer_function, iterations=10)

0 comments on commit 9329133

Please sign in to comment.