Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: optimize retries #854

Merged
merged 89 commits into from
Aug 17, 2023
Merged

feat: optimize retries #854

merged 89 commits into from
Aug 17, 2023

Commits on Jul 24, 2023

  1. added function for populating shard

    @daniel-sanche
    daniel-sanche committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    18650af View commit details
    Browse the repository at this point in the history

  2. Merge branch 'v3' into benchmarks_prod

    @daniel-sanche
    daniel-sanche committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    df05171 View commit details
    Browse the repository at this point in the history

  3. got table population working

    @daniel-sanche
    daniel-sanche committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    4db25b3 View commit details
    Browse the repository at this point in the history

  4. finished table population

    @daniel-sanche
    daniel-sanche committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    28cc1f4 View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2023

  1. pulled out table setup fixtures from system tests

    @daniel-sanche
    daniel-sanche committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    9e97ac0 View commit details
    Browse the repository at this point in the history

  2. get system tests working

    @daniel-sanche
    daniel-sanche committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    52550cd View commit details
    Browse the repository at this point in the history

  3. added some basic throughput code

    @daniel-sanche
    daniel-sanche committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    a191236 View commit details
    Browse the repository at this point in the history

  4. count operations

    @daniel-sanche
    daniel-sanche committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    31dbbd9 View commit details
    Browse the repository at this point in the history

  5. added splits to table

    @daniel-sanche
    daniel-sanche committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    eac206a View commit details
    Browse the repository at this point in the history

  6. extracted cluster config

    @daniel-sanche
    daniel-sanche committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    ea771de View commit details
    Browse the repository at this point in the history

  7. chanted test name

    @daniel-sanche
    daniel-sanche committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    2eda0ac View commit details
    Browse the repository at this point in the history

  8. improved benchmark output

    @daniel-sanche
    daniel-sanche committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    f0649fd View commit details
    Browse the repository at this point in the history

  9. fixed population to use new format

    @daniel-sanche
    daniel-sanche committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    fbf2636 View commit details
    Browse the repository at this point in the history

  10. added sharded scan benchmark

    @daniel-sanche
    daniel-sanche committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    21df0c2 View commit details
    Browse the repository at this point in the history

  11. more accurate time tracking

    @daniel-sanche
    daniel-sanche committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    91ef008 View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2023

  1. added script to deploy to GCE

    @daniel-sanche
    daniel-sanche committed Jul 26, 2023
    Configuration menu
    Copy the full SHA
    f91750f View commit details
    Browse the repository at this point in the history

  2. limit to one qualifier

    @daniel-sanche
    daniel-sanche committed Jul 26, 2023
    Configuration menu
    Copy the full SHA
    0236f85 View commit details
    Browse the repository at this point in the history

  3. added separate test for point reads

    @daniel-sanche
    daniel-sanche committed Jul 26, 2023
    Configuration menu
    Copy the full SHA
    fd94350 View commit details
    Browse the repository at this point in the history

  4. made row size configurable

    @daniel-sanche
    daniel-sanche committed Jul 26, 2023
    Configuration menu
    Copy the full SHA
    82ecaf2 View commit details
    Browse the repository at this point in the history

  5. don't timeout benchmarks at deadline

    @daniel-sanche
    daniel-sanche committed Jul 26, 2023
    Configuration menu
    Copy the full SHA
    a35f985 View commit details
    Browse the repository at this point in the history

  6. added fastapi benchmark

    @daniel-sanche
    daniel-sanche committed Jul 26, 2023
    Configuration menu
    Copy the full SHA
    a440d76 View commit details
    Browse the repository at this point in the history

  7. added driver for fastapi class

    @daniel-sanche
    daniel-sanche committed Jul 26, 2023
    Configuration menu
    Copy the full SHA
    d0fcdb9 View commit details
    Browse the repository at this point in the history

Commits on Jul 27, 2023

  1. delete VM on complete; improved formatting

    @daniel-sanche
    daniel-sanche committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    0bb70ef View commit details
    Browse the repository at this point in the history

  2. improved fastapi benchmark

    @daniel-sanche
    daniel-sanche committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    3f3f59a View commit details
    Browse the repository at this point in the history

  3. run tests concurrently

    @daniel-sanche
    daniel-sanche committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    5a6ffad View commit details
    Browse the repository at this point in the history

  4. saved test_system changes

    @daniel-sanche
    daniel-sanche committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    604071f View commit details
    Browse the repository at this point in the history

  5. added profling to point reads

    @daniel-sanche
    daniel-sanche committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    3686ef6 View commit details
    Browse the repository at this point in the history

  6. add duration to run script

    @daniel-sanche
    daniel-sanche committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    c42f596 View commit details
    Browse the repository at this point in the history

Commits on Jul 28, 2023

  1. build async generator manually

    @daniel-sanche
    daniel-sanche committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    e2ac258 View commit details
    Browse the repository at this point in the history

  2. lazy parse cell data

    @daniel-sanche
    daniel-sanche committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    1228c8a View commit details
    Browse the repository at this point in the history

  3. added profiling to scans

    @daniel-sanche
    daniel-sanche committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    1a3a8e7 View commit details
    Browse the repository at this point in the history

  4. removed iscouroutine check

    @daniel-sanche
    daniel-sanche committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    0659161 View commit details
    Browse the repository at this point in the history

  5. added legacy benchmark

    @daniel-sanche
    daniel-sanche committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    ce2b733 View commit details
    Browse the repository at this point in the history

  6. added thread to legacy tests

    @daniel-sanche
    daniel-sanche committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    4714e8a View commit details
    Browse the repository at this point in the history

  7. cut out row adapter

    @daniel-sanche
    daniel-sanche committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    3111543 View commit details
    Browse the repository at this point in the history

  8. lazy load rows

    @daniel-sanche
    daniel-sanche committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    c717a0b View commit details
    Browse the repository at this point in the history

Commits on Jul 29, 2023

  1. simplify generator layers for non-stremaing

    @daniel-sanche
    daniel-sanche committed Jul 29, 2023
    Configuration menu
    Copy the full SHA
    b5cb253 View commit details
    Browse the repository at this point in the history

  2. change pool size

    @daniel-sanche
    daniel-sanche committed Jul 29, 2023
    Configuration menu
    Copy the full SHA
    aa4bed4 View commit details
    Browse the repository at this point in the history

Commits on Jul 30, 2023

  1. disabled lazy loading (75k rps)

    @daniel-sanche
    daniel-sanche committed Jul 30, 2023
    Configuration menu
    Copy the full SHA
    613b032 View commit details
    Browse the repository at this point in the history

  2. only check for last_scanned when there are no chunks (81k rps)

    @daniel-sanche
    daniel-sanche committed Jul 30, 2023
    Configuration menu
    Copy the full SHA
    8e37967 View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2023

  1. refactored chunk parsing (80k rps)

    @daniel-sanche
    daniel-sanche committed Jul 31, 2023
    Configuration menu
    Copy the full SHA
    6775e2b View commit details
    Browse the repository at this point in the history

  2. removed subclass for row

    @daniel-sanche
    daniel-sanche committed Jul 31, 2023
    Configuration menu
    Copy the full SHA
    537f4ff View commit details
    Browse the repository at this point in the history

  3. added igor's row merger

    @daniel-sanche
    daniel-sanche committed Jul 31, 2023
    Configuration menu
    Copy the full SHA
    58b3fbb View commit details
    Browse the repository at this point in the history

Commits on Aug 7, 2023

  1. use stream by default

    @daniel-sanche
    daniel-sanche committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    a7478ab View commit details
    Browse the repository at this point in the history

  2. use our models

    @daniel-sanche
    daniel-sanche committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    689f072 View commit details
    Browse the repository at this point in the history

  3. wrapped in object

    @daniel-sanche
    daniel-sanche committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    107b8c5 View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2023

  1. updated row merger for acceptance tests

    @daniel-sanche
    daniel-sanche committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    d76a7dc View commit details
    Browse the repository at this point in the history

  2. improve performance

    @daniel-sanche
    daniel-sanche committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    1d9a475 View commit details
    Browse the repository at this point in the history

  3. removed HasField from family

    @daniel-sanche
    daniel-sanche committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    37bc3a0 View commit details
    Browse the repository at this point in the history

  4. added request revision logic

    @daniel-sanche
    daniel-sanche committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    ea95dcc View commit details
    Browse the repository at this point in the history

  5. added back split cell verifications

    @daniel-sanche
    daniel-sanche committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    9b60693 View commit details
    Browse the repository at this point in the history

  6. got all acceptance tests passing

    @daniel-sanche
    daniel-sanche committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    24de29e View commit details
    Browse the repository at this point in the history

  7. track last yielded key

    @daniel-sanche
    daniel-sanche committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    9c9ba5e View commit details
    Browse the repository at this point in the history

  8. track reamining count

    @daniel-sanche
    daniel-sanche committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    5d49ad8 View commit details
    Browse the repository at this point in the history

  9. added retries

    @daniel-sanche
    daniel-sanche committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    5ecfa3e View commit details
    Browse the repository at this point in the history

  10. added exception conversion

    @daniel-sanche
    daniel-sanche committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    13603ea View commit details
    Browse the repository at this point in the history

  11. moved into single function

    @daniel-sanche
    daniel-sanche committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    b40e1cb View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2023

  1. Fixed style issues

    @daniel-sanche
    daniel-sanche committed Aug 9, 2023
    Configuration menu
    Copy the full SHA
    0f58066 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'v3' into optimize_read_rows_2

    @daniel-sanche
    daniel-sanche committed Aug 9, 2023
    Configuration menu
    Copy the full SHA
    07341da View commit details
    Browse the repository at this point in the history

  3. fixed row and helpers tests

    @daniel-sanche
    daniel-sanche committed Aug 9, 2023
    Configuration menu
    Copy the full SHA
    90b85b3 View commit details
    Browse the repository at this point in the history

  4. fixed some read rows tests

    @daniel-sanche
    daniel-sanche committed Aug 9, 2023
    Configuration menu
    Copy the full SHA
    4298e38 View commit details
    Browse the repository at this point in the history

  5. comments and cleanup

    @daniel-sanche
    daniel-sanche committed Aug 9, 2023
    Configuration menu
    Copy the full SHA
    62f1bf7 View commit details
    Browse the repository at this point in the history

  6. fixed issues with conformance tests

    @daniel-sanche
    daniel-sanche committed Aug 9, 2023
    Configuration menu
    Copy the full SHA
    98510c3 View commit details
    Browse the repository at this point in the history

  7. improved proto conversion

    @daniel-sanche
    daniel-sanche committed Aug 9, 2023
    Configuration menu
    Copy the full SHA
    4cf4aab View commit details
    Browse the repository at this point in the history

Commits on Aug 10, 2023

  1. fixed read_rows tests

    @daniel-sanche
    daniel-sanche committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    0f0ccee View commit details
    Browse the repository at this point in the history

  2. fixed tests

    @daniel-sanche
    daniel-sanche committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    42f44da View commit details
    Browse the repository at this point in the history

  3. optimizing query class

    @daniel-sanche
    daniel-sanche committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    3d1804c View commit details
    Browse the repository at this point in the history

  4. keep filter and limit out of proto storage

    @daniel-sanche
    daniel-sanche committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    b0882af View commit details
    Browse the repository at this point in the history

  5. fixed tests

    @daniel-sanche
    daniel-sanche committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    2f2286d View commit details
    Browse the repository at this point in the history

  6. skip check when not necessary

    @daniel-sanche
    daniel-sanche committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    fbe298e View commit details
    Browse the repository at this point in the history

  7. experiment: yield in row batches

    @daniel-sanche
    daniel-sanche committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    2a82343 View commit details
    Browse the repository at this point in the history

  8. Revert "experiment: yield in row batches" 

    This reverts commit 2a82343.
    @daniel-sanche
    daniel-sanche committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    4e6dd6f View commit details
    Browse the repository at this point in the history

  9. removed benchmarks dir

    @daniel-sanche
    daniel-sanche committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    daa7a59 View commit details
    Browse the repository at this point in the history

  10. fixed type annotation

    @daniel-sanche
    daniel-sanche committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    e91d693 View commit details
    Browse the repository at this point in the history

  11. added slots to query

    @daniel-sanche
    daniel-sanche committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    cb28451 View commit details
    Browse the repository at this point in the history

Commits on Aug 14, 2023

  1. use separate instances for each run

    @daniel-sanche
    daniel-sanche committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    1dd4d0c View commit details
    Browse the repository at this point in the history

  2. made _ReadRowOperation into slots

    @daniel-sanche
    daniel-sanche committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    6d2dab6 View commit details
    Browse the repository at this point in the history

  3. clean up style issues

    @daniel-sanche
    daniel-sanche committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    309405a View commit details
    Browse the repository at this point in the history

  4. removed commented test

    @daniel-sanche
    daniel-sanche committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    e36d70b View commit details
    Browse the repository at this point in the history

Commits on Aug 15, 2023

  1. use optimizations from retries

    @daniel-sanche
    daniel-sanche committed Aug 15, 2023
    Configuration menu
    Copy the full SHA
    eb4bcfa View commit details
    Browse the repository at this point in the history

  2. removed layer of wrapping

    @daniel-sanche
    daniel-sanche committed Aug 15, 2023
    Configuration menu
    Copy the full SHA
    f6e8cd2 View commit details
    Browse the repository at this point in the history

  3. fixed tests

    @daniel-sanche
    daniel-sanche committed Aug 15, 2023
    Configuration menu
    Copy the full SHA
    5037891 View commit details
    Browse the repository at this point in the history

  4. updated retryable stream submodule

    @daniel-sanche
    daniel-sanche committed Aug 15, 2023
    Configuration menu
    Copy the full SHA
    ea5b4f9 View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2023

  1. updated dependency version

    @daniel-sanche
    daniel-sanche committed Aug 16, 2023
    Configuration menu
    Copy the full SHA
    9456196 View commit details
    Browse the repository at this point in the history

  2. added todo

    @daniel-sanche
    daniel-sanche committed Aug 16, 2023
    Configuration menu
    Copy the full SHA
    b9f8cdf View commit details
    Browse the repository at this point in the history

Commits on Aug 17, 2023

  1. Merge branch 'v3' into optimize_retries

    @daniel-sanche
    daniel-sanche authored Aug 17, 2023
    Configuration menu
    Copy the full SHA
    fa0734c View commit details
    Browse the repository at this point in the history

  2. moved exception builder into own method

    @daniel-sanche
    daniel-sanche committed Aug 17, 2023
    Configuration menu
    Copy the full SHA
    3323a66 View commit details
    Browse the repository at this point in the history

  3. updated constraints

    @daniel-sanche
    daniel-sanche committed Aug 17, 2023
    Configuration menu
    Copy the full SHA
    76d6df7 View commit details
    Browse the repository at this point in the history

  4. fixed lint issues

    @daniel-sanche
    daniel-sanche committed Aug 17, 2023
    Configuration menu
    Copy the full SHA
    bf669ad View commit details
    Browse the repository at this point in the history