[RLlib; Offline RL] Implement CQL algorithm logic in new API stack.#47000

Merged

sven1977 merged 41 commits intoray-project:masterfrom simonsays1980:cql-algorithm

Aug 19, 2024

+338-54

Commits on Aug 2, 2024

Added CQLLearner and CQLTorchLearner.
simonsays1980
committed

Commits on Aug 7, 2024

Commits on Aug 12, 2024

Moved optimizing from 'compute_loss_for_module' to 'compute_gradients'.
simonsays1980
committed
Changed logging training iterations to a simpler logic proposed by @sven1977 .
simonsays1980
committed
Merge branch 'master' into cql-learner
simonsays1980
committed
Merge branch 'master' into cql-algorithm
simonsays1980
committed
Merge branch 'cql-learner' into cql-algorithm
simonsays1980
committed
Added '__init__.py' to 'cql/torch'.
simonsays1980
committed
Moved connector pipeline setup to 'build_learner_connector' from learners for CQL, DQN, SAC, and MARWIL. Remvoed CQLLearner. Added further comments fromn @sven1977 's review.
simonsays1980
committed
Changed 'joinpath' to '/' in poaths for examples of BC, MARWIL, CQL.
simonsays1980
committed

Commits on Aug 13, 2024

Commits on Aug 14, 2024

[RLlib; Offline RL] Implement twin-Q net option for CQL. (ray-project#47105 )
simonsays1980
committed
[core] remove unused GcsAio(Publisher|Subscriber) methods and subclasses. (ray-project#47057 )

rynewang
authored and
simonsays1980
committed
[Core] Fix a bug where we submit the actor creation task to the wrong submitter (ray-project#47109 )

jjyao
authored and
simonsays1980
committed
[doc][build] Update all changed files timestamp to latest (ray-project#47115 )

khluu
authored and
simonsays1980
committed
[serve] split test_proxy.py into unit and e2e tests (ray-project#47112 )

zcin
authored and
simonsays1980
committed
[Utility] add env_float utility into ray._private.ray_constants (ray-project#47117 )

hongpeng-guo
authored and
simonsays1980
committed
[Data] Fix progress bars not showing % progress (ray-project#47120 )

scottjlee
authored and
simonsays1980
committed
[data] change data17 to datal (ray-project#47082 )

aslonnie
authored and
simonsays1980
committed
[ci] change data build for all python versions to arrow 17 (ray-project#47121 )

can-anyscale
authored and
simonsays1980
committed
Fixed a small buck that came with a change in validation.
simonsays1980
committed

Commits on Aug 15, 2024

Commits on Aug 16, 2024

[core] make a note GetThreadContext() can't be used in fiber context (ray-project#47151 )

hongchaodeng
authored and
simonsays1980
committed
docs: introduce quickstart button for image classification (ray-project#46715 )

saihaj
authored and
simonsays1980
committed
[Data] Fix validation bug when size=0 in ActorPoolStrategy (ray-project#47072 )

xingyu-long
authored and
simonsays1980
committed
[Data] Fix exception in async map (ray-project#47110 )

Bye-legumes
authored and
simonsays1980
committed
[Data] Progress Bar: Sort sample in "rows" and remove the duplicate Sort sample. (ray-project#47106 )

Bye-legumes
authored and
simonsays1980
committed
Add iceberg datasource (ray-project#46889 )

authored and
simonsays1980
committed
[observability][export-api] Add event data schema for nodes (ray-project#47086 )

nikitavemuri
authored and
simonsays1980
committed
[autoscaler] fix import not used lint error on typing (ray-project#47163 )

aslonnie
authored and
simonsays1980
committed
Merge branch 'master' into cql-algorithm
simonsays1980
authored