Upstream sync 2024 06 12 #302

robertgshaw2-redhat · 2024-06-12T02:19:18Z

Upstream sync 2024 06 12 (#302) - v0.5.0 of vllm

SUMMARY:

Merge commits from vllm-project@114332b to vllm-project@8f89d72
Our GCP test instances do not have gcc or clang installed. All of the triton kernels rely on the gcc and clang to generate JITs. These are still disabled (cc @andy-neuma). All are marked with:

@pytest.mark.skip("C compiler not installed in NM automation. "
                  "This codepath follows a triton pathway, which "
                  "JITs using clang or gcc. Since neither are installed "
                  "in our test instances, we need to skip this for now.")

Note that vllm-project@114332b is NOT included in this merge.

COMPARE vs UPSTREAM:

https://github.com/neuralmagic/nm-vllm/compare/upstream-sync-2024-06-11..vllm-project:vllm:v0.5.0

Signed-off-by: kevin <[email protected]>

…ned --max-logprobs (vllm-project#5312) Co-authored-by: DarkLight1337 <[email protected]>

Co-authored-by: DarkLight1337 <[email protected]> Co-authored-by: ywang96 <[email protected]>

…llm-project#5324) Co-authored-by: simon-mo <[email protected]>

…e == 1 (vllm-project#5254)

…se (vllm-project#5230) Co-authored-by: Antoni Baum <[email protected]>

mgoin

LGTM!

mgoin and others added 18 commits June 12, 2024 02:17

[Doc] Add documentation for FP8 W8A8 (vllm-project#5388)

1535153

[ci] Fix Buildkite agent path (vllm-project#5392)

10e0353

Signed-off-by: kevin <[email protected]>

[Misc] Various simplifications and typing fixes (vllm-project#5368)

c098739

[Bugfix] OpenAI entrypoint limits logprobs while ignoring server defi…

74eb6ab

…ned --max-logprobs (vllm-project#5312) Co-authored-by: DarkLight1337 <[email protected]>

[Bugfix][Frontend] Cleanup "fix chat logprobs" (vllm-project#5026)

4be9ac5

[Doc] add debugging tips (vllm-project#5409)

cf83abb

[Doc][Typo] Fixing Missing Comma (vllm-project#5403)

4ec29eb

[Misc] Remove VLLM_BUILD_WITH_NEURON env variable (vllm-project#5389)

76796b1

[CI] docfix (vllm-project#5410)

c8e3f5b

Co-authored-by: DarkLight1337 <[email protected]> Co-authored-by: ywang96 <[email protected]>

[Speculative decoding] Initial spec decode docs (vllm-project#5400)

41de77c

[Doc] Add an automatic prefix caching section in vllm documentation (v…

cf489b2

…llm-project#5324) Co-authored-by: simon-mo <[email protected]>

[Docs] [Spec decode] Fix docs error in code example (vllm-project#5427)

fc135fa

[Bugfix] Fix MultiprocessingGPUExecutor.check_health when world_siz…

7407079

…e == 1 (vllm-project#5254)

[Bugfix] fix lora_dtype value type in arg_utils.py (vllm-project#5398)

cbb4376

[Frontend] Customizable RoPE theta (vllm-project#5197)

5c2abcf

[Core][Distributed] add same-node detection (vllm-project#5369)

7c70e49

[Core][Doc] Default to multiprocessing for single-node distributed ca…

a925bdf

…se (vllm-project#5230) Co-authored-by: Antoni Baum <[email protected]>

[Doc] add common case for long waiting time (vllm-project#5430)

041b5c1

mgoin approved these changes Jun 12, 2024

View reviewed changes

robertgshaw2-redhat added 3 commits June 12, 2024 17:57

fix test compressed.py

cb285f4

fix calling test_same_node

1646e18

format

415d6f7

robertgshaw2-redhat merged commit 5aaec10 into main Jun 12, 2024
37 checks passed

robertgshaw2-redhat deleted the upstream-sync-2024-06-12 branch June 12, 2024 22:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upstream sync 2024 06 12 #302

Upstream sync 2024 06 12 #302

robertgshaw2-redhat commented Jun 12, 2024 •

edited

Loading

mgoin left a comment

Upstream sync 2024 06 12 #302

Upstream sync 2024 06 12 #302

Conversation

robertgshaw2-redhat commented Jun 12, 2024 • edited Loading

mgoin left a comment

Choose a reason for hiding this comment

robertgshaw2-redhat commented Jun 12, 2024 •

edited

Loading