Code

[VLM][Model] TP support for ViTs#7186

Code

Merged

WoosukKwon merged 33 commits intovllm-project:mainvllm-project/vllm:mainfrom ChristopherCho:tp-support-for-vitChristopherCho/vllm:tp-support-for-vitCopy head branch name to clipboardAug 30, 2024

+340-289

Commits

Commits on Aug 6, 2024

• feat: replace siglipattention with tp'ed one

  ChristopherCho

  committed
  0919bd1

  Copy full SHA for 0919bd1

• feat: tp blip attention

  ChristopherCho

  committed
  7cfc98c

  Copy full SHA for 7cfc98c

• feat: clip attention replaced

  ChristopherCho

  committed
  079c53f

  Copy full SHA for 079c53f

• fix: style

  ChristopherCho

  committed
  9a9af50

  Copy full SHA for 9a9af50

• fix: provide qunatization config

  ChristopherCho

  committed
  8176c8e

  Copy full SHA for 8176c8e

• fix: return value of attention

  ChristopherCho

  committed
  b3bdbef

  Copy full SHA for b3bdbef

• fix: add tp config in clip attention

  ChristopherCho

  committed
  8927414

  Copy full SHA for 8927414

• feat: tp attention in intern vit

  ChristopherCho

  committed
  22a0a84

  Copy full SHA for 22a0a84

• fix: style fix

  ChristopherCho

  committed
  f6063b1

  Copy full SHA for f6063b1

• feat: weight loading for clip based models

  ChristopherCho

  committed
  8e54ef6

  Copy full SHA for 8e54ef6

• feat: weight loading for siglip based models

  ChristopherCho

  committed
  87043e4

  Copy full SHA for 87043e4

• feat: weight loading for blip based models

  ChristopherCho

  committed
  2c18f3a

  Copy full SHA for 2c18f3a

• fix: bug in clip weight loading

  ChristopherCho

  committed
  c6015f5

  Copy full SHA for c6015f5

• fix: bug in clip attention

  ChristopherCho

  committed
  be0e190

  Copy full SHA for be0e190

• fix: bug in blip attention

  ChristopherCho

  committed
  0121445

  Copy full SHA for 0121445

• fix: blip does not require sharding

  ChristopherCho

  committed
  414040f

  Copy full SHA for 414040f

Commits on Aug 7, 2024

• fix: style

  ChristopherCho

  committed
  f28aec3

  Copy full SHA for f28aec3

• fix: phi3v weight loading logic fixed

  ChristopherCho

  committed
  734fcb1

  Copy full SHA for 734fcb1

• fix: make intern vit working

  ChristopherCho

  committed
  f1329c9

  Copy full SHA for f1329c9

• fix: fix for tp input

  ChristopherCho

  committed
  c776662

  Copy full SHA for c776662

Commits on Aug 8, 2024

• fix: minor refactoring

  ChristopherCho

  committed
  5ad0d22

  Copy full SHA for 5ad0d22

Commits on Aug 26, 2024

• Merge branch 'main' into tp-support-for-vit

  ywang96

  authored
  0a9968e

  Copy full SHA for 0a9968e

Commits on Aug 27, 2024

• format

  ywang96

  committed
  70580a6

  Copy full SHA for 70580a6

Commits on Aug 28, 2024

• cleanup TODO

  ywang96

  committed
  0593206

  Copy full SHA for 0593206

Commits on Aug 30, 2024

• Merge branch 'main' into tp-support-for-vit

  ChristopherCho

  committed
  ccc49dc

  Copy full SHA for ccc49dc

• doc: Todo for adding prefix in clip load weights

  ChristopherCho

  committed
  31af673

  Copy full SHA for 31af673

• Fix: use view rather than reshape and contiguous

  Show description for 3652cb9
  

  ChristopherCho

  and

  ywang96

  authored
  3652cb9

  Copy full SHA for 3652cb9

• Fix: use view rather than reshape and contiguous in clip

  Show description for 21a146d
  

  ChristopherCho

  and

  ywang96

  authored
  21a146d

  Copy full SHA for 21a146d

• Fix: use view rather than reshape and contiguous in siglip

  Show description for 2d1f639
  

  ChristopherCho

  and

  ywang96

  authored
  2d1f639

  Copy full SHA for 2d1f639

• Merge branch 'tp-support-for-vit' of https://github.com/ChristopherCho/vllm into tp-support-for-vit

  ChristopherCho

  committed
  300e8a9

  Copy full SHA for 300e8a9

• feat: option for disabling bias in blip

  ChristopherCho

  committed
  659adc5

  Copy full SHA for 659adc5

• Merge branch 'vllm-project:main' into tp-support-for-vit

  ChristopherCho

  authored
  ebf3503

  Copy full SHA for ebf3503

• patch internvl

  ywang96

  committed
  ffb176b

  Copy full SHA for ffb176b