Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bugfix] Cleanup Pixtral HF code #11333

Merged
merged 1 commit into from
Dec 19, 2024
Merged

[Bugfix] Cleanup Pixtral HF code #11333

merged 1 commit into from
Dec 19, 2024

Conversation

DarkLight1337
Copy link
Member

@DarkLight1337 DarkLight1337 commented Dec 19, 2024 •
edited
Loading

This PR fixes incorrect max image tokens for Pixtral HF, and also cleans up unused code that are already integrated into the merged multi-modal processor.

@DarkLight1337 DarkLight1337 added the ready ONLY add when PR is ready to merge/full CI is needed label Dec 19, 2024
@github-actions GitHub Actions
Copy link

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

  • Add ready label to the PR
  • Enable auto-merge.

🚀

@DarkLight1337 DarkLight1337 requested a review from mgoin December 19, 2024 10:36
@DarkLight1337
Cleanup Pixtral HF code
e9eddeb 
Signed-off-by: DarkLight1337 <[email protected]>
Isotr0py
Isotr0py approved these changes Dec 19, 2024
View reviewed changes
Copy link
Collaborator

@Isotr0py Isotr0py left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the cleanup!

@Isotr0py Isotr0py enabled auto-merge (squash) December 19, 2024 11:32
@Isotr0py Isotr0py merged commit a0f7d53 into vllm-project:main Dec 19, 2024
57 checks passed
mgoin
mgoin reviewed Dec 19, 2024
View reviewed changes
Copy link
Member

@mgoin mgoin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for getting rid of some code! Just one question

vllm/model_executor/models/pixtral.py
item_size=image_feature_size)
}
# Consider the image_break_token
return (grid_length + 1) * grid_length
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this have a plus one at the end for the end token?

Copy link
Member Author

@DarkLight1337 DarkLight1337 Dec 19, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is intentional. The reason is that now we include the image_break_token in the multi-modal placeholder tokens.

@DarkLight1337 DarkLight1337 deleted the cleanup-pixtral-hf branch December 19, 2024 14:32
BKitor pushed a commit to BKitor/vllm that referenced this pull request Dec 30, 2024
@DarkLight1337 @BKitor
[Bugfix] Cleanup Pixtral HF code (vllm-project#11333)
2772fba 
Signed-off-by: DarkLight1337 <[email protected]>
joennlae pushed a commit to 44ai-labs/vllm that referenced this pull request Jan 19, 2025
@DarkLight1337
[Bugfix] Cleanup Pixtral HF code (vllm-project#11333)
9470cb7 
Signed-off-by: DarkLight1337 <[email protected]>
joennlae pushed a commit to 44ai-labs/vllm that referenced this pull request Jan 19, 2025
@DarkLight1337
[Bugfix] Cleanup Pixtral HF code (vllm-project#11333)
6fc4bdb 
Signed-off-by: DarkLight1337 <[email protected]>
abmfy pushed a commit to abmfy/vllm-flashinfer that referenced this pull request Jan 24, 2025
@DarkLight1337 @abmfy
[Bugfix] Cleanup Pixtral HF code (vllm-project#11333)
9afd3e4 
Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: Bowen Wang <[email protected]>
abmfy pushed a commit to abmfy/vllm-flashinfer that referenced this pull request Jan 24, 2025
@DarkLight1337 @abmfy
[Bugfix] Cleanup Pixtral HF code (vllm-project#11333)
8995ecb 
Signed-off-by: DarkLight1337 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mgoin mgoin mgoin left review comments

@Isotr0py Isotr0py Isotr0py approved these changes

Assignees
No one assigned
Labels
ready ONLY add when PR is ready to merge/full CI is needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@DarkLight1337 @mgoin @Isotr0py