Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: use webgpu for FlowerCaptioner #135

Draft
wants to merge 22 commits into
base: master
Choose a base branch
from
Draft

feat: use webgpu for FlowerCaptioner #135

wants to merge 22 commits into from

Conversation

cristianglezm
Copy link
Owner

@cristianglezm cristianglezm commented Oct 19, 2024 •
edited
Loading

TODO

  • cache manager (to delete models files)
  • make model options component for settings
  • refactor settings - split into components
    • AppOptions
    • CreationParams
    • mutationsRates
    • AppActions
    • make new layout for settings
  • change xenova/transformers to huggingface/transformers
  • test all models
    • fp32 works on CPU(it takes ~13s), GPU (it takes ~1m30s)
    • fp16 gives exception
    • q8 works on CPU (Firefox, Chrome, Edge)
    • q8 gives garbled output on Chrome and Edge (GPU)
    • int8 gives error on session creation (Can't create a session. ERROR_CODE: 9, ERROR_MESSAGE: Could not find an implementation for ConvInteger(10) node with name '/embeddings/patch_embeddings/projection/Conv_quant')
    • uint8 works on CPU
    • q4 works on CPU (it takes ~24s)
    • q4f16 gives exception.
    • bnb4 - works on CPU (it takes ~24s)
  • fix not working on Chrome or Edge GPU (Garbled descriptions)
  • Firefox webGPU not supported yet. (wait for it)

@cristianglezm cristianglezm self-assigned this Oct 19, 2024
@cristianglezm cristianglezm added enhancement New feature or request javascript Pull requests that update Javascript code labels Oct 19, 2024
@cristianglezm cristianglezm force-pushed the hf-transformers-v3 branch 2 times, most recently from 86b532a to 79f3f59 Compare November 15, 2024 16:20
@cristianglezm
changed xenova/transfomers to huggingface/transformers
da58b6a
@cristianglezm
Bump @huggingface/transformers from 3.0.0-alpha.15 to 3.0.0
07b42fb
@cristianglezm
adapted to use webgpu or cpu if gpu not available.
a997334
@cristianglezm
added code to test local onnx model.
d190362
@cristianglezm
update TODO.md
7a37611
@cristianglezm
added default value to pagination component prop anchor
ae28ca9
@cristianglezm
Bump @huggingface/transformers from 3.0.0 to 3.0.1
612429c
@cristianglezm
added ModelOptions to AIStore
6c78a68 
* added some utility functions to Captioner
@cristianglezm
fixed mobile style error for errorModal.
e2958fc
@cristianglezm
check if Captioner model is loaded instead of loadModel option in on.
35c8300 
* LoadModel option is used to load the model after website has loaded.
* fixed code style error.
@cristianglezm
added CacheManager component
d1aaadd
@cristianglezm
added ModelOptions component for settings
f9d5040
@cristianglezm
updated TODO
8ad84e9
@cristianglezm
updated TODO
980cfdd
@cristianglezm
added reload fn to CacheManager
1e0a510 
* fixed delete host confirm message
@cristianglezm
Bump @huggingface/transformers from 3.0.1 to 3.0.2
32ed797
@cristianglezm
refactored settings
1328a7a 
* added isModelInCache to ModelOptions.vue
* added HasFiles to CacheManager class
* added AppActions
* added AppOptions
* added CreationParams
* added MutationRates
@cristianglezm
refactored Captioner worker
4b75a19
@cristianglezm
updated TODO
b1c100a
@cristianglezm
fixed DescriptionModal overflowing window
bd31db5
@cristianglezm
updated TODO
4135e7b
@cristianglezm cristianglezm mentioned this pull request Nov 22, 2024
Open
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@cristianglezm cristianglezm

Labels
enhancement New feature or request javascript Pull requests that update Javascript code
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@cristianglezm