Addition of test code for GPTNeoX Flax support #24002

gojiteji · 2023-06-05T02:53:15Z

I have added a test code for the GPTNeoX Flax support #22950.

I implemented it based on a fork at https://github.com/OhadRubin/transformers the above PR and Flax GPT-Neo test code.

During the execution of the tests based on the doc, the log displayed the following output:

platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
rootdir: /myhomedir/transformers
configfile: setup.cfg
plugins: anyio-3.6.2
collected 43 items

tests/models/gpt_neox/test_modeling_flax_gpt_neox.py sssssssssssssssssssssssssssssssssssssssssss [100%]

=================================================== 43 skipped in 1.97s ===================================================

HuggingFaceDocBuilderDev · 2023-06-05T03:09:55Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

sanchit-gandhi · 2023-06-05T16:51:37Z

Hey @gojiteji! Thanks for picking-up the Flax GPT Neo PR! Would you mind rebasing onto main:

git fetch upstream
git rebase upstream main

And then force pushing the changes:

git push -f origin fix_flax_gpt_neox

This will then isolate the changes from your PR amongst the other ones

…uggingface#23235) * Ratio option for `logging_steps`, `eval_steps`, `save_steps` * Add guards if arguments are not set * Add more detailed comments + formatting * Update src/transformers/training_args.py Co-authored-by: Sylvain Gugger <[email protected]> * Update src/transformers/training_args.py Co-authored-by: Sylvain Gugger <[email protected]> * Update src/transformers/training_args.py Co-authored-by: Sylvain Gugger <[email protected]> * Convert args values to `int` if bigger than 1 * `black` * `make fixup` --------- Co-authored-by: Sylvain Gugger <[email protected]>

* intiial commit * new styling * update * just run doctest in CI * remove more test for fast dev * update * update refs * update path and fetch upstream * update documentatyion trests * typo * parse pwd * don't check for files that are in hidden folders * just give paths relative to transformers * update * update * update * major refactoring * make sure options is ok * lest test that mdx is tested * doctest glob * nits * update doctest nightly * some cleaning * run correct test on diff * debug * run on a single worker * skip_cuda_test tampkate * updates * add rA and continue on failure * test options * parse `py` codeblock? * we don't need to replace ignore results, don't remember whyu I put it * cleanup * more cleaning * fix arg * more cleaning * clean an todo * more pre-processing * doctest-module has none so extra `- ` is needed * remove logs * nits * doctest-modules .... * oups * let's use sugar * make dataset go quiet * add proper timeout * nites * spleling timeout * update * properly skip tests that have CUDSA * proper skipping * cleaning main and get tests to run * remove make report? * remove tee * some updates * tee was removed but is the full output still available? * [all-test] * only our tests * don't touch tee in this PR * no atee-sys * proper sub * monkey * only replace call * fix sub * nits * nits * fix invalid syntax * add skip cuda doctest env variable * make sure all packages are installed * move file * update check repo * revert changes * nit * finish cleanup * fix re * findall * update don't test init files * ignore pycache * `-ignore-pycache` when running pytests * try to fix the import missmatch error * install dec * pytest is required as doctest_utils imports things from it * the only log issues were dataset, ignore results should work * more cleaning * Update .circleci/create_circleci_config.py Co-authored-by: Sylvain Gugger <[email protected]> * Apply suggestions from code review Co-authored-by: amyeroberts <[email protected]> * [ydshieh] empty string if cuda is found * [ydshieh] fix condition * style * [ydshieh] fix * Add comment * style * style * show failure * trigger CI --------- Co-authored-by: Sylvain Gugger <[email protected]> Co-authored-by: Yih-Dar <[email protected]> Co-authored-by: amyeroberts <[email protected]> Co-authored-by: ydshieh <[email protected]>

Revert "[Doctests] Refactor doctests + add CI (huggingface#22987)" This reverts commit 627f447.

fix

* Remove nestedness in tool config * Really do it * Use remote tools descriptions * Work * Clean up eval * Changes * Tools * Tools * tool * Fix everything * Use last result/assign for evaluation * Prompt * Remove hardcoded selection * Evaluation for chat agents * correct some spelling * Small fixes * Change summarization model (huggingface#23172) * Fix link displayed * Update description of the tool * Fixes in chat prompt * Custom tools, custom prompt * Tool clean up * save_pretrained and push_to_hub for tool * Fix init * Tests * Fix tests * Tool save/from_hub/push_to_hub and tool->load_tool * Clean push_to_hub and add app file * Custom inference API for endpoints too * Clean up * old remote tool and new remote tool * Make a requirements * return_code adds tool creation * Avoid redundancy between global variables * Remote tools can be loaded * Tests * Text summarization tests * Quality * Properly mark tests * Test the python interpreter * And the CI shall be green. * fix loading of additional tools * Work on RemoteTool and fix tests * General clean up * Guard imports * Fix tools * docs: Fix broken link in 'How to add a model...' (huggingface#23216) fix link * Get default endpoint from the Hub * Add guide * Simplify tool config * Docs * Some fixes * Docs * Docs * Docs * Fix code returned by agent * Try this * Match args with signature in remote tool * Should fix python interpreter for Python 3.8 * Fix push_to_hub for tools * Other fixes to push_to_hub * Add API doc page * Docs * Docs * Custom tools * Pin tensorflow-probability (huggingface#23220) * Pin tensorflow-probability * [all-test] * [all-test] Fix syntax for bash * PoC for some chaining API * Text to speech * J'ai pris des libertés * Rename * Basic python interpreter * Add agents * Quality * Add translation tool * temp * GenQA + LID + S2T * Quality + word missing in translation * Add open assistance, support f-strings in evaluate * captioning + s2t fixes * Style * Refactor descriptions and remove chain * Support errors and rename OpenAssistantAgent * Add setup * Deal with typos + example of inference API * Some rename + README * Fixes * Update prompt * Unwanted change * Make sure everyone has a default * One prompt to rule them all. * SD * Description * Clean up remote tools * More remote tools * Add option to return code and update doc * Image segmentation * ControlNet * Gradio demo * Diffusers protection * Lib protection * ControlNet description * Cleanup * Style * Remove accelerate and try to be reproducible * No randomness * Male Basic optional in token * Clean description * Better prompts * Fix args eval in interpreter * Add tool wrapper * Tool on the Hub * Style post-rebase * Big refactor of descriptions, batch generation and evaluation for agents * Make problems easier - interface to debug * More problems, add python primitives * Back to one prompt * Remove dict for translation * Be consistent * Add prompts * New version of the agent * Evaluate new agents * New endpoints agents * Make all tools a dict variable * Typo * Add problems * Add to big prompt * Harmonize * Add tools * New evaluation * Add more tools * Build prompt with tools descriptions * Tools on the Hub * Let's chat! * Cleanup * Temporary bs4 safeguard * Cache agents and clean up * Blank init * Fix evaluation for agents * New format for tools on the Hub * Add method to reset state * Remove nestedness in tool config * Really do it * Use remote tools descriptions * Work * Clean up eval * Changes * Tools * Tools * tool * Fix everything * Use last result/assign for evaluation * Prompt * Remove hardcoded selection * Evaluation for chat agents * correct some spelling * Small fixes * Change summarization model (huggingface#23172) * Fix link displayed * Update description of the tool * Fixes in chat prompt * Custom tools, custom prompt * Tool clean up * save_pretrained and push_to_hub for tool * Fix init * Tests * Fix tests * Tool save/from_hub/push_to_hub and tool->load_tool * Clean push_to_hub and add app file * Custom inference API for endpoints too * Clean up * old remote tool and new remote tool * Make a requirements * return_code adds tool creation * Avoid redundancy between global variables * Remote tools can be loaded * Tests * Text summarization tests * Quality * Properly mark tests * Test the python interpreter * And the CI shall be green. * Work on RemoteTool and fix tests * fix loading of additional tools * General clean up * Guard imports * Fix tools * Get default endpoint from the Hub * Simplify tool config * Add guide * Docs * Some fixes * Docs * Docs * Fix code returned by agent * Try this * Docs * Match args with signature in remote tool * Should fix python interpreter for Python 3.8 * Fix push_to_hub for tools * Other fixes to push_to_hub * Add API doc page * Fixes * Doc fixes * Docs * Fix audio * Custom tools * Audio fix * Improve custom tools docstring * Docstrings * Trigger CI * Mode docstrings * More docstrings * Improve custom tools * Fix for remote tools * Style * Fix repo consistency * Quality * Tip * Cleanup on doc * Cleanup toc * Add disclaimer for starcoder vs openai * Remove disclaimer * Small fixed in the prompts * 4.29 * Update src/transformers/tools/agents.py Co-authored-by: Lysandre Debut <[email protected]> * Complete documentation * Small fixes * Agent evaluation * Note about gradio-tools & LC * Clean up agents and prompt * Apply suggestions from code review Co-authored-by: Patrick von Platen <[email protected]> * Apply suggestions from code review Co-authored-by: Patrick von Platen <[email protected]> * Note about gradio-tools & LC * Add copyrights and address review comments * Quality * Add all language codes * Add remote tool tests * Move custom prompts to other docs * Apply suggestions from code review Co-authored-by: amyeroberts <[email protected]> * TTS tests * Quality --------- Co-authored-by: Lysandre <[email protected]> Co-authored-by: Patrick von Platen <[email protected]> Co-authored-by: Philipp Schmid <[email protected]> Co-authored-by: Connor Henderson <[email protected]> Co-authored-by: Lysandre <[email protected]> Co-authored-by: Lysandre <[email protected]> Co-authored-by: amyeroberts <[email protected]>

…e#23243) trainer parameters changed to save tokenizer in addition to feature_extractor

trainer parameters fixed

* Improve docs * correct tip format * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Correct grammer & spelling * Improve code style * make style ruff * make style final

* Metadata update * Make fixup

* Update Image segmentation description * prompt -> label

* pong TF prob * fix --------- Co-authored-by: ydshieh <[email protected]>

* refine documentation for Tools * + one bugfix

* Try on a couple of blocks to see * Build the doc please * Build the doc please * Build the doc please * add more * Finish with all * Style

* chore: allow protobuf 3.20.3 Allow latest bugfix release for protobuf (3.20.3) * chore: update auto-generated dependency table update auto-generated dependency table * run in subprocess * Apply suggestions from code review Co-authored-by: amyeroberts <[email protected]> * Apply suggestions --------- Co-authored-by: ydshieh <[email protected]> Co-authored-by: Yih-Dar <[email protected]> Co-authored-by: amyeroberts <[email protected]>

…#23271) * Revert "Revert "[Doctests] Refactor doctests + add CI" (huggingface#23245)" This reverts commit 69ee462. * try not expose HfDocTestParser * move into testing_utils.py * remove pytest install --------- Co-authored-by: ydshieh <[email protected]>

* fix gpt2 inference * fixup * no need to be in `_keys_to_ignore_on_load_missing`

…ace#23257) * Temp tol fix for flaky whipser test * Add equivalent update to TF tests

…uggingface#22787) * update min k_value of conditional detr post-processing * feat: add top_k arg to post processing of deformable and conditional detr * refactor: revert changes to deprecated methods * refactor: move prob reshape to improve code clarity and reduce repetition

Wrong parantheses

Make `huggingface-tools` to [`huggingface-tools`](https://huggingface.co/huggingface-tools)

… json (huggingface#23268) * convert numpy array to list before writing to json per_category_iou and per_category_accuracy are ndarray in the eval_metrics * code reformatted with make style

Add first draft

…ace#23919)

…huggingface#23932) consistentcy

…ng (huggingface#23939) rename encode input to match docstring

Make TF inv_freq non-trainable like PyTorch

Revert "Update stale.yml to use HuggingFaceBot (huggingface#23941)" This reverts commit 5929f86.

* huggingface#23675 Registering Malay language * removing untranslated files * some translate * more updates to toctree * inc index * additional translations for toctree * translations of more sections * removing untranslated file * translated index.mdx to malay

…huggingface#23922) * Modify device map behavior for 4/8 bits model * Remove device_map arg for training 4/8 bit model * Remove index Co-authored-by: Sylvain Gugger <[email protected]> * Add Exceptions * Modify comment Co-authored-by: Sylvain Gugger <[email protected]> * Fix formatting * Get current device with accelerate * Revert "Get current device with accelerate" This reverts commit 46f0079. * Fix Exception * Modify quantization doc * Fix error Co-authored-by: Sylvain Gugger <[email protected]> --------- Co-authored-by: Sylvain Gugger <[email protected]>

use _make_causal_mask in clip models

…gface#23944) * Fix 'ReduceLROnPlateau' object has no attribute 'get_last_lr' * fix style

…dapter to Wav2Vec2 (huggingface#23813) * add fine-tuned with adapter layer * Add set_target_lang to tokenizer * Implement load adapter * add tests * make style * Apply suggestions from code review * Update src/transformers/models/wav2vec2/tokenization_wav2vec2.py * make fix-copies * Apply suggestions from code review * make fix-copies * make style again * mkae style again * fix doc string * Update tests/models/wav2vec2/test_tokenization_wav2vec2.py * Apply suggestions from code review * fix * Correct wav2vec2 adapter * mkae style * Update src/transformers/models/wav2vec2/modeling_wav2vec2.py Co-authored-by: Sanchit Gandhi <[email protected]> * add more nice docs * finish * finish * Apply suggestions from code review Co-authored-by: amyeroberts <[email protected]> * Apply suggestions from code review Co-authored-by: amyeroberts <[email protected]> * Apply suggestions from code review * all finish --------- Co-authored-by: Sanchit Gandhi <[email protected]> Co-authored-by: amyeroberts <[email protected]>

* generated code from add-new-model-like * Add code for modeling, config, and weight conversion * add tests for image-classification, update modeling and config * add code, tests for semantic-segmentation * make style, make quality, make fix-copies * make fix-copies * Update modeling_mobilevitv2.py fix bugs * Update _toctree.yml * update modeling, config fix bugs * Edit docs - fix bug MobileViTv2v2 -> MobileViTv2 * Update mobilevitv2.mdx * update docstrings * Update configuration_mobilevitv2.py make style * Update convert_mlcvnets_to_pytorch.py remove unused options * Update convert_mlcvnets_to_pytorch.py make style * Add suggestions from code review Co-Authored-By: amyeroberts <[email protected]> * make style, make quality * Add suggestions from code review Co-Authored-By: amyeroberts <[email protected]> * Add suggestions from code review Remove MobileViTv2ImageProcessor Co-Authored-By: amyeroberts <[email protected]> * make style * Add suggestions from code review Rename MobileViTv2 -> MobileViTV2 Co-Authored-By: amyeroberts <[email protected]> * Add suggestions from code review Co-Authored-By: amyeroberts <[email protected]> * Update modeling_mobilevitv2.py make style * Update serialization.mdx * Update modeling_mobilevitv2.py --------- Co-authored-by: amyeroberts <[email protected]>

) * translated object_detection.mdx Co-Authored-By: Hyeonseo Yun <[email protected]> Co-Authored-By: Nayeon Han <[email protected]> Co-Authored-By: simso <[email protected]> Co-Authored-By: Gabriel Yang <[email protected]> Co-Authored-By: Wonhyeong Seo <[email protected]> Co-Authored-By: Jungnerd <[email protected]> * Apply suggestions from code review Co-authored-by: Hyeonseo Yun <[email protected]> Co-authored-by: Wonhyeong Seo <[email protected]> Co-authored-by: Sohyun Sim <[email protected]> --------- Co-authored-by: Hyeonseo Yun <[email protected]> Co-authored-by: Nayeon Han <[email protected]> Co-authored-by: simso <[email protected]> Co-authored-by: Gabriel Yang <[email protected]> Co-authored-by: Wonhyeong Seo <[email protected]> Co-authored-by: Jungnerd <[email protected]> Co-authored-by: Hyeonseo Yun <[email protected]> Co-authored-by: Sohyun Sim <[email protected]>

…gingface#23952) Trainer: fixed KeyError on evaluate for ReduceLROnPlateau Co-authored-by: Claudius Kienle <[email protected]>

…huggingface#23945)

* Add an option to reduce compile() console spam * Add annotations to the example scripts * Add notes to the quicktour docs as well * minor fix

* added blogs to docs * removed new-line

…s/decision_transformer (huggingface#23964) Bump cryptography in /examples/research_projects/decision_transformer Bumps [cryptography](https://github.com/pyca/cryptography) from 39.0.1 to 41.0.0. - [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst) - [Commits](pyca/cryptography@39.0.1...41.0.0) --- updated-dependencies: - dependency-name: cryptography dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

sanchit-gandhi · 2023-06-06T17:21:50Z

Hey @gojiteji - not sure if you pushed or force pushed? See previous comment: #24002 (comment)

Let's see if we can revive the commit history here. In the case that we can't, we probably need to open a new PR for this

github-actions · 2023-07-05T15:02:22Z

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

sanchit-gandhi · 2023-07-05T17:23:14Z

Hey @gojiteji - feel free to open a new PR for this if you still want to continue the integration. Currently not sure which bits are new since the commit history is broken, but am more than happy to help with any questions / queries on a fresh PR!

github-actions · 2023-07-30T15:02:37Z

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

konstantinjdobler and others added 28 commits June 6, 2023 02:49

v4.30.0.dev0

d3d843e

Revert "[Doctests] Refactor doctests + add CI" (huggingface#23245)

01a81b9

Revert "[Doctests] Refactor doctests + add CI (huggingface#22987)" This reverts commit 627f447.

Fix from_config (huggingface#23246)

c6fc02c

fix

CTC example: updated trainer parameters to save tokenizer (huggingfac…

85bd8bf

…e#23243) trainer parameters changed to save tokenizer in addition to feature_extractor

[docs] Audio task guides fixes (huggingface#23239)

a0ab2ff

trainer parameters fixed

Metadata update (huggingface#23259)

11efaa4

* Metadata update * Make fixup

Update Image segmentation description (huggingface#23261)

f61d1c7

* Update Image segmentation description * prompt -> label

pin tensorflow-probability in docker files (huggingface#23260)

e6125fc

* pong TF prob * fix --------- Co-authored-by: ydshieh <[email protected]>

Refine documentation for Tools (huggingface#23266)

9bcc1da

* refine documentation for Tools * + one bugfix

Fix new line bug in chat mode for agents (huggingface#23267)

1eb40ca

Render custom tool docs a bit better (huggingface#23269)

1c672e1

* Try on a couple of blocks to see * Build the doc please * Build the doc please * Build the doc please * add more * Finish with all * Style

Fix link displayed for custom tools (huggingface#23274)

4e6c141

Remove missplaced test file (huggingface#23275)

b14bc6d

[gpt] Gpt2 fix half precision causal mask (huggingface#23256)

96f90a5

* fix gpt2 inference * fixup * no need to be in `_keys_to_ignore_on_load_missing`

Temporary tolerance fix for flaky whipser PT-TF equiv. test (huggingf…

8735cd9

…ace#23257) * Temp tol fix for flaky whipser test * Add equivalent update to TF tests

transformers-cli -> huggingface-cli (huggingface#23276)

7c611ec

Temporarily increase tol for PT-FLAX whisper tests (huggingface#23288)

3a5055f

Added missing " in CHAT_PROMPT_TEMPLATE (huggingface#23287)

80f2b59

Update custom_tools.mdx: fix link (huggingface#23292)

90a5b09

Wrong parantheses

Update transformers_agents.mdx (huggingface#23289)

b113a55

Make `huggingface-tools` to [`huggingface-tools`](https://huggingface.co/huggingface-tools)

Convert numpy arrays to lists before saving the evaluation metrics as…

4a8f7f1

… json (huggingface#23268) * convert numpy array to list before writing to json per_category_iou and per_category_accuracy are ndarray in the eval_metrics * code reformatted with make style

NielsRogge and others added 23 commits June 6, 2023 02:49

[PushToHub] Make it possible to upload folders (huggingface#23920)

b085c73

Add first draft

Skip device placement for past key values in decoder models (huggingf…

790e661

…ace#23919)

[Flax Whisper] Update decode docstring (huggingface#23908)

c753137

Effectively allow encoder_outputs input to be a tuple in pix2struct (…

0318d46

…huggingface#23932) consistentcy

Fix doc string nits (huggingface#23929)

c0d07c9

Pin rhoknp (huggingface#23937)

8e5475d

rename DocumentQuestionAnsweringTool parameter input to match docstri…

eff4eda

…ng (huggingface#23939) rename encode input to match docstring

Update stale.yml to use HuggingFaceBot (huggingface#23941)

14708b5

Make TF ESM inv_freq non-trainable like PyTorch (huggingface#23940)

8a8322a

Make TF inv_freq non-trainable like PyTorch

Revert "Update stale.yml to use HuggingFaceBot" (huggingface#23943)

ea8587b

Revert "Update stale.yml to use HuggingFaceBot (huggingface#23941)" This reverts commit 5929f86.

use _make_causal_mask in clip/vit models (huggingface#23942)

ad8a583

use _make_causal_mask in clip models

Fix ReduceLROnPlateau object has no attribute 'get_last_lr' (huggin…

c08d95a

…gface#23944) * Fix 'ReduceLROnPlateau' object has no attribute 'get_last_lr' * fix style

add new mms functions to doc (huggingface#23954)

984391e

Trainer: fixed evaluate raising KeyError for ReduceLROnPlateau (hug…

248e844

…gingface#23952) Trainer: fixed KeyError on evaluate for ReduceLROnPlateau Co-authored-by: Claudius Kienle <[email protected]>

[Whisper Tokenizer] Skip special tokens when decoding with timestamps (…

8f6431f

…huggingface#23945)

Add an option to reduce compile() console spam (huggingface#23938)

3355769

* Add an option to reduce compile() console spam * Add annotations to the example scripts * Add notes to the quicktour docs as well * minor fix

Added time-series blogs to the models (huggingface#23857)

307edf6

* added blogs to docs * removed new-line

gojiteji force-pushed the fix_flax_gpt_neox branch from 7cf36e4 to 80605bf Compare June 5, 2023 17:56

HeegyuKim mentioned this pull request Aug 6, 2023

Bug Fixed GPTNeoX Flax supports #25334

Closed

github-actions bot closed this Aug 7, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Addition of test code for GPTNeoX Flax support #24002

Addition of test code for GPTNeoX Flax support #24002

gojiteji commented Jun 5, 2023 •

edited

Loading

HuggingFaceDocBuilderDev commented Jun 5, 2023

sanchit-gandhi commented Jun 5, 2023

sanchit-gandhi commented Jun 6, 2023

github-actions bot commented Jul 5, 2023

sanchit-gandhi commented Jul 5, 2023

github-actions bot commented Jul 30, 2023

Addition of test code for GPTNeoX Flax support #24002

Addition of test code for GPTNeoX Flax support #24002

Conversation

gojiteji commented Jun 5, 2023 • edited Loading

HuggingFaceDocBuilderDev commented Jun 5, 2023

sanchit-gandhi commented Jun 5, 2023

sanchit-gandhi commented Jun 6, 2023

github-actions bot commented Jul 5, 2023

sanchit-gandhi commented Jul 5, 2023

github-actions bot commented Jul 30, 2023

gojiteji commented Jun 5, 2023 •

edited

Loading