Let's test what you learned in this chapter!
<Question choices={[ { text: "From top to bottom", explain: "Try again -- although most other programming languages print the exception at the top, Python is special in this regard." }, { text: "From bottom to top", explain: "Correct! One advantage of Python's tracebacks showing the exception at the bottom is that it's easier to debug when you're working in the terminal and this is the last line you see.", correct: true } ]} />
<Question choices={[ { text: "A simple implementation of a Transformer architecture from a research article", explain: "Although it is very educational to implement your own Transformer models from scratch, this is not what we're talking about here." }, { text: "A compact and self-contained block of code that can be run without any external dependencies on private files or data", explain: "Correct! Minimal reproducible examples help the library's maintainers reproduce the problem you are having, so they can find solutions faster.", correct: true }, { text: "A screenshot of the Python traceback", explain: "Try again -- although it is tempting to include a screenshot of the error you are facing when filing an issue, this makes it very difficult for others to reproduce the error." }, { text: "A notebook that contains your whole analysis, including parts unrelated to the error", explain: "Not quite -- although it can be helpful to share a Google Colab notebook that shows the error, make sure it is short and only contains the relevant code." } ]} />
from transformers import GPT3ForSequenceClassification
# ImportError: cannot import name 'GPT3ForSequenceClassification' from 'transformers' (/Users/lewtun/miniconda3/envs/huggingface/lib/python3.8/site-packages/transformers/__init__.py)
# ---------------------------------------------------------------------------
# ImportError Traceback (most recent call last)
# /var/folders/28/k4cy5q7s2hs92xq7_h89_vgm0000gn/T/ipykernel_30848/333858878.py in <module>
# ----> 1 from transformers import GPT3ForSequenceClassification
# ImportError: cannot import name 'GPT3ForSequenceClassification' from 'transformers' (/Users/lewtun/miniconda3/envs/huggingface/lib/python3.8/site-packages/transformers/__init__.py)Which of the following might be a good choice for the title of a forum topic to ask for help?
<Question
choices={[
{
text: "ImportError: cannot import name 'GPT3ForSequenceClassification' from 'transformers' (/Users/lewtun/miniconda3/envs/huggingface/lib/python3.8/site-packages/transformers/init.py)",
explain: "Including the last line of the traceback can be descriptive, but this is better reserved for the main body of the topic. Try again!"
},
{
text: "Problem with from transformers import GPT3ForSequenceClassification",
explain: "Try again -- although this provides useful information, it's probably best reserved for the main body of the text.",
},
{
text: "Why can't I import GPT3ForSequenceClassification?",
explain: "Good choice! This title is concise and gives the reader a clue about what might be wrong (i.e., that GPT-3 is not supported in 🤗 Transformers).",
correct: true
},
{
text: "Is GPT-3 supported in 🤗 Transformers?",
explain: "Good one! Using questions as topic titles is a great way to communicate the problem to the community.",
correct: true
}
]}
/>
4. Suppose you've tried to run trainer.train() and are faced with a cryptic error that doesn't tell you exactly where the error is coming from. Which of the following is the first place you should look for errors in your training pipeline?
<Question choices={[ { text: "The optimization step where we compute gradients and perform backpropagation", explain: "Although there may be bugs in your optimizer, this is usually several steps into the training pipeline, so there are other things to check first. Try again!" }, { text: "The evaluation step where we compute metrics", explain: "Evaluation is usually what you do after training for a full epoch, so you should first check somewhere earlier in the training pipeline.", }, { text: "The datasets", explain: "Correct! Looking at your data is almost always the first thing you should do, to make sure the text is encoded appropriately, has the expected features, and so on.", correct: true }, { text: "The dataloaders", explain: "Try again -- this is very close to the first thing you should check. Do you remember what object we pass to the dataloaders?" } ]} />
<Question choices={[ { text: "Post the error message on the forums or GitHub.", explain: "That won't help anyone as CUDA error messages are usually very uninformative." }, { text: "Execute the same code on the CPU.", explain: "Exactly, that should give you a better error message!", correct: true }, { text: "Read the traceback to find out what caused the error.", explain: "That's what you would do for any other error, but CUDA errors are usually not raised where they happened because most CUDA operations are asynchronous." }, { text: "Reduce the batch size.", explain: "Reducing the batch size is usually a good strategy for handling CUDA out-of-memory errors, but not for this particular problem. Try again!" }, { text: "Restart the Jupyter kernel.", explain: "Try again -- restarting the kernel won't make the error magically go away!", } ]} />
<Question choices={[ { text: "Post a full reproducible example of the bug.", explain: "Yes, that's the best way to help the maintainers find your bug. What else should you do?", correct: true }, { text: "Ask every day for an update.", explain: "That's unlikely to get you any help; people will probably ignore you more.", }, { text: "Inspect the source code around the bug and try to find the reason why it happens. Post the results in the issue.", explain: "That will definitely help the maintainers! And if you do find the source of the bug and a fix, you can even open a pull request. What else should you do?", correct: true } ]} />
<Question choices={[ { text: "It isn't; overfitting is always bad and should be avoided.", explain: "When training over the whole dataset, overfitting can indeed be a sign that your model won't generalize well to new examples. For debugging, though, we don't usually train over the whole dataset. Try again!" }, { text: "It allows us to verify that the model is able to reduce the loss to zero.", explain: "Correct! With a small batch with as little as two examples, we can quickly verify whether the model is capable of learning.", correct: true }, { text: "It allows us to verify that the tensor shapes of our inputs and labels are correct.", explain: "Try again -- if your tensor shapes are misaligned, then you certainly won't be able to train, even on a single batch.", } ]} />
8. Why is it a good idea to include details on your compute environment with transformers-cli env when creating a new issue in the 🤗 Transformers repo?
<Question choices={[ { text: "It allows the maintainers to understand which version of the library you're using.", explain: "Correct! Since each major version of the library may have changes in the API, knowing which specific version you are using can help narrow down the problem. What are the other benefits?", correct: true }, { text: "It allows the maintainers to know whether you're running code on Windows, macOS, or Linux.", explain: "Correct! Errors can sometimes be caused by the specific operating system you are using, and knowing this helps the maintainers reproduce them locally. That's not the only reason, though.", correct: true }, { text: "It allows the maintainers to know whether you're running code on a GPU or CPU.", explain: "Correct! As we've seen in this chapter, code ran on GPUs or CPUs may produce diffferent results or errors, and knowing which hardware you're using can help focus the maintainers' attention. But this isn't the only benefit...", correct: true } ]} />