Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Github release assets model autodownload #711

Merged
merged 6 commits into from
Aug 12, 2020

Conversation

Ownmarc
Copy link
Contributor

@Ownmarc Ownmarc commented Aug 11, 2020

πŸ› οΈ PR Summary

Made with ❀️ by Ultralytics Actions

🌟 Summary

Simplified and robust pretrained model downloading in Ultralytics YOLOv5.

πŸ“Š Key Changes

  • Removed deprecated Google Drive download logic for pretrained YOLOv5 models.
  • Added direct download from GitHub releases as the primary method.
  • Included a fallback to Google Cloud Storage if GitHub download fails.
  • Implemented assertions and improved error handling for download integrity checks.

🎯 Purpose & Impact

  • πŸ”„ Streamlining Download Process: The update aims to simplify how users download pretrained models by using GitHub, which may offer a more reliable service compared to the previous method.
  • πŸ›  Enhanced Reliability: By asserting file existence and size after download, users are less likely to encounter issues due to incomplete or failed downloads.
  • πŸ› Better Error Messaging: Clear error messages can help diagnose problems during the download process, improving user experience.
  • πŸ”— Fallback Option: In case of GitHub issues, the code now attempts to download from Google Cloud Storage, providing an alternative to ensure that the process is less prone to failure.

@glenn-jocher glenn-jocher changed the title dowload from github assets as 3rd autodownload fallback Github release assets model autodownload Aug 11, 2020
@glenn-jocher
Copy link
Member

glenn-jocher commented Aug 11, 2020

Ok I think this looks good now, but unfortunately I'm seeing an error locally that doesn't show up in the checks. I get this on a python detect.py command.

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/urllib/request.py", line 1350, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/http/client.py", line 1255, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/http/client.py", line 1301, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/http/client.py", line 1250, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/http/client.py", line 1010, in _send_output
    self.send(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/http/client.py", line 950, in send
    self.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/http/client.py", line 1424, in connect
    self.sock = self._context.wrap_socket(self.sock,
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ssl.py", line 1040, in _create
    self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/glennjocher/PycharmProjects/yolov5/detect.py", line 171, in <module>
    detect()
  File "/Users/glennjocher/PycharmProjects/yolov5/detect.py", line 33, in detect
    model = attempt_load(weights, map_location=device)  # load FP32 model
  File "/Users/glennjocher/PycharmProjects/yolov5/models/experimental.py", line 136, in attempt_load
    attempt_download(w)
  File "/Users/glennjocher/PycharmProjects/yolov5/utils/google_utils.py", line 31, in attempt_download
    hub.download_url_to_file('https://github.com/ultralytics/yolov5/releases/download/v2.0/' + file, weights)
  File "/Users/glennjocher/PycharmProjects/yolov5/venv/lib/python3.8/site-packages/torch/hub.py", line 379, in download_url_to_file
    u = urlopen(req)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/urllib/request.py", line 542, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/urllib/request.py", line 1393, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/urllib/request.py", line 1353, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)>

It looks like the problem is specific to MacOS with python requests (I'm running on a Macbook). We may have to ditch the torch downloader for the existing curl commands, which I've never seen anyone report problems with.
https://stackoverflow.com/questions/42098126/mac-osx-python-ssl-sslerror-ssl-certificate-verify-failed-certificate-verify

@glenn-jocher
Copy link
Member

I dropped all this into a try, except, finally clause. Try GitHub, except GCP, finally check and return

@glenn-jocher
Copy link
Member

CI is passing, everything seems good, merging.

@glenn-jocher glenn-jocher merged commit 1f92422 into ultralytics:master Aug 12, 2020
burglarhobbit pushed a commit to burglarhobbit/yolov5 that referenced this pull request Jan 1, 2021
* assets autodownload

* Update google_utils.py

* Update google_utils.py

* Update google_utils.py

* Update google_utils.py

* Update google_utils.py

Co-authored-by: Glenn Jocher <[email protected]>
KMint1819 pushed a commit to KMint1819/yolov5 that referenced this pull request May 12, 2021
* assets autodownload

* Update google_utils.py

* Update google_utils.py

* Update google_utils.py

* Update google_utils.py

* Update google_utils.py

Co-authored-by: Glenn Jocher <[email protected]>
BjarneKuehl pushed a commit to fhkiel-mlaip/yolov5 that referenced this pull request Aug 26, 2022
* assets autodownload

* Update google_utils.py

* Update google_utils.py

* Update google_utils.py

* Update google_utils.py

* Update google_utils.py

Co-authored-by: Glenn Jocher <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants