Fix device fallback in serialization

[thejumpman2323]

Description

fix #1158

Related Issues

Checklist

• Is this code covered by new or existing unit tests or integration tests?
• Did you run make test successfully?
• Do new classes, functions, methods and parameters all have docstrings?
• Were existing docstrings updated, if necessary?
• Was external documentation updated, if necessary?

Additional Notes or Comments

[codecov-commenter]

Codecov Report

Attention: 48 lines in your changes are missing coverage. Please review.

Comparison is base (34830a7) 80.33% compared to head (3cad92b) 80.44%.
Report is 12 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1240      +/-   ##
==========================================
+ Coverage   80.33%   80.44%   +0.10%     
==========================================
  Files          95       97       +2     
  Lines        6602     6683      +81     
==========================================
+ Hits         5304     5376      +72     
- Misses       1298     1307       +9     

Files	Coverage Δ
superduperdb/base/config.py	100.00% <ø> (ø)
superduperdb/components/model.py	93.66% <100.00%> (+11.00%)	⬆️
superduperdb/ext/openai/model.py	96.09% <ø> (+0.64%)	⬆️
superduperdb/ext/torch/model.py	76.80% <ø> (ø)
superduperdb/misc/serialization.py	85.86% <100.00%> (+0.92%)	⬆️
superduperdb/misc/superduper.py	76.25% <ø> (ø)
superduperdb/base/build.py	65.85% <66.66%> (-1.65%)	⬇️
superduperdb/base/logger.py	86.95% <80.00%> (-3.05%)	⬇️
superduperdb/cli/serve.py	39.28% <50.00%> (+4.28%)	⬆️
superduperdb/cli/stack.py	55.55% <55.55%> (ø)
... and 1 more

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[blythed]

Do we need a unit-test?

[thejumpman2323]

@blythed I don't think unit test is possible on ci as ci only have one device -- ok but we can have something that runs on Mac M1 -- or we could mock this?