Metadata.to_yaml() fails with error "cannot represent an object" when encountering numpy datatypes

[khughitt]

Overview

If one attempts to include non-simple numeric data types (e.g. np.int64 instead of int), yaml creation fails with a similar error message to that described in issue #453.

This is due to the fact that the default loader for recent versions of pyyaml no longer handle non-simple data types.

A solution, is to use the "unsafe" loader, e.g.:

import numpy as np

# works
yaml.unsafe_load(yaml.dump({'value': np.float64(1.234)}))

# doesn't work
yaml.full_load(yaml.dump({'value': np.float64(1.234)}))

Do you think it would make sense to switch to the "unsafe" loader for frictionless? This way, numpy datatypes could be automatically handled.

The question is -- can you think of any scenario where frictionless.Metadata.to_yaml() could be passed actually unsafe code?

If there is such a potential, then perhaps it's best to continue using the full_loader and leave the type-casting up to the user / mark this issue as "won't fix". Up to you.

Cheers,
Keith

Related:

• FIX: pyyaml.load xpdAcq/xpdAcq#553 (comment)
• Yaml-5.1 fails with Numpy objects yaml/pyyaml#286 (comment)

Versions

• Python 3.8.6
• frictionless 3.48.0
• pyyaml 5.3.1

---

Please preserve this line to notify @roll (lead of this repository)

[roll]

Hi @khughitt,

Yes, I would preserve loading only safe types by default but we can create an option to support a unsafe loader.

I've created a feature request for it - #612