Considering using the C yaml loader in Python

[esmet]

Please describe your use case / problem.

The default python yaml implementation is verty slow compared to native implementations. When Ambassador needs to manage many service (1000 or more) objects, the yaml data can get large enough to the point where updates take several seconds of CPU time or more.

Describe the solution you'd like

Use the CSafeLoader from pyyaml and specify Loader=CSafeLoader on calls to yaml.load and yaml.load_all. Note that yaml.load etc is typically unsafe with the default loader because it allows for arbitrary object instantiation in the python interpreter. The CSafeLoader has the word safe in it, but I tested this case manually anyway. Forunately, CSafeLoader does in fact prevent you from instantiating python objects, so I think there are no new security concerns by using this loader over the existing yaml.safe_load_all.

In practice, I observed that 20k service objects with the standard implementation takes 70s to load. With the C loader, it takes 6.5 seconds.

I can provide a patch for the simplest possible case if necessary.

Describe alternatives you've considered

The alternative would be to migrate Ambassador's internal python components to operate on JSON only. The JSON parser in Python is already fast.

Additional context

More context on the original performance investigation here: #1292

[richarddli]

Good call, this is in #1312

[kflynn]

We use the C parser in 0.52.0.