In crawler cronjob, take declarations directly from postgres

[michielbdejong]

No description provided.

[michielbdejong]

This is an adaptation of milestone 1.2 which reads:

Adapt the ToS;DR webapp to obtain the list of services and documents to annotate from the API instead of the ToS;DR server database

[michielbdejong]

Given the trouble we ran into with the use of very large git repositories, basically trying to use GitHub as a database server, we decided it's better if the crawl engine writes its metadata directly to the postgres database from which ToS;DR is already loading its data.

[michielbdejong]

Working on this in the ota-to-postgres branch of this repo , and the postgres-backend branch of our fork of the OTA engine.

Got a test script working, reading the declarations from pg. Next step: use this as the declarations source for npx ota track