Support for ActorPacks

[cybersec-code]

An initial set of features to support ActorPack validation and insertion into the database. A new taxonomy was introduced (country) as well as new tables to the schema.

[soad003]

Thanks for the PR.

Are there any test actorpacks to test with in the repo? And some tags that use those actors? Would be great for testing (tests/testfiles).

I have not done a full review yet but here are a couple of things I noticed.

Problems

• tagpack-tool taxonomy insert country does not work (invalid choice: 'country' (choose from 'abuse', 'entity', 'confidence')
  ) but tagpack-tool taxonomy insert tries to insert a coutry taxomony.

Nits

• Readme could use an update e.g (Ingest taxonomies, Ingest TagPacks)
• Files to test the actorpacks would be great

[soad003]

is this constraint to the concepts table intended?

[soad003]

Maybe i misunderstand the schema. Have seen that the tests also treat coutries as concepts.

[soad003]

Is this quality view something we want to keep up to date in general? If so maybe add call to this to tp-tool tagstore refresh_views

[cybersec-code]

Thanks for the feedback,

• I added an actors file to the tagpacks repo for testing. We will add more ActorPacks to this folder in the near future.
• We are still defining how to integrate actors with tags, so there are no tests for it at the moment.
• We added country as a taxonomy, that's why the constraint references concept(id) in the schema.
• tagpack-tool taxonomy insert country is working now.
• I think for now it's not necessary to keep up to date the quality view, because we want to take "snapshots" of those results at different points in time. But maybe in the future this changes.
• I'm updating the README file

[soad003]

Thanks for the clarification and updates.

I have tested what I can test so far (import of actors, computing quality stats etc.) and quickly skimmed over the source-code.

I think its got to go like it is right now. Any objections?

[mdragaschnig]

I tested validation and insert for tagpacks and actorpacks, looking good so far.

[soad003]

I have integrated recent changes from develop. I think it all looks fine. Will merge into develop now.

@cybersec-code: there have been some changes regarding folder structure of the repo and some others infrastructure change like enforced formatting and linting via pre-commit hooks. If you have any questions or troubles while developing, let me know.

Thanks everyone for the effort.