core + web integration: dev validation

[noahehall]

C

• bootstrap the entire platform
  • greenfield SLA: 5ms
  • brownfield SLA: 30s
• many detours and refactors initiatied and resolved as a pre-req to this ticket

---

• complete the following tickets before continuing:
  • CD: setup bootstrap scripts for all compose services using command: poop and move away from command to ease integration with nomad #28
  • CD: refactor Haproxy config to support increased application complexity networking#10 -
    • we cant use 99% of the vars that come from dev as its all open source
  • CD: integrate nomad with vault #36
    • we cant use 99% of the vars that come from dev as its all open source
• apps are currently loading successfully via nomad but need to be reachable over the network without hardcoding IPs/DNS names or replicating docker dev networks
  • haproxy is setup for dynamic dns resolution through the docker resolver at 127.0.0.11:53

T

• integration: core + web: via nomad & consul
  • move most things out of conf files and into env vars where appropriate
    • this eases transition from docker compose > nomad where we should align with idiomatic nomad conventions without hampering development nor operation
    • haha this is going to be a pain, but you only need to do it once
    • basicaly check consul & vault -h and map that shiz to NOMAD_POOP_BOOP
    • for terraform you should be able to query that directly from the cli
  • map jobspec envvars to task application envvars
  • docker registry script
    • update container volume source: destination:
      • destination: etc/ssl/certs/live/dev.nirv.ai/*.pem > /etc/ssl/certs/dev.nirv.ai/*.pem
      • source: ./apps/nirvai-core-letsencrypt/.... > /etc/ssl/certs/dev.nirv.ai/*.pem
  • ensure all services have the correct consul intentions
• sanity check: web + core running?
• completed on one of the other tickets
  • think through vault task in devcore jobspec
  • this entire thoughtstorm is stale, think through it again after consul integration ticket is complete
  • currenlty vault is in the dev_core jobspec as a group & task, with the other dev services
    • this creates a cyclical dependency as nomad needs the vault server to be running in order to create the tokens for the other tasks
  • we could
    • leave things as they are and rely on the 30s retry policy, which doesnt make sense if you say it out loud
    • see if there is dependency management between groups in a job spec, which wouldnt require having a separate jobspec
    • move vault into its own jobspec
• verify system aligns with hashicorp consul deployment guide

A

• https://github.com/see https://github.com/nirv-ai/docs/blob/main/_architecture_decision_records_/core-IaC.png
• https://developer.hashicorp.com/nomad/docs/integrations/vault-integration
• https://www.hashicorp.com/resources/nomad-networking-demystified
• https://developer.hashicorp.com/nomad/docs/job-specification/network
• https://mrkaran.dev/posts/nomad-networking-explained/
• Nomad bridge network and docker0 bridge network should be able to talk to each other. hashicorp/nomad#10217
• https://developer.hashicorp.com/nomad/tutorials/load-balancing/load-balancing-haproxy
• https://developer.hashicorp.com/nomad/docs/job-specification/network
• postgres data permission denied: @see Host-Volumes + SELinux result in permission denied.  hashicorp/nomad#9123
• seems like theres no way push off the consul integration :( @see [question] Using environment variables from another group hashicorp/nomad#5455