Databases: best practices

[pat-s]

With the recent additions of tarchetypes::tar_date(), some help of Will and browsing related discussions here and in {drake} I currently do it as follows

tar_target(
  table_heartbeat,
  get_from_database("heartbeat", 1000000L),
  format = "fst",
  # rerun target once a day (can be adapted to any time diff)
  cue = tarchetypes::tar_cue_age(
    name = table_heartbeat,
    age = as.difftime(1, units = "days")
  )
)

With the following DB helper

get_from_database <- function(table, nrow, ...) {
  checkmate::check_int(nrow)
  con <- DBI::dbConnect()
  on.exit(DBI::dbDisconnect(con))
  # alternatively read the full table with dbGetTable(con, <table>)
  DBI::dbGetQuery(con, sprintf("select TOP %s * from %s", nrow, table))
}

[wlandau]

Nice, thanks for sharing! Great way to pull from a database at least once per day. Relevant to #269, cc @MxNl and @malcolmbarrett.

[malcolmbarrett]

Thanks for sharing @wlandau and @pat-s!