Schema support #220

krlmlr · 2018-01-29T07:55:07Z

New dbListObject() generic, returns a data frame with columns table (list of Id objects), id (a SQL vector), and is_prefix (a logical, indicates if this is a schema prefix or a table/view)
New dbUnquoteIdentifier() generic, seems necessary e.g. for dbExistsTable()
Export Id()
- Requires all arguments to be named
- Does not depend on the connection
- Only when calling dbQuoteIdentifier() the connection is used

codecov-io · 2018-01-29T08:01:46Z

Codecov Report

Merging #220 into master will decrease coverage by 2.26%.
The diff coverage is 9.52%.

@@            Coverage Diff             @@
##           master     #220      +/-   ##
==========================================
- Coverage   40.71%   38.44%   -2.27%     
==========================================
  Files          16       16              
  Lines         560      593      +33     
==========================================
  Hits          228      228              
- Misses        332      365      +33

Impacted Files	Coverage Δ
R/interpolate.R	`99.25% <ø> (ø)`	⬆️
R/deprecated.R	`0% <ø> (ø)`	⬆️
R/DBDriver.R	`3.22% <ø> (ø)`	⬆️
R/data.R	`26.19% <ø> (ø)`	⬆️
R/DBResult.R	`0% <ø> (ø)`	⬆️
R/transactions.R	`0% <ø> (ø)`	⬆️
R/DBConnection.R	`0% <0%> (ø)`	⬆️
R/table.R	`0% <0%> (ø)`	⬆️
R/quote.R	`32.18% <4%> (-12.27%)`	⬇️
R/data-types.R	`41.17% <50%> (ø)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 13d4c55...1dd0ebc. Read the comment docs.

hadley · 2018-01-29T13:14:13Z

R/DBConnection.R

@@ -362,6 +362,36 @@ setGeneric("dbListTables",
  valueClass = "character"
 )

+#' List remote objects
+#'
+#' Returns the names of remote tables accessible through this connection


tables -> objects ?

hadley · 2018-01-29T13:14:36Z

R/DBConnection.R

+#'
+#' Returns the names of remote tables accessible through this connection
+#' (possibly via a prefix) as a data frame.
+#' This should, where possible, include temporary objects.


Maybe "This should include temporary objects, but not all database drivers support listing temporary objects"

Perhaps include a sentence describing how this is different to dbListTables()?

hadley · 2018-01-29T13:15:04Z

R/DBConnection.R

+#' @inheritSection DBItest::spec_sql_list_objects Additional arguments
+#'
+#' @inheritParams dbGetQuery
+#' @param prefix An optional prefix, passed to [dbUnquoteIdentifier()].


This is worded too generically for me to understand

hadley · 2018-01-29T13:16:12Z

R/quote.R

-#' use in a query as a column name, to make sure that you
-#' generate valid SQL and avoid SQL injection.
+#' use in a query as a column or table name, to make sure that you
+#' generate valid SQL and avoid SQL injection. The inverse operation is


injection -> injection attacks.

hadley · 2018-01-29T13:17:09Z

R/quote.R

@@ -100,6 +106,9 @@ setGeneric("dbQuoteIdentifier",

 quote_identifier <-
  function(conn, x, ...) {
+    if (is.list(x)) {


I think auto-vectorisation over lists is best avoided because it violates type-stability (and we generally avoid it elsewhere)

hadley · 2018-01-29T13:18:15Z

R/quote.R

+
+#' Unquote identifiers
+#'
+#' Call this method to convert a [SQL] object created by [dbQuoteIdentifier()]


After reading this documentation I don't know what this is going to return. It sounds like it's going to return table objects which doesn't seem right?

Return values are part of the specs in DBItest which I haven't adapted yet.

The method is returning a Table object. Why does this seem wrong?

Root of my confusion here is that Table is a reference/identifier, not a table.

hadley · 2018-01-29T13:20:22Z

R/table.R

@@ -3,21 +3,30 @@ setClass("Table", slots = list(name = "character"))

 #' Refer to a table nested in a hierarchy (e.g. within a schema)
 #'
+#' Objects of class `Table` have a single slot `name`, which is a named


Calling this a table seems slightly confusing since it's the name of a table, not a table itself. Maybe TableRef?

And doesn't every other object start with db?

Table() doesn't need the connection, it's just a dumb object like SQL(). We could use Identifier() too.

I would suggest Id()

Id() looks very prone to collisions with other packages...

This search doesn't find packages that export crossing: https://github.com/search?utf8=%E2%9C%93&q=user%3Acran+%22export+crossing%22+file%3ANAMESPACE&type=Code . I remember a recent discussion on Twitter about searching for packages that export a function?

SqlId() looks inconsistent to SQL(), and SQLId() looks ... interesting.

Sorry, it needs to be filename: not file:. https://github.com/search?utf8=%E2%9C%93&q=user%3Acran+filename%3ANAMESPACE+%22export+Id%22&type= is the correct search, and again no conflicts for Id, the only functions are lowercase id.

Why doesn't it find "nest"? https://github.com/search?utf8=%E2%9C%93&q=user%3Acran+filename%3ANAMESPACE+%22export+nest%22&type=

I am not sure, but it seems like GitHub limits the length of time a query can run. Anyway I ran the search on my Local CRAN mirror, there are no CRAN packages that export Id.

You can call it Identifier if you like it better.

Confirmed (from a scraping of my local CRAN mirror) that no package exports an Id class either. Renaming.

jimhester · 2018-01-29T14:05:29Z

R/quote.R

+  if (is(x, "SQL")) {
+    . <- strsplit(as.character(x), '^"|"$|"[.]"')
+    . <- lapply(., `[`, -1L)
+    split <- .


I realize you likely converted this from a magrittr pipeline, but I think this would be better as

split <- lapply(strsplit(as.character(x), '^"|"$|"[.]"'), `[`, -1L)

And actually I think it is more explicit to use tail than [ here, but that is largely a matter of taste.

I don't like nested expressions, and the dot looks like the lesser of all evils here to me, because I don't need to invent names for intermediates I don't care about.

To be honest, I really don't like the use of . here. It feels so different to the style of the rest of the code.

jimhester · 2018-01-29T14:06:55Z

R/table.R

@@ -3,21 +3,30 @@ setClass("Table", slots = list(name = "character"))

 #' Refer to a table nested in a hierarchy (e.g. within a schema)
 #'
+#' Objects of class `Table` have a single slot `name`, which is a named


I would suggest Id()

jimhester · 2018-01-29T14:11:38Z

R/quote.R

+#'
+#' dbUnquoteIdentifier(
+#'   ANSI(),
+#'   SQL(c("Schema"."Table", "UnqualifiedTable"))


Unless I am missing something this does not parse. I think you are missing a set of quotes.

SQL(c("Schema"."Table", "UnqualifiedTable")) #> Error: unexpected symbol in "SQL(c("Schema"."

jimhester · 2018-01-29T14:18:46Z

R/table.R

 Table <- function(...) {
-  new("Table", name = c(...))
+  components <- c(...)
+  if (is.null(names(components)) || any(names(components) == "")) {


I am not sure we want to force naming, particularly for the most common case (schema & table).

Which comes first -- schema or table? If we force naming, that confusion won't arise.

Yes, but it makes it much more cumbersome to use the function. If it is cumbersome to use no one is going to use it. Having Table("foo", "bar") map to 'foo'.'bar' seems like an easy solution, and if your backend uses different names you can instead use named arguments.

if (length(components) == 2) && (is.null(names(components)) || all(names(components) == ""))) { names(components) <- c("schema", "table") }

Backends are free to implement their own wrappers with named arguments, and probably should too.

For instance, in BigQuery, "schema" means something entirely different. I'd rather keep things strict for the first implementation, we can always relax later but it's more difficult to impose restrictions on a function that has been exported.

…ListObjects()` and `dbUnquoteIdentifier()`, methods for `Id` that call `dbQuoteIdentifier()` and then forward (#220). - `dbListConnections()` is soft-deprecated by documentation. - Default implementations of `dbQuoteIdentifier()` and `dbQuoteLiteral()` preserve names, default implementation of `dbQuoteString()` strips names (#173). - Breaking change: If the `names` argument is unset, `SQL()` strips the names from the output.

@yutannihilation

…ps the names from the output if the `names` argument is unset. - The `dbReadTable()`, `dbWriteTable()`, `dbExistsTable()`, `dbRemoveTable()`, and `dbListFields()` generics now specialize over the first two arguments to support implementations with the `Id` S4 class as type for the second argument. Some packages may need to update their documentation to satisfy R CMD check again. New generics ------------ - Schema support: Export `Id()`, new generics `dbListObjects()` and `dbUnquoteIdentifier()`, methods for `Id` that call `dbQuoteIdentifier()` and then forward (#220). - New `dbQuoteLiteral()` generic. The default implementation uses switchpatch to avoid dispatch ambiguities, and forwards to `dbQuoteString()` for character vectors. Backends may override methods that also dispatch on the second argument, but in this case also an override for the `"SQL"` class is necessary (#172). Default implementations ----------------------- - Default implementations of `dbQuoteIdentifier()` and `dbQuoteLiteral()` preserve names, default implementation of `dbQuoteString()` strips names (#173). - Specialized methods for `dbQuoteString()` and `dbQuoteIdentifier()` are available again, for compatibility with clients that use `getMethod()` to access them (#218). - Add default implementation of `dbListFields()`. - The default implementation of `dbReadTable()` now has `row.names = FALSE` as default and also supports `row.names = NULL` (#186). API changes ----------- - The `SQL()` function gains an optional `names` argument which can be used to assign names to SQL strings. Deprecated generics ------------------- - `dbListConnections()` is soft-deprecated by documentation. - `dbListResults()` is deprecated by documentation (#58). - `dbGetException()` is soft-deprecated by documentation (#51). - The deprecated `print.list.pairs()` has been removed. Bug fixes --------- - Fix `dbDataType()` for `AsIs` object (#198, @yutannihilation). - Fix `dbQuoteString()` and `dbQuoteIdentifier()` to ignore invalid UTF-8 strings (r-dbi/DBItest#156). Documentation ------------- - Help pages for generics now contain a dynamic list of methods implemented by DBI backends (#162). - `sqlInterpolate()` now supports both named and positional variables (#216, @hannesmuehleisen). - Point to db.rstudio.com (@wibeasley, #209). - Reflect new 'r-dbi' organization in `DESCRIPTION` (@wibeasley, #207). Internal -------- - Using switchpatch on the second argument for default implementations of `dbQuoteString()` and `dbQuoteIdentifier()`.

krlmlr added 8 commits January 28, 2018 18:57

use signature() for all methods

c0590c2

new dbUnquoteIdentifier()

9a9eb8d

declare dbListObjects()

984d549

dbQuoteIdentifier() works for lists

1f7101a

implement toString.Table

18c8e85

provide subsetting operators for class SQL

679bc6a

document slots of Table

ad9fd0f

export Table()

2fa6811

krlmlr mentioned this pull request Jan 29, 2018

Schema support r-dbi/RPostgres#171

Closed

krlmlr requested review from hadley and jimhester January 29, 2018 07:57

hadley reviewed Jan 29, 2018

View reviewed changes

require all arguments to be named

dd7ce92

jimhester reviewed Jan 29, 2018

View reviewed changes

krlmlr added 5 commits January 30, 2018 08:15

fix docs

9105b5c

type stability

69d7aa8

get rid of .

adee1b0

Table -> Id

14008e4

oops

b13a857

krlmlr mentioned this pull request Feb 2, 2018

Allowing the user to specify enclosing collections via dbQuoteIdentifier #222

Closed

fix missing bits and pieces

1dd0ebc

krlmlr merged commit e59d1cd into master Feb 2, 2018

krlmlr deleted the f-#24-schema branch February 3, 2018 00:56

etiennebr mentioned this pull request Mar 20, 2018

Support for schemas using DBI::Id tidyverse/dplyr#3452

Closed

github-actions bot locked and limited conversation to collaborators Oct 9, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Schema support #220

Schema support #220

krlmlr commented Jan 29, 2018 •

edited

Loading

codecov-io commented Jan 29, 2018 •

edited

Loading

hadley Jan 29, 2018

hadley Jan 29, 2018

hadley Jan 29, 2018

hadley Jan 29, 2018

hadley Jan 29, 2018

hadley Jan 29, 2018

hadley Jan 29, 2018

krlmlr Jan 29, 2018

hadley Jan 29, 2018

hadley Jan 29, 2018

hadley Jan 29, 2018

krlmlr Jan 29, 2018

jimhester Jan 29, 2018

krlmlr Jan 30, 2018

krlmlr Jan 30, 2018

jimhester Jan 30, 2018

krlmlr Jan 30, 2018

jimhester Jan 30, 2018

krlmlr Jan 31, 2018

jimhester Jan 29, 2018

jimhester Jan 29, 2018

krlmlr Jan 30, 2018

hadley Jan 30, 2018

jimhester Jan 29, 2018

jimhester Jan 29, 2018

jimhester Jan 29, 2018

krlmlr Jan 30, 2018

jimhester Jan 30, 2018

krlmlr Jan 30, 2018

Schema support #220

Schema support #220

Conversation

krlmlr commented Jan 29, 2018 • edited Loading

codecov-io commented Jan 29, 2018 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

krlmlr commented Jan 29, 2018 •

edited

Loading

codecov-io commented Jan 29, 2018 •

edited

Loading