Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Schema support #220

Merged
merged 15 commits into from
Feb 2, 2018
7 changes: 7 additions & 0 deletions NAMESPACE
Original file line number Diff line number Diff line change
@@ -1,9 +1,13 @@
# Generated by roxygen2: do not edit by hand

S3method("[",SQL)
S3method("[[",SQL)
S3method(toString,Table)
export(.SQL92Keywords)
export(ANSI)
export(SQL)
export(SQLKeywords)
export(Table)
export(dbBegin)
export(dbBind)
export(dbBreak)
Expand All @@ -29,6 +33,7 @@ export(dbHasCompleted)
export(dbIsValid)
export(dbListConnections)
export(dbListFields)
export(dbListObjects)
export(dbListResults)
export(dbListTables)
export(dbQuoteIdentifier)
Expand All @@ -41,6 +46,7 @@ export(dbSendQuery)
export(dbSendStatement)
export(dbSetDataMappings)
export(dbUnloadDriver)
export(dbUnquoteIdentifier)
export(dbWithTransaction)
export(dbWriteTable)
export(fetch)
Expand Down Expand Up @@ -74,6 +80,7 @@ exportMethods(dbQuoteLiteral)
exportMethods(dbQuoteString)
exportMethods(dbReadTable)
exportMethods(dbSendStatement)
exportMethods(dbUnquoteIdentifier)
exportMethods(dbWithTransaction)
exportMethods(show)
exportMethods(sqlAppendTable)
Expand Down
32 changes: 31 additions & 1 deletion R/DBConnection.R
Original file line number Diff line number Diff line change
Expand Up @@ -362,6 +362,36 @@ setGeneric("dbListTables",
valueClass = "character"
)

#' List remote objects
#'
#' Returns the names of remote tables accessible through this connection
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tables -> objects ?

#' (possibly via a prefix) as a data frame.
#' This should, where possible, include temporary objects.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe "This should include temporary objects, but not all database drivers support listing temporary objects"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps include a sentence describing how this is different to dbListTables()?

#'
#' @template methods
#' @templateVar method_name dbListObjects
#'
#' @inherit DBItest::spec_sql_list_objects return
#' @inheritSection DBItest::spec_sql_list_objects Additional arguments
#'
#' @inheritParams dbGetQuery
#' @param prefix An optional prefix, passed to [dbUnquoteIdentifier()].
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is worded too generically for me to understand

#' If given the method will return all objects accessible through this prefix.
#' @family DBIConnection generics
#' @export
#' @examples
#' con <- dbConnect(RSQLite::SQLite(), ":memory:")
#'
#' dbListObjects(con)
#' dbWriteTable(con, "mtcars", mtcars)
#' dbListObjects(con)
#'
#' dbDisconnect(con)
setGeneric("dbListObjects",
def = function(conn, prefix = NULL, ...) standardGeneric("dbListObjects"),
valueClass = "data.frame"
)

#' Copy data frames from database tables
#'
#' Reads a database table to a data frame, optionally converting
Expand Down Expand Up @@ -394,7 +424,7 @@ setGeneric("dbReadTable",

#' @rdname hidden_aliases
#' @export
setMethod("dbReadTable", c("DBIConnection", "character"),
setMethod("dbReadTable", signature("DBIConnection", "character"),
function(conn, name, ..., row.names = FALSE, check.names = TRUE) {
sql_name <- dbQuoteIdentifier(conn, x = name, ...)
if (length(sql_name) != 1L) {
Expand Down
4 changes: 2 additions & 2 deletions R/DBDriver.R
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ setMethod("dbDriver", signature("character"),
#' @rdname hidden_aliases
#' @param object Object to display
#' @export
setMethod("show", "DBIDriver", function(object) {
setMethod("show", signature("DBIDriver"), function(object) {
tryCatch(
# to protect drivers that fail to implement the required methods (e.g.,
# RPostgreSQL)
Expand Down Expand Up @@ -251,6 +251,6 @@ setGeneric("dbDataType",

#' @rdname hidden_aliases
#' @export
setMethod("dbDataType", "DBIObject", function(dbObj, obj, ...) {
setMethod("dbDataType", signature("DBIObject"), function(dbObj, obj, ...) {
dbiDataType(obj)
})
6 changes: 3 additions & 3 deletions R/DBResult.R
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ setClass("DBIResult", contains = c("DBIObject", "VIRTUAL"))
#' @rdname hidden_aliases
#' @param object Object to display
#' @export
setMethod("show", "DBIResult", function(object) {
setMethod("show", signature("DBIResult"), function(object) {
# to protect drivers that fail to implement the required methods (e.g.,
# RPostgreSQL)
tryCatch(
Expand Down Expand Up @@ -101,7 +101,7 @@ setGeneric("dbFetch",

#' @rdname hidden_aliases
#' @export
setMethod("dbFetch", "DBIResult", function(res, n = -1, ...) {
setMethod("dbFetch", signature("DBIResult"), function(res, n = -1, ...) {
fetch(res, n = n, ...)
})

Expand Down Expand Up @@ -303,7 +303,7 @@ setGeneric("dbGetRowCount",
#' [dbGetRowsAffected()], and [dbHasCompleted()].
NULL
#' @rdname hidden_aliases
setMethod("dbGetInfo", "DBIResult", function(dbObj, ...) {
setMethod("dbGetInfo", signature("DBIResult"), function(dbObj, ...) {
list(
statement = dbGetStatement(dbObj),
row.count = dbGetRowCount(dbObj),
Expand Down
22 changes: 11 additions & 11 deletions R/data-types.R
Original file line number Diff line number Diff line change
Expand Up @@ -30,14 +30,14 @@ as_is_data_type <- function(x) {
setOldClass("difftime")
setOldClass("AsIs")

setMethod("dbiDataType", "data.frame", data_frame_data_type)
setMethod("dbiDataType", "integer", function(x) "INT")
setMethod("dbiDataType", "numeric", function(x) "DOUBLE")
setMethod("dbiDataType", "logical", function(x) "SMALLINT")
setMethod("dbiDataType", "Date", function(x) "DATE")
setMethod("dbiDataType", "difftime", function(x) "TIME")
setMethod("dbiDataType", "POSIXct", function(x) "TIMESTAMP")
setMethod("dbiDataType", "character", varchar_data_type)
setMethod("dbiDataType", "factor", varchar_data_type)
setMethod("dbiDataType", "list", list_data_type)
setMethod("dbiDataType", "AsIs", as_is_data_type)
setMethod("dbiDataType", signature("data.frame"), data_frame_data_type)
setMethod("dbiDataType", signature("integer"), function(x) "INT")
setMethod("dbiDataType", signature("numeric"), function(x) "DOUBLE")
setMethod("dbiDataType", signature("logical"), function(x) "SMALLINT")
setMethod("dbiDataType", signature("Date"), function(x) "DATE")
setMethod("dbiDataType", signature("difftime"), function(x) "TIME")
setMethod("dbiDataType", signature("POSIXct"), function(x) "TIMESTAMP")
setMethod("dbiDataType", signature("character"), varchar_data_type)
setMethod("dbiDataType", signature("factor"), varchar_data_type)
setMethod("dbiDataType", signature("list"), list_data_type)
setMethod("dbiDataType", signature("AsIs"), as_is_data_type)
2 changes: 1 addition & 1 deletion R/data.R
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ setGeneric("sqlData",

#' @rdname hidden_aliases
#' @export
setMethod("sqlData", "DBIConnection", function(con, value, row.names = NA, ...) {
setMethod("sqlData", signature("DBIConnection"), function(con, value, row.names = NA, ...) {
value <- sqlRownamesToColumn(value, row.names)

# Convert factors to strings
Expand Down
4 changes: 2 additions & 2 deletions R/deprecated.R
Original file line number Diff line number Diff line change
Expand Up @@ -144,13 +144,13 @@ setGeneric("SQLKeywords",
)

#' @rdname hidden_aliases
setMethod("SQLKeywords", "DBIObject",
setMethod("SQLKeywords", signature("DBIObject"),
definition = function(dbObj, ...) .SQL92Keywords,
valueClass = "character"
)

#' @rdname hidden_aliases
setMethod("SQLKeywords", "missing",
setMethod("SQLKeywords", signature("missing"),
definition = function(dbObj, ...) .SQL92Keywords,
valueClass = "character"
)
Expand Down
4 changes: 2 additions & 2 deletions R/interpolate.R
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ setGeneric("sqlInterpolate",

#' @rdname hidden_aliases
#' @export
setMethod("sqlInterpolate", "DBIConnection", function(conn, sql, ..., .dots = list()) {
setMethod("sqlInterpolate", signature("DBIConnection"), function(conn, sql, ..., .dots = list()) {
pos <- sqlParseVariables(conn, sql)

if (length(pos$start) == 0) {
Expand Down Expand Up @@ -110,7 +110,7 @@ setGeneric("sqlParseVariables",

#' @rdname hidden_aliases
#' @export
setMethod("sqlParseVariables", "DBIConnection", function(conn, sql, ...) {
setMethod("sqlParseVariables", signature("DBIConnection"), function(conn, sql, ...) {
sqlParseVariablesImpl(
sql,
list(
Expand Down
97 changes: 88 additions & 9 deletions R/quote.R
Original file line number Diff line number Diff line change
Expand Up @@ -61,16 +61,22 @@ setMethod("show", "SQL", function(object) {
cat(paste0("<SQL> ", [email protected], collapse = "\n"), "\n", sep = "")
})

#' @export
`[.SQL` <- function(x, ...) SQL(NextMethod())

#' @export
`[[.SQL` <- function(x, ...) SQL(NextMethod())

#' Quote identifiers
#'
#' Call this method to generate a string that is suitable for
#' use in a query as a column name, to make sure that you
#' generate valid SQL and avoid SQL injection.
#' use in a query as a column or table name, to make sure that you
#' generate valid SQL and avoid SQL injection. The inverse operation is
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

injection -> injection attacks.

#' [dbUnquoteIdentifier()].
#'
#' @param conn A subclass of [DBIConnection-class], representing
#' an active connection to an DBMS.
#' @param x A character vector to quote as identifier.
#' @param x A character vector, [SQL] or [Table] object to quote as identifier.
#' @param ... Other arguments passed on to methods.
#'
#' @template methods
Expand Down Expand Up @@ -100,6 +106,9 @@ setGeneric("dbQuoteIdentifier",

quote_identifier <-
function(conn, x, ...) {
if (is.list(x)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think auto-vectorisation over lists is best avoided because it violates type-stability (and we generally avoid it elsewhere)

return(vapply(x, function(xx) list(dbQuoteIdentifier(conn, xx)), list(1)))
}
if (is(x, "SQL")) return(x)
if (is(x, "Table")) {
return(SQL(paste0(dbQuoteIdentifier(conn, x@name), collapse = ".")))
Expand All @@ -121,17 +130,87 @@ quote_identifier <-

#' @rdname hidden_aliases
#' @export
setMethod("dbQuoteIdentifier", c("DBIConnection"), quote_identifier)
setMethod("dbQuoteIdentifier", signature("DBIConnection"), quote_identifier)

# Need to keep other method declarations around for now, because clients might
# use getMethod(), see e.g. https://github.com/r-dbi/odbc/pull/149
#' @rdname hidden_aliases
#' @export
setMethod("dbQuoteIdentifier", c("DBIConnection", "character"), quote_identifier)
setMethod("dbQuoteIdentifier", signature("DBIConnection", "character"), quote_identifier)

#' @rdname hidden_aliases
#' @export
setMethod("dbQuoteIdentifier", c("DBIConnection", "SQL"), quote_identifier)
setMethod("dbQuoteIdentifier", signature("DBIConnection", "SQL"), quote_identifier)

#' Unquote identifiers
#'
#' Call this method to convert a [SQL] object created by [dbQuoteIdentifier()]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After reading this documentation I don't know what this is going to return. It sounds like it's going to return table objects which doesn't seem right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Return values are part of the specs in DBItest which I haven't adapted yet.

The method is returning a Table object. Why does this seem wrong?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Root of my confusion here is that Table is a reference/identifier, not a table.

#' back to a list of [Table] objects.
#'
#' @param conn A subclass of [DBIConnection-class], representing
#' an active connection to an DBMS.
#' @param x An [SQL] or [Table] object or character vector, or a list of such
#' objects, to unquote.
#' @param ... Other arguments passed on to methods.
#'
#' @template methods
#' @templateVar method_name dbUnquoteIdentifier
#'
#' @inherit DBItest::spec_sql_unquote_identifier return
#' @inheritSection DBItest::spec_sql_unquote_identifier Specification
#'
#' @family DBIResult generics
#' @export
#' @examples
#' # Unquoting allows to understand the structure of a possibly complex quoted
#' # identifier
#'
#' dbUnquoteIdentifier(
#' ANSI(),
#' SQL(c("Schema"."Table", "UnqualifiedTable"))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless I am missing something this does not parse. I think you are missing a set of quotes.

SQL(c("Schema"."Table", "UnqualifiedTable"))
#> Error: unexpected symbol in "SQL(c("Schema"."

#' )
#'
#' # Character vectors are wrapped in a list
#' dbQuoteIdentifier(
#' ANSI(),
#' c(schema = "Schema", table = "Table")
#' )
#'
#' # Lists of character vectors are returned unchanged
#' dbQuoteIdentifier(
#' ANSI(),
#' list(c(schema = "Schema", table = "Table"), "UnqualifiedTable")
#' )
setGeneric("dbUnquoteIdentifier",
def = function(conn, x, ...) standardGeneric("dbUnquoteIdentifier")
)

#' @rdname hidden_aliases
#' @export
setMethod("dbUnquoteIdentifier", signature("DBIConnection"), function(conn, x, ...) {
if (is.list(x)) {
return(vapply(x, dbUnquoteIdentifier, conn = conn, list(1)))
}
if (is(x, "SQL")) {
. <- strsplit(as.character(x), '^"|"$|"[.]"')
. <- lapply(., `[`, -1L)
split <- .
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realize you likely converted this from a magrittr pipeline, but I think this would be better as

split <- lapply(strsplit(as.character(x), '^"|"$|"[.]"'), `[`, -1L)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And actually I think it is more explicit to use tail than [ here, but that is largely a matter of taste.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like nested expressions, and the dot looks like the lesser of all evils here to me, because I don't need to invent names for intermediates I don't care about.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest, I really don't like the use of . here. It feels so different to the style of the rest of the code.

tables <- lapply(split, Table)
quoted <- lapply(tables, dbQuoteIdentifier, conn = conn)
bad <- quoted != x
if (any(bad)) {
stop("Can't unquote ", x[bad][[1L]], call. = FALSE)
}
return(tables)
}
if (is(x, "Table")) {
return(list(x))
}
if (is.character(x)) {
return(list(do.call(Table, as.list(x))))
}
stop("x must be character, SQL or Table, or a list of such objects", call. = FALSE)
})

#' Quote literal strings
#'
Expand Down Expand Up @@ -194,15 +273,15 @@ quote_string <-
# use getMethod(), see e.g. https://github.com/r-dbi/odbc/pull/149
#' @rdname hidden_aliases
#' @export
setMethod("dbQuoteString", c("DBIConnection"), quote_string)
setMethod("dbQuoteString", signature("DBIConnection"), quote_string)

#' @rdname hidden_aliases
#' @export
setMethod("dbQuoteString", c("DBIConnection", "character"), quote_string)
setMethod("dbQuoteString", signature("DBIConnection", "character"), quote_string)

#' @rdname hidden_aliases
#' @export
setMethod("dbQuoteString", c("DBIConnection", "SQL"), quote_string)
setMethod("dbQuoteString", signature("DBIConnection", "SQL"), quote_string)

#' Quote literal values
#'
Expand Down
21 changes: 17 additions & 4 deletions R/table.R
Original file line number Diff line number Diff line change
Expand Up @@ -3,21 +3,34 @@ setClass("Table", slots = list(name = "character"))

#' Refer to a table nested in a hierarchy (e.g. within a schema)
#'
#' Objects of class `Table` have a single slot `name`, which is a named
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Calling this a table seems slightly confusing since it's the name of a table, not a table itself. Maybe TableRef?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And doesn't every other object start with db?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Table() doesn't need the connection, it's just a dumb object like SQL(). We could use Identifier() too.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest Id()

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Id() looks very prone to collisions with other packages...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This search doesn't find packages that export crossing: https://github.com/search?utf8=%E2%9C%93&q=user%3Acran+%22export+crossing%22+file%3ANAMESPACE&type=Code . I remember a recent discussion on Twitter about searching for packages that export a function?

SqlId() looks inconsistent to SQL(), and SQLId() looks ... interesting.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, it needs to be filename: not file:. https://github.com/search?utf8=%E2%9C%93&q=user%3Acran+filename%3ANAMESPACE+%22export+Id%22&type= is the correct search, and again no conflicts for Id, the only functions are lowercase id.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure, but it seems like GitHub limits the length of time a query can run. Anyway I ran the search on my Local CRAN mirror, there are no CRAN packages that export Id.

You can call it Identifier if you like it better.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed (from a scraping of my local CRAN mirror) that no package exports an Id class either. Renaming.

#' character vector.
#'
#' @param ... Components of the hierarchy, e.g. `schema`, `table`,
#' or `cluster`, `catalog`, `schema`, `table`.
#' For more on these concepts, see
#' \url{http://stackoverflow.com/questions/7022755/}
#' @export
Table <- function(...) {
new("Table", name = c(...))
components <- c(...)
if (is.null(names(components)) || any(names(components) == "")) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure we want to force naming, particularly for the most common case (schema & table).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which comes first -- schema or table? If we force naming, that confusion won't arise.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but it makes it much more cumbersome to use the function. If it is cumbersome to use no one is going to use it. Having Table("foo", "bar") map to 'foo'.'bar' seems like an easy solution, and if your backend uses different names you can instead use named arguments.

if (length(components) == 2) && (is.null(names(components)) || all(names(components) == ""))) {
  names(components) <- c("schema", "table")
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Backends are free to implement their own wrappers with named arguments, and probably should too.

For instance, in BigQuery, "schema" means something entirely different. I'd rather keep things strict for the first implementation, we can always relax later but it's more difficult to impose restrictions on a function that has been exported.

stop("All arguments to Table() must be named.", call. = FALSE)
}
new("Table", name = components)
}

#' @rdname hidden_aliases
#' @param object Table object to print
#' @export
setMethod("show", "Table", function(object) {
cat("<Table> ", paste0(object@name, collapse = "."), "\n", sep = "")
setMethod("show", signature("Table"), function(object) {
cat(toString(object), "\n", sep = "")
})

#' @export
toString.Table <- function(x, ...) {
paste0("<Table> ", paste(x@name, collapse = "."))
}

#' @rdname hidden_aliases
#' @export
setMethod("dbQuoteIdentifier", c("DBIConnection", "Table"), quote_identifier)
setMethod("dbQuoteIdentifier", signature("DBIConnection", "Table"), quote_identifier)
2 changes: 1 addition & 1 deletion R/transactions.R
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@ setGeneric("dbWithTransaction",

#' @rdname hidden_aliases
#' @export
setMethod("dbWithTransaction", "DBIConnection", function(conn, code) {
setMethod("dbWithTransaction", signature("DBIConnection"), function(conn, code) {
## needs to be a closure, because it accesses conn
rollback_because <- function(e) {
call <- dbRollback(conn)
Expand Down
Loading