Skip to content

Commit

Permalink
checkpoint
Browse files Browse the repository at this point in the history
  • Loading branch information
radeusgd committed Jun 1, 2023
1 parent 34be879 commit c060224
Show file tree
Hide file tree
Showing 5 changed files with 163 additions and 19 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ from Standard.Base.Metadata.Choice import Option
import Standard.Base.Metadata.Display

import Standard.Table.Data.Table.Table as Materialized_Table
import Standard.Table.Data.Type.Value_Type.Value_Type

import project.Data.SQL_Query.SQL_Query
import project.Data.SQL_Statement.SQL_Statement
Expand Down Expand Up @@ -191,6 +192,43 @@ type Connection
read self query limit=Nothing =
self.query query . read max_rows=limit

## PRIVATE
Creates a new empty table in the database.

Arguments:
- table_name: the name of the table to create. If not provided, a random
name will be generated for temporary tables. If `temporary=False`, then
a name must be provided.
- structure: the structure of the table. This can be provided as a vector
of pairs of column names and types or an existing `Table` to copy the
structure from it. Note that if a `Table` is provided, only its column
structure is inherited - no table content is copied.
- primary_key: the names of the columns to use as the primary key. The
first column from the table is used by default. If it is set to
`Nothing` or an empty vector, no primary key will be created.
- temporary: if set to `True`, the table will be temporary, meaning that
it will be dropped once the `connection` is closed. Defaults to
`False`.
- on_problems: the behavior to use when encountering non-fatal problems.
Defaults to reporting them as warning.

! Error Conditions

- If a table with the given name already exists, then a
`Table_Already_Exists` error is raised.
- If a column type is not supported and is coerced to a similar
supported type, an `Inexact_Type_Coercion` problem is reported
according to the `on_problems` setting.
- If a column type is not supported and there is no replacement (e.g.
native Enso types), an `Unsupported_Type` error is raised.
- If the provided primary key columns are not present in table
structure provided, `Missing_Input_Columns` error is raised.
- An `SQL_Error` may be reported if there is a failure on the database
side.
create_table : Text|Nothing -> Table | Vector (Pair Text Value_Type) -> Boolean -> Boolean -> Problem_Behavior -> Table ! Table_Already_Exists
create_table (table_name : Text | Nothing = Nothing) (structure : Vector | Table | Materialized_Table) (primary_key : Vector Text = first_column_in_structure structure) (temporary : Boolean = False) (allow_existing : Boolean = False) (on_problems : Problem_Behavior = Problem_Behavior.Report_Warning) =
create_table_structure connection table_name structure primary_key temporary allow_existing on_problems

## PRIVATE
Internal read function for a statement with optional types.

Expand Down Expand Up @@ -260,3 +298,8 @@ make_table_name_selector : Connection -> Widget
make_table_name_selector connection =
tables_to_display = connection.tables.at "Name" . to_vector
Single_Choice display=Display.Always values=(tables_to_display.map t-> Option t t.pretty)

## PRIVATE
first_column_in_structure structure = case structure of
_ : Vector -> structure.first.first
_ -> structure.columns_names.first
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,6 @@ from project.Internal.Upload_Table import all
empty vector, no primary key will be created.
- temporary: if set to `True`, the table will be temporary, meaning that it
will be dropped once the `connection` is closed. Defaults to `False`.
- structure_only: if set to `True`, the created table will inherit the
structure (column names and types) of the source table, but no rows will be
inserted. Defaults to `False`.
- on_problems: the behavior to use when encountering non-fatal problems.
Defaults to reporting them as warning.

Expand All @@ -47,11 +44,11 @@ from project.Internal.Upload_Table import all

If an error has been raised, the table is not created (that may not always
apply to `SQL_Error`).
In_Memory_Table.select_into_database_table : Connection -> Text|Nothing -> (Vector Text) | Nothing -> Boolean -> Boolean -> Problem_Behavior -> Database_Table ! Table_Already_Exists | Inexact_Type_Coercion | Missing_Input_Columns | Non_Unique_Primary_Key | SQL_Error | Illegal_Argument
In_Memory_Table.select_into_database_table self connection table_name=Nothing primary_key=[self.columns.first.name] temporary=False structure_only=False on_problems=Problem_Behavior.Report_Warning =
In_Memory_Table.select_into_database_table : Connection -> Text|Nothing -> (Vector Text) | Nothing -> Boolean -> Problem_Behavior -> Database_Table ! Table_Already_Exists | Inexact_Type_Coercion | Missing_Input_Columns | Non_Unique_Primary_Key | SQL_Error | Illegal_Argument
In_Memory_Table.select_into_database_table self connection table_name=Nothing primary_key=[self.columns.first.name] temporary=False on_problems=Problem_Behavior.Report_Warning =
Panic.recover SQL_Error <|
connection.jdbc_connection.run_within_transaction <|
upload_in_memory_table source_table connection table_name primary_key temporary structure_only on_problems
upload_in_memory_table source_table connection table_name primary_key temporary on_problems

## Creates a new database table from this table.

Expand All @@ -66,9 +63,6 @@ In_Memory_Table.select_into_database_table self connection table_name=Nothing pr
empty vector, no primary key will be created.
- temporary: if set to `True`, the table will be temporary, meaning that it
will be dropped once the `connection` is closed. Defaults to `False`.
- structure_only: if set to `True`, the created table will inherit the
structure (column names and types) of the source table, but no rows will be
inserted. Defaults to `False`.
- on_problems: the behavior to use when encountering non-fatal problems.
Defaults to reporting them as warning.

Expand All @@ -90,11 +84,11 @@ In_Memory_Table.select_into_database_table self connection table_name=Nothing pr

If an error has been raised, the table is not created (that may not always
apply to `SQL_Error`).
Database_Table.select_into_database_table : Connection -> Text|Nothing -> (Vector Text) | Nothing -> Boolean -> Boolean -> Problem_Behavior -> Database_Table ! Table_Already_Exists | Inexact_Type_Coercion | Missing_Input_Columns | Non_Unique_Primary_Key | SQL_Error | Illegal_Argument
Database_Table.select_into_database_table self connection table_name=Nothing primary_key=[self.columns.first.name] temporary=False structure_only=False on_problems=Problem_Behavior.Report_Warning = Panic.recover SQL_Error <|
Database_Table.select_into_database_table : Connection -> Text|Nothing -> (Vector Text) | Nothing -> Boolean -> Problem_Behavior -> Database_Table ! Table_Already_Exists | Inexact_Type_Coercion | Missing_Input_Columns | Non_Unique_Primary_Key | SQL_Error | Illegal_Argument
Database_Table.select_into_database_table self connection table_name=Nothing primary_key=[self.columns.first.name] temporary=False on_problems=Problem_Behavior.Report_Warning = Panic.recover SQL_Error <|
Panic.recover SQL_Error <|
connection.jdbc_connection.run_within_transaction <|
upload_database_table source_table connection table_name primary_key temporary structure_only on_problems
upload_database_table source_table connection table_name primary_key temporary on_problems

## Updates the target table with the contents of this table.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -105,12 +105,22 @@ type Postgres_Connection
tables self name_like=Nothing database=self.database schema=Nothing types=self.dialect.default_table_types all_fields=False =
self.connection.tables name_like database schema types all_fields

## Set up a query returning a Table object, which can be used to work with data within the database or load it into memory.
## Set up a query returning a Table object, which can be used to work with
data within the database or load it into memory.

Arguments:
- query: name of the table or sql statement to query.
If supplied as `Text`, the name is checked against the `tables` list to determine if it is a table or a query.
If supplied as `Text`, the name is checked against the `tables` list to
determine if it is a table or a query.
- alias: optionally specify a friendly alias for the query.

! Error Conditions

- If provided with a `Raw_SQL` query or `Text` that looks like a query, if
any SQL error occurs when executing the query, a `SQL_Error` error is
raised.
- If provided with a `Table_Name` or a text short-hand and the table is
not found, a `Table_Not_Found` error is raised.
@query make_table_name_selector
query : Text | SQL_Query -> Text -> Database_Table
query self query alias="" = self.connection.query query alias
Expand All @@ -125,6 +135,42 @@ type Postgres_Connection
read : Text | SQL_Query -> Integer | Nothing -> Materialized_Table
read self query limit=Nothing = self.connection.read query limit

## Creates a new empty table in the database.

Arguments:
- table_name: the name of the table to create. If not provided, a random
name will be generated for temporary tables. If `temporary=False`, then
a name must be provided.
- structure: the structure of the table. This can be provided as a vector
of pairs of column names and types or an existing `Table` to copy the
structure from it. Note that if a `Table` is provided, only its column
structure is inherited - no table content is copied.
- primary_key: the names of the columns to use as the primary key. The
first column from the table is used by default. If it is set to
`Nothing` or an empty vector, no primary key will be created.
- temporary: if set to `True`, the table will be temporary, meaning that
it will be dropped once the `connection` is closed. Defaults to
`False`.
- on_problems: the behavior to use when encountering non-fatal problems.
Defaults to reporting them as warning.

! Error Conditions

- If a table with the given name already exists, then a
`Table_Already_Exists` error is raised.
- If a column type is not supported and is coerced to a similar
supported type, an `Inexact_Type_Coercion` problem is reported
according to the `on_problems` setting.
- If a column type is not supported and there is no replacement (e.g.
native Enso types), an `Unsupported_Type` error is raised.
- If the provided primary key columns are not present in table
structure provided, `Missing_Input_Columns` error is raised.
- An `SQL_Error` may be reported if there is a failure on the database
side.
create_table : Text|Nothing -> Table | Vector (Pair Text Value_Type) -> Boolean -> Boolean -> Problem_Behavior -> Table ! Table_Already_Exists
create_table (table_name : Text | Nothing = Nothing) (structure : Vector | Table | Materialized_Table) (primary_key : Vector Text = first_column_in_structure structure) (temporary : Boolean = False) (allow_existing : Boolean = False) (on_problems:Problem_Behavior = Problem_Behavior.Report_Warning) =
self.connection.create_table table_name structure primary_key temporary allow_existing on_problems

## ADVANCED

Executes a raw update query. If the query was inserting, updating or
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -99,12 +99,22 @@ type SQLite_Connection
tables self name_like=Nothing database=self.database schema=Nothing types=self.dialect.default_table_types all_fields=False =
self.connection.tables name_like database schema types all_fields

## Set up a query returning a Table object, which can be used to work with data within the database or load it into memory.
## Set up a query returning a Table object, which can be used to work with
data within the database or load it into memory.

Arguments:
- query: name of the table or sql statement to query.
If supplied as `Text`, the name is checked against the `tables` list to determine if it is a table or a query.
If supplied as `Text`, the name is checked against the `tables` list to
determine if it is a table or a query.
- alias: optionally specify a friendly alias for the query.

! Error Conditions

- If provided with a `Raw_SQL` query or `Text` that looks like a query, if
any SQL error occurs when executing the query, a `SQL_Error` error is
raised.
- If provided with a `Table_Name` or a text short-hand and the table is
not found, a `Table_Not_Found` error is raised.
@query make_table_name_selector
query : Text | SQL_Query -> Text -> Database_Table
query self query alias="" = self.connection.query query alias
Expand All @@ -119,6 +129,42 @@ type SQLite_Connection
read : Text | SQL_Query -> Integer | Nothing -> Materialized_Table
read self query limit=Nothing = self.connection.read query limit

## Creates a new empty table in the database.

Arguments:
- table_name: the name of the table to create. If not provided, a random
name will be generated for temporary tables. If `temporary=False`, then
a name must be provided.
- structure: the structure of the table. This can be provided as a vector
of pairs of column names and types or an existing `Table` to copy the
structure from it. Note that if a `Table` is provided, only its column
structure is inherited - no table content is copied.
- primary_key: the names of the columns to use as the primary key. The
first column from the table is used by default. If it is set to
`Nothing` or an empty vector, no primary key will be created.
- temporary: if set to `True`, the table will be temporary, meaning that
it will be dropped once the `connection` is closed. Defaults to
`False`.
- on_problems: the behavior to use when encountering non-fatal problems.
Defaults to reporting them as warning.

! Error Conditions

- If a table with the given name already exists, then a
`Table_Already_Exists` error is raised.
- If a column type is not supported and is coerced to a similar
supported type, an `Inexact_Type_Coercion` problem is reported
according to the `on_problems` setting.
- If a column type is not supported and there is no replacement (e.g.
native Enso types), an `Unsupported_Type` error is raised.
- If the provided primary key columns are not present in table
structure provided, `Missing_Input_Columns` error is raised.
- An `SQL_Error` may be reported if there is a failure on the database
side.
create_table : Text|Nothing -> Table | Vector (Pair Text Value_Type) -> Boolean -> Boolean -> Problem_Behavior -> Table ! Table_Already_Exists
create_table (table_name : Text | Nothing = Nothing) (structure : Vector | Table | Materialized_Table) (primary_key : Vector Text = first_column_in_structure structure) (temporary : Boolean = False) (allow_existing : Boolean = False) (on_problems:Problem_Behavior = Problem_Behavior.Report_Warning) =
self.connection.create_table table_name structure primary_key temporary allow_existing on_problems

## ADVANCED

Executes a raw update query. If the query was inserting, updating or
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,17 +14,32 @@ import project.Data.Update_Action.Update_Action
import project.Internal.In_Transaction.In_Transaction
import project.Internal.IR.Query.Query
import project.Internal.IR.SQL_Expression.SQL_Expression
from project.Connection.Connection import all_known_table_names
from project.Errors import all

## PRIVATE
create_table_structure connection table_name structure primary_key temporary allow_existing on_problems =
case table_name.is_nothing.not && all_known_table_names connection . contains table_name of
True ->
if allow_existing then Nothing else Error.throw (Table_Already_Exists.Error table_name)
False ->
effective_table_name = resolve_effective_table_name table_name temporary
# TODO fix this structure not necessarily being a Table
# TODO migrate select into to use this method
resolved_primary_key = resolve_primary_key structure primary_key
create_table_statement = prepare_create_table_statement connection effective_table_name self.columns resolved_primary_key temporary on_problems
create_table_statement.if_not_error <|
connection.execute_update create_table_statement

## PRIVATE
A helper that can upload a table from any backend to a database.
It should be run within a transaction.
upload_table source_table connection table_name primary_key temporary structure_only on_problems =
upload_table source_table connection table_name primary_key temporary on_problems =
case source_table of
_ : In_Memory_Table ->
upload_in_memory_table source_table connection table_name primary_key temporary structure_only on_problems
upload_in_memory_table source_table connection table_name primary_key temporary on_problems
_ : Database_Table ->
upload_database_table source_table connection table_name primary_key temporary structure_only on_problems
upload_database_table source_table connection table_name primary_key temporary on_problems
_ ->
Panic.throw <| Illegal_Argument.Error ("Unsupported table type: " + Meta.get_qualified_type_name source_table)

Expand Down

0 comments on commit c060224

Please sign in to comment.