Skip to content

Commit

Permalink
Update rename_columns to new API design, add first_row, `second_r…
Browse files Browse the repository at this point in the history
…ow` and `last_row` functions to the table. (#5719)

- Updates the `rename_columns` API.
- Add `first_row`, `second_row` and `last_row` to the Table types.
- New option for reading only last row of ResultSet.
  • Loading branch information
jdunkerley authored Feb 23, 2023
1 parent b8158d0 commit 652b8d5
Show file tree
Hide file tree
Showing 21 changed files with 305 additions and 231 deletions.
5 changes: 4 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -323,6 +323,8 @@
- [`File.parent` may return `Nothing`.][5699]
- [Removed non-regex functionality from `is_match`, `match`, and `match_all`,
and renamed them to `match`, `find`, `find_all` (respectively).][5721]
- [Updated `rename_columns` to new API. Added `first_row`, `second_row` and
`last_row` to Table types][5719]

[debug-shortcuts]:
https://github.com/enso-org/enso/blob/develop/app/gui/docs/product/shortcuts.md#debug
Expand Down Expand Up @@ -493,9 +495,10 @@
[5645]: https://github.com/enso-org/enso/pull/5645
[5646]: https://github.com/enso-org/enso/pull/5646
[5656]: https://github.com/enso-org/enso/pull/5656
[5678]: https://github.com/enso-org/enso/pull/5678
[5679]: https://github.com/enso-org/enso/pull/5679
[5699]: https://github.com/enso-org/enso/pull/5699
[5678]: https://github.com/enso-org/enso/pull/5678
[5719]: https://github.com/enso-org/enso/pull/5719
[5721]: https://github.com/enso-org/enso/pull/5721

#### Enso Compiler
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
import project.Data.Locale.Locale
import project.Data.Text.Regex
import project.Data.Text.Text

from project.Data.Boolean import Boolean, True, False

Expand Down Expand Up @@ -29,9 +31,19 @@ type Case_Sensitivity
TextFoldingStrategy.caseInsensitiveFold locale.java_locale

## PRIVATE
Is case insensitive.
is_case_insensitive : Boolean
is_case_insensitive self = case self of
Is case insensitive when in memory.
is_case_insensitive_in_memory : Boolean
is_case_insensitive_in_memory self = case self of
Case_Sensitivity.Default -> False
Case_Sensitivity.Sensitive -> False
Case_Sensitivity.Insensitive _ -> True

## PRIVATE
Create matcher function
create_match_function : Boolean -> (Text -> Text -> Boolean)
create_match_function self use_regex=False = case use_regex of
True -> (name-> pattern-> Regex.compile pattern case_insensitive=self.is_case_insensitive_in_memory . matches name)
False -> case self of
Case_Sensitivity.Default -> (==)
Case_Sensitivity.Sensitive -> (==)
Case_Sensitivity.Insensitive locale -> (name-> criterion-> name.equals_ignore_case criterion locale)
Original file line number Diff line number Diff line change
Expand Up @@ -229,7 +229,7 @@ Text.characters self =
"aabbbbccccaaBcaaaa".find "a[ab]c" Case_Sensitivity.Insensitive
Text.find : Text -> Case_Sensitivity -> Match | Nothing ! Compile_Error
Text.find self pattern=".*" case_sensitivity=Case_Sensitivity.Sensitive =
case_insensitive = case_sensitivity.is_case_insensitive
case_insensitive = case_sensitivity.is_case_insensitive_in_memory
Regex.compile pattern case_insensitive=case_insensitive . match self Matching_Mode.First

## Finds all the matches of the regular expression `pattern` in `self`,
Expand All @@ -251,7 +251,7 @@ Text.find self pattern=".*" case_sensitivity=Case_Sensitivity.Sensitive =
"aABbbbccccaaBCaaaa".find_all "a[ab]+c" Case_Sensitivity.Insensitive
Text.find_all : Text -> Case_Sensitivity -> Vector Match ! Compile_Error
Text.find_all self pattern=".*" case_sensitivity=Case_Sensitivity.Sensitive =
case_insensitive = case_sensitivity.is_case_insensitive
case_insensitive = case_sensitivity.is_case_insensitive_in_memory
case Regex.compile pattern case_insensitive=case_insensitive . match self Regex_Mode.All of
Nothing -> []
matches -> matches
Expand All @@ -278,7 +278,7 @@ Text.find_all self pattern=".*" case_sensitivity=Case_Sensitivity.Sensitive =
"[email protected]".match regex Case_Sensitivity.Insensitive
Text.match : Text -> Case_Sensitivity -> Boolean ! Compile_Error
Text.match self pattern=".*" case_sensitivity=Case_Sensitivity.Sensitive =
case_insensitive = case_sensitivity.is_case_insensitive
case_insensitive = case_sensitivity.is_case_insensitive_in_memory
compiled_pattern = Regex.compile pattern case_insensitive=case_insensitive
compiled_pattern.matches self

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ type Pattern
Splits the `input` text based on the pattern described by `self`.

Arguments:
- input: The text to splut based on the pattern described by `self`.
- input: The text to split based on the pattern described by `self`.
- mode: The splitting mode to use. This must default to `Regex_Mode.All`.

This method will _always_ return a vector. If no splits take place, the
Expand Down
5 changes: 3 additions & 2 deletions distribution/lib/Standard/Base/0.0.0-dev/src/Data/Vector.enso
Original file line number Diff line number Diff line change
Expand Up @@ -1034,12 +1034,13 @@ type Builder
builder = Vector.new_builder
builder . append_vector_range [20, 30, 40, 50] 1 3 . to_vector == [30, 40]
append_vector_range : Vector Any ! Error -> Integer -> Integer -> Builder ! Error
append_vector_range self vector start end =
append_vector_range self vector start=0 end=vector.length =
subrange = vector.slice start end
## This workaround is needed because
`self.java_builder.addAll subrange.to_array` fails with
`Unsupported argument types: [Array]`.
self.java_builder.appendTo subrange.to_array
append_result = self.java_builder.appendTo subrange.to_array
append_result.if_not_error self

## PRIVATE
Appends a new element into this builder.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
from Standard.Base import all
import Standard.Base.Error.Illegal_State.Illegal_State

from Standard.Table import Column_Name_Mapping
import Standard.Table.Data.Table.Table as Materialized_Table

import project.Data.SQL_Query.SQL_Query
Expand Down Expand Up @@ -103,7 +102,7 @@ type Connection
table = result_set_to_table <|
metadata.getTables database schema name_like types_array

renamed = table.rename_columns (Column_Name_Mapping.By_Name name_map)
renamed = table.rename_columns name_map
if all_fields then renamed else
renamed.select_columns ["Database", "Schema", "Name", "Type", "Description"]

Expand Down Expand Up @@ -141,11 +140,26 @@ type Connection

## PRIVATE
Internal read function for a statement with optional types.

Arguments:
- statement: SQL_Statement to execute.
- expected_types: Optional vector of expected types for each column.
read_statement : SQL_Statement -> (Nothing | Vector SQL_Type) -> Materialized_Table
read_statement self statement expected_types=Nothing =
self.jdbc_connection.with_prepared_statement statement stmt->
result_set_to_table stmt.executeQuery expected_types

## PRIVATE
Internal read function for a statement with optional types returning just last row.

Arguments:
- statement: SQL_Statement to execute.
- expected_types: Optional vector of expected types for each column.
read_last_row : SQL_Statement -> (Nothing | Vector SQL_Type) -> Materialized_Table
read_last_row self statement expected_types=Nothing =
self.jdbc_connection.with_prepared_statement statement stmt->
result_set_to_table stmt.executeQuery expected_types True

## ADVANCED

Executes a raw update query. If the query was inserting, updating or
Expand Down
60 changes: 44 additions & 16 deletions distribution/lib/Standard/Database/0.0.0-dev/src/Data/Table.enso
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import Standard.Base.Error.Illegal_State.Illegal_State
import Standard.Base.Error.Incomparable_Values.Incomparable_Values
import Standard.Base.Error.Unimplemented.Unimplemented

from Standard.Table import Auto_Detect, Aggregate_Column, Data_Formatter, Column_Name_Mapping, Column_Selector, Sort_Column, Match_Columns, Position, Set_Mode
from Standard.Table import Auto_Detect, Aggregate_Column, Data_Formatter, Column_Selector, Sort_Column, Match_Columns, Position, Set_Mode
import Standard.Table.Data.Column_Type_Selection.Column_Type_Selection
import Standard.Table.Data.Expression.Expression
import Standard.Table.Data.Expression.Expression_Error
Expand Down Expand Up @@ -359,12 +359,22 @@ type Table
other, a `Duplicate_Output_Column_Names`.

> Example
Rename the first column to "FirstColumn"
Rename the "Alpha" column to "Beta"

table.rename_columns (Column_Name_Mapping.By_Position ["FirstColumn"])
table.rename_columns (Map.from_vector [["Alpha", "Beta"]])

> Example
Rename the first column to "FirstColumn" passed as a Vector
Rename the last column to "LastColumn"

table.rename_columns (Map.from_vector [[-1, "LastColumn"]])

> Example
Rename the "Alpha" column to "Beta" and last column to "LastColumn"

table.rename_columns (Map.from_vector [["Alpha", "Beta"], [-1, "LastColumn"]])

> Example
Rename the first column to "FirstColumn"

table.rename_columns ["FirstColumn"]

Expand All @@ -376,18 +386,12 @@ type Table
> Example
For all columns starting with the prefix `name=`, replace it with `key:`.

table.rename_columns (Column_Name_Mapping.By_Name (Map.from_vector [["name=(.*)", "key:$1"]]) Regex_Matcher.Value)
rename_columns : Map | Vector Text | Column_Name_Mapping -> Boolean -> Problem_Behavior -> Table ! Missing_Input_Columns | Column_Indexes_Out_Of_Range | Ambiguous_Column_Rename | Too_Many_Column_Names_Provided | Invalid_Output_Column_Names | Duplicate_Output_Column_Names
rename_columns self (column_map=(Column_Name_Mapping.By_Position ["Column"])) (error_on_missing_columns=True) (on_problems=Report_Warning) = case column_map of
_ : Vector ->
self.rename_columns (Column_Name_Mapping.By_Position column_map) error_on_missing_columns on_problems
_ : Map ->
self.rename_columns (Column_Name_Mapping.By_Name column_map) error_on_missing_columns on_problems
_ ->
case Table_Helpers.rename_columns internal_columns=self.internal_columns mapping=column_map error_on_missing_columns=error_on_missing_columns on_problems=on_problems of
new_names ->
new_columns = self.internal_columns.map_with_index i->c->(c.rename (new_names.at i))
self.updated_columns new_columns
by_name = Column_Selector.By_Name "name=(.*)" Case_Sensitivity.Sensitive use_regex=True
table.rename_columns (Map.from_vector [[by_name, "key:$1"]])
rename_columns : Map (Text | Integer | Column_Selector) Text | Vector Text -> Boolean -> Problem_Behavior -> Table ! Missing_Input_Columns | Column_Indexes_Out_Of_Range | Ambiguous_Column_Rename | Too_Many_Column_Names_Provided | Invalid_Output_Column_Names | Duplicate_Output_Column_Names
rename_columns self column_map=["Column"] (error_on_missing_columns=True) (on_problems=Report_Warning) =
new_names = Table_Helpers.rename_columns internal_columns=self.internal_columns mapping=column_map error_on_missing_columns=error_on_missing_columns on_problems=on_problems
new_names.if_not_error (self.updated_columns (self.internal_columns.map c-> c.rename (new_names.at c.name)))

## PRIVATE

Expand Down Expand Up @@ -702,6 +706,30 @@ type Table
rows self max_rows=1000 =
self.read max_rows=max_rows . rows

## Returns the first row of the table.

In the database backend, it first materializes the table to in-memory.
first_row : Row ! Index_Out_Of_Bounds
first_row self =
self.read max_rows=1 . rows . first

## Returns the second row of the table.

In the database backend, it first materializes the table to in-memory.
second_row : Row ! Index_Out_Of_Bounds
second_row self =
self.read max_rows=2 . rows . second

## Returns the last row of the table.

In the database backend, it first materializes the table to in-memory.
last_row : Row ! Index_Out_Of_Bounds
last_row self =
if self.internal_columns.is_empty then Error.throw (Illegal_Argument.Error "Cannot create a table with no columns.") else
sql = self.to_sql
expected_types = self.internal_columns.map .sql_type
self.connection.read_last_row sql expected_types . rows . first

## ALIAS sort
Sorts the rows of the table according to the specified columns and order.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ polyglot java import java.util.Properties
polyglot java import java.sql.Connection
polyglot java import java.sql.DatabaseMetaData
polyglot java import java.sql.PreparedStatement
polyglot java import java.sql.ResultSet
polyglot java import java.sql.SQLException
polyglot java import java.sql.SQLTimeoutException

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -115,10 +115,24 @@ type Postgres_Connection

## PRIVATE
Internal read function for a statement with optional types.

Arguments:
- statement: SQL_Statement to execute.
- expected_types: Optional vector of expected types for each column.
read_statement : SQL_Statement -> (Nothing | Vector SQL_Type) -> Materialized_Table
read_statement self statement expected_types=Nothing =
self.connection.read_statement statement expected_types

## PRIVATE
Internal read function for a statement with optional types returning just last row.

Arguments:
- statement: SQL_Statement to execute.
- expected_types: Optional vector of expected types for each column.
read_last_row : SQL_Statement -> (Nothing | Vector SQL_Type) -> Materialized_Table
read_last_row self statement expected_types=Nothing =
self.connection.read_last_row statement expected_types

## ADVANCED

Executes a raw update query. If the query was inserting, updating or
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@ read_column result_set column_name =

## PRIVATE
Converts a ResultSet into a Materialized_Table.
result_set_to_table : ResultSet -> (Vector | Nothing) -> Materialized_Table
result_set_to_table result_set expected_types=Nothing =
result_set_to_table : ResultSet -> (Vector | Nothing) -> Boolean -> Materialized_Table
result_set_to_table result_set expected_types=Nothing last_row_only=False =
metadata = result_set.getMetaData
ncols = metadata.getColumnCount
column_names = Vector.new ncols ix-> metadata.getColumnName ix+1
Expand All @@ -35,13 +35,25 @@ result_set_to_table result_set expected_types=Nothing =
typeid = metadata.getColumnType ix+1
name = metadata.getColumnTypeName ix+1
SQL_Type.Value typeid name
column_builders = column_types.map typ->
create_builder typ
go has_next = if has_next.not then Nothing else
column_builders.map_with_index ix-> builder->
builder.fetch_and_append result_set ix+1
@Tail_Call go result_set.next
go result_set.next
column_builders = column_types.map create_builder
case last_row_only of
True ->
## Not using the `ResultSet.last` as not supported by all connection types.
go has_next current = case has_next.not of
True ->
if current.is_nothing then Nothing else
column_builders.each_with_index ix-> builder-> builder.append (current.at ix)
Nothing
False ->
values = column_builders.map_with_index ix-> builder-> builder.fetch_value result_set ix+1
@Tail_Call go result_set.next values
go result_set.next Nothing
False ->
go has_next = if has_next.not then Nothing else
column_builders.map_with_index ix-> builder->
builder.fetch_and_append result_set ix+1
@Tail_Call go result_set.next
go result_set.next
columns = column_builders.zip column_names builder-> name->
builder.make_column name
Materialized_Table.new columns
Expand Down Expand Up @@ -105,25 +117,29 @@ type Builder
- i: the index of the column to fetch from (starting from 1 as is the
ResultSet convention).
fetch_and_append : ResultSet -> Integer -> Nothing
fetch_and_append self rs i = case self of
Builder.Builder_Inferred _ ->
obj = rs.getObject i
self.java_builder.append obj
Builder.Builder_Boolean _ ->
bool = rs.getBoolean i
case rs.wasNull of
True -> self.java_builder.appendNulls 1
False -> self.java_builder.appendBoolean bool
Builder.Builder_Long _ ->
long = rs.getLong i
case rs.wasNull of
True -> self.java_builder.appendNulls 1
False -> self.java_builder.appendLong long
Builder.Builder_Double _ ->
double = rs.getDouble i
case rs.wasNull of
True -> self.java_builder.appendNulls 1
False -> self.java_builder.appendDouble double
fetch_and_append self rs i =
value = self.fetch_value rs i
self.append value

## PRIVATE
Fetches the value of ith column from the current row of the result set
fetch_value : ResultSet -> Integer -> Any
fetch_value self rs i =
value = case self of
Builder.Builder_Inferred _ -> rs.getObject i
Builder.Builder_Boolean _ -> rs.getBoolean i
Builder.Builder_Long _ -> rs.getLong i
Builder.Builder_Double _ -> rs.getDouble i
if rs.wasNull then Nothing else value

## PRIVATE
append : Any -> Nothing
append self value = if value.is_nothing then self.java_builder.appendNulls 1 else
case self of
Builder.Builder_Inferred _ -> self.java_builder.append value
Builder.Builder_Boolean _ -> self.java_builder.appendBoolean value
Builder.Builder_Long _ -> self.java_builder.appendLong value
Builder.Builder_Double _ -> self.java_builder.appendDouble value

## PRIVATE

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -108,10 +108,24 @@ type SQLite_Connection

## PRIVATE
Internal read function for a statement with optional types.

Arguments:
- statement: SQL_Statement to execute.
- expected_types: Optional vector of expected types for each column.
read_statement : SQL_Statement -> (Nothing | Vector SQL_Type) -> Materialized_Table
read_statement self statement expected_types=Nothing =
self.connection.read_statement statement expected_types

## PRIVATE
Internal read function for a statement with optional types returning just last row.

Arguments:
- statement: SQL_Statement to execute.
- expected_types: Optional vector of expected types for each column.
read_last_row : SQL_Statement -> (Nothing | Vector SQL_Type) -> Materialized_Table
read_last_row self statement expected_types=Nothing =
self.connection.read_last_row statement expected_types

## ADVANCED

Executes a raw update query. If the query was inserting, updating or
Expand Down
Loading

0 comments on commit 652b8d5

Please sign in to comment.