diff --git a/README.md b/README.md index 3154fb5..1faf462 100644 --- a/README.md +++ b/README.md @@ -53,9 +53,11 @@ The TableauDatasource class uses the `TDEFileGenerator` and/or the `HyperFileGen * 4.8.0 Introduces the RestJsonRequest object and _json plural querying methods for passing JSON responses to other systems * 4.9.0 API 3.3 (2019.1) compatibility, as well as ability to swap in static files using TableauDocument and other bug fixes. * 5.0.0 Python 3.6+ native rewrite. Completely re-organized in the backend, with two models for accessing Rest API methods: TableauRestApiConnection (backwards-compatible) and TableauServerRest, with subclasses grouping the methods. -## --- Table(au) of Contents --- + +--- Table(au) of Contents --- ------ +- [0. Installing and Getting Started](#0-installing-and-getting-started) * [0.0 tableau_tools Library Structure](#00-tableau_tools-library-structure) * [0.1 Importing tableau_tools library](#01-importing-tableau_tools-library) * [0.2 Logger class](#02-logger-class) @@ -1530,7 +1532,7 @@ The dictionary should be a simple mapping of the caption from the template to th ds.columns.translate_captions(english_dict) new_eng_filename = tab_file.save_new_file('English Version') # Reload template again - tab_file = TableauFile('template_file.tds') + tab_file = TableauFileManager('template_file.tds') dses = tab_file.datasources for ds in dses: ds.columns.translate_captions(german_dict) @@ -1541,48 +1543,55 @@ The way that Tableau stores the relationships between the various tables, stored The long and short of it is that the first / left-most table you see in the Data Connections pane in Tableau Desktop is the "main table" which other relations connect to. At the current time , tableau_tools can consistently identify and modify this "main table", which suffices for the vast majority of data source swapping use cases. +There is a TableRelations (note not TableauRelations) object within each TableauDatasource object, representing the table names and their relationships, accesible through the `.tables` property. Any methods for modifying will go through `TableauDatasource.tables` + You can access the main table relationship using -`TableauDatasource.main_table_relation` +`TableauDatasource.tables.main_table` -To determine the type, use: +To see if the datasource is connected to a Stored Procedure, check -`TableauDatasource.main_table_relation.get('type')` +`TableauDatasource.is_stored_procedure: bool` -which should return either `'table', 'stored-proc' or 'text'` (which represents Custom SQL) +like: -##### 2.5.5.1 Database Table Relations -If the type is 'table', then unsurprisingly this relation represents a real table or view in the database. The actual table name in the database is stored in the 'table' attribute, with brackets around the name (regardless of the database type). + tab_file = TableauFileManager('template_file.tds') + dses = tab_file.datasources + for ds in dses: + if ds.is_stored_procedure: + # do stored proc things + +You can also determine the type of the main table, using: -Access it via: -`TableauDatasource.main_table_relation.get('table')` +`TableauDatasource.main_table_type` -Set it via: -`TableauDatasource.main_table_relation.set('table', table_name_in_brackets)` +which should return either `'table', 'stored-proc' or 'custom-sql'` + +##### 2.5.5.1 Database Table Relations +If the type is 'table', then unsurprisingly this relation represents a real table or view in the database. The actual table name in the database is stored in a 'table' attribute, with brackets around the name (regardless of the database type). + +Access and set via property: +`TableauDatasource.tables.main_table_name` Ex. for ds in dses: - if ds.main_table_relation.get('type') == 'table': - if ds.main_table_relation.get('table') == '[Test Table]': - ds.main_table_relation.set('table','[Real Data]') + if ds.main_table_type == 'table': + if ds.tables.main_table_name == '[Test Table]': + ds.tables.main_table_name = '[Real Data]' ##### 2.4.5.2 Custom SQL Relations Custom SQL relations are stored with a type of 'text' (don't ask me, I didn't come up with it). The text of the query is stored as the actual text value of the relation tag in the XML, which is also unusual for the Tableau XML files. -To retrieve the Custom SQL itself, use: - -`TableauDatasource.main_table_relation.text` - -And to set it, use: +To retrieve the Custom SQL itself, use the property to get or set: -`TableauDatasource.main_table_relation.text = new_custom_sql` +`TableauDatasource.tables.main_custom_sql` Ex. for ds in dses: - if ds.main_table_relation.get('type') == 'text': - ds.main_table_relation.text = 'SELECT * FROM my_cool_table' + if ds.is_custom_sql: + ds.tables.main_custom_sql = 'SELECT * FROM my_cool_table' ##### 2.4.5.3 Stored Procedure Relations Stored Procedures are thankfully referred to as 'stored-proc' types in the XML, so they are easy to find. Stored Procedures differ from the other relation types by having parameters for the input values of the Stored Procedure. They also can only connect to that one Stored Procedure (no JOINing of other tables or Custom SQL). This means that a Stored Procedure Data Source only has one relation, the `main_table_relation`. @@ -1590,10 +1599,10 @@ Stored Procedures are thankfully referred to as 'stored-proc' types in the XML, There are actually two ways to set Stored Procedure Parameters in Tableau -- either with Direct Value or linking them to a Tableau Parameter. Currently, tableau_tools allows you to set Direct Values only. To see the current value of a Stored Procedure Parameter, use (remember to search for the exact parameter name. If SQL Server or Sybase, include the @ at the beginning): -`TableauDatasource.get_stored_proc_parameter_value_by_name(parameter_name)` +`TableauDatasource.tables.get_stored_proc_parameter_value_by_name(parameter_name)` To set the value: -`TableauDatasource.set_stored_proc_parameter_value_by_name(parameter_name, parameter_value)` +`TableauDatasource.tables.set_stored_proc_parameter_value_by_name(parameter_name, parameter_value)` For time or datetime values, it is best to pass in a `datetime.date` or `datetime.datetime` variable, but you can also pass in unicode text in the exact format that Tableau's XML uses: @@ -1603,9 +1612,9 @@ date: '#YYYY-MM-DD#' Ex. for ds in dses: - if ds.main_table_relation.get('type') == 'stored-proc': - ds.set_stored_proc_parameter_value_by_name('@StartDate', datetime.date(2018, 1, 1)) - ds.set_stored_proc_parameter_value_by_name('@EndDate', "#2019-01-01#") + if ds.is_stored_proc: + ds.tables.set_stored_proc_parameter_value_by_name('@StartDate', datetime.date(2018, 1, 1)) + ds.tables.set_stored_proc_parameter_value_by_name('@EndDate', "#2019-01-01#") \*\*\*NOTE: From this point on, things become increasingly experimental and less supported. However, I can assure you that many Tableau customers do these very things, and we are constantly working to improve the functionality for making datasources dynamically. diff --git a/examples/tableau_documents_tests.py b/examples/tableau_documents_tests.py index f653f37..2d7c084 100644 --- a/examples/tableau_documents_tests.py +++ b/examples/tableau_documents_tests.py @@ -13,6 +13,7 @@ def live_db_connection_changes(): print(ds) print(ds.connections) ds.ds_name = 'New Datasource Name' + ds.set for conn in ds.connections: conn.connection_name = 'Changed Connection Name' t_file.save_new_file('New TDS') diff --git a/tableau_documents/tablea_relations.py b/tableau_documents/tablea_relations.py new file mode 100644 index 0000000..a79980e --- /dev/null +++ b/tableau_documents/tablea_relations.py @@ -0,0 +1,296 @@ +import xml.etree.ElementTree as ET +from typing import Union, Any, Optional, List, Dict, Tuple +import random +from xml.sax.saxutils import quoteattr, unescape + +from tableau_tools.tableau_exceptions import * + +# This represents the classic Tableau data connection window relations +# Allows for changes in JOINs, Stored Proc values, and Custom SQL +class TableRelations(): + # + # Reading existing table relations + # + def __init__(self, relation_xml_obj: ET.Element): + self.relation_xml_obj = relation_xml_obj + self.main_table: ET.Element + self.table_relations: List[ET.Element] + self.join_relations = [] + self.ns_map = {"user": 'http://www.tableausoftware.com/xml/user', 't': 'http://tableau.com/api'} + ET.register_namespace('t', self.ns_map['t']) + + def _read_existing_relations(self): + # Test for single relation + relation_type = self.relation_xml_obj.get('type') + if relation_type != 'join': + self.main_table_relation = self.relation_xml_obj + self.table_relations = [self.relation_xml_obj, ] + + else: + table_relations = self.relation_xml_obj.findall('.//relation', self.ns_map) + final_table_relations = [] + # ElementTree doesn't implement the != operator, so have to find all then iterate through to exclude + # the JOINs to only get the tables, stored-procs and Custom SQLs + for t in table_relations: + if t.get('type') != 'join': + final_table_relations.append(t) + self.main_table_relation = final_table_relations[0] + self.table_relations = final_table_relations + + # Read any parameters that a stored-proc might have + if self.main_table_relation.get('type') == 'stored-proc': + self._stored_proc_parameters_xml = self.main_table_relation.find('.//actual-parameters') + + @property + def main_custom_sql(self) -> str: + if self.main_table_relation.get('type') == 'stored-proc': + return self.main_table_relation.text + else: + raise InvalidOptionException('Data Source does not have Custom SQL defined') + + @main_custom_sql.setter + def main_custom_sql(self, new_custom_sql: str): + if self.main_table_relation.get('type') == 'stored-proc': + self.main_table_relation.text = new_custom_sql + else: + raise InvalidOptionException('Data Source does not have Custom SQL defined') + + @property + def main_table_name(self) -> str: + if self.main_table_relation.get('type') == 'table': + return self.main_table_relation.get('table') + else: + raise InvalidOptionException('Data Source main relation is not a database table (or view). Possibly Custom SQL or Stored Procedure') + + @main_table_name.setter + def main_table_name(self, new_table_name:str): + if self.main_table_relation.get('type') == 'table': + self.main_table_relation.set('table', new_table_name) + else: + raise InvalidOptionException( + 'Data Source main relation is not a database table (or view). Possibly Custom SQL or Stored Procedure') + + # + # For creating new table relations + # + def set_first_table(self, db_table_name: str, table_alias: str, connection: Optional[str] = None, + extract: bool = False): + self.ds_generator = True + # Grab the original connection name + if self.main_table_relation is not None and connection is None: + connection = self.main_table_relation.get('connection') + self.main_table_relation = self.create_table_relation(db_table_name, table_alias, connection=connection, + extract=extract) + + def set_first_custom_sql(self, custom_sql: str, table_alias: str, connection: Optional[str] = None): + self.ds_generator = True + if self.main_table_relation is not None and connection is None: + connection = self.main_table_relation.get('connection') + self.main_table_relation = self.create_custom_sql_relation(custom_sql, table_alias, connection=connection) + + def set_first_stored_proc(self, stored_proc_name: str, table_alias: str, connection: Optional[str] = None): + self.ds_generator = True + if self.main_table_relation is not None and connection is None: + connection = self.main_table_relation.get('connection') + self.main_table_relation = self.create_stored_proc_relation(stored_proc_name, table_alias, connection=connection) + + def get_stored_proc_parameter_value_by_name(self, parameter_name: str) -> str: + if self._stored_proc_parameters_xml is None: + raise NoResultsException('There are no parameters set for this stored proc (or it is not a stored proc)') + param = self._stored_proc_parameters_xml.find('../column[@name="{}"]'.format(parameter_name)) + if param is None: + raise NoMatchFoundException('Could not find Stored Proc parameter with name {}'.format(parameter_name)) + else: + value = param.get('value') + + # Maybe add deserializing of the dates and datetimes eventally? + + # Remove the quoting and any escaping + if value[0] == '"' and value[-1] == '"': + return unescape(value[1:-1]) + else: + return unescape(value) + + def set_stored_proc_parameter_value_by_name(self, parameter_name: str, parameter_value: str): + # Create if there is none + if self._stored_proc_parameters_xml is None: + self._stored_proc_parameters_xml = ET.Element('actual-parameters') + # Find parameter with that name (if exists) + param = self._stored_proc_parameters_xml.find('.//column[@name="{}"]'.format(parameter_name), self.ns_map) + + if param is None: + # create_stored... already converts to correct quoting + new_param = self.create_stored_proc_parameter(parameter_name, parameter_value) + + self._stored_proc_parameters_xml.append(new_param) + else: + if isinstance(parameter_value, str): + final_val = quoteattr(parameter_value) + elif isinstance(parameter_value, datetime.date) or isinstance(parameter_value, datetime.datetime): + time_str = "#{}#".format(parameter_value.strftime('%Y-%m-%d %H-%M-%S')) + final_val = time_str + else: + final_val = str(parameter_value) + param.set('value', final_val) + + @staticmethod + def create_stored_proc_parameter(parameter_name: str, parameter_value: Any) -> ET.Element: + c = ET.Element('column') + # Check to see if this varies at all depending on type or whatever + c.set('ordinal', '1') + if parameter_name[0] != '@': + parameter_name = "@{}".format(parameter_name) + c.set('name', parameter_name) + if isinstance(parameter_value, str): + c.set('value', quoteattr(parameter_value)) + elif isinstance(parameter_value, datetime.date) or isinstance(parameter_value, datetime.datetime): + time_str = "#{}#".format(parameter_value.strftime('%Y-%m-%d %H-%M-%S')) + c.set('value', time_str) + else: + c.set('value', str(parameter_value)) + return c + + @staticmethod + def create_random_calculation_name() -> str: + n = 19 + range_start = 10 ** (n - 1) + range_end = (10 ** n) - 1 + random_digits = random.randint(range_start, range_end) + return 'Calculation_{}'.format(str(random_digits)) + + @staticmethod + def create_table_relation(db_table_name: str, table_alias: str, connection: Optional[str] = None, + extract: bool = False) -> ET.Element: + r = ET.Element("relation") + r.set('name', table_alias) + if extract is True: + r.set("table", "[Extract].[{}]".format(db_table_name)) + else: + r.set("table", "[{}]".format(db_table_name)) + r.set("type", "table") + if connection is not None: + r.set('connection', connection) + return r + + @staticmethod + def create_custom_sql_relation(custom_sql: str, table_alias: str, connection: Optional[str] = None) -> ET.Element: + r = ET.Element("relation") + r.set('name', table_alias) + r.text = custom_sql + r.set("type", "text") + if connection is not None: + r.set('connection', connection) + return r + + # UNFINISHED, NEEDS TESTING TO COMPLETE + @staticmethod + def create_stored_proc_relation(stored_proc_name: str, connection: Optional[str] = None, actual_parameters=None): + r = ET.Element("relation") + r.set('name', stored_proc_name) + r.set("type", "stored-proc") + if connection is not None: + r.set('connection', connection) + if actual_parameters is not None: + r.append(actual_parameters) + return r + + # on_clauses = [ { left_table_alias : , left_field : , operator : right_table_alias : , right_field : , },] + @staticmethod + def define_join_on_clause(left_table_alias, left_field, operator, right_table_alias, right_field): + return {"left_table_alias": left_table_alias, + "left_field": left_field, + "operator": operator, + "right_table_alias": right_table_alias, + "right_field": right_field + } + + def join_table(self, join_type: str, db_table_name: str, table_alias: str, join_on_clauses: List[Dict], + custom_sql: Optional[str] = None): + full_join_desc = {"join_type": join_type.lower(), + "db_table_name": db_table_name, + "table_alias": table_alias, + "on_clauses": join_on_clauses, + "custom_sql": custom_sql} + self.join_relations.append(full_join_desc) + + def generate_relation_section(self, connection_name: Optional[str] = None) -> ET.Element: + # Because of the strange way that the interior definition is the last on, you need to work inside out + # "Middle-out" as Silicon Valley suggests. + # Generate the actual JOINs + #if self.relation_xml_obj is not None: + # self.relation_xml_obj.clear() + #else: + rel_xml_obj = ET.Element("relation") + # There's only a single main relation with only one table + + if len(self.join_relations) == 0: + for item in list(self.main_table_relation.items()): + rel_xml_obj.set(item[0], item[1]) + if self.main_table_relation.text is not None: + rel_xml_obj.text = self.main_table_relation.text + + else: + prev_relation = None + + # We go through each relation, build the whole thing, then append it to the previous relation, then make + # that the new prev_relationship. Something like recursion + #print(self.join_relations) + for join_desc in self.join_relations: + + r = ET.Element("relation") + r.set("join", join_desc["join_type"]) + r.set("type", "join") + if len(join_desc["on_clauses"]) == 0: + raise InvalidOptionException("Join clause must have at least one ON clause describing relation") + else: + and_expression = None + if len(join_desc["on_clauses"]) > 1: + and_expression = ET.Element("expression") + and_expression.set("op", 'AND') + for on_clause in join_desc["on_clauses"]: + c = ET.Element("clause") + c.set("type", "join") + e = ET.Element("expression") + e.set("op", on_clause["operator"]) + + e_field1 = ET.Element("expression") + e_field1_name = '[{}].[{}]'.format(on_clause["left_table_alias"], + on_clause["left_field"]) + e_field1.set("op", e_field1_name) + e.append(e_field1) + + e_field2 = ET.Element("expression") + e_field2_name = '[{}].[{}]'.format(on_clause["right_table_alias"], + on_clause["right_field"]) + e_field2.set("op", e_field2_name) + e.append(e_field2) + if and_expression is not None: + and_expression.append(e) + else: + and_expression = e + c.append(and_expression) + r.append(c) + if prev_relation is not None: + r.append(prev_relation) + + if join_desc["custom_sql"] is None: + # Append the main table first (not sure this works for more deep hierarchies, but let's see + main_rel_xml_obj = ET.Element('relation') + for item in list(self.main_table_relation.items()): + main_rel_xml_obj.set(item[0], item[1]) + if self.main_table_relation.text is not None: + main_rel_xml_obj.text = self.main_table_relation.text + main_rel_xml_obj.set('connection', connection_name) + r.append(main_rel_xml_obj) + + new_table_rel = self.create_table_relation(join_desc["db_table_name"], + join_desc["table_alias"], connection=connection_name) + else: + new_table_rel = self.create_custom_sql_relation(join_desc['custom_sql'], + join_desc['table_alias'], connection=connection_name) + r.append(new_table_rel) + prev_relation = r + #prev_relation = copy.deepcopy(r) + #rel_xml_obj.append(copy.deepcopy(r)) + rel_xml_obj = copy.deepcopy(prev_relation) + return rel_xml_obj \ No newline at end of file diff --git a/tableau_documents/tableau_datasource.py b/tableau_documents/tableau_datasource.py index 7503b70..f0a65b5 100644 --- a/tableau_documents/tableau_datasource.py +++ b/tableau_documents/tableau_datasource.py @@ -16,6 +16,7 @@ from tableau_documents.tableau_connection import TableauConnection from tableau_documents.tableau_document import TableauDocument from tableau_documents.tableau_columns import TableauColumns +from tableau_documents.tablea_relations import TableRelations # Meant to represent a TDS file, does not handle the file opening @@ -49,7 +50,7 @@ def __init__(self, datasource_xml: Optional[ET.Element] = None, logger_obj: Opti self.column_instances = [] self.main_table_relation = None self.main_table_name = None - self.table_relations = None + self._table_relations = None self._connection_root = None self._stored_proc_parameters_xml = None @@ -171,6 +172,7 @@ def __init__(self, datasource_xml: Optional[ET.Element] = None, logger_obj: Opti # Skip the relation if it is a Parameters datasource. Eventually, build out separate object if self.xml.get('name') != 'Parameters': self.relation_xml_obj = self.xml.find('.//relation', self.ns_map) + self._table_relations = TableRelations(relation_xml_obj=self.relation_xml_obj) self._read_existing_relations() else: self.log('Found a Parameters datasource') @@ -227,6 +229,27 @@ def columns(self) -> TableauColumns: def is_published(self) -> bool: return self._published + @property + def tables(self): + return self._table_relations + + @property + def is_stored_proc(self) -> bool: + if self.tables.main_table.get('type') == 'stored-proc': + return True + else: + return False + + @property + def main_table_type(self) -> str: + raw_type = self.tables.main_table.get('type') + if raw_type == 'stored-proc': + return 'stored-proc' + elif raw_type == 'text': + return 'custom-sql' + elif raw_type == 'table': + return 'table' + @property def published_ds_site(self) -> str: if self.repository_location.get("site"): @@ -539,255 +562,7 @@ def generate_extract_section(self) -> Union[ET.Element, bool]: raise InvalidOptionException('tableau_tools 5.0 is Python 3.6+ compatible, but the TDE generating library is only available for Python 2.7. Please upgrade to a newer version of Tableau with Hyper extract engine (10.5+)') return e - # - # Reading existing table relations - # - def _read_existing_relations(self): - # Test for single relation - relation_type = self.relation_xml_obj.get('type') - if relation_type != 'join': - self.main_table_relation = self.relation_xml_obj - self.table_relations = [self.relation_xml_obj, ] - - else: - table_relations = self.relation_xml_obj.findall('.//relation', self.ns_map) - final_table_relations = [] - # ElementTree doesn't implement the != operator, so have to find all then iterate through to exclude - # the JOINs to only get the tables, stored-procs and Custom SQLs - for t in table_relations: - if t.get('type') != 'join': - final_table_relations.append(t) - self.main_table_relation = final_table_relations[0] - self.table_relations = final_table_relations - - # Read any parameters that a stored-proc might have - if self.main_table_relation.get('type') == 'stored-proc': - self._stored_proc_parameters_xml = self.main_table_relation.find('.//actual-parameters') - - # - # For creating new table relations - # - def set_first_table(self, db_table_name: str, table_alias: str, connection: Optional[str] = None, - extract: bool = False): - self.ds_generator = True - # Grab the original connection name - if self.main_table_relation is not None and connection is None: - connection = self.main_table_relation.get('connection') - self.main_table_relation = self.create_table_relation(db_table_name, table_alias, connection=connection, - extract=extract) - - def set_first_custom_sql(self, custom_sql: str, table_alias: str, connection: Optional[str] = None): - self.ds_generator = True - if self.main_table_relation is not None and connection is None: - connection = self.main_table_relation.get('connection') - self.main_table_relation = self.create_custom_sql_relation(custom_sql, table_alias, connection=connection) - - def set_first_stored_proc(self, stored_proc_name: str, table_alias: str, connection: Optional[str] = None): - self.ds_generator = True - if self.main_table_relation is not None and connection is None: - connection = self.main_table_relation.get('connection') - self.main_table_relation = self.create_stored_proc_relation(stored_proc_name, table_alias, connection=connection) - - def get_stored_proc_parameter_value_by_name(self, parameter_name: str) -> str: - if self._stored_proc_parameters_xml is None: - raise NoResultsException('There are no parameters set for this stored proc (or it is not a stored proc)') - param = self._stored_proc_parameters_xml.find('../column[@name="{}"]'.format(parameter_name)) - if param is None: - raise NoMatchFoundException('Could not find Stored Proc parameter with name {}'.format(parameter_name)) - else: - value = param.get('value') - - # Maybe add deserializing of the dates and datetimes eventally? - - # Remove the quoting and any escaping - if value[0] == '"' and value[-1] == '"': - return unescape(value[1:-1]) - else: - return unescape(value) - - def set_stored_proc_parameter_value_by_name(self, parameter_name: str, parameter_value: str): - # Create if there is none - if self._stored_proc_parameters_xml is None: - self._stored_proc_parameters_xml = ET.Element('actual-parameters') - # Find parameter with that name (if exists) - param = self._stored_proc_parameters_xml.find('.//column[@name="{}"]'.format(parameter_name), self.ns_map) - - if param is None: - # create_stored... already converts to correct quoting - new_param = self.create_stored_proc_parameter(parameter_name, parameter_value) - - self._stored_proc_parameters_xml.append(new_param) - else: - if isinstance(parameter_value, str): - final_val = quoteattr(parameter_value) - elif isinstance(parameter_value, datetime.date) or isinstance(parameter_value, datetime.datetime): - time_str = "#{}#".format(parameter_value.strftime('%Y-%m-%d %H-%M-%S')) - final_val = time_str - else: - final_val = str(parameter_value) - param.set('value', final_val) - - @staticmethod - def create_stored_proc_parameter(parameter_name: str, parameter_value: Any) -> ET.Element: - c = ET.Element('column') - # Check to see if this varies at all depending on type or whatever - c.set('ordinal', '1') - if parameter_name[0] != '@': - parameter_name = "@{}".format(parameter_name) - c.set('name', parameter_name) - if isinstance(parameter_value, str): - c.set('value', quoteattr(parameter_value)) - elif isinstance(parameter_value, datetime.date) or isinstance(parameter_value, datetime.datetime): - time_str = "#{}#".format(parameter_value.strftime('%Y-%m-%d %H-%M-%S')) - c.set('value', time_str) - else: - c.set('value', str(parameter_value)) - return c - - @staticmethod - def create_random_calculation_name() -> str: - n = 19 - range_start = 10 ** (n - 1) - range_end = (10 ** n) - 1 - random_digits = random.randint(range_start, range_end) - return 'Calculation_{}'.format(str(random_digits)) - - @staticmethod - def create_table_relation(db_table_name: str, table_alias: str, connection: Optional[str] = None, - extract: bool = False) -> ET.Element: - r = ET.Element("relation") - r.set('name', table_alias) - if extract is True: - r.set("table", "[Extract].[{}]".format(db_table_name)) - else: - r.set("table", "[{}]".format(db_table_name)) - r.set("type", "table") - if connection is not None: - r.set('connection', connection) - return r - - @staticmethod - def create_custom_sql_relation(custom_sql: str, table_alias: str, connection: Optional[str] = None) -> ET.Element: - r = ET.Element("relation") - r.set('name', table_alias) - r.text = custom_sql - r.set("type", "text") - if connection is not None: - r.set('connection', connection) - return r - - # UNFINISHED, NEEDS TESTING TO COMPLETE - @staticmethod - def create_stored_proc_relation(stored_proc_name: str, connection: Optional[str] = None, actual_parameters=None): - r = ET.Element("relation") - r.set('name', stored_proc_name) - r.set("type", "stored-proc") - if connection is not None: - r.set('connection', connection) - if actual_parameters is not None: - r.append(actual_parameters) - return r - - # on_clauses = [ { left_table_alias : , left_field : , operator : right_table_alias : , right_field : , },] - @staticmethod - def define_join_on_clause(left_table_alias, left_field, operator, right_table_alias, right_field): - return {"left_table_alias": left_table_alias, - "left_field": left_field, - "operator": operator, - "right_table_alias": right_table_alias, - "right_field": right_field - } - def join_table(self, join_type: str, db_table_name: str, table_alias: str, join_on_clauses: List[Dict], - custom_sql: Optional[str] = None): - full_join_desc = {"join_type": join_type.lower(), - "db_table_name": db_table_name, - "table_alias": table_alias, - "on_clauses": join_on_clauses, - "custom_sql": custom_sql} - self.join_relations.append(full_join_desc) - - def generate_relation_section(self, connection_name: Optional[str] = None) -> ET.Element: - # Because of the strange way that the interior definition is the last on, you need to work inside out - # "Middle-out" as Silicon Valley suggests. - # Generate the actual JOINs - #if self.relation_xml_obj is not None: - # self.relation_xml_obj.clear() - #else: - rel_xml_obj = ET.Element("relation") - # There's only a single main relation with only one table - - if len(self.join_relations) == 0: - for item in list(self.main_table_relation.items()): - rel_xml_obj.set(item[0], item[1]) - if self.main_table_relation.text is not None: - rel_xml_obj.text = self.main_table_relation.text - - else: - prev_relation = None - - # We go through each relation, build the whole thing, then append it to the previous relation, then make - # that the new prev_relationship. Something like recursion - #print(self.join_relations) - for join_desc in self.join_relations: - - r = ET.Element("relation") - r.set("join", join_desc["join_type"]) - r.set("type", "join") - if len(join_desc["on_clauses"]) == 0: - raise InvalidOptionException("Join clause must have at least one ON clause describing relation") - else: - and_expression = None - if len(join_desc["on_clauses"]) > 1: - and_expression = ET.Element("expression") - and_expression.set("op", 'AND') - for on_clause in join_desc["on_clauses"]: - c = ET.Element("clause") - c.set("type", "join") - e = ET.Element("expression") - e.set("op", on_clause["operator"]) - - e_field1 = ET.Element("expression") - e_field1_name = '[{}].[{}]'.format(on_clause["left_table_alias"], - on_clause["left_field"]) - e_field1.set("op", e_field1_name) - e.append(e_field1) - - e_field2 = ET.Element("expression") - e_field2_name = '[{}].[{}]'.format(on_clause["right_table_alias"], - on_clause["right_field"]) - e_field2.set("op", e_field2_name) - e.append(e_field2) - if and_expression is not None: - and_expression.append(e) - else: - and_expression = e - c.append(and_expression) - r.append(c) - if prev_relation is not None: - r.append(prev_relation) - - if join_desc["custom_sql"] is None: - # Append the main table first (not sure this works for more deep hierarchies, but let's see - main_rel_xml_obj = ET.Element('relation') - for item in list(self.main_table_relation.items()): - main_rel_xml_obj.set(item[0], item[1]) - if self.main_table_relation.text is not None: - main_rel_xml_obj.text = self.main_table_relation.text - main_rel_xml_obj.set('connection', connection_name) - r.append(main_rel_xml_obj) - - new_table_rel = self.create_table_relation(join_desc["db_table_name"], - join_desc["table_alias"], connection=connection_name) - else: - new_table_rel = self.create_custom_sql_relation(join_desc['custom_sql'], - join_desc['table_alias'], connection=connection_name) - r.append(new_table_rel) - prev_relation = r - #prev_relation = copy.deepcopy(r) - #rel_xml_obj.append(copy.deepcopy(r)) - rel_xml_obj = copy.deepcopy(prev_relation) - return rel_xml_obj def add_table_column(self, table_alias: str, table_field_name: str, tableau_field_alias: str): # Check to make sure the alias has been added