diff --git a/README.md b/README.md index 066cb3d..4eeac0a 100644 --- a/README.md +++ b/README.md @@ -50,6 +50,7 @@ The TableauDatasource class uses the `TDEFileGenerator` and/or the `HyperFileGen * 4.4.0: A lot of improvements to the tableau_documents library and its documentation * 4.5.0: 2018.1 (API 3.0) compatibility. All requests for a given connection using a single HTTP session, and other improvements to the RestXmlRequest class. * 4.7.0: Dropping API 2.0 (Tableau 9.0) compatibility. Any method that is overwritten in a later version will not be updated in the TableauRestApiConnection class going forward. Also implemented a direct_xml_request parameter for Add and Update methods, allowing direct submission of an ElementTree.Element request to the endpoints, particularly for replication. +* 4.8.0 Introduces the RestJsonRequest object and _json plural querying methods for passing JSON responses to other systems ## --- Table(au) of Contents --- ------ @@ -116,6 +117,7 @@ The TableauDatasource class uses the `TDEFileGenerator` and/or the `HyperFileGen * [2.12 Creating a TableauDatasource from Scratch](#212-creating-a-tableaudatasource-from-scratch) * [2.13 Creating and Modifying Parameters](#213-creating-and-modifying-parameters) + [2.13.1 TableauParameter class](#2131-tableauparameter-class) + * [2.14 HyperFileGenerator and TDEFileGenerator Classes](#214-) - [3 tabcmd](#3-tabcmd) * [3.1 Tabcmd Class](#31-tabcmd-class) * [3.2 Triggering an Extract Refresh](#32-triggering-an-extract-refresh) @@ -133,6 +135,7 @@ tableau_tools * permissions * published_content (Project, Workbook, Datasource) * rest_xml_request + * rest_json_request * sort * tableau_rest_api_server_connection * tableau_rest_api_server_connection21 @@ -143,6 +146,9 @@ tableau_tools * tableau_rest_api_server_connection26 * tableau_rest_api_server_connection27 * tableau_rest_api_server_connection28 + * tableau_rest_api_server_connection30 + * tableau_rest_api_server_connection31 + * tableau_rest_api_server_connection32 * url_filter * tableau_documents * tableau_connection @@ -225,6 +231,10 @@ tableau_tools 4.0+ implements the different versions of the Tableau Server REST `TableauRestApiConnection30: 2018.1` +`TableauRestApiConnection31: 2018.2` + +`TableauRestApiConnection32: 2018.3` + You need to initialize at least one object of this class. Ex.: @@ -275,7 +285,7 @@ The simplest method for getting information from the REST API are the "plural" q `TableauRestApiConnection.query_groups()` -`TableauRestApiConnection.query_users()` +`TableauRestApiConnection.query_users(username_or_luid)` `TableauRestApiConnection.query_workbooks()` @@ -283,6 +293,9 @@ The simplest method for getting information from the REST API are the "plural" q `TableauRestApiConnection.query_datasources()` +`TableauRestApiConnection.query_workbook_views()` + + These will all return an ElementTree object representing the results from the REST API call. This can be useful if you need all of the information returned, but most of your calls to these methods will be to get a dictionary of names : luids you can use for lookup. There is a simple static method for this conversion `TableauRestApiConnection.convert_xml_list_to_name_id_dict(xml_obj)` @@ -297,6 +310,25 @@ Ex. for group_name in groups_dict: print "Group name {} is LUID {}".format(group_name, groups_dict[group_name]) +There are also equivalent JSON querying methods for the plural methods, which return a Python json object, using the Tableau REST API's native method for requesting JSON responses rather than XML. + +The JSON plural query methods allow you to specify the page of results using the page= optional parameter(starting at 1). If you do not specify a page, tableau_tools will automatically paginate through all results and combine them together. + + +`TableauRestApiConnection.query_groups_json(page_number=None)` + +`TableauRestApiConnection.query_users_json(page_number=None)` + +`TableauRestApiConnection.query_workbooks_json(username_or_luid, page_number=None)` + +`TableauRestApiConnection.query_projects_json(page_number=None)` + +`TableauRestApiConnection.query_datasources_json(page_number=None)` + +`TableauRestApiConnection.query_workbook_views_json(page_number=None)` + + + ##### 1.2.2.1 Filtering and Sorting (Tableau Server 9.3+) `TableauRestApiConnection22` implements filtering and sorting for the methods where it is allowed. Singular lookup methods are programmed to take advantage of this automatically for improved perofrmance, but the plural querying methods can use the filters to bring back specific sets. @@ -1522,6 +1554,31 @@ Ex. param.set_allowable_values_to_list(allowable_values) param.set_current_value(u'Spring 2018') +### 2.14 HyperFileGenerator and TDEFileGenerator Classes +The "add extract" functionality in tableau_tools uses the Extract API/Tableau SDK (they are the same thing, the names changed back and forth over time). The HyperFileGenerator and TDEFileGenerator classes are replicas of one another, but HyperFileGenerator uses the Extract API 2.0, which is capable of creating Hyper files. + +You can use them in conjunction with PyODBC (install via pip) to create extracts very simply from a query to an ODBC connection. In essence, the classes will map any of the PyODBC data types to the Extract API data types automatically when you provide a PyODBC cursor from an executed query. + +`HyperFileGenerator(logger_obj)` + +There are two steps to creating an extract. You first must create a Table Definition, then you insert the rows of data. + +The most basic way to set a Table Definition is defining a dict in the form of {'column_name' : 'data_type'}. + +`HyperFileGenerator.set_table_definition(column_name_type_dict, collation=Collation.EN_US)` + +However, you can use the a pyodbc cursor to the same effect, which basically lets you just write a query and pass everything through directly: + +`HyperFileGenerator.create_table_definition_from_pyodbc_cursor(pydobc_cursor, collation=Collation.EN_US)` + +This will return the TableDefinition object from the Extract API, but it also sets the internal table_definition for the particular instance of the HyperFileGenerator object so you don't need to do anything other than run this method and anything afterward you do will take the current TableDefinition. + +To generate the extract: +`HyperFileGenerator.create_extract(tde_filename, append=False, table_name=u'Extract', pyodbc_cursor=None)` + +You do need to specify an actual filename for it to write to, because the Extract API always works on a file on disk. You can specify multiple tables within this file by giving different table names, and you can even append by specifying append=True while using the same table_name that previously has been created within the file. The pyodbc_cursor= optional parameter will run through all of the rows from the cursor and add them to the Extract. + +At the current time, the only exposed method to add data to the extract is the pyodbc cursor. ## 3 tabcmd The Tableau Server REST API can do most of the things that the tabcmd command line tool can, but if you are using older versions of Tableau Server, some of those features may not have been implemented yet. If you need a functionality from tabcmd, the `tabcmd.py` file in the main part of tableau_tools library wraps most of the commonly used functionality to allow for easier scripting of calls (rather than doing it directly on the command line or using batch files) diff --git a/examples/archive_site.py b/examples/archive_site.py new file mode 100644 index 0000000..1813064 --- /dev/null +++ b/examples/archive_site.py @@ -0,0 +1,44 @@ +# -*- coding: utf-8 -*- + +from tableau_tools.tableau_rest_api import * +from tableau_tools import * +import os + + +def archive_tableau_site(save_to_directory, server, username, password, site_content_url): + # The last two digits of this constructor match to the version of API available on the Tableau Server + t = TableauRestApiConnection30(server=server, username=username, + password=password, site_content_url=site_content_url) + t.signin() + all_projects = t.query_projects() + all_projects_dict = t.convert_xml_list_to_name_id_dict(all_projects) + + # This gives you the Project name; the values of the dict are the LUIDs + for project in all_projects_dict: + # Create directory for projects + try: + print(u'Making directory {}'.format(project)) + os.mkdir(u'{}/{}'.format(save_to_directory, project)) + except OSError as e: + print(u'Directory already exists') + + print(u'Downloading datasources for project {}'.format(project)) + # Get All Data sources + dses_in_project = t.query_datasources(project_name_or_luid=all_projects_dict[project]) + for ds in dses_in_project: + ds_luid = ds.get(u'id') + ds_content_url = ds.get(u'contentUrl') + print(u'Downloading datasource {}'.format(ds_content_url)) + t.download_datasource(ds_name_or_luid=ds_luid, + filename_no_extension=u"{}/{}/{}".format(save_to_directory, project, ds_content_url), + include_extract=False) + + print(u'Downloading workbooks for project {}'.format(project)) + wbs_in_project = t.query_workbooks_in_project(project_name_or_luid=all_projects_dict[project]) + for wb in wbs_in_project: + wb_luid = wb.get(u'id') + wb_content_url = wb.get(u'contentUrl') + print(u'Downloading workbook {}'.format(wb_content_url)) + t.download_workbook(wb_name_or_luid=wb_luid, + filename_no_extension=u"{}/{}/{}".format(save_to_directory, project, wb_content_url), + include_extract=False) \ No newline at end of file diff --git a/examples/create_site_sample.py b/examples/create_site_sample.py index 1decbf0..d5f1fc0 100644 --- a/examples/create_site_sample.py +++ b/examples/create_site_sample.py @@ -12,19 +12,20 @@ new_site_name = u'Sample Test Site' # Choose the API version for your server. 10.2 = 25 -default = TableauRestApiConnection26(server, username, password) +default = TableauRestApiConnection26(server=server, username=username, password=password, site_content_url=u'default') try: default.signin() - default.create_site(new_site_name, new_site_content_url) + default.create_site(new_site_name=new_site_name, new_content_url=new_site_content_url) except AlreadyExistsException as e: - print e.msg - print u"Cannot create new site, it already exists" + print(e.msg) + print(u"Cannot create new site, it already exists") exit() time.sleep(4) -t = TableauRestApiConnection26(server, username, password, new_site_content_url) +t = TableauRestApiConnection26(server=server, username=username, password=password, + site_content_url=new_site_content_url) t.signin() logger = Logger(u'create_site_sample.log') # Enable logging after sign-in to hide credentials @@ -34,45 +35,49 @@ groups_to_create = [u'Administrators', u'Executives', u'Managers', u'Line Level Workers'] for group in groups_to_create: - t.create_group(group) + t.create_group(group_name=group) # Remove all permissions from Default Project time.sleep(4) -default_proj = t.query_project(u'Default') +default_proj = t.query_project(project_name_or_luid=u'Default') default_proj.lock_permissions() default_proj.clear_all_permissions() # This clears all, including the defaults # Add in any default permissions you'd like at this point -admin_perms = default_proj.create_project_permissions_object_for_group(u'Administrators', role=u'Project Leader') +admin_perms = default_proj.create_project_permissions_object_for_group(group_name_or_luid=u'Administrators', + role=u'Project Leader') default_proj.set_permissions_by_permissions_obj_list([admin_perms, ]) -admin_perms = default_proj.create_workbook_permissions_object_for_group(u'Administrators', role=u'Editor') -admin_perms.set_capability(u'Download Full Data', u'Deny') +admin_perms = default_proj.create_workbook_permissions_object_for_group(group_name_or_luid=u'Administrators', + role=u'Editor') +admin_perms.set_capability(capability_name=u'Download Full Data', mode=u'Deny') default_proj.workbook_defaults.set_permissions_by_permissions_obj_list([admin_perms, ]) -admin_perms = default_proj.create_datasource_permissions_object_for_group(u'Administrators', role=u'Editor') +admin_perms = default_proj.create_datasource_permissions_object_for_group(group_name_or_luid=u'Administrators', + role=u'Editor') default_proj.datasource_defaults.set_permissions_by_permissions_obj_list([admin_perms, ]) # Change one of these -new_perms = default_proj.create_project_permissions_object_for_group(u'Administrators', role=u'Publisher') +new_perms = default_proj.create_project_permissions_object_for_group(group_name_or_luid=u'Administrators', + role=u'Publisher') default_proj.set_permissions_by_permissions_obj_list([new_perms, ]) # Create Additional Projects projects_to_create = [u'Sandbox', u'Data Source Definitions', u'UAT', u'Finance', u'Real Financials'] for project in projects_to_create: - t.create_project(project, no_return=True) + t.create_project(project_name=project, no_return=True) # Set any additional permissions on each project # Add Users users_to_add = [u'user_1', u'user_2', u'user_3'] for user in users_to_add: - t.add_user(user, user, u'Publisher') + t.add_user(username=user, fullname=user, site_role=u'Publisher') time.sleep(3) # Add Users to Groups -t.add_users_to_group(u'user_1', u'Managers') -t.add_users_to_group(u'user_2', u'Administrators') -t.add_users_to_group([u'user_2', u'user_3'], u'Executives') +t.add_users_to_group(username_or_luid_s=u'user_1', group_name_or_luid=u'Managers') +t.add_users_to_group(username_or_luid_s=u'user_2', group_name_or_luid=u'Administrators') +t.add_users_to_group(username_or_luid_s=[u'user_2', u'user_3'], group_name_or_luid=u'Executives') diff --git a/examples/extract_refresh_pre_10_3_sample.py b/examples/extract_refresh_pre_10_3_sample.py index c372b74..79f3b73 100644 --- a/examples/extract_refresh_pre_10_3_sample.py +++ b/examples/extract_refresh_pre_10_3_sample.py @@ -5,6 +5,7 @@ # This script uses the tabcmd object to trigger an extract refresh # Please use the 10_3 script if on a 10.3+ version of Tableau Server, which uses the REST API commands +# Should be totally deprecated at this point logger = Logger(u'extract_refresh.log') diff --git a/examples/move_extracts_from_server_to_server.py b/examples/move_extracts_from_server_to_server.py index 2f7a692..13ddae5 100644 --- a/examples/move_extracts_from_server_to_server.py +++ b/examples/move_extracts_from_server_to_server.py @@ -4,6 +4,12 @@ from tableau_tools import * import time +# This example is for a specific use case: +# When the Originating Tableau Server can connect to the live data source to generate the new extract, but the +# Destination Tableau Server cannot. This particular pattern downloads the updated TDSX file from the Originating Server +# then pushes it to the Destination Server without saving credentials. With no credentials (and no Extract Refresh +# scheduled), the version on the Destination Server will remain static until another version is pushed + o_server = u'http://' o_username = u'' o_password = u'' @@ -16,7 +22,7 @@ d_password = u'' d_site_content_url = u'' -t = TableauRestApiConnection26(o_server, o_username, o_password, o_site_content_url) +t = TableauRestApiConnection28(o_server, o_username, o_password, o_site_content_url) t.signin() t.enable_logging(logger) downloaded_filename = u'File Name' @@ -24,7 +30,7 @@ proj_name = u'Default' t.download_workbook(u'WB Name on Server', downloaded_filename, proj_name_or_luid=proj_name) -d = TableauRestApiConnection25(d_server, d_username, d_password, d_site_content_url) +d = TableauRestApiConnection28(d_server, d_username, d_password, d_site_content_url) d.signin() d.enable_logging(logger) proj = d.query_project(u'Default') diff --git a/examples/permissions_auditing.py b/examples/permissions_auditing.py index a5aba00..c0c44e8 100644 --- a/examples/permissions_auditing.py +++ b/examples/permissions_auditing.py @@ -3,6 +3,8 @@ from tableau_tools import * import csv +# There are differences between Python 2.7 and Python 3, so this may not run on Python 3 yet + username = '' password = '' server = 'http://localhost' @@ -36,7 +38,7 @@ output_writer.writerow(headers) for site_content_url in site_content_urls: - t = TableauRestApiConnection25(server, username, password, site_content_url) + t = TableauRestApiConnection28(server, username, password, site_content_url) t.enable_logging(logger) t.signin() projects = t.query_projects() diff --git a/examples/permissions_changing.py b/examples/permissions_changing.py index 1ae312d..9e27280 100644 --- a/examples/permissions_changing.py +++ b/examples/permissions_changing.py @@ -3,32 +3,6 @@ from tableau_tools.tableau_rest_api import * from tableau_tools import * -capabilities_to_set = {u"Download Full Data": u"Deny"} -tableau_group_name = u'All Users' - -server = u'http://' -username = u'username' -password = u'secure_password' -site = u'a_site' - -logger = Logger(u'Permissions.log') - -t = TableauRestApiConnection25(server, username, password, site) -t.signin() -t.enable_logging(logger) - -projects = t.query_projects() -projects_dict = t.convert_xml_list_to_name_id_dict(projects) - - -# Determine the identifer (LUID) of the Group -try: - all_users_group_luid = t.query_group_luid(tableau_group_name) -except NoMatchFoundException: - print(u"No group found using the name provided") - exit() - - def update_workbook_permissions(project_obj, published_workbook_object, group_luid, capabilities_dict): """ :type project_obj: Project @@ -63,33 +37,68 @@ def update_workbook_permissions(project_obj, published_workbook_object, group_lu print(u'No permissions found for group, adding new permissions') published_workbook_object.set_permissions_by_permissions_obj_list([new_perm_obj, ]) +capabilities_to_set = {u"Download Full Data": u"Deny"} +tableau_group_name = u'All Users' + +server = u'http://' +username = u'username' +password = u'secure_password' +site = u'a_site' + +logger = Logger(u'Permissions.log') + +t = TableauRestApiConnection28(server=server, username=username, password=password, site_content_url=site) +t.signin() +t.enable_logging(logger) + +projects = t.query_projects() +projects_dict = t.convert_xml_list_to_name_id_dict(projects) + + +# Determine the identifer (LUID) of the Group +try: + all_users_group_luid = t.query_group_luid(group_name=tableau_group_name) + for project in projects_dict: + # List of projects we want to search in. Uncomment if you want to limit which projects are affected: + # projects_to_change = [u'Project A', u'Project C'] + # if project not in projects_to_change: + # continue + + # Update the workbook_defaults in the project + + # Get the project as an object so permissions are available + try: + project_object = t.query_project(project_name_or_luid=project) + workbook_defaults_obj = project_object.workbook_defaults + print(u"Updating the Project's Workbook Defaults") + update_workbook_permissions(project_obj=project_object, published_workbook_object=workbook_defaults_obj, + group_luid=all_users_group_luid, capabilities_dict=capabilities_to_set) + + # Update the workbooks themselves (if the permissions aren't locked, because this would be a waste of time) + if project_object.are_permissions_locked() is False: + wbs_in_project = t.query_workbooks_in_project(project_name_or_luid=project) + wbs_dict = t.convert_xml_list_to_name_id_dict(wbs_in_project) + for wb in wbs_dict: + # Second parameter project_name is unecessary when passing a LUID + # That is why you reference wbs_dict[wb], rather than wb directly, which is just the name + wb_obj = t.get_published_workbook_object(workbook_name_or_luid=wbs_dict[wb], + project_name_or_luid=u"") + print(u'Updating workbook with LUID {}'.format(wbs_dict[wb])) + update_workbook_permissions(project_obj=project_object, published_workbook_object=wb_obj, + group_luid=all_users_group_luid, capabilities_dict=capabilities_to_set) + + except NoMatchFoundException: + print(u"No project found with the given name, check the log") + exit() + +except NoMatchFoundException: + print(u"No group found using the name provided") + exit() + + + + + + + -for project in projects_dict: - # List of projects we want to search in. Uncomment if you want to limit which projects are affected: - # projects_to_change = [u'Project A', u'Project C'] - # if project not in projects_to_change: - # continue - - # Update the workbook_defaults in the project - - # Get the project as an object so permissions are available - try: - project_object = t.query_project(project) - except NoMatchFoundException: - print(u"No project found with the given name, check the log") - exit() - - workbook_defaults_obj = project_object.workbook_defaults - print(u"Updating the Project's Workbook Defaults") - update_workbook_permissions(project_object, workbook_defaults_obj, all_users_group_luid, capabilities_to_set) - - # Update the workbooks themselves (if the permissions aren't locked, because this would be a waste of time) - if project_object.are_permissions_locked() is False: - wbs_in_project = t.query_workbooks_in_project(project) - wbs_dict = t.convert_xml_list_to_name_id_dict(wbs_in_project) - for wb in wbs_dict: - # Second parameter project_name is unecessary when passing a LUID - # That is why you reference wbs_dict[wb], rather than wb directly, which is just the name - wb_obj = t.get_published_workbook_object(wbs_dict[wb], u"") - print(u'Updating workbook with LUID {}'.format(wbs_dict[wb])) - update_workbook_permissions(project_object, wb_obj, all_users_group_luid, capabilities_to_set) diff --git a/examples/template_publish_sample.py b/examples/template_publish_sample.py index ea1ee2a..c4d8a21 100644 --- a/examples/template_publish_sample.py +++ b/examples/template_publish_sample.py @@ -74,7 +74,7 @@ def promote_from_dev_to_test(logger_obj=None): new_project = test.query_project(u'Promoted Content') test.publish_datasource(u'Temp TDSX.tdsx', ds, new_project, overwrite=True, save_credentials=True) # If you have credentials to publish - # test.publish_datasource(temp_filename, ds, new_project, connection_username=u'', connection_password=u'', overwrite=True, save_credentials=True) + #test.publish_datasource(temp_filename, ds, new_project, connection_username=u'', connection_password=u'', overwrite=True, save_credentials=True) os.remove(u'Temp TDSX.tdsx') # promote_from_dev_to_test(logger) diff --git a/examples/user_sync_sample.py b/examples/user_sync_sample.py index da3945b..0de0bbe 100644 --- a/examples/user_sync_sample.py +++ b/examples/user_sync_sample.py @@ -3,6 +3,11 @@ from tableau_rest_api.tableau_rest_api_connection import * +# This is example code showing a sync process from a database with users in it +# It uses psycopg2 to connect to a PostgreSQL database +# You can substitute in any source of the usernames and groups +# The essential logic is that you do a full comparison on who should exist and who doesn't, and both add and remove + psycopg2.extensions.register_type(psycopg2.extensions.UNICODE) psycopg2.extensions.register_type(psycopg2.extensions.UNICODEARRAY) @@ -13,7 +18,7 @@ password = u'' server = u'http://' -t = TableauRestApiConnection(server, username, password, site_content_url='') +t = TableauRestApiConnection28(server=server, username=username, password=password, site_content_url=u'default') t.enable_logging(logger) t.signin() @@ -31,11 +36,11 @@ # Loop through the results for row in cur: if row[0] not in groups_dict: - print 'Creating group {}'.format(row[0]) - luid = t.create_group(row[0]) + print('Creating group {}'.format(row[0])) + luid = t.create_group(group_name=row[0]) groups_dict[row[0]] = luid -print groups_dict +print(groups_dict) # Create all @@ -50,17 +55,17 @@ # Loop through users, make sure they exist for row in cur: if row[0] not in users_dict: - print 'Creating user {}'.format(row[0].encode('utf8')) - luid = t.add_user(row[0], row[1], site_role=u'Publisher') + print('Creating user {}'.format(row[0].encode('utf8'))) + luid = t.add_user(username=row[0], fullname=row[1], site_role=u'Publisher') users_dict[row[0]] = luid -print users_dict +print(users_dict) # Create projects for each user for user in users_dict: proj_obj = t.create_project(u"My Saved Reports - {}".format(user)) user_luid = users_dict[user] - perms_obj = proj_obj.create_project_permissions_object_for_user(user_luid, u'Publisher') + perms_obj = proj_obj.create_project_permissions_object_for_user(username_or_luid=user_luid, role=u'Publisher') proj_obj.set_permissions_by_permissions_obj_list([perms_obj, ]) @@ -84,8 +89,8 @@ groups_and_users[group_luid] = [] groups_and_users[group_luid].append(user_luid) - print 'Adding user {} to group {}'.format(row[0].encode('utf8'), row[2].encode('utf8')) - t.add_users_to_group(user_luid, group_luid) + print('Adding user {} to group {}'.format(row[0].encode('utf8'), row[2].encode('utf8'))) + t.add_users_to_group(username_or_luid_s=user_luid, group_name_or_luid=group_luid) # Determine if any users are in a group who do not belong, then remove them for group_luid in groups_and_users: @@ -96,8 +101,8 @@ # values() are the LUIDs in these dicts for user_luid in users_in_group_on_server_dict.values(): if user_luid not in groups_and_users[group_luid]: - print 'Removing user {} from group {}'.format(user_luid, group_luid) - t.remove_users_from_group(user_luid, group_luid) + print('Removing user {} from group {}'.format(user_luid, group_luid)) + t.remove_users_from_group(username_or_luid_s=user_luid, group_name_or_luid=group_luid) # Determine if there are any users who are in the system and not in the database, set them to unlicsened users_on_server = t.query_users() @@ -107,9 +112,9 @@ continue if user_on_server.get("name") not in usernames: if user_on_server.get("siteRole") not in [u'ServerAdministrator', u'SiteAdministrator']: - print 'User on server {} not found in security table, set to Unlicensed'.format(user_on_server.get("name").encode('utf8')) + print('User on server {} not found in security table, set to Unlicensed'.format(user_on_server.get("name").encode('utf8'))) # Just set them to 'Unlicensed' - t.update_user(user_on_server.get("name"), site_role=u'Unlicensed') + t.update_user(username_or_luid=user_on_server.get("name"), site_role=u'Unlicensed') # You can check that content permissions all match their project permissions if necessary diff --git a/setup.py b/setup.py index 644d8d9..43cc6c1 100644 --- a/setup.py +++ b/setup.py @@ -2,7 +2,7 @@ setup( name='tableau_tools', - version='4.7.2', + version='4.8.0', packages=['tableau_tools', 'tableau_tools.tableau_rest_api', 'tableau_tools.tableau_documents', 'tableau_tools.examples'], url='https://github.com/bryantbhowell/tableau_tools', license='', diff --git a/tableau_rest_api/rest_json_request.py b/tableau_rest_api/rest_json_request.py new file mode 100644 index 0000000..420a742 --- /dev/null +++ b/tableau_rest_api/rest_json_request.py @@ -0,0 +1,316 @@ +from ..tableau_base import * +from ..tableau_exceptions import * +import xml.etree.cElementTree as etree +# from HTMLParser import HTMLParser +# from StringIO import StringIO +from io import BytesIO +import re +import math +import copy +import requests +import sys +import json + + +# Handles all of the actual HTTP calling +class RestJsonRequest(TableauBase): + def __init__(self, url=None, token=None, logger=None, ns_map_url='http://tableau.com/api', + verify_ssl_cert=True): + """ + :param url: + :param token: + :param logger: + :param ns_map_url: + """ + super(self.__class__, self).__init__() + + # requests Session created to minimize connections + self.session = requests.Session() + self.session.headers.update({u'Content-Type': u'application/json', u'Accept': u'application/json'}) + + self.__defined_response_types = (u'xml', u'png', u'binary', u'json') + self.__defined_http_verbs = (u'post', u'get', u'put', u'delete') + self.url = url + self._xml_request = None + self.token = token + self.__raw_response = None + self.__last_error = None + self.__last_url_request = None + self.__last_response_headers = None + self.__json_object = None + self.ns_map = {'t': ns_map_url} + etree.register_namespace('t', ns_map_url) + self.logger = logger + self.log(u'RestJsonRequest intialized') + self.__publish = None + self.__boundary_string = None + self.__publish_content = None + self._http_verb = None + self.__response_type = None + self.__last_response_content_type = None + self.__luid_pattern = self.luid_pattern + self.__verify_ssl_cert = verify_ssl_cert + + try: + self.http_verb = 'get' + self.set_response_type(u'json') + except: + raise + + @property + def token(self): + return self._token + + @token.setter + def token(self, token): + self._token = token + # Requests documentation says setting a dict value to None will remove it. + self.session.headers.update({'X-tableau-auth': token, u'Content-Type': u'application/json', + u'Accept': u'application/json'}) + + @property + def xml_request(self): + return self._xml_request + + @xml_request.setter + def xml_request(self, xml_request): + """ + :type xml_request: Element + :return: boolean + """ + self._xml_request = xml_request + + @property + def http_verb(self): + return self._http_verb + + @http_verb.setter + def http_verb(self, verb): + verb = verb.lower() + if verb in self.__defined_http_verbs: + self._http_verb = verb + else: + raise InvalidOptionException(u"HTTP Verb '{}' is not defined for this library".format(verb)) + + def set_response_type(self, response_type): + response_type = response_type.lower() + if response_type in self.__defined_response_types: + self.__response_type = response_type + else: + raise InvalidOptionException(u"Response type '{}' is not defined in this library".format(response_type)) + #if response_type == u'json': + #self.session.headers.update({'Content-Type': 'application/json'}) + + # Must set a boundary string when publishing + def set_publish_content(self, content, boundary_string): + if content is None and boundary_string is None: + self.__publish = False + else: + self.__publish = True + self.__boundary_string = boundary_string + self.__publish_content = content + + def get_raw_response(self): + return self.__raw_response + + def get_last_error(self): + return self.__last_error + + def get_last_url_request(self): + return self.__last_url_request + + def get_last_response_content_type(self): + return self.__last_response_content_type + + def get_response(self): + if self.__response_type == u'json' and self.__json_object is not None: + self.log_debug(u"JSON Object Response: {}".format(json.dumps(self.__json_object))) + return self.__json_object + else: + return self.__raw_response + + # Larger requests require pagination (starting at 1), thus page_number argument can be called. + def __make_request(self, page_number=1): + url = self.url + if page_number > 0: + param_separator = '?' + # If already a parameter, just append + if '?' in url: + param_separator = '&' + url += "{}pageNumber={}".format(param_separator, str(page_number)) + + self.__last_url_request = url + + request_headers = {} + + if self.__publish is True: + request_headers['Content-Type'] = 'multipart/mixed; boundary={}'.format(self.__boundary_string) + + # Need to handle binary return for image somehow + self.log(u"Request {} {}".format(self._http_verb.upper(), url)) + + # Log the XML request being sent + encoded_request = u"" + if self.xml_request is not None: + self.log(u"Request XML: {}".format(etree.tostring(self.xml_request, encoding='utf-8').decode('utf-8'))) + if isinstance(self.xml_request, str): + encoded_request = self.xml_request.encode('utf-8') + else: + encoded_request = etree.tostring(self.xml_request, encoding='utf-8') + if self.__publish_content is not None: + encoded_request = self.__publish_content + try: + if self.http_verb == u'get': + response = self.session.get(url, headers=request_headers, verify=self.__verify_ssl_cert) + elif self.http_verb == u'delete': + response = self.session.delete(url, headers=request_headers, verify=self.__verify_ssl_cert) + elif self.http_verb == u'post': + response = self.session.post(url, data=encoded_request, headers=request_headers, + verify=self.__verify_ssl_cert) + elif self.http_verb == u'put': + response = self.session.put(url, data=encoded_request, headers=request_headers, + verify=self.__verify_ssl_cert) + else: + raise InvalidOptionException(u'Must use one of the http verbs: get, post, put or delete') + # To match previous exception handling pattern with urllib2 + response.raise_for_status() + + # Tableau 9.0 doesn't return real UTF-8 but escapes all unicode characters using numeric character encoding + # initial_response = response.read() # Leave the UTF8 decoding to later + initial_response = response.content # Leave the UTF8 decoding to later + self.log(u'Response is of type {}'.format(type(initial_response))) + + # self.__last_response_content_type = response.info().getheader('Content-Type') + self.__last_response_content_type = response.headers.get(u'Content-Type') + + self.log_debug(u"Content type from headers: {}".format(self.__last_response_content_type)) + + # Don't bother with any extra work if the response is expected to be binary + if self.__response_type == u'binary': + self.__raw_response = initial_response + return initial_response + + + # Use HTMLParser to get rid of the escaped unicode sequences, then encode the thing as utf-8 + # parser = HTMLParser() + # unicode_raw_response = parser.unescape(initial_response) + unicode_raw_response = initial_response + self._set_raw_response(unicode_raw_response) + return True + + # Error detection + except requests.exceptions.HTTPError as e: + self._handle_http_error(response, e) + + def _handle_http_error(self, response, e): + status_code = response.status_code + # No recovering from a 500 (although this can happen for other reasons, possible worth expanding) + if status_code >= 500: + raise e + # REST API returns 400 type errors that can be recovered from, so handle them + raw_error_response = response.content + self.log(u"Received a {} error, here was response:".format(unicode(status_code))) + self.log(raw_error_response.decode('utf8')) + + utf8_parser = etree.XMLParser(encoding='utf-8') + xml = etree.parse(BytesIO(raw_error_response), parser=utf8_parser) + try: + tableau_error = xml.findall(u'.//t:error', namespaces=self.ns_map) + error_code = tableau_error[0].get('code') + tableau_detail = xml.findall(u'.//t:detail', namespaces=self.ns_map) + detail_text = tableau_detail[0].text + # This is to capture an error from the old API version when doing tests + except IndexError: + old_ns_map = {'t': 'http://tableausoftware.com/api'} + tableau_error = xml.findall(u'.//t:error', namespaces=old_ns_map) + error_code = tableau_error[0].get('code') + tableau_detail = xml.findall(u'.//t:detail', namespaces=old_ns_map) + detail_text = tableau_detail[0].text + detail_luid_match_obj = re.search(self.__luid_pattern, detail_text) + if detail_luid_match_obj: + detail_luid = detail_luid_match_obj.group(0) + else: + detail_luid = False + self.log(u'Tableau REST API error code is: {}'.format(error_code)) + # If you are not signed in + if error_code == u'401000': + raise NotSignedInException(u'You must sign in first') + # Everything that is not 400 can potentially be recovered from + if status_code in [401, 402, 403, 404, 405, 409]: + # If 'not exists' for a delete, recover and log + if self._http_verb == 'delete': + self.log(u'Delete action attempted on non-exists, keep going') + if status_code == 409: + self.log(u'HTTP 409 error, most likely an already exists') + raise RecoverableHTTPException(status_code, error_code, detail_luid) + else: + raise e + + def _set_raw_response(self, unicode_raw_response): + if sys.version_info[0] < 3: + try: + self.__raw_response = unicode_raw_response.encode('utf-8') + # Sometimes it appears we actually send this stuff in UTF8 + except UnicodeDecodeError: + self.__raw_response = unicode_raw_response + unicode_raw_response = unicode_raw_response.decode('utf-8') + else: + self.__raw_response = unicode_raw_response + unicode_raw_response = unicode_raw_response.decode('utf-8') + + if self.__response_type == 'xml': + self.log_debug(u"Raw Response: {}".format(unicode_raw_response)) + + def request_from_api(self, page_number=None): + + if page_number is not None: + self.__make_request(page_number) + full_json_obj = json.loads(self.__raw_response) + self.__json_object = full_json_obj + self.log_debug(u"Logging the JSON object for page {}".format(page_number)) + self.log_debug(json.dumps(self.__json_object)) + self.log(u"Request succeeded") + return True + else: + self.__make_request(1) + if self.__response_type == u'json': + if self.__raw_response == '' or self.__raw_response is None or len(self.__raw_response) == 0: + return True + + full_json_obj = json.loads(self.__raw_response) + + total_pages = 1 + for level_1 in full_json_obj: + + if level_1 == u'pagination': + + # page_number = int(pagination.get('pageNumber')) + page_size = int(full_json_obj[u'pagination'][u'pageSize']) + total_available = int(full_json_obj[u'pagination'][u'totalAvailable']) + total_pages = int(math.ceil(float(total_available) / float(page_size))) + self.log_debug(u'{} pages of content found'.format(total_pages)) + else: + combined_json_obj = copy.deepcopy(full_json_obj[level_1]) + if total_pages > 1: + self.log_debug(u'Working on the pages') + for i in xrange(2, total_pages + 1): + self.log_debug(u'Starting on page {}'.format(i)) + self.__make_request(i) # Get next page + + full_json_obj = json.loads(self.__raw_response) + for l1 in full_json_obj: + if l1 != u'pagination': + for main_element in full_json_obj[l1]: + # One level in to get to the list + for list_element in full_json_obj[l1][main_element]: + for e in combined_json_obj: + combined_json_obj[e].append(copy.deepcopy(list_element)) + + self.__json_object = combined_json_obj + self.log_debug(u"Logging the combined JSON object") + self.log_debug(json.dumps(self.__json_object)) + self.log(u"Request succeeded") + return True + elif self.__response_type in ['binary', 'png', 'csv']: + self.log(u'Non XML response') + return True diff --git a/tableau_rest_api/rest_xml_request.py b/tableau_rest_api/rest_xml_request.py index 15094bd..6a1c617 100644 --- a/tableau_rest_api/rest_xml_request.py +++ b/tableau_rest_api/rest_xml_request.py @@ -171,8 +171,8 @@ def __make_request(self, page_number=1): response.raise_for_status() # Tableau 9.0 doesn't return real UTF-8 but escapes all unicode characters using numeric character encoding - # initial_response = response.read() # Leave the UTF8 decoding to lxml - initial_response = response.content # Leave the UTF8 decoding to lxml + # initial_response = response.read() # Leave the UTF8 decoding to later + initial_response = response.content # Leave the UTF8 decoding to later self.log(u'Response is of type {}'.format(type(initial_response))) # self.__last_response_content_type = response.info().getheader('Content-Type') diff --git a/tableau_rest_api/tableau_rest_api_connection.py b/tableau_rest_api/tableau_rest_api_connection.py index 6d3e209..3e43b7d 100644 --- a/tableau_rest_api/tableau_rest_api_connection.py +++ b/tableau_rest_api/tableau_rest_api_connection.py @@ -8,6 +8,7 @@ from ..tableau_documents.tableau_datasource import TableauDatasource from ..tableau_exceptions import * from rest_xml_request import RestXmlRequest +from rest_json_request import RestJsonRequest from published_content import Project20, Project21, Project28, Workbook, Datasource import copy @@ -39,6 +40,7 @@ def __init__(self, server, username, password, site_content_url=""): self._last_response_content_type = None self._request_obj = None # type: RestXmlRequest + self._request_json_obj = None # type: RestJsonRequest # All defined in TableauBase superclass self._site_roles = self.site_roles @@ -309,6 +311,27 @@ def query_resource(self, url_ending, server_level=False): self.end_log_block() return xml + # baseline method for any get request. appends to base url + def query_resource_json(self, url_ending, server_level=False, page_number=None): + """ + :type url_ending: unicode + :type server_level: bool + :type page_number: int + :rtype: json + """ + self.start_log_block() + api_call = self.build_api_url(url_ending, server_level) + if self._request_json_obj is None: + self._request_json_obj = RestJsonRequest(token=self.token, logger=self.logger, + verify_ssl_cert=self.verify_ssl_cert) + self._request_json_obj.http_verb = u'get' + self._request_json_obj.url = api_call + self._request_json_obj.request_from_api(page_number=page_number) + json_response = self._request_json_obj.get_response() # return JSON as string + self._request_obj.url = None + self.end_log_block() + return json_response + def query_single_element_from_endpoint(self, element_name, name_or_luid, server_level=False): """ :type element_name: unicode @@ -493,6 +516,16 @@ def query_datasources(self, project_name_or_luid=None): self.end_log_block() return dses + def query_datasources_json(self, page_number=None): + """ + :type page_number: int + :rtype: json + """ + self.start_log_block() + datasources = self.query_resource_json(u"datasources", page_number=page_number) + self.end_log_block() + return datasources + # Tries to guess name or LUID, hope there is only one def query_datasource(self, ds_name_or_luid, proj_name_or_luid=None): """ @@ -595,6 +628,21 @@ def query_groups(self): # # No basic verb for querying a single group, so run a query_groups + def query_groups_json(self, page_number=None): + """ + :type page_number: int + :rtype: json + """ + self.start_log_block() + groups = self.query_resource_json(u"groups", page_number=page_number) + #for group in groups: + # # Add to group-name : luid cache + # group_luid = group.get(u"id") + # group_name = group.get(u'name') + # self.group_name_luid_cache[group_name] = group_luid + self.end_log_block() + return groups + def query_group(self, group_name_or_luid): """ :type group_name_or_luid: unicode @@ -664,6 +712,16 @@ def query_projects(self): self.end_log_block() return projects + def query_projects_json(self, page_number=None): + """ + :type page_number: int + :rtype: json + """ + self.start_log_block() + projects = self.query_resource_json(u"projects", page_number=page_number) + self.end_log_block() + return projects + def query_project(self, project_name_or_luid): """ :type project_name_or_luid: unicode @@ -721,6 +779,16 @@ def query_sites(self): self.end_log_block() return sites + def query_sites_json(self, page_number=None): + """ + :type page_number: int + :rtype: json + """ + self.start_log_block() + sites = self.query_resource_json(u"sites", server_level=True, page_number=page_number) + self.end_log_block() + return sites + # Methods for getting info about the sites, since you can only query a site when you are signed into it # Return list of all site contentUrls @@ -771,6 +839,25 @@ def query_users(self): self.end_log_block() return users + # The reference has this name, so for consistency adding an alias + def get_users_json(self, page_number=None): + """ + :type page_number: int + :rtype: json + """ + return self.query_users_json(page_number=page_number) + + def query_users_json(self, page_number=None): + """ + :type page_number: int + :rtype: json + """ + self.start_log_block() + users = self.query_resource_json(u"users", page_number=page_number) + #self.log(u'Found {} users'.format(unicode(len(users)))) + self.end_log_block() + return users + def query_user(self, username_or_luid): """ :type username_or_luid: unicode @@ -866,6 +953,23 @@ def query_workbooks(self, username_or_luid=None, project_name_or_luid=None): self.end_log_block() return wbs + def query_workbooks_json(self, username_or_luid=None, page_number=None): + """ + :type username_or_luid: unicode + :type page_number: int + :rtype: json + """ + self.start_log_block() + if username_or_luid is None: + user_luid = self.user_luid + elif self.is_luid(username_or_luid): + user_luid = username_or_luid + else: + user_luid = self.query_user_luid(username_or_luid) + wbs = self.query_resource_json(u"users/{}/workbooks".format(user_luid), page_number=page_number) + self.end_log_block() + return wbs + # Because a workbook can have the same pretty name in two projects, requires more logic def query_workbook(self, wb_name_or_luid, proj_name_or_luid=None, username_or_luid=None): """ @@ -924,7 +1028,7 @@ def query_workbook_luid(self, wb_name, proj_name_or_luid=None, username_or_luid= wb_luid = workbooks_with_name[0].get("id") self.end_log_block() return wb_luid - elif len(workbooks_with_name) > 1 and proj_name_or_luid is not False: + elif len(workbooks_with_name) > 1 and proj_name_or_luid is not None: if self.is_luid(proj_name_or_luid): wb_in_proj = workbooks.findall(u'.//t:workbook[@name="{}"]/t:project[@id="{}"]/..'.format(wb_name, proj_name_or_luid), self.ns_map) else: @@ -979,6 +1083,29 @@ def query_workbook_views(self, wb_name_or_luid, proj_name_or_luid=None, username self.end_log_block() return vws + def query_workbook_views_json(self, wb_name_or_luid, proj_name_or_luid=None, username_or_luid=None, usage=False, + page_number=None): + """ + :type wb_name_or_luid: unicode + :type proj_name_or_luid: unicode + :type username_or_luid: unicode + :type usage: bool + :type page_number: int + :rtype: json + """ + self.start_log_block() + if usage not in [True, False]: + raise InvalidOptionException(u'Usage can only be set to True or False') + if self.is_luid(wb_name_or_luid): + wb_luid = wb_name_or_luid + else: + wb_luid = self.query_workbook_luid(wb_name_or_luid, proj_name_or_luid, username_or_luid) + vws = self.query_resource_json(u"workbooks/{}/views?includeUsageStatistics={}".format(wb_luid, + str(usage).lower()), + page_number=page_number) + self.end_log_block() + return vws + def query_workbook_view(self, wb_name_or_luid, view_name_or_luid=None, view_content_url=None, proj_name_or_luid=None, username_or_luid=None, usage=False): """ diff --git a/tableau_rest_api/tableau_rest_api_connection_22.py b/tableau_rest_api/tableau_rest_api_connection_22.py index 7ae88e4..de2d4d6 100644 --- a/tableau_rest_api/tableau_rest_api_connection_22.py +++ b/tableau_rest_api/tableau_rest_api_connection_22.py @@ -24,6 +24,16 @@ def query_schedules(self): self.end_log_block() return schedules + def query_schedules_json(self, page_number=None): + """ + :type page_number: int + :rtype: json + """ + self.start_log_block() + schedules = self.query_resource_json(u"schedules", server_level=True, page_number=page_number) + self.end_log_block() + return schedules + def query_extract_schedules(self): """ :rtype: etree.Element @@ -78,8 +88,6 @@ def query_extract_refresh_tasks_by_schedule(self, schedule_name_or_luid): self.end_log_block() return tasks - - # # End Scheduler Querying Methods # @@ -96,5 +104,18 @@ def query_views(self, usage=False): self.end_log_block() return vws + def query_views_json(self, usage=False, page_number=None): + """ + :type usage: bool + :rtype: json + """ + self.start_log_block() + if usage not in [True, False]: + raise InvalidOptionException(u'Usage can only be set to True or False') + vws = self.query_resource_json(u"views?includeUsageStatistics={}".format(str(usage).lower()), + page_number=page_number) + self.end_log_block() + return vws + # Did not implement any variations of query_workbook_views as it's still necessary to know the workbook to narrow # down to that particular view \ No newline at end of file diff --git a/tableau_rest_api/tableau_rest_api_connection_23.py b/tableau_rest_api/tableau_rest_api_connection_23.py index 607e050..f282aa9 100644 --- a/tableau_rest_api/tableau_rest_api_connection_23.py +++ b/tableau_rest_api/tableau_rest_api_connection_23.py @@ -153,6 +153,56 @@ def query_single_element_luid_from_endpoint_with_filter(self, element_name, name self.end_log_block() raise NoMatchFoundException(u"No {} found with name {}".format(element_name, name)) + # These are the new basic methods that use the Filter functionality introduced + def query_resource_json(self, url_ending, server_level=False, filters=None, sorts=None, additional_url_ending=None, + page_number=None): + """ + :type url_ending: unicode + :type server_level: bool + :type filters: list[UrlFilter] + :type sorts: list[Sort] + :type additional_url_ending: unicode + :type page_number: int + :rtype: etree.Element + """ + self.start_log_block() + if filters is not None: + if len(filters) > 0: + filters_url = u"filter=" + for f in filters: + filters_url += f.get_filter_string() + u"," + filters_url = filters_url[:-1] + + if sorts is not None: + if len(sorts) > 0: + sorts_url = u"sort=" + for sort in sorts: + sorts_url += sort.get_sort_string() + u"," + sorts_url = sorts_url[:-1] + + if sorts is not None and filters is not None: + url_ending += u"?{}&{}".format(sorts_url, filters_url) + elif sorts is not None: + url_ending += u"?{}".format(sorts_url) + elif filters is not None and len(filters) > 0: + url_ending += u"?{}".format(filters_url) + elif additional_url_ending is not None: + url_ending += u"?" + if additional_url_ending is not None: + url_ending += additional_url_ending + + api_call = self.build_api_url(url_ending, server_level) + if self._request_json_obj is None: + self._request_json_obj = RestJsonRequest(token=self.token, logger=self.logger, + verify_ssl_cert=self.verify_ssl_cert) + self._request_json_obj.http_verb = u'get' + self._request_json_obj.url = api_call + self._request_json_obj.request_from_api(page_number=page_number) + json_response = self._request_json_obj.get_response() # return JSON as string + self._request_obj.url = None + self.end_log_block() + return json_response + # Check method for filter objects @staticmethod def _check_filter_objects(filter_checks): diff --git a/tableau_rest_api/tableau_rest_api_connection_24.py b/tableau_rest_api/tableau_rest_api_connection_24.py index 391806e..5c2344e 100644 --- a/tableau_rest_api/tableau_rest_api_connection_24.py +++ b/tableau_rest_api/tableau_rest_api_connection_24.py @@ -35,6 +35,14 @@ def query_api_version(self): # grab api version number def query_views(self, usage=False, created_at_filter=None, updated_at_filter=None, tags_filter=None, sorts=None): + """ + :type usage: bool + :type created_at_filter: UrlFilter + :type updated_at_filter: UrlFilter + :type tags_filter: UrlFilter + :type sorts: list[Sort] + :rtype: etree.Element + """ self.start_log_block() if usage not in [True, False]: raise InvalidOptionException(u'Usage can only be set to True or False') @@ -46,6 +54,29 @@ def query_views(self, usage=False, created_at_filter=None, updated_at_filter=Non self.end_log_block() return vws + def query_views_json(self, usage=False, created_at_filter=None, updated_at_filter=None, tags_filter=None, + sorts=None, page_number=None): + """ + :type usage: bool + :type created_at_filter: UrlFilter + :type updated_at_filter: UrlFilter + :type tags_filter: UrlFilter + :type sorts: list[Sort] + :type page_number: int + :rtype: json + """ + self.start_log_block() + if usage not in [True, False]: + raise InvalidOptionException(u'Usage can only be set to True or False') + filter_checks = {u'updatedAt': updated_at_filter, u'createdAt': created_at_filter, u'tags': tags_filter} + filters = self._check_filter_objects(filter_checks) + + vws = self.query_resource_json(u"views", filters=filters, sorts=sorts, + additional_url_ending=u"includeUsageStatistics={}".format(str(usage).lower()), + page_number=page_number) + self.end_log_block() + return vws + def query_view(self, vw_name_or_luid): """ :type vw_name_or_luid: @@ -88,4 +119,25 @@ def query_datasources(self, project_name_or_luid=None, updated_at_filter=None, c self.end_log_block() return dses + def query_datasources_json(self, updated_at_filter=None, created_at_filter=None, + tags_filter=None, datasource_type_filter=None, sorts=None, page_number=None): + """ + :type updated_at_filter: UrlFilter + :type created_at_filter: UrlFilter + :type tags_filter: UrlFilter + :type datasource_type_filter: UrlFilter + :type sorts: list[Sort] + :type page_number: int + :rtype: json + """ + self.start_log_block() + filter_checks = {u'updatedAt': updated_at_filter, u'createdAt': created_at_filter, u'tags': tags_filter, + u'type': datasource_type_filter} + filters = self._check_filter_objects(filter_checks) + + datasources = self.query_resource_json(u'datasources', filters=filters, sorts=sorts, page_number=page_number) + + self.end_log_block() + return datasources + # query_datasource and query_datasource_luid can't be improved because filtering doesn't take a Project Name/LUID \ No newline at end of file diff --git a/tableau_rest_api/tableau_rest_api_connection_25.py b/tableau_rest_api/tableau_rest_api_connection_25.py index dd0414d..ad9fc13 100644 --- a/tableau_rest_api/tableau_rest_api_connection_25.py +++ b/tableau_rest_api/tableau_rest_api_connection_25.py @@ -27,6 +27,21 @@ def query_user_favorites(self, username_or_luid): self.end_log_block() return favorites + def query_user_favorites_json(self, username_or_luid, page_number=None): + """ + :type username_or_luid: unicode + :rtype: json + """ + self.start_log_block() + if self.is_luid(username_or_luid): + user_luid = username_or_luid + else: + user_luid = self.query_user_luid(username_or_luid) + favorites = self.query_resource_json(u"favorites/{}/".format(user_luid), page_number=page_number) + + self.end_log_block() + return favorites + def create_project(self, project_name=None, project_desc=None, locked_permissions=True, publish_samples=False, no_return=False, direct_xml_request=None): """ @@ -251,6 +266,66 @@ def query_single_element_luid_from_endpoint_with_filter(self, element_name, name self.end_log_block() raise NoMatchFoundException(u"No {} found with name {}".format(element_name, name)) + # These are the new basic methods that use the Filter functionality introduced + def query_resource_json(self, url_ending, server_level=False, filters=None, sorts=None, additional_url_ending=None, + fields=None, page_number=None): + """ + :type url_ending: unicode + :type server_level: bool + :type filters: list[UrlFilter] + :type sorts: list[Sort] + :type additional_url_ending: unicode + :type fields: list[unicode] + :type page_number: int + :rtype: json + """ + self.start_log_block() + url_endings = [] + if filters is not None: + if len(filters) > 0: + filters_url = u"filter=" + for f in filters: + filters_url += f.get_filter_string() + u"," + filters_url = filters_url[:-1] + url_endings.append(filters_url) + if sorts is not None: + if len(sorts) > 0: + sorts_url = u"sort=" + for sort in sorts: + sorts_url += sort.get_sort_string() + u"," + sorts_url = sorts_url[:-1] + url_endings.append(sorts_url) + if fields is not None: + if len(fields) > 0: + fields_url = u"fields=" + for field in fields: + fields_url += u"{},".format(field) + fields_url = fields_url[:-1] + url_endings.append(fields_url) + if additional_url_ending is not None: + url_endings.append(additional_url_ending) + + first = True + if len(url_endings) > 0: + for ending in url_endings: + if first is True: + url_ending += u"?{}".format(ending) + first = False + else: + url_ending += u"&{}".format(ending) + + api_call = self.build_api_url(url_ending, server_level) + if self._request_json_obj is None: + self._request_json_obj = RestJsonRequest(token=self.token, logger=self.logger, + verify_ssl_cert=self.verify_ssl_cert) + self._request_json_obj.http_verb = u'get' + self._request_json_obj.url = api_call + self._request_json_obj.request_from_api(page_number=page_number) + json_response = self._request_json_obj.get_response() # return JSON as string + self._request_obj.url = None + self.end_log_block() + return json_response + def query_datasources(self, project_name_or_luid=None, all_fields=True, updated_at_filter=None, created_at_filter=None, tags_filter=None, datasource_type_filter=None, sorts=None, fields=None): """ @@ -291,6 +366,36 @@ def query_datasources(self, project_name_or_luid=None, all_fields=True, updated_ self.end_log_block() return dses + def query_datasources_json(self, project_name_or_luid=None, all_fields=True, updated_at_filter=None, + created_at_filter=None, tags_filter=None, datasource_type_filter=None, sorts=None, + fields=None, page_number=None): + """ + :type project_name_or_luid: unicode + :type all_fields: bool + :type updated_at_filter: UrlFilter + :type created_at_filter: UrlFilter + :type tags_filter: UrlFilter + :type datasource_type_filter: UrlFilter + :type sorts: list[Sort] + :type fields: list[unicode] + :type page_number: int + :rtype: json + """ + self.start_log_block() + if fields is None: + if all_fields is True: + fields = [u'_all_'] + + filter_checks = {u'updatedAt': updated_at_filter, u'createdAt': created_at_filter, u'tags': tags_filter, + u'type': datasource_type_filter} + filters = self._check_filter_objects(filter_checks) + + datasources = self.query_resource_json(u'datasources', filters=filters, sorts=sorts, fields=fields, + page_number=page_number) + + self.end_log_block() + return datasources + def query_workbooks(self, username_or_luid=None, project_name_or_luid=None, all_fields=True, created_at_filter=None, updated_at_filter=None, owner_name_filter=None, tags_filter=None, sorts=None, fields=None): """ @@ -337,6 +442,45 @@ def query_workbooks(self, username_or_luid=None, project_name_or_luid=None, all_ self.end_log_block() return wbs + def query_workbooks_json(self, username_or_luid=None, project_name_or_luid=None, all_fields=True, + created_at_filter=None, updated_at_filter=None, owner_name_filter=None, + tags_filter=None, sorts=None, fields=None, page_number=None): + """ + :type username_or_luid: unicode + :type all_fields: bool + :type created_at_filter: UrlFilter + :type updated_at_filter: UrlFilter + :type owner_name_filter: UrlFilter + :type tags_filter: UrlFilter + :type sorts: list[Sort] + :type fields: list[Sort] + :type page_number: int + :rtype: json + """ + self.start_log_block() + if fields is None: + if all_fields is True: + fields = [u'_all_'] + + if username_or_luid is None: + user_luid = self.user_luid + elif self.is_luid(username_or_luid): + user_luid = username_or_luid + else: + user_luid = self.query_user_luid(username_or_luid) + + filter_checks = {u'updatedAt': updated_at_filter, u'createdAt': created_at_filter, u'tags': tags_filter, + u'ownerName': owner_name_filter} + filters = self._check_filter_objects(filter_checks) + + if username_or_luid is not None: + wbs = self.query_resource_json(u"users/{}/workbooks".format(user_luid), page_number=page_number) + else: + wbs = self.query_resource(u"workbooks".format(user_luid), sorts=sorts, filters=filters, fields=fields) + + self.end_log_block() + return wbs + # Alias added in def get_users(self, all_fields=True, last_login_filter=None, site_role_filter=None, sorts=None, fields=None): """ @@ -375,6 +519,47 @@ def query_users(self, all_fields=True, last_login_filter=None, site_role_filter= self.end_log_block() return users + def get_users_json(self, all_fields=True, last_login_filter=None, site_role_filter=None, sorts=None, fields=None, + page_number=None): + """ + :type all_fields: bool + :type last_login_filter: UrlFilter + :type site_role_filter: UrlFilter + :type sorts: list[Sort] + :type fields: list[unicode] + :type page_number: int + :rtype: json + """ + return self.query_users_json(all_fields=all_fields, last_login_filter=last_login_filter, + site_role_filter=site_role_filter, sorts=sorts, fields=fields, page_number=page_number) + + # New methods with Filtering + def query_users_json(self, all_fields=True, last_login_filter=None, site_role_filter=None, sorts=None, fields=None, + username_filter=None, page_number=None): + """ + :type all_fields: bool + :type last_login_filter: UrlFilter + :type site_role_filter: UrlFilter + :type username_filter: UrlFilter + :type sorts: list[Sort] + :type fields: list[unicode] + :type page_number: int + :rtype: json + """ + self.start_log_block() + if fields is None: + if all_fields is True: + fields = [u'_all_'] + + filter_checks = {u'lastLogin': last_login_filter, u'siteRole': site_role_filter, u'name': username_filter} + filters = self._check_filter_objects(filter_checks) + + users = self.query_resource_json(u"users", filters=filters, sorts=sorts, fields=fields, page_number=page_number) + + self.log(u'Found {} users'.format(unicode(len(users)))) + self.end_log_block() + return users + def query_user(self, username_or_luid, all_fields=True): """ :type username_or_luid: unicode @@ -406,12 +591,12 @@ def query_user_luid(self, username): # Do not include file extension. Without filename, only returns the response def download_datasource(self, ds_name_or_luid, filename_no_extension, proj_name_or_luid=None, - download_extract=True): + include_extract=True): """" :type ds_name_or_luid: unicode :type filename_no_extension: unicode :type proj_name_or_luid: unicode - :type download_extract: bool + :type include_extract: bool :return Filename of the saved file :rtype: unicode """ @@ -421,7 +606,7 @@ def download_datasource(self, ds_name_or_luid, filename_no_extension, proj_name_ else: ds_luid = self.query_datasource_luid(ds_name_or_luid, project_name_or_luid=proj_name_or_luid) try: - if download_extract is False: + if include_extract is False: url = self.build_api_url(u"datasources/{}/content?includeExtract=False".format(ds_luid)) else: url = self.build_api_url(u"datasources/{}/content".format(ds_luid)) diff --git a/tableau_rest_api/tableau_rest_api_connection_27.py b/tableau_rest_api/tableau_rest_api_connection_27.py index 9ebd46e..d784ae8 100644 --- a/tableau_rest_api/tableau_rest_api_connection_27.py +++ b/tableau_rest_api/tableau_rest_api_connection_27.py @@ -90,6 +90,36 @@ def query_groups(self, name_filter=None, domain_name_filter=None, domain_nicknam self.end_log_block() return groups + def query_groups_json(self, name_filter=None, domain_name_filter=None, domain_nickname_filter=None, + is_local_filter=None, user_count_filter=None, minimum_site_role_filter=None, + sorts=None, page_number=None): + """ + :type name_filter: UrlFilter + :type domain_name_filter: UrlFilter + :type domain_nickname_filter: UrlFilter + :type is_local_filter: UrlFilter + :type user_count_filter: UrlFilter + :type minimum_site_role_filter: UrlFilter + :type sorts: list[Sort] + :type page_number: int + :rtype: etree.Element + """ + filter_checks = {u'name': name_filter, u'domainName': domain_name_filter, + u'domainNickname': domain_nickname_filter, u'isLocal': is_local_filter, + u'userCount': user_count_filter, u'minimumSiteRole': minimum_site_role_filter} + + filters = self._check_filter_objects(filter_checks) + + self.start_log_block() + groups = self.query_resource_json(u"groups", filters=filters, sorts=sorts, page_number=page_number) + for group in groups: + # Add to group-name : luid cache + group_luid = group.get(u"id") + group_name = group.get(u'name') + self.group_name_luid_cache[group_name] = group_luid + self.end_log_block() + return groups + # # No basic verb for querying a single group, so run a query_groups def query_group(self, group_name_or_luid): @@ -155,6 +185,31 @@ def query_projects(self, name_filter=None, owner_name_filter=None, updated_at_fi self.end_log_block() return projects + def query_projects_json(self, name_filter=None, owner_name_filter=None, updated_at_filter=None, + created_at_filter=None, owner_domain_filter=None, owner_email_filter=None, sorts=None, + page_number=None): + """ + :type name_filter: UrlFilter + :type owner_name_filter: UrlFilter + :type updated_at_filter: UrlFilter + :type created_at_filter: UrlFilter + :type owner_domain_filter: UrlFilter + :type owner_email_filter: UrlFilter + :type sorts: list[Sort] + :type page_number: int + :rtype: etree.Element + """ + filter_checks = {u'name': name_filter, u'ownerName': owner_name_filter, + u'updatedAt': updated_at_filter, u'createdAt': created_at_filter, + u'ownerDomain': owner_domain_filter, u'ownerEmail': owner_email_filter} + + filters = self._check_filter_objects(filter_checks) + + self.start_log_block() + projects = self.query_resource_json(u"projects", filters=filters, sorts=sorts, page_number=None) + self.end_log_block() + return projects + def query_project_luid(self, project_name): """ :type project_name: unicode diff --git a/tableau_rest_api/tableau_rest_api_connection_28.py b/tableau_rest_api/tableau_rest_api_connection_28.py index 07cbeea..b8374da 100644 --- a/tableau_rest_api/tableau_rest_api_connection_28.py +++ b/tableau_rest_api/tableau_rest_api_connection_28.py @@ -1,4 +1,6 @@ from tableau_rest_api_connection_27 import * +import csv +import urllib class TableauRestApiConnection28(TableauRestApiConnection27): @@ -255,7 +257,7 @@ def save_view_pdf(self, wb_name_or_luid, view_name_or_luid, filename_no_extensio self.end_log_block() raise - def save_view_data_as_csv(self, wb_name_or_luid, view_name_or_luid, filename_no_extension=None, + def save_view_data_as_csv(self, wb_name_or_luid=None, view_name_or_luid=None, filename_no_extension=None, proj_name_or_luid=None, view_filter_map=None): """ :type wb_name_or_luid: unicode @@ -293,10 +295,9 @@ def save_view_data_as_csv(self, wb_name_or_luid, view_name_or_luid, filename_no_ save_file.write(data) save_file.close() self.end_log_block() + return else: - self.end_log_block() - # Do we need to do a codec conversion to make this unicode text? - return data + raise InvalidOptionException(u'This method is for saving response to file. Must include filename_no_extension parameter') # You might be requesting something that doesn't exist except RecoverableHTTPException as e: @@ -308,6 +309,46 @@ def save_view_data_as_csv(self, wb_name_or_luid, view_name_or_luid, filename_no_ self.end_log_block() raise + def query_view_data(self, wb_name_or_luid=None, view_name_or_luid=None, proj_name_or_luid=None, + view_filter_map=None): + """ + :type wb_name_or_luid: unicode + :type view_name_or_luid: unicode + :type proj_name_or_luid: unicode + :type view_filter_map: dict + :rtype: csv + """ + self.start_log_block() + + if self.is_luid(view_name_or_luid): + view_luid = view_name_or_luid + else: + if wb_name_or_luid is None: + raise InvalidOptionException(u'If looking up view by name, must include workbook') + view_luid = self.query_workbook_view_luid(wb_name_or_luid, view_name=view_name_or_luid, + proj_name_or_luid=proj_name_or_luid) + try: + if view_filter_map is not None: + final_filter_map = {} + for key in view_filter_map: + new_key = u"vf_{}".format(key) + final_filter_map[new_key] = view_filter_map[key] + + additional_url_params = u"?" + urllib.urlencode(final_filter_map) + else: + additional_url_params = u"" + url = self.build_api_url(u"views/{}/data{}".format(view_luid, additional_url_params)) + # Raw response should be UTF-8 encoded plain text CSV + data = self.send_binary_get_request(url) + # Convert to CSV object? + return data + + # You might be requesting something that doesn't exist + except RecoverableHTTPException as e: + self.log(u"Attempt to request data results in HTTP error {}, Tableau Code {}".format(e.http_code, e.tableau_error_code)) + self.end_log_block() + raise + def update_datasource_now(self, ds_name_or_luid, project_name_or_luid=False): """ :type ds_name_or_luid: unicode