4.8.0 introduces RestJsonRequest object and _json methods for queryin…

…g lists. Other bug fixes and documentation changes as well

README.md

@@ -50,6 +50,7 @@ The TableauDatasource class uses the `TDEFileGenerator` and/or the `HyperFileGen
 * 4.4.0: A lot of improvements to the tableau_documents library and its documentation
 * 4.5.0: 2018.1 (API 3.0) compatibility. All requests for a given connection using a single HTTP session, and other improvements to the RestXmlRequest class.
 * 4.7.0: Dropping API 2.0 (Tableau 9.0) compatibility. Any method that is overwritten in a later version will not be updated in the TableauRestApiConnection class going forward. Also implemented a direct_xml_request parameter for Add and Update methods, allowing direct submission of an ElementTree.Element request to the endpoints, particularly for replication.
+* 4.8.0 Introduces the RestJsonRequest object and _json plural querying methods for passing JSON responses to other systems
 ## --- Table(au) of Contents ---
 ------
 
@@ -116,6 +117,7 @@ The TableauDatasource class uses the `TDEFileGenerator` and/or the `HyperFileGen
   * [2.12 Creating a TableauDatasource from Scratch](#212-creating-a-tableaudatasource-from-scratch)
   * [2.13 Creating and Modifying Parameters](#213-creating-and-modifying-parameters)
     + [2.13.1 TableauParameter class](#2131-tableauparameter-class)
+  * [2.14 HyperFileGenerator and TDEFileGenerator Classes](#214-)
 - [3 tabcmd](#3-tabcmd)
   * [3.1 Tabcmd Class](#31-tabcmd-class)
   * [3.2 Triggering an Extract Refresh](#32-triggering-an-extract-refresh)
@@ -133,6 +135,7 @@ tableau_tools
     * permissions
     * published_content (Project, Workbook, Datasource)
     * rest_xml_request
+    * rest_json_request
     * sort
     * tableau_rest_api_server_connection
     * tableau_rest_api_server_connection21
@@ -143,6 +146,9 @@ tableau_tools
     * tableau_rest_api_server_connection26
     * tableau_rest_api_server_connection27
     * tableau_rest_api_server_connection28
+    * tableau_rest_api_server_connection30
+    * tableau_rest_api_server_connection31
+    * tableau_rest_api_server_connection32
     * url_filter
 * tableau_documents
     * tableau_connection
@@ -225,6 +231,10 @@ tableau_tools 4.0+ implements the different versions of the Tableau Server REST
 
 `TableauRestApiConnection30: 2018.1`
 
+`TableauRestApiConnection31: 2018.2`
+
+`TableauRestApiConnection32: 2018.3`
+
 You need to initialize at least one object of this class. 
 Ex.:
 
@@ -275,14 +285,17 @@ The simplest method for getting information from the REST API are the "plural" q
 
 `TableauRestApiConnection.query_groups()`
 
-`TableauRestApiConnection.query_users()`
+`TableauRestApiConnection.query_users(username_or_luid)`
 
 `TableauRestApiConnection.query_workbooks()`
 
 `TableauRestApiConnection.query_projects()`
 
 `TableauRestApiConnection.query_datasources()`
 
+`TableauRestApiConnection.query_workbook_views()`
+
+
 These will all return an ElementTree object representing the results from the REST API call. This can be useful if you need all of the information returned, but most of your calls to these methods will be to get a dictionary of names : luids you can use for lookup. There is a simple static method for this conversion
 
 `TableauRestApiConnection.convert_xml_list_to_name_id_dict(xml_obj)`
@@ -297,6 +310,25 @@ Ex.
     for group_name in groups_dict:
         print "Group name {} is LUID {}".format(group_name, groups_dict[group_name])
 
+There are also equivalent JSON querying methods for the plural methods, which return a Python json object, using the Tableau REST API's native method for requesting JSON responses rather than XML. 
+
+The JSON plural query methods allow you to specify the page of results using the page= optional parameter(starting at 1). If you do not specify a page, tableau_tools will automatically paginate through all results and combine them together. 
+
+
+`TableauRestApiConnection.query_groups_json(page_number=None)`
+
+`TableauRestApiConnection.query_users_json(page_number=None)`
+
+`TableauRestApiConnection.query_workbooks_json(username_or_luid, page_number=None)`
+
+`TableauRestApiConnection.query_projects_json(page_number=None)`
+
+`TableauRestApiConnection.query_datasources_json(page_number=None)`
+
+`TableauRestApiConnection.query_workbook_views_json(page_number=None)`
+
+
+
 ##### 1.2.2.1 Filtering and Sorting (Tableau Server 9.3+)
 `TableauRestApiConnection22` implements filtering and sorting for the methods where it is allowed. Singular lookup methods are programmed to take advantage of this automatically for improved perofrmance, but the plural querying methods can use the filters to bring back specific sets.
 
@@ -1522,6 +1554,31 @@ Ex.
     param.set_allowable_values_to_list(allowable_values)
     param.set_current_value(u'Spring 2018')
 
+### 2.14 HyperFileGenerator and TDEFileGenerator Classes
+The "add extract" functionality in tableau_tools uses the Extract API/Tableau SDK (they are the same thing, the names changed back and forth over time). The HyperFileGenerator and TDEFileGenerator classes are replicas of one another, but HyperFileGenerator uses the Extract API 2.0, which is capable of creating Hyper files. 
+
+You can use them in conjunction with PyODBC (install via pip) to create extracts very simply from a query to an ODBC connection. In essence, the classes will map any of the PyODBC data types to the Extract API data types automatically when you provide a PyODBC cursor from an executed query.
+
+`HyperFileGenerator(logger_obj)`
+
+There are two steps to creating an extract. You first must create a Table Definition, then you insert the rows of data.
+
+The most basic way to set a Table Definition is defining a dict in the form of {'column_name' : 'data_type'}.
+
+`HyperFileGenerator.set_table_definition(column_name_type_dict, collation=Collation.EN_US)`
+
+However, you can use the a pyodbc cursor to the same effect, which basically lets you just write a query and pass everything through directly:
+
+`HyperFileGenerator.create_table_definition_from_pyodbc_cursor(pydobc_cursor, collation=Collation.EN_US)`
+
+This will return the TableDefinition object from the Extract API, but it also sets the internal table_definition for the particular instance of the HyperFileGenerator object so you don't need to do anything other than run this method and anything afterward you do will take the current TableDefinition.
+
+To generate the extract:
+`HyperFileGenerator.create_extract(tde_filename, append=False, table_name=u'Extract', pyodbc_cursor=None)`
+
+You do need to specify an actual filename for it to write to, because the Extract API always works on a file on disk. You can specify multiple tables within this file by giving different table names, and you can even append by specifying append=True while using the same table_name that previously has been created within the file. The pyodbc_cursor= optional parameter will run through all of the rows from the cursor and add them to the Extract. 
+
+At the current time, the only exposed method to add data to the extract is the pyodbc cursor.
 
 ## 3 tabcmd
 The Tableau Server REST API can do most of the things that the tabcmd command line tool can, but if you are using older versions of Tableau Server, some of those features may not have been implemented yet. If you need a functionality from tabcmd, the `tabcmd.py` file in the main part of tableau_tools library wraps most of the commonly used functionality to allow for easier scripting of calls (rather than doing it directly on the command line or using batch files)

examples/archive_site.py

@@ -0,0 +1,44 @@
+# -*- coding: utf-8 -*-
+
+from tableau_tools.tableau_rest_api import *
+from tableau_tools import *
+import os
+
+
+def archive_tableau_site(save_to_directory, server, username, password, site_content_url):
+    # The last two digits of this constructor match to the version of API available on the Tableau Server
+    t = TableauRestApiConnection30(server=server, username=username,
+                                   password=password, site_content_url=site_content_url)
+    t.signin()
+    all_projects = t.query_projects()
+    all_projects_dict = t.convert_xml_list_to_name_id_dict(all_projects)
+
+    # This gives you the Project name; the values of the dict are the LUIDs
+    for project in all_projects_dict:
+        # Create directory for projects
+        try:
+            print(u'Making directory {}'.format(project))
+            os.mkdir(u'{}/{}'.format(save_to_directory, project))
+        except OSError as e:
+            print(u'Directory already exists')
+
+        print(u'Downloading datasources for project {}'.format(project))
+        # Get All Data sources
+        dses_in_project = t.query_datasources(project_name_or_luid=all_projects_dict[project])
+        for ds in dses_in_project:
+            ds_luid = ds.get(u'id')
+            ds_content_url = ds.get(u'contentUrl')
+            print(u'Downloading datasource {}'.format(ds_content_url))
+            t.download_datasource(ds_name_or_luid=ds_luid,
+                                  filename_no_extension=u"{}/{}/{}".format(save_to_directory, project, ds_content_url),
+                                  include_extract=False)
+
+        print(u'Downloading workbooks for project {}'.format(project))
+        wbs_in_project = t.query_workbooks_in_project(project_name_or_luid=all_projects_dict[project])
+        for wb in wbs_in_project:
+            wb_luid = wb.get(u'id')
+            wb_content_url = wb.get(u'contentUrl')
+            print(u'Downloading workbook {}'.format(wb_content_url))
+            t.download_workbook(wb_name_or_luid=wb_luid,
+                                filename_no_extension=u"{}/{}/{}".format(save_to_directory, project, wb_content_url),
+                                include_extract=False)

examples/create_site_sample.py

@@ -12,19 +12,20 @@
 new_site_name = u'Sample Test Site'
 
 # Choose the API version for your server. 10.2 = 25
-default = TableauRestApiConnection26(server, username, password)
+default = TableauRestApiConnection26(server=server, username=username, password=password, site_content_url=u'default')
 try:
     default.signin()
-    default.create_site(new_site_name, new_site_content_url)
+    default.create_site(new_site_name=new_site_name, new_content_url=new_site_content_url)
 except AlreadyExistsException as e:
-    print e.msg
-    print u"Cannot create new site, it already exists"
+    print(e.msg)
+    print(u"Cannot create new site, it already exists")
     exit()
 
 time.sleep(4)
 
 
-t = TableauRestApiConnection26(server, username, password, new_site_content_url)
+t = TableauRestApiConnection26(server=server, username=username, password=password,
+                               site_content_url=new_site_content_url)
 t.signin()
 logger = Logger(u'create_site_sample.log')
 # Enable logging after sign-in to hide credentials
@@ -34,45 +35,49 @@
 groups_to_create = [u'Administrators', u'Executives', u'Managers', u'Line Level Workers']
 
 for group in groups_to_create:
-    t.create_group(group)
+    t.create_group(group_name=group)
 
 # Remove all permissions from Default Project
 time.sleep(4)
-default_proj = t.query_project(u'Default')
+default_proj = t.query_project(project_name_or_luid=u'Default')
 default_proj.lock_permissions()
 
 default_proj.clear_all_permissions() # This clears all, including the defaults
 
 
 # Add in any default permissions you'd like at this point
-admin_perms = default_proj.create_project_permissions_object_for_group(u'Administrators', role=u'Project Leader')
+admin_perms = default_proj.create_project_permissions_object_for_group(group_name_or_luid=u'Administrators',
+                                                                       role=u'Project Leader')
 default_proj.set_permissions_by_permissions_obj_list([admin_perms, ])
 
-admin_perms = default_proj.create_workbook_permissions_object_for_group(u'Administrators', role=u'Editor')
-admin_perms.set_capability(u'Download Full Data', u'Deny')
+admin_perms = default_proj.create_workbook_permissions_object_for_group(group_name_or_luid=u'Administrators',
+                                                                        role=u'Editor')
+admin_perms.set_capability(capability_name=u'Download Full Data', mode=u'Deny')
 default_proj.workbook_defaults.set_permissions_by_permissions_obj_list([admin_perms, ])
 
-admin_perms = default_proj.create_datasource_permissions_object_for_group(u'Administrators', role=u'Editor')
+admin_perms = default_proj.create_datasource_permissions_object_for_group(group_name_or_luid=u'Administrators',
+                                                                          role=u'Editor')
 default_proj.datasource_defaults.set_permissions_by_permissions_obj_list([admin_perms, ])
 
 # Change one of these
-new_perms = default_proj.create_project_permissions_object_for_group(u'Administrators', role=u'Publisher')
+new_perms = default_proj.create_project_permissions_object_for_group(group_name_or_luid=u'Administrators',
+                                                                     role=u'Publisher')
 default_proj.set_permissions_by_permissions_obj_list([new_perms, ])
 
 # Create Additional Projects
 projects_to_create = [u'Sandbox', u'Data Source Definitions', u'UAT', u'Finance', u'Real Financials']
 for project in projects_to_create:
-    t.create_project(project, no_return=True)
+    t.create_project(project_name=project, no_return=True)
 
 # Set any additional permissions on each project
 
 # Add Users
 users_to_add = [u'user_1', u'user_2', u'user_3']
 for user in users_to_add:
-    t.add_user(user, user, u'Publisher')
+    t.add_user(username=user, fullname=user, site_role=u'Publisher')
 
 time.sleep(3)
 # Add Users to Groups
-t.add_users_to_group(u'user_1', u'Managers')
-t.add_users_to_group(u'user_2', u'Administrators')
-t.add_users_to_group([u'user_2', u'user_3'], u'Executives')
+t.add_users_to_group(username_or_luid_s=u'user_1', group_name_or_luid=u'Managers')
+t.add_users_to_group(username_or_luid_s=u'user_2', group_name_or_luid=u'Administrators')
+t.add_users_to_group(username_or_luid_s=[u'user_2', u'user_3'], group_name_or_luid=u'Executives')

examples/extract_refresh_pre_10_3_sample.py

@@ -5,6 +5,7 @@
 
 # This script uses the tabcmd object to trigger an extract refresh
 # Please use the 10_3 script if on a 10.3+ version of Tableau Server, which uses the REST API commands
+# Should be totally deprecated at this point
 
 logger = Logger(u'extract_refresh.log')
 

examples/move_extracts_from_server_to_server.py

@@ -4,6 +4,12 @@
 from tableau_tools import *
 import time
 
+# This example is for a specific use case:
+# When the Originating Tableau Server can connect to the live data source to generate the new extract, but the
+# Destination Tableau Server cannot. This particular pattern downloads the updated TDSX file from the Originating Server
+# then pushes it to the Destination Server without saving credentials. With no credentials (and no Extract Refresh
+# scheduled), the version on the Destination Server will remain static until another version is pushed
+
 o_server = u'http://'
 o_username = u''
 o_password = u''
@@ -16,15 +22,15 @@
 d_password = u''
 d_site_content_url = u''
 
-t = TableauRestApiConnection26(o_server, o_username, o_password, o_site_content_url)
+t = TableauRestApiConnection28(o_server, o_username, o_password, o_site_content_url)
 t.signin()
 t.enable_logging(logger)
 downloaded_filename = u'File Name'
 wb_name_on_server = u'WB Name on Server'
 proj_name = u'Default'
 t.download_workbook(u'WB Name on Server', downloaded_filename, proj_name_or_luid=proj_name)
 
-d = TableauRestApiConnection25(d_server, d_username, d_password, d_site_content_url)
+d = TableauRestApiConnection28(d_server, d_username, d_password, d_site_content_url)
 d.signin()
 d.enable_logging(logger)
 proj = d.query_project(u'Default')

examples/permissions_auditing.py

@@ -3,6 +3,8 @@
 from tableau_tools import *
 import csv
 
+# There are differences between Python 2.7 and Python 3, so this may not run on Python 3 yet
+
 username = ''
 password = ''
 server = 'http://localhost'
@@ -36,7 +38,7 @@
     output_writer.writerow(headers)
 
     for site_content_url in site_content_urls:
-        t = TableauRestApiConnection25(server, username, password, site_content_url)
+        t = TableauRestApiConnection28(server, username, password, site_content_url)
         t.enable_logging(logger)
         t.signin()
         projects = t.query_projects()