Skip to content

Commit

Permalink
re-impl docs in docusaurus
Browse files Browse the repository at this point in the history
  • Loading branch information
renardeinside committed Jan 13, 2025
1 parent 6d39f4f commit dde2508
Show file tree
Hide file tree
Showing 37 changed files with 1,102 additions and 987 deletions.
130 changes: 0 additions & 130 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,136 +21,6 @@ See [contributing instructions](CONTRIBUTING.md) to help improve this project.

[![build](https://github.com/databrickslabs/ucx/actions/workflows/push.yml/badge.svg)](https://github.com/databrickslabs/ucx/actions/workflows/push.yml) [![codecov](https://codecov.io/github/databrickslabs/ucx/graph/badge.svg?token=p0WKAfW5HQ)](https://codecov.io/github/databrickslabs/ucx) ![linesofcode](https://aschey.tech/tokei/github/databrickslabs/ucx?category=code)

<!-- TOC -->
* [Databricks Labs UCX](#databricks-labs-ucx)
* [Installation](#installation)
* [Installation requirements](#installation-requirements)
* [Authenticate Databricks CLI](#authenticate-databricks-cli)
* [Install UCX](#install-ucx)
* [Installation resources](#installation-resources)
* [Installation folder](#installation-folder)
* [Readme notebook](#readme-notebook)
* [Debug notebook](#debug-notebook)
* [Debug logs](#debug-logs)
* [Installation configuration](#installation-configuration)
* [Advanced installation options](#advanced-installation-options)
* [Force install over existing UCX](#force-install-over-existing-ucx)
* [Installing UCX on all workspaces within a Databricks account](#installing-ucx-on-all-workspaces-within-a-databricks-account)
* [Installing UCX with company hosted PYPI mirror](#installing-ucx-with-company-hosted-pypi-mirror)
* [Upgrading UCX for newer versions](#upgrading-ucx-for-newer-versions)
* [Uninstall UCX](#uninstall-ucx)
* [Migration process](#migration-process)
* [Table migration process](#table-migration-process)
* [Table mapping](#table-mapping)
* [Step 1 : Create the mapping file](#step-1--create-the-mapping-file)
* [Step 2: Update the mapping file](#step-2-update-the-mapping-file)
* [Data access](#data-access)
* [Step 1 : Map cloud principals to cloud storage locations](#step-1--map-cloud-principals-to-cloud-storage-locations)
* [Step 2 : Create or modify cloud principals and credentials](#step-2--create-or-modify-cloud-principals-and-credentials)
* [Step 3: Create the "uber" Principal](#step-3-create-the-uber-principal)
* [New Unity Catalog resources](#new-unity-catalog-resources)
* [Step 0: Attach a metastore](#step-0-attach-a-metastore)
* [Step 1: Create external Locations](#step-1-create-external-locations)
* [Step 2: Create Catalogs and Schemas](#step-2-create-catalogs-and-schemas)
* [Migrate Hive metastore data objects](#migrate-hive-metastore-data-objects)
* [Odds and Ends](#odds-and-ends)
* [Skip migrating schemas, tables or views](#skip-migrating-schemas-tables-or-views)
* [Move data objects](#move-data-objects)
* [Alias data objects](#alias-data-objects)
* [Revert migrated data objects](#revert-migrated-data-objects)
* [Workflows](#workflows)
* [Assessment workflow](#assessment-workflow)
* [Group migration workflow](#group-migration-workflow)
* [Table migration workflows](#table-migration-workflows)
* [Migrate tables](#migrate-tables)
* [Migrate external Hive SerDe tables](#migrate-external-hive-serde-tables)
* [Migrate external tables CTAS](#migrate-external-tables-ctas)
* [Post-migration data reconciliation workflow](#post-migration-data-reconciliation-workflow)
* [[LEGACY] Scan tables in mounts Workflow](#legacy-scan-tables-in-mounts-workflow)
* [<b>Always run this workflow AFTER the assessment has finished</b>](#balways-run-this-workflow-after-the-assessment-has-finishedb)
* [[LEGACY] Migrate tables in mounts Workflow](#legacy-migrate-tables-in-mounts-workflow)
* [[EXPERIMENTAL] Migration Progress Workflow](#experimental-migration-progress-workflow)
* [Dashboards](#dashboards)
* [Linter message codes](#linter-message-codes)
* [`cannot-autofix-table-reference`](#cannot-autofix-table-reference)
* [`catalog-api-in-shared-clusters`](#catalog-api-in-shared-clusters)
* [`changed-result-format-in-uc`](#changed-result-format-in-uc)
* [`direct-filesystem-access-in-sql-query`](#direct-filesystem-access-in-sql-query)
* [`direct-filesystem-access`](#direct-filesystem-access)
* [`dependency-not-found`](#dependency-not-found)
* [`jvm-access-in-shared-clusters`](#jvm-access-in-shared-clusters)
* [`legacy-context-in-shared-clusters`](#legacy-context-in-shared-clusters)
* [`not-supported`](#not-supported)
* [`notebook-run-cannot-compute-value`](#notebook-run-cannot-compute-value)
* [`python-udf-in-shared-clusters`](#python-udf-in-shared-clusters)
* [`rdd-in-shared-clusters`](#rdd-in-shared-clusters)
* [`spark-logging-in-shared-clusters`](#spark-logging-in-shared-clusters)
* [`sql-parse-error`](#sql-parse-error)
* [`sys-path-cannot-compute-value`](#sys-path-cannot-compute-value)
* [`table-migrated-to-uc`](#table-migrated-to-uc)
* [`to-json-in-shared-clusters`](#to-json-in-shared-clusters)
* [`unsupported-magic-line`](#unsupported-magic-line)
* [Utility commands](#utility-commands)
* [`logs` command](#logs-command)
* [`ensure-assessment-run` command](#ensure-assessment-run-command)
* [`update-migration-progress` command](#update-migration-progress-command)
* [`repair-run` command](#repair-run-command)
* [`workflows` command](#workflows-command)
* [`open-remote-config` command](#open-remote-config-command)
* [`installations` command](#installations-command)
* [`report-account-compatibility` command](#report-account-compatibility-command)
* [`export-assessment` command](#export-assessment-command)
* [Metastore related commands](#metastore-related-commands)
* [`show-all-metastores` command](#show-all-metastores-command)
* [`assign-metastore` command](#assign-metastore-command)
* [`create-ucx-catalog` command](#create-ucx-catalog-command)
* [Table migration commands](#table-migration-commands)
* [`principal-prefix-access` command](#principal-prefix-access-command)
* [Access for AWS S3 Buckets](#access-for-aws-s3-buckets)
* [Access for Azure Storage Accounts](#access-for-azure-storage-accounts)
* [`create-missing-principals` command (AWS Only)](#create-missing-principals-command-aws-only)
* [`delete-missing-principals` command (AWS Only)](#delete-missing-principals-command-aws-only)
* [`create-uber-principal` command](#create-uber-principal-command)
* [`migrate-credentials` command](#migrate-credentials-command)
* [`validate-external-locations` command](#validate-external-locations-command)
* [`migrate-locations` command](#migrate-locations-command)
* [`create-table-mapping` command](#create-table-mapping-command)
* [`skip` command](#skip-command)
* [`unskip` command](#unskip-command)
* [`create-catalogs-schemas` command](#create-catalogs-schemas-command)
* [`assign-owner-group` command](#assign-owner-group-command)
* [`migrate-tables` command](#migrate-tables-command)
* [`revert-migrated-tables` command](#revert-migrated-tables-command)
* [`move` command](#move-command)
* [`alias` command](#alias-command)
* [Code migration commands](#code-migration-commands)
* [`lint-local-code` command](#lint-local-code-command)
* [`migrate-local-code` command](#migrate-local-code-command)
* [`migrate-dbsql-dashboards` command](#migrate-dbsql-dashboards-command)
* [`revert-dbsql-dashboards` command](#revert-dbsql-dashboards-command)
* [Cross-workspace installations](#cross-workspace-installations)
* [`sync-workspace-info` command](#sync-workspace-info-command)
* [`manual-workspace-info` command](#manual-workspace-info-command)
* [`create-account-groups` command](#create-account-groups-command)
* [`validate-groups-membership` command](#validate-groups-membership-command)
* [`validate-table-locations` command](#validate-table-locations-command)
* [`cluster-remap` command](#cluster-remap-command)
* [`revert-cluster-remap` command](#revert-cluster-remap-command)
* [`upload` command](#upload-command)
* [`download` command](#download-command)
* [`join-collection` command](#join-collection-command)
* [collection eligible command](#collection-eligible-command)
* [Common Challenges and the Solutions](#common-challenges-and-the-solutions)
* [Network Connectivity Issues](#network-connectivity-issues)
* [Insufficient Privileges](#insufficient-privileges)
* [Version Issues](#version-issues)
* [Authentication Issues](#authentication-issues)
* [Multiple Profiles in Databricks CLI](#multiple-profiles-in-databricks-cli)
* [Workspace has an external Hive Metastore (HMS)](#workspace-has-an-external-hive-metastore-hms)
* [Verify the Installation](#verify-the-installation)
* [Star History](#star-history)
* [Project Support](#project-support)
<!-- TOC -->

# Installation

Expand Down
10 changes: 7 additions & 3 deletions CONTRIBUTING.md → docs/ucx/docs/dev/contributing.mdx
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
# Contributing
---
sidebar_position: 4
---

# Contributing to UCX

## First Principles

Expand Down Expand Up @@ -214,7 +218,7 @@ with the [Python plugin (Community Edition)](https://plugins.jetbrains.com/plugi
IntelliJ CE, then it would work in PyCharm. Debugging capabilities are essential for troubleshooting and diagnosing issues during
development. Please make sure that your test setup allows for easy debugging by following best practices.

![debugging tests](docs/debugging-tests.gif)
![debugging tests](/img/debugging-tests.gif)

Adhering to these guidelines ensures that our integration tests are robust, efficient, and easily maintainable. This,
in turn, contributes to the overall reliability and quality of our software.
Expand Down Expand Up @@ -263,7 +267,7 @@ HATCH_PYTHON="$(which python3.10)" make clean dev test
```

Configure your IDE to use `.venv/bin/python` from the virtual environment when developing the project:
![IDE Setup](docs/hatch-intellij.gif)
![IDE Setup](/img/hatch-intellij.gif)


Verify installation with
Expand Down
65 changes: 65 additions & 0 deletions docs/ucx/docs/dev/docs_authoring.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
import Admonition from '@theme/Admonition';

# Authoring Documentation

This document provides guidelines for writing documentation for the UCX project.


## Tech Stack

The UCX documentation is built using [Docusaurus](https://docusaurus.io/), a modern static site generator. Docusaurus is a project of Facebook Open Source and is used by many open-source projects to build their documentation websites.
We also use [MDX](https://mdxjs.com/) to write markdown files that include JSX components. This allows us to write markdown files with embedded React components.
For styling, we use [Tailwind CSS](https://tailwindcss.com/), a utility-first CSS framework for rapidly building custom designs.

## Writing Documentation

Most of the documentation is written in markdown files with the `.mdx` extension.
The markdown files are located in the `docs` directory of the UCX project.

## Prerequisites

Before you start writing documentation, make sure you have the following tools installed on your machine:
- [Node.js](https://nodejs.org/en/)
- [Yarn](https://yarnpkg.com/)

On macOS, you can install Node.js and Yarn using [Homebrew](https://brew.sh/):

```bash
brew install node
npm install --global yarn
```


## Setup

To set up the documentation locally, follow these steps:

1. Clone the UCX repository
2. Navigate to the `docs/ucx` directory:

```bash
cd docs/ucx
```

3. Start the development server:

```bash
yarn start
```

4. Open your browser and navigate to `http://localhost:3000/` to view the documentation site.


## Adding images

To add images to your documentation, place the image files in the `static/img` directory.

To include an image in your markdown file, use the following syntax:

```markdown
![Alt text](/img/image-name.png)
```

<Admonition type="tip" title="Tip" emoji="💡">
Images support zooming features out of the box.
</Admonition>
7 changes: 7 additions & 0 deletions docs/ucx/docs/dev/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
sidebar_position: 4
---

# Contributing to UCX

This section is for contributors to the UCX toolkit. It contains information on how to contribute to the toolkit, including how to submit issues, pull requests, and how to contribute to the documentation.
Loading

0 comments on commit dde2508

Please sign in to comment.