Merge pull request percona#4537 from patrickbirch/ps-8.0-7960

PS-7960 Document percona-202104 and percona-202105 new functionality(…

doc/source/index.rst

@@ -176,6 +176,7 @@ Percona MyRocks
 
    MyRocks Introduction <myrocks/index>
    MyRocks Installation <myrocks/install>
+   MyRocks Supported Features <myrocks/added-features>
    MyRocks Limitations <myrocks/limitations>
    MyRocks Differences <myrocks/differences>
    MyRocks Information Schema Tables <myrocks/information-schema-tables>

doc/source/myrocks/added-features.rst

@@ -0,0 +1,21 @@
+.. _added-features:
+
+================================================================================
+Updated Supported Features
+================================================================================
+
+The following is a list of the latest supported features:
+
+* **Percona Server for MySQL** 8.0.27-18 adds support for ``SELECT FOR UPDATE SKIP LOCKED/NOWAIT``. The transaction isolation level must be ``READ COMMITTED``.
+
+* **Percona Server for MySQL** 8.0.27-18 adds the ability to cancel ongoing manual compactions. The cancel methods are the following:
+
+  * Using either Control+C (from a session) or KILL (from another session) for client sessions running manual compactions by ``SET GLOBAL rocksdb_compact_cf (variable)``.
+  * Using a global variable ``rocksdb_cancel_manual_compactions`` to cancel all ongoing manual compactions.
+
+* **Percona Server for MySQL** 8.0.23-14 adds supported for `Generated Columns <https://dev.mysql.com/doc/refman/8.0/en/create-table-generated-columns.html>`_ and index are supported. Generated columns are not supported in versions earlier than 8.0.23-14.
+
+* **Percona Server for MySQL** 8.0.23-14 adds support for `explicit DEFAULT value expressions <https://dev.mysql.com/doc/refman/8.0/en/data-type-defaults.html>`__. From version 8.0.13-3 to version 8.0.22-13, MyRocks did not support these expressions.
+
+
+

doc/source/myrocks/limitations.rst

@@ -6,21 +6,20 @@ MyRocks Limitations
 
 The MyRocks storage engine lacks the following features compared to InnoDB:
 
-   * `Online DDL <https://dev.mysql.com/doc/refman/8.0/en/innodb-online-ddl.html>`_ is not supported due to the lack of atomic DDL support. 
-       - There is no ``ALTER TABLE ... ALGORITHM=INSTANT`` functionality
-       - A partition management operation only supports the ``COPY`` algorithms, which rebuilds the partition table and moves the data based on the new ``PARTITION ... VALUE`` definition. In the case of ``DROP PARTITION``, the data not moved to another partition is deleted.
-   * `ALTER TABLE .. EXCHANGE PARTITION <https://dev.mysql.com/doc/refman/8.0/en/partitioning-management-exchange.html>`_. 
-   * `SAVEPOINT <https://dev.mysql.com/doc/refman/8.0/en/savepoint.html>`_
-   * `Transportable tablespace <https://dev.mysql.com/doc/refman/8.0/en/innodb-table-import.html>`_
-   * `Foreign keys <https://dev.mysql.com/doc/refman/8.0/en/create-table-foreign-keys.html>`_
-   * `Spatial indexes <https://dev.mysql.com/doc/refman/8.0/en/using-spatial-indexes.html>`_
-   * `Fulltext indexes <https://dev.mysql.com/doc/refman/8.0/en/innodb-fulltext-index.html>`_
-   * `Gap locks <https://dev.mysql.com/doc/refman/8.0/en/innodb-locking.html#innodb-gap-locks>`_
-   * `Group Replication <https://dev.mysql.com/doc/refman/8.0/en/group-replication.html>`_
-   * `Partial Update of LOB in InnoDB <https://mysqlserverteam.com/mysql-8-0-optimizing-small-partial-update-of-lob-in-innodb/>`_
+* `Online DDL <https://dev.mysql.com/doc/refman/8.0/en/innodb-online-ddl.html>`_ is not supported due to the lack of atomic DDL support.
+    - There is no ``ALTER TABLE ... ALGORITHM=INSTANT`` functionality
+    - A partition management operation only supports the ``COPY`` algorithms, which rebuilds the partition table and moves the data based on the new ``PARTITION ... VALUE`` definition. In the case of ``DROP PARTITION``, the data not moved to another partition is deleted.
+* `ALTER TABLE .. EXCHANGE PARTITION <https://dev.mysql.com/doc/refman/8.0/en/partitioning-management-exchange.html>`_.
+* `SAVEPOINT <https://dev.mysql.com/doc/refman/8.0/en/savepoint.html>`_
+* `Transportable tablespace <https://dev.mysql.com/doc/refman/8.0/en/innodb-table-import.html>`_
+* `Foreign keys <https://dev.mysql.com/doc/refman/8.0/en/create-table-foreign-keys.html>`_
+* `Spatial indexes <https://dev.mysql.com/doc/refman/8.0/en/using-spatial-indexes.html>`_
+* `Fulltext indexes <https://dev.mysql.com/doc/refman/8.0/en/innodb-fulltext-index.html>`_
+* `Gap locks <https://dev.mysql.com/doc/refman/8.0/en/innodb-locking.html#innodb-gap-locks>`_
+* `Group Replication <https://dev.mysql.com/doc/refman/8.0/en/group-replication.html>`_
+* `Partial Update of LOB in InnoDB <https://mysqlserverteam.com/mysql-8-0-optimizing-small-partial-update-of-lob-in-innodb/>`_
 
 
-As of |Percona Server| version 8.0.23-14, `Generated Columns <https://dev.mysql.com/doc/refman/8.0/en/create-table-generated-columns.html>`_ and index are supported. Generated columns are not supported in versions earlier than 8.0.23-14.
 
 You should also consider the following:
 
@@ -43,17 +42,8 @@ You should also consider the following:
   then descending scan will be faster and ascending scan will be slower.
   Note that InnoDB also imposes a cost
   when the index is scanned in the opposite order.
-
-* MyRocks does not support operating as either a source or a replica
-  in any replication topology that is not exclusively row-based.
-  Statement-based and mixed-format binary logging is not supported.
-  For more information, see `Replication Formats
-  <https://dev.mysql.com/doc/refman/8.0/en/replication-formats.html>`_.
-
-* As of 8.0.17, InnoDB supports `multi-valued indexes <https://dev.mysql.com/doc/refman/8.0/en/create-index.html#create-index-multi-valued>`__. MyRocks does not support this feature.
-
-* As of 8.0.17, InnoDB supports the use of the `Clone Plugin <https://dev.mysql.com/doc/refman/8.0/en/clone-plugin.html>`__ and the Clone Plugin API. MyRocks tables do not support either these features.
-
+
+
 * When converting from large MyISAM/InnoDB tables, either by using the
   ``ALTER`` or ``INSERT INTO SELECT`` statements it's recommended that you
   check the :ref:`Data loading <myrocks_data_loading>` documentation and
@@ -71,8 +61,8 @@ You should also consider the following:
 
    .. warning::
 
-      If you are loading large data without enabling :variable:`rocksdb_bulk_load`
-      or :variable:`rocksdb_commit_in_the_middle`, please make sure transaction
+      If you are loading large data without enabling `rocksdb_bulk_load`
+      or `rocksdb_commit_in_the_middle`, please make sure transaction
       size is small enough. All modifications of the ongoing transactions are
       kept in memory.
 
@@ -84,40 +74,45 @@ You should also consider the following:
      |MySQL| Documentation: Preparing Your Installation for Upgrade
         https://dev.mysql.com/doc/refman/8.0/en/upgrade-prerequisites.html
 
-* The |MyRocks| storage engine does not support the |sql.no-wait| and
-  |sql.skip-locked| modifiers introduced in the |InnoDB| storage
-  engine with |MySQL| 8.0.
-
 .. include:: ../.res/replace.concept.txt
 
-* |Percona Server| 8.0 and Unicode 9.0.0 standards have defined a change in the
+* **Percona Server for MySQL** 8.0 and Unicode 9.0.0 standards have defined a change in the
   handling of binary collations. These collations are handled as NO PAD,
   trailing spaces are included in key comparisons. A binary collation comparison
   may result in two unique rows inserted and does not generate a`DUP_ENTRY`
   error. MyRocks key encoding and comparison does not account for this
   character set attribute.
 
-*  As of |Percona Server| version 8.0.23-14, MyRocks supports `explicit DEFAULT value expressions <https://dev.mysql.com/doc/refman/8.0/en/data-type-defaults.html>`__. From version 8.0.13-3 to version 8.0.22-13, MyRocks did not support these expressions.
 
-* |Percona Server| 8.0.16 does not support encryption for the MyRocks
-  storage engine. At this time, during an ``ALTER TABLE`` operation, MyRocks mistakenly detects all InnoDB tables as encrypted. Therefore, any attempt to ``ALTER`` an InnoDB table to MyRocks fails.
+Not Supported on MyRocks
+============================================
 
-  As a workaround, we recommend a manual move of the table. The following  steps are the same as the ``ALTER TABLE ... ENGINE=...`` process:
+MyRocks does not support the following:
 
-* Use ``SHOW CREATE TABLE ...`` to return the InnoDB table definition.
+* Operating as either a source or a replica
+  in any replication topology that is not exclusively row-based.
+  Statement-based and mixed-format binary logging is not supported.
+  For more information, see `Replication Formats
+  <https://dev.mysql.com/doc/refman/8.0/en/replication-formats.html>`_.
 
-* With the table definition as the source, perform
-  a ``CREATE TABLE ... ENGINE=RocksDB``.
+* Using `multi-valued indexes <https://dev.mysql.com/doc/refman/8.0/en/create-index.html#create-index-multi-valued>`__. Implemented in **Percona Server for MySQL** 8.0.17, InnoDB supports this feature.
 
-* In the new table, use ``INSERT INTO <new table> SELECT * FROM <old table>``.
+* Using `spatial data types <https://dev.mysql.com/doc/refman/8.0/en/spatial-type-overview.html>`__ .
 
-  .. note::
+* Using the `Clone Plugin <https://dev.mysql.com/doc/refman/8.0/en/clone-plugin.html>`__ and the Clone Plugin API.  As of **Percona Server for MySQL** 8.0.17, InnoDB supports either these features.
 
-    With MyRocks and with large tables, it is recommended to set the session variable ``rocksdb_bulk_load=1`` during the load to prevent running out of memory. This recommendation is because of the MyRocks large transaction limitation.
+* Using encryption in tables. At this time, during an ``ALTER TABLE`` operation, MyRocks mistakenly detects all InnoDB tables as encrypted. Therefore, any attempt to ``ALTER`` an InnoDB table to MyRocks fails.
 
-  .. seealso::
+  As a workaround, we recommend a manual move of the table. The following  steps are the same as the ``ALTER TABLE ... ENGINE=...`` process:
 
-    MyRocks Data Loading
-    https://www.percona.com/doc/percona-server/8.0/myrocks/data_loading.html
+  1. Use ``SHOW CREATE TABLE ...`` to return the InnoDB table definition.
+
+  #. With the table definition as the source, perform a ``CREATE TABLE ... ENGINE=RocksDB``.
+
+  #. In the new table, use ``INSERT INTO <new table> SELECT * FROM <old table>``.
+
+    .. note::
+
+        With MyRocks and with large tables, it is recommended to set the session variable ``rocksdb_bulk_load=1`` during the load to prevent running out of memory. This recommendation is because of the MyRocks large transaction limitation. For more information, see `MyRocks Data Loading <https://www.percona.com/doc/percona-server/8.0/myrocks/data_loading.html>`__
 
-* MySQL has `spatial data types <https://dev.mysql.com/doc/refman/8.0/en/spatial-type-overview.html>`__ . These data types are not supported by MyRocks.
+

doc/source/myrocks/variables.rst

@@ -79,6 +79,10 @@ Also, all variables can exist in one or both of the following scopes:
      - Yes
      - Yes
      - Global
+   * - :variable:`rocksdb_bulk_load_partial_index`
+     - Yes
+     - Yes
+     - Local
    * - :variable:`rocksdb_block_restart_interval`
      - Yes
      - No
@@ -119,6 +123,10 @@ Also, all variables can exist in one or both of the following scopes:
      - Yes
      - No
      - Global
+   * - :variable:`rocksdb_cancel_manual_compactions`
+     - Yes
+     - Yes
+     - Global
    * - :variable:`rocksdb_checksums_pct`
      - Yes
      - Yes
@@ -443,6 +451,10 @@ Also, all variables can exist in one or both of the following scopes:
      - Yes
      - No
      - Global
+   * - :variable:`rocksdb_partial_index_sort_max_mem`
+     - Yes
+     - Yes
+     - Local
    * - :variable:`rocksdb_pause_background_work`
      - Yes
      - Yes
@@ -919,6 +931,17 @@ for example, when setting up a new MyRocks instance from a MySQL dump.
 Enabling this variable will also enable
 the :variable:`rocksdb_commit_in_the_middle` variable.
 
+.. variable:: rocksdb_bulk_load_partial_index
+
+  :version 8.0.27-17: Implemented
+  :cli: ``--rocksdb-bulk-load-partial-index``
+  :dyn: Yes
+  :scope: Local
+  :vartype: Boolean
+  :default: ``ON``
+
+Materialize partial index during bulk load, instead of leaving the index empty.
+
 .. variable:: rocksdb_bulk_load_size
 
   :cli: ``--rocksdb-bulk-load-size``
@@ -975,6 +998,17 @@ Enabled by default.
 If you disable this feature,
 RocksDB will allocate additional memory to maintain these data blocks.
 
+.. variable:: rocksdb_cancel_manual_compactions
+
+  :version 8.0.27-17: Implemented
+  :cli: ``--rocksdb-cancel-manual-compactions``
+  :dyn: Yes
+  :scope: Global
+  :vartype: Boolean
+  :default: ``OFF``
+
+Cancels all ongoing manual compactions.
+
 .. variable:: rocksdb_checksums_pct
 
   :cli: ``--rocksdb-checksums-pct``
@@ -2098,6 +2132,17 @@ Specifies whether MyRocks should re-read the data file
 as soon as it is created to verify correctness.
 Enabled by default.
 
+.. variable:: rocksdb_partial_index_sort_max_mem
+
+  :version 8.0.27-17: Implemented
+  :cli: ``--rocksdb-partial-index-sort-max-mem``
+  :dyn: Yes
+  :scope: Local
+  :vartype: Unsigned Integer
+  :default: ``0``
+
+Maximum memory to use when sorting an unmaterialized group for partial indexes. The 0(zero) value is defined as no limit.
+
 .. variable:: rocksdb_pause_background_work
 
   :cli: ``--rocksdb-pause-background-work``