Transport refactor

changelog/61161.fixed

@@ -0,0 +1 @@
+Re-factor transport to make them more plug-able

doc/topics/channels/index.rst

@@ -0,0 +1,32 @@
+.. _channels:
+
+=============
+Salt Channels
+=============
+
+One of the fundamental features of Salt is remote execution. Salt has two basic
+"channels" for communicating with minions. Each channel requires a client
+(minion) and a server (master) implementation to work within Salt. These pairs
+of channels will work together to implement the specific message passing
+required by the channel interface. Channels use :ref:`Transports <transports>`
+for sending and receiving messages.
+
+
+Pub Channel
+===========
+The pub (or pubish) channel is how a master sends a job (payload) to a
+minion. This is a basic pub/sub paradigm, which has specific targeting semantics.
+All data which goes across the publish system should be encrypted such that only
+members of the Salt cluster can decrypt the published payloads.
+
+
+Req Channel
+===========
+The req channel is how the minions send data to the master. This interface is
+primarily used for fetching files and returning job returns. The req channels
+have two basic interfaces when talking to the master. ``send`` is the basic
+method that guarantees the message is encrypted at least so that only minions
+attached to the same master can read it-- but no guarantee of minion-master
+confidentiality, whereas the ``crypted_transfer_decode_dictentry`` method does
+guarantee minion-master confidentiality. The req channel is also used by the
+salt cli to publish jobs to the master.

doc/topics/configuration/index.rst

@@ -30,6 +30,7 @@ secure and troubleshoot, and how to perform many other administrative tasks.
     ../tutorials/cron
     ../hardening
     ../../security/index
+    ../channels/index
     ../transports/index
     ../master_tops/index
     ../../ref/returners/index

doc/topics/transports/index.rst

@@ -1,33 +1,37 @@
- .. _transports:
+.. _transports:
 
 ==============
 Salt Transport
 ==============
 
-One of fundamental features of Salt is remote execution. Salt has two basic
-"channels" for communicating with minions. Each channel requires a
-client (minion) and a server (master) implementation to work within Salt. These
-pairs of channels will work together to implement the specific message passing
-required by the channel interface.
-
-
-Pub Channel
-===========
-The pub channel, or publish channel, is how a master sends a job (payload) to a
-minion. This is a basic pub/sub paradigm, which has specific targeting semantics.
-All data which goes across the publish system should be encrypted such that only
-members of the Salt cluster can decrypt the publishes.
-
-
-Req Channel
-===========
-The req channel is how the minions send data to the master. This interface is
-primarily used for fetching files and returning job returns. The req channels
-have two basic interfaces when talking to the master. ``send`` is the basic
-method that guarantees the message is encrypted at least so that only minions
-attached to the same master can read it-- but no guarantee of minion-master
-confidentiality, whereas the ``crypted_transfer_decode_dictentry`` method does
-guarantee minion-master confidentiality.
+
+Transports in Salt are used by :ref:`Channels <channels>` to send messages between Masters, Minions,
+and the Salt CLI. Transports can be brokerless or brokered. There are two types
+of server / client implementations needed to implement a channel.
+
+
+Publish Server
+==============
+
+The publish server implements a publish / subscribe paradigm and is used by
+Minions to receive jobs from Masters.
+
+Publish Client
+==============
+
+The publish client subscribes to, and receives messages from a Publish Server.
+
+
+Request Server
+==============
+
+The request server implements a request / reply paradigm. Every request sent by
+the client must recieve exactly one reply.
+
+Request Client
+==============
+
+The request client sends requests to a Request Server and recieves a reply message.
 
 
 .. toctree::

doc/topics/transports/tcp.rst

@@ -2,7 +2,7 @@
 TCP Transport
 =============
 
-The tcp transport is an implementation of Salt's channels using raw tcp sockets.
+The tcp transport is an implementation of Salt's transport using raw tcp sockets.
 Since this isn't using a pre-defined messaging library we will describe the wire
 protocol, message semantics, etc. in this document.
 
@@ -83,17 +83,17 @@ Crypto
 The current implementation uses the same crypto as the ``zeromq`` transport.
 
 
-Pub Channel
-===========
-For the pub channel we send messages without "message ids" which the remote end
-interprets as a one-way send.
+Publish Server and Client
+=========================
+For the publish server and client we send messages without "message ids" which
+the remote end interprets as a one-way send.
 
 .. note::
 
     As of today we send all publishes to all minions and rely on minion-side filtering.
 
 
-Req Channel
-===========
-For the req channel we send messages with a "message id". This "message id" allows
-us to multiplex messages across the socket.
+Request Server and Client
+=========================
+For the request server and client we send messages with a "message id". This
+"message id" allows us to multiplex messages across the socket.

doc/topics/transports/zeromq.rst

@@ -1,26 +1,27 @@
 ================
-Zeromq Transport
+ZeroMQ Transport
 ================
 
 .. note::
 
-    Zeromq is the current default transport within Salt
+    ZeroMQ is the current default transport within Salt
 
-Zeromq is a messaging library with bindings into many languages. Zeromq implements
+ZeroMQ is a messaging library with bindings into many languages. ZeroMQ implements
 a socket interface for message passing, with specific semantics for the socket type.
 
 
-Pub Channel
-===========
-The pub channel is implemented using zeromq's pub/sub sockets. By default we don't
-use zeromq's filtering, which means that all publish jobs are sent to all minions
-and filtered minion side. Zeromq does have publisher side filtering which can be
-enabled in salt using :conf_master:`zmq_filtering`.
+Publish Server and Client
+=========================
+The publish server and client are implemented using ZeroMQ's pub/sub sockets. By
+default we don't use ZeroMQ's filtering, which means that all publish jobs are
+sent to all minions and filtered minion side. ZeroMQ does have publisher side
+filtering which can be enabled in salt using :conf_master:`zmq_filtering`.
 
 
-Req Channel
-===========
-The req channel is implemented using zeromq's req/rep sockets. These sockets
-enforce a send/recv pattern, which forces salt to serialize messages through these
-socket pairs. This means that although the interface is asynchronous on the minion
-we cannot send a second message until we have received the reply of the first message.
+Request Server and Client
+=========================
+The request server and client are implemented using ZeroMQ's req/rep sockets.
+These sockets enforce a send/recv pattern, which forces salt to serialize
+messages through these socket pairs. This means that although the interface is
+asynchronous on the minion we cannot send a second message until we have
+received the reply of the first message.

kitchen.yml

@@ -38,7 +38,7 @@ provisioner:
     - 139
   max_retries: 2
   remote_states:
-    name: git://github.com/saltstack/salt-jenkins.git
+    name: https://github.com/saltstack/salt-jenkins.git
     branch: master
     repo: git
     testingdir: /testing

salt/auth/__init__.py

@@ -18,11 +18,11 @@
 import time
 from collections.abc import Iterable, Mapping
 
+import salt.channel.client
 import salt.config
 import salt.exceptions
 import salt.loader
 import salt.payload
-import salt.transport.client
 import salt.utils.args
 import salt.utils.dictupdate
 import salt.utils.files
@@ -511,7 +511,7 @@ def _send_token_request(self, load):
             salt.utils.zeromq.ip_bracket(self.opts["interface"]),
             str(self.opts["ret_port"]),
         )
-        with salt.transport.client.ReqChannel.factory(
+        with salt.channel.client.ReqChannel.factory(
             self.opts, crypt="clear", master_uri=master_uri
         ) as channel:
             return channel.send(load)