.. _internals-fileserver-client: The Salt Fileserver and Client ============================== Introduction ------------ Salt has a modular fileserver, and multiple client classes which are used to interact with it. This page serves as a developer's reference, to help explain how the fileserver and clients both work. Fileserver ---------- The fileserver is not a daemon, so the fileserver and client are not a true server and client in the traditional sense. Instead, the fileserver is simply a class (``salt.fileserver.Fileserver``), located in `salt/fileserver/__init__.py`_. This class has access to the configured fileserver backends via a loader instance, referenced as ``self.servers``. When a request comes in from the fileclient, it will ultimately result in a ``Fileserver`` class function being run. The functions in this class will run corresponding functions in the configured fileserver backends to perform the requested action. So, in summary: 1. A fileclient class makes a request... 2. which triggers the fileserver to run a function... 3. which runs a named function in each of the configured backends. Not all of the functions will always execute on every configured backend. For instance, the ``find_file`` function in the fileserver will stop when it finds a match, so if it finds a match for the desired path in the first configured backend, it won't proceed and try to find the file in the next backend in the list. Additionally, not all backends implement all functions in the ``salt.fileserver.Fileserver`` class. For instance, there is a function called ``update``, which exists to update remote fileservers such as the ``git``, ``hg``, and ``svn`` backends. This action has no use however in the ``roots`` backend, so it is simply not implemented there, and thus the ``roots`` backend will be skipped if the ``update`` function is run on the fileserver. Backends for the fileserver are located in `salt/fileserver/`_ (the files not named ``__init__.py``). .. _`salt/fileserver/__init__.py`: https://github.com/saltstack/salt/tree/develop/salt/fileserver/__init__.py .. _`salt/fileserver/`: https://github.com/saltstack/salt/tree/develop/salt/fileserver Fileclient ---------- There are three fileclient classes: salt.fileclient.RemoteClient ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This client is used when :conf_minion:`file_client` is set to ``remote``. This is how minions request files from the master. Functions in this client will craft a payload and send it to the master via the transport channel. This is the same way that the minion asks the minion to do other things, such as updating and requesting data from the mine. The payload will be a dictionary with a key called ``cmd``, and other values as needed. Payloads sent via the transport channel are processed my an MWorker instance on the master, and the MWorker's ``_handle_aes()`` function will execute the command. The command will be a function attribute of the ``salt.master.AESFuncs`` class. The AESFuncs class' ``__setup_fileserver()`` function instantiates a ``salt.fileserver.Fileserver`` instance and maps its member functions to AESFuncs attributes. This is what makes the fileserver functions available remotely. The result of the function is returned back through the transport channel to the minion. Transporting files is done in chunks, the size of which is decided by the ``file_buffer_size`` config option. If you look at the ``serve_file()`` function in any of the fileserver backends, you can see how the ``loc`` value in the payload determines the offset so that an intermediate chunk of the file can be served. The RemoteClient's ``get_file()`` function will loop until the end of the file is reached, retrieving one chunk at a time. salt.fileclient.FSClient ~~~~~~~~~~~~~~~~~~~~~~~~ This client is used when :conf_minion:`file_client` is set to ``local``. This is how masterless minions request files. This class inherits from the RemoteClient, but instead of using a transport channel (zmq, tcp, etc.), it uses a "fake" transport channel (``salt.fileserver.FSChan``), which implements its own ``send()`` function. Thus, when a function that the FSClient inherits from the RemoteClient runs ``self.channel.send()``, it's actually calling ``salt.fileserver.FSChan.send()``, which calls corresponding functions in the ``salt.fileserver.Fileserver()`` class. The result is that local file requests use the same code as remote file requests, they just bypass sending them through an actual transport channel and instead call them on the FSChan's Fileserver instance. salt.fileclient.LocalClient ~~~~~~~~~~~~~~~~~~~~~~~~~~~ This client is now used exclusively by Pillar. This used to be used when :conf_minion:`file_client` was set to ``local``, but the ``FSChan`` class was written to allow minions with ``file_client: local`` to access the full set of backends. This class will probably be renamed at some point as it is often confused with ``salt.client.LocalClient``. The :mod:`cp ` Module -------------------------------------- Most of the user-facing interaction with the fileclient happens via the :mod:`cp ` module. The functions in this module instantiate a fileclient instance (if one is not already saved to the ``__context__`` dunder) and run fileclient functions. Updating the Fileserver ----------------------- The master daemon spawns a process dedicated to routine maintenance tasks upon startup. This process runs an instance of ``salt.master.Maintenance``, which loops forever, running a series of functions and then sleeping for a length of time determined by the :conf_master:`loop_interval` config option. One of the maintenance tasks is to update the fileserver, and it essentially runs ``salt.fileserver.Fileserver.update()``, which as we know from above will run all configured backends' ``update()`` functions, if present. This is now remote fileservers like ``git``, ``hg``, and ``svn`` stay up-to-date. For the local file_client (FSClient), since it does not interact with the master, upon spawning of its FSChan it will update the fileserver.