salt.modules.highstate_doc

Creating a document collection

From the master we can run the following script to creates a collection of all your minion documents.

salt '*' state.apply makereadme

#!/bin/python
import os
import salt.client
s = salt.client.LocalClient()
# NOTE: because of issues with `cp.push` use `highstate_doc.read_file`
o = s.cmd('*', 'highstate_doc.read_file', ['/root/README.md'])
for m in o:
    d = o.get(m)
    if d and not d.endswith('is not available.'):
        # mkdir m
        #directory = os.path.dirname(file_path)
        if not os.path.exists(m):
            os.makedirs(m)
        with open(m + '/README.md','wb') as f:
            f.write(d)
        print('ADDED: ' + m + '/README.md')

Once the master has a collection of all the README files. You can use pandoc to create HTML versions of the markdown.

# process all the readme.md files to readme.html
if which pandoc; then echo "Found pandoc"; else echo "** Missing pandoc"; exit 1; fi
if which gs; then echo "Found gs"; else echo "** Missing gs(ghostscript)"; exit 1; fi
readme_files=$(find $dest -type f -path "*/README.md" -print)
for f in $readme_files ; do
    ff=${f#$dest/}
    minion=${ff%%/*}
    echo "process: $dest/${minion}/$(basename $f)"
    cat $dest/${minion}/$(basename $f) |             pandoc --standalone --from markdown_github --to html             --include-in-header $dest/style.html             > $dest/${minion}/$(basename $f).html
done

It is also nice to put the help files in source control.

# git init git add -A git commit -am 'updated docs' git push -f

Other hints

If you wish to customize the document format:

# you could also create a new `processor` for perhaps reStructuredText
# highstate_doc.config:
#     processor: doc_custom.processor_rst

# example `salt://makereadme.jinja`
"""
{{opts['id']}}
==========================================

{# lowstates is set from highstate_doc.render() #}
{# if lowstates is missing use salt.highstate_doc.process_lowstates() #}
{% for s in lowstates %}
{{s.id}}
-----------------------------------------------------------------
{{s.function}}

{{s.markdown.requisite}}
{{s.markdown.details}}

{%- endfor %}
"""

# example `salt://makereadme.sls`
{% import_text "makereadme.jinja" as makereadme %}
{{sls}} or:
    file.managed:
        - name: /root/README_other.md
        - contents: {{salt.highstate_doc.render(jinja_template_text=makereadme)|json}}
        - mode: '0640'

Some replace_text_regex values that might be helpful:

CERTS
-----

``'-----BEGIN RSA PRIVATE KEY-----[\r\n\t\f\S]{0,2200}': 'XXXXXXX'``
``'-----BEGIN CERTIFICATE-----[\r\n\t\f\S]{0,2200}': 'XXXXXXX'``
``'-----BEGIN DH PARAMETERS-----[\r\n\t\f\S]{0,2200}': 'XXXXXXX'``
``'-----BEGIN PRIVATE KEY-----[\r\n\t\f\S]{0,2200}': 'XXXXXXX'``
``'-----BEGIN OPENSSH PRIVATE KEY-----[\r\n\t\f\S]{0,2200}': 'XXXXXXX'``
``'ssh-rsa .* ': 'ssh-rsa XXXXXXX '``
``'ssh-dss .* ': 'ssh-dss XXXXXXX '``

DB
--

``'DB_PASS.*': 'DB_PASS = XXXXXXX'``
``'5432:*:*:.*': '5432:*:XXXXXXX'``
``"'PASSWORD': .*": "'PASSWORD': 'XXXXXXX',"``
``" PASSWORD '.*'": " PASSWORD 'XXXXXXX'"``
``'PGPASSWORD=.* ': 'PGPASSWORD=XXXXXXX'``
``"_replication password '.*'":  "_replication password 'XXXXXXX'"``

OTHER
-----

``'EMAIL_HOST_PASSWORD =.*': 'EMAIL_HOST_PASSWORD =XXXXXXX'``
``"net ads join -U '.*@MFCFADS.MATH.EXAMPLE.CA.* ": "net ads join -U '.*@MFCFADS.MATH.EXAMPLE.CA%XXXXXXX "``
``"net ads join -U '.*@NEXUS.EXAMPLE.CA.* ": "net ads join -U '.*@NEXUS.EXAMPLE.CA%XXXXXXX "``
``'install-uptrack .* --autoinstall': 'install-uptrack XXXXXXX --autoinstall'``
``'accesskey = .*': 'accesskey = XXXXXXX'``
``'auth_pass .*': 'auth_pass XXXXXXX'``
``'PSK "0x.*': 'PSK "0xXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'``
``'SECRET_KEY.*': 'SECRET_KEY = XXXXXXX'``
``"password=.*": "password=XXXXXXX"``
``'<password>.*</password>': '<password>XXXXXXX</password>'``
``'<salt>.*</salt>': '<salt>XXXXXXX</salt>'``
``'application.secret = ".*"': 'application.secret = "XXXXXXX"'``
``'url = "postgres://.*"': 'url = "postgres://XXXXXXX"'``
``'PASS_.*_PASS': 'PASS_XXXXXXX_PASS'``

HTACCESS
--------

``':{PLAIN}.*': ':{PLAIN}XXXXXXX'``

salt.modules.highstate_doc.markdown_basic_jinja_template(**kwargs)

Return text for a simple markdown jinja template

This function can be used from the highstate_doc.render modules jinja_template_function option.

salt.modules.highstate_doc.markdown_default_jinja_template(**kwargs)

Return text for a markdown jinja template that included a header

This function can be used from the highstate_doc.render modules jinja_template_function option.

salt.modules.highstate_doc.markdown_full_jinja_template(**kwargs)

Return text for an advanced markdown jinja template

This function can be used from the highstate_doc.render modules jinja_template_function option.

salt.modules.highstate_doc.process_lowstates(**kwargs)

return processed lowstate data that was not blacklisted

render_module_function is used to provide your own. defaults to from_lowstate

salt.modules.highstate_doc.processor_markdown(lowstate_item, config, **kwargs)

Takes low state data and returns a dict of processed data that is by default used in a jinja template when rendering a markdown highstate_doc.

This lowstate_item_markdown given a lowstate item, returns a dict like:

vars:       # the raw lowstate_item that was processed
id:         # the 'id' of the state.
id_full:    # combo of the state type and id "state: id"
state:      # name of the salt state module
function:   # name of the state function
name:       # value of 'name:' passed to the salt state module
state_function:    # the state name and function name
markdown:          # text data to describe a state
    requisites:    # requisite like [watch_in, require_in]
    details:       # state name, parameters and other details like file contents

salt.modules.highstate_doc.read_file(name)

output the contents of a file:

this is a workaround if the cp.push module does not work. https://github.com/saltstack/salt/issues/37133

help the master output the contents of a document that might be saved on the minions filesystem.

#!/bin/python
import os
import salt.client
s = salt.client.LocalClient()
o = s.cmd('*', 'highstate_doc.read_file', ['/root/README.md'])
for m in o:
    d = o.get(m)
    if d and not d.endswith('is not available.'):
        # mkdir m
        #directory = os.path.dirname(file_path)
        if not os.path.exists(m):
            os.makedirs(m)
        with open(m + '/README.md','wb') as fin:
            fin.write(d)
        print('ADDED: ' + m + '/README.md')

salt.modules.highstate_doc.render(jinja_template_text=None, jinja_template_function='highstate_doc.markdown_default_jinja_template', **kwargs)

Render highstate to a text format (default Markdown)

if jinja_template_text is not set, jinja_template_function is used.

jinja_template_text: jinja text that the render uses to create the document. jinja_template_function: a salt module call that returns template text.

Options

highstate_doc.markdown_basic_jinja_template highstate_doc.markdown_default_jinja_template highstate_doc.markdown_full_jinja_template