salt.cache.redis_cache

Redis

Redis plugin for the Salt caching subsystem.

As Redis provides a simple mechanism for very fast key-value store, in order to privde the necessary features for the Salt caching subsystem, the following conventions are used:

• A Redis key consists of the bank name and the cache key separated by /, e.g.: $KEY_minions/alpha/stuff where minions/alpha is the bank name and stuff is the key name.

• As the caching subsystem is organised as a tree, we need to store the caching path and identify the bank and its offspring. At the same time, Redis is linear and we need to avoid doing keys <pattern> which is very inefficient as it goes through all the keys on the remote Redis server. Instead, each bank hierarchy has a Redis SET associated which stores the list of sub-banks. By default, these keys begin with $BANK_.

• In addition, each key name is stored in a separate SET of all the keys within a bank. By default, these SETs begin with $BANKEYS_.

For example, to store the key my-key under the bank root-bank/sub-bank/leaf-bank, the following hierarchy will be built:

127.0.0.1:6379> SMEMBERS $BANK_root-bank
1) "sub-bank"
127.0.0.1:6379> SMEMBERS $BANK_root-bank/sub-bank
1) "leaf-bank"
127.0.0.1:6379> SMEMBERS $BANKEYS_root-bank/sub-bank/leaf-bank
1) "my-key"
127.0.0.1:6379> GET $KEY_root-bank/sub-bank/leaf-bank/my-key
"my-value"

There are three types of keys stored:

• $BANK_* is a Redis SET containing the list of banks under the current bank

• $BANKEYS_* is a Redis SET containing the list of keys under the current bank

• $KEY_* keeps the value of the key

These prefixes and the separator can be adjusted using the configuration options:

bank_prefix: $BANK

The prefix used for the name of the Redis key storing the list of sub-banks.

bank_keys_prefix: $BANKEYS

The prefix used for the name of the Redis keyt storing the list of keys under a certain bank.

key_prefix: $KEY

The prefix of the Redis keys having the value of the keys to be cached under a certain bank.

separator: _

The separator between the prefix and the key body.

The connection details can be specified using:

host: localhost

The hostname of the Redis server.

port: 6379

The Redis server port.

cluster_mode: False

Whether cluster_mode is enabled or not

cluster.startup_nodes:

A list of host, port dictionaries pointing to cluster members. At least one is required but multiple nodes are better

cache.redis.cluster.startup_nodes
  - host: redis-member-1
    port: 6379
  - host: redis-member-2
    port: 6379

cluster.skip_full_coverage_check: False

Some cluster providers restrict certain redis commands such as CONFIG for enhanced security. Set this option to true to skip checks that required advanced privileges.

Note

Most cloud hosted redis clusters will require this to be set to True

db: '0'

The database index.

Note

The database index must be specified as string not as integer value!

password:

Redis connection password.

unix_socket_path:

New in version 2018.3.1.

Path to a UNIX socket for access. Overrides host / port.

Configuration Example:

cache.redis.host: localhost
cache.redis.port: 6379
cache.redis.db: '0'
cache.redis.password: my pass
cache.redis.bank_prefix: #BANK
cache.redis.bank_keys_prefix: #BANKEYS
cache.redis.key_prefix: #KEY
cache.redis.separator: '@'

Cluster Configuration Example:

cache.redis.cluster_mode: true
cache.redis.cluster.skip_full_coverage_check: true
cache.redis.cluster.startup_nodes:
  - host: redis-member-1
    port: 6379
  - host: redis-member-2
    port: 6379
cache.redis.db: '0'
cache.redis.password: my pass
cache.redis.bank_prefix: #BANK
cache.redis.bank_keys_prefix: #BANKEYS
cache.redis.key_prefix: #KEY
cache.redis.separator: '@'

salt.cache.redis_cache.contains(bank, key)

Checks if the specified bank contains the specified key.

salt.cache.redis_cache.fetch(bank, key)

Fetch data from the Redis cache.

salt.cache.redis_cache.flush(bank, key=None)

Remove the key from the cache bank with all the key content. If no key is specified, remove the entire bank with all keys and sub-banks inside. This function is using the Redis pipelining for best performance. However, when removing a whole bank, in order to re-create the tree, there are a couple of requests made. In total:

• one for node in the hierarchy sub-tree, starting from the bank node

• one pipelined request to get the keys under all banks in the sub-tree

• one pipeline request to remove the corresponding keys

This is not quite optimal, as if we need to flush a bank having a very long list of sub-banks, the number of requests to build the sub-tree may grow quite big.

An improvement for this would be loading a custom Lua script in the Redis instance of the user (using the register_script feature) and call it whenever we flush. This script would only need to build this sub-tree causing problems. It can be added later and the behaviour should not change as the user needs to explicitly allow Salt inject scripts in their Redis instance.

salt.cache.redis_cache.init_kwargs(kwargs)

salt.cache.redis_cache.list_(bank)

Lists entries stored in the specified bank.

salt.cache.redis_cache.store(bank, key, data)

Store the data in a Redis key.