salt.returners.cassandra_cql_return

Return data to a cassandra server

maintainer

Corin Kochenower<ckochenower@saltstack.com>

maturity

new as of 2015.2

depends

salt.modules.cassandra_cql

depends

DataStax Python Driver for Apache Cassandra https://github.com/datastax/python-driver pip install cassandra-driver

platform

all

configuration

To enable this returner, the minion will need the DataStax Python Driver for Apache Cassandra ( https://github.com/datastax/python-driver ) installed and the following values configured in the minion or master config. The list of cluster IPs must include at least one cassandra node IP address. No assumption or default will be used for the cluster IPs. The cluster IPs will be tried in the order listed. The port, username, and password values shown below will be the assumed defaults if you do not provide values.:

cassandra:
  cluster:
    - 192.168.50.11
    - 192.168.50.12
    - 192.168.50.13
  port: 9042
  username: salt
  password: salt

Use the following cassandra database schema:

CREATE KEYSPACE IF NOT EXISTS salt
    WITH replication = {'class': 'SimpleStrategy', 'replication_factor' : 1};

CREATE USER IF NOT EXISTS salt WITH PASSWORD 'salt' NOSUPERUSER;

GRANT ALL ON KEYSPACE salt TO salt;

USE salt;

CREATE TABLE IF NOT EXISTS salt.salt_returns (
    jid text,
    minion_id text,
    fun text,
    alter_time timestamp,
    full_ret text,
    return text,
    success boolean,
    PRIMARY KEY (jid, minion_id, fun)
) WITH CLUSTERING ORDER BY (minion_id ASC, fun ASC);
CREATE INDEX IF NOT EXISTS salt_returns_minion_id ON salt.salt_returns (minion_id);
CREATE INDEX IF NOT EXISTS salt_returns_fun ON salt.salt_returns (fun);

CREATE TABLE IF NOT EXISTS salt.jids (
    jid text PRIMARY KEY,
    load text
);

CREATE TABLE IF NOT EXISTS salt.minions (
    minion_id text PRIMARY KEY,
    last_fun text
);
CREATE INDEX IF NOT EXISTS minions_last_fun ON salt.minions (last_fun);

CREATE TABLE IF NOT EXISTS salt.salt_events (
    id timeuuid,
    tag text,
    alter_time timestamp,
    data text,
    master_id text,
    PRIMARY KEY (id, tag)
) WITH CLUSTERING ORDER BY (tag ASC);
CREATE INDEX tag ON salt.salt_events (tag);

Required python modules: cassandra-driver

To use the cassandra returner, append '--return cassandra_cql' to the salt command. ex:

salt '*' test.ping --return_cql cassandra

Note: if your Cassandra instance has not been tuned much you may benefit from altering some timeouts in cassandra.yaml like so:

# How long the coordinator should wait for read operations to complete
read_request_timeout_in_ms: 5000
# How long the coordinator should wait for seq or index scans to complete
range_request_timeout_in_ms: 20000
# How long the coordinator should wait for writes to complete
write_request_timeout_in_ms: 20000
# How long the coordinator should wait for counter writes to complete
counter_write_request_timeout_in_ms: 10000
# How long a coordinator should continue to retry a CAS operation
# that contends with other proposals for the same row
cas_contention_timeout_in_ms: 5000
# How long the coordinator should wait for truncates to complete
# (This can be much longer, because unless auto_snapshot is disabled
# we need to flush first so we can snapshot before removing the data.)
truncate_request_timeout_in_ms: 60000
# The default timeout for other, miscellaneous operations
request_timeout_in_ms: 20000

As always, your mileage may vary and your Cassandra cluster may have different needs. SaltStack has seen situations where these timeouts can resolve some stacktraces that appear to come from the Datastax Python driver.

salt.returners.cassandra_cql_return.event_return(events)

Return event to one of potentially many clustered cassandra nodes

Requires that configuration be enabled via 'event_return' option in master config.

Cassandra does not support an auto-increment feature due to the highly inefficient nature of creating a monotonically increasing number across all nodes in a distributed database. Each event will be assigned a uuid by the connecting client.

salt.returners.cassandra_cql_return.get_fun(fun)

Return a dict of the last function called for all minions

salt.returners.cassandra_cql_return.get_jid(jid)

Return the information returned when the specified job id was executed

salt.returners.cassandra_cql_return.get_jids()

Return a list of all job ids

salt.returners.cassandra_cql_return.get_load(jid)

Return the load data that marks a specified jid

salt.returners.cassandra_cql_return.get_minions()

Return a list of minions

salt.returners.cassandra_cql_return.prep_jid(nocache, passed_jid=None)

Do any work necessary to prepare a JID, including sending a custom id

salt.returners.cassandra_cql_return.returner(ret)

Return data to one of potentially many clustered cassandra nodes

salt.returners.cassandra_cql_return.save_load(jid, load, minions=None)

Save the load to the specified jid id