data_pipeline - Create and manage AWS Datapipelines

• Synopsis
  • Requirements
• Parameters
• Notes
• Examples
• Return Values
• Status
  • Author

Synopsis

• Create and manage AWS Datapipelines. Creation is not idempotent in AWS, so the uniqueId is created by hashing the options (minus objects) given to the datapipeline.
• The pipeline definition must be in the format given here http://docs.aws.amazon.com/datapipeline/latest/APIReference/API_PutPipelineDefinition.html#API_PutPipelineDefinition_RequestSyntax.
• Also operations will wait for a configurable amount of time to ensure the pipeline is in the requested state.

Requirements

The below requirements are needed on the host that executes this module.

• boto
• boto3
• python >= 2.6

Parameters

Notes

Examples

# Note: These examples do not set authentication details, see the AWS Guide for details.

# Create pipeline
- data_pipeline:
    name: test-dp
    region: us-west-2
    objects: "{{pipelineObjects}}"
    parameters: "{{pipelineParameters}}"
    values: "{{pipelineValues}}"
    tags:
      key1: val1
      key2: val2
    state: present

# Example populating and activating a pipeline that demonstrates two ways of providing pipeline objects
- data_pipeline:
  name: test-dp
  objects:
    - "id": "DefaultSchedule"
      "name": "Every 1 day"
      "fields":
        - "key": "period"
          "stringValue": "1 days"
        - "key": "type"
          "stringValue": "Schedule"
        - "key": "startAt"
          "stringValue": "FIRST_ACTIVATION_DATE_TIME"
    - "id": "Default"
      "name": "Default"
      "fields": [ { "key": "resourceRole", "stringValue": "my_resource_role" },
                  { "key": "role", "stringValue": "DataPipelineDefaultRole" },
                  { "key": "pipelineLogUri", "stringValue": "s3://my_s3_log.txt" },
                  { "key": "scheduleType", "stringValue": "cron" },
                  { "key": "schedule", "refValue": "DefaultSchedule" },
                  { "key": "failureAndRerunMode", "stringValue": "CASCADE" } ]
  state: active

# Activate pipeline
- data_pipeline:
    name: test-dp
    region: us-west-2
    state: active

# Delete pipeline
- data_pipeline:
    name: test-dp
    region: us-west-2
    state: absent

Return Values

Common return values are documented here, the following are the fields unique to this module:

Status

This module is flagged as preview which means that it is not guaranteed to have a backwards compatible interface.

Author

• Raghu Udiyar <raghusiddarth@gmail.com> (@raags)
• Sloane Hertel <shertel@redhat.com>