# AWS Tasks

This module contains a collection of tasks for interacting with AWS resources.

All AWS related tasks can be authenticated using the AWS_CREDENTIALS Prefect Secret that should be a dictionary with two keys: "ACCESS_KEY" and "SECRET_ACCESS_KEY". See Third Party Authentication for more information.

# S3Download

class

prefect.tasks.aws.s3.S3Download

(bucket=None, boto_kwargs=None, **kwargs)[source]

Task for downloading data from an S3 bucket and returning it as a string. Note that all initialization arguments can optionally be provided or overwritten at runtime.

For authentication, there are two options: you can set the AWS_CREDENTIALS Prefect Secret containing your AWS access keys which will be passed directly to the boto3 client, or you can configure your flow's runtime environment for boto3.

Args:

bucket (str, optional): the name of the S3 Bucket to download from
boto_kwargs (dict, optional): additional keyword arguments to forward to the boto client.
**kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:

methods:
prefect.tasks.aws.s3.S3Download.run (key, credentials=None, bucket=None, compression=None, as_bytes=False)[source]
Task run method. Args: `key (str)`: the name of the Key within this bucket to retrieve `credentials (dict, optional)`: your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: `ACCESS_KEY` and `SECRET_ACCESS_KEY` which will be passed directly to `boto3`. If not provided here or in context, `boto3` will fall back on standard AWS rules for authentication. `bucket (str, optional)`: the name of the S3 Bucket to download from `compression (str, optional)`: specifies a file format for decompression, decompressing data upon download. Currently supports `'gzip'`. `as_bytes (bool, optional)`: If true, result will be returned as `bytes` instead of `str`. Defaults to False. Returns: `str`: the contents of this Key / Bucket, as a string or bytes

prefect.tasks.aws.s3.S3Download.run

(key, credentials=None, bucket=None, compression=None, as_bytes=False)[source]

Task run method.

Args:

key (str): the name of the Key within this bucket to retrieve
credentials (dict, optional): your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY which will be passed directly to boto3. If not provided here or in context, boto3 will fall back on standard AWS rules for authentication.
bucket (str, optional): the name of the S3 Bucket to download from
compression (str, optional): specifies a file format for decompression, decompressing data upon download. Currently supports 'gzip'.
as_bytes (bool, optional): If true, result will be returned as bytes instead of str. Defaults to False.

Returns:

str: the contents of this Key / Bucket, as a string or bytes

# S3Upload

class

prefect.tasks.aws.s3.S3Upload

(bucket=None, boto_kwargs=None, **kwargs)[source]

Task for uploading string data (e.g., a JSON string) to an S3 bucket. Note that all initialization arguments can optionally be provided or overwritten at runtime.

For authentication, there are two options: you can set a Prefect Secret containing your AWS access keys which will be passed directly to the boto3 client, or you can configure your flow's runtime environment for boto3.

Args:

bucket (str, optional): the name of the S3 Bucket to upload to
boto_kwargs (dict, optional): additional keyword arguments to forward to the boto client.
**kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:

methods:
prefect.tasks.aws.s3.S3Upload.run (data, key=None, credentials=None, bucket=None, compression=None)[source]
Task run method. Args: `data (str)`: the data payload to upload `key (str, optional)`: the Key to upload the data under; if not provided, a random `uuid` will be created `credentials (dict, optional)`: your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: `ACCESS_KEY` and `SECRET_ACCESS_KEY` which will be passed directly to `boto3`. If not provided here or in context, `boto3` will fall back on standard AWS rules for authentication. `bucket (str, optional)`: the name of the S3 Bucket to upload to `compression (str, optional)`: specifies a file format for compression, compressing data before upload. Currently supports `'gzip'`. Returns: `str`: the name of the Key the data payload was uploaded to

prefect.tasks.aws.s3.S3Upload.run

(data, key=None, credentials=None, bucket=None, compression=None)[source]

Task run method.

Args:

data (str): the data payload to upload
key (str, optional): the Key to upload the data under; if not provided, a random uuid will be created
credentials (dict, optional): your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY which will be passed directly to boto3. If not provided here or in context, boto3 will fall back on standard AWS rules for authentication.
bucket (str, optional): the name of the S3 Bucket to upload to
compression (str, optional): specifies a file format for compression, compressing data before upload. Currently supports 'gzip'.

Returns:

str: the name of the Key the data payload was uploaded to

# S3List

class

prefect.tasks.aws.s3.S3List

(bucket=None, **kwargs)[source]

Task for listing files from an S3 bucket. Note that all initialization arguments can optionally be provided or overwritten at runtime.

Args:

bucket (str, optional): the name of the S3 Bucket to list the files of.
**kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:

methods:
prefect.tasks.aws.s3.S3List.run (prefix, delimiter="", page_size=None, max_items=None, credentials=None, bucket=None, last_modified_begin=None, last_modified_end=None)[source]
Task run method. Args: `prefix (str)`: the name of the prefix within this bucket to retrieve objects from `delimiter (str)`: indicates the key hierarchy `page_size (int)`: controls the number of items returned per page of each result `max_items (int)`: limits the maximum number of total items returned during pagination `credentials (dict, optional)`: your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: `ACCESS_KEY` and `SECRET_ACCESS_KEY` which will be passed directly to `boto3`. If not provided here or in context, `boto3` will fall back on standard AWS rules for authentication. `bucket (str, optional)`: the name of the S3 Bucket to list the files of `last_modified_begin (str, optional)`: keep items with `LastModified` greater than or equal to given value. timestamp should be in `RFC 3339`, `ISO 8601` or any pendulum supported format. `last_modified_end (str, optional)`: keep items with `LastModified` less than or equal to given value. timestamp should be in `RFC 3339`, `ISO 8601` or any pendulum supported format. Returns: `list[str]`: A list of keys that match the given prefix.

prefect.tasks.aws.s3.S3List.run

(prefix, delimiter="", page_size=None, max_items=None, credentials=None, bucket=None, last_modified_begin=None, last_modified_end=None)[source]

Task run method.

Args:

prefix (str): the name of the prefix within this bucket to retrieve objects from
delimiter (str): indicates the key hierarchy
page_size (int): controls the number of items returned per page of each result
max_items (int): limits the maximum number of total items returned during pagination
credentials (dict, optional): your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY which will be passed directly to boto3. If not provided here or in context, boto3 will fall back on standard AWS rules for authentication.
bucket (str, optional): the name of the S3 Bucket to list the files of
last_modified_begin (str, optional): keep items with LastModified greater than or equal to given value. timestamp should be in RFC 3339, ISO 8601 or any pendulum supported format.
last_modified_end (str, optional): keep items with LastModified less than or equal to given value. timestamp should be in RFC 3339, ISO 8601 or any pendulum supported format.

Returns:

list[str]: A list of keys that match the given prefix.

# LambdaCreate

class

prefect.tasks.aws.lambda_function.LambdaCreate

(function_name, runtime, role, handler, zip_file=None, bucket="", bucket_key="", object_version=None, description="", function_timeout=3, memorysize=128, publish=True, subnet_ids=None, security_group_ids=None, dead_letter_config=None, environment_variables=None, kms_key_arn="", function_tags=None, tracing_config="PassThrough", layers=None, boto_kwargs=None, **kwargs)[source]

Task for creating a Lambda function.

Args:

function_name (str): name of the Lambda function to create
runtime (str): the identifier of the function's runtime
role (str): the Amazon Resource Name of the function's execution role
handler (str): the name of the method within your code that Lambda calls to execute your function
zip_file (str): path to zip file containing code for Lambda function, either zip_file or (bucket and bucket_key) must be passed
bucket (str): an S3 bucket in the same AWS region as your function
bucket_key (str): the Amazon S3 key of the deployment package
object_version (str, optional): for versioned S3 objects, the version of the deployment package to use
description (str, optional): description of Lambda function
function_timeout (int, optional): Lambda function timeout in seconds, default is 3 seconds
memorysize (int, optional): amount of memory that Lambda function has access to in MB, must be a multiple of 64 MB, default is 128
publish (bool, optional): set to True to publish the first version of the function during creation, defaults to True
subnet_ids (List[str], optional): list of subnet ids for vpc configuration
security_group_ids (List[str], optional): list of security group ideas for vpc configuration
dead_letter_config (dict, optional): a dead letter queue configuration that specifies the queue or topic where Lambda sends asynchronous events when they fail processing
environment_variables (dict, optional): key-value pairs of environment variables to pass to the Lambda function
kms_key_arn (str, optional): the ARN of the AWS key management service used to encrypt your function's environment variables, if not provided, AWS Lambda uses a default service key
function_tags (dict, optional): a list of tags to apply to the function, string to string map
tracing_config (str, optional): set to Active to samle and trace a subset of incoming requests with Amazon X-Ray
layers (List[str], optional): a list of function layers to add to the function's execution environment, specify each layer by its ARN
boto_kwargs (dict, optional): additional keyword arguments to forward to the boto client.
**kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:

methods:
prefect.tasks.aws.lambda_function.LambdaCreate.run (credentials=None)[source]
Task run method. Creates Lambda function. Args: `credentials (dict, optional)`: your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: `ACCESS_KEY` and `SECRET_ACCESS_KEY` which will be passed directly to `boto3`. If not provided here or in context, `boto3` will fall back on standard AWS rules for authentication. Returns: `json`: response from AWS CreateFunction endpoint

prefect.tasks.aws.lambda_function.LambdaCreate.run

(credentials=None)[source]

Task run method. Creates Lambda function.

Args:

credentials (dict, optional): your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY which will be passed directly to boto3. If not provided here or in context, boto3 will fall back on standard AWS rules for authentication.

Returns:

json: response from AWS CreateFunction endpoint

# LambdaDelete

class

prefect.tasks.aws.lambda_function.LambdaDelete

(function_name, qualifier="", boto_kwargs=None, **kwargs)[source]

Task for deleting a Lambda function.

Args:

function_name (str): name of the Lambda function to delete
qualifier (str, optional): specify a version to delete, if not provided, the function will be deleted entirely
boto_kwargs (dict, optional): additional keyword arguments to forward to the boto client.
**kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:

methods:
prefect.tasks.aws.lambda_function.LambdaDelete.run (credentials=None)[source]
Task run method. Deletes Lambda function. Args: `credentials (dict, optional)`: your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: `ACCESS_KEY` and `SECRET_ACCESS_KEY` which will be passed directly to `boto3`. If not provided here or in context, `boto3` will fall back on standard AWS rules for authentication. Returns: `dict`: response from AWS DeleteFunction endpoint

prefect.tasks.aws.lambda_function.LambdaDelete.run

(credentials=None)[source]

Task run method. Deletes Lambda function.

Args:

credentials (dict, optional): your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY which will be passed directly to boto3. If not provided here or in context, boto3 will fall back on standard AWS rules for authentication.

Returns:

dict: response from AWS DeleteFunction endpoint

# LambdaInvoke

class

prefect.tasks.aws.lambda_function.LambdaInvoke

(function_name, invocation_type="RequestResponse", log_type="None", client_context=None, payload="null", qualifier="$LATEST", boto_kwargs=None, **kwargs)[source]

Task to invoke a Lambda function.

Args:

function_name (str): the name of the Lambda funciton to invoke
invocation_type (str, optional): the invocation type of Lambda function, default is RequestResponse other options include Event and DryRun
log_type (str, optional): set to 'Tail' to include the execution log in the response
client_context (dict, optional): data to pass to the function in the context object, dict object will be transformed into base64 encoded json automatically
payload (bytes or seekable file-like object): the JSON provided to Lambda function as input
qualifier (str, optional): specify a version or alias to invoke a published version of the function, defaults to $LATEST
boto_kwargs (dict, optional): additional keyword arguments to forward to the boto client.
**kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:

methods:
prefect.tasks.aws.lambda_function.LambdaInvoke.run (function_name=None, payload=None, credentials=None)[source]
Task run method. Invokes Lambda function. Args: `function_name (str)`: the name of the Lambda funciton to invoke `payload (bytes or seekable file-like object)`: the JSON provided to Lambda function as input `credentials (dict, optional)`: your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: `ACCESS_KEY` and `SECRET_ACCESS_KEY` which will be passed directly to `boto3`. If not provided here or in context, `boto3` will fall back on standard AWS rules for authentication. Returns: `dict`: response from AWS Invoke endpoint

prefect.tasks.aws.lambda_function.LambdaInvoke.run

(function_name=None, payload=None, credentials=None)[source]

Task run method. Invokes Lambda function.

Args:

function_name (str): the name of the Lambda funciton to invoke
payload (bytes or seekable file-like object): the JSON provided to Lambda function as input
credentials (dict, optional): your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY which will be passed directly to boto3. If not provided here or in context, boto3 will fall back on standard AWS rules for authentication.

Returns:

dict: response from AWS Invoke endpoint

# LambdaList

class

prefect.tasks.aws.lambda_function.LambdaList

(master_region="ALL", function_version="ALL", marker=None, max_items=50, boto_kwargs=None, **kwargs)[source]

Task to list Lambda functions.

Args:

master_region (str, optional): for Lambda@Edge functions, the AWS region of the master function
function_version (str, optional): the version of a function, default is 'ALL'
marker (str, optional): specify the pagination token that's returned by a previous request to retreive the next page of results
max_items (int, optional): specify a value between 1 and 50 to limit the number of functions in the response
boto_kwargs (dict, optional): additional keyword arguments to forward to the boto client.
**kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:

methods:
prefect.tasks.aws.lambda_function.LambdaList.run (credentials=None)[source]
Task fun method. Lists all Lambda functions. Args: `credentials (dict, optional)`: your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: `ACCESS_KEY` and `SECRET_ACCESS_KEY` which will be passed directly to `boto3`. If not provided here or in context, `boto3` will fall back on standard AWS rules for authentication. Returns: `dict`: a list of Lambda functions from AWS ListFunctions endpoint

prefect.tasks.aws.lambda_function.LambdaList.run

(credentials=None)[source]

Task fun method. Lists all Lambda functions.

Args:

credentials (dict, optional): your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY which will be passed directly to boto3. If not provided here or in context, boto3 will fall back on standard AWS rules for authentication.

Returns:

dict: a list of Lambda functions from AWS ListFunctions endpoint

# StepActivate

class

prefect.tasks.aws.step_function.StepActivate

(state_machine_arn=None, execution_name=None, execution_input="{}", boto_kwargs=None, **kwargs)[source]

Task for activating an AWS Step Function.

Args:

state_machine_arn (str): the Amazon Resource Name (ARN) of the state machine to execute
execution_name (str): the name of the execution, this name must be unique for your AWS account, region, and state machine for 90 days
execution_input (str, optional, DEPRECATED): string that contains the JSON input data for the execution, defaults to "{}". Setting execution_input through the constructor is deprecated. Set it when running this task instead.
boto_kwargs (dict, optional): additional keyword arguments to forward to the boto client.
**kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:

methods:
prefect.tasks.aws.step_function.StepActivate.run (credentials=None, state_machine_arn=None, execution_name=None, execution_input=None, boto_kwargs=None)[source]
Task run method. Activates AWS Step function. Args: `credentials (dict, optional)`: your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: `ACCESS_KEY` and `SECRET_ACCESS_KEY` which will be passed directly to `boto3`. If not provided here or in context, `boto3` will fall back on standard AWS rules for authentication. `state_machine_arn (str)`: the Amazon Resource Name (ARN) of the state machine to execute `execution_name (str)`: the name of the execution, this name must be unique for your AWS account, region, and state machine for 90 days `execution_input (str, optional)`: string that contains the JSON input data for the execution, defaults to `"{}"` `boto_kwargs (dict, optional)`: additional keyword arguments to forward to the boto client. Returns: `dict`: response from AWS StartExecution endpoint

prefect.tasks.aws.step_function.StepActivate.run

(credentials=None, state_machine_arn=None, execution_name=None, execution_input=None, boto_kwargs=None)[source]

Task run method. Activates AWS Step function.

Args:

credentials (dict, optional): your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY which will be passed directly to boto3. If not provided here or in context, boto3 will fall back on standard AWS rules for authentication.
state_machine_arn (str): the Amazon Resource Name (ARN) of the state machine to execute
execution_name (str): the name of the execution, this name must be unique for your AWS account, region, and state machine for 90 days
execution_input (str, optional): string that contains the JSON input data for the execution, defaults to "{}"
boto_kwargs (dict, optional): additional keyword arguments to forward to the boto client.

Returns:

dict: response from AWS StartExecution endpoint

# AWSSecretsManager

class

prefect.tasks.aws.secrets_manager.AWSSecretsManager

(secret=None, boto_kwargs=None, **kwargs)[source]

Task for retrieving secrets from an AWS Secrets Manager and returning it as a dictionary. Note that all initialization arguments can optionally be provided or overwritten at runtime.

Args:

secret (str, optional): the name of the secret to retrieve
boto_kwargs (dict, optional): additional keyword arguments to forward to the boto client.
**kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:

methods:
prefect.tasks.aws.secrets_manager.AWSSecretsManager.run (secret=None, credentials=None)[source]
Task run method. Args: `secret (str)`: the name of the secret to retrieve `credentials (dict, optional)`: your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: `ACCESS_KEY` and `SECRET_ACCESS_KEY` which will be passed directly to `boto3`. If not provided here or in context, `boto3` will fall back on standard AWS rules for authentication. Returns: `dict`: the contents of this secret, as a dictionary

prefect.tasks.aws.secrets_manager.AWSSecretsManager.run

(secret=None, credentials=None)[source]

Task run method.

Args:

secret (str): the name of the secret to retrieve
credentials (dict, optional): your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY which will be passed directly to boto3. If not provided here or in context, boto3 will fall back on standard AWS rules for authentication.

Returns:

dict: the contents of this secret, as a dictionary

# BatchSubmit

class

prefect.tasks.aws.batch.BatchSubmit

(job_name=None, job_definition=None, job_queue=None, boto_kwargs=None, **kwargs)[source]

Task for submitting a job to AWS batch.

Args:

job_name (str, optional): The AWS batch job name.
job_definition (str, optional): The AWS batch job definition.
job_queue (str, optional): Name of the AWS batch job queue.
boto_kwargs (dict, optional): additional kekyword arguments to forward to the boto client.
**kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:

methods:
prefect.tasks.aws.batch.BatchSubmit.run (job_name=None, job_definition=None, job_queue=None, batch_kwargs=None, credentials=None)[source]
Submit a job to the AWS Batch job service. Args: `job_name (str, optional)`: The AWS batch job name. `job_definition (str, optional)`: The AWS batch job definition. `job_queue (str, optional)`: Name of the AWS batch job queue. `batch_kwargs (dict, optional)`: Additional keyword arguments to pass to the boto3 `submit_job` function. See the submit_job # noqa documentation for more details. `credentials (dict, optional)`: your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: `ACCESS_KEY` and `SECRET_ACCESS_KEY` which will be passed directly to `boto3`. If not provided here or in context, `boto3` will fall back on standard AWS rules for authentication.

prefect.tasks.aws.batch.BatchSubmit.run

(job_name=None, job_definition=None, job_queue=None, batch_kwargs=None, credentials=None)[source]

Submit a job to the AWS Batch job service.

Args:

job_name (str, optional): The AWS batch job name.
job_definition (str, optional): The AWS batch job definition.
job_queue (str, optional): Name of the AWS batch job queue.
batch_kwargs (dict, optional): Additional keyword arguments to pass to the boto3 submit_job function. See the submit_job # noqa documentation for more details.
credentials (dict, optional): your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY which will be passed directly to boto3. If not provided here or in context, boto3 will fall back on standard AWS rules for authentication.

# AWSClientWait

class

prefect.tasks.aws.client_waiter.AWSClientWait

(client=None, waiter_name=None, waiter_definition=None, boto_kwargs=None, **kwargs)[source]

Task for waiting on a long-running AWS job. Uses the underlying boto3 waiter functionality.

Args:

client (str, optional): The AWS client on which to wait (e.g., 'batch', 'ec2', etc)
waiter_name (str, optional): The name of the waiter to instantiate. Can be a boto-supported waiter or one of prefect's custom waiters. Currently, prefect offers three additional waiters for AWS Batch: "JobExists" waits for a job to be instantiated, "JobRunning" waits for a job to start running, and "JobComplete" waits for a job to finish. You can find the definitions for all prefect-defined waiters here. # noqa You may also use a custom waiter name, if you supply an accompanying waiter definition dict.
waiter_definition (dict, optional): A valid custom waiter model, as a dict. Note that if you supply a custom definition, it is assumed that the provided 'waiter_name' is contained within the waiter definition dict.
boto_kwargs (dict, optional): additional kekyword arguments to forward to the boto client.
**kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:

methods:
prefect.tasks.aws.client_waiter.AWSClientWait.run (client=None, waiter_name=None, waiter_definition=None, waiter_kwargs=None, credentials=None)[source]
Task for waiting on a long-running AWS job. Uses the underlying boto3 waiter functionality. Args: `client (str)`: The AWS client on which to wait (e.g., 'batch', 'ec2', etc) `waiter_name (str, optional)`: The name of the waiter to instantiate. Can be a boto-supported waiter or one of prefect's custom waiters. Currently, prefect offers three additional waiters for AWS Batch: `"JobExists"` waits for a job to be instantiated, `"JobRunning"` waits for a job to start running, and `"JobComplete"` waits for a job to finish. You can find the definitions for all prefect-defined waiters here. # noqa You may also use a custom waiter name, if you supply an accompanying waiter definition dict. `waiter_definition (dict, optional)`: A valid custom waiter model, as a dict. Note that if you supply a custom definition, it is assumed that the provided 'waiter_name' is contained within the waiter definition dict. `waiter_kwargs (dict, optional)`: Arguments to pass to the `waiter.wait(...)` method. Will depend upon the specific waiter being called. `credentials (dict, optional)`: your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: `ACCESS_KEY` and `SECRET_ACCESS_KEY` which will be passed directly to `boto3`. If not provided here or in context, `boto3` will fall back on standard AWS rules for authentication.

prefect.tasks.aws.client_waiter.AWSClientWait.run

(client=None, waiter_name=None, waiter_definition=None, waiter_kwargs=None, credentials=None)[source]

Task for waiting on a long-running AWS job. Uses the underlying boto3 waiter functionality.

Args:

client (str): The AWS client on which to wait (e.g., 'batch', 'ec2', etc)
waiter_name (str, optional): The name of the waiter to instantiate. Can be a boto-supported waiter or one of prefect's custom waiters. Currently, prefect offers three additional waiters for AWS Batch: "JobExists" waits for a job to be instantiated, "JobRunning" waits for a job to start running, and "JobComplete" waits for a job to finish. You can find the definitions for all prefect-defined waiters here. # noqa You may also use a custom waiter name, if you supply an accompanying waiter definition dict.
waiter_definition (dict, optional): A valid custom waiter model, as a dict. Note that if you supply a custom definition, it is assumed that the provided 'waiter_name' is contained within the waiter definition dict.
waiter_kwargs (dict, optional): Arguments to pass to the waiter.wait(...) method. Will depend upon the specific waiter being called.
credentials (dict, optional): your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY which will be passed directly to boto3. If not provided here or in context, boto3 will fall back on standard AWS rules for authentication.

This documentation was auto-generated from commit n/a
on February 23, 2022 at 19:26 UTC

← Asana Tasks Azure Tasks →