# AWS Tasks

This module contains a collection of tasks for interacting with AWS resources.

All AWS related tasks can be authenticated using the AWS_CREDENTIALS Prefect Secret that should be a dictionary with two keys: "ACCESS_KEY" and "SECRET_ACCESS_KEY". See Third Party Authentication for more information.

# S3Download

class

prefect.tasks.aws.s3.S3Download

(bucket=None, boto_kwargs=None, **kwargs)[source]

Task for downloading data from an S3 bucket and returning it as a string. Note that all initialization arguments can optionally be provided or overwritten at runtime.

For authentication, there are two options: you can set the AWS_CREDENTIALS Prefect Secret containing your AWS access keys which will be passed directly to the boto3 client, or you can configure your flow's runtime environment for boto3.

Args:

bucket (str, optional): the name of the S3 Bucket to download from
boto_kwargs (dict, optional): additional keyword arguments to forward to the boto client.
**kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:

methods:
prefect.tasks.aws.s3.S3Download.run (key, credentials=None, bucket=None, compression=None, as_bytes=False)[source]
Task run method. Args: `key (str)`: the name of the Key within this bucket to retrieve `credentials (dict, optional)`: your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: `ACCESS_KEY` and `SECRET_ACCESS_KEY` which will be passed directly to `boto3`. If not provided here or in context, `boto3` will fall back on standard AWS rules for authentication. `bucket (str, optional)`: the name of the S3 Bucket to download from `compression (str, optional)`: specifies a file format for decompression, decompressing data upon download. Currently supports `'gzip'`. `as_bytes (bool, optional)`: If true, result will be returned as `bytes` instead of `str`. Defaults to False. Returns: `str`: the contents of this Key / Bucket, as a string or bytes

prefect.tasks.aws.s3.S3Download.run

(key, credentials=None, bucket=None, compression=None, as_bytes=False)[source]

Task run method.

Args:

key (str): the name of the Key within this bucket to retrieve
credentials (dict, optional): your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY which will be passed directly to boto3. If not provided here or in context, boto3 will fall back on standard AWS rules for authentication.
bucket (str, optional): the name of the S3 Bucket to download from
compression (str, optional): specifies a file format for decompression, decompressing data upon download. Currently supports 'gzip'.
as_bytes (bool, optional): If true, result will be returned as bytes instead of str. Defaults to False.

Returns:

str: the contents of this Key / Bucket, as a string or bytes

# S3Upload

class

prefect.tasks.aws.s3.S3Upload

(bucket=None, boto_kwargs=None, upload_options=None, **kwargs)[source]

Task for uploading string data (e.g., a JSON string) to an S3 bucket. Note that all initialization arguments can optionally be provided or overwritten at runtime.

For authentication, there are two options: you can set a Prefect Secret containing your AWS access keys which will be passed directly to the boto3 client, or you can configure your flow's runtime environment for boto3.

Args:

bucket (str, optional): the name of the S3 Bucket to upload to
boto_kwargs (dict, optional): additional keyword arguments to forward to the boto client.
upload_options (dict, optional): Additional options for s3 client upload_fileobj() method ExtraArgs argument
**kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:

methods:
prefect.tasks.aws.s3.S3Upload.run (data, key=None, credentials=None, bucket=None, compression=None, upload_options=None)[source]
Task run method. Args: `data (str)`: the data payload to upload `key (str, optional)`: the Key to upload the data under; if not provided, a random `uuid` will be created `credentials (dict, optional)`: your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: `ACCESS_KEY` and `SECRET_ACCESS_KEY` which will be passed directly to `boto3`. If not provided here or in context, `boto3` will fall back on standard AWS rules for authentication. `bucket (str, optional)`: the name of the S3 Bucket to upload to `compression (str, optional)`: specifies a file format for compression, compressing data before upload. Currently supports `'gzip'`. `upload_options (dict, optional)`: Additional options for s3 client `upload_fileobj()` method `ExtraArgs` argument Returns: `str`: the name of the Key the data payload was uploaded to

prefect.tasks.aws.s3.S3Upload.run

(data, key=None, credentials=None, bucket=None, compression=None, upload_options=None)[source]

Task run method.

Args:

data (str): the data payload to upload
key (str, optional): the Key to upload the data under; if not provided, a random uuid will be created
credentials (dict, optional): your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY which will be passed directly to boto3. If not provided here or in context, boto3 will fall back on standard AWS rules for authentication.
bucket (str, optional): the name of the S3 Bucket to upload to
compression (str, optional): specifies a file format for compression, compressing data before upload. Currently supports 'gzip'.
upload_options (dict, optional): Additional options for s3 client upload_fileobj() method ExtraArgs argument

Returns:

str: the name of the Key the data payload was uploaded to

# S3List

class

prefect.tasks.aws.s3.S3List

(bucket=None, boto_kwargs=None, **kwargs)[source]

Task for listing files from an S3 bucket. Note that all initialization arguments can optionally be provided or overwritten at runtime.

Args:

bucket (str, optional): the name of the S3 Bucket to list the files of.
boto_kwargs (dict, optional): additional keyword arguments to forward to the boto client.
**kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:

methods:
prefect.tasks.aws.s3.S3List.run (prefix, delimiter="", page_size=None, max_items=None, credentials=None, bucket=None, last_modified_begin=None, last_modified_end=None)[source]
Task run method. Args: `prefix (str)`: the name of the prefix within this bucket to retrieve objects from `delimiter (str)`: indicates the key hierarchy `page_size (int)`: controls the number of items returned per page of each result `max_items (int)`: limits the maximum number of total items returned during pagination `credentials (dict, optional)`: your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: `ACCESS_KEY` and `SECRET_ACCESS_KEY` which will be passed directly to `boto3`. If not provided here or in context, `boto3` will fall back on standard AWS rules for authentication. `bucket (str, optional)`: the name of the S3 Bucket to list the files of `last_modified_begin (str, optional)`: keep items with `LastModified` greater than or equal to given value. timestamp should be in `RFC 3339`, `ISO 8601` or any pendulum supported format. `last_modified_end (str, optional)`: keep items with `LastModified` less than or equal to given value. timestamp should be in `RFC 3339`, `ISO 8601` or any pendulum supported format. Returns: `list[str]`: A list of keys that match the given prefix.

prefect.tasks.aws.s3.S3List.run

(prefix, delimiter="", page_size=None, max_items=None, credentials=None, bucket=None, last_modified_begin=None, last_modified_end=None)[source]

Task run method.

Args:

prefix (str): the name of the prefix within this bucket to retrieve objects from
delimiter (str): indicates the key hierarchy
page_size (int): controls the number of items returned per page of each result
max_items (int): limits the maximum number of total items returned during pagination
credentials (dict, optional): your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY which will be passed directly to boto3. If not provided here or in context, boto3 will fall back on standard AWS rules for authentication.
bucket (str, optional): the name of the S3 Bucket to list the files of
last_modified_begin (str, optional): keep items with LastModified greater than or equal to given value. timestamp should be in RFC 3339, ISO 8601 or any pendulum supported format.
last_modified_end (str, optional): keep items with LastModified less than or equal to given value. timestamp should be in RFC 3339, ISO 8601 or any pendulum supported format.

Returns:

list[str]: A list of keys that match the given prefix.

# LambdaCreate

class

prefect.tasks.aws.lambda_function.LambdaCreate

(function_name, runtime, role, handler, zip_file=None, bucket="", bucket_key="", object_version=None, description="", function_timeout=3, memorysize=128, publish=True, subnet_ids=None, security_group_ids=None, dead_letter_config=None, environment_variables=None, kms_key_arn="", function_tags=None, tracing_config="PassThrough", layers=None, boto_kwargs=None, **kwargs)[source]

Task for creating a Lambda function.

Args:

function_name (str): name of the Lambda function to create
runtime (str): the identifier of the function's runtime
role (str): the Amazon Resource Name of the function's execution role
handler (str): the name of the method within your code that Lambda calls to execute your function
zip_file (str): path to zip file containing code for Lambda function, either zip_file or (bucket and bucket_key) must be passed
bucket (str): an S3 bucket in the same AWS region as your function
bucket_key (str): the Amazon S3 key of the deployment package
object_version (str, optional): for versioned S3 objects, the version of the deployment package to use
description (str, optional): description of Lambda function
function_timeout (int, optional): Lambda function timeout in seconds, default is 3 seconds
memorysize (int, optional): amount of memory that Lambda function has access to in MB, must be a multiple of 64 MB, default is 128
publish (bool, optional): set to True to publish the first version of the function during creation, defaults to True
subnet_ids (List[str], optional): list of subnet ids for vpc configuration
security_group_ids (List[str], optional): list of security group ideas for vpc configuration
dead_letter_config (dict, optional): a dead letter queue configuration that specifies the queue or topic where Lambda sends asynchronous events when they fail processing
environment_variables (dict, optional): key-value pairs of environment variables to pass to the Lambda function
kms_key_arn (str, optional): the ARN of the AWS key management service used to encrypt your function's environment variables, if not provided, AWS Lambda uses a default service key
function_tags (dict, optional): a list of tags to apply to the function, string to string map
tracing_config (str, optional): set to Active to samle and trace a subset of incoming requests with Amazon X-Ray
layers (List[str], optional): a list of function layers to add to the function's execution environment, specify each layer by its ARN
boto_kwargs (dict, optional): additional keyword arguments to forward to the boto client.
**kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:

methods:
prefect.tasks.aws.lambda_function.LambdaCreate.run (credentials=None)[source]
Task run method. Creates Lambda function. Args: `credentials (dict, optional)`: your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: `ACCESS_KEY` and `SECRET_ACCESS_KEY` which will be passed directly to `boto3`. If not provided here or in context, `boto3` will fall back on standard AWS rules for authentication. Returns: `json`: response from AWS CreateFunction endpoint

prefect.tasks.aws.lambda_function.LambdaCreate.run

(credentials=None)[source]

Task run method. Creates Lambda function.

Args:

credentials (dict, optional): your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY which will be passed directly to boto3. If not provided here or in context, boto3 will fall back on standard AWS rules for authentication.

Returns:

json: response from AWS CreateFunction endpoint

# LambdaDelete

class

prefect.tasks.aws.lambda_function.LambdaDelete

(function_name, qualifier="", boto_kwargs=None, **kwargs)[source]

Task for deleting a Lambda function.

Args:

function_name (str): name of the Lambda function to delete
qualifier (str, optional): specify a version to delete, if not provided, the function will be deleted entirely
boto_kwargs (dict, optional): additional keyword arguments to forward to the boto client.
**kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:

methods:
prefect.tasks.aws.lambda_function.LambdaDelete.run (credentials=None)[source]
Task run method. Deletes Lambda function. Args: `credentials (dict, optional)`: your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: `ACCESS_KEY` and `SECRET_ACCESS_KEY` which will be passed directly to `boto3`. If not provided here or in context, `boto3` will fall back on standard AWS rules for authentication. Returns: `dict`: response from AWS DeleteFunction endpoint

prefect.tasks.aws.lambda_function.LambdaDelete.run

(credentials=None)[source]

Task run method. Deletes Lambda function.

Args:

credentials (dict, optional): your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY which will be passed directly to boto3. If not provided here or in context, boto3 will fall back on standard AWS rules for authentication.

Returns:

dict: response from AWS DeleteFunction endpoint

# LambdaInvoke

class

prefect.tasks.aws.lambda_function.LambdaInvoke

(function_name, invocation_type="RequestResponse", log_type="None", client_context=None, payload="null", qualifier="$LATEST", boto_kwargs=None, **kwargs)[source]

Task to invoke a Lambda function.

Args:

function_name (str): the name of the Lambda funciton to invoke
invocation_type (str, optional): the invocation type of Lambda function, default is RequestResponse other options include Event and DryRun
log_type (str, optional): set to 'Tail' to include the execution log in the response
client_context (dict, optional): data to pass to the function in the context object, dict object will be transformed into base64 encoded json automatically
payload (bytes or seekable file-like object): the JSON provided to Lambda function as input
qualifier (str, optional): specify a version or alias to invoke a published version of the function, defaults to $LATEST
boto_kwargs (dict, optional): additional keyword arguments to forward to the boto client.
**kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:

methods:
prefect.tasks.aws.lambda_function.LambdaInvoke.run (function_name=None, payload=None, credentials=None)[source]
Task run method. Invokes Lambda function. Args: `function_name (str)`: the name of the Lambda funciton to invoke `payload (bytes or seekable file-like object)`: the JSON provided to Lambda function as input `credentials (dict, optional)`: your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: `ACCESS_KEY` and `SECRET_ACCESS_KEY` which will be passed directly to `boto3`. If not provided here or in context, `boto3` will fall back on standard AWS rules for authentication. Returns: `dict`: response from AWS Invoke endpoint

prefect.tasks.aws.lambda_function.LambdaInvoke.run

(function_name=None, payload=None, credentials=None)[source]

Task run method. Invokes Lambda function.

Args:

function_name (str): the name of the Lambda funciton to invoke
payload (bytes or seekable file-like object): the JSON provided to Lambda function as input
credentials (dict, optional): your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY which will be passed directly to boto3. If not provided here or in context, boto3 will fall back on standard AWS rules for authentication.

Returns:

dict: response from AWS Invoke endpoint

# LambdaList

class

prefect.tasks.aws.lambda_function.LambdaList

(master_region="ALL", function_version="ALL", marker=None, max_items=50, boto_kwargs=None, **kwargs)[source]

Task to list Lambda functions.

Args:

master_region (str, optional): for Lambda@Edge functions, the AWS region of the master function
function_version (str, optional): the version of a function, default is 'ALL'
marker (str, optional): specify the pagination token that's returned by a previous request to retreive the next page of results
max_items (int, optional): specify a value between 1 and 50 to limit the number of functions in the response
boto_kwargs (dict, optional): additional keyword arguments to forward to the boto client.
**kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:

methods:
prefect.tasks.aws.lambda_function.LambdaList.run (credentials=None)[source]
Task fun method. Lists all Lambda functions. Args: `credentials (dict, optional)`: your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: `ACCESS_KEY` and `SECRET_ACCESS_KEY` which will be passed directly to `boto3`. If not provided here or in context, `boto3` will fall back on standard AWS rules for authentication. Returns: `dict`: a list of Lambda functions from AWS ListFunctions endpoint

prefect.tasks.aws.lambda_function.LambdaList.run

(credentials=None)[source]

Task fun method. Lists all Lambda functions.

Args:

credentials (dict, optional): your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY which will be passed directly to boto3. If not provided here or in context, boto3 will fall back on standard AWS rules for authentication.

Returns:

dict: a list of Lambda functions from AWS ListFunctions endpoint

# StepActivate

class

prefect.tasks.aws.step_function.StepActivate

(state_machine_arn=None, execution_name=None, execution_input="{}", boto_kwargs=None, **kwargs)[source]

Task for activating an AWS Step Function.

Args:

state_machine_arn (str): the Amazon Resource Name (ARN) of the state machine to execute
execution_name (str): the name of the execution, this name must be unique for your AWS account, region, and state machine for 90 days
execution_input (str, optional, DEPRECATED): string that contains the JSON input data for the execution, defaults to "{}". Setting execution_input through the constructor is deprecated. Set it when running this task instead.
boto_kwargs (dict, optional): additional keyword arguments to forward to the boto client.
**kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:

methods:
prefect.tasks.aws.step_function.StepActivate.run (credentials=None, state_machine_arn=None, execution_name=None, execution_input=None, boto_kwargs=None)[source]
Task run method. Activates AWS Step function. Args: `credentials (dict, optional)`: your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: `ACCESS_KEY` and `SECRET_ACCESS_KEY` which will be passed directly to `boto3`. If not provided here or in context, `boto3` will fall back on standard AWS rules for authentication. `state_machine_arn (str)`: the Amazon Resource Name (ARN) of the state machine to execute `execution_name (str)`: the name of the execution, this name must be unique for your AWS account, region, and state machine for 90 days `execution_input (str, optional)`: string that contains the JSON input data for the execution, defaults to `"{}"` `boto_kwargs (dict, optional)`: additional keyword arguments to forward to the boto client. Returns: `dict`: response from AWS StartExecution endpoint

prefect.tasks.aws.step_function.StepActivate.run

(credentials=None, state_machine_arn=None, execution_name=None, execution_input=None, boto_kwargs=None)[source]

Task run method. Activates AWS Step function.

Args:

credentials (dict, optional): your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY which will be passed directly to boto3. If not provided here or in context, boto3 will fall back on standard AWS rules for authentication.
state_machine_arn (str): the Amazon Resource Name (ARN) of the state machine to execute
execution_name (str): the name of the execution, this name must be unique for your AWS account, region, and state machine for 90 days
execution_input (str, optional): string that contains the JSON input data for the execution, defaults to "{}"
boto_kwargs (dict, optional): additional keyword arguments to forward to the boto client.

Returns:

dict: response from AWS StartExecution endpoint

# AWSSecretsManager

class

prefect.tasks.aws.secrets_manager.AWSSecretsManager

(secret=None, boto_kwargs=None, **kwargs)[source]

Task for retrieving secrets from an AWS Secrets Manager and returning it as a dictionary. Note that all initialization arguments can optionally be provided or overwritten at runtime.

Args:

secret (str, optional): The name of the secret to retrieve.
boto_kwargs (dict, optional): Additional keyword arguments to forward to the boto client.
**kwargs (dict, optional): Additional keyword arguments to pass to the Task constructor.

methods:

methods:
prefect.tasks.aws.secrets_manager.AWSSecretsManager.run (secret=None, credentials=None)[source]
Task run method. Args: `secret (str)`: The name of the secret to retrieve. `credentials (dict, optional)`: Your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: `ACCESS_KEY` and `SECRET_ACCESS_KEY` which will be passed directly to `boto3`. If not provided here or in context, `boto3` will fall back on standard AWS rules for authentication. Returns: `Union[dict, str]`: The contents of this secret. Either as a dictionary or as a string.

prefect.tasks.aws.secrets_manager.AWSSecretsManager.run

(secret=None, credentials=None)[source]

Task run method.

Args:

secret (str): The name of the secret to retrieve.
credentials (dict, optional): Your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY which will be passed directly to boto3. If not provided here or in context, boto3 will fall back on standard AWS rules for authentication.

Returns:

Union[dict, str]: The contents of this secret. Either as a dictionary or as a string.

# AWSParametersManager

class

prefect.tasks.aws.parameter_store_manager.AWSParametersManager

(parameter_name=None, boto_kwargs=None, **kwargs)[source]

Task for retrieving values from AWS SSM Parameters Store and returning the parameter value. Note that all initialization arguments can optionally be provided or overwritten at runtime. For authentication, there are two options: you can set the AWS_CREDENTIALS Prefect Secret containing your AWS access keys, which will be passed directly to the boto3 client, or you can configure your flow's runtime environment for boto3.

Args:

parameter_name (str, optional): The name of the parameter to retrieve via SSM.
boto_kwargs (dict, optional): Additional keyword arguments to forward to the boto client.
**kwargs (dict, optional): Additional keyword arguments to pass to the Task constructor.

methods:

methods:
prefect.tasks.aws.parameter_store_manager.AWSParametersManager.run (parameter_name=None, credentials=None, boto_kwargs=None)[source]
Task run method. Args: `parameter_name (str)`: The name of the parameter to retrieve from SSM. `credentials (dict, optional)`: Your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: `ACCESS_KEY` and `SECRET_ACCESS_KEY` which will be passed directly to `boto3`. If not provided here or in context, `boto3` will fall back on standard AWS rules for authentication. `boto_kwargs (dict, optional)`: Additional keyword arguments to forward to the boto client. Returns: `str`: The parameter value, as a string.

prefect.tasks.aws.parameter_store_manager.AWSParametersManager.run

(parameter_name=None, credentials=None, boto_kwargs=None)[source]

Task run method.

Args:

parameter_name (str): The name of the parameter to retrieve from SSM.
credentials (dict, optional): Your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY which will be passed directly to boto3. If not provided here or in context, boto3 will fall back on standard AWS rules for authentication.
boto_kwargs (dict, optional): Additional keyword arguments to forward to the boto client.

Returns:

str: The parameter value, as a string.

# BatchSubmit

class

prefect.tasks.aws.batch.BatchSubmit

(job_name=None, job_definition=None, job_queue=None, boto_kwargs=None, **kwargs)[source]

Task for submitting a job to AWS batch.

Args:

job_name (str, optional): The AWS batch job name.
job_definition (str, optional): The AWS batch job definition.
job_queue (str, optional): Name of the AWS batch job queue.
boto_kwargs (dict, optional): additional kekyword arguments to forward to the boto client.
**kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:

methods:
prefect.tasks.aws.batch.BatchSubmit.run (job_name=None, job_definition=None, job_queue=None, batch_kwargs=None, credentials=None)[source]
Submit a job to the AWS Batch job service. Args: `job_name (str, optional)`: The AWS batch job name. `job_definition (str, optional)`: The AWS batch job definition. `job_queue (str, optional)`: Name of the AWS batch job queue. `batch_kwargs (dict, optional)`: Additional keyword arguments to pass to the boto3 `submit_job` function. See the submit_job # noqa documentation for more details. `credentials (dict, optional)`: your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: `ACCESS_KEY` and `SECRET_ACCESS_KEY` which will be passed directly to `boto3`. If not provided here or in context, `boto3` will fall back on standard AWS rules for authentication.

prefect.tasks.aws.batch.BatchSubmit.run

(job_name=None, job_definition=None, job_queue=None, batch_kwargs=None, credentials=None)[source]

Submit a job to the AWS Batch job service.

Args:

job_name (str, optional): The AWS batch job name.
job_definition (str, optional): The AWS batch job definition.
job_queue (str, optional): Name of the AWS batch job queue.
batch_kwargs (dict, optional): Additional keyword arguments to pass to the boto3 submit_job function. See the submit_job # noqa documentation for more details.
credentials (dict, optional): your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY which will be passed directly to boto3. If not provided here or in context, boto3 will fall back on standard AWS rules for authentication.

# AWSClientWait

class

prefect.tasks.aws.client_waiter.AWSClientWait

(client=None, waiter_name=None, waiter_definition=None, boto_kwargs=None, **kwargs)[source]

Task for waiting on a long-running AWS job. Uses the underlying boto3 waiter functionality.

Args:

client (str, optional): The AWS client on which to wait (e.g., 'batch', 'ec2', etc)
waiter_name (str, optional): The name of the waiter to instantiate. Can be a boto-supported waiter or one of prefect's custom waiters. Currently, prefect offers three additional waiters for AWS Batch: "JobExists" waits for a job to be instantiated, "JobRunning" waits for a job to start running, and "JobComplete" waits for a job to finish. You can find the definitions for all prefect-defined waiters here. # noqa You may also use a custom waiter name, if you supply an accompanying waiter definition dict.
waiter_definition (dict, optional): A valid custom waiter model, as a dict. Note that if you supply a custom definition, it is assumed that the provided 'waiter_name' is contained within the waiter definition dict.
boto_kwargs (dict, optional): additional kekyword arguments to forward to the boto client.
**kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:

methods:
prefect.tasks.aws.client_waiter.AWSClientWait.run (client=None, waiter_name=None, waiter_definition=None, waiter_kwargs=None, credentials=None)[source]
Task for waiting on a long-running AWS job. Uses the underlying boto3 waiter functionality. Args: `client (str)`: The AWS client on which to wait (e.g., 'batch', 'ec2', etc) `waiter_name (str, optional)`: The name of the waiter to instantiate. Can be a boto-supported waiter or one of prefect's custom waiters. Currently, prefect offers three additional waiters for AWS Batch: `"JobExists"` waits for a job to be instantiated, `"JobRunning"` waits for a job to start running, and `"JobComplete"` waits for a job to finish. You can find the definitions for all prefect-defined waiters here. # noqa You may also use a custom waiter name, if you supply an accompanying waiter definition dict. `waiter_definition (dict, optional)`: A valid custom waiter model, as a dict. Note that if you supply a custom definition, it is assumed that the provided 'waiter_name' is contained within the waiter definition dict. `waiter_kwargs (dict, optional)`: Arguments to pass to the `waiter.wait(...)` method. Will depend upon the specific waiter being called. `credentials (dict, optional)`: your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: `ACCESS_KEY` and `SECRET_ACCESS_KEY` which will be passed directly to `boto3`. If not provided here or in context, `boto3` will fall back on standard AWS rules for authentication.

prefect.tasks.aws.client_waiter.AWSClientWait.run

(client=None, waiter_name=None, waiter_definition=None, waiter_kwargs=None, credentials=None)[source]

Task for waiting on a long-running AWS job. Uses the underlying boto3 waiter functionality.

Args:

client (str): The AWS client on which to wait (e.g., 'batch', 'ec2', etc)
waiter_name (str, optional): The name of the waiter to instantiate. Can be a boto-supported waiter or one of prefect's custom waiters. Currently, prefect offers three additional waiters for AWS Batch: "JobExists" waits for a job to be instantiated, "JobRunning" waits for a job to start running, and "JobComplete" waits for a job to finish. You can find the definitions for all prefect-defined waiters here. # noqa You may also use a custom waiter name, if you supply an accompanying waiter definition dict.
waiter_definition (dict, optional): A valid custom waiter model, as a dict. Note that if you supply a custom definition, it is assumed that the provided 'waiter_name' is contained within the waiter definition dict.
waiter_kwargs (dict, optional): Arguments to pass to the waiter.wait(...) method. Will depend upon the specific waiter being called.
credentials (dict, optional): your AWS credentials passed from an upstream Secret task; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY which will be passed directly to boto3. If not provided here or in context, boto3 will fall back on standard AWS rules for authentication.

This documentation was auto-generated from commit bd9182e
on July 31, 2024 at 18:02 UTC

← Asana Tasks Azure Tasks →