Prefect Docs

# Result Subclasses

Result subclasses are the backbone of tracking the value, type and optional persistence method of return values from tasks by exposing a read / write / exists interface for common storage backends. Results can also be instantiated directly by the user in a task to use those methods to interact with persistent storage and track data besides a task's return value.

A results read / write / exists methods depends on a result's location attribute, which can be a concrete string or a templated string that will be formatted at time of write using prefect.context.

Note that a result's read and write methods return new Result instances, the former with their location attribute formatted, and the latter with the value attribute hydrated.

For example, here is how you would use a result in a task directly to read and write arbitrary pieces of data:

import prefect
from prefect import task
from prefect.engine.results import S3Result

MY_RESULTS = S3Result(bucket='my_bucket', location="{task_name}.txt")

@task(name="my_example_task")
def my_example_task():
    # read data from a file in the bucket.
    my_task_data = MY_RESULTS.read(location="some_data_in_my_bucket.csv")
    print(my_task_data.value) # is the deserialized data that was in the file s3://my_bucket/some_data_in_my_bucket.csv

    # write data to the templated location in the bucket using prefect context
    data = 3
    my_task_data = MY_RESULTS.write(data, **prefect.context)
    print(my_task_data.value) # is the value `3
    print(my_task_data.location) # is "my_example_task.txt"

Results will only persist return data if checkpointing is turned on. To learn more about how to use results and how to configure checkpointing, read our tutorial on Using Results.

# PrefectResult

class

prefect.engine.results.prefect_result.PrefectResult

(**kwargs)[source]

Hook for storing and retrieving JSON serializable Python objects that can safely be stored directly in a Prefect database.

Args:

  • **kwargs (Any, optional): any additional Result initialization options

methods:                                                                                                                                                       

prefect.engine.results.prefect_result.PrefectResult.exists

(location, **kwargs)[source]

Confirms that the provided value is JSON deserializable.

Args:

  • location (str): the value to test
  • **kwargs (Any): unused, for compatibility with the interface
Returns:
  • bool: whether the provided string can be deserialized

prefect.engine.results.prefect_result.PrefectResult.read

(location)[source]

Returns the underlying value regardless of the argument passed.

Args:

  • location (str): an unused argument
Returns:
  • Result: a new result instance with the data represented by the location

prefect.engine.results.prefect_result.PrefectResult.write

(value_, **kwargs)[source]

JSON serializes self.value and returns self.

Args:

  • value_ (Any): the value to write; will then be stored as the value attribute of the returned Result instance
  • **kwargs (optional): unused, for compatibility with the interface
Returns:
  • Result: returns a new Result with both value and location attributes


# GCSResult

class

prefect.engine.results.gcs_result.GCSResult

(bucket=None, **kwargs)[source]

Result that is written to and read from a Google Cloud Bucket.

To authenticate with Google Cloud, you need to ensure that your flow's runtime environment has the proper credentials available (see https://cloud.google.com/docs/authentication/production for all the authentication options).

You can also optionally provide your service account key to prefect.context.secrets.GCP_CREDENTIALS for automatic authentication - see Third Party Authentication for more information.

To read more about service account keys see https://cloud.google.com/iam/docs/creating-managing-service-account-keys. To read more about the JSON representation of service account keys see https://cloud.google.com/iam/docs/reference/rest/v1/projects.serviceAccounts.keys.

Args:

  • bucket (str): the name of the bucket to write to / read from
  • **kwargs (Any, optional): any additional Result initialization options

methods:                                                                                                                                                       

prefect.engine.results.gcs_result.GCSResult.exists

(location, **kwargs)[source]

Checks whether the target result exists.

Does not validate whether the result is valid, only that it is present.

Args:

  • location (str): Location of the result in the specific result target. Will check whether the provided location exists
  • **kwargs (Any): string format arguments for location
Returns:
  • bool: whether or not the target result exists.

prefect.engine.results.gcs_result.GCSResult.read

(location)[source]

Reads a result from a GCS bucket and returns a corresponding Result instance.

Args:

  • location (str): the GCS URI to read from
Returns:
  • Result: the read result

prefect.engine.results.gcs_result.GCSResult.write

(value_, **kwargs)[source]

Writes the result value to a location in GCS and returns the resulting URI.

Args:

  • value_ (Any): the value to write; will then be stored as the value attribute of the returned Result instance
  • **kwargs (optional): if provided, will be used to format the location template to determine the location to write to
Returns:
  • Result: a new Result instance with the appropriately formatted location


# LocalResult

class

prefect.engine.results.local_result.LocalResult

(dir=None, validate_dir=True, **kwargs)[source]

Result that is written to and retrieved from the local file system.

Note: "local" refers to where the flow's tasks execute, which is not necessarily the same as the place that flow.run() runs from. So, for example, if you use a LocalEnvironment with a DaskExecutor pointed at a remote Dask cluster, LocalResult files will be written to the Dask workers' file system.

Note: If this result raises a PermissionError that could mean it is attempting to write results to a directory that it is not permissioned for. In that case it may be helpful to specify a specific dir for that result instance.

Args:

  • dir (str, optional): the absolute path to a directory for storing all results; defaults to ${prefect.config.home_dir}/results
  • validate_dir (bool, optional): a boolean specifying whether to validate the provided directory path; if True, the directory will be converted to an absolute path and created. Defaults to True
  • **kwargs (Any, optional): any additional Result initialization options

methods:                                                                                                                                                       

prefect.engine.results.local_result.LocalResult.exists

(location, **kwargs)[source]

Checks whether the target result exists in the file system.

Does not validate whether the result is valid, only that it is present.

Args:

  • location (str): Location of the result in the specific result target. Will check whether the provided location exists
  • **kwargs (Any): string format arguments for location
Returns:
  • bool: whether or not the target result exists

prefect.engine.results.local_result.LocalResult.read

(location)[source]

Reads a result from the local file system and returns the corresponding Result instance.

Args:

  • location (str): the location to read from
Returns:
  • Result: a new result instance with the data represented by the location

prefect.engine.results.local_result.LocalResult.write

(value_, **kwargs)[source]

Writes the result to a location in the local file system and returns a new Result object with the result's location.

Args:

  • value_ (Any): the value to write; will then be stored as the value attribute of the returned Result instance
  • **kwargs (optional): if provided, will be used to format the location template to determine the location to write to
Returns:
  • Result: returns a new Result with both value and location attributes


# S3Result

class

prefect.engine.results.s3_result.S3Result

(bucket, boto3_kwargs=None, upload_options=None, **kwargs)[source]

Result that is written to and retrieved from an AWS S3 Bucket.

For authentication, there are two options: you can provide your AWS credentials to prefect.context.secrets.AWS_CREDENTIALS for automatic authentication or you can configure your flow's runtime environment for boto3.

See Third Party Authentication for more information.

Args:

  • bucket (str): the name of the bucket to write to / read from
  • boto3_kwargs (dict, optional): keyword arguments to pass on to boto3 when the client session is initialized.
  • upload_options (dict, optional): Additional options for s3 client upload_fileobj() method ExtraArgs argument
  • **kwargs (Any, optional): any additional Result initialization options

methods:                                                                                                                                                       

prefect.engine.results.s3_result.S3Result.exists

(location, **kwargs)[source]

Checks whether the target result exists in the S3 bucket.

Does not validate whether the result is valid, only that it is present.

Args:

  • location (str): Location of the result in the specific result target.
  • **kwargs (Any): string format arguments for location
Returns:
  • bool: whether or not the target result exists.

prefect.engine.results.s3_result.S3Result.read

(location)[source]

Reads a result from S3, reads it and returns a new Result object with the corresponding value.

Args:

  • location (str): the S3 URI to read from
Returns:
  • Any: the read result

prefect.engine.results.s3_result.S3Result.write

(value_, **kwargs)[source]

Writes the result to a location in S3 and returns the resulting URI.

Args:

  • value_ (Any): the value to write; will then be stored as the value attribute of the returned Result instance
  • **kwargs (optional): if provided, will be used to format the location template to determine the location to write to
Returns:
  • Result: a new Result instance with the appropriately formatted S3 URI


# AzureResult

class

prefect.engine.results.azure_result.AzureResult

(container, connection_string=None, connection_string_secret=None, **kwargs)[source]

Result for writing to and reading from an Azure Blob storage.

Note that your flow's runtime environment must be able to authenticate with Azure; there are currently two supported options: provide a connection string either at initialization or at runtime through an environment variable, or set your Azure connection string as a Prefect Secret. Using an environment variable is the recommended approach.

Args:

  • container (str): the name of the container to write to / read from
  • connection_string (str, optional): an Azure connection string for communicating with Blob storage. If not provided the value set in the environment as AZURE_STORAGE_CONNECTION_STRING will be used
  • connection_string_secret (str, optional): the name of a Prefect Secret which stores your Azure connection string
  • **kwargs (Any, optional): any additional Result initialization options

methods:                                                                                                                                                       

prefect.engine.results.azure_result.AzureResult.exists

(location, **kwargs)[source]

Checks whether the target result exists.

Does not validate whether the result is valid, only that it is present.

Args:

  • location (str): Location of the result in the specific result target. Will check whether the provided location exists
  • **kwargs (Any): string format arguments for location
Returns:
  • bool: whether or not the target result exists.

prefect.engine.results.azure_result.AzureResult.initialize_service

()[source]

Initialize a Blob service.

prefect.engine.results.azure_result.AzureResult.read

(location)[source]

Reads a result from an Azure Blob container and returns a corresponding Result instance.

Args:

  • location (str): the Azure blob location to read from
Returns:
  • Result: the read result

prefect.engine.results.azure_result.AzureResult.write

(value_, **kwargs)[source]

Writes the result value to a blob storage in Azure.

Args:

  • value_ (Any): the value to write; will then be stored as the value attribute of the returned Result instance
  • **kwargs (optional): if provided, will be used to format the location template to determine the location to write to
Returns:
  • Result: a new Result instance with the appropriately formatted location


# SecretResult

class

prefect.engine.results.secret_result.SecretResult

(secret_task, **kwargs)[source]

Hook for storing and retrieving sensitive task results from a Secret store. Only intended to be used for Secret Tasks - each call to "read" will actually rerun the underlying secret task and re-retrieve the secret value.

Args:

  • secret_task (Task): the Secret Task this result wraps
  • **kwargs (Any, optional): additional kwargs to pass to the Result initialization

methods:                                                                                                                                                       

prefect.engine.results.secret_result.SecretResult.read

(location)[source]

Returns the Secret Value corresponding to the passed name.

Args:

  • location (str): the name of the Secret to retrieve
Returns:
  • Result: a new result instance with the data represented by the location

prefect.engine.results.secret_result.SecretResult.write

(value_, **kwargs)[source]

Secret results cannot be written to; provided for interface compatibility.

Args:

  • value_ (Any): unused, for interface compatibility
  • **kwargs (optional): unused, for interface compatibility
Raises:
  • ValueError: SecretResults cannot be written to


# ConstantResult

class

prefect.engine.results.constant_result.ConstantResult

(**kwargs)[source]

Hook for storing and retrieving constant Python objects. Only intended to be used internally. The "backend" in this instance is the class instance itself.

Args:

  • **kwargs (Any, optional): any additional Result initialization options

methods:                                                                                                                                                       

prefect.engine.results.constant_result.ConstantResult.exists

(location, **kwargs)[source]

As all Python objects are valid constants, always returns True.

Args:

  • location (str): for interface compatibility
  • **kwargs (Any): string format arguments for location
Returns:
  • bool: True, confirming the constant exists.

prefect.engine.results.constant_result.ConstantResult.read

(location)[source]

Will return the underlying value regardless of the argument passed.

Args:

  • location (str): an unused argument

prefect.engine.results.constant_result.ConstantResult.write

(value_, **kwargs)[source]

Will return the repr of the underlying value, purely for convenience.

Args:

  • value_ (Any): unused, for interface compatibility
  • **kwargs (optional): unused, for interface compatibility
Raises:
  • ValueError: ConstantResults cannot be written to


This documentation was auto-generated from commit bd9182e
on July 31, 2024 at 18:02 UTC

← Result Serializers →