Cloud

---

CloudFlowRunner

class

prefect.engine.cloud.flow_runner.CloudFlowRunner

(flow, state_handlers=None)[source]

FlowRunners handle the execution of Flows and determine the State of a Flow before, during and after the Flow is run.

In particular, through the FlowRunner you can specify which tasks should be the first tasks to run, which tasks should be returned after the Flow is finished, and what states each task should be initialized with.

Args:

• flow (Flow): the Flow to be run
• state_handlers (Iterable[Callable], optional): A list of state change handlers that will be called whenever the flow changes state, providing an opportunity to inspect or modify the new state. The handler will be passed the flow runner instance, the old (prior) state, and the new (current) state, with the following signature:

            state_handler(
                flow_runner: FlowRunner,
                old_state: State,
                new_state: State) -> State

If multiple functions are passed, then the new_state argument will be the result of the previous handler.

Note: new FlowRunners are initialized within the call to Flow.run() and in general, this is the endpoint through which FlowRunners will be interacted with most frequently.

Example:

@task
def say_hello():
    print('hello')

with Flow("My Flow") as f:
    say_hello()

fr = FlowRunner(flow=f)
flow_state = fr.run()

methods:
prefect.engine.cloud.flow_runner.CloudFlowRunner.call_runner_target_handlers (old_state, new_state)[source]
A special state handler that the FlowRunner uses to call its flow's state handlers. This method is called as part of the base Runner's handle_state_change() method. Args: old_state (State): the old (previous) state new_state (State): the new (current) state Returns: State: the new state
prefect.engine.cloud.flow_runner.CloudFlowRunner.check_for_cancellation ()[source]
Contextmanager used to wrap a cancellable section of a flow run.
prefect.engine.cloud.flow_runner.CloudFlowRunner.initialize_run (state, task_states, context, task_contexts, parameters)[source]
Initializes the Task run by initializing state and context appropriately. If the provided state is a Submitted state, the state it wraps is extracted. Args: state (Optional[State]): the initial state of the run task_states (Dict[Task, State]): a dictionary of any initial task states context (Dict[str, Any], optional): prefect.Context to use for execution to use for each Task run task_contexts (Dict[Task, Dict[str, Any]], optional): contexts that will be provided to each task parameters(dict): the parameter values for the run Returns: NamedTuple: a tuple of initialized objects: (state, task_states, context, task_contexts)
prefect.engine.cloud.flow_runner.CloudFlowRunner.run (state=None, task_states=None, return_tasks=None, parameters=None, task_runner_state_handlers=None, executor=None, context=None, task_contexts=None)[source]
The main endpoint for FlowRunners. Calling this method will perform all computations contained within the Flow and return the final state of the Flow. Args: state (State, optional): starting state for the Flow. Defaults to Pending task_states (dict, optional): dictionary of task states to begin computation with, with keys being Tasks and values their corresponding state return_tasks ([Task], optional): list of Tasks to include in the final returned Flow state. Defaults to None parameters (dict, optional): dictionary of any needed Parameter values, with keys being strings representing Parameter names and values being their corresponding values task_runner_state_handlers (Iterable[Callable], optional): A list of state change handlers that will be provided to the task_runner, and called whenever a task changes state. executor (Executor, optional): executor to use when performing computation; defaults to the executor specified in your prefect configuration context (Dict[str, Any], optional): prefect.Context to use for execution to use for each Task run task_contexts (Dict[Task, Dict[str, Any]], optional): contexts that will be provided to each task Returns: State: State representing the final post-run state of the Flow.

---

CloudTaskRunner

class

prefect.engine.cloud.task_runner.CloudTaskRunner

(task, state_handlers=None, flow_result=None)[source]

TaskRunners handle the execution of Tasks and determine the State of a Task before, during and after the Task is run.

In particular, through the TaskRunner you can specify the states of any upstream dependencies, and what state the Task should be initialized with.

Args:

• task (Task): the Task to be run / executed
• state_handlers (Iterable[Callable], optional): A list of state change handlers that will be called whenever the task changes state, providing an opportunity to inspect or modify the new state. The handler will be passed the task runner instance, the old (prior) state, and the new (current) state, with the following signature: state_handler(TaskRunner, old_state, new_state) -> State; If multiple functions are passed, then the new_state argument will be the result of the previous handler.
• flow_result: the result instance configured for the flow (if any)

methods:
prefect.engine.cloud.task_runner.CloudTaskRunner.call_runner_target_handlers (old_state, new_state)[source]
A special state handler that the TaskRunner uses to call its task's state handlers. This method is called as part of the base Runner's handle_state_change() method. Args: old_state (State): the old (previous) state new_state (State): the new (current) state Returns: State: the new state
prefect.engine.cloud.task_runner.CloudTaskRunner.check_task_is_cached (state, inputs)[source]
Checks if task is cached in the DB and whether any of the caches are still valid. Args: state (State): the current state of this task inputs (Dict[str, Result]): a dictionary of inputs whose keys correspond to the task's run() arguments. Returns: State: the state of the task after running the check Raises: ENDRUN: if the task is not ready to run
prefect.engine.cloud.task_runner.CloudTaskRunner.initialize_run (state, context)[source]
Initializes the Task run by initializing state and context appropriately. Args: state (Optional[State]): the initial state of the run context (Dict[str, Any]): the context to be updated with relevant information Returns: tuple: a tuple of the updated state, context, and upstream_states objects
prefect.engine.cloud.task_runner.CloudTaskRunner.load_results (state, upstream_states)[source]
Given the task's current state and upstream states, populates all relevant result objects for this task run. Args: state (State): the task's current state. upstream_states (Dict[Edge, State]): the upstream state_handlers Returns: Tuple[State, dict]: a tuple of (state, upstream_states)
prefect.engine.cloud.task_runner.CloudTaskRunner.run (state=None, upstream_states=None, context=None, is_mapped_parent=False)[source]
The main endpoint for TaskRunners. Calling this method will conditionally execute self.task.run with any provided inputs, assuming the upstream dependencies are in a state which allow this Task to run. Additionally, this method will wait and perform Task retries which are scheduled for <= 10 minutes in the future. Args: state (State, optional): initial State to begin task run from; defaults to Pending() upstream_states (Dict[Edge, State]): a dictionary representing the states of any tasks upstream of this one. The keys of the dictionary should correspond to the edges leading to the task. context (dict, optional): prefect Context to use for execution is_mapped_parent (bool): a boolean indicating whether this task run is the run of a parent mapped task Returns: State object representing the final post-run state of the Task
prefect.engine.cloud.task_runner.CloudTaskRunner.set_task_run_name (task_inputs)[source]
Sets the name for this task run by calling the set_task_run_name mutation. Args: task_inputs (Dict[str, Result]): a dictionary of inputs whose keys correspond to the task's run() arguments.

---

This documentation was auto-generated from commit bd9182e
on July 31, 2024 at 18:02 UTC