# Handling Failure

Prefect 2

With general availability of Prefect 2, we recommend new users start with Prefect 2.

Follow along in the Terminal

cd examples/tutorial
python 04_handle_failures.py

# If at first you don't succeed...

Now that we have a working ETL flow let's take further steps to ensure its robustness. The extract_* tasks are making web requests to external APIs in order to fetch the data. What if the API is unavailable for a short period? Or if a single request times out for unknown reasons? Prefect Tasks can be retried on failure; let's add this to our extract_* tasks:




from datetime import timedelta
import aircraftlib as aclib
from prefect import task, Flow, Parameter

@task(max_retries=3, retry_delay=timedelta(seconds=10))
def extract_reference_data():
    # same as before ...

@task(max_retries=3, retry_delay=timedelta(seconds=10))
def extract_live_data(airport, radius, ref_data):
    # same as before ...

This is a simple measure that helps our Flow gracefully handle transient errors in only the tasks we specify. Now if there are any failed web requests, a maximum of 3 attempts will be made, waiting 10 seconds between each attempt.

More Ways to Handle Failures

There are other mechanisms Prefect provides to enable specialized behavior around failures:

Up Next!

Schedule our Flow to run periodically or on a custom schedule.