# Contributing
Contributing to Prefect's task library is a great way to assist in open source development! Generally users have contributed tasks to the task library that accomplish a specific goal or action.
There are a few key reasons why users would want to contribute tasks to the task library:
- Gain experience contributing to an open source project
- Increase adoption for libraries, tools, and frameworks by making an easy route for users of Prefect to interact with
- Allow for tasks to evolve with Prefect meaning that as paradigms and abstractions change in Prefect the task in the open source library will change with it
- Open up collaboration to thousands of other developers who could use your task (they might fix bugs in the task you weren't aware of!)
Not to mention that we occasionally also send Prefect swag to some of our open source contributors!
# Task Structure
In order to build a task for the task library you need to define the task's __init__ and run
functions. The __init__ of the task will be called before the flow runs and the run function is where
any of your task's logic will live to be executed at runtime. Allowing for kwargs to be set both during
initialization and at runtime is key to improving a task's functionality.
One example of this separation in action would be initializing a ShellTask with a specific shell and
then passing in a different command at runtime. This will create two tasks in the flow, each with
different commands, without having to redefine the shell type.
my_shell = ShellTask(shell="bash")
with Flow("shell_commands") as flow:
task1 = my_shell(command="ls")
task2 = my_shell(command="ls | wc -l")
(For more in depth information on the components of Prefect tasks take a look at The Anatomy of a Prefect Task guide.)
This snippet below is the general structure of a task contained in the task library.
from prefect import Task
from prefect.utilities.tasks import defaults_from_attrs
class YourTask(Task):
"""
Task for doing something
Args:
- your_kwarg (string, optional): an optional kwarg
- **kwargs: additional keyword arguments to pass to the Task constructor
"""
def __init__(
self,
your_kwarg: str = None,
**kwargs: Any
):
self.your_kwarg = your_kwarg
super().__init__(**kwargs)
@defaults_from_attrs("your_kwarg")
def run(self, your_kwarg: str = None) -> str:
"""
Run your task
Args:
- your_kwarg (string, optional): an optional kwarg
Returns:
- string: description of what it returned
"""
# Place your specific task run logic inside this function
return use_your_library(your_kwarg)
In the snippet above there is a special decorator defaults_from_attrs. This decorator serves the purpose
of reducing the amount of boilerplate code in the task. If a value is set via initialization of the task
and is not set again at runtime then the value set at initialization will be used in place of the absent
runtime value. However, values set at runtime will always override those set during initialization.
For more examples of how the other tasks in the task library look check out the directory containing all of the task library code.
For more information on contributing to the Prefect library as whole check out the development documentation.
# Secrets and Authentication
It is common for tasks in the task library to require some sort of authentication when interacting with
services. Prefect has a desired implementation when it comes to using credentials in a task and that is
through the use of a Secret task. A PrefectSecret is a special type
of task for interacting with Prefect secrets that securely represents
the retrieval of sensitive data.
Secret tasks are only able to retrieve secret data during runtime therefore it is required that your
secret values be passed into your tasks through the run kwargs:
class YourTask(Task):
def __init__(self, your_kwarg: str = None, **kwargs: Any):
...
# Allows init kwargs to be passed to the run function if they are not overridden
@defaults_from_attrs("your_kwarg")
def run(self, your_kwarg: str = None, your_secret: str = None) -> str:
authenticate_with_your_service(your_secret)
...
This allows users of the task to use Prefect Secrets to securely pass sensitive information to the task using whatever secret storage mechanism they prefer:
from prefect import Flow
from prefect.tasks.secrets import PrefectSecret
from prefect.tasks.your_framework import YourTask
your_task = YourTask(your_kwarg="init kwarg")
with Flow("your-flow") as flow:
your_secret = PrefectSecret("your_secret_name")
your_task(your_secret=your_secret)
# Testing
Due to the nature of the tasks in the task library interacting with a wide range of services from various contributors it is not always possible for Prefect to maintain tests that communicate with all of these services. Because of this, it is integral that users who are contributing tasks to the task library test their tasks themselves against whichever services the tasks interact with.
However, we do still encourage the contribution of unit tests! The unit tests for tasks in the task library generally test that proper variables are set and used with accompanying mocks for the services that they interact with. For examples of how some of the other tasks in the task library are tested check out the tasks testing directory.
# Documentation
Tasks in the task library follow Prefect's standard documentation practices as outlined in the development
page on Documentation. This means that kwargs in the task's
__init__ and run function must be documented in the docstring. Check out any of the other
tasks in the task library as a
point of reference!
In order for new tasks to appear in the API documentation they need to be added to the
outline.toml file in the docs
directory:
[pages.tasks.your_task]
title = "Your Task"
module = "prefect.tasks.your_task"
classes = ["YourTask"]