Prefect Agents are lightweight processes for orchestrating flow runs. Agents run inside a user's architecture, and are responsible for starting and monitoring flow runs. During operation the agent process queries the Prefect API for any scheduled flow runs, and allocates resources for them on their respective deployment platforms.
Note that both Prefect Agents (and flow runs) only send requests out to the Prefect API, and never receive requests themselves. This is part of our Hybrid Execution Model, and helps keep your code and data safe.
A single agent can manage many concurrent flow runs - the only reason to have multiple active agents is if you need to support flow runs on different deployment platforms.
# Agent Types
Prefect supports several different agent types for deploying on different platforms.
Local: The Local Agent executes flow runs as local processes.
Docker: The Docker Agent executes flow runs in docker containers.
See their respective documentation for more information on each type.
Prefect agents can be started via the CLI, using
prefect agent <AGENT TYPE> start. For example, to start a local agent:
prefect agent local start
Alternatively, all Prefect Agents can also be run using the Python API.
from prefect.agent.local import LocalAgent LocalAgent().start()
# Common Configuration Options
The following configuration options are shared for all agents.
# Authentication Cloud
Prefect agents rely on the use of an API key from Prefect Cloud. For information on how to create and configure API keys, see the API key documentation.
In addition to the methods outlined there, you may pass an API key via CLI to an agent
$ prefect agent <agent-type> start --key "API_KEY"
# Prefect API Address
Prefect agents query the API for any pending flow runs. By default the address used is:
https://api.prefect.iofor Prefect Cloud
http://localhost:4200for Prefect Server
If needed, you can manually configure the address through the CLI:
prefect agent <AGENT TYPE> start --api <API ADDRESS>
Agents have an optional
labels argument which allows for separation of
execution when using multiple agents. This is especially useful for teams
wanting to run specific flows on different clusters. For more information on
labels and how to use them see the
Run Configuration docs.
By default, agents have no set labels and will only pick up runs from flows with no specified labels. Labels can be provided to an agent through a few methods:
# Environment Variables
All agents have a
--env flag for configuring environment variables to set on
all flow runs managed by that agent. This can be useful for things you want
applied to all flow runs, whereas the
env option in a flow's
RunConfig only applies to runs of a
# Agent Automations Cloud
Users on Standard or Enterprise licenses in Cloud can create an agent automation to notify them if all agents from a configuration group (agent config ids can be added to multiple agents) have not queried for work in a certain time frame. To do so go to the automations tab of the dashboard in the UI and set up an agent configuration then copy the agent config id that is provided once your automation is created. You can then provide the agent configuration to your agent using the --agent-config-id flag:
prefect agent <AGENT TYPE> start --agent-config-id <AGENT CONFIG ID>
Note - Agent automations can only be added as a flag when starting an agent at present. They can not be added at install.
# Health Checks
Agents can optionally run a private HTTP server for use as a health check.
Health checks can be used by common orchestration services (e.g.
kubernetes, ...) to check that the agent is
running properly and take actions (such as restarting the agent) if it's not.
A few ways to configure:
If enabled, the HTTP health check will be available via the
route at the configured agent address. This route returns
200 OK if the
agent is healthy, and will error otherwise.