Prefect Docs

# Vertex Agent

Looking for the latest Prefect 2 release? Prefect 2 and Prefect Cloud 2 have been released for General Availability. See https://docs.prefect.io/ for details.

The Vertex Agent executes flow runs as Vertex Custom Jobs. Vertex describes these as "training" jobs, but they can be used to run any kind of flow.

# Requirements

The required dependencies for the Vertex Agent aren't installed by default. You'll need to add the gcp extra via pip.

pip install prefect[gcp]

Prefect Server

In order to use this agent with Prefect Server the server's GraphQL API endpoint must be accessible. This may require changes to your Prefect Server deployment and/or configuring the Prefect API address on the agent.

# Flow Configuration

The Vertex Agent will deploy flows using either a UniversalRun (the default) or VertexRun run_config. Using a VertexRun object lets you customize the deployment environment for a flow (exposing env, image, machine_type, etc...):

from prefect.run_configs import VertexRun

# Configure extra environment variables for this flow,
# and set a custom image and machine type
flow.run_config = VertexRun(
    env={"SOME_VAR": "VALUE"},
    image="my-custom-image",
    machine_type="e2-highmem-16",
)

See the VertexRun documentation for more information.

# Agent Configuration

The Vertex agent can be started from the Prefect CLI as

prefect agent vertex start

API Keys Cloud

When using Prefect Cloud, this will require a service account API key, see here for more information.

Below we cover a few common configuration options, see the CLI docs for a full list of options.

# Project

By default the agent will deploy flow run tasks into the current project (as defined by google.auth.default) You can specify a different project using the --project option:

prefect agent vertex start --project my-project

This can be a different project than the agent is running in, as long as the account has permissions to start Vertex Custom Jobs in the specified project.

# Region

Vertex requires a region in which to run the flow, and will default to us-central1 You can specify a different region using the --region-name option:

prefect agent vertex start --region-name us-east1

# Service Account

Vertex jobs can run as a specified service account. Vertex provides a default, but specifying a specific account can give you more control over what resources the flow runs are allowed to access. You can specify a non-default account using the --service-account option:

prefect agent vertex start --service-account my-account@my-project.iam.gserviceaccount.com

← ECS Agent →