Prefect Docs

# Census Tasks

Verified by Prefect

These tasks have been tested and verified by Prefect.

This module contains a task for starting and monitoring Census sync jobs

# CensusSyncTask

class

prefect.tasks.census.census.CensusSyncTask

(api_trigger=None, **kwargs)[source]

Task for running Census connector sync jobs.

This task assumes the user has a Census sync already configured and is attempting to orchestrate the sync using Prefect task to send a post to the API within a prefect flow. Copy and paste from the api trigger section on the configuration page in the api_trigger param to set a default sync.

Args:

  • api_trigger (str, optional): Default sync to trigger, if none is specified in run. The API trigger URL for a sync can be found on sync's configuration page (https://app.getcensus.com/syncs/{sync_id}/configuration) under Sync Triggers > API.
  • **kwargs (dict, optional): Additional kwargs to pass to the base Task constructor.
Example: Trigger a Census sync with an api_trigger stored in a Prefect secret:

    from prefect import Flow
    from prefect.tasks.census import CensusSyncTask
    from prefect.tasks.secrets import PrefectSecret

    sync_census = CensusSyncTask()

    with Flow("Run Census Sync") as flow:
        api_trigger = PrefectSecret('CENSUS_API_TRIGGER')
        results = sync_census(api_trigger=api_trigger)
methods:                                                                                                                                                       

prefect.tasks.census.census.CensusSyncTask.check_invalid_api

(api_trigger)[source]

Makes sure the URL for the API trigger matches the Census format specified below. If it does not, it will raise a ValueError.

Format of api_trigger: - https://bearer:secret-token:{secret}@app.getcensus.com/api/v1/syncs/{sync_id}/trigger

Args:

  • api_trigger (str): If specified in the constructor, will call this validation there
Returns:
  • confirmed_pattern (Match Object - https😕/docs.python.org/3/library/re.html#match-objects)

prefect.tasks.census.census.CensusSyncTask.run

(api_trigger, poll_status_every_n_seconds=60)[source]

Task run method for Census syncs.

An invocation of run will attempt to start a sync job for the specified api_trigger. run will poll Census for the sync status, and will only complete when the sync has completed or when it receives an error status code from the trigger API call.

Args:

  • api_trigger (str): Ff not specified in run, it will pull from the default for the CensusSyncTask constructor. The API trigger URL for a sync can be found on sync's configuration page (https://app.getcensus.com/syncs/{sync_id}/configuration) under Sync Triggers > API.
  • poll_status_every_n_seconds (int, optional): This task polls the Census API for the sync's status. If provided, this value will override the default polling time of 60 seconds and it has a minimum wait time of 5 seconds. Keyword argument.
Returns:
  • dict: Dictionary of statistics returned by Census on the specified sync in following structure: 
            {
                'error_message': None / str,
                'records_failed': int / None,
                'records_invalid': int / None,
                'records_processed': int / None,
                'records_updated': int / None,
                'status': 'completed'/'working'/'failed'
            }


This documentation was auto-generated from commit bd9182e
on July 31, 2024 at 18:02 UTC

← Collection Tasks →