Prefect Docs

# Census Tasks

This module contains a task for starting and monitoring Census sync jobs

# CensusSyncTask

class

prefect.tasks.census.census.CensusSyncTask

(api_trigger=None, **kwargs)[source]

Task for running Census connector sync jobs.

This task assumes the user has a Census sync already configured and is attempting to orchestrate the sync using Prefect task to send a post to the API within a prefect flow. Copy and paste from the api trigger section on the configuration page in the api_trigger param to set a default sync.

Args:

  • api_trigger (str, optional): default sync to trigger, if none is specified in run
  • **kwargs (dict, optional): additional kwargs to pass to the base Task constructor

methods:                                                                                                                                                       

prefect.tasks.census.census.CensusSyncTask.check_invalid_api

(api_trigger)[source]

Makes sure the url for the api trigger matches the Census format specified below. If it does not, it will raise a ValueError before returning.

Format of api_trigger: - "https://bearer:secret-token:s3cr3t@app.getcensus.com/api/v1/syncs/123/trigger"

Args:

  • api_trigger (str): if specified in the constructor, will call this validation there
Returns:
  • confirmed_pattern (Match Object: https://docs.python.org/3/library/re.html#match-objects)

prefect.tasks.census.census.CensusSyncTask.run

(api_trigger, poll_status_every_n_seconds=60)[source]

Task run method for Census syncs.

An invocation of run will attempt to start a sync job for the specified api_trigger. run will poll Census for the sync status, and will only complete when the sync has completed or when it receives an error status code from the trigger API call.

Args:

  • api_trigger (str): if not specified in run, it will pull from the default for the CensusSyncTask constructor. Keyword argument.
  • poll_status_every_n_seconds (int, optional): this task polls the Census API for the sync's status. If provided, this value will override the default polling time of 60 seconds and it has a minimum wait time of 5 seconds. Keyword argument.
Returns:
  • dict: dictionary of statistics returned by Census on the specified sync, structure below.
Structure: 
    {
        'error_message': None / str,
        'records_failed': int / None,
        'records_invalid': int / None,
        'records_processed': int / None,
        'records_updated': int / None,
        'status': 'completed'/'working'/'failed'
    }


This documentation was auto-generated from commit n/a
on February 23, 2022 at 19:26 UTC

← Collection Tasks →