Prefect Docs

# Shell Tasks

# ShellTask

class

prefect.tasks.shell.ShellTask

(command=None, env=None, helper_script=None, shell="bash", return_all=False, log_stderr=False, stream_output=False, **kwargs)[source]

Task for running arbitrary shell commands.

NOTE: This task combines stderr and stdout because reading from both streams without blocking is tricky.

Args:

  • command (string, optional): shell command to be executed; can also be provided post-initialization by calling this task instance
  • env (dict, optional): dictionary of environment variables to use for the subprocess; can also be provided at runtime
  • helper_script (str, optional): a string representing a shell script, which will be executed prior to the command in the same process. Can be used to change directories, define helper functions, etc. when re-using this Task for different commands in a Flow; can also be provided at runtime
  • shell (string, optional): shell to run the command with; defaults to "bash"
  • return_all (bool, optional): boolean specifying whether this task should return all lines of stdout as a list, or just the last line as a string; defaults to False
  • log_stderr (bool, optional): boolean specifying whether this task should log the output in the case of a non-zero exit code; defaults to False. This actually logs both stderr and stdout and will only log the last line of output unless return_all is True
  • stream_output (Union[bool, int, str], optional): specifies whether this task should log the output as it occurs, and at what logging level. If True is passed, the logging level defaults to INFO; otherwise, any integer or string value that's passed will be treated as the log level, provided the logging library can successfully interpret it. If enabled, log_stderr will be ignored as the output will have already been logged. defaults to False
  • **kwargs: additional keyword arguments to pass to the Task constructor
Raises:
  • TypeError: if stream_output is passed in as a string, but cannot successfully be converted to a numeric value by logging.getLevelName()
Example:

    from prefect import Flow
    from prefect.tasks.shell import ShellTask

    task = ShellTask(helper_script="cd ~")
    with Flow("My Flow") as f:
        # both tasks will be executed in home directory
        contents = task(command='ls')
        mv_file = task(command='mv .vimrc /.vimrc')

    out = f.run()
methods:                                                                                                                                                       

prefect.tasks.shell.ShellTask.run

(command=None, env=None, helper_script=None)[source]

Run the shell command.

Args:

  • command (string): shell command to be executed; can also be provided at task initialization. Any variables / functions defined in helper_script will be available in the same process this command runs in
  • env (dict, optional): dictionary of environment variables to use for the subprocess
  • helper_script (str, optional): a string representing a shell script, which will be executed prior to the command in the same process. Can be used to change directories, define helper functions, etc. when re-using this Task for different commands in a Flow
Returns:
  • output (string): if return_all is False (the default), only the last line of output is returned, otherwise all lines are returned, which is useful for passing result of shell command to other downstream tasks. If there is no output, None is returned.
Raises:
  • prefect.engine.signals.FAIL: if command has an exit code other than 0


This documentation was auto-generated from commit bd9182e
on July 31, 2024 at 18:02 UTC

← Snowflake Tasks →