Basic Task Structure

This section is designed to show how tasks are generally constructed in luisy. Other than in luigi, luisy tasks are mainly defined through decorators. Python decorators are adding functionality to existing functions of a class. luisy offers a wide range of decorators that will help you to make your task as you need it.

A basic task in luigi needs to have some decorators attached to be a working task inside a luisy pipeline:

• Output

  Every luisy.tasks.base.Task has a corresponding output, that is located somewhere on your local machine. The type of your output can be defined with one of the output decorators that luisy offers. If no output decorator is attached to your task, the default (pickle_output()) will be used

• Filename

  The output decorator only tells luisy which fileending to use and how to save your object. The basename of your outputfile, is usually created by decorating your task with auto_filename(). If you choose to have a specific filename, you have to overwrite the method get_file_name().

• requires/inherits

  These decorators are originally taken from luigi (See luigi .util). luisy tries to encourage the user to use requires() and inherits to prevent a lot of boilerplate code in your pipelines.

• (optional) Subdir layer

  Best practice to manage your pipeline in raw/interim/final subdirectories. luisy offers decorators to define tasks in their specific locations (raw(), interim(), final()). For more information check out getting_started.