freckelize

Description

freckelize is an application that downloads a remote code or data repository (a ‘freckle’) that is structured according to one or some conventions. After download, freckelize will execute pre-defined tasks appropriate for the type of freckle in question.

In order to handle those different data/code profiles, freckelize can be extended with so called adapters. For example, if the freckle is a python project, freckelize will use the python-dev adapter which will create a virtualenv (after, if necessary, downloading everything that is needed to create virtualenvs in the first place), download and install all dependencies it can find in any potential requirement_*.txt files, and then execute either pip install -e . or python setup.py develop inside the created virtualenv.

Or, the freckle is a folder containing subfolders which in turn contain dotfiles (that’s what configuration files are called in Unix-land). freckelize will download this repo, install potentially configured applications that relate to the configuration files, and symbolically link those configuration files to the appropriate places.

Application interface

(Autogenerated) cli help

Adapters (& profiles)

Overview

As freckelize is designed to handle folders of data/code of different types, each using a different structure, it relies on plugins (called ‘adapters’) to process those different types of data (a data ‘profile’).

Currently, only very few types/profiles of data are ‘officially’ supported by freckelize: dotfiles and python dev projects (although, at the time you are reading this this might have changed). To encourage the creation of lots of adapters – in order to support lots of different data profilesfreckelize tries to make it as easy as possible to create new adapters, and to add those adapters to your environment.

A freckelize adapter consists of two to three files, which need to located in the same folder:

adapter-name.adapter.freckle (required)
contains metadata about the adapter. The file is yaml-formated and contains 3 main sections: doc (help text), role-dependencies (Ansible roles that this adapter needs as dependencies), and args (optional, arguments to enable additional user input client-side)
adapter-name.init.freckle (optional)
a list of tasks (in the format of an Ansible tasks file, or playbook) to execute once per run, per adapter.
adapter-name.tasks.freckle (optional)
a list of tasks (in the format of an Ansible tasks file, or playbook) to execute once for every single freckle in a run.

Only one of those latter two needs to exist (which one doesn’t matter), but it’s also possible for both of those to be there. init.freckle contains tasks that are ‘adapter-specific’ (tasks that need to be done the same way – and only once – for every freckle that is processed), tasks.freckle contains tasks that are ‘freckle-specific’ (tasks that need to be done for every freckle), using the freckle specific metadata.

As an example for a reasonably complex adapter, check out the source-code of the python-dev one on github: https://github.com/makkus/freckles/tree/master/freckles/external/default_adapter_repo/python-dev

Adapter folders

By default freckelize comes with a (small) set of ‘officially supported’ adapters which are always available (see below). In addition, by default it checks one other folder for more available adapters: $HOME/.freckles/adapters. Additional locations can be specified by adding either git repository urls or local paths to the trusted-repos config option of the freckles config file ($HOME/.freckles/config.yml). To easily add and retrieve an existing git repo that contains adapters (or roles, for that matter), you can use the enable-repo frecklecutable, e.g.:

frecklecute enable-repo gh:makkus/freckles_roles_and_adapters

This will add the git repo url to the trusted-repos key in $HOME/.freckles/config.yml, and check out the repository into a location using a unique path ($HOME/.local/share/freckles/repos/https/github/com/makkus/freckles/roles/and/adapters/git in this case) where freckelize will find it in subsequent runs.

freckelize will look at all files in the configured folders and check if any of them contains a file that ends with the string .adapter.freckle. If one (or several) is found, it’ll assume the name of the adapter is the first part of the file-name (everything before .adapter.freckle). Then it’ll look for two other files in the same folder, starting with the (same) adapter name and ending with either .init.freckle or tasks.freckle (see above).

Available adapters

Supported adapters

Those are adapters that ship with the freckles python package.

freckles.io adapter repository

There is a repository of community created and maintained freckle adapters, which can easily enabled using the enable-repo frecklecutable:

frecklecutable enable-repo freckles-io

This will check out the freckelize adapters git repo (into: ~/.local/share/freckles/repos/freckles_io), and mark it as trusted in the ~/.freckles/config.yml config file.

Locally available adapters

Which adapters are available to you locally depends on which additional repositories you specify as ‘trusted’ in the freckles config, and whether you have any custom adapters in ~/.freckles/adapters. To quickly list all available adapters, execute freckelize with the --help option:

$ freckelize --help
Usage: freckelize [OPTIONS] ADAPTER1 [ARGS]... [ADAPTER2 [ARGS]...]...

  Downloads a remote dataset or code (called a 'freckle') and sets up
  ...
  ...

                         * more output *

  ...
  ...
Commands:
  debug-freckle  helper adapter, for developing other adapter
  dotfiles       installs packages, stows dotfiles
  python-dev     prepares a python development environment

Blueprints

TBD

Development

freckelize’s goal is not primarily to manage e.g. dotfiles or python development project, but to facilitate handling a multitude of different data profiles. That’s why freckelize is designed in a way to make it easy to create new adapters and add them to a freckelize execution context.

While creating adapters is not as easy as using them, it should still be manageable for people with a rudimentary amount of programming skills. This is mostly due to the fact that Ansible itself, on top of which freckles is built, has a fairly flat learning curve, esp. for a configuration management system. If you already have some experience with Ansible, writing a freckelize adapter should be easy.

If you are keen to get started, check out the relevant documentation: