freckelize

Description

freckelize is an application that downloads a remote code or data repository (a ‘freckle’) that is structured according to one or some conventions. After download, freckelize will execute pre-defined tasks appropriate for the type of freckle in question.

In order to handle those different data/code profiles, freckelize can be extended with so called adapters. For example, if the freckle is a python project, freckelize will use the python-dev adapter which will create a virtualenv (after, if necessary, downloading everything that is needed to create virtualenvs in the first place), download and install all dependencies it can find in any potential requirement_*.txt files, and then execute either pip install -e . or python setup.py develop inside the created virtualenv.

Or, the freckle is a folder containing subfolders which in turn contain dotfiles (that’s what configuration files are called in Unix-land). freckelize will download this repo, install potentially configured applications that relate to the configuration files, and symbolically link those configuration files to the appropriate places.

Application interface

(Autogenerated) cli help

freckelize

Downloads a remote dataset or code (called a ‘freckle’) and sets up your local environment to be able to handle the data, according to the data’s profile.

Ideally the remote dataset includes all the metadata that is needed to setup the environment, but it’s possible to provide some directives using commandline options globally (–target, –include, –exclude), or per adapter (use the –help function on each adapter to view those).

Locally available adapters for supported profiles are listed below, each having their own configuration. You can specify a ‘global’ url by adding it’s ‘–freckle’ option before any of the subsequently specified adapters, prompting any of those adapters to apply their tasks to it. Or you can assign one (or multiple) freckle to an adapter by providing it after the adapter name.

For more details, visit the online documentation: https://docs.freckles.io/en/latest/freckelize_command.html

freckelize [OPTIONS] ADAPTER1 [ARGS]... [ADAPTER2 [ARGS]...]...

Options

-f, --freckle <freckle>

the url or path to the freckle(s) to use, if specified here, before any commands, all profiles will be applied to it

-v, --extra-vars <extra_vars>

extra vars, overlayed on top of potential folder/adapter variables

-t, --target-folder <target_folder>

target folder for freckle checkouts (if remote url provided), defaults to folder ‘freckles’ in users home

-i, --include <include>

if specified, only process folders that end with one of the specified strings, only applicable for multi-freckle folders

-e, --exclude <exclude>

if specified, omit process folders that end with one of the specified strings, takes precedence over the include option if in doubt, only applicable for multi-freckle folders

-pw, --ask-become-pass <ask_become_pass>

whether to force ask for a password, force ask not to, or let freckles decide (which might not always work)

--non-recursive

whether to exclude all freckle child folders, default: false

--host <host>

host(s) to freckelize (defaults to ‘localhost’)

-r, --use-repo <use_repo>

extra context repos to use

-o, --output <output>

format of the output

--no-run

don’t execute frecklecute, only prepare environment and print task list

--version

prints the version of freckles

Adapters (& profiles)

Overview

As freckelize is designed to handle folders of data/code of different types, each using a different structure, it relies on plugins (called ‘adapters’) to process those different types of data (a data ‘profile’).

Currently, only very few types/profiles of data are ‘officially’ supported by freckelize: dotfiles and python dev projects (although, at the time you are reading this this might have changed). To encourage the creation of lots of adapters – in order to support lots of different data profilesfreckelize tries to make it as easy as possible to create new adapters, and to add those adapters to your environment.

A freckelize adapter consists of two to three files, which need to located in the same folder:

adapter-name.adapter.freckle (required)
contains metadata about the adapter. The file is yaml-formated and contains 3 main sections: doc (help text), role-dependencies (Ansible roles that this adapter needs as dependencies), and args (optional, arguments to enable additional user input client-side)
adapter-name.init.freckle (optional)
a list of tasks (in the format of an Ansible tasks file, or playbook) to execute once per run, per adapter.
adapter-name.tasks.freckle (optional)
a list of tasks (in the format of an Ansible tasks file, or playbook) to execute once for every single freckle in a run.

Only one of those latter two needs to exist (which one doesn’t matter), but it’s also possible for both of those to be there. init.freckle contains tasks that are ‘adapter-specific’ (tasks that need to be done the same way – and only once – for every freckle that is processed), tasks.freckle contains tasks that are ‘freckle-specific’ (tasks that need to be done for every freckle), using the freckle specific metadata.

As an example for a reasonably complex adapter, check out the source-code of the python-dev one on github: https://github.com/makkus/freckles/tree/master/freckles/external/default_adapter_repo/python-dev

Adapter folders

By default freckelize comes with a (small) set of ‘officially supported’ adapters which are always available (see below). In addition, by default it checks one other folder for more available adapters: $HOME/.freckles/adapters. Additional locations can be specified by adding either git repository urls or local paths to the trusted-repos config option of the freckles config file ($HOME/.freckles/config.yml). To easily add and retrieve an existing git repo that contains adapters (or roles, for that matter), you can use the enable-repo frecklecutable, e.g.:

frecklecute enable-repo gh:makkus/freckles_roles_and_adapters

This will add the git repo url to the trusted-repos key in $HOME/.freckles/config.yml, and check out the repository into a location using a unique path ($HOME/.local/freckles/repos/https/github/com/makkus/freckles/roles/and/adapters/git in this case) where freckelize will find it in subsequent runs.

freckelize will look at all files in the configured folders and check if any of them contains a file that ends with the string .adapter.freckle. If one (or several) is found, it’ll assume the name of the adapter is the first part of the file-name (everything before .adapter.freckle). Then it’ll look for two other files in the same folder, starting with the (same) adapter name and ending with either .init.freckle or tasks.freckle (see above).

Available adapters

Supported adapters

Those are adapters that ship with the freckles python package.

freckles.io adapter repository

There is a repository of community created and maintained freckle adapters, which can easily enabled using the enable-repo frecklecutable:

frecklecutable enable-repo freckles-io

This will check out the freckelize adapters git repo (into: ~/.local/freckles/repos/freckles_io), and mark it as trusted in the ~/.freckles/config.yml config file.

Locally available adapters

Which adapters are available to you locally depends on which additional repositories you specify as ‘trusted’ in the freckles config, and whether you have any custom adapters in ~/.freckles/adapters. To quickly list all available adapters, execute freckelize with the --help option:

$ freckelize --help
Usage: freckelize [OPTIONS] ADAPTER1 [ARGS]... [ADAPTER2 [ARGS]...]...

  Downloads a remote dataset or code (called a 'freckle') and sets up
  ...
  ...

                         * more output *

  ...
  ...
Commands:
  debug-freckle  helper adapter, for developing other adapter
  dotfiles       installs packages, stows dotfiles
  python-dev     prepares a python development environment

Blueprints

TBD

Development

freckelize’s goal is not primarily to manage e.g. dotfiles or python development project, but to facilitate handling a multitude of different data profiles. That’s why freckelize is designed in a way to make it easy to create new adapters and add them to a freckelize execution context.

While creating adapters is not as easy as using them, it should still be manageable for people with a rudimentary amount of programming skills. This is mostly due to the fact that Ansible itself, on top of which freckles is built, has a fairly flat learning curve, esp. for a configuration management system. If you already have some experience with Ansible, writing a freckelize adapter should be easy.

If you are keen to get started, check out the relevant documentation: