Environment

Fabric tasks for defining environments.

Fabric environment dictionary

An integral part of Fabric is what is know as “environments”: a Python dictionary subclass which is used as a combination settings registry and shared inter-task data namespace. Most of Fabric’s and Invenio Fabrics behaviour is modifiable through env variables.

The environment is accessible through fabric.api.env, and allows e.g. a task access to the current user etc (user is called an env variable):

from fabric.api import env
@task
def sometask():
    print env.user
    print env.host

For more information on Fabric environment dictionary please see http://fabric.readthedocs.org/en/latest/usage/env.html

Invenio Fabric environments

The tasks in this module helps you define the env variables which all other Invenio Fabric tasks depends on. Secondly, it provide means for loading env variables, but without putting them in fabric.api.env. This is useful for tasks like fab int mysql_copy:prod which would copy the database from production to integration, and thus need access to host, user, password etc from both the integration and production environment at the same time.

Usage

Defining a new environment is simple:

from fabric.api import task
from inveniofab.api import *

@task
def prod(activate=True):
    # Step 1: Create env with default values
    env = env_create('prod', activate=activate)

    # Step 2: Modify some values
    env.CFG_XYZ = ...

    # Step 3: Return env
    return env

Things to note:

  • The task must take a keyword argument activate which defaults to True. This is used to control if the environment is activated or not (i.e. copied to fabric.api.env.
  • First parameter to env_create(), must match the task name (prod in this case). Otherwise the environment cannot be loaded without activating it, and template overriding will not work properly (see later in this chapter).
  • You must return the environment variable at the end of the task.

Running tasks

As mentioned above, the Invenio Fabric tasks depends on an environment being defined. The pattern to run Invenio Fabric task is:

fab <env> <task1> <task2>

In the example above, you could for instance run:

fab prod bootstrap

to bootstrap the production environment. I.e. you always call an environment task as the first task.

Also note, that some task takes parameters:

fab <env> <task1>:<arg1>,<arg2>,<kw1>=<val>,<kw2>=<val> <task2> ...
fab prod bootstrap:quite=1

Roles

By default the following roles are defined:

  • web - frontend webservers
  • lb - load balancers
  • db-master - master databases
  • db-slave - slave databases
  • workers - worker nodes

Some task will use these roles to decide which machine to execute a given command on. For more on Fabric roles please see http://fabric.readthedocs.org/en/1.4.3/usage/execution.html#roles

You may override these roles in your environment task:

@task
def prod(activate=True):
    env = env_create('prod', activate=activate)
    env.roledefs['web'] = [...]
    return env

Creating tasks

When creating task you can access the variables simply by loading the Fabric env dictionary from fabric.api.env. All Invenio Fabric env variables are upper-case (for now, please look in the source code of env_defaults() to see, which variables are defined).

Example:

from fabric.api import env

@task
def sometask():
    print env.CFG_INVENIO_PREFIX
    local("%(CFG_INVENIO_PREFIX)s/bin/python" % env)

Template loading

The Invenio Fabric environment by default also configures a Jinja2 environment which can be used to load and render templates.

Example:

from fabric.api import env

@task
def sometask():
    tpl = env.jinja.get_template("etc/invenio-local.conf")
    output = tpl.render(env)
    ...

Above code will look for the template first in <env>/etc/invenio-local.conf, then common/etc/invenio-local.conf. So assuming your directory structure looks like below:

fabfile.py
common/etc/invenio-local.conf
int/etc/invenio-local.conf

Running fab int sometask would use the template int/etc/invenio-local.conf, while fab prod sometask and fab loc sometask would use the template common/etc/invenio-local.conf.

This allows you to define common templates for configuration files, and selectively override them for certain environments if needed.

Relation to invenio-local.conf

Many variables map directly to their Invenio counterpart - e.g. CFG_DATABASE_NAME can be defined in invenio-local.conf as well as in Invenio Fabric env.CFG_DATABASE_NAME.

Usually when deploying or working with a specific Invenio project, some configuration variables depend only on the project (e.g. CFG_WEBSTYLE_TEMPLATE_SKIN), while others depend on the deployment environment (e.g. CFG_DATABASE_HOST).

As a general rule, project-wide configuration should go in a common invenio-local.conf template, while deployment dependent configuration should be defined in a Fabric environment task. The invenio-local.conf can then be render with the Fabric env variables.

Tasks

inveniofab.env.env_create(envname, defaults_func=None, activate=True, **kwargs)

Setup a new environment (e.g. integration, production, local).

See module documentation above for detailed explanation of enviroments.

Parameters:
  • envname – str, Name of environment (must be same as task envname).
  • defaults_func – callable taking a dictionary as argument and returns the same dictionary. Used to setup defaults in the environment. By default the env_defaults() is used. Take great care if overriding this, since many tasks expects specific variables to be defined.
  • activate – True to activate the environment, or False to just load it (i.e. should config be put in fabric.api.env or not).
inveniofab.env.env_defaults(env, name='invenio', prefix=None, python=None, **kwargs)

Setup defaults in environment.

The method will by default try to guess

Parameters:
  • name –
  • prefix –
  • python –
inveniofab.env.env_get(name)

Get environment by name (does not activate the environment).

An environment is defined in a local task. This task will look for a task with the name <name> and execute. Hence the task defining the environment.

Example:

from fabric.api import env

@task
def sometask():
    another_env = env_get('prod')
    print env.CFG_SITE_URL
    print another_env.CFG_SITE_URL

Running fab int sometask would print first the integration CFG_SITE_URL, then the production CFG_SITE_URL.

inveniofab.env.env_override(env, this_repo, this_ref, override={}, global_overrides=None)

Override default values for repository

inveniofab.env.env_make_name(prefix, python, ref)

Generate a MySQL friendly environment name.