The Craftr build system¶
Craftr is a next generation build system based on Ninja and Python that features modular and cross-platform build definitions at the flexibility of a Python script and provides access to multiple levels of build automation abstraction.
Features | |
---|---|
Cross-platform | Use Craftr anywhere Ninja and Python can run |
Modular build scripts | Import other build scripts as Python modules, synergizes well
with git submodule |
Performance | Craftr outperforms traditional tools like Make, CMake or Meson |
Language-independent | Don't be tied to a single language, use Craftr for anything you want! |
Extensive standard library | High-level interfaces to modern C/C++ compilers, C+, Java, Protobuff, Yacc, Cython, Flex, NVCC (OpenCL coming soon) |
Everything under your control | Use the lowlevel API when- and wherever you need it and manully define the exact build commands |
Consequent out-of-tree builds | Craftr never builds in your working tree (unless you tell it to) |
Contents¶
Getting Started¶
Craftr is built around Python-ish modules that we call Craftr modules or Craftfiles (though this name usually refers to the first type of Craftr modules). There are two ways a Craftr module can be created:
- A file named
Craftfile.py
with a# craftr_module(...)
declaration - A file named
craftr.ext.<module_name>.py
where<module_name>
is of course the name of your Craftr module
By default, Craftr will execute the Craftfile.py
from the current
working directy if no different main module is specified with the -m
option. Below you can find a simple Craftfile that can build a C++ program
on any platform (that is supported by the Craftr STL).
# craftr_module(my_project)
from craftr import path
from craftr.ext import platform
# Create object files for each .cpp file in the src/ directory.
obj = platform.cxx.compile(
sources = path.glob('src/*.cpp'),
std = 'c++11',
)
# Link all object files into an executable called "main".
program = platform.ld.link(
inputs = obj,
output = 'main'
)
Below is a sample invokation on Windows. We pass the -v
flag for
additional debug output by Craftr and full command-line output from Ninja.
λ craftr -v
detected ninja v1.6.0
cd "build"
load 'craftr.ext.my_project'
(craftr.ext.my_project, line 9): unused options for compile(): {'std'}
exporting 'build.ninja'
$ ninja -v
[1/2] cl /nologo /c c:\users\niklas\desktop\test\src\main.cpp /Foc:\users\niklas\desktop\test\build\my_project\obj\main.obj /EHsc /W4 /Od /showIncludes
[2/2] link /nologo c:\users\niklas\desktop\test\build\my_project\obj\main.obj /OUT:c:\users\niklas\desktop\test\build\my_project\main.exe
λ ls build build\my_project\
build:
build.ninja my_project/
build\my_project\:
main.exe* obj/
Installation¶
pip install craftr-build
To install from the Git repository, use the -e
flag to be able to update
Craftr by simply pulling the latest changes from the remote repository.
git clone https://github.com/craftr-build/craftr.git && cd craftr
pip install -e .
Targets¶
Craftr describes builds with the craftr.Target
class. Similar to
rules in Makefiles, a target has input and output files and a command to
produce the output files. Note that in Craftr, targets can also represents
Tasks which can be used to embed real Python functions into the build
graph.
Using the Target
class directly is usually not
necessary unless you have very specific requirements and need control
over the exact commands that will be executed. Or if you’re just being
super lazy and need the easiest script to compile a C program:
# craftr_module(super_lazy)
from craftr import Target, path
main = Target(
command = 'gcc $in -o $out',
inputs = path.local(['src/main.c', 'src/util.c']),
outputs = 'main'
)
The substition of $in
and $out
is conveniently done by Ninja.
$ craftr .main
[1/1] gcc /home/niklas/Desktop/example/src/main....til.c -o /home/niklas/Desktop/example/build/main
Tasks¶
Tasks were initially designed as functions doing convenient operations that can be invoked from the command-line, they can also be used to export any function as a “command” to the Ninja manifest and have the production of output files implemented in Python.
A common use-case for tasks is to generate an archive from the build
products to have it ready for distribution. Below you can find a simple
example using the archive
and git
extension modules:
#craftr_module(myapp)
from craftr import path, task, info
from craftr.ext import archive, git, platform
git = git.Git(project_dir)
obj = platform.cc.compile(sources = path.glob('src/*.c'))
bin = platform.ld.linkn(inputs = obj, output = 'myapp')
@task(requires = [bin])
def archive():
archive = Archive(prefix = 'myapp-{}'.format(git.describe()))
archive.add('res') # Add a directory to the archive
archive.add(bin.outputs) # Add the produced binary
archive.save()
info('archive saved: {!r}'.format(archive.name))
Note
Craftr is clever enough to run a task directly if it doesn’t
need any Ninja targets to be built before it can be executed.
For example, the following task via craftr .hello
@task
def hello():
info('Hello, World!')
See also
Tasks invoked by Ninja are executed through the Craftr RTS.
Generator Functions¶
Most of the time you don’t want to be using Targets directly but instead use functions to produce them with a high-level interface. It is sometimes useful to create such a target generator function first and then use it to reduce the complexity of the build script.
The Craftr standard library provides an extensive set of functions and classes that generate targets for you, most notably the C/C++ compiler toolsets.
See also
Since C/C++ builds are very complex and strongly vary between platforms, Craftr defines a standard interface for compiling C/C++ source files as well as the linking and archiving steps.
Functions that generate targets use the craftr.TargetBuilder
that does a lot of useful preprocessing and, as the name suggests,
make building Targets much easier.
Frameworks¶
The craftr.Framework
is in fact just a dictionary (with an
additional name
attribute) that represents
a set of options for anything build related. How the data is interpreted
depends on the generator function.
Frameworks are useful for grouping build information. They were designed for C/C++ builds but may find other uses as well. For example, there might be a framework for a C++ library that specifies the include paths, preprocessor definitions, linker search path and other libraries required for the library to be used in a C++ application.
For example, the Craftfile for a header-only C++ library might look as simple as this:
from craftr import Framework, path
from craftr.ext.libs.some_library import some_library
my_library = Framework(
frameworks = [some_library],
include = [path.local('include')],
libs = ['zip'],
)
As you can see in the example above, frameworks can also be nested.
Targets there were generated by helper functions (as described in
the Generator Functions section) list up the frameworks that have
been used to produce the target in the Target.frameworks
attribute. Passing a target directly as input to another generator
function will automatically inherit the frameworks of that target!
fw = Framework(
include = [path.local('vendor/include'),
libpath = [path.local('vendor/lib')],
libs = ['vendorlib1', 'vendorlib2']
)
obj = cc.compiler(
sources = path.glob('src/*.c'),
frameworks = [fw]
)
bin = ld.link(
inputs = obj
# we don't need to specify "fw" again, it is inherited from "obj"
)
Build Options¶
Options for the build process are entirely read from environment variables.
The craftr.options.get()
function is a convenient method to read the
options from the environment.
In Craftr, options can be specified local for a module or globally for all modules. A local option is actually prefixed by the full name of the Craftr module.
#craftr_module(app)
from craftr import options
name = options.get('name')
debug = options.get_bool('debug')
info('Hello {}, you want a {} build?'.format(name, 'debug' if debug else 'release'))
The options can be specified locally using the following methods:
craftr -D.name="John Doe" -D.debug
craftr -Dapp.name="John Doe" -Dapp.debug
app.name="John Doe" app.debug="true" craftr # assuming your shell supports this syntax
They can be set globally like this:
craftr -Dname="John Doe" -Ddebug
name="John Doe" debug="true" craftr # assuming your shell supports this syntax
Options and environment variables can also be set from craftrc.py
files.
Oh, and say hello to John!
Hello John Doe, you want a debug build?
craftrc.py Files¶
Before anything, Craftr will execute a craftrc.py
file if any exist. This
file can exist in the current working directory and/or the users home directory.
Both will be executed if both exist! You can prevent Craftr from executing
these files by passing --no-rc
. You can also tell it to execute a specific
file with the --rc
parameter (can be combined).
This file is not executed in a Craftr module context and hence should not declare any targets, but it can be used to set up environment variables and options.
For example, for using the craftr.ext.qt5 module on Windows, you could
use this craftrc.py
file in the home directory to let the Craftr Qt5
module know where the Qt5 headers and libraries are located.
from os import environ
if 'Qt5Path' not in environ:
environ['Qt5Path'] = 'D:\\lib\\Qt\\5.5\\msvc2013_64'
Note that you can still specify a different Qt5Path
via the command
line that will override the value set in the craftrc.py
file because
the environment variables are set in the following order:
- Variables from the parent process/shell
- Variables prefixed on the command-line (like
VAR=VALUE craftr ...
) if your shell supports it craftrc.py
files that modify thecraftr.environ
- Options passed via the
-D, --define
command-line parameter - Craftr modules that modify the
craftr.environ
Colorized Output¶
Craftr colorizes output by default if it is attached to a TTY. If it is not
but colorized output is still desired, CRAFTR_ISATTY
can be set to true
in the environment. Also, colorized output can be disabled by setting the
variable to false
instead. For any other value, default behaviour applies.
Debugging¶
Not only can you debug your Craftr build scripts with the pdb
module, but you can also increase the verbosity level for more verbose
output. This is very useful for tracing down warnings or locations of
errors in the output, eg.:
λ craftr --skip-build
you really shouldn't do it that way!
To find the location of that line, we can pass -v
.
λ craftr --skip-build -v
detected ninja v1.6.0
cd "build"
load 'craftr.ext.test'
(craftr.ext.test, line 4): you really shouldn't do it that way!
exporting 'build.ninja'
Now if you’re really having trouble finding out how the Python script
actually gets there, you can enable a stacktrace with each line that
is output with -vv
.
λ craftr --skip-build -vv
detected ninja v1.6.0
cd "build"
load 'craftr.ext.test'
(craftr.ext.test, line 4): you really shouldn't do it that way!
In <module> (F:\Python34\Scripts\craftr-script.py, line 9)
In main() (c:\users\niklas\repos\craftr-build\craftr\craftr\__main__.py, line 256)
In import_module() (f:\python34\lib\importlib\__init__.py, line 109)
In load_module() (c:\users\niklas\repos\craftr-build\craftr\craftr\ext.py, line 245)
In <craftr.ext.test> (Craftfile.py, line 4)
exporting 'build.ninja'
This output is also nicely colorized if you’re in a terminal that supports ANSI color codes.