poudriere.8

.\" Copyright (c) 2012 Baptiste Daroussin <bapt@FreeBSD.org>
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions
.\" are met:
.\" 1. Redistributions of source code must retain the above copyright
.\"    notice, this list of conditions and the following disclaimer.
.\" 2. Redistributions in binary form must reproduce the above copyright
.\"    notice, this list of conditions and the following disclaimer in the
.\"    documentation and/or other materials provided with the distribution.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
.\" ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
.\" SUCH DAMAGE.
.\"
.\" $FreeBSD$
.\"
.\" Note: The date here should be updated whenever a non-trivial
.\" change is made to the manual page.
.Dd November 10, 2012
.Dt POUDRIERE 8
.Os FreeBSD
.Sh NAME
.Nm poudriere
.Nd bulk package builder and port tester
.Sh SYNOPSIS
.Nm
.Cm command
.Cm subcommand
.Op Ar options
.Sh DESCRIPTION
The
.Nm
tool is used to build packages from the FreeBSD ports tree.
It can also be used to test a single port.
.Pp
Its goals are to use modern facilities present in FreeBSD (as ZFS,
jails), to be easy to use and to depend only on base.
.Sh COMMANDS
The first argument to
.Nm
must be a
.Cm command
from the following list:
.Bl -tag -width "-f conffile"
.It bulk
This command makes a ready-to-export package tree, and fills it with
binary packages built from a given list of ports.
During the build, hit ^T to send
.Dv SIGINFO
and get stats and progress back.
.It jail
This command gives you control to manage the jails used by
.Nm
to provide building environment (different FreeBSD versions, different
architectures).
.It ports
This command allows you to manage different portstrees which will be used
by
.Nm
(create, update and delete portstrees).
.It testport
This command, mainly targeted at FreeBSD ports developers, launches a
test on a given port (useful before submitting/committing a port).
.It options
This command allows to configure the options for a given port
.It distclean
This command will cleanup old distfiles
.It queue
This command allows a non-root user to queue
.Nm
commands.
.It cron
This command does not take arguments and runs jobs from the queue
command of
.Nm ,
and is designed to be run from
.Xr cron 8 .
.It version
Show version of poudriere.
.Nm.
.El
.Sh SUBCOMMANDS
Here are the list of subcommands and associated options supported by
.Nm ,
sorted by
.Cm command
order.
.Ss bulk
.Pp
There is only one subcommand for bulk command.
.Bl -tag -width "-f conffile"
.It Fl f Ar file
Absolute path to a file which contains the list of ports to build.
Ports must be specified in the form
.Ar category/port
and shell-style comments are allowed.
.El
.Pp
See
.Sx CUSTOMISATION
to know how to build binary packages with options that differs from
defaults.
.Pp
The list of ports can also be provided by command line arguments.
.Pp
Here are the options associated with the
.Cm bulk
command.
.Bl -tag -width "-f conffile"
.It Fl a
Build all ports in the tree.
.It Fl c
This flag specifies to clean the previous built binary packages.
.It Fl C
Clean previous packages for the given list to build.
.It Fl j Ar name
If given, run the bulk build only on the jail named
.Ar name .
.It Fl J Ar number
This argument specifies how many
.Ar number
jobs will run in parallel for a bulk build.
.It Fl p Ar tree
This flag specifies on which ports
.Ar tree
the bulk build will be done.
.It Fl s
Skip sanity tests.
Sanity tests are made to check if the ports exists,
does not have an increased version number, and if the compiled options
match the current options from the make.conf files and /var/db/options.
.It Fl t
Add some testing.
Currently uninstalls the port, and disable parallel
jobs for make.
.It Fl w
Save WRKDIR on build failure.
The WRKDIR will be tarred up into
.Sy ${POUDRIERE_DATA}/wrkdirs .
.It Fl v
This will show more information during the build.
Specify twice to enable debug output.
.It Fl z Ar set
This specifies which SET to use for the build.
See
.Sx CUSTOMISATION
for examples of how this is used.
.El
.Ss jail
.Pp
These subcommands are mutually exclusive.
.Bl -tag -width "-f conffile"
.It Fl c
Creates a jail.
.It Fl d
Deletes a jail.
.It Fl l
List all available jails.
.It Fl s
Starts a jail.
.It Fl k
Kills a jail (stops it).
.It Fl i
Show information about the jail.
.El
.Pp
Except for
.Fl l ,
all of the subcommands require the
.Fl j
option (see below).
.Pp
Here are the options associated with the
.Cm jail
command.
.Bl -tag -width "-f conffile"
.It Fl J Ar number
This argument specifies how many
.Ar number
jobs will run in parallel for buildworld.
.It Fl q
Remove the header when
.Fl l
is the specified mandatory option.
Otherwise, it has no effect.
.It Fl j Ar name
Specifies the
.Ar name
of the jail.
.It Fl v Ar version
Specifies which
.Ar version
of FreeBSD to use in the jail.
If you are using method ftp then the
.Ar version
should in the form of: 9.0-RELEASE.
If you are using method csup then the
.Ar version
should be in the form of cvs branches: RELENG_9 or . for current.
If you are using method svn then the
.Ar version
should be in the form of svn branches: stable/9 or head for current.
.It Fl a Ar architecture
Specifies which
.Ar architecture
of FreeBSD to use in the jail. (Default: same as host)
.It Fl m Ar method
Specifies which
.Ar method
to use to create the jail.
Could be csup, svn{,+http,+ssh), ftp, allbsd (Default: ftp).
.It Fl f Ar filesystem
Specifies the
.Ar filesystem
name (${ZPOOL}/jails/filesystem).
.It Fl M Ar mountpoint
Gives an alternative
.Ar mountpoint
when creating jail.
.It Fl t Ar version
instead of upgrading to the latest security fix of the jail version, you can
jump to the new specified
.Ar version .
.El
.Ss ports
.Pp
These subcommands are mutually exclusive.
.Bl -tag -width "-f conffile"
.It Fl c
Creates a ports tree.
.It Fl d
Deletes a ports tree.
.It Fl u
Updates a ports tree.
.It Fl l
List all available ports trees.
.El
.Pp
Except for
.Fl l ,
all of the subcommands require the
.Fl p
switch (see below).
.Pp
Here are the options associated with the
.Cm ports
command.
.Bl -tag -width "-f conffile"
.It Fl q
Remove the header when
.Fl l
is the specified subcommand.
Otherwise, it has no effect.
.It Fl p Ar name
Specifies the
.Ar name
of the ports tree to use.
.It Fl F
When used with
.Fl c ,
only create the needed ZFS file systems and directories, but do not
populate them.
.It Fl f Ar filesystem
Specifies the
.Ar filesystem
name (${ZPOOL}/jails/filesystem).
.It Fl M Ar mountpoint
Gives an alternative
.Ar mountpoint
when creating ports tree.
.It Fl m Ar method
Specifies which
.Ar method
to use to create the ports tree.
Could be portsnap, git, svn{,+http,+ssh} (Default:
portsnap).
.El
.Ss testport
.Pp
These subcommands are mutually exclusive.
.Bl -tag -width "-f conffile"
.It Fl d Ar path
Specifies the path to the port to test.
.It Fl o Ar origin
Specifies an origin in the ports tree
.El
.Pp
Here are the options associated with the
.Cm testport
command.
.Bl -tag -width "-f conffile"
.It Fl c
Run make config for the given port.
.It Fl j Ar name
Runs only inside the jail named
.Ar name .
.It Fl J Ar number
This argument specifies how many
.Ar number
jobs will run in parallel for building the dependencies.
.It Fl n
Do not use custom prefix.
.It Fl p Ar tree
Specifies which ports
.Ar tree
to use.
.It Fl v
This will show more information during the build.
Specify twice to enable debug output.
.It Fl z Ar set
This specifies whith SET to use for the build.
See
.Sx CUSTOMISATION
for examples of how this is used.
.El
.Ss distclean
This command accepts the following options:
.Bl -tag -width "-f conffile"
.It Fl J Ar number
This argument specifies how many
.Ar number
jobs will run in parallel for gathering distfile information.
.It Fl n
Dry run, do not actually delete anything.
.It Fl p Ar tree
Specifies which ports
.Ar tree
to use.
.It Fl y
Assume yes, do not confirm and just delete the files.
.It Fl v
This will show more information during the build.
Specify twice to enable debug output.
.El
.Pp
.Ss options
This command accepts the following options:
.Bl -tag -width "-f conffile"
.It Fl c
Use 'config' target, which will always show the dialog for the given ports.
.It Fl C
Use 'config-conditional' target, which will only bring up the dialog on new options for the given ports.
(This is the default)
.It Fl j Ar jailname
If given, configure the options only for the given jail.
.It Fl p Ar portstree
Run the configuration inside the given ports tree (by default uses default)
.It Fl f Ar filename
List of ports to configure
.It Fl n
Do not be recursive
.It Fl r
Remove port options instead of configuring them
.It Fl s
Show port options instead of configuring them
.It Fl z Ar set
This specifies whith SET to use for the build.
See
.Sx CUSTOMISATION
for examples of how this is used.
.El
.Pp
The
.Cm options
subcommand can also take the list of ports to configure thought command line
arguments instead of the using a file list.
.Ss queue
.Pp
This command takes a
.Nm
command in argument.
.Pp
There are no options associated with the
.Cm queue
command.
.Sh ENVIRONMENT
The
.Nm
command does not use any
environment variable.
.Sh FILES
.Bl -tag -width ".Pa /usr/local/etc/poudriere.conf" -compact
.It Pa /usr/local/etc/poudriere.conf
See self-documented
.Ar /usr/local/etc/poudriere.conf.sample
for example.
.It Pa /usr/local/etc/poudriere.d
This directory contains
.Ar make.conf
files for your different jails.
.El
.Sh EXIT STATUS
.Ex -std
.Sh EXAMPLES
.Ss bulk build of binary packages
This first example provides a guide on how to use
.Nm
for bulk build packages.
.Pp
.Bq Prepare infrastructure
.Pp
First you have to create a jail, which will hold all the building
infrastructure needs.
.Pp
.Dl "poudriere jail -c -v 8.2-RELEASE -a amd64 -j 82amd64"
.Pp
A jail will take approximately 3GB of space.
.Pp
Of course you can use another version of FreeBSD, regarless on what
version you are running.
amd64 users can choose i386 arch like in this
example:
.Pp
.Dl "poudriere jail -c -v 8.1-RELEASE -a i386 -j 81i386"
.Pp
This command will fetch and install a minimal jail, small (~400MB) so
you can create a lot of them.
It will install the jail under the pool
you have chosen, at poudriere/jailname.
.Pp
You also need to have at least one ports tree to build packages from it,
so let us take the default configuration by creating a ports tree.
.Pp
.Dl "poudriere ports -c"
.Pp
A ports tree will take approximately 4GB of space.
.Pp
.Bq Specify a list of ports you want to build
.Pp
Create a flat text file in which you put the ports you want to see
built by poudriere.
.Pp
.Dl "echo 'sysutils/screen' > ~/pkglist"
.Dl "echo 'editors/vim' >> ~/pkglist"
.Pp
Any line starting with the hash sign will be treated as a comment.
.Pp
.Bq Launch the bulk build
.Pp
Now you can launch the bulk build.
You can specify to build for only one
arch/version ; by default it will make the bulk build on all the jails
created by poudriere.
.Dl "poudriere bulk -f ~/pkglist -j 81i386"
.Pp
.Bq Find your packages
.Pp
Once the bulk build is over, you can meet your shiny new packages here:
.Pp
.Dl "/usr/local/poudriere_data/packages/81i386"
.Pp
with 81i386 as the name of the jail.
.Ss test a single port
This second example show how to use
.Nm
for a single port.
.Pp
Let's take the example of building a single port;
.Pp
.Dl "poudriere testport -d ~/ports-cvs/mybeautifulporttotest"
.Pp
all the tests will be done in all the jails in alphabetical order.
.Pp
It starts the jail, then mount the ports tree (nullfs), then mounts the
package dir (pourdriere_data/packages/<jailname>-<tree>-<setname>), then it mounts the
~/ports-cvs/mybeautifulporttotest (nullfs) it builds all the dependencies
(except runtime ones) and log it to
poudriere_data/logs/testport/jailname/default/mybeautifulporttotest.log).
.Pp
If packages for the dependencies already exists it will use them
.Pp
When all the dependencies are built, packages for them are created so
that next time it will be faster.
.Pp
All the dependency phase is done with PREFIX == LOCALBASE.
.Pp
After that it will build the port itself with LOCALBASE != PREFIX
and log the build to
poudriere_data/logs/testport/jailname/default/mybeautifulporttotest.log
.Pp
It will try to:
install it,
create a package from it,
deinstall it,
check for cruft left behind and
propose the line to add to pkg-plist if needed.
.Pp
It is very easy to extend it so that we can easily add other tests if
wanted.
.Sh CUSTOMISATION
For bulk building, you can customize binary packages produced by
.Nm
by changing build options port by port, and you can also specify
building directives in a make.conf file.
.Ss Custom build options
Before building a package,
.Nm
can mount a directory containing option files if available.
.Nm
will check for any of these directories in this order:
.Pp
.Dl /usr/local/etc/poudriere.d/<jailname>-<setname>-options
.Dl /usr/local/etc/poudriere.d/<setname>-options
.Dl /usr/local/etc/poudriere.d/<jailname>-options
.Dl /usr/local/etc/poudriere.d/options
.Pp
If a directory with this name exists, it is null-mounted into the
/var/db/ports/ directory of the jail, thus allowing to build package
with custom OPTIONS.
.Pp
The
.Cm options
subcommand can be used to easily configure options in the correct directory.
.Pp
This directory has the usual layout for options: it contains one directory per
port (the name of the port) containing an 'options' file with lines similar to:
.Pp
.Dl WITH_FOO=true
.Dl WITHOUT_BAR=true
.Pp
As a starter, you may want to copy an existing /var/db/ports/ to
/usr/local/etc/poudriere.d/options.
.Ss Create optional make.conf
You can also specify a global make.conf which will be used for all the
jails, and also add a per-jail, per-set, or per-jail-set make.conf.
They will all be used in the jail, in the order shown:
.Pp
.Dl /usr/local/etc/poudriere.d/make.conf
.Dl /usr/local/etc/poudriere.d/<setname>-make.conf
.Dl /usr/local/etc/poudriere.d/<tree>-make.conf
.Dl /usr/local/etc/poudriere.d/<jailname>-make.conf
.Dl /usr/local/etc/poudriere.d/<jailname>-<tree>-make.conf
.Dl /usr/local/etc/poudriere.d/<jailname>-<setname>-make.conf
.Sh COMPATIBILITY
The
.Nm
command must be used on a recent version of FreeBSD, i.e. a version
which has ZFS >= v15, and a zpool.
.Sh CAVEATS
.Ss Jailname
.Fl j
is the name of the jail which will also be the name of the zfs
filesystem.
.Pp
Be careful to respects the names supported by jail(8):
.Pp
.Bd -literal
    "This is an arbitrary string that identifies a jail (except it
     may not contain a '.')"
.Ed
.Pp
Be also careful to not begin the name of the jail by a number if you are
not in -stable or current:
.Pp
.Lk http://svn.freebsd.org/viewvc/base?view=revision&revision=209820
.Sh BUGS
In case of bugs, feel free to fill
.Pp
.Lk http://fossil.etoilebsd.net/poudriere/reportlist
.Sh AUTHORS
.An Baptiste Daroussin Aq bapt@FreeBSD.org
.An Bryan Drewery Aq bdrewery@FreeBSD.org