Early in Conary development, we made a conscious decision to focus on making the normal case easy. Good assumptions simplify. When possible, the tools should do the right thing for the most common case, making it as easy as reasonably possible to specify exceptions to the common case. As we find more patterns of "normal cases", we have been able to automate more and more of those "normal cases".

There is always room for improvement.

We have broadened our perspective. Building individual packages is generally quite easy in Conary. Building an image on rBuilder is not that hard. Doing release management with Conary utilizes two commands. But the whole process, put together, takes too many steps. We have made it so easy to build and maintain custom images (virtual appliances, software appliances, and software stacks for hardware appliances) that it no longer requires relatively esoteric skills—which, oddly enough, makes it clear that the process is still too complex.

Reducing most of the work to a series of commands was really impressive, and yet it's not enough—there are too many commands to run, with different command line arguments; fundamentally, our instructions for how to create a product with rPath's tools have too much verbatim boilerplate. Our highest-level tools expose too many options for different ways of doing things. Don't get me wrong, we don't want to remove the options. We just don't want to force every user to think about each of the options. As James Thurber so aptly put it, "Sixty minutes of thinking of any kind is bound to lead to confusion and unhappiness."

Introducing rBuild

rBuild will provide a single command line program for developing on the rPath platform. After you define a product in rBuilder, you will have one command-line program for packaging, release management, and building images. For normal use, you will not need to run the cvc or rmake at all. To make this possible, rBuild depends on the new product concept in rBuilder—you need to create a product in rBuilder, then you check out the product in rBuild to work on it. The product that you create in rBuilder gives rBuild the information it needs to automate all these processes.

rBuild is alpha-quality software. It has bugs. As alpha-quality software, rBuild is not yet ready to be your first experience with rPath's software products. This is especially true because many cases of unexpected input result in a traceback rather than a nice error message. Experienced Conary packagers and appliance developers can help us turn rBuild into a truly usable software development tool, but there absolutely will be frustrating bumps along the way. Many thanks in advance for your patience! But more than patience, we want to hear your story about using rBuild. Talk to us on the #conary channel, or file bug reports in our issue tracking system.

After rBuild 1.0 is released, the official rPath Application to Appliance guide will be updated to use rBuild instead of cvc and rmake commands.

More warnings: We expect to continue to update the schema for product definitions, and so when we update rBuilder Online you might sometimes need to update rBuild to match. It is likely that newer versions of rBuild will depend on new features in the most recent version of rMake and Conary when they are released. We might come out with multiple versions of rBuild in one day. Expect incompatible interface changes until rBuild 1.0 is released.

Enough Already! Let's Go!

For the moment, I build alpha releases of rBuild in my own rBuilder Online repository. To install rBuild, make sure that you have the most recent versions of Conary and rMake installed, then, run this command:
$ conary update {decorator,rpath-{common,xmllib,product-definition},rbuild}=mkj.rpath.org@rpl:2

Now you are ready to try things out. First, rBuild needs to interview you to find out what you want to do:
$ rbuild config
...
URL to use to contact rBuilder (start with http:// or https://): https://www.rpath.org/rbuilder/
rBuilder contacted ok.
Your rbuilder user name: YOUR_RBUILDER_ONLINE_USER_NAME_HERE
Your rbuilder password: YOUR_RBUILDER_ONLINE_PASSWORD_HERE
Name to display when committing: Your Name Here
Contact - usually email or url: http://your.url/here
rBuild configuration complete. To rerun this configuration test run rbuild config --ask, or simply edit ~/.rbuildrc.
 
You should now begin working with a product by running 'rbuild init <short name> <version>'
...


Unfortunately, the suggested rbuild init <short name> <version> does not work right now with rBuilder Online, so you will have to use the alternative form of just specifying the label you are building on. (Remember what I said about bugs?)

Here follows an example. Try this process with a product that you have defined in rBuilder Online:


$ rbuild init rbuild.rpath.org@rpl:rbuild-1
Created checkout for rbuild.rpath.org@rpl:rbuild-1 at rbuild-1
$ cd rbuild-1
$ ls -l
total 12
drwxrwxr-x 2 johnsonm johnsonm 4096 2008-08-29 15:21 Development
drwxrwxr-x 2 johnsonm johnsonm 4096 2008-08-29 15:21 QA
drwxrwxr-x 2 johnsonm johnsonm 4096 2008-08-29 15:21 Release


Conveniently, "Development", "QA", and "Release" sort the same way in alphabetical and chronological order. These are the stages that are defined automatically for products created in rBuilder. The general idea is that you build packages in the Development stage, and build groups that work. Then you promote your work to the next stage, where you do QA on the entire group of packages together. When you are satisfied, you promote the things you have made sure are high enough quality to Release.


$ cd Development
$ ls -l
total 0
$ rbuild checkout rbuild
Checked out existing package 'rbuild'
$ cd rbuild
$ ls -l
total 8
-rw-rw-r-- 1 johnsonm johnsonm 426 2008-08-29 15:28 CONARY
-rw-rw-r-- 1 johnsonm johnsonm 559 2008-08-18 06:25 rbuild.recipe
$ rbuild build packages
...
Added Job 12
  decorator:source=rbuild.rpath.org@rpl:rbuild-1-devel/2.2.0-3
  info-user:source=rbuild.rpath.org@rpl:rbuild-1-devel/1-1
  rbuild:source=malcolm.rdu.rpath.com@local:rbuild-1-devel/0.2-5
  rpath-common:source=rbuild.rpath.org@rpl:rbuild-1-devel/0.1-4
  rpath-product-definition:source=rbuild.rpath.org@rpl:rbuild-1-devel/0.1-6
  rpath-xmllib:source=rbuild.rpath.org@rpl:rbuild-1-devel/0.1-5
...


If you are following along with the product on rbuild.rpath.org instead of with your own product, this will require that you are building on a 64-bit system, because the rbuild.rpath.org product has images defined for both 32-bit and 64-bit systems. Because these different flavors of images are defined, rBuild automatically fires off builds for all the different flavors. 64-bit builds will fail on a 32-bit system.

Things to note:

• You do not need to have any ~/.conaryrc or .rmakerc configuration to use rBuild.
• You do not need to specify the packages that you need to build; rBuild uses your group and your checked-out packages to determine what to build.
• You do not need to specify which flavors to build; rBuild determines which flavors to build by looking at the list of images you want to build.

Help!

rBuild documentation is presented as inline help. That help is still incomplete. We need to know where to put more useful information. Please file issue reports in our issue tracking system telling us where the help was insufficient, or suggesting improvements to the help text.

Just to help you get started, here's how you use the inline help:

$ build help
rbuild: Conary-based Product Development Tool
 
Common Commands
  build     Build the various components of a product
  checkout  Check out packages and groups for editing
  config    Print the rbuild configuration
  init      Create a directory for for working with a product
  promote   Promote groups and packages to next stage
  rebase    Update product to most recent platform version
  show      Shows details about the result of rbuild operations
  status    Print status summary relative to the current directory
  update    Update working directories from repository
  watch     Watches details about the result of rbuild operations
 
Information Display
  help      Display help information
 
$ rbuild help build
usage: rbuild build <subcommand> [options]
 
    Build commands for building the various components of a product.
 
Subcommands:
     groups    Build groups for this stage
     images    Build images for this stage
     packages  Build edited packages for this stage
     platform  Create a platform usable by others from this product
 
(Use 'rbuild help build <subcommand>' for help on a subcommand)
 
options:
 
  Command Options:
    --no-commit
    --no-watch


If this sounds interesting, please join us on the #conary channel and discuss it! Thanks!