As many of you have noticed, the alpha release of Ubuntu Delivered by rPath is now available as a platform selection on rBuilder Online.

As part of taking Ubuntu Delivered by rPath beyond its current alpha status, we have been making changes to our infrastructure to better support this platform.

Because of the nature of these changes, updates to existing appliances based on Ubuntu Delivered by rPath will fail. Therefore, It will be necessary to recreate the appliance using the latest version of Ubuntu Delivered by rPath, and to then redeploy the appliance.

If you created your appliance using rBuilder Online's web interface:

• Select the appropriate version of your product
• Click "Edit product version"
• Click "Update" next to the Appliance Platform listing
• Select "Ubuntu Hardy Delivered by rPath" from the drop down menu
• Click "Update Appliance Platform"
• Click "Manage Appliance" on the left of the page
• Click "Revise Appliance"
• Click "No archive to package" at the bottom of the page
• Click "Continue" at the bottom of the page
• Click "Update Contents"
• Click "Build Appliance"
• Click "Generate Appliance Images"

If you have created your appliance using rPath's command-line tools:

• Use "cvc checkout" to obtain a copy of the appliance's recipe
• Change any occurrences of "ubuntu.rpath.org" to "ubuntu.rpath.com"
• Commit your changes
• Cook groups and build new images

rPath apologizes for this inconvenience, but these changes were necessary to ensure that this highly-popular platform will be adequately supported by rBuilder Online.

A Christmas gift some years ago of The Dream Hunters introduced me to Neil Gaiman's work. As an amateur of folk tales, I was entranced and delighted by the beauty and craft of The Dream Hunters, and then started reading earlier works in the Sandman series; my introduction to the graphic novel. Neverwhere and Stardust next captured my attention, followed quickly by most of the rest of Gaiman's books.

In my second and third readings of Stardust, I grew conscious of one of the characteristics of Gaiman's works that I find deeply satisfying: What I first saw as "loose threads" were actually precisely woven through the text.

For many authors, it appears to me that internal structural integrity is achieved in a tradeoff against narrative flow; or, more often, that narrative flow is achieved by ruthlessly discarding internal structural integrity. Speaking as an engineer, this is a reasonable tradeoff, since most readers seem not to care about structural integrity. But internal inconsistencies drive me up the wall even when perpetrated by my favorite authors. (I have read — and own and love — dozens of C. S. Lewis's books, but try piecing together the chronology of Narnia!)

Gaiman is in a delightful but rare class of writers who have the means and inclination to combine compelling narrative with extraordinary structural integrity. I consider Dorthy L. Sayers, J. R. R. Tolkien, and Rudyard Kipling well-known exemplars. Each time I re-read most of their published works, I generally notice additional internal consistency, rather than become annoyed by discovering small inconsistencies. This adds delight to re-reading, and helps me slow down while I read, increasing my joy in reading. A virtuous cycle.

Imagine, then, my delight to receive a copy of Gaiman's The Graveyard Book. I paid almost no attention to the title, as usual. I dove in and read the whole book, rudely ignoring everyone around me for hours. As I finished it, I took a deep breath of satisfaction, leaned back in my seat, and said to myself, "Wow. The Jungle Book, set in a graveyard... Oh." I am a little slow on the uptake, I guess.

Like several of Gaiman's other works, you probably won't like it if you are squeamish. There is Death, and Blood, and Violence, and even War. It differs from prime time television news in three ways: it involves magic, it tells of creatures that are literally rather than figuratively inhuman, and it is well worth your time.

Several readers of advanced copies have complained that The Graveyard Book is episodic; almost a collection of short stories. That complaint misses the point in several respects. The first is that Gaiman is a master of the short story; a few words tell a long story. The second is that this story is particularly well-suited to this presentation. It is short enough not to scare away a (say) middle school reader — good! If you want the story to last longer, read slower! True to form, Gaiman's narrative and construction reward careful reading. Finally, this book, in this form, is an homage to another great work worth re-reading, and worth reading out loud.

• Amazon
• Barnes and Noble

Conary 2.0.27 is a maintenance release.

• Metadata information can now be extracted from debian packages. (CNY-2995)

• The "conary rollback" command now handles --info and --interactive flags, which behave similarly to the "conary update" flags. (CNY-820)

• Conary no longer checks for the existence of a db journal on startup, making it possible to perform cvc operations while updating your system. (CNY-2958)
• Files with (only) unsatisfied Java requirements that are filtered out using r.Requires(exceptDeps) no longer have their provides removed. (CNY-3001)
• A bug that caused the frozen list of trove script compatibility information (created if using the "toClass=" keyword argument when adding a trove script) to be recorded in a random order has been fixed. Because sorting the list causes a different Trove signature digest to be generated, this version of Conary introduces version 2 trove signatures. Conary will ignore version 1 trove signatures on old troves that have multiple trove script compatibility entries. (CNY-2997)

• A new type of metadata information, storing key-value pairs, is now available. (CNY-2983)

conary-policy 1.0.22 is a maintenance release.

• A bug causing PHPRequires to fail if buildRequires are not present in the repository has been fixed. (CNP-160)
• PythonEggs now allows directories that end in .egg to be packaged. (CNP-148)
• The NormalizePythonEggs has been added. It automatically unpacks python .egg files that are placed in standard python module locations. The resulting directory is put in place of the .egg file. (CNP-149)
• The SymlinkTargetRequires policy has been added. This policy will supress the DanglingSymlinks policy in favor of adding requires when appropriate. (CNP-130)
• The PHPRequires policy now caches negative results. (CNP-164)
• The PHPRequires policy now looks for binaries named "php5" as well as "php". Previous PHP5 handling looked only for packages named "php5" but still expected to see binaries named "php". (CNP-165)
• The ResetKeyValueMetadata policy has been added. It selectively deletes certain key-value metadata fields. (CNY-2983)

Foresight Linux has gotten a lot more popular recently, in part due to the release of the new Foresight Mobile Edition and Foresight Linux Kid's Edition.

Every silver cloud has a dark lining, it seems. The new attention has caused the system that runs the Foresight repository to run out of resources, and get much, much slower — particularly as Foresight developers are simultaneously doing major rebuilds in preparation for the combined GNOME 2.24 and Foresight 2.0.5 release.

We are now doing emergency maintenance on the Foresight repository system, applying more resources to handle the additional load. Unfortunately, this means that Foresight system updates are not available during some or all of this maintenance. This process will take several hours to complete.

Thanks for your patience!

Update: The Foresight repository is now available for reading, but not for committing. System updates will work. However, the repository will again go offline briefly when the maintenance is completed in order to transition the load.

Another update: The repository is now backed by a more capable database server (more memory, more and faster disk) that seems to be handling the load much better — we were essentially simply running out of memory. We'll be watching performance. And working on the software side too for better scaling.

Conary 2.0.26 is a maintenance release.

• The way that factories are cooked has been changed so that buildRequires are set (for building in rMake) and so that platforms may override the default build requirements. (CNY-2986)

• A bug causing cooks to fail when Perl was not installed on the build system has been fixed. (CNY-2989)
• A bug that prevented Java dependencies from being parsed correctly in some cases, for example when given on a "conary rq --what-provides" command line, has been fixed. Previously a "bad java dependency: flags required" error message was generated. (CNY-2990)

Conary 2.0.25 is a maintenance release.

• "conary emerge" no longer emits unnecessary warnings. (CNY-2835)
• A bug that prevented proper ownership and permissions from being recorded when using r.addArchive(..., perserveOwnership=True) and the archive had a prefix of "./" has been fixed. Additionally, a bug that could cause permissions to be assigned to files with similar paths as those created by addArchive has been fixed. (CNY-2953)
• Promote in 2.0.24 could traceback instead of printing an error message if no troves were present to clone.
• A bug causing addSource to use the wrong file if there was a name collision between a filename in an rpm and the source directory has been fixed. (CNY-2627)
• A bug causing createNewSourceTrove to erroneously change pathIds has been fixed. (CNY-2971)
• When installing with the --root option and a trailing '/' character provided, Conary previously wrote new entries in the ld.so.conf without a leading '/' character. This has been fixed. (CNY-2982)

• Cross-compiling packages that include perl modules can now provide perl: dependencies; the perl @INC path needs to be provided via the r.Requires(bootstrapPerlIncPath=...) keyword argument. (CNY-2943)
• Packaging a version of perl that requires libraries in the destdir to start but does not use RPATH now succeeds. (CNY-2955)
• When the perl @INC path includes elements that are symlinks in the destdir, the symlinks are now resolved for correct perl: provides. (CNY-2949)
• When discovering perl: dependencies using a bootstrap perl, Conary now sets LD_LIBRARY_PATH. (CNY-2959)
• ComponentRequires now handles :cil, :java, :perl, :python, and :ruby components in the same manner it handles :lib. (CNY-2935)
• The r.addArchive() method can now extract file contents from dpkg .deb files when the data is bzip2-compressed. (CNY-2967)
• Promote now uses hasTroves to check to check the completeness of a promote instead of fetching group troves for improved performance.
• Builtin recipes have been broken out so that there's no need to commit conary internals to repository based recipes. BaseRequiresRecipe has been introduced. (CNY-2898)
• Files in /var/log, /var/run, and /var/cache are now automatically marked InitialContents. (CNY-2578)
• Explicit calls to r.Requires are now honored in derived recipes. (CNY-1760)
• The r.addArchive() method can now extract control contents from dpkg .deb archive files. (CNY-2967)

• Shim commitChangeSet() and getFileContents() calls now fall back to network access properly

• Added --lsprof argument to standalone server
• If a user's mirror permission was granted by the second or subsequent role, the anonymous permissions were inappropriately granted to the user (CNY-2964)

• Reimplemented dependency freezing in C
• DependencySet objects now store the frozen depenedency by default and thaw it only when the thawed objects are needed (CNY-2410)
• Intern frozen dependency and frozen StringCollection strings (CNY-2410)
• DependenciesStream and FlavorStream objects now subclass from DependenciesSet instead of having a DependenciesSet object as an attribute. This saves one object per instance. (CNY-2410)

The scheduled maintenance of rBuilder Online is complete.

Introducing Platform Selection

rBuilder Online now allows you to select an appliance platform when creating a new product and/or product version. This selection is currently limited to "rPath Linux 2" for the time being, but stay tuned: more choices are coming very soon.

Some users who created products and product versions in the last few months may see that their appliance is based upon a "custom appliance platform from conary.rpath.com@rpl:2-devel". It is recommended that you update your appliance platform for any product versions that are based on this custom platform by clicking on the Update button in the edit product version page and selecting "rPath Linux 2". Doing so will ensure that you are on a supported path for future platform upgrades.

Do Your Part

Help us make rBuilder Online better. We welcome your feedback on the rBuilder Online forum. If you find any bugs, please file an issue.

Thank you for your interest in rBuilder Online.

rBuilder Online will undergo scheduled maintenance starting Thursday September 11th at 21:00 EDT (-0400) and ending at 22:00. During this time, rBuilder Online and all rpath.org repositories will not be available. A message will be posted when the scheduled maintenance period has been completed.

Thank you for your interest in rBuilder Online.

I have welcomed the opportunity to do real-world testing of rBuild by working on LochDNS again — this time with a tool that makes the process take less time and attention. I'm pleased to announce LochDNS 2.0.

Again, we're fixing bugs throughout the ecosystem. I even managed to expose a long-standing corner-case bug in Conary that no one had hit for years.

One of the interesting things about LochDNS is that I build lots of images so that people can use it to demonstrate rPath's technology on lots of different platforms. The thought of manually clicking through lots of individual web forms to build about 20 images tended to keep me from spending the time to update LochDNS in the past. My odd case caught one more bug shared between rMake (RMK-907) and rBuild (RBLD-92) that we'll be able to fix shortly.

Having worked all the way through the process once with rBuild, and fixed the relevant bugs in rBuild along the way, I expect that future updates to LochDNS should be easy; mostly a background task.

Update: I have rebuilt rMake for RMK-907 with the critical fix so that all the images will be built. Later, I'll fix rBuild so that it reports which images it is building, instead of a generic Image_1, Image_2, etc. This change to rBuild will require the API change made in rMake for RMK-907 to function, so it won't work with released rMake. In short, update rMake before/if you update rBuild.

Ken VanDine has been using rBuild to build all sorts of projects. He has filed a few bugs, but has also been really enthusiastic about how easy it makes the whole process. Among his projects have been several variants of Foresight Linux. Derived packages are one of Ken's favorite Conary features and are perfect for making lightly-customized variants.

Unfortunately, rBuild didn't deal very well with derived packages. It turned out that this was due to two bugs; one in rBuild proper (RBLD-86) and one in rMake (RMK-906). With those two bugs fixed, Ken is much happier!

I released rBuild 0.5 alpha and built an rmake snapshot. Update to or install the latest versions of the stack with:

conary update {conary,rmake,decorator,rpath-{common,xmllib,product-definition},rbuild}=mkj.rpath.org@rpl:2


Remember, these are a combination of alpha code and un-validated snapshots. They worked for me in a few tests. Your milage may vary — and if it does, please do tell us about it.

In the past week of using rBuild for some real tasks, we've discovered several bugs. In particular:

• rBuild being reluctant to rebuild groups (RMK-903)
• rBuild not building images in rBuilder Online (RMK-897)
• rbuild rebase not working in non-standard label case (RPCL-36)
• rbuild rebase failing to update product definition (CNY-2971)

We have fixed all those bugs (in the current source code) without touching rBuild itself — note that all those issue IDs are for other components: rMake, rPath Common Libraries, and Conary.

In addition, we have found other bugs that we haven't yet fixed. For example, rbuild update stage not correctly updating packages (RBLD-89), and cannot publish certain rBuilder images built with rbuild build images (RBL-3386)

I have built some new packages, including conary and rmake snapshots with conary and rmake bug fixes included:

conary update {conary,rmake,decorator,rpath-{common,xmllib,product-definition},rbuild}=mkj.rpath.org@rpl:2


Remember, these are a combination of alpha code and un-validated snapshots. They worked for me in a single minor test. Your milage may vary — and if it does, please do tell us about it.

Conary 2.0.24 is a maintenance release.

• includeConfigFile now interprets paths relative to the file currently being processed, for files and URIs. (CNY-2950)
• includeConfigFile now supports a ~/path syntax to substitute the current user's home directory. (CNY-1650)

• Added support for commitChangeSet and commitChangeSetFile to ShimNetClient (CNY-2545)
• During promote operations use the cloned history for packages for the components of those packages as well. (CNY-1867)
• Promote now uses group weak references to learn about all of the components for a package instead of downloading all of the package troves for the source. (CNY-1867)

• The ccs2tar script now correctly handles changeset files containing directories, symlinks, and other non-regular files. (CNY-2954)

• More of the file installation path was moved into C for improved performance
• Added misc.pack() and use it to freeze TroveRefsFilesStream and StreamCollection objects
• Initial step of promote takes advantage of weak references to avoid downloading so many packages (CNY-1867)
• The shim network client now supports CONARY_CLIENT_LOG environment variable
• Added support for getFileContents call to ShimNetClient (CNY-1669)
• Conary update no longer caches all hostnames immediately on update - instead hostnames are cached as they are accessed. This avoids unnecessary caches when installing from changesets. (CNY-2832)
• Track CLONEDFROMLIST in troveInfo now as well as CLONEDFROM. This keeps track of the full clone history of a trove, with the most recent parent at the end of the list.
• Use CLONEDFROMLIST instead of CLONEDFROM where available to remove the need to follow the cloned from history in time. (CNY-1867)