Changes in 1.0.14:
Client Changes:

• Added a "rmake load" command and "--to-file" parameter to build command
  to allow the information about a job to be encapsulated and sent
  between rmake instances. (RMK-522 )
• A "--show-config" flag was added to "rmake query". This displays the
  configuration used for the specified job. (RMK-624 )
• Added --clear-prebuilt-list option for "rmake restart". This option
  will run the same job as before but not mark any packages as
  prebuilt (RMK-729 ).
• Removed rmake-xml + rmake.desktop files used for integration w/
  rBuilder. That integration is being removed and something else will
  replace it. (RMK-749 )

Bug Fixes:

• Fixed a traceback when building multiple packages without a parent
  branch (RMK-732 )
• Added back compatibility with conary earlier than 1.2.6. (RMK-741 )
• restart now will not update the version of groups used to recurse
  when --no-update is used (RMK-735 )
• Prebuilt packages are now displayed in query --troves output. (RMK-707 )
• The Conary database from the chroot is removed before attempting to
  clean up the chroot itself. This helps when the latter fails by not
  leaving a Conary database that could potentially confuse rmake if it
  chooses to reuse the chroot. (RMK-698 )
• the currently running rmake no longer installs itself into
  /usr/lib/python2.4/site-pacakges/rmake, using
  /usr/share/rmake/rmake instead.
• The --config option will now correctly override values set in a
  context. (RMK-748 )
• For conary versions 1.2.16 or greater, rMake will do a better job
  of correctly recording the build requirements that it uses when more
  than one version of a package is installed into a chroot. (CNY-2427 )
• Fixed a bug in dependency resolution that would affect rmake repositories
  where an older x86_64 build version was competing with a newer x86
  version built in the same job. (RMK-761 )
• Chroots that have been removed will no longer show up in "list chroot"
  output (RMK-724 )

Bongo has reached its M3 milestone! and with the release of 0.3.0, we've updated the appliance. The appliance is now available in both 32 and 64 bit as well.

More information can be found at http://bongo-project.org/

Congrats Bongoteers!

The first beta test snapshot toward rPath Linux 2 has been released.

rPath Linux 2 beta releases are available on the conary.rpath.com@rpl:2-qa label.

This release contains:

• VMware support integrated into the default kernel; open-vm-tools included in vmware group flavors.
• The group-appliance superclass is available as group-appliance=conary.rpath.com@rpl:2-qa to enable porting appliances from rPath Linux 1 to rPath Linux 2. (We recommend doing this in parallel testing branches until rPath Linux 2 is released.)
• SYSLINUX's EXTLINUX is the default bootloader, though grub continues to be available.
• Conary 2.0 is included, which provides several optimizations and improvements.
• Short group flavors (no more need to specify flavor elements that are really irrelevant to your appliance when you cook groups). This is enabled by the move to Conary 2.0
• Appliance Installer ISO images (fast, lower memory requirements, no package selection available) for 32-bit and 64-bit systems
• Installable ISO images (slower, larger memory requirements, package selection available) for 32-bit and 64-bit systems
• Appliance Installer ISO images for 32-bit and 64-bit Xen dom0

rPath Linux 2 will be an appliance operating system. In keeping with this focus, the installer ISO images for rPath Linux 2 Beta 1 are intended primarily for testing the installer, and secondarily as a platform for testing appliance development. This is not a general purpose operating system.

Migrating appliances from an rPath Linux 1 base to an rPath Linux 2 Beta base has been only minimally tested so far. Please test migrations, but assume that they may break in ways that make it inconvenient or even impractical to recover the test system. Most of our internal appliance testing so far has been of creating working systems based on rPath Linux 2 Beta, not of migrating.

Security update announcements will be made available for rPath Linux 2 Beta, but security updates for beta releases will be lower priority than security updates for generally available releases. Because security-announce@lists.rpath.com is intended to be very low-traffic, security update announcements will be sent only to the update-announce@lists.rpath.com mailing list. Group change summaries will be provided only on the wiki, not sent via email to the update-announce list. Note that most changes to the beta will not reference an advisory.

If you are not yet familiar with rPath Linux or the ground-breaking Conary technology, we recommend that you use our supported and maintained rPath Linux 1 product to become familiar with both technologies before experiencing this beta test snapshot.

Now these are the laws of good Software,
Unwritten and varied they be;
And he who is wise will observe them,
When writing code in script or in C.

As naught may outwit the compiler,
So it is with syntax and its grip,
For the core of programming is syntax,
And the bonds of syntax give the pip.

Take heed what ye pass as directives,
Be your code written simply and plain,
Lest preprocessors choke on the input,
And so ye shall write it again.

If ye labor from morn until even',
And meet with reproof for your toil,
'Tis well that your editor idle;
The compiler--its trousers show soil.

On the strength of all lines in the function,
Dependeth successful return.
Who knows when some misguided caller,
Correct sizing of bufs it will spurn?

When a hacker who's tired returneth,
With some bugs in his code glaring mean;
We debugger new codes for good reason,
Few first passes at coding are clean!
So shall thou, and lest perchance thou grow mired,
In the uttermost parts of the C;
Pray your editor balances braces,
As much and as oft as may be.

Ignore not good documentation
But rather to write it aspire;
Though elder hacks may just use the source
There cometh, perchance, a new hire!

[Inspired by Rear Admiral Ronald A. Hopwood's (C.B., Royal Navy) 1896 masterpiece of naval wisdom. Kipling, please forgive us both.]

Conary 1.2.16 is a maintenance release.

• The rollback API now raises a RollbackError instead of returning non-zero on errors. (CNY-1643)

• When file conflicts occur while cooking groups, the deps that caused a package to be pulled in are listed. (CNY-2308)
• Config policy will automatically append a newline to non-binary files. Files that are marked as Config and appear to be binary will continue to trigger an error as they have previously. (CNY-2422)
• Perl requirements that are not present on the system or provided by the package being built are dropped. (CNY-2180)
• Two expansion functions have been introduced to package recipes: r.glob and r.regexp. These functions return an object that can be substituted for any API parameter that calls for a string based regexp or glob. (CNY-2222)
• Group recipe actions like "r.addAll" and "r.addCopy" now record the version of the group that is being copied from into the newly created group. (CNY-2359)
• Add code to display new-style metadata through rq, q, and showchangeset, as well as propagate that metadata via cooking, promoting, shadowing, and committing source packages. Entering this metadata must still be done through scripts. (CNY-1808)
• Added a hook that allows build requirements to be overridden by rMake. (CNY-2427)
• Build actions that don't match anything (r.Move, r.Copy etc) will now log exactly what they were trying to do. (CNY-2216)

• Conary no longer tracebacks when building a package that contains a pkgconfig reference to a file in the current package that is in a symlinked directory. (CNY-2455)
• The order in which configuration files are read when a glob was passed to includeConfigFile is now deterministic. (CNY-2483)

Conary 2.0.5 is a maintenance release.

• Conary now attempts to provide hints about flavor combinations that would work if it cannot find a flavor that matches your exact request (CNY-1920).
• The rollback API now raises a RollbackError instead of returning non-zero on errors. (CNY-1643)

• When file conflicts occur while cooking groups, the deps that caused a package to be pulled in are listed. (CNY-2308)
• Config policy will automatically append a newline to non-binary files. Files that are marked as Config and appear to be binary will continue to trigger an error as they have previously. (CNY-2422)
• Perl requirements that are not present on the system or provided by the package being built are dropped. (CNY-2180)
• Two expansion functions have been introduced to package recipes: r.glob and r.regexp. These functions return an object that can be substituted for any API parameter that calls for a string based regexp or glob. (CNY-2222)
• Group recipe actions like "r.addAll" and "r.addCopy" now record the version of the group that is being copied from into the newly created group. (CNY-2359)
• Add code to display new-style metadata through rq, q, and showchangeset, as well as propagate that metadata via cooking, promoting, shadowing, and committing source packages. Entering this metadata must still be done through scripts. (CNY-1808)
• Added a hook that allows build requirements to be overridden by rMake. (CNY-2427)

• Conary no longer tracebacks when building a package that contains a pkgconfig reference to a file in the current package that is in a symlinked directory. (CNY-2455)
• The order in which configuration files are read when a glob was passed to includeConfigFile is now deterministic. (CNY-2483)

SCaLE is fast approaching. I'll be manning the Bongo booth. If you're in the area, please stop by and say hi. Bongo is currently a featured appliance on rBuilder Online.

rBuilder Online's move to the new data center has been completed. Thank you for your patience.

We expect to place rBuilder Online in read-only mode for approximately 1 hour to synchronize data starting around 9:00 PM EST on January 19, 2008. After this completes, we will shut down the servers and move them to the new data center. It's possible the outage window will shift depending on the status moving other rPath services to the new data center. We will keep you updated if the schedule changes.

Conary 1.2.15 is a maintenance release.

• Moving an unmodified shadow to tip via cvc promote no longer causes an error. (CNY-2441)
• Recipes using addSvnSnapshot no longer modify the conary configuration object's tmpDir setting (CNY-2401)
• Fixed a rare bug in which dependency resolution would fail when a dependency that used to be provided by one installed package is now provided by two new packages. (CNY-2459)

• Conary will now use the proxy settings stored in its local configuration when loading remote configuration files. (CNY-2363)

• Policies that move files in destdir now track path changes they make so that files will end up in the correct package or component after being moved, when "package=" or "component=" has been used in a build action. (CNY-1679)

Conary 2.0.4 is a maintenance release.

• Moving an unmodified shadow to tip via cvc promote no longer causes an error. (CNY-2441)
• Recipes using addSvnSnapshot no longer modify the conary configuration object's tmpDir setting (CNY-2401)
• Fixed a rare bug in which dependency resolution would fail when a dependency that used to be provided by one installed package is now provided by two new packages. (CNY-2459)

• Conary will now use the proxy settings stored in its local configuration when loading remote configuration files. (CNY-2363)
• PGP keys having some self signatures that fail to pass are no longer failing, as long as at least one self signature passes. (CNY-2439)
• The client enforces the trust model using the internal implementation of OpenPGP. (CNY-1895)

• Policies that move files in destdir now track path changes they make so that files will end up in the correct package or component after being moved, when "package=" or "component=" has been used in a build action. (CNY-1679)

Since its launch on September 15, 2005, rBuilder Online has grown to host over 9,000 projects. rPath designed rBuilder to scale as needed, so the hardware configuration for rBuilder Online is largely unchanged from its original deployment. The hardware on which rBuilder Online runs has been extremely reliable and well-performing throughout the site's tremendous growth. However, we have grown beyond the capabilities of our current Internet Service Provider.

Therefore, rBuilder Online hardware, along with other rPath co-located servers, is moving to a new facility. The new Internet Service Provider has higher quality connectivity to Europe and Asia, which has been a problem for the old provider. Many services will migrate from a virtual server in the old data center to a new virtual server in the new data center. As a result, there should be a relatively short downtime for such resources as wiki.rpath.com, issues.rpath.com, and blogs.rpath.com.

Due to the vast amount of data stored in rBuilder Online, the only practical way to relocate the service is to physically move the servers. The new data center is relatively close to the old one, so the total downtime should be under 24 hours.

Because our customers rely heavily on three particular repositories currently served by the transitioning hardware, rPath is temporarily moving repository services for them to its Raleigh office. This will provide zero downtime for critical services associated with the conary.rpath.com, products.rpath.com, and rap.rpath.com repositories.

The co-location move will take place this weekend, January 18-20, 2008. A specific move window for rBuilder Online will be established when there is more infrastructure in place at the new data center.

I just realized that I never (shame on me!) blogged about the fact that I'm giving a Conary packaging talk called "Conary Packaging: Simply Powerful" at FOSDEM 2008.

I'm excited about this talk, because previous times when I've talked about this material in an ad-hoc fashion (such as the BoF session at last year's Ohio LinuxFest) there have been lots of good questions and the audience have been engaged participants.

I'm also looking forward to meeting for the first time several Foresight contributors with whom I've been working online for years.

So, what will I talk about?

I will present Conary packaging, first demonstrating how simple the process is, then providing technical details of features that make packaging a simple, easy, repeatable process. I will show how Conary handles common cases automatically, while making it easy to override its defaults for unusual cases. I will demonstrate how Conary improves packaging quality, by making packaging consistent and by providing tools to analyze packages and collections of packages. I will explain how Conary's dependency management is different from legacy package management, encoding more thorough dependency information without requiring cascading piles of irrelevant packages.

I'm looking forward to lots of fun discussions about Conary. Come join me!

Conary 2.0.3 is a maintenance release. Conary 2.0.x is not currently included in rPath Linux 1, though it is currently part of rPath Linux development.

• PGP version 3 keys are now supported for verification of package signatures. According to RFC4880, version 3 keys are deprecated. (CNY-2420)
• Superclasses are now loaded from the filesystem. Superclasses can now be cooked. Doing so will make a changeset that installs the recipe itself on the filesystem. (CNY-983)
• When building packages, Conary now reads /etc/ld.so.conf.d/*.conf files to determine whether to include the path in the dependency. (CNY-2433)
• The /etc/ld.so.conf.d/*.conf handling added to Conary 1.2.13 introduced a bug that could erase necessary entries from ld.so.conf in some circumstances. This bug has been resolved. (CNY-2440)
• Spaces in URLs are now automatically escaped. (CNY-2389)
• The Requires() policy now inspects Lib: and Lib.private: pkg-config keywords in .pc files to find library files, and where it finds them, it adds appropriate Conary trove requirements to the .pc files. (CNY-2370)

• The implementation of the util.mkdirChain function has been changed to no longer use exceptions internally to signal that the directory already exists. Raising exceptions is a relatively expensive operation that slows down the data store. Exceptions are still used internally in the very infrequent case of intermediate directories not existing. (CNY-2405)
• Conary will not downgrade packages if no version is specified by the user, but, due to an out of date mirror or other reasons, the update available for a package is older than the currently installed package. (CNY-2402)
• Conary now recognizes /etc/ld.so.conf.d/*.conf files, adding an include line to /etc/ld.so.conf if they exist, and does not add new duplicate entries to /etc/ld.so.conf for directories already specified in /etc/ld.so.conf.d/*.conf files. (CNY-2432)
• A getDatabase() method has been added to the ConaryClient class. This method returns the local system database object. Code that needs to query the local system database should use this method to obtain the database object. (CNY-2316)

• A bug that caused an unhandled exception when adding a new role in the Conary repository web interface has been fixed.

Conary 1.2.14 is a bug fix release.

• The /etc/ld.so.conf.d/*.conf handling added to Conary 1.2.13 introduced a bug that could erase necessary entries from ld.so.conf in some circumstances. This bug has been resolved. (CNY-2440)

Conary 1.2.13 is a maintenance release.

• Conary will not downgrade packages if no version is specified by the user, but, due to an out of date mirror or other reasons, the update available for a package is older than the currently installed package. (CNY-2402)
• Conary now recognizes /etc/ld.so.conf.d/*.conf files, adding an include line to /etc/ld.so.conf if they exist, and does not add new duplicate entries to /etc/ld.so.conf for directories already specified in /etc/ld.so.conf.d/*.conf files. (CNY-2432)

• Superclasses are now loaded from the filesystem. Superclasses can now be cooked. Doing so will make a changeset that installs the recipe itself on the filesystem. (CNY-983)
• When building packages, Conary now reads /etc/ld.so.conf.d/*.conf files to determine whether to include the path in the dependency. (CNY-2433)

The scheduled maintenance of rBuilder Online has been completed.

This maintenance was to install and reconfigure the hardware for rBuilder Online, therefore there are no user visible changes.

Thank you for your interest in rBuilder Online.

rBuilder Online will undergo scheduled maintenance starting Tuesday, January 8th at 10:00pm EDT (-0500 UTC) and ending Wednesday, January 9th at 01:00am EDT. During this time, rBuilder Online and all rpath.org repositories will not be available.

Thank you for your interest in rBuilder Online.

Conary 2.0.2 is a bug fix release. Conary 2.0.x is not currently included in rPath Linux 1, though it is currently part of rPath Linux development.

• PGP version 2 signatures are now properly parsed. Version 2 signatures are documented in the outdated RFC1991 and are considered deprecated, but some PGP keys contain them. (CNY-2417)

• A bug that caused an unhandled exception when adding a new role in the Conary repository web interface has been fixed.

Conary 2.0.1 is a bug fix release. Conary 2.0.x is not currently included in rPath Linux 1, though it is currently part of rPath Linux development.

• The Requires() policy now inspects Lib: and Lib.private: pkg-config keywords in .pc files to find library files, and where it finds them, it adds appropriate Conary trove requirements to the .pc files. (CNY-2370)

• An update bug that could result in a trove integrity error has been fixed. The issue would occur when updating packages where, for some files, the only changes are to file versions. (CNY-2403)

Conary 1.2.12 is a maintenance release.

• A getDatabase() method has been added to the ConaryClient class. This method returns the local system database object. Code that needs to query the local system database should use this method to obtain the database object. (CNY-2316)

• Spaces in URLs are now automatically escaped. (CNY-2389)
• The Requires() policy now inspects Lib: and Lib.private: pkg-config keywords in .pc files to find library files, and where it finds them, it adds appropriate Conary trove requirements to the .pc files. (CNY-2370)

• The PackageSpec documentation incorrectly stated that you could pass both package and component information to it; this has been corrected to properly redirect to ComponentSpec for this usage. (CNY-2387)
• An update bug that could result in a trove integrity error has been fixed. The issue would occur when updating packages where, for some files, the only changes are to file versions. (CNY-2403)

Conary 2.0.0 is a major new feature release. Conary 2.0.x is not currently included in rPath Linux 1, though it is currently part of rPath Linux development.

• The way Conary handles architecture flavors has been changed, primarily for better support for multilib systems that support having both 32-bit and 64-bit packages installed.
• The Conary repository Access Control List capabilities have been significantly upgraded to support per-trove ACLs. Trove permissions are cached and recalculated whenever the ACLs change or new troves are added to the repository, for better scaling and faster changeset retrieval.
• All dependencies on the GnuPG "gpg" program have been removed; Conary now implements the required OpenPGP functionality internally.

• Conary 2.0 is significantly faster for many repository commit operations. Some "cvc promote" operations, in particular, have a 200% performance improvement. Some group commit operations are 75% faster. Creating a new shadow can be as much as 100% faster. Details of some of the changes that provide the improvement are below.
• Committing to repositories has been reworked to pull all modified streams from the repository or database with one query. (CNY-2053)
• Unchanged stream sets now return None on diff. This prevents the Conary repository from having to do unnecessary work to retrieve the "old" version of a stream and apply a diff when nothing actually changed.
• Unchanged file streams are now 2-byte, backwards compatible, sequences instead of more complex representations of "nothing changed".
• The commit code (both client and server) recognizes unchanged files and does not merge stream sets for them.
• Distributed changeset creation no longer recompresses file contents.
• "cvc promote" no longer recompresses non-config files during changeset assembly. (CNY-2202)
• "cvc promote" now gets file contents and streams from changesets when a lot of them are needed from the same trove. (CNY-2202)
• Changesets that represent a "cvc promote" operation have been changed to be relative to either the source of the promote (if promoting within a server) or to the current version on the target. (CNY-2202)
• The "cvc shadow" command now commits relative changesets when the shadow is made to the same repository as the original trove.
• The Conary client now tries to perform update jobs as close to updateThreshold (without going over) as practical for a noticeable performance improvement. (CNY-2283)
• Committing changesets to the repository has been modified to check for files which don't have contents available (during a promote or shadow, for example) with one SQL query. Previously, a single SQL query was used to check for each file. (CNY-2053)

• Conary will now ignore flavoring and requirements from ELF libraries that are built for architectures other than the architecture that is being built for. This avoids a common problem where a single extra (unused) sparc file library causes an entire package to be flavored to be installed on systems that support both x86 and sparc instruction sets, as that configuration is not meaningful. (CNY-1717)
• The rarely used "cvc describe" command line interface has been removed. (CNY-2357)
• The rarely used "cvcdesc" script has been removed. (CNY-2357)
• Spaces in URLs provided in source actions such as addArchive are now automatically escaped to make them legal URLs. (CNY-2389)
• The "cvc promote" command now uses the labels and branches specified in the "from" section of any promote as the place to search for packages (CNY-2235).
• The PackageSpec documentation incorrectly stated that you could pass both package and component information to it; this has been corrected to properly redirect to ComponentSpec for this usage. (CNY-2387)

• The "getTroveLatestByLabel" client-side call has been added.
• Most repository exceptions are demarshalled using logic in the exception class itself rather than in a large if/elif block. (CNY-747)
• The "troveNames" and "troveNamesByServer" methods now accept a "troveTypes" argument, and by default return only troves that are present. (CNY-1838)
• The mirror client supports a --fast-sync flag which will only scan for new troves in the source and skip the time-expensive scans for changed trove info records. (CNY-1756)
• The mirror client supports a --absolute flag which will make it use only absolute changesets to mirror content. (CNY-1755)
• The rollback stack code has been split into a separate class. (CNY-2061)
• A new "conary rmrollback" command that removes old rollbacks has been added. (CNY-2061)
• Rollback objects now have an "isLocal" method to tell whether applying that rollback will require repository access. (CNY-2077)
• Conary now allows a biarch system to have one flavor that expresses both the x86 and x86_64 support. The flavorPreferences configuration option informs conary to prefer x86_64 packages. This change allows group building to automatically resolve x86 packages. (CNY-525)
• Conary no longer uses gnupg for OpenPGP key management. (CNY-2349)
• Conary clients no longer send an absolute URI when talking directly to a repository server. (CNY-2324)
• Many repository permission handling methods have been renamed for consistency. For a complete list, see doc/PROTOCOL.versions in the Conary source code. (CNY-2298)
• Setting/resetting the admin field permission is now handled by the setUserGroupIsAdmin() call instead of addAcl/editAcl. (CNY-1782)

• The scoring for target flavor sets has been fixed. (CNY-1539)
• The (unimplemented) concept of caps for permissions has been removed.
• Server methods have been decorated with the @requireClientProtocol decorator.
• The "usergroups" term has been replaced with "roles" in the repository web user interface. (CNY-1973)
• Server exceptions have been reworked to have marshalling logic in the exception class instead of in a large if/elif block. (CNY-747)
• Server exceptions are marshalled in the proxy layer now instead of in both the proxy and server layers.
• When the repository database is locked, "RepositoryLocked" exceptions are now returned for all code paths. (CNY-1596)
• The repository call log now tracks changeset cache misses. (CNY-1843)
• Repositories no longer pass anonymous hints to the proxy layer. The hint has been False since Conary 1.1.29.
• The "troveNames" method now filters by using the "troveTypes" argument. (CNY-1838)
• The "getPackageBranchPathIds" method no longer requires access to the entire version history of a package; it returns the pathIds and fileIds for troves the user is allowed to see. Previously, an InsufficientPermission exception was raised if part of the version history of a package was not visible to the user. (CNY-2038)
• The repository call log now records the time required to service the request. (CNY-2305)
• A "serializeCommits" boolean server configuration option has been added. This is best turned on for repositories used for development, but is unneeded for repositories that are mirrors. (CNY-2285)

• The XML-RPC protocol now allows passing keyword arguments for exceptions. (CNY-747)
• XML-RPC return values no longer include the "useAnonymous" flag.