Conary News
Archives for: 2009
Conary 2.1.1 is a maintenance release.
New Features- Added repair command as an experimental feature. (CNY-1800)
- Files marked as "missingok" in encapsulated RPMs now have that status represented by a "MissingOkay" flag on the corresponding Conary file object. (CNY-3306)
- Files from encapsulated RPMs now have the mtime from the RPM recorded in troveInfo. The mtime of last change is still used to calculate fileIds, preserving knowledge of when meaningful file information has changed. (CNY-3307)
- Added support in verify and update paths for prelinked binaries. (CNY-3298)
- Changeset reassembly for capsules ignores directories. (CNY-3305)
- Encapsulated symbolic links are represented as having the mode that will exist on the filesystem, even if that varies from the mode recorded in the capsule. (CNY-3304)
- Previously, getTroveLeavesByPath() didn't return leaves; it returned the most recent trove which includes that path. It now returns only leaves (most recent versions) that include that path, and if the path is only included in older versions, no troves are returned. (CNP-184)
- Database connection objects now have a close_fork() method that closes the connection without notifying the server.
Conary 2.1.0 is a new feature release.
New Features- Conary now allows packages to be installed using foreign package managers via wrapped content called capsules. RPM capsule support is included in this release. (CNY-3217, CNY-3213)
- Added ignoreDependencies configuration option, which allows users to specify certain dependency classes which should be ignored during installs. The default is "abi rpmlib".
- The xz compression file format is now supported for decompressing sources. (CNY-3207)
- The changemail.py repository commit action module now takes an optional --maxsize argument and truncates changemail messages to fit. (CNY-3205)
- The Run build action now does path-based build requirement discovery like most build actions. (CNY-3222)
- Conary now allows identical files to be shared during installs and updates. Both troves own the shared files, and they must be updated at the same time when the shared files change.
- Conary now honors the no_proxy environment variable to allow direct connections to specific hosts and domains. (CNY-3246)
- Added excludeCapsuleContents config option for servers.
- Added ChangeSet.removeCommitted() to filter already-committed troves from a changeset.
- The bootstrap python module dependency discovery proxy now provides additional useful informational messages to aid debugging.
- Conary proxies are responsible for reassembling the content for changesets containing capsules. (CNY-3228, CNY-3259)
- The --capsules command line option modifies showchangeset, repquery, and query commands to display file information for capsules instead of for installed files.
- A new Conary proxy setting, injectCapsuleContentServers, controls changeset content injection for proxies talking only to hosts in this list. (CNY-3261)
- Conary no longer uses regular expressions to represent manifest files and other internally-generated lists of files matched by policy, and so will not raise OverflowError for manifests with many files, or from policy generated from source objects, on platforms with a restricted regular expression code size. (CNY-3282)
- The python dependency finder client has been modified to resolve incompatibilities on the client side with python 2.3. (CNY-3285)
- Conary can now read cpio archives directly. (CNY-3280)
- The Conary repository now provides a method for obtaining commit progress updates. "conary commit" now provides progress output. (CNY-3290)
- When cooking groups where checkOnlyByDefaultDeps is set to False, added dependencies will now inherit the byDefault setting of the depending component. (CNY-3195)
- The imageGroup flag in group recipes now implies checkOnlyByDefaultDeps=False. (CNY-3195)
- Group search paths may now contain lists or tuples of troves to be considered as a single "bucket". (CNY-3286)
- In-core changesets previously could be merged only once into a ReadOnlyChangeSet. After multiple merges, reset() previously failed to reset the underlying in-core changeset. This has been fixed.
- Conary no longer exits with an unhandled KeyError exception if an unknown dependency type is encountered in the local database. (CNY-3232)
- Decompression with xz is preferred over unlzma, as CNY-3194 intended. (CNY-3231)
- rpm2cpio now correctly handles rpm packages compressed with xz. (CNY-3234)
- Unpacking an RPM with addArchive or addCapsule now functions correctly even if the CPIO archive in the RPM contains a non-traversable directory. (CNY-3244)
- When running an external entitlement generator, use /dev/null for stdin instead of closing the file descriptor.
- Added logging to several source actions to ensure that some output is provided to make debugging build failures easier.
- Call lstat() instead of stat() to deal with circular links. (CNY-3251)
- Bootstrapping python dependencies now uses python source intead of a version-specific compiled python file.
- The addCapsule source action properly verifies GPG keys when a keyid argument is supplied. (CNY-3258)
- Commiting signed troves to repositories without signatures no longer causes repository tracebacks.
- Repositories no longer store duplicate file path information when a path is committed for the first time by a job which references it multiple times.
- Replacing a symlink with a regular file no longer creates the regular file as the target of the symlink.
- Local rollbacks properly restore file contents when the file was replaced by a non-regular file (such as a symlink)
- Cooks will no longer block indefinitely if a subprocess forks and does not exit. (CNY-3284)
- When startGroup is called, flags will now be properly copied from the parent group. (CNY-3287)
- Replacing a symlink which was part of a loop no longer causes a traceback.
- A repository's internal getfile function now avoids any database access when no files have been requested.
- Loading troveinfo data uses bulk operations, speeding up repository commits.
- Committing changesets now iterates over troves the changeset is relative to, rather than getting them one at a time, significantly reducing the number of sql calls.
- Conary now uses a single call when validating that file objects exist in the repository during a commit.
- Repository commits now handle file paths and file streams separately, in order to reduce the size of the temporary tables and the commit path.
- The number of SQL calls used to create the flavors needed by a repository commit has been dramatically reduced.
- No-operation SQL calls during repository commits when no redirects are being commited were eliminated.
- Repository commits now merge all dependencies from the temporary tables into the final tables at once rather than once per trove.
- Repository commits now update permission tables for all new troves at once.
- Updates to latest tables are now done for all troves at once, and are based on the current latest entries and the new trove. Formerly they were done one at a time based on everything in the repository (through a view).
- The rpmhelper.Header object can now produce dependency sets that represent what an RPM requires and provides.
- A new RPM dependency class has been added. This class is used to represent dependencies expressed in RPM packages that cannot be mapped directly to a native Conary dependency.
- Colons in dependency flags are now escaped as \: to allow deterministic thawing.
- CIL dependency parsing uses xml.etree.ElementTree, if available, instead of elementree.ElementTree.
- The LDFLAGS setting is now honored when linking binaries.
- Trove and TroveChangeSet objects now represent paths as directory/basename pairs.
- Changeset dict of absolute file object changes are now indexed more efficiently.
- StreamCollection classes can now choose to ignore the skipSet when freezing for a performance boost
- Dirnames/basenames are added as a single operation at the beginning of a repository commit to make the ids available for the files build removing the need for large, complex joins later.
conary-policy 1.0.26 is a maintenance release.
- When "file:" requirements are not not explicitly met by "file:" provides but the path exists in a trove, the new ResolveFileDependencies policy replaces the unsatisfied "file:" requirements with "trove:" requirements. (CNP-184)
- The XinetdConfigRequires policy now creates file: dependencies instead of trove: dependencies by default, depending on the ResolveFileDependencies policy to change to trove: dependencies on platforms for which the /usr/sbin/xinetd path is not explicitly provided. (CNP-183)
conary-policy 1.0.25 is a new feature release.
- Policies have been made capsule-aware and assume that the capsule will handle essentially all operations involving files in capsules. (PFM-506)
- The RemoveBootstrapTroveDependencies policy removes trove dependencies that are not satisfied within the packages being built as a bootstrap because they may be computed against an incompatibly-named set of host packages. (PFM-506)
- The PHPRequires policy now reports adding dependencies. (CNP-181)
Conary 2.0.50 is a maintenance release.
Bug Fixes:- cvc rdiff now functions correctly on trove versions where no previous version exists. (CNY-3218)
- Updates of multiple flavored groups containing pre-update scripts no longer fail after the critical update is applied. The fixed Conary issue is related to CNY-3216, but only manifests itself when multiple groups are present. (CNY-3219)
- Path-based action build requirement discovery now functions correctly when spaces are present in environment variables or adjacent spaces are present before the name of the executable in a command string. (CNY-3224)
- When building local changes, conary would sometimes look through the database by fileId to get the path for a file it needed, instead of just looking in the filesystem directly. That caused a failure when two paths had the same fileId but one of them had been removed from the system. (CNY-3226)
- Building from the repository will no longer attempt to re-download autosourced files. (CNY-3221)
- Conary will now link against an embedded copy of sqlite if a specific version of sqlite has previously been configured and built under the conary/pysqlite3/ directory. At this time, the supported version is 3.6.18.
Conary 2.0.48 is a bug-fix release.
Bug Fixes
- A bug introduced in the previous release that caused installations of device nodes as a non-root user to crash has been fixed. (CNY-2038)
- Directory permissions are no longer unnecessarily updated if they have not changed. (CNY-3202)
- A race condition in restoring files that share content has been fixed. The race involved changing the contents of the restored file before references to that file are restored. (CNY-3203)
Conary 2.0.46 is a maintenance release.
New Features- If python-keyutils is available, cvc will try to interact with the user's session keyring. (CNY-3190)
- Conary now uses xz in preference to unlzma to decompress lzma data. (CNY-3194)
- A new function, parseChangeSpec, that parses a single trove update specification such as name=ver1[flv1]--ver2[flv2], was added to cmdline.py. (CNY-3197)
- The code path leading to getChangeSet was reworked to use significantly fewer repository calls, resulting in a massive speedup for uncached, large requests.
- PGP passphrases, required for signing troves, are now fetched using a callback. (CNY-3189)
- Repositories running on postgresql can properly handle queries for labels sharing the hostname portion. (CNY-3192)
- The sha1 for files in a repository could be improperly left as NULL for some cases involving shadows. This has been fixed, and migrating existing repositories to version 17.3 will fix existing missing sha1 checksums.
- Group pre-execution scripts now run exactly before any package that is part of the group. Previously, pre-execution scripts were run at the very beginning of the update, where the probability of their interpreter not being installed was high, especially for pre-install scripts. (CNY-2705)
- Group pre-execution scripts no longer unnecessarily fetch information they already have from the repository. (CNY-3196)
- Password information for XMLRPC server URLs is now hidden in tracebacks. (CNY-3198)
Conary 2.0.45 is a maintenance release.
New Features:- The standalone Conary repository server now prints a message if it fails to import the Conary REST api.
- Added the restore command, which restores file ownership to troves from which they were removed (because of a remove operation or --replace-files).
- The addGitSnapshot action now accepts a branch= argument. (CNY-2368)
- Fetching path hashes from a changeset now uses absolute trove info when available (which it is for any recent enough server), instead of merging differential trove info with database trove info, which significantly speeds up update planning.
- Database operations now use the TRUNCATE command where available, which may speed up some repository queries.
- An internal API used for emulating repository access by certain roles previously did not function when given a specific set of roles; this has been corrected.
conary-policy 1.0.24 is a maintenance release.
- Libraries (ELF files containing DT_SONAME entries, not symbolic links) with file names ending in .so (instead of .so.something) are now tagged as shared libraries. (CNP-175)
- TagLocale no longer looks at files in %(debuglibdir)s or %(debugsrcdir)s, since it would only complain about them and ignore them anyway. (CNP-177)
Conary 2.0.44 is a maintenance release.
New Features:- Conary can now be configured with a set of certificate authorities (trustedCerts) against which all secure repository access will be validated. (CNY-2735)
- Conary now logs the starting, output, and error exit codes of trove scripts to the /var/log/conary file. (CNY-3150)
- The "conary updateall" command now honors the --just-db option. (CNY-3167)
- The EULA_Conary.txt file has been renamed to EULA_Conary_Dual_License.txt in order to highlight the contents of the beginning of the file, which describe a choice of license terms. The contents of the file are not changed, and the licenses under which Conary may be used are not changed. (CNY-3168)
- The SetModes build action now takes an optional allowNoMatch keyword argument, that causes a warning to be printed instead of failing the build when the target file does not exist. (CNY-3171)
- Added removeInvalidRollbacks() to client API. (CNY-2933)
- The conary.lib.mainhandler.MainHandler.main() method now sets sys.excepthook only if the setSysExcepthook method variable is True (as it is by default), allowing subclasses to take responsibility for setting sys.excepthook differently. (CNY-3170)
- An internal function used to install files no longer leaks a file descriptor on error. (CNY-3152)
- Unknown errors while retrieving PGP keys for archive verification are now gracefully handled. (CNY-3120)
- The cvc refresh command now ignores negative cache entries. (CNY-3157)
- The cvc sign command properly handles keys specified by key ID, in addition to fingerprints. (CNY-3139)
- The conary updateall --items command now honors --labels, --full-versions and --flavors flags. (CNY-3138)
- The --lsprof option now writes out profiling information even if an exception occurs, for both the conary and cvc commands.
- Dependency numbers in temporary dependency checking tables no longer overlap dependency numbers in the persistent database.
- Dependencies with multiple flags previously were not marked as satisfied during dependency resolution passes, forcing extra work on subsequent passes.
- Local rollbacks previously failed to restore contents when a file's metadata changed but the contents stayed the same.
- Generating local rollbacks previously failed for non-root users when the file was unreadable due to permissions.
- The ccs2tar script uses tarfile.open() in a way that is compatible with both python 2.4 and 2.6 (CNY-3160)
- Brace expansion produces results similar to bash. (CNY-3158)
- Trove searches with invalid (non-ASCII) input no longer cause a crash on PostgreSQL-backed repositories. (CNY-3165)
- Flavor processing in depSetFreeze previously did not properly handle return codes from depFreezeRaw, which could lead to a segmentation fault. (CNY-3166)
- Path normalization no longer produces erroneous double-slash entries in /etc/ld.so.conf. (CNY-3142)
- Accessing a server through a Conary proxy that injects entitlements now correctly forces the use of SSL. (CNY-3176)
- Update handling now looks for preerase trove scripts in the local database instead of instantiating full troves from the database and looking in those troves.
- Dependency tables for removed troves are no longer dropped and rebuilt during each iteration of dependency solving; they are now incrementally built during subsequent iterations.
- Dependency ordering code now looks in the local database for trove references instead of instantiating troves to get that information.
- Code which decides what referenced troves need to be erased when
collections are erased now:
- looks in the local database for trove references instead of instantiating troves to get that information
- batches reference checks, pin checks, and presence checks across troves instead of checking those for each trove individually
- Fast-pathed code which matches old troves with new troves for the case where only a single old trove and a single new trove with a given name exist.
- Graph calculations for dependency ordering now only occur when the results are needed instead of for every pass through the dependency resolver.
- Reworked rdiff to not download a full changeset from the repository. It now downloads only the config (text) files and diffs them on the client. This allows it to show diffs for changes across repositories and yields a nice speedup. (CNY-3039)
- Reworked "conary remove" functionality to generate a changeset for the file remove and commit that through the normal update path.
- During dependency checks for new troves, no longer include every trove which requires a given dependency in the SQL tables. Instead, include that dependency only once and expand the matched requirements when we order troves (which is the only time it matters).
- Added simple caching of path hashes from the database to improve performance of large system updates.
Conary 2.0.43 is a maintenance release.
Internal Changes:- Additional information is now included when Conary receives corrupt changesets, to assist in diagnosing the cause of the changeset corruption. (CNY-2629)
- File paths from changesets are now properly normalized, and the rest of the code correctly handles the single-slash prefix in the case of installing to the main root directory. This fix completes the change introduced in Conary 2.0.41 and reverted in Conary 2.0.42, that caused tag handlers to malfunction when run by Conary 2.0.41. (CNY-3142)
- Conary now puts libtool .la files in :devellib components by default, rather than in :lib components, for better multilib development environment support. (CNY-3143)
- An internal function, mkdirChain, used to create directories, could be subject to a race condition on Python 2.4. This has been corrected. (CNY-3145)
- The documentation for the mode= argument to several build actions has been improved to make it clearer to specify an octal number. (CNY-3146)
- The cvc refresh command properly handles URL strings that contain username and password specifications. (CNY-3108)
Conary 2.0.42 is a bug fix release.
Bug Fixes:- A previous change that normalized paths before being passed into tag handlers has been reverted, because it caused tag handlers to malfunction. (CNY-3141)
Conary 2.0.41 is a maintenance release.
Bug Fixes:- When a derived packages was created incorrectly, Conary could sometimes trace back; in these cases, Conary will now print a more useful error message. (CNY-3125)
- Displaying the output from verify spent a significant amount of time querying repositories for information which was available in the local database. All data accesses for verify are now satisfied locally for a significant (10x) speedup in performance).
- The output for conary verify is more succinct; instead of two summary lines per component validated, it now prints one line per component that has differences to display. (CNY-2419)
- Using --verbose on verify now gives some progress indication.
- Removed improper options for localcs from the man page.
- Verify significantly sped up.
- Turned off threading in recreatedb, because threading broke for some manifests. (CNY-3128)
- Rewrote SQL query which removes old dependencies for better performance.
- The cvc file cache now understand file mode changes as well as content changes. (CNY-3082)
- Key signatures for archives in source components are no longer checked when the source is in the repository, but rather checked at commit time. (CNY-3130)
- Fixed bug which caused files to be dropped from builds when two paths shared the same pathId somewhere in the history of that trove. (CNY-3133)
- Updates and local changeset generation handle troves with renames, though cases with multiple versions of such troves being installed simultaneously have known shortcomings. (CNY-3134)
- Updating source directories to versions which added or removed a factory now updates the CONARY file properly. (CNY-3073)
- Verify now relies on file size and mtime to determine whether files have been modified. The --hash option has been added to verify to force sha1s to be used instead.
- The system database can now return file objects along with troves to avoid double walking of the file tables.
- TroveSource.walkTroveSet now walks troves in a sorted, depth first order.
Conary 2.0.40 is a maintenance release.
Bug Fixes:- keepRequired now recognizes that local troves may have had files removed, preventing file conflicts with incoming troves
- Fixed formatting error in cvc explain that caused policy actions to be listed twice. (CNY-3107)
- Conary ignores configuration file errors for certain local operations including conary query, rblist, and showcs. (CNY-3111)
- Python flags on a Requires dep now use exactly the flags on the corresponding Provides, rather than being trimmed to match flags on python:runtime's Requires deps. (CNY-3030)
- Disallow characters '"\\()[] in labels
- Added getLabelsForHost call, and bumped server protocol to 66. (CNY-2771)
- Added getFilesFromTrove call, and bumped server protocol to 67. (CNY-3100)
- Repositories now check the database to see which files already exist instead of looking in the filesystem for a noticeable speedup in commit time for systems with NFS based data stores.
- In a GroupRecipe, r.remove now takes a use flag to determine if a trove should be removed. This matches the behavior of r.add*. (CNY-3115)
- Added types to allow creation of sockets and named pipes via conaryclient/newtrove.py code paths
- The "migrate" command now accepts the "--from-file" option (CNY-3114)
- Standalone and apache servers now pull in REST api from the crest project if the code is available. Note that restlib and xobj are required for the crest implementation, and that the "baseUri" must be specified in the server config file (it's normally /conary).
- Repository commits now take advantage of knowing which parts of the commit have not changed since the old version for better performance
- The internal OpenPGP implementation can now change passphrases on secret keys. (CNY-3123)
Conary 2.0.39 is a maintenance release.
Bug Fixes:- Fixed potential segfault in sqlite bindings when running under python 2.6. (CNY-3103)
- Promoting a binary foo from a to b, and then promoting a group containing foo from b to a will no longer cause foo to be unnecessarily repromoted. (CNY-3088)
- Fixed an interaction with loadRecipe overrides used by rmake and autoloadRecipes that would cause rmake to not find any matches for autoloadRecipes that were included in biarch flavored superclasses. (CNY-3090)
- Redirect components should not have build requires set (CNY-3069)
- Updates are no longer halted by the signal SIGWINCH, or SIGURG, which are normally ignored. (CNY-3098)
- getTrovesBySource now returns a list of the accessible binaries associated with a source even if the source is not accessible. (CNY-3099)
- Changeset-based migrations no longer fail when a flavor is dropped from the group that is being migrated. (CNY-3101)
- Attempting to promote a binary will no longer reclone the binary if a binary is already the latest at the target branch, even if there is a later source committed on the target branch. (CNY-3089)
Conary Policy 1.0.23 is a maintenance release.
Changes:- A bug in SymlinkTargetRequires that caused a traceback when a symlink path contained a special regex character, such as +, has been fixed. (CNP-169)
- The NormalizePythonInterpreterVersion policy has been modified to apply to all python scripts, regardless of whether they are executable, so that it applies to python modules as well as scripts. (CNP-170)
- Improved policy compatibility with python 2.6 (CNP-173)
- A bug in PHPRequires policy that caused builds to fail or take an extremely long time to complete when both php and php5 were available has been fixed. (CNP-174)
Conary 2.0.38 is a bug fix release.
Bug Fixes:- Conary versions 1.1.20 (April 2007) through 2.0.37, inclusive, use a SHA256 implementation provided by the pycrypto package, which contained an implementation that generates faulty digests for certain message lengths. To compensate, a backward compatibility module that implements bug-for-bug compatibility with the incorrect SHA256 algorithm has been added, in order to inter-operate (generate and validate signatures) with Conary versions which use the incorrect SHA256 implementation in pycrypto. (CNY-3086)
Conary 2.0.37 is a bug fix release.
Bug Fixes:- Resolved a bug introduced in the 2.0.36 sha1Uncompress function that could cause update operations to fail. (CNY-3084)
Conary 2.0.36 is a maintenance release.
New Features:- Policy can now be added per distribution in the directory /usr/lib/conary/distro/policy. (CNY-3071)
- More fixes for flavored superclass groups - you can now commit recipes when autoloading flavored superclass groups. (CNY-3061)
- Schema migrations are now transactional, meaning that any failure during migration will roll back cleanly where possible. (CNY-2789)
- If a user removes a file, and then recreates the file as a directory, the warning message previously did not include the affected path. This has been fixed. (CNY-3076)
- Path conflicts generated during group cooks correctly print. (CNY-3079)
- You can now display buildreqs using --show-buildreqs for other types of recipes other than packages (CNY-3075)
- You can now add archives with files with % in them when perserving ownership. (CNY-3080)
- All troveInfo is now used to calculate a changeset fingerprint, regardless of the client version specified. This allows a proxy to expire its cached version when extended trove info is updated. (CNY-3083)
- Bugs that could cause erroneous error messages in conary.lib.misc.sha1Copy() and conary.lib.misc.sha1Uncompress() have been fixed.
Conary 2.0.35 is a maintenance release.
Bug Fixes:- setTroveInfo no longer sends extended metadata information to update services that cannot understand it. (CNY-3050)
Conary 2.0.34 is a maintenance release.
Bug Fixes:- You can now have flavored groups for superclass groups - these can be created when superclass groups are created as a part of another group recipe. This release includes additional fixes over those added in 2.0.33. (CNY-3061)
- Post-mortem debugging sessions with recipes using --debug-all now start at the point of the innermost exception, making debugging easier.
Conary 2.0.33 is a maintenance release.
New Features:- When adding an XML file (filename ending in ".xml") to a source component, it will be added as a text (config) file by default, and so changes will be managed with diff/patch merging. Specify the --binary option to avoid merging. (CNY-3045)
- When adding groups via addAll, conary now reports the exact name, version, and flavor of the group being added. (CNY-2977)
- Fixed reporting of errors when using glob objects in actions such as r.Remove. (CNY-3055)
- Promotion of source components where files have been renamed across repositories has been fixed. (CNY-3059)
- Preserving ownership in tar archives containing files with missing intermediate directories now functions. (CNY-3060)
- Added support for python module discovery in python 2.6, which uses a new PREFIXES variable to determine where to search for modules. (CNY-3063)
- You can now have flavored groups for superclass groups - these can be created when superclass groups are created as a part of another group recipe. (CNY-3061)
- Metadata promotion is more robust in the context of other inaccessible or missing older troves. (CNY-2611)
- Promotion of inidividual components is no longer allowed. (CNY-2941)
- Fixed a case conaryclient.makeSourceTrove could fail to create a trove because it tried to assign it a previously existing version. (CNY-3028)
- A bug that caused an IOError when a repository attempted to commit a changeset larger than 2 GiB has been fixed (CNY-3065)