| layout | title |
|---|---|
default |
Release Notes |
This is a list of release notes for various releases, you should review these before upgrading as any potential problems and backward incompatible changes will be highlighted here.
- TOC Placeholder {:toc}
This is a maintenance release in the production series of MCollective and is a recommended upgrade for all users of 1.2.0.
- Improve error handling in the inventory application
- Fix compatablity problems with RedHat 4 init scripts
- Allow . in Fact names
- Allow applications to use the exit method
- Correct parsing of the MCOLLECTIVE_EXTRA_OPTS environment variable
This release should be 100% backward compatable with version 1.2.0
| Date | Description | Ticket |
|---|---|---|
| 2011/06/02 | Correct parsing of MCOLLECTIVE_EXTRA_OPTS in cases where no config related settings were set | 7755 |
| 2011/05/23 | Allow applications to use the exit method as would normally be expected | 7626 |
| 2011/05/16 | Allow . in fact names | 7532 |
| 2011/05/16 | Fix compatability issues with RH4 init system | 7448 |
| 2011/05/15 | Handle failures from remote nodes better in the inventory app | 7524 |
| 2011/05/06 | Revert unintended changes to the Debian rc script | 7420 |
| 2011/05/06 | Remove the test agent that was accidentally checked in | 7425 |
This is a release in the development series of mcollective. It features major new features, some bug fixes and internal structure refactoring.
This release is for early adopters, production users should consider the 1.2.x series.
- Agents can now programatically declare if they should work on a node
- Applications can now use the exit method as normal and clean disconnects will be done
- The target collective for registration messages is configurable. In the past it defaulted to main_collective
- Error reporting in applications, agents and mcolletive core has been improved
- The RC script works better on Red Hat 4 based systems
- The connector layer is being improved to make it easier to use other middleware. This release starts this process but it's far from complete.
- The sshkey plugin was removed from core and moved to the plugins project
If you were using the sshkey plugin you need to ensure your CM system is copying it out prior to this upgrade as the packages will not contain it anymore.
If you have your own connectors other than the STOMP one we supply you should wait to upgrade till 1.3.1 at which point you will need to make extensive changes to your plugins internals. If your CM is copying out the connector you have to ensure that when this version of MCollective start that the new plugin is in place.
| Date | Description | Ticket |
|---|---|---|
| 2011/06/07 | Exceptions raised during option parsing were not handled and resulted in stack traces | 7796 |
| 2011/06/06 | Remove the sshkey, it's being moved to the plugin repository | 7794 |
| 2011/06/02 | Correct parsing of MCOLLECTIVE_EXTRA_OPTS in cases where no config related settings were set | 7755 |
| 2011/05/31 | Disconnect from the middleware when an application calls exit | 7712 |
| 2011/05/29 | Validations failure in RPC agents will now raise the correct exceptions as documented | 7711 |
| 2011/05/25 | Make the target collective for registration messages configurable | 7650 |
| 2011/05/24 | Rename the connector plugins send method to publish to avoid issues ruby Object#send | 7623 |
| 2011/05/23 | Log a warning when the CF file parsing fails rather than raise a whole ruby exception | 7627 |
| 2011/05/23 | Allow applications to use the exit method as would normally be expected | 7626 |
| 2011/05/22 | Refactor subscribe and unsubscribe so that middleware structure is entirely contained in the connectors | 7620 |
| 2011/05/21 | Add the ability for agents to programatically declare if they should work on a node | 7583 |
| 2011/05/20 | Improve error reporting in the single application framework | 7574 |
| 2011/05/16 | Allow . in fact names | 7532 |
| 2011/05/16 | Fix compatability issues with RH4 init system | 7448 |
| 2011/05/15 | Handle failures from remote nodes better in the inventory app | 7524 |
| 2011/05/06 | Revert unintended changes to the Debian rc script | 7420 |
| 2011/05/06 | Remove the test agent that was accidentally checked in | 7425 |
This is the next production release of MCollective. It brings to an end active support for versions 1.1.4 and older.
This release brings to general availability all the features added in the 1.1.x development series.
- The concept of sub-collectives were introduced that help you partition your MCollective traffic for network isolation, traffic management and security
- The single executable framework has been introduced replacing the old
mc-
*commands - A new AES+RSA security plugin was added that provides strong encryption, client authentication and message security
- New fact matching operators <=, >=, <, >, !=, == and =~.
- Actions can be written in external scripts and therefore other languages than Ruby, wrappers exist for PHP, Perl and Python
- Plugins can now be configured using the plugins.d directory
- A convenient and robust exec wrapper has been written to assist in calling external scripts
- The MCOLLECTIVE_EXTRA_OPTS environment variable has been added that will add options to all client scripts
- Network timeout handling has been improved to better take account of latency
- Registration plugins can elect to skip sending of registration data by returning nil, previously nil data would be published
- Multiple libdirs are supported
- The logging framework is pluggable and easier to use
- Fact plugins can now force fact cache invalidation. The YAML plugin will force a cache clear as soon as the source YAML file updates
- The ping application now supports filters
- Network payload can now be Base64 encoded avoiding issues with Unicode characters in older Stomp gems
- All fact plugins are now cached and only updated every 300 seconds
- The progress bar now resizes based on terminal dimensions
- DDL files with missing output blocks will not invalidate the whole DDL
- Display of DDL assisted complex data has been improved to be more readable
- Stomp messages can have a priority header added for use with recent versions of ActiveMQ
- Almost 300 unit tests have been written, lots of old code and any new code being written is subject to continuos testing on Ruby 1.8.5, 1.8.6 and 1.9.2
- Improved the Red Hat RC script to be more compliant with distribution policies and to reuse the builtin functions
- The old mc-
*commands are being removed in favor for the new mco command. The old style is still available and your existing scripts will keep working but porting to the new single executable system is very easy and encouraged. - MCOLLECTIVE_TIMEOUT and MCOLLECTIVE_DTIMEOUT were removed in favor of MCOLLECTIVE_EXTRA_OPTS
- mc-controller could exit all mcollectived instances, this feature was not ported to the new mco controller application
- mcollectived and all of the standard supplied client scripts now disconnects cleanly from the middleware avoiding exceptions in the ActiveMQ logs
- Communications with the middleware has been made robust by adding a timeout while sending
- Machines that do not pass security validation are now handled as having not responded at all
- When a fire and forget request was sent, replies were still sent, they are now suppressed
This release can communicate with machines running older versions of mcollective there are though a few steps to take to ensure a smooth upgrade.
{% highlight ini %} topicprefix = /topic/mcollective {% endhighlight %}
This has to change to:
{% highlight ini %} topicprefix = /topic/ main_collective = mcollective collectives = mcollective {% endhighlight %}
The interface for the encodereply method on the security plugins have changed if you are using any of the community plugins or wrote your own you should update them with the latest at the time you upgrade to 1.2.0
The interface to the fact plugins have been greatly simplified, this means you need to update to new plugins at the time you upgrade to 1.2.0
You can place these new plugins into the plugindir before upgrading. The old mcollective will not use these plugins and the new one will not touch the old ones. This will allow for a clean rollback.
Once the new version is deployed you will immediately have caching on all fact types at 300 seconds you can tune this using the fact_cache_time setting in the configuration file.
The new fact selectors are only available on newer versions of mcollective. If a client attempts to use them and an older version of the server is on the network those older servers will treat all fact lookups as ==
| Date | Description | Ticket |
|---|---|---|
| 2011/05/03 | Improve Red Hat RC script by using distro builtin functions | 7340 |
| 2011/05/01 | Support setting a priority on Stomp messages | 7246 |
| 2011/04/30 | Handle broken and incomplete DDLs better and improve the format of DDL output | 7191 |
| 2011/04/23 | Encode the target agent and collective in requests | 7223 |
| 2011/04/20 | Make the SSL Cipher used a config option | 7191 |
| 2011/04/20 | Add a clear method to the PluginManager that deletes all plugins, improve test isolation | 7176 |
| 2011/04/19 | Abstract the creation of request and reply hashes to simplify connector plugin development | 5701 |
| 2011/04/15 | Improve the shellsafe validator and add a Util method to do shell escaping | 7066 |
| 2011/04/14 | Update Rakefile to have a mail_patches task | 6874 |
| 2011/04/13 | Update vendored systemu library for Ruby 1.9.2 compatibility | 7067 |
| 2011/04/12 | Fix failing tests on Ruby 1.9.2 | 7067 |
| 2011/04/11 | Update the DDL documentation to reflect the mco help command | 7042 |
| 2011/04/11 | Document the use filters on the CLI | 5917 |
| 2011/04/11 | Improve handling of unknown facts in Util#has_fact? to avoid exceptions about nil#clone | 6956 |
| 2011/04/11 | Correctly set timeout on the discovery agent to 5 seconds as default | 7045 |
| 2011/04/11 | Let rpcutil#agent_inventory supply unknown for missing values in agent meta data | 7044 |
This is a release in the development series of mcollective. It features major new features and some bug fixes.
This release is for early adopters, production users should consider the 1.0.x series.
We have implemented the ability to write actions in languages other than Ruby. This is done via simple JSON API documented in in our docs
The ext directory on GitHub hosts wrappers for PHP, Perl and Python that makes using this interface easier.
{% highlight ruby %} action "test" do implemented_by "/some/external/script" end {% endhighlight %}
Special thanks to the community members who contributed the wrappers.
- Actions can now be written in any language
- Plugin configuration can be kept in /etc/mcollective/plugin.d
- mco inventory now shows collective and sub-collective membership
- mc-controller has been deprecated for mco controller
- Agents are now ran using new instances of the classes rather than reuse the exiting one to avoid concurrency related problems
- When mcollectived exits it now cleanly disconnects from the Middleware
- The rpcutil agent is less strict about valid Fact names
- The default configuration files have been updated for sub-collectives
This release will be backward compatible with version 1.1.3 for compatibility with earlier releases see the notes for 1.1.3 and the sub collective related configuration changes.
| Date | Description | Ticket |
|---|---|---|
| 2011/03/28 | Correct loading of vendored JSON gem | 6877 |
| 2011/03/28 | Show collective and sub collective info in the inventory application | 6872 |
| 2011/03/23 | Disconnect from the middleware when mcollectived disconnects | 6821 |
| 2011/03/21 | Update rpcutil ddl file to be less strict about valid fact names | 6764 |
| 2011/03/22 | Support reading configuration from configfir/plugin.d for plugins | 6623 |
| 2011/03/21 | Update default configuration files for sub-collectives | 6741 |
| 2011/03/16 | Add the ability to implement actions using external scripts | 6705 |
| 2011/03/15 | Port mc-controller to the Application framework and deprecate the exit command | 6637 |
| 2011/03/13 | Only cache registration and discovery agents, handle the rest as new instances | 6692 |
| 2011/03/08 | PluginManager can now create new instances on demand for a plugin type | 6622 |
This is a release in the development series of mcollective. It features major new features and some bug fixes.
This release is for early adopters, production users should consider the 1.0.x series.
- Add the ability to partition collectives into sub-collectives for security and network traffic management
- Add a exec wrapper for agents that provides unique environments and cwds in a thread safe manner as well as avoid zombie processes
- Automatically pass Application options to rpcclient when options are not specifically provided
- Rename /usr/sbin/mc to /usr/bin/mco
- Missing libdirs will not cause crashes anymore
- Parse MCOLLECTIVE
_EXTRA_OPTS correctly with multiple options - file
_logger failures are handled better - Improve middleware communication in unreliable settings by adding timeouts around middleware operations
The configuration format has changed slightly to accomodate the concept of collective names and sub-collectives.
In older releases the configuration was:
{% highlight ini %} topicprefix = /topic/mcollective {% endhighlight %}
This has to change to:
{% highlight ini %} topicprefix = /topic/ main_collective = mcollective collectives = mcollective {% endhighlight %}
When setup as above a old and new version will be compatible but as soon as you start configuring the new sub-collective feature you will loose compatiblity between versions.
Various defaults apply, if you configure it with these exactly topic and
collective names you can leave off the main_collective and collectives
directives as the above settings would be their defaults
| Date | Description | Ticket |
|---|---|---|
| 2011/03/04 | Rename /usr/sbin/mc to /usr/bin/mco | 6578 |
| 2011/03/01 | Wrap rpcclient in applications ensuring that options is always set | 6308 |
| 2011/02/28 | Make communicating with the middleware more robust by including send calls in timeouts | 6505 |
| 2011/02/28 | Create a wrapper to safely run shell commands avoiding zombies | 6392 |
| 2011/02/19 | Introduce Sub-collectives for network partitioning | 5967 |
| 2011/02/19 | Improve error handling when parsing arguments in the rpc application | 6388 |
| 2011/02/19 | Fix error logging when file_logger creation fails | 6387 |
| 2011/02/17 | Correctly parse MCOLLECTIVE_EXTRA_OPTS in the new unified binary framework | 6354 |
| 2011/02/15 | Allow the signing key and Debian distribution to be customized | 6321 |
| 2011/02/14 | Remove inadvertently included package.ddl | 6313 |
| 2011/02/14 | Handle missing libdirs without crashing | 6306 |
This is a minor bug fix release.
- The YAML fact plugin failed to remove deleted facts from memory
- The - character is now allowed in Fact names for the rpcutil agent
- Machines that fali security validations were not reported correctly, they are now treated as having not responded at all
- Timeouts on RPC requests were too aggressive and did not allow for slow networks
This release will be backward compatible with older releases.
| Date | Description | Ticket |
|---|---|---|
| 2011/02/02 | Include full Apache 2 license text | 6113 |
| 2011/01/29 | The YAML fact plugin kept deleted facts in memory | 6056 |
| 2012/01/04 | Use the LSB based init script on SUSE | 5762 |
| 2010/12/30 | Allow - in fact names | 5727 |
| 2010/12/29 | Treat machines that fail security validation like ones that did not respond | 5700 |
| 2010/12/25 | Allow for network and fact source latency when calculating client timeout | 5676 |
| 2010/12/25 | Increase the rpcutil timeout to allow for slow facts | 5679 |
This is a release in the development series of mcollective. It features minor bug fixes and features.
This release is for early adopters, production users should consider the 1.0.x series.
- The main fix in this release is a packaging bug in Debian systems that prevented both client and server from being installed on the same machine.
- Backwards compatibility fix for fact filters that are empty strings
- Registration plugins can now return nil which will skip that specific registration message. This will enable plugins to decide based on some node state if a message should be sent or not.
| Date | Description | Ticket |
|---|---|---|
| 2011/02/13 | Surpress replies to SimpleRPC clients who did not request results | 6305 |
| 2011/02/11 | Fix Debian packaging error due to the same file in multiple packages | 6276 |
| 2011/02/11 | The application framework will now disconnect from the middleware for consistency | 6292 |
| 2011/02/11 | Returning nil from a registration plugin will skip registration | 6289 |
| 2011/02/11 | Set loglevel to warn by default if not specified in the config file | 6287 |
| 2011/02/10 | Fix backward compatibility with empty fact strings | 6278 |
This is a release in the development series of mcollective. It features major new features and numerous bug fixes. Please pay careful attention to the upgrading section as there is some changes that are not backward compatible.
This release is for early adopters, production users should consider the 1.0.x series.
A new security plugin that encrypts the payloads, uniquely identify senders and secure replies from inspection by other people on the collective has been written. The plugin can re-use Puppet certificates and supports distributing of public keys if you wish.
This plugin and its deployment is very complex and it has a visible performance impact but we felt it was a often requested feature and so decided to implement it.
Full documentation for this plugin can be found in our docs, please read them very carefully should you choose to deploy this plugin.
In the past a lot of the CLI tools have behaved inconsistently as the mc scripts were mostly just written to serve immediate needs, we are starting a process of improving these scripts and making them more robust.
The first step is to create a new framework for CLI commands, we call these Single Executable Applications. A new executable called mc is being distributed with this release:
{% highlight console %} $ mc The Marionette Collective version 1.1.1
/usr/sbin/mc: command (options)
Known commands: rpc filemgr inventory facts ping find help {% endhighlight %}
{% highlight console %} $ mc help The Marionette Collection version 1.1.1
facts Reports on usage for a specific fact filemgr Generic File Manager Client find Find hosts matching criteria help Application list and RPC agent help inventory Shows an inventory for a given node ping Ping all nodes rpc Generic RPC agent client application {% endhighlight %}
{% highlight console %} $ mc rpc package status package=zsh Determining the amount of hosts matching filter for 2 seconds .... 51
- [ ============================================================> ] 51 / 51
test.com: Properties: {:provider=>:yum, :release=>"3.el5", :arch=>"x86_64", :version=>"4.2.6", :epoch=>"0", :name=>"zsh", :ensure=>"4.2.6-3.el5"} {% endhighlight %}
You can see these commands behave just like their older counter parts but is more integrated and discovering available commands is much easier.
Agent help that was in the past available through mc-rpc --ah agentname is now available through mc help agentname and error reporting is short single line reports by default but by adding -v to the command line you can get full Ruby backtraces.
We've maintained backward compatibility by creating wrappers for all the old mc scripts but these might be deprecated in future.
These application live in the normal plugin directories and should make it much easier to distribute plugins in future.
We will port the scripts for plugins to this framework and encourage you to do the same when writing new CLI tools. An example of a ported CLI can be seen in the filemgr agent.
Find the documentation for these plugins here.
The logging system has been ra-efactored to not use a Signleton, logging messages are now simply:
{% highlight ruby %} MCollective::Log.notice("hello world") {% endhighlight %}
A backwards compatible wrapper exist to prevent existing code from breaking.
In some cases - like when using MCollective from within Rails - the STOMP gem would fail to decode the payloads. We've worked with the authors and a new release was made that makes this more robust but we've also enabled Base64 encoding on the Stomp connector for those who can't upgrade the Gem and who are running into this problem.
- Machines that do not pass security checks are handled as having not responded so that these are listed in the usual stat for non responsive hosts
- The - character is now allowed in Fact names by the DDL for rpcutil
- Version 1.1.0 introduced a bug with reloading agents from disks using USR1 and mc-controller
- New AES+RSA based security plugin was added
- Create a new single executable framework and port several mc scripts
- Security plugins have access to the callerid they are responding to
- The logging methods have been improved by removing the use of Singletons
- The STOMP connector can now Base64 encode all sent data to deal with en/decoding issues by the gem
- The rpcutil agent has a new ping action
- the mc ping client now supports standard filters
- DDL documentation has been updated to show you can disable type validations in the DDL
- Fact plugins can now force fact cache invalidation, the YAML plugin will immediately load new facts when mtime on the file change
- Improve 1.0.0 compatibility for foo=/bar/ style fact matches at the expense of 1.1.0 compatibility
Upgrading should be mostly painless as most things are backward compatible.
We discovered that we broke backward compatibility with 1.0.0 and 0.4.x Fact filters. A filter in the form foo=/bar/ would be treated as an equality filter and not a regular expression.
This releases fixes this compatibility with older versions at the expense of compatibility with 1.1.0. If you are upgrading from 1.1.0 keep this in mind and plan accordingly, once you've upgraded a client its requests that contain these filters will not be correctly parsed on servers running 1.1.0.
The security plugins have changed slightly, if you wrote your own security plugin the interface to encodereply has changed slightly. All the bundled security plugins have been updated already and older ones will just keep working.
| Date | Description | Ticket |
|---|---|---|
| 2011/02/02 | Load the DDL from disk once per printrpc call and not for every result | 5958 |
| 2011/02/02 | Include full Apache 2 license text | 6113 |
| 2011/01/31 | Create a new single executable application framework | 5897 |
| 2011/01/30 | Fix backward compatibility with old foo=/bar/ style fact searches | 5985 |
| 2011/01/30 | Documentation update to reflect correct default identity behavior | 6073 |
| 2011/01/29 | Let the YAML file force fact reloads when the files update | 6057 |
| 2011/01/29 | Add the ability for fact plugins to force fact invalidation | 6057 |
| 2011/01/29 | Document an approach to disable type validation in the DDL | 6066 |
| 2011/01/19 | Add basic filters to the mc-ping command | 5933 |
| 2011/01/19 | Add a ping action to the rpcutil agent | 5937 |
| 2011/01/17 | Allow MC::RPC#printrpc to print single results | 5918 |
| 2011/01/16 | Provide SimpleRPC style results when accessing the MC::Client results directly | 5912 |
| 2011/01/11 | Add an option to Base64 encode the STOMP payload | 5815 |
| 2011/01/11 | Fix a bug with forcing all facts to be strings | 5832 |
| 2011/01/08 | When using reload_agents or USR1 signal no agents would be reloaded | 5808 |
| 2011/01/04 | Use the LSB based init script on SUSE | 5762 |
| 2011/01/04 | Remove the use of a Singleton in the logging class | 5749 |
| 2011/01/02 | Add AES+RSA security plugin | 5696 |
| 2010/12/31 | Security plugins now have access to the callerid of the message they are replying to | 5745 |
| 2010/12/30 | Allow - in fact names | 5727 |
| 2010/12/29 | Treat machines that fail security validation like ones that did not respond | 5700 |
This is the first in a new development series, as such there will be rapid changes and new features. We cannot guarantee the changes will be backward compatible but we will as before try to keep these releases solid and production quality.
Production users who do not wish to have rapid change should use release 1.0.0.
This release focus mainly on getting all the community contributed code into a release and addressing some issues I had but wasn't comfortable fixing them late in the previous development series.
Please read these notes carefully we are removing some old functionality and changing some internals, you need to carefully review the text below.
- The progress bar will now try hard to detect screen size and adjust itself, failing back to a dumb mode if it can't work it out.
- rpcutil timeout was too short when considering slow facts and network latency
- libdir can now be multiple directories specified with : separation - Thanks to Richard Clamp
- Logging is now pluggable, 3 logger types are supported - file, syslog and console. Thanks to Nicolas Szalay for the initial Syslog code
- A new experimental ssh agent based security system. Thanks to Jordan Sissel
- New fact matching operators <=, >=, <, >, !=, == and =~. Thanks to Mike Pountney
- SimpleRPC fact_filter method can now take any valid fact string as input in addition to the old format
- A message gets logged at startup showing mcollective version and logging level
- The fact plugin format has been changed, simplified, made thread safe and global caching added. This breaks backward compatibility with old fact sources
- Creating options hashes has been simplified by adding a helper that creates them for you
- Calculating the client timeout has been improved by including for latency and fact source slowness
- Audit log lines are now on one line and include a ISO 8601 format date
- The old MCOLLECTIVE_TIMEOUT and MCOLLECTIVE_DTIMEOUT were removed, a new MCOLLECTIVE_EXTRA_OPTS was added which should allow much more flexibility. Supply any command line options in this var
Upgrading should be easy the only backward incompatible change is the Facts format. If you only use the included YAML plugin the upgrade will just work if you use the packages. If you use either the facter or ohai plugins you will need to download new plugins from the community plugin page.
If you wrote your own Facts plugin you will need to change it a bit:
- The old get_facts method should now be load_facts_from_source
- The class for facts have to be in the form MCollective::Facts::Foo_facts and the filename should match
This is all, your facts can now be much simpler as threading and caching is handled in the base class.
You can place these new plugins into the plugindir before upgrading. The old mcollective will not use these plugins and the new one will not touch the old ones. This will allow for a clean rollback.
Once the new version is deployed you will immediately have caching on all fact types at 3000 seconds you can tune this using the fact_cache_time setting in the configuration file.
| Date | Description | Ticket |
|---|---|---|
| 2010/12/28 | Adjust the logfile audit format to include local time and all on one line | 5694 |
| 2010/12/26 | Improve the SimpleRPC fact_filter helper to support new fact operators | 5678 |
| 2010/12/25 | Increase the rpcutil timeout to allow for slow facts | 5679 |
| 2010/12/25 | Allow for network and fact source latency when calculating client timeout | 5676 |
| 2010/12/25 | Remove MCOLLECTIVE_TIMEOUT and MCOLLECTIVE_DTIMEOUT environment vars in favor of MCOLLECTIVE_EXTRA_OPTS | 5675 |
| 2010/12/25 | Refactor the creation of the options hash so other tools don't need to know the internal formats | 5672 |
| 2010/12/21 | The fact plugin format has been changed and simplified, the base now provides caching and thread safety | 5083 |
| 2010/12/20 | Add parameters <=, >=, <, >, !=, == and =~ to fact selection | 5084 |
| 2010/12/14 | Add experimental sshkey security plugin | 5085 |
| 2010/12/13 | Log a startup message showing version and log level | 5538 |
| 2010/12/13 | Add a console logger | 5537 |
| 2010/12/13 | Logging is now pluggable and a syslog plugin was provided | 5082 |
| 2010/12/13 | Allow libdir to be an array of directories for agents and ddl files | 5253 |
| 2010/12/13 | The progress bar will now intelligently figure out the terminal dimensions | 5524 |
This is a bug fix and minor improvement release.
We will maintain the 1.0.x branch as a stable supported branch. The features currently in the branch will be frozen and we'll only do bug fixes.
A new 1.1.x series of releases will be done where we will introduce new features. Once the 1.1.x code base reaches a mature point it will become the new stable release and so forth.
- Settings like retry times were ignored in the Stomp connector
- The default init script had incorrect LSB comments
- The rpcutil DDL has better validation and will now match all facts
- You can now send RPC requests to a subset of discovered nodes
- SimpleRPC custom_request can now be used to create fire and forget requests
- Clients can now cleanly disconnect from the middleware. Bundled clients have been updated. This should cause fewer exceptions in ActiveMQ logs
- Rather than big exceptions many clients will now log errors only
- mc-facts has been reworked to be a SimpleRPC client, this speeds it up significantly
- Add get_config_item to rpcutil to retrieve a running config value from a server
- YAML facts are now forced to be all strings and is thread safe
- On RedHat based systems the requirement for the LSB packages has been removed
The first feature is a major new feature in SimpleRPC. If you expose a service redundantly on your network using MCollective you wouldn't always want to send requests to all the nodes providing the service. You can now limit the requests to an arbitrary amount using the new --limit-nodes option which will also take a percentage. A shortcut -1 has been added that is the equivalent to --limit-nodes 1
This release will be backward compatible with older releases.
| Date | Description | Ticket |
|---|---|---|
| 2010/12/04 | Remove the LSB requirements for RedHat systems | 5451 |
| 2010/11/23 | Make the YAML fact source thread safe and force all facts to strings | 5377 |
| 2010/11/23 | Add get_config_item to rpcutil to retrieve a running config value from a server | 5376 |
| 2010/11/20 | Convert mc-facts into a SimpleRPC client | 5371 |
| 2010/11/18 | Added GPG signing to Rake packaging tasks (SIGNED=1) | 5355 |
| 2010/11/17 | Improve error messages from clients in the case of failure | 5329 |
| 2010/11/17 | Add helpers to disconnect from the middleware and update all bundled clients | 5328 |
| 2010/11/16 | Correct LSB provides and requires in default init script | 5222 |
| 2010/11/16 | Input validation on rpcutil has been improved to match all valid facts | 5320 |
| 2010/11/16 | Add the ability to limit the results to a subset of hosts | 5306 |
| 2010/11/15 | Add fire and forget mode to SimpleRPC custom_request | 5305 |
| 2010/11/09 | General connection settings to the Stomp connector was ignored | 5245 |
This is a bug fix and minor improvement release.
- Multiple RPC proxy classes in the same script would not all share the same command line options
- Ruby 1.9.x compatibility has been improved
- A major bug in registration has been fixed, any exception in the registration logic would have resulted in a high CPU consuming loop
The last bug is a major issue it will result in the mcollectived consuming lots of CPU, updating to this version of MCollective is strongly suggested. Should you run into this problem on a large scale you can use mc-controller exit to exit all your mcollectived processes at the same time.
- The PSK security plugin can now be configured to set the callerid to a few different values useful for cases where you want to do group based RPC Authorization for example.
- Info logging has been minimised by demoting the 'not targeted at us' message to debug
- Document the 'exit' option to mc-controller
This release will be backward compatible with older releases.
| Date | Description | Ticket |
|---|---|---|
| 2010/10/18 | Document exit command to mc-controller | 152 |
| 2010/10/13 | Log messages that don't pass the filters at debug level | 149 |
| 2010/10/03 | Preserve options in cases where RPC::Client instances exist in the same program | 148 |
| 2010/09/30 | Add the ability to set different types of callerid in the PSK plugin | 145 |
| 2010/09/30 | Improve Ruby 1.9.x compatibility | 142 |
| 2010/09/29 | Improve error handling in registration to avoid high CPU usage loops | 143 |
This is a bug fix and minor improvement release.
- Internal data structure related to Agent meta data has been fixed, no user impact from this
- When using per-user config files the rpc-help.erb template could not be found
- The log files will now rotate by default keeping 5 x 2MB files
- The config were parsed multiple times in complex scripts, this has been eliminated
- MCollective::RPC loaded a bunch of unneeded stuff into Object, this has been cleaned up
- Various packaging related tweaks were done
- We ship a new agent called rpcutil with the base system, you can use this agent to get inventory etc from your mcollectived. mc-inventory has been rewritten to use this agent and should serve as a good reference for what you can get from the agent.
- The DDL now support :boolean style inputs, mc-rpc also turn true/false on the command line into booleans when needed
This release will be backward compatible with older releases.
| Date | Description | Ticket |
|---|---|---|
| 2010/09/20 | Improve Debian packaging task | 140 |
| 2010/09/20 | Add :boolean type support to the DDL | 138 |
| 2010/09/19 | Refactor MCollective::RPC to add less unneeded stuff to Object | 137 |
| 2010/09/18 | Prevent duplicate config loading with multiple clients active | 136 |
| 2010/09/18 | Rotate the log file by default, keeping 5 x 2MB files | 135 |
| 2010/09/18 | Write a overview document detailing security of the collective | 131 |
| 2010/09/18 | Add MCollective.version, set it during packaging and include it in the rpcutil agent | 134 |
| 2010/09/13 | mc-inventory now use SimpleRPC and the rpcutil agent and display server stats | 133 |
| 2010/09/13 | Make the path to the rpc-help.erb configurable and set sane default | 130 |
| 2010/09/13 | Make the configfile used available in the Config class and add to rpcutil | 132 |
| 2010/09/12 | Rework internal statistics and add a rpcutil agent | 129 |
| 2010/09/12 | Fix internal memory structures related to agent meta data | 128 |
| 2010/08/24 | Update the OpenBSD port for changes in 0.4.8 tarball | 125 |
| 2010/08/23 | Fix indention/block error in M:R:Stats | 124 |
| 2010/08/23 | Fix permissions in the RPM for files in /etc | 123 |
| 2010/08/23 | Fix language in two error messages | 122 |
This is a bug fix and minor improvement release.
- The RPM packages now require redhat-lsb since our RC scripts need it
- The rake tasks do not attempt to build rpms on all platforms
- Some plugin missing related exceptions are now handled gracefully
- The Rakefile had a few warnings cleaned up
-
Users can now have a ~/.mcollective file which will be preferred over over /etc/mcollective/client.cfg if it exists. You can still use --config to override.
-
The SSL Security plugin can now use "either YAML or Marshal for serialization":/reference/plugins/security_ssl.html#serialization_method, this means other programming languages can be used as clients. A sample Perl client is included in the ext directory. Marshal remains the default for backwards compatibility
-
mc-inventory can now be used to create "custom reports using a small reporting DSL":/reference/ui/nodereports.html, this enable you to build custom reports listing all your machines and gives you access to facts, agents and classes lists.
-
The log level for the mcollectived can be adjusted during run time using the USR2 unix process signal.
This release will be backward compatible with older releases. If you choose to use YAML in the SSL plugin you need matching versions on the client.
| Date | Description | Ticket |
|---|---|---|
| 2010/08/19 | Fix missing help template in debian packages | 90 |
| 2010/08/18 | Clean up some hardlink warnings in the Rakefile | 117 |
| 2010/08/18 | Include the website in the main repo and add a simple Rake task | 118 |
| 2010/08/17 | Handle exceptions for missing plugins better | 115 |
| 2010/08/17 | Add support for ~/.mcollective as a config file | 114 |
| 2010/08/07 | SSL security plugin can use either YAML or Marshal | 94 |
| 2010/08/06 | Multiple YAML files can now be used as fact source | 112 |
| 2010/08/06 | Allow log level to be adjusted at run time with USR2 | 113 |
| 2010/07/31 | Add basic report scripting support to mc-inventory | 111 |
| 2010/07/06 | Removed 'rpm' from the default rake task | 109 |
| 2010/07/06 | Add redhat-lsb to the server RPM dependencies | 108 |
This is a bug fix and incremental improvement release focusing on small improvements in the DDL mostly.
We've extended the use of the DDL in the RPC client. We've integrated the DDL into printrpc helper. The output is dynamic showing field names in human readable format rather than hash dumps.
We're also using color to improve the display of the results, the color display can be disabled with the new color configuration option.
A "screencast of the DDL integration":http://mcollective.blip.tv/file/3799653 and color usage has been recorded.
A serious issue has been fixed with regard to complex agents. If you attempted to use multiple agents from the same script errors such as duplicate discovery results or simply not working.
The default fact source has been changed to YAML, it was inadvertently set to Facter in the past.
Some previously unhandled exceptions are now being handled correctly and passed onto the clients as failed requests rather than no responses at all.
This release will be backward compatible with older releases. The change to YAML fact source by default might impact you if you did not previously specify a fact source in the configuration files.
| Date | Description | Ticket |
|---|---|---|
| 2010/06/27 | Change default factsource to YAML | 106 |
| 2010/06/27 | Added VIM snippets to create DDLs and Agents | 102 |
| 2010/06/26 | DDL based help now works better with Symbols in in/output | 105 |
| 2010/06/23 | Whitespace at the end of config lines are now stripped | 100 |
| 2010/06/22 | printrpc will now inject some colors into results | 99 |
| 2010/06/22 | Recover from syntax and other errors in agents | 98 |
| 2010/06/17 | The agent a MC::RPC::Client is working on is now available | 97 |
| 2010/06/17 | Integrate the DDL with data display helpers like printrpc | 92 |
| 2010/06/15 | Avoid duplicate topic subscribes in complex clients | 95 |
| 2010/06/15 | Catch some unhandled exceptions in RPC Agents | 96 |
| 2010/06/15 | Fix missing help template file from RPM | 90 |
This release is a major feature release.
We're focusing mainly on the Stomp connector and on the SimpleRPC agents in this release though a few smaller additions were made.
We've historically been stuck just using RubyGem Stomp 1.1 due to multi threading bugs in the newer releases. All attempts to contact the authors failed. Recently though I had some luck and these issues are resolved in the RubyGem Stomp 1.1.6 release.
This means we can take advantage of a lot of new features such as connection pooling for failover/ha and also SSL TLS between nodes and ActiveMQ server.
See "Stomp Connector":/reference/plugins/connector_stomp.html for details.
I've been working since around February on building introspection, automatically generated documentation and the ability for user interfaces to be auto generated for agents, even ones you write your self.
This feature is documented in "DDL":/simplerpc/ddl.html but a quick example of a DDL document might help make it clear:
- mc-facts now take all the standard filters so you can make reports for just subsets of machines
- A new utility mc-inventory has been added, it will show agents, facts and classes for a node
- mc-rpc has a new option --agent-help that will use the DDL and display auto generated documentation for an agent.
- mc-facts output is sorted for better readability
This release will be backward compatible with older releases, the new Stomp features though require the newer Ruby Gem.
| Date | Description | Ticket |
|---|---|---|
| 2010/06/12 | Qualify the Process class to avoid clashes in the discovery agent | 88 |
| 2010/06/12 | Add mc-inventory which shows agents, classes and facts for a node | 87 |
| 2010/06/11 | mc-facts now supports standard filters | 86 |
| 2010/06/11 | Add connection pool retry options and SSL for connection | 85 |
| 2010/06/11 | Add support for specifying multiple stomp hosts for failover | 84 |
| 2010/06/10 | Tighten up handling of filters to avoid nil's getting into them | 83 |
| 2010/06/09 | Sort the mc-facts output to be more readable | 82 |
| 2010/06/08 | Fix deprecation warnings in newer Stomp gems | 81 |
This release is a major feature release.
The focus of this release is to finish up some of the more enterprise like features, we now have fine grained Authorization and Authentication and a new security model that uses SSL keys.
Vladimir Vuksan contributed the base code of a new "SSL based security plugin":/reference/plugins/security_ssl.html . This plugin builds on the old PSK plugin but gives each client a unique certificate pair. The nodes all share a certificate and only store client public keys. This means that should one node be compromised it cannot be used to control the rest of the network.
We've developed new hooks and plugins for SimpleRPC that enable you to write "fine grained authorization systems":/simplerpc/authorization.html . You can do authorization on every aspect of the request and you'll have access to the caller userid as provided by the security plugin. Combined with the above SSL plugin this can be used to build powerful and secure Authentication and Authorization systems.
A sample plugin can be found "here":http://code.google.com/p/mcollective-plugins/wiki/ActionPolicy
Web apps doesn't tie in well with our discover, request, wait model. We've made it easier for web apps to maintain their own cached discovery data using the "Registration:/reference/plugins/registration.html system and then based on that do requests that would not require any discovery.
It is often desirable to just send a request and not wait for any reply. We've made it easy to do requests like this] with the addition of a new request parameter on the SimpleRPC client class.
Requests like this will not take any time to do discovery and you will not be able to get results back from the agents.
To make it a bit easier to manage daemons and agents you can now send the mcollectived a USR1 signal and it will re-read all it's agents from disk.
This release when used with the old style PSK plugin should be perfectly backward compatible with your existing agents. To use some of the newer features like authorization will require config and/or agent changes.
| Date | Description | Ticket |
|---|---|---|
| 2010/06/01 | Improve the main discovery agent by adding facts and classes to its inventory action | 79 |
| 2010/05/30 | Add various helpers to get reports as text instead of printing them | 43 |
| 2010/05/30 | Add a custom_request method to call SimpleRPC agents with your own discovery | 75 |
| 2010/05/30 | Refactor RPC::Client to be more generic and easier to maintain | 75 |
| 2010/05/29 | Fix a small scoping issue in Security::Base | 76 |
| 2010/05/25 | Add option --no-progress to disable progress bar for SimpleRPC | 74 |
| 2010/05/23 | Add some missing dependencies to the RPMs | 72 |
| 2010/05/22 | Add an option :process_results to the client | 71 |
| 2010/05/13 | Fix help output that still shows old branding | 70 |
| 2010/04/27 | The supplied generic stompclient now accepts STOMP_PORT in the environment | 68 |
| 2010/04/26 | Add a SimpleRPC Client helper to reset filters | 64 |
| 2010/04/26 | Listen for signal USR1 and reload all agents from disk | 65 |
| 2010/04/12 | Add SimpleRPC Authorization support | 63 |
This release is mostly a bug fix release.
We've cleaned up the logs a bit so you'll see fewer exceptions logged, also if you have Rails + Puppet on a node the logs will stay in the standard format. We are handling some exceptions better which has improved stability of the registration system.
If you do some slow queries in your discovery agent you can bump the timeout up in the config using plugin.discovery.timeout.
All the scripts now use /usr/bin/env ruby rather than hardcoded paths to deal better with Ruby's in weird places.
Several other small annoyances was fixed or improved.
We've always had a tool that let you control a network of mcollective instances remotely, it lagged behind a bit with the core, we've fixed it up and documented it "here":/reference/basic/daemon.html . You can use it to reload agents from disk without restarting the daemon for example or get stats or shut down the entire mcollective network.
No changes that impacts backward compatibility has been made.
| Date | Description | Ticket |
|---|---|---|
| 2010/03/27 | Make it easier to construct SimpleRPC requests to use with the standard client library | 60 |
| 2010/03/27 | Manipulating the filters via the helper methods will force rediscovery | 59 |
| 2010/03/23 | Prevent Activesupport when brought in by Facter from breaking our logs | 57 |
| 2010/03/23 | Clean up logging for messages not targeted at us | 56 |
| 2010/03/19 | Add exception handling to the registration base class | 55 |
| 2010/03/03 | Use /usr/bin/env ruby instead of hardcoded paths | 54 |
| 2010/02/17 | Improve mc-controller and document it | 46 |
| 2010/02/08 | Remove some close coupling with Stomp to easy creating of other connectors | 49 |
| 2010/02/01 | Made the discovery agent timeout configurable using plugin.discovery.timeout | 48 |
| 2010/01/25 | mc-controller now correctly loads/reloads agents. | 45 |
| 2010/01/25 | Building packages has been improved to ensure rdocs are always included | 44 |
This release fixes a few bugs and introduce a major new SimpleRPC feature for auditing requests.
We've created an "auditing framework for SimpleRPC":/simplerpc/auditing.html, each request gets passed to an audit plugin for processing. We ship one that simply logs to a file on each node and there's a "community plugin":http://code.google.com/p/mcollective-plugins/wiki/AuditCentralRPCLog that logs everything on a central logging host.
In future we might add auditing to the client libraries so requests will be logged where they are sent as well as auditing of replies being sent, this will be driven by requests from the community though.
Till now while writing agents you can use the fail method to set statuses in the reply, this however did not also raise exceptions and terminate execution of the agent immediately.
Often the existing behavior is required but it did lead to some awkward code when you did want to just exit the agent immediately as well as set a fail status. We've added a fail! method that works just like fail except it stops execution of your agent immediately.
No changes that impacts backward compatibility has been made.
| Date | Description | Ticket |
|---|---|---|
| 2010/01/23 | Handle ctrl-c during discovery without showing exceptions to users | 34 |
| 2010/01/21 | Force all facts in the YAML fact source to be strings | 41 |
| 2010/01/19 | Add SimpleRPCAuditing audit logging to SimpleRPC clients and Agents | |
| 2010/01/18 | The SRPM we provide will now build outside of the Rake environment | 40 |
| 2010/01/18 | Add a fail! method to RPC::Agent | 37 |
| 2010/01/18 | mc-rpc can now be used without supplying arguments | 38 |
| 2010/01/18 | Don't raise an error if no user/pass is given to the stomp connector, try unauthenticated mode | 35 |
| 2010/01/17 | Improve error message when Regex validation failed on SimpleRPC input | 36 |
This release fixes a few bugs, add some command line improvements and brings major changes to the Debian packaging.
Firstly we've had some amazing work done by Riccardo Setti to make us Debian packages that complies with Debian and Ubuntu policy, this release use these new packages. It has some unfortunate changes to file layout detailed below but overall I think it's a big win to get us in line with Distribution policies and standards.
The only major change is that in the past we used /usr/libexec/mcollective as the libdir, but Debian does not have this directory and it is not in the LFHS anymore so we now use /usr/share/mcollective/plugins as the lib dir. You need to move your plugins there and update both client and server configs.
The RedHat packages will move to this convention too in the next release since I think it's the better location and complies with LFHS.
We've streamlined the command line a bit, nothings changed we've just added some flags.
The --with-class, --with-fact, --with-agent and --with-identity now all have a short form -C, -F, -A and -I respectively.
We've added a new filter option --with and a short form -W that combines --with-class and --with-fact into one filter type, use case would be:
{% highlight console %} % mc-find-hosts -W "/centreon/ country=de roles::dev_server/" {% endhighlight %}
This would find hosts with class regex matched /centreon/, class roles::dev_server and fact matching country=de. Hopefully this saves on some typing.
You can also now set the environment variables MCOLLECTIVE_TIMEOUT and MCOLLECTIVE_DTIMEOUT which saves you from typing --timeout and --discovery-timeout often, especially useful on very fast networks.
- We've added the COPYING file to all the packages
- We've made the init script more LSB compliant
- A bug related to discovery in SimpleRPC was fixed
The only backwards issue is the Debian packages. They've been tested to upgrade cleanly but you need to change the config as above.
| Date | Description | Ticket |
|---|---|---|
| 2010/01/13 | New packaging for Debian provided by Riccardo Setti | 29 |
| 2010/01/07 | Improved LSB compliance of the init script - thanks Riccardo Setti | 32 |
| 2010/01/07 | Multiple calls to SimpleRPC client would reset discovered hosts | 31 |
| 2010/01/04 | Timeouts can now be changed with MCOLLECTIVE_TIMEOUT and MCOLLECTIVE_DTIMEOUT environment vars | 25 |
| 2010/01/04 | Specify class and fact filters easier with the new -W or --with option | 27 |
| 2010/01/04 | Added COPYING file to RPMs and tarball | 28 |
| 2010/01/04 | Make shorter filter options -C, -I, -A and -F | 26 |
This is a bug fix release to address some shortcomings and issues found in Simple RPC.
The main issue is around handling of meta data in agents, the documented approach did not work, we've now solved this by adding a number of hooks into the processing of Simple RPC agents.
We've also made logging and config retrieval a bit easier in agents - and documented this.
You can now call the mc-rpc command a bit easier:
{% highlight console %} % mc-rpc --agent helloworld --action echo --argument msg="hello world" % mc-rpc helloworld echo msg="hello world" {% endhighlight %}
The 2 calls are the same, you can pass as many arguments in key=val pairs as needed at the end.
No issues with backward compatibility, should be a simple upgrade.
| Date | Description | Ticket |
|---|---|---|
| 2010/01/02 | Added hooks to plug into the processing of requests, also enabled setting meta data and timeouts | 14 |
| 2010/01/02 | Created readers for @config and @logger in the SimpleRPC agent | 23 |
| 2009/12/30 | Don't send out any requests if no nodes were discovered | 17 |
| 2009/12/30 | Added :discovered and :discovered_nodes to client stats | 20 |
| 2009/12/30 | Add a empty_filter? helper to the RPC mixin | 18 |
| 2009/12/30 | Fix formatting bug with progress bar | 21 |
| 2009/12/29 | Simplify mc-rpc command line | 16 |
| 2009/12/29 | Fix layout issue when printing hosts that did not respond | 15 |
This release introduced a major new feature - Simple RPC - a framework for easily building clients and agents. More than that it's a set of conventions and standards that will help us build generic clients like web based ones capable of talking to all agents.
We think this feature is ready for wide use, it's well documented and we've done extensive testing. We'll be porting some of our own code over to it once this release is out and we do anticipate there might be some 0.4.x releases to round off a few issues that might remain. We do not currently have any open tickets against Simple RPC.
We've also added the ability to create more complex queries such as:
{% highlight console %} --with-class /dev_server/ --with-class /rails/ {% endhighlight %}
This does an AND operation on the puppet classes on the node and finds only nodes with both /dev_server/ AND /rails/ classes. This new functionality applies to all types of filter.
We've made the --with-class filters more generic in comments, documentation etc with an eye to be more usable in Chef and other Configuration Management environments.
Unfortunately introducing the new filtering methods has some backward compatibility issues, if you had clients/agents with code like:
{% highlight ruby %} options[:filter]["agent"] = "some agent" {% endhighlight %}
You should now change that to:
{% highlight ruby %} options[:filter]["agent"] << "some agent" {% endhighlight %}
As each filter is an array now. If you do not change the code it will still work as before but you will not be able to use the compound filtering feature on filter types that you've forced to be a string and there might be some other undesired side effects. We've tried though to at least not break old code, they just can't use the new features.
You were also able to test easily in the past if you're running unfiltered using something like:
{% highlight ruby %} if options[:filter] == {} {% endhighlight %}
Now that's much harder and we've added a helper to make this easier:
{% highlight ruby %} if MCollective::Util.empty_filter?(options[:filter]) {% endhighlight %}
This new method is compatible with both the old and new filter method so you can start using it before you finish the first issue mentioned here.
We've also made the class filter more generic, in the past you did class filters like this:
{% highlight ruby %} options[:filter]["puppet_class"] << /apache/ {% endhighlight %}
Now you have to adjust it to:
{% highlight ruby %} options[:filter]["cf_class"] << /apache/ {% endhighlight %}
Old code will keep working but you should change to this name for filters to be consistent with the rest of the code base.
| Date | Description | Ticket |
|---|---|---|
| 2009/12/28 | Add support for other configuration management systems like chef in the --with-class filters | 13 |
| 2009/12/28 | Add a Util.empty_filter? to test for an empty filter |
|
| 2009/12/27 | Create a new client framework SimpleRPCIntroduction | 6 |
| 2009/12/27 | Add support for multiple filters of the same type | 3 |
Primarily a bug fix release. Only new feature is to allow the user to create MCollective::Util::* classes and put those in the plugins directory. This is useful for more complex agents and clients.
This release should not break any older code, if it does it's a bug.
| Date | Description | Ticket |
|---|---|---|
| 2009/12/16 | Improvements for newer versions of Ruby where TERM signal was not handled | 7 |
| 2009/12/07 | MCollective::Util is now a module and plugins can drop in util classes in the plugin dir | |
| 2009/12/07 | The Rakefile now works with rake provided on Debian 4 systems | 2 |
| 2009/12/07 | Improvements in the RC script for Debian and older Ubuntu systems | 5 |
First numbered release
n/a
n/a