Could not autoload puppet/provider/network_config/interfaces: no such file to load -- puppetx/filemapper

[kotowicz]

I installed the package by calling puppet module install adrien-network
I've added a simple static entry to a node:

network_config { 'lo':
    ensure => 'present',
    family => 'inet',
    method => 'loopback',
    onboot => 'true',
}

deployment fails with the following error:

Could not autoload puppet/provider/network_config/interfaces: no such file to load -- puppetx/filemapper
Could not autoload puppet/type/network_config: Could not autoload puppet/provider/network_config/interfaces: no such file to load -- puppetx/filemapper
Could not autoload puppet/type/network_config: Could not autoload puppet/provider/network_config/interfaces: no such file to load -- puppetx/filemapper on node xxxx

filemapper is installed inside /etc/puppet/modules/filemapper

I'd be happy to help debug the problem. I'm using puppet 3.4.3 & puppet-master 3.4.3.

List of installed packages:
$ sudo puppet module list
/etc/puppet/modules
├── adrien-boolean (v1.0.1)
├── adrien-filemapper (v1.1.2)
├── adrien-network (v0.4.1)

[gonzalobrandan]

Hi kotowicz, I had the same problem, reboot the master and then worked OK

[kotowicz]

rebooting did not solve my problem.

[NorseGaud]

Ditto. Debian Wheezy here, followed the instructions and see:

root@soap:~# puppet agent --test
Info: Retrieving pluginfacts
Info: Retrieving plugin
Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Could not autoload puppet/type/network_config: Could not autoload puppet/provider/network_config/interfaces: cannot load such file -- puppetx/filemapper on node soap.local.dev
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

[jhoblitt]

@ConnarPierce Do you have pluginsync disabled? Pluginsync needs to be working in order for custom types to function.

[NorseGaud]

@jhoblitt Where do I find that out? I just did a fresh install and am brand new to all of this.

EDIT: I just rebooted the agent and it's now working:

root@soap:~# puppet agent --test
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Caching catalog for soap.local.dev
Info: Applying configuration version '1408731618'
Notice: /Stage[main]/Main/Node[soap.local.dev]/Network_config[eth1]/onboot: onboot changed '' to 'true'
Notice: Finished catalog run in 0.06 seconds

If you could still clarify the pluginsync thing, I'd appreciate it.

[vchepkov]

run 'puppet plugin download' on your master. Would be nice if module somehow did it itself.

[kotowicz]

@vchepkov this doesn't help either:

$ puppet plugin download
Error: /File[/var/lib/puppet/lib]: Failed to generate additional resources using 'eval_generate': getaddrinfo: Temporary failure in name resolution

Error: /File[/var/lib/puppet/lib]: Could not evaluate: Could not retrieve file metadata for puppet://puppet/plugins: getaddrinfo: Temporary failure in name resolution
Wrapped exception:
getaddrinfo: Temporary failure in name resolution

Error: /File[/var/lib/puppet/facts.d]: Failed to generate additional resources using 'eval_generate': getaddrinfo: Temporary failure in name resolution

Error: /File[/var/lib/puppet/facts.d]: Could not evaluate: Could not retrieve file metadata for puppet://puppet/pluginfacts: getaddrinfo: Temporary failure in name resolution
Wrapped exception:
getaddrinfo: Temporary failure in name resolution

After restarting puppetmaster, I keep getting the same error message as before:

Could not autoload puppet/provider/network_config/interfaces: no such file to load -- puppetx/filemapper
Could not autoload puppet/type/network_config: Could not autoload puppet/provider/network_config/interfaces: no such file to load -- puppetx/filemapper
Could not autoload puppet/type/network_config: Could not autoload puppet/provider/network_config/interfaces: no such file to load -- puppetx/filemapper on node xxxx

[igalic]

Error: /File[/var/lib/puppet/lib]: Could not evaluate: Could not retrieve file metadata for puppet://puppet/plugins: getaddrinfo: Temporary failure in name resolution

you'll have to execute that as root

[adrienthebo]

Per the dependencies section of the readme (https://github.com/puppet-community/puppet-network#dependencies) you need to be able to pluginsync the master for this to work; the error that you're seeing indicates that you have faulty DNS or Puppet configuration. Make sure that pluginsync works and that should fix your issue.

[rnelson0]

I see this during an rspec run, which I wouldn't expect to have anything to do with a master.

$ rspec spec/classes/base/linux/routes_spec.rb
.FFFFFFFFFFFFFFF

Failures:

  1) profile::base::linux::routes with defaults for all parameters should contain Network_route[A.B.C.D/21]
     Failure/Error: it { is_expected.to contain_network_route('A.B.C.D/21') }
     Puppet::Error:
       Could not autoload puppet/type/network_route: Could not autoload puppet/provider/network_route/redhat: cannot load such file -- puppetx/filemapper
     # ./spec/fixtures/modules/network/lib/puppet/provider/network_route/redhat.rb:2:in `<top (required)>'
     # ./spec/fixtures/modules/network/lib/puppet/type/network_route.rb:3:in `<top (required)>'
     # ./spec/classes/base/linux/routes_spec.rb:17:in `block (3 levels) in <top (required)>'

  2) profile::base::linux::routes with defaults for all parameters should contain Network_route[E.F.G.H/22]
     Failure/Error: it { is_expected.to contain_network_route('E.F.G.H/22') }
       expected that the catalogue would contain Network_route[E.F.G.H/22]
     # ./spec/classes/base/linux/routes_spec.rb:18:in `block (3 levels) in <top (required)>'

After the first autoload error the remaining routes don't complain about it. Kind of weird. Fixtures and filesystem check out.

$ cat .fixtures.yml
<snip>
    boolean: 'https://github.com/adrienthebo/puppet-boolean'
    filemapper: 'https://github.com/adrienthebo/puppet-filemapper'
    network: 'git://github.com/puppet-community/puppet-network'

$ ls spec/fixtures/modules/filemapper/lib/puppetx/filemapper.rb
spec/fixtures/modules/filemapper/lib/puppetx/filemapper.rb

Is this related or new behavior?

[rnelson0]

When I push the change to the master and run on an agent in noop mode, I get an error but it looks like it wants to try and add the routes. A little concerned about taking noop off, though:

Info: Loading facts
Info: Caching catalog for dur-mantisdev01.gcsc.att.com
Info: Applying configuration version '1430837697'
Error: Could not prefetch network_route provider 'redhat': Malformed redhat route file, cannot instantiate network_route resources
Notice: /Stage[main]/Profile::Base::Linux::Routes/Network_route[A.B.C.D/21]/ensure: current_value absent, should be present (noop)
...

[igalic]

as a first guess, i would've suspected rspec-puppet, but who knows.

[rnelson0]

Good and bad news! The good: this appears related to LOAD_PATH and rodjek/rspec-puppet#237. This can be tested by modifying a few files (two files in my use case - anywhere there's a "require 'puppetx/filemapper'" line) and telling them where to find puppetx/filemapper. Example:

$ head -2 spec/fixtures/modules/network/lib/puppet/provider/network_route/routes.rb
require 'ipaddr'
require File.expand_path(File.join(File.dirname(__FILE__), '../../../../../filemapper/lib', 'puppetx/filemapper'))
[rn7284@chi-build02 profile]$ head -2 spec/fixtures/modules/network/lib/puppet/provider/network_route/redhat.rb
require 'ipaddr'
require File.expand_path(File.join(File.dirname(__FILE__), '../../../../../filemapper/lib', 'puppetx/filemapper'))

The bad news: that's clear as mud and only required for rspec-puppet - until the other PR is merged and (hopefully) fixes the LOAD_PATH issue. Modifying all these files could then potentially cause issues in production if the filemapper module is not in the same modulepath as network (one is in the environment and the other in /etc/puppet/modules or such).

[rnelson0]

A better short term fix is to add this to spec/spec_helper.rb

$LOAD_PATH << 'spec/fixtures/modules/filemapper/lib'

It's still a bit ... weird ... but has less side effects.

[ffrank]

As far as workarounds go, this one looks alright to me.

[JimPanic]

Invoking the tests like this works for me:

bundle exec spec/travis_rspec.rb --color --format documentation -b -I $(puppet module list | head -n1)/filemapper/lib

[rski]

This seems to be this puppet bug, which has existed for a fair amount of time. If I understood it correctly, require 'puppetx/filemapper' is not enough to make puppet put filemapper in the $LOAD_PATH. Filemapper only happens to be there sometimes by chance.

My guess is if filemapper were shipped as part of puppet-network it would load it every time since puppet would load puppet-network's puppetx folder.

Thoughts?

[njm506]

It is indeed that bug. Please upvote it, as nobody seems to be paying it any attention at the moment.

The problem is that $LOAD_PATH isn't set up before types/providers are loaded, so they can't find their dependencies within modules. Ideally, Puppet would either fix up $LOAD_PATH before loading things, or would provide an alternative to require that temporarily searches the appropriate locations.

While distributing filemapper with the module would work, it could easily conflict with any other versions installed elsewhere, as the namespace will be shared.

[rski]

There seems to be some activity in the puppet bug, sprint and team assignments. Things are looking good.

[igalic]

\o/

[jurim76]

The bug still exists

puppet-3.8.7
puppet-network-0.7.0
puppet-filemapper-2.1.0

Error 400 on SERVER: Could not autoload puppet/type/network_config: Could not autoload puppet/provider/network_config/redhat: cannot load such file -- puppetx/filemapper

[jhoblitt]

Has the puppetserver config been changed to allow loading of PuppetX?

https://tickets.puppetlabs.com/browse/SERVER-973

[jurim76]

Do you mean "pluginsync" parameter?
pluginsync set to "true"

[jhoblitt]

I mean adjusting the ruby-load-path similar to https://github.com/jenkinsci/puppet-jenkins/blob/master/NATIVE_TYPES_AND_PROVIDERS.md#puppetserver

[njm506]

On Puppet 3.8 I'm currently working around this with a hack module, adding a parser function which affects the Puppetmaster's Ruby environment:

require 'puppet/util/autoload'

module Puppet::Parser::Functions
  newfunction(:libfix) do |args|
    libpath_all_modules = args.shift
    libpath_modules     = args.shift

    env = Puppet.lookup(:current_environment)
    moddirs = Puppet::Util::Autoload.module_directories(env)

    if libpath_all_modules
      add_paths = moddirs
    else
      add_paths = moddirs.find_all do |path|
        modname = path.split('/')[-2]
        libpath_modules.include? modname
      end
    end

    add_paths.each do |path|
      unless $LOAD_PATH.include? path
        $LOAD_PATH << path
      end
    end
  end
end

I've then got a manifest which looks like this:

# Class to fix LOAD_PATH
class libfix(
  $libpath_modules = [],
  $libpath_all_modules = true,
) {
  # This function doesn't do anything, but it needs to be parsed
  libfix($libpath_all_modules, $libpath_modules)
}

So you can either include libfix or declare it like a resource and select particular modules to place on the load path (in case you need to work around problematic modules). It's a relatively nasty hack but it works for us. There's absolutely no guarantee whether it'd work on any other version than it was designed for.

Also bear in mind that this will cause pollution between environments as LOAD_PATH is shared (unless you have something that segregates master instances for each environment).

[oniGino]

FYI to anyone with this issue, and your attempting to run this module inside of a branch. You need to make sure your puppet master also gets a run on that same branch or else the necessary PuppetFile modules will not exist server side, even though the ruby code might be on the agent side, it will still fail.

[jcharaoui]

I've had this issue today: after a reload of the puppetserver, the first run would always fail with this error message. Subsequent runs would succeed, however.

To avoid the initial failure, I found that simply running puppet generate types in the environment directory solved this issue. Therefore I don't think this is a bug specifically related to this module.