Archive for January, 2016

Munin and “Can’t use an undefined value as an ARRAY reference”  

Posted at 10:45 am in Uncategorized

I recently came across a rather obscure and vague error in Munin:

Can't use an undefined value as an ARRAY reference
 at /usr/share/perl5/Munin/Master/


It seems there are quite a few error reports on this, with very different suggestions on how to solve the problem – and for some, the problem was never solved.

In this case, we found that the FastCGI process that contructs the HTML structure (munin-cgi-html) was not able to read the file /var/lib/munin/htmlconf.storable due to a strict system umask (027, instead of the more normal 022). The htmlconf.storable file is written with system umask and ownership munin:munin, so the FastCGI process running as the user www-data will not be allowed to read this file.

File permissions with umask 022:

-rw-rw-r-- 1 munin munin [...] /var/lib/munin/htmlconf.storable


File permissions with umask 027:

-rw-rw---- 1 munin munin [...] /var/lib/munin/htmlconf.storable


There’s a recent ticket for making the process responsible for writing this file (and some others) set its own umask instead of relying on the system umask. That would have fixed this issue. In the meantime, the issue can be circumvented by changing directory ownership on /var/lib/munin, specifying umask for the Munin user, or running the FastCGI process as the Munin user.

Written by bjorn on January 28th, 2016

Tagged with , ,

Auditd in Ubuntu not listening?  

Posted at 8:36 am in Uncategorized

So you planned on using auditd for receiving logs from other auditd installations? And you’re using Ubuntu? Well, it could prove difficult. In the Ubuntu package, the maintainers have chosen – on everyone’s behalf – that no-one needs this. My setup is Ubuntu 14.04 (“Trusty”), with audit version 2.3.2, but it seems this has been the default for some time.

As documented, the auditd software can be set up for accepting logs from remote installations by configuring it with a listening port, more specifically by adding tcp_listen_port and a port number to /etc/audit/auditd.conf. After some troubleshooting and scratching my head why it still didn’t work, I started auditd in foreground mode, and this is what I found:

tcp_listen_port_parser called with: 5000
Listener support is not enabled, ignoring value at line 25
tcp_listen_queue_parser called with: 5
Listener support is not enabled, ignoring value at line 26
tcp_max_per_addr_parser called with: 1
Listener support is not enabled, ignoring value at line 27
tcp_client_max_idle_parser called with: 0
Listener support is not enabled, ignoring value at line 29


Listener support is not enabled? It seems some decisions were made in this mailing list exchange, but the principle of least surprise should still be maintained: “The auditd listener code is still enabled by default so that existing distro packaging recipes will not need to be updated.”. Building as two different auditd packages was also suggested – one for running as a server, and one for running as a client. So far so good.

I also found another blog post mentioning that this behaviour changed with Ubuntu 13.10. Time to check the packaging rules, and indeed, there it was:

 dh_auto_configure -- \
 --sbindir=/sbin \
 --libdir=/lib/${DEB_HOST_MULTIARCH} \
 --enable-shared=audit \
 --enable-gssapi-krb5 \
 --disable-listener \


Basically, Ubuntu has decided that you should not use auditd for log collection from remote servers, even when you explicitly configure it for doing so. Note that it doesn’t listen by default, you would have to make deliberate configuration changes, but still you are simply not allowed to. They didn’t document this, either – the least that could be done was add some comments to the configuration file where the listener configuration exists, letting you know that making certain configuration changes will be silently ignored. This behaviour is still the default with Ubuntu 15.10.

There are some ways to come around this: You can build your own auditd package, you can set up a different log transport and not rely on auditd – or you can choose another Linux distribution than Ubuntu for your log collection requirements.

Debian, for example, has not neutered this package. In Debian Jessie, with auditd 2.4, listener support is still perfectly possible.

Written by bjorn on January 26th, 2016

Tagged with , , ,