Ubuntu has a certain really annoying property. Alright it has several, but the one that I'm talking about right now is the insistence that it start services upon installation. While I'm never a fan, there are certain times when it chafes more than normal.
Here's the deal. I'm using the PuppetLabs RabbitMQ module, and I'm trying to spin up an instance in Vagrant. The initialization is simple enough:
class { 'rabbitmq':
port => '5672',
service_manage => true,
environment_variables => {
'RABBITMQ_NODENAME' => 'server',
'RABBITMQ_SERVICENAME' => 'rabbitMQ'
}
}
This works - mostly. The package gets installed, but when puppet tries to manage the service, it fails:
The reason for the failure is actually because the port is in use. If you connect into the machine and try to start the service manually, you get this:
vagrant@precise64:~$ sudo /etc/init.d/rabbitmq-server start
* Starting message broker rabbitmq-server
* FAILED - check /var/log/rabbitmq/startup_\{log, _err\}
...fail!
When you check the start_log, you find this:
Error description:
{could_not_start_tcp_listener,{"::",5672}}
The reason it can't start the listener is because that port is already in use!
vagrant@precise64:~$ sudo lsof -P | grep LISTEN | grep 5672
beam.smp 2211 rabbitmq 16u IPv6 11556 0t0 TCP *:5672 (LISTEN)
Sure enough, 'ps auxf' shows this:
rabbitmq 2211 0.4 8.3 1073128 31160 ? Sl 10:07 0:02 \_ /usr/lib/erlang/erts-5.8.5/bin/beam.smp -W w -K true -A30 -P 1048576 -- -root /usr/lib/erlang -progname erl -- -home /var/lib/rabbitmq -- -pa /usr/lib/rabbitmq/lib/rabbitmq_server-3.2.3/sbin/../ebin -noshell -noinput -s rabbit boot -sname rabbit@precise64 -boot start_sasl -kernel inet_default_connect_options [{nodelay,true}] -sasl errlog_type error -sasl sasl_error_logger false -rabbit error_logger {file,"/var/log/rabbitmq/rabbit@precise64.log"} -rabbit sasl_error_logger {file,"/var/log/rabbitmq/rabbit@precise64-sasl.log"} -rabbit enabled_plugins_file "/etc/rabbitmq/enabled_plugins" -rabbit plugins_dir "/usr/lib/rabbitmq/lib/rabbitmq_server-3.2.3/sbin/../plugins" -rabbit plugins_expand_dir "/var/lib/rabbitmq/mnesia/rabbit@precise64-plugins-expand" -os_mon start_cpu_sup false -os_mon start_disksup false -os_mon start_memsup false -mnesia dir "/var/lib/rabbitmq/mnesia/rabbit@precise64"
As it turns out, of course, starting the service upon installation is by design.
So here's my question. Presumably, other people have done this. The service is documented as tested under Ubuntu 12.04, which is what I'm running. How do you make this work? The only thing I've seen that makes the puppet run not fail is to set "service_manage => false" and I don't want that. I want the service to be managed and I want the initial default running-right-after-installation instance to die.
What's the right way to do that? For what it's worth, I searched on ServerFault and found someone else with the same problem, but there hadn't been an answer after a month yet, which is why I brought it here.
Can you give me a hand? Thanks in advance!
EDIT
Got it! I popped onto the #rabbitmq IRC channel on freenode and started asking questions, and I was going down the complete wrong line of thinking, when one of the local denizens, bob235, asked me a crucial question:
so when you install other services (e.g. webservers) they don't start up automatically? any idea how they manage to do that?
Right. Good call. Why, when I install apache via puppet doesn't it fail. Well, it's because Puppet checks to see if Apache is running before it starts, by doing '/etc/init.d/apache2 status' and learning that the service is already running. So why wasn't that happening here?
I booted a generic vagrant instance, and manually installed rabbitmq using apt-get. It ran after installation, as I expected it would. Running '/etc/init.d/rabbitmq-server status' showed that it was running:
root@precise64:~# /etc/init.d/rabbitmq-server status
Status of node rabbit@precise64 ...
[{pid,2284},
{running_applications,[{rabbit,"RabbitMQ","2.7.1"},
{mnesia,"MNESIA CXC 138 12","4.5"},
{os_mon,"CPO CXC 138 46","2.2.7"},
{sasl,"SASL CXC 138 11","2.1.10"},
{stdlib,"ERTS CXC 138 10","1.17.5"},
{kernel,"ERTS CXC 138 10","2.14.5"}]},
{os,{unix,linux}},
{erlang_version,"Erlang R14B04 (erts-5.8.5) [source] [64-bit]
[smp:2:2] [rq:2] [async-threads:30] [kernel-poll:true]\n"},
{memory,[{total,24525920},
{processes,9676480},
{processes_used,9672808},
{system,14849440},
{atom,1124441},
{atom_used,1120225},
{binary,92152},
{code,11134417},
{ets,733392}]},
{vm_memory_high_watermark,0.3999999984343938},
{vm_memory_limit,153295257}]
...done.
root@precise64:~# /etc/init.d/rabbitmq-server status
Status of node precise64@precise64 ...
Error: unable to connect to node precise64@precise64: nodedown
DIAGNOSTICS
===========
nodes in question: [precise64@precise64]
hosts, their running nodes and ports:
- precise64: [{rabbit,45074},{rabbitmqctl2835,51120}]
current node details:
- node name: rabbitmqctl2835@precise64
- home dir: /var/lib/rabbitmq
- cookie hash: ovNnSahEs2CWYKS80bcf5w==
Yeah, it definitely does't see it running. But wait, did you catch it? There's a subtle difference.
In the output from installing it manually, the first line was "Status of node rabbit@precise64", but after the puppet run, it was "precise64@precise64". Puppet is changing the node name on disk, but the running instance is under the original name!
I edited /etc/rabbitmq/rabbitmq-env.conf to change the node name back to 'rabbit', then ran /etc/init.d/rabbitmq-server status, and sure enough, it showed identical output to when I installed it manually.
To finish it off, I edited the puppet config to change the node name back to 'rabbit', then restarted the vagrant box. It started perfectly:
End result: If you want to use the RabbitMQ module from puppetlabs, my advice is to not change the node name on the server from 'rabbit'.