Tuesday, July 3, 2012

rabbitmq cluster problem : version_mismatch

Goal: a rabbitmq cluster ready for HA mirrored queues.
Problem: "Error: version_mismatch" during cluster command.

I went through the steps in the clustering guide and they worked fine on my freshly installed staging servers. I'm getting an error running through them on my production instance, trying to cluster a newly created EC2 instance in with my current production rabbitmq server.

Error: version_mismatch. Specifically one of them has 'topic_trie_node'.

I created the second server ( 10.7.203.145 ) from the first by creating a new volume from a snapshot of the disk from the first and then mounting the volume on a new EC2 instance.

Dear LazyWeb: Any ideas?

[andrew@ip-10-7-203-145]% sudo rabbitmqctl stop_app                               :) ~
Stopping node 'ip-10-7-203-145@ip-10-7-203-145' ...
...done.
[andrew@ip-10-7-203-145]% sudo rabbitmqctl reset                                  :) ~
Resetting node 'ip-10-7-203-145@ip-10-7-203-145' ...
...done.
[andrew@ip-10-7-203-145]% sudo rabbitmqctl cluster ip-10-7-203-145@ip-10-7-203-145 ip-10-7-203-85@ip-10-7-203-85Clustering node 'ip-10-7-203-145@ip-10-7-203-145' with ['ip-10-7-203-145@ip-10-7-203-145',
                                    'ip-10-7-203-85@ip-10-7-203-85'] ...
Error: {version_mismatch,[add_ip_to_listener,exchange_event_serial,
                          exchange_scratch,gm,ha_mirrors,mirrored_supervisor,
                          remove_user_scope,semi_durable_route,topic_trie,
                          user_admin_to_tags,add_queue_ttl,
                          multiple_routing_keys],
                         [add_queue_ttl,multiple_routing_keys,
                          add_ip_to_listener,exchange_event_serial,
                          exchange_scratch,gm,ha_mirrors,mirrored_supervisor,
                          remove_user_scope,semi_durable_route,topic_trie,
                          topic_trie_node,user_admin_to_tags]}
[andrew@ip-10-7-203-145]%                                                     2 :( ~

I agree with you prompt, sad-face indeed.

2 comments:

Simon MacMullen said...

You are trying to cluster together different versions of RabbitMQ - this will not work.

Andrew Grangaard said...

I'm running 2.7.1-1 on each as installed from rabbitmq.com/debian. Both have the same plugins installed and same erlang rev. !?!?

"deb http://www.rabbitmq.com/debian/ testing main"

[andrew@ip-10-7-203-145]% aptitude show rabbitmq-server :) ~
Package: rabbitmq-server
State: installed
Automatically installed: no
Version: 2.7.1-1
Priority: extra
Section: net
Maintainer: RabbitMQ Team
Uncompressed Size: 3,736 k
Depends: erlang-nox (>= 1:12.b.3), adduser, logrotate
Description: An AMQP server written in Erlang
RabbitMQ is an implementation of AMQP, the emerging standard for high performance enterprise messaging. The
RabbitMQ server is a robust and scalable implementation of an AMQP broker.
Homepage: http://www.rabbitmq.com/



[andrew@ip-10-7-203-85]% aptitude show rabbitmq-server :) ~
Package: rabbitmq-server
State: installed
Automatically installed: no
Version: 2.7.1-1
Priority: extra
Section: net
Maintainer: RabbitMQ Team
Uncompressed Size: 3,736 k
Depends: erlang-nox (>= 1:12.b.3), adduser, logrotate
Description: An AMQP server written in Erlang
RabbitMQ is an implementation of AMQP, the emerging standard for high performance enterprise messaging. The
RabbitMQ server is a robust and scalable implementation of an AMQP broker.
Homepage: http://www.rabbitmq.com/


[{pid,22943},
{running_applications,
[{rabbitmq_management,"RabbitMQ Management Console","2.7.1"},
{rabbitmq_management_agent,"RabbitMQ Management Agent","2.7.1"},
{amqp_client,"RabbitMQ AMQP Client","2.7.1"},
{rabbit,"RabbitMQ","2.7.1"},
{os_mon,"CPO CXC 138 46","2.2.5"},
{sasl,"SASL CXC 138 11","2.1.9.3"},
{rabbitmq_mochiweb,"RabbitMQ Mochiweb Embedding","2.7.1"},
{webmachine,"webmachine","1.7.0-rmq2.7.1-hg"},
{mochiweb,"MochiMedia Web Server","1.3-rmq2.7.1-git"},
{inets,"INETS CXC 138 49","5.5.2"},
{mnesia,"MNESIA CXC 138 12","4.4.17"},
{stdlib,"ERTS CXC 138 10","1.17.3"},
{kernel,"ERTS CXC 138 10","2.14.3"}]},
{os,{unix,linux}},
{erlang_version,
"Erlang R14B02 (erts-5.8.3) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:30] [kernel-poll:true]\n"},