WebFaction
Community site: login faq

Hi everyone,

I ran into some difficulties in the migration from CentOS 6 to CentOS 7 when it came to getting RabbitMQ up and running again.

My Erlang/RabbitMQ setup largely follows the instructions found elsewhere, notably here: http://raichev.net/adding-celery-with-rabbitmq-to-a-django-project-on-webfaction.html

However, when attempting to run RabbitMQ on CentOS 7, I was running into this error:

[me@centos7 ~]$ src/rabbitmq_server-3.6.9/sbin/rabbitmq-server
Failed to create thread: Resource temporarily unavailable (11)
src/rabbitmq_server-3.6.9/sbin/rabbitmq-server: line 252: 9322 Aborted start_rabbitmq_server "$@"

We tracked this down to the number of threads being created in the above step. The old CentOS 6 server had 8 processors. The shiny new CentOS 7 has 48. The default number of additional threads to create is (16*number_of_processors) which equals 768 on web602, making me sail over the 300 thread limit! See here for a discussion on thread count.

I have 2 differences between my setup and Alex's in the first link, both in sbin/rabbitmq-defaults:

#CONFIG_FILE=/home/ontheday/src/rabbitmq_server-3.6.9/sbin
CONFIG_FILE=/home/ontheday/src/rabbitmq_server-3.6.9/etc/rabbitmq/rabbitmq
LOG_BASE=/home/ontheday/logs/user/rabbitmq
MNESIA_BASE=/home/ontheday/src/rabbitmq_server-3.6.9/sbin

IO_THREAD_POOL_SIZE=129

First, I'm referencing etc/rabbitmq/rabbitmq.config which is below. I needed this for rabbit to listen on the correct port, despite the instructions from Alex:

[me@centos7 rabbitmq]$ more rabbitmq.config
[
  {rabbit, [{tcp_listeners, [port-number]}]}
].

Second, I have an additional line in sbin/rabbitmq-defaults to deal with the thread count:

[me@centos7 rabbitmq]$ ps -u [username] -o rss,command
100464 /home/me//lib/erlang/erts-8.3/bin/beam.smp -W w -A 129 -P 1048576 -t 5000000

Note the "-A 129" which is picked up from the IO_THREAD_POOL_SIZE setting. I set it odd so it is more obviously not a default setting. Webfaction support raised my thread limit so we could debug - before I added the IO_THREAD_POOL_SIZE setting this was showing as "-A 768" as expected given the default calculation.

This question is marked "community wiki".

asked 09 Apr, 16:22

Sunday%20Afternoon's gravatar image

Sunday After...
1114
accept rate: 0%

edited 09 Apr, 16:43

Be the first one to answer this question!
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×877
×12
×6
×5
×3

question asked: 09 Apr, 16:22

question was seen: 447 times

last updated: 09 Apr, 16:43

WEBFACTION
REACH US
SUPPORT
AFFILIATE PROGRAM
LEGAL
© COPYRIGHT 2003-2016 SWARMA LIMITED - WEBFACTION IS A SERVICE OF SWARMA LIMITED
REGISTERED IN ENGLAND AND WALES 5729350 - VAT REGISTRATION NUMBER 877397162
5TH FLOOR, THE OLD VINYL FACTORY, HAYES, UB3 1HA, UNITED KINGDOM