[Bug 1124947] New: devel:languages:erlang:Factory/erlang: Bug with EPMD
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947 Bug ID: 1124947 Summary: devel:languages:erlang:Factory/erlang: Bug with EPMD Classification: openSUSE Product: openSUSE.org Version: unspecified Hardware: Other OS: Other Status: NEW Severity: Normal Priority: P5 - None Component: 3rd party software Assignee: matwey.kornilov@gmail.com Reporter: gsantomaggio@suse.de QA Contact: bnc-team-screening@forge.provo.novell.com Found By: --- Blocker: --- We are having several problems with the EPMD systemd service. Often the epmd process is activated by an Elixir or Erlang process, this causes problems for other services that need Erlang as RabbitMQ. Here is a very easy way to reproduce the problem ( using openSUSE-Tumbleweed, but I can reproduce the problem also with other suse/open distributions): ```` vagrant init opensuse/openSUSE-Tumbleweed-x86_64 vagrant up vagrant ssh ``` Install rabbitmq: ``` sudo zypper in rabbitmq-server sudo rabbitmqct status ``` Check the EPMD: ``` ps aux | grep epmd vagrant 3605 0.0 0.0 26820 216 ? S 10:12 0:00 /usr/lib64/erlang/erts-10.1.3/bin/epmd -daemon ``` So, epmd is not running under systemd. Try to run RabbitMQ: ``` sudo systemctl start rabbitmq-server A dependency job for rabbitmq-server.service failed. See 'journalctl -xe' for details. ``` does not start, because of: ``` feb 11 10:29:09 vagrant-openSUSE-Tumbleweed systemd[3955]: epmd.socket: Failed to create listening socket: Address already in use feb 11 10:29:09 vagrant-openSUSE-Tumbleweed systemd[1]: epmd.socket: Failed to receive listening socket: Input/output error feb 11 10:29:09 vagrant-openSUSE-Tumbleweed systemd[1]: epmd.socket: Failed to listen on sockets: Input/output error feb 11 10:29:09 vagrant-openSUSE-Tumbleweed systemd[1]: epmd.socket: Failed with result 'resources'. feb 11 10:29:09 vagrant-openSUSE-Tumbleweed systemd[1]: Failed to listen on Erlang Port Mapper Daemon Activation Socket. ``` by killing "/usr/lib64/erlang/erts-10.1.3/bin/epmd -daemon" it works ``` ps aux | grep epmd epmd 3993 0.0 0.1 26820 1860 ? Ss 10:33 0:00 /usr/bin/epmd -systemd ``` Unfortunately, we have to deal with this problem in our OpenStack installations. I know that the epmd service has been added to bind the epmd socket to localhost, but when it is not running inside systemd it is listening to all the network interfaces: ``` ss -tulpen | grep epmd tcp LISTEN 0 128 0.0.0.0:4369 ``` it is causing lot of problems during startup/upgrade RabbitMQ installations (also in OpenStack) proposal: Can we remove epmd.service ? I am/was also mainteiner for other erlang packages as: https://github.com/rabbitmq/erlang-rpm Where we don't have anything about epmd, and also we don't have problems :) Thank you -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947#c1
--- Comment #1 from Matwey Kornilov
We are having several problems with the EPMD systemd service.
Often the epmd process is activated by an Elixir or Erlang process, this causes problems for other services that need Erlang as RabbitMQ.
Here is a very easy way to reproduce the problem ( using openSUSE-Tumbleweed, but I can reproduce the problem also with other suse/open distributions):
```` vagrant init opensuse/openSUSE-Tumbleweed-x86_64 vagrant up vagrant ssh ```
Install rabbitmq:
``` sudo zypper in rabbitmq-server sudo rabbitmqct status ```
Check the EPMD:
``` ps aux | grep epmd vagrant 3605 0.0 0.0 26820 216 ? S 10:12 0:00 /usr/lib64/erlang/erts-10.1.3/bin/epmd -daemon ```
First of all. Why are you running epmd from vagrant user? Probably, instead of running epmd from systemd service you manually start another one instance which ate required port number.
So, epmd is not running under systemd.
Try to run RabbitMQ:
``` sudo systemctl start rabbitmq-server A dependency job for rabbitmq-server.service failed. See 'journalctl -xe' for details. ```
does not start, because of: ``` feb 11 10:29:09 vagrant-openSUSE-Tumbleweed systemd[3955]: epmd.socket: Failed to create listening socket: Address already in use feb 11 10:29:09 vagrant-openSUSE-Tumbleweed systemd[1]: epmd.socket: Failed to receive listening socket: Input/output error feb 11 10:29:09 vagrant-openSUSE-Tumbleweed systemd[1]: epmd.socket: Failed to listen on sockets: Input/output error feb 11 10:29:09 vagrant-openSUSE-Tumbleweed systemd[1]: epmd.socket: Failed with result 'resources'. feb 11 10:29:09 vagrant-openSUSE-Tumbleweed systemd[1]: Failed to listen on Erlang Port Mapper Daemon Activation Socket. ```
by killing "/usr/lib64/erlang/erts-10.1.3/bin/epmd -daemon"
it works ``` ps aux | grep epmd epmd 3993 0.0 0.1 26820 1860 ? Ss 10:33 0:00 /usr/bin/epmd -systemd ```
Unfortunately, we have to deal with this problem in our OpenStack installations.
I know that the epmd service has been added to bind the epmd socket to localhost, but when it is not running inside systemd it is listening to all the network interfaces:
``` ss -tulpen | grep epmd tcp LISTEN 0 128 0.0.0.0:4369 ```
it is causing lot of problems during startup/upgrade RabbitMQ installations (also in OpenStack)
proposal: Can we remove epmd.service ?
I am/was also mainteiner for other erlang packages as: https://github.com/rabbitmq/erlang-rpm Where we don't have anything about epmd, and also we don't have problems :)
Thank you
-- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947#c2
--- Comment #2 from Matwey Kornilov
We are having several problems with the EPMD systemd service.
Often the epmd process is activated by an Elixir or Erlang process, this causes problems for other services that need Erlang as RabbitMQ.
Here is a very easy way to reproduce the problem ( using openSUSE-Tumbleweed, but I can reproduce the problem also with other suse/open distributions):
```` vagrant init opensuse/openSUSE-Tumbleweed-x86_64 vagrant up vagrant ssh ```
Install rabbitmq:
``` sudo zypper in rabbitmq-server sudo rabbitmqct status ```
Check the EPMD:
``` ps aux | grep epmd vagrant 3605 0.0 0.0 26820 216 ? S 10:12 0:00 /usr/lib64/erlang/erts-10.1.3/bin/epmd -daemon ```
So, epmd is not running under systemd.
Try to run RabbitMQ:
``` sudo systemctl start rabbitmq-server A dependency job for rabbitmq-server.service failed. See 'journalctl -xe' for details. ```
does not start, because of: ``` feb 11 10:29:09 vagrant-openSUSE-Tumbleweed systemd[3955]: epmd.socket: Failed to create listening socket: Address already in use feb 11 10:29:09 vagrant-openSUSE-Tumbleweed systemd[1]: epmd.socket: Failed to receive listening socket: Input/output error feb 11 10:29:09 vagrant-openSUSE-Tumbleweed systemd[1]: epmd.socket: Failed to listen on sockets: Input/output error feb 11 10:29:09 vagrant-openSUSE-Tumbleweed systemd[1]: epmd.socket: Failed with result 'resources'. feb 11 10:29:09 vagrant-openSUSE-Tumbleweed systemd[1]: Failed to listen on Erlang Port Mapper Daemon Activation Socket. ```
by killing "/usr/lib64/erlang/erts-10.1.3/bin/epmd -daemon"
it works ``` ps aux | grep epmd epmd 3993 0.0 0.1 26820 1860 ? Ss 10:33 0:00 /usr/bin/epmd -systemd ```
Unfortunately, we have to deal with this problem in our OpenStack installations.
I know that the epmd service has been added to bind the epmd socket to localhost, but when it is not running inside systemd it is listening to all the network interfaces:
``` ss -tulpen | grep epmd tcp LISTEN 0 128 0.0.0.0:4369 ```
it is causing lot of problems during startup/upgrade RabbitMQ installations (also in OpenStack)
proposal: Can we remove epmd.service ?
I am/was also mainteiner for other erlang packages as: https://github.com/rabbitmq/erlang-rpm Where we don't have anything about epmd, and also we don't have problems :)
Thank you
I'll look into it later. My suggestion is that epmd.socket is not started at erlang package installation for some reason. Listening IP addresses is better to discuss with the Security team. It were they who asked not to listen on 0.0.0.0 by default. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947
Antonio Ojea
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947#c3
--- Comment #3 from Gabriele Santomaggio
First of all. Why are you running epmd from vagrant user? I wanted to provide an easy way to reproduce the problem.
We have this problem in general, from Bare-metal servers to virtual machines/vagrant etc. The problem is that EPMD is running automatically by Erlang when you run an erlang node with the network name ( using -sname or -name parameter). the same bug can be reproduced using: ``` erl -sname hello ``` it will execute epmd as daemon, bound to all the interfaces: /erlang/erts-10.1.3/bin/epmd -daemon -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947#c4
--- Comment #4 from Matwey Kornilov
First of all. Why are you running epmd from vagrant user? I wanted to provide an easy way to reproduce the problem.
We have this problem in general, from Bare-metal servers to virtual machines/vagrant etc.
The problem is that EPMD is running automatically by Erlang when you run an erlang node with the network name ( using -sname or -name parameter).
the same bug can be reproduced using:
``` erl -sname hello ```
it will execute epmd as daemon, bound to all the interfaces: /erlang/erts-10.1.3/bin/epmd -daemon
Exactly! But the full algorithm is the following: 1) Try to connect to EPMD 2) Spawn new EPMD if 1) failed When the system is properly configured, we always should finish on the step 1 and never try step 2. Because epmd.socket service must be always active. Could you please try the following? ``` vagrant init opensuse/openSUSE-Tumbleweed-x86_64 vagrant up vagrant ssh sudo zypper in rabbitmq-server sudo systemctl status epmd.socket ``` -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947#c5
--- Comment #5 from Gabriele Santomaggio
Could you please try the following?
``` vagrant init opensuse/openSUSE-Tumbleweed-x86_64 vagrant up vagrant ssh
sudo zypper in rabbitmq-server sudo systemctl status epmd.socket ```
``` vagrant@vagrant-openSUSE-Tumbleweed:~> sudo systemctl status epmd.socket ● epmd.socket - Erlang Port Mapper Daemon Activation Socket Loaded: loaded (/usr/lib/systemd/system/epmd.socket; enabled; vendor preset: enabled) Active: inactive (dead) Listen: 127.0.0.1:4369 (Stream) ``` -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947#c6
--- Comment #6 from Matwey Kornilov
(In reply to Matwey Kornilov from comment #4)
Could you please try the following?
``` vagrant init opensuse/openSUSE-Tumbleweed-x86_64 vagrant up vagrant ssh
sudo zypper in rabbitmq-server sudo systemctl status epmd.socket ```
``` vagrant@vagrant-openSUSE-Tumbleweed:~> sudo systemctl status epmd.socket ● epmd.socket - Erlang Port Mapper Daemon Activation Socket Loaded: loaded (/usr/lib/systemd/system/epmd.socket; enabled; vendor preset: enabled) Active: inactive (dead) Listen: 127.0.0.1:4369 (Stream) ```
It is dead, but it should be alive. I suppose there is an issue in post install scripts in erlang-epmd package. Could you please try? ``` vagrant init opensuse/openSUSE-Tumbleweed-x86_64 vagrant up vagrant ssh sudo zypper in rabbitmq-server sudo reboot sudo systemctl status epmd.socket ``` I suppose it will be active after the reboot and rest of the things will be work correctly. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947#c7
--- Comment #7 from Gabriele Santomaggio
Could you please try?
``` vagrant init opensuse/openSUSE-Tumbleweed-x86_64 vagrant up vagrant ssh sudo zypper in rabbitmq-server sudo reboot sudo systemctl status epmd.socket ```
I suppose it will be active after the reboot and rest of the things will be work correctly.
Yes. ``` vagrant@vagrant-openSUSE-Tumbleweed:~> sudo systemctl status epmd.socket ● epmd.socket - Erlang Port Mapper Daemon Activation Socket Loaded: loaded (/usr/lib/systemd/system/epmd.socket; enabled; vendor preset: enabled) Active: active (listening) since Mon 2019-02-11 11:26:59 CET; 55s ago Listen: 127.0.0.1:4369 (Stream) Tasks: 0 (limit: 1158) CGroup: /system.slice/epmd.socket ``` -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947#c8
--- Comment #8 from Matwey Kornilov
(In reply to Matwey Kornilov from comment #6)
Could you please try?
``` vagrant init opensuse/openSUSE-Tumbleweed-x86_64 vagrant up vagrant ssh sudo zypper in rabbitmq-server sudo reboot sudo systemctl status epmd.socket ```
I suppose it will be active after the reboot and rest of the things will be work correctly.
Yes. ``` vagrant@vagrant-openSUSE-Tumbleweed:~> sudo systemctl status epmd.socket ● epmd.socket - Erlang Port Mapper Daemon Activation Socket Loaded: loaded (/usr/lib/systemd/system/epmd.socket; enabled; vendor preset: enabled) Active: active (listening) since Mon 2019-02-11 11:26:59 CET; 55s ago Listen: 127.0.0.1:4369 (Stream) Tasks: 0 (limit: 1158) CGroup: /system.slice/epmd.socket ```
I've found that the same issue presents for cups.socket After cups is installed, cups.socket is inactive. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947#c9
--- Comment #9 from Matwey Kornilov
(In reply to Gabriele Santomaggio from comment #7)
(In reply to Matwey Kornilov from comment #6)
Could you please try?
``` vagrant init opensuse/openSUSE-Tumbleweed-x86_64 vagrant up vagrant ssh sudo zypper in rabbitmq-server sudo reboot sudo systemctl status epmd.socket ```
I suppose it will be active after the reboot and rest of the things will be work correctly.
Yes. ``` vagrant@vagrant-openSUSE-Tumbleweed:~> sudo systemctl status epmd.socket ● epmd.socket - Erlang Port Mapper Daemon Activation Socket Loaded: loaded (/usr/lib/systemd/system/epmd.socket; enabled; vendor preset: enabled) Active: active (listening) since Mon 2019-02-11 11:26:59 CET; 55s ago Listen: 127.0.0.1:4369 (Stream) Tasks: 0 (limit: 1158) CGroup: /system.slice/epmd.socket ```
I've found that the same issue presents for cups.socket After cups is installed, cups.socket is inactive.
People from maillist say that inactive services are intentionally due to security and stability reasons. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947#c10
--- Comment #10 from Gabriele Santomaggio
People from maillist say that inactive services are intentionally due to security and stability reasons. Thank you Matwey, But actually, it does not solve the problem :) Wich maililist btw?
-- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947#c11
--- Comment #11 from Matwey Kornilov
(In reply to Matwey Kornilov from comment #9)
People from maillist say that inactive services are intentionally due to security and stability reasons. Thank you Matwey, But actually, it does not solve the problem :) Wich maililist btw?
Sorry, forgot to attach the reference link: https://lists.opensuse.org/opensuse-packaging/2019-02/msg00012.html -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947#c12
--- Comment #12 from Gabriele Santomaggio
Sorry, forgot to attach the reference link: https://lists.opensuse.org/opensuse-packaging/2019-02/msg00012.html
Thank you Matwey, it seems there is a sort of deadlock for this issue :)! Starting epmd.socket service (after install erlang) solves the problem or at least reduce the possibility. ``` sudo zypper in rabbitmq-server sudo systemctl start epmd.socket ``` then if I try to execute an erlang node, it uses the epmd systemd service -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947
http://bugzilla.opensuse.org/show_bug.cgi?id=1124947#c13
--- Comment #13 from Matwey Kornilov
participants (1)
-
bugzilla_noreply@novell.com