[opensuse-buildservice] Build Service: delayed_job memory consumption

newer
[opensuse-buildservice] All Debian...

Matwey V. Kornilov

1 Oct 2019 1 Oct '19

15:48

Hello, I am running private OBS instance on Leap 15.1. I use obs-api-2.10-lp151.10.1.noarch I see that delayed_job ruby scripts consume about 100MB of RAM each on my setup: PID RSS %MEM CMD 93019 93184 1.1 delayed_job.1000 93051 97220 1.1 delayed_job.1040 93013 106740 1.3 delayed_job.1060 93026 107380 1.3 delayed_job.1030 92997 111040 1.3 delayed_job.1010 92984 114296 1.4 delayed_job.1050 92965 117524 1.4 delayed_job.1020 93003 119332 1.4 delayed_job.1 93034 123532 1.5 delayed_job.2 92791 123664 1.5 clockworkd.clock 93040 128580 1.5 delayed_job.0 As far as I understand, the scripts are just delayed job workers, they pop the jobs from the database and execute some stuff then. Is it possible to tune memory consumption somehow? -- With best regards, Matwey V. Kornilov -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

Show replies by date

Henne Vogelsang

2 Oct 2 Oct

10:45

Hey, On 01.10.19 17:48, Matwey V. Kornilov wrote:

...

I am running private OBS instance on Leap 15.1. I use obs-api-2.10-lp151.10.1.noarch

I see that delayed_job ruby scripts consume about 100MB of RAM each on my setup: ... As far as I understand, the scripts are just delayed job workers, they pop the jobs from the database and execute some stuff then.

In the context of the application. That's the reason they load it into memory.

...

Is it possible to tune memory consumption somehow?

Not really no. Henne -- Henne Vogelsang http://www.opensuse.org Everybody has a plan, until they get hit. - Mike Tyson -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

Matwey V. Kornilov

10:48

ср, 2 окт. 2019 г. в 13:45, Henne Vogelsang :

...

Hey,

On 01.10.19 17:48, Matwey V. Kornilov wrote:

...
I am running private OBS instance on Leap 15.1. I use obs-api-2.10-lp151.10.1.noarch

I see that delayed_job ruby scripts consume about 100MB of RAM each on my setup: ... As far as I understand, the scripts are just delayed job workers, they pop the jobs from the database and execute some stuff then.

In the context of the application. That's the reason they load it into memory.

So, each process has a copy of the same compiled application code in memory? Right?

...

...
Is it possible to tune memory consumption somehow?

Not really no.

Henne

-- Henne Vogelsang http://www.opensuse.org Everybody has a plan, until they get hit. - Mike Tyson

-- With best regards, Matwey V. Kornilov -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

Henne Vogelsang

11:55

Hey, On 02.10.19 12:48, Matwey V. Kornilov wrote:

...

ср, 2 окт. 2019 г. в 13:45, Henne Vogelsang :

...
On 01.10.19 17:48, Matwey V. Kornilov wrote:

...
I am running private OBS instance on Leap 15.1. I use obs-api-2.10-lp151.10.1.noarch

I see that delayed_job ruby scripts consume about 100MB of RAM each on my setup: ... As far as I understand, the scripts are just delayed job workers, they pop the jobs from the database and execute some stuff then.

In the context of the application. That's the reason they load it into memory.

So, each process has a copy of the same compiled application code in memory? Right?

Nothing is compiled, but in general yes. All of them are instances of the OBS Ruby on Rails app. Henne -- Henne Vogelsang http://www.opensuse.org Everybody has a plan, until they get hit. - Mike Tyson -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

Matwey V. Kornilov

12:06

ср, 2 окт. 2019 г. в 14:55, Henne Vogelsang :

...

Hey,

On 02.10.19 12:48, Matwey V. Kornilov wrote:

...
ср, 2 окт. 2019 г. в 13:45, Henne Vogelsang :

...
On 01.10.19 17:48, Matwey V. Kornilov wrote:

...
I am running private OBS instance on Leap 15.1. I use obs-api-2.10-lp151.10.1.noarch

I see that delayed_job ruby scripts consume about 100MB of RAM each on my setup: ... As far as I understand, the scripts are just delayed job workers, they pop the jobs from the database and execute some stuff then.

In the context of the application. That's the reason they load it into memory.

So, each process has a copy of the same compiled application code in memory? Right?

Nothing is compiled, but in general yes. All of them are instances of the OBS Ruby on Rails app.

Thanks. I mean that ruby needs to compile sources into some internal byte-code representation.

...

Henne

-- Henne Vogelsang http://www.opensuse.org Everybody has a plan, until they get hit. - Mike Tyson

-- With best regards, Matwey V. Kornilov -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

Matwey V. Kornilov

13 Oct 13 Oct

09:55

ср, 2 окт. 2019 г. в 13:45, Henne Vogelsang :

...

Hey,

On 01.10.19 17:48, Matwey V. Kornilov wrote:

...
I am running private OBS instance on Leap 15.1. I use obs-api-2.10-lp151.10.1.noarch

I see that delayed_job ruby scripts consume about 100MB of RAM each on my setup: ... As far as I understand, the scripts are just delayed job workers, they pop the jobs from the database and execute some stuff then.

In the context of the application. That's the reason they load it into memory.

...
Is it possible to tune memory consumption somehow?

Not really no.

As far as I understand, currently delayed_job uses a process per queue. In the docs I see that it is allowed to start workers to serve any queue ("--pool=*:2"). So, having a pool of fixed small number of workers serving any jobs would reduce number of processes (and memory consuming). But I am not sure about possible job deadlocks. What do you think?

...

Henne

-- Henne Vogelsang http://www.opensuse.org Everybody has a plan, until they get hit. - Mike Tyson

-- With best regards, Matwey V. Kornilov -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

Henne Vogelsang

14 Oct 14 Oct

10:23

Hey, On 13.10.19 11:55, Matwey V. Kornilov wrote:

...

As far as I understand, currently delayed_job uses a process per queue.

At least one yes. We have one queue (quick) that supports multiple workers.

...

I am not sure about possible job deadlocks.

All the jobs in queues, except "quick", do **not** support parallel jobs and rely on running sequential. Henne -- Henne Vogelsang http://www.opensuse.org Everybody has a plan, until they get hit. - Mike Tyson -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org

1663

Age (days ago)

1676

Last active (days ago)

List overview

Download

6 comments

2 participants

participants (2)

Henne Vogelsang
Matwey V. Kornilov