Created attachment 828734 [details]
MWE for testing multithreading performance.

I am trying out threading performance in C++/OpenMP and Julia.

On a 6 core 32GB RAM Dell Precision 7540 laptop with Tumbleweed (kernel
5.4.10), multithreading speed up unexpectedly significantly decreases with
problem size.

I attach the test program to be compiled with

   g++ -Ofast -fopenmp threads.cxx -o threads

Usage is 

   threads N nrepeat

Problem size aka array length aka loop range size can be given as the command
line parameter N. nrepeat is the number of repeats of the measurment (to allow
watching CPU usage)

The typical output is as follows:
$threads 100000 1
  nthreads=4
  scalar=0.004519, para=0.001273, speedup=3.550611
I get 400% CPU usage in top (when setting nrepeat to 100000)

$threads 100000 1
  nthreads=4
  scalar=0.056637, para=0.046533, speedup=1.217147

In this case CPU usage in top is limited to 266% 

I observe this behavior with
 g++-10
 g++-7
 julia 1.3.1 (with a similar test program)

I don't observe this behavior on a compute server with Leap 5.10, kernel
4.12.14 and  g++-7:

$threads 100000 1
  nthreads=4
  scalar=0.008033, para=0.002451, speedup=3.278010

$threads 1000000 1
  nthreads=4
  scalar=0.069559, para=0.021247, speedup=3.273745


When I copy the executable from the server to the laptop I get the same results
as for the executable compiled on the laptop. I suspect this to be a problem
with thread scheduling, memory management or something related.
And yes, mitigations are already off, and hyperthreading is on (though I don't
use it here).

J�rgen