Mailinglist Archive: opensuse-packaging (182 mails)

< Previous Next >
[opensuse-packaging] Re: --as-needed and pthread
  • From: Michael Matz <matz@xxxxxxx>
  • Date: Fri, 19 Jun 2009 05:29:54 +0200 (CEST)
  • Message-id: <Pine.LNX.4.64.0906190446100.29566@xxxxxxxxxxxxx>
Hi,

[including some random mailing list, so that the text isn't lost :) ]

On Thu, 18 Jun 2009, Cristian Rodríguez wrote:

there are some packages that fail to build because they miss phtread
symbols. I have always have the doubt about why there is the
"-pthread" gcc flag... somewhere I read that's the correct thing to
do, that using -lpthread is wrong, but without explanations. What
should I use in these cases? I suppose use -lpthread would be the
equivalent to what they did without --as-needed... but perhaps the
behavior without --as-needed was already wrong.

matz is the right person to answer your question, as far as I know, the
right way is using -lpthread instead of -pthread , is that correct ?

Nope, -pthread is better than -lpthread (see below). It's completely
possible that something is broken with as-needed vs. pthreads. Those
cases have to be identified and then worked around
(export SUSE_AS_NEEDED=0), or better, analyzed for the root cause.

I didn't get the complete thread, only this forward, so I don't know what
the nature of the breakage is, so I can give only speculations:
I would think that if something with pthread and as-needed breaks it
doesn't get exposed as missing symbols or the like (so that apps don't
even start) but rather mysterious breakages related to non-thread-aware
functions being used even though -pthread was given as compile and link
flag.


[background]

Regarding the explanation of -pthread/-lpthread: Unfortunately the initial
threading support for UNIX was a bit like bolted onto the existing
(non thread-aware) libc. For various reasons it was decided that users
that aren't interested in threading are not supposed to pay the cost of
_potential_ thread safety. This cost is mostly associated with locking
some shared data structures for libc and the process output (e.g. output
of printf() ).

Now, given this criteria, there's not much choice. Basically the user has
to announce to the system if or if not the application is thread aware, so
that it (the system) can chose between thread-aware or non thread-aware
functions. This announcement is done by the -pthread flag. The
difference to the -lpthread flag is that it is not only a link time flag
(saying that the libpthread library has to be put into the dependence
libraries) but also a compile time flag. So for compiling .c to .o files
for a thread aware program it's necessary to include -pthread into the
compile flags. The effect of that option is system dependend. Usually it
will result in the compiler defining some preprocessor symbols
(_REENTRANT) so that system headers can choose between different
implementations of various posix functions (where the API doesn't make a
difference between thread-safe and -unsafe functions) and internal data
structures.

So, for memoizing the difference: -lpthread is a link time only flag,
simply telling the linker to include libpthread into the application.
-pthread (without the "l") is a link time _and compile time flag_.
Because sometimes different code has to be emitted (for the runtime system
including libc) for programs supposed to be thread-safe, this can't be
announced only at the link phase, the compiler has to know about this.

[more magic]

Now, for linux there's a bit magic involved. To make the cost for non
thread-safe programs really low some routines in libc _are not
thread-safe_. That's fine, those programs aren't linked against
libpthread, so they have announced to be not interested in threads.
libpthread.so provides thread-aware variants of some functions that are
also provided by libc itself. For instance open() and friends is such
function. The libpthread variant includes some locking for shared data
structures.

(Apart from these overriding functions it also includes the usual pthread
functions, pthread_*, e.g. pthread_spin_lock).

How this all works is because ELF symbols resolution rules say so. If an
application calls "open" and is linked against libpthread and libc (which
both provide a version of "open", one thread-aware the other not) then the
dynamic linker will chose the first it finds. Other things (gcc .spec
files) make sure that libpthread is linked in _before_ libc when -pthread
is given. This can't be ensured for -lpthread, the latter being simply a
request for a random other library, as far as the system knows. So only
-pthread, not -lpthread, makes really sure that libpthread is linked at
the right place vs. libc, so that it can override the non thread-aware
functions of it.

[less magic]

Having said all this, it usually is just fine to simply link against
libpthread (via -lpthread) with linux, as long as you don't mention -lc
before. This is because most headers don't care about the _REENTRANT
preprocessor define, i.e. they don't change their content depending on the
flag, and because ELF rules will still make the functions from libpthread
be preferred. But for cross-platform compatibility it's important to use
-pthread, _not_ -lpthread. And as it's important for others, and at least
good under linux, you can just as well use it always if you need.


Ciao,
Michael.
< Previous Next >