On 9/8/21 11:07 AM, Michael Matz wrote:
Hello,
On Wed, 8 Sep 2021, Zebediah Figura wrote:
Well, if you're fearing that, then you should also fear symbol clashes, right? I.e. you not only need to rename the lib basename, but also all exported symbols from your version. (After all, programs could also crawl the export lists). At that point it all becomes cumbersome.
How do deal with that problem currently? Why would that change with the file format of your helper libs? Why should the libfreetype that wine is using for internal purposes be loaded visibly into the process image at all? After all the whole purpose of an emulator is to, well, emulate the target as faithfully as possible, and one of those things would be to provide the target process with only the list of loaded DLLs it would also see on a normal Windows system. I.e. one without freetype or any other libs that you mentioned. The code and data of those libs can of course be loaded if you need them in-target-process, but all the meta data of it should not be readily linked into the normal data structures available to the target process.
Like any problem of emulation, it's a matter of trying to figure out what's "good enough".
True.
And the root of the problem here, what we're concerned about, is that the amount of work required to make dynamic libraries "good enough"—by our estimation—is a lot more than it is for static libraries.
I might see that, but is it really the case? You're saying static libs would work for you. That means static ELF libs linked into the wine executable. They aren't seen by the target PE process then, and whatever makes use of the code in those static libs (i.e. code in wine itself) also directly refers to symbols within them with the link editor resolving those references. I.e. I assume that you don't expose any of those symbols to the target process (would require trampolines and custom loader code when you link statically).
Now, for some reason that I don't see right now you don't want to use dynamic ELF libs anymore. What is that reason? Linking the wine executable(s) dynamically to the system ELF libs that are static above and would be dynamic then (or are already) has exactly the same properties: symbols and lib metadata stays internal to the wine executable and isn't exposed directly to the target process, which is exactly what you should want.
Your solution to this wish of avoiding ELF libs is to instead create and load PE libs into the wine executable. That immediately creates the problem you're worried about, namely namespace pollution for the target process. Now, you also don't want to adjust your PE loader to support something like an internal namespace for those wine-internal libs, because of complexity (I wouldn't call such internal namespace a hack, btw). But it seems all this complexity only comes from your wish of loading PE libs when that doesn't seem necessary. My normal answer to that would be "don't do that then". It also avoids the need to actually create PE libs with a mingw toolchain and the necessary maintenance of everything needed to cross compile those libraries.
Think of it this way: assume for a moment the target process is a different architecture, i.e. that wine wouldn't only be an API emulator but also a CPU emulator, say for aarch64 with wine running on x86_64. Now your idea would be equivalent to compiling libfreetype for aarch64, and then implementing all kinds of complexity within wine to actually run that code on x86_64 just to make use of the facilities provided by libfreetype for wine purposes, instead of just using a libfreetype compiled for x86_64 to start with.
Your current train of thoughts seems to go from a working and natural solution (link against system libs in host format for the host executables) to complex and unnatural solutions (create host code in target file format, but load it into the host program with the target loader but also somewhat hide it from the target process).
So, what, _exactly_ is the reason that you don't want to link against ELF system libraries for the wine executables that are ELF files themself? I clearly must be missing something of the picture to see why going through these hoops even occurred to you :-)
Currently host libraries are loaded in ELF format, and they're stored somewhere in wine internals. win32 code can see them, if they knew how to look for them, but of course they don't. The idea with PE libraries is that, since we of course already have a PE loader, we just use it for the code we can.
But why? You also have an ELF loader that you don't even need to maintain yourself (in ld.so), why would you want to use the target object loader (PE) for the host binaries (the wine code) when you can load host files perfectly fine already?
Right, I omitted the explanation for the sake of simplicity (not that it helped very much, my mail still ended up very long). Let me try to explain in detail; please feel free to ask for further clarification if necessary. The basic reason is that Wine is essentially trying to split itself into "PE" and "Unix" parts. The idea is that all of the "user" code is built in PE format, and only calls into unix code by making a sort of fake "syscall". The "kernel" code is built in ELF format (or whatever the host system loads). There are a few basic reasons for doing this: * Many applications, mostly anti-cheat and anti-tamper engines, depend on our libraries actually being in PE format on disk, and matching that in memory. We used to have "fake" DLLs for this, but they weren't verisimilar enough. This by itself basically only matters for actual Windows libraries (and in practice applications mostly only care about ones they expect to be tampered with, which isn't that many.) However... * Some of the important hosts for Wine have dropped support for 32-bit libraries or are going to drop it. Mac is the obvious example here, but many commercial Linux distributions also want to drop support. By limiting the surface through which code can transition between PE and Unix code, it becomes feasible to do 32-to-64 translation, where previously this was quite infeasible. * There's some demand for running Win32 debuggers under wine. These have always worked more than a little tenuously, but a couple of them are causing problems when they try to break in and unwind from Unix code. By acting like we're making actual syscalls, things work much better. Of the several dozen modules we have that currently call into Unix libraries, about half of them aren't going to use PE dependencies. Most of these have already been fully split into PE and Unix parts, including things like the socket layer and winegstreamer. As you point out, we could just call into the Unix parts for the other half as well. There are a few reasons we don't want to do this: * Making fake syscalls is, like real syscalls, not cheap. It involves a full context switch into and out of Unix code. * Making callbacks from Unix code is difficult, and not cheap either. Some libraries don't do callbacks, and so this doesn't matter, but others do, and it makes things ugly. * Writing the wrappers between Unix and PE code involves some nontrivial work in itself. The smaller the interface is, the easier things are for us. ἔρρωσθε, Zebediah