Dynamic linking of rust libraries
Hi, I have some questions regarding dynamic rust packaging. As most of you probably know, Rust supports building a library crate crate as a shared object (--crate-type dylib). Those objects contain everything needed to build & link against them but can be shared by multiple dependents unlike rust's rlibs that are statically linked into the resulting binary. Additionally invoking rustc with -C prefer-dynamic links the rust std dynamically (/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd- <fingerprint>.so) rather than staticly in the resulting binary and thus its filesize a lot smaller. I was wondering if this could be used to build rust packages, because currently all rust crate packages just provide the sourcecode, and cargo/rustc builds a static binary (only links against glibc). Additionally devel:languages:rust:crates seems kinda dead (last updated package seven months ago), but dynamic linking could offer an advantage over vendoring the dependencies into your package. The downside to dynamic linking in rust is, that if a library or the std gets updated but a dependency didn't it'll break the dependency. However, given that the buildservice will recompile any dependents if a package gets updated, the only case where this could become an issue is in case of a partial update. If anyone wants to experiment with rust's dynamic librarys, heres how I got it to work: 1. Build any library you depend on with --crate-type dylib 2. Drop the library you've just built from the dependencies section 3. Rebuild the project with RUSTFLAGS="-C prefer-dynamic --extern depname=../path/to/built/library/libdepname.so" cargo -v build 4. To run the built binary make sure that the libdepname.so is in your LD search path. Regards, Florian "sp1rit" -- Matrix: @sp1rit:tchncs.de <sp1rit@disroot.org> D248BF2F4C6A82A1E0569D897D8C1CD573166D09 <sp1rit@national.shitposting.agency> BBDE032EAAFBFC627FB7E635B1F4055D8460CE34
On Mon, Jun 7, 2021 at 8:02 AM sp1rit <sp1rit@national.shitposting.agency> wrote:
Hi, I have some questions regarding dynamic rust packaging.
As most of you probably know, Rust supports building a library crate crate as a shared object (--crate-type dylib). Those objects contain everything needed to build & link against them but can be shared by multiple dependents unlike rust's rlibs that are statically linked into the resulting binary. Additionally invoking rustc with -C prefer-dynamic links the rust std dynamically (/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd- <fingerprint>.so) rather than staticly in the resulting binary and thus its filesize a lot smaller.
I was wondering if this could be used to build rust packages, because currently all rust crate packages just provide the sourcecode, and cargo/rustc builds a static binary (only links against glibc). Additionally devel:languages:rust:crates seems kinda dead (last updated package seven months ago), but dynamic linking could offer an advantage over vendoring the dependencies into your package.
It's mostly dead due to Sasi and myself having to work on other things before we can get back to bootstrapping that setup. I hope we can make progress on that someday soon.
The downside to dynamic linking in rust is, that if a library or the std gets updated but a dependency didn't it'll break the dependency. However, given that the buildservice will recompile any dependents if a package gets updated, the only case where this could become an issue is in case of a partial update.
If anyone wants to experiment with rust's dynamic librarys, heres how I got it to work: 1. Build any library you depend on with --crate-type dylib 2. Drop the library you've just built from the dependencies section 3. Rebuild the project with RUSTFLAGS="-C prefer-dynamic --extern depname=../path/to/built/library/libdepname.so" cargo -v build 4. To run the built binary make sure that the libdepname.so is in your LD search path.
Yes, because of OBS auto-rebuilding, it'd be good to be able to do. We probably want a toggle for switching between static and dynamic in the macros for OBS to flip. -- 真実はいつも一つ！/ Always, there's only one truth!
On 7 Jun 2021, at 21:52, sp1rit <sp1rit@national.shitposting.agency> wrote:
Hi, I have some questions regarding dynamic rust packaging.
As most of you probably know, Rust supports building a library crate crate as a shared object (--crate-type dylib). Those objects contain everything needed to build & link against them but can be shared by multiple dependents unlike rust's rlibs that are statically linked into the resulting binary. Additionally invoking rustc with -C prefer-dynamic links the rust std dynamically (/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd- <fingerprint>.so) rather than staticly in the resulting binary and thus its filesize a lot smaller.
I was wondering if this could be used to build rust packages, because currently all rust crate packages just provide the sourcecode, and cargo/rustc builds a static binary (only links against glibc). Additionally devel:languages:rust:crates seems kinda dead (last updated package seven months ago), but dynamic linking could offer an advantage over vendoring the dependencies into your package.
The downside to dynamic linking in rust is, that if a library or the std gets updated but a dependency didn't it'll break the dependency. However, given that the buildservice will recompile any dependents if a package gets updated, the only case where this could become an issue is in case of a partial update.
If anyone wants to experiment with rust's dynamic librarys, heres how I got it to work: 1. Build any library you depend on with --crate-type dylib 2. Drop the library you've just built from the dependencies section 3. Rebuild the project with RUSTFLAGS="-C prefer-dynamic --extern depname=../path/to/built/library/libdepname.so" cargo -v build 4. To run the built binary make sure that the libdepname.so is in your LD search path.
Nope - sadly this isn't possible. dylib's only expose a C interface. Rust isn't able to use this because of monomorphisation of generics. IE when you have a Vec<T>, you only know what T is and how to size that memory lay out at final compilation and compile time. So you can't use a dylib from another rust program, you would need to consume it via it's (unsafe) cffi interface instead, and you lose tons of value. This is why all rust binaries are staticly linked. If you build a dylib, it's intent and purpose is to be consumed by *non* rust code IE C, python, other. So in this way, it's not possible to use dynamic linking. This is why vendoring is currently required because all rust binaries require a full source tree to perform these compiler operations. Hope that helps,
Regards, Florian "sp1rit"
-- Matrix: @sp1rit:tchncs.de
<sp1rit@disroot.org> D248BF2F4C6A82A1E0569D897D8C1CD573166D09 <sp1rit@national.shitposting.agency> BBDE032EAAFBFC627FB7E635B1F4055D8460CE34
— Sincerely, William Brown Senior Software Engineer, 389 Directory Server SUSE Labs, Australia
On 7 Jun 2021, at 21:52, sp1rit <sp1rit@national.shitposting.agency> wrote:
Hi, I have some questions regarding dynamic rust packaging.
As most of you probably know, Rust supports building a library crate crate as a shared object (--crate-type dylib). Those objects contain everything needed to build & link against them but can be shared by multiple dependents unlike rust's rlibs that are statically linked into the resulting binary. Additionally invoking rustc with -C prefer-dynamic links the rust std dynamically (/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd- <fingerprint>.so) rather than staticly in the resulting binary and thus its filesize a lot smaller.
I was wondering if this could be used to build rust packages, because currently all rust crate packages just provide the sourcecode, and cargo/rustc builds a static binary (only links against glibc). Additionally devel:languages:rust:crates seems kinda dead (last updated package seven months ago), but dynamic linking could offer an advantage over vendoring the dependencies into your package.
The downside to dynamic linking in rust is, that if a library or the std gets updated but a dependency didn't it'll break the dependency. However, given that the buildservice will recompile any dependents if a package gets updated, the only case where this could become an issue is in case of a partial update.
If anyone wants to experiment with rust's dynamic librarys, heres how I got it to work: 1. Build any library you depend on with --crate-type dylib 2. Drop the library you've just built from the dependencies section 3. Rebuild the project with RUSTFLAGS="-C prefer-dynamic --extern depname=../path/to/built/library/libdepname.so" cargo -v build 4. To run the built binary make sure that the libdepname.so is in your LD search path.
Nope - sadly this isn't possible. dylib's only expose a C interface. Rust isn't able to use this because of monomorphisation of generics. IE when you have a Vec<T>, you only know what T is and how to size that memory lay out at final compilation and compile time. So you can't use a dylib from another rust program, you would need to consume it via it's (unsafe) cffi interface instead, and you lose tons of value. This is why all rust binaries are staticly linked. If you build a dylib, it's intent and purpose is to be consumed by *non* rust code IE C, python, other. So in this way, it's not possible to use dynamic linking. This is why vendoring is currently required because all rust binaries require a full source tree to perform these compiler operations. Hope that helps,
Regards, Florian "sp1rit"
-- Matrix: @sp1rit:tchncs.de
<sp1rit@disroot.org> D248BF2F4C6A82A1E0569D897D8C1CD573166D09 <sp1rit@national.shitposting.agency> BBDE032EAAFBFC627FB7E635B1F4055D8460CE34
— Sincerely, William Brown Senior Software Engineer, 389 Directory Server SUSE Labs, Australia
participants (3)
-
Neal Gompa -
sp1rit -
William Brown