dlopen many cgo libs #3845
dlopen many cgo libs #3845
Conversation
0x00A5
force-pushed
the
dlopen-many
branch
3 times, most recently
from
d1b0295 to
e5f07a0
Compare
0x00A5
force-pushed
the
dlopen-many
branch
2 times, most recently
from
d758e03 to
584281e
Compare
| go version | ||
| echo "Building Go shared server library..." | ||
| go build -buildmode=c-shared -o "$SERVER_SO_FILE" "$SERVER_GO_FILE" | ||
|
|
||
| echo "Building Go shared file ops library..." | ||
| go build -buildmode=c-shared -o "$FILEOPS_SO_FILE" "$FILEOPS_GO_FILE" |
There was a problem hiding this comment.
Shouldn't we test shared libs from different Go versions? Was that the user's case?
There was a problem hiding this comment.
The user uses same version v1.24.11. I also found that (gdb) info functions Syscall6 only returns one internal/runtime.syscall.Syscall6 despite that this test app load two libs. And it always returns the one from the first lib the app dlopen'd (I flipped the order of my dlopen calls to verify). So, I am not sure what will happen if different go versions are used 💀
| void* handle = dlopen(so_path.c_str(), RTLD_LAZY); | ||
| if (!handle) { | ||
| // These dlopen() flags are provided by the user. | ||
| void* server_so_handle = dlopen(server_so_path.c_str(), RTLD_LAZY | RTLD_NODELETE | RTLD_DEEPBIND); |
There was a problem hiding this comment.
Huh. RTLD_NODELETE makes me thing that maybe they're loading and unloading the same library multiple times 👀
We should test it probably
There was a problem hiding this comment.
Huh. Also RTLD_LAZY looks fishy as hell 🤔
Not sure why it even works with RTLD_LAZY right now. Not sure how exactly layer "hooks" the symbol either (does it overwrite pointer in the symbol table? does it overwrite instructions at the start of the hooked function itself?). But RTLD_LAZY sounds like there are other trampolines and function replacements in the game.
There was a problem hiding this comment.
RTLD_NODELETE
Yes, worth asking the user about this in my call with them next week. dlclose() will unload the go runtime code, maybe they don't want that to happen.
RTLD_LAZY
The one we hook is from static library, thus the symbols are bind already when the go lib is compiled? This flag affects all functions in dynamic library I think.
| go version | ||
| echo "Building Go shared server library..." | ||
| go build -buildmode=c-shared -o "$SERVER_SO_FILE" "$SERVER_GO_FILE" | ||
|
|
||
| echo "Building Go shared file ops library..." | ||
| go build -buildmode=c-shared -o "$FILEOPS_SO_FILE" "$FILEOPS_GO_FILE" |
There was a problem hiding this comment.
The user uses same version v1.24.11. I also found that (gdb) info functions Syscall6 only returns one internal/runtime.syscall.Syscall6 despite that this test app load two libs. And it always returns the one from the first lib the app dlopen'd (I flipped the order of my dlopen calls to verify). So, I am not sure what will happen if different go versions are used 💀
| void* handle = dlopen(so_path.c_str(), RTLD_LAZY); | ||
| if (!handle) { | ||
| // These dlopen() flags are provided by the user. | ||
| void* server_so_handle = dlopen(server_so_path.c_str(), RTLD_LAZY | RTLD_NODELETE | RTLD_DEEPBIND); |
There was a problem hiding this comment.
RTLD_NODELETE
Yes, worth asking the user about this in my call with them next week. dlclose() will unload the go runtime code, maybe they don't want that to happen.
RTLD_LAZY
The one we hook is from static library, thus the symbols are bind already when the go lib is compiled? This flag affects all functions in dynamic library I think.
Added a new cgo lib that does basic file ops. The cpp app now dlopen both cgo libs.
The address needs to seat in the middle of the function so stack frame unwind works normally.
1. Correct warning message about missing main.go files. 2. Use go 1.24 to build test app in CI. 3. Fix wrong comments on assembly code.
CPP main app dlopen CPP shared object that wraps Go c-archive library.
2 participants
This PR implements go detours for go v1.24.*. These detours are used only for Linux amd64 dlopen hook.
The go detours use the same approach in our 1.25 detours, but do not use the
gosave_systemstack_switchfunction resolved from the go binary. Instead, the whole call stack is implemented with Rust naked functions.This approach shall handle edge cases when multiple c-shared go libs are opened.