The venerable master Qc Na was walking with his student, Anton. Hoping to prompt the master into a discussion, Anton said "Master, I have heard that objects are a very good thing - is this true?" Qc Na looked pityingly at his student and replied, "Foolish pupil - objects are merely a poor man's closures."

Chastised, Anton took his leave from his master and returned to his cell, intent on studying closures. He carefully read the entire "Lambda: The Ultimate..." series of papers and its cousins, and implemented a small Scheme interpreter with a closure-based object system. He learned much, and looked forward to informing his master of his progress.

On his next walk with Qc Na, Anton attempted to impress his master by saying "Master, I have diligently studied the matter, and now understand that objects are truly a poor man's closures." Qc Na responded by hitting Anton with his stick, saying "When will you learn? Closures are a poor man's object." At that moment, Anton became enlightened.

Anton van Straaten, 4 Jun 2003

Consequently, I had no choice but to intercept pthread_create to incorporate logic for registering the thread in the garbage collector (GC). Achieving this in a satisfactory manner was a lengthy quest, so I would like to share the end result here. Furthermore, the solution is not specific to pthread_create and can be applied to intercept most system library calls, such as malloc and free.

The calls we aim to intercept originate from shared objects supplied by the system, such as libpthread.so or libc.so. Since the function we intend to intercept lacks an implementation in our program, we can provide one, and it will take precedence over the system's implementation. This is fantastic news, but a significant problem remains: we still need to forward the call to the system's pthread_create so that a new thread will indeed be created.

To achieve this, we manually resolve the pthread_create function using dlsym. To ensure we do not resolve the same method again and again, we use the RTLD_NEXT functionality, which ensures the lookup only happens in subsequent objects.

#include <dlfcn.h>
#include <pthread.h>
#include <stdio.h>

#define RTLD_DEFAULT ((void*)0)
#define RTLD_NEXT ((void*)-1ll)

typedef int (*pthread_create_type)(pthread_t* thread,
                                   const pthread_attr_t* attr,
                                   void* (*start_routine)(void*), void* arg);

int pthread_create(pthread_t* thread, const pthread_attr_t* attr,
                   void* (*start_routine)(void*), void* arg) {
    printf("pthread_create intercepted!");

    pthread_create_type real_pthread_create =
        (pthread_create_type) dlsym(RTLD_NEXT, "pthread_create");
    return real_pthread_create(thread, attr, start_routine, arg);
}

This solution still has a couple of problems. First it does resolve the symbol every time, which can be costly. We could cache the result of the resolution and check the cache before resolving, but we can do better. We are doing an indirect call no matter what, so we might as well take advantage of this and create a trampoline function, that resolves and then update the indirect call directly. While we are at it, we can add some error handling code when the real pthread_create fails to resolve.

#include <dlfcn.h>
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>

typedef int (*pthread_create_type)(pthread_t* thread,
                                   const pthread_attr_t* attr,
                                   void* (*start_routine)(void*), void* arg);

static int resolve_pthread_create(pthread_t* thread,
                                  const pthread_attr_t* attr,
                                  void* (*start_routine)(void*), void* arg);
static pthread_create_type pthread_create_trampoline = resolve_pthread_create;

static int resolve_pthread_create(pthread_t* thread,
                                  const pthread_attr_t* attr,
                                  void* (*start_routine)(void*), void* arg) {
    pthread_create_type real_pthread_create =
        (pthread_create_type) dlsym(RTLD_NEXT, "pthread_create");
    if (!real_pthread_create) {
        printf("Failed to locate pthread_create!");
        exit(1);
    }

    pthread_create_trampoline = real_pthread_create;
    return real_pthread_create(thread, attr, start_routine, arg);
}

int pthread_create(pthread_t* thread, const pthread_attr_t* attr,
                   void* (*start_routine)(void*), void* arg) {
    printf("pthread_create intercepted!");
    return pthread_create_trampoline (thread, attr, start_routine, arg);
}

This is good, and we could stop there, however, there one more major problem: sanitizers. Sanitizers such as asan, tsan, ubsan and so on leverage a similar technique to intercept system library calls. If we stop there, we’d be unable to use them, which would be a major problem. Thanksfully, they also provide a way to work around this: we can resolve __interceptor_function_we_want_to_intercept instead of the function itself. Detailed explainations can be found in the intercept library’s source.

Introducing this two stage resolution, we have something that does pretty much everything we want.

typedef int (*pthread_create_type)(pthread_t* thread,
                                   const pthread_attr_t* attr,
                                   void* (*start_routine)(void*), void* arg);

static int resolve_pthread_create(pthread_t* thread,
                                  const pthread_attr_t* attr,
                                  void* (*start_routine)(void*), void* arg);
static pthread_create_type pthread_create_trampoline = resolve_pthread_create;

static int resolve_pthread_create(pthread_t* thread,
                                  const pthread_attr_t* attr,
                                  void* (*start_routine)(void*), void* arg) {
    pthread_create_type real_pthread_create = (pthread_create_type)
        dlsym(RTLD_DEFAULT, "__interceptor_pthread_create");
    if (real_pthread_create) {
        goto Forward;
    }

    real_pthread_create =
        (pthread_create_type) dlsym(RTLD_NEXT, "pthread_create");
    if (!real_pthread_create) {
        printf("Failed to locate pthread_create!");
        exit(1);
    }

Forward:
    pthread_create_trampoline = real_pthread_create;
    return real_pthread_create(thread, attr, start_routine, arg);
}

int pthread_create(pthread_t* thread, const pthread_attr_t* attr,
                   void* (*start_routine)(void*), void* arg) {
    printf("pthread_create intercepted!");
    return pthread_create_trampoline(thread, attr, start_routine, arg);
}

While this snippet of code is rather short and easy to understand, most of this is undocumented and it took a fair amount of research to figure it out.

If this is interesting or useful to you, let me know.