luajit - Lua yielding across C-call boundary

Question

I'm trying to call lua_yield inside a debug hook, and get this error in my output. I'm wanting to yield after a certain number of instructions have been processed and was hoping this was the way to do it.

I'm writing this using some Python ctypes bindings.

yielding
b'test.lua:1: attempt to yield across C-call boundary'

I assumed this should work since I'm using LuaJIT and it has a fully resumable VM.

@lua_Hook
def l_dbg_count(L: lua_State_p, ar: ctypes.POINTER(lua_Debug)):
    if ar.contents.event == EventCode.HookCount:
        print("yielding")
        lua_yield(L, 0)

#main method
def main():
    ...
    lua_sethook(L, l_dbg_count, DebugEventMask.Count, 1)
    luaL_loadfile(L, b"test.lua")
    ret = lua_pcall(L, 0, 0, 0)
    while True:
        if ret != LuaError.Ok and ret != LuaError.Yield:
            print(lua_tostring(L, -1))
            break
        elif ret == LuaError.Yield:
            print("resuming")
            ret = lua_resume(L, None, 0)
    lua_close(L)

Answer 1

I first must push a new thread using lua_newthread, then calling luaL_loadfile and instead of lua_pcall, calling lua_resume.

I rewrote this in C to check if there was possible stack unwinding issues from Lua to Python.

void l_dbg_count(lua_State *L, lua_Debug *ar) {
    if(ar->event == LUA_HOOKCOUNT) {
        printf("yielding
");
        lua_yield(L, 0);
    }
}

...

int main(int argc, char **argv) {
    lua_State *L = luaL_newstate();
    luaL_openlibs(L);
    lua_sethook(L, l_dbg_count, LUA_MASKCOUNT, 5);

    lua_State *L_t = lua_newthread(L);
    luaL_loadfile(L_t, "test.lua");
    int ret = lua_resume(L_t, 0);
    while(true) {
        if(ret != 0 && ret != LUA_YIELD) {
            fprintf(stderr, "%s", lua_tostring(L_t, -1));
            break;
        } else if(ret == LUA_YIELD) {
            printf("resuming
");
            ret = lua_resume(L_t, 0);
        } else {
            break;
        }
    }

    lua_close(L);
    return EXIT_SUCCESS;
}

This however does break the coroutine library from working it seems, so currently looking into a possible fix for that.