Skip to content

Conversation

@pborges
Copy link

@pborges pborges commented Oct 29, 2025

Add Custom Module Loader Support

Summary

Exposes QuickJS's JS_SetModuleLoaderFunc to Go, enabling custom module loading from memory, HTTP, databases, or any Go-accessible source.

Motivation

Previously, the library only supported loading ES6 modules from the file system. This limitation prevented:

  • Loading virtual modules from memory
  • Fetching modules from HTTP/network sources
  • Loading modules from databases or embedded resources
  • Implementing custom module resolution logic
  • Controlling module access for security/sandboxing

Changes

New API

Runtime Method:

func (r *Runtime) SetModuleLoaderFunc(loaderFunc ModuleLoaderFunc)

Loader Function Signature:

type ModuleLoaderFunc func(ctx *Context, moduleName string) (string, error)

Return Values:

  • (source, nil) - Compile and load the provided source code
  • ("", error) - Throw error as JavaScript exception
  • ("", nil) - Fall back to default file system loader

Example Usage

Virtual Modules from Memory:

modules := map[string]string{
    "math-utils": `export function add(a, b) { return a + b; }`,
    "config": `export const API_URL = 'https://api.example.com';`,
}

rt.SetModuleLoaderFunc(func(ctx *qjs.Context, moduleName string) (string, error) {
    if source, ok := modules[moduleName]; ok {
        return source, nil  // Load from memory
    }
    return "", nil  // Fall back to file system
})

Access Control:

rt.SetModuleLoaderFunc(func(ctx *qjs.Context, moduleName string) (string, error) {
    if !isAllowed(moduleName) {
        return "", fmt.Errorf("module '%s' not allowed", moduleName)
    }
    return "", nil  // Load from file system
})

Implementation Details

  1. C Layer (qjswasm/module_loader.c):
    - GoModuleLoaderProxy - C callback registered with QuickJS
    - QJS_SetModuleLoaderCallback - Sets up the loader with callback ID
    - Uses QuickJS's opaque pointer pattern (idiomatic C)
  2. Go Layer (proxy.go):
    - JsModuleLoaderProxy - WASM host function that bridges to Go
    - createModuleLoaderProxyWithRegistry - Creates the proxy with registry
    - readStringFromWasmMem - Utility to read C strings from WASM memory
  3. API Layer (runtime.go):
    - SetModuleLoaderFunc - Public API method
    - Registers Go function in proxy registry
    - Calls C layer to set up the loader

@pborges
Copy link
Author

pborges commented Oct 29, 2025

just wanted to start by saying I'm super impressed with this architecture, I hate dynamic dependencies, wrapping quickjs in WASM is so freaking cool.

I've been working on a side project that involves having multiple JavaScript runtimes that need to communicate and call each other. Because of this, I needed a way to resolve module imports dynamically at runtime.

This PR is my initial attempt to introduce the underlying changes needed to enable that custom resolution logic. I'd love to contribute if this feature is something you think would be useful to others.

I'm not totally sure if I've followed all the existing idioms and architectural patterns you're forming in the library yet, but am very open to any suggestions you may have.

Thank you!

@ngocphuongnb
Copy link
Contributor

@pborges Hey! Thanks so much for putting this together!
Being able to load modules from different places (not just files) is a great idea, opens up a lot of cool possibilities.
I'll take a closer look soon. Really appreciate you working on this!

@codecov
Copy link

codecov bot commented Oct 30, 2025

Codecov Report

❌ Patch coverage is 66.66667% with 18 lines in your changes missing coverage. Please review.
✅ Project coverage is 99.13%. Comparing base (09df7fa) to head (77ca2ae).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
proxy.go 69.04% 7 Missing and 6 partials ⚠️
context.go 0.00% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master      #26      +/-   ##
==========================================
- Coverage   99.75%   99.13%   -0.62%     
==========================================
  Files          15       15              
  Lines        2831     2885      +54     
==========================================
+ Hits         2824     2860      +36     
- Misses          3       15      +12     
- Partials        4       10       +6     
Files with missing lines Coverage Δ
runtime.go 99.06% <100.00%> (+0.03%) ⬆️
context.go 97.32% <0.00%> (-2.68%) ⬇️
proxy.go 91.44% <69.04%> (-8.56%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@pborges
Copy link
Author

pborges commented Oct 30, 2025

Of course! If you want me to do some more work on it let me know.

@ngocphuongnb ngocphuongnb marked this pull request as ready for review November 3, 2025 09:50
@ngocphuongnb
Copy link
Contributor

@pborges I see there's a lint CI error that needs to be fixed first. Once that's sorted out, we can get it merged.

@pborges
Copy link
Author

pborges commented Nov 4, 2025

@ngocphuongnb

@pborges
Copy link
Author

pborges commented Nov 4, 2025

I guess I am confused at this point the only linter errors I see are in unchanged files :/

@ngocphuongnb
Copy link
Contributor

@pborges Got it, I'll go ahead and merge it and try fixing the lint error on my side.
Thanks a lot for contributing!

@pborges
Copy link
Author

pborges commented Nov 5, 2025

Thanks! I think I took care of the linter errors (even the ones not in the affected files)

Not 100% why the linter was concerned with those files though

@pborges pborges marked this pull request as draft November 7, 2025 17:12
@pborges
Copy link
Author

pborges commented Nov 7, 2025

I am not sure retuning a string is the best bet... I am experimenting with returning a qjs.Value for the module which might make more sense, I wouldn't bother merging this, might be a code smell

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants