ncnn_mp — ncnn C API bindings for MicroPython

[Willaaaaaaa]

Repository: ncnn_mp

Brief Introduction

What is ncnn_mp

ncnn_mp is an external C module developed for the MicroPython environment, designed to port Tencent's high-performance neural network inference framework, ncnn, to resource-constrained microcontrollers (MCUs).

Why ncnn_mp

Although there exist Python bindings for ncnn, ncnn_mp specifically targets MicroPython and MCUs.
By encapsulating the ncnn C API in an object-oriented way, ncnn_mp enables developers to use ncnn features with a natural and Pythonic API on embedded platforms.

How it works

• The workflow from source code to firmware:
  Select the target platform (Unix, ESP32 ...)
  -> micropython.cmake or micropython.mk
  -> Compile the ncnn_mp.c source code (external C module)
  -> ncnn_mp.c calls the ncnn C apis through c_api.h
  -> A MicroPython firmware containing the ncnn_mp module is generated
• core file ncnn_mp.c:
  1. Register ncnn_mp module
  2. Define module contents (classes, mothods ...)
  3. Defining a Python class: Taking Allocator as an Example
  4. Implement attribute handling, constructors, destructors, and other methods within classes.

Difficulties

Overview

• API Design: Convert ncnn's native, procedural, C-style API (handle-based) to a object-oriented, Pythonic API. This requires to deal with error handling, resource management, and data structures.
• Development Experience: Beyond core functionality, efforts were made in code style, the .pyi stub file, and CI workflows.
• Docs Reading: MicroPython's documentation is quite concise, and many apis and internal behaviours aren't fully documented, which means reading through the source code, existing modules, and examples is necessary.

Practice

1. Converting from C-style handles to Python objects

• Description: ncnn exposes handles and procedural APIs. The challenge here is to expose these constructs as Python objects with natural constructors, methods, and automatic resource management.
• Solution: Implement a MicroPython custom type for each core ncnn structure: Allocator, Option, Mat, Blob, ParamDict, DataReader, ModelBin, Layer, Net, and Extractor. Each C struct in ncnn_mp.c contains a pointer to the corresponding ncnn C object. This object-oriented approach automates resource management by MicroPython, which automatically calls the __del__ method to safely release the underlying ncnn resources.

2. Resolving the Unexpected MicroPython Instance Attribute Lookup Routine

• Description: Defining dynamic computed attributes (for example, Mat.w and Mat.h) together with regular methods (for example, Mat.fill()) led to unexpected AttributeError when accessing methods from instances. The behaviour was inconsistent: direct function-style calls (Mat.fill(a, ...)) worked, while a.fill() raised an error.
• Solution: MicroPython's attribute lookup gives priority to a type's attr function. If attr returns a value or indicates the attribute is handled, lookup stops. If attr does not find the attribute, it must explicitly signal that lookup should continue to the instance's locals dict. The fix was to modify the attr function so that when a requested attribute is not one of the dynamic attributes it recognizes, it sets dest[0] = MP_OBJ_NULL to allow normal lookup in the locals_dict. After this change, both dynamic attributes and normal method lookups behave correctly.

3. Improving the Python Development Experience

• Description: After initial code writing, the C module was functional but provided a poor developer experience. Without support for modern IDE features, developers had no access to autocompletion, type hints, or inline documentation, making the API difficult to use.
• Solution: A .pyi stub file was written manually. This file provides complete type definitions and docstrings for all classes and methods, enabling full support for static analysis, IntelliSense, and parameter information in modern editors like VS Code.

4. Bridging Python binary data and ncnn Mat structures

• Description: ncnn expects tensor/matrix data in native memory formats, whereas Python commonly uses bytes and bytearray for binary data. There was no native C API to directly construct a Mat from a continuous Python buffer.
• Solution:
  • Implemented Mat.from_bytes() to populate an existing an ncnn Mat from a bytes-like object. The implementation validates buffer size against w * h * c * sizeof(element) and copies data into Mat’s memory. On mismatch, a ValueError is raised.
  • Implemented Mat.to_bytes() to return a byte object containing data. The method allocates a Python-managed buffer, copies data into it, and returns it to the caller. Memory ownership and lifetime are handled by MicroPython’s GC to avoid leaks.

5. Other problems&solutions

• See issues and prs in my repo.

Innovations

• Pythonic API design
• Provide real-world application examples as a guide (such as mnist in ./example)
• Developer experience focus (provide a .pyi file and docstrings)
• Cross-platform CI
• Practical extensions

Further work

Update on 9.4: All is done now.
See

• ncnn_mp Modification Roadmap Willaaaaaaa/ncnn_mp#1
• Support for Custom Layers Defined in Python Willaaaaaaa/ncnn_mp#4