Pycardium Module Development¶

This is a step-by-step guide on augmenting Pycardium with new modules. These module are either written in Python or in C:

Python Modules: Python modules live in pycardium/modules/py and are pre-compiled and packed into Pycardium at build-time. Python modules are meant to be used for creating abstractions and helpers based on lower level functionality. The rationale is that it is easier to write pythonic modules in python instead of having to wrap all that in C code. Though care has to be taken as these modules can eat quite a lot of space.
C Modules: C modules allow MicroPython code to interface with the rest of the system or allow a much faster implementation of performance-critical code. For interfacing with card10, these modules are mostly meant to make use of the Epicardium API.

Python Modules¶

Adding a new Python module¶

Add a file in pycardium/modules/py. The name of the file dictates the name of the module. Next, add you file to the list in pycardium/modules/py/meson.build to make the build-system aware of it. Rebuild, and you should see your module pop up in the list shown by

help("modules")

Size Consideration¶

Python modules can get quite big. If you want to know exact sizes, take a look at the output of

ls -lh build/pycardium/*@@frozen.c@*/

If your module contains some kind of big lookup-table or data-block, consider pushing that into external flash. You can then read the data using standard Python open() and if you need to decode it, use the ustruct or ujson module.

Creating a new C module¶

Module Source¶

Create a new file in pycardium/modules. The basic structure of a C module looks like this:

#include "py/obj.h"

/* Define a function in this module */
static mp_obj_t mp_example_hello(mp_obj_t number)
{
        int num = mp_obj_get_int(number);
        printf("Hello from example: %d\r\n", num);
        return mp_const_none;
}
/* Create an object for this function */
static MP_DEFINE_CONST_FUN_OBJ_1(example_hello_obj, mp_example_hello);

/* The globals table for this module */
static const mp_rom_map_elem_t example_module_globals_table[] = {
        {MP_ROM_QSTR(MP_QSTR___name__), MP_ROM_QSTR(MP_QSTR_example)},
        {MP_ROM_QSTR(MP_QSTR_hello), MP_ROM_PTR(&example_hello_obj)},
};
static MP_DEFINE_CONST_DICT(example_module_globals, example_module_globals_table);

const mp_obj_module_t example_module = {
        .base    = {&mp_type_module},
        .globals = (mp_obj_dict_t*)&example_module_globals,
};

/* This is a special macro that will make MicroPython aware of this module */
/* clang-format off */
MP_REGISTER_MODULE(MP_QSTR_example, example_module, MODULE_EXAMPLE_ENABLED);

Note

Please keep the /* clang-format off */. It ensures that clang-format won’t touch the following line. This is necessary because MicroPython’s codegen scripts can’d deal with newlines in the MP_REGISTER_MODULE macro.

Build Integration¶

Next, you need to add your module to pycardium/meson.build. There is a list of module sources at the very top called modsrc where you need to add your file (e.g. modules/example.c).

QSTR Definitions¶

If you now run ninja -C build/, you will hit a few errors regarding missing QSTR definitions. With the example module above, they will look similar to this:

../pycardium/modules/example.c:15:46: error: 'MP_QSTR_example' undeclared here (not in a function)
   15 |  {MP_ROM_QSTR(MP_QSTR___name__), MP_ROM_QSTR(MP_QSTR_example)},

To fix these errors, you need to add all QSTRs your module needs to pycardium/modules/qstrdefs.h. Add a section for your module where you define all QSTRs you need:

/* example */
Q(example)
Q(hello)

Each Q(...) will define into a corresponding MP_QSTR_.... So Q(example) corresponds to MP_QSTR_example.

Enable Module¶

The last step is to actually enable inclusion of your module into the firmware. Do this by adding a define in pycardium/mpconfigport.h:

#define MODULE_EXAMPLE_ENABLED              (1)

The name of the define is the one from the last line in the module source above.

Wrapping Epicardium API¶

Most modules will probably make use of the Epicardium API. Doing so does not require any extra work, you can just call the API from your module code. You should check the input that your module got from MicroPython before sending data off to Epicardium. For example, raise a ValueError if an integer is too big to fit into the type specified by the API. You should also gracefully handle errors returned by API calls. As most API calls use errno codes, you can just wrap them in an OSError:

int ret = epic_bma_get_accel(&values);

if (ret < 0) {
       mp_raise_OSError(-ret);
}

QSTRs¶

QSTRs are so called “interned strings”. This means they are not allocated like normal python objects but instead live in flash and are indexed. This allow MicroPython to very efficiently use them as identifiers. According to them, comparing two QSTR is as fast as comparing integers.

Unfortunately, the way these QSTRs are collected from the source files is quite weird. MicroPython comes with a few python scripts (namely makeqstrdefs.py and makeqstrdata.py) that parse the C source files and search for uses of MP_QSTR_*. These are then sorted and indexed into a header file called qstrdefs.collected.h. This is closely tied in with their Makefiles.

As we use our own build system, we had to somehow wrap this generation to work for us as well. This is done using a few scripts in lib/micropython. Currently, our build system does not parse the module sources to search for QSTRs. Instead all QSTRs needed by modules need to be defined in the header pycardium/modules/qstrdefs.h.

Functions for MicroPython¶

As shown in the example above, you can create functions that can be called from MicroPython code. These functions always have one of the following signatures. To create a MicroPython object for a function, you need the macro call shown after each signature. Please place these calls directly after the function body as shown above.

/* Function with 0 arguments */
mp_obj_t mp_example_fun0(void);
static MP_DEFINE_CONST_FUN_OBJ_0(example_fun0_obj, mp_example_fun0);

/* Function with 1 argument */
mp_obj_t mp_example_fun1(mp_obj_t arg0_in);
static MP_DEFINE_CONST_FUN_OBJ_1(example_fun1_obj, mp_example_fun0);

/* Function with 2 arguments */
mp_obj_t mp_example_fun2(mp_obj_t arg0_in, mp_obj_t arg1_in);
static MP_DEFINE_CONST_FUN_OBJ_2(example_fun2_obj, mp_example_fun0);

/* Function with 3 arguments */
mp_obj_t mp_example_fun3(mp_obj_t arg0_in, mp_obj_t arg1_in, mp_obj_t arg2_in);
static MP_DEFINE_CONST_FUN_OBJ_3(example_fun3_obj, mp_example_fun0);

/* Function with 4 or more arguments */
mp_obj_t mp_example_fun4(size_t n_args, mp_obj_t *args);
static MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN(example_fun4_obj, 4, 4, mp_example_fun4);

For functions with 4 or more arguments, you need to use the MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN macro and instead of directly accessing the arguments, you get an array. The macro gets two numbers (they are the same in the example above): The minimum and maximum number of arguments.

MicroPython Objects¶

TL;DR: Look at lib/micropython/micropython/py/obj.h. It contains most functions needed to create, access, and modify MicroPython objects.

For C modules to interface with MicroPython, you need to be able to interface with MicroPython objects. The generic type of an object is mp_obj_t. As you can see in the example above, this is also what a function gets as arguments and returns back to MicroPython. You can cast mp_obj_t to concrete object types if you made sure it is actually the correct type.

Booleans¶

True and False are const objects, called mp_const_true and mp_const_false respectively. A usual equality check can be used:

#include "py/obj.h"
#include "py/runtime.h"

if (bool_obj == mp_const_true) {
    /* is a boolean true */
} else if (bool_obj == mp_const_false) {
    /* is a boolean false */
} else {
    mp_raise_TypeError("arg 0 is not a boolean");
}

Integers¶

As long as your integers stay within 31-bit limits, integers are stored very efficiently and can be accessed and created like this:

#include "py/obj.h"

/* Create a new integer which is < 2^31 */
mp_obj_t int_obj = MP_OBJ_NEW_SMALL_INT(0xc0ffee);

/* Check if an integer is small and if so, extract it */
if (mp_obj_is_small_int(int_obj)) {
        int int_value = MP_OBJ_SMALL_INT_VALUE(int_obj);
}

For bigger integers or if you are uncertain about the limits, use the following functions:

#include "py/obj.h"

/* Create new integer objects in various sizes and signedness */
mp_obj_t int0_obj = mp_obj_new_int((mp_int_t)value);
mp_obj_t int1_obj = mp_obj_new_int_from_uint((mp_uint_t)value);
mp_obj_t int2_obj = mp_obj_new_int_from_ll((long long)value);
mp_obj_t int3_obj = mp_obj_new_int_from_ull((unsigned long long)value);

/* Check if a value is an integer */
if (mp_obj_is_integer(int_obj)) {

        /* Get an integer */
        int int0 = mp_obj_get_int(int_obj);
        int int0 = mp_obj_get_int_truncated(int_obj);
}

int value;
if (!mp_obj_get_int_maybe(int_obj, &value)) {
   /* Not an integer! */
}

To get really big integers you have to use

#include "py/obj.h"
#include "py/objint.h"

long long value;
mp_obj_int_to_bytes_impl(int_obj, 8, (byte*)&value);

Strings¶

As in CPython, MicroPython also has multiple string representations: There is str, bytes, and bytearray but also the above mentions QSTRs. Ideally, code should work with as many of these as possible and to ensure this, please use these generic functions:

#include "py/obj.h"

/* Create a new string object */
char buf[] = "Hello MicroPython!";
mp_obj_t str_obj = mp_obj_new_str(buf, sizeof(buf));

/* Check if an object is a string */
if (mp_obj_is_str(str_obj)) {
        /* Either str or QSTR */
}
if (mp_obj_is_str_or_bytes) {
        /* Either str, QSTR, or bytes */
}

/*
 * Get a char array from a string object.
 * CAUTION! This string is not necessarily null terminated!
 */
char *str_data;
size_t str_len;
str_data = mp_obj_str_get_data(str_obj, &str_len);

Lists & Tuples¶

While lists and tuples can be accessed using type specific functions, you should refrain from doing so: Use generic indexing functions instead, as they also allow types derived from lists or tuples or types with custom indexing implementations (i.e. duck-typing).

#include "py/obj.h"
#include "py/runtime.h"

/* Get the length of a list-like object */
mp_int_t len = mp_obj_get_int(mp_obj_len(list_obj));

/* Get element at a specific index */
mp_obj_t elem5 = mp_obj_subscr(
        list_obj,
        MP_OBJ_NEW_SMALL_INT(5),
        MP_OBJ_SENTINEL
);

To create a list or tuple:

mp_obj_t values[] = {
        MP_OBJ_NEW_SMALL_INT(0xde),
        MP_OBJ_NEW_SMALL_INT(0xad),
        MP_OBJ_NEW_SMALL_INT(0xbe),
        MP_OBJ_NEW_SMALL_INT(0xef),
};

/* Create a tuple */
mp_obj_t tuple = mp_obj_new_tuple(4, values);

/* Create a list */
mp_obj_t list = mp_obj_new_list(4, values);