While working on REL, I learned a lot about Bazel and its usage as build system in open source projects. In a series of blog posts, I will share these learnings and describe different approaches. Today’s blog post addresses the integration of C++-based Python modules into Bazel and the modelling of dependencies towards the corresponding Python-based tests.

As part of REL’s Python integration, I created a cc_library called rel_py which includes the core REL C++ library and the necessary Python binding (using the ingenious pybind11 framework). If the library is built as dynamic library, the resulting librel_py.so file can directly be imported in every Python script via import statement. It took me quite some time, though, to model the dependency between a py_test rule and the mentioned cc_library. My goal was to add an integration test to Bazel, which uses REL within Python, to read a toy model and test the basic functionality, like accessing all type instances, checking the API etc.

The integration test itself is a simple python script, that imports librel_py.so and interacts with the API. I wrapped it into a py_test rule. At the moment, it is not possible, though, to model a dependency (deps) in Bazel from py_test towards cc_library, as Bazel only allows dependencies towards rules from the py_ family. Therefore I tried the data - attribute, which allows specifying arbitrary dependencies, e.g. to test data. Unfortunately, with this approach, I was not able to specify the correct import paths for the Python runtime. During test execution, Python always complained, that the module that shall be imported cannot be found.

After searching on Stackoverflow and the Bazel bugtracker, I finally figured out the following approach, to get the dependencies right: Apparently it is necessary to define a dummy py_library first, which is modeled as dependency within the py_test. The py_library then uses the data attribute to point to a cc_binary rule, which is located in the same folder as the two py-rules, and is actually a copy of the original cc_library. The disadvantage of this solution is definitely, that the Bazel model is partially duplicated. Nevertheless, the obvious advantage, it now works and I can run an integration test via bazel test, that builds the Python binding library/binary of REL and runs a Python script, to test the functionality.

My solution in Bazel can be found here: https://github.com/sscit/rel/blob/main/relpy/test/BUILD

Bazel Bugtracker issues related to this topic, that contain additional details: