Optimize Build System for Local Development

[arboleya]

A thousand years ago, we had a local environment that did not require building packages for dev/test purposes. This neat setup was introduced by @pedronauck even before I joined Fuel; it was fast, easy, and a bliss to work with. After some time, we had so many problems with CJS/ESM incompatibilities, especially when using all the myriad different bundlers out there, that it became impossible to keep it that way and our sanity simultaneously.

That was a sad day. We exchanged development speed for better compatibility, which means we got a more solid product at a high price: a slow development environment. At first glance, it was worth it. Problem solved.

Fast-forward to now. Over two years have passed, and the repo has grown exponentially regarding LOC, docs, and different test strategies. Now, it is impossible to do anything real without rebuilding the entire repo. Every. Single. Time. Without mentioning local caching issues and problems alike. Something that initially seemed tolerable for a noble reason (compatibility) has become unbearable. We need a change.

A few months back, I started exploring hybrid solutions to tackle this issue — could we regain development speed while maintaining compatibility? I did a PoC, and it worked. Such a result gave me hope, but priorities were different with all the optimizations we had going on, and I couldn't pursue it until the end.

This issue is so that I can soon resurrect that PoC and try revamping our build system one final time.

Unlike the first setup that only worked without building packages, this final revision must be composed and usable in three different modes:

1. Local Env without build step: instantaneous development
2. Local Env with build step: simulate CI locally
3. CI Env with build step (mandatory!): our compatibility gatekeeper

Now that the team is very lean, we need this more than ever.

This should speed things up with no downsides whatsoever.

Wish me luck.

[MFarabi619]

@arboleya I'm the maintainer for Mira, and would be interested in helping look into this. I've worked with Turborepo and Nx thus far. Briefly explored Moonrepo and Bazel on the side. Always interested in learning more.

[arboleya]

Hey @MFarabi619, this was already ongoing on my end!

I've put a PoC here if you're interested:

• https://github.com/arboleya/acme-monorepo

Bun and Biome are optional.

The trick is around two parts:

1. The publiConfigs property for package.json files
2. The drypub script

We will try applying this concept to this repo in the next few days.