On Tech Stacks
I just merged (my own) pull request swapping out the Go-based backend with a replacement written in Rust in SilverBullet. Similarly, a few months earlier I replaced the frontend build process that used to run through Deno to Node.js.
What the hell am I doing?
I think this warrants a little explanation, and it’s also an opportunity to discuss how I think about deciding on technology stacks in general — even beyond SilverBullet. As always, nothing happens in a vacuum.
Let’s start with the Deno to Node.js swap-a-roo.
Node to Deno and back again
The Deno to Node transition is a transition that happened a few releases back.
The very first prototypes of SilverBullet were built using Node as backend, and built the frontend with Node-tooling as well. Around this time, I had this killer idea of allowing for a sandboxed plugin mechanism where plugins could transparently either be run in the browser (in an iframe or webworker) or on the server. I managed to get this to work on the server somewhat with Node, but it was clear that Node was not really designed with server-side sandboxing support in mind.
Deno was a promising, more modern take on the JavaScript server runtime that was about to hit 1.0-ish around this time, and I liked it a lot. It was effectively Node “done right” from the OG Node author (Ryan Dahl) 10 years later, and their CTO (Bert) used to be a colleague back at Cloud9 IDE. And all built on Rust! Deno offers some sort of limited permission support for workers out of the box, which was exactly what I needed. Great!
And for many years it was a sensible choice. However, it’s also been a source of distraction. The Deno team moves fast and sometimes breaks things, and over the years I’ve had to report various regressions. Admittedly, they were quickly fixed, but still a distraction. As a Deno user, I prefer to spend less of my time keeping up with the thinking of the day on how imports and package management should work, and Deno changed tactics quite a lot on package management (HTTP imports, import maps, JSR, varying levels of supporting npm) that resulted in churn trying to just stay things up to date with the latest best practices. Then there were a bunch of cool features like Deno KV that started out promising, but were not very deeply developed. Sometimes seemingly abandoned.
In the mean time, time dragged on, and a few relevant developments had taken place:
Node had woken up, and started to catch up on support of various browser APIs (probably under pressure from Deno and Bun), like fetch and others, and there may be typescript(ish) support there at some point.
The other development is that as of SilverBullet v2, the need for a back-end in SilverBullet had become almost become symbolic. It effectively became a dumb data store. Case in point: I rewrote the (almost) entirity of it from Deno to Go probably in a week or so (and now again to Rust — spoiler alert, although this one took a bit longer).
In terms of maturity of toolchains, there’s a difference. Deno comes with a bunch of nice things out of the box (like fmt, lint etc.) but the Node ecosystem is richer. Given that I only really have the client (frontend) to care about now, SilverBullet gets effectively no value out anything specific that Deno offers. However, it is a barrier to entry in terms of onboarding new people, because a lot more people know node/npm than know Deno (as similar as they may be).
That’s why a year ago or so I decided I’d slowly step away from the Deno dependency and a release or two back managed to swap out the Deno dependency for a Node.js one.
Since this happend, I got few complaints, I do not think people have really noticed the change. However, it has enabled more easily integrate various “node.js ecosystem” tools like biome for code formatting and more advanced linting, and also unlocked finally building up a Playwright based set of end-to-end tests.
I still absolutely believe Deno is the better implementation of a JavaScript runtime. It gets a lot of things right, more so than Node. However, for SilverBullet’s use it’s not betterer enough to make the unfamiliarity and lack of tooling/ecosystem worth it. As an aside, it’s not clear to me what the long term future of Deno looks like. Its competitor Bun recently got sold to Anthropic and has started doing interesting things like — I don’t know — rewrite its entire code base from Zig to Rust in one fell swoop. I mean, who does a thing like that?
Gettin’ Rusty
Maybe half a year ago I started the development of SilverBullet+, SilverBullet.md’s desktop BFF. Whereas most of your “desktop apps” are likely wrapped web-apps built on Electron (each weighing in at about half a gigabyte of diskspace as that same amount in memory use), I wanted to take a more light-weight route for SilverBullet. RAM and disk space is getting expensive lately, and we all need to do our part to not waste it too much.
Enter Tauri.
Rather than bundling an entire Chrome instance with every single app, Tauri apps leverage whatever browser rendering engine already ships with your operating system. This means WebKit on macOS and Linux, and (Edge’s) Chrome on Windows. The result: significantly smaller app bundles and less memory waste. Great!
True, I now still need to deal with Webkit vs Chrome incompatibilities, but this was already part of my day-to-day life given a lot of people are running SilverBullet.md through Safari (Webkit) anyway. A lot of obscure Webkit-specific bugs got ironed out as a result (don’t ask 🤦).
One more catch: whereas Electron is an ecosystem that is more or less entirely driven through JavaScript, with Tauri apps you are gently (or less gently) pushed to write a bunch of plumbing code in Rust. Luckily, I’m a recovering programming language designer. I’m happy to pick up unfamiliar languages and had attempting adopting Rust a bit in the past, but never really had a valid excuse to dive too deeply into it. However, I decided to do this for the purposes of shipping SilverBullet+ with Tauri. And science. You’re welcome, world.
As part of this effort, and not wanting to ship a Frankenstein combination of TypeScript compiled and wrapped in a Go binary in turn wrapped in a Rust binary, I ended up porting a large chunk of the SilverBullet backend to Rust as well. Luckily this is doable, because SilverBullet’s HTTP server basically primarily lists, reads and writes files to disk. Almost all (interesting) application logic runs inside the browser engine, courtesey of my v2 pivot (which also made Deno redundant).
While achieving the mission, it has become a bit of a pain to maintain two backends: a Go version for SilverBullet.md and a Rust version for SB+. My build process now involves a mix of node.js, Go and Rust.
To simplify my life once more, a backport of the backend from SB+ into SilverBullet.md started to make sense and the result of that is now merged — the next release will likely ship it.
What does this means for you as a user?
From an end-user perspective likely not much. For now.
Unless you’re a resource minner [sic] I suppose — somebody obsessed with reducing resource usage in a world that seems to trend towards maxxing everything (looks, tokens).
The Rust server binaries got smaller (by a few MB), and yes, there’s one fewer garbage collector running (Go had one, Rust doesn’t) on your server now. Likely memory use is slightly less too.
Is resource minning an actual thing? I don’t know, I just invented it. Maybe it should be.
Jokes aside, what I enjoy in this whole 4 year SilverBullet journey so far is that every technology or architectural change tends to have cascading effects that are hard to predict upfront. One thing this particular one may — and I have to emphasize may — open up again is a CRDT-based sync engine, or at least the ability to do real-time collaboration in some capacity.
Previously I considered this a no-go because of the lack of maturity of the CRDT-space in Go land, but the story is different in the Rust world. So… who knows.
Can’t wait (for some reason)?
Did I fail in underselling the user impact of this change and you still want to try this without waiting for a release? Can’t wait for a fresh whiff of Rust on your server? Fine, you can do so as follows:
If you’re a SilverBullet.md user running the binary directly (it’s still a statically compiled single binary): you can pull binaries for your platform from the edge release on github (or use the silverbullet upgrade-edge command) to upgrade to an edge build. If you’re using docker images, pull the latest :v2 or :v2-runtime-api tags, they’ve already been updated as well.
If you’re a SilverBullet+ Pro user (paid tier), you’ll have a release channel selector in your “Pro” tab in the dashboard where you can switch to the “Edge” release channel. This will soon also be based on the newly refactored Rust code, although it was already all Rust before (except for the CLI).
The CTO Hat
My day job is being the CTO at Fixly.pl. Part of that job is to make wise (sounding) technology choices that have a high probability of standing the test of time.
Being the primary developer of SilverBullet gives me that same responsibility. Therefore I try to apply similar long-term facing decisions for it, too. Although with a parkle more of risk taking, because there’s not hundreds of thousand users depending on it (yet).
Here are some criteria I apply for both, although I weigh them slightly differently:
- Avoid technology zoos: fewer technologies is often better. Use the right tool for the right job, sure, but within reason. Less to master, less to keep up on, easier to “hire” for. The SilverBullet(+) stack went from a mix of Rust, Go and TypeScript to just Rust and TypeScript. While the OSS community may not have noticed, for me this is a big win.
- Maturity: old and boring is better than new and flashy. Sure, you can attract a particular audience through interesting techology choices, but that will only work short term, until everybody jumps on whatever the next hot new technology is. Rust is likely still considered more “hot” than Go, although it’s also become kind of a new default choice for a lot of serious system programming over the years. Node.js is definitely the more mature choice compared to Deno or Bun.
- Leverage: when choosing one technology over another, there are valid reasons to pick a (more) flashy technology if it gives you a significant enough benefit. With Deno that benefit used to be there for SilverBullet, but had faded. For Tauri over Electron, this is more of a gamble. I like that SilverBullet+ downloads are fairly small, and the app starts quickly. But is that significant enough long term? Time will have to tell. However, with a Rust backend supporting CRDTs, this investment may actually result in whole new opportunities.
- Fun: Lower on the list, but still present: is the technology attractive enough and fun to use? I’ve done prototypes with Electron, and I have to say: I did not enjoy myself. With Tauri, much more so.
We’ll see how the story continues to unfold. The size of the project remains managable enough (and was simplified again as a result of this transition) to be able to make technological pivots like this. Requirements will continue to shift.
And technology wise, we all know there’s no silver bullet. Except SilverBullet.
Subscribe to No SilverBullet
Get new posts by email. No spam, unsubscribe anytime.
Prefer a feed? Grab the RSS feed.