I have been blogging in English for some time, but this is my first time participating in an indie web conversation. So I think I should introduce myself first.
I’m Eltrac. I’m mainly a web developer but I like all kinds of thing about programming. I code in Go, Clojure and Lua, and I explore new tech all the time. I usually blog more than three times a week, but most of my posts are written in Chinese. I recently became an active member of
Codeberg e.V.
and have been working on my own FOSS project
weepinbell
for a while. It’s a new Webmention receiver written in Clojure.
In this post, I will talk about why I developed weepinbell, how I designed its architecture, what I think of the IndieWeb movement and many things.
Why weepinbell?
I wrote to
Brennan
recently and he was really nice to mention weepinbell
on his blog
. In that email, I talked about how I set a new goal for weepinbell (i.e. an alternative to webmention.io) because of Aaron Parecki rewriting webmention.io with Claude. Later, in
Aaron’s response
to
Brennan’s new post
, he mentioned weepinbell as “a direct response to my rewrite”.
I would like to make things clear. I didn’t start developing weepinbell because of Aaron’s rewrite. I started early in this May to learn Clojure. I’ve been blogging (in Chinese) for years and of course I love the concept of IndieWeb. Naturally, my practice project for learning a new language involves IndieWeb technologies.
The motive is that I want to get my hands dirty with the Webmention standard. I read through the
W3C specification document
and started putting things in parentheses. You can still find snippets from that document in my code as comments. I bascially code against the specification. It was a fun experience for me.
(if-let [error-msg (check-request source target allowlist)]
;; respond if request verification failed (bad-request {:status400:message error-msg})
;; proceed if request is ok (do;; The receiver SHOULD check that target is a valid;; resource for which it can accept Webmentions.;; This check SHOULD happen synchronously to reject;; invalid Webmentions before more in-depth verification begins.;;;; Webmention verification SHOULD be handled asynchronously;; to prevent DoS (Denial of Service) attacks. (put! wmc new-wm)
;; respond with 201 when Webmention passed checks and;; was sent into queue, include a Location header (let [wm-id (digest (str source target))
wm-url (str (:base-url config) "/api/mention/" wm-id)
response {:status201:message"Webmention is accepted and pending for verification.":url wm-url}]
(created wm-url response))))))
After the initial quick and rough implementation, I stopped working on weepinbell for a few months. And it had been receiving Webmentions for my blog during that time. Everything worked and I was content. It was bascially a personal tool.
Then the webmention.io rewrite came to my attention.
Why does it bother me? I don’t even use webmention.io any more. I see some of my friends seeking alternatives, and… there aren’t many to choose from! There is
webmentiond
by Horst Gutmann and some other implementations here and there. My friend
Chlorine
developed
CircleAt
in Rust. And I’m sure there are many more. But frankly, none of them truly count as “alternative to webmention.io”.
Don’t get me wrong. I think many of them are great for their purpose, i.e. small service one can self-host. However, they can be quite difficult for the non-tech indie web folks and those who simply do not own a server. They need a public instance.
Refactoring for scalability
I decided to continue development of weepinbell and make it “the next webmention.io”, at least that’s what I’m going for.
So I took a closer look at my code. They’re not that messy, well partitioned (for the most part) and maintainable. I can swap some of the parts easily because I used hook functions and vtables.1 The architecture is flexible enough, and the Lisp power also makes modifying software really easy.
The following part will be somewhat technical. Feel free to jump to the next section if you don’t want to read about system design. The bottom line is: weepinbell would be lightweight enough for people to self-host; it would also be scalable enough to support large traffic.
I’m a Go programmer by the living.2 And Clojure’s
core.async
library greatly resembles Go’s concurrey programming style, which means my skills of building scalable applications in Go translates.
Webmention is not big data. Storage is definitely not the bottleneck. In fact, up until
a few commits ago
, weepinbell does not even support any type of database. It unloaded everything in a single file and then loaded it back when needed. And it just… worked, for months.
Of course this is neither performant nor safe. If I want to build something reliable, I need a database. If you have taken a look at that commit I linked, you might notice that the business logic is untouched. All that’s added is a new storage implementation, a new config option, and a few lines in the main component (core.clj).
I’ll probably elaborate on architecture flexibility in another post. That’s something I’m really proud of. The gist is that using a new type of database and ORM is easy, and will be just as easy if I ever decided to redo the storage layer again.
;; all I need to make sure is that the new storage implementation;; returns a valid storage vtable, like this:(defn use-store [db-file]
(let [ds (make-datasource db-file)]
(create-tables ds)
{:append (partial insert-webmention ds)
:update (partial update-webmention ds)
:delete (partial delete-webmention ds)
:get (partial get-webmention ds)
:get-by-domain (partial get-webmentions-by-domain ds)}))
Beside a new SQLite implementation, I added a storage worker that handles I/O asynchronously. So when processing a new webmention, the processor won’t have to wait for DB transaction (or appending to a file; it does not care about what’s going on underneath) to end. It simply delegates to the storage worker and keeps processing the next webmention.
The processor delegates by putting an operation object into a channel. It’s similar to what I already have for receiver. When a new webmention request is sent to the web server, receiver does some basic checks (e.g. does source and target look like real URLs?) and then put the request into the webmention queue. A processor will pick it up and make sure source has truly mentioned target, parse microformats, wrap data into a valid record, and hand it over to storage worker.
Reading from a channel is thread-safe. And since I/O is handled internally by storage worker, it can also be thread-safe, without the need for a mutex lock. So we can add as many processors as we want to handle webmentions concurrently.
The overall architecture looks like this:
Architecture of the receiver part of weepinbell, drawn using Excalidraw
That’s horizontal scaling. For self-hosters, simply set pipeline-concurrency to 1, and there will be only one processor handling webmentions, using a reasonable amount of RAM and CPU.
For large groups or someone who wants to host a public instance, change a number in config and you’re good to go.
It’s just regular HTML with standardized class names like h-card and p-name. It’s easy to add a few classes to your HTML. However, it’s still hard to parse structural data from HTML with just a few more classes.
Good thing there are already some parsers available… right?
Production ready parsers are available for: Go, Node, PHP, Python, Ruby.
Right… I’m using Clojure…
Clojure can use Java code, since it runs on top of it. I found
mf2j
, an “In-development Java parser for microformats2”. It hasn’t been updated since… 10 years ago.
There is a
Clojure implementation
. It’s archived, with its last update committed 11 years ago.
Do I implement a new one myself? Do I pick up where they left off? How do I make sure I don’t burn out and abandon it as well? After all, Clojure is a niche language and IndieWeb is also kind of niche. I can’t imagine there being another Clojurist building something that relies on microformats. At least, I don’t know anyone like that.
I briefly considered running a micro-service that wraps the Go parser, and weepinbell can talk to this service using RESTful API or gRPC. But it feels heavy and everything just won’t fit in a single .jar.
So how can Clojure run programs written in a non-JVM language? What else runs on JVM? Scala, Groovy… Wait, will WebAssembly work?
Yes! WebAssembly runs on JVM. There is a native JVM WebAssembly runtime called
chicory
.
I spent an afternoon studying
Extism
and chirocy. Then I wrote a tiny little Go wrapper of Will Norris’
parser library
.
package main
import (
"bytes""github.com/extism/go-pdk""willnorris.com/go/microformats")
//go:wasmexport parsefuncparse() int32 {
input := pdk.Input()
r := bytes.NewReader(input)
data := microformats.Parse(r, nil)
pdk.OutputJSON(data)
return0}
Then I compiled it to .wasm using
tinygo
compiler, bundled the .wasm inside .jar, and called the Extism plugin from Clojure using chicory. Voilà! Here’s
mf2clj
, a Clojure wrapper of WebAssembly compiled from a Go wrapper of an existing microformats parser.
By the way, I made a little logo just for fun.
You use it just like a normal library in Clojure. And it’s surprisingly fast.
More surprisingly, bundling a .wasm binary and the WebAssembly runtime inside the .jar is not as heavy as I thougt it would be. The .jar only grew 2 MB in size. Am I really running Java?
Nevertheless, the nightmare isn’t over yet. In the begining, I settled for the incomplete mf2j library and used it for a while. At that time, I called the JSON output “data structure from Hell”. I still do —— Everything is an array! Everything is deep inside something’s properties or children! Nothing is deterministic! Every website’s format is different!
In the end, I had to write a smaller parser to parse output a the original parser. I called this feature microformats reduction. I remember webmention.io having something similar.
If something does not exists, it’ll just be null. The format is not final yet, I will make some adjustments. This can be turned off by ?microformats=raw when querying webmentions.
Also, I think more than half of the bugs I found in weepinbell have something to do with microformats-related features. Maybe we should switch to Open Graph and JSON-LD, folks?
Anyway, I’m just glad it’s finally working.
Slowing down
At this point, I’ve almost finished everything I have planned for the v1.0.0 release: a proper database, horizontal scalability, up-to-date microformats parser. As for hosting a public instance, that’ll involve authenticating users and making a dashboard for them to manage webmentions. That would be a separate project.
I plan to develop a service that sits in front of weepinbell, handling logging in and stuffs. I mentioned the idea in the project’s
PHILOSOPHY.md
.
I want to keep the core of weepinbell minimal. Those who want to be their own sysadmins don’t have to care about authentication. They don’t even need a frontend. They only need to self-host one .jar and use API to query webmentions, use RSS to receive notifications and use webhook to automate stuff. These are the built-ins.
A platform like webmention.io needs more than that. If I want to keep weepinbell “for both individuals and large groups”, I have to separate things. Also, this makes everything modular and swappable, a pretty good architecture decision.
My writing to Brennan was an attempt to get more Clojure developers on the team. Well, that didn’t go well. For the new platform, which will be named Ringabell4, I want to use more mainstream stack, probably Go, so that it’s easier for people to get involved.
For now, I’m still working on weepinbell’s test suite. The current test coverage is more than 83%, but there are still some edge cases that haven’t been covered by tests. I had to test a lot of paths by hand. Also it’ll take some time for the new SQLite implementation to be stable.
I don’t think we need to or should move fast when it comes to developing reliable software, with or without LLMs.
Independence, and now decentralization
In reply to Brennan’s recent question,
“What do we do now?”
, I think it’s finanlly time to get more decentralized.
Decentralization is never about moving away from the corporate web (that’s independence). It’s about not relying on a
single point
, no matter what the point is and who it involves.
I agree with Brennan on not conflating IndieWeb and the lower-case indie web. The
IndieWeb camp
is a nice community and a great introduction to the indie web. But I think we should form more indie web communities other than the IndieWeb camp.
omg.lol
is an example. It’s a fun web community by
Neatnik
. They provide infrastructure like a simple profile page, Fediverse instance and email forwarding. Brennan is also building
folk.zone
, which hosts a fork of webmention.io and many more.
Those two may not be exactly alternatives to the IndieWeb camp, but they can be. And there can be more for people to choose from.
Tech folks can self-host their own infrastructure if they want. For the non-tech folks, there should be more alternatives to what the IndieWeb camp has to offer. Poeple who are capable can host public services and take a small community.
After all, communities are what brings the indie web together. We can and should build more communities (this is probably more like polycentricity rather than decentralization). We can have the LispyWeb for lisp programmers, ArtsyWeb for artists, and Indie Indie Literature Club!5 for book lovers. Those are just some random names I came up with.
Perhaps the upcoming Ringabell might be the next cozy small community. Who knows? To everyone who’s nice enough to finish reading this, I hope we can all find our people.
A vtable in Clojure is basically a hashmap whose values are functions. It’s a technique mentioned in one of Uncle Bob’s books, Functional Design. ↩︎
Actually I’m at the very begining of my career. Don’t think of me as an expert of anything. XD ↩︎
Clojure runs on JVM, so it compiles to JVM bytecode, just like Java. ↩︎
标照 Rich 本人对简单的定义,标准库里的 map、filter、reduce 等操作序列的函数,都不够简单,因为他们不仅涉及做什么操作(what to do),还涉及了怎么做(how to do it)——预设了惰性,预设了数据操作的顺序(序列都是从前往后线性遍历的),预设了数据的表示方式(返回的是序列)。
如果 map、filter、mapcat 等函数,能够与数据类型(序列)解耦呢?也就是说,如果 map 只定义做什么操作(what to do) ,把怎么做(how to do it)交给其他组件呢?
人工智能研究员 Nick Cammarata 发表了一篇长文,我英文不好,也不是技术领域专家。粗略读过,感觉意思是说:以前大家以为的神经网络不可理喻,其实是人类的认知错误——神经网络本身逻辑清晰。
拿神经网络里的一个具体神经元 4e:55 举例子,它既被猫脸激活也会被汽车前脸激活。人们误以为神经网络用高维空间搞出了猫和车的叠加态,总想区分开,找到一个专门的被猫激活的「猫神经元」。Nick 则认为完全没必要,也不必训练新模型来展开折叠区分猫和车,而是应该直接研究神经网络本身,按照神经网络的逻辑来理解,根本不存在叠加,分类清晰,猫下巴和车前脸都归属于 V 形这个几何意象类别……
关于这篇文章,显然我远没能清晰解读它。不过,我好像突然知道为什么 AI 总是让人们欣喜又警惕、摇摆不定了。人类制造了不趁手的工具时就会改造它,让它顺手,诚然工具的材质、属性,也要选择合适的来使用,但那些是自然的规律、上苍的造物。而大语言模型,不是因为电子的属性,不是因为硬盘的材质,就只是作为人造物、工具本身,却需要人类学习它的逻辑,就像学习如何顺应水流、接受光照一样。欣喜或警惕,都是很自然的反应。
你更偏向哪一类? ① 管理一群人、看小人故事、防御袭击(环世界风格) ② 搭管线、物理热力学工程(缺氧风格) ③ 全自动工厂流水线(异星工厂)
没错:矮人要塞(Dwarf Fortress)就是殖民地涌现模拟这一支的祖师爷
2002年兄弟二人启动开发,2006公开Alpha版,比环世界早差不多10年。环世界作者Tynan自己公开说,环世界就是简化、现代化的矮人要塞,核心灵感来源就是DF。 泰拉瑞亚、MC、监狱建筑师、缺氧全都受它启发,但不是完全照搬,只是吸收它的设计思想:程序生成完整世界、生物精细模拟、人物独立情绪记忆,游戏自己“演故事”,口号 Losing is Fun(失败即是乐趣)。
22.79.117.157:55212 - - [13/Oct/2026:16:26:03] "GET /media/packages/2026/10/__UNI__8813F79_1007202252.apk" 200 79556639
113.219.202.124:5113 - - [13/Oct/2026:16:26:03] "GET /media/packages/2026/10/__UNI__8813F79_1007202252.apk" 200 79556639
113.219.202.2:12879 - - [13/Oct/2026:16:26:04] "GET /media/packages/2026/10/__UNI__8813F79_1007202252.apk" 200 79556639
113.219.202.3:46581 - - [13/Oct/2026:16:26:04] "GET /media/packages/2026/10/__UNI__8813F79_1007202252.apk" 200 79556639
222.79.116.209:27848 - - [13/Oct/2026:16:26:04] "GET /media/packages/2026/10/__UNI__8813F79_1007202252.apk" 200 79556639
222.79.116.184:59270 - - [13/Oct/2026:16:26:04] "GET /media/packages/2026/10/__UNI__8813F79_1007202252.apk" 200 79556639
2026-10-13 16:26:25,486 WARNING Application instance <Task pending name='Task-6311' coro=<ASGIHandler.__call__() running at /home/obaby/soft/baby-claw/upgrade-server/venv/lib/python3.10/site-package
s/django/core/handlers/asgi.py:170> wait_for=<Future finished result=None>> for connection <WebRequest at 0x7f5ee03d73d0 method=GET uri=/media/packages/2026/10/__UNI__8813F79_1007202252.apk clientpro
to=HTTP/1.1> took too long to shut down and was killed.
2026-10-13 16:26:25,542 WARNING Application instance <Task pending name='Task-6348' coro=<ASGIHandler.__call__() running at /home/obaby/soft/baby-claw/upgrade-server/venv/lib/python3.10/site-package
s/django/core/handlers/asgi.py:170> wait_for=<Future finished result=None>> for connection <WebRequest at 0x7f5ee0255390 method=GET uri=/media/packages/2026/10/__UNI__8813F79_1007202252.apk clientpro
to=HTTP/1.1> took too long to shut down and was killed.
2026-10-13 16:26:25,543 WARNING Application instance <Task pending name='Task-6350' coro=<ASGIHandler.__call__() running at /home/obaby/soft/baby-claw/upgrade-server/venv/lib/python3.10/site-package
s/django/core/handlers/asgi.py:170> wait_for=<Future finished result=None>> for connection <WebRequest at 0x7f5ee03a3c10 method=GET uri=/media/packages/2026/10/__UNI__8813F79_1007202252.apk clientpro
to=HTTP/1.1> took too long to shut down and was killed.
2026-10-13 16:26:25,543 WARNING Application instance <Task pending name='Task-6351' coro=<ASGIHandler.__call__() running at /home/obaby/soft/baby-claw/upgrade-server/venv/lib/python3.10/site-package
s/django/core/handlers/asgi.py:170> wait_for=<Future finished result=None>> for connection <WebRequest at 0x7f5ee01ebfd0 method=GET uri=/media/packages/2026/10/__UNI__8813F79_1007202252.apk clientpro
to=HTTP/1.1> took too long to shut down and was killed.
2026-10-13 16:26:25,543 WARNING Application instance <Task pending name='Task-6357' coro=<ASGIHandler.__call__() running at /home/obaby/soft/baby-claw/upgrade-server/venv/lib/python3.10/site-package
s/django/core/handlers/asgi.py:170> wait_for=<Future pending cb=[Task.task_wakeup()]>> for connection <WebRequest at 0x7f5ee03d7760 method=GET uri=/media/packages/2026/10/__UNI__8813F79_1007202252.ap
k clientproto=HTTP/1.1> took too long to shut down and was killed.
2026-10-13 16:26:25,548 WARNING Application instance <Task pending name='Task-6368' coro=<ASGIHandler.__call__() running at /home/obaby/soft/baby-claw/upgrade-server/venv/lib/python3.10/site-package
s/django/core/handlers/asgi.py:170> wait_for=<Future finished result=None>> for connection <WebRequest at 0x7f5ee01892d0 method=GET uri=/media/packages/2026/10/__UNI__8813F79_1007202252.apk clientpro
to=HTTP/1.1> took too long to shut down and was killed.
2026-10-13 16:26:25,548 WARNING Application instance <Task pending name='Task-6373' coro=<ASGIHandler.__call__() running at /home/obaby/soft/baby-claw/upgrade-server/venv/lib/python3.10/site-package
s/django/core/handlers/asgi.py:170> wait_for=<Future pending cb=[shield.<locals>._outer_done_callback() at /usr/lib/python3.10/asyncio/tasks.py:864, Task.task_wakeup()]>> for connection <WebRequest a
t 0x7f5ee0058310 method=GET uri=/media/packages/2026/10/__UNI__8813F79_1007202252.apk clientproto=HTTP/1.1> took too long to shut down and
上一篇
我提及了如何用
Ring
开发 HTTP 服务器,也提到 Ring 提供了同步 API 和异步 API。那么,什么是异步编程?为什么需要异步?在什么情况下需要让一个程序同时做很多事情?
同步、异步、并发和并行
同步(synchronization)的中文名字可能会让人误以为是多个实体同时进行操作,实际上同步的意思是指系统中所有事件都依照相同的顺序进行协调。可以用工厂流水线举例,假如流水线中一个工人需要等上一个工人交付一部分工作之后才能干活,整个流水线都要所有人一起干活才能转动起来,这个流水线就是同步的。反之,假如某些工作可以分开做,比如一部分人负责生产 A 零件,另一部分人负责生产 B 零件,两者互不干扰,而最终把 A 和 B 组装在一起的又是另一拨人,这个系统就是异步的(asynchronous)。
;; A function that simulates a long-running process by calling Thread/sleep:(defn long-running-job [n]
(Thread/sleep3000) ; wait for 3 seconds (+ n 10))
;; Use `doall` to eagerly evaluate `map`, which evaluates lazily by default.;; With `map`, the total elapsed time is just under 4 * 3 seconds:user=> (time (doall (map long-running-job (range 4))))
"Elapsed time: 11999.235098 msecs"(10111213)
;; With `pmap`, the total elapsed time is just over 3 seconds:user=> (time (doall (pmap long-running-job (range 4))))
"Elapsed time: 3200.001117 msecs"(10111213)
core.async 几乎和 Go 语言异步编程的要素一模一样,尤其是 go block——把语句放在 go block 里面,语句就会异步执行。把消息队列 chan 当作一等公民的做法,也如出一辙。根据
Clojure 官方的博客文章
,这种异步编程风格其实有一段历史了,Go 也是借鉴并现代化了这种做法。
The roots of this style go back at least as far as Hoare’s Communicating Sequential Processes (CSP), followed by realizations and extensions in e.g. occam, Java CSP and the Go programming language.
无论如何,让我们来看看怎么用 go 和 chan 实现异步编程吧。如果你已经会写 Go,那么接下来的代码会很熟悉。
user=> (def c (chan1))
#'user/cuser=> (take! c
(fn [x]
(println "Clojure callback value " x)))
nil
user=> (put! c "XYZ")
Clojure callback value XYZ
true
user=> (put! c "XYZ")
true