On the Judicious Use of core.async

I remember when core.async was released. It was heralded as an example of the power of Lisp, because being a library it added an asynchronous programming model to a language, something that would usually be done by extending the compiler with syntax. This is certainly true.

It was also celebrated for bringing CSP to Clojure. There are probably many interesting remarks, both positive and negative, that could be made about the flavor of CSP that core.async implements, but I would like to cover a few practicalities that I have learned using core.async. First the bad:

core.async tends towards imperative style. A go block is like a function in that it can capture local values in a closure, but it is not a function. A go block is not referentially transparent. It can have zero or more mutable input channels and zero or more mutable output channels. A go block encourages the use of loop (the most imperative construct in Clojure).

core.async is infectious. This is especially true in ClojureScript where there is no blocking take. Any function that spawns a core.async process will generally return a value on a channel, and to read the channel you have to be in a go block, so they will propagate across your code base.

core.async does no error handling for you. I understand why this is¹, but it still feels kind of crappy when you discover that your application is deadlocked because a process is stuck waiting on a channel that will never have any data put to it, because another process died silently of an exception.

core.async scrambles stack traces. Because of the way the go macro re-compiles your code into lightweight processes this is probably unavoidable, but again, it hits you right in the feels.

core.async tends toward large, hairy go blocks. The go macro has to be able to see (lexically speaking) any use of <! or >!, so you will be gently encouraged to make incomprehensibly big go blocks.

Of course like anything, there is another side. The good:

Similar in both clojure and clojurescript. It is pretty cool to be able to write similar code in both Clojure and ClojureScript. You can even write cross platform code in cljc files.

Transducers on channels. Transducers are very cool, and the fact that you can define a transformation process and apply it to a channel is very nice.

Lightweight processes. It is useful to have the option to start 10k processes without consuming as many resources as threads.

I think the drawbacks of core.async can be summarized thusly: it is too low-level. You end up specifying how and where instead of just specifying what.

Whatever the drawbacks, core.async is very much worth using, so here are some ideas for how to use it judiciously:

1. Use it sparingly. core.async processes are lightweight, but like any threading model, there is overhead. Break up the work your application does into chunks that are neither too large nor too small.

2. Do not use it in libraries. There may be exceptions, but most libraries should not use core.async. Once you do, you are forcing a particular concurrency model on users of your library (and one that is infectious).

3. Build abstractions that use core.async. Think about what your application is doing, and write code that tells the computer what to do (not how). Is your application a processing pipeline? Model it as stages that are sewn together. Model using data structures or even records and protocols that are implemented with core.async hidden behind your abstraction.

4. Be very, very conscientious about exceptions. This is another reason to build abstractions instead of using core.async directly. If you are not eternally vigilant about exceptions, you will deadlock your application, and the stack trace you get will be useless for debugging.

I think of core.async like manual memory management or writing multi-threaded programs with locks: too low-level to be used directly except in exceptional circumstances.

Interestingly, at Clojure/West 2017 Tim Baldridge also had some practical tips about using core.async usage. You might checkout his talk.

Footnotes: