feierabendprojekte

High-Level vs. Low-Level

Recently I went to a coding dojo where we tried to solve the eight queen puzzle. My Clojure solution can be found here. Three things during the dojo made me think about low-level vs. high-level.

The thing is that when I started learning Clojure it felt extremely low-level. This mainly due to the fact that Clojure uses regular data structures for everything, instead of elaborate constructs like objects. But now it’s kind of the opposite. Clojure feels much more high-level than Java. Why is that?

Edit: I am not talking about low-level and high-level programming languages. I try to compare Clojure to Java code. At first Clojure felt like low-level Java code because the use of lists, vectors and maps reminded me of array usage.

Higher-Order Functions

Another participant and a co-worker tried to build a solution in Java, based on arrays. During the retrospective they said they had to fix several ArrayIndexOutOfBounds Exceptions. I found this curious because I did not even try to avoid those in my solution, they just did not occur. And it made me think back and try to determine when I got my last OutOfBounds-exception. Honestly I can’t even remember, but it must be years.

So, no such Exceptions for years? How does that work? Higher order functions. We’ve been having them since Java 8 with streams and the according map and filter functions. Around the time I started using streams, OOB exceptions just vanished from my day-to-day work.

So functional programming with higher order functions is in the mainstream now. And I think they are extremely valuable. If you’re not using them in you work take a look now.

This is why functional programming feels a lot more high-level than imperative Java code from 10 years ago.

But even if you are using streams in Java they feel low-level compared to functional languages. So much is missing or (partly) incompatible with each other (Function, Predicate etc.). Functional composition in a functional language feels more like legos than anything with streams, because it’s more flexible and more general.

REPL

The idea during the dojo was to use a TDD workflow. Frankly I did not strictly follow the workflow in the sense of „write a unit test first“. But I did test things in the REPL. The REPL (Read Eval Print Loop) is a way to interact with the Clojure runtime as well as your program while it’s running. So I wrote expressions that compared expected values with the return values of the actual functions. Which is kind of like unit testing. But I also just executed functions and evaluated their return value interactively. With the right tools you can even turn those REPL interaction into test-cases.

Tests still make sense in Clojure but the „instant feedback“ part of TDD is delivered by the REPL in a much more flexible way. As Stu Halloway puts it „Coding alive, against a tangible runtime, where you’re in your program invoking your tools, instead of living in your tools invoking your program.“ (please watch the whole talk).

To put it bluntly: a compile-run cycle feels backwards and low-level compared to an interactive runtime environment.

Data-orientation and reuse

Ok I admit it, I used a library while coding. math.combinatorics proved extremely handy in solving the eight queens puzzle. The alternative would have been to write eight nested for loops, but then my algorithm wouldn’t be able to calculate solutions for different board-dimensions (e.g. 6×6).

Clojure’s data orientation is the key to this. When you use the set of built-in datastructure abstractions (map, set, list, vector) then the majority of Clojure libraries are compatible with your code. There might be differences that require mappings of course. But in most cases things just compose well and reuse is incredibly high. This is contrast to libraries in Java that in my experience are either:

In Clojure there is no difference between the high-level and low-level code I talked about. Everything works with data and everything can interoperate. I could use math.combinatorics from any code with extremely low impedance mismatch.

This is the third thing why Java feels low-level: the more general a solution a Java library provides, the more low-level the interface has to be (think smallest common denominator). The more objecty and elaborate a library gets, the more it tends to be specialized and not to be reusable for your case or at least incur a high cost. There are of course libraries/frameworks that are both elaborate and general. But they tend to be monsters, e.g. Spring.

In Clojure you get both: the generality/low-level-ness and the interoperability/reuse in higher-level code.

Edit: This really means idiomatic Clojure code is more comparable to high-level Java code. But the reuse you get is way better than that and more like low-level Java code.

Conclusion

To me Clojure felt low-level at first because of its use of basic datastructures. It now feels like high-level because of the functional paradigm, its interactive environment but also its use of basic data structures that allow for a tremendous amount generality of reuse. Ironic, don’t you think?

Edit: If you’re a newcomer to Clojure don’t feel discouraged by how low-level Clojure might feel. The more you get to know it, the more sense it makes and the more you start to see that using basic data structures can be really high-level in the sense of generality and reuse.

Edit 2: A fellow redditor mentioned that Clojure with its default immutability and persistent datastructures abstracts away the mutability of the machine it’s running on. This is absolutely true and yet another reason to see Clojure as more high-level than Java where mutablity is everywhere.

Edit 3: Found a great quote: “A programming language is low level when its programs require attention to the irrelevant.”—Alan Perlis, Epigrams on programming