I wrote this as the final pieces of Ooloi's backend architecture were falling into place. What began as a meditation on infrastructure and isolation turned into something more personal about mastery, loss, and the strange kind of solitude that comes with finishing something no one else can see. This isn't documentation. It's a reflection. / Peter There's a peculiar melancholy that settles over you when you near the completion of something genuinely complex, something that has consumed many months of concentrated thought and represents the synthesis of decades of accumulated understanding. I find myself in precisely this position with Ooloi's backend architecture, and the psychological reality proves a bit more complicated than I'd anticipated. It's rather like the post-coital moment after particularly intense sex: that strange combination of satisfaction, exhaustion, and existential emptiness when the driving urgency suddenly lifts. You've achieved something profound, yet find yourself staring at the aftermath wondering what, precisely, comes next. I'm smoking a conceptual cigarette, as it were, contemplating the peculiar loneliness that follows architectural completion. In a matter of days, I'll complete the final piece: the endpoint resolution system for slurs and ties that uses the framework I've spent months building. Once that's finished, the backend will be conceptually complete – 15,000+ tests passing, STM transactions handling 100,000+ operations per second, Vector Path Descriptors enabling elegant client-server communication, and a transducer-based piece-walker that coordinates musical time with mathematical precision. The piece-walker literally performs the musical score, traversing it in time just as I once performed Vierne at the organ. To anyone versed in these technical domains, that represents serious work. To everyone else, it's incomprehensible gobbledygook happening 'under the hood' of something they might one day use to write music. And therein lies the first layer of loneliness: having solved genuinely difficult problems that almost nobody can fully appreciate. The Weight of Invisible Architecture Software architecture, when done properly, is invisible to its eventual users. They should never know about the STM transaction coordination that keeps their concurrent edits from colliding, or the VPD system that allows them to reference musical elements without direct object pointers, or the careful functional design that ensures their work remains consistent across complex operations. This invisibility is precisely the point – and precisely the problem. I've spent months solving challenges that required rather more thought than I'd initially anticipated, creating abstractions that handle the full complexity of musical notation whilst remaining elegant enough to extend indefinitely. Yet once complete, this work vanishes into infrastructure. The better I've done my job, the less visible it becomes. There's something profoundly isolating about completing work that embodies your best thinking but can never be fully shared. The musicians who will eventually use Ooloi might appreciate its responsiveness or reliability, but they'll never see the polymorphic dispatch system that makes complex musical operations feel effortless, or understand why the pure tree structure with ID references elegantly solves problems that have plagued notation software for decades. Clojure for ClosureThe choice of Clojure wasn't merely technical: it was also psychological. Having started programming in Lisp in 1976, having built Common Lisp compilers and interpreters, having spent $7.5 million of investor money and then having unresolved feelings about Igor Engraver's death for a quarter of a century, returning to a Lisp dialect feels like completing a circle that's been open far too long. Clojure for closure, if you will. But this completion reveals its own complexity. I'm 64, carrying more than five decades of programming experience and a parallel career as an internationally performed composer – an intersection that doesn't exactly suffer from overcrowding. The same mind that wrote what apparently is the internationally most played Swedish opera now architects STM concurrency patterns. The same hands that have performed French romantic organ works now implement temporal traversal through transducers. This convergence of domains should feel like triumph. Instead, it often feels like exile – not belonging entirely to the musical world I've moved beyond, nor quite fitting into the tech world that didn't shape me. I don't belong anywhere, really. The isolation isn't just professional; it's existential. The Economics of Art and Pragmatism I must confess something that still sits uneasily: I've essentially given up composing, despite international success, because conditions in Sweden for composers have deteriorated to the point where I had to prioritise my pension. There's an unwritten opera I'd like to complete – I have the text ready – but it will likely never come to fruition. Whether this represents economic necessity or conscious rejection of a cultural environment I found increasingly superficial and performative, I honestly can't say. Perhaps both. The exact proportion remains unclear even to myself, and I've learned to be comfortable with that ambiguity. Life rarely offers the clean motivations we prefer in retrospect. What I can say is this: the creative energy that might have gone into that final opera has found other expression. The same understanding of temporal flow, structural relationship, and expressive possibility that shaped my musical work now manifests in software architecture. It's sublimation in the deepest sense: not compromise, but transformation. The Paradox of CompletionHere's what nobody tells you about completing something genuinely substantial: the moment of architectural completion isn't triumph, it's vertigo. All those months of wrestling with complex problems, of holding intricate systems in your head, of solving puzzles that demanded your full intellectual capacity – suddenly that pressure lifts, and you're left staring at what you've built with a strange mixture of satisfaction and emptiness. The backend is nearly finished. The hard problems are solved. The foundation is solid. And now comes the work that should be 'easier': creating user interfaces, handling the cultural and aesthetic dimensions of human interaction, making decisions about visual design and workflow that seem trivial after months of STM transaction coordination but are actually far more treacherous. Technical problems have logical solutions. Human interface problems have cultural solutions, psychological solutions, aesthetic solutions; domains where being right isn't enough, where the same mind that can architect transducer pipelines struggles with questions like 'should this button be blue or green?' not because the technical challenge is greater, but because the criteria for success shift from mathematical to cultural. The Transition ChallengeMoving from backend completion to frontend implementation isn't just a technical transition. It's a psychological one. After months of building infrastructure that only I can see, I must now create experiences that others will judge. After solving problems where elegance and correctness align, I must now solve problems where user perception and technical reality often diverge. The loneliness of architectural completion isn't just about having done complex work in isolation. It's something else entirely. The 'easy' work ahead may be harder in ways that have nothing to do with computational complexity. It's about moving from mathematical elegance to human messiness, from logical purity to cultural compromise. Most acutely, it's about the strange position of being someone who carries irreplaceable knowledge – the synthesis of decades in both musical and computational domains – and wondering how to encode that understanding into forms that others can inherit and extend. Not just the technical patterns, but the aesthetic judgements, the performance intuitions, the hard-won understanding of how creative work actually happens. What Comes NextIn a couple of weeks, when the final endpoint resolution system is working and the backend architecture is truly complete, I'll begin the gRPC implementation that bridges backend and frontend. Then comes the 'Hello World' window – Ooloi's first visible manifestation, however simple.
The psychological challenge isn't technical uncertainty. I've built user interfaces before, in a previous technological era. It's the weight of transition: from solving invisible problems to creating visible experiences, from mathematical elegance to cultural navigation, from the loneliness of architectural completion to the different loneliness of human interface design. The work continues, but its nature changes completely. After months of building the engine, it's time to build the car. And to discover what new forms of isolation await when mathematical precision meets human perception. For now, I sit with the strange melancholy of nearly completing something that matters enormously but whose full significance can be communicated to virtually no one. It's a peculiar form of creative isolation – not the romantic loneliness of the misunderstood artist, but the technical loneliness of someone who happens to carry knowledge that exists at intersections most people never visit. Clojure for closure, indeed. But it turns out that closure reveals as much as it resolves. Time for a smoke.
0 Comments
How solving a real music notation problem revealed the perfect transducer use case The Problem That Started It All I found myself confronting what appeared to be a deceptively simple requirement for Ooloi: 'Resolve slur endpoints across the musical structure'. Rather straightforward, one might assume: simply traverse the musical structure and locate where slurs terminate. But then, as so often is the case, the requirements revealed added complexity:
It became apparent that I needed a general-purpose piece traversal utility: something handling temporal coordination whilst remaining flexible enough for multiple applications. Rather than construct something bespoke (and likely regrettable), I researched the available approaches within Clojure's ecosystem. That's when I recognised this as precisely what transducers were designed for. The Architecture RecognitionAllow me to demonstrate the pattern I anticipated avoiding. Without a general traversal utility, each application would require its own approach: Three functions, identical traversal logic, different transformations. Exactly the architectural smell I wanted to avoid from the outset. This was precisely Rich Hickey's transducer insight made manifest: "What if the transformation was separate from the collection?" The Transducer RevelationWhat if I could write the temporal traversal once, then apply different transformations to it? Objective achieved: one traversal algorithm, many applications. But its architectural reach turned out to be even more profound. The Architectural InsightThe design decision hinged upon recognising that I was conflating two distinct concerns: the mechanism of traversal and the logic of transformation. This wasn't merely about avoiding the tedium of duplicated code (though that would have been reason enough) but rather about establishing clean architectural boundaries that would serve the system's long-term evolution. Consider the conceptual shift this separation enabled: Rather than thinking in terms of specific operations upon musical structures:
The transducer approach encouraged thinking in terms of composed processes:
The traversal thus became reusable infrastructure, whilst the transformation became pluggable logic. This distinction would prove invaluable as the system's requirements expanded. The Broader ApplicationsWhat I hadn't anticipated was how broadly applicable the resulting abstraction would prove. After implementing the piece-walker for attachment resolution, I discovered it elegantly supported patterns I hadn't originally considered, each demonstrating the composability that emerges naturally from separating traversal concerns: Each is built from simple, testable pieces. And they all inherit the same temporal coordination guarantee. This composability emerged naturally from the transducer design: a pleasant architectural bonus. The Performance Characteristics As one would expect from a well-designed transducer, memory usage remained constant regardless of piece size: a particularly crucial consideration when dealing with the sort of orchestral scores that might contain hundreds of thousands of musical elements. Consider the alternative approach, which would create intermediate collections at each processing step: The transducer version processes one item at a time: Same result, constant memory usage. This exemplifies what Rich meant by 'performance without compromising composability'. Demystifying Transducers Transducers suffer from an unfortunate reputation for complexity, often relegated to 'advanced topics' when they needn't be. This is particularly galling given that they're fundamentally straightforward when you encounter the right use case, which the musical domain provides in abundance. Think of transducers as 'transformation pipelines' that work with any data source, much as one might design AWS data processing workflows that operate regardless of whether the data arrives from S3 buckets, database queries, or API streams: The pipeline stays the same. The data source changes. In Ooloi: Why This Matters Beyond MusicThe piece-walker solved a universal software problem: How does one avoid duplicating traversal logic whilst maintaining performance and composability? This pattern applies everywhere:
Transducers provide the infrastructure for "traverse once, transform many ways." The Bigger PictureBuilding the piece-walker demonstrated that transducers aren't an abstract functional programming concept. They're a practical design pattern for a specific architectural problem: separating the concerns of traversal from transformation. The musical domain made this separation particularly clear because the temporal coordination requirements are so explicit. When you need the same traversal logic applied with different transformations, transducers provide the elegant answer. This separation makes code:
What's Next?The piece-walker is documented thoroughly in our Architecture Decision Record for those wanting technical details. But the real value lies not in the musical specifics but in observing how transducers address genuine architectural challenges with apparent effortlessness. The next time you find yourself contemplating similar data processing logic across multiple contexts, you might ask: 'What if the transformation was separate from the collection?' You may well recognise your own perfectly suitable transducer use case. References and Further ReadingRich Hickey's Essential Talks
Official Documentation
Educational Resources
Advanced Topics
|
AuthorPeter Bengtson –composer, organist, programmer, cloud architect. Currently windsurfing through parentheses. Archives
July 2025
Categories
All
|
|
Ooloi is a modern, open-source music notation software designed to handle complex musical scores with ease. It is designed to be a flexible and powerful music notation software tool providing professional, high-quality results. The core functionality includes inputting music notation, formatting scores and their parts, and printing them. Additional features can be added as plugins, allowing for a modular and customizable user experience.
|