When I started programming in Lisp in 1979, after reading an article in BYTE Magazine, I hardly imagined that 45 years later I'd be embarking on a new Lisp adventure. Yet here we are, with FrankenScore (to be renamed Ooloi upon open-source release) – a modern music notation software built with Clojure. It's a project that brings together all my lifelong passions: music, programming, and the pursuit of elegant solutions to complex problems. The Path from Common Lisp to ClojureMy journey with Lisp began in an era when optimising Common Lisp compilers were cutting-edge technology. I cut my teeth implementing Common Lisp interpreters and compilers (as one did in those days), delving into the intricacies of a truly original programming language. This experience shaped my understanding of what a powerful, flexible programming language could be. And now in 2024 I find myself in the world of Clojure, a modern Lisp dialect that runs on the Java Virtual Machine. The transition feels both familiar and novel. Clojure's emphasis on immutability and its handling of concurrency through Software Transactional Memory (STM) aligns with the functional programming principles I've long appreciated in Lisp. But it's not just about the language. The ecosystem around Clojure – the JVM, the interoperability with Java libraries, the rich set of tools and frameworks – provides a robust foundation that we could only dream of back in the Common Lisp days. CLOS Thinking in a Clojure WorldOne of the more interesting aspects of this transition has been adapting CLOS-style thinking to Clojure's more data-centric approach. CLOS, with its powerful multiple inheritance and method combination features, encouraged a certain way of modelling problems. In FrankenScore, I've found myself reaching for these familiar patterns, but implementing them in Clojure's more functional style. For instance, the use of Clojure's protocols and multimethods, combined with hierarchies and the Methodical library, allows us to achieve CLOS-like polymorphism. It's a different approach, but one that feels natural once you embrace Clojure's philosophy. Clojure's deliberate avoidance of traditional object-oriented features felt immediately familiar and refreshing. It resonates with CLOS's approach, which many, including myself, have long regarded as transcending traditional OOP. Composition over inheritance, a principle I always valued even in the CLOS days, is not just a best practice in Clojure but the very fabric of its design philosophy. This alignment between CLOS's advanced features and Clojure's functional paradigm makes the transition feel natural and even inevitable. Changes in ThinkingPerhaps the most significant shift has been in embracing Clojure's emphasis on immutable data structures and pure functions. While these concepts weren't foreign in Common Lisp, they're central to Clojure's design. This shift encourages a style of programming that's inherently more thread-safe and easier to reason about – crucial for a complex application like FrankenScore. Another major change has been adapting to Clojure's more minimalist standard library compared to Common Lisp. This has led to a greater appreciation for carefully chosen, interoperable libraries and a more modular design approach. SimilaritiesDespite the differences, there are of course similarities in the overall approach. The emphasis on interactive development, the power of macros for domain-specific languages and the elimination of boilerplate code, plus the satisfaction of working in a dynamic, expressive language – these are all as present in my Clojure work as they were in my Common Lisp days. Moreover, the focus on solving complex problems through abstraction and composition remains. Whether it's CLOS or Clojure, the goal is still to create systems that are powerful, flexible, and pleasant to work with. Closing ThoughtsThis journey from Common Lisp to Clojure, from Igor Engraver to FrankenScore/Ooloi, is both challenging and rewarding. It's a testament to the enduring power of Lisp's ideas and the continued evolution of programming languages.
As I continue to develop FrankenScore, I'm captivated by the possibilities that Clojure and its ecosystem offer. While creating a powerful music notation software is the immediate goal, the project's scope extends far beyond that. It's an exploration of the synergies between music, technology, and open-source collaboration – a playground where these elements intersect and interact in novel ways. To those considering a similar journey, I'd say: embrace the change, but don't forget the lessons of the past. The parentheses may look familiar, but the world inside them is ever-evolving.
0 Comments
In the past weeks, I've been focused on FrankenScore's core architecture. I'm not rushing to open-source this; instead, I'm taking my time to craft a solid platform that will do the heavy lifting for future users and collaborators. All the complexities involving data representation and manipulation in a multi-threaded environment must be solved so collaborators can concentrate on the essentials. Clojure is ideal here, just as Common Lisp was the clear choice for Igor Engraver back in 1996.
Key developments: 1. The API is now fully polymorphic and can be used in the same way internally in the backend as in the frontend. There is a system of pointerless vector path descriptors (VPDs) implemented for this purpose that all API operations can accept as part of their polymorphic setup. I wouldn't be surprised if core collaborators will use the API for internal purposes as well, as it is highly efficient and exposes the underlying functionality in an abstract, domain-specific way. There should be little need to go directly to the underlying data structures, at least not for speed - and certainly not for expressivity. This also bodes well for plugin development in other languages than Clojure, which is an important feature. 2. This beast is fast. Clojure's STM facilities ensure high-speed ACID-compliant transactions with automatic retries. They are also composable. This means that plugins can bombard the backend with hundreds of thousands of mutation requests, for instance to implement MusicXML, with the same efficiency as the pure Clojure backend. 3. Piece Manager Implementation: There's now a Piece Manager, providing functions for storing, retrieving, and resolving pieces from IDs. This allows for multiple clients to work simultaneously on the same piece in a distributed arrangement. The FrankenScore backend can run in the cloud with multiple people collaborating on the same piece. Multiple pieces can be open simultaneously to allow copy-and-paste operations between them. My next steps involve implementing file persistence (saving and opening music files), as well as tackling printing. These are foundational features, not mere add-ons. Persistence forces a clear definition of the data model and enables easier testing. Printing isn't just about output; it's about representation and serves as a sanity check on the entire system design. Both will likely inform further refinements of the core architecture, potentially revealing oversights or opportunities for optimisation. Additionally, sequencing is a crucial part of the core platform. And by sequencing I mean support for converting musical representations to timed sound events - though not necessarily via MIDI; a software synth may use direct means of control, for instance. The core sequencer can be used by plugins to generate MIDI, or to input MIDI, but the actual MIDI implementation will be done in the plugin layer. But that's a whole blog post of its own. Twenty-five years ago, I embarked on a journey to revolutionise music notation software with Igor Engraver. Today, I'm resurrecting that spirit with FrankenScore. But why now? Why breathe new life into a project that's been dormant for a quarter-century? A Vision Deferred Igor Engraver was always meant to be freeware, a tool for musicians and composers to express their creativity without financial barriers. Commercial considerations, however, steered us away from that vision. Now, with FrankenScore, we're returning to those roots by embracing open-source development. This aligns with my original intentions and the spirit of accessibility that drove Igor Engraver's creation. The Tech Landscape: Then and Now Back in '96, when Igor Engraver was born, the technological landscape was vastly different:
Today, we have multi-core processors, cross-platform development tools, and languages like Clojure that offer powerful abstractions and concurrent programming models. These advancements allow us to build FrankenScore as a more robust, efficient, and flexible tool than was possible with Igor Engraver. The State of Music Notation Software Igor Engraver was conceived because the available options at the time – Finale (as user-friendly as a cactus) and Sibelius (marginally better) – weren't up to the task. They fell short in usability, flexibility, and output quality. I hated using Finale (and I've written an entire opera in it). Instead of enhancing your creativity – which, at the end of the day, is what a music processor should do – Finale and all other similar programs hampered your creativity. Surprisingly, a quarter-century later, the field hasn't progressed as much as you might expect. While there have been improvements – some of them clearly inspired by Igor Engraver! – there's still a significant gap between what's available and what's possible. Why FrankenScore, Why Now? The time is ripe for FrankenScore, and I can't help but feel a sense of excitement and purpose. We're at a unique intersection of technological readiness and persistent unmet needs in the music notation world. The tools and platforms available to us now make it possible to build something truly revolutionary – a modern, efficient, and cross-platform solution that was merely a dream when Igor Engraver was conceived. What strikes me is how, despite the passage of time, the music notation software landscape still leaves much to be desired, especially in terms of usability and flexibility. It's both frustrating and motivating. But here's the kicker – we now have this thriving open-source ecosystem at our fingertips. It's the perfect environment for collaborative development and continuous improvement, something I could only have wished for back in the day. There's also a personal element to this timing. I feel a renewed focus, unburdened by the commercial constraints that ultimately derailed Igor Engraver. We can, as a community, now pour our energy into creating the best possible tool for musicians and composers, staying true to the original vision of accessibility and innovation. And you know what? Those years weren't wasted. The experiences from Igor Engraver – our successes, our setbacks, the lessons learned – they're all invaluable insights that we're bringing to FrankenScore's development. It's like we're picking up where we left off, but with 25 years of additional wisdom and technological advancements in our toolkit. FrankenScore isn't just a revival; it's a reimagining. We're taking the core ideas that made Igor Engraver revolutionary and implementing them with modern technology and development practices. Our goal is to create a music notation tool that's not just incrementally better, but fundamentally transforms how musicians interact with notation software. We're excited to embark on this journey, and we invite you – musicians, developers, and enthusiasts – to join us in shaping the future of music notation software. Together, let's bring Igor Engraver's vision to life in FrankenScore. (Oh, and by the way, FrankenScore is just a pre-release working name. When we open the repo, make it open source and invite collaborators to participate, we will switch to Ooloi, just like the domain you're on right now. I'll explain the reasons in a later blog posting.) I should perhaps say something about how Generative AI is used in the FrankenScore project. First of all, I have a prompt of about 4100 lines which prefaces every conversation with the AI chat client. The prompt consists of project documentation, background, design principles and goals, coding principles and conventions, explanations of central code and code examples. It also includes a major part of the source. This allows the AI to:
The copy on this website was almost entirely created by AI means, often using multiple iterations until I arrived at something suitable for publication. There remain a few passages that slipped me by as the AI produced text that reads a little too self-congratulatory on my part, but it was simply the opinion of the AI (though it is of course nice that it likes the code). I'll fix that during the days to come. Also, the technical comparison with other software is a bit too speculative and monotone. I'll change that, too. In terms of code, I've found that Claude 3.5 Sonnet reasons better at depth about Clojure code than GPT-4o and consequently is the superior choice for complex coding. GPT-4o is still useful for producing text, though. It isn't exactly bad at coding, but it has a tendency to vomit code at you at every opportunity, which is both tiresome and expensive. Also, it kind of loses track when conversations get very long. And they do; the chains of thought are sometimes complex, and a meandering AI can get costly. Therefore using Claude saves money in the long run. By the way, it's easy to tell when I am writing. Just look for signs of British English. You know, -ise and colour and whilst and so forth. The AI invariably produces American English. As I windsurf through parentheses on my holiday, reviving the spirit of Igor Engraver in the form of FrankenScore, I'm struck by a profound realisation: this is how programming should always feel. Free. Uplifting. Intellectually stimulating. A far cry from being shackled to the oars of enterprise galleys, with some middle manager shouting "ATTACK SPEED!" at bewildered code monkeys. But why should this freedom be a holiday exception? As programmers (not "developers," please!), we should be grounded in computer science thinking. We need to regularly return to these ancient founts of wisdom, like Lisp, and apply their lessons to our everyday work. Otherwise, we're just highly paid button-pushers in a digital sweatshop. Remember when computer science curricula started with Scheme? It wasn't about the language; it was about learning to think algorithmically. Then Oracle, in its infinite wisdom (read: hunger for "cannon fodder"), saw Scheme replaced by Java Enterprise. And thus began the great shitshow that's lasted for decades. Yet, for all its faults, we must tip our hats to Java for gifting us the JVM. And here's where Clojure enters, marrying Lisp's elegance with the JVM's robustness and interoperability. It's like finding out your eccentric uncle and strait-laced aunt had a brilliant love child. But thanks to the JVM, your weird uncle can now fit into the enterprise world. Diving into Clojure led me to Rich Hickey's talks. The man veers into philosophical territory faster than a Silicon Valley startup pivots to blockchain. He ponders things like what names are, and why we use them - essential musings for any first-class programmer. It reminds me of my friend Niklas Derouche, architect and coder extraordinaire, who insists you must read Derrida to be a proper architect. Because nothing says "I understand this codebase" like a healthy dose of deconstruction theory. And he is right. Make no mistake. In three weeks of holiday hacking, I've made more progress and felt more fulfilled than in months of enterprise work. It's a stark reminder of what's possible when we shed unnecessary constraints and return to first principles. So, fellow coders, I challenge you: When was the last time you felt truly free in your programming? Perhaps it's time we all took a holiday to rediscover the Lisp arts. Who knows, you might just find your programming parentheses - I mean, paradigms - shifted. P.S. If you're about to comment that 'modern' languages and frameworks are just as good, save your breath. I'd sooner believe in the tooth fairy than in the supposed superiority of JavaScript or the 'agility' of SAFe. P.P.S. If you missed the Ben Hur reference (you uncultured git), this is sprint execution according to SAFe, with the CTO watching: |
AuthorPeter Bengtson –composer, organist, programmer, cloud architect. Currently windsurfing through parentheses. Archives
September 2024
Categories
All
|
|
FrankenScore is a modern, open-source music notation software designed to handle complex musical scores with ease. It is designed to be a flexible and powerful music notation software tool providing professional, extremely high-quality results. The core functionality includes inputting music notation, formatting scores and their parts, and printing them. Additional features can be added as plugins, allowing for a modular and customizable user experience.
|