Michael Gogins
Introduction
From time to time the Csound list hosts discussion of proposed changes to Csound: making the language easier to use, rewriting Csound itself in C++, adding new features, and so on.
The following presents my suggestions for modernizing Csound. I have offered some of these at various times on the Csound list, and other places. This document updates my suggestions, adds to them, and, I hope, makes them more understandable and more consistent.
My Background
First, allow me to outline some relevant aspects of my background in the field, not to impress, for many musicians and programmers are more accomplished, but to show how my suggestions arise from experience.
To began with, I have created some computer music, some of which has been performed in public venues (at the 1998 International Computer Music Conference, among other places). And I have published articles on algorithmic composition (in Computer Music Journal, among other places).
I have been using Csound since 1988. I have developed alternative versions of Csound for my own use, including an ActiveX, re-entrant version of Csound that could be used as a plugin in Visual Basic or C++ projects. I did some work on the Extended Csound project, during which time I discussed some of the suggestions here with Barry Vercoe. I have actually implemented some of these suggestions in my own versions of Csound.
Finally, I have developed completely from scratch Silence, a computer music system in pure Java, designed for both algorithmic composition and algorithmic synthesis. The synthesizer in Silence features ports of several well-known software instruments from other systems, including Csound instruments and some of Perry Cook's Synthesis Tool Kit (STK) instruments, all rewritten in pure Java. I have used Silence to compose some of my pieces, and I am now using it to synthesize pieces as well. I am currently adding a Java port of Christopher Penrose's phase vocoder to Silence, and when I am done, Silence will have most of the synthesis and signal processing facilities of Csound or Cmix, run on all Java platforms, and be completely user-extensible by means of plugins for compositional algorithms, signal processing effects, and software instruments.
What this means is that I actually have done all the work for Silence that would have to be done to modernize Csound according to my suggestions.
As for programming, I learned to program by doing computer music, not the other way round. Currently, I am employed as a senior software engineer at SunGard Trading Systems, where I am the technical lead on a new project, an on-line foreign currency trading system built using Java server and client programming and C++ server programming.
Musical and Engineering Background of Csound
Barry Vercoe wrote Csound in 1985, when C was just being standardized and C++ hardly existed. Even though Csound is written entirely in C, at least one aspect is actually object-oriented. The OPDS structure for opcodes not only has function pointers as members, but also serves as the base class from which new opcodes can be derived. Therefore, Csound's opcodes are classes in every sense of the world.
Musicians and programmers from all over the world soon began contributing to Csound, first under the oversight of Barry Vercoe, lately with John Fitch coordinating. Major improvements include:
Although Csound has become the most functional single piece of music software in existence and arguably the best musical instrument ever created, the internals of Csound have been repeatedly refined, but never thoroughly re-engineered on modern principles of software engineering. Csound continues to be essentially a 1980's era Unix command, a single statically linked hunk of code, and it has a maze of conditional compilation macros to enable it to run on different platforms.
In the meantime, software engineering concepts and practices have significantly evolved. The most important change has been, not object-oriented programming or new languages as one might think, but rather components. Software components range from the shared libraries that make up operating systems, to plugins in Web browsers, to custom controls embedded in documents and programs, to plugin audio input/output drivers like ASIO, to signal processing plugins in audio editing programs, like DirectSound plugins. Components factor out relatively independent chunks of functionality from software, and make it easier to write new software by re-using existing components.
What is a component? A component is simply a shared library with a well-designed and well-defined external interface.
Plugins are the most useful components. A plugin is simply a component that is automatically loaded and linked by its host program at run time. Usually, many different plugins are written to implement one abstract interface that is called by the host, for example the ASIO drivers and effects plugins used by VST programs. Plugins not only make it easier to write new programs, just like other components, but more importantly, they enable new functionality to be added to the host program without having to modify it, by independently writing new plugins instead.
Another significant development in software engineering is the gradually increasing dominance of Unix and Linux (essentially a variant of Unix) for serious development, and the concomitant growth of the open source movement. The new Mac OS uses a Unix kernel (Free BSD), and it becomes easier and easier to develop Unix-like software on Windows using Cygwin or Nutcracker. Furthermore, Unix (and increasingly, Linux) continue to dominate academic computer music.
As far as musical software engineering is concerned, Csound now has a great deal of competition, with more popping up every day. There are now literally dozens of software synthesizers and signal processing programs, both academic and commercial, and several competing sound processing languages, notably Cmix, SuperCollider, and Structured Audio Orchestra Language (SAOL). This last is potentially the most important, for Barry Vercoe's student Eric Scheirer designed it, it has a more modern language design, and it is already an international standard (MPEG-4) with several implementations.
Commercial and pop music oriented software programs such as Cubase, Reaktor, and Buzz have developed their own sophistication, with graphical patch languages and robust, excellent-sounding instruments and effects. The Cubase digital audio sequencer, the Cool Edit Pro hard disk recorder and sound editor, and the Buzz tracker already use a plugin architecture. Cubase and other current digital audio sequencers, and Buzz, actually accept both plugin software synthesizers and plugin signal processing effects.
In spite of all this, for various reasons none of Csound's competition has managed to dethrone it for "serious" computer music (yet).
SAOL would be capable of replacing Csound if it did not require an external C compiler, but so far there is no efficient self-contained SAOL runtime. Furthermore, SAOL lacks a built-in phase vocoder, which is one of Csound's greatest strengths.
Cmix is very capable, and offers most of the functionality of Csound, but it runs only on Unix and Linux, and for instrument development it too requires an external C compiler.
SuperCollider runs only on the Mac OS.
Commercial and pop-music oriented software synthesizers, most notably Reaktor and Buzz, in spite of their advanced software engineering, still lack the phase vocoder, and remain too closely tied to the MIDI standard or the "tracker" concept for actually representing scores, which is a crippling limitation. Both MIDI and trackers quantize time and pitch far too coarsely for serious composition.
Some other sound processing languages are essentially dialects of LISP and will not find a wide base of users in today's environment, regardless of their capabilities.
In short, any program capable of replacing Csound must satisfy at least the following requirements:
There is no immediate prospect of such a program, so Csound's dominance of our little world can be expected to continue. And as long as it does, contributors will continue to ensure that the latest developments in sound synthesis, signal processing, and so on are added to Csound.
Proposal for Modernizing Csound
In the first place, it is vital to maintain backward compatibility with existing scores, orchestras, instrument definitions, and samples. Any changes to the Csound language should be incremental changes: additions to what exists, not of what exists. If radical changes are desired, look to other systems such as Cmix (which uses C), SuperCollider, SAOL, or my own Silence (which uses Java).
Furthermore, it would be unwise to rewrite more of Csound than absolutely necessary. In spite of the advantages of modernizing Csound, the original design and the existing code are both of unusually high quality. The existing opcodes represent a staggering amount of work, which would difficult indeed to replace. Several highly skilled man-decades reside in those opcodes. To rewrite and debug them at the going rate would cost millions of dollars. As for the orchestra and score compilers, they are not broken, so there is no reason to fix them.
Even more valuable work resides in the instrument definitions themselves, which amount to a virtual encyclopedia of sound synthesis and signal processing, as demonstrated in The Csound Book by Richard Boulanger.
Taking all of the above into account, I propose modernizing Csound as follows:
I will now discuss the first two proposals in more detail. The details might require change or amplification, but I am confident that the basic ideas are sound.
Notice that I make no proposal regarding a GUI. If Csound becomes a plugin, writing a Csound GUI becomes much simpler than it is today.
Csound Plugin Interfaces
The Csound plugins should be declared in C, using object-oriented C, not C++. The GNU libtldl library, which works across many platforms, should be used for loading plugins. As mentioned above, Csound components should be implemented by adapting the existing Csound code, not by rewriting the opcodes or compilers.
Extensive, but not complex, changes to the Csound code are required to create these components in the best way.
To begin with, it is necessary to actually finish, and begin using, the RESET code contributed by Richard Dobson. Only this will enable Csound to be restarted after it has stopped without having to reload Csound. I did this work without difficulty for my own versions of Csound.
After that, it is desirable to make all current global structures, variables, and functions, such as O, opcodlst, and midiin, members of a central Csound structure or class, which would require global functions to take a pointer to this structure as their first parameter. This would make Csound into a completely object-oriented program (though it would still be written in C). This is the best way to enable multiple Csound plugins to be used in the same process at the same time.
Fortunately, these changes, although tedious, are quite straightforward and can safely be made without changing the existing logic or breaking any code.
The Csound plugins should be declared roughly as follows. Note that if the second suggestion (making Csound completely object-oriented) is not followed, the csound parameter can be omitted from many functions.
typedef struct{
// Functions and structures to be set by Csound itself.
OPARMS O;
OENTRY *opcodlst;
FUNC *ftlst;
int (sensMidi*)(Csound *csound);
int (sensFMidi*)(Csound *csound);
int (sensLine*)(Csound *csound);
int (sensOrcEvent*)(Csound *csound);
// All other existing global objects and functions
// also become members of this...
// New functions to manage plugins inside Csound.
int (loadOpcodes*)(Csound *csound, char *path);
char *(getOpcode*)(Csound *csound, int index);
int (loadFunctions*)(Csound *csound, char *path);
char *(getFunction*)(Csound *csound, int index);
int (loadPinstrs*)(Csound *csound, char *path);
char *(getPinstr*)(Csound *csound, int index);
int (loadMidiDrivers*)(Csound *csound, char *path);
char *(getMidiDriver*)(Csound *csound, int index);
int (setMidiDriver*)(Csound *csound, int index);
int (loadAudioDrivers*)(Csound *csound, char *path);
char *(getAudioDriver*)(Csound *csound, int index);
int (setAudioDriver*)(Csound *csound, int index);
int (loadLineDrivers*)(Csound *csound, char *path);
char *(getLineDriver*)(Csound *csound, int index);
int (setLineDriver*)(Csound *csound, int index);
// New functions for operating Csound (supplementing main()).
void (open*)(Csound *csound, FILE *csdFile);
int (start*)(Csound *csound);
int (stop*)(Csound *csound);
int (tick*)(Csound *csound);
char* (getOption*)(Csound csound*, char *option);
void (setOption*)(Csound csound*, char *option, char *parameter);
void (main*)(Csound csound*, int argc, char *argv[]);
void (printUsage*)(Csound csound*, FILE *file);
void (destroy*)(Csound *csound);
} Csound;
typedef Csound *(CsoundFactory*)();
// Used in all places in Csound where MIDI, audio, or control data is
// read from the system or external drivers (except for files).
typedef struct{
int (open*)(CsoundReader *reader);
int (read*)(CsoundReader * reader, int bytes, void *data);
int (control*)(CsoundReader *reader, int cmd, int size, void *data);
int (close*)(CsoundReader *reader);
void (destroy*)(CsoundReader *reader);
} CsoundReader;
// Used in all places in Csound where MIDI, audio, or control data is
// written to the system or external drivers (except for files).
typedef struct{
int (open*)(CsoundWriter *writer);
int (write*)( CsoundWriter *writer, int bytes, void *data);
int (control*)(CsoundWriter *writer, int cmd, int size, void *data);
int (close*)(CsoundWriter *writer);
void (destroy*)(CsoundWriter *reader);
} CsoundWriter;
Note that these driver interfaces represent the lower halves of the drivers, that is, functions in the drivers to be called by the Csound kernel.
The upper halves of the drivers consist of functions or structures in Csound, such as the sensmidi flag, to be accessed by the drivers. These are available to the drivers via the Csound object passed in the FACTORY function. I have not defined upper half functions and structures here because, for most part, they already exist and would only need to be declared as such.
It must be admitted that moving the existing driver code into independent plugins probably represents the most difficult part of implementing my suggestions, mainly because it would need to be tested on many different platforms and configurations.
// All plugins for Csound create an instance of themselves // via this function, whose return value must be type cast to the // appropriate plugin function or structure. typedef void *(FACTORY*)(Csound *csound); // Each plugin library must export one or more of these functions, // which are called by Csound to discover the plugin classes // or plugin factories. // Csound calls each function repeatedly until it returns 0 // to enumerate available plugins. // Note: actual loading of plugins is done using the ltdl library. int csRegisterOpcode(Csound *csound, int i, OENTRY **oentry, char **name); int csRegisterFunction(Csound *csound, int i, SUBR *gensub, char **name); int csRegisterMidiReader(Csound *csound int i, FACTORY *create, char **name); int csRegisterMidiWriter(Csound *csound int i, FACTORY *create, char **name); int csRegisterAudioReader(Csound *csound int i, FACTORY *create, char **name); int csRegisterAudioWriter(Csound *csound int i, FACTORY *create, char **name); int csRegisterLineReader(Csound *csound int i, FACTORY *create, char **name); int csRegisterPinstr(Csound *csound int i, FACTORY *create, char **name);
I have already implemented a version of Csound with plugin opcodes, using a form of the csRegisterOpcode function, and this code is still being maintained in Gabriel Maldonado's DirectCsound. The main change required to my existing plugin code is to use the GNU libtldl facility for loading the plugin libraries and accessing their registration and creation functions.
Note the tick function in the Csound declaration. The purpose of this function is to enable external programs hosting Csound as a plugin to control the timing of synthesis, by requesting the computation of one ksmps worth of sample frames at a time. Only something like this would make it possible, e.g., to use the Csound kernel in a simple and straightforward way to create a VST2 plugin version of Csound, which would work as a synthesizer or effect in Cubase. This is because Cubase and other such hosts need to be in control of their plugins. For example, Cubase calls VST2 instruments and expects to receive a filled-in sound buffer in return.
I am familiar with this issue because I have already tried and failed to create a Csound VST2 plugin. I found that, in order to get my plugin to work, I would have had either (a) to change the Csound playevents function to be driven from outside Csound, or (b) to introduce a mutex into all of Csound's output routines, to be signaled by the VST2 host. I was not confident the second approach would work well enough. Furthermore, I was not willing to maintain these changes if they were not adopted by canonical Csound, and I did not think that was likely, so I quit.
This change would require some modifications of the musmon and playevents functions.
Note that a VST2 Csound plugin would vastly exceed the capabilities of all other VST2 plugins put together: it would enable one to use any Csound instrument inside Cubase, to easily program one's own filters and reverbs, or to schedule and mix phase vocoder sounds using music notation and the Cubase software mixer, and so on, and so on....
To illustrate the use of plugins, the following C program, which is written and compiled the same way on all GNU platforms, exactly reproduces the behavior of the current Csound program, using the Csound kernel plugin:
#include <csound.h>
#include <ldtl.h>
int main(int argc, char *argv[])
{
lt_dlhandle csoundLibrary = lt_dlopen(libCsound.so);
CsoundFactory csoundFactory = (CsoundFactory) lt_dlsym(csoundLibrary, "csoundFactory");
Csound *csound = csoundFactory();
csound->main(csound, argc, argv);
csound->destroy(csound);
}
Additions to the Csound Score and Orchestra Languages
A new m statement should be added to Csound's score language. This statement would represent MIDI channel messages by translating them in real time into score line events, with the following pfields:
In other words, real-time MIDI performance control messages would enter Csound as real-time score events using regular pfields, not using the MIDI opcodes (which, however, would be retained for backward compatibility).
A similar facility exists in SAOL, and I have implemented this idea in Silence.
For example, playing middle C for a quarter note on a MIDI keyboard might result in the following real-time score line events:
m 1 0.5 -1 144 60 72 m 1 0.75 0 128 60 0
However, m statements could also be used in place of i statements. The advantage is that both real-time and non real-time performance have the same semantics. Therefore, the same instrument definition can be used for real-time MIDI performance, MIDI file performance, and non real-time score performance. This greatly reduces the work required to maintain a versatile library of instrument definitions.
For example, the m statements
m 1 0.5 -1 144 60 72 m 1 0.75 0 128 60 0
would be functionally equivalent to the m statement
m 1 0.5 0.75 144 60 72
which would be functionally equivalent to the i statement
i 1 0.5 0.75 144 60 72
A new pinstr statement should be added to the Csound orchestra language (for "plugin instr"). The instrument definitions themselves would be plugins similar to opcodes. The purpose of pinstr is to make it possible to write Csound instrument definitions directly in C or C++, which are more powerful than the Csound orchestra language.
The pinstr newInstance function will return a finished INSDS structure, to be inserted directly into the playlist without having to be compiled
The pinstr statement would accept initialization parameters in the orchestra file, for example:
pinstr 4 ; Klangumwanderized Rhodes piano ringModulatorGain = 0.25 ringModulatorFrequency = 440 vibratoDepth = 0.5 vibratoFrequency = 5.3 endin
This change would require modifications to oload, rdorch, otran, and musmon. In addition, the OENTRY and OPDS structures need a new function
SUBR dopadr
to be called by Csound when an instrument instance is deallocated on reset. This will enable plugin instruments or opcodes to perform their own memory allocation and deallocation, and to release allocated memory and other system resources on reset.
Simplifying Opcode and Pinstr Programming
In order to simplify the development of new Csound opcodes and plugin instruments (pinstrs), I propose creating two C++ base classes with some convenience functions, declared as follows.
class Opcode : public OPDS
{
// Variables that used to go into opcode structures (such as FOSC)
// become members of classes derived from this,
// as FLOAT * values, with outputs first and inputs second.
public:
// Automatically wire the C++ functions
// into the Csound structures.
// By default, Opcodes are arate,
// but krate can be wired in by derived classes.
Opcode(Csound *csound)
{
OPDS::iopadr = &iopadrDispatcher;
OENTRY::iopadr = &iopadrDispatcher;
kopadr = &kopadrDispatcher;
aopadr = &aopadrDispatcher;
OPDS::opadr = &aopadrDispatcher;
dopadr = &dopadrImplementation;
}
virtual ~Opcode(){}
static void iopadrDispatcher(OPDS *opds)
{
((Opcode *)opds)->iopadrImplementation();
}
virtual void iopadrImplementation(){};
static void kopadrDispatcher(OPDS *opds)
{
((Opcode *)opds)->kopadrImplementation();
}
virtual void kopadrImplementation(){};
static void aopadrDispatcher(OPDS *opds)
{
((Opcode *)opds)->aopadrImplementation();
}
virtual void aopadrImplementation(){};
static void iopadrDispatcher(OPDS *opds)
{
((Opcode *)opds)->iopadrImplementation();
}
virtual void dopadrImplementation(){};
// Returns an OENTRY structure for this.
virtual OENTRY *getOENTRY()=0;
// Convenience functions.
FUNC *getFunction(int number);
int getSR();
int getKR();
int getKSMPS();
FLOAT getESR();
FLOAT getKSR();
};
With this design, a new opcode can be created simply by deriving a new class from Opcode, adding member variables for data, and overriding the virtual iopadrImplementation, etc., functions to do the actual work.
class Pinstr : public Opcode
{
Csound *csound;
FLOAT *signal;
public:
` Pinstr(Csound *_csound) : csound(_csound)
{
}
virtual ~Pinstr(){}
// Called by Csound to initialize patch parameters from the orc file.
virtual initialize(char *name, char *value)=0;
virtual output()
{
csound->out(csound, signal);
}
// Called by Csound to get a new instance for the playlist.
virtual INSDS *newInstance()=0;
};
The purpose of this class is to enable Csound to call the newInstance function during performance to create a new instance of the instrument, and to call initialize for each initialization parameter in the orchestra file, in other words to create a new copy of the instrument template. The output function is called by the instrument's Opcode::iopadrImplementation function, to output its audio signal using one of the Csound output opcodes in its C form.
I have implemented a similar facility in Silence, which has a Unit base class for opcode plugins, a SignalFlowUnit base class for signal processing effects plugins, and an Instrument base class for software instrument plugins.
Conclusion
If my suggestions were implemented, Csound would become modern software without changing its existing logic or unnecessarily changing existing code.
Csound would remain backwardly compatible with all existing Csound instruments and music.
It would be possible to link Csound into other programs on Unix, Linux, Windows, or the Macintosh without any effort. For example, it would be possible to add Csound as a completely functional software synthesizer to a custom-written algorithmic composition system or program, and it would be possible to write a VST2 instrument version of Csound, or to use Csound itself to write specialized VST2 instruments.
It would be possible to start Csound, stop it, and restart it for the same or a different piece without reloading it.
It would be possible to write new MIDI, audio, or line input and output drivers for Csound on any computer with a minimum of effort, and they would begin to work in Csound without having to modify the Csound kernel itself.
It would be possible to use the same instrument definitions for all kinds of performance: real-time MIDI control, MIDI file input, score input, and line input.
And it would be possible to quickly write new opcodes, and even new instrument definitions, in C++, and they too would begin to work in Csound without having to modify the Csound kernel itself.