Realtime Systems 1

    CSOUND JOURNAL ISSUE 23

    INDEX | ABOUT | LINKS






    Introduction

    Working with live electronics and real-time in Csound is a large subject area. In a way, all creative and performance practice with Csound can be said to fall within the area of live electronics —from simple playback of audio files, to the use of the same audio signal to trigger other events. When designing Csound systems for live performance, we must consider an number of issues and context-dependent requirements. This article discusses these issues and showcases a number of professional, and commercially released, solutions that were developed to realize two classic compositions: one by John Cage and another by Karlheinz Stockhausen. These complex working systems, whose requirements were set by their composers a long time before Csound existed, will help the reader to better understand how to manage their own real-time performance systems using Csound.

    I. Field of Intervention Csound-based Systems for Real-Time Interactive Chamber Music Performance

    In recent years I have been researching some of the possibilities of utilizing Csound for signal routing in real-time in the concert hall, and I realized that there are several applications for this research. Considering my training and also as I am basically an interpreter of music, I am particularly attracted to the idea of the use of an audio signal that has been produced by a musical instrument and coming from a microphone, being used to set in motion a kind of one-man performance.

    Wanting to define this more clearly, I saw and I experienced some possible areas of the application of Csound in live electronics:

      I. The signal captured with one or more microphones, and optionally:
        A. a returned unaltered signal (but perhaps with some routing to direct the sound output), and
        B. signal synchronization with other events, perhaps with suitably prepared sounds.

      II. The signal captured with one or more microphones, which is passed into Csound and then utilized or changed by molding and shaping through:
       A. the application of effects (FX), such as: ring modulation, phaser, flanger, amplitude following filter, chorus, harmonizer, reverb, stretching, pvs opcodes, glitching, distortion, spatial processing, granulation, pitch change and others, or
       B. the application of effects (FX) on the signal repeated at various time intervals with delay, utilizing opcodes such as fout, or diskin2 in this task.

      III. The signal captured with one or more microphones, analyzed for its basic parameters (mainly the amplitude and frequency) and employed as:
       A. a monitoring tool to trigger other events,
       B. or used as a switch, which could be defined as a choice of possible options in the sequence or path to an algorithm; something similar to a switch or to a switch that opens the door to different choices in real-time.

      IV. The mix of the various fields utilizing those from the above (I, II, or III).

    We can leave out the operation of sound location in real-time. I personally believe both in acousmatic music, and when the instrumentalist or singer is merely superimposed on a musical event (on the sounds of support, or on tape), crystallized and already decided in advance, that it is less sensible to speak of live electronics. Because in that case in fact there is no reaction, and the attention is unbalanced on the executioner, that behaves almost like an automaton to synchronize (more or less freely within the constraints imposed in the score) its interventions on what, in principle, remains unchangeable. I believe more in a concept of live electronics in which the relationship between performer and machine is focused on interactivity, and not only on technology. Thus the performance is based on an intelligent musical reaction as well as one technological.

    Examples from the literature of live electronics will be discussed in this article (and some with tutorials), with more or less detailed reference to the Csound language adopted. These are: Solo [Nr.19] by K.Stockhausen (digital setup in Csound by E.Francioni) based on delay lines, Ryoanji by J.Cage (digital setup in Csound by E.Francioni) based on synchronization with sound support, Cluster by E.Francioni - based on the timeline (timeinsts) and the delay-lines, eSpace by E.Francioni , an example of spatial signal quadraphonic, Accompanist by J.Fitch, based on the reaction to rhythmic impulses, Claire by M. Ingalls, based on the pitch and the randomness of the reaction, Etude 3, an experiment of B. McKinney, based on pvspitch, and Im Rauschen, cantabile by L.Antunes Pena, based on keyboard commands (sensekey).

    II. Some general questions on Live Csound

    Before doing any live electronics with Csound you need to know, and keep in mind some basic settings. These are settings used to optimize the system. For all frontends commonly used (such as CsoundQt, Cabbage, WinXound, Blue, etc...) we propose preliminary settings that must be met for a successful performance in terms of the rationalization of operating system resources, and reducing signal latency.

    In addition to the general settings, there are specific settings dedicated, in detail, to what we are trying to achieve.

    It is also necessary to respect the basic norms of coding efficiency for compiling Csound. This will contribute greatly to the rationalization of resources, and the smoothness and speed of reading the code by Csound.

    For this purpose, it seemed particularly important what Heintz[1] also suggests about the settings and precautions to keep in mind for a success in live performance.

      1. Instruments are the main building blocks in Csound. For a good design of Live Csound, it is important to decide which task should be assigned to each instrument. If an instrument has to receive any control message, like MIDI or OSC, it usually has to be always on for listening. If a trigger signal is received and another instrument is turned on, which may just flash some light in the GUI, or set some value, the called instrument should be turned off immediately after having done its job[1].

      2. It is crucial for this design to understand one idiosyncrasy of Csound instruments which is not always transparent, even for experienced Csound users: the difference between the initialization and the performance passes. Briefly, this means once an instrument is called, all i- and k-variables are initialized. While the i-variables stay as they are, the k-variables will be updated in every performance cycle, i.e. every ksmps samples[1].

    This list from [1] continues also with guidelines below for Live Csound:

      3. "Use different instruments for the different jobs which are to be done in your live configuration"[1].

      4. "Use the init pass of an instrument for setting values whenever possible, and turn off any instrument as soon as its job is done"[1].

      5. "Split jobs if the instrument becomes too complex"[1].

      6. "Use User-Defined Opcodes (UDO's) and macros to modularize your code"[1].

      7. "Tweak the performance and the latency with the vector size (ksmps) and the buffer size settings using the command line flags -B, and -b"[1].

      8. Avoid some opcodes which are not intended for real-time use. For example, do not use the old pv opcodes for Fourier transform/phase vocoding, like pvoc or pvcross. Use the pvs opcodes instead (pvsvoc, or pvcscross, etc). For another example, do not use opcodes for convolution which are not good for real-time performance, like convolve or dconv. Use pconvolve or ftconfv instead[1].

      9. Csound's performance for live applications depends mainly on its vector and buffer sizes. As mentioned above, the ksmps constant defines the internal vector size. A value of ksmps = 32 should be adequate for most live situations. The software and hardware buffer sizes must not be kept at Csound's defaults, but should be set to lower values to avoid audio latency. CsoundQt offers an easy way to adjust the Buffer settings in the configuration panel[1].

    In terms of a one-man-performance it is also of considerable importance to use an effective and functional graphical user interface (GUI) that allows the performer some basic controls such as prior checking of parameters, a Play/Stop/Pause control during the performance (if it is made possible by the one-man performance), the ability to view useful parameters (metronomes, chronometers, lines to display, etc.), synchronization space/time with other events, the possibility to test only parts or sections of the piece under study, the ability to modify the speed of the execution of the code under study, and the possibility to insert presets on the type of execution to be used.

    III. Csound for Cage's Ryoanji - A possible solution for the Sound System. The generation and management of the Sound System: in deferred-time and real-time

    Among the historical works dedicated to live electronics one cannot forget Ryoanji by John Cage, a work already described in my article Csound for Cage's Royanji in Csound Journal[2], in which reference is made especially for the part relating to the description of historical and interpretive aspects. The article also features several pages of the original score. The following paragraphs illustrate the basic steps for thinking of a possible solution for the installation of the sound system, as well as the design, implementation and use for Cage's work. While I have not yet created an App of this algorithm, it is my intention to do this in the near future.

    The project

    In this discussion we will analyze the version for a live contrabass and sound support. Sound support, is simply a tape, made up of three double basses, percussion and vocalise ad libitum. Considering that the reader is probably already familiar with this piece by Cage, the first question is "What do we want?". The answer is simple. We want to set up the sound system for Ryoanji but include the use of Csound. How is the sound system set up with the use of Csound? The performer needs a patch and a graphical interface to prepare the exhibition. He has the control of the preparatory material and then he performs the piece! What are the problems we face and what things do we need to know for doing that? The way I develop this project, regardless of the version that we have decided to set up, is that Csound cannot be reduced to a simple player for playing sound support (tape). Csound can be used instead in a more creative and imaginative way especially for the choice of materials and the type of control over the sound that we want to activate. We could write an algorithm that allows us to perform at least two activities: a preparatory performance (in deferred time) to be followed, and one dedicated to real-time.

    In any case the patch that I propose for Ryoanji that I will comment on below in some basic steps, has been prepared to satisfy both needs from above. Basically the larger task will be the preparation of the sounds of support, and we can decide to carry out that task in various ways.

    For working with a live contrabass soloist there will be particular issues with the coding for Csound. Thinking of a one-man performance, it is important that the soloist has control of the various elements before and during the performance. For this reason it is essential to have the help of a GUI for a preliminary calibration of execution and access to some basic controls such as the tools to carry out the preparation phase, a Timer (to understand where you are in execution), the metronome (in this case set to 60 MM), the skiptime (which under study allows us to be able to perform the song from chosen points), and the possibility of excluding some controls to save cycles in the CPU.

    The patch may be set in the way that the three parts of the contrabass of support can be implemented, at the discretion of the interpreter, using one of the following methods:

      • recording studio audio tracks of bass (2, 3 and 4) for later import into the algorithm in real-time (for example with diskin2, soundin or mp3in),
      • recording of audio samples in deferred time, and stretching it in real-time,
      • or analysis of audio samples in deferred time, and resynthesis timed and stretched in real-time.

    It is very important to be aware that the last two methods above, if using the algorithm in real-time. That is, during the performance, those willrequire a significant amount of resources in terms of CPU .

    For the realization of the pulses of the necessary percussion, accompanying the whole piece, you could used two separate libraries of audio samples of sounds of metal and sounds of a timber (as shown by the author), perhaps bringing into play a collection of physical models utilizing Csound.

    The vocalise ad libitum is notated in the score with horizontal lines graphically differentiated as shown below in figure 1.

    ______ / _ _ _ _ / . . . . . / _ . _ . _ .

    Figure 1. example ad libitum notation in Cage's Ryoanji.

    These could be realized by following the strategy adopted for the three basses, then loaded and recalled also as sounds for support in order to facilitate a one-man performance.

    At a higher level of observation presented here is a design as the basics of an algorithm in the following block diagram.

    For the realization of the successful execution of our dedicated algorithms in real-time and using live electronics, it is essential to keep in mind the points already set out in the second Section of this article, to which I would add, in this case, the following recommendations:

      • all files retrieved from the code (wav, mp3, pvx, txt) necessary for performance, are in the same folder as the csd,
      • and in this case it is recommended to set the header during record, using the following values:

    sr = 44100
    ksmps = 260 

    Then set the header during the playback to the follow values, omitting the kr.

    sr = 44100
    ksmps = 64

    The patch can be thought of as divided into two parts: one part of the code for recording, and the other for playback. This will appear to the user as two different types of operation. A record tool has the task of recording and saving the audio samples (.wav) of the bass where the pitch C2 is on all four strings, as well as audio samples (.wav), and simultaneously files pvx, such as voice (vowels I, O, A, E corresponding to the four parts of the vocalise). Below is the fundamental body of the code.

    instr	rec	; recording file wav and pvx (one track at a time)
    
    aIn		inch	1
    
    if		i(gkrec)	==	1 	then
    Sfile		=	"/Users/enrico/Desktop/Ryoanji/myCbC2_I.wav"
    fout		Sfile, 2, aIn
    elseif	i(gkrec)	==	2	then
    Sfile		=	"/Users/enrico/Desktop/Ryoanji/myCbC2_II.wav"
    fout		Sfile, 2, aIn
    elseif	i(gkrec)	==	3	then
    Sfile		=	"/Users/enrico/Desktop/Ryoanji/myCbC2_III.wav"
    fout		Sfile, 2, aIn
    elseif	i(gkrec)	==	4	then
    Sfile		=	"/Users/enrico/Desktop/Ryoanji/myCbC2_IV.wav"
    fout		Sfile, 2, aIn
    
    elseif	i(gkrec)	==	5	then
    Sfile		=	"/Users/enrico/Desktop/Ryoanji/myVx_I.wav"
    fout		Sfile, 2, aIn
    fss pvsanal  aIn, 1024,256,1024,0
    pvsfwrite fss, "/Users/enrico/Desktop/Ryoanji/myVx_I.pvx"
    elseif	i(gkrec)	==	6	then
    Sfile		=	"/Users/enrico/Desktop/Ryoanji/myVx_O.wav"
    fout		Sfile, 2, aIn 
    fss pvsanal  aIn, 1024,256,1024,0
    pvsfwrite fss, "/Users/enrico/Desktop/Ryoanji/myVx_O.pvx"
    elseif	i(gkrec)	==	7	then
    Sfile		=	"/Users/enrico/Desktop/Ryoanji/myVx_A.wav"
    fout		Sfile, 2, aIn
    fss pvsanal  aIn, 1024,256,1024,0
    pvsfwrite fss, "/Users/enrico/Desktop/Ryoanji/myVx_A.pvx"
    elseif	i(gkrec)	==	8	then
    Sfile		=	"/Users/enrico/Desktop/Ryoanji/myVx_E.wav"
    fout		Sfile, 2, aIn
    fss pvsanal  aIn, 1024,256,1024,0
    pvsfwrite fss, "/Users/enrico/Desktop/Ryoanji/myVx_E.pvx"
    endif
    
    endin
    

    It is recommended that the dialogue between the performer and the various parts of the orchestra always happen through the Widgets Panel, not only for this but also generally in works dedicated to real-time.

    For playback (Play) there at least four sectors of code:

      • the soloist contrabass,
      • the three basses of support,
      • percussion and
      • the vocalise (ad libitum)

    Regarding the bass solo part, it is "suggested" to be employed in the live performance, perhaps with signal corrections, but the three basses of support are required or fixed parts of Ryoanji. It was decided to choose two possibilities among the choices for the intended audio material to realize the three audio tracks, utilizing one mp3 for each part of bass as previously recorded. This approach will resemble more a performance like solo + sound support.

    By implementing this option the work of the algorithm becomes much more complex, challenging and creative. Code is required for every bass's dedicated calls (schedule) of another instrument that performs stretching of the samples and passing all the values needed to do it. Below is shown this step in regards to the code for the contrabass 2 part.

    instr CB2
    
    if 	i(gkCB234) = 1 then
    gaCB2		soundinmac	"/Users/enrico/Desktop/Ryoanji/CB_02-mn.mp3", \
    		1, i(gkskiptim)
    gkled2=0
    elseif 	i(gkCB234) = 2 then
    clear	gaCB2
    gkled2 = 1
    
    ;	       p1	p2	p3	p4	p5	p6	p7	p8	p9	p10	p11	p12	p13	p14	p15	p16	p17	p18	p19
    schedule	"cb2",	57.85,	7.85,  3,	1,	2,	1,	0,	1,	1,	2,	.3,	.3,	2,	4,	4,	10,	8,	1	
    schedule	"cb2",	82.50,	4.4,	2,	4,	1.5,	-.5,	-1,	-1,	-1,	-1,	.3,	.3,	2,	4,	4,	10,	8,	1
    ;[omissis]
    
    endif
    

    The parameters of schedule (from p4 to p19) refer to the actual values for the arguments of the opcode mincer that will implement the real-time stretching of the samples and the resulting sound output.

    gaCB1	mincer atime, kamp, kpitch, itab, ilock 

    As we said above regarding the sounds of percussion, Csound gives us some choices of semi-physical models present between opcodes, such as tambourine, cabasa, sleighbells, guiro, bamboo, cabasa. The performer should be given a chance to access these (in addition to their own samples) with a menu in the Widgets Panel. I have suggested a solution that seems effective to invoke samples of the percussion at intervals of time set out in the score. The code uses the opcode loop_le, for making timed calls connected to the instrument pc3 required for the output of the sound of the percussion.

    gir27 ftgen 27, 0, -77, -2, 0,14,26,38,51,64,76,88,101,115,130,143,157,171,186,198,211,226,241,255,269,284,297,310,324,339,351,
    365,378,392,407,422,434,447,461,475,490,505,519,534,546,561,574,589,604,619,634,647,662,676,691,704,719,734,749,762,775,787,801,813,827,842,854,866,880,895,908,923,937,951,963,977,991,1003,1017,1029,
    1043	
    ; function values gir27 mark the actions-time (p2) of the \
    	percussion, then past a table and then to schedule
    
    istart	= 0 	; i-rate variable to count loop.
    incr	= 1	; value to increment the loop
    iend	= 77	; maximum value of loop index
    
    looppoint:
    ip2  table  0, gir27 
    schedule "pc3", ip2, 10 
    loop_le istart,incr,iend,looppoint
    

    For the four double basses, one live and three from the sounds of support, and the four lines of the ad libitum part, I suggest a spatial quadraphonic setup, that could be implemented with the knowledge and use of the opcode space. In the specific case for the location of the signals of support, it seems practical that the locations of the signal are already established before the performance employing the function set in ifn.

    gir1	ftgen	1,	0,	0,	-28,	"CB01.txt" 
    ; GEN 28 delegated to the move, which refers to the text file "CB01.txt" \
    	which are written in the movements of the signal in time
    
    ifn	=	1	
    ktime	line	0, idur1, idur1
    kreverbsend	= .5
    kx	= 	0
    ky	=	0
    a1, a2, a3, a4	 space	asig, ifn, ktime, kreverbsend, kx, ky
    

    IV. Inside SOLO [Nr.19] , the Digital setup for Stockhausen's Composition

    Another important work to which you can apply the term "historic" and one dedicated to the production of live electronics is certainly Solo für Melodieninstrument mit Rückkopplung by Karlheinz Stockhausen (1966) . Published by Universal Vienna, this composition has unique importance. From the 80s to today, there were numerous approaches to solve transposition in the digital environment (Max/MSP, Pd, Kyma/Capybara, and even with the use of multitrack editors), the original analog layout, and the use of magnetic tape to obtain delays of the signal included between 6 and 45.6 seconds.

    Analog Setup

    Before becoming an App, the csd file for SOLO [Nr.19] went through a long period of trial. The App for iPad, iPod and iPhone, realized thanks to the valuable contribution of Alessandro Petrolati - apeSoft - and distributed on the App Store, is devoted mainly to the performer who wants to deal with the performance of the piece, but also for the student who wishes to examine carefully the point of view of style and composition . This App can provide away for various performers to create more personal versions dedicated to different solo instruments (violin, saxophone, contrabass) and also to voice, with a certain aptitude in the resolution of the initial difficulties of the technical setup .

    Also in this Section of SOLO, and the same for Ryoanji, I will assume that the reader has already acquired a certain familiarity with the score and the technical setup[3]. After a careful observation of the flow chart and the Form-Schema of the original, it is necessary to understand that in the digital transfer of the original technical setup, some basic requirements should be met, such as listed below.

      • Implement delay-line delays programmed in time.
      • Implement the automated system of envelopes of the four Assistants.
      • Create the possibility of entering the FX signal (Timbres).
      • Create a useful graphical performance tool that contains controls needed before and during the performance (metronomes, time-line, count-down, timers, etc).

    On a more refined level of program, we should also take care with the items listed below.

      • Ensure that the user can perform with the same app for all six SOLO Versions.
      • Insert a master metro control as general time keeper and also for skiptime for the purposes of the preparatory study of the performance.
      • Include the possibility to change the Beats per Period.
      • Enter the system of random perforations.

    Below we can summarize the technical setup of SOLO simply thinking of a signal that passes through a delay-line with the addition of a feedback as well as adding temporal envelopes.

    A simplification of the algorithm is shown below, which is an incomplete code example. This represents a possible solution for the Form-Schema (only the left channel) in which the incoming signal is subjected to a delay-line in feedback, also adding envelopes for Assistants 1, 2, 3 and 4: Microphone (1) Feedback (2), Playback (3) and Delay (4).

    gkenl1 init 0	;line envelope Assistant 1 \
    	(at instr en_L1 for envelopes of Assistant 1 left channel)
    gkenl2 init 0	;line envelope Assistant 2 \
    	(at instr en_L2 for envelopes of Assistant 2 left channel)
    gkenl3 init 0	;line envelope Assistant 3 (at instr en_L3 \
    	for envelopes of Assistant 3 left channel)
    gk_tdel init 0	;variable delay time gk_tdel \
    	(at instr Tdel - Assistant 4 - for left and right channels)
    ;===========
    ;instr setup
    ;===========
    imax =	p4		; maximum value of the delay-line; for the \
    			Version I p4 = 25.3 (seconds)
    igain_A	= .8	; attenuation value (g), before delay-line
    amic inch 1		; audio signal from the microphone
    asig = amic*gkenL1 	; microphone signal \
    			* line envelope Assistant 1 (left channel)
    adummyl delayr imax ; reads the maximum value of the delay-line
    adeL deltapi gk_tdel ; tapped signal, the variable delay time \
    			gk_tdel - Assistant 4
    awrite = (asig+(adeL*gkenL2))*igain_A ; signal + (delayed signal \
    			* envelope line Assistant 2 left channel) * attenuation value (g)
    	delayw awrite ; write in the delay line awrite
    
    aoutL = adeL*gkenL3 ; delayed signal * line envelope Assistant 3 (left channel)
    aleft = aoutL + amic ; processed signal + direct-line
    out	aleft ; output
    endin
    

    Shown below, in the partial example, is the left channel - First Assistant (Microphone) with times of only Cycle A.

    gi1	ftgen	1,	0,	4,	-2,	18,	18,	30	; etc.
    gi2	ftgen	2,	0,	4,	-2,	6,	6,	18	; etc.
    
    ; instantaneous values by score
    ia 	= p4
    ib 	= p5
    ifn1 	= p6
    ifn2 	= p7
    
    indx	init	- 1		; initialization of the index table
    ilen	=	ftlen(ifn1)	; here you get the length of the table ifn1
    reset:					; label attached to reinit
    indx		=	indx + 1		; increase in the index
    ; conditional step: if the index is less than or equal to -1 ilen then go to the label again
    if	indx <= ilen - 1  goto forth			
    turnoff				; or, if the condition occurs, turn off 
    forth:				; label attached to goto, if the condition is satisfied
    idur	 table	indx, ifn1	; reading the values idur from the table
    idur_env table	indx, ifn2	; reading the values idur_env from the table
    ; it performs another conditional jump: from time 0 will jump to label continue and continue to cook for idur seconds
    		timout 0, idur, continue			
    		reinit reset					
    ; at this point csound temporarily stops the execution and executes a further step initialization
    		
    continue:		; label attached to timout, if the condition is satisfied
    ; generation line of envelope, as a global variable 
    gkenl1	linseg 0, ia, 1, idur_env - (ia + ib), 1, ib, 0	
    
    endin
    

    As for the Timbres, or the FX signal, I suggest the use of a UDO (User-Defined opcode). The example below is the code that will be called as a spectral arpeggiator as opcode SpecArp.

    opcode SpecArp, a, akkk
    setksmps 64
     
    asigL, klfo, krange, kdepth xin
    ifftsize	=		512
    ioverlap	=		ifftsize / 4
    iwinsize	=		ifftsize
    iwinshape	=		1	; von-Hann window
     
    fftinL	pvsanal	 asigL, ifftsize, ioverlap, iwinsize, iwinshape
    kbin  	oscili	 krange, klfo, gifn510, 0	; ftable 0-1
    ftpsL  	pvsarp  fftinL, kbin, kdepth, 3  	; arpeggiate it (range 220.5 - 2425.5)
    aOutL 	pvsynth ftpsL				; synthesise it
    	xout aOutL
    endop
    

    Obviously for the Timbres to be applied to the signal the programmer can feel free to choose within the opcodes of Csound, UDO libraries available online, or by using totally customized patches. Now let us look at some aspects of the GUI . As mentioned above, it is essential to provide the interpreter with at least some essential controls for the initial setting for the one-man performance.

      • Metronomes which candisplay and scan Beats, Periods and Cycles.

    The latter instrument of control is important during the study phase for partial executions, and to apply a skiptime with a pitch for Cycles (a kind of advance statement).

      • Display of the time-line of Periods and Cycles.

      • Management of Beats per Period and the master meter.

    On an advanced level, one of the major efforts that will have to be made by the Csounder is to set the algorithm for performing all six Versions of SOLO. In this case, I prefer talking about a multi-version.

    What follows is an outline, very brief, of the entire csd.

    The process of integration of the orchestra for SOLO is simple. It is necessary to make only a few minor changes to make it compatible with the Csound API (i.e. Application Programming Interface).

    The first improvement concerning the flags of Csound, as you can see, is to disable rtaudio and rtmidi since the IO audio is managed by iOs.

    
    -o dac
    -+rtmidi=null
    -+rtaudio=null
    -d
    -+msg_color=0
    --expression-opt
    -M0
    -m0
    -i adc
    

    The second improvement concerns the use of utilizing the opcodes chnget and chnset instead of the opcodes invalue and outvalue due to better Csound API support.

    Since the architecture of SOLO requires the extensive use of the control structure (i.e. k-rate variables), and since all the processes related to the servers are automated for reasons of optimizing the performance of the first generation of iDevices, it is best to set the ksmps of Csound identical to bufferFrame of the iOs. Since Csound does not allow the dynamic setting of these general parameters (sr and ksmps), it was necessary to introduce a workaround that is to place a textual level of values on sr and ksmps. The implementation is in Objective-C and will not be discussed in this text.

    For clarification, in the orchestra's header for SOLO we can see the placeholders for the sr and ksmps.

    sr    = 44100
    ksmps = 512
    
    ;;;;SR;;;;    //strings replaced from app with own values
    ;;;;KSMPS;;;;
    
    nchnls	= 2
    0dbfs 	= 1

    The interaction between the UI in Objective-C and the Csound orchestra code is simple. We are using the methods for set and get for writing a value from the UI in Csound (set) or the other way to read a value from Csound (get). Since most of the interactions is of type get, to obtain values from Csound, it was utilized to implement the mechanism of the timer pulling, using the methods of the protocol csoundObjStarted and csoundObjCompleted for activating and deactivating a timer accordingly. The last calls for regular time intervals is a function delegated to sample values by the software bus Csound, using chnset, which also works with the display and updates the graphical interface.

    -(void)csoundObjStarted {
    
    //…
        /* Enable Pull Timer */
        if (!pullCsoundTimer) {
            pullCsoundTimer = [NSTimer scheduledTimerWithTimeInterval:0.05
             target:self
             selector:@selector(pullValuesFromCsound)
             userInfo:nil
             repeats:YES];
        }
    //…
    }
    
    -(void)csoundObjStarted {
    
    //…
        /* Disable Pull Timer */
       if (pullCsoundTimer) {
    
            [pullCsoundTimer invalidate];
            pullCsoundTimer = nil;
        }
    //…
    }
     

    Finally, with the function call from the NSTimer, values are read from the software bus by Csound. The address of the software bus is expressed as a string. Therefore, in the example, NSString @ "time" refers to instrument 5, and in particular to the code shown below.

    ktinst	timeinsts
    chnset ktinst, "time"
    

    As you can see by the opcode chnset the value is written to the ktinst software bus "time" which is read by the function csoundGetChannelPtr. This function in the C programming language of the Csound API copies the pointer MYFLT to the variable channelPtr_time. Now channelPtr_time, the pointer, points to the value of ktinst. Since channelPtr_time is a pointer, according to the syntax of the C programming language, it must be de-referenced in order to read the actual data in memory *channelPtr_time.

    -(void)pullValuesFromCsound {
    //…
    
    	MYFLT* channelPtr_time = nil;
    	csoundGetChannelPtr(_cs, &channelPtr_time,
                                [@"time" cStringUsingEncoding:NSASCIIStringEncoding],
                                CSOUND_CONTROL_CHANNEL | CSOUND_OUTPUT_CHANNEL);
            
    	if (channelPtr_time) {
    		MYFLT timeValue = *channelPtr_time / 66.0;            
    		time.progress = timeValue;            
    		[time_label setText:[NSString stringWithFormat:@"Time = %d", (int) *channelPtr_time]];
       }
    //…
    }
    

    In the example above, the value is used to update the native object UIProgressView and to update the text of a UILabel.

    V. Exploiting the Characteristics of the Signal in Real-time

    The signal can become an element that can be used to do anything. A simple approach is processing the same signal. You can take advantage of the characteristics of the analyzed signal and use that in turn to do something else. The k values output relative to amplitude and to pitch of the signal are valuable data and information to be referenced to make subsequent choices in the path of the algorithm.

    Csound provides opcodes which in addition to being exploited to achieve tonal effects or FX directly on the signal, can also be allocated to capture essential components of a signal such as amplitude, frequency, or spectrum.

      ptrack — Tracks the pitch of a signal pitch.
      pitchamdf — Follows the pitch of a signal based on the AMDF method.
      pvspitch — Track the pitch and amplitude of a PVS signal.
      follow — Envelope follower unit generator.
      follow2 — Another controllable envelope extractor.

    We will look at a patch employing some of these opcodes to exploit the signal.

    Some time ago I heard Three Breaths in three movements by Iain McCurdy[4] and I was captivated by the interest aroused in me in this direction. After contacting the author about his composition, he replied[5]:

    “In the 2nd movement I used pitch tracking so that different sound files would be triggered depending upon what note the player played. The sound files used were short recordings of the player making vocal sounds.

    In the 3rd movement I used pitch tracking to control the interval of a pvs pitch shifter - by doing this I could have a pitch shifted version of the input sound that sustained a constant pitch even while the source sound was playing different notes. There were actually up to three of these pitch shifters running simultaneously so i could create drones and chords.

    (The 1st movement did not use pitch tracking.) […].

    I think the challenge when using pitchamdf and pitch (and pvspitch) is to get a steady reading for pitch. Noise artefacts, very often present at the onset of the note, can 'confuse' the pitch tracker. There are ways in which you can try and address this, like lowpass filtering the input sound before the pitch tracker or delaying reading the input signal to 'miss out' the note attack.

    I did find using pitch tracking in this piece quite a challenge! I might in the future do some deeper research in comparing Csound's pitch trackers (pitch, pitchamdf, pvspitch, there may be others?). […] If you are using a bass guitar it might be quite a good source for pitch tracking as the sound is quite pure.

    Also think about what you could do with just amplitude tracking - this can be very expressive and is much less temperamental than pitch tracking.”

    Regarding pitch tracking, in the case of using opcodes such as pitchamdf or ptrack, but also with follow and follow2, utilize the last two opcodes where you want to extract only the profile amplitude of the signal.

    McCurdy also provides some valuable example .csds[6]. Utilizing those, it is interesting to analyze the opcodes pitchamdf and ptrack which I also transcribed for use in CsoundQt.

    I experimented withthese long .csds utilizing signals of different types: synthetic, audio files, or from a microphone, but these tests focused on some problems.

      •After the release of the microphone signal, or the signal from audio files, the CPU is being exploited to the maximum and then remained stable. However this does not happen if you are using a synthesis signal.

      •If after the start of the algorithm, there is no stress from the signal, the CPU immediately falls to normal values.

      • If you are using a synthesis signal with amplitude = 0, the variable kcps omits a maximum value (ihmaxcps).

    To avoid these three issues, I considered it appropriate to never put the amplitude of the signal equal to 0. Instead, I add a small amount (offset) to the signal's amplitude.

    The code snippet below shows the case for the synthesis signal.

    aL	oscili	kamp+.01, 440, 1
    ;and in the case of the audio signal (or from the microphone):
    aSig = (aL*kgateport) + .00001

    The next example is a very good one to show the use of the port opcode to prevent the annoying clicks for opening and closing of the signal threshold.

    For the opcode ptrack [Syntax: kcps, kamp ptrack asig, ihopsize[,ipeaks]] you can start from the example by Victor Lazzarini[7] (which appears in the Csound Manual) to help with understanding the real meaning of its potential functions.

    <CsoundSynthesizer>
    <CsOptions>
    ; Select audio/midi flags here according to platform
    ; Audio out   Audio in    No display
    -odac           -iadc     -d     ;;;RT audio I/O
    </CsOptions>
    <CsInstruments>
    
    sr	= 44100
    ksmps	= 16
    nchnls	= 1
    
    ;Example by Victor Lazzarini 2007
    
    instr 1
      a1 inch 1              ; take an input signal
      kf,ka ptrack a1, 512   ; pitch track with winsize=1024
      kcps port kf, 0.01     ; smooth freq
      kamp port ka, 0.01     ; smooth amp
    
      ; drive an oscillator
      aout oscili ampdb(kamp)*0dbfs, kcps, 1
    
      out  aout
    endin
    
    </CsInstruments>
    <CsScore>
    ; simple sine wave
    f 1 0 4096 10 1
    
    i 1 0 3600
    e
    </CsScore>
    </CsoundSynthesizer>
    

    McCurdy also provides an example of using the opcode ptrack[8] , similar to the one dedicated to the opcode pitchamdf[9]. This opcode is noteworthy for greater effectiveness in achieving the result and employs a simpler syntax .

    In general when using pitch and pitchamdf, but also the use of pvspitch, to obtain a reading as constant as possible for the values of frequency, it is necessary to adopt certain strategies of intervention to try to overcome such problems as external noise when employed with a microphone and signal characteristics in the initial phase of the reading.

    Below are a listed a few principles, in that case.

      • Apply lowpass filters before tracking pitch.
      • Delay the reading of the attack of the note, which is the most critical phase and in which the pitch tracking does not know what decisions to make, given the impurities and the frequency of tone that characterizes this instant.

    In the last section below, let us make a list that will help to make a summary of what was said about the use of the two opcodes (ptrack and pitchamdf).

    VI. Strong Points and Remedies for Weaknesses

    The opcode ptrack no signal values kcps stationed at the last value given.

    The opcode ptrack is faster to use than pitchamdf.

    There is no need to use frequency bands as in pitchamdf, requiring more data for input to the opcode.

    The opcode pitchamdf can restrict the frequency range and has a greater success and accuracy.

    Just after start and render, pitchamdf produces values (i.e. 147, 0), and then is given ihmaxcps to ensure delayed output values of the kcps.

    If the code is running with no signal (asig = 0), the kcps position is on the maximum value of pitchamdf (ihmaxcps). In this situation the temporal values of kcps should be ignored by a jump execution and then stabilized at the last value of kcps.

    When the kgateport passes 1, and then opens for a few milliseconds, kcps still passes the maximum value (ihmaxcps). It then passes the value of the incoming signal. We should delay the output of the values of kcps of a few milliseconds, because the kcps is slower than kgateport .

    With attention to these details I tried to write a few examples that exploited the pitch of the signal. This has given rise to a patch showing that with calibrating the parameters well and adopting some measures, it is possible to achieve good results aimed at live electronics. Hoping to bring forth some curiosity in the reader, the following provides examples of those patches where the first patch is designed for the contrabass as a solo instrument and uses the opcode ptrack. You can download and view the complete tutorial examples from the following link: tutorials.zip.

    gaMc	init	0 	; global variable of the audio signal
    gkint	init	1	; on/off switch (managed by the tool/event that will be triggered)
    gkcps	init	0	; frequency ptrack
    
    ;===========================
    instr 1				; signal - microphone
    ;===========================
    	
    aMc	inch	1		; MICROPHONE
    gaMc	=	aMc		; global
    endin
    
    ;===========================
    instr 2				; with ptrack
    ;===========================
    
    ; FILTRO
    	
    icpsmin	= cpspch(5.00)		; frequency of the lowest tone of the contrabass (V chord C1)
    aFlt1	butterbp gaMc*gkint, gkcps, 30	; applying a band-pass filter
    aFlt	atonex	aFlt1, icpsmin, 5	; application of a high-pass filter
    
    ; RESCALE
    
    kMcIn	port	gkMcIn, .2		; smooth
    aL	= aFlt * kMcIn * gkint		; rescale the amplitude and/or turn off the signal
    
    ; AUDIO_GATE
    
    afollow  follow2 aL, .0001, .03	; profile amplitude; attack = .0001, decay = .03 seconds
    kfollow	downsamp afollow		; creates version k-rate profile of signal amplitude
    gksoglia invalue	"soglia"	; value from 0 to 1 that is set in the GUI
    kgate	= (kfollow< i(gksoglia)?0:1)	; creates a signal GATE
    kgateport port	kgate,.05		; smooth
    aSig = aL * kgateport * gkint		; I apply the gating signal
    	
    ; PTRACKING
    
    iSize	pow	2, 10			; 1024
    gkcps,kA 	ptrack 	aSig, iSize 	; pitch track with winsize...
    
    In the final part of instr 2,  I joined this other patch that has the function of making a mean (average) every N samples; creating a buffer of N samples to k, that makes a sum and then divides by the number of samples:
    
    kindex	init	0
    kfreq	init	0		; ...first pitch-detector
    gkfreq	init	100
    
    kindex = kindex + 1	
    
    kfreq = kfreq + gkcps  
    
    icps	 = (2^7)+1		; samples (128 + 1)
    if kindex == icps then	
    kindex = 0			; kindex reset to 0
    gkfreq = kfreq / icps		; It gkfreq be the average of icps samples - in this case we have a resolution of ksmps * icps
    kfreq	  = 0			; kfreq = 0, reset to 0
    endif
    	endin
    

    The variable gkfreq constitutes useful data for subsequent choices, creating the conditions to turn on (trigger) various planned events.

    In the second patch I compared three types of signals using pitchamdf employing microphone, synthetic and audio files. The test was also performed on the audio signal with ptrack.

    gktrigger	init	0
    
    ;=======================
    instr 1
    ;=======================
    
    ksig	invalue	"signal"
    
    if 	ksig	= 1	then		; SIGNAL #1 - MICROPHONE
    a1		inch		1
    ap		butterbp 	a1, 250, 200	; filter
    apf1		tonex		ap, 20, 4	; filter (Smoothing dell'inviluppo)
    
    ; Extraction of the pitch and the rms signal (very important to hit the limits)		
    ;				sig	min	max	icps	imedi	idwns	iexcps	
    kcps, krms	pitchamdf  	apf1,	150,	350,	150,	51,	1, 	56		
    apf2		=	0
    apf3		=	0
    apf4		=	0
    ;=======================
    
    elseif	ksig	= 2	then
    apf2	soundin	"test4.wav"		; SIGNAL #2 – AUDIO FILE
    
    ; Extraction of the pitch and the rms signal
    ;				sig	min	max	icps	imedi	idwns	iexcps	
    kcps, krms	pitchamdf  	apf2,	150,	350,	150,	51,	1, 	0 
    apf1		=	0
    apf3		=	0
    apf4		=	0
    ;=======================
    
    elseif	ksig	= 3	then		; SIGNAL #3 – SYNTHETIC SOUND
    kfr linseg 150, p3/2, 350, p3/2, 150
    apf3	oscili	.5, kfr, 2
    
    ; Extraction of the pitch and the rms signal
    ;				sig	min	max	icps	imedi	idwns	iexcps	
    kcps, krms	pitchamdf  	apf3,	150,	350,	150,	51,	1, 	0
    apf1		=	0
    apf2		=	0
    apf4		=	0
    ;=======================
    
    elseif	ksig = 4	then		; SIGNAL #2 – AUDIO FILE
    apf4 soundin "test4.wav"		; signal
    
    ; Extraction of the pitch, the signal
    kcp, krms ptrack apf4,	2^12		; size 4096
    kcps delayk kcp, .1		
    ; delayk has the function of delaying the values kcps not to \
    	have a frequency of 100 at the beginning (arbitrarily defined)
    
    apf1		=	0
    apf2		=	0
    apf3		=	0
    endif
    ;=======================
    
    apf	sum	apf1, apf2, apf3, apf4, a1	; sum of the audio signals
    
    At this point the algorithm shows us  how  could be the call of other events:
    
    First possibility:
    
    if kcps > 250 then
    gktrigger	= 1
    else
    gktrigger	= 0
    endif
    
    ; CALL TOOLS / EVENTS
    		
    schedkwhen gktrigger, -1, 1, 2, 0, -1
    
    ; OUTPUT
    
    ken	linen		.1, 1, p3, 1
    	outs		apf*ken, apf*0
    	endin
    
    Second possibility:
    
    ; WITH THE TRIGGER
    
    isoglia	= 250					; trigger threshold
    kmode	= 2
    ; remember the three ways in which works trigger:
    ; kmode = 0 turns on ascending /
    ; kmode = 1 turn descending \
    ; kmode = 2 turns on both ascending to descending / \
    
    ktrig	trigger	 abs(kcps), isoglia, kmode	; absolute value of kcps \
    						and trigger
    	
    ; FOLLOWING LINES NEED TO TURN (PASS 1) WHEN VALUES GO BEYOND THE PITCH \
    	FOR THE FIRST TIME (in ascending) and OFF (0 PASS) WHEN VALUES PITCH \
    	GO ON FOR THE SECOND TIME (in descending order)
    
    kInverter init -1
    
    if (ktrig != 0) then			; if ktrig is not 0 then ...
    kInverter = -1 * kInverter		;kInverter = -1 * (-1) kInverter (that is 1)
    gktrigger = (kInverter == -1 ? 0 : kInverter)	
    
    ;if the condition is true and that is that kInverter equals -1, \
    	gktrigger assumes value 0 if the condition is false gktrigger \
    	assumes that value kInverter 1
    
    endif
    
    ; CALL TOOLS / EVENTS
    		
    schedkwhen gktrigger, -1, 1, 2, 0, -1
    
    ; OUTPUT
    
    ken	linen		.1, 1, p3, 1
    	outs		apf*ken, apf*0
    	endin
    
    Now schedkwhen call instr 2 will produce any other events
    
    ;=======================
    instr 2
    ;======================= 
    
    if	gktrigger	!= 0 goto cont
    turnoff
    cont:
    
    kamp 	linsegr 0, .05, 1, p3, 1, .2, 0 
    a1 	oscili kamp, 600, 1
    	outs 	a1*0, a1
    endin
    

    By making a debugging variable for kcps, we can note that once the algorithm is started, with the opcode ptrack there is delay in the transmission of values of pitch. This translates into a greater accuracy and security for the interpreter stage of the code.

    Finally there are some new opcodes that we should also take into account and take advantage of the analysis and management of real-time signals for those opcodes with Csound:

      plltrack - Tracks the pitch of a signal.
      pvscent - Calculate the spectral centroid of a signal.
      centroid - Calculate the spectral centroid of a signal.

    Acknowledgments

    I would like to thank John Fitch, Eugenio Giordani, Joachim Heintz, Matt Ingalls, Iain McCurdy, Bruce McKinney, Alessandro Petrolati, Luis Antunes Pena, Vittoria Verdini, and Rory Walsh for helping me in various ways on the realization of this article.

    References

    [1]Joachim Heintz, 2012. "Using Csound as a Real-time Application in Pd and CsoundQt." [Online]. Available: http://csoundjournal.com/issue17/heintz.html. [Accessed May 16, 2016].

    [2]Enrico Francioni, August 3, 2013. "Csound for Cage's Ryoanji A possible solution for the Sound System," in Csound Journal, Issue 18. [Online]. Available: http://www.csoundjournal.com/issue18/francioni.html. [Accessed October 17, 2016].

    [3]Enrico Francioni, 2008. Omaggio a Stockhausen, in "ATTI dei Colloqui di Informatica Musicale", XVII CIM Proceedings, Venezia, October 15-17, 2008 (it). [Online]. Available: http://smc.dei.unipd.it/cim_proceedings/2008_CIM_XVII_Atti.pdf. [Accessed October 19, 2016].

    [4]Iain McCurdy, 2009. Three Breaths. [Online]. Available: http://iainmccurdy.org/compositions.html. [Accessed October 21, 2016].

    [5]Iain McCurdy, February 2011. [e-mail]. Communication on Three Breaths.

    [6]Iain McCurdy, 2011. Realtime Score Generation. [Online]. Available: http://iainmccurdy.org/csound.html#RealtimeScoreGeneration. [Accessed October 22, 2016].

    [7]Barry Vercoe, et. al. "ptrack." The Canonical Csound Reference Manual, Version 6.07. [Online]. Available: http://csound.github.io/docs/manual/ptrack.html. [Accessed October 21, 2016].

    [8]Iain McCurdy, 2011. "ptrack.csd." Realtime Score Generation. [Online]. Available: http://iainmccurdy.org/CsoundRealtimeExamples/PitchTracking/ptrack.csd. [Accessed October 21, 2016].

    [9]Iain McCurdy, 2011. "pitchamdf.csd." Realtime Score Generation. [Online]. Available: http://iainmccurdy.org/CsoundRealtimeExamples/PitchTracking/pitchamdf.csd. [Accessed October 22, 2016].

    Bibliography

    J. Heintz, J. Aikin, and I. McCurdy, et. al. FLOSS Manuals, Csound. Amsterdam, Netherlands: Floss Manuals Foundation, [online document]. Available: http://www.flossmanuals.net/csound/ [Accessed May 25, 2013].

    Riccardo Bianchini and Alessandro Cipriani, "Il Suono Virtuale," (Italian Edition). Roma, IT: ConTempo s.a.s., 1998.

    Richard Boulanger, The Csound Book, The MIT Press-Cambridge, 2000.

    Enrico Francioni, Omaggio a Stockhausen, technical set-up digitale per una performance di "Solo für Melodieninstrument mit Rückkopplung, Nr. 19", in AIMI (Associazione Informatica Musicale Italiana), "ATTI dei Colloqui di Informatica Musicale", XVII CIM Proceedings, Venezia, October 15-17, 2008 (it).

    Karlheinz Stockhausen, Solo, für Melodieinstrument mit Rückkopplung. Nr. 19, , UE 14789, Universal Edition-Wien, 1969.

    Essential Sitography

    AIMI (Italian Association of Computer Music). Internet: http://www.aimi-musica.org/, 2015 [Accessed October 16, 2016].

    A. Cabrera, et. al. "QuteCsound." Internet: http://qutecsound.sourceforge.net/index.html, [Accessed May 16, 2016].

    apeSoft. Internet: www.apesoft.it, 2016 [Accessed October 16, 2016].

    Boulanger Labs. cSounds.com. CSOUNDS.COM: The Csound Community, 2016 [Accessed May 16, 2016].

    I. McCurdy and J. Hearon, eds. "Csound Journal". Internet: csoundjournal.com [Accessed May 16, 2016].

    E. Francioni. "Csound for Cage's Ryoanji A possible solution for the Sound System," in Csound Journal, Issue 18. Internet: http://www.csoundjournal.com/issue18/francioni.html , August 3, 2013 [Accessed October 17, 2016].

    E. Francioni. "Cage-Ryoanji_(stereo_version)_[selection]," (on SoundCloud). Internet: https://soundcloud.com/kolox/cage-ryoanji-stereo_version , [Accessed October 17, 2016].

    E. Francioni: "Francioni-CLUSTER V (demo-2ch)," (for a percussionist, live electronics and sound support). Internet: https://soundcloud.com/enrico_francioni/cluster-v-demo-2ch, [Accessed October 17, 2016].

    E. Francioni. "Stockhausen-SOLO_[Nr.19]_für_Melodieninstrument_mit_Rückkopplung," (Version I). Internet: https://soundcloud.com/enrico_francioni/kstockhausen-solo_nr19_fur_melodieninstrument_mit_ruckkopplung, [Accessed October 17, 2016].

    E Francioni. "SOLO_MV_10.1 Solo Multiversion for Stockhausen's Solo [N.19]," in Csound Journal, Issue 13. Internet: http://www.csoundjournal.com/issue17/solo_mv_10_1.html , May 23, 2010 [Accessed October 17, 2016].

    iTunes. "SOLO [Nr.19]," by apeSoft. Internet: https://itunes.apple.com/us/app/solo-nr.19/id884781236?mt=8, [Accessed October 17, 2016].

    J. Heintz. "Live Csound, Using Csound as a Real-time Application in Pd and CsoundQt." in Csound Journal, Issue 17. Internet: http://www.csounds.com/journal/issue17/heintz.html , Nov. 10, 2012 [Accessed October 17, 2016].

    Biography

    Francioni imageEnrico Francioni graduated in Electronic Music and double-bass at the Rossini-Pesaro. His works are in Oeuvre-Ouverte, Cinque Giornate per la Nuova Musica, FrammentAzioni, CIM, EMUfest, VoxNovus, ICMC, Bass2010, and Acusmatiq, etc.
    He performed the world premiere of the Suite I by F.Grillo. As author and soloist he has awards in national and international competitions. He has recorded for Dynamic, Agora, Orfeo and others. He is dedicated to teaching and has taught double-bass at the Rossini-Pesaro.

    email: francioni61021 AT libero dot it