Image processing and manipulation

Computer image processing involves the use of algorithms to perform operations on digital images to enhance, analyze, or manipulate them. This involves a range of techniques used to convert images into a more desirable form, extract information, or create visual effects.

Working with bitmap data (images and videos)

To load images and videos into the TD network the MovieFile In TOP is used.

Here's an example of how to make an image selector that reacts to SuperCollider sending OSC messages:

TouchDesigner

SuperCollider

s.boot;
s.quit;

// see available audio devices
ServerOptions.outDevices;

// define output device
Server.default.options.outDevice_("Scarlett 8i6 USB");

// Set up NetAddr for OSC communication
n = NetAddr("localhost", 12000);

// Define the SynthDef

SynthDef(\randomFilterSaw, {
    arg out=0, freq=220, amp=0.3;
    var sig, filterFreq, trigger;

    // Create trigger for random timing (between 0.5 and 3 seconds)
    trigger = Dust.kr(LFNoise0.kr(0.1).range(1/3, 2));

    // Generate random filter frequency (between 80 and 300 Hz)
    filterFreq = TRand.kr(80, 300, trigger);

    // Saw oscillator
    sig = Saw.ar(freq);

    // Apply low pass filter with random cutoff
    sig = LPF.ar(sig, filterFreq);

    // Apply amplitude envelope
    sig = sig * amp;

    // Send OSC message with current filter frequency
    SendReply.kr(trigger, '/filter_freq', filterFreq);

    Out.ar(out, sig!2);
}).add;

OSCdef(\filterFreqReporter, {|msg|
    var freq = msg[3];
    "Current filter frequency: % Hz".format(freq).postln;
    n.sendMsg("/filter_freq", freq);
}, '/filter_freq');




// Start synth on first output (channel 0)
x = Synth(\randomFilterSaw, [\out, 0]);

x.free;

Manipulation of values in the bitmap: Color adjustments + Manipulation of the bitmap structure

TouchDesigner provides many operators to easily adjust and manipulate bitmap data. The functionality of these operators is quite similar as in equivalent tools in image editing programms like Photoshop or video editors.

TOPs: HSV Adjust, Level, Monochrome, Blend

TouchDesigner

SuperCollider

s.boot;
s.quit;

// see available audio devices
ServerOptions.outDevices;

// define output device
Server.default.options.outDevice_("Scarlett 8i6 USB");

// Set up NetAddr for OSC communication
n = NetAddr("localhost", 12000);

// Define the SynthDef

SynthDef(\randomFilterSaw, {
    arg out=0, freq=220, amp=0.3;
    var sig, filterFreq, trigger;

    // Create trigger for random timing (between 0.5 and 3 seconds)
    trigger = Dust.kr(LFNoise0.kr(0.1).range(1/3, 2));

    // Generate random filter frequency (between 80 and 300 Hz)
    filterFreq = TRand.kr(80, 300, trigger);

    // Saw oscillator
    sig = Saw.ar(freq);

    // Apply low pass filter with random cutoff
    sig = LPF.ar(sig, filterFreq);

    // Apply amplitude envelope
    sig = sig * amp;

    // Send OSC message with current filter frequency
    SendReply.kr(trigger, '/filter_freq', filterFreq);

    Out.ar(out, sig!2);
}).add;

OSCdef(\filterFreqReporter, {|msg|
    var freq = msg[3];
    "Current filter frequency: % Hz".format(freq).postln;
    n.sendMsg("/filter_freq", freq);
}, '/filter_freq');




// Start synth on first output (channel 0)
x = Synth(\randomFilterSaw, [\out, 0]);

x.free;


SynthDef(\randomFilterNoise, {
    arg out=0, amp=0.3;
    var sig, filterFreq, trigger;

    // Create trigger for random timing (between 0.5 and 3 seconds)
    trigger = Dust.kr(LFNoise0.kr(0.1).range(1/3, 2));

    // Generate random filter frequency (between 100 and 1000 Hz)
    filterFreq = TRand.kr(100, 1000, trigger);

    // White noise oscillator
    sig = WhiteNoise.ar();

    // Apply low pass filter with random cutoff
    sig = LPF.ar(sig, filterFreq);

    // Apply amplitude envelope
    sig = sig * amp;

    // Send OSC message with current filter frequency to localhost:12000
    SendReply.kr(trigger, '/noise_freq', filterFreq);

    // Output routing
    Out.ar(out, sig!2);
}).add;



// OSC responder
OSCdef(\noiseFreqReporter, {|msg|
    var freq = msg[3];
    "Current filter frequency: % Hz".format(freq).postln;
    n.sendMsg("/noise_freq", freq);
}, '/noise_freq');


y = Synth(\randomFilterNoise);
y.free;

Assignment:

Experiment with different texture operators to manipulate bitmap data. Find out about them autonomously. Pair them via OSC with sounds generated in SuperCollider.

Duration: ~30 minutes

Example for detecting edges, extracting 2d-vector points and using them for generative visuals (coded in Processing, but similarely possible in TouchDesigner): http://andreaspirchner.com/generative-image-based-illustration/