Archive | September 2011

3D Spectrogram with Processing and Microphone

3D spectrogram created with Processing.  Realtime visualization of sound using a microphone or line in.  Modified code to allow for realtime recording using line in or microphone. Based on code by John Locke. Link

Video of 3D spectrogram:

Code:

//3D Spectrogram with Microphone Input
//Modified by kylejanzen 2011 - https://kylejanzen.wordpress.com
//Based on script wwritten by John Locke 2011 - http://gracefulspoon.com

//Output .DXF file at any time by pressing "r" on the keyboard

import processing.dxf.*;
import ddf.minim.analysis.*;
import ddf.minim.*;

FFT fftLin;
FFT fftLog;

Waveform audio3D;

Minim minim;
AudioInput microphone;

boolean record;

PFont font;

float camzoom;
float maxX = 0;float maxY = 0;float maxZ = 0;
float minX = 0;float minY = 0;float minZ = 0;

void setup()
{
  size(1250,750,P3D); //screen proportions
  noStroke();
  minim = new Minim(this);
  microphone = minim.getLineIn(Minim.STEREO, 4096); //repeat the song

  background(255);

  fftLog = new FFT(microphone.bufferSize(),microphone.sampleRate());
  fftLog.logAverages(1,2);  //adjust numbers to adjust spacing;
  float w = float (width/fftLog.avgSize());
  float x = w;
  float y = 0;
  float z = 50;
  float radius = 10;
  audio3D = new Waveform(x,y,z,radius);
}
void draw()
{
  background(0);
  directionalLight(126,126,126,sin(radians(frameCount)),cos(radians(frameCount)),1);
  ambientLight(102,102,102);

  if (frameCount>200)
  {
    for(int i = 0; i < fftLog.avgSize(); i++){
      float zoom = 1;
      float jitter = (fftLog.getAvg(i)*2);
      //println(jitter);
      PVector foc = new PVector(audio3D.x+jitter, audio3D.y+jitter, 0);
      PVector cam = new PVector(zoom, zoom, -zoom);
      camera(foc.x+cam.x+50,foc.y+cam.y+50,foc.z+cam.z,foc.x,foc.y,foc.z,0,0,1);
    }
  }
  //play the song
  fftLog.forward(microphone.mix);

  audio3D.update();
  audio3D.textdraw();

  if(record)
  {
    beginRaw(DXF, "output.dxf");
  }
  audio3D.plotTrace();

  if(record)
  {
    endRaw();
    record = false;
    println("It's Done Bitches! Find your DXF!");
  }
}
void stop()
{
  // always close Minim audio classes when you finish with them
  microphone.close();
  // always stop Minim before exiting
  minim.stop();
  super.stop();
}
class Waveform
{
  float x,y,z;
  float radius;

  PVector[] pts = new PVector[fftLog.avgSize()];

  PVector[] trace = new PVector[0];

  Waveform(float incomingX, float incomingY, float incomingZ, float incomingRadius)
  {
    x = incomingX;
    y = incomingY;
    z = incomingZ;
    radius = incomingRadius;
  }
  void update()
  {
    plot();
  }
  void plot()
  {
    for(int i = 0; i < fftLog.avgSize(); i++)
    {
      int w = int(width/fftLog.avgSize());

      x = i*w;
      y = frameCount*5;
      z = height/4-fftLog.getAvg(i)*4; //change multiplier to reduces height default '10'

      stroke(0);
      point(x, y, z);
      pts[i] = new PVector(x, y, z);
      //increase size of array trace by length+1
      trace = (PVector[]) expand(trace, trace.length+1);
      //always get the next to last
      trace[trace.length-1] = new PVector(pts[i].x, pts[i].y, pts[i].z);
    }
  }
  void textdraw()
  {
    for(int i =0; i<fftLog.avgSize(); i++){
      pushMatrix();
      translate(pts[i].x, pts[i].y, pts[i].z);
      rotateY(PI/2);
      rotateZ(PI/2);

      fill(255,200);
      text(round(fftLog.getAvg(i)*100),0,0,0);
      popMatrix();
    }
  }
  void plotTrace()
  {
    stroke(255,80);
    int inc = fftLog.avgSize();

    for(int i=1; i<trace.length-inc; i++)
    {
      if(i%inc != 0)
      {
        beginShape(TRIANGLE_STRIP);

        float value = (trace[i].z*100);
        float m = map(value, -500, 20000, 0, 255);
        fill(m*2, 125, -m*2, 140);
        vertex(trace[i].x, trace[i].y, trace[i].z);
        vertex(trace[i-1].x, trace[i-1].y, trace[i-1].z);
        vertex(trace[i+inc].x, trace[i+inc].y, trace[i+inc].z);
        vertex(trace[i-1+inc].x, trace[i-1+inc].y, trace[i-1+inc].z);
        endShape(CLOSE);
      }
    }
  }
}
void keyPressed()
{
  if (key == 'r') record = true;
}
Advertisements

Graphical Observations of Sound

Contrasting the acoustic studies of Helmholtz; who listened to resonance with the ear, Koenig and his colleagues in Paris became fascinated with a way to view sound.

Phonautograph by Scott (1857), Pantalony, pg. 42.

…the phonautograph, an instrument that recorded sounds directly from the air… marked a lasting shift in acoustics from reliance on the to the eye. ~ David Pantalony,  Altered Sensations: Rudolph Koenig’s Acoustical Workshop in the Nineteenth-Century Paris, pg. 41.

Recording instruments like Koenig and Scott’s phonautograph provided a means for a group to study sound without the individual subjectivity related to hearing.

Self-recording instruments… with an emphasis on automation (replacing human skills with machines), objectively in instrumentation (making it possible for data to be viewed and shared by several witnesses at once) and the investigation of previously unobservable patterns and effects (extension of the senses).  ~ David Pantalony,  Altered Sensations: Rudolph Koenig’s Acoustical Workshop in the Nineteenth-Century Paris, pg. 41.

The phonautograph provided acoustics with means to observe and graphically record sound.  This allowed for the measurement and analysis of sound without the temporal limitations.  The records could be shared graphically at anytime.  Unlike Helmholtz’s focus on hearing, the machine became the device for observing sound.  Acoustics were no longer dependent on the subjective nature of the the human ear; the machine (even with its limitations) provided a new objective approach.

Les Phonogrammes Traces by Koenig (1882), Pantalony, pg. 48.

Koenig’s graphical recordings (les phonogrammes) became tools for studying and resolving error in his devices. They aided him with discovering subtleties which were difficult to perceive by ear.  This allowed for a refinement of many of his devices that previously was not available.

Visualizing Sound – Bell Labs (1950)

Neon lamp used to track physical movement of sound.

Neon Lamp Traces Sound Wave, Bell Labs, 1950. Link

Neon Lamp Traces Sound Wave’s Picture
That’s a sound wave you see in the picture above. Here demonstrating how an acoustic lens focuses sound from a horn, the wave was made visible with the device at left – an aluminum rod with a microphone and a neon lamp at the end. A small motor swings the rod in a wide arc, scanning the area. The microphone picks up the sound and turns it into electric current to feed the lamp. Wherever the sound is strongest, the light is brightest, and the wave is traced out. A complete sound photo, such as this from Bell Labs, takes 10 minutes exposure. ~ Bell Labs, Link

Ballet Mécanique (1924)

Ballet Mécanique, Directed by Léger & Murphy, Music by  Anthiel, Cinematography by Murphy & Man Ray, 1924. Link

Man Ray (1890 – 1976)

Man Ray, Carl Van Vechten, 1934. Link

Man Ray, Le Retour à la Raison, 1923. Link

Rayograph, Man Ray, 1925. Link

Rayograph, Man Ray, 1927. Link

Photocell Audio Recording of Laser

Audio recording of laser created by a computer mouse.  Recorded with two photocells and a Zoom H2. 3D Spectrogram created using Processing based on code by John Locke.

Rendered images of 3D spectrogram:

Photocell Laser 3D 20110918
Photocell Laser 3D Close 20110918
Photocell Laser 3D Perspective 20110918

Video of 3D spectrogram:

Video of the spectrogram:

Spectrogram of the recording:

Photocell Laser 20110918

“chez Koenig”

Rudolph Koenig, Public Domain, 1890. Link

Between 1859 and 1901 a Prussian immigrant named Rudolph Koenig ran one of the more popular scientific ateliers in Paris. It was a place singularly devoted to sound. Visitors bought instruments, performed experiments, learned about acoustics, discussed the instrument trade in Paris, witnessed demonstrations, and stayed for an evening of food, drink, music and literature. Many of the apparatus which adorned his atelier became the foundation of modern acoustics. There were graphical instruments for recording sound, manometric flame instruments for making sound waves visible, sirens, tuning forks for precision experiments, and a variety of demonstration instruments.  ~ David Pantalony,  Altered Sensations: Rudolph Koenig’s Acoustical Workshop in the Nineteenth-Century Paris.

Koenig’s studio became a place for exploration of acoustics and sound. Instruments for producing, observing, and recording sound were introduced here by him and his colleagues; many focusing on “making for pure motives” as opposed to commercial success.  As described by Pantalony, ‘Koenig’s atelier’ was a multi-use space:

Workshops entailed the spaces where instruments were actually made; showrooms (or boutiques or studios) involved the business activities of instrument making; laboratories supported experimentalactivity; and living quarters related to the daily aspects of life as a scientific artisan.  ~ David Pantalony,  Altered Sensations: Rudolph Koenig’s Acoustical Workshop in the Nineteenth-Century Paris.

The work/live studio culture promoted by Koenig is very interesting to me.

Koenig experimented with and created many devices for producing sound which I intend to study further. Among many devices, Koenig developed a mechanical means to visualize sound using his manometer;  a device that depicts sound waves using a flame and a rotating mirror.

Flame Manometer, the Whipple Museum. Link

Rotary mirror and manometric flame according to Koenig, from Physiologische Graphik, Langendorff, Oskar. 1891. Link 

Illustration of two manometric traces, from Koenig’s Acoustic Catalogue, 1865. Link

At the moment, I am interested in Koenig’s exploration into the visualization of sound. I intend to further explore this topic by studying the interactions between light and sound. Pursuing techniques of listening to light, recording light, visualizing sound using light, and modulating sound with light.