Editor's Note: This was Project 4 of my New Media Arts Class

Original Title: Project 4 - What if you can make music with just your hands?

Original Description: What if you could make music just by waving your hands in front of a sensor? This system makes music based on the position of your hands in front of a pair of ultrasonic sensors using pentatonic scales.

The Device

I created this device in a few hours using the base of an old keyboard I took apart a long time ago, and a lot of Sticky Putty.

Project Information

This was the final project in my MART380 class.

Motivation

I have recently been researching more about Computer Music, and decided that this would be a cool concept to peruse for my final project in this class. I had some experience in working with the Distance Sensor for an Arduino, so I decided to go with that.

Music

I decided to go with using a Pentatonic Scale, as well as 7th chords with added 2nds to make my music. There are a few distinct parts of the music generation process: The Arpeggio, Chords, Melody, and Bass. The Melody is randomly generated by picking one of the 5 notes to play. Chords change on the user putting their hands close to the device, which causes a new Chord to be randomly selected.

System Requirements

Currently I only support linux, but in theory this project can be modified to work on other platforms, such as Windows or MacOS.

Software Required:

• Pure Data (In particular the l2ork version, which can be found here)

  • The following Packages Must be installed:

    • Comport (serial communication module)

    • Reverb

• Jack Audio Connection Kit (for Linux users to route MIDI and Audio streams through software, link here)

• ZynAddSubFX (Audio Synthesizer, makes the sound for the project. Download Here. Please Download the Non-ZynFusion version.)

• USB 2.0 Connections, or otherwise able to communicate with an arduino through a serial interface.

Tools Needed

In order to setup this system, you will need:

• 2 Arduinos

• 2 Arduino Ultrasonic Sensors

• Wires and USB cables as appropriate

Setup Procedure

This is a brief guide to setting up the Arduino software and hardware, as well as Pure Data.

Arduino

1. Connect the Arduino to the Distance Sensor.

NOTE: Be sure that the ultrasonic sensors point opposite of each other, otherwise interference may occur.

2. Connect Arduinos to PC

3. Write Program to both Arduinos

PC

1. Start Jack

a. Note that QJackCtl has a nice GUI to help with setup if you are into that type of thing

2. Start Pure Data

• Turn on the DSP

• Open up mart380_final.pd

3. Start ZynAddSubFX

• Load inst.xmz

4. Make connections

5. In the Opened Pure Data Project, click on the large Bang Object (looks like a button) at the top-left corner of the window, where there is a comment saying "Start".

Use

By default, the system should be quiet, as the reverb is on full-blast, and the notes are very short. Place hands in front of the sensors, about 20 inches away, and move around, probably closer to the sensor, until the L LED on the Arduino illuminates. When this light is lit, you will know that your hand is within the range of the sensor, and a non-zero or 127 value will be sent to the computer.

First, try moving your hands closer to the sensors and listen for the music to get louder. To change Chords, move your hands closely in front of the sensors, until the lights go out (from your hands being too close). After a few ticks, the chord should randomly change.

Modification

This program can be modified to accommodate different instruments, sensors, or chords.