Software Subsystem

Our software features an LCD screen, with a user interface that lets the user pick a song. The software then played the chosen song by controlling the stepper motors to move the finger boxes to the correct place for each note and playing the note by using the servo to press the finger down.

Firmware Component

Our project’s firmware component was the Arduino, which was attached to a version 3 CNC motor shield. We used the Servo library built into the IDE program and the SpeedyStepper Library to create functions to move our “finger” system and strike the key for a specific duration, which we will explain in the sections below.

Arrays

An important part of our code are the arrays, which can show which numbers represent which notes. One array has note names, one has the same notes with the numbers MIDI uses to represent them, and one has the location of the notes on the keyboard in steps.

Using these arrays, the code can take the midi input note and convert it to the stepper location, by finding the index of MIDI input symbol in the midiNotes array, and telling the stepper to go to the location found in that index. The programmer can do the same matching with the note names, or we could make a function to do it for the note names too.

Playing Notes

The playNote() function has two inputs: note location and duration. It uses this information to move to that location, sweep the servo to the down position, and then sweep it back to the up position. Playing a song can be done by stringing together several of the playNote() function with the correct notes and relative durations. The goal was, that by combining an array made from the MIDI conversion code we found on GitHub, the midiConvert() function, and the playNote() function, we could play from MIDI. Unfortunately, we ran into some difficulties with MIDO and pySerial, and were unable to fully incorporate the MIDI, and therefore, it is not part of our final product.

Calibration

We needed to find the correct position for the servo to go down and up to, the correct distance between keys on the keyboard, and the amount of time between notes. Rather than hard-code these values, we made them variables at the beginning of the code after testing them using calibration code. Our calibration code just told the servo to move down and back up, and we tested the distances and locations that were the best values, before inputting those into the variables.

Moving the Fingers

Our fingers were given inputs for their location and note duration. Each finger had a separate function corresponding to it for convenience. We used an absolute motion function in SpeedyStepper that set our steps to millimeters and implemented this to move the fingers to the correction position. At the end of each song, these functions were also used to return the structure to its initial position, allowing for consistent replayability. When the structure arrived at the note location, the servo would write to the calibrated down position, and remain there for the note duration, before retracting and moving to the next position.

User Interface

The user interface consists on an LCD screen programmed to display our song options. There are two buttons on the side allow the user to view available songs, and select a song to play. This is implemented by creating a state for each song, and a second state within each song that controls whether it is playing or not. When a song is selected and played, the Arduino is told to send signals to the steppers and servos to play that song.