Code: Viewable on GitHub
Software: Python Image Processing
Available on our repository as binarize.py and spiral.py.
Prerequisites: OpenCV, Numpy, PySerial.
The python code handles the process of receiving an image, binarizing it, converting it into a series of executable commands, and passing those commands off to the Arduino. It starts with the code in binarize.py, and the Binarize object. This object loads the image at a given file path into OpenCV in grayscale. Then, using Otsu's Method, the binary_split method generates binary thresholds, and creates a new binarized image based off of those thresholds.
Prerequisites: OpenCV, Numpy, PySerial.
The python code handles the process of receiving an image, binarizing it, converting it into a series of executable commands, and passing those commands off to the Arduino. It starts with the code in binarize.py, and the Binarize object. This object loads the image at a given file path into OpenCV in grayscale. Then, using Otsu's Method, the binary_split method generates binary thresholds, and creates a new binarized image based off of those thresholds.
The other python file, spiral.py, and the PolarImageConverter object handles the process of transforming the binarized image into executable instructions. It loads the numpy array that represents the binarized image and converts it into an array of 0s and 1s (as opposed to 0s and 255s). The cropImage method crops the array to be square with an odd-number of rows and columns-- so that we can find one exact center pixel. Next, we convert the original array into a list of values mapped to x and y coordinates. That is, if the center pixel is bright, we append the value (0, 0, 1), and if the pixel two above it and one to the right is dark, we append (1, 2, 0).
With a given number of steps per rotation (360, by the number of stripes on our visual encoder), and a number of total rotations, we can convert this cartesian list of coordinates into a similar list of spiral commands-- the difference between the two being that instead of mapping the values to x and y coordinates, we instead map them to the number of steps done in a simulated archimedean spiral. For example, the cartesian command (0, 0, 1) would instead map to (0, 1), and the command (0, 1, 1) would map to (360, 1) -- for that point is one full spiral out from the (0, 0) point. We additionally append one more integer to the command, which signifies which marker we would be using, if we chose to break the image down by multiple shades.
Finally, the constructSpiralTraversalDirections method takes the spiral commands and condenses it into a much more concise format, by removing commands that would do nothing different from the last. This means that if there are two commands in sequence that tell us that the pixels are empty, we can keep just the first and continue removing instructions until we reach one that tells us to put a marker down.
With our fully realized spiral commands, we can begin interfacing with the Arduino. This is done in the sendSerial method, which first sends the information about how many total rotations, and then the sorted instructions, fifty at a time, over a serial connection. This is done to make sure that the Arduino doesn't run out of memory-- once it has executed all of the commands, it sends back a character that lets the python program know to send more. The commands are sent one at a time, with comma-separated values representing angle, up/down, and marker number, and semicolons to signify the end of a whole command. Once we have sent all the commands, our drawing is finished: we can end the program.
With a given number of steps per rotation (360, by the number of stripes on our visual encoder), and a number of total rotations, we can convert this cartesian list of coordinates into a similar list of spiral commands-- the difference between the two being that instead of mapping the values to x and y coordinates, we instead map them to the number of steps done in a simulated archimedean spiral. For example, the cartesian command (0, 0, 1) would instead map to (0, 1), and the command (0, 1, 1) would map to (360, 1) -- for that point is one full spiral out from the (0, 0) point. We additionally append one more integer to the command, which signifies which marker we would be using, if we chose to break the image down by multiple shades.
Finally, the constructSpiralTraversalDirections method takes the spiral commands and condenses it into a much more concise format, by removing commands that would do nothing different from the last. This means that if there are two commands in sequence that tell us that the pixels are empty, we can keep just the first and continue removing instructions until we reach one that tells us to put a marker down.
With our fully realized spiral commands, we can begin interfacing with the Arduino. This is done in the sendSerial method, which first sends the information about how many total rotations, and then the sorted instructions, fifty at a time, over a serial connection. This is done to make sure that the Arduino doesn't run out of memory-- once it has executed all of the commands, it sends back a character that lets the python program know to send more. The commands are sent one at a time, with comma-separated values representing angle, up/down, and marker number, and semicolons to signify the end of a whole command. Once we have sent all the commands, our drawing is finished: we can end the program.
Firmware: Arduino Control
Available on our repository as polar_printing_press.ino.
The Arduino begins by setting up all of its input and output pins. It resets the location of the marker holder by moving it toward the center until the assembly clicks the limit switch at the center of the bar.
The Arduino needs to be initialized when it is begun. What this means is that when the Arduino is beginning, it first needs to read certain variables from the serial port. The very first thing to read is the number of rotations that the platter will perform in the entire image. The Arduino, after the code is uploaded, simply waits until it gets this value. After it receives this value, it initializes some variables related to how many steps the stepper motor will need to do in total and how many steps the stepper will need to take per degree that the platter rotates.
After initialization, the Arduino begins doing the work required to draw the picture. On every loop, the Arduino reads the IR sensor, which is on the bottom of the platter and reads a 360-degree black and white alternating pattern in order to know how far the platter has rotated. Each time the IR sensor changes from reading the current color to the other color, the Arduino knows that the platter has rotated 1 degree. The platter also has a limit switch on its underside, which is pressed by a hump on the platter for each rotation that it does. Arduino also checks this button's state on every loop. If the button is being pressed, the Arduino resets the variable that holds the current degrees rotated through. Only if the Arduino is executing, it increments the variable that holds how many total rotations the platter has done. I'll explain why it does this later.
The Arduino has two control states: executing and receiving commands. The Arduino is executing when it has commands to perform. In this state, it is tracking the rotation of the wheel, stepping the stepper, and moving the pen up and down in order to draw the image.
The Arduino holds three lists that represent the current commands, which hold angle data, which marker should actuate, and the marker's new state. It also holds a variable that represents the current command that Arduino is on. The next command is found by simply using the current command as the index of the angles array. When executing commands, Arduino calculates the current number of degrees that the platter has rotated through by multiplying the total rotations by 360 and adding the number of degrees past that the platter is. If the next command's angle is this value, Arduino executes the command by writing to the relevant solenoid pin, and then incrementing the current command variable.
While the Arduino is executing, it also needs to step the stepper motor outward in order to spiral the pen outwards. Because we want an Archimedean spiral, the stepper needs to step linearly with the rotations that the platter has done. What our code does is calculate the total number of steps needed based on how much the platter has rotated in total. It also has a variable that holds the total number of steps that it has already done. Thus, the Arduino can calculate the number of steps that it needs to do, and it adds this to the step queue. i'll talk about the step queue in the next paragraph.
The stepping is a little more complex because the stepper motor needs a pin to be set high and then low in order to execute a step. The normal way of doing this is to add a delay between writing to the pin. However, this delay is blocking, so if we used delays, our steps would prevent our IR sensor from reading the platter's position properly. We solved this problem using a queue of steps and time calculations to step the motor. Each time through its loop function, the Arduino will call the stepIfNeeded() function, which will set the stepper's pin to the proper value. If the stepper's current state is low, the stepper is waiting for a step command, and if it's high, it is executing a step. If the stepper is waiting, and there are steps in the step queue, Arduino will write high to the stepper and save what time that it toggled that pin into a variable. If the stepper pin is set to high, it can only be set to low if it has been writing high for a long enough time for the stepper to read the command. Each time in the stepIfNeeded function, if the stepper pin is high, Arduino checks if it has waited enough time since it wrote high to the pin. If it's been enough time, Arduino sets the pin to low and saves the time that it did that so that it can step the stepper again after sufficient time has passed. It also subtracts 1 from the step queue. Basically, if the stepper is ready and there is a step in the step queue, Arduino steps it by setting pin state and then calling that function over and over again until enough time has passed to reset the stepper pin state. Because the function is called in every loop, but executes quickly and doesn't delay, it doesn't block any other commands, but it is still able to step quickly.
The Arduino will change state from executing to reading if it executes all of its commands. The Arduino knows the the total number of commands that it has, and the number of commands that it has executed since last receiving commands. If it has executed all of the commands that it has, it will change its state to begin reading the serial port to listen for commands. When the Arduino is first initialized, the variable that keeps track of the number of commands executed is set to be more than the total number of commands, so that the Arduino knows that it needs to receive commands from the serial connection before executing.
Arduino reads the serial for commands by calling the readSerialCommands() function. This function simply reads from the serial. It grabs a single command by reading until it sees a semicolon. Then it splits the command into its three components by splitting the command where there are commas. The three components are the total angle that Arduino should perform the command, the marker that should be actuated, and the new marker state. When Arduino receives the letter "d" from the serial connection, the Arduino knows that it has read all of the commands, so it sets the current number of commands executed back to 0.
Some work needs to be done to mitigate the blocking nature of the serial read command. Because the serial read function is blocking, the Arduino hasn't been able to read the IR sensor to determine its position. If the Arduino immediately began executing, it would not be in the correct position. Also during this serial read time, the Arduino hasn't been able to execute commands, so it doesn't step the stepper motor to move the marker holder.
The Arduino needs to know when it returns to the position where it first started listening for commands, so that it can begin executing again in the same place where it stopped executing. To do this, when the Arduino changes state from executing to reading, it saves the current number of degrees past the limit switch that it is. Then, when the Arduino has received all of the commands from serial, it waits for the limit switch to be pressed, and then begins counting the degrees that the platter spins. When that number of degrees is equal to the degree that it was at when it stopped executing, it begins executing again.
The Arduino continues alternating states like this until it the Python code closes the serial connection when it has sent all of the commands. At this point, it is done drawing the image, and resets itself.
The Arduino begins by setting up all of its input and output pins. It resets the location of the marker holder by moving it toward the center until the assembly clicks the limit switch at the center of the bar.
The Arduino needs to be initialized when it is begun. What this means is that when the Arduino is beginning, it first needs to read certain variables from the serial port. The very first thing to read is the number of rotations that the platter will perform in the entire image. The Arduino, after the code is uploaded, simply waits until it gets this value. After it receives this value, it initializes some variables related to how many steps the stepper motor will need to do in total and how many steps the stepper will need to take per degree that the platter rotates.
After initialization, the Arduino begins doing the work required to draw the picture. On every loop, the Arduino reads the IR sensor, which is on the bottom of the platter and reads a 360-degree black and white alternating pattern in order to know how far the platter has rotated. Each time the IR sensor changes from reading the current color to the other color, the Arduino knows that the platter has rotated 1 degree. The platter also has a limit switch on its underside, which is pressed by a hump on the platter for each rotation that it does. Arduino also checks this button's state on every loop. If the button is being pressed, the Arduino resets the variable that holds the current degrees rotated through. Only if the Arduino is executing, it increments the variable that holds how many total rotations the platter has done. I'll explain why it does this later.
The Arduino has two control states: executing and receiving commands. The Arduino is executing when it has commands to perform. In this state, it is tracking the rotation of the wheel, stepping the stepper, and moving the pen up and down in order to draw the image.
The Arduino holds three lists that represent the current commands, which hold angle data, which marker should actuate, and the marker's new state. It also holds a variable that represents the current command that Arduino is on. The next command is found by simply using the current command as the index of the angles array. When executing commands, Arduino calculates the current number of degrees that the platter has rotated through by multiplying the total rotations by 360 and adding the number of degrees past that the platter is. If the next command's angle is this value, Arduino executes the command by writing to the relevant solenoid pin, and then incrementing the current command variable.
While the Arduino is executing, it also needs to step the stepper motor outward in order to spiral the pen outwards. Because we want an Archimedean spiral, the stepper needs to step linearly with the rotations that the platter has done. What our code does is calculate the total number of steps needed based on how much the platter has rotated in total. It also has a variable that holds the total number of steps that it has already done. Thus, the Arduino can calculate the number of steps that it needs to do, and it adds this to the step queue. i'll talk about the step queue in the next paragraph.
The stepping is a little more complex because the stepper motor needs a pin to be set high and then low in order to execute a step. The normal way of doing this is to add a delay between writing to the pin. However, this delay is blocking, so if we used delays, our steps would prevent our IR sensor from reading the platter's position properly. We solved this problem using a queue of steps and time calculations to step the motor. Each time through its loop function, the Arduino will call the stepIfNeeded() function, which will set the stepper's pin to the proper value. If the stepper's current state is low, the stepper is waiting for a step command, and if it's high, it is executing a step. If the stepper is waiting, and there are steps in the step queue, Arduino will write high to the stepper and save what time that it toggled that pin into a variable. If the stepper pin is set to high, it can only be set to low if it has been writing high for a long enough time for the stepper to read the command. Each time in the stepIfNeeded function, if the stepper pin is high, Arduino checks if it has waited enough time since it wrote high to the pin. If it's been enough time, Arduino sets the pin to low and saves the time that it did that so that it can step the stepper again after sufficient time has passed. It also subtracts 1 from the step queue. Basically, if the stepper is ready and there is a step in the step queue, Arduino steps it by setting pin state and then calling that function over and over again until enough time has passed to reset the stepper pin state. Because the function is called in every loop, but executes quickly and doesn't delay, it doesn't block any other commands, but it is still able to step quickly.
The Arduino will change state from executing to reading if it executes all of its commands. The Arduino knows the the total number of commands that it has, and the number of commands that it has executed since last receiving commands. If it has executed all of the commands that it has, it will change its state to begin reading the serial port to listen for commands. When the Arduino is first initialized, the variable that keeps track of the number of commands executed is set to be more than the total number of commands, so that the Arduino knows that it needs to receive commands from the serial connection before executing.
Arduino reads the serial for commands by calling the readSerialCommands() function. This function simply reads from the serial. It grabs a single command by reading until it sees a semicolon. Then it splits the command into its three components by splitting the command where there are commas. The three components are the total angle that Arduino should perform the command, the marker that should be actuated, and the new marker state. When Arduino receives the letter "d" from the serial connection, the Arduino knows that it has read all of the commands, so it sets the current number of commands executed back to 0.
Some work needs to be done to mitigate the blocking nature of the serial read command. Because the serial read function is blocking, the Arduino hasn't been able to read the IR sensor to determine its position. If the Arduino immediately began executing, it would not be in the correct position. Also during this serial read time, the Arduino hasn't been able to execute commands, so it doesn't step the stepper motor to move the marker holder.
The Arduino needs to know when it returns to the position where it first started listening for commands, so that it can begin executing again in the same place where it stopped executing. To do this, when the Arduino changes state from executing to reading, it saves the current number of degrees past the limit switch that it is. Then, when the Arduino has received all of the commands from serial, it waits for the limit switch to be pressed, and then begins counting the degrees that the platter spins. When that number of degrees is equal to the degree that it was at when it stopped executing, it begins executing again.
The Arduino continues alternating states like this until it the Python code closes the serial connection when it has sent all of the commands. At this point, it is done drawing the image, and resets itself.