Arduino Graph

Posted: Monday, January 26
Due: Thursday, February 5 (we will demo our work during classtime)

All assignments in this class will be emphasizing the theme of tangible interactive computing. With IA01, you will get a taste of the Arduino platform--the current de facto open hardware platform for rapid prototyping. According to the Arduino webpage

"Arduino is an open-source electronics prototyping platform based on flexible, easy-to-use hardware and software. It's intended for artists, designers, hobbyists, and anyone interested in creating interactive objects or environments."

Arduino makes working with sensors and actuators significantly easier than before it existed. This is largely due to the enormous support and ample code examples from the online Arduino community. There are many examples of Arduino projects on blogs, forums, and YouTube that can help get you started on your own projects. Before starting this assignment, I encourage you to search for "cool Arduino projects" to get a sense, first, of the enormous variety of inventions that have been built by hobbyists, hackers, and professionals with the Arduino and, second, to gain an appreciation for the variety of types of people working on these projects from professional digital media artists building interactive art installations to musicians building experimental instruments to engineers prototyping electronics projects.

Assignment Overview

The Arduino is a powerful platform because it enables a wider range of people to build computational artifacts that enable new interactive experiences in the physical world relatively simply and quickly. Perhaps because of this simplicity, however, the development environment is rather impoverished compared to other development tools. The Arduino programming environment lacks basic debugging functionality like stack tracing, breakpoints, memory inspectors, etc. Thus, you have to return to the "old school" way of debugging your program, namely println()! Specifically, Serial.println(). For example, the code below is reads from A0, which is hooked up to a potentiometer, and writes out to D9, which is connected to the LED. The only way to investigate what these values are is via a Serial.println() function call.
int ledPin = 9;    // LED connected to digital pin 9
 
void setup()  {
  // nothing happens in setup
  Serial.begin(9600);
  analogWrite(ledPin, 255);
}
 
void loop()  {
  //read in from Analog0 (which is connected to the potentiometer)
  int sensorValue = analogRead(0);
 
  //remember that our analogWrite has a min value of 0 and a max value of 255,
  //so we should map our input range to our output range
  int analogOut = map(sensorValue, 0, 1023, 0, 255);
  analogWrite(ledPin, analogOut);
 
  //print out the data!
  String printOut = "sensorValue=" + String(sensorValue) + ", analogOut=" + String(analogOut);
  Serial.println(printOut);
}
See the println at the end of the loop function? (quick tip: remember to initialize the serial port in your setup() function with Serial.begin(baudRate)). The Arduino prints the formatted string (the variable printOut) to the serial port, which your computer reads off of USB. You can then open the Serial Monitor in the Arduino IDE to view the incoming serial port data. It will look something like this:

SerialMonitorPotentiometer.png

So, while this can be very useful, it doesn't scale with program complexity and it certainly pales in sophistication and usefulness compared to normal IDEs. This lack of feedback from the Arduino IDE is frustrating (we are spoiled!). Plus, it can obfuscate what is actually happening with your prototype during execution and, I think, can negatively impact learning (e.g., it's not easy to answer some of the "what is going wrong with my circuit?" questions).

Assignment Goal(s)

To help us with improving Arduino's runtime transparency and to help answer those "what is happening?" questions, we will build an Arduino visualizer that can graph data off the serial port in real time. While Arduino connects to the PC using an USB port, that USB connection is actually used like an old serial (RS232) connection. Your job will be to simply grab data of the serial port, and graph it. You will also create a more 'abstract' visualization of the data that changes based on analog and digital input.

This assignment should serve as a nice introduction to Arduino, the Arduino IDE, some simple I/O functionality, some basic sensors, and creating interactive graphics in Processing. Moreover, you will be creating a useful debugging tool for yourselves that you can use the rest of the semester! Note: we will also learn other strategies for debugging our hardware designs such as using a multimeter, but this is a good start.

I have included screenshots from my own implementation; however, this should not bias you! There are countless ways to approach this assignment and I look forward to your creative solutions!

Simple Arduino Graph Example

I chose to separate the analog visualization area (top) from the digital visualization area (bottom), the analog area has a labeled y-axis and legend, and each input signal is unique colored. Warning: my implementation may not fulfill all the required criteria I've listed for you as I change this assignment each semester, but it's a reasonable example in helping illustrate this assignment.
ArduinoGraphScreenshot3.png


Simple Arduino Abstract Representation Example

In this simple abstract visualizer, I represent each digital and analog input channel as an individual particle emitter in a particle system. For the analog inputs, the size of each particle is proportional to the analog value. For the digital inputs, the emitter is either 'on' or 'off' depending on the 'HIGH' or 'LOW' digital input state. The analog values are simple circles while the digital values are rotating pins. The color mappings are the same as the above Arduino Graph. The particle emitter flow rates are constant.

ArduinoGraph7AbstractScreenshot5.png


What to Do

There are a few steps to this assignment, please follow them closely (you don't want to lose needless points for not following directions).
  1. You must use Processing to create the visualizations. The Arduino IDE is actually based on and inspired by Processing--they also have similar goals: to lower the barrier to rapidly prototyping designs. So, it's good to learn this environment a bit. While Arduino is based on the 'C' programming language, Processing is based on Java. You should have already downloaded both the the Arduino development environment (1.5.8 beta) and the Processing development environment (3.0a5). If not, do so now.

  2. Your Arduino Visualizer (or oscilliscope or grapher or whatever you want to call it) must visualize a temporal representation of Analog In and Digital In. The y-axis should be the value of the analog or digital input and the x-axis should be temporality (either clock time or just sequence). The y-axis should be labeled.

  3. The visualizer should distinguish between analog vs. digital. It's up to you how you want to do this. For example, you could use color, multiple simultaneous graphs, etc. Your program should support graphing all input off of the Arduino Uno or Leonardo simultaneously. Your program should dynamically sense which input channels are active and only display those inputs.

  4. Your visualizer should be in a resizable window and the graphics should redraw accordingly.

  5. In order for your visualizer to understand the data off of the serial port, you will need to define some sort of standardized communication protocol. For example, you might program your visualizer to ignore any data off the serial port except for lines prefixed by a certain character and then these prefixed lines have a certain structure. As another example, maybe you decide that your communication protocol is a simple comma separated format like: A0I=X, A1I=Y, .... A5I=Z (for analog in), D0I=X, D1I=Y,....D13I=Z (for digital in).

  6. Once you've completed your graph visualizer--which offers a utilitarian and hopefully helpful view of the data--I would like you to create an abstract visualization of the incoming analog and digital data. This abstract visualization could be artistic, silly, fun, clever. It's up to you. The key is to play with Processing's functionality, to push yourself creatively, and to have fun! Think about the unique ways you can use animation, color, size, shape, and other visual/motion properties to convey the analog and digital inputs. Some ideas: build a particle system like my example above but hopefully way cooler like this, this, this reactive particle system, or this perlin flow field particle system (you can find lots of Processing particle systems online--use them as inspiration or as a template to remix with your own particularly implementation), build flocking behavior that reacts to the inputs like this or this, or create a generative tree like this or this (imagine, for example, that each input channel is its own tree and input values change the branching factor, leaf color, tree height, etc.)

  7. Like most of the assignments in this class, you must host your code on GitHub using Git, a version control system. Please commit everything you need to run your program including your Arduino code and your Processing code. In undergrad/grad, I used CVS and Subversion extensively, so I am more familiar with those version control systems; Git is different and is, apparently, the future! So, we'll use it in this class (maybe some of you can teach us more about it!). Git is unique in that it is a completely distributed version control system, has cheap local branching, etc. To learn more about Git, see these resources: Getting Started (on the Git website), Github bootcamp, and Github's list of resources for learning Git and Github. Given that I am a Git novice, I don't use the command line, instead, I rely on a GUI tool called GitHub for Windows and GitHub for Mac. I suggest you start using Git right away as soon as you begin working on the project and commiting code as you progress/iterate on the assignment. We will also be using Git for our mini-projects.

  8. You must create a GitHub readme.md that describes your program ('md' stands for markdown, which allows you to write easy-to-write plain text that converts to valid HTML for viewing on GitHub; see this link). In particular, your readme.md should provide a brief summary of the functionality/purpose of your program and should describe your communication protocol between Arduino and your visualizer, your graph visualizer as well as your abstract visualization (with screenshots), and include enough details such that an interested technical person could download and use your program.

  9. Finally, you must link to your GitHub project in the Deliverables section below. I have started a template for you to follow. You do not have to use your real name to post to the wiki but if you use a pseudonym, please send this to me. Similarly, you do not have to use your pre-existing GitHub account. You can create a dummy (more anonymous) account for the purposes of this class if you'd like. I realize that some students like to protect their online privacy.

Advanced Functionality (Optional)

If you finish with the above, and still want to add features, how about exploring the following:
  • Because only one computer program can read from the Serial port at a time, how about adding "Serial Monitor" functionality to your visualizer that simply prints the raw data off the serial port.

  • How about adding support for analog out and digital out? How would you do this? Is this as useful for debugging? Why or why not?

  • Can you perform some simple signal processing on the incoming waveforms such as threshold detection, peak detection?

  • Can you detect significant changes in values and annotate the graph with the raw printed value?

  • Can you apply some simple signal processing techniques to transform and visualize the incoming waveform into a different type of waveform (e.g., bandpass filter, sliding window averaging filter)? Try visualizing this new waveform in a window directly below or overlaid on the original signal.
  • Can you add support for specific kinds of sensors and render custom visualizations for those sensors? For example, to do this, you could add in sensor type into your communication protocol (e.g., D0I=X|SensorType=Accel|Name='X'|Label='X-Axis', D1I=Y|SensorType=Accel|Name='Y'|Label='Y-Axis', D2I=Z|SensorType=Accel|Name='Z'|Label='Z-Axis') and, in the case of an accelerometer, you could draw a 3D-representation of its state depicting roll, yaw, and pitch (see Project 21 from Tom Igoe's Making Things Talk book).

  • Semi-super advanced: can you print out a scrolling spectrogram of the incoming data?

Related Links

Here are some potentially useful links. Disclaimer: I have not looked into these very deeply, so read at your own risk. :) Feel free to post other examples to Piazza or critiques of the links below. While you can use libraries in this assignment, the visualization system as a whole must be your own. If you pull from or remix any code that is not your own, please clearly indicate this.

Deliverables

We will be demonstrating our work during class on Thursday, February 5th so your code must be complete by then and fully committed to GitHub. During your presentation, I would like you to show a live demo that shows off both your graph visualizer and abstract representation using at least three different kinds of sensors. You will have roughly 4 minutes to present your two visualizations with various inputs (both analog and digital), to discuss your approach and rationale for your design, to describe any challenges you faced and how you overcame them (these sorts of things are useful to the class as we are all learning!), and to take a bit of Q/A.

Listeners will provide feedback on new ideas and suggest areas for improvement. When providing feedback to one other, remember that verbalizing useful criticism is a skill that requires practice and thought. As Scott Berkun says, "good criticism serves one purpose: to give the creator of the work more perspective and help them make their next set of choices" (link) To add to this, good criticism helps a designer improve their artifact and learn. That's what we want in this class!

Note: 4 minutes is super short I realize but we have 12 presentations (12 x (4 + 1 min for setup) = 60 mins), so we really can't dedicate any more time. To help us with coordination, I will use my iPhone timer. When it goes off, we'll have to move on to the next presenter.

Please list your GitHub link and two program screenshots below. You do not have to use your personal name below. Instead, you can use a pseudonym if you'd like. Similarly, you do not have to use your personal Github account. You can create one just for this class with a pseudonym. I value and respect your privacy. This decision is up to you.

Jon Froehlich

link to GitHub project
ArduinoGraphScreenshot1.pngArduinoGraph7AbstractScreenshot3.png

Sriram Karthik Badam

Github link

Sensor Visualization.pngAbstract Visualization.png

Seokbin Kang

link to GitHub project
visualization1.pngvisualization2.png

Beth McNany

link to GitHub project
external image QuqQpJq.png external image XgYT2mI.png

Jonggi Hong

link to GitHub project
a2_graph.pnga2_abstract.png

Majeed Kazemitabaar

link to GitHub project
visualizer.pngabstract vis.png

Raul Guerra

link to GitHub project

Screen Shot 2015-02-05 at 8.02.07 AM.png

Kentaro Matsumoto

Github link
KM_IA02_ScreenShot.png

Luis Santos

Github: Grapher; Drawuino
external image graph.png
external image ChangeTransparencyExample.pngexternal image changeColorExample.png

Philip Dasler

Github: Arduino Visualizer
Screen Shot 2015-02-05 at 11.03.43 AM.pngScreen Shot 2015-02-05 at 11.12.20 AM.png

Brian B.

Github: IA2-Brian

StandardPicBB.jpg

InkpenPicBB.jpg

Notes for Next Time

These notes are for the next time I teach this course (or use this assignment):
  • Make it a requirement that the visualizers should automatically adapt to an arbitrary number of inputs or, at least, the 11 digital inputs and 6 analog inputs off the Uno.
  • In addition to screenshots, have students take little movies of their visualizers
  • Make it more clear that the visualizer windows need to be resizable and the data should adapt accordingly