Connecting Arduino to JavaScript / p5js

The idea of this project: Build a bridge between an Arduino Microcontroller and Computer Generated Art created in JavaScript and p5.js. This will let the visitors of your digital exhibition or installation generate graphics and patterns using buttons, sensors, switches, potentiometers, gyroscopes etc.

From processing to p5js

For several years there have been libraries and examples on how to accomplish this using processing. Nowadays, many designers, artists and developers have headed on to p5.js which offers many of processing’s features, but is completely based on web technologies (JavaScript, HTML, CSS). This makes it easier to learn and combine with other web technologies.

The problem: Make Arduino communicate with JavaScript

Without going too far in depth: The main problem is that JavaScript was never really intended to communicate with Arduino. The sketches you program and upload to Arduino are in C++. But on the Arduino there is basically no RAM – so doing “fancy stuff” an the Board itself is not an option.

This is why, when thinking of an art installation, in most cases you will want to reduce the Arduino to a device that will read the sensors, buttons, sliders, temperature etc. - but not do any heavy calculations or graphic generation. This will be the job of your Desktop computer.

Communication via Serial Port

But how does Arduino communicate with the desktop computer? It uses something that feels like it’s from the past century: the “serial port”. This is like a virtual cable that can only send bytes one-by-one, in a long row. The bytes get buffered in the wire until the other side (the computer) reads and deletes them from the queue.

The communication is very low-level: If the Arduino reads the position of a potentiometer, it is actually just reading the currency (voltage) at a certain pin. Arduino then pushes this value (a number between 0 and 1024 in the Serial Port and the computer grabs it from there and does something with it.

As long as we are using a programming language on our computer that can read bytes from the serial port, this is pretty straightforward and there are many libraries and frameworks out there to accomplish this. Two projects you should have heard of are:

If you feel at home in web technologies than working with the two can be a little “frustrating”. When styling things or rendering text or markup they have limits. On the other hand, learning p5.js is really fun for a web developer because you can use all your “secret” JavaScript, HTML and CSS magic to produce great and creative results - in 2D and even WebGL / 3D.

The problem is: JavaScript (for the web) can not read bytes from the Serial Port. This might change in the future - but currently the only way to get Arduino hooked up to JavaScript is by using a “babelfish” – a small program that understands “both sides”: It can read bytes from the serialport and then pass the values on to the JavaScript application.

This is the reason why most of the projects of this kind ask you to launch an additional application or start a special server to act as the “translator” and build the bridge between the serial port and your JavaScript / p5js application.

Keeping things simple

The main concept behind this project is:

Get the Board connected as fast and smart as possible
Have autodetect features for the hardware, reducing the need to write long configurations
Keep the methods to wire a physical sensor to p5js as simple and intuitive as createButton()
Make it easy to deploy the project as a standalone application, i.e. for exhibits or art installations
Use JSON to communicate between NodeJS or JavaScript and the Arduino Board

Software you’ll needed for your project

Download	Needed for…
Arduino IDE	software for uploading scripts to the Arduino board
nnarduino-sketch	the sketch that is uploaded on the Arduino
nnarduino for Mac or Windows	the App to run your p5js sketch

Example: Simple Push-Button

To demonstrate how easy it is, to connect a sensor with p5.js, have a look at the following sketch. Note that this is not an Arduino Sketch in C++ - it is written in JavaScript with p5js.

The physical button is connected to PIN 2 of the Arduino. To make p5js “watch” the button, all we need to do is write button = connectSensor( SENSOR_BUTTON, 2 );. The library will automatically search for the right port, connect with the Microcontroller, configure the pin and add an EventListener.

In the sketch, pressing the button will reduce the size of the circle on the canvas. Nothing really creative… but it saved you the headache that makes you uncreative!

let button;

function setup () {
   createCanvas(400, 400);
   button = connectSensor( SENSOR_BUTTON, 2 );
}

function draw () {
   background( 255 );
   if (button.value() == 1) {
      circle( 200, 200, 50 );
   } else {
      circle( 200, 200, 100 );
   }
}

Alternative projects

Of course there are many other project that aim towards a similar goal. During your research you probably came accross at least one of following projects and are thinking: “Why another one?”. Here is why:

Johnny-Five

Johnny-Five is probably one of the most famous and renowned projects to get the JavaScript hooked up with an Arduino and other Microcontrollers.

Personally I had massive problems installing Johnny-Five. The Serial communication simply wouldn’t work. This partly had to do with security updates in NodeJS / electron and partly with the Serial Communication itself. Even a really deep dive into the Firmata protocol which is heavily inspired by the Midi Message Format could not help solve the problems.

p5.bots

p5.bots - runs a NodeJS based p5bots-server that bridges p5js with the Arduino using the StandardFirmata Firmware.

This library needs a NodeJS script running in the background to function that needs to be started manually. Goal of my project was to have the option to build an executable app for Mac and Windows that can be launched with a simple double-click or automatically after restarting the computer, i.e. if it is part of an exhibition and must reboot automatically after a power failure.

p5.serialcontrol

p5.serialcontrol - a desktop app with GUI, similar to the p5bots-server. A tutorial can be found here

The two-component solution is great – but again: My project should stay as “compact” as possible in one single application.

Web Serial API

While capabilities of webbrowsers are evolving, Chrome will soon support the Web Serial Api. In some near future it will be possible to let Chrome communicate via serial port without any “Babelfish” in between. A nice blog entry describes the possibilities and has a demo.

For a standalone version (an installation in an exhibit) this could become interesting in the future. At the time of writing this, the application wrapped up with electron did not support this feature yet. But let’s keep eyes open!

Who’s behind this project?

I am a lecturer at the Hochschule Rhein-Main / University of Applied Sciences in Wiesbaden, Germany where I teach grafic design / visual arts. And at 99° I do design and build things.

In my graduate courses, I introduce design students to programming and try to show them that design and development are not as separable today as they were a few years ago. “Real designers write code. Always have. Always will.”

Since many of the students have little to no experience in coding, a good “gateway drug” is Generative Design. In few other disciplines does code directly result in such a visual outcome. Students already have a great sense of achievement in the first few lectures. Some catch fire with the subject, dive deep and create exciting corporate designs, posters or artistic graphics by the end of the semester.

“Could we do hardware?”

Last semester, some students approached me. They had set out to build a “machine” as part of their bachelor’s thesis. Their idea was to have switches, sensors and controllers that would create generative graphics.

I gave them some tips and links to literature and further info. Although they were quite fit in JavaScript and web technologies, they despaired at the absolute basics. What sounded rather simple at first turned out to be an odyssey through different techniques, programming languages and worlds: Starting with C++ on the Arduino, which today feels like programming in the Stone Age for a web developer. Then the serial communication from the Arduino to NodeJS and from Node to JavaScript or p5js.

Actually they got a lot further than I thought they would - but for me, that was the motivation to take all the stress away from my designers and give them a framework where they can be creative again without spending 90% of their time dealing with installation and compatibility issues.

Since I thought that other students might be interested in the topic, I made a course out of it this semester, which I called “Design Machine”. I seemed to have hit a zeitgeist with the topic - the demand was so overwhelming that I had to put off 3/4 of the students until the next semester.

Connect!

If you like this project or have any improvements, please contact me. If you found a bug, find the issue tracker here.