June 22, 2018 by Zac Jackson
Learn how to connect your iPhone to a Processing sketch and using the Tramontana app and the WebSockets library.
Article by Pierluigi Dalla Rosa and Ankkit Modi
Have you ever wanted to use your iPhone as an input or output device for a project?
Or thought about using your iPhone as a reactive output or as an input device and connect it to a Processing sketch?
Using Tramontana, you can connect your iPhone to a Processing sketch and tap into all the sensor data from the iPhone like the accelerometer, microphone, etc. You can even use it as an output device and display what you want through its screen. And it’s all possible through Processing, without having to even touch a line of swift code or get into app development.
With Tramontana, it's simple to architect and choreograph interactive spaces, interactive prototypes or multi-device experiences.
In this article, we will build the first step towards creating any of the above. We'll go over how to connect your computer to your device and have them talk to each other.
Have you ever wanted to use your iPhone as an input or output device? Or thought about using your iPhone as a reactive output or as an input device and connecting it to a Processing sketch?
In this project, we'll do exactly this.
We're going to break the old paradigm of having to build a custom application for each device. You can write just one application and have it run or connect to multiple devices. You can also choose to write that application in Javascript or openFrameworks or Processing. In this example, we will go with Processing.
Our goal for this project is to change the screen color on the iOS device by key presses on the keyboard.
If you are not familiar with writing Processing programs or programming of any kind, you can browse the Processing Foundation's tutorials to get a jump start.
For this project, you first need to make sure your computer and device are on the same Wi-Fi network. Tramontana will not work if your devices are on different networks.
Second, have the Tramontana app installed on your iOS device from the app store.
And, finally, you need to have a version of Processing installed on your computer. You now need to install two dependencies:
Both libraries can be found on the “Import Library > Add Library” option under the Sketch menu. Once you find and install them, you are ready to write the program.
Below are screenshots of importing Tramontana and the WebSocket library:
You can follow along the installation process in the video below:
Create a new sketch by hitting cmd+n on your keyboard or by clicking File>New.
Import the library by typing
import websockets.*;
import tramontana.library.*;
or by clicking on the sketch menu and navigating to the Import Library and clicking on the Tramontana and repeating the same for WebSocket.
Create an object of class in Tramontana that is the representation of your physical device.
So, in this case, I will type:
[CODE]
Tramontana iPhone;
[/CODE]As you can learn from the Processing tutorials, a Processing sketch has a setup method and a draw method.
We are going to instantiate the Tramontana object that we just created in the setup method:
[CODE]
iPhone = new Tramontana(this, "192.168.2.14");
[/CODE]You can find the IP address of your device directly on the starting screen on the Tramontana app. IP addresses are made of four numbers between 0 and 255, separated by dots.
After we have instantiated the object, the connection to the device is opened and you can send commands to Tramontana.
In our first tutorial, we want to change the color of the iPhone while we press different keys on the computer keyboard.
You can interact with keyboard events with keyPressed.
[CODE]
void keyPressed() {
}
[/CODE]We are going to add an if condition, where if we press a key—for instance ‘a’—we change the color to red and if we press another key—say, ‘s’—we change the color to blue.
We can accomplish this with the following code:
[CODE]
void keyPressed() {
if(key=='a') {
iPhone.setColor (255,0,0,255);
}
else if(key=='s') {
iPhone.setColor (0,0,255,255);
}
}
[/CODE]Hit play and you should see the tramontana console on your phone stating that the connection is open.
And after you type 'a' or 's' on your keyboard you should see your screen change color.
This is how it will work if you have set everything up correctly:
Congratulations! You just created your first multi-device interactive experience with Tramontana.
From here, you can explore other cool interactions that Tramontana can unlock for you and start building amazing prototypes in your iPhone in a matter of a few minutes without touching a line of swift code.
