Miłosz Orzeł

Time for the third episode of "Out of Boredom" series :) There was a Sonar project and something about shooting paintballs... This time you will learn how to use Arduino and .NET 4.5 to receive input from joystick and use it to control servos that move a webcam horizontally and vertically!

• Here you can see a video of the project outcome: Vimeo
• This repository contains all the code: GitHub

How it works? Short version: joystick (in my case Logitech Extreme 3D Pro) is connected to laptop with Windows 7 via USB. Desktop application (.NET 4.5/WinForms) uses SharpDX library (managed DirectX API) to pull position information from joystick. This info is presented in numerical and graphical way on UI (C# 5.0 async helps here). Joy position is then translated into desired pan&tilt servo angles and that data is sent to Arduino Uno through serial port. Arduino receives the data, and thanks to its Servo library, commands servos to move... 

Longer description is split into 2 parts. The first describes desktop app, the second shows Arduino sketch...

1. Desktop application

Main task of the program is to read joystick position. This is easy thanks to SharpDX 2.6.2.0 library that wraps DirectX API so it can be conveniently operated with C#. SharpDX and SharpDX.DirectInput NuGet packages are used in the app. This is the class that contains all the code necessary to monitor joystick movements:

using SharpDX.DirectInput;
using System;
using System.Linq;
using System.Threading;

namespace ServoJoyApp
{
    public class JoystickMonitor
    {
        private string _joystickName;

        public JoystickMonitor(string joystickName)
        {
            _joystickName = joystickName;
        }

        public void PollJoystick(IProgress<JoystickUpdate> progress, CancellationToken cancellationToken)
        {
            var directInput = new DirectInput();
            
            DeviceInstance device = directInput.GetDevices(DeviceType.Joystick, DeviceEnumerationFlags.AttachedOnly)
                                        .SingleOrDefault(x => x.ProductName == _joystickName);

            if (device == null)
            {
                throw new Exception(string.Format("No joystick with \"{0}\" name found!", _joystickName));
            }
            
            var joystick = new Joystick(directInput, device.InstanceGuid);
            
            joystick.Properties.BufferSize = 128;
            joystick.Acquire();

            while (!cancellationToken.IsCancellationRequested)
            {
                joystick.Poll();
                JoystickUpdate[] states = joystick.GetBufferedData();
                
                foreach (var state in states)
                {
                    progress.Report(state);                    
                }
            }
        }
    }
}

DirectInput class lets us obtain access to joystick. Its GetDevices method is used to look for an attached joystick with particular name. If such device is found, object of Joystick class gets created. Joystick class has Poll method that fills a buffer containing information about joysticks states. State info comes in a form of JoystickUpdate structure. Such structure can be used to determine what button was pushed or what is the current position in y-axis for example.

Here's an example of reading current joystick position on x-axis:

if (state.Offset == JoystickOffset.X)
{
      int xAxisPosition = state.Value;
}

Position is kept in Value property but before using it you have to check what that value means. This can be done by comparing Offset property to desired JoystickOffset enum value. See the docs of JoystickOffset to see what kind of values you can read.

PollJoystick method presented earlier has fallowing signature:

public void PollJoystick(IProgress<JoystickUpdate> progress, CancellationToken cancellationToken)

IProgress generic interface was introduced in .NET 4.5 to allow methods to report task progress. PollJoystick method uses it to notify the rest of the program about changes in joystick state. This is done by progress.Report(state) call. The second parameter (with CancellationToken type) lets as stop joystick polling any time we want. PollJoystick method does it when IsCancellationRequested property of CancellationToken structure is set to true. Is it necessary to use this async-related stuff to poll joystick data? No - it's possible to put joystick polling loop directly in button event handler but then all the work will be executed in UI thread and it will make application unresponsive! Here's how you can run joystick monitoring in modern C#:

private async void btnJoystickMonitorStart_Click(object sender, EventArgs e)
{
    try
    {
        btnJoystickMonitorStart.Enabled = false;
        btnJoystickMonitorStop.Enabled = true;

        var joystickMonitor = new JoystickMonitor(txtJoystickName.Text.Trim());

        _joystickMonitorCancellation = new CancellationTokenSource();
        var progress = new Progress<JoystickUpdate>(s => ProcessJoystickUpdate(s));
        await Task.Run(() => joystickMonitor.PollJoystick(progress, _joystickMonitorCancellation.Token), _joystickMonitorCancellation.Token);
    }
    catch (Exception ex)
    {
        MessageBox.Show(ex.Message, "Oh no :(", MessageBoxButtons.OK, MessageBoxIcon.Error);
    }
}

Notice that button even handler is marked with async keyword. Before PollJoystick task is started a new cancellation token is created and ProcessJoystickUpdate is set as a handler for asynchronous task progress notifications. When this setup is done joystick monitoring task is started with await Task.Run call...

This is a part of code responsible for handling joystick state changes:

private void ProcessJoystickUpdate(JoystickUpdate state)
{
    if (state.Offset == JoystickOffset.X)
    {
        int xAxisPercent = GetAxisValuePercentage(XAxisMax, state.Value);
        pnlXAxisPercent.Width = (int)xAxisPercent;
        lblXAxisPercent.Text = xAxisPercent + "%";
        lblXAxisValue.Text = state.Value.ToString();

        if (rbPanOnXAxis.Checked)
        {
            _panServoPosition = MapAxisValueToPanServoPosition(state.Value, XAxisMax);         
            lblPanServoPosition.Text = _panServoPosition.ToString();
        }
    }
	
	// ... more ...

As you can see JoysticUpdate structure is used to determine current position in x-axis. UI elements are updated and desired servo position is calculated... If you've looked carefully you might be wondering why if (rbPanOnXAxis.Checked) exists. This is done because the app lets its users decide whether pan (horizontal movement) servo should be bound to x-axis (right-left stick movement) or to zRotation-axis (controlled by twisting wrist on joystick - not all joysticks have this feature).

private byte MapAxisValueToPanServoPosition(double axisValue, double axisMax)
{            
    byte servoValue = (byte)Math.Round((axisValue / axisMax) * (PanServoMax - PanServoMin) + PanServoMin);
    return chkPanInvert.Checked ? (byte)(PanServoMax - servoValue) : servoValue;
}

Values of my joystick position are reported in 0 to 65535 range but only numbers from 0 to 180 are meaningful for servos I've used. That's why method MapAxisValueToPanServoPosition presented above was created...

Ok, so we are done with detecting joystick movements! Now we need to send desired servo position to Arduino. Fortunately this is really simple thanks to SerialPort component that you can use in WinForms programs. Just drag this component from toolbox and use code as below to control connection with Arduino (spArduino is the name I've given to SerialPort component):

private void btnArduinoConnectionToggle_Click(object sender, EventArgs e)
{
    try
    {
        if (spArduino.IsOpen)
        {
            spArduino.Close();

            btnArduinoConnectionToggle.Text = "Connect";
        }
        else
        {
            spArduino.BaudRate = (int)nudBaudRate.Value;
            spArduino.PortName = txtPortName.Text;

            spArduino.Open();
            btnArduinoConnectionToggle.Text = "Disconnect";
        }
    }
    catch (Exception ex)
    {
        MessageBox.Show(ex.Message, "Oh no :(", MessageBoxButtons.OK, MessageBoxIcon.Error);
    }
}

In my case Arduino is accessible via COM3 port and baud rate of 9600 is sufficient. That's right - despite the fact that the device is connected to PC with USB cable it is accessible via COM port.

Sending servo position to Arduino is really easy:

private void tmServoPosition_Tick(object sender, EventArgs e)
{
    if (spArduino.IsOpen)
    {
        spArduino.Write(new byte[] { _panServoPosition, _tiltServoPosition, SerialPackagesSeparator }, 0, 3);
    }
}

spArduino.Write call is used to send an array of three bytes to Arduino. Two values are for requested servo potions and the last one is used to separate the pairs so servo controlling program can always distinguish between pan and tilt values. Writing over serial port is executed inside Tick method of Timer component. This time I didn't bothered with manual creation of background task. I just dragged Timer component and adjusted its Enabled and Interval properties to make the app communicate with Arduino every 10 milliseconds...

2. Arduino sketch

We've discussed an program that can detect joystick movements and send servo position requests over serial port. Now time for a piece of microcontroller software that can actually force servos to move. This is the whole code:

#include <Servo.h>  

const byte setupReadyLedPin = 8;
const byte panServoPin = 10;
const byte tiltServoPin = 12;
const byte separator = 255;

Servo panServo; 
Servo tiltServo; 

void setup() {  
    pinMode(setupReadyLedPin, OUTPUT);
         
    panServo.attach(panServoPin);   
    tiltServo.attach(tiltServoPin);   
    
    Serial.begin(9600); // Open connection with PC
    
    digitalWrite(setupReadyLedPin, HIGH);
}

void loop() {      
    if (Serial.available() > 2) {            
        byte panAngle = Serial.read();
        byte tiltAngle = Serial.read();
        byte thirdByte = Serial.read();
         
        if (panAngle != separator && tiltAngle != separator && thirdByte == separator) {         
            // Moving servos
            panServo.write(panAngle);
            tiltServo.write(tiltAngle);
        }
    }       
}

Yup, it's that easy! Servo library is included to allow for panServo and tiltServo objects to be created. These objects (of type Servo) make it possible to command servos to move into desired positions. This is done by calling write method with desired angle, like this:

panServo.write(panAngle);

Before it can be done however, servos have to be assigned to Arduino's output pins. This is achieved by calls to attach method seen in setup function. Digital servos are controlled by duration of ON pulse calculated in 20ms intervals, 1.5ms ON pulse should command a servo to move to the middle... But Servo library does all the heavy lifting for you so you don't have to create proper control signals manually. All you need to do is connect 3 cables each servo has. The servos I own use brown cable for ground, red for plus and orange for control signal. Sonar project used single micro servo so the only power supply needed was the one included in USB. This time two servos are utilized so you should add external power supply. I've connected 1A AC/DC power adapter through a plug that is included on Arduino Uno board and the servos worked really well. Arduino has a built-in fuse that protects USB port from overcurrent (it's a resettable fuse that doesn't allow current bigger than 500mA)...

Communication with .NET application is implemented with Serial class. First, in setup function, a connection is established with Serial.begin(9600) call. Then inside a loop Serial.available method is used to check if packet with servo positions request has arrived from PC. If so, pan&tilt servo angles are read and servos are ordered to move.

This is all that is necessary to control two servos with joystick connected to a computer :) In my project I've used DGServo S3003 servos with pan&tilt bracket to move A4tech PK-910H webcam and I'm really happy with the results! While watching the clip you may think that camera is moving to much compared to joystick movements. Keep in mind however, that servos move in 180 degrees but my joystick operates in smaller range. This is why small stick movement results in big camera swing. Despite that, I was able to control servos position with 1-degree precision quite easily... 

Update 2015-01-11: Click here to see a video of my recent project that has pan and tilt servos used to build gun turret!

My first Arduino based project was Sonar with C#, JS and HTML5. Now I continue the "Out of Boredom" series with a setup that allows firing a paintball marker (gun) with a command sent from a computer :) This time software stack is simpler - just a small Arduino sketch and uncomplicated WinForms (.NET/C#) application, but hardware requires a bit of electronics knowledge. Don't worry though, nothing too complicated - I'm definitely not an expert in this field... 

The project is based around an electromechanical relay. Such relay is basically an electronically controlled switch that allows you to turn powerful devices on and off by sending a signal from Arduino's output pins. You can control very large motors or light bulbs for example. The beauty of this components is the fact that you can govern external circuits - there is no physical link between your control circuit and the thing you want to turn on/off. You can use a relay for devices that require huge current but you can also control more subtle equipment, and that's what I decided to do. I play paintball/speedball and I happen to own high-end electro-pneumatic marker called DM13. Such marker has electronically operated trigger and uses solenoid valve to shoot... I thought: "Wouldn't it be cool to press enter on my laptop and make this gun fire nearly 20 balls per second?"... See this video to see how it worked :)

This is a list of hardware parts used:

And this is the circuit diagram:

LED connected to Pin 13 is used to signal that device is ready, LED attached to Pin 12 indicates firing (closed trigger circuit). LEDs are of course not connected to Arduino directly, there are resistors protecting them from overcurrent. Buzzer is there to make one tone when device is ready and another (higher) tone when device is firing. The purpose of a switch is to make your life easier while testing. Buzzer sound can get annoying quickly so you can turn it off...

These are the boring bits, the more interesting stuff is on the left side of the diagram. Pin 2 is connected (via resistor) to a base of NPN transistor and collector is attached to relay coil. The transistor is needed because the coil, which controls the switching function, needs more power then Arduino output pins can supply. In this circuit transistor is used not as amplifier but as a switch. When Pin 2 is set to HIGH (base-emitter voltage = 5V) the transistor reaches its fully-on state and current flows through collector energizing the coil. Putting HIGH state on Pin 2 results in a connection between relay's COM (Common) and NO (Normally Open) pins. I've checked my marker with a multimeter (in resistance mode) and I was able to see that pulling the trigger resulted in a closed circuit between middle and lower pins of the trigger switch. Attaching one cable between middle pin of the switch and COM pin, and another cable between lower switch pin and Normally Open pin gives the ability to simulate trigger pull. In other words: HIGH state on Arduino's Pin 2 equals trigger pull as far as maker is concerned*. It's quite simple but as you've seen on the video it works really well! One more thing: look on the diagram on the right-hand side of JZC-11F relay - there's a signal diode and it's purpose is to protect the transistor from voltage spikes that appear when Pin 2 to is put to LOW state (when supply voltage is removed from relay's coil). Such diode usage is called "flyback" or "freewheeling"... I first created this circuit on a breadboard and then soldered it on a universal board... Keep in mind that there are multiple Arduino relay shields available so you don't really have to create such transistor based circuit yourself. I did it because I think that it's a nice way to refresh some very basic electronics knowledge. Ok, we are done with hardware!

Now time for software (this GitHub repository contains all the code)!

Here's complete Arduino sketch:

const byte fireRelayPin = 2;
const byte fireBuzzerPin = 11;
const byte fireLedPin = 12;
const byte readyLedPin = 13;

const byte readyToFireMessage = 6; // ASCII ACK
const byte fireCommand = 70; // ASCII F
const byte triggerPullDelayInMs = 30;
const byte fireBuzzerHz = 1000;
const byte readyBuzzerHz = 400;

void setup() {  
    pinMode(fireRelayPin, OUTPUT);
    pinMode(fireBuzzerPin, OUTPUT);
    pinMode(fireLedPin, OUTPUT);
    pinMode(readyLedPin, OUTPUT);
  
    Serial.begin(9600);
    
    tone(fireBuzzerPin, readyBuzzerHz);  
    digitalWrite(readyLedPin, HIGH); 

    Serial.write(readyToFireMessage);     
}

void loop() {
   if (Serial.available()) {
        byte data = Serial.read(); 
        
        if (data == fireCommand) {
            pullTrigger();
            delay(triggerPullDelayInMs);  
            releaseTrigger();
        }
    }  
}

void pullTrigger() {
    digitalWrite(fireLedPin, HIGH);    
    digitalWrite(fireRelayPin, HIGH);  
    tone(fireBuzzerPin, fireBuzzerHz);     
}

void releaseTrigger() {
    digitalWrite(fireLedPin, LOW);    
    digitalWrite(fireRelayPin, LOW);  
    tone(fireBuzzerPin, readyBuzzerHz);  

    Serial.write(readyToFireMessage);     
}

Nothing complicated :) First, the usual (good) practice of creating constants for pin numbers and other useful values. Then there is a setup method that configures pin modes, initializes serial connection (used to communicate with PC), and turns on the "ready" LED. There is also a call to tune function. tune is used to generate sound by making piezoelectric element in buzzer vibrate at a designated frequency. In loop function program waits for data sent by PC via serial port and checks if this data means a fire command (ASCII letter 'F'). If so, a trigger pull is simulated followed by a slight delay and trigger release. The digitalWrite(fireRelayPin, HIGH); line is what makes paintball gun fire. If you've looked carefully ,you've noticed Serial.write(readyToFireMessage); calls in setup and releaseTrigger functions. These exist to let computer know that Arduino is ready to receive first fire command or is ready to process a new one.

This is the .NET 4.5 WinForms application build to control paintball marker:

Pressing "Fire" button once makes the marker fire one shot, pressing and holding it makes it fire in a series. How many balls per second will be fired depends of course on triggerPullDelayInMs value, relay speed, marker settings (mine DM13 was setup in semi mode capped to around 20bps), paintball loader speed etc. Shooting is only possible when "Device is ready to fire" shows "YES" and "Safe" checkbox is not ticked.

Here's the code behind the CommandWindow show above (few particularity dull lines removed):

using System;
using System.Drawing;
using System.IO.Ports;
using System.Threading;
using System.Windows.Forms;

namespace PbFireApp
{
    public partial class CommandWindow : Form
    {
        private const byte ReadyToFireMessage = 6; // ASCII ACK
        private const byte FireCommand = 70; // ASCII F

        private bool _isReadyToFire;
        private bool IsReadyToFire
        {
            get
            {
                return _isReadyToFire;
            }
            set
            {
                _isReadyToFire = value;
                 SetIsReadyToFireLabel();
            }
        }

        public CommandWindow()
        {
            InitializeComponent();
            
            IsReadyToFire = false;
            spArduino.DataReceived += new SerialDataReceivedEventHandler(DataReceivedHandler);
        }

        private void DataReceivedHandler(object sender, SerialDataReceivedEventArgs e)
        {
            byte data = (byte)spArduino.ReadByte();
            IsReadyToFire = data == ReadyToFireMessage;          
        }

        private void Fire()
        {
            if (!chkSafe.Checked && IsReadyToFire)
            {
                IsReadyToFire = false;
                spArduino.Write(new byte[] { FireCommand }, 0, 1);
            }
        }

        private void btnConnect_Click(object sender, EventArgs e)
        {
            try
            {
                if (spArduino.IsOpen)
                {
                    spArduino.Close();

                    btnConnect.Text = "Connect";                
                    gbFire.Enabled = false;
                }
                else
                {
                    spArduino.BaudRate = (int)nudBaudRate.Value;
                    spArduino.PortName = txtPortName.Text;

                    spArduino.Open();

                    btnConnect.Text = "Disconnect";            
                    gbFire.Enabled = true;
                }
            }
            catch (Exception ex)
            {
                MessageBox.Show(ex.ToString(), "Oh no :(", MessageBoxButtons.OK, MessageBoxIcon.Error);
            }
        }
		
        // ... more ...

        delegate void SetIsReadyToFireLabelCallback();

        private void SetIsReadyToFireLabel()
        {
            if (lblIsReadyToFire.InvokeRequired)
            {
                SetIsReadyToFireLabelCallback d = new SetIsReadyToFireLabelCallback(SetIsReadyToFireLabel);
                Invoke(d, new object[] { });
            }
            else
            {
                lblIsReadyToFire.Text = IsReadyToFire ? "YES" : "NO";
                lblIsReadyToFire.ForeColor = IsReadyToFire ? Color.Orange : Color.Black;
            }
        }
    }
}

The app is so simple that I decided to put all the code in to form's cs file... SerialPort component is used to communicate with Arduino (I've written a bit more about serial communication in Sonar project posts). Paintball marker is instructed to fire when 'F' command (short for "Fire", ASCII code 70) is sent to Arduino. This line is responsible for it: 

spArduino.Write(new byte[] { FireCommand }, 0, 1);

DataReceivedHandler method is used to set IsReadyToFire property to true when ACK (Acknowledge, ASCII 6) message is obtained from Arduino...

The code is quite obvious except for the SetIsReadyToFireLabel method. Why it is needed? The color and text of a Label control are changed when IsReadyToFire property is set. And that can happen when DataReceivedHandler is executed. We can't directly change UI elements from SerialPort.DataReceived event handler because this will result in an InvalidOperationException with a message such as this: "Cross-thread operation not valid: Control 'lblIsReadyToFire' accessed from a thread other than the thread it was created on.".

And that's it, second "Out of Boredom" project is complete :) 

1. It should be possible to control solenoid valve directly but that would be more complicated and might result in damaging my precious DM13.…