morzel.net

.net, js, html, arduino, java... no rants or clickbaits.

Save yourself some troubles with TortoiseGit pre-commit hook

INTRO

Back in 2013, when I was using SVN, I wrote the post about creating a TortoiseSVN pre-commit hook that can prevent someone from committing code which is not meant to be shared (e. g. some hack done for troubleshooting). The idea was to mark “uncommittable” code with a comment containing NOT_FOR_REPO text and block the commit if such text is found in any of the modified or added files… The technique saved me a few times and proved to be useful to others…

This days I’m mostly using Git, and with Git’s decentralized nature and cheap branching the above technique is less needed but might still be helpful. The good news is that the same hook can be used in both TortoiseSVN and in TortoiseGit (I like to do commits with Tortoise UI and reserve command line for things like interactive rebase)…

First I will show you how to implement a pre-commit hook (I will use C# but you can use anything that Windows can run) and then you will see how to setup the hook in TortoiseGit Settings... 

 

TORTOISE PRE-COMMIT HOOK IN C#

You can find full code sample in this GitHub repository (it's a C# 6 console project from Visual Studio 2015, targeting .NET 4.5.2). Below is the class that implements the hook:

using System;
using System.IO;
using System.Text.RegularExpressions;

namespace DontLetMeCommit
{
    class Program
    {
        const string NotForRepoMarker = "NOT_FOR_REPO";

        static void Main(string[] args)
        {
            string[] affectedPaths = File.ReadAllLines(args[0]);
                        
            foreach (string path in affectedPaths)
            {
                if (ShouldFileBeChecked(path) && HasNotForRepoMarker(path))
                {
                    string errorMessage = $"{NotForRepoMarker} marker found in {path}";
                    Console.Error.WriteLine(errorMessage); // Notice write to Error output stream!
                    Environment.Exit(1);
                }
            }
        }

        static bool ShouldFileBeChecked(string path)
        {
            // Here we are choosing to check only selected file types but you may want to check
            // all the files except specified types or skip filtering altogether...
            Regex filePattern = new Regex(@"^.*\.(cs|js|xml|config)$", RegexOptions.IgnoreCase);

            // List of files affected by the commit might include (re)moved files so we check if file exists...
            return File.Exists(path) && filePattern.IsMatch(path);
        }

        static bool HasNotForRepoMarker(string path)
        {
            using (StreamReader reader = File.OpenText(path))
            {
                string line = reader.ReadLine();

                while (line != null)
                {
                    if (line.Contains(NotForRepoMarker)) 
                        return true; // "Uncommittable" code marker found - let's block the commit!

                    line = reader.ReadLine();
                }
            }

            return false;
        }
    }
}

How it works?

When Tortoise calls a pre-commit hook it passes a path to temporary file as a the first argument (args[0]). Each line in that file contains a path to a file that is affected by the commit. Hook reads all the lines (paths) from tmp file and checks if NOT_FOR_REPO text appears in any of them. If that's the case the commit is blocked by ending the program with non-zero code (call to Environment.Exit). Before that happens, a message is printed to Error stream (Tortoise will present this message to user). HasNotForRepoMarker method checks file by reading it line-by-line (via StreamReader) and stopping as soon as the marker is found. On my laptop full scan of 100 MB text file with one million lines takes about half a second so I guess its fast enough :) ShouldFileBeChecked method is there to decide if a path is interesting for us. We definitely don't want to check paths of removed files, hence the File.Exists call. I've also added Regex file name pattern matching to show you that you can be quite picky about which files you wish to check... That's it, compile it and you can use it as a hook!

 

ENABLING THE HOOK IN TORTOISEGIT SETTINGS

To set the hook first right click any folder and open TortoiseGit | Settings menu (I'm using TortoiseGit 2.1.0.0):

Menu step 1

Then go to Hook Scripts section and click Add... button:

Menu step 2... Click to enlarge...

Now choose Pre-Commit Hook type, next choose a Working Tree Path (select a folder which you want to protect with the hook - its subdirectories will be covered too!), and then choose Command Line To Execute (in case of C# hook this is an *.exe file). Make sure that what Wait for the script to finish and Hide script while running checkboxes are ticked (first checkbox is to make sure that commit is not going to complete unit all files are scanned and the second prevents console window from appearing). Her's how my settings look like:

Menu step 3... Click to enlarge...

Now click OK and voila - you have a pre-commit hook. Let's test it...

 

TESTING THE HOOK 

To check if the hook is working I've added NOT_FOR_REPO comment in one of the files from C:\Examples\Repos\blog-post-sonar Git repository:

namespace SonarServer
{
    class Program
    {
        const byte DataSampleStartMarker = 255; // NOT_FOR_REPO
        static List<byte> rawSonarDataBuffer = new List<byte>();
		

 I also did some other modification in a different file and removed one file, so my commit window looked like this:

Git commit... Click to enlarge...

After clicking Commit button the hook did it's job and blocked the commit:

Git commit blocked

Cool, and what if you really want to commit this even if the NOT_FOR_REPO marker is present? In that case use can do the commit through Git command line because TortoiseGit hook is something different than a "native" Git hook (from .git/.hooks)

And here's a proof that the same hook works when used with TortoiseSVN:

SVN commit blocked... Click to enlarge...

TortoiseSVN window looks a bit nicer and has Retry without hooks option...

 

 

Easy way to fix outdated links (URL rewrite rule in Web.config)

INTRO

I’ve recently moved my site from BlogEngine.NET 2.0 to 3.3 – thanks to the great work done by BlogEngine.NET team the migration was easy... The only serious problem I’ve noticed was with post links ending with .aspx. For example when Google or CodeProject had a link to such URL:

http://en.morzel.net/post/2014/09/24/OoB-Sonar-with-Arduino-C-JavaScript-and-HTML5-(Part-2).aspx

the post was not found. If .aspx suffix was removed from the address:

http://en.morzel.net/post/2014/09/24/OoB-Sonar-with-Arduino-C-JavaScript-and-HTML5-(Part-2) 

everything was working fine! Fortunately fixing it didn’t require any BlogEngine code changes – all thanks to URL Rewrite Module 2.0 (available since IIS 7) and system system.webServer/rewrite Web.config section.

URL Rewrite is a big topic. Checkout http://www.iis.net/learn/extensions/url-rewrite-module docs if you want to know all the details – you can even do things like setting HTTP headers or server variables! In this post I will focus on how to fix the .aspx link problem and I will also note some issues you might face while trying to setup you own URL rewrite rules…

 

SETTING THE RULES

I’ve added such rewrite section inside system.webServer node in Web.config file:

<rewrite>
    <rules>
        <rule name="FixOldAspxLinks" stopProcessing="true">
            <match url="^(.*post/.+)\.aspx$" />
            <action type="Redirect" url="{R:1}" redirectType="Permanent" />
        </rule>
    </rules>
</rewrite>

It has a single rule that matches all addresses that contain post/ and end with .aspx and triggers redirect action to the same address but with .aspx part dropped.

The rule

Rule has a name (something describing the purpose of the rule is welcome) and stopProcessing=”true” setting which instructs IIS to skip any further rules for matched URL (yes, there's only one rule but having stopProcessing=”true” makes our intention clear).

The match

If you are familiar with regular expressions the url="^(.*post/.+)\.aspx$" attribute should be obvious, if not - don’t worry, it’s simpler than it looks:

  • ^ – means beginning of URL
  • $ – means the end of URL
  • .* – means any character zero or more times
  • .+ – means any character at least once
  • \. – means a literal dot (in regexes . stands for any character so if we literally want to look for a dot we need to escape the special meaning by preceding it with backslash)
  • () – parentheses denote the text (capturing group) what we are going to reference in action element by using {R:1} 

The matching expression could be written in many ways but the one I’ve used solves the problem without going overboard with URL pattern recognition…

The action

We want the browser to look for a new address hence type="Redirect" is set.
New address is specified with url="{R:1}". The {R:1} is a reference to the group captured by matching expression – its value is the text found between parentheses. In our case it’s everything that preceded the .aspx suffix. redirectType="Permanent" instructs the server to issue a 301 Moved Permanently response to the browser. When HTTP client receives permanent redirect it will use the new URL each time it sees a link to the old URL…

Ok, so the above rewrite should be all that’s needed to make .apsx problem disappear! Doesn’t work on your machine? Read on!

 

POSSIBLE ISSUES

No URL Rewrite module installed

Before pushing any changes to remote server I wanted to check rewrite settings on my local IIS 7.5 on Windows 7 x64. I did it and instead of redirect I got HTTP Error 500.19 – Internal Server Error. The error page was useless as it didn’t show any hint on what was wrong with the config... If you face the same issue you probably don’t have IIS Rewrite module installed (it is not added by default). Quick way to find out if you have the module is to check if this file exists: 

%windir%\System32\inetsrv\config\schema\rewrite_schema.xml

I got the installer from here: https://www.microsoft.com/en-us/download/details.aspx?id=7435. After module was added to IIS the rewrite rule started to work :)

Redirect caching

Rewrite rule is setup to redirectType="Permanent" because we want to teach HTTP clients that the resource is moved for good, right? It's all ok unless you are during development and do some changes to the rule – if browser already received 301 response for particular URL your modified rule will not get a chance to work! To solve this problem you can clear the cache but I prefer to have Chrome's dev tools open (with caching disabled) or try to open the page in fresh incognito window…

Pattern testing

Regular expressions are powerful tool but it's very easy to make a mistake while working with them. Fortunately IIS Rewrite Module has it's own panel (snap-in) in IIS Manager:

URL Rewrite module in IIS Manager... Click to enlarge...

that lists rewrite rules used for the site:

Rewrite rule in IIS Manager... Click to enlarge...

If you double click a rule, you will see a window that lets you change rule properties without manual modifications to Web.config. Pressing "Test pattern..." button opens the window in which you can quickly test your regular expression:

Pattern test in IIS Manager... Click to enlarge...

Differences in Java and C#: protected

Java and C# are very similar languages so if you have to switch between the two it’s easy to overlook subtle differences. One of the tricky bits is the meaning of protected access modifier. 

In C#, if you mark a field with protected keyword it will be available to the class that owns it and to its derived classes. In Java access will be broader. Not only the owner and derived classes will be able to access the field but also all classes defined in the same package. In C# similar effect can be achieved by assigning protected internal access level. Member marked like that has access which is a union of internal (same assembly) and protected levels. Important thing to note is that concepts of Java package and C# assembly are not equivalent. C# assembly can span multiple namespaces and is related to physical unit (EXE, DLL) that keeps intermediate code and metadata. Package in Java is more similar to namespace in C# with key (not only) difference that it has an impact on accessibility…

Below are two projects that show protected access level differences between Java and C# (both are available in this GitHub repository). C# program was made in Visual Studio 2015 Community and targets .NET 4.5.2. Java program was made in IntelliJ IDEA 15 Community Edition and is set to use Java 8. Version of Java/.NET is not relevant and there’s nothing special in the projects - could’ve been easily done in notepad but isn’t it great that this days we can get such awesome IDEs for free? :) 

Java and C# projects... Click to enlarge...

Java

Base.java

package com.example;

public class Base {
    protected int someProtectedFiled = 123;

    public void testAccessInBaseClass() {
        // SAME BEHAVIOR IN JAVA AND C#
        // The class can access its own protected field
        System.out.println(someProtectedFiled);
    }
}

Derived.java

package com.example;

public class Derived extends Base {
    public void testAccessInDerivedClass() {
        // SAME BEHAVIOR IN JAVA AND C#
        // The class can access inherited protected field
        System.out.println(someProtectedFiled);

        // DIFFERENT BEHAVIOR IN JAVA AND C#!
        // Access to protected field is possible because classes are in the same package
        // (notice access through qualifier of type Base instead of Derived)
        System.out.println(new Base().someProtectedFiled);
        // In C# the field would have to be public or protected internal (otherwise CS1540 error is produced)
    }
}

NotDerived.java

package com.example;

public class NotDerived {
    public void testAccessInNotDerivedClass() {
        // DIFFERENT BEHAVIOR IN JAVA AND C#!
        // Access to protected field is possible because classes are in the same package
        System.out.println(new Base().someProtectedFiled);
        // In C# the field would have to be public or protected internal (otherwise CS0122 error is produced)
    }
}

DerivedInAnotherPackage.java

package com.example.another;

import com.example.Base;

public class DerivedInAnotherPackage extends Base {
    public void testAccessInDerivedClassFromAnotherPackage() {
        // SAME BEHAVIOR IN JAVA AND C#
        // The class can access inherited protected field even from another package
        System.out.println(someProtectedFiled);

        // SAME BEHAVIOR IN JAVA AND C#
        // For the below to work (instead of compilation error) the field would have to be public
        // (notice access through qualifier of type Base instead of DerivedInAnotherPackage):
        // System.out.println(new Base().someProtectedFiled);
    }
}

C#

Base.cs

namespace Protected
{
    public class Base
    {
        protected int someProtectedFiled = 123;

        public void TestAccessInBaseClass()
        {
            // SAME BEHAVIOR IN JAVA AND C#
            // The class can access its own protected field            
            System.Console.WriteLine(someProtectedFiled);
        }
    }
}

Derived.cs

namespace Protected
{
    public class Derived : Base
    {
        public void TestAccessInDerivedClass()
        {
            // SAME BEHAVIOR IN JAVA AND C#
            // The class can access inherited protected field           
            System.Console.WriteLine(someProtectedFiled);

            // DIFFERENT BEHAVIOR IN JAVA AND C#!
            // For the below to work (instead of CS1540 compilation error) the field would have to be public            
            // or protected internal (notice access through qualifier of type Base instead of Derived):
            // System.Console.WriteLine(new Base().someProtectedFiled);
        }
    }
}

NotDerived.cs

namespace Protected
{
    public class NotDerived
    {
        public void TestAccessInNotDerivedClass()
        {
            // DIFFERENT BEHAVIOR IN JAVA AND C#!
            // For the below to work (instead of CS0122 compilation error) the field would
            // have to be public or protected internal:
            // System.Console.WriteLine(new Base().someProtectedFiled);
        }
    }
}

DerivedInAnotherAssembly.cs

using Protected;

namespace AnotherAssembly // Namespace doesn't matter
{
    public class DerivedInAnotherAssembly: Base
    {
        public void TestAccessInDerivedClassFromAnotherAssembly()
        {
            // SAME BEHAVIOR IN JAVA AND C#
            // The class can access inherited protected field even from another assembly            
            System.Console.WriteLine(someProtectedFiled);

            // SAME BEHAVIOR IN JAVA AND C#
            // For the below to work (instead of CS1540 compilation error) the field would have to be public
            // (notice access through qualifier of type Base instead of DerivedInAnotherAssembly):
            // System.Console.WriteLine(new Base().someProtectedFiled);           
        }
    }
}

I hope that comments in the code make everything clear and this dry topic is exhausted... I've just orderd USB shield for my Arduino so if it works the next post will be about moving this thing with PlayStation controller :)

Update (2016-03-14):

Unfortunately I don’t have time to write the post mentioned above but the good news is that you definitely can use SparkFun USB Host Shield with USB Host Shield 2.0 library to steer servos with PS3 controller :) Here’s a short video of how it worked on my paintball turret. And here’s a clip of test circuit (hardware diagram).

Setting version of assemblies in ASP.NET MVC application with TeamCity build feature

INTRO

Last five posts (1, 2, 3, 4, 5) were all about fun stuff with Arduino. Now it’s time for something more mundane ;) In this post I will show you how to create TeamCity build that automatically sets version information in all assemblies produced by ASP.NET application. It's nothing new but I hope to give you some useful background info and note a few gotchas you may face...

Complete code of sample application is available in this GitHub repository.

TeamCity has a build feature called AssemblyInfo patcher that makes setting assembly version easy... This feature is usable on any type of .NET project because it works by updating AssemblyInfo files. Content of such files is used to create version information that .NET Framework uses for picking up correct version of referenced assemblies. Version data is also shown in Windows file properties... Here's a part of AssemblyInfo.cs file which is automatically added by Visual Studio when you create a new project:

// Version information for an assembly consists of the following four values:
//
//      Major Version
//      Minor Version
//      Build Number
//      Revision
//
// You can specify all the values or you can default the Revision and Build Numbers
// by using the '*' as shown below:
[assembly: AssemblyVersion("1.0.0.0")]
[assembly: AssemblyFileVersion("1.0.0.0")]

It contains two attributes: AssemblyVersion and AssemblyFileVersion along with a comment that describes numbering pattern recommended by Microsoft. We will also use another attribute: AssemblyInformationalVersion which is not added by default. AssemblyVersion sets version number that is recognized by .NET for dependency resolution. AssemblyFileVersion is used for file version as seen by Windows and AssemblyInformationalVersion is meant more for human consumption as it can contain strings (we will make use of it for holding Git commit hash)... Detailed description of the meaning of these attributes is outside scope of this post but check this great SO answer if you want to know more.

 

SAMPLE APPLICATION

My test application was created in Visual Studio Community 2013 by using ASP.NET Web Application / MVC project template (C#/.NET 4.5). Two additional projects of Class Library type were added. Here’s how the full solution looks:

Visual Studio solution... Click to enlarge...

Home/Index.cshtml view generated by VS was modified to present version information pulled from three .NET assemblies that are produced by the solution (one is for main web app project and two other are for class libraries). Such div was added to the view:

<div class="row text-primary">
    <div class="col-md-12">
        <dl>
            <dt>Core assembly info:</dt>
            <dd>@ViewBag.CoreAssemblyInfo</dd>
            <dt>DataAccess assembly:</dt>
            <dd>@ViewBag.DataAccessAssemblyInfo</dd>
            <dt>Web assembly info:</dt>
            <dd>@ViewBag.WebAssemblyInfo</dd>
        </dl>
    </div>
</div>

You can see some Bootstrap classes there since nowadays Visual Studio templates use Bootstrap framework for styling...

This is how rendered view looks before TeamCity processes AssemblyInfo.cs files:

Version information in web app before TC build... Click to enlarge...

And here's how version info looks after version attributes are modified by build feature:

Version information in web app after TC build... Click to enlarge...

If you wonder how the view gets version info here's HomeController.Index action method: 

public ActionResult Index()
{
    ViewBag.CoreAssemblyInfo = SomeCoreClass.GetAssemblyInfo();
    ViewBag.DataAccessAssemblyInfo = SomeDataAccessClass.GetAssemblyInfo();

    Assembly assembly = Assembly.GetExecutingAssembly();
    string webAsseblyInfo = string.Format("Full Name = \"{0}\"; Informational Version = \"{1}\"",
                            assembly.FullName, FileVersionInfo.GetVersionInfo(assembly.Location).ProductVersion);
    ViewBag.WebAssemblyInfo = webAsseblyInfo;

    return View();
}

You can see how the most important assembly version number (the one used by .NET and designated by AssemblyVersion attribute) is a part of assembly's FullName. Informational version (the one that can have strings) is taken with the help of FileVersionInfo class. You can get the number form AssemblyFileVersion attribute too - just check all the interesting stuff that GetVersionInfo method returns... The same kind of code is used in GetAssemblyInfo methods in SomeCoreClass and SomeDataAccessClass.

Ok, so we have our test application - full code is here. Note: I’ve pushed all used Nuget packages to the repository – that takes some space in the repo and might be against recommended way of using Git but it makes TeamCity setup easier. If packages folder is not committed you can expect this type of error during build:

[Csc] App_Start\BundleConfig.cs(2, 18): error CS0234: The type or namespace name 'Optimization' does not exist in the namespace 'System.Web' (are you missing an assembly reference?)

To solve it you would have to restore Nuget packages during build (here’s some info on how to do it).

 

TEAMCITY CONFIGURATION

Now time for build server config! I assume that you have some working knowledge about setting TeamCity build for .NET application so I will discuss only the steps relevant to versioning. I’ve used TeamCity 9.1.3 but don't worry if you have a bit older TC (AssemblyInfo patcher feature exists for a while). I used TC to build code from Git repository checkout on my local drive...

Before setting up AssemblyInfo patcher, add two new build parameters: Minor and Major. These are meant to represent two initial segments of version number and should be set manually - it's your (technical/marketing?) decision whether to name your next version 1.9 or 2.0, right? 

Major and Minor build parameters... Click to enlarge...

Next step is to add AssemblyInfo patcher build feature:

AssemblyInfo patcher build feature... Click to enlarge settings...

And set its properties:

AssemblyInfo patcher settings... Click to enlarge...

I've decided to use such settings:

  • AssemblyVersion:   %Major%.%Minor%.%build.number%
  • AssemblyFileVersion:   %Major%.%Minor%.%build.number%
  • AssemblyInformationalVersion:   %Major%.%Minor%.%build.number%.%build.vcs.number%

You can see that our Major and Minor parameters are used. You can also see the use of TeamCity built-in parameter named build.number. Last attribute contains another TeamCity param: build.vcs.number. It gets version control revision id. I'm using Git so this is a long alphanumerical SHA-1 hash. It means that it cannot be used in setting AssemblyVersion attribute. If you try to do so you will get an error like this:

[Csc] Properties\AssemblyInfo.cs(35, 12): error CS0647: Error emitting 'System.Reflection.AssemblyVersionAttribute' attribute -- 'The version specified '2.1.13.536ea0163412325ab7962957ce1cec777799d587' is invalid'

If you try to use it for AssemblyFileVersion you can expect a warning: 

[Csc] CSC warning CS1607: Assembly generation -- The version '2.1.12.536ea0163412325ab7962957ce1cec777799d587' specified for the 'file version' is not in the normal 'major.minor.build.revision' format

But you can safely use it in AssemblyInformationalVersion as .NET doesn't care if you put letters there... Note: If you work with SVN instead of Git you are lucky because value returned for build.vcs.number is an integer and can be used in all three version-related attributes. If you really need to set revision in AssemblyVersion while using Git you might need to add a custom build step for creating integer id... Let's keep it simple here and leave the last part of version number intact (as 0)...

Once you have AssemblyInfo patcher feature configured and you run the build, you can expect such entries in the build log: 

[22:46:02]Step 1/1: Visual Studio (sln) (2s)
[22:46:02][Step 1/1] Update assembly versions: Scanning checkout directory for assembly information related files to update version to 2.1.14
[22:46:02][Update assembly versions] Updating assembly version in C:\TeamCity\buildAgent\work\8c2a410f7087e36b\.NET\AssemblyInfoTest\AssemblyInfoTest\Properties\AssemblyInfo.cs
[22:46:02][Update assembly versions] Updating assembly version in C:\TeamCity\buildAgent\work\8c2a410f7087e36b\.NET\AssemblyInfoTest\Core\Properties\AssemblyInfo.cs
[22:46:02][Update assembly versions] Updating assembly version in C:\TeamCity\buildAgent\work\8c2a410f7087e36b\.NET\AssemblyInfoTest\DataAccess\Properties\AssemblyInfo.cs

If all went ok your log should also contain something like this:

[22:46:05]Reverting patched assembly versions
[22:46:05][Reverting patched assembly versions] Restoring C:\TeamCity\buildAgent\work\8c2a410f7087e36b\.NET\AssemblyInfoTest\AssemblyInfoTest\Properties\AssemblyInfo.cs
[22:46:05][Reverting patched assembly versions] Restoring C:\TeamCity\buildAgent\work\8c2a410f7087e36b\.NET\AssemblyInfoTest\Core\Properties\AssemblyInfo.cs
[22:46:05][Reverting patched assembly versions] Restoring C:\TeamCity\buildAgent\work\8c2a410f7087e36b\.NET\AssemblyInfoTest\DataAccess\Properties\AssemblyInfo.cs

Don't worry, reverting takes place only in build agent work files. The build artifacts contain properly versioned assemblies. You've seen a proof of that rendered on HTML page, you can also check DLL files properties:

File properties with version information... Click to enlarge...

Properties window shows version set by AssemblyFileVersion and AssemblyInformationalVersion. I have Polish Windows so the properties are labeled Wersja pliku (it means File version) and Wersja produktu (it means Product version).

Keep in mind that AssemblyInfo patcher will not work if version attribute has non-standard format (or AssemblyInfo files are in unusual locations).

Let's say you have something like this (because you keep information about your product in static class constants):

[assembly: AssemblyVersion(ProductInfo.Version)]

You can expect such warning during build:

[Update assembly versions] Assembly info version was specified, but couldn't be patched in file C:\TeamCity\buildAgent\work\8c2a410f7087e36b\.NET\AssemblyInfoTest\AssemblyInfoTest\Properties\AssemblyInfo.cs. Is necessary attribute missing?

 

SUMMARY

And that's all! We have a TeamCity build that sets version information in ASP.NET MVC application assemblies :) 

If somebody will be interested I can write a little supplement to this post in which I will describe how to add version info into zip package (artifact) and how to display it on Team City UI...

[OoB] Controlling gun turret with Spring Boot REST service (Java/Arduino)

This is the fifth post in my little "Out of Boredom" series dedicated to hobby projects with Arduino. Previous posts were all .NET based:

Now it's time for a bit of Java!

I will show you how to use Spring Boot framework to quickly create RESTful web service that can receive JSON requests and send commands to Arduino. The aim is to control a servo and relay based gun turret such as this one:

Gun turret prototype... Click to enlarge...

Click here to get an idea of how such turret operates. The video shows a prototype based on ASG pistol. Don't worry, my aim is quite peaceful: I want to build paintball gun turret :)

I will not discuss any electronics setup or Arduino code in this post. Take a look at the articles linked above to see information about PC-Arduino communication, controlling servos and building relay based circuit... I assume that you know what Spring Boot is, but advanced knowledge is not required to follow this post.

Full code of the project is available on GitHub (includes both Java application and Arduino sketch)...

The easiest way to create basic working Spring Boot project is to use Spring Initializr. I started my project by filling the form like this:

Spring Initializr settings... Click to enlarge...

Note that Gradle is used to create (fat) Jar package, Java 1.8 is used and the only required dependency is Web. This setup will produce an application running on Tomcat Embedded so there's no need for installing any web server. The app has support for REST controllers so we will be able to handle JSON based communication in a clean way with very little effort...

 

Without further ado, here is the piece of code responsible for receiving messages from clients:

package springarduino;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class TurretController {
    private final ArduinoConnection arduino;

    @Autowired
    public TurretController(ArduinoConnection arduino) {
        this.arduino = arduino;
    }

    @RequestMapping(value = "turret/execute", method = RequestMethod.POST, consumes = "application/json")
    public TurretResponse executeTurretAction(@RequestBody TurretRequest request) {
        if (request.getPan() < 0 || request.getPan() > 180) {
            throw new IllegalArgumentException("Pan out of 0..180 range (" + request.getPan() + ")");
        }

        if (request.getTilt() < 0 || request.getTilt() > 180) {
            throw new IllegalArgumentException("Tilt out of 0..180 range (" + request.getTilt() + ")");
        }

        boolean sent = arduino.controlTurret(request.getPan(), request.getTilt(), request.isFire());
        if (!sent) {
            throw new RuntimeException("Command not sent :(");
        }

        return new TurretResponse(request.getId(), "Command sent :)");
    }
}

TurretController class has @RestController annotation and a public method marked with @RequestMapping. The mapping specifies that executeTurretAction method should be invoked whenever a client makes a POST request to .../turret/execute URL. Any HTTP client capable of sending POST with Content-Type="application/json" can communicate with such Spring controller method. It can be some desktop application or a simple HTML page with a bit of jQuery for example. I was using Postman Chrome App to prepare requests. In the next post I will describe how to communicate with such controller from smartphone that runs PhoneGap/AngularJS based application...

executeTurretAction methods expects one argument of type TurretRequest:

package springarduino;

public class TurretRequest {
    private int id;
    private int pan;
    private int tilt;
    private boolean fire;

    // public getters and setters hidden for brevity
}

If client sends JSON payload such as this:

{
    "id": "311",
    "pan": "111",
    "tilt": "99",
    "fire": "true"
}

Spring will take care of creating TurretRequest object. Our service method returns TurretResponse:

package springarduino;

public class TurretResponse {
    private int id;
    private String message;

    public TurretResponse(int id, String message) {
        this.id = id;
        this.message = message;
    }

    // public getters and setters hidden for brevity
}

If everything goes well, this kind of data will be sent back to client:

{
    "id": 19,
    "message": "Command sent :)"
}

You don't have to do anything special to make this happen. Spring chooses HttpMessageConverter implementation to create response in a format that is expected by the client. Another nice feature of our @RestController is the error handling. Let's say that client provides invalid value of tilt angle - this is what is sent back as response:

{
    "timestamp": 1424382893952,
    "status": 500,
    "error": "Internal Server Error",
    "exception": "java.lang.IllegalArgumentException",
    "message": "Tilt out of 0..180 range (222)",
    "path": "/turret/execute"
}

Such message is easy to handle in error callbacks (thanks to 500 HTTP status code) and contains useful properties such as message and exception.

Notice that arduino object of ArduinoConnection type is used inside executeTurretAction method. ArduinoConnecton is a Spring bean responsible for communicating with Arduino over serial port.

Our controller gets reference to proper object thanks to Spring's IoC container. TurretController constructor is annotated with @Autowired so Spring knows that ArduinoConnection object needs to be injected.

This is the class responsible for talking to Arduino:

package springarduino;

import gnu.io.NRSerialPort;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;

import javax.annotation.PostConstruct;
import javax.annotation.PreDestroy;
import java.io.DataOutputStream;

@Component
public class ArduinoConnection {
    private static final int MESSAGE_SEPARATOR = 255;

    private final Logger log = LoggerFactory.getLogger(this.getClass());

    @Value("${arduinoPortName}")
    private String portName;

    @Value("${arduinoBaudRate}")
    private int baudRate;

    private NRSerialPort serial;


    @PostConstruct
    public void connect() {
        log.info("ArduinoConnection PostConstruct callback: connecting to Arduino...");

        serial = new NRSerialPort(portName, baudRate);
        serial.connect();

        if (serial.isConnected()) {
            log.info("Arduino connection opened!");
        }
    }

    @PreDestroy
    public void disconnect() {
        log.info("ArduinoConnection PreDestroy callback: disconnecting from Arduino...");

        if (serial != null && serial.isConnected()) {
            serial.disconnect();

            if (!serial.isConnected()) {
                log.info("Arduino connection closed!");
            }
        }
    }

    public boolean controlTurret(int pan, int tilt, boolean fire){
        try {
            // Actual values sent to Arduino will be in proper unsigned byte range (0..255)
            byte[] message = new byte[]{(byte) pan, (byte) tilt, (byte) (fire ? 1 : 0), (byte) MESSAGE_SEPARATOR};

            DataOutputStream stream = new DataOutputStream(serial.getOutputStream());
            stream.write(message);

            log.info("Turret control message sent (pan={}, tilt={}, fire={})!", pan, tilt, fire);
            return  true;
        } catch (Exception ex) {
            log.error("Error while sending control message: ", ex);
            return false;
        }
    }
}

@Component annotation is there to show Spring that ArduinoConnection is a bean and as such Spring should take care of its lifecycle and usage as dependency. By default Spring creates beans in singleton scope. This is fine for us - we only need one such object. connect method is marked with @PostConstruct. This makes the method an initialization callback that gets invoked when ArduinoConnection object is created (it will happen when application is started). @PreDestroy is used on disconnect method to make sure that connection to serial port is released when the program is closed.

controlTurret method is the piece of code that is responsible for sending gun turret action request to Arduino. That method is used in TurretController.executeTurretAction, remember? it uses instance of NRSerialPort to communicate over serial port (gnu.io.NRSerialPort package makes it possible). It comes form NeuronRobotics/nrjavaserial library which is a fork of RXTX that greatly simplifies serial port access. nrjavaserial takes care of loading proper native library needed to access the port (it worked well on my Windows 7 x64). As stated before, I'm not going to discuss Arduino communication and microcontroller code in this post. I will just note that you don't have to worry about casting int to byte while message array is created. It's sad but Java doesn't have unsigned bytes so it will show (byte)MESSAGE_SEPARATOR (which is 255) as -1 while debugging, but a correct value will go over wire to Arduino. Take a look at this screen shot from Free Device Monitoring Studio (you can use this tool to check what data is sent to Arduino through serial port):

Bytes sent to Arduino... Click to enlarge...

Let's get back to ArduinoConnection class: portName and baudRate properties are marked with @Value annotations. This way it is very easy to take the settings from configuration file. All you have to do is to create a file named application.properties in /src/main/resources directory and values will be automatically loaded from config. Here's the content of application.properties file: 

server.address = 192.168.0.17
server.port = 8090

arduinoPortName = COM3
arduinoBaudRate = 9600

Apart from aforementioned settings for Arduino, there are two other elements, namely: sever.address and server.port. By default Spring Boot runs the application on localhost:8080. I've changed this to custom settings to make it easy to access the application from devices connected to my WiFi network... Accessing TurretController from such devices is the reason why I added following CorsFilter class to the project:

package springarduino;

import org.springframework.stereotype.Component;

import javax.servlet.*;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;

@Component
public class CorsFilter implements Filter {

    public void doFilter(ServletRequest req, ServletResponse res, FilterChain chain) throws IOException, ServletException {
        HttpServletResponse response = (HttpServletResponse) res;
        response.setHeader("Access-Control-Allow-Origin", "*");
        response.setHeader("Access-Control-Allow-Methods", "POST, GET, OPTIONS");
        response.setHeader("Access-Control-Max-Age", "3600");
        response.setHeader("Access-Control-Allow-Headers", "X-Requested-With");
        chain.doFilter(req, res);
    }

    public void init(FilterConfig filterConfig) {}

    public void destroy() {}
}

Thanks to this Cross-Origin Resource Sharing filter it's easy to make ajax calls to executeTurretAction method without using some trickery like JSONP to circumvent same-origin policy restrictions.

And that's it! All interesting elements of the Java web app were discussed. Full code is accessible on GitHub. Since this is Gradle based project, running it is as easy as typing gradlew run. I've included Gradle Wrapper in the repo so you don't even need to have Gradle installed...