Miłosz Orzeł

TL;DR

You can use measureText method to calculate widths of text and then use setColumnWidth to adjust columns. This the demo and this is the repo. It's quite likely that you don't need this technique (suppressColumnVirtualisation might be enough), but read on if your grid has huge amount of columns and users need high data density...

VIRTUALIZATION VS RESIZE

I've been porting some old UI with grids based on HTML table to ag-Grid. Users appreciate the features added by ag-Grid (such as filters, pinning, sorting, resizing, reordering, grouping...) but noticed one change for worse: not all columns were automatically sized to take the smallest amount of space possible. In enterprise applications data density is really important (not recognizing that is a recipe for failure). Too much white space means that people might miss important information or have to waste time scrolling...

ag-Grid is is capable of handing huge amount of rows and columns thanks to virtualization. The gird creates elements only for rows and columns which are currently visible (plus some buffer) to avoid producing bloated DOM (too many elements severely degrade performance). Virtualization means that the API methods designed to automatically resize columns are only capable of resizing the rendered columns. ag-Grid has suppressColumnVirtualisation option but in my case girds can have as much as 300 columns so turning virtualization off is not possible for performance reasons.

THE SOLUTION

So is it possible to resize all columns while enjoying the benefits of virtualization? If your grid cells contain only text and you know the font then yes, it's possible.

You can calculate the width of text for each grid cell and collect the minimal width needed for each of the gird columns. ag-Grid API might then be used to set desired column sizes. Text width calculation could be done efficiently with measureText method on Canvas.

Here's a module that can collect minimal width needed for each column (it doesn't have any dependency on React):

const canvas = document.createElement('canvas');
const context = canvas.getContext('2d');

const collectMaxWidth = (text, group, maxWidths) => {
    const width = context.measureText(text).width;

    const maxWidth = maxWidths.get(group);

    if (maxWidth === undefined || width > maxWidth) {
        maxWidths.set(group, width);
    }
};

const collectMaxWidthCached = (text, group, maxWidths, widthsCache) => {
    const cachedWidth = widthsCache.get(text);

    let width;

    if (cachedWidth === undefined) {
        width = context.measureText(text).width;
        widthsCache.set(text, width);
    } else {
        width = cachedWidth;
    }

    const maxWidth = maxWidths.get(group);

    if (maxWidth === undefined || width > maxWidth) {
        maxWidths.set(group, width);
    }
};

const calculateColumnWidths = config => {
    console.time('Column widths calculation');

    const maxWidths = new Map()

    if (config.measureHeaders) {
        context.font = config.headerFont;

        config.columnDefs.forEach(column => {
            collectMaxWidth(column.headerName, column.field, maxWidths);
        });
    }

    context.font = config.rowFont;

    config.rowData.forEach(row => {
        config.columnDefs.forEach(column => {
            if (config.cache) {
                collectMaxWidthCached(row[column.field], column.field, maxWidths, config.cache);
            } else {
                collectMaxWidth(row[column.field], column.field, maxWidths);
            }
        });
    });

    const updatedColumnDefs = config.columnDefs.map(cd => ({
        ...cd,
        width: Math.ceil(maxWidths.get(cd.field) + config.padding)
    }));

    console.timeEnd('Column widths calculation');

    return updatedColumnDefs;
};

export default calculateColumnWidths;

The module creates canvas element and then a 2d context is retrieved from it. The context is used in collectMaxWidth function to measure size of provided text by invoking context.measureText(text).width. 

The module also has collectMaxWidthCached function which can offer a performance improvement based on the fact that grid data is often repetitive. If some string was already measured, there's no need to use canvas API again - taking the value from JavaScript Map is super quick. So unless you are extremely worried about memory limits, use the cached version.

The module exports calculateColumnWidths function which takes config object with following properties:

• columnDefs - Array used to specify ag-Grid columns (limit this if you have hidden columns).
• rowData - Array of grid records (limit this if you use paging or filtering).
• measureHeaders - If true then column headers should be taken into account (caveat: header icons are ignored). 
• headerFont - Determines column header font.
• rowFont - Determines normal grid cell font.
• padding - Additional width added to measured text (you need to choose it experimentally, mind varying header icons).
• cache - JS Map used for speed boost (length of each unique non-header text is measured only once), pass null to skip caching.

Below is an example of button's click handler from my demo app what uses calculateColumnWidths:

const handleResizeWithCustomClick = () => {
    console.time('Resize all columns (including widths calculation)');

    if (gridApi && gridColumnApi) {
        // Here ALL columns and rows are used because there are no hidden columns
        // and the the grid has neither paging nor filtering enabled!
        const updatedColumDefs = calculateColumnWidths({
            columnDefs,
            rowData,
            measureHeaders: true,
            headerFont: 'bold 12px Arial',
            rowFont: 'normal 12px Arial',
            padding: 30,
            cache: useWidthsCache ? textWidthsCache : null
        });

        // Setting width by setColumnWidth has the advantage of preserving column
        // changes done by user such as sorting or filters. The disadvantage is that
        // initial resize might be slow if the grid was scrolled towards later columns
        // before resizing was invoked (bug in the gird?).
        // Resize by gridApi.setColumnDefs(updatedColumDefs) or setColumnDefs(updatedColumDefs)
        // should be faster but columns settings could be reset (mind deltaColumnMode)...            
        updatedColumDefs.forEach(def => gridColumnApi.setColumnWidth(def.field, def.width));
    }

    console.timeEnd('Resize all columns (including widths calculation)');
};

Notice how setColumnWidth from ag-Grid Column API is used to apply calculated width. Mind the comments about paging/filtering and the difference between using setColumnWidth and calling Grid API setColumnDefs or updating state bound to grid's columnDefs property...

DEMO APP

Live demo: https://morzel85.github.io/blog-post-ag-grid-full-resize 
Source code on GitHub: https://github.com/morzel85/blog-post-ag-grid-full-resize

The app uses React 16.13.1 and ag-Grid Community 23.0.1 (I've tested it in Chrome 80, Firefox 74, Edge 44).
Clone the repo, do npm install and npm start to run the app locally (just like with any other thing started with create-react-app)...

Usage:
Click on "Generate data (100 rows with 300 columns)" button to create 30K grid cells. You should then click the "Resize columns with columnApi.autoSizeColumns" button and scroll to the right to notice that only visible columns were resized:

Now reload the page, click data generation again and press "Resize columns with custom text measure" and scroll the gird horizontally. You should notice that all columns were resized:

Clicking resizing button the the second time should give you better performance because of text measure caching and because ag-Grid itself has less work to do. These are the timings from Chrome: 

I would say that 100ms for 30K cell measure is quite fast! With caching it drops to 5ms! You may also notice that most of the time is not really spent in my text measuring function but in the ag-Grid which handles columns resize. 

I've noticed one weird performance issue: if you scroll to the right of the grid before clicking the resize button, the time spent by ag-Grid handling columnApi.autoSizeColumns calls increases significantly. Doing bulk resize with gridApi.setColumnDefs or by updating array used in for columnDefs property solves the performance issue at the cost of column settings reset (exact behavior depends on deltaColumnMode)...

Update 2020-13-04:
ag-Grid team confirmed that there's indeed an issue with setColumnWidth performance and the fix should be ready later this month.

TL;DR

Utilize useState or useRef hooks and gridReady event to maintain access to Grid API and Column API. Check here for code samples.

AG-GRID IN REACT 16.8.0+

ag-Grid has an impressive set of features, matched by unusually comprehensive documentation. The issue is that at this moment (February 2020), most of the React-specific examples are written for class components. Because of this, using the grid inside a function (functional) component could be slightly challenging. One of the things that are not so obvious is how to access Grid API and Column API...

Grid and Column APIs have over 150 methods that let you manipulate everything from cell focus to column width. The docs suggest obtaining and saving references to these APIs inside gridReady event handler:

class GridExample extends Component {
  // Irrelevant parts removed for brevity... 

  onGridReady = params => {
    this.gridApi = params.api;
    this.gridColumnApi = params.columnApi;
  }

  render() {
    return (
		<AgGridReact          
            onGridReady={this.onGridReady}            
        />  
    );
  }	
}

Such approach works nicely in a class component but doesn't work when component is "just" a function. Worry not! useState and useRef hooks (part of React 16.8.0) can help you access the APIs in a function component and it's all quite easy! 

I've asked ag-Grid team if they have a preference between useState and useRef and they don't, so I will show you the two options and you can pick the one that suites you.

Here's a component that uses selectAll method from Grid API and moveColumnByIndex method from Column API with the help of useState hook:

import React, { useState } from "react";
import { AgGridReact } from "ag-grid-react";

import { columnDefs } from "./columnDefs";
import { rowData } from "./rowData";

import "ag-grid-community/dist/styles/ag-grid.css";
import "ag-grid-community/dist/styles/ag-theme-balham.css";

const AgGridWithUseState = () => {
  console.log("AgGridWithUseState Render");

  const [gridApi, setGridApi] = useState();
  const [columnApi, setColumnApi] = useState();

  return (
    <div className="example">
      <h1>API access with useState</h1>
      <div>
        <button
          onClick={() => gridApi && gridApi.selectAll()}>
          Grid API selectAll()
        </button>
        <button 
          onClick={() => columnApi && columnApi.moveColumnByIndex(0, 1)}>
          Column API moveColumnByIndex(0, 1)
        </button>
      </div>
      <div className="ag-theme-balham">
        <AgGridReact
          columnDefs={columnDefs}
          rowData={rowData}
          rowSelection="multiple"
          onGridReady={params => {
            console.log("AgGridWithUseState Grid Ready");
            setGridApi(params.api);
            setColumnApi(params.columnApi);
          }}
        />
      </div>
    </div>
  );
};

export default AgGridWithUseState;

and below is the exact same thing done with useRef:

import React, { useRef } from "react";
import { AgGridReact } from "ag-grid-react";

import { columnDefs } from "./columnDefs";
import { rowData } from "./rowData";

import "ag-grid-community/dist/styles/ag-grid.css";
import "ag-grid-community/dist/styles/ag-theme-balham.css";

const AgGridWithUseRef = () => {
  console.log("AgGridWithUseRef Render");

  const gridApi = useRef();
  const columnApi = useRef();

  return (
    <div className="example">
      <h1>API access with useRef</h1>
      <div>
        <button
          onClick={() => gridApi.current && gridApi.current.selectAll()}>
          Grid API selectAll()
        </button>
        <button
          onClick={() => columnApi.current && columnApi.current.moveColumnByIndex(0, 1)}>
          Column API moveColumnByIndex(0, 1)
        </button>
      </div>
      <div className="ag-theme-balham">
        <AgGridReact
          columnDefs={columnDefs}
          rowData={rowData}
          rowSelection="multiple"
          onGridReady={params => {
            console.log("AgGridWithUseRef Grid Ready");
            gridApi.current = params.api;
            columnApi.current = params.columnApi;
          }}
        />
      </div>
    </div>
  );
};

export default AgGridWithUseRef;

Check the function assigned to onGridReady property to find out how to retrieve and keep references to the APIs. Button's onClick handlers show you how to use the APIs.

USEREF VS USESTATE (DEMO)

Live demo: https://morzel85.github.io/blog-post-ag-grid-api-access-with-hooks 
Source code on GitHub: https://github.com/morzel85/blog-post-ag-grid-api-access-with-hooks

The app uses React 16.12.0 and ag-Grid Community 22.1.1 (I've tested it in Chrome 80, Firefox 73, Edge 44).
Clone the repo, do npm install and npm start to run the app locally (just like with any other thing started with create-react-app)...

Demo shows access to selectAll method from Grid API and moveColumnByIndex method from Column API. This application has two grids that differ only in the way the APIs are accessed (one uses useState hook, the other goes with useRef hook).

What's the difference between useState and useRef version?

It boils down to the way hooks are designed to work. useState causes component rerender and useRef doesn't. You can see the difference if you open the console and check the log:

Notice two "AgGridWithUseState Render" lines after "AgGridWithUseState Grid Ready".

This happens because Grid and Column APIs are set by two calls to state hook setters and that causes component rerenders. Sounds bad? Remember that in React, a component rerender doesn't necessary mean any DOM change has to be applied. ag-Grid is also smart enough to avoid unnecessary work. useState triggers rendering by design and it lets you write the component in more declarative way. For example you could keep rowData and columnDefs in state and have the grid update itself without explicit API calls...

How about the useRef approach? Use it if you don't need to run any action as a result of API reference being set.

It's October 2018 so I should probably write something about React 16.5, Angular 7.0 or Blazor 0.6... But... How about some fun with iframe-infested legacy application instead? ;)

TL;DR

You can pass a message to embedded iframe with:

someIframe.contentWindow.postMessage({ a: 'aaa', b: 'bbb' }, '*');

and to parent of an iframe like this:

window.parent.postMessage({ a: 'aaa', b: 'bbb' }, '*');

This is how you can receive the message:

window.addEventListener('message', function (e) {
    // Do something with e.data...         
});  

Mind the security! See my live example and its code on GitHub or go for framebus library if you want some more features.

THE IFRAMES

Unless you work for a startup, chances are that part of your duties is to keep some internal web application alive and this app remembers the glorious era of IE 6. It’s called work for a reason, right? ;) In the old days iframes were used a lot. Not only for embedding content from other sites, cross domain ajax or hacking an overlay that covered selects but also to provide boundaries between page zones or mimic desktop-like windows layout…

So let’s assume that you have site with nested iframes where you need to modify state of one iframe based on action that happened in another iframe:

In the example above, top iframe 0 has a text field and if Update User Name button is clicked we should modify User Name labels in nested iframe 1a and iframe 1b. When Update Account Number button is pressed Account Number in deeply nested iframe 2a should change. Clicking on Update News should modify text in iframe 2b. That last iframe contains a Clear News button, and when it's clicked a notification should be passed to top iframe 0...

DIRECT ACCESS (THE BAD WAY)

One way of implementing interactions between iframes is through direct access to nested/parent iframe's DOM elements (if same-origin policy allows). State of element in nested iframe can modified be such code:

document.getElementById('someIframe').contentWindow.document.getElementById('someInput').value = 'test';

and reaching element in parent can be done with:

window.parent.document.getElementById('someInput').value = 'test';

The problem with this approach is that it tightly couples iframes and that’s unfortunate since the iframes were likely used to provide some sort of encapsulation. Direct DOM access has another flaw: it gets really nasty in case of deep nesting: window.parent.parent.parent.document...

MESSAGING (THE GOOD WAY)

Window.postMessage metod was introduced into browsers to enable safe cross-origin communication between Window objects. The method can be used to pass data between iframes. In this post I’m assuming that the application with iframes is old but it can be run in Internet Explorer 11, which is the last version that Microsoft released (in 2013). From what I’ve seen it’s often the case that IE has to be supported but at least it’s the latest version of it. Sorry if that assumption doesn’t work for you, I’ve suffered my share of old IE support... 

Thanks to postMessage method it’s very easy to create a mini message bus so events triggered in one iframe can be handled in another if the target iframe chooses to take an action. Such approach reduces coupling between iframes as one frame doesn't need to know any details about elements of the other...

Take a look at an example function that can send a messages down to all directly nested iframes:

const sendMessage = function (type, value) {
     console.log('[iframe0] Sending message down, type: ' + type + ', value: ' + value);

     var iframes = document.getElementsByTagName('iframe');
     for (var i = 0; i < iframes.length; i++) {
         iframes[i].contentWindow.postMessage({ direction: 'DOWN', type: type, value: value }, '*');
     }
};

In the code above, iframes are found with document.getElementsByTagName and then a message is sent to each of them through contentWindow.postMessage call. First parameter of postMessage method is the message (data) we want to pass. Browser will take care of its serialization and its up to you to decide what needs to be passed. I've chosen to pass an object with 3 properties: first designate in which direction message should go (UP or DOWN), second states the message type (UPDATE_USER for example) and the last one contains the payload of the message. In the case of our sample app it will be a text user put into input and which should affect elements in nested iframes. The '*' value passed to contentWindow method determines how browser dispatches the event. Asterisk means no restrictions - it's ok for our code sample but in real world you should consider providing an URI as the parameter value so browser will be able to restrict the event based on scheme, host name and port number. This is a must in case you need to pass sensitive data (you don't want to show it to any site that got loaded into iframe)!

This is how sendMessage function can be used to notify nested iframes about the need to update user info:

document.getElementById('updateUserName').addEventListener('click', function (event) {
    sendMessage('UPDATE_USER', document.getElementById('textToSend').value);
});

Code shown above belongs to iframe 0 which contains two nested iframes: 1a and 1b. Below is the code from iframe 1b which can do two things: handle a message in case it is interested in it or just pass it UP or DOWN:

window.addEventListener('message', function (e) {
    console.log('[iframe1b] Message received');

    if (e.data.type === 'UPDATE_USER') {
        console.log('[iframe1b] Handling message - updating user name to: ' + e.data.value);
        document.getElementById('userName').innerText = e.data.value;
    } else {
        if (e.data.direction === 'UP') {
            console.log('[iframe1b] Passing message up');
            window.parent.postMessage(e.data, '*');
        } else {
            console.log('[iframe1b] Passing message down');
            document.getElementById('iframe2b').contentWindow.postMessage(e.data, '*');
        }
    }               
});

You can see that messages can be captured by listening to message event on window object. Passed message is available in event's data field, hence the check for e.data.type is done to see if code should handle the message or just pass it. Passing UP is done with window.parent.postMessage, passing DOWN works with contentWindow.postMessage called on an iframe element. 

iframe 2b has a button with following click handler:

document.getElementById('clearNews').addEventListener('click', function () {
   document.getElementById('news').innerText = '';

   console.log('[iframe2b] News cleared, sending message up, type: NEWS_CLEARED');
   window.parent.postMessage({ direction: 'UP', type: 'NEWS_CLEARED' }, '*');
);

It clears news text and sends notification to parent window (iframe). This message will be received by iframe 1b and passed up to iframe 0 which will handle it by displaying 'News cleared' text: 

window.addEventListener('message', function (e) {
    console.log('[iframe0] Message received');

    if (e.data.type === 'NEWS_CLEARED') {
        console.log('[iframe0] Handling message - notifying about news clear');
        document.getElementById('newsClearedNotice').innerText = 'News cleared!';
    }
});

Notice that this time message handler is quite simple. This is because in the case of top iframe 0 we don't want to pass received messages. 

EXAMPLE

That's it. Here's a working sample of iframe "rich" page. Open browser console to see how messages fly around. Check the repo to see the code, it's vanilla JS with no fancy features since we assumed that IE 11 has to be directly supported (checked also in Firefox 62, Chrome 69 and Edge 42).

Part 1 described the general idea behind Sonar project, hardware components used and Arduino sketch... This second post in "Out of Boredom" series is about C# and JavaScript programs that make it possible to display ultrasonic range sensor data in web browsers. The role of .NET application is to receive messages from Arduino over serial port and broadcast it to clients using SignalR library. JS/HTML5 clients use jquery.signalR lib to obtain information about servo position with distance to obstacles and use this data to render sonar image on canvas:

These links are in previous post, but just to remind you:

• This GitHub repository contains all the code of Sonar project.
• This short video shows working Sonar.

1. SonarServer 

SonarServer is a .NET 4.5 console app created in Visual Studio Express 2013 for Windows Desktop. It uses Microsoft.AspNet.SignalR.SelfHost and Microsoft.Owin.Cors NuGet packages to create self-hosted SignalR server. ASP.NET SignalR is a library designed to make it easy to create applications that are able to push data to clients running in web browsers. This is in contrast to normal web pages/apps behavior where the client (browser) asks server for action by issuing a request (such as GET or POST). SignalR allows clients to listen for messages send by a server... If possible SignalR will use WebSockets to enable efficient bi-directorial connection. If that option is not available due to either browser or server limitations, it will automatically switch to other push techniques like long polling or Server-Sent Events. When I tested the code on my laptop with Windows 7 Home Premium SP1, long polling was used on IE 11 and SSE in Chrome 37. Server was sending about 20 messages per second and clients didn't have any problems with handling that load (communication was on localhost). Self-hosting means that SignalR server doesn't have to be run on a web server such as IIS - it can exist in plain old console project! If you are completely new to SignalR check this tutorial... 

This is SonarServer project structure:

SonarData.cs file contains such struct:

namespace SonarServer
{
    public struct SonarData
    {
        public byte Angle { get; set; }
        public byte Distance { get; set; }
    }
}

Server will send a list of such objects to clients.

SonarHub.cs contains a class derived form Hub. It doesn't declare any methods but is nonetheless useful. Library will use it to generate JavaScript proxy objects...

using Microsoft.AspNet.SignalR;

namespace SonarServer
{
    public class SonarHub : Hub
    {

    }
}

Startup.cs file looks like this:

using Microsoft.Owin.Cors;
using Owin;

namespace SonarServer
{
    class Startup
    {
        public void Configuration(IAppBuilder app)
        {
            app.UseCors(CorsOptions.AllowAll);
            app.MapSignalR();
        }
    }
}

It configures the SignalR server, letting it support cross-domain connection thanks to UseCors call.

Below are the most important bits of Program.cs file. This code:

using (SerialPort sp = new SerialPort())
 {
     sp.PortName = "COM3";
     sp.ReceivedBytesThreshold = 3;
     sp.DataReceived += new SerialDataReceivedEventHandler(DataReceivedHandler);

     sp.Open();

     Console.WriteLine("Serial port opened!");

     using (WebApp.Start<Startup>("http://localhost:8080/"))
     {
         Console.WriteLine("Server running!");
         Console.ReadKey();
     }
 }

is responsible for opening a connection with Arduino over serial port named "COM3" and starting SignalR server on localhost:8080. ReceivedBytesThreshold property allows us to control the amount of bytes received from Arduino before DataReceivedHandler is called. You can increase this value if you want bigger packages of data to be broadcasted by server and rendered by clients. This is the part of DataReceivedHandler method that loads serial port data into a byte array:

int count = sp.BytesToRead;
[] data = new byte[count];
sp.Read(data, 0, count);

Such array of bytes is latter on added to custom buffer and processed to create a list of SonarData objects sent to SignalR clients. Part 1 mentioned that Arduino sends data to PC in bytes (array) packages containing three elements: [255, angle, distance]. The purpose of special 255 value is to separate angle-distance pairs of values which are used to create sonar image. We can't just send [angle, distance] stream from Arduino to PC, because the Server could easily loose track of which value is angle and which is distance. This might happen due to delays, buffering etc. Sure it's not a bulletproof protocol but it work well when I tested it. Lot's not get crazy with that - it's a hobby project, remember? :) Check ProcessSonarData method in repository if you want to see how an array of bytes is turned into SonarData list (with buffering taken into account)... 

The last missing piece of SonarServer puzzle is SendSonarDataToClients method:

private static void SendSonarDataToClients(List<SonarData> sonarDataForClients)
{
    var hub = GlobalHost.ConnectionManager.GetHubContext<SonarHub>();
    hub.Clients.All.sonarData(sonarDataForClients);

    Console.WriteLine("Sonar data items sent to clients. Samples count=" + sonarDataForClients.Count);
}

This is the thing that actually broadcasts data to clients running in web browsers. You may be wondering why SonarHub instance is not created directly with new operator and instead GetHubContext method is used. This is because SignalR is responsible for its hubs life cycle. Such code:

SonarHub sonarHub = new SonarHub();
sonarHub.Clients.All.sonarData(sonarDataForClients);

would result in the exception: System.InvalidOperationException: Using a Hub instance not created by the HubPipeline is unsupported." .

2. SonarClient

SonarClient is the subproject responsible for drawing sonar image. It's not a Visual Studio solution - just a few files:

I've tested SonarClient code in IE 11 and Chrome 37 and it worked really well. The assumption is that you run modern browser too. I didn't bother with any feature detection, it's enough for me that I have to write for IE9 at work - well, at least it's not IE 6, huh? ;) But if you want to do such thing I can recommend Modernizr library...

This is content of index.html file (with some boring parts removed for brevity):

<!DOCTYPE html>
<html>
<head>
    <title>Sonar - sample code from morzel.net blog post</title>
    <style>
        /* more */
    </style>
</head>
<body>
    <div>
        <canvas id="sonarImage" width="410" height="210"></canvas>

        <table>
              <!-- more -->            
        </table>
    </div>

    <a href="http://morzel.net" target="_blank">morzel.net</a>

    <script src="lib/jquery-1.6.4.js"></script>
    <script src="lib/jquery.signalR-2.1.1.js"></script>
    <script src="http://localhost:8080/signalr/hubs"></script>
    <script src="sonarStats.js"></script>
    <script src="sonarImage.js"></script>
    <script src="sonarConnection.js"></script>

    <script>
        $(function () {          
            sonarImage.init('sonarImage');
            sonarConnection.init('http://localhost:8080/signalr');
        });
    </script>
</body>
</html>

Most important bit of this HTML5 markup is the canvas element used to create sonar image. The page imports jquery and jquery.signalR libraries that make it possible to communicate with the Server. That line is particularly interesting:

 <script src="http://localhost:8080/signalr/hubs"></script>

SingalR automatically creates JavaScript proxy objects for server-client messaging, that line lets us load them into page. Last three script references are for JS modules responsible for: displaying info about data received from SonarServer, rendering sonar image and communication with server, respectively. Later there's a short script which initializes the modules after pages DOM is ready.

I will skip the description of sonarStats.js file (nothing fancy there - just filling some table cells). But sonarConnection.js should be interesting for you. This is the whole content:

var sonarConnection = (function () {
    'use strict';

    var sonarHub, startTime, numberOfMessages, numberOfSamples;

    var processSonarData = function (sonarData) {
        numberOfMessages++;
        $.each(sonarData, function (index, item) {
            numberOfSamples++;
            sonarImage.draw(item.Angle, item.Distance);
            sonarStats.fillTable(item.Angle, item.Distance, startTime, numberOfMessages, numberOfSamples);
        });
    };

    return {
        init: function (url) {
            $.connection.hub.url = url;

            sonarHub = $.connection.sonarHub;

            if (sonarHub) {
                sonarHub.client.sonarData = processSonarData;
                               
                startTime = new Date();
                numberOfMessages = 0;
                numberOfSamples = 0;

                $.connection.hub.start();
            } else {
                alert('Sonar hub not found! Are you sure the server is working and URL is set correctly?');
            }
        }
    };
}());

The init method sets SignalR hub URL along with sonarData handler and starts a connection with the .NET app. There is also some very basic hub availability check (jquery.signalR library has extensive support for connection related events but lets keep things simple here). processSonarData is invoked in response to Server calling hub.Clients.All.sonarData(sonarDataForClients). The processSonarData function receives an array of objects containing information about servo angle and distance to obstacles. SignalR takes care of proper serialization/deserialization of data - you don't have to play with JSON yourself. $.each function (part of jQuery) is used to invoke sonarImage.draw and sonarStats.fillTable methods for every item in sonarData array...

And here comes the module that changes pairs of angle-distance values into o nice sonar image (whole code of sonarImage.js):

var sonarImage = (function () {
    'use strict';

    var maxDistance = 100;
    var canvas, context;

    var fadeSonarLines = function () {
        var imageData = context.getImageData(0, 0, canvas.width, canvas.height),
            pixels = imageData.data,
            fadeStep = 1,
            green,
            fadedGreen;

        for (var i = 0; i < pixels.length; i += 4) {
            green = pixels[i + 1];

            fadedGreen = green - fadeStep;
            pixels[i + 1] = fadedGreen;
        }

        context.putImageData(imageData, 0, 0);
    };

    return {
        init: function (canvasId) {
            canvas = document.getElementById(canvasId);
            context = canvas.getContext('2d');

            context.lineWidth = 2;
            context.strokeStyle = '#00FF00';
            context.fillStyle = "#000000";

            context.fillRect(0, 0, canvas.width, canvas.height);

            context.translate(canvas.width / 2, 0);
            context.scale(2, 2);
        },

        draw: function (angle, distance) {
            context.save();

            context.rotate((90 - angle) * Math.PI / 180);
            context.beginPath();
            context.moveTo(0, 0);
            context.lineTo(0, distance || maxDistance); // Treat 0 as above range
            context.stroke();

            context.restore();

            fadeSonarLines();
        }
    };
}());

Again we have an init method. It obtains 2d drawing context from canvas and uses it to set line width and color (green). It also sets inner color (black) and fills the whole canvas with it. context.translate call is used to move origin of coordinate system to the middle of canvas (horizontally). By default it sits in upper left corner. context.scale is used to make image two times bigger than it would be drawn in default settings. Read this post if you want to know more about canvas coordinates.

The draw method is invoked for each data sample (angle-distance pair) produced by Arduino and broadcasted with SonarServer. The distance to obstacles measured by HC-SR04 sensor is represented as a line. The bigger the distance the longer the line. This happens thanks to beginPath, moveTo, lineTo, and stroke calls. context.rotate method is responsible for showing the angle in which the sensor was pointing while measuring distance (angle of servo arm). As the servo moves around we want to change the direction in which distance line is drawn. Notice that code responsible for drawing the line is surrounded by context.save and context.restore calls. These two ensure that rotation transformation doesn't accumulate between draw method calls...

fadeSonarLines function is responsible for creating the effect of older sonar lines disappearing in a nice gradual way. context.getImageData method returns an array of RGBA values representing the pixels that create current canvas image. The function loops through image data and progressively reduces the intensity of green color component. This way sonar lines fade to black.

And viola - sonar image can be rendered in a browser :)

Isn't it amazing what is now possible on the web platform? I'm not exactly a dinosaur but I remember when displaying a div overlay on a page required use of hidden iframe (because select elements were otherwise rendered above the overlay)... Crazy times ;)

This post marks the beginning of "Out of Boredom" series. It will be about creating stuff with my recently purchased Arduino Uno. Let's have a break from chores of professional programming and create something just for fun :)

My first Arduino based project is Sonar. It utilizes ultrasonic range sensor, servo, SignalR and canvas to create sonar image:

I am splitting the description into two posts. First post will focus on hardware components and Arduino sketch and the second will be about .NET and JavaScript applications. You can get complete code in this GitHub repository. You can also click here to see short video of the whole thing working.

Here are hardware elements used:

That's it, just a few cheap components! Virtually no electronics skills are required to complete this project. I assume, however, that you have basic Arduino knowledge and you know a bit about C# and JavaScript.

Here is the software stack:

Above might sound a bit overwhelming but I assure you that the code is short and not that complicated.

The basic idea goes like this: HC-SR04 sensor measures time it takes an ultrasonic signal to bounce from obstacles and this gives as a chance to calculate distance to these obstacles. Position of the sensor is controlled by servo. Information about distance to objects and direction in which the sensor is pointing is sent to PC that is running console application with SignalR sever. PC receives the data and sends it to JavaScript clients that are capable of presenting sonar data in nice visual way using HTML5 canvas element...

More details!

The main component (except for the Arduino of course) is the HC-SR04 Ultrasonic Ranging Module. This sensor works by sending sound signal at 40 kHz (so above human perception limits) and detecting the echo. That's why this project is called "Sonar" and not "Radar" - it uses sound waves (not radio waves) do detect objects. The sensor should work in ranges from 2cm up to 400cm at accuracy of few millimetres. But keep in mind that the shape an material of objects might affect performance. I tested it at maximum distance of about 2 meters and was happy with the results. You can use this sensor without any libraries. That requires doing things like putting HIGH value on Trig pin for 10uS to emit ultrasonic signal, measuring duration of HIGH pulse on Echo pin and calculating distance knowing that speed of sound in the air is around 340m/s... But there's a better way: you can use NewPing lib (link) to get the distance. If you don't know how to include new library in your Arduino sketch click here.

The second important component is the servo. HC-SR04 sensor has measuring angle of about 15 degrees. But if we move it around by attaching it to servo's arm we can easily get 180 degree view. I won't get into details on how servo works and how it is controlled in this post. I plan to make another post about shooting paintball marker with Arduino+laptop and I will describe it then. For now all you need to know is that Arduino comes with Servo library which makes it very easy to move servo into desired position (angle)... I utilized 9g Tower Pro Micro Servo in this project. It's powerful enough to move the sensor yet can be powered directly from Arduino's +5V pin.

Last physical components are LED used to signal the ready state (that is when setup function had completed) with its accompanying resistor. Making a diode shine is electronics equivalent of "Hello World!" so I'm sure you know how to handle LED. Even if not, you can always use the tiny built-in LED connected to pin 13 of Arduino Uno...

This diagram shows how hardware components should be connected:

Here's the whole code that should be uploaded to Arduino:  

#include <NewPing.h>
#include <Servo.h>  

const byte setupReadyLedPin = 8;
const byte triggerPin = 10;
const byte echoPin = 11;
const byte servoPin = 12;

const byte maxDistanceInCm = 100;

byte angle;
byte angleStep;
byte angleStepDelayInMs = 50;

NewPing sonar(triggerPin, echoPin, maxDistanceInCm); 
Servo servo; 

void setup() {  
    pinMode(setupReadyLedPin, OUTPUT);
    
    angle = 0;
    angleStep = 1;
    
    servo.attach(servoPin);   
    servo.write(angle); 
    
    Serial.begin(9600); // Open connection with PC
    
    digitalWrite(setupReadyLedPin, HIGH);
}

void loop() {      
    alterServoMoveDirection();    
    
    measureAndSendDistance();
    
    angle += angleStep;
    servo.write(angle); // Move servo
    
    delay(angleStepDelayInMs);   
}

void alterServoMoveDirection() {
    if (angle == 180) {
       angleStep = -1; 
    } else if (angle == 0) {
       angleStep = 1;
    }
}

void measureAndSendDistance() {
    byte distanceInCm = sonar.ping_cm(); // Use ultrasound to measure distance   
      
    byte sonarData[] = {255, angle, distanceInCm};
    Serial.write(sonarData, 3); // Send data to PC
}

As stated before, I assume that you know something about Arduino programming and things like const, pinMode, delay, setup and loop don't require explanation...

First lines which should capture your attention are:

#include <NewPing.h>
#include <Servo.h>  

NewPing sonar(triggerPin, echoPin, maxDistanceInCm); 
Servo servo;

Above lines let us use NewPing and Servo classes to measure distance and move the sensor. Notice also that setup function has such lines:

servo.attach(servoPin);   
servo.write(angle);

These exist to set the pin used to control the servo and to move the servo into initial position at 0 degrees.

This line:

Serial.begin(9600);

allowes Arduino to talk to PC (in my case a laptop with Windows 7) over serial port. That's right, even though Arduino Uno is connected to computer via USB cable it actually uses COM port to communicate. On my machine its called "COM3" (screen shown below comes from Device Manager - my Windows is in Polish):

The value passed to begin method determinates baud rate (communication speed). It's important to have the same value used in software that communicats with Arduino.

The loop function moves servo and invokes measureAndSendDistance function which uses NewPing to calculate distance and Serial to send data to PC. This is how easy it is to get distance in cm thanks to NewPing lib:

byte distanceInCm = sonar.ping_cm()

If measured distance exceeds the maximum value specified as last parameter to NewPing constructor the value of 0 is returned. Check NewPing docs to see other useful functions of this lib.

And finally this is how Arduino sends data to PC:

byte sonarData[] = {255, angle, distanceInCm};
Serial.write(sonarData, 3); // Send data to PC

The first array element (255) is used as a marker/separator to easily distinguish pairs of angle-distance values. Its role will become clear in the second post which will describe SignalR server and clients... I assume that maxDistanceInCm const will never be set above 200 so distanceInCm will never have value of 255. 255 will never be sent as an angle too because our servo moves in 0..180 degrees range. Sure it might be a good idea to create const and avoid 255 magic number. Some validation would be useful too... But screw it, this project is just for fun! :)

Ok, you survived to the end of the first post about Arduino/.NET/JS/HTML sonar. The second post should be ready in about a week.

Update 2014-09-29: Here's the Part 2.