Miłosz Orzeł

.net, js, html, arduino, java... no rants or clickbaits.

BadImageFormatException, x86 i x64

Have you ever seen BadImageFormatException or “An attempt was made to load a program with an incorrect format” message?

If so, maybe the program that you tried to run hasn’t been compiled with /platform:x86 option. You are probably wondering why, while writing in C# you should think about what kind of machine (x86 or x64) your program will execute on. Well… in most cases you don’t have to care about it. If your application doesn’t contain any unsafe code and doesn’t import any native modules that are destined for specific CPU architecture, then you can forget about 32/64 bit dilemma. After all C# code gets compiled into intermediate language (CIL) and upon execution framework’s just-in-time compiler will emit native instructions suitable for current platform. Great, but there’s a catch…

Imagine a situation when you import 32 bit DLL in your application. If you run that software on 32 bit operating system everything works as expected. Unfortunately on x64 machine you can get BadImageFormatException while trying to load DLL. Why? Your assembly may be compiled with /platform:anycpu setting. It means that on 32 bit systems your code will be executed with 32 bit CLR version. On 64 bit systems a 64 bit process will be used – and it will cause problems with loading 32 bit DLL. If you provide /platform:x86 setting, then 32 bit version of the CLR will be choosen even on the x64 versions of Windows (utilizing WoW64: Windows 32-bit on Windows 64-bit).

In Visual Studio 2010 you can set targeted platform in project “Properties” window (on “Build” tab). To get there, right click on project file in Solution Explorer and choose “Properties” or use main menu “Project | <you project name> Properties…”.

Setting targeted platform in Visual Studio (click to enlarge)

Microsoft has created useful tool called CorFlags which can be used to show or set the targeted platform of an managed assembly. You can access this program by using Visual Studio Command Prompt or find it directly on your disk (I have it under C:\Program Files\Microsoft.NET\SDK\v2.0\Bin\CorFlags.exe).

Here are some examples of what you may see after checking an EXE file created with various /platform option values (use CorFlags file.name command to check a file):

anycpu x86 x64
Version   : v4.0.30319
CLR Header: 2.5
PE        : PE32
CorFlags  : 1
ILONLY    : 1
32BIT     : 0
Signed    : 0
Version   : v4.0.30319
CLR Header: 2.5
PE        : PE32
CorFlags  : 3
ILONLY    : 1
32BIT     : 1
Signed    : 0
Version   : v4.0.30319
CLR Header: 2.5
PE        : PE32+
CorFlags  : 1
ILONLY    : 1
32BIT     : 0
Signed    : 0

For the scope of this post 2 rows of the CorFlags output are important: PE and 32BIT.

  • PE: PE32 means that file can be executed on both x86 and x64
  • PE: PE32+ means that it can only be run on 64 bit version of OS
  • 32BIT: 1 means that program must be executed on x86 environment.

Understanding 32BIT: 1 meaning is really important if you want to avoid problems with importing 32 bit DLL on 64 bit systems. If 32BIT flag is set and you run PE32 file on x64, then your app will be executed in 32 bit process (under WoW), hence will have a chance to properly import 32 bit DLL. Without this setting 64 bit environment will be chosen which will cause problems.

The good news is that you can easily modify 32BIT flag with CorFlags tool, to do this execute that command:

CorFlags file.exe /32BIT+

And to remove it, try this

CorFlags file.exe /32BIT-

So even if you cannot recompile problematic assembly with proper /platform option you still have a chance of using legacy 32 bit DLLs on 64 bit Windows :)

Why the use of GetPixel and SetPixel is so inefficient!

Bitmap class provides two simple methods: GetPixel and SetPixel used respectively to retrieve a point of image (as the Color structure) and set a point of image. The following code illustrates how to retrieve/set all the pixels in the bitmap:

private void GetSetPixel(Bitmap image) {
   for (int x = 0; x < image.Width; x++) {
      for (int y = 0; y < image.Height; y++) {
         Color pixel = image.GetPixel(x, y);
         image.SetPixel(x, y, Color.Black);
      }
   } 
}

As shown, review and modification of pixels is extremely simple. Unfortunately behind the simplicity of the code lies a serious performance trap. While for a small number of references to image points, the speed at which GetPixel and SetPixel work is good enough, for larger images it is not the case. Graph presented below can serve as a proof of that. It shows results of 10 tests* which consisted of 10-fold invocation of previously shown GetSetPixel method for images 100x100 and 1000x1000 pixels in size.

Wyniki testów prędkości operacji na pikselach obrazu z użyciem metod GetPixel i SetPixel klasy Bitmap.

The average test time for an image measuring 100 by 100 pixels was 543 milliseconds. This speed is acceptable if the image processing is not done frequently. Performance problem is, however, clearly visible when you try to use an image of size 1000 per 1000 pixels. Execution of the test in this case takes an average of more than 41 seconds - more than 4 sec. on a single call to GetSetPixel (seriously!).

Why so slow?

Low efficiency is due to the fact that access to the pixel is not a simple reference to a memory area. Each getting or setting of color is associated with invocation of .NET Framework method, which is a wrapper for native function contained in gdiplus.dll. This call is through the mechanism of P/Invoke (Platform Invocation), which is used to communicate from managed code to unmanaged API (API outside of the .NET Framework). So for a bitmap of 1000x1000 pixels there will be 1 million calls to GetPixel method that besides the validation of parameters uses the native GdipBitmapGetPixel function. Before returning color information, GDI+ function has to perform such operations as calculating the position of bytes responsible for description of desired pixel… Similar situation occurs in the case SetPixel method.

Look at the following code of Bitmap.GetPixel method obtained with the .NET Reflector (System.Drawing.dll, .NET Framework 2.0):

public Color GetPixel(int x, int y) {
   int argb = 0;
   if ((x < 0) || (x >= base.Width)) {
      throw new ArgumentOutOfRangeException(“x”, SR.GetString(“ValidRangeX”));
   }
   if ((y < 0) || (y >= base.Height)) {
      throw new ArgumentOutOfRangeException(“y”, SR.GetString(“ValidRangeY”));
   }
   
   int status = SafeNativeMethods.Gdip.GdipBitmapGetPixel(new HandleRef(this, base.nativeImage), x, y, out argb);
   if (status != 0) {
      throw SafeNativeMethods.Gdip.StatusException(status);
   }
   return Color.FromArgb(argb);
}

Here is import of GDI + function:

[DllImport(“gdiplus.dll”, CharSet=CharSet.Unicode, SetLastError=true, 
ExactSpelling=true)]
internal static extern int GdipBitmapGetPixel(HandleRef bitmap, int x, int y, out int argb);

Update 2013-07-10: Unfortunately I haven't found time to write an article about a solution to this performance problem but there are some useful hints in my comment.

Update 2013-11-07: I've written an article (...finally) about fast pixel operations. No need to use crappy Get/SetPixel anymore :) Click here.

Update 2018-01-08: If you really want to do some complex and efficient image processing then you should use specialized library like OpenCV. Few months ago I've written "Detecting a Drone - OpenCV in .NET for Beginners (Emgu CV 3.2, Visual Studio 2017)" blog post series that will help you do it...

* I have tested on such laptop: HP Pavilion dv5, AMD Turion X2 Dual-Core Mobile RM-70, 3 GB RAM, Vista Home Premium

IBM.WMQ.MQMessage.ReadString and EndOfStreamException

Recently during modification of a program to communicate with WebSphere MQ (v6.0.2.7) I noticed that logs contain some exceptions of type EndOfStreamException. Since the adapter code was rather complex it took a while before I found a trivial cause of the problems ;)

System.IO.EndOfStreamException: Nie można odczytać danych spoza końca
strumienia.
   w System.IO.__Error.EndOfFile()
   w System.IO.BinaryReader.ReadByte()
   w System.IO.BinaryReader.Read7BitEncodedInt()
   w System.IO.BinaryReader.ReadString()
   w IBM.WMQ.MQMessage.ReadString(Int32 length)

The error was reported, because sometimes in two different locations there was a call to ReadString method on the same MQMessage object:

string text = message.ReadString(message.MessageLength);

To get rid of the trouble simply add one line of code:

string text = message.ReadString(message.MessageLength);
message.Seek(0);

What's the problem?

ReadString is a method that scans stream of bytes and converts it to a string*. After successful reading of entire message content, marker of current position was left on the end of a stream. So next call to ReadString had to end up with EndOfStreamException exception. Why it had to happen? Internally ReadString (IBM.WMQ.MQMessage) uses data stored in MemoryStream object. While getting text, depending on current value of message’s DataLength property, there may be a  call to .NET Framework’s System.IO.BinaryReader.ReadString method. To read the text, it must first get the encoded length of that text  - this is why method Read7BitEncodedInt is visible on the stack trace. This routine in turn uses ReadByte method which after hitting the end of a stream throws discussed exception.

* Conversion takes place using message's CharacterSet (CCSID) property.

Why strings are immutable and what are the implications of it?

String type (System.String) stores text values as a sequence of char (System.Char) elements that represent Unicode characters (encoded in UTF-16). Usually one char element stands for one symbol.

When working with text one has to remember that strings in .NET are immutable! This simply means that once created, strings cannot be modified (without reflection or unsafe code), and the methods that apparently modify a string, really return a new object with the desired value.

Immutability of strings has many advantages (more about it soon), but it can cause problems if programmer forgets that any "change" to the string actually causes creation of a new instance of String class. Although the CLR treats strings in a special way, they are still a reference type, for which the memory is allocated on the managed heap.

Operation of this loop will create 10 000 string variables, all of which except the last are trash that will need to be collected by Garbage Collector:

string s = string.Empty;

for (int i = 0; i < 10000; i++)
{
    s += "x";
}

The following picture shows part of the "Histogram by Size for Allocated Objects" window from CLR Profiler application. You can see how subsequent iterations caused heap allocations for ever larger strings.

To avoid creating many unwanted objects use StringBuilder class, which allows you to modify the text without making new String class instances.

StringBuilder sb = new StringBuilder();

for (int i = 0; i < 10000; i++)
{
    sb.Append("x");
}

string x = sb.ToString();

This simple change has a huge impact on the amount of allocated and freed memory. See the following comparison of fragments of Profiler’s “Summary” window:

Why do designers of .NET (just like the creators of Java) decided to implement immutable text strings?

For optimization reasons (mainly because of comparison speed), texts can be stored in a special table (intern pool) maintained by the CLR. The idea is to avoid creating a number of variables defining the same string. Below is a piece of code proving that variables that have the same string value can point to the same* object:

string a = "xx";
string b = "xx";
string c = "x";
string d = String.Intern(c + c);

Console.WriteLine((object)a == (object)b); // True
Console.WriteLine((object)a == (object)d); // True

If the strings were modifiable, change to the value of variable a, would also change the value of b and d.

Immutability of strings has positive significance in multithreaded applications – any text amendment causes creation of a new variable so there is no need to set up the lock to avoid conflicts while multiple threads simultaneously access text. This is really important because quite often authorization of some operations is based on particular string value (for example, you can block specific functionality of a service based on client’s address).

Another important reason for immutability is the widespread use of strings as keys in hash tables. Would calculation of position of the element in table make any sense if it would be possible to modify the value of a key? The following listing shows that the "change" to variable used as the key does not affect the operation of the hash table:

string key = "abc";
Hashtable ht = new Hashtable();
ht.Add(key, 123);

key = "xbc";

Console.WriteLine(key); // xbc
Console.WriteLine(ht["abc"]); // 123

What would happen in the case of modifiable strings can be seen through a block of code that uses unsafe mode to actually change the string used as key:

unsafe
{
    string key = "abc";
    Hashtable ht = new Hashtable();
    ht.Add(key, 123);

    fixed (char* p = key)
    {
        p[0] = 'x';
    }

    Console.WriteLine(key); // xbc
    Console.WriteLine(ht["abc"]); // Not found!
}

Immutability is also related to the fact that the string is stored internally as an array - the data structure representing a continuous address space (which for performance reasons does not support insert operation).

* Whether a text literal is automatically added to the pool may be dependent on the use of ngen.exe tool or CompilationRelaxations settings...