This entry is part 1 of 17 in the series blImageAPI -- BarbatoLabs Image API

blImageAPI -- BarbatoLabs Image API

Introduction

Back in the good old days of OpenCV 1.1 and earlier, the main image structure used throughout their algorithms was IplImage. Playing with IplImage requires skillful manipulation of data in memory, and thus makes it very difficult for beginners to play.  Even experts run into the dreaded memory leaks caused by an image they forget to release somewhere in their code.

Then of course, you want to access image pixels, and that’s where it gets tough.  OpenCV does provide get and set functions, but the performance hits start to accumulate when building real-time systems.

In this article, with the help of boost::shared_ptr, I design blImage, a simple, very fast, secure, yet scalable image structure, which I will use for all my work.

Here’s some sample code showing the potential of what will be possible with blImage:

int main(int argc, char *argv[])
{
    // Create a blank 8-bit color image
    blImage< blColor3<unsigned char> > MyImage;

    // Load an image from file
    MyImage.LoadImageFromFile("C:\\tux.png");

    // Create a floating point grayscale
    // version of our colored image
    blImage<unsigned char> MyImageFloatGray = MyImage;

    // Create two windows and show
    // both images side by side
    cvNamedWindow("ColorImage",CV_WINDOW_AUTOSIZE);
    cvNamedWindow("GrayscaleImage",CV_WINDOW_AUTOSIZE);
    cvShowImage("ColorImage",MyImage);
    cvShowImage("GrayscaleImage",MyImageFloatGray);

    // wait for a key
    cvWaitKey(0);
    return 0;
}

This is what we get when we run the above snippet

Dependencies

  1. boost::shared_ptr from the boost c++ libraries
    1. boost::weak_ptr could be used instead if you like
  2. IplImage from the OpenCV library (obviously)
  3. CvScalar from the OpenCV library (for loading images into blImage)

Design Philosophy

Accessibility is the mother of all catalysts when it comes to invention. The more people have access to more tools, the faster we move forward in a never ending journey of advancing technology. But, as the world would have it, easy access to technology can lead to a chaotic soup of unsustainable and unstable systems.

Thus, our image structure abides by the following principles:

  1. Simplicity
    1. Easy to read/understand code (a kid should be able to use it)
    2. Follows a well documented layout
  2. Scalability
    1. Easy to derive from to add functionality
    2. Could be used for small or large programs
    3. Could be used as a generalized Matrix structure
  3. Performance
    1. Incur as little cost as possible compared to a raw IplImage* pointer
  4. Robustness
    1. Account for possible errors and provide feedback

Image Depth

Before we build our image structure, we have to account for the different depths that we could need. For example, we might be playing with an 8-bit image in one algorithm, and a 32-bit image in another algorithm, or be mixing and matching images of different depths.

In this design, we account for the following depths:

  1. “bool” — (1-bit)
  2. “unsigned char” and “char” — (8-bits unsigned and signed)
  3. “unsigned short” and “short” — (16-bits unsigned and signed)
  4. “int” — (32-bits signed, we Only account for the 32-bit signed)
  5. “float” — (32-bits floating point)
  6. “double” — (64-bits floating point)

Note that for each depth, we have a range for the possible pixel values:

  1. 1-bit — [0,1]
  2. 8-bit unsigned — [0,255]
  3. 8-bit signed — [-128,127]
  4. 16-bit unsigned — [0,65535]
  5. 16-bit signed — [-32768,32767]
  6. 32-bit signed — [-2147483648,2147483647]
  7. 32-bit float — [0,1]
  8. 64-bit double — [0,1]

Image Channels

Of course, we can’t have an image without having color. We represent color by introducing the idea of channels. OpenCV uses a BGR sequence as opposed to the more common RGB to store values. Also, since our image structure can be used as a generalized matrix, we need to account for several cases:

  1. One Channel — One dimensional color
    1. Grayscale images
    2. Single-valued matrices (Linear Algebra)
  2. Two Channels –Two dimensional color
    1. For special type images
    2. Complex valued matrices (Complex Algebra)
  3. Three Channels — Three dimensional color
    1. Colored images (BGR or HSV images)
  4. Four dimensional color
    1. Colored images with alpha transparency values (ex. BGRA)

Representation of Color

For One-Channel images we’re going to use c++ basic data types corresponding to the depth of the images.

For Two-Channel images we’re going to use std::complex<T> with T corresponding to the depth of the images.

For Three-Channel images, we’re going to build a basic three-color structure, remembering that IplImage uses a BGR sequence instead of the widespread RGB sequence. We’re calling this color structure blColor3, and saving it in a header file called blColor3.hpp

blColor3.hpp (Click to see code…)

For Four-Channel images, we’re going to derive from the three-color structure, and add an Alpha channel

blColor4.hpp (Click to see code…)

Clean Up Resources Functor

Since we’re going to use boost::shared_ptr to hold our images, we’re going to need a simple functor for shared_ptr to properly release our images after we’re done using them. We’re going to put the functor in a separate file called blCleanResources.hpp, which will serve as a depository for other clean up code.

blCleanResources.hpp (Click to see code…)

Our Image Structure

Finally, we have come to our basic image structure. We’re going to separate the image structure into two classes:

  1. blImg — (Base class used to wrap raw IplImage pointers)
    1. Automatic garbage collection (with the use of boost::shared_ptr)
    2. Generic image coding (through the use of template meta-programming)
    3. Direct, efficient and easy access to pixels.
  2. blImage — (Derived class used to add basic image functionality)
    1. Provides constructors and functions to build, load and copy images from several sources, such as image files, images in memory, images from webcams, generalized matrix data and

The base blImg class is defined in a header file called blImg.hpp. This is a very long file that contains all the template specialization code to handle different image depths and number of channels

blImg.hpp (Click to see code…)

The derived blImage class is defined in a header file called blImage.hpp

blImage.hpp (Click to see code…)

Using the code

Using our image structure could not be any easier. This design super-simplifies the use of IplImage* pointers.  The following code shows some of the possible uses of blImage. Notice how easy it is to access pixel data without having to worry about types and depths. Pixel access is also very fast as it provides direct access to the pixel data.

int main(int argc, char *argv[])
{
    // Create a blank 8-bit color image
    blImage< blColor3<unsigned char> > MyImage;
    // Load an image from file
    MyImage.LoadImageFromFile("C:\\tux.png");
    // Create a floating point grayscale
    // version of our colored image
    blImage<float> MyImage2 = MyImage;
    // We can easily access pixel data very efficiently
    for(int i = 0; i < MyImage2.size1(); ++i)
        for(int j = 0; j < MyImage2.size2(); ++j)
            MyImage2[i][j] += 100;
    blImage<float> MyImage3 = MyImage2;
    // Let's take the first derivative in both directions
    cvSobel(MyImage2,MyImage2,1,1);
    cvNamedWindow("tux",CV_WINDOW_AUTOSIZE);
    cvShowImage("tux",MyImage3);
    cvWaitKey(0);
    return 0;
}

Running this little snippet I get the following image:

Downloads

I have put all the files into a zip file which can be downloaded here.  All you have to do is extract it somewhere, let’s say in a directory called blImageAPI, and then include the blImageAPI.hpp file as follows:

#include "blImageAPI/blImageAPI.hpp"
using namespace blImageAPI;

Note: Everything is declared in a namespace blImageAPI, and such you would use it as: blImageAPI::blImage

blImageAPI.zip -- Ver Jun/06/2011 1:43pm (1954)

Updates

Good Luck and have fun. I will be releasing subsequent algorithms and data structures for image analysis, so be sure to check back in and get the updated code files.

  1. I have changed the blImg.hpp file to clean out the template specialization code.  Much more efficient and clean now. I also have overloaded operators to handle colors in a natural way.
  2. I had used a virtual destructor for the blColor3 and blColor4 structures, but I had to change it to a normal destructor, as declaring it virtual added an extra 8 bytes of size to the structures.
  3. Fixed errors caused by converting pictures. Turns out that cvCvtColor function only works with input images of type “unsigned char”, “unsigned short” or “float”, but fails for the rest.
  4. Oct/28/2010 — Updated the blColor3 and blColor4 structures to include a default constructor that takes one value only. This makes them compatible with other types when writing generic image algorithms using the blImageAPI.
  5. Nov/10/2010 — Added a simple “GetImageDataCast()” function to the blImg.hpp file to get a pointer to the raw image data, but cast to the type of the image.
    1. This function allows us to get a casted pointer to the raw image data, for those of you that prefer using raw pointers.
      1. GetImageData() — returns a const char* pointer
      2. GetImageDataCast() — returns a const blType* pointer, where blType is the type of the image
  6. Nov/11/2010 — Fixed a bug in the “LoadImageFromFile” function. Now it works perfectly
  7. Nov/16/2010 — Changed the blColor4 constructor to have a default value for alpha. I had left the alpha value as a required value before.
  8. Nov/30/2010 — Added two operators to blColor3 and blColor4 to multiply and divide two colors. Now, when two colors are multiplied or divided, the individual color components are multiplied or divided.
  9. Dec/28/2010 — Added an assignment operator to the blImg class. Without it, when using the assignment operator, the copied image referred to the same IplImage, now the assignment operator clones the original image.
  10. Jan/02/2011
    1. Changed the destructors from virtual to normal
    2. Fixed a bug in the LoadImage function when converting a single channel image to a color image when the single channel image was of type double.
  11. May/25/2011
    1. Added a null deleter functor to blCleanResources so that we can wrap a pointer with a shared_ptr without calling its destructor when it goes out of scope. For example we can wrap the “this” pointer with a shared_ptr doing the following: 
      // Let's say we're inside the blImage class, we
// can send a shared_ptr of the "this" pointer doing
// the following:
shared_ptr< blImage<unsigned char> > MyPointer = shared_ptr< blImage<unsigned char> >(this,null_deleter());
    2. Added a constructor and a LoadMatrix function to the blImage class so that we can load CvMat* structures into a blImage structure. For example we can do this: 
      // Let's say we have a CvMat*
// structure from somewhere in
// our code like this
CvMat* MyMatrix;
// Then later in our code we want to
// create a blImage out of that matrix
// we can do the following
blImage<unsigned char> MyImage(MyMatrix);
// Or we could do
blImage<unsigned char> MyImage = MyMatrix;
// Or maybe this
blImage<unsigned char> MyImage;
MyImage.LoadMatrix(MyMatrix);
    3. Added a function to blImg to get retrieve the size in byte of an image like the following: 
      // We want to get the size of a blImage
blImage<unsigned char> MyImage;
cout << "The size of the image = " << MyImage.GetSizeInBytes();
  12. Jun/06/2011
    1. Added a function to blImage called WrapIplImage — It wraps an IplImage with a blImage and NULLIFIES the IplImage pointer.  In case the depth and number of channels don’t match, the IplImage is copied and the IplImage pointer is released and NULLIFIED.
Series NavigationblCaptureDevice — A simple data structure to handle video sources in OpenCV