blVideoThread — A simple class to capture video using opencv in a parallel thread

This entry is part 14 of 17 in the series blImageAPI -- BarbatoLabs Image API

Introduction

This is a super short post about a simple class I made to capture video in a parallel thread. Usually, the problem is that the video capturing hardware, like webcams, is limited to thirty frames per second. If we were to capture video in series with the main program, at best, the main program would be as fast as thirty frames per second.

To get around this problem, we can have a parallel thread capture the video, and when we want to, we can grab the captured frame and use it in our main program.

In this article, I present a super simple class, derived from blCaptureDevice and the thread class from the sfml library sf::thread.

Idea

The idea behind this video thread class is simple:

  1. We connect to the webcam or video source.
  2. The parallel thread will capture one frame and then set a flag letting us know that the frame has been captured
    1. The thread will not capture another frame while the flag is set
  3. The main program checks if a frame has been captured
    1. If No, the program moves on, and will check back later.
    2. If Yes, we grab the frame, and remove the flag.
  4. Once the flag has been removed, the video thread will be able to capture another frame, and the cycle will start over.

Note:  While the program is running, we can pause the capturing, (we set a flag, the thread is actually still running)

The code

I have saved the class in a file named: blVideoThread.hpp, which follows.

blVideoThread.hpp (Click to see code...)
#ifndef BL_VIDEOTHREAD_HPP
#define BL_VIDEOTHREAD_HPP

//-------------------------------------------------------------------
// FILE:            blVideoThread.hpp
// CLASS:           blVideoThread
// BASE CLASS:      sf::Thread,blCaptureDevice
//
// PURPOSE:         Based on sf::Thread and blCaptureDevice, it
//                  allows us to retrieve frames from a video source,
//                  which is usually limited to 30 frames per second,
//                  in a parallel thread, allowing the main application
//                  to run at much faster rates
//
// AUTHOR:          Vincenzo Barbato
//                  http://www.barbatolabs.com
//                  navyenzo@gmail.com
//
// LISENSE:         MIT-LICENCE
//                  http://www.opensource.org/licenses/mit-license.php
// DEPENDENCIES:    sf::Thread -- Derived from sf::Thread to handle
//                                threading in a cross-platform way
//
//                  blCaptureDevice -- We use a capture device to
//                                     grab frames from a video source
//                  blImage -- We use a blImage to store the
//                             last captured frame
//
// NOTES:
//
// DATE CREATED:    Feb/03/2011
// DATE UPDATED:
//-------------------------------------------------------------------

//-------------------------------------------------------------------
// Includes and libs needed for this file
//-------------------------------------------------------------------
//-------------------------------------------------------------------

//-------------------------------------------------------------------
// Enums needed for this file
//-------------------------------------------------------------------
//-------------------------------------------------------------------

//-------------------------------------------------------------------
class blVideoThread : public sf::Thread,
                      public blCaptureDevice
{
public: // Constructors and destructors

    // Default constructor
    blVideoThread();

    // Destructor
    ~blVideoThread()
    {
        StopCapturingThread();
    }

public: // Public functions

    // Functions used to start/stop/pause/unpause the thread
    void            StartCapturingThread();
    void            StopCapturingThread();
    void            PauseCapturingThread();

    // Function used to know whether
    // there's a new image available
    const bool&     IsNewFrameAvailable()const;

    // Function used to tell the video
    // thread that we grabbed the frame
    // and the it should go ahead and
    // grab another one
    void            LetThreadQueryAnotherFrame();

    // Function that gets called
    // when thread is running
    virtual void    Run();

    // Function used to get the captured frame
    const blImage< blColor3<unsigned char> >&   GetFrame()const;

protected: // Protected variables

    // Frame image used in this thread
    blImage< blColor3<unsigned char> >          m_Frame;

    // Boolean variable used to
    // continue/terminate the thread
    bool                m_IsCapturingThreadToBeTerminated;

    // Boolean variable used for
    // pausing the video grabbing
    bool                m_IsCapturingThreadPaused;

    // Boolean variable used to check if
    // new image is available from this thread
    bool                m_IsNewFrameAvailable;
};
//-------------------------------------------------------------------

//-------------------------------------------------------------------
inline blVideoThread::blVideoThread()
{
    // Default all the booleans
    m_IsCapturingThreadToBeTerminated = true;
    m_IsCapturingThreadPaused = false;
    m_IsNewFrameAvailable = false;
}
//-------------------------------------------------------------------

//-------------------------------------------------------------------
inline void blVideoThread::StartCapturingThread()
{
    // Let's first make sure that we
    // don't try to re-launch the thread
    // after it's already been started
    if(!m_IsCapturingThreadToBeTerminated)
    {
        // Error -- Tried to restart a thread
        //          that had already been started
        return;
    }

    // Check if the capture device has been set
    if(this->IsConnected())
    {
        // Since we have a capture device, we start the thread
        this->Launch();

        // Set boolean so that thread will run
        m_IsCapturingThreadToBeTerminated = false;
        return;
    }

    // Error -- In this case, we don't yet
    // have a capture device so we just return

    // Set the boolean in order
    // to terminate the thread
    m_IsCapturingThreadToBeTerminated = true;

    return;
}
//-------------------------------------------------------------------

//-------------------------------------------------------------------
inline void blVideoThread::StopCapturingThread()
{
    // Set the boolean in order
    // to terminate the thread
    m_IsCapturingThreadToBeTerminated = true;

    // Here we just terminate
    // the running thread
    this->Terminate();
}
//-------------------------------------------------------------------

//-------------------------------------------------------------------
inline void blVideoThread::Run()
{
    while(!m_IsCapturingThreadToBeTerminated)
    {
        if(!m_IsCapturingThreadPaused &&
           !m_IsNewFrameAvailable &&
           !m_IsCapturingThreadToBeTerminated)
        {
            // If we cannot successfully query
            // a frame, then we just stop the
            // capturing thread
            if(this->IsConnected())
            {
                // Query a new frame and
                // signal that a new frame
                // is available
                this->QueryFrame(m_Frame);
                m_IsNewFrameAvailable = true;
            }
            else
            {
                // The capturing device got
                // disconnected, so we stop
                // this capturing thread
                m_IsCapturingThreadToBeTerminated = true;
            }
        }
    }
}
//-------------------------------------------------------------------

//-------------------------------------------------------------------
inline void blVideoThread::PauseCapturingThread()
{
    // If the thread is currently paused, then
    // we unpause otherwise we pause it
    if(m_IsCapturingThreadPaused)
        m_IsCapturingThreadPaused = false;
    else
        m_IsCapturingThreadPaused = true;
}
//-------------------------------------------------------------------

//-------------------------------------------------------------------
inline const blImage< blColor3<unsigned char> >& blVideoThread::GetFrame()const
{
    return m_Frame;
}
//-------------------------------------------------------------------

//-------------------------------------------------------------------
inline const bool& blVideoThread::IsNewFrameAvailable()const
{
    return m_IsNewFrameAvailable;
}
//-------------------------------------------------------------------

//-------------------------------------------------------------------
inline void blVideoThread::LetThreadQueryAnotherFrame()
{
    m_IsNewFrameAvailable = false;
}
//-------------------------------------------------------------------

#endif // BL_VIDEOTHREAD_HPP

Usage

Using this class is very simple as illustrated by the following snippet:

// We have to include the blImageAPI
// library and define the following macro
// to use the video thread
#include "blImageAPI/blImageAPI"
using namespace blImageAPI;

#define USE_BL_VIDEOTHREAD

// This code should go somewhere at the
// beginning of the program, when first creating
// the video thread
// Here we initialize the video thread, connect it
// to a webcam and start the parallel capturing
blVideoThread MyVideoThread;
MyVideoThread.ConnectToWebcam(0);
MyVideoThread.StartCapturingThread();

// We then somewhere in a loop, capture
// the frame and do whatever we want with
// it, here we create a texture with it using
// blTexture
if(MyVideoThread.IsNewFrameAvailable())
{
    // Load the frame into a texture
    m_RollerTexture->LoadImageToTexture(m_VideoThread.GetFrame(),true,false);
    // Tell the video thread to capture another frame
    // (Here we set the flag mentioned above)
    m_VideoThread.LetThreadQueryAnotherFrame();
}

Downloads

I have put all the files into a zip file which can be downloaded here. All you have to do is extract it somewhere, let’s say in a directory called blImageAPI, and then include the blImageAPI.hpp file as follows:

#include "blImageAPI/blImageAPI.hpp"<br />
using namespace blImageAPI;<br />

Note:  Everything is declared in a namespace blImageAPI, and such you would use it as: blImageAPI::blImage

Note2: Also, to use the video thread, you have to define the macro USE_BL_VIDEOTHREAD (I did this so that in case you don’t have the sfml library, if you don’t define the macro, the compiler won’t yell at you).
blImageAPI.zip -- Ver Jun/06/2011 1:43pm (4992)

About Vincenzo Barbato

Known to his friends as Enzo, he's an outside-the-box engineer/researcher whose interests and expertise span many fields, including controls systems, multi-physics simulations, mechatronics, oil technologies, data analysis and machine vision just to name a few.

Refusing to grow up, he's on a continuous journey to develop simple and creative solutions that have the power of disrupting industries by optimizing systems and processes.

Married to a beautiful wife, with two beautiful daughters and two identical twin boys, his home is a never ending chaotic fountain of inspiration.

His outlook on life:

"Never blindly accept what you're told, listen, but then question, with curiosity, creativity and collaboration we can change the world"

About Enzo

Known to his friends as Enzo, he's an outside-the-box engineer/researcher whose interests and expertise span many fields, including controls systems, multi-physics simulations, mechatronics, oil technologies, data analysis and machine vision just to name a few. Refusing to grow up, he's on a continuous journey to develop simple and creative solutions that have the power of disrupting industries by optimizing systems and processes. Married to a beautiful wife, with two beautiful daughters and two identical twin boys, his home is a never ending chaotic fountain of inspiration. His outlook on life: "Never blindly accept what you're told, listen, but then question, with curiosity, creativity and collaboration we can change the world"

2 Comments

  1. Pingback: blVideoThread2 — Another way to capture video using opencv in a parallel thread | BarbatoLabs

  2. Pingback: blTexture — Load an IplImage into an opengl texture and create webcam and video textures using the blImageAPI | BarbatoLabs

Leave a Reply

Your email address will not be published. Required fields are marked *