Two Different Frames of Same VidioCapture

OpenCV Challenge


    I believe, this bug is going to be one of the most challenging I have worked on. Not because of complexity of the work, not because of the scale involved, but because the part of the code is brand new to me.
STDMETHODIMP SourceReaderCB::OnReadSample(HRESULT hrStatus, DWORD dwStreamIndex, DWORD dwStreamFlags, LONGLONG llTimestamp, IMFSample *pSample)
{
    CV_UNUSED(llTimestamp);
    HRESULT hr = 0;
    cv::AutoLock lock(m_mutex);
    if (SUCCEEDED(hrStatus))
    {
        if (pSample)
       
            CV_LOG_DEBUG(NULL, "videoio(MSMF): got frame at " << llTimestamp);
            IMFSample* prev = m_lastSample.Get();
            if (prev)
            {
                CV_LOG_DEBUG(NULL, "videoio(MSMF): drop frame (not processed)");
            }
            m_lastSample = pSample;
        }
    }
    else
    {
        CV_LOG_WARNING(NULL, "videoio(MSMF): OnReadSample() is called with error status: " << hrStatus);
    }

    Sooo, here we have a function... which takes 5 parameters... one of which seems to be a pointer... and then magic happens inside of the function.



    But first, let me refresh your memory on the issue itself. The bug crawls in OpenCV repo, and it concerns VideoStream: when a user plugs in an external video cam (it is supposed to have MSMF (Microsoft Media Foundation) background), takes a number of frames from the stream and, before one would take another frame, plugs out the cam, the OpenCV read() function will return a flag signaling about unsuccessful frame take, but it would also return non-empty frame, which logically would need to be empty.
    The bug is in C++ style of the code and shouldn't be so hard to spot. Yet, the fact that we are dealing with MSMF implies that the deep inside in the .cpp logic, the code will take forms of the one we can see in <windows.h> and such.
    “It's a dangerous business, Frodo, going out your door. You step onto the road, and if you don't keep your feet, there's no knowing where you might be swept off to.” (LotR).
    The piece of the code from the above is the one responsible for throwing an error upon trying to read a frame while having the camera being unplugged.


    Here's the piece of the code that can produce the issue:

VideoCapture m_camera;
    Mat m_originalFrame;

    m_camera.open(1 + CAP_MSMF); // Note manual change in the type of the camera
    bool success = m_camera.read(m_originalFrame);
    if (success)
    {
        cout << "Success\n";
        cout << "Please unplugged your camera \n";
        imshow("UNplugged your camera", m_originalFrame);
        waitKey();
        success = m_camera.read(m_originalFrame);
        if (!success || m_originalFrame.empty())
        {
            cout << "second non success\n";
            m_camera.release();
        }
        else
        {
            cout << "If camera is unplugged you shouldn't read this message";
        }
    }

        And the replicated issue can be seen on the following image:



    As to what approach should I take to fix this, I am currently thinking about looking for a default entity and the value it should take and try to return it instead of non-empty frame.

Issue can be found here.



Comments

Popular posts from this blog

Node.js: fs.open()

Good friend penguinV