OpenCV: first version up on NuGet

  • Article

I got OpenCV to work via NuGet! Here are the steps to consume it:

 

1. File > New > C++ > UWP

 

2. References > Manage NuGet References > Add OpenCV.UWP.native.imgcodecs

This package is currently in pre-release, so it only shows up if you check the "Include Pre-release" checkbox.

It brings in two other dependencies: imgproc and imgcore.

 

3. Write some code! Here are some starters: add these to your #include and your using sections in MainPage.xaml.cpp:

#include <robuffer.h>
#include <opencv2\imgcodecs.hpp>
#include <opencv2\imgproc.hpp>

using namespace Windows::UI::Xaml::Media::Imaging;
using namespace Windows::Storage::Streams;
using namespace Microsoft::WRL;

Next add this inside your MainPage constructor, just after InitializeComponents:

cv::Mat bmp1 = cv::imread("demo.jpg");
cv::Mat bmp2 = cv::Mat(bmp1.rows, bmp1.cols, CV_8UC4);
cv::cvtColor(bmp1, bmp2, CV_BGR2BGRA);
WriteableBitmap^ wbmp = ref new WriteableBitmap(bmp2.cols, bmp2.rows);
IBuffer^ buffer = wbmp->PixelBuffer;
unsigned char* dstPixels;
ComPtr<IBufferByteAccess> pBufferByteAccess;
ComPtr<IInspectable> pBuffer((IInspectable*)buffer);
pBuffer.As(&pBufferByteAccess);
pBufferByteAccess->Buffer(&dstPixels);
memcpy(dstPixels, bmp2.data, bmp2.step.buf[1] * bmp2.cols * bmp2.rows);
image1->Source = wbmp;

Next, add an Image to your MainPage.xaml:

<Image x:Name="image1"/>

Next find a picture, rename it "demo.jpg", and add it to your project. (By default when you drag a jpg into SolutionExplorer it gets placed in the Assets folder; the code above assumes instead that it's at top level).

 

4. Run it!

What's exciting is that this will run on all x86|x64|ARM configurations, and on all Debug|Release. It will pick the correct OpenCV binaries+libraries for the configuration you select.

 

 

5. Debugging: download the symbols

I tried to upload the PDBs to symbolsource.org as recommended in the NuGet docs, but symbolsource.org seems to be down or not accepting symbols. So instead, download them from here, and unzip onto your desktop:

  • OpenCV.UWP.native.symbols.zip [145mb; right-click > save-as].
  • Once it's downloaded, right-click > Properties > Unblock and then right-click > ExtractAll

 

6. Debugging: step into the OpenCV source code

The steps here are kind of fiddly, so that video might be the easiest way to follow them. It shows every single step explicitly.

  1. The first time you debug, go to Debug > Windows > Modules, sort this list alphabetically by the Name column, look for opencv_*, right-click > LoadSymbolInformation, and point it to the symbol directory you unzipped.
  2. Tools > Options > Debugging > uncheck the "Just My Code" button
  3. Tools > Options > Debugging > "Enable source server support".

Once that was set up, I wanted to F11 into my call to "cv::Mat(bmp1.rows, bmp1.cols, CV_8UC4);". At first when I F11 it just stepped through header files, until it got to this line in mat.inl.hpp:

create(2, sz, _type);

This particular function isn't inline in a header file; it resides in the DLL. There's something I don't understand about C++. How come, when I F11 now it just skips over that function? And when I right-click step-into-specific then it says no source is available? but then when I hit F11 a second time then it jumps into the source code proper? Anyway, here's how to make ti work. I did right-click > Step Into Specific > cv::Mat::create. This claims the source isn't available, so I click on view disassembly, to this line:

00398F48 jmp dword ptr [__imp_cv::Mat::create (03F2490h)]

If I hit F11 a second time then it downloads the OpenCV source code file automatically from GitHub (using the "enable source server support" feature that you checked earlier, and a clever technique by Hamish Graham to use GitHub itself as the source server). You can close the disassembly now and double-click on the top frame in the callstack. Now you can step through the source code of the DLL.

 

 

 So far I've only put three of the 15 OpenCV modules on NuGet: core, imgproc and imgcodecs. The work I did to generate these NuPkgs is here on GitHub.