NDepend 6.0

NDepend 6.0 was released a few months ago, and due to my workload I have been unable to blog about it until now. This version contains a ton of new features. I have been working with this product since 2009, I think (Sorry but I cannot remember the version back then) and I must say that with every new version the product gets better. It has been two and a bit years since version 5 came out, so many of you must be wondering what’s new in this version?

The first new feature you’ll notice is right there on the welcome screen and the additional integrations they have added to the product. It used to be just “Install Visual Studio Extension” that has been available since version 3, but now it integrates into TFS, TeamCity, SonarQube and Reflector.

 

image

 

NDepend is a very powerful tool and if you haven’t used it before, fear not. It provides a pseudo-wizard that allows developers to select what to do with the source code that’s been analyzed.

 

NDepend beginner dialog

 

The resulting report (post-analysis) has improved a lot by adding “how to fix” information to rule failures.

 

image

 

They have also added support for async/await methods, specially in the code coverage analysis.  The metrics view now can be resized and even add this kind of “heat map” color style to it.

 

 

Rules now can be exported and share and I must say it’s a pretty handy feature when there are more developers doing code analysis. This feature provides the flexibility of creating a rule file outside the project file.

 

image

 

Once you have created a custom rules file, it can be referenced from your project. It is preferable to use relative paths so it works the same on your development machine and the build server.

 

image

 

In summary, it’s a great tool and must have for every .NET developer’s toolbox. It keeps getting better and better. At the same time, if you need to analyze C++ code they also have CppDepend which is awesome as well. I must publicly thank Patrick for always providing me with a personal license.

 

Angel

Microsoft MVP Award recipient for 2016

Hi community,

It gives me a great joy to announce that Microsoft has awarded me with an MVP recognition once again, in this opportunity in the “Visual Studio and Development Technologies” category . Since the MVP Program has been around for more than two decades, Microsoft have recently made changes to it, therefore, I can contribute to anything that’s related to Visual Studio and not only Visual C++ centric. This change is benefitial for everyone, and it’s something that I’ve been doing for a long time ago when posting stuff around interop between managed and native code, as well as different technologies. This is my eleventh consecutive time as an MVP, and during this time I’ve been able to see how Microsoft and their technologies have change for better I would say. Their approach towards FOSS and cross-platform have made them a better and open organization than ever before.

MVP 2016

 

These days I spend my days mostly working in architecture design, building frameworks and solutions for customers and R&D which I find it very appealing because I can do cool stuff like building native code that leverages AWS C++ SDK with Visual StudioCLion or Qt Creator using C++ that I can deploy to my WindowsLinux or Raspberry Pi without any issues and that’s one of C++’s main strengths, portability and efficiency.
None of my successes as professional or individual could have been possible without God and his son Jesus, my family (my wife and two daughters), Microsoft and you, because you all keep my motivation to learn and keep myself improving everyday.

Thank-you very much all.

Angel

Integration between Qt application running on Linux and Microsoft SQL Server

I love computers, technology and software that’s for sure. I fell in love with them at a very early age and I still got the passion Smile

To me, when I’m able to make one system talk or communicate and interact with another it’s always fun and challenging, but more importantly it’s rewarding as.

I have always been a Microsoft dude, but at the same time I have always been supporter of FOSS and everything it’s got to offer. On a separate note but in the same context (kind of) I am passionate about standards, patterns and languages.

Talking of languages… People always come to me asking stuff about C++ and their misconceptions on the language. C++ allows me to do what any other language can’t… that simple. I am afraid that they way it’s be taught at universities differs big time from modern C++ features and best practices, but I’m not the best person to make that judgment and that’s not the intent of this post anyways.

From now onwards, I will start publishing articles on Qt, FOSS and how to integrate them into the Microsoft Eco-System. Today, I will start describing how to query a SQL Server Database from Ubuntu Linux via a Qt application.

The beauty of standards is that no matter who the consumer is, the expected outcome will always be the same (e.g.: Surfing the web from any browser regardless of the operating system, for instance).

Qt natively provides support for data aware applications, therefore developers can greatly benefit from this by implementing MVC based applications. MVC support was introduced in Qt 4. In this post, however I will not discuss any MVC related topic but a simple application running on Ubuntu that connects and pulls data from SQL Server. In order to do so, some artifacts and configuration steps are required in Linux.

 

In linux we will require to have UnixODBC driver, FreeTDS libraries and for debugging purposes Wireshark to check whether our requests are being sent over the network. We must configure wireshark in order to enable listening on network cards.

angel@ubuntu:~$ sudo apt-get install wireshark

angel@ubuntu:~$ sudo groupadd wireshark

angel@ubuntu:~$ sudo usermod -a -G wireshark $USER

angel@ubuntu:~$ sudo chgrp wireshark /usr/bin/dumpcap

angel@ubuntu:~$ sudo setcap cap_net_raw,cap_net_admin=eip /usr/bin/dumpcap

angel@ubuntu:~$ sudo apt-get -y install freetds-bin tdsodbc unixodbc

 

 

 

 

Once we have all the pre-requisites installed, we have to make changes to odbc.ini, odbcinst.ini and freetds.conf. These changes are shown below

image

Freetds.conf

 

image

odbc.ini

 

image

odbcinst.ini

 

Qt Creator uses the odbc.ini and odbcinst.ini files that are not the ones we have made changes to, so we either create a symbolic link to them or just copy them to the expected folder.

angel@ubuntu:~$ sudo cp /etc/odbcinst.ini /usr/local/etc/odbcinst.ini

angel@ubuntu:~$ sudo cp /etc/odbc.ini /usr/local/etc/odbc.ini

 

In order to test that our Linux environment has been properly configured and we can effectively connect and query SQL Server, we can use tsql utility that’s part of FreeTDS. It’s important to specify UNICODE (UTF-8) as the client charset in freetds.conf otherwise the client won’t be able to understand some of the data returned by SQL Server.

image

 

The issue with UNICODE not being properly configured comes up in Wireshark as Unknown Packet Type.

 

image

 

Back to my introduction on standards, TDS is an application layer protocol that was initially designed and developed by Sybase, but later on it’s been enhanced and maintained by Microsoft for their SQL Server. The protocol definition can be found here

 

We have described the prerequisite and configuration steps, now let’s move on to the sample application. As mentioned earlier, we will not make this a MVC app where we can bind the model directly to the view (native Qt widget) but we’ll pull data based on a query and then we populate a DTO that’s a vector (collection) of EmployeeDto class.

 

 

std::vector<EmployeeDto> EmployeeDal::GetPeople() {

    std::vector<EmployeeDto> retval;

    auto db = Database_get();

 

    if (db.open()) {

        QString queryStr("Select  distinct top 100  'ID'=A.BusinessEntityID, 'FirstName'=A.FirstName, 'LastName'=A.LastName ");

        queryStr.append("From  [AdventureWorks2012].[Person].[Person] A ");

        queryStr.append("Order by LastName");

        QSqlQueryModel query;

        query.setQuery(queryStr, db);

 

        auto count = query.rowCount();

 

        for (auto r = 0; r < (count > 0 ? count : RowCount); r++) {

            auto record = query.record(r);

 

            if (!record.isEmpty()) {

                EmployeeDto newRecord;

                newRecord.Id_set(record.field("ID").value().toInt());

                newRecord.FirstName_set(QString(record.field("FirstName").value().toString()));

                newRecord.LastName_set(QString(record.field("LastName").value().toString()));

                retval.push_back(newRecord);

            }

        }

        db.close();

    }

 

    return retval;

 

The way Qt handles events might resemble to .NET, but they’re quite different. Qt has the concept of Signals and Slots which makes it easy for developers to raise events to notify consumers that something has occurred and needs to be handled. The code snippet below calls our DAL which in turn pulls data from SQL Server. Qt uses any C++ toolchain available, and when running in Windows it uses MSVC. In my Windows development machine, I have Visual Studio, Qt Creator and CLION but I only have Qt Creator and CLION in my Linux environment, and once again the beauty of standards is that I can compile a project in Windows and the same project can also be compiled in a different environment. In Linux I mainly use the C++ compiler in GCC thus I can have C++ code with new features (e.g.: Lambdas) and benefit from whichever environment I’m using.

void MainWindow::on_btnRunQuery_clicked() {

    auto index = 0;

    EmployeeDal dalObj;

    auto results = dalObj.GetPeople();

    auto lstView = findChild<QTableWidget*>("lstQueryResult");

 

    if (results.size() > 0 && lstView != nullptr) {

        lstView->clearContents();

 

        lstView->setRowCount(results.size());

 

        std::for_each(results.begin(), results.end(), [&](EmployeeDto& employee) {

            lstView->setItem(index, 0, new QTableWidgetItem(QString::number(employee.Id_get())));

            lstView->setItem(index, 1, new QTableWidgetItem(employee.FirstName_get()));

            lstView->setItem(index, 2, new QTableWidgetItem(employee.LastName_get()));

            index++;

        });

    }

}

 

void MainWindow::on_btnClose_clicked() {

    auto code =  [&]() {close();};

    Messenger("Are you sure you want to quit?", reinterpret_cast<const QMainWindow*>(this), code);

}

 

Lambda expressions in C++ allow us to pass a functor as a parameter or we can also use  the function class template which is a general-purpose polymorphic function wrapper (This would be the equivalent of Action or Func in C#)

void MainWindow::SomeFunction(std::function<bool(void)>& ptr) {

    if (ptr != nullptr)

        ptr();

}

 

void MainWindow::on_Something_triggered() {

    std::function<bool(void)> ptr = std::bind(&MainWindow::DoSomething, this);

    SomeFunction(ptr);

}

In our example I have a template function called “Messenger” which displays a Messagebox that executes a lambda passed as parameter.

#include <QMessageBox>

 

template<class T>

void Messenger(const QString& text, const QMainWindow* window, T&& functor) {

    QMessageBox msgBox(window->parentWidget());

    msgBox.setText(text);

    msgBox.setStandardButtons(QMessageBox::Yes | QMessageBox::No);

    msgBox.setDefaultButton(QMessageBox::Yes);

 

    if (msgBox.exec() == QMessageBox::Yes)

        functor();

}

 

The images depicted below correspond to the application running and the MessageBox displayed by invoking our template function.

image image

 

If we start a trace in SQL Server Profiler we can see the request that has been received from the application

 

image

 

Sample demo source code here

 

Regards,

 

Angel

Whatis Utility for Windows

Hi Community,

I’ve just come back to the office after rolling off from an engagement, so this week while being back to the bench I’ve been practicing  and studying some Qt for my upcoming certification, doing a few things around Raspberry Pi + Kinect + OpenKinect on Raspbian but at the same time I also built an utility that’s been missing in Windows, however available in any other *nix operating system and I’m referring to whatis command which I find very useful.

whatis displays information stored in manual pages description, so I thought that I needed to provide similar functionality, hence I chose to use ESENT that’s built-in to Windows. ESENT is a NoSQL ISAM database primarily for C/C++ developers that can be also used from managed code (ESENT Managed Interop on codeplex, for example).

My version of whatis stores information about any binary (DLL or EXE) that contains a VERSIONINFO resource in an ESENT database. This information is stored the first time the utility is called to query a file, and then subsequent calls retrieve it from the database. The solution contains a few classes being the ESENTCore the most important one because it acts as DAL to interact with ESENT. If the database is deleted, do not worry because it will get created again.

I thought it’d be a great idea to dissect the solution by firstly describing the method that creates the table used by whatis

vector<ColumnInfo> EsentCore::GetColumnDefinition() {

    vector<ColumnInfo> retval;

 

    retval.push_back(ColumnInfo{0, wstring(Pk_Id_Column), JET_COLUMNDEF{

        sizeof(JET_COLUMNDEF), NULL, JET_coltypLong, NULL, NULL, NULL, NULL,

        GetLengthInBytes(4), JET_bitColumnAutoincrement}});

 

    retval.push_back(ColumnInfo{0, wstring(Name_Column), JET_COLUMNDEF{

        sizeof(JET_COLUMNDEF), NULL, JET_coltypText, NULL, NULL, NULL, NULL,

        GetLengthInBytes(50), JET_bitColumnFixed | JET_bitColumnNotNULL}});

 

    retval.push_back(ColumnInfo{0, wstring(Location_Column), JET_COLUMNDEF{

        sizeof(JET_COLUMNDEF), NULL, JET_coltypText, NULL, NULL, NULL, NULL,

        GetLengthInBytes(MAX_PATH), JET_bitColumnFixed | JET_bitColumnNotNULL}});

 

    retval.push_back(ColumnInfo{0, wstring(Description_Column), JET_COLUMNDEF{

        sizeof(JET_COLUMNDEF), NULL, JET_coltypLongText , NULL, NULL, NULL, NULL,

        GetLengthInBytes(MAX_PATH * 5), JET_bitColumnMaybeNull}});

 

    return retval;

}

It is a simple structure (Primary key that’s auto numeric, file name, file location and file description). Data operations with ESENT have to be performed in the context of a transaction,  and the database must be prepared for these operations to occur, a good example can be seen in the InsertRecord method shown below

bool EsentCore::InsertRecord(const vector<ColumnType>& columns) {

    auto colIndex = 0;

    auto retval = false;

    auto columnInfo = GetColumnIds();

    auto newColumns = make_unique<JET_SETCOLUMN[]>(modifiableColumnCount);

 

    if (columns.size() > 0) {

        if (SUCCEEDED(JetBeginTransaction(m_sessionId)) && SUCCEEDED(JetPrepareUpdate(m_sessionId, m_tableId, JET_prepInsert))) {

            ZeroMemory(newColumns.get(), sizeof(JET_SETCOLUMN) * modifiableColumnCount);

            for_each(columns.begin(), columns.end(), [&](const ColumnType& column) {

                if (column.ColumnName != Pk_Id_Column && column.ColumnName != CreatedOn_Column) {

                    auto colInfo = columnInfo.at(column.ColumnName);

                    newColumns[colIndex] = JET_SETCOLUMN{0};

                    newColumns[colIndex].columnid = colInfo.columnid;

                    auto ptrData = make_unique<char[]>(MAX_PATH * 5);

 

                    if (colInfo.coltyp == JET_coltypText || colInfo.coltyp == JET_coltypLongText) {

                        auto wstr = reinterpret_cast<wchar_t*>(const_cast<void*>(column.ColumnData.pvData));

                        auto size = wcslen(wstr);

                        wcstombs(ptrData.get(), wstr, size);

                        newColumns[colIndex].pvData = malloc(size);

                        memcpy(const_cast<void*>(newColumns[colIndex].pvData), ptrData.get(), size);

                        newColumns[colIndex].cbData = size;

                    }

 

                    newColumns[colIndex].err = JET_errSuccess;

                    colIndex++;

                }

            });

 

            retval = SUCCEEDED(JetSetColumns(m_sessionId, m_tableId, newColumns.get(), modifiableColumnCount)) &&

                SUCCEEDED(JetUpdate(m_sessionId, m_tableId, nullptr, NULL, NULL));

 

            // Free memory

            for (auto nIndex = 0; nIndex < colIndex; nIndex++)

                free(const_cast<void*>(newColumns[nIndex].pvData));

 

            // Commit or rollback transaction

            if (retval)

                JetCommitTransaction(m_sessionId, NULL);

            else  JetRollback(m_sessionId, NULL);

        }

    }

 

    return retval;

}

I’m a generic programming aficionado that’s why I enjoy using templates in C++ or generics in .NET and they’re not the same in case you’re wondering as I had mentioned here. Speaking of which, there’s a template function (similar to a generic method in .NET) that based on the input parameter it behaves differently yet it returns the same data type

template <typename T>

WhatIsRecord EsentCore::GetRecord(const T* cols) {

    WhatIsRecord retval;

    vector<void*> values;

    JET_RETRIEVECOLUMN* colValues = nullptr;

    vector<ColumnType>* colDef = nullptr;

    strstream name, location, description;

    auto colType = typeid(T) == typeid(JET_RETRIEVECOLUMN);

 

    if (colType)

        colValues = (JET_RETRIEVECOLUMN*)cols;

    else {

        colDef = (vector<ColumnType>*) cols;

 

        for (auto index = 0; index < 3; index++) {

            auto colData = colDef->at(index + 1).ColumnData;

            auto ptrData = make_unique<char[]>(MAX_PATH * 5);

            auto wstr = reinterpret_cast<wchar_t*>(const_cast<void*>(colData.pvData));

            auto size = wcslen(wstr);

            wcstombs(ptrData.get(), wstr, size);

            values.push_back(malloc(colData.cbData));

            ZeroMemory(values.at(index), colData.cbData);

            memcpy(values.at(index), ptrData.get(), size);

        }

    }

 

    name << reinterpret_cast<char*>(const_cast<void*>(colType ? colValues[1].pvData : values.at(0))) << endl;

    retval.Name = Trim(name);

    location << reinterpret_cast<char*>(const_cast<void*>(colType ? colValues[2].pvData : values.at(1))) << endl;

    retval.Location = Trim(location);

    description << reinterpret_cast<char*>(const_cast<void*>(colType ? colValues[3].pvData : values.at(2))) << endl;

    retval.Description = Trim(description);

 

    for_each(values.begin(), values.end(), [&](void* block) {free(block); });

 

    return retval;

}

The method mentioned above uses the typeid operator that’s one of the mechanisms in the language responsible to provide RTTI. This is the closest we can get to typeof operator in .NET. 

Strings are one of the trickiest things to deal with in native code, that’s something most managed code developers take for granted and they can easily use and handle, but fear not, in C++  we have STL’s strings and also streams to make our lives easier as shown in my implementation of Trim method below

string EsentCore::Trim(strstream& text) {

    string retval;

    string newString(text.str());

    

    for (auto index = 0; isprint((unsigned char) newString[index]); index++) 

        retval.append(1, newString[index]);

 

    retval.erase(retval.find_last_not_of(' ') + 1);

 

    return retval;

}

I’ve also created a project on codeplex – http://whatis.codeplex.com in case you want to use the utility or extend it… and yes, I know it’s got a very funny  URL Smile 

There’s one tool I use every time I have to work with an  ESENT database. It’s free, awesome and lightweight and that’s ESEDatabaseView by NirSoft. The image depicted below show the objects in my whatis database

image

as well as some of the contents of WhatIsCatalogue (Table used by whatis) with information loaded by running a PowerShell script described next

image
 
The following script retrieves all the executable images in System32, loops through the collection and with every iteration it adds the file information to whatis.db

$files = Get-ChildItem "C:\Windows\System32" -Filter *.exe

 

for ($i=0; $i -lt $files.Count; $i++) {

    $exe = "c:\windows\system32\whatis.exe"

    &;$exe  $files[$i].FullName

}

image

As mentioned earlier, images below correspond to whatis running on my three different development environments (Ubuntu LinuxOS X El Capitan and our custom version of the utility running on Windows 10

image

image

image

C Language – Tutorial

Hi Community,

I would like to share with you an interesting tutorial on C language kindly provided by the guys from Udemy. Long story short, I received an email from a nice lady called “Molly Elizabeth” (which I originally thought to be a bot) asking me whether I was keen on publishing their tutorial that I’ve found succinct and well-explained. Please find the tutorial here – C Tutorial: Learn C in 20 Minutes

Regards,

Angel

Windows 10 and Visual Studio 2015 Resources

Hi Community,

Windows 10 and Visual Studio 2015 were successfully launched a few days ago, and with them a bunch of new features were introduced, being Universal Windows Apps one of the most exciting and expected features; so what I’ve done is to compile a table that contains links to useful resources and even offers available on these new technologies, please find it below

Reference Comments

Developer Tools Download

Download any Visual Studio SKUs, including VS 2015

AzureCon – September 29th Event

Join Live Q&As and interact with the architects and engineers who are building the latest features

.NET Native – What it means for Universal Windows Platform (UWP) developers

Great blog for UWP devs to understand .NET Native

Free Xamarin Subscriptions  for Windows Phone Developers

Offers expire on August 31, 2015

A Developer’s Guide to Windows 10

The latest Windows 10 developer training contents

How Microsoft Edge and Internet Explorer 11 on Windows 10 work better together in the Enterprise

 

* Developer for Windows 10 – What’s new
* Get Started
* Design
* Develop
* Publish

All the information developers may need to write applications for Windows 10

Windows 10 courses in MVA

Great site to get online courses on Windows 10

Channel 9

Another great online resource for Windows 10 related videos

MetaFS (Metadata File System for Windows)

FileSystems are the storage backbone for any computer system, in Today’s operating systems most of them provide a common functionality set like journaling, security and metadata among others. In my humble opinion, metadata is one of the most important ones, reason being is that they can tell users about the contents of the file without even opening it. Therefore, the information entered has to be accurate in order to make metadata important, otherwise it just doesn’t make sense to have it at all.

Starting with the principle of “allowing users capture the information the consider important in a flexible and customizable way” I started working on this project called “MetaFS”  a few months back (currently available on codeplex.com). I can humbly say that’s 85% percent complete, and I’m confident that the remaining bits can be completed with the help from developers in the community.

Unlike WinFS, MetaFS can have any database as metadata repository, but more importantly any given organization can create templates to capture the information that’s relevant to them, thus providing a personal touch to the solution. These templates can easily be customized per file types and even per user by extending the template which is based in XML.

The solution is comprised of the following elements:

image

  • MetaFsDriver: It’s a minifilter driver that monitors changes to the filesystem (similar to the way Antivirus and scanning software work) and if changes are detected, an event is raised to notify Windows Service which in turn interacts with Data Access layer (built with Entity Framework) to update metadata information in the database. Every file in windows has its own identifier FltQueryInformationFile function from the mini driver and then use the ObjectId member that contains this unique value. In MetaFS that has not been implemented yet but it’s the plan to have it, therefore it’s easier to update the metadata of a given file instead of using filenames (which is the current implementation). 

    image

  • MetaFSAgent:  It’s a Windows Service that starts the mini driver but more importantly allows kernel mode notifications to be raised to user mode. I explain this on this post and this communication channel is enabled through a symbolic link. The service is also responsible for calling the data access layer to update metadata information in the database.

    image

  • Shell Extension (Dynamic Property Page):  it’s a “dynamic” property  page (or sheet) that renders template definition at runtime. It’s dynamic in nature because the UI varies based on the template.  It’s an implementation of IShellPropSheetExt interface in the form of an ATL COM Library. This shell extension is loaded by the shell of the OS (Windows Explorer).

    image

  • Managed Code: This component is responsible for data access and rendering the UI based on template definition in XML. So, what effectively happens is that Windows Explorer loads the custom shell extension (property page) that in turns creates an instance and embeds a .NET user control in it and in doing so Windows Explorer creates an AppDomain thus expecting a config file (explorer.exe.config) that contains the connection string to connect to our database.   The images below depict our template rendered at runtime, and the beauty is that if changes are made to the template definition those changes are reflected the next time I’ll right click on the file to see its properties.

    image

If we start Spy++ we can inspect all the visual elements that comprise our custom template, and as you can see they’re all standard Windows Form controls. The purpose of this project was to enable and help users capture metadata through a customizable template approach, but this also opens a few opportunities to integrate with other systems (e.g.: To upload a file directly to SharePoint, for instance).

image

There are a few technical aspects in the solution, but there’s one however, that allows to intercept and replace the Dialog procedure so enabling calling .NET code from any of the existing defined buttons of the dialog (OK, Cancel and Apply). This code is shown below

/////////////////

// C++ Code    // 

/////////////////

 

 

#import "..\..\MetaFSPropPage\bin\x64\Debug\MetaFSPropPage.tlb" 

 

WNDPROC pWndProc;

 

LRESULT CALLBACK CustomWndProc(HWND hwndDlg, UINT msg, WPARAM wParam, LPARAM lParam) {

    FindChild* findChildWnd;

    auto retval = CallWindowProc(pWndProc, hwndDlg, msg, wParam, lParam);

 

    switch (msg) {

    case WM_COMMAND:

        auto selectedButton = LOWORD(wParam);

        if (selectedButton == IDOK || selectedButton == IDAPPLY || selectedButton == IDCANCEL) {

            auto enumChildProc = [](HWND hwnd, LPARAM lParam)-> BOOL {

                auto retval = TRUE;

                wchar_t buffer[MAX_PATH];

                auto lParamValues(reinterpret_cast<FindChild*>(lParam));

 

                if (GetClassName(hwnd, buffer, wcslen(buffer)) > 0) {

                    wstring className(buffer);

                    if (className.find(lParamValues->windowName) != wstring::npos) {

                        lParamValues->nMatchCount++;

 

                        if (lParamValues->FoundTargetIndex()) {

                            lParamValues->hFound = hwnd;

                            retval = FALSE;

                        }

                    }

                }

                return retval;

            };

 

            EnumChildWindows(hwndDlg, enumChildProc, reinterpret_cast<LPARAM>((findChildWnd = new FindChild)));

 

            switch (selectedButton) {

            case IDOK:

                SendMessage(findChildWnd->hFound, WM_APP, IDOK, NULL);

                break;

            case IDAPPLY:

                SendMessage(findChildWnd->hFound, WM_APP, IDAPPLY, NULL);

                break;

            case IDCANCEL:

                SendMessage(findChildWnd->hFound, WM_APP, IDCANCEL, NULL);

                break;

            }

 

            delete findChildWnd;

        }

        break;

    }

 

    return retval;

}

/////////////

// C# Code //

/////////////

 

 

protected override void WndProc(ref Message m) {

            switch (m.Msg) {

                case WM_IME_SETCONTEXT:

                    if (Controls[0].Controls.Count == 0)

                        UpdateOrRetrieveMetadata(SelectedFile);

                    break;

                case WM_APP:

                    if (m.WParam.ToInt32().Equals(IDAPPLY)) {

                        if (HasChanges)

                            UpdateOrRetrieveMetadata();

 

                    } else if (m.WParam.ToInt32().Equals(IDOK)) {

                        UpdateOrRetrieveMetadata();

                    }

                    break;

            }

            base.WndProc(ref m);

        }

Another important point to mention is the Windows Service registration, in .NET  Visual Studio does all of the scaffolding for us hence it provides the Service Installer but in MetaFS case,  our service is native so how can we install and register it? Fear not… That’s very easy to do Smile

//////////////////////////////////////

//// Service Creation/Registration ///

//////////////////////////////////////

 

sc create "MetaFSAgent" binPath="C:\Code\Community\VisualC\MetaFS\x64\Debug\MetaFsService.exe" DisplayName="MetaFS Agent" obj="MyPC\MetaFSAgentUser" password="MyPassword"

Sc description "MetaFSAgent" "Metadata File System Agent - Kernel Mode/User Mode Interaction"

 

//////////////////////////////////////

//// Service Deletion/Uninstall    ///

//////////////////////////////////////

 

Sc delete "MetaFSAgent"

and that’s pretty much it. Please feel free to contribute to the project here http://metafs.codeplex.com

Happy coding!

Angel

Building Native Windows Applications with CLION, GCC and CygWin

As a software developer and architect, I’m always looking at options for my customers whether they’re around technologies, tools, hosting, security and tenancy model (being the last three mandatory in a “cloudy” world) among some other factors and considerations. To me crafting software it’s more than cracking code and ensure it compiles, it’s more about building smart solutions that will help people, as well as enjoying the building process. What really makes it enjoyable are two things (in my humble and personal opinion: First, what the challenge and requirements are;  Second, technology to use and lastly, the IDE).

In regards to points 2 & 3, I have to say that C++ is an awesome and powerful language and similarly to C, they both have been standard and platform independent since their very first versions, something that was achieved by Java and subsequently by .NET (and its Linux implementation, Mono for instance). So what’s all this introductory fuss about then? well… This post is about building a very simple native Windows application using non-Microsoft technologies:

Our source code Today is pretty straightforward, it’s classic Win32 development and if you’re like me and keen on learning how to do this kind of development without any frameworks, I can mention two books that helped me a lot when I was learning (They’re not recent or new though and the OS has changed a lot since the books were originally published – They are a relic, actually)

petzold Norton

The code and a screenshot of the application running are shown below

#include <windows.h> HINSTANCE hInst; const char g_szClassName[] = "bonafideideasWindowClass"; int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, LPSTR lpCmdLine, int nCmdShow){ MSG Msg; HWND hwnd; WNDCLASSEX wc; wc.lpszMenuName = NULL; wc.hInstance = hInstance; wc.lpszClassName = g_szClassName; wc.cbSize = sizeof(WNDCLASSEX); wc.cbClsExtra = wc.cbWndExtra = 0; wc.style = CS_HREDRAW | CS_VREDRAW ; wc.hbrBackground = (HBRUSH)(COLOR_WINDOW+1); wc.hCursor = LoadCursor(NULL, IDC_ARROW); wc.hIcon = LoadIcon(NULL, IDI_APPLICATION); wc.hIconSm = LoadIcon(NULL, IDI_APPLICATION); wc.lpfnWndProc = [=](HWND hwnd, UINT msg, WPARAM wParam, LPARAM lParam) -> LRESULT { HDC hdc; RECT rect; TEXTMETRIC tm; PAINTSTRUCT ps; switch(msg) { case WM_CLOSE: if (MessageBox(NULL, "Are you sure you want to quit?", "Confirmation", MB_ICONQUESTION | MB_YESNO) == IDYES) DestroyWindow(hwnd); break; case WM_DESTROY: PostQuitMessage(0); break; case WM_PAINT: hdc = BeginPaint (hwnd, &ps) ; GetClientRect (hwnd, &rect) ; DrawText (hdc, TEXT ("Native Windows Development with CygWin and CLion."), -1, &rect, DT_SINGLELINE | DT_CENTER | DT_VCENTER) ; EndPaint (hwnd, &ps) ; return 0 ; default: return DefWindowProc(hwnd, msg, wParam, lParam); } return 0; }; if(!RegisterClassEx(&wc)) { MessageBox(NULL, "Window Registration Failed", "Error", MB_ICONEXCLAMATION | MB_OK); return 0; } hwnd = CreateWindowEx( WS_EX_CLIENTEDGE, g_szClassName, "Simplest Windows Native App built with CLion", WS_OVERLAPPEDWINDOW, CW_USEDEFAULT, CW_USEDEFAULT, 500, 250, NULL, NULL, hInstance, NULL); if(hwnd == NULL) { MessageBox(NULL, "Window Creation Failed", "Error", MB_ICONEXCLAMATION | MB_OK); return 0; } ShowWindow(hwnd, nCmdShow); UpdateWindow(hwnd); while(GetMessage(&Msg, NULL, 0, 0) > 0) { TranslateMessage(&Msg); DispatchMessage(&Msg); } return Msg.wParam; }

 

AppRunning

Now, let’s talk a bit of CLion… As previously mentioned it’s a cross-platform IDE (I’m writing a few things on Linux with it, actually).  It’s very similar to IntelliJ IDEA (Keep in mind it’s from the same company). The UI is easy to use and  functional (It’s not fair to compare to more mature products) and just to give you a glimpse

image

Besides providing with a good and decent Intellisense  support and fast parsing oh header files and libraries, good debugging experience and it also supports most of the new features in C++11 and C++14 Standards (This is because of using GCC as compiler).

image

Happy coding!,

Angel

Instrumenting SharePoint code via Performance Counters

Instrumentation is an important and integral part of every system because it allows (among some other things) to monitor the health of the system and/or components of it.  On a recent engagement, I had to architect a SharePoint solution that integrates with a core LOB system. The SharePoint solution comprises a few workflows that in turn do specific operations with the external system. Even when SEH, Tracing and Logging was implemented I needed something else, performance counters to provide IT staff with real-time information of the solution and in doing so, I leveraged one feature in the operating system as well as giving metrics on the performance of the solution.

Performance counters have been available in .NET since 1.0, however, it’s unusual to use them in a SharePoint solution. The way I approached this is described below:

  1. I usually use DI/IOC in my solutions (please note, not every solution benefits from this pattern though so implementing it might cause more headaches than expected results). Therefore, I defined my instrumentation capabilities in an interface IInstrumentationManager

    public interface IInstrumentationManager { /// <summary> /// Gets the perf counters. /// </summary> /// <value> /// The perf counters. /// </value> IPerfCounters PerfCounters { get; } /// <summary> /// Gets the logger. /// </summary> /// <value> /// The logger. /// </value> ILogger<ILogEntry> Logger { get; } }

  2. The performance counters are defined in IPerfCounters
    public interface IPerfCounters { /// <summary> /// Gets the created configuration. /// </summary> /// <value> /// The created configuration. /// </value> DateTime CreatedOn { get; } /// <summary> /// Gets the instance unique identifier. /// </summary> /// <value> /// The instance unique identifier. /// </value> Guid InstanceId { get; } /// <summary> /// Gets the <see cref="PerformanceCounter"/> with the specified counter name. /// </summary> /// <value> /// The <see cref="PerformanceCounter"/>. /// </value> /// <param name="counterName">Name of the counter.</param> /// <returns></returns> PerformanceCounter this[PerfCounter counterName] { get; } }

  3. Please note I have an indexer that uses an enum to access a specific performance counter.  The concrete class implements  IPerfcounters and IDisposable.
    public class PerfCounters : IPerfCounters, IDisposable { /// <summary> /// Gets the created configuration. /// </summary> /// <value> /// The created configuration. /// </value> public DateTime CreatedOn { get; private set; } /// <summary> /// Gets the instance unique identifier. /// </summary> /// <value> /// The instance unique identifier. /// </value> public Guid InstanceId { get; private set; } /// <summary> /// Gets the <see cref="PerformanceCounter" /> with the specified counter name. /// </summary> /// <value> /// The <see cref="PerformanceCounter" />. /// </value> /// <param name="counterName">Name of the counter.</param> /// <returns></returns> public PerformanceCounter this[PerfCounter counterName] { get { var retval = CountersAvailable.ContainsKey(counterName) ? CountersAvailable[counterName] : null; return retval; } } /// <summary> /// Gets or sets the counters available. /// </summary> /// <value> /// The counters available. /// </value> protected Dictionary<PerfCounter, PerformanceCounter> CountersAvailable { get; set; } /// <summary> /// Gets the counter data collection. /// </summary> /// <value> /// The counter data collection. /// </value> protected CounterCreationDataCollection CounterDataCollection { get; private set; } /// <summary> /// The disposed /// </summary> /// <value> /// <c>true</c> if this instance is disposed; otherwise, <c>false</c>. /// </value> protected bool IsDisposed { get; private set; } /// <summary> /// Gets or sets the configuration manager. /// </summary> /// <value> /// The configuration manager. /// </value> protected IConfigManager ConfigManager { get; set; } /// <summary> /// Gets or sets the impersonate helper. /// </summary> /// <value> /// The impersonate helper. /// </value> protected IImpersonateHelper ImpersonateHelper { get; set; } /// <summary> /// Initializes a new instance of the <see cref="PerfCounters" /> class. /// </summary> /// <param name="configMgr">The configuration MGR.</param> /// <param name="impersonateHelper">The impersonate helper.</param> public PerfCounters(IConfigManager configMgr, IImpersonateHelper impersonateHelper) { CreatedOn = DateTime.Now; ConfigManager = configMgr; InstanceId = Guid.NewGuid(); ImpersonateHelper = impersonateHelper; CounterDataCollection = new CounterCreationDataCollection(); CountersAvailable = new Dictionary<PerfCounter, PerformanceCounter>(); CreateOrInitializePerfCounters(); } /// <summary> /// Finalizes an instance of the <see cref="PerfCounters"/> class. /// </summary> ~PerfCounters() { Dispose(false); } /// <summary> /// Creates the original initialize perf counters. /// </summary> protected void CreateOrInitializePerfCounters() { NetworkCredential credential; var localAdmin = ConfigManager[Consts.LocalAdminKey]; if (!string.IsNullOrEmpty(localAdmin) && (credential = localAdmin.ConvertToNetworkCredential()) != null) ImpersonateHelper.Impersonate(CreatePerfCounters, credential); AddCountersToInstance(); } /// <summary> /// Creates the perf counters. /// </summary> private void CreatePerfCounters() { // Let's create Performance counter category if (!PerformanceCounterCategory.Exists(Consts.PerformanceCounterCategory)) { CounterDataCollection.AddRange(new[] { new CounterCreationData(Consts.TotalErrorCounter, string.Empty, PerformanceCounterType.NumberOfItems64), new CounterCreationData(Consts.TotalUnhandledErrorCounter, string.Empty, PerformanceCounterType.NumberOfItems64), new CounterCreationData(Consts.TotalTicketsProcessedCounter, string.Empty, PerformanceCounterType.NumberOfItems64), }); PerformanceCounterCategory.Create(Consts.PerformanceCounterCategory, Consts.PerformanceCounterCategory, PerformanceCounterCategoryType.MultiInstance, CounterDataCollection); } } /// <summary> /// Adds the counters to instance. /// </summary> private void AddCountersToInstance() { // Let's ensure category exists and create custom performance counters if (PerformanceCounterCategory.Exists(Consts.PerformanceCounterCategory)) { CountersAvailable.Add(PerfCounter.ErrorCount, new PerformanceCounter(Consts.PerformanceCounterCategory, Consts.TotalErrorCounter, Consts.PerformanceCounterCategory, false)); CountersAvailable.Add(PerfCounter.UnhandledExceptions, new PerformanceCounter(Consts.PerformanceCounterCategory, Consts.TotalUnhandledErrorCounter, Consts.PerformanceCounterCategory, false)); CountersAvailable.Add(PerfCounter.TotalTicketsProcessed, new PerformanceCounter(Consts.PerformanceCounterCategory, Consts.TotalTicketsProcessedCounter, Consts.PerformanceCounterCategory, false)); } } /// <summary> /// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources. /// </summary> public void Dispose() { Dispose(true); GC.SuppressFinalize(this); } /// <summary> /// Releases unmanaged and - optionally - managed resources. /// </summary> /// <param name="isDisposing"><c>true</c> to release both managed and unmanaged resources; <c>false</c> to release only unmanaged resources.</param> protected virtual void Dispose(bool isDisposing) { if (!IsDisposed) { if (isDisposing) { if (CountersAvailable != null && CountersAvailable.Count > 0) CountersAvailable.ToList().ForEach(x => { x.Value.Close(); x.Value.Dispose(); }); } IsDisposed = true; } } }

  4. Performance counters must be created and registered on the computer before they can be used, and in order to do that the user creating them must meet a certain criteria, this is usually done as a separate step in the deployment of the solution using the performance counters, however, I was able to do it by storing encrypted credentials of  user with right permissions on a SharePoint List, and this list entry was present then performance counters were created through impersonation of this user, if the list entry was missing nothing occurred.  The code responsible for impersonating user was passed in as a callback
    public class ImpersonateHelper : IImpersonateHelper { #region "Consts" private const int Logon32Interactive = 2; private const int Logon32ProviderDefault = 0; #endregion #region "Imported functions" /// <summary> /// Logons the user. /// </summary> /// <param name="lpszUsername">The LPSZ username.</param> /// <param name="lpszDomain">The LPSZ domain.</param> /// <param name="lpszPassword">The LPSZ password.</param> /// <param name="dwLogonType">Type of the dw logon.</param> /// <param name="dwLogonProvider">The dw logon provider.</param> /// <param name="phToken">The ph token.</param> /// <returns></returns> [DllImport("advapi32.dll", SetLastError = true)] private static extern bool LogonUser(string lpszUsername, string lpszDomain, string lpszPassword, int dwLogonType, int dwLogonProvider, ref IntPtr phToken); /// <summary> /// Closes the handle. /// </summary> /// <param name="handle">The handle.</param> /// <returns></returns> [DllImport("kernel32.dll", CharSet = CharSet.Auto)] private extern static bool CloseHandle(IntPtr handle); #endregion #region "Impersonate code" #endregion /// <summary> /// Impersonates the specified code to impersonate. /// </summary> /// <param name="codeToImpersonate">The code to impersonate.</param> /// <param name="credential">The credential.</param> /// <returns></returns> /// <exception cref="System.NotImplementedException"></exception> public IExecutionResult Impersonate(Action codeToImpersonate, NetworkCredential credential) { var tokenHandle = IntPtr.Zero; var retval = new ExecutionResult() {Success = false}; if (codeToImpersonate != null && credential != null) { try { if (LogonUser(credential.UserName, credential.Domain, credential.Password, Logon32Interactive, Logon32ProviderDefault, ref tokenHandle)) { using (var newId = new WindowsIdentity(tokenHandle)) { using (var impersonatedUser = newId.Impersonate()) { codeToImpersonate.Invoke(); impersonatedUser.Undo(); } } retval.Success = true; } } catch (Exception ex) { retval.ErrorCondition = ex; } finally { CloseHandle(tokenHandle); } } return retval; } }

  5. Now, we’ve got so far out performance counters (creation and registration) but you might be wondering, where are we going to store it? Answer is very simple – in the  AppDomain. AppDomain is created when the CLR starts a new instance of any .NET application, at the same time developers can store information that’s not persisted once the application has been terminated, so it’s kind of a “property bag” in our application’s process memory.
    var defaultCounters = new PerfCounters(container.Resolve<IConfigManager>(), container.Resolve<IImpersonateHelper>()); RegisterPerfCounters(defaultCounters); container.RegisterInstance(typeof(IPerfCounters), defaultCounters); /// <summary> /// Registers the perf counters. // </summary> /// <typeparam name="T"></typeparam> /// <param name="instance">The instance.</param> /// <param name="instanceName">Name of the instance.</param> private void RegisterPerfCounters<T>(T instance, string instanceName = Consts.PerfCounterInstanceName) where T : IPerfCounters { if (AppDomain.CurrentDomain.GetData(instanceName) == null) AppDomain.CurrentDomain.SetData(instanceName, instance); }

  6. The InstrumentationManager then retrieves our IPerfCounters instance from the AppDomain through Unity
    public class InstrumentationManager : IInstrumentationManager { /// <summary> /// Gets the perf counters. /// </summary> /// <value> /// The perf counters. /// </value> /// <exception cref="System.NotImplementedException"></exception> public IPerfCounters PerfCounters { get { return AppDomain.CurrentDomain .GetData(Consts.PerfCounterInstanceName) as IPerfCounters; } } /// <summary> /// Gets the logger. /// </summary> /// <value> /// The logger. /// </value> /// <exception cref="System.NotImplementedException"></exception> public ILogger<ILogEntry> Logger { get { return GlobalService.Current.Container.Resolve<ILogger<ILogEntry>>(); } } }

  7. Then we increment or decrement our performance counter by accessing the right counter through an indexer that takes an enum
    var InstrumentationManager = GlobalService.Current .Container.Resolve<IInstrumentationManager>(); InstrumentationManager.PerfCounters[PerfCounter.TotalTicketsProcessed] .Increment();

When our solution is running, we can then open perfmon and see how it’s behaving

perfmon

Communication between user and kernel-mode

As a follow-up post on Windows Services and Minifilters, Today I’ll explain one way we can send notifications from kernel-mode to an user-mode application.  Let’s remember Windows provides fault tolerance protection in the form of rings, thus an offending code in one of the rings should not compromise the stability of the system. My metadata filesystem project has a minifilter driver that notifies a Windows service when an important event (event I’m interested in) has occurred. 

The basics to do this are:

  1. Minifilter driver creates a device by calling IoCreateDevice and also creates a symbolic link through IoCreateSymbolicLink

    NTSTATUS DriverEntry(_In_ PDRIVER_OBJECT DriverObject, _In_ PUNICODE_STRING RegistryPath) { NTSTATUS retval = STATUS_SUCCESS; UNICODE_STRING DeviceName; UNICODE_STRING LinkName; PDEVICE_OBJECT EventDrvDeviceObject; UNREFERENCED_PARAMETER(DriverObject); UNREFERENCED_PARAMETER(RegistryPath); KdPrint(("MetaFsDriver::DriverEntry - Entered.\n")); g_FilterDriverObject = DriverObject; // Create Dispatch Entry Points. for (int x = 0; x <= IRP_MJ_MAXIMUM_FUNCTION; ++x) DriverObject->MajorFunction[x] = MetaFsDispatchPassThrough; DriverObject->DriverUnload = MetaFsDriverUnload; DriverObject->MajorFunction[IRP_MJ_CREATE] = MetaFsDispatchCreate; DriverObject->MajorFunction[IRP_MJ_DEVICE_CONTROL] = MetaFsDispatchDeviceControl; // Create device object RtlInitUnicodeString(&DeviceName, MetaFs_NT_DEVICE_NAME); retval = IoCreateDevice( DriverObject, 0, &DeviceName, FILE_DEVICE_UNKNOWN, 0, FALSE, &EventDrvDeviceObject); if (!NT_SUCCESS(retval)) { return retval; } KdPrint(("MetaFsDriver::DriverEntry - Device Created.\n")); // Create symbolic links RtlInitUnicodeString(&LinkName, MetaFs_WIN32_DEVICE_NAME); retval = IoCreateSymbolicLink(&LinkName, &DeviceName); if (!NT_SUCCESS(retval)) { IoDeleteDevice(EventDrvDeviceObject); return retval; } KdPrint(("MetaFsDriver::DriverEntry - Symbolic link created.\n")); // Set FastIO dispatch table DriverObject->FastIoDispatch = &g_FastIoDispatch; retval = IoRegisterFsRegistrationChange(DriverObject, MetaFsNotificationCallback); KdPrint(("MetaFsDriver::DriverEntry - Exited.\n")); return retval; }

    Since code above is running in kernel mode, it’s a bit tricky to send any output to Visual Studio output window, but fear not we can always make use of KdPrint to send messages to the kernel debugger, at the same time we also need to use DebugView to  capture this messages

    image

    Our minidriver details can be seen via DeviceTree

    image

  2. Once our minifilter driver is running and it’s created a symbolic link, the user mode application  (in our case, a Windows service) creates an event by calling CreateEvent and then uses DeviceIoControl to send the event handle to the driver. In order to send our control code, we use the name of the symbolic link that was created by minifilter driver.

    VOID InitializeCommunicationWithDriver() { HANDLE hFile; DWORD cbSize; if ((hFile = CreateFile(TARGET_DRIVER, GENERIC_READ | GENERIC_WRITE, 0, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL)) != INVALID_HANDLE_VALUE) { if ((g_filterEventHandle = CreateEvent(NULL, FALSE, FALSE, NULL)) != NULL) { if (DeviceIoControl(hFile, IOCTL_REGISTER_EVENT, &g_filterEventHandle, sizeof(g_filterEventHandle), NULL, NULL, &cbSize, NULL)) WriteEvent(EventInformation(EventType::INFORMATION_TYPE, L"Event successfully created....")); } CloseHandle(hFile); } }

  3. The call to CreateEvent creates a kernel mode KEVENT object and makes an entry into the application process’s handle table that points to the KEVENT. The HANDLE value returned to the application is essentially an index into the handle table. The handle isn’t directly useful to a WDM driver, though, for two reasons. First of all, there isn’t a documented kernel-mode interface for setting an event, given just a handle. Second, and most important, the handle is useful only in a thread that belongs to the same process. If driver code runs in an arbitrary thread (as it often does), it will be unable to reference the event by using the handle.

  4. It’s required to convert the handle to a pointer to the underlying KEVENT object. To handle a METHOD_BUFFERED control operation that the application uses to register an event with the driver

    HANDLE hEvent = *(PHANDLE) Irp->AssociatedIrp.SystemBuffer; PKEVENT pevent; NTSTATUS status = ObReferenceObjectByHandle(hEvent, EVENT_MODIFY_STATE, *ExEventObjectType, Irp->RequestorMode, (PVOID*) &pevent, NULL);

  5. ObReferenceObjectByHandle looks up hEvent in the handle table for the current process and stores the address of the associated kernel object in the pevent variable. If the RequestorMode in the IRP is UserMode, this function also verifies that hEvent really is a handle to something, that the something is an event object, and that the handle was opened in a way that includes the EVENT_MODIFY_STATE privilege.

  6. The Windows service can wait for the event to occur (This takes place in a separate thread created in ServiceMain)

    WaitForSingleObject(hEvent, INFINITE);

  7. The driver signals the event in the usual way
    KeSetEvent(pevent, EVENT_INCREMENT, FALSE);

  8. Eventually, the application cleans up by calling CloseHandle. The driver has a separate reference to the event object, which it must release by calling ObDereferenceObject. The object manager won’t destroy the event object until both these things occur.

That pretty much describes one way to communicate between user and kernel-mode. Two pieces of advice in regards to driver development:

  1. Wrap every entry point in your code
    1. DLL entry points
    2. Driver entry points
    3. Around all accesses to buffers passed to the driver
  2. Never trust a pointer
  3. Take care to protect data structures from thread conflicts
  4. Unload drivers properly and cancel any pending operations
  5. Test your drivers on a virtual machine, please!!!!

if you don’t do driver development the right way then you’ll see quite a few images like the one depicted below

image

My ramblings on computers and architecture