.NET Mocking Framework Survey Results

Two weeks ago, I posted a survey (using Excel Surveys available in SkyDrive) asking you what your preferred/primary .NET mocking framework was. We plan to use this data to help drive out which frameworks we want to evaluate further.

The survey closed yesterday and, out of 147 responses, here is how everything shook out:

147 responses may not seem like a lot and, in the grand scheme of all .NET developers across the world, it’s not.  There are several ways you can look at this number:  Out of everyone that viewed the survey request, the majority of them didn’t want to take the time to give their input; they don’t use .NET-based technologies and/or mocking frameworks; I just didn’t reach a wide enough audience and/or provide enough time to get the word out about the survey; or there was some other reason they didn’t bother with the survey that I’m just not thinking of at the moment.

In regards to this penultimate point above, here was my general approach for getting the survey link out:

• Posted to my blog – just over 225 page views (as of a few moments ago) – not a huge number by any stretch
• Posted to my twitter feed – roughly 250 followers – again, not a huge number. This one is a little harder to judge because I don’t know how many times it was re-tweeted, copied and tweeted, etc.
• Posted to Google+ – don’t know the impact – likely very, very small
• Posted on my LinkedIn timeline – 279 connections
• Posted on two separate LinkedIn groups: .NET Developers & .NET Professional with .NET Developers having a membership of about 148,000
• Various e-mail distribution lists – not sure of the count

Click here to view the full survey results.

[Updated: Fixed broken survey link]

Which .NET Mocking Framework Do You Prefer?

We are preparing to evaluate mocking frameworks that support .NET to determine if our current mocking framework is still the best solution for us.  Before we actually start the evaluation, we’d like to get a general idea of which .NET mocking frameworks are popular amongst developers out in the rest of the world.

To aid in this endeavor, we have created a simple survey with two questions (or three, depending on how you answer the first question).  Essentially, the survey asks which of the following .NET mocking frameworks is your primary (or preferred) framework.  The list of frameworks is comprised of the following:

• EasyMock.NET
• FakeItEasy
• JustMock (Telerik)
• Microsoft Fakes
• Moq
• NMock2
• NMock3
• NSubstitute
• Rhino Mocks
• Simple.Mocking
• Typemock
• Other (list in comments)

If you select “Other” then the second question is there to list the name.  The third and final question is simply for any general comments you’d like to make.

We plan to keep the survey open until Tuesday, January 15th.  Once the survey has closed, I will post the results back to this blog.  So, please help us determine what the preferred mocking frameworks are that are in use today.  The survey should (literally) take less than 30 seconds.  Your contribution will be greatly appreciated!

This Survey is Now Closed.  Click Here to view the final results.

Also, please take a moment and pass the word by tweeting, blogging, etc. the above survey link (or the link to this post).  The more responses we get, the better.

Visual Studio Load Test Feature Pack

Last August, Microsoft announced they would be including the lab agents (i.e. licenses) with TFS 2010 as well as Visual Studio 2010 Ultimate with MSDN and Visual Studio 2010 Test Professional with MSDN (link).  As of today, Microsoft has made another great licensing announcement – the Visual Studio Load Test Virtual User Feature Pack 2010.

With this feature pack, you can simulate as many virtual users as you like without the need for purchasing additional virtual user licenses.  This change only adds to the already great value provided with the Visual Studio 2010 Ultimate with MSDN subscription.

More Information

You can find more information regarding this release via these links:

• Visual Studio Load Test Virtual User Feature Pack 2010
• Unlimited Load Testing (Brian Harry)
• Visual Studio 2010 enhancements

Testing Tool Enhancements Coming

Brian Harry has blogged about the forthcoming Feature Pack 2.  If you make use of Coded UI Tests and/or Microsoft Test Runner, then you will be interested in this Feature Pack.  This Feature Pack, once released, will contain the following features:

• The ability to test Silverlight 4.0 applications (currently, browser-hosted applications only – out-of-browser applications will be supported in the future).
• Playback of recorded tests in Firefox.  You still have to record the test scripts using Internet Explorer (v7.0 or later), however, you can play them back in either browser.
• Coded UI Test Editor.  This new editor will allow you to manage certain aspects of your coded UI tests easier than before.

To get the full details, read Brian’s post here.

Visual Studio 2010 Testing Tools Round Up

Yesterday, I co-presented a session covering some of the testing tools available in Visual Studio 2010.  Overall, the talk seemed to go very well.  In fact, the majority of the feedback responses were highly positive.  However, there was one negative comment – “This talk really was about Team Foundation Server.  Without this expensive application, the information provided is useless.”

This comment was, no doubt, formulated based on a statement I had made near the start of my talk.  I pointed out that Microsoft Test Manager relied on Team Foundation Server (TFS) and that it could not be utilized without it.  In hindsight, I should have clearly pointed out that the remaining test tools found within the various SKUs of Visual Studio 2010 are not reliant on TFS to be functional.  Also, although it wasn’t directly relevant to the topic at hand, I should have pointed out that TFS is now included with Visual Studio Ultimate/Test Professional (w/MSDN) – thereby possibly addressing the “…expensive application…” portion of the above statement.

To address some of the confusion around the various testing tools available within the various Visual Studio 2010 SKUs, I’ve create the following graphic (note: each of the SKUs listed below are assumed to have a corresponding MSDN subscription):

For full comparison of Visual Studio features, click here

If you’re not familiar with all of these testing features found in Visual Studio 2010, check out the links below for more details.

• Lab Management – REQUIRES TFS – Lab Management lets you manage a collection of virtual machines, templates, and virtual environments for your Team Foundation team project. You can use these environments to develop, test, or run your application.  Many of the Lab Management features can be integrated into your automated builds as well.
• Microsoft Test Manager - REQUIRES TFS – this is a brand new product with Visual Studio 2010 providing the following capabilities:
  • Create Test Plans
  • Create/Manage Test Suites
  • Create/Manage Test Configurations (e.g. Operating System, Web Browser, etc.)
  • Create/Manage Test Cases
  • Manual Testing – with the ability to record test actions
  • “Fast Forward” capabilities – the ability to automatically run through a selected set of manual test steps that have been previously recorded.
  • Lots more
• Basic Unit Tests – Unit tests give developers and testers a quick way to look for logic errors in the methods of classes in Visual C#, Visual Basic, and Visual C++ projects.  TFS is not required for unit tests to operate in Visual Studio.  However, if you have TFS, the results can be published for later review.
• Database Unit Tests – You can use database unit tests to establish a baseline state for your database and then to verify any subsequent changes that you make to database objects.  NOTE: In Visual Studio 2010 Professional, you can run database unit tests but you cannot create or modify tests in the designer.
• Generic Unit Tests - A generic test typically wraps an automated test or tool that was previously created outside the Visual Studio Team System framework. Using generic tests, therefore, is a means for taking advantage of an existing test infrastructure.
• Ordered Tests - An ordered test contains other tests (e.g. unit, web, generic, manual, and ordered tests, but not load tests) that are meant to be run in a specified order.
• Coded UI Tests - You can use Visual Studio 2010 Ultimate or Visual Studio 2010 Premium to create automated tests of the user interface (UI) known as coded UI tests.
• Load Tests - You use a load test to encapsulate non-manual tests, that is, unit, Web, generic, ordered tests, and then run them simultaneously by using virtual users. Running these tests under load generates test results, including performance and other counters, in tables and in graphs.
• Web Performance Tests - Web tests consist of an ordered series of HTTP Requests that you record in a browser session using Microsoft Internet Explorer. You can also create a coded Web test, in which you can add more advanced features, such as flow control.
• Test Impact Analysis - By using Test Impact Analysis during code development, you can identify the methods in a test project that have been affected by code changes in your managed code solution.
• Code Coverage - You can use the code coverage feature of Visual Studio Application Lifecycle Management (ALM) to determine what proportion of your project's code is actually being tested using unit tests.