Showing posts with label xUnit. Show all posts
Showing posts with label xUnit. Show all posts

Sunday, February 26, 2017

Run .NET Core xUnit tests from ReSharper in VS2015

Visual Studio 2017 is literally only a few days away from release; so it might be a little late, but I finally figured out how to run .NET Core xUnit tests from ReSharper in VS2015! Good News: If you can't upgrade to VS2017 right away, then at least you can still run your unit tests!

Just make sure that the following is included in your project.json file (with the appropriate runtime):

{
  "testRunner": "xunit",
 
  "dependencies": {
    "dotnet-test-xunit": "2.2.0-preview2-build1029",
    "xunit": "2.2.0"
  },
 
  "frameworks": {
    "netcoreapp1.0": {
      "imports": "dnxcore50"
    }
  },
 
  "runtimes": {
    "win10-x64": {}
  }
}

Enjoy,
Tom

Sunday, April 24, 2016

How to Order xUnit Tests and Collections

xUnit is an extremely extensible unit testing framework!

If you need to control the order of your unit tests, then all you have to do is implement an ITestCaseOrderer. Once implemented, you just add a TestCaseOrdererAttribute to the top of your test class to use it. Below we use a custom OrderAttribute to order the tests.

To control the order of the test collections you can do a very similar trick by implementing an ITestCollectionOrderer. However, an ITestCollection is not neccessarily associated with a specific class, so to to use attributes to order them you need to use a little reflection. Check out the sample below for details.

Implementation

/// <summary>
/// Used by CustomOrderer
/// </summary>
public class OrderAttribute : Attribute
{
    public int I { get; }
 
    public OrderAttribute(int i)
    {
        I = i;
    }
}
 
/// <summary>
/// Custom xUnit test collection orderer that uses the OrderAttribute
/// </summary>
public class CustomTestCollectionOrderer : ITestCollectionOrderer
{
    public const string TypeName = "xUnitCustom.CustomTestCollectionOrderer";
 
    public const string AssembyName = "xUnitCustom";
 
    public IEnumerable<ITestCollection> OrderTestCollections(
        IEnumerable<ITestCollection> testCollections)
    {
        return testCollections.OrderBy(GetOrder);
    }
 
    /// <summary>
    /// Test collections are not bound to a specific class, however they
    /// are named by default with the type name as a suffix. We try to
    /// get the class name from the DisplayName and then use reflection to
    /// find the class and OrderAttribute.
    /// </summary>
    private static int GetOrder(
        ITestCollection testCollection)
    {
        var i = testCollection.DisplayName.LastIndexOf(' ');
        if (i <= -1)
            return 0;
 
        var className = testCollection.DisplayName.Substring(i + 1);
        var type = Type.GetType(className);
        if (type == null)
            return 0;
 
        var attr = type.GetCustomAttribute<OrderAttribute>();
        return attr?.I ?? 0;
    }
}

Thursday, March 31, 2016

xUnit.net: Extensions Config v3.3.0

I recently made several updates to the xunit.extensions.config library, which allows you to configure theory data from your app.config file. Here are the links to the source and the NuGet package:

New Features

  • Named Parameter Support

You no longer need to configure your data by parameter index. You can now name your data for each parameter, making the configuration much easier to read and understand.

  • AppSettings Support

You can now use the standard AppSettings section of the App.config to configure your data. If no settings are found, then the framework will fallback to trying to use the standard config section.

  • Default Namespace Option

You can now provide a default namespace for your tests. This reduced the amount of redundant text in your config file, and makes test names much more concise and easy to read.

  • Extensible Data Provider

Don't want to use the existing data providers? Would you rather use a database? Now you can! Just add an AppSettings key for "TestData.ServiceFactory" that provides the fully qualified name of a static method that returns an IConfigTestDataService, and the framework will try to use that to load configuration data.

Wednesday, June 24, 2015

Capture xUnit Test Output with NLog and Common Logging

I recently blogged about How To Capture Test Output in xUnit 2.0. This is great, but how can we pass the ITestOutputHelper into our code to capture log output?

You could just wrap the xUnit helper in an ILog or ILogger, but we can also take it a step further and get all of the NLog features too! By creating an NLog target that wraps the ITestOutputHelper we can enable ourselves to use multiple targets, layouts, variables, verbosity levels, and more.

Sample Unit Test

public class NLogTests : IDisposable
{
    private readonly ILogger _logger;
 
    public NLogTests(ITestOutputHelper outputHelper)
    {
        _logger = outputHelper.GetNLogLogger();
    }
 
    public void Dispose()
    {
        _logger.RemoveTestOutputHelper();
    }
 
    [Fact]
    public void Hello()
    {
        _logger.Trace("World Trace");
        _logger.Debug("World Debug");
        _logger.Warn("World Warn");
        _logger.Error("World Error");
    }
}

Saturday, May 23, 2015

How To Capture Test Output in xUnit 2.0

As of xUnit 2.0 the test framework no longer captures any native outputs, this is by design. Now the question is: where do you write your test output?

You now have to write all test output to an interface, ITestOutputHelper, that is injected into your test's constructor. This design seems to have polarized developers a bit, as not everyone enjoys being require to add a constructor to your test classes. Personally, as a huge fan of dependency injection, I really like this solution.

Spoilers: My next blog post will be about how to combine this with Common.Logging and NLog.

Sample Test

using System.Threading.Tasks;
using Xunit;
using Xunit.Abstractions;
 
namespace XUnitDemo
{
    public class OutputTests
    {
        private readonly ITestOutputHelper _testOutput;
 
        public OutputTests(ITestOutputHelper testOutput)
        {
            _testOutput = testOutput;
        }
 
        [Fact]
        public async Task WriteLine()
        {
            _testOutput.WriteLine("Hello");
            _testOutput.WriteLine("World");
 
            await Task.Delay(1000);
            
            _testOutput.WriteLine("Goodnight");
            _testOutput.WriteLine("Moon");
        }
    }
}

Enjoy,
Tom

Thursday, April 9, 2015

xUnit.net: Extensions Config v2.0

Last year I open sourced some code that allowed you to power your xUnit theories from a custom section in your application configuration file. I have now updated that project to support xUnit 2.0, and also to allow for an optional name attribute to be set on each data set.

<testData>
  <tests>
    <add name="SampleProject.Class1.Main">
      <data>
        <add index="0" name="Optional" p0="Hello" p1="World" />
        <add index="1" name="Cows" p0="Goodnight" p1="Moon" />
      </data>
    </add>
  </tests>
</testData>

Enjoy,
Tom

Thursday, March 26, 2015

xUnit 2.0 has been Released!

It has been a long time coming, but xUnit 2.0 is finally here! My favorite new features:

  • Integrated Theory Support - Theories are one of my favorite xUnit features, and I could not be more excited that they are now a first class citizen of the framework!
  • Parallelism - I have not used it yet, but I am extremely excited at the prospect of quicker test runs on my build server.
  • Assert.ThrowsAsync - I write a lot of async code these days, and now I can write my tests to be async as well!
  • Migration to GitHub - Because Git!

I'm so excited,
Tom

PS: I have already updated my xUnit configuration project to support 2.0.

Sunday, September 28, 2014

xUnit Theory Data from Configuration

I've said it before and I'll say it again, I love xUnit!

In particular, I love xUnits support for data driven tests. It offers several different options for ways to power a data driven unit test right out of the box. Best of all, xUnit allows for easy extensibility.

I have written some simple extensions for xUnit that allow you to power your data driven tests from your configuration file. Not only that, but it allows you to optionally provide default data using inline attributes when no configuration is available.

Sample Config

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <configSections>
    <section name="testData" 
type="Xunit.Extensions.Configuration.TestDataSection,xunit.extensions.config"/>
  </configSections>
  <testData>
    <tests>
      <add name="Demo.Tests.Basic">
        <data>
          <add index="0" p0="1" />
        </data>
      </add>
      <add name="Demo.Tests.FromConfig">
        <data>
          <add index="0" p0="4" />
        </data>
      </add>
    </tests>
  </testData>
</configuration>

Sample Tests

namespace Demo
{
    public class Tests
    {
        [Theory]
        [ConfigData]
        public void Basic(int i)
        {
            // This theory data comes from the config file.
            Assert.Equal(1, i);
        }
 
        [Theory]
        [ConfigOrInlineData(2)]
        public void FromInline(int i)
        {
            // This theory data comes from the attribute.
            Assert.Equal(2, i);
        }
 
        [Theory]
        [ConfigOrInlineData(3)]
        public void FromConfig(int i)
        {
            // This theory data comes from the config file
            // instead of the attribute.
            Assert.Equal(4, i);
        }
    }
}

Enjoy,
Tom

Sunday, August 24, 2014

xUnit Console Runner - Filter by Test Name

You can now filter by test name with the xUnit Console Runner.

Sample Code

namespace DemoProject
{
    public class ExampleTests
    {
        [Fact]
        public void HelloWorld()
        {
            Assert.True(true);
        }
 
        [Fact]
        public void GoodnightMoon()
        {
            Assert.True(false);
        }
    }
}

Sample Command Line

C:\>xunit.console.exe DemoProject.dll -testName "DemoProject.ExampleTests.HelloWorld"
xUnit.net console test runner (64-bit .NET 4.0.30319.18449)
Copyright (C) 2014 Outercurve Foundation.

Starting:  DemoProject.dll
Finished: DemoProject.dll

=== TEST EXECUTION SUMMARY ===
   DemoProject.dll  Total: 1, Failed: 0, Skipped: 0, Time: 0.276s, Errors: 0

Enjoy,
Tom

Sunday, July 13, 2014

Use RavenDB to power Data Driven xUnit Theories

I love xUnit's data driven unit tests, I also really enjoy working with RavenDB, and now I can use them together!

Data driven unit tests are very powerful tools that allow you to execute the same test code against multiple data sets. Testing frameworks such as xUnit makes this extremely easy to develop by offering an out of the box set attributes to quickly and easily annotate your test methods with dynamic data sources.

Below is some simple code that adds a RavenDataAttribute to xUnit. This attribute will pull arguments from a document database and pass them into your unit test, using the fully qualified method name as a key.

Example Unit Tests

public class RavenDataTests
{
    [Theory]
    [RavenData]
    public void PrimitiveArgs(int number, bool isDivisibleBytwo)
    {
        var remainder = number % 2;
        Assert.Equal(isDivisibleBytwo, remainder == 0);
    }
 
    [Theory]
    [RavenData]
    public void ComplexArgs(ComplexArgsModel model)
    {
        var remainder = model.Number % 2;
        Assert.Equal(model.IsDivisibleByTwo, remainder == 0);
    }
 
    [Fact(Skip = "Only run once for setup")]
    public void Setup()
    {
        var type = typeof(RavenDataTests);
 
        var primitiveArgsMethod = type.GetMethod("PrimitiveArgs");
        var primitiveArgs = new object[] { 3, false };
        RavenDataAttribute.SaveData(primitiveArgsMethod, primitiveArgs);
 
        var complexArgsMethod = type.GetMethod("ComplexArgs");
        var complexArgsModel = new ComplexArgsModel
        {
            IsDivisibleByTwo = true,
            Number = 4
        };
        RavenDataAttribute.SaveData(complexArgsMethod, complexArgsModel);
    }
 
    public class ComplexArgsModel
    {
        public int Number { get; set; }
        public bool IsDivisibleByTwo { get; set; }
    }
}

Monday, November 18, 2013

XUnit.PhantomQ v.1.2 Released

Want to run client side QUnit tests from Visual Studio or your build server? Now it is easier than ever; just grab newly updated XUnit.PhantomQ v1.2 from NuGet!

XUnit.PhantomQ will allow you to execute your QUnit tests as XUnit tests. It supports both library and web projects, and features the ability to easily specify test files and their dependencies by real relative path from the root of your project.

XUnit.PhantomQ on NuGet
XUnit.PhantomQ Source on GitHub

Change Log for v1.2

My thanks to James M Greene and the other authors of the PhantomJS Runner; their work served as the model for this version's improved test result information.

  • Significantly improved test result information and error details.
  • Added console.log support.
  • Added test timeout configuration support.
  • Added QUnit module support.
  • Added QUnit result details to QUnitTest.Context

Sunday, October 20, 2013

Unit Testing and Dependency Injection, with xUnit InlineData and Unity

Inversion of control is great because it makes your code more testable; but you usually still have to write tests for each implementation of your interfaces. So what if your unit testing framework could just work directly with your container to make testing even easier? Well, xUnit can!

Below we define a custom data source for our xUnit theories by extending the InlineDataAttribute. This allows our data driven unit tests to resolve types via a container, and then inject those resolved objects straight into our unit tests.

Bottom line: This allows us to test more with less code!

The rest of post is very code heavy, so I strongly recommend that you start out by taking a look at sections 1 and 2 to get an idea of what we are trying to accomplish. :)

  1. Example Interfaces and Classes
  2. Example Unit Tests
  3. IocInlineDataResolver
  4. UnityInlineDataAttribute

Saturday, August 24, 2013

XUnit.PhantomQ v1.1

I recently blogged about how to Use XUnit to Run QUnit Tests. The initial v1.0 release of XUnit.PhantomQ did not support error messages, but now in v1.1 it supports the must have feature of bringing error messages back with failed test results.

XUnit.PhantomQ on NuGet
XUnit.PhantomQ Source on GitHub

Enjoy,
Tom

Sunday, July 14, 2013

Use XUnit to Run QUnit Tests

I read a great article recently about Unit testing JavaScript in VisualStudio with ReSharper, written by Chris Seroka. As cool as this feature is, it left me with two questions:

  1. What about developers that do not have ReSharper?
  2. How do I run my JavaScript unit tests on a builder server?

It is no secret that I, absolutely, love, xUnit! Thus I decided to extend xUnit theories to be able to run QUnit tests by implementing a new DataAttribute.

Introducing XUnit.PhantomQ

XUnit.PhantomQ is a little NuGet package you can install to get access to the QUnitDataAttribute (see below for an example). This library will allow you to execute your QUnit tests as XUnit tests.

XUnit.PhantomQ supports both library and web projects, and features the ability to easily specify test files and their dependencies by real relative path to the root of your project.

XUnit.PhantomQ on NuGet
XUnit.PhantomQ Source on GitHub

QUnitData Attribute

Here is an example of writing some JavaScript, a file of QUnit tests, and then using an xUnit theory and a QUnitData Attribute to execute all of those tests right inside of Visual Studio.

// Contents of Demo.js
function getFive() {
    return 5;
}
 
// Contents of Tests.js
test('Test Five', function() {
    var actual = getFive();
    equal(actual, 5);
});
test('Test Not Four', function () {
    var actual = getFive();
    notEqual(actual, 4);
});
 
// Contents of QUnitTests.cs
public class QUnitTests
{
    [Theory, QUnitData("Tests.js", "Demo.js")]
    public void ReturnFiveTests(QUnitTest test)
    {
        test.AssertSuccess();
    }
}
 

Integrating XUnit, PhantomJS, and QUnit

So, how does this thing work under the hood? Below is the complete pipeline, step by step, of whenever the tests are executed:

  1. The XUnit test runner identifies your theory tests.
  2. The QUnitDataAttribute is invoked.
  3. A static helper locates PhantomJS.exe and the root folder of your project.
    • It will automatically walk up the folder structure and try to find PhantomJS.exe in your packages folder. However, you can override this and explicitly set the full path by adding an AppSetting to your config file:
      <add key="PhantomQ.PhantomJsExePath" value="C:/PhantomJS.exe" />
    • The same goes for the root of your project, you can override this location with another AppSetting to your config:
      <add key="PhantomQ.WorkingDirectory" value="C:/Code/DemoProject" />
  4. PhantomJS.exe is invoked as a separate process.
  5. The server loads up XUnit.PhantomQ.Server.js
  6. The server now opens XUnit.PhantomQ.html
  7. QUnit is set to autoRun false.
  8. Event handlers are added to QUnit for testDone and done.
  9. All of the dependencies are now added to the page as script tags.
  10. The test file itself is loaded.
  11. QUnit.start is invoked.
  12. The server waits for the test to complete.
  13. Upon completion the server reads the results out of the page.
  14. The tests and their results are serialized to a JSON dictionary.
  15. The result dictionary is written to the standard output.
  16. The resulting JSON string is read in from the process output.
  17. The results are deserialized using Newtonsoft.JSON
  18. The results are loaded into QUnitTest objects.
  19. The QUnitTest array is passed back from the DataAttribute
  20. Each test run finally calls the AssertSuccess and throws on failure.

...and that's (finally) all folks! If you have any questions, comments, or thoughts on how this could be improved, please let me know!

Shout it

Enjoy,
Tom

Friday, October 26, 2012

xUnit Visual Studio Integration

Good news, everyone! It is actually very easy to get xUnit completely integrated with Visual Studio. You only need to install two plugins...

VS2010 - xUnit Test Runner Extension

This will support running tests with a Visual Studio test project.
This includes all of the VS features, such as code coverage!

https://github.com/quetzalcoatl/xvsr10/downloads

ReSharper - xUnit Contrib Plugin

This will allow ReSharper to detect and run xUnit tests.

http://xunitcontrib.codeplex.com/releases/view/92101
(If you are still running Resharper 6, then you will need the latest: v6.1.1)

Team Build (TFS) Integration

Integrating with xUnit your Team Foundation Server is a very tricky proposition, but it can be done. That, however, is a (rather long) blog post for another day!

Shout it

Enjoy,
Tom

Thursday, August 30, 2012

WebDrivers in Parallel

Here is a post that is long over due, and is finally getting published by request!

Concurrent WebDrivers

A WebDriver is not thread safe, but it is not required to be a singleton either. If you instantiate multiple drivers, they can all be run at the same time. If you cast a series of drivers into a collection of the IWebDriver interface, and then throw that into a Parallel ForEach, you have yourself one fun toy to play with!

So what are some uses of this?

Admittedly it is not a particularly common or practical use case to have multiple automated browser sessions running at the same time, but it can still come in handy. One fun application is to test website concurrency. You can do a minor load test of a webpage, or your can have two sessions trying to race each other for a limited resource.

...also, running WebDrivers in parallel can make for a killer demo!

Sample Code (xUnit Test)

public class ParallelDemo : IDisposable
{
    public IList<IWebDriver> Drivers { get; private set; }
 
    public ParallelDemo()
    {
        Drivers = new List<IWebDriver>
        {
            new InternetExplorerDriver(),
            new FirefoxDriver(),
            new ChromeDriver()
        };
    }
 
    public void Dispose()
    {
        foreach (var driver in Drivers)
        {
            driver.Close();
            driver.Dispose();
        }
    }
 
    [Fact]
    public void ParallelSearch()
    {
        Parallel.ForEach(Drivers, SearchForTom);
    }
 
    private static void SearchForTom(IWebDriver driver)
    {
        driver.Url = "https://www.google.com/#q=tom+dupont";
        var firstResult = driver.FindElement(By.CssSelector("h3 > a.l"));
        Assert.Contains("Tom DuPont .NET", firstResult.Text);
    }
}
Shout it

Enjoy,
Tom

Saturday, April 14, 2012

xUnit Theory, the Data Driven Unit Test

Update: I have also written a post about NUnit's Data Driven TestCaseAttribute.

Do you like copying and pasting code? Neither do I.

A good set of unit tests often end up reusing the same code with varied inputs. Rather than copy and paste that test code over and over, we can use the pattern of data driven unit tests to help streamline our test fixtures. This is the practice of having a single test definition be invoked and count as multiple tests at run time. This also enables us to do other dynamic things, such as configuring our unit tests from external sources. :)

I frequently use MSTest, but it's data driven tests inconveniently require you to define a DataSource. (Updated) Come to find out NUnit does offer data driven unit tests with their TestCaseSource attribute. Meanwhile xUnit offers several lightweight and simple options for defining data driven tests, which it refers to as theories.

Let's take a look at some of xUnit's Theory data sources:

InlineData Example

public class StringTests1
{
    [Theory,
    InlineData("goodnight moon", "moon", true),
    InlineData("hello world", "hi", false)]
    public void Contains(string input, string sub, bool expected)
    {
        var actual = input.Contains(sub);
        Assert.Equal(expected, actual);
    }
}

PropertyData Example

public class StringTests2
{
    [Theory, PropertyData("SplitCountData")]
    public void SplitCount(string input, int expectedCount)
    {
        var actualCount = input.Split(' ').Count();
        Assert.Equal(expectedCount, actualCount);
    }
 
    public static IEnumerable<object[]> SplitCountData
    {
        get
        {
            // Or this could read from a file. :)
            return new[]
            {
                new object[] { "xUnit", 1 },
                new object[] { "is fun", 2 },
                new object[] { "to test with", 3 }
            };
        }
    }
}

ClassData Example

public class StringTests3
{
    [Theory, ClassData(typeof(IndexOfData))]
    public void IndexOf(string input, char letter, int expected)
    {
        var actual = input.IndexOf(letter);
        Assert.Equal(expected, actual);
    }
}
 
public class IndexOfData : IEnumerable<object[]>
{
    private readonly List<object[]> _data = new List<object[]>
    {
        new object[] { "hello world", 'w', 6 },
        new object[] { "goodnight moon", 'w', -1 }
    };
 
    public IEnumerator<object[]> GetEnumerator()
    { return _data.GetEnumerator(); }
 
    IEnumerator IEnumerable.GetEnumerator()
    { return GetEnumerator(); }
}
kick it on DotNetKicks.com

Happy testing!
Tom

Sunday, April 8, 2012

Migrating from NUnit to xUnit

I recently started using xUnit, and I have been really enjoying it!

If you are currently using NUnit to write your unit tests, then it is not at all difficult to migrate to using xUnit. The philosophical difference between the two is simply this: with xUnit you need to think of your tests as objects, rather than of methods. Or if you prefer acronyms: put some more OOP into your TDD. *bu-dum, tish!*

Anyway, here is a visual representation of equivalent commands between NUnit and xUnit:

xUnit

NUnit

[NUnit.Framework.TestFixture]
public class TestFixture
{
  [NUnit.Framework.TestFixtureSetUp]
  public void TestFixtureSetUp()
  {
    // 1) Set up test fixture  -------->
  }
  [NUnit.Framework.TestFixtureTearDown]
  public void TestFixtureTearDown()
  {
    // 8) Tear down test fixture  ----->
  }
 
 
 
  [NUnit.Framework.SetUp]
  public void SetUp()
  {
    // 2) Set up TestA  --------------->
    // 5) Set up TestB  --------------->
  }
  [NUnit.Framework.Test]
  public void TestA()
  {
    // 3) Run TestA  ------------------>
  }
  [NUnit.Framework.Test]
  public void TestB()
  {
    // 6) Run TestB.  ----------------->
  }
  [NUnit.Framework.TearDown]
  public void TearDown()
  {
    // 4) Tear down TestA  ------------>
    // 7) Tear down TestB  ------------>
  }
}
 
public class TestData : IDisposable
{
 
  public TestData()
  {
    // 1) Set up test fixture
  }
 
  public void Dispose()
  {
    // 8) Tear down test fixture
  }
}
public class TestClass
  : IDisposable, Xunit.IUseFixture
{
  public TestClass()
  {
    // 2) Set up TestA
    // 5) Set up TestB
  }
  [Xunit.Fact]
  public void TestA()
  {
    // 3) Run TestA
  }
  [Xunit.Fact]
  public void TestB()
  {
    // 6) Run TestB
  }
 
  public void Dispose()
  {
    // 4) Tear down TestA
    // 7) Tear down TestB
  }
  public TestData TestData { get; set; }
  public void SetFixture(TestData data)
  {
    // 2.5) Set fixture data for TestA
    // 5.5) Set fixture data for TestB
    TestData = data;
  }
}
kick it on DotNetKicks.com

Happy testing!
~Tom

Real Time Web Analytics