Last year I open sourced some code that allowed you to power your xUnit theories from a custom section in your application configuration file. I have now updated that project to support xUnit 2.0, and also to allow for an optional name attribute to be set on each data set.
<testData>
<tests>
<add name="SampleProject.Class1.Main">
<data>
<add index="0" name="Optional" p0="Hello" p1="World" />
<add index="1" name="Cows" p0="Goodnight" p1="Moon" />
</data>
</add>
</tests>
</testData>
Enjoy,
Tom
I've said it before and I'll say it again, I love xUnit!
In particular, I love xUnits support for data driven tests. It offers several different options for ways to power a data driven unit test right out of the box. Best of all, xUnit allows for easy extensibility.
I have written some simple extensions for xUnit that allow you to power your data driven tests from your configuration file. Not only that, but it allows you to optionally provide default data using inline attributes when no configuration is available.
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<configSections>
<section name="testData"
type="Xunit.Extensions.Configuration.TestDataSection,xunit.extensions.config"/>
</configSections>
<testData>
<tests>
<add name="Demo.Tests.Basic">
<data>
<add index="0" p0="1" />
</data>
</add>
<add name="Demo.Tests.FromConfig">
<data>
<add index="0" p0="4" />
</data>
</add>
</tests>
</testData>
</configuration>
namespace Demo
{
public class Tests
{
[Theory]
[ConfigData]
public void Basic(int i)
{
// This theory data comes from the config file.
Assert.Equal(1, i);
}
[Theory]
[ConfigOrInlineData(2)]
public void FromInline(int i)
{
// This theory data comes from the attribute.
Assert.Equal(2, i);
}
[Theory]
[ConfigOrInlineData(3)]
public void FromConfig(int i)
{
// This theory data comes from the config file
// instead of the attribute.
Assert.Equal(4, i);
}
}
}
Enjoy,Tom
I love xUnit's data driven unit tests, I also really enjoy working with RavenDB, and now I can use them together!
Data driven unit tests are very powerful tools that allow you to execute the same test code against multiple data sets. Testing frameworks such as xUnit makes this extremely easy to develop by offering an out of the box set attributes to quickly and easily annotate your test methods with dynamic data sources.
Below is some simple code that adds a RavenDataAttribute to xUnit. This attribute will pull arguments from a document database and pass them into your unit test, using the fully qualified method name as a key.
public class RavenDataTests
{
[Theory]
[RavenData]
public void PrimitiveArgs(int number, bool isDivisibleBytwo)
{
var remainder = number % 2;
Assert.Equal(isDivisibleBytwo, remainder == 0);
}
[Theory]
[RavenData]
public void ComplexArgs(ComplexArgsModel model)
{
var remainder = model.Number % 2;
Assert.Equal(model.IsDivisibleByTwo, remainder == 0);
}
[Fact(Skip = "Only run once for setup")]
public void Setup()
{
var type = typeof(RavenDataTests);
var primitiveArgsMethod = type.GetMethod("PrimitiveArgs");
var primitiveArgs = new object[] { 3, false };
RavenDataAttribute.SaveData(primitiveArgsMethod, primitiveArgs);
var complexArgsMethod = type.GetMethod("ComplexArgs");
var complexArgsModel = new ComplexArgsModel
{
IsDivisibleByTwo = true,
Number = 4
};
RavenDataAttribute.SaveData(complexArgsMethod, complexArgsModel);
}
public class ComplexArgsModel
{
public int Number { get; set; }
public bool IsDivisibleByTwo { get; set; }
}
}
A while back I wrote a blog post about data driven unit testing with xUnit. Back then a reader had to correct me because I did not think that NUnit had support for such things.
NUnit 2.5 added a slew of great features for authoring your own data driven unit tests. Perhaps best of all is the amazing support that ReSharper offers for the NUnit test cases.
You really should be using these amazing features when authoring your unit tests!
// Here is a simple example that is the equivalent of an
// inline data attribute from xUnit.
[TestCase(1, 2, 3)]
[TestCase(2, 3, 5)]
public void SimpleSumCase(int a, int b, int expected)
{
var actual = a + b;
Assert.AreEqual(expected, actual);
}
Mocking is a necessary evil for unit testing.
Fortunately frameworks like NSubstitute make it painless setup your mock services. NSubstitute offers a fluent API that requires few lambdas and no calls to an Object property. You just get back the interface that you are substituting and work with it directly. Frankly, NSubstitute is so easy to work with that it almost seems like magic!
Below is a visual representation of equivalent commands between Moq and NSubstitute:
[Fact]
public void NSubstitute()
{
var tester = Substitute.For<ITester>();
--------------------------------------------
var voidArg = String.Empty;
tester
.When(t => t.Void(Arg.Any<string>()))
.Do(i => voidArg = i.Arg<string>());
tester.Void("A");
Assert.Equal("A", voidArg);
--------------------------------------------
tester
.Bool()
.Returns(true);
var boolResult = tester.Bool();
Assert.Equal(true, boolResult);
--------------------------------------------
tester
.Int
.Returns(1);
var intResult = tester.Int;
Assert.Equal(1, intResult);
--------------------------------------------
tester.Received(1).Void("A");
--------------------------------------------
tester.DidNotReceive().Void("B");
}
[Fact]
public void Moq()
{
var tester = new Mock<ITester>();
// Setup a callback for a void method. -------
var voidArg = String.Empty;
tester
.Setup(t => t.Void(It.IsAny<string>()))
.Callback<string>(s => voidArg = s);
tester.Object.Void("A");
Assert.Equal("A", voidArg);
// Setup the result of a method. -------------
tester
.Setup(t => t.Bool())
.Returns(true);
var boolResult = tester.Object.Bool();
Assert.Equal(true, boolResult);
// Setup the result of a property. -----------
tester
.SetupGet(t => t.Int)
.Returns(1);
var intResult = tester.Object.Int;
Assert.Equal(1, intResult);
// Ensure that a function was called. --------
tester.Verify(m => m.Void("A"), Times.Once);
// Ensure that a function was NOT called. ----
tester.Verify(m => m.Void("B"), Times.Never);
}
Enjoy,
Tom
Want to run client side QUnit tests from Visual Studio or your build server? Now it is easier than ever; just grab newly updated XUnit.PhantomQ v1.2 from NuGet!
XUnit.PhantomQ will allow you to execute your QUnit tests as XUnit tests. It supports both library and web projects, and features the ability to easily specify test files and their dependencies by real relative path from the root of your project.
XUnit.PhantomQ on NuGet
XUnit.PhantomQ Source on GitHub
My thanks to James M Greene and the other authors of the PhantomJS Runner; their work served as the model for this version's improved test result information.
Inversion of control is great because it makes your code more testable; but you usually still have to write tests for each implementation of your interfaces. So what if your unit testing framework could just work directly with your container to make testing even easier? Well, xUnit can!
Below we define a custom data source for our xUnit theories by extending the InlineDataAttribute. This allows our data driven unit tests to resolve types via a container, and then inject those resolved objects straight into our unit tests.
Bottom line: This allows us to test more with less code!
The rest of post is very code heavy, so I strongly recommend that you start out by taking a look at sections 1 and 2 to get an idea of what we are trying to accomplish. :)
I recently blogged about how to Use XUnit to Run QUnit Tests. The initial v1.0 release of XUnit.PhantomQ did not support error messages, but now in v1.1 it supports the must have feature of bringing error messages back with failed test results.
XUnit.PhantomQ on NuGet
XUnit.PhantomQ Source on GitHub
Enjoy,
Tom
Did you know that Selenium already supports PhantomJS?
WebDriver is a specification for controlling the behavior of a web browser. PhantomJS is a headless WebKit scriptable with a JavaScript API. Ghost Driver is a WebDriver implementation that uses PhantomJS for its back-end. Selenium is a software testing framework for web applications. Selenium WebDriver is the successor to Selenium RC. The Selenium WebDriver NuGet Package is a .NET client for for Selenium WebDriver that includes support for PhantomJs via GhostDriver.
You need only install two NuGet packages in order to use PhantomJS with WebDriver. You will probably also want which ever Unit Testing framework you prefer. As always, I suggest xUnit.
After installing those, using the PhantomJSDriver is as easy as any other WebDriver!
const string PhantomDirectory =
@"..\..\..\packages\phantomjs.exe.1.8.1\tools\phantomjs";
[Fact]
public void GoogleTitle()
{
using (IWebDriver phantomDriver = new PhantomJSDriver(PhantomDirectory))
{
phantomDriver.Url = "http://www.google.com/";
Assert.Contains("Google", phantomDriver.Title);
}
}
Enjoy,
Tom
I read a great article recently about Unit testing JavaScript in VisualStudio with ReSharper, written by Chris Seroka. As cool as this feature is, it left me with two questions:
It is no secret that I, absolutely, love, xUnit! Thus I decided to extend xUnit theories to be able to run QUnit tests by implementing a new DataAttribute.
XUnit.PhantomQ is a little NuGet package you can install to get access to the QUnitDataAttribute (see below for an example). This library will allow you to execute your QUnit tests as XUnit tests.
XUnit.PhantomQ supports both library and web projects, and features the ability to easily specify test files and their dependencies by real relative path to the root of your project.
XUnit.PhantomQ on NuGet
XUnit.PhantomQ Source on GitHub
Here is an example of writing some JavaScript, a file of QUnit tests, and then using an xUnit theory and a QUnitData Attribute to execute all of those tests right inside of Visual Studio.
// Contents of Demo.js
function getFive() {
return 5;
}
// Contents of Tests.js
test('Test Five', function() {
var actual = getFive();
equal(actual, 5);
});
test('Test Not Four', function () {
var actual = getFive();
notEqual(actual, 4);
});
// Contents of QUnitTests.cs
public class QUnitTests
{
[Theory, QUnitData("Tests.js", "Demo.js")]
public void ReturnFiveTests(QUnitTest test)
{
test.AssertSuccess();
}
}
So, how does this thing work under the hood? Below is the complete pipeline, step by step, of whenever the tests are executed:
...and that's (finally) all folks! If you have any questions, comments, or thoughts on how this could be improved, please let me know!
Enjoy,
Tom
What if you could spin up an entire database in memory for every unit test? You can!
RavenDB offers an EmbeddableDocumentStore NuGet Package that allows you to create a complete in memory instance of RavenDB. This makes writing integration tests for your custom Indexes extremely easy.
The Hibernating Rhinos team makes full use of this feature by including a full suite of unit tests in their RavenDB solution. They even encourage people to submit pull requests to their GitHub repository so that they can pull those tests directly into their source. This is a BRILLIANT integration of all these technologies to both encourage testing and provide an extremely stable product.
So then, how do you test your RavenDB indexes? Great question; let's get into the code!
public class Doc
{
public int InternalId { get; set; }
public string Text { get; set; }
}
public class Index : AbstractIndexCreationTask<Doc>
{
public Index()
{
Map = docs => from doc in docs
select new
{
doc.InternalId,
doc.Text
};
Analyzers.Add(d => d.Text, "Raven.Extensions.AlphanumericAnalyzer");
}
}
In this example I am creating an abstract base class for my unit tests. The GetDocumentStore method provides an EmbeddableDocumentStore that comes pre-initialized with the default RavenDocumentsByEntityName index, your custom index, and a complete set of documents that have already been inserted. The Documents come from an abstract Documents property, which we will see implemented below in step 5.
protected abstract ICollection<TDoc> Documents { get; }
private EmbeddableDocumentStore NewDocumentStore()
{
var documentStore = new EmbeddableDocumentStore
{
Configuration =
{
RunInUnreliableYetFastModeThatIsNotSuitableForProduction = true,
RunInMemory = true
}
};
documentStore.Initialize();
// Create Default Index
var defaultIndex = new RavenDocumentsByEntityName();
defaultIndex.Execute(documentStore);
// Create Custom Index
var customIndex = new TIndex();
customIndex.Execute(documentStore);
// Insert Documents from Abstract Property
using (var bulkInsert = documentStore.BulkInsert())
foreach (var document in Documents)
bulkInsert.Store(document);
return documentStore;
}
These tests are testing a custom Alphanumeric analyzer. They will take in a series of lucene queries and assert that they match the correct internal Ids. These documents are being defined by our abstract Documents property from Step 4.
NOTE: Do not forget to include the WaitForNonStaleResults method on your queries, as your index may not be done building the first time you run your tests.
[Theory]
[InlineData(@"Text:Hello", new[] {0})]
[InlineData(@"Text:file_name", new[] {2})]
[InlineData(@"Text:name*", new[] {2, 3})]
[InlineData(@"Text:my AND Text:txt", new[] {2, 3})]
public void Query(string query, int[] expectedIds)
{
int[] actualIds;
using (var documentStore = NewDocumentStore())
using (var session = documentStore.OpenSession())
{
actualIds = session.Advanced
.LuceneQuery<Doc>("Index")
.Where(query)
.SelectFields<int>("InternalId")
.WaitForNonStaleResults()
.ToArray();
}
Assert.Equal(expectedIds, actualIds);
}
protected override ICollection<Doc> Documents
{
get
{
return new[]
{
"Hello, world!",
"Goodnight...moon?",
"my_file_name_01.txt",
"my_file_name01.txt"
}
.Select((t, i) => new Doc
{
InternalId = i,
Text = t
})
.ToArray();
}
}
Enjoy,
Tom
Update: I have also written a post about NUnit's Data Driven TestCaseAttribute.
Do you like copying and pasting code? Neither do I.
A good set of unit tests often end up reusing the same code with varied inputs. Rather than copy and paste that test code over and over, we can use the pattern of data driven unit tests to help streamline our test fixtures. This is the practice of having a single test definition be invoked and count as multiple tests at run time. This also enables us to do other dynamic things, such as configuring our unit tests from external sources. :)
I frequently use MSTest, but it's data driven tests inconveniently require you to define a DataSource. (Updated) Come to find out NUnit does offer data driven unit tests with their TestCaseSource attribute. Meanwhile xUnit offers several lightweight and simple options for defining data driven tests, which it refers to as theories.
Let's take a look at some of xUnit's Theory data sources:
InlineData Example
public class StringTests1
{
[Theory,
InlineData("goodnight moon", "moon", true),
InlineData("hello world", "hi", false)]
public void Contains(string input, string sub, bool expected)
{
var actual = input.Contains(sub);
Assert.Equal(expected, actual);
}
}
PropertyData Example
public class StringTests2
{
[Theory, PropertyData("SplitCountData")]
public void SplitCount(string input, int expectedCount)
{
var actualCount = input.Split(' ').Count();
Assert.Equal(expectedCount, actualCount);
}
public static IEnumerable<object[]> SplitCountData
{
get
{
// Or this could read from a file. :)
return new[]
{
new object[] { "xUnit", 1 },
new object[] { "is fun", 2 },
new object[] { "to test with", 3 }
};
}
}
}
ClassData Example
public class StringTests3
{
[Theory, ClassData(typeof(IndexOfData))]
public void IndexOf(string input, char letter, int expected)
{
var actual = input.IndexOf(letter);
Assert.Equal(expected, actual);
}
}
public class IndexOfData : IEnumerable<object[]>
{
private readonly List<object[]> _data = new List<object[]>
{
new object[] { "hello world", 'w', 6 },
new object[] { "goodnight moon", 'w', -1 }
};
public IEnumerator<object[]> GetEnumerator()
{ return _data.GetEnumerator(); }
IEnumerator IEnumerable.GetEnumerator()
{ return GetEnumerator(); }
}
Happy testing!
Tom
I recently started using xUnit, and I have been really enjoying it!
If you are currently using NUnit to write your unit tests, then it is not at all difficult to migrate to using xUnit. The philosophical difference between the two is simply this: with xUnit you need to think of your tests as objects, rather than of methods. Or if you prefer acronyms: put some more OOP into your TDD. *bu-dum, tish!*
Anyway, here is a visual representation of equivalent commands between NUnit and xUnit:
xUnit
NUnit
[NUnit.Framework.TestFixture]
public class TestFixture
{
[NUnit.Framework.TestFixtureSetUp]
public void TestFixtureSetUp()
{
// 1) Set up test fixture -------->
}
[NUnit.Framework.TestFixtureTearDown]
public void TestFixtureTearDown()
{
// 8) Tear down test fixture ----->
}
[NUnit.Framework.SetUp]
public void SetUp()
{
// 2) Set up TestA --------------->
// 5) Set up TestB --------------->
}
[NUnit.Framework.Test]
public void TestA()
{
// 3) Run TestA ------------------>
}
[NUnit.Framework.Test]
public void TestB()
{
// 6) Run TestB. ----------------->
}
[NUnit.Framework.TearDown]
public void TearDown()
{
// 4) Tear down TestA ------------>
// 7) Tear down TestB ------------>
}
}
public class TestData : IDisposable
{
public TestData()
{
// 1) Set up test fixture
}
public void Dispose()
{
// 8) Tear down test fixture
}
}
public class TestClass
: IDisposable, Xunit.IUseFixture
{
public TestClass()
{
// 2) Set up TestA
// 5) Set up TestB
}
[Xunit.Fact]
public void TestA()
{
// 3) Run TestA
}
[Xunit.Fact]
public void TestB()
{
// 6) Run TestB
}
public void Dispose()
{
// 4) Tear down TestA
// 7) Tear down TestB
}
public TestData TestData { get; set; }
public void SetFixture(TestData data)
{
// 2.5) Set fixture data for TestA
// 5.5) Set fixture data for TestB
TestData = data;
}
}
Happy testing!
~Tom
Are you getting this error when trying to use the InternetExplorerDriver for WebDriver (Selenium 2.0)?
System.InvalidOperationException
"Unexpected error launching Internet Explorer. Protected Mode must be set to the same value (enabled or disabled) for all zones. (NoSuchDriver)"
Don't worry, it's easier to fix than it sounds: Simply go into your Internet Options of Internet Explorer, select the Security tab, and all four zones to have the same Protected Mode value (either all on or all off). That's it!
Sample Code
using NUnit.Framework;
using OpenQA.Selenium;
using OpenQA.Selenium.IE;
namespace WebDriverTests
{
public abstract class WebDriverTestBase
{
public IWebDriver WebDriver { get; set; }
[TestFixtureSetUp]
public void TestFixtureSetUp()
{
WebDriver = new InternetExplorerDriver();
}
[TestFixtureTearDown]
public void TestFixtureTearDown()
{
if (WebDriver != null)
{
WebDriver.Close();
WebDriver.Dispose();
}
}
[SetUp]
public void SetUp()
{
WebDriver.Url = "about:blank";
}
}
[TestFixture]
public class GoogleTests : WebDriverTestBase
{
[Test]
public void SearchForTom()
{
WebDriver.Url = "http://www.google.com/";
IWebElement searchBox = WebDriver
.FindElement(By.Id("lst-ib"));
searchBox.SendKeys("Tom DuPont");
searchBox.Submit();
IWebElement firstResult = WebDriver
.FindElement(By.CssSelector("#search cite"));
Assert.AreEqual("www.tomdupont.net/", firstResult.Text);
}
}
}
Enjoy,
Tom
