Testing shared resources with xUnit and Moq

Source code

Introduction

xUnit is widely used testing library and the post focuses attention how to write unit tests for classes that uses static properties or shared resources. Several scenarios are considered in the article.

Background

Solution uses C#7, .Net 4.6.1, NuGet packages Unity and CommonServiceLocator to implement ServiceLocator pattern, and NuGet packages xUnit, Moq, FluentAssertions and Ikc5.TypeLibrary to write tests.

Issue

Let’s imagine that the application has classes which use some shared resource like static property or shared container. Good written unit test should initialize necessary data and resources, call test routine, assert what is expected, and clear resources. In addition it is expected to use the minimal set of required dependencies and resolved it to mocked instances where it is possible. Due to by default xUnit runs tests in parallel, in the case of shared resources several tests could simultaneously change state or access shared resource that leads to the case when some tests get unexpected value and fail. But let’s note that at the same time each single test is successful, and fails only when all tests are run. The following image demonstrates such schema:

Tests over shared resource
Tests over shared resource

According to the scheme solution could be searched in the following ways:

  • disable parallelization feature and run test sequentially;
  • change code to remove call to the shared resource from tested classes;
  • keep parallelization feature but add synchronization between test threads.

Let’s discuss it below.

In this post we consider two boundary cases – the class with value type static property and implementation of service locator pattern. ServiceLocator is one of widely used patterns for implementing dependency injection. Container is filled up with dependencies, and application classes use static instance of IServiceLocator to call Resolve or GetInstance methods to resolve dependencies.

Solution

Console application

Provided solution contains ConsoleApp console application and several ConsoleApp.*Tests test libraries that demonstrate considered approaches. Classes and test classes are put to folders Resource and Locator depends on the implemented case. The Main method ProcessValue initializes static property and Unity container and calls client classes’ methods.

Static property

ConsoleApp contains resource class that exposes static property SharedProperty

public class Resources
{
	public string InstanceProperty { get; set; } = "Instance value";

	public static int SharedProperty { get; set; } = 17;
}

and client class that consumes it

public class Class1
{
	// … other methods

	public string GetValues()
	{
		return Resources.SharedProperty.ToString("D3");
	}
}

Service Locator pattern

ConsoleApp contains three similar interfaces with single method GetValue, corresponding implementation classes where this method returns string constant, and two classes such that each consumes two of three services.

The interface IService and its implementations Service are listed below

public interface IService1
{
	string GetValue();
}

public class Service1 : IService1
{
	public string GetValue()
	{
		return "Service1";
	}
}

In order to implement ServiceLocator pattern, application contains UnityServiceLocator class that is derived from ServiceLocatorImplBase with default implementation.

public class UnityServiceLocator : ServiceLocatorImplBase
{
	private readonly IUnityContainer _unityContainer;

	public UnityServiceLocator(IUnityContainer unityContainer)
	{
		_unityContainer = unityContainer;
	}

	protected override object DoGetInstance(Type serviceType, string key)
	{
		return _unityContainer.Resolve(serviceType, key);
	}

	protected override IEnumerable DoGetAllInstances(Type serviceType)
	{
		return _unityContainer.ResolveAll(serviceType);
	}
}

Further, the application contains two client classes Class1 and Class2 with GetServiceValues method that returns a collection of values from two services: Service1 and Service2 for Class1, and Service2 and Service3 for Class2 by resolving service instances via service locator. It is used to demonstrate different required services for Class1Tests and Class2Tests tests.

public class Class1
{
	// … other methods

	public IEnumerable GetServiceValues()
	{
		var service1 = ServiceLocator.Current.GetInstance();
		yield return service1.GetValue();

		var service2 = ServiceLocator.Current.GetInstance();
		yield return service2.GetValue();
	}
}

Test library

Let’s consider ConsoleApp.FailedTests test library. It contains tests for both cases separated to folders Resource and Locator. According to the library name, these tests demonstrates the main issue – each single test is successful, but two of them, one per case, are always fail when all tests are run.

Static property tests

The library contains Collection1Tests and Collection2Tests classes with tests for client class that uses static property. All tests look in the similar way

[Fact]
public void Class1_ShouldReturn_InitValue()
{
	const string expectedValue = "017";
	Resources.SharedProperty = 17;
	Thread.Sleep(100);

	var class1 = new Class1();

	// test routine
	var actualValue = class1.GetValues();

	// assert
	actualValue.Should().NotBeNullOrEmpty();
	actualValue.Should().Be(expectedValue);
}

Let’s note that tests include the statement Thread.Sleep(100) to emulate some set up work.

Service Locator tests

Similarly, the library contains Class1Tests and Class2Tests classes with tests for client classes that use service locator.

[Fact]
public void Class1_ShouldReturn_TwoServiceMockValues()
{
	const string expectedService1Value = "Service1Mock";
	const string expectedService2Value = "Service2Mock";

	// set services
	var service1 = Mock.Of(mock => mock.GetValue() == expectedService1Value);
	var service2 = Mock.Of(mock => mock.GetValue() == expectedService2Value);

	// set container
	var container = new UnityContainer();

	container
		.RegisterInstance(typeof(IService1), service1, new ExternallyControlledLifetimeManager())
		.RegisterInstance(typeof(IService2), service2, new ExternallyControlledLifetimeManager());

	var locator = new UnityServiceLocator(container);
	ServiceLocator.SetLocatorProvider(() => locator);

	// test routine
	var class1 = new Class1();
	var values = class1.GetServiceValues()?.ToList();

	// assert
	values.Should().NotBeNull();
	values.Count.Should().Be(2);
	values[0].Should().Be(expectedService1Value);
	values[1].Should().Be(expectedService2Value);
}

Approaches

There are different approaches to solve the issue, but each of them has advantages and disadvantages. Let’s analyze how code or tests could be changed.

Control test order

According to Running Tests in Parallel documentation, by default xUnit runs tests in parallel. To remove simultaneous calls from tests to the shared resource, test classes could be updated in the following way:

  • Keeping all tests that use shared resource in the same collection by adding CollectionAttribute attribute to the test classes. ConsoleApp.CollectionTests library demonstrates this approach:

    [Collection("Resource Collection")]
    public class Class1Tests()
    {
        // ...
    }
    

    and

    [Collection("Resource Collection")]
    public class Class2Tests()
    {
        // ...
    }
    

    Simultaneously, tests in Locator folder are marked by the attribute [Collection("Locator Collection")];

  • Put all tests that use the same shared resource to the same test class. Due to this tests will be included to the same collection automatically and runs sequentially;
  • Separate tests to different assemblies.

After such changes all tests are successful. The main disadvantages of this approach is obvious – usually real-world application contains a lot of unit tests and executing them sequentially could dramatically increase total time of the execution. Moreover, tested method could use more then one shared resource that complicates separating to several collections.

Update code to use dependencies

The issue could be solved if remove calls to shared resources from classes and add middle layer such that it allows different implementation: implementation with calls to shared resources in real application and mocked object in test environment. The main idea is to add dependencies to client class, that is instance or interface of middle layer class. Various implementations are possible, and let’s consider some of them, that is implemented in ConsoleApp.FixedCodeTests.

Static property tests

Implement IResourcesWrapper interface that repeats public properties and methods of Resources class

public interface IResourcesWrapper
{
	string InstanceProperty { get; set; }

	int SharedProperty { get; set; }
}

and add constructor parameter to the client class

public class Class1
{
	private readonly IResourcesWrapper _resourcesWrapper;

	// dependency injection in constructor
	public Class1(IResourcesWrapper resourcesWrapper)
	{
		_resourcesWrapper = resourcesWrapper;
	}

	public string GetValues()
	{
		// replace call to static property by the call to property of the instance
		return _resourcesWrapper.SharedProperty.ToString("D3");
	}
}

In the application, new class ResourcesWrapper is created and it implements ResourcesWrapper and returns value of static property

public class ResourcesWrapper : IResourcesWrapper
{
	private readonly Resources _resources = new Resources();

	// … other methods and properties

	public int SharedProperty
	{
		get => Resources.SharedProperty;
		set => Resources.SharedProperty = value;
	}
}

On the other hand, tests are changed accordingly, i.e. create mock of IResourcesWrapper and set static property to expected value

[Fact]
public void Class1_ShouldReturn_InitValue()
{
	// ...
	var resourcesWrapper = new Mock();
	resourcesWrapper.SetupGet(mock => mock.SharedProperty).Returns(17);

	var class1 = new Class1(resourcesWrapper.Object);

	// test routine - the same
	// ...
}

This approach works for static methods and static classes, too.

In addition, the similar solution could be implemented without interface, where instance of ResourceWrapper class is injected to Class1 class constructor. Then in the test library, a developer could create derived class TestSharedResourceWrapper that returns test value or mocked object. But previous implementation is better according to dependency inversible principle.

Service Locator tests

Analyze code and find out that shared resource returns object that implements some interface. For example, in the case of service locator implementation, ServiceLocator.Current is an object that implements IServiceLocator interface that could be injected to the application classes. To implement this idea, let’s update Class1:

public class Class1
{
	private readonly IServiceLocator _serviceLocator;

	public Class1(IServiceLocator serviceLocator)
	{
		_serviceLocator = serviceLocator;
	}

	public IEnumerable GetServiceValues()
	{
		var service1 = _serviceLocator.GetInstance();
		yield return service1.GetValue();
		// ...
	}
}

Then tests are changed slightly, i.e. pass IServiceLocator instance to the class constructor

[Fact]
public void Class1_ShouldReturn_TwoServiceMockValues()
{
	// set container and services - as before
	// ...
	var locator = new UnityServiceLocator(container);
	var class1 = new Class1(locator);

	// test routine - the same
	// ...
}

Now each class uses its own copy of container without accessing it via static property.

Summarizing

Again, after these changes all tests are successful. The main advantage of this approach is that it is one more step toward loose class coupling and applying dependency injection pattern. The disadvantages of this approach are also quite obvious – it could be hard to implement and it requires a lot of code changes as usually real-world application contains many classes with probably complex hierarchy. Constructors could be restricted for modifications, added dependencies could dramatically increase number of constructor parameters, and changes affect other classes. In addition, often business is not ready for additional development time just for writing unit tests.

Synchronize test threads

One more solution is to synchronize access of tests to the shared resources, as is shown in ConsoleApp.SynchronizedTests. The main idea is to use EventWaitHandle to control access to the shared resource. Before test runs the tested method, it waits for signaled state of EventWaitHandle instance, sets unsignaled state, uses shared resource, and then sets signaled state back as soon as possible. Let’s consider implementation of this approach in our cases.

Static property tests

Let’s create class over event wait handle. Each instance opens handle by its name, which is stored in static property. But let’s use generic class as it allows to keep unique value of event wait handle name per combination of type arguments. Method Lock waits for EventWaitHandle and throw an exception after fixed wait time if event wait handle don’t send a signal. Dispose method set handle to signaled state and frees resources.

public class SynhronizeTest : DisposableObject
{
	/// ...
	private EventWaitHandle EventWaitHandle { get; set; }
	private static string EventWaitHandleName { get; }

	/// ...
	private static EventWaitHandle GetEventWaitHandle()
	{
		if (EventWaitHandle.TryOpenExisting(EventWaitHandleName, out var eventWaitHandle))
			return eventWaitHandle;
		// TRUE to set signaled state that allows the first thread continue 
		return new EventWaitHandle(true, EventResetMode.AutoReset, EventWaitHandleName);
	}

	public void Lock()
	{
		EventWaitHandle = GetEventWaitHandle();
		if (!EventWaitHandle.WaitOne(TimeSpan.FromMilliseconds(WaitTime)))
			throw new ArgumentException(/*…*/);
	}

	protected override void Dispose(bool disposing)
	{
		if (disposing)
		{
			if (EventWaitHandle != null)
			{
				EventWaitHandle.Set();
				EventWaitHandle.Close();
				EventWaitHandle = null;
			}
		}
		base.Dispose(disposing);
	}
}

Then tests should create the instance of SynhronizeTest and disposes it after tested method have been executed.

[Fact]
public void Class1_ShouldReturn_InitValue()
{
	// ...
	using (var synchronizeTest = new SynhronizeTest())
	{
		synchronizeTest.Lock();
		Resources.SharedProperty = 17;

		// test routine - the same
		// ...
	}

	// assert - the same
	// ...
}

Service Locator tests

For the case of service locator pattern let’s use the similar approach. At first, create class TestableServiceLocator that is not a generic class, keep unity container in class property and set service locator in Lock.

public abstract class TestableServiceLocator : DisposableObject
{
	/// ...
	private EventWaitHandle EventWaitHandle { get; set; }
	protected IUnityContainer Container { get; private set; }
	private static string EventWaitHandleName { get; }

	/// ...
	public void Lock()
	{
		EventWaitHandle = GetEventWaitHandle();
		if (!EventWaitHandle.WaitOne(TimeSpan.FromMilliseconds(WaitTime)))
			throw new ArgumentException(/*...*/);

		var locator = new UnityServiceLocator(Container);
		ServiceLocator.SetLocatorProvider(() => locator);
	}

	// ...
}

As there are set of tests that requires the same container, let’s derive classes from TestableServiceLocator that keeps defined container:

internal class Class1ServiceLocator : TestableServiceLocator
{
	// ...

	public Class1ServiceLocator()
		: base(new UnityContainer())
	{
		// set services
		var service1 = Mock.Of(mock => mock.GetValue() == Service1Value);
		// ...
		// set container
		Container
			.RegisterInstance(typeof(IService1), service1, new ExternallyControlledLifetimeManager())
		// ...
	}

	// ...
}

As the last step, tests should create the instance of this derived class, runs tested method and dispose locking instance.

[Fact]
public void Class1_ShouldReturn_TwoServiceMockValues()
{
	IList values;
	using (var serviceLocator = new Class1ServiceLocator())
	{
		serviceLocator.Lock();

		// test routine - the same
		// ...
	}

	// assert - the same
	// ...
}

Summarizing

Again, after these changes all tests are successful. The main advantage of this approach is that it is doesn’t require changes of the application and all things are done in test library. In addition, this approach is good for large number of unit tests and in come kind is compromise solution between previous solutions. On the other hand, additional time is added due to thread synchronization, so it could be considered as disadvantage of this approach.

Run All tests
Run All tests

Conclusions

Writing unit tests that use shared resource is a challenge and in this post we consider various approaches how to solve possible issues. Each method has it disadvantages and advantages and the decision which method to use depends on various factors including business needs, complexity of systems, used tools.


1. All used IP-addresses, names of servers, workstations, domains, are fictional and are used exclusively as a demonstration only.
2. Information is provided «AS IS».

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.