Design, Test and Develop like it's heaven on Earth - Part 8

Published on 7/16/2019

In this part we are going to start implementing the syndication plan from part 7. There’s quite a bit to do, so make sure you’re familiar with the discussions and concepts from the previous post.

Filling in the blanks

I start by defining the abstractions we discussed to put the required services in place. The service worker:


public interface IBlogBackgroundService : IHostedService, IDisposable
{
    Task ExecuteAsync(CancellationToken cancellationToken);
}

This background service interface is simply a way to give me an abstraction of it while I don’t have an actual implementation. And to be honest, the actual implementation will probably not even live in the domain layer. It basically fulfils all the definitions of the BackgroundService base class that was mentioned in the IHostedService documentation. Next, the queue:


public interface IBlogSyndicationQueue
{
    Task Enqueue(IBlogSyndication blogSyndication);
    Task<IBlogSyndication%gt; Dequeue();
}

Red

The remaining blanks we will fill in through unit testing. Let’s start with the blog service:


[TestMethod]
public async Task Publish_Enqueue()
{
    //arrange
    string title = "hello_test";
    Blog blog = new Blog() { Title = "Hello Test!" };
    _dbAdaptorMock.Setup(x => x.Read(title))
        .ReturnsAsync(blog);

    //act
    await Service.Publish(title);

    //assert
    _blogSyndicationQueueMock.Verify(x => x.Enqueue(??));
}

That’s as far as I got on my initial pass. Let’s take it apart a bit. We arrange the adaptor as before, but then we do a verify check on a new mock, _blogSyndicationQueueMock. This is set up as follows:


readonly Mock<IBlogSyndicationQueue> _blogSyndicationQueueMock = new Mock<IBlogSyndicationQueue>();

In order for the service to know about it, we will have to pass it through the constructor, just like the adaptor:


public BlogService Service => new BlogService(_dbAdaptorMock.Object, _blogSyndicationQueueMock.Object);

And then, of course, we need to change the service in order to accept this new constructor injection:


readonly IBlogSyndicationQueue _blogSyndicationQueue;

public BlogService(IBlogDatabaseAdaptor dbAdaptor, IBlogSyndicationQueue blogSyndicationQueue)
{
    _dbAdaptor = dbAdaptor;
    _blogSyndicationQueue = blogSyndicationQueue;
}

But now we are left with the missing parameter for the Enqueue() method on the mock. It takes an instance of the IBlogSyndication abstraction. We discussed how we will create these using a factory object, based on configuration. This means that we will have to provide a mock for the factory which we can set up to produce whatever we need for the test. But first, we need to define the factory. It’s a very basic structure:


public interface IBlogSyndicationFactory
{
    IBlogSyndication GetInstance(string provider);
}

If we stick this into the test (and into the constructor) we have something that we can set up to get us an instance of the syndication abstraction:


readonly Mock<IBlogSyndicationFactory> _blogSyndicationFactoryMock = new Mock<IBlogSyndicationFactory>();

public BlogService Service => new BlogService(_dbAdaptorMock.Object, _blogSyndicationQueueMock.Object, _blogSyndicationFactoryMock.Object);

And then in the unit test, we add an additional arrange bit:


Mock<IBlogSyndication> syndicationMock = new Mock<IBlogSyndication>();
_blogSyndicationFactoryMock.Setup(x => x.GetInstance("provider"))
    .Returns(syndicationMock.Object);

Here we just have a local mock, since it’s not something that will be injected into the service at all. It is important to have this locally in the test method itself so that your unit test effectively documents everything that is required to happen in the correct scope. Looking at the unit test later on then informs the reader as well as acts as a guard against future changes.

Now we can complete our assert of the unit test:


_blogSyndicationQueueMock.Verify(x => x.Enqueue(syndicationMock.Object));

Green

Let's see if we can pass this unit test. We make two code changes to the service:


IBlogSyndication syndication = _blogSyndicationFactory.GetInstance("provider");
await _blogSyndicationQueue.Enqueue(syndication);

Concept: Configuration

We’ve passed our test now, but we hardcoded the provider string. It is time that we bring in the configuration as we discussed in our plan. Configuration is central to most programming environments and runtimes today. It is also absolutely key in various other technologies like docker and is the backbone of infrastructure as code. It comes in various different flavours, from plain old text files (.ini) or old XML (.xml) to JSON (.json) or yaml (.yml). Configuration is defined first and then overridden later through various mechanisms, typically environment variables or secrets vault services. The .NET Core configuration system is absolutely key and central to even the framework provided functionality. Most things come with overloads for options, and these options can typically be defined by configuration. It’s a good idea to familiarize yourself with these systems and their conventions - whichever one is applicable to you.

Blue

For this we’ll be using the options pattern, so let’s define the syndication option delegate classes. We start by figuring out what the actual configuration will look like first. Probably something like this:


   "syndications":[
       {
           "provider":"RSS"
       },
       {
           "provider":"Twitter",
           "apiKey":""
       },
       {
           "provider":"Facebook",
           "apiKey":""
       }
   ]

From here we can derive two classes: One for the list, and one for a provider. Some properties will obviously be null for some of them, like RSS.


public class BlogSyndicationOption
{
    public string Provider { get; set; }
    public string ApiKey { get; set; }
}

public class BlogSyndicationOptionCollection : List<BlogSyndicationOption>
{

}

Red

Now that we’ve got these defined it’s possible to use them in our service in order to properly drive our factory. We need to inject the options into the constructor of course.


readonly Mock<IOptionsMonitor<BlogSyndicationOptionCollection>> _blogSyndicationOptionsMock = new Mock<IOptionsMonitor<BlogSyndicationOptionCollection>>();

public BlogService Service => new BlogService(_dbAdaptorMock.Object, _blogSyndicationQueueMock.Object, _blogSyndicationFactoryMock.Object, _blogSyndicationOptionsMock.Object);

Because the options pattern uses an abstraction IOptionsMonitor (alternatively IOptions), we can set it up per test through the mock. For our plan, we need to test that the service correctly enqueues as many syndications as are configured, so we set up a collection with two entries. We verify for each enqueued item that the blog was correctly set against it. This introduces a new property Blog on the IBlogSyndication abstraction. We also change our factory mock setup to work for any provided string value.


_blogSyndicationOptionsMock.SetupGet(x => x.CurrentValue)
    .Returns(new BlogSyndicationOptionCollection()
    {
        new BlogSyndicationOption() { Provider = "RSS" },
        new BlogSyndicationOption() { Provider = "Twitter" }
    });
_blogSyndicationFactoryMock.Setup(x => x.GetInstance(It.IsAny<string>()))
    .Returns(syndicationMock.Object);

This means we can now assert that our factory is called for exactly this configuration and that two items were hydrated and enqueued:


_blogSyndicationFactoryMock.Verify(x => x.GetInstance("RSS"));
_blogSyndicationFactoryMock.Verify(x => x.GetInstance("Twitter"));
_blogSyndicationQueueMock.Verify(x => x.Enqueue(syndicationMock.Object), Times.Exactly(2));
syndicationMock.VerifySet(x => x.Blog = blog, Times.Exactly(2));

Green

The code changes we have to make in order to pass this test involve iterating over the collection and process each configured entry:


public async Task Publish(string title)
{
    var blog = await _dbAdaptor.Read(title);

    blog.IsPublished = true;

    Validate(blog);

    _syndicationCollection?.ForEach(x =>
    {
        IBlogSyndication syndication = _blogSyndicationFactory.GetInstance(x.Provider);
        _blogSyndicationQueue.Enqueue(syndication);
    });
}

I added the null-coalescing operator before the .ForEach() so that none of the other unit tests fail (since no options are configured for those tests). However, I feel that this is a much more important consideration than just something to “deal with”.

Concept: Logging

I will probably want to know when no syndications were configured (or some other problem causes it to be null), so it’s something that I want to log. Why is this important? This is part of an automated process, and there is no effective and simple way for the API to inform the UI that something happened without blocking and breaking the entire process flow. Also, it’s really not the user’s concern that some back-end misconfiguration is causing a problem; the user simply wants to publish the blog and move on. This sort of thing is what system logs are meant for. Logging informs us of operational exceptions, and in the classic 80/20 law of customer service, 80% of the exceptional circumstances can be handled offline and out of process, and these should be reported on by the system (as opposed to the customer or user calling in to complain about an error on the screen).

Blue

We have to refactor this code to log when we have no syndications to process but to do that we need the logging service. So let’s inject that into the constructor also:


readonly ILogger _logger;

public BlogService(IBlogDatabaseAdaptor dbAdaptor, IBlogSyndicationQueue blogSyndicationQueue, IBlogSyndicationFactory blogSyndicationFactory, IOptionsMonitor<BlogSyndicationOptionCollection> syndicationCollectionOptions, ILoggerFactory loggerFactory)
{
    _dbAdaptor = dbAdaptor;
    _blogSyndicationQueue = blogSyndicationQueue;
    _blogSyndicationFactory = blogSyndicationFactory;
    _syndicationCollection = syndicationCollectionOptions.CurrentValue;
    _logger = loggerFactory?.CreateLogger<BlogService>();
}

The mock setup for the logger is non-trivial. We have to mock both the factory and the actual logger, and set up the factory mock to return the logger mock.


readonly Mock<ILoggerFactory> _loggerFactoryMock = new Mock<ILoggerFactory>();
readonly Mock<ILogger> _loggerMock = new Mock<ILogger>();

public BlogServiceTests()
{
    _loggerFactoryMock.Setup(x => x.CreateLogger(It.IsAny<string>())).Returns(_loggerMock.Object);
}

Now we can check for the exceptional case:


public async Task Publish(string title)
{
    var blog = await _dbAdaptor.Read(title);

    blog.IsPublished = true;

    Validate(blog);

    if (_syndicationCollection == null || _syndicationCollection.Count == 0)
    {
        _logger?.LogWarning("No syndications configured for processing");
    }
    else
    {
        _syndicationCollection.ForEach(x =>
        {
            IBlogSyndication syndication = _blogSyndicationFactory.GetInstance(x.Provider);
            _blogSyndicationQueue.Enqueue(syndication);
        });
    }
}

This logging creates a branch, and we will need to add a test that verifies the logger was called.


[TestMethod]
public async Task Publish_EnqueueSyndications_NullOrNoSyndications_Logs()
{
    //arrange
    string title = "hello_test";
    Blog blog = new Blog() { Title = "Hello Test!" };
    _dbAdaptorMock.Setup(x => x.Read(title))
        .ReturnsAsync(blog);
    _blogSyndicationOptionsMock.SetupGet(x => x.CurrentValue)
        .Returns((BlogSyndicationOptionCollection)null);

    //act
    await Service.Publish(title);

    //re-arrange
    _blogSyndicationOptionsMock.SetupGet(x => x.CurrentValue)
        .Returns(new BlogSyndicationOptionCollection() { });

    //act
    await Service.Publish(title);

    //assert
    _loggerMock.Verify(x => x.Log(LogLevel.Warning, It.IsAny<EventId>(), It.IsAny<object>(), It.IsAny<Exception>(), It.IsAny<Func<object, Exception, string>>()), Times.Exactly(2));
}

It is really two tests in one since it re-configures the mock to return an empty collection to cover the logical OR of the if-statement. Because we acted twice, we expect two invocations of the logger.

The next problem I see here is that this service constructor is becoming very long. It accepts a whole slew of abstractions in order to perform its functions. Related to this is the fact that the Publish() method is also getting rather contrived. It deals with reading the blog and the validation stuff, and then also runs through the syndication stuff (and I now notice that we never implemented the Update call to save the blog back after setting the flag). We can refactor the syndication stuff out into a method to get back to a single orchestration purpose of course.


public async Task Publish(string title)
{
    var blog = await _dbAdaptor.Read(title);

    blog.IsPublished = true;

    Validate(blog);

    //TODO save

    Syndicate(blog);
}

private void Syndicate(Blog blog)
{
    if (_syndicationCollection == null || _syndicationCollection.Count == 0)
    {
        _logger?.LogWarning("No syndications configured for processing");
    }
    else
    {
        _syndicationCollection.ForEach(x =>
        {
            IBlogSyndication syndication = _blogSyndicationFactory.GetInstance(x.Provider);
            syndication.Blog = blog;
            _blogSyndicationQueue.Enqueue(syndication);
        });
    }
}

This is a good start and informs us of the required parameters that we’ll need to pass, but this still requires all the constructor parameters. Even the unit test class as a whole is becoming a very cluttered place with all these new mock declarations. It is starting to look like all this syndication stuff should be its own service.

Conclusion

I’ll draw the line here for part 8. We have quite a bit of work to do in the next part, moving around and rewiring a lot of what we did now to better restructure and modularize everything in order to keep the implementations clean and the responsibilities single. And then we still need to move on to the “other side” of the queue abstraction, testing that items are pulled off of it and actually processed.

In this part we saw two very important concepts: Configuration and Logging. We saw how your program’s behavior can be driven by configuration (see also the strategy pattern). We saw how we want to isolate the user from runtime errors during automated processes and how we should deal with those cases.

We also wrote our most complicated unit test (and it’s test number 10!). How do I measure that? It has by far the biggest number of mock configurations. This is how it illustrates to the reader of the unit test what the underlying requirements are. It now becomes simple for the reader to build a mental list of all the systems or abstractions that are used in the functionality this test covers, and if you name things appropriately, it also becomes easy for the reader to find and navigate to those underlying systems.