SpecFlow is an extremely powerful testing framework, but if you don’t know how to use it, it can end up being more of a liability than an asset.
When SpecFlow was first introduced on my team over a year ago, we used it exclusively for unit testing. A few months ago we started using it for feature testing in one of our microservices, and I am now finally starting to refactor our automation framework to use it.
My point with all this is that I’ve experienced trying SpecFlow out in a variety of situations, and I think I’ve come up with a pretty good idea of when and how to use it. I haven’t found many other blog posts where people talk about how they write their SpecFlow frameworks, so I figured I’d share mine.
When to use SpecFlow
SpecFlow is not the end-all, be-all tool for testing. There are some situations where it is extremely valuable and other places where the costs greatly outweigh the benefits of using it.
- Use SpecFlow when writing feature tests or automation tests.
- Don’t use SpecFlow when writing unit tests.
These statements seem obvious since the tool is created for feature testing and not unit testing, but it is worth mentioning since I’ve actually tried it in both situations and seen the results.
SpecFlow for feature and automation testing
SpecFlow is good for feature and automation testing for the simple fact that that’s what it’s designed for. When talking about features, the development, QA and product teams all communicate using the same language. This is necessary to ensure that everyone is on the same page about what is being built. And it’s exactly this shared language that SpecFlow is designed to use.
When you write feature tests using this shared language, you provide a lot of benefits to everyone:
- Anyone can run your tests and understand which pieces of your application are working and which ones aren’t. It’s no longer limited to just the development team.
- Shared languages lend themselves really well to resuability. Multiple features and scenarios within your application will typically be comprised of a set of common steps. Examples of this would be:
“Given I have logged in”
“When I import <x> file”
“Then an error message should show.”
By writing your tests using a shared language, you can start to re-use pieces of tests you’ve already written instead of having to re-write them for every new test.
- The readability of your tests is improved immensely since re-using steps brings consistency to how your tests are written. Instead of every developer writing their tests in their own way, a common way of writing them is introduced.
- When a test fails, you can quickly assess what is failing and what is the potential impact without having to delve down into the nuts and bolts of what the test is doing.
SpecFlow for unit testing
I don’t like SpecFlow for unit tests at all. In my experience, testing individual units of code does not fit well with the Gherkin syntax that SpecFlow uses. The main reason is because when you’re writing unit tests, you have to set up and mock everything else that you’re not testing, and there doesn’t seem to be a good way of doing this within the context of a SpecFlow scenario without it reading in a really weird way.
Let me illustrate with an example:
- Scenario: When I make a successful request to save data then I should recieve an OK result
- Given I have a request to save some data
- And I have set up the data repository to return resource id “CCB80401-908B-4868-82EB-EDF598FA03CE”
- When I send the request to save the data
- Then I should return a success response with resource id “CCB80401-908B-4868-82EB-EDF598FA03CE”
In the above scenario, we’re testing a REST controller endpoint. We want to make sure that the controller will return the correct status code and resource id when we make a successful POST request.
The problem I have with this is the second Given step, where we have to mock our data repository to return a specific value that we can test against. The whole idea behind the Gherkin language is to provide a standardized way of communicating software features that the development team, business analysis/product owners, and QA team. When you start writing SpecFlow steps like this one, you’re no longer writing with business requirements; you’ve now gone into the realm of developer nomenclature.
Working effectively with SpecFlow is a skill
One last thing I think is worth pointing out is that SpecFlow is a skill that will need to be learned. As I mentioned earlier, it’s really powerful, but only once you understand how it works and how to make use of it. It’s not something that the development team can just pick up and start working with just becuse they’re familiar with unit testing.
I’m mentioning this because I’ve seen some real implications of new teammates working with SpecFlow without having sufficient training or actually caring about using it. There are a lot of lazy developers out there – especially when it comes to testing – so real damage can be done if you try to impose SpecFlow on the wrong team or at the wrong time. Here’s some of what I’ve seen:
- A lot of poorly written specs get written which do not read like business requirements at all, and instead read like unit tests. Essentially what will happen is developers will continue writing tests in a way that they’re familiar with, which is usually a unit testing style.
- No consideration is given for reusability, so your framework will end up having 10 step definitions written in slightly different ways that all do the same thing. This makes your testing framework impossible to maintain.
- Refactoring tests becomes a nightmare because people “hack” their tests to share data across their step definitions and get their (mediocre) assertions working.
To summarize, SpecFlow gives you what you need to succeed but doesn’t necessarily give you the how. This is where it becomes important to introduce a framework for SpecFlow testing, something I haven’t seen much discussion on. Working within a SpecFlow framework is the same concept as all other frameworks; it makes the developer’s job easier by introducing a set of standards and best practices.
Now I’ll start talking about the framework which I’ve found effective and not too hard to understand. Let me start by giving an overview of the important pieces:
- Organization: All your test materials – data, step definitions, feature files, etc. – must be organized in a logical and easy to understand manner. This is the only way to make step and data re-use easy. If people don’t know where to find your steps or data, they can’t re-use them. If they can’t re-use them then this is how you end up with an unmaintainable mess.
- Data: the data that your tests use should be placed in a “Data” folder and subfolders to isolate it from your step definitions.
- Context objects: information must be shared across step definitions in order to make SpecFlow work. There are a few ways of doing it, but the best way is to use context objects and SpecFlow’s context injection. These objects are basically POCOs that hold the current state that your application is in at a given point in time.
- Interfaces: in addition to injecting context objects into your tests, you want to inject interfaces as well. For example, if you have a shared library that you use to make calls to one of your APIs, you don’t want your steps to be bound to some concrete implementation of this service. Instead you want to inject an interface. This will allow you to change the implementation later on without having to refactor all your tests.
- Utilities: utilities are classes that you will create within your test framework which can be re-used across multiple steps. Using utilities will help trim down your step definitions by a lot.
- FluentAssertions: Using the FluentAssertions library makes your assertions much cleaner, easier to understand, and provides a great deal of detail about what went wrong when your tests fail.
The easiest way for me to explain this framework is to show an example, which I do in the video below.