Jun 25, 2021
Announcing HMock 0.2
I’ve just released a small update to HMock, the Haskell mock testing framework which I first released following Zurihac. Here’s what’s new in the new version.
Rejecting ambiguous expectations
First, HMock can now reject ambiguous expectations. Suppose had a mock filesystem monad with HMock, and you wrote something like this:
expect $ ReadFile_ anything |-> "some content"
expect $ ReadFile "foo.txt" |-> "foo content"x <- readFile "foo.txt"
With HMock 0.1, x is "foo content". The more recent expectation matches before earlier expectations. This is often what you want. But Svenningsson et al argue very strongly in An Expressive Semantics of Mocking that this is error-prone, and it would be better to reject this code because it sets up two expectations that both match the same call. Starting in HMock 0.2, you can use setAmbiguityCheck True in MockT to enable ambiguity checking, causing HMock to reject the example above.
If you do this, you’ll have to think about how to express your desired behavior in non-ambiguous ways. There are some other changes to HMock that help with this. First, the desugaring of expectN has been changed to avoid creating unnecessary ambiguities. Second, there’s now a new operation, allowUnexpected, that allows you to tell HMock that certain calls should succeed without an expectation and, optionally, what they should return.
You might think allowUnexpected is just ambiguity by another name! But the true problem is that previously one had to say “expect any number of calls” when what one really meant was “don’t make me expect all these calls”. It’s not that the calls are really expected at all; just that you don’t care about them. Crucially, allowUnexpected is limited: it cannot be nested in other expectations, for instance. The intended overlap behavior in this limited case is clear. By defining it not to be an ambiguity, we can still say what we want, but prevent other less benign sorts of ambiguity.
MockSetup monad
Second, I’ve moved setup to a new restricted monad called MockSetup to fix a race condition.
Setup is the special action called setupMockable that automatically runs before the first time that HMock touches a class. It’s often used to set defaults for methods. When the first thread adds an expectation about a class, or tries to delegate a method of that class to HMock, HMock first runs setupMockable to set up defaults and such, and then proceeds.
The problem is that when you have multiple threads, one thread may be in the process of initializing a class when the other one hits. What do you do then? Well, you can block. But the setup process frequently needs to touch the class it’s initializing, and that can’t block! So… block unless the thread you would block is doing the initializing? Ugly, and what if the second thread holds a resource that the first (initializing) thread needs to make progress? Deadlock again!
This is fundamentally a difficult problem to solve using standard concurrency techniques. I solved the problem by moving the mock state to software transactional memory. But to be a real solution, setupMockable needs to run in the STM monad. That wasn’t previously possible. Now it is.
What this means is that setupMockable can no longer:
- Perform arbitrary I/O.
- Add expectations.
It would be possible to avoid the second limitation; but I’ve left it in place because I don’t think setupMockable is the right place for adding expectations anyway. Instead, you should use allowUnexpected to allow the calls without an expectation, if that’s what you mean.
Nesting MockT
Third, I’ve added a new operation called nestMockT, which lets you create a nested semi-independent block within a mock test. This block:
- Has its own options, such as defaults and ambiguity checking, which are originally the same as the parent. Changing them takes effect in the nested block, but the changes are reverted as soon as the nested block completes.
- Has its own expectations. The expectations of the nested block may be interleaved with the expectations of its parent, but when the block finishes, its own expectations must be satisfied.
The idea here is that you can test some part of your code without worrying that the expectations you set up will stick around and match where they aren’t expected later. In general, you should be reluctant to use nestMockT, and instead write more shorter tests if possible. However, when that’s not possible, you have an option for managing that complexity.
Module Structure
The original HMock release shoved most things into only a couple modules: Test.HMock and Test.HMock.TH. This was too much. I’ve now reorganized the implementation, split up behemoth modules, and divided everything into what I think are logical units. You can import just the parts you need, or import Test.HMock to get almost everything.
Iavor Diatchki has built a really useful library called graphmod, which can draw a graph of your modules and dependency structure using GraphViz! After pruning unnecessary arrows to get a nicer visualization, here’s what HMock looks like now:
There are some cyclic module dependencies involving Test.HMock.Internal.State, but otherwise the dependencies are clear.
Caution! Unstable API
I still have not adapted HMock to work with effect systems or other frameworks, as I want to do eventually. Because of this, the API should still be considered unstable. You’ll be fine if you add upper bounds on your dependency versions a la the PVP; but if not, your code might break with future updates. You’ve been warned!
Anyway, let me know what you think.
