If you look at the original property-based testing library, the Haskell version of QuickCheck,
tests are very closely tied to types: The way you typically specify a property is by inferring
the data that needs to be generated from the types the test function expects for its arguments.
In writing it though I forgot that there was a halfway house which is
also somewhat bad (but significantly less so) that you see in a couple
This is when the shrinking is not type based, but still follows the
classic shrinking API that takes a value and returns a lazy list of
shrinks of that value. Examples of libraries that do this are
This works reasonably well and solves the major problems with type
directed shrinking, but it’s still somewhat fragile and importantly
does not compose nearly as well as the approaches that Hypothesis
or test.check take.
Ideally, as well as not being based on the types of the values being
generated, shrinking should not be based on the actual values generated
This may seem counter-intuitive, but it actually works pretty well.
One of the big differences between Hypothesis and Haskell QuickCheck is
how shrinking is handled.
Specifically, the way shrinking is handled in Haskell QuickCheck is bad
and the way it works in Hypothesis (and also in test.check and EQC) is
good. If you’re implementing a property based testing system, you should
use the good way. If you’re using a property based testing system and it
doesn’t use the good way, you need to know about this failure mode.
Unfortunately many (and possibly most) implementations of property based
testing are based on Haskell’s QuickCheck and so make the same mistake.