├── .gitignore ├── instructions.md ├── lib └── scalacheck_2.9.2-1.10.0.jar └── src └── test └── scala └── pp └── scalacheck ├── IntSpecification.scala ├── ListSpecification.scala ├── MapSpecification.scala ├── StringSpecification.scala ├── TreeSpecification.scala └── VectorSpecification.scala /.gitignore: -------------------------------------------------------------------------------- 1 | 2 | # Eclipse pleasantries 3 | 4 | bin/ 5 | .settings/ 6 | .cache 7 | .classpath 8 | .project 9 | -------------------------------------------------------------------------------- /instructions.md: -------------------------------------------------------------------------------- 1 | ScalaCheck is a tool for testing Scala and Java programs, based on property specifications and 2 | automatic test data generation. The basic idea is that you define a property that specifies the 3 | behaviour of a method or some unit of code, and ScalaCheck checks that the property holds. 4 | All test data are generated automatically in a random fashion, so you don't have to worry about 5 | any missed cases. 6 | 7 | In this tutorial, we explain the basics of ScalaCheck, how it works and several interesting 8 | use-cases. You can learn more about it at https://github.com/rickynils/scalacheck/wiki/User-Guide. 9 | 10 | ## Prerequisites 11 | 12 | - Download and install Eclipse Indigo (3.7): http://www.eclipse.org/indigo/ 13 | - Download and install Scala IDE for version 2.9.x: http://scala-ide.org/download/current.html 14 | - Download the ScalaCheck 1.10.0 testing library: https://oss.sonatype.org/index.html#nexus-search;quick~scalacheck_2.9.2 15 | 16 | ## Instructions 17 | 18 | We first do the following preparatory steps. 19 | 20 | 1. Create a new Scala project in Eclipse. 21 | 2. Create a folder `lib`. 22 | 3. Copy the ScalaCheck 1.10.0 JAR file into the `lib` folder. 23 | 4. Go to Project -> Properties -> Java Build Path -> Add JARs and select the ScalaCheck JAR from the `lib` folder. 24 | 5. Create a source folder `src/test/scala`. 25 | 6. Create a new package `pp.scalacheck` in the source folder above. 26 | 7. Create a new file `StringSpecification.scala` in the package above. 27 | 28 | Alternatively, you can download the entire project from: http://github.com/axel22/scalacheck-tutorial 29 | 30 | ### Testing `String`s 31 | 32 | We will now write a few ScalaCheck tests that test certain `String` properties. 33 | We import the contents of the ScalaCheck package and its `forAll` statement: 34 | 35 | package pp.scalacheck 36 | 37 | import org.scalacheck._ 38 | import Prop.forAll 39 | 40 | Next, we define an object `StringSpecification` which will contain multiple properties 41 | of `String`s. The first such property will test that the concatenation of two strings 42 | `x` and `y` always starts with the string `x`. 43 | 44 | object StringSpecification extends Properties("String") { 45 | 46 | property("startsWith") = forAll { (x: String, y: String) => 47 | (x + y).startsWith(x) 48 | } 49 | 50 | // ... more properties will follow ... 51 | 52 | } 53 | 54 | We defined the property named `"startsWith"` saying that 55 | it must be true that for all `String`s `x` and `y` their concatenation is 56 | starts with `x`. 57 | 58 | Each `Properties` object is a complete application in ScalaCheck - it has a `main` 59 | method defined, which simply tests all the properties. 60 | This means that we can run it in Eclipse, so lets do exactly that. 61 | We click Run -> Run As -> Scala Application to obtain the following output: 62 | 63 | + String.startsWith: OK, passed 100 tests. 64 | 65 | We get an output saying that ScalaCheck generated `100` different `String`s 66 | and that our property was true for all of them. 67 | 68 | Lets write another property. This one states that the length of then concatenation of 69 | two `String`s is always longer than one of those `String`s. 70 | 71 | property("concatenation length") = forAll { (x: String, y: String) => 72 | x.length < (x + y).length 73 | } 74 | 75 | Running the test suite again we get: 76 | 77 | ! String.concatenation length: Falsified after 0 passed tests. 78 | > ARG_0: "" 79 | > ARG_1: "" 80 | 81 | Ooops! ScalaCheck is telling us that this seemingly simple property of `String`s 82 | isn't true at all. In fact, it even produces a small counterexample -- what if we 83 | have 2 empty `String`s? In that case our property does not hold. 84 | Let lets fix it to state that the concatenation is longer or equal to one of the 85 | two `String`s: 86 | 87 | property("concatenation length") = forAll { (x: String, y: String) => 88 | x.length <= (x + y).length 89 | } 90 | 91 | Running the tests now yields: 92 | 93 | + String.startsWith: OK, passed 100 tests. 94 | + String.concatenation length: OK, passed 100 tests. 95 | 96 | You can try writing a couple of your own properties. 97 | 98 | 99 | ### Testing Collections 100 | 101 | Automatic generation of input data is particularly useful for testing collections, 102 | so lets write a couple of collection properties. 103 | 104 | We create a new file in `pp.scalacheck` called `ListSpecification.scala` and write 105 | the following property saying that reversing a list twice gives back the original 106 | list: 107 | 108 | object ListSpecification extends Properties("List") { 109 | 110 | property("double reverse") = forAll { (lst: List[Int]) => 111 | lst.reverse.reverse == lst 112 | } 113 | 114 | } 115 | 116 | ScalaCheck is smart enough to know how the generate arbitrary lists of 117 | integers for us. Running the tests yields: 118 | 119 | + List.double reverse: OK, passed 100 tests. 120 | 121 | Lets write another property involving lists, stating that reversing two lists 122 | and zipping them is the same as first zipping them and then reversing the 123 | resulting list of pairs (for example, for `1 :: 2 :: Nil` and `2 :: 4 :: Nil` 124 | we get `(2, 4) :: (1, 2) :: Nil`). 125 | 126 | property("zip reverse") = forAll { (a: List[Int], b: List[Int]) => 127 | (a.reverse zip b.reverse) == (a zip b).reverse 128 | } 129 | 130 | It's not apparent at first sight that this property does not always hold: 131 | 132 | ! List.zip reverse: Falsified after 3 passed tests. 133 | > ARG_0: List("0") 134 | > ARG_1: List("-1", "0") 135 | 136 | In fact, the property always fails when the lists don't have the same length. 137 | We can fix the property by ensuring that both lists do have the same length: 138 | 139 | property("zip reverse") = forAll { (a: List[Int], b: List[Int]) => 140 | val a1 = a.take(b.length) 141 | val b1 = b.take(a.length) 142 | (a1.reverse zip b1.reverse) == (a1 zip b1).reverse 143 | } 144 | 145 | ScalaCheck can test the properties of other collections as well. For example, 146 | the following property of `Map`s says that after you add a key-value pair to 147 | any `Map`, the size of the resulting map will increase: 148 | 149 | property("+ and size") = forAll { (m: Map[Int, Int], key: Int, value: Int) => 150 | (m + (key -> value)).size == (m.size + 1) 151 | } 152 | 153 | Try to find a counterexample. Then use ScalaCheck to find check if your 154 | counterexample is similar. 155 | 156 | 157 | ### Testing Custom Datatypes 158 | 159 | We've seen so far that ScalaCheck is pretty smart in practice with generating the 160 | right data that can lead to counterexamples. 161 | However, ScalaCheck cannot by default generate input data for datatypes it does not 162 | know about. 163 | For example, lets define a custom datatype called `Vector` which describes two-dimensional 164 | mathematical vectors. We also define 2 operations -- scalar multiplication and vector length. 165 | 166 | case class Vector(x: Int, y: Int) { 167 | def *(s: Int) = Vector(x * s, y * s) 168 | def length: Double = math.sqrt(x * x + y * y) 169 | } 170 | 171 | Lets try to write a property involving this vector, for example, one that says that 172 | if we multiply a vector with a scalar larger than `1.0`, its length will increase. 173 | Otherwise, its length will decrease. 174 | 175 | property("* and length") = forAll { (v: Vector, s: Int) => 176 | if (s >= 1.0) (v * s).length >= v.length 177 | else (v * s).length < v.length 178 | } 179 | 180 | The snippet above does not compile -- instead it produces some strange error message 181 | about not finding an implicit value: 182 | 183 | - could not find implicit value for parameter a1: 184 | org.scalacheck.Arbitrary[pp.scalacheck.VectorSpecification.Vector] 185 | 186 | What the compiler is trying to tell us here is that the ScalaCheck library knows nothing 187 | about our user-defined `Vector` datatype, so it can't generate input examples for it. 188 | 189 | This is where ScalaCheck generators come into play. 190 | A generator (`Gen`) is an object which describes how input data is generated for 191 | a certain type. 192 | ScalaCheck comes with a range of predefined generators for basic datatypes. 193 | For example, the generator `Arbitrary.arbitrary[Double]` generates arbitrary real numbers: 194 | 195 | val doubles = Arbitrary.arbitrary[Double] 196 | 197 | The generator `Gen.oneOf` generates the specified values, in the example below `true` or `false`: 198 | 199 | val booleans = Gen.oneOf(true, false) 200 | 201 | Another predefined generator `Gen.choose` picks integer numbers within a given range: 202 | 203 | val twodigits = Gen.choose(-100, 100) 204 | 205 | How to use a generator in a property? We simply have to specify it after the `forAll`: 206 | 207 | property("sqrt") = forAll(ints) { d => 208 | math.sqrt(d * d) == d 209 | } 210 | 211 | Btw, the above property is not always true. Try to figure out a counterexample. 212 | 213 | Finally, several basic generators can be combined into more complex generators, 214 | and this is a recommended strategy to obtain generators for more custom datatypes. 215 | This is done using for-comprehensions which you already saw on collections: 216 | 217 | val vectors: Gen[Vector] = 218 | for { 219 | x <- Gen.choose(-100, 100) 220 | y <- Gen.choose(-100, 100) 221 | } yield Vector(x, y) 222 | 223 | The above should be read as -- we define a generators called `vectors` which generates 224 | `Vector`s as follows: use a generator `choose` to generate a value `x` in the range from 225 | `-100` to `100`, and another generator `choose` to generate a value `y` in the same range, 226 | then use these values to create `Vector(x, y)`. 227 | 228 | This (intuitive) for-comprehension is translated into a chain of (unintuitive) `map`/`flatMap` 229 | calls. Try to figure out what the desugared expression looks like! 230 | 231 | Now that we have the generator for vectors, we can use it: 232 | 233 | property("* and length") = forAll(vectors, ints) { (v: Vector, s: Int) => 234 | if (s >= 1.0) (v * s).length >= v.length 235 | else (v * s).length < v.length 236 | } 237 | 238 | Alas, our property still fails because `s` might be negative. Fix it to use only 239 | positive scalars! 240 | 241 | We conclude this short tutorial by showing how you can write an even more complex 242 | generator. Lets create a custom datatype for binary trees of integers: 243 | 244 | trait Tree 245 | case class Node(left: Tree, right: Tree) extends Tree 246 | case class Leaf(x: Int) extends Tree 247 | 248 | val ints = Gen.choose(-100, 100) 249 | 250 | def leafs: Gen[Leaf] = for { 251 | x <- ints 252 | } yield Leaf(x) 253 | 254 | def nodes: Gen[Node] = for { 255 | left <- trees 256 | right <- trees 257 | } yield Node(left, right) 258 | 259 | def trees: Gen[Tree] = Gen.oneOf(leafs, nodes) 260 | 261 | Just like the tree datatype, its generator consists of several cases and is mutually recursive. 262 | Try to figure out how it works. 263 | 264 | Then define a method `depth` on `Tree`s which returns the depth of the tree. 265 | Write a ScalaCheck test to verify if for any two trees `x` and `y`, the tree 266 | `Node(x, y)` is deeper than both `x` and `y`. 267 | 268 | 269 | 270 | 271 | 272 | 273 | 274 | 275 | -------------------------------------------------------------------------------- /lib/scalacheck_2.9.2-1.10.0.jar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/axel22/scalacheck-tutorial/e10e68abb323d5e8a5fe65999fa054f7a840e6bb/lib/scalacheck_2.9.2-1.10.0.jar -------------------------------------------------------------------------------- /src/test/scala/pp/scalacheck/IntSpecification.scala: -------------------------------------------------------------------------------- 1 | package pp.scalacheck 2 | 3 | import org.scalacheck._ 4 | import Prop.forAll 5 | 6 | object IntSpecification extends Properties("Int") { 7 | 8 | val ints = Gen.choose(-100, 100) 9 | 10 | property("sqrt") = forAll(ints) { d => 11 | math.sqrt(d * d) == d 12 | } 13 | 14 | } -------------------------------------------------------------------------------- /src/test/scala/pp/scalacheck/ListSpecification.scala: -------------------------------------------------------------------------------- 1 | package pp.scalacheck 2 | 3 | import org.scalacheck._ 4 | import Prop.forAll 5 | 6 | object ListSpecification extends Properties("List") { 7 | 8 | property("double reverse") = forAll { (lst: List[Int]) => 9 | lst.reverse.reverse == lst 10 | } 11 | 12 | // property("zip reverse") = forAll { (a: List[Int], b: List[Int]) => 13 | // (a.reverse zip b.reverse) == (a zip b).reverse 14 | // } 15 | 16 | // property("zip reverse") = forAll { (a: List[Int], b: List[Int]) => 17 | // val a1 = a.take(b.length) 18 | // val b1 = b.take(a.length) 19 | // (a1.reverse zip b1.reverse) == (a1 zip b1).reverse 20 | // } 21 | 22 | } -------------------------------------------------------------------------------- /src/test/scala/pp/scalacheck/MapSpecification.scala: -------------------------------------------------------------------------------- 1 | package pp.scalacheck 2 | 3 | import org.scalacheck._ 4 | import Prop.forAll 5 | 6 | object MapSpecification extends Properties("Map") { 7 | 8 | property("+ and size") = forAll { (m: Map[Int, Int], key: Int, value: Int) => 9 | (m + (key -> value)).size == (m.size + 1) 10 | } 11 | 12 | } -------------------------------------------------------------------------------- /src/test/scala/pp/scalacheck/StringSpecification.scala: -------------------------------------------------------------------------------- 1 | package pp.scalacheck 2 | 3 | import org.scalacheck._ 4 | import Prop.forAll 5 | 6 | object StringSpecification extends Properties("String") { 7 | 8 | property("startsWith") = forAll { (x: String, y: String) => 9 | (x + y).startsWith(x) 10 | } 11 | 12 | // property("concatenation length") = forAll { (x: String, y: String) => 13 | // x.length < (x + y).length 14 | // } 15 | 16 | // property("concatenation length") = forAll { (x: String, y: String) => 17 | // x.length <= (x + y).length 18 | // } 19 | 20 | // property("substring") = forAll { (a: String, b: String, c: String) => 21 | // (a+b+c).substring(a.length, a.length+b.length) == b 22 | // } 23 | 24 | } -------------------------------------------------------------------------------- /src/test/scala/pp/scalacheck/TreeSpecification.scala: -------------------------------------------------------------------------------- 1 | package pp.scalacheck 2 | 3 | import org.scalacheck._ 4 | import Prop.forAll 5 | 6 | object TreeSpecification extends Properties("Tree") { 7 | 8 | trait Tree 9 | case class Node(left: Tree, right: Tree) extends Tree 10 | case class Leaf(x: Int) extends Tree 11 | 12 | val ints = Gen.choose(-100, 100) 13 | 14 | def leafs: Gen[Leaf] = for { 15 | x <- ints 16 | } yield Leaf(x) 17 | 18 | def nodes: Gen[Node] = for { 19 | left <- trees 20 | right <- trees 21 | } yield Node(left, right) 22 | 23 | def trees: Gen[Tree] = Gen.oneOf(leafs, nodes) 24 | 25 | } -------------------------------------------------------------------------------- /src/test/scala/pp/scalacheck/VectorSpecification.scala: -------------------------------------------------------------------------------- 1 | package pp.scalacheck 2 | 3 | import org.scalacheck._ 4 | import Prop.forAll 5 | 6 | object VectorSpecification extends Properties("Vector") { 7 | 8 | case class Vector(x: Int, y: Int) { 9 | def *(s: Int) = Vector(x * s, y * s) 10 | def length: Double = math.sqrt(x * x + y * y) 11 | } 12 | 13 | // property("* and length") = forAll { (v: Vector, s: Int) => 14 | // if (s >= 1.0) (v * s).length >= v.length 15 | // else (v * s).length < v.length 16 | // } 17 | 18 | val ints = Gen.choose(-100, 100) 19 | 20 | val vectors: Gen[Vector] = for { 21 | x <- ints 22 | y <- ints 23 | } yield Vector(x, y) 24 | 25 | // property("* and length") = forAll(vectors, ints) { (v: Vector, s: Int) => 26 | // if (s >= 1.0) (v * s).length >= v.length 27 | // else (v * s).length < v.length 28 | // } 29 | 30 | // property("* and length") = forAll(vectors, ints) { (v: Vector, s0: Int) => 31 | // val s = math.abs(s0) 32 | // if (s >= 1.0) (v * s).length >= v.length 33 | // else (v * s).length < v.length 34 | // } 35 | 36 | // implicit val arbitraryVector = Arbitrary(vectors) 37 | // 38 | // implicit val arbitraryInts = Arbitrary(ints) 39 | // 40 | // property("* and length") = forAll { (v: Vector, s0: Int) => 41 | // val s = math.abs(s0) 42 | // if (s >= 1.0) (v * s).length >= v.length 43 | // else (v * s).length < v.length 44 | // } 45 | 46 | 47 | 48 | } --------------------------------------------------------------------------------