├── .gitignore ├── README.md ├── bench-all-gee ├── bench-all-lambda ├── bench-all-lamppc18 ├── bench-all-maglite ├── bench-all-moxie ├── bench-all-wolf ├── lib ├── high-scale-lib.jar └── scalatest-1.4-SNAPSHOT.jar-backup ├── project ├── build.properties └── build │ ├── benchtask.scala.disabled │ └── projdef.scala ├── src ├── bench │ └── scala │ │ └── ctries │ │ ├── FixedLookupInserts.scala │ │ ├── FlatHash.scala │ │ ├── Global.scala │ │ ├── Iteration.scala │ │ ├── Lengths.scala │ │ ├── LookupInserts.scala │ │ ├── MemoryOccupancyTest.scala │ │ ├── MultiThreadInsert.scala │ │ ├── MultiThreadLookup.scala │ │ ├── MultiThreadReinsert.scala │ │ ├── MultiThreadRemove.scala │ │ ├── MultiThreadUpdates.scala │ │ ├── PageRank.scala │ │ ├── SingleThreadInsert.scala │ │ └── Snapshots.scala ├── main │ ├── java │ │ ├── Foo.java │ │ ├── ctries │ │ │ ├── CNodeBase.java │ │ │ ├── ConcurrentTrieBase.java │ │ │ └── INodeBase.java │ │ └── ctries2 │ │ │ ├── BasicNode.java │ │ │ ├── CNodeBase.java │ │ │ ├── ConcurrentTrieBase.java │ │ │ ├── GenOld.java │ │ │ ├── INodeBase.java │ │ │ └── MainNode.java │ └── scala │ │ ├── ctries │ │ └── ConcurrentTrie.scala │ │ └── ctries2 │ │ └── ConcurrentTrie.scala └── test │ └── scala │ ├── ctries │ └── ctries.scala │ └── ctries2 │ ├── DumbHash.scala │ ├── Wrap.scala │ ├── basics.scala │ ├── concmap.scala │ ├── ctries2.scala │ ├── iteratorspec.scala │ ├── lnodes.scala │ └── snapshots.scala └── tools ├── bench-all ├── bench-insert ├── bench-iter ├── bench-lookup ├── bench-pagerank ├── bench-remove ├── bench-snapshot ├── bench-snapshot-lookup ├── bench-update ├── toplot └── totikz /.gitignore: -------------------------------------------------------------------------------- 1 | # project dependencies 2 | project/boot 3 | 4 | # target dir contents 5 | target 6 | 7 | # garbage 8 | *~ 9 | *.*~ 10 | **/*.*~ 11 | **/*~ 12 | tmp 13 | tmp/ 14 | lib_managed/ 15 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Ctrie - A Lock-Free Concurrent Hash Array Mapped Trie 2 | 3 | A concurrent hash-trie or Ctrie is a concurrent thread-safe lock-free implementation of a hash array mapped trie. It is used to implement the concurrent map abstraction. It has particularly scalable concurrent insert and remove operations and is memory-efficient. It supports O(1), atomic, lock-free snapshots which are used to implement linearizable lock-free size, iterator and clear operations. The cost of evaluating the (lazy) snapshot is distributed across subsequent updates, thus making snapshot evaluation horizontally scalable. 4 | 5 | The implementation is written in Scala (http://www.scala-lang.org/), a statically compiled language for the JVM. It is fully compliant with the collection package from the Scala standard library. 6 | 7 | As the Ctrie is a part of the Scala standard library since the version 2.10, clients using versions 2.9.x can use this implementation. 8 | 9 | More info about Ctries: 10 | 11 | - http://infoscience.epfl.ch/record/166908/files/ctries-techreport.pdf - this is a nice introduction to Ctries, along with a correctness proof 12 | - http://lamp.epfl.ch/~prokopec/ctries-snapshot.pdf - a more up-to-date writeup (more coherent with the current version of the code) which describes the snapshot operation 13 | 14 | 15 | ## How to run 16 | 17 | __1) Requirements__ 18 | 19 | - JDK1.6 or higher 20 | - SBT (simple build tool) - see http://code.google.com/p/simple-build-tool/wiki/Setup 21 | 22 | __2) Run SBT__ 23 | 24 | Once the `sbt` command is run in the root directory of the project, it will download the appropriate Scala library and compiler from Maven. It will start the SBT interactive shell. Generally, to run SBT tasks, you have to start the SBT shell. 25 | 26 | __3) Update dependencies__ 27 | 28 | Within the SBT shell type the command: 29 | 30 | > update 31 | 32 | and hit ENTER. This will resolve all the library dependencies from Maven (e.g. the ScalaTest library). 33 | 34 | __4) Compile__ 35 | 36 | Within the SBT shell: 37 | 38 | > compile 39 | 40 | This will compile the project. After this, you can run tests using the `test` command or run benchmarks using the `bench` command. 41 | 42 | 43 | ### Packages 44 | 45 | The `ctries2` package contains an up-to-date version of the Ctrie data structure. 46 | 47 | The `ctries` package contains a previous version of the Ctrie data structure. The difference is that the previous version uses one I-node per a stored key-value pair, resulting in an additional indirection and increased memory usage. This previous version is useful to compare performance and memory usage against the preferred version in the `ctries2` package. 48 | 49 | 50 | ### Branches 51 | 52 | The `master` branch contains an up-to-date version of the Ctrie data structure with snapshot support implemented. 53 | 54 | The `no-snapshots` branch contains the version of the Ctrie data structure without snapshot support. This branch is useful to compare snapshot support overhead (in my experiments 10-20% slowdown on a 1M element dataset). 55 | 56 | 57 | ### Tests 58 | 59 | Tests are run using the `test` command in the SBT shell. They can also be run selectively like this: 60 | 61 | > test-only ctries2.LNodeSpec 62 | 63 | which runs the tests associated with L-nodes. 64 | 65 | 66 | ### Benchmarks 67 | 68 | Benchmarks are located in the `src/bench/scala` directory. They are run by using the `bench` command in the SBT shell, which will start a new JVM instance, do the warmup and run the selected snippet a number of times. Generally, you will have to inspect what parameters the specific benchmark expects in order to run it. For example, to run the insertion benchmark for 500k elements, using 4 threads and repeat this test 6 times, you have to write: 69 | 70 | > bench -Dsz=500000 -Dpar=4 ctries.MultiInsertCtrie2 6 1 71 | 72 | This specific benchmark will evenly distribute the work of inserting `sz` elements into an empty Ctrie between `par` threads. 73 | 74 | It's also possible to run benchmarks in batches to produce a range of results using the `bench-batch` task in the SBT shell. This will run a Perl script to gather the data and convert them to Latex tikz format used for figures. 75 | 76 | All benchmarks presented in the paper are available within this project. 77 | -------------------------------------------------------------------------------- /bench-all-gee: -------------------------------------------------------------------------------- 1 | tools/bench-all gee 1,2,4,6,8,10,12,14,16,18,20,24,28,32,36,40,44,48,52,56,60,64 1000000 5 4 1 1000000 non-uniform 12 0.6 32000 2 | -------------------------------------------------------------------------------- /bench-all-lambda: -------------------------------------------------------------------------------- 1 | tools/bench-all lambda 1,2,4,6,8,10,12,14,16 32000,125000,500000,1000000 5 4 1 15000000 non-uniform 12 0.6 32000 2 | -------------------------------------------------------------------------------- /bench-all-lamppc18: -------------------------------------------------------------------------------- 1 | tools/bench-all lamppc18 1,2,4,6,8 32000,125000,500000,1000000 5 4 1 1200000 non-uniform 12 0.6 32000 2 | -------------------------------------------------------------------------------- /bench-all-maglite: -------------------------------------------------------------------------------- 1 | tools/bench-all maglite 1,2,4,6,8,10,12,14,16,18,20,24,28,32,36,40,44,48,52,56,60,64 1000000 5 4 1 1000000 non-uniform 12 0.6 32000 2 | -------------------------------------------------------------------------------- /bench-all-moxie: -------------------------------------------------------------------------------- 1 | tools/bench-all moxie 1,2,4,6,8,10,12,14,16,18,20,24,28,32,36,40,44,48,52,56,60,64,68,72,76,80,84,88,92,96,100,104,108,112,116,120,124,128 1000000 5 4 1 1000000 non-uniform 12 0.6 32000 2 | -------------------------------------------------------------------------------- /bench-all-wolf: -------------------------------------------------------------------------------- 1 | tools/bench-all wolf 1,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,36,40,44,48,52,56,60,64 1000000 5 4 1 1000000 non-uniform 12 0.6 32000 2 | -------------------------------------------------------------------------------- /lib/high-scale-lib.jar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/axel22/Ctries/2f9afb396a122632dcea221c9d6f8a9c9f18cb36/lib/high-scale-lib.jar -------------------------------------------------------------------------------- /lib/scalatest-1.4-SNAPSHOT.jar-backup: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/axel22/Ctries/2f9afb396a122632dcea221c9d6f8a9c9f18cb36/lib/scalatest-1.4-SNAPSHOT.jar-backup -------------------------------------------------------------------------------- /project/build.properties: -------------------------------------------------------------------------------- 1 | #Project properties 2 | #Tue Mar 15 16:11:45 CET 2011 3 | project.organization=epfl 4 | project.name=CTries 5 | sbt.version=0.7.4 6 | project.version=1.0 7 | build.scala.versions=2.9.1 8 | project.initialize=false 9 | -------------------------------------------------------------------------------- /project/build/benchtask.scala.disabled: -------------------------------------------------------------------------------- 1 | def benchLabel = "bench" 2 | 3 | def benchCompileDirectoryName = benchLabel 4 | 5 | def benchCompilePath = outputPath / benchCompileDirectoryName 6 | 7 | def benchClasses = (benchCompilePath ##) ** "*.class" 8 | 9 | def benchPath = benchClasses 10 | 11 | def benchJarName = artifactBaseName + "-bench" + ".jar" 12 | 13 | def benchJarPath = outputPath / benchJarName 14 | 15 | def benchSourcePath = sourcePath / benchLabel 16 | 17 | def benchSourceScala = benchSourcePath / scalaDirectoryName 18 | 19 | class BenchCompileConfig extends BaseCompileConfig { 20 | def baseCompileOptions = testCompileOptions 21 | def label = benchLabel 22 | def sourceRoots = benchSourceScala 23 | def sources = (benchSourceScala ##) ** "*.scala" 24 | def outputDirectory = benchCompilePath 25 | def classpath = testClasspath 26 | def analysisPath = outputPath / "bench-analysis" 27 | def javaOptions = testJavaCompileOptions.map(_.asString) 28 | def fingerprints = Fingerprints(Nil, Nil) 29 | } 30 | 31 | lazy val packageBench = packageTask(benchPath, benchJarPath, packageOptions).dependsOn(compileBench) 32 | 33 | lazy val compileBench = task { 34 | (new CompileConditional(new BenchCompileConfig, buildCompiler)).run 35 | } dependsOn (`package`) 36 | 37 | -------------------------------------------------------------------------------- /project/build/projdef.scala: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | import sbt._ 5 | import Process._ 6 | import java.io._ 7 | 8 | 9 | 10 | class Ctries(info: ProjectInfo) extends DefaultProject(info) { 11 | 12 | /* deps */ 13 | 14 | val scalatest = "org.scalatest" %% "scalatest" % "1.6.1" 15 | val scalacheck = "org.scala-tools.testing" %% "scalacheck" % "1.9" 16 | 17 | /* config */ 18 | 19 | override def testSourceRoots = super.testSourceRoots +++ (sourcePath / "bench") 20 | 21 | override def compileOptions = super.compileOptions ++ compileOptions("-Yinline") 22 | 23 | /* helpers */ 24 | 25 | def loginfo(msg: String) = log.log(Level.Info, msg) 26 | 27 | def runsync(comm: String) { 28 | loginfo("running: %s".format(comm)) 29 | comm !; 30 | } 31 | 32 | def libsPath = ("lib" ** "*.jar").get.mkString(":") 33 | 34 | def benchcomm(args: String) = "java -Xmx2048m -Xms2048m -server -cp %s:%s:%s:%s %s".format( 35 | packageTestJar, 36 | buildScalaInstance.libraryJar, 37 | jarPath, 38 | libsPath, 39 | args) 40 | 41 | val plotresfile = File.createTempFile("plot", "out") 42 | plotresfile.deleteOnExit(); 43 | 44 | val gplotflag = "--gnuplot" 45 | 46 | val totruns = 25 47 | 48 | def generatePlots(cases: Seq[String], paramtups: List[(String, List[String])], input: String, output: String) = for (((pname, _), i) <- paramtups.zipWithIndex) { 49 | loginfo("plotting against %s".format(pname)) 50 | 51 | val plotfile = new File("tmp/generateplot") 52 | val contents = """ 53 | #!/usr/bin/gnuplot 54 | 55 | set terminal png 56 | set grid 57 | set xlabel "%s" 58 | set ylabel "time" 59 | set out "./tmp/%s" 60 | 61 | plot %s 62 | """.format( 63 | pname.split('=')(0), 64 | output + i + ".png", 65 | (for (j <- 0 until cases.length) yield { 66 | """ "%s" using %d:%d title "%s" with linespoints""".format( 67 | input, 68 | i + 1, 69 | paramtups.length + 1 + j, 70 | cases(j) 71 | ) 72 | }).mkString(", ") 73 | ) 74 | 75 | val out = new java.io.FileWriter(plotfile) 76 | out.write(contents) 77 | out.close 78 | 79 | "gnuplot %s".format(plotfile) !; 80 | } 81 | 82 | /* tasks */ 83 | 84 | lazy val justTestOnly = testQuickMethod(testCompileConditional.analysis, testOptions)(o => testTask(testFrameworks, testClasspath, testCompileConditional.analysis, o)) 85 | 86 | lazy val justTest = testTask(testFrameworks, testClasspath, testCompileConditional.analysis, testOptions) dependsOn (compile) 87 | 88 | lazy val bench = task { args => 89 | task { 90 | runsync(benchcomm(args.mkString(" "))) 91 | None 92 | } dependsOn (`package`, packageTest) 93 | } 94 | 95 | lazy val benchBatch = task { args => 96 | task { 97 | val (params, nonparams) = args.partition(_.startsWith("-D")) 98 | val (other, benches) = nonparams.partition(_.startsWith("--")) 99 | val othermap = Map(other.map(_.split('=')).map(x => (x(0), x(1))): _*) 100 | 101 | val paramtups = for { 102 | p <- params.toList 103 | val pts = p.split('=') 104 | val (name, vals) = (pts(0), pts(1).split(',')) 105 | } yield (p, vals.toList) 106 | 107 | def forparams(paramlist: List[(String, List[String])], args: List[String], accs: String)(f: (List[String], String) => Unit): Unit = paramlist match { 108 | case head :: tail => for (v <- head._2) forparams(tail, args ::: List(v), "%s %s=%s".format(accs, head._1.split('=')(0), v))(f) 109 | case Nil => f(args, accs) 110 | } 111 | 112 | if (othermap contains (gplotflag)) { 113 | val gpf = ("tmp" / othermap(gplotflag)).asFile 114 | gpf.delete() 115 | gpf.createNewFile() 116 | 117 | forparams(paramtups, Nil, "") { (lst, s) => 118 | plotresfile.delete() 119 | plotresfile.createNewFile() 120 | for (bench <- benches) { 121 | loginfo("running for: %s, %s".format(bench, s)) 122 | val bcomm = benchcomm("%s %s %d 1".format(s, bench, totruns)) 123 | //loginfo("command: %s".format(bcomm)) 124 | bcomm #| """sed -e s/[^0-9\t]//g """.format(bench) #>> plotresfile !; 125 | } 126 | //"cat %s".format(plotresfile.getAbsolutePath) !; 127 | "echo -ne %s ".format(lst.mkString(" ")) #>> gpf !; 128 | runsync("tools/toplot %s %s".format(plotresfile.getAbsolutePath, gpf.toString)) 129 | } 130 | 131 | generatePlots(benches, paramtups, gpf.getAbsolutePath, othermap(gplotflag)) 132 | } else { 133 | forparams(paramtups, Nil, "") { (lst, s) => 134 | for (bench <- benches) { 135 | runsync(benchcomm("%s %s %d 1".format(s, bench, totruns))) 136 | } 137 | } 138 | } 139 | 140 | None 141 | } dependsOn (`package`, packageTest) 142 | } 143 | 144 | } 145 | -------------------------------------------------------------------------------- /src/bench/scala/ctries/FixedLookupInserts.scala: -------------------------------------------------------------------------------- 1 | package ctries 2 | 3 | 4 | 5 | import Global._ 6 | import scala.testing.Benchmark 7 | 8 | 9 | 10 | object FixedLookupInsertsCHM extends Benchmark { 11 | import java.util.concurrent.ConcurrentHashMap 12 | val chm = new ConcurrentHashMap[Elem, Elem] 13 | 14 | def run() { 15 | val p = par.get 16 | val step = sz / p 17 | 18 | val ins = for (i <- 0 until p) yield new Worker(chm, i, step) 19 | 20 | for (i <- ins) i.start() 21 | for (i <- ins) i.join() 22 | } 23 | 24 | class Worker(chm: ConcurrentHashMap[Elem, Elem], n: Int, step: Int) extends Thread { 25 | override def run() { 26 | val ratio = lookupratio.get 27 | var i = n * step 28 | val until = (n + 1) * step 29 | val e = elems 30 | while (i < until) { 31 | // do an insert 32 | chm.put(e(i), e(i)) 33 | i += 1 34 | 35 | // do some lookups 36 | var j = 0 37 | while (j < ratio) { 38 | chm.get(e(math.abs(j * 0x9e3775cd) % i)) 39 | j += 1 40 | } 41 | } 42 | } 43 | } 44 | } 45 | 46 | 47 | object FixedLookupInsertsCtrie extends Benchmark { 48 | val ct = new ctries.ConcurrentTrie[Elem, Elem] 49 | 50 | def run() { 51 | val p = par.get 52 | val step = sz / p 53 | 54 | val ins = for (i <- 0 until p) yield new Worker(ct, i, step) 55 | 56 | for (i <- ins) i.start() 57 | for (i <- ins) i.join() 58 | } 59 | 60 | class Worker(ct: ctries.ConcurrentTrie[Elem, Elem], n: Int, step: Int) extends Thread { 61 | override def run() { 62 | val ratio = lookupratio.get 63 | var i = n * step 64 | val until = (n + 1) * step 65 | val e = elems 66 | while (i < until) { 67 | // do an insert 68 | ct.insert(e(i), e(i)) 69 | i += 1 70 | 71 | // do some lookups 72 | var j = 0 73 | while (j < ratio) { 74 | ct.lookup(e(math.abs(j * 0x9e3775cd) % i)) 75 | j += 1 76 | } 77 | } 78 | } 79 | } 80 | } 81 | 82 | 83 | object FixedLookupUpdatesCtrie2 extends Benchmark { 84 | val ct = new ctries2.ConcurrentTrie[Elem, Elem] 85 | 86 | def run() { 87 | val p = par.get 88 | val step = sz / p 89 | 90 | val ins = for (i <- 0 until p) yield new Worker(ct, i, step) 91 | 92 | for (i <- ins) i.start() 93 | for (i <- ins) i.join() 94 | } 95 | 96 | class Worker(ct: ctries2.ConcurrentTrie[Elem, Elem], n: Int, step: Int) extends Thread { 97 | override def run() { 98 | val ratio = lookupratio.get 99 | var i = n * step 100 | val until = (n + 1) * step 101 | val e = elems 102 | while (i < until) { 103 | // do an update 104 | ct.update(e(i), e(i)) 105 | i += 1 106 | 107 | // do some lookups 108 | var j = 0 109 | while (j < ratio) { 110 | ct.lookup(e(math.abs(j * 0x9e3775cd) % i)) 111 | j += 1 112 | } 113 | } 114 | } 115 | } 116 | } 117 | 118 | -------------------------------------------------------------------------------- /src/bench/scala/ctries/FlatHash.scala: -------------------------------------------------------------------------------- 1 | package ctries 2 | 3 | 4 | 5 | import Global._ 6 | import scala.testing.Benchmark 7 | 8 | 9 | 10 | object FlatHash extends Benchmark { 11 | import java.util.concurrent.atomic.AtomicReferenceArray 12 | 13 | var arr: AtomicReferenceArray[Elem] = null 14 | val loadFactor = 0.45 15 | 16 | override def setUp { 17 | arr = null 18 | arr = new AtomicReferenceArray[Elem]((sz / loadFactor).toInt) 19 | Runtime.getRuntime.gc() 20 | } 21 | 22 | def run() { 23 | val p = par.get 24 | val step = sz / p 25 | 26 | val ins = for (i <- 0 until p) yield new Inserter(arr, i, step) 27 | 28 | for (i <- ins) i.start() 29 | for (i <- ins) i.join() 30 | } 31 | 32 | class Inserter(arr: AtomicReferenceArray[Elem], n: Int, step: Int) extends Thread { 33 | override def run() { 34 | var i = n * step 35 | val until = (n + 1) * step 36 | val e = elems 37 | 38 | while (i < until) { 39 | flat_insert(arr, e(i)) 40 | i += 1 41 | } 42 | } 43 | } 44 | 45 | @inline def flat_insert(arr: AtomicReferenceArray[Elem], e: Elem) { 46 | val len = arr.length 47 | var idx = e.hashCode % len 48 | if (idx < 0) idx = -idx 49 | 50 | do { 51 | while (arr.get(idx) != null) idx = (idx + 1) % len 52 | if (arr.weakCompareAndSet(idx, null, e)) idx = -1 53 | } while (idx != -1) 54 | } 55 | 56 | } 57 | 58 | 59 | 60 | object FlatHashNoComm extends Benchmark { 61 | import java.util.concurrent.atomic.AtomicReferenceArray 62 | 63 | def run() { 64 | val p = par.get 65 | val step = sz / p 66 | 67 | val ins = for (i <- 0 until p) yield new Inserter(0, i, step) 68 | 69 | for (i <- ins) i.start() 70 | for (i <- ins) i.join() 71 | } 72 | 73 | class Inserter(var s: Int = 0, n: Int, step: Int) extends Thread { 74 | override def run() { 75 | var i = n * step 76 | val until = (n + 1) * step 77 | val e = elems 78 | 79 | while (i < until) { 80 | s += i * i % (n + 1) + step 81 | i += 1 82 | } 83 | } 84 | } 85 | 86 | } 87 | 88 | -------------------------------------------------------------------------------- /src/bench/scala/ctries/Global.scala: -------------------------------------------------------------------------------- 1 | package ctries 2 | 3 | 4 | 5 | 6 | 7 | 8 | case class Elem(i: Int) extends Comparable[Elem] { 9 | def compareTo(y: Elem) = i - y.i 10 | 11 | override def hashCode = { 12 | // var hc = i * 0x9e3775cd 13 | // hc = java.lang.Integer.reverseBytes(hc) 14 | // hc * 0x9e3775cd 15 | 16 | var hc = i * 0x9e3775cd 17 | hc 18 | 19 | //var hc = i * 0x9e3775cd 20 | //hc + (hc << 16) + i 21 | 22 | //i 23 | } 24 | } 25 | 26 | 27 | object Global { 28 | val sz = System.getProperty("sz").toInt 29 | val par = Option(System.getProperty("par")).map(_.toInt) 30 | val lookups = Option(System.getProperty("lookups")).map(_.toInt) 31 | val inserts = Option(System.getProperty("inserts")).map(_.toInt) 32 | val removes = Option(System.getProperty("removes")).map(_.toInt) 33 | val totalops = Option(System.getProperty("totalops")).map(_.toInt) 34 | val lookupratio = Option(System.getProperty("lookupratio")).map(_.toInt) 35 | val damping = Option(System.getProperty("damping")).map(_.toDouble) 36 | val maxlinks = Option(System.getProperty("maxlinks")).map(_.toInt) 37 | val pagegenerator = Option(System.getProperty("pagegenerator")) 38 | val updateFilled = Option(System.getProperty("updateFilled")).map(_.toBoolean).getOrElse(false) 39 | val debug = Option(System.getProperty("debug")).map(_.toBoolean).getOrElse(false) 40 | val elems = (for (i <- 0 until (sz * 2)) yield Elem(i)).toArray 41 | } 42 | 43 | 44 | -------------------------------------------------------------------------------- /src/bench/scala/ctries/Iteration.scala: -------------------------------------------------------------------------------- 1 | package ctries 2 | 3 | 4 | 5 | import Global._ 6 | import scala.testing.Benchmark 7 | 8 | 9 | 10 | object IterationCtrie2 extends Benchmark { 11 | import ctries2.ConcurrentTrie 12 | 13 | val ct = new ConcurrentTrie[Elem, Elem] 14 | for (i <- 0 until sz) ct.put(elems(i), elems(i)) 15 | 16 | def run() { 17 | val it = ct.iterator 18 | 19 | while (it.hasNext) it.next() 20 | } 21 | 22 | } 23 | 24 | 25 | // should be the same as iteration of a ctrie2 26 | object IterationCtrie2ReadOnly extends Benchmark { 27 | import ctries2.ConcurrentTrie 28 | 29 | val ct = new ConcurrentTrie[Elem, Elem] 30 | for (i <- 0 until sz) ct.put(elems(i), elems(i)) 31 | val ro = ct.readOnlySnapshot() 32 | 33 | def run() { 34 | val it = ro.iterator 35 | 36 | while (it.hasNext) it.next() 37 | } 38 | 39 | } 40 | 41 | 42 | object IterationSkipList extends Benchmark { 43 | val csl = new java.util.concurrent.ConcurrentSkipListMap[Elem, Elem] 44 | for (i <- 0 until sz) csl.put(elems(i), elems(i)) 45 | 46 | def run() { 47 | val it = csl.entrySet.iterator 48 | 49 | while (it.hasNext) it.next() 50 | } 51 | 52 | } 53 | 54 | 55 | object IterationCHM extends Benchmark { 56 | val chm = new java.util.concurrent.ConcurrentHashMap[Elem, Elem] 57 | for (i <- 0 until sz) chm.put(elems(i), elems(i)) 58 | 59 | def run() { 60 | val it = chm.entrySet.iterator 61 | 62 | while (it.hasNext) it.next() 63 | } 64 | 65 | } 66 | 67 | 68 | object IterationHashTrie extends Benchmark { 69 | var ht = collection.immutable.HashMap[Elem, Elem]() 70 | for (i <- 0 until sz) ht = ht + ((elems(i), elems(i))) 71 | 72 | def run() { 73 | val it = ht.iterator 74 | 75 | while (it.hasNext) it.next() 76 | } 77 | } 78 | 79 | 80 | object IterationHashTable extends Benchmark { 81 | val hm = collection.mutable.HashMap[Elem, Elem]() 82 | for (i <- 0 until sz) hm(elems(i)) = elems(i) 83 | 84 | def run() { 85 | val it = hm.iterator 86 | 87 | while (it.hasNext) it.next() 88 | } 89 | } 90 | 91 | 92 | object IterationList extends Benchmark { 93 | var lst = List[(Elem, Elem)]() 94 | for (i <- 0 until sz) lst ::= (elems(i), elems(i)) 95 | 96 | def run() { 97 | val it = lst.iterator 98 | 99 | while (it.hasNext) it.next() 100 | } 101 | } 102 | 103 | 104 | object IterationArray extends Benchmark { 105 | var arr = new Array[(Elem, Elem)](sz) 106 | for (i <- 0 until sz) arr(i) = (elems(i), elems(i)) 107 | 108 | def run() { 109 | val it = arr.iterator 110 | 111 | while (it.hasNext) it.next() 112 | } 113 | } 114 | -------------------------------------------------------------------------------- /src/bench/scala/ctries/Lengths.scala: -------------------------------------------------------------------------------- 1 | package ctries 2 | 3 | 4 | 5 | import Global._ 6 | 7 | 8 | 9 | object Lengths { 10 | import ctries2._ 11 | 12 | val ct = new ConcurrentTrie[Elem, Elem] 13 | for (i <- 0 until sz) ct.put(elems(i), elems(i)) 14 | 15 | def maxdepth[K, V](ct: ConcurrentTrie[K, V], in: INode[K, V], acc: Int): Int = { 16 | in.GCAS_READ(ct) match { 17 | case cn: CNode[K, V] => (for (elem <- cn.array) yield elem match { 18 | case sin: INode[K, V] => maxdepth(ct, sin, acc + 1) 19 | case sn: SNode[K, V] => acc + 1 20 | }).max 21 | case tn: TNode[K, V] => acc + 1 22 | case ln: LNode[K, V] => acc + 1 23 | case null => acc + 1 24 | } 25 | } 26 | 27 | def avgdepth[K, V](ct: ConcurrentTrie[K, V], in: INode[K, V], d: Long): (Int, Long) = { 28 | in.GCAS_READ(ct) match { 29 | case cn: CNode[K, V] => (for (elem <- cn.array) yield elem match { 30 | case sin: INode[K, V] => avgdepth(ct, sin, d + 1) 31 | case sn: SNode[K, V] => (1, d) 32 | }).reduceLeft((a, b) => (a._1 + b._1, a._2 + b._2)) 33 | case tn: TNode[K, V] => (1, d) 34 | case ln: LNode[K, V] => (1, d) 35 | case null => (1, d) 36 | } 37 | } 38 | 39 | def avgnodes[K, V](ct: ConcurrentTrie[K, V], in: INode[K, V], d: Long): (Int, Long) = { 40 | in.GCAS_READ(ct) match { 41 | case cn: CNode[K, V] => (for (elem <- cn.array) yield elem match { 42 | case sin: INode[K, V] => avgnodes(ct, sin, d + 2) 43 | case sn: SNode[K, V] => (1, d + 2) 44 | }).reduceLeft((a, b) => (a._1 + b._1, a._2 + b._2)) 45 | case tn: TNode[K, V] => (1, d + 1) 46 | case ln: LNode[K, V] => (1, d + 1) 47 | case null => (1, d + 1) 48 | } 49 | } 50 | 51 | def histogram[K, V](ct: ConcurrentTrie[K, V], in: INode[K, V], d: Int, hist: Array[Long]) { 52 | in.GCAS_READ(ct) match { 53 | case cn: CNode[K, V] => for (elem <- cn.array) elem match { 54 | case sin: INode[K, V] => histogram(ct, sin, d + 2, hist) 55 | case sn: SNode[K, V] => hist(d + 2) += 1 56 | } 57 | case tn: TNode[K, V] => hist(d + 1) += 1 58 | case ln: LNode[K, V] => hist(d + 1) += 1 59 | case null => hist(d + 1) += 1 60 | } 61 | } 62 | 63 | val HSZ = 24 64 | val HGT = 40 65 | 66 | def printHist(hist: Array[Long]) { 67 | val max = hist.max 68 | val bars = for (i <- 0 until HSZ) yield { 69 | val hgt = (1.0 * hist(i) / max * HGT).toInt 70 | (Seq.fill(HGT - hgt)(" ")) ++ 71 | (Seq.fill(hgt)(" * ")) ++ 72 | (Seq("|" + (" " * (2 - i.toString.length)) + i + " ")) 73 | } 74 | val lines = bars.transpose 75 | val decorlines = lines.reverse.zipWithIndex.map { 76 | case (line, i) => 77 | val num = (1.0 * i / HGT * max).toInt.toString 78 | Seq(" " * (9 - num.length) + num + " |") ++ line 79 | }.reverse 80 | for (line <- decorlines) { 81 | for (elem <- line) print(elem) 82 | println() 83 | } 84 | } 85 | 86 | def main(args: Array[String]) { 87 | val maxd = maxdepth(ct, ct.RDCSS_READ_ROOT(), 1) 88 | println("Maximum inode: %d".format(maxd)) 89 | 90 | val (sntotal, snpath) = avgdepth(ct, ct.RDCSS_READ_ROOT(), 1L) 91 | println("Average inode: %f = %d / %d".format(1.0 * snpath / sntotal, snpath, sntotal)) 92 | 93 | val (ndtotal, ndpath) = avgnodes(ct, ct.RDCSS_READ_ROOT(), 1L) 94 | println("Average nodes: %f = %d / %d".format(1.0 * ndpath / ndtotal, ndpath, ndtotal)) 95 | 96 | val hist = new Array[Long](HSZ) 97 | histogram(ct, ct.RDCSS_READ_ROOT(), 1, hist) 98 | printHist(hist) 99 | 100 | if (sz < 128) println(ct.string) 101 | } 102 | 103 | } 104 | -------------------------------------------------------------------------------- /src/bench/scala/ctries/LookupInserts.scala: -------------------------------------------------------------------------------- 1 | package ctries 2 | 3 | 4 | 5 | import Global._ 6 | import scala.testing.Benchmark 7 | 8 | 9 | 10 | object LookupInsertsCHM extends Benchmark { 11 | import java.util.concurrent.ConcurrentHashMap 12 | 13 | def run() { 14 | val chm = new ConcurrentHashMap[Elem, Elem] 15 | val p = par.get 16 | val step = sz / p 17 | 18 | val ins = for (i <- 0 until p) yield new Worker(chm, i, step) 19 | 20 | for (i <- ins) i.start() 21 | for (i <- ins) i.join() 22 | } 23 | 24 | class Worker(chm: ConcurrentHashMap[Elem, Elem], n: Int, step: Int) extends Thread { 25 | override def run() { 26 | val ratio = lookupratio.get 27 | var i = n * step 28 | val until = (n + 1) * step 29 | val e = elems 30 | while (i < until) { 31 | // do an insert 32 | chm.put(e(i), e(i)) 33 | i += 1 34 | 35 | // do some lookups 36 | var j = 0 37 | while (j < ratio) { 38 | chm.get(e(math.abs(j * 0x9e3775cd) % i)) 39 | j += 1 40 | } 41 | } 42 | } 43 | } 44 | } 45 | 46 | 47 | object LookupInsertsSkipList extends Benchmark { 48 | import java.util.concurrent.ConcurrentSkipListMap 49 | 50 | def run() { 51 | val skiplist = new ConcurrentSkipListMap[Elem, Elem] 52 | val p = par.get 53 | val step = sz / p 54 | 55 | val ins = for (i <- 0 until p) yield new Worker(skiplist, i, step) 56 | 57 | for (i <- ins) i.start() 58 | for (i <- ins) i.join() 59 | } 60 | 61 | class Worker(skiplist: ConcurrentSkipListMap[Elem, Elem], n: Int, step: Int) extends Thread { 62 | override def run() { 63 | val ratio = lookupratio.get 64 | var i = n * step 65 | val until = (n + 1) * step 66 | val e = elems 67 | while (i < until) { 68 | // do an insert 69 | skiplist.put(e(i), e(i)) 70 | i += 1 71 | 72 | // do some lookups 73 | var j = 0 74 | while (j < ratio) { 75 | skiplist.get(e(math.abs(j * 0x9e3775cd) % i)) 76 | j += 1 77 | } 78 | } 79 | } 80 | } 81 | } 82 | 83 | 84 | object LookupInsertsCtrie extends Benchmark { 85 | def run() { 86 | val ct = new ctries.ConcurrentTrie[Elem, Elem] 87 | val p = par.get 88 | val step = sz / p 89 | 90 | val ins = for (i <- 0 until p) yield new Worker(ct, i, step) 91 | 92 | for (i <- ins) i.start() 93 | for (i <- ins) i.join() 94 | } 95 | 96 | class Worker(ct: ctries.ConcurrentTrie[Elem, Elem], n: Int, step: Int) extends Thread { 97 | override def run() { 98 | val ratio = lookupratio.get 99 | var i = n * step 100 | val until = (n + 1) * step 101 | val e = elems 102 | while (i < until) { 103 | // do an insert 104 | ct.insert(e(i), e(i)) 105 | i += 1 106 | 107 | // do some lookups 108 | var j = 0 109 | while (j < ratio) { 110 | ct.lookup(e(math.abs(j * 0x9e3775cd) % i)) 111 | j += 1 112 | } 113 | } 114 | } 115 | } 116 | } 117 | 118 | 119 | object LookupUpdatesCtrie2 extends Benchmark { 120 | def run() { 121 | val ct = new ctries2.ConcurrentTrie[Elem, Elem] 122 | val p = par.get 123 | val step = sz / p 124 | 125 | val ins = for (i <- 0 until p) yield new Worker(ct, i, step) 126 | 127 | for (i <- ins) i.start() 128 | for (i <- ins) i.join() 129 | } 130 | 131 | class Worker(ct: ctries2.ConcurrentTrie[Elem, Elem], n: Int, step: Int) extends Thread { 132 | override def run() { 133 | val ratio = lookupratio.get 134 | var i = n * step 135 | val until = (n + 1) * step 136 | val e = elems 137 | while (i < until) { 138 | // do an update 139 | ct.update(e(i), e(i)) 140 | i += 1 141 | 142 | // do some lookups 143 | var j = 0 144 | while (j < ratio) { 145 | ct.lookup(e(math.abs(j * 0x9e3775cd) % i)) 146 | j += 1 147 | } 148 | } 149 | } 150 | } 151 | } 152 | 153 | 154 | -------------------------------------------------------------------------------- /src/bench/scala/ctries/MemoryOccupancyTest.scala: -------------------------------------------------------------------------------- 1 | package ctries 2 | 3 | 4 | 5 | import Global._ 6 | import scala.testing.Benchmark 7 | 8 | 9 | 10 | object AfterDeleteCHM extends Benchmark { 11 | import java.util.concurrent.ConcurrentHashMap 12 | 13 | def run() { 14 | val chm = new ConcurrentHashMap[Elem, Elem] 15 | val e = elems 16 | 17 | for (i <- 0 until sz) chm.put(e(i), e(i)) 18 | for (i <- 0 until sz) chm.remove(e(i)) 19 | 20 | Runtime.getRuntime.gc() 21 | while (true) {} 22 | } 23 | } 24 | 25 | 26 | object AfterDeleteSkipList extends Benchmark { 27 | import java.util.concurrent.ConcurrentSkipListMap 28 | 29 | def run() { 30 | val skiplist = new ConcurrentSkipListMap[Elem, Elem] 31 | val e = elems 32 | 33 | for (i <- 0 until sz) skiplist.put(e(i), e(i)) 34 | for (i <- 0 until sz) skiplist.remove(e(i)) 35 | 36 | Runtime.getRuntime.gc() 37 | while (true) {} 38 | } 39 | } 40 | 41 | 42 | object AfterDeleteCtrie extends Benchmark { 43 | def run() { 44 | val ctrie = new ctries.ConcurrentTrie[Elem, Elem] 45 | val e = elems 46 | 47 | for (i <- 0 until sz) ctrie.insert(e(i), e(i)) 48 | for (i <- 0 until sz) ctrie.remove(e(i)) 49 | 50 | Runtime.getRuntime.gc() 51 | while (true) {} 52 | } 53 | } 54 | 55 | 56 | object AfterDeleteCtrie2 extends Benchmark { 57 | def run() { 58 | val ctrie = new ctries2.ConcurrentTrie[Elem, Elem] 59 | val e = elems 60 | 61 | for (i <- 0 until sz) ctrie.update(e(i), e(i)) 62 | for (i <- 0 until sz) ctrie.remove(e(i)) 63 | 64 | Runtime.getRuntime.gc() 65 | while (true) {} 66 | } 67 | } 68 | 69 | 70 | object MemoryCHM extends Benchmark { 71 | import java.util.concurrent.ConcurrentHashMap 72 | 73 | def run() { 74 | val chm = new ConcurrentHashMap[Elem, Elem] 75 | val e = elems 76 | 77 | for (i <- 0 until sz) chm.put(e(i), e(i)) 78 | 79 | Runtime.getRuntime.gc() 80 | while (true) {} 81 | } 82 | } 83 | 84 | 85 | object MemorySkipList extends Benchmark { 86 | import java.util.concurrent.ConcurrentSkipListMap 87 | 88 | def run() { 89 | val skiplist = new ConcurrentSkipListMap[Elem, Elem] 90 | val e = elems 91 | 92 | for (i <- 0 until sz) skiplist.put(e(i), e(i)) 93 | 94 | Runtime.getRuntime.gc() 95 | while (true) {} 96 | } 97 | } 98 | 99 | 100 | object MemoryCtrie extends Benchmark { 101 | def run() { 102 | val ctrie = new ctries.ConcurrentTrie[Elem, Elem] 103 | val e = elems 104 | 105 | for (i <- 0 until sz) ctrie.insert(e(i), e(i)) 106 | 107 | Runtime.getRuntime.gc() 108 | while (true) {} 109 | } 110 | } 111 | 112 | 113 | object MemoryCtrie2 extends Benchmark { 114 | def run() { 115 | val ctrie = new ctries2.ConcurrentTrie[Elem, Elem] 116 | val e = elems 117 | 118 | for (i <- 0 until sz) ctrie.update(e(i), e(i)) 119 | 120 | Runtime.getRuntime.gc() 121 | while (true) {} 122 | } 123 | } 124 | 125 | 126 | -------------------------------------------------------------------------------- /src/bench/scala/ctries/MultiThreadInsert.scala: -------------------------------------------------------------------------------- 1 | package ctries 2 | 3 | 4 | 5 | import Global._ 6 | import scala.testing.Benchmark 7 | 8 | 9 | 10 | object MultiInsertCHM extends Benchmark { 11 | import java.util.concurrent.ConcurrentHashMap 12 | 13 | def run() { 14 | val chm = new ConcurrentHashMap[Elem, Elem] 15 | val p = par.get 16 | val step = sz / p 17 | 18 | val ins = for (i <- 0 until p) yield new Inserter(chm, i, step) 19 | 20 | for (i <- ins) i.start() 21 | for (i <- ins) i.join() 22 | } 23 | 24 | class Inserter(chm: ConcurrentHashMap[Elem, Elem], n: Int, step: Int) extends Thread { 25 | override def run() { 26 | var i = n * step 27 | val until = (n + 1) * step 28 | val e = elems 29 | 30 | while (i < until) { 31 | chm.put(e(i), e(i)) 32 | i += 1 33 | } 34 | } 35 | } 36 | } 37 | 38 | 39 | object MultiInsertSkipList extends Benchmark { 40 | import java.util.concurrent.ConcurrentSkipListMap 41 | 42 | def run() { 43 | val skiplist = new ConcurrentSkipListMap[Elem, Elem] 44 | val p = par.get 45 | val step = sz / p 46 | 47 | val ins = for (i <- 0 until p) yield new Inserter(skiplist, i, step) 48 | 49 | for (i <- ins) i.start() 50 | for (i <- ins) i.join() 51 | } 52 | 53 | class Inserter(skiplist: ConcurrentSkipListMap[Elem, Elem], n: Int, step: Int) extends Thread { 54 | override def run() { 55 | var i = n * step 56 | val until = (n + 1) * step 57 | val e = elems 58 | 59 | while (i < until) { 60 | skiplist.put(e(i), e(i)) 61 | i += 1 62 | } 63 | } 64 | } 65 | } 66 | 67 | 68 | object MultiInsertCtrie extends Benchmark { 69 | def run() { 70 | val ct = new ctries.ConcurrentTrie[Elem, Elem] 71 | val p = par.get 72 | val step = sz / p 73 | 74 | val ins = for (i <- 0 until p) yield new Inserter(ct, i, step) 75 | 76 | for (i <- ins) i.start() 77 | for (i <- ins) i.join() 78 | } 79 | 80 | class Inserter(ct: ctries.ConcurrentTrie[Elem, Elem], n: Int, step: Int) extends Thread { 81 | override def run() { 82 | var i = n * step 83 | val until = (n + 1) * step 84 | val e = elems 85 | 86 | while (i < until) { 87 | ct.insert(e(i), e(i)) 88 | i += 1 89 | } 90 | } 91 | } 92 | } 93 | 94 | 95 | object MultiInsertCtrie2 extends Benchmark { 96 | def run() { 97 | val ct = new ctries2.ConcurrentTrie[Elem, Elem] 98 | val p = par.get 99 | val step = sz / p 100 | 101 | val ins = for (i <- 0 until p) yield new Updater(ct, i, step) 102 | 103 | for (i <- ins) i.start() 104 | for (i <- ins) i.join() 105 | } 106 | 107 | class Updater(ct: ctries2.ConcurrentTrie[Elem, Elem], n: Int, step: Int) extends Thread { 108 | override def run() { 109 | var i = n * step 110 | val until = (n + 1) * step 111 | val e = elems 112 | 113 | while (i < until) { 114 | ct.update(e(i), e(i)) 115 | i += 1 116 | } 117 | } 118 | } 119 | } 120 | 121 | 122 | object MultiInsertCliff extends Benchmark { 123 | import org.cliffc.high_scale_lib._ 124 | 125 | def run() { 126 | val hm = new NonBlockingHashMap[Elem, Elem] 127 | val p = par.get 128 | val step = sz / p 129 | 130 | val ins = for (i <- 0 until p) yield new Updater(hm, i, step) 131 | 132 | for (i <- ins) i.start() 133 | for (i <- ins) i.join() 134 | } 135 | 136 | class Updater(hm: NonBlockingHashMap[Elem, Elem], n: Int, step: Int) extends Thread { 137 | override def run() { 138 | var i = n * step 139 | val until = (n + 1) * step 140 | val e = elems 141 | 142 | while (i < until) { 143 | hm.put(e(i), e(i)) 144 | i += 1 145 | } 146 | } 147 | } 148 | } 149 | 150 | 151 | -------------------------------------------------------------------------------- /src/bench/scala/ctries/MultiThreadLookup.scala: -------------------------------------------------------------------------------- 1 | package ctries 2 | 3 | 4 | 5 | import Global._ 6 | import scala.testing.Benchmark 7 | 8 | 9 | 10 | object MultiLookupCHM extends Benchmark { 11 | import java.util.concurrent.ConcurrentHashMap 12 | val chm = new ConcurrentHashMap[Elem, Elem] 13 | for (i <- 0 until sz) chm.put(elems(i), elems(i)) 14 | 15 | def run() { 16 | val p = par.get 17 | val step = sz / p 18 | 19 | val ins = for (i <- 0 until p) yield new Looker(chm, i, step) 20 | 21 | for (i <- ins) i.start() 22 | for (i <- ins) i.join() 23 | } 24 | 25 | class Looker(chm: ConcurrentHashMap[Elem, Elem], n: Int, step: Int) extends Thread { 26 | override def run() { 27 | var i = n * step 28 | val until = (n + 1) * step 29 | val e = elems 30 | 31 | while (i < until) { 32 | chm.get(e(i)) 33 | i += 1 34 | } 35 | } 36 | } 37 | } 38 | 39 | 40 | object MultiLookupSkipList extends Benchmark { 41 | import java.util.concurrent.ConcurrentSkipListMap 42 | val skiplist = new ConcurrentSkipListMap[Elem, Elem] 43 | for (i <- 0 until sz) skiplist.put(elems(i), elems(i)) 44 | 45 | def run() { 46 | val p = par.get 47 | val step = sz / p 48 | 49 | val ins = for (i <- 0 until p) yield new Looker(skiplist, i, step) 50 | 51 | for (i <- ins) i.start() 52 | for (i <- ins) i.join() 53 | } 54 | 55 | class Looker(skiplist: ConcurrentSkipListMap[Elem, Elem], n: Int, step: Int) extends Thread { 56 | override def run() { 57 | var i = n * step 58 | val until = (n + 1) * step 59 | val e = elems 60 | 61 | while (i < until) { 62 | skiplist.get(e(i)) 63 | i += 1 64 | } 65 | } 66 | } 67 | } 68 | 69 | 70 | object MultiLookupCtrie extends Benchmark { 71 | val ct = new ctries.ConcurrentTrie[Elem, Elem] 72 | for (i <- 0 until sz) ct.insert(elems(i), elems(i)) 73 | 74 | def run() { 75 | val p = par.get 76 | val step = sz / p 77 | 78 | val ins = for (i <- 0 until p) yield new Looker(ct, i, step) 79 | 80 | for (i <- ins) i.start() 81 | for (i <- ins) i.join() 82 | } 83 | 84 | class Looker(ct: ctries.ConcurrentTrie[Elem, Elem], n: Int, step: Int) extends Thread { 85 | override def run() { 86 | var i = n * step 87 | val until = (n + 1) * step 88 | val e = elems 89 | 90 | while (i < until) { 91 | ct.lookup(e(i)) 92 | i += 1 93 | } 94 | } 95 | } 96 | } 97 | 98 | 99 | object MultiLookupCtrie2 extends Benchmark { 100 | val ct = new ctries2.ConcurrentTrie[Elem, Elem] 101 | for (i <- 0 until sz) ct.update(elems(i), elems(i)) 102 | 103 | def run() { 104 | val p = par.get 105 | val step = sz / p 106 | 107 | val ins = for (i <- 0 until p) yield new Looker(ct, i, step) 108 | 109 | for (i <- ins) i.start() 110 | for (i <- ins) i.join() 111 | } 112 | 113 | class Looker(ct: ctries2.ConcurrentTrie[Elem, Elem], n: Int, step: Int) extends Thread { 114 | override def run() { 115 | var i = n * step 116 | val until = (n + 1) * step 117 | val e = elems 118 | 119 | while (i < until) { 120 | ct.lookup(e(i)) 121 | i += 1 122 | } 123 | } 124 | } 125 | } 126 | 127 | 128 | object MultiLookupCliff extends Benchmark { 129 | import org.cliffc.high_scale_lib._ 130 | 131 | val hm = new NonBlockingHashMap[Elem, Elem] 132 | for (i <- 0 until sz) hm.put(elems(i), elems(i)) 133 | 134 | def run() { 135 | val p = par.get 136 | val step = sz / p 137 | 138 | val ins = for (i <- 0 until p) yield new Looker(hm, i, step) 139 | 140 | for (i <- ins) i.start() 141 | for (i <- ins) i.join() 142 | } 143 | 144 | class Looker(hm: NonBlockingHashMap[Elem, Elem], n: Int, step: Int) extends Thread { 145 | override def run() { 146 | var i = n * step 147 | val until = (n + 1) * step 148 | val e = elems 149 | 150 | while (i < until) { 151 | hm.get(e(i)) 152 | i += 1 153 | } 154 | } 155 | } 156 | } 157 | 158 | -------------------------------------------------------------------------------- /src/bench/scala/ctries/MultiThreadReinsert.scala: -------------------------------------------------------------------------------- 1 | package ctries 2 | 3 | 4 | 5 | import Global._ 6 | import scala.testing.Benchmark 7 | 8 | 9 | 10 | object MultiReinsertCHM extends Benchmark { 11 | import java.util.concurrent.ConcurrentHashMap 12 | 13 | var chm = new ConcurrentHashMap[Elem, Elem] 14 | 15 | override def setUp { 16 | chm = new ConcurrentHashMap[Elem, Elem] 17 | for (i <- 0 until sz) chm.put(elems(i), elems(i)) 18 | Runtime.getRuntime.gc() 19 | } 20 | 21 | def run() { 22 | val p = par.get 23 | val step = sz / p 24 | 25 | val ins = for (i <- 0 until p) yield new Inserter(chm, i, step) 26 | 27 | for (i <- ins) i.start() 28 | for (i <- ins) i.join() 29 | } 30 | 31 | class Inserter(chm: ConcurrentHashMap[Elem, Elem], n: Int, step: Int) extends Thread { 32 | override def run() { 33 | var i = n * step 34 | val until = (n + 1) * step 35 | val e = elems 36 | 37 | while (i < until) { 38 | chm.put(e(i), e(i)) 39 | i += 1 40 | } 41 | } 42 | } 43 | } 44 | 45 | 46 | object MultiReinsertSkipList extends Benchmark { 47 | import java.util.concurrent.ConcurrentSkipListMap 48 | 49 | var skiplist = new ConcurrentSkipListMap[Elem, Elem] 50 | 51 | override def setUp { 52 | skiplist = new ConcurrentSkipListMap[Elem, Elem] 53 | for (i <- 0 until sz) skiplist.put(elems(i), elems(i)) 54 | Runtime.getRuntime.gc() 55 | } 56 | 57 | def run() { 58 | val p = par.get 59 | val step = sz / p 60 | 61 | val ins = for (i <- 0 until p) yield new Inserter(skiplist, i, step) 62 | 63 | for (i <- ins) i.start() 64 | for (i <- ins) i.join() 65 | } 66 | 67 | class Inserter(skiplist: ConcurrentSkipListMap[Elem, Elem], n: Int, step: Int) extends Thread { 68 | override def run() { 69 | var i = n * step 70 | val until = (n + 1) * step 71 | val e = elems 72 | 73 | while (i < until) { 74 | skiplist.put(e(i), e(i)) 75 | i += 1 76 | } 77 | } 78 | } 79 | } 80 | 81 | 82 | object MultiReinsertCtrie extends Benchmark { 83 | var ct = new ctries.ConcurrentTrie[Elem, Elem] 84 | 85 | override def setUp { 86 | ct = new ctries.ConcurrentTrie[Elem, Elem] 87 | for (i <- 0 until sz) ct.insert(elems(i), elems(i)) 88 | Runtime.getRuntime.gc() 89 | } 90 | 91 | def run() { 92 | val p = par.get 93 | val step = sz / p 94 | 95 | val ins = for (i <- 0 until p) yield new Inserter(ct, i, step) 96 | 97 | for (i <- ins) i.start() 98 | for (i <- ins) i.join() 99 | } 100 | 101 | class Inserter(ct: ctries.ConcurrentTrie[Elem, Elem], n: Int, step: Int) extends Thread { 102 | override def run() { 103 | var i = n * step 104 | val until = (n + 1) * step 105 | val e = elems 106 | 107 | while (i < until) { 108 | ct.insert(e(i), e(i)) 109 | i += 1 110 | } 111 | } 112 | } 113 | } 114 | 115 | 116 | object MultiReupdateCtrie2 extends Benchmark { 117 | var ct = new ctries2.ConcurrentTrie[Elem, Elem] 118 | 119 | override def setUp { 120 | ct = new ctries2.ConcurrentTrie[Elem, Elem] 121 | for (i <- 0 until sz) ct.update(elems(i), elems(i)) 122 | Runtime.getRuntime.gc() 123 | } 124 | 125 | def run() { 126 | val p = par.get 127 | val step = sz / p 128 | 129 | val ins = for (i <- 0 until p) yield new Updateer(ct, i, step) 130 | 131 | for (i <- ins) i.start() 132 | for (i <- ins) i.join() 133 | } 134 | 135 | class Updateer(ct: ctries2.ConcurrentTrie[Elem, Elem], n: Int, step: Int) extends Thread { 136 | override def run() { 137 | var i = n * step 138 | val until = (n + 1) * step 139 | val e = elems 140 | 141 | while (i < until) { 142 | ct.update(e(i), e(i)) 143 | i += 1 144 | } 145 | } 146 | } 147 | } 148 | 149 | 150 | -------------------------------------------------------------------------------- /src/bench/scala/ctries/MultiThreadRemove.scala: -------------------------------------------------------------------------------- 1 | package ctries 2 | 3 | 4 | 5 | import Global._ 6 | import scala.testing.Benchmark 7 | 8 | 9 | // uncomment setUp 10 | object MultiRemoveCHM extends Benchmark { 11 | import java.util.concurrent.ConcurrentHashMap 12 | var chm = new ConcurrentHashMap[Elem, Elem] 13 | 14 | override def setUp() { 15 | chm = new ConcurrentHashMap[Elem, Elem] 16 | for (i <- 0 until sz) chm.put(elems(i), elems(i)) 17 | } 18 | 19 | def run() { 20 | val p = par.get 21 | val step = sz / p 22 | 23 | val ins = for (i <- 0 until p) yield new Remover(chm, i, step) 24 | 25 | for (i <- ins) i.start() 26 | for (i <- ins) i.join() 27 | } 28 | 29 | var last: AnyRef = null 30 | override def tearDown { 31 | last = chm 32 | } 33 | 34 | class Remover(chm: ConcurrentHashMap[Elem, Elem], n: Int, step: Int) extends Thread { 35 | override def run() { 36 | var i = n * step 37 | val until = (n + 1) * step 38 | val e = elems 39 | 40 | while (i < until) { 41 | chm.remove(e(i)) 42 | i += 1 43 | } 44 | } 45 | } 46 | } 47 | 48 | 49 | object MultiRemoveSkipList extends Benchmark { 50 | import java.util.concurrent.ConcurrentSkipListMap 51 | var skiplist = new ConcurrentSkipListMap[Elem, Elem] 52 | 53 | override def setUp() { 54 | skiplist = new ConcurrentSkipListMap[Elem, Elem] 55 | for (i <- 0 until sz) skiplist.put(elems(i), elems(i)) 56 | } 57 | 58 | def run() { 59 | val p = par.get 60 | val step = sz / p 61 | 62 | val ins = for (i <- 0 until p) yield new Remover(skiplist, i, step) 63 | 64 | for (i <- ins) i.start() 65 | for (i <- ins) i.join() 66 | } 67 | 68 | class Remover(skiplist: ConcurrentSkipListMap[Elem, Elem], n: Int, step: Int) extends Thread { 69 | override def run() { 70 | var i = n * step 71 | val until = (n + 1) * step 72 | val e = elems 73 | 74 | while (i < until) { 75 | skiplist.remove(e(i)) 76 | i += 1 77 | } 78 | } 79 | } 80 | } 81 | 82 | 83 | object MultiRemoveCtrie extends Benchmark { 84 | var ct = new ctries.ConcurrentTrie[Elem, Elem] 85 | 86 | override def setUp() { 87 | ct = new ctries.ConcurrentTrie[Elem, Elem] 88 | for (i <- 0 until sz) ct.insert(elems(i), elems(i)) 89 | } 90 | 91 | def run() { 92 | val p = par.get 93 | val step = sz / p 94 | 95 | val ins = for (i <- 0 until p) yield new Remover(ct, i, step) 96 | 97 | for (i <- ins) i.start() 98 | for (i <- ins) i.join() 99 | } 100 | 101 | class Remover(ct: ctries.ConcurrentTrie[Elem, Elem], n: Int, step: Int) extends Thread { 102 | override def run() { 103 | var i = n * step 104 | val until = (n + 1) * step 105 | val e = elems 106 | 107 | while (i < until) { 108 | ct.remove(e(i)) 109 | i += 1 110 | } 111 | } 112 | } 113 | } 114 | 115 | 116 | object MultiRemoveCtrie2 extends Benchmark { 117 | var ct = new ctries2.ConcurrentTrie[Elem, Elem] 118 | 119 | override def setUp() { 120 | ct = new ctries2.ConcurrentTrie[Elem, Elem] 121 | for (i <- 0 until sz) ct.update(elems(i), elems(i)) 122 | } 123 | 124 | def run() { 125 | val p = par.get 126 | val step = sz / p 127 | 128 | val ins = for (i <- 0 until p) yield new Remover(ct, i, step) 129 | 130 | for (i <- ins) i.start() 131 | for (i <- ins) i.join() 132 | } 133 | 134 | class Remover(ct: ctries2.ConcurrentTrie[Elem, Elem], n: Int, step: Int) extends Thread { 135 | override def run() { 136 | var i = n * step 137 | val until = (n + 1) * step 138 | val e = elems 139 | 140 | while (i < until) { 141 | ct.remove(e(i)) 142 | i += 1 143 | } 144 | } 145 | } 146 | } 147 | 148 | 149 | object MultiRemoveCliff extends Benchmark { 150 | import org.cliffc.high_scale_lib._ 151 | 152 | var hm = new NonBlockingHashMap[Elem, Elem] 153 | 154 | override def setUp() { 155 | hm = new NonBlockingHashMap[Elem, Elem] 156 | for (i <- 0 until sz) hm.put(elems(i), elems(i)) 157 | } 158 | 159 | def run() { 160 | val p = par.get 161 | val step = sz / p 162 | 163 | val ins = for (i <- 0 until p) yield new Remover(hm, i, step) 164 | 165 | for (i <- ins) i.start() 166 | for (i <- ins) i.join() 167 | } 168 | 169 | class Remover(hm: NonBlockingHashMap[Elem, Elem], n: Int, step: Int) extends Thread { 170 | override def run() { 171 | var i = n * step 172 | val until = (n + 1) * step 173 | val e = elems 174 | 175 | while (i < until) { 176 | hm.remove(e(i)) 177 | i += 1 178 | } 179 | } 180 | } 181 | } 182 | 183 | 184 | -------------------------------------------------------------------------------- /src/bench/scala/ctries/MultiThreadUpdates.scala: -------------------------------------------------------------------------------- 1 | package ctries 2 | 3 | 4 | 5 | import Global._ 6 | import scala.testing.Benchmark 7 | 8 | 9 | 10 | object MultiUpdateCHM extends Benchmark { 11 | import java.util.concurrent.ConcurrentHashMap 12 | 13 | var chm = new ConcurrentHashMap[Elem, Elem] 14 | val array = Array.fill(lookups.get)(0) ++ Array.fill(inserts.get)(1) ++ Array.fill(removes.get)(2) 15 | 16 | override def setUp() { 17 | chm = new ConcurrentHashMap[Elem, Elem] 18 | if (updateFilled) for (i <- 0 until sz) chm.put(elems(i), elems(i)) 19 | } 20 | 21 | def run() { 22 | val p = par.get 23 | val howmany = totalops.get / p 24 | 25 | val ws = for (i <- 0 until p) yield new Worker(chm, i, howmany) 26 | 27 | for (i <- ws) i.start() 28 | for (i <- ws) i.join() 29 | } 30 | 31 | class Worker(chm: ConcurrentHashMap[Elem, Elem], n: Int, howmany: Int) extends Thread { 32 | override def run() { 33 | var i = 0 34 | val until = howmany 35 | val e = elems 36 | val arr = array 37 | val arrlen = array.length 38 | val s = sz 39 | 40 | while (i < until) { 41 | val imodsz = i % s 42 | array(i % arrlen) match { 43 | case 0 => chm.get(e(imodsz)) 44 | case 1 => chm.put(e(imodsz), e(imodsz)) 45 | case 2 => chm.remove(e(imodsz)) 46 | } 47 | 48 | i += 1 49 | } 50 | } 51 | } 52 | } 53 | 54 | 55 | object MultiUpdateSkipList extends Benchmark { 56 | import java.util.concurrent.ConcurrentSkipListMap 57 | 58 | var csl = new ConcurrentSkipListMap[Elem, Elem] 59 | val array = Array.fill(lookups.get)(0) ++ Array.fill(inserts.get)(1) ++ Array.fill(removes.get)(2) 60 | 61 | override def setUp() { 62 | csl = new ConcurrentSkipListMap[Elem, Elem] 63 | if (updateFilled) for (i <- 0 until sz) csl.put(elems(i), elems(i)) 64 | } 65 | 66 | def run() { 67 | val p = par.get 68 | val howmany = totalops.get / p 69 | 70 | val ws = for (i <- 0 until p) yield new Worker(csl, i, howmany) 71 | 72 | for (i <- ws) i.start() 73 | for (i <- ws) i.join() 74 | } 75 | 76 | class Worker(csl: ConcurrentSkipListMap[Elem, Elem], n: Int, howmany: Int) extends Thread { 77 | override def run() { 78 | var i = 0 79 | val until = howmany 80 | val e = elems 81 | val arr = array 82 | val arrlen = array.length 83 | val s = sz 84 | 85 | while (i < until) { 86 | val imodsz = i % s 87 | array(i % arrlen) match { 88 | case 0 => csl.get(e(imodsz)) 89 | case 1 => csl.put(e(imodsz), e(imodsz)) 90 | case 2 => csl.remove(e(imodsz)) 91 | } 92 | 93 | i += 1 94 | } 95 | } 96 | } 97 | } 98 | 99 | 100 | object MultiUpdateCtrie2 extends Benchmark { 101 | import ctries2.ConcurrentTrie 102 | 103 | var ct = new ConcurrentTrie[Elem, Elem] 104 | val array = Array.fill(lookups.get)(0) ++ Array.fill(inserts.get)(1) ++ Array.fill(removes.get)(2) 105 | 106 | override def setUp() { 107 | ct = new ConcurrentTrie[Elem, Elem] 108 | if (updateFilled) for (i <- 0 until sz) ct.put(elems(i), elems(i)) 109 | } 110 | 111 | def run() { 112 | val p = par.get 113 | val howmany = totalops.get / p 114 | 115 | val ws = for (i <- 0 until p) yield new Worker(ct, i, howmany) 116 | 117 | for (i <- ws) i.start() 118 | for (i <- ws) i.join() 119 | } 120 | 121 | class Worker(ct: ConcurrentTrie[Elem, Elem], n: Int, howmany: Int) extends Thread { 122 | override def run() { 123 | var i = 0 124 | val until = howmany 125 | val e = elems 126 | val arr = array 127 | val arrlen = array.length 128 | val s = sz 129 | 130 | while (i < until) { 131 | val imodsz = i % s 132 | array(i % arrlen) match { 133 | case 0 => ct.lookup(e(imodsz)) 134 | case 1 => ct.update(e(imodsz), e(imodsz)) 135 | case 2 => ct.remove(e(imodsz)) 136 | } 137 | 138 | i += 1 139 | } 140 | } 141 | } 142 | } 143 | 144 | 145 | object MultiUpdateCliff extends Benchmark { 146 | import org.cliffc.high_scale_lib._ 147 | 148 | var hm = new NonBlockingHashMap[Elem, Elem] 149 | val array = Array.fill(lookups.get)(0) ++ Array.fill(inserts.get)(1) ++ Array.fill(removes.get)(2) 150 | 151 | override def setUp() { 152 | hm = new NonBlockingHashMap[Elem, Elem] 153 | if (updateFilled) for (i <- 0 until sz) hm.put(elems(i), elems(i)) 154 | } 155 | 156 | def run() { 157 | val p = par.get 158 | val howmany = totalops.get / p 159 | 160 | val ws = for (i <- 0 until p) yield new Worker(hm, i, howmany) 161 | 162 | for (i <- ws) i.start() 163 | for (i <- ws) i.join() 164 | } 165 | 166 | class Worker(hm: NonBlockingHashMap[Elem, Elem], n: Int, howmany: Int) extends Thread { 167 | override def run() { 168 | var i = 0 169 | val until = howmany 170 | val e = elems 171 | val arr = array 172 | val arrlen = array.length 173 | val s = sz 174 | 175 | while (i < until) { 176 | val imodsz = i % s 177 | array(i % arrlen) match { 178 | case 0 => hm.get(e(imodsz)) 179 | case 1 => hm.put(e(imodsz), e(imodsz)) 180 | case 2 => hm.remove(e(imodsz)) 181 | } 182 | 183 | i += 1 184 | } 185 | } 186 | } 187 | } 188 | 189 | 190 | -------------------------------------------------------------------------------- /src/bench/scala/ctries/PageRank.scala: -------------------------------------------------------------------------------- 1 | package ctries 2 | 3 | 4 | 5 | import Global._ 6 | import scala.testing.Benchmark 7 | import scala.collection.parallel._ 8 | 9 | 10 | 11 | // a parallel ctrie created using the parallel collections framework 12 | class ParCtrie[K, V](szest: Int) extends mutable.ParMap[K, V] { 13 | val seq = new ctries2.ConcurrentTrie[K, V] 14 | 15 | def clear() = throw new UnsupportedOperationException 16 | 17 | final def +=(kv: (K, V)) = { 18 | seq += kv 19 | this 20 | } 21 | 22 | final def remove(k: K): Option[V] = seq.remove(k) 23 | 24 | final def -=(k: K) = { 25 | seq -= k 26 | this 27 | } 28 | 29 | def splitter: IterableSplitter[(K, V)] = 30 | new CtrieSplitter(szest, seq.readOnlySnapshot().asInstanceOf[ctries2.ConcurrentTrie[K, V]]) with SCPI 31 | 32 | def get(k: K): Option[V] = seq.get(k) 33 | 34 | def put(key: K, value: V): Option[V] = seq.put(key, value) 35 | 36 | def size = szest 37 | 38 | type SCPI = SignalContextPassingIterator[CtrieSplitter] 39 | 40 | class CtrieSplitter(szestimate: Int, ct: ctries2.ConcurrentTrie[K, V]) 41 | extends ctries2.CtrieIterator[K, V](ct) with ParIterator { 42 | self: SCPI => 43 | def remaining = szestimate // not using these ops 44 | def dup = throw new UnsupportedOperationException // not using views 45 | def split: Seq[CtrieSplitter] = subdivide.map { // probably won't use LNodes 46 | case ci: ctries2.CtrieIterator[K, V] => 47 | val cs = new CtrieSplitter(szestimate / 2, ct) with SCPI 48 | cs.stack = ci.stack 49 | cs.stackpos = ci.stackpos 50 | cs.depth = ci.depth 51 | cs.current = ci.current 52 | cs 53 | } 54 | } 55 | 56 | } 57 | 58 | 59 | // the total number of links on the page 60 | // indices of the pages linked by this page 61 | // and the indices of the pages linking to this page 62 | case class Page(name: Int, linksnum: Int, var outgoing: Array[Int], var incoming: Array[Int]) { 63 | override def toString = "Page(%d, %d, %s, %s)".format(name, linksnum, outgoing.toList, incoming.toList) 64 | } 65 | 66 | 67 | object Page { 68 | val rand = new util.Random(324132L) 69 | 70 | def calcIncoming(pages: Array[Page]) { 71 | val sz = pages.length 72 | val incomings = new Array[collection.Set[Int]](sz) 73 | 74 | for (i <- 0 until sz) incomings(i) = collection.mutable.HashSet[Int]() 75 | for (i <- 0 until sz; linkto <- pages(i).outgoing) incomings(linkto) += i 76 | for (i <- 0 until sz) pages(i).incoming = incomings(i).toArray.filter(_ != i) 77 | } 78 | 79 | // creates a set of pages with most links to initial pages 80 | def initialGauss(sz: Int) = { 81 | val pages = (for (i <- 0 until sz) yield { 82 | val linknum = rand.nextInt(maxlinks.get) 83 | val outgoing = for (j <- 0 until linknum) yield { 84 | val lnk = (math.abs(rand.nextGaussian() / 4) * sz) max 0 min (sz - 1) 85 | lnk.toInt 86 | } 87 | Page(i, linknum, outgoing.toArray, null) 88 | }).toArray 89 | 90 | calcIncoming(pages) 91 | 92 | pages 93 | } 94 | 95 | def groups(sz: Int) = { 96 | val pages = (for (i <- 0 until sz) yield { 97 | val linknum = rand.nextInt(maxlinks.get) 98 | val outgoing = for (j <- 0 until linknum) yield { 99 | val lnk = (rand.nextGaussian() * maxlinks.get + i) max 0 min (sz - 1) 100 | lnk.toInt 101 | } 102 | Page(i, linknum, outgoing.toArray, null) 103 | }) toArray 104 | 105 | calcIncoming(pages) 106 | 107 | pages 108 | } 109 | 110 | def nonUniform(sz: Int) = { 111 | val pages = (for (i <- 0 until sz) yield { 112 | val linknum = rand.nextInt(maxlinks.get) 113 | val group = (i / maxlinks.get) * maxlinks.get 114 | val linksin = for (j <- 0 until linknum) yield { 115 | val lnk = (rand.nextGaussian() * maxlinks.get + group) max 0 min (sz - 1) 116 | lnk.toInt 117 | } 118 | val linksprev = for (j <- 0 until linknum) yield { 119 | val lnk = (rand.nextGaussian() * maxlinks.get + group - maxlinks.get) max 0 min (sz - 1) 120 | lnk.toInt 121 | } 122 | val linkshalf = for (j <- 0 until linknum) yield { 123 | val lnk = (rand.nextGaussian() * maxlinks.get + group / 2) max 0 min (sz - 1) 124 | lnk.toInt 125 | } 126 | Page(i, linknum, (linksin ++ linksprev ++ linkshalf).toArray, null) 127 | }) toArray 128 | 129 | calcIncoming(pages) 130 | 131 | pages 132 | } 133 | 134 | def pages(sz: Int) = pagegenerator.get match { 135 | case "gauss-init" => initialGauss(sz) 136 | case "groups" => groups(sz) 137 | case "non-uniform" => nonUniform(sz) 138 | } 139 | 140 | } 141 | 142 | 143 | object SeqPageRank extends Benchmark { 144 | import ctries2.{ConcurrentTrie => Ctrie} 145 | 146 | val pages = Page.pages(sz) 147 | val prob1 = new Array[Double](sz) 148 | val prob2 = new Array[Double](sz) 149 | var ct: Ctrie[Int, Page] = null 150 | 151 | if (debug) println("Page set constructed.") 152 | 153 | override def setUp() { 154 | val in = 1.0 / sz 155 | for (i <- 0 until sz) prob1(i) = in 156 | for (i <- 0 until sz) prob2(i) = in 157 | ct = new Ctrie[Int, Page] 158 | for (i <- 0 until sz) ct.put(i, pages(i)) 159 | } 160 | 161 | def run() { 162 | val d = damping.get 163 | val epsilon = d / sz / 1000 164 | var iter = 0 165 | while (ct.nonEmpty) { 166 | val (last, next) = if (iter % 2 == 0) (prob1, prob2) else (prob2, prob1) 167 | for ((name, page) <- ct) { 168 | var sum = 0.0 169 | for (pind <- page.incoming) sum += last(pind) / pages(pind).linksnum 170 | next(name) = (1 - d) / sz + d * sum 171 | if (next(name) - last(name) < epsilon) ct.remove(name) 172 | } 173 | iter += 1 174 | 175 | if (debug) if (iter % 1 == 0) println("Iteration %d, size %d".format(iter, ct.size)) 176 | } 177 | 178 | if (debug) println("No. iterations: " + iter) 179 | } 180 | } 181 | 182 | 183 | object ParPageRank extends Benchmark { 184 | val pages = Page.pages(sz) 185 | val prob1 = new Array[Double](sz) 186 | val prob2 = new Array[Double](sz) 187 | var pct: ParCtrie[Int, Page] = null 188 | 189 | collection.parallel.ForkJoinTasks.defaultForkJoinPool.setParallelism(par.get) 190 | 191 | if (debug) println("Page set constructed.") 192 | 193 | override def setUp() { 194 | val in = 1.0 / sz 195 | for (i <- 0 until sz) prob1(i) = in 196 | for (i <- 0 until sz) prob2(i) = in 197 | pct = new ParCtrie[Int, Page](1 << 20) 198 | for (i <- 0 until sz) pct.put(i, pages(i)) 199 | } 200 | 201 | def run() { 202 | val d = damping.get 203 | val epsilon = d / sz / 1000 204 | var iter = 0 205 | while (pct.seq.nonEmpty) { 206 | val (last, next) = if (iter % 2 == 0) (prob1, prob2) else (prob2, prob1) 207 | for ((name, page) <- pct) { 208 | var sum = 0.0 209 | for (pind <- page.incoming) sum += last(pind) / pages(pind).linksnum 210 | next(name) = (1 - d) / sz + d * sum 211 | if (next(name) - last(name) < epsilon) pct.remove(name) 212 | } 213 | iter += 1 214 | 215 | if (debug) if (iter % 10 == 0) println("Iteration %d, size %d".format(iter, pct.seq.size)) 216 | } 217 | 218 | if (debug) println("No. iterations: " + iter) 219 | } 220 | } 221 | 222 | 223 | object FilterPageRank extends Benchmark { 224 | val pages = Page.pages(sz) 225 | val prob1 = new Array[Double](sz) 226 | val prob2 = new Array[Double](sz) 227 | val initialpages = pages.map(p => (p.name, p)).toMap 228 | 229 | collection.parallel.ForkJoinTasks.defaultForkJoinPool.setParallelism(par.get) 230 | 231 | if (debug) println("Page set constructed.") 232 | 233 | override def setUp() { 234 | val in = 1.0 / sz 235 | for (i <- 0 until sz) prob1(i) = in 236 | for (i <- 0 until sz) prob2(i) = in 237 | } 238 | 239 | def run() { 240 | val d = damping.get 241 | val epsilon = d / sz / 1000 242 | var remainingpages = initialpages.par 243 | var iter = 0 244 | while (remainingpages.nonEmpty) { 245 | val (last, next) = if (iter % 2 == 0) (prob1, prob2) else (prob2, prob1) 246 | for ((name, page) <- remainingpages) { 247 | var sum = 0.0 248 | for (pind <- page.incoming) sum += last(pind) / pages(pind).linksnum 249 | next(name) = (1 - d) / sz + d * sum 250 | } 251 | remainingpages = remainingpages.filter(kv => next(kv._1) - last(kv._1) >= epsilon) 252 | iter += 1 253 | 254 | if (debug) if (iter % 10 == 0) println("Iteration %d, size %d".format(iter, remainingpages.size)) 255 | } 256 | 257 | if (debug) println("No. iterations: " + iter) 258 | } 259 | 260 | } 261 | 262 | 263 | 264 | 265 | 266 | 267 | 268 | 269 | -------------------------------------------------------------------------------- /src/bench/scala/ctries/SingleThreadInsert.scala: -------------------------------------------------------------------------------- 1 | package ctries 2 | 3 | 4 | 5 | import Global._ 6 | import scala.testing.Benchmark 7 | 8 | 9 | 10 | object SingleInsertCHM extends Benchmark { 11 | import java.util.concurrent.ConcurrentHashMap 12 | 13 | def run() { 14 | val chm = new ConcurrentHashMap[Elem, Elem] 15 | 16 | var i = 0 17 | val until = sz 18 | val e = elems 19 | while (i < until) { 20 | chm.put(e(i), e(i)) 21 | i += 1 22 | } 23 | } 24 | 25 | } 26 | 27 | 28 | object SingleInsertCtrie extends Benchmark { 29 | def run() { 30 | val ct = new ConcurrentTrie[Elem, Elem] 31 | 32 | var i = 0 33 | val until = sz 34 | val e = elems 35 | while (i < until) { 36 | ct.insert(e(i), e(i)) 37 | i += 1 38 | } 39 | } 40 | } 41 | 42 | 43 | -------------------------------------------------------------------------------- /src/bench/scala/ctries/Snapshots.scala: -------------------------------------------------------------------------------- 1 | package ctries 2 | 3 | 4 | 5 | import Global._ 6 | import scala.testing.Benchmark 7 | 8 | 9 | 10 | object SingleRemovingCtrie2 extends Benchmark { 11 | import ctries2.ConcurrentTrie 12 | 13 | val ct = new ConcurrentTrie[Elem, Elem] 14 | 15 | override def setUp() { 16 | for (i <- 0 until sz) ct.put(elems(i), elems(i)) 17 | } 18 | 19 | def run() { 20 | for (i <- 0 until sz) ct.remove(elems(i)) 21 | } 22 | 23 | } 24 | 25 | 26 | object SingleRemovingCtrie2Snapshot extends Benchmark { 27 | import ctries2.ConcurrentTrie 28 | 29 | val ct = new ConcurrentTrie[Elem, Elem] 30 | for (i <- 0 until sz) ct.put(elems(i), elems(i)) 31 | 32 | def run() { 33 | val snap = ct.snapshot() 34 | for (i <- 0 until sz) snap.remove(elems(i)) 35 | } 36 | 37 | } 38 | 39 | 40 | object SingleInsertionCtrie2 extends Benchmark { 41 | import ctries2.ConcurrentTrie 42 | 43 | var ct = new ConcurrentTrie[Elem, Elem] 44 | 45 | override def setUp() { 46 | ct = new ConcurrentTrie[Elem, Elem] 47 | for (i <- 0 until sz) ct.update(elems(i), elems(i)) 48 | } 49 | 50 | def run() { 51 | for (i <- 0 until sz) ct.update(elems(i), elems(i)) 52 | } 53 | } 54 | 55 | 56 | object SingleInsertionCtrie2Snapshot extends Benchmark { 57 | import ctries2.ConcurrentTrie 58 | 59 | val ct = new ConcurrentTrie[Elem, Elem] 60 | for (i <- 0 until sz) ct.update(elems(i), elems(i)) 61 | 62 | def run() { 63 | val snap = ct.snapshot() 64 | for (i <- 0 until sz) snap.update(elems(i), elems(i)) 65 | } 66 | } 67 | 68 | 69 | object MultiRemovingCtrie2 extends Benchmark { 70 | import ctries2.ConcurrentTrie 71 | 72 | val ct = new ConcurrentTrie[Elem, Elem] 73 | 74 | override def setUp() { 75 | for (i <- 0 until sz) ct.update(elems(i), elems(i)) 76 | } 77 | 78 | def run() { 79 | val p = par.get 80 | val step = sz / p 81 | 82 | val ins = for (i <- 0 until p) yield new Remover(ct, i, step) 83 | 84 | for (i <- ins) i.start() 85 | for (i <- ins) i.join() 86 | } 87 | 88 | class Remover(ct: ConcurrentTrie[Elem, Elem], n: Int, step: Int) extends Thread { 89 | override def run() { 90 | var i = n * step 91 | val until = (n + 1) * step 92 | val e = elems 93 | 94 | while (i < until) { 95 | ct.remove(e(i)) 96 | i += 1 97 | } 98 | } 99 | } 100 | } 101 | 102 | 103 | object MultiRemovingCtrie2Snapshot extends Benchmark { 104 | import ctries2.ConcurrentTrie 105 | 106 | val ct = new ConcurrentTrie[Elem, Elem] 107 | 108 | override def setUp() { 109 | for (i <- 0 until sz) ct.update(elems(i), elems(i)) 110 | } 111 | 112 | def run() { 113 | val p = par.get 114 | val step = sz / p 115 | val snap = ct.snapshot() 116 | 117 | val ins = for (i <- 0 until p) yield new Remover(snap, i, step) 118 | 119 | for (i <- ins) i.start() 120 | for (i <- ins) i.join() 121 | } 122 | 123 | class Remover(snap: ConcurrentTrie[Elem, Elem], n: Int, step: Int) extends Thread { 124 | override def run() { 125 | var i = n * step 126 | val until = (n + 1) * step 127 | val e = elems 128 | 129 | while (i < until) { 130 | snap.remove(e(i)) 131 | i += 1 132 | } 133 | } 134 | } 135 | } 136 | 137 | 138 | object MultiLookingCtrie2 extends Benchmark { 139 | import ctries2.ConcurrentTrie 140 | 141 | val ct = new ConcurrentTrie[Elem, Elem] 142 | for (i <- 0 until sz) ct.update(elems(i), elems(i)) 143 | 144 | def run() { 145 | val p = par.get 146 | val step = sz / p 147 | 148 | val ins = for (i <- 0 until p) yield new Looker(ct, i, step) 149 | 150 | for (i <- ins) i.start() 151 | for (i <- ins) i.join() 152 | } 153 | 154 | class Looker(ct: ConcurrentTrie[Elem, Elem], n: Int, step: Int) extends Thread { 155 | override def run() { 156 | var i = n * step 157 | val until = (n + 1) * step 158 | val e = elems 159 | 160 | while (i < until) { 161 | ct.lookup(e(i)) 162 | i += 1 163 | } 164 | } 165 | } 166 | } 167 | 168 | 169 | object MultiLookingCtrie2Snapshot extends Benchmark { 170 | import ctries2.ConcurrentTrie 171 | 172 | val ct = new ConcurrentTrie[Elem, Elem] 173 | for (i <- 0 until sz) ct.update(elems(i), elems(i)) 174 | 175 | def run() { 176 | val p = par.get 177 | val step = sz / p 178 | val snap = ct.snapshot() 179 | 180 | val ins = for (i <- 0 until p) yield new Looker(snap, i, step) 181 | 182 | for (i <- ins) i.start() 183 | for (i <- ins) i.join() 184 | } 185 | 186 | class Looker(snap: ConcurrentTrie[Elem, Elem], n: Int, step: Int) extends Thread { 187 | override def run() { 188 | var i = n * step 189 | val until = (n + 1) * step 190 | val e = elems 191 | 192 | while (i < until) { 193 | snap.lookup(e(i)) 194 | i += 1 195 | } 196 | } 197 | } 198 | } 199 | 200 | 201 | 202 | 203 | 204 | -------------------------------------------------------------------------------- /src/main/java/Foo.java: -------------------------------------------------------------------------------- 1 | package ctries; 2 | 3 | 4 | 5 | public class Foo { 6 | public final String a = "init"; 7 | public Foo() { 8 | System.out.println("before: " + a); 9 | try { 10 | Foo.class.getField("a").set(this, "new value"); 11 | } catch (Exception e) { 12 | System.out.println(e); 13 | } 14 | System.out.println(a); 15 | } 16 | } -------------------------------------------------------------------------------- /src/main/java/ctries/CNodeBase.java: -------------------------------------------------------------------------------- 1 | package ctries; 2 | 3 | 4 | 5 | 6 | 7 | public abstract class CNodeBase { 8 | 9 | public int bitmap = 0; 10 | 11 | public INode[] array = null; 12 | 13 | } -------------------------------------------------------------------------------- /src/main/java/ctries/ConcurrentTrieBase.java: -------------------------------------------------------------------------------- 1 | package ctries; 2 | 3 | 4 | 5 | 6 | 7 | public abstract class ConcurrentTrieBase { 8 | 9 | public volatile INode root = null; 10 | 11 | protected Object RESTART = INodeBase.RESTART; 12 | 13 | } -------------------------------------------------------------------------------- /src/main/java/ctries/INodeBase.java: -------------------------------------------------------------------------------- 1 | package ctries; 2 | 3 | 4 | 5 | 6 | 7 | public abstract class INodeBase { 8 | 9 | public static final Object NOTFOUND = new Object(); 10 | 11 | public static final Object RESTART = new Object(); 12 | 13 | public volatile Object mainnode = null; 14 | 15 | } -------------------------------------------------------------------------------- /src/main/java/ctries2/BasicNode.java: -------------------------------------------------------------------------------- 1 | package ctries2; 2 | 3 | 4 | 5 | 6 | 7 | 8 | public abstract class BasicNode { 9 | 10 | public abstract String string(int lev); 11 | 12 | } -------------------------------------------------------------------------------- /src/main/java/ctries2/CNodeBase.java: -------------------------------------------------------------------------------- 1 | package ctries2; 2 | 3 | 4 | 5 | 6 | 7 | // TODO remove this - this is no longer the base class of CNode 8 | public abstract class CNodeBase extends MainNode { 9 | 10 | public int bitmap = 0; 11 | 12 | public BasicNode[] array = null; 13 | 14 | } -------------------------------------------------------------------------------- /src/main/java/ctries2/ConcurrentTrieBase.java: -------------------------------------------------------------------------------- 1 | package ctries2; 2 | 3 | 4 | 5 | 6 | 7 | // TODO no longer need this 8 | public abstract class ConcurrentTrieBase { 9 | 10 | public volatile Object root = null; 11 | 12 | //protected Object RESTART = INodeBase.RESTART; 13 | 14 | } -------------------------------------------------------------------------------- /src/main/java/ctries2/GenOld.java: -------------------------------------------------------------------------------- 1 | package ctries2; 2 | 3 | 4 | 5 | 6 | 7 | 8 | // TODO this can go to Scala files 9 | public final class GenOld { 10 | } 11 | -------------------------------------------------------------------------------- /src/main/java/ctries2/INodeBase.java: -------------------------------------------------------------------------------- 1 | package ctries2; 2 | 3 | 4 | 5 | import java.util.concurrent.atomic.AtomicReferenceFieldUpdater; 6 | 7 | 8 | 9 | public abstract class INodeBase extends BasicNode { 10 | 11 | public static final AtomicReferenceFieldUpdater updater = AtomicReferenceFieldUpdater.newUpdater(INodeBase.class, MainNode.class, "mainnode"); 12 | 13 | public static final Object RESTART = new Object(); 14 | 15 | public volatile MainNode mainnode = null; 16 | 17 | public final Gen gen; 18 | 19 | public INodeBase(Gen generation) { 20 | gen = generation; 21 | } 22 | 23 | public BasicNode prev() { 24 | return null; 25 | } 26 | 27 | } -------------------------------------------------------------------------------- /src/main/java/ctries2/MainNode.java: -------------------------------------------------------------------------------- 1 | package ctries2; 2 | 3 | 4 | 5 | import java.util.concurrent.atomic.AtomicReferenceFieldUpdater; 6 | 7 | 8 | 9 | public abstract class MainNode extends BasicNode { 10 | 11 | public static final AtomicReferenceFieldUpdater updater = AtomicReferenceFieldUpdater.newUpdater(MainNode.class, MainNode.class, "prev"); 12 | 13 | public volatile MainNode prev = null; 14 | 15 | public boolean CAS_PREV(MainNode oldval, MainNode nval) { 16 | return updater.compareAndSet(this, oldval, nval); 17 | } 18 | 19 | public void WRITE_PREV(MainNode nval) { 20 | updater.set(this, nval); 21 | } 22 | 23 | // do we need this? unclear in the javadocs... 24 | public MainNode READ_PREV() { 25 | return updater.get(this); 26 | } 27 | 28 | } -------------------------------------------------------------------------------- /src/main/scala/ctries/ConcurrentTrie.scala: -------------------------------------------------------------------------------- 1 | package ctries 2 | 3 | 4 | 5 | import java.util.concurrent.atomic._ 6 | import annotation.tailrec 7 | import annotation.switch 8 | 9 | 10 | 11 | final class INode[K, V](private val updater: AtomicReferenceFieldUpdater[INodeBase, AnyRef]) extends INodeBase { 12 | 13 | import INodeBase._ 14 | 15 | @inline final def CAS(old: AnyRef, n: AnyRef) = updater.compareAndSet(this, old, n) 16 | 17 | @tailrec final def insert(k: K, v: V, hc: Int, lev: Int, parent: INode[K, V]): Boolean = { 18 | val m = mainnode 19 | 20 | // this node is never initially empty - it will always contain at least 1 value when created 21 | // if the inode contains a null, that means its contents have been removed 22 | m match { 23 | case cn: CNode[K, V] => // 2) multiple values at this node 24 | // 2) calculate the position in the bitmap 25 | val idx = (hc >>> lev) & 0x1f 26 | val bmp = cn.bitmap 27 | val flag = 1 << idx 28 | val mask = flag - 1 29 | val pos = Integer.bitCount(bmp & mask) 30 | if ((bmp & flag) != 0) { 31 | // 2a) there is a binding at the position and it's not null - descend 32 | cn.array(pos).insert(k, v, hc, lev + 5, this) 33 | } else { // 2b) no binding at the position - create a new node 34 | val len = cn.array.length 35 | val arr = new Array[INode[K, V]](len + 1) 36 | val ncnode = new CNode[K, V](bmp | flag, arr) 37 | Array.copy(cn.array, 0, arr, 0, pos) 38 | arr(pos) = s_inode(new SNode(k, v, hc, false)) 39 | Array.copy(cn.array, pos, arr, pos + 1, len - pos) 40 | CAS(cn, ncnode) 41 | } 42 | case sn: SNode[K, V] if !sn.tomb => // 1) a singleton node - we rely on the fact that singletons below the root are never compressed 43 | val yk = sn.k 44 | val yhc = sn.hc 45 | // 1a) if they have the same keys, replace the old binding 46 | if (yhc == hc && yk == k) CAS(sn, new SNode(k, v, hc, false)) 47 | // 1b) if they have different keys, create a new INode and put it here 48 | else CAS(sn, cnode(new SNode(k, v, hc, false), hc, sn.copy, yhc, lev)) 49 | case sn: SNode[K, V] if sn.tomb => // 3) a tomb singleton - update this position through the parent 50 | // lemma: a tomb-inode no longer changes its value 51 | clean(parent) 52 | false 53 | case null if parent ne null => // 4) a deleted node - update this position through the parent 54 | // lemma: a null-inode no longer changes its value 55 | clean(parent) 56 | false 57 | case null => // 5) root null - retry from root 58 | false 59 | } 60 | } 61 | 62 | private def clean(parent: INode[K, V]) { 63 | val parentm = parent.mainnode 64 | parentm match { 65 | case cn: CNode[K, V] => 66 | parent.CAS(cn, cn.toCompressed) 67 | case _ => 68 | } 69 | } 70 | 71 | private def cnode(x: SNode[K, V], xhc: Int, y: SNode[K, V], yhc: Int, lev: Int): CNode[K, V] = if (lev < 35) { 72 | val xidx = (xhc >>> lev) & 0x1f 73 | val yidx = (yhc >>> lev) & 0x1f 74 | val bmp = (1 << xidx) | (1 << yidx) 75 | if (xidx == yidx) { 76 | val subinode = new INode[K, V](updater) 77 | subinode.mainnode = cnode(x, xhc, y, yhc, lev + 5) 78 | new CNode(bmp, Array(subinode)) 79 | } else { 80 | if (xidx < yidx) new CNode(bmp, Array(s_inode(x), s_inode(y))) 81 | else new CNode(bmp, Array(s_inode(y), s_inode(x))) 82 | } 83 | } else sys.error("list nodes not supported yet") 84 | 85 | private def s_inode(sn: SNode[K, V]) = { 86 | val in = new INode[K, V](updater) 87 | in.mainnode = sn 88 | in 89 | } 90 | 91 | @tailrec 92 | final def lookup(k: K, hc: Int, lev: Int, m: AnyRef, parent: INode[K, V]): AnyRef = { 93 | m match { 94 | case cn: CNode[K, V] => // 2) a multinode 95 | val idx = (hc >>> lev) & 0x1f 96 | val bmp = cn.bitmap 97 | val flag = 1 << idx 98 | if ((bmp & flag) == 0) null // 2a) bitmap shows no binding 99 | else { // 2b) bitmap contains a value - descend 100 | val pos = Integer.bitCount(bmp & (flag - 1)) 101 | val subinode = cn.array(pos) 102 | subinode.lookup(k, hc, lev + 5, subinode.mainnode, this) 103 | } 104 | case sn: SNode[K, V] if !sn.tomb => // 1) singleton node 105 | if (sn.hc == hc && sn.k == k) sn.v.asInstanceOf[AnyRef] 106 | else null 107 | case sn: SNode[K, V] => // 3) non-live node 108 | clean(parent) 109 | throw RestartException 110 | case null if parent ne null => 111 | clean(parent) 112 | throw RestartException 113 | case null => null 114 | } 115 | } 116 | 117 | final def remove(k: K, hc: Int, lev: Int, parent: INode[K, V]): Option[V] = { 118 | var m = mainnode 119 | 120 | m match { 121 | case sn: SNode[K, V] if !sn.tomb => // 1) singleton node 122 | if (sn.hc == hc && sn.k == k) { 123 | if (CAS(sn, null)) Some(sn.v) else null 124 | } else None 125 | case cn: CNode[K, V] => // 2) a multinode 126 | val idx = (hc >>> lev) & 0x1f 127 | val flag = 1 << idx 128 | val bmp = cn.bitmap 129 | if ((bmp & flag) == 0) None // 2a) binding not found 130 | else { // 2b) there is a binding - go lower 131 | val pos = Integer.bitCount(bmp & (flag - 1)) 132 | val res = cn.array(pos).remove(k, hc, lev + 5, this) 133 | 134 | // 2b) start compression 135 | res match { 136 | case null => null // restarting 137 | case None => None // no changes, since nothing was found 138 | case _ => 139 | // do a contraction until you succeed or this becomes a non-multiway 140 | @tailrec def contract(): Unit = { 141 | mainnode match { 142 | case old: CNode[K, V] => if (!CAS(old, old.toContracted)) contract() 143 | case _ => 144 | } 145 | } 146 | 147 | // if this is possibly a single tip, do a tomb markup 148 | @tailrec def tombmarkup(): Int = { 149 | mainnode match { 150 | case old: CNode[K, V] => 151 | val r = old.tryTombSingleton() 152 | if (r == 0) tombmarkup() else r 153 | case _ => -2 154 | } 155 | } 156 | 157 | // OLD VERSION 158 | // tombmarkup() 159 | // contract() 160 | // res 161 | 162 | // TODO optimize this 163 | // finds a singleton and tries to tomb it 164 | // -2 - if this isn't a cnode at all 165 | // -1 - if there are only null-inodes below 166 | // 1 - if tomb succeeded 167 | // 2 - a singleton, and already tombed 168 | // 3 - if there is no single singleton, and no null-inodes below 169 | // 4 - if there is no single singleton, but there are null-inodes below 170 | (tombmarkup(): @switch) match { 171 | case -2 => 172 | res // retry contract higher up 173 | case -1 => 174 | contract() 175 | res 176 | case 1 => 177 | contract() 178 | res 179 | case 2 => 180 | contract() 181 | res 182 | case 3 => 183 | res // could avoid retrying contract with new Result(res), but it doesn't get much better 184 | case 4 => 185 | contract() 186 | res 187 | case _ => sys.error("unreachable") 188 | } 189 | } 190 | } 191 | case sn: SNode[K, V] if sn.tomb => // 3) tomb node - compress parent and restart 192 | if (sn.hc == hc && sn.k == k) { // we've found a candidate 193 | val parentm = parent.mainnode 194 | parentm match { 195 | case cn: CNode[K, V] => 196 | // try to reach sn once more from the parent 197 | val idx = (hc >>> (lev - 5)) & 0x1f 198 | val flag = 1 << idx 199 | val bmp = cn.bitmap 200 | val pos = Integer.bitCount(bmp & (flag - 1)) 201 | if ((bmp & flag) != 0 && cn.array(pos) == this) { // tomb is still reachable - try to rewrite the parent without this node 202 | val arr = cn.array 203 | val nlen = arr.length - 1 204 | val nbmp = bmp ^ flag 205 | val narr = new Array[INode[K, V]](nlen) 206 | Array.copy(arr, 0, narr, 0, pos) 207 | Array.copy(arr, pos + 1, narr, pos, nlen - pos) 208 | val ncnode = new CNode(nbmp, narr) 209 | if (parent.CAS(cn, ncnode)) Some(sn.v) // rewritten the parent - success! 210 | else null // a modification on the parent - restart 211 | } else null // a modification on the parent - restart 212 | case _ => null // a modification on the parent - restart 213 | } 214 | } else None // we've found no such binding 215 | case null if parent ne null => 216 | clean(parent) 217 | null // restart 218 | case null => None // 5) root null node, no need to recompress, but restart 219 | } 220 | } 221 | 222 | // does this inode hold a tombed singleton 223 | // if true, guaranteed to stay that way 224 | @inline final def isTombed = { 225 | val m = mainnode 226 | m match { 227 | case sn: SNode[_, _] if sn.tomb => true 228 | case _ => false 229 | } 230 | } 231 | 232 | @inline final def isLive = { 233 | val m = mainnode 234 | m match { 235 | case sn: SNode[_, _] if sn.tomb => true 236 | case null => true 237 | case _ => false 238 | } 239 | } 240 | 241 | @inline final def nonLive = !isLive 242 | 243 | @inline final def isSingleton = { 244 | val m = mainnode 245 | m match { 246 | case sn: SNode[_, _] => true 247 | case _ => false 248 | } 249 | } 250 | 251 | private[ctries] def string(lev: Int) = "%sINode -> %s".format(" " * lev, mainnode match { 252 | case null => "" 253 | case sn: SNode[_, _] => "SNode(%s, %s, %d, %c)".format(sn.k, sn.v, sn.hc, if (sn.tomb) '!' else '_') 254 | case cn: CNode[_, _] => cn.string(lev) 255 | }) 256 | 257 | } 258 | 259 | 260 | final class CNode[K, V](bmp: Int, a: Array[INode[K, V]]) extends CNodeBase[K, V] { 261 | assert(Integer.bitCount(bmp) == a.length) // TODO comment this line 262 | bitmap = bmp 263 | array = a 264 | 265 | private[ctries] def string(lev: Int): String = "CNode %x\n%s".format(bitmap, array.map(_.string(lev + 1)).mkString("\n")) 266 | 267 | private def resurrect(inode: INode[K, V], inodemain: AnyRef) = inodemain match { 268 | case sn: SNode[_, _] if sn.tomb => 269 | val newinode = new INode[K, V](ConcurrentTrie.inodeupdater) 270 | newinode.mainnode = sn.copyUntombed 271 | newinode 272 | case _ => inode 273 | } 274 | 275 | @inline private def isSNode(n: AnyRef) = (n ne null) && n.isInstanceOf[SNode[_, _]] 276 | 277 | @inline private def asSNode(n: AnyRef) = n.asInstanceOf[SNode[K, V]] 278 | 279 | // finds a singleton and tries to tomb it 280 | // -1 - if there are only null-inodes below 281 | // 0 - if singleton was found and tomb failed 282 | // 1 - if tomb succeeded 283 | // 2 - a singleton, and already tombed 284 | // 3 - if there is no single singleton, and no null-inodes below 285 | // 4 - if there is no single singleton, but there are null-inodes below 286 | final def tryTombSingleton(): Int = { 287 | val arr = array 288 | val len = arr.length 289 | var totalnulls = 0 290 | var i = 0 291 | var x: INode[K, V] = null 292 | var m: AnyRef = null 293 | while (i < len) { 294 | val inode = arr(i) 295 | val main = inode.mainnode 296 | if (main ne null) { // found a non-null-inode 297 | if (x eq null) { // found first non-null-inode below 298 | x = inode 299 | m = main 300 | } else return 3 // found more than a single one 301 | } else totalnulls += 1 // found a null-inode 302 | i += 1 303 | } 304 | if (x ne null) m match { 305 | case sn: SNode[K, V] if !sn.tomb => if (x.CAS(sn, sn.copyTombed)) 1 else 0 306 | case sn: SNode[K, V] => 2 307 | case _ => if (totalnulls == 0) 3 else 4 308 | } else -1 309 | } 310 | 311 | // - will remove at least the null-inodes and tomb-inodes present at the beginning 312 | // and possibly some that appear during the compression 313 | // - if it detects a singleton 314 | final def toCompressed = { 315 | var bmp = bitmap 316 | val maxsubnodes = Integer.bitCount(bmp) // !!!this ensures lock-freedom!!! 317 | if (maxsubnodes == 1 && array(0).isTombed) asSNode(array(0).mainnode).copyUntombed 318 | else { 319 | var nbmp = 0 320 | var i = 0 321 | val arr = array 322 | var nsz = 0 323 | val tmparray = new Array[INode[K, V]](arr.length) 324 | while (bmp != 0) { // construct new bitmap 325 | val lsb = bmp & (-bmp) 326 | val inode = arr(i) 327 | val inodemain = inode.mainnode 328 | if (inodemain ne null) { 329 | nbmp |= lsb 330 | tmparray(nsz) = resurrect(inode, inodemain) 331 | nsz += 1 332 | } 333 | bmp ^= lsb 334 | i += 1 335 | } 336 | 337 | if (nsz > 0) { 338 | val narr = new Array[INode[K, V]](nsz) 339 | Array.copy(tmparray, 0, narr, 0, nsz) 340 | new CNode(nbmp, narr) 341 | } else null 342 | } 343 | } 344 | 345 | // - will remove at least the null-inodes present at the beginning 346 | // and possibly some that appear during the compression 347 | // - will not resurrect tomb singletons 348 | // - if this cnode is part of a tomb-tip, it will contract the node, shortening the branch 349 | final def toContracted = { 350 | val arr = array 351 | val len = array.length 352 | var i = 0 353 | var bmp = bitmap 354 | val tmparray = new Array[INode[K, V]](arr.length) 355 | var nsz = 0 356 | var nbmp = 0 357 | while (bmp != 0) { // construct new bitmap 358 | val lsb = bmp & (-bmp) 359 | val inode = arr(i) 360 | val inodemain = inode.mainnode 361 | if (inodemain ne null) { 362 | nbmp |= lsb 363 | tmparray(nsz) = inode 364 | nsz += 1 365 | } 366 | bmp ^= lsb 367 | i += 1 368 | } 369 | 370 | if (nsz == 1 && tmparray(0).isTombed) { 371 | asSNode(tmparray(0).mainnode).copyUntombed 372 | } else if (nsz > 0) { 373 | val narr = new Array[INode[K, V]](nsz) 374 | Array.copy(tmparray, 0, narr, 0, nsz) 375 | new CNode(nbmp, narr) 376 | } else null 377 | } 378 | } 379 | 380 | 381 | final class SNode[K, V](final val k: K, final val v: V, final val hc: Int, final val tomb: Boolean) { 382 | final def copy = new SNode(k, v, hc, tomb) 383 | final def copyTombed = new SNode(k, v, hc, true) 384 | final def copyUntombed = new SNode(k, v, hc, false) 385 | } 386 | 387 | 388 | // TODO final class LNode 389 | 390 | 391 | class ConcurrentTrie[K, V] extends ConcurrentTrieBase[K, V] { 392 | root = new INode[K, V](ConcurrentTrie.inodeupdater) 393 | private val rootupdater = AtomicReferenceFieldUpdater.newUpdater(classOf[ConcurrentTrieBase[_, _]], classOf[INode[_, _]], "root") 394 | 395 | @inline private def computeHash(k: K): Int = { 396 | k.hashCode 397 | } 398 | 399 | final def insert(k: K, v: V) { 400 | val hc = computeHash(k) 401 | inserthc(k, hc, v) 402 | } 403 | 404 | @tailrec private def inserthc(k: K, hc: Int, v: V) { 405 | val r = root 406 | 407 | // 0) check if the root node contains a null reference - if so, allocate a new root 408 | if (r.mainnode eq null) { 409 | val nroot = new INode[K, V](ConcurrentTrie.inodeupdater) 410 | nroot.mainnode = new SNode(k, v, hc, false) 411 | if (!rootupdater.compareAndSet(this, r, nroot)) inserthc(k, hc, v) 412 | } else if (!root.insert(k, v, hc, 0, null)) inserthc(k, hc, v) 413 | } 414 | 415 | final def lookup(k: K): V = { 416 | val hc = computeHash(k) 417 | lookuphc(k, hc).asInstanceOf[V] 418 | } 419 | 420 | //@tailrec 421 | private def lookuphc(k: K, hc: Int): AnyRef = { 422 | val r = root 423 | val rm = r.mainnode 424 | try { 425 | r.lookup(k, hc, 0, rm, null) 426 | } catch { 427 | case RestartException => lookuphc(k, hc) 428 | } 429 | } 430 | 431 | final def lookupOpt(k: K): Option[V] = { 432 | val hc = computeHash(k) 433 | Option(lookuphc(k, hc)).asInstanceOf[Option[V]] 434 | } 435 | 436 | final def remove(k: K): Option[V] = { 437 | val hc = computeHash(k) 438 | removehc(k, hc) 439 | } 440 | 441 | @tailrec private def removehc(k: K, hc: Int): Option[V] = { 442 | val r = root 443 | 444 | // 0) check if the root node contains a null reference 445 | if (r.mainnode eq null) None 446 | else { 447 | val res = r.remove(k, hc, 0, null) 448 | if (res eq null) removehc(k, hc) else res 449 | } 450 | } 451 | 452 | private[ctries] def string = root.string(0) 453 | 454 | } 455 | 456 | 457 | object ConcurrentTrie { 458 | val inodeupdater = AtomicReferenceFieldUpdater.newUpdater(classOf[INodeBase], classOf[AnyRef], "mainnode") 459 | } 460 | 461 | 462 | object RestartException extends util.control.ControlThrowable 463 | 464 | -------------------------------------------------------------------------------- /src/main/scala/ctries2/ConcurrentTrie.scala: -------------------------------------------------------------------------------- 1 | package ctries2 2 | 3 | 4 | 5 | import java.util.concurrent.atomic._ 6 | import collection.Map 7 | import collection.mutable.ConcurrentMap 8 | import collection.immutable.ListMap 9 | import annotation.tailrec 10 | import annotation.switch 11 | 12 | 13 | 14 | final class INode[K, V](bn: MainNode[K, V], g: Gen) extends INodeBase[K, V](g) { 15 | import INodeBase._ 16 | 17 | WRITE(bn) 18 | 19 | def this(g: Gen) = this(null, g) 20 | 21 | @inline final def WRITE(nval: MainNode[K, V]) = INodeBase.updater.set(this, nval) 22 | 23 | @inline final def CAS(old: MainNode[K, V], n: MainNode[K, V]) = INodeBase.updater.compareAndSet(this, old, n) 24 | 25 | @inline final def GCAS_READ(ct: ConcurrentTrie[K, V]): MainNode[K, V] = { 26 | val m = /*READ*/mainnode 27 | val prevval = /*READ*/m.prev 28 | if (prevval eq null) m 29 | else GCAS_Complete(m, ct) 30 | } 31 | 32 | @tailrec private def GCAS_Complete(m: MainNode[K, V], ct: ConcurrentTrie[K, V]): MainNode[K, V] = if (m eq null) null else { 33 | // complete the GCAS 34 | val prev = /*READ*/m.prev 35 | val ctr = ct.RDCSS_READ_ROOT(true) 36 | 37 | prev match { 38 | case null => 39 | m 40 | case fn: FailedNode[_, _] => // try to commit to previous value 41 | if (CAS(m, fn.prev)) fn.prev 42 | else GCAS_Complete(/*READ*/mainnode, ct) 43 | case vn: MainNode[_, _] => 44 | // Assume that you've read the root from the generation G. 45 | // Assume that the snapshot algorithm is correct. 46 | // ==> you can only reach nodes in generations <= G. 47 | // ==> `gen` is <= G. 48 | // We know that `ctr.gen` is >= G. 49 | // ==> if `ctr.gen` = `gen` then they are both equal to G. 50 | // ==> otherwise, we know that either `ctr.gen` > G, `gen` < G, 51 | // or both 52 | if ((ctr.gen eq gen) && ct.nonReadOnly) { 53 | // try to commit 54 | if (m.CAS_PREV(prev, null)) m 55 | else GCAS_Complete(m, ct) 56 | } else { 57 | // try to abort 58 | m.CAS_PREV(prev, new FailedNode(prev)) 59 | GCAS_Complete(/*READ*/mainnode, ct) 60 | } 61 | } 62 | } 63 | 64 | @inline final def GCAS(old: MainNode[K, V], n: MainNode[K, V], ct: ConcurrentTrie[K, V]): Boolean = { 65 | n.WRITE_PREV(old) 66 | if (CAS(old, n)) { 67 | GCAS_Complete(n, ct) 68 | /*READ*/n.prev eq null 69 | } else false 70 | } 71 | 72 | @inline private def inode(cn: MainNode[K, V]) = { 73 | val nin = new INode[K, V](gen) 74 | nin.WRITE(cn) 75 | nin 76 | } 77 | 78 | @inline final def copyToGen(ngen: Gen, ct: ConcurrentTrie[K, V]) = { 79 | val nin = new INode[K, V](ngen) 80 | val main = GCAS_READ(ct) 81 | nin.WRITE(main) 82 | nin 83 | } 84 | 85 | /** Inserts a key value pair, overwriting the old pair if the keys match. 86 | * 87 | * @return true if successful, false otherwise 88 | */ 89 | @tailrec final def rec_insert(k: K, v: V, hc: Int, lev: Int, parent: INode[K, V], startgen: Gen, ct: ConcurrentTrie[K, V]): Boolean = { 90 | val m = GCAS_READ(ct) // use -Yinline! 91 | 92 | m match { 93 | case cn: CNode[K, V] => // 1) a multiway node 94 | val idx = (hc >>> lev) & 0x1f 95 | val flag = 1 << idx 96 | val bmp = cn.bitmap 97 | val mask = flag - 1 98 | val pos = Integer.bitCount(bmp & mask) 99 | if ((bmp & flag) != 0) { 100 | // 1a) insert below 101 | cn.array(pos) match { 102 | case in: INode[K, V] => 103 | if (startgen eq in.gen) in.rec_insert(k, v, hc, lev + 5, this, startgen, ct) 104 | else { 105 | if (GCAS(cn, cn.renewed(startgen, ct), ct)) rec_insert(k, v, hc, lev, parent, startgen, ct) 106 | else false 107 | } 108 | case sn: SNode[K, V] => 109 | if (sn.hc == hc && sn.k == k) GCAS(cn, cn.updatedAt(pos, new SNode(k, v, hc), gen), ct) 110 | else { 111 | val rn = if (cn.gen eq gen) cn else cn.renewed(gen, ct) 112 | val nn = rn.updatedAt(pos, inode(CNode.dual(sn, sn.hc, new SNode(k, v, hc), hc, lev + 5, gen)), gen) 113 | GCAS(cn, nn, ct) 114 | } 115 | } 116 | } else { 117 | val rn = if (cn.gen eq gen) cn else cn.renewed(gen, ct) 118 | val ncnode = rn.insertedAt(pos, flag, new SNode(k, v, hc), gen) 119 | GCAS(cn, ncnode, ct) 120 | } 121 | case tn: TNode[K, V] => 122 | clean(parent, ct, lev - 5) 123 | false 124 | case ln: LNode[K, V] => // 3) an l-node 125 | val nn = ln.inserted(k, v) 126 | GCAS(ln, nn, ct) 127 | } 128 | } 129 | 130 | /** Inserts a new key value pair, given that a specific condition is met. 131 | * 132 | * @param cond null - don't care if the key was there; KEY_ABSENT - key wasn't there; KEY_PRESENT - key was there; other value `v` - key must be bound to `v` 133 | * @return null if unsuccessful, Option[V] otherwise (indicating previous value bound to the key) 134 | */ 135 | @tailrec final def rec_insertif(k: K, v: V, hc: Int, cond: AnyRef, lev: Int, parent: INode[K, V], startgen: Gen, ct: ConcurrentTrie[K, V]): Option[V] = { 136 | val m = GCAS_READ(ct) // use -Yinline! 137 | 138 | m match { 139 | case cn: CNode[K, V] => // 1) a multiway node 140 | val idx = (hc >>> lev) & 0x1f 141 | val flag = 1 << idx 142 | val bmp = cn.bitmap 143 | val mask = flag - 1 144 | val pos = Integer.bitCount(bmp & mask) 145 | if ((bmp & flag) != 0) { 146 | // 1a) insert below 147 | cn.array(pos) match { 148 | case in: INode[K, V] => 149 | if (startgen eq in.gen) in.rec_insertif(k, v, hc, cond, lev + 5, this, startgen, ct) 150 | else { 151 | if (GCAS(cn, cn.renewed(startgen, ct), ct)) rec_insertif(k, v, hc, cond, lev, parent, startgen, ct) 152 | else null 153 | } 154 | case sn: SNode[K, V] => cond match { 155 | case null => 156 | if (sn.hc == hc && sn.k == k) { 157 | if (GCAS(cn, cn.updatedAt(pos, new SNode(k, v, hc), gen), ct)) Some(sn.v) else null 158 | } else { 159 | val rn = if (cn.gen eq gen) cn else cn.renewed(gen, ct) 160 | val nn = rn.updatedAt(pos, inode(CNode.dual(sn, sn.hc, new SNode(k, v, hc), hc, lev + 5, gen)), gen) 161 | if (GCAS(cn, nn, ct)) None 162 | else null 163 | } 164 | case INode.KEY_ABSENT => 165 | if (sn.hc == hc && sn.k == k) Some(sn.v) 166 | else { 167 | val rn = if (cn.gen eq gen) cn else cn.renewed(gen, ct) 168 | val nn = rn.updatedAt(pos, inode(CNode.dual(sn, sn.hc, new SNode(k, v, hc), hc, lev + 5, gen)), gen) 169 | if (GCAS(cn, nn, ct)) None 170 | else null 171 | } 172 | case INode.KEY_PRESENT => 173 | if (sn.hc == hc && sn.k == k) { 174 | if (GCAS(cn, cn.updatedAt(pos, new SNode(k, v, hc), gen), ct)) Some(sn.v) else null 175 | } else None 176 | case otherv: V => 177 | if (sn.hc == hc && sn.k == k && sn.v == otherv) { 178 | if (GCAS(cn, cn.updatedAt(pos, new SNode(k, v, hc), gen), ct)) Some(sn.v) else null 179 | } else None 180 | } 181 | } 182 | } else cond match { 183 | case null | INode.KEY_ABSENT => 184 | val rn = if (cn.gen eq gen) cn else cn.renewed(gen, ct) 185 | val ncnode = rn.insertedAt(pos, flag, new SNode(k, v, hc), gen) 186 | if (GCAS(cn, ncnode, ct)) None else null 187 | case INode.KEY_PRESENT => None 188 | case otherv: V => None 189 | } 190 | case sn: TNode[K, V] => 191 | clean(parent, ct, lev - 5) 192 | null 193 | case ln: LNode[K, V] => // 3) an l-node 194 | @inline def insertln() = { 195 | val nn = ln.inserted(k, v) 196 | GCAS(ln, nn, ct) 197 | } 198 | cond match { 199 | case null => 200 | val optv = ln.get(k) 201 | if (insertln()) optv else null 202 | case INode.KEY_ABSENT => 203 | ln.get(k) match { 204 | case None => if (insertln()) None else null 205 | case optv => optv 206 | } 207 | case INode.KEY_PRESENT => 208 | ln.get(k) match { 209 | case Some(v0) => if (insertln()) Some(v0) else null 210 | case None => None 211 | } 212 | case otherv: V => 213 | ln.get(k) match { 214 | case Some(v0) if v0 == otherv => if (insertln()) Some(otherv) else null 215 | case _ => None 216 | } 217 | } 218 | } 219 | } 220 | 221 | /** Looks up the value associated with the key. 222 | * 223 | * @return null if no value has been found, RESTART if the operation wasn't successful, or any other value otherwise 224 | */ 225 | @tailrec final def rec_lookup(k: K, hc: Int, lev: Int, parent: INode[K, V], startgen: Gen, ct: ConcurrentTrie[K, V]): AnyRef = { 226 | val m = GCAS_READ(ct) // use -Yinline! 227 | 228 | m match { 229 | case cn: CNode[K, V] => // 1) a multinode 230 | val idx = (hc >>> lev) & 0x1f 231 | val flag = 1 << idx 232 | val bmp = cn.bitmap 233 | if ((bmp & flag) == 0) null // 1a) bitmap shows no binding 234 | else { // 1b) bitmap contains a value - descend 235 | val pos = if (bmp == 0xffffffff) idx else Integer.bitCount(bmp & (flag - 1)) 236 | val sub = cn.array(pos) 237 | sub match { 238 | case in: INode[K, V] => 239 | if (ct.isReadOnly || (startgen eq in.gen)) in.rec_lookup(k, hc, lev + 5, this, startgen, ct) 240 | else { 241 | if (GCAS(cn, cn.renewed(startgen, ct), ct)) rec_lookup(k, hc, lev, parent, startgen, ct) 242 | else return RESTART // used to be throw RestartException 243 | } 244 | case sn: SNode[K, V] => // 2) singleton node 245 | if (sn.hc == hc && sn.k == k) sn.v.asInstanceOf[AnyRef] 246 | else null 247 | } 248 | } 249 | case tn: TNode[K, V] => // 3) non-live node 250 | def cleanReadOnly(tn: TNode[K, V]) = if (ct.nonReadOnly) { 251 | clean(parent, ct, lev - 5) 252 | RESTART // used to be throw RestartException 253 | } else { 254 | if (tn.hc == hc && tn.k == k) tn.v.asInstanceOf[AnyRef] 255 | else null 256 | } 257 | cleanReadOnly(tn) 258 | case ln: LNode[K, V] => // 5) an l-node 259 | ln.get(k).asInstanceOf[Option[AnyRef]].orNull 260 | } 261 | } 262 | 263 | /** Removes the key associated with the given value. 264 | * 265 | * @param v if null, will remove the key irregardless of the value; otherwise removes only if binding contains that exact key and value 266 | * @return null if not successful, an Option[V] indicating the previous value otherwise 267 | */ 268 | final def rec_remove(k: K, v: V, hc: Int, lev: Int, parent: INode[K, V], startgen: Gen, ct: ConcurrentTrie[K, V]): Option[V] = { 269 | val m = GCAS_READ(ct) // use -Yinline! 270 | 271 | m match { 272 | case cn: CNode[K, V] => 273 | val idx = (hc >>> lev) & 0x1f 274 | val bmp = cn.bitmap 275 | val flag = 1 << idx 276 | if ((bmp & flag) == 0) None 277 | else { 278 | val pos = Integer.bitCount(bmp & (flag - 1)) 279 | val sub = cn.array(pos) 280 | val res = sub match { 281 | case in: INode[K, V] => 282 | if (startgen eq in.gen) in.rec_remove(k, v, hc, lev + 5, this, startgen, ct) 283 | else { 284 | if (GCAS(cn, cn.renewed(startgen, ct), ct)) rec_remove(k, v, hc, lev, parent, startgen, ct) 285 | else null 286 | } 287 | case sn: SNode[K, V] => 288 | if (sn.hc == hc && sn.k == k && (v == null || sn.v == v)) { 289 | val ncn = cn.removedAt(pos, flag, gen).toContracted(lev) 290 | if (GCAS(cn, ncn, ct)) Some(sn.v) else null 291 | } else None 292 | } 293 | 294 | if (res == None || (res eq null)) res 295 | else { 296 | @tailrec def cleanParent(nonlive: AnyRef) { 297 | val pm = parent.GCAS_READ(ct) 298 | pm match { 299 | case cn: CNode[K, V] => 300 | val idx = (hc >>> (lev - 5)) & 0x1f 301 | val bmp = cn.bitmap 302 | val flag = 1 << idx 303 | if ((bmp & flag) == 0) {} // somebody already removed this i-node, we're done 304 | else { 305 | val pos = Integer.bitCount(bmp & (flag - 1)) 306 | val sub = cn.array(pos) 307 | if (sub eq this) nonlive match { 308 | case tn: TNode[K, V] => 309 | val ncn = cn.updatedAt(pos, tn.copyUntombed, gen).toContracted(lev - 5) 310 | if (!parent.GCAS(cn, ncn, ct)) 311 | if (ct.RDCSS_READ_ROOT().gen == startgen) cleanParent(nonlive) 312 | } 313 | } 314 | case _ => // parent is no longer a cnode, we're done 315 | } 316 | } 317 | 318 | if (parent ne null) { // never tomb at root 319 | val n = GCAS_READ(ct) 320 | if (n.isInstanceOf[TNode[_, _]]) 321 | cleanParent(n) 322 | } 323 | 324 | res 325 | } 326 | } 327 | case tn: TNode[K, V] => 328 | clean(parent, ct, lev - 5) 329 | null 330 | case ln: LNode[K, V] => 331 | if (v == null) { 332 | val optv = ln.get(k) 333 | val nn = ln.removed(k) 334 | if (GCAS(ln, nn, ct)) optv else null 335 | } else ln.get(k) match { 336 | case optv @ Some(v0) if v0 == v => 337 | val nn = ln.removed(k) 338 | if (GCAS(ln, nn, ct)) optv else null 339 | case _ => None 340 | } 341 | } 342 | } 343 | 344 | private def clean(nd: INode[K, V], ct: ConcurrentTrie[K, V], lev: Int) { 345 | val m = nd.GCAS_READ(ct) 346 | m match { 347 | case cn: CNode[K, V] => nd.GCAS(cn, cn.toCompressed(ct, lev, gen), ct) 348 | case _ => 349 | } 350 | } 351 | 352 | final def isNullInode(ct: ConcurrentTrie[K, V]) = GCAS_READ(ct) eq null 353 | 354 | /* this is a quiescent method! */ 355 | def string(lev: Int) = "%sINode -> %s".format(" " * lev, mainnode match { 356 | case null => "" 357 | case tn: TNode[_, _] => "TNode(%s, %s, %d, !)".format(tn.k, tn.v, tn.hc) 358 | case cn: CNode[_, _] => cn.string(lev) 359 | case ln: LNode[_, _] => ln.string(lev) 360 | case x => "".format(x) 361 | }) 362 | 363 | } 364 | 365 | 366 | object INode { 367 | val KEY_PRESENT = new AnyRef 368 | val KEY_ABSENT = new AnyRef 369 | 370 | def newRootNode[K, V] = { 371 | val gen = new Gen 372 | val cn = new CNode[K, V](0, new Array(0), gen) 373 | new INode[K, V](cn, gen) 374 | } 375 | } 376 | 377 | 378 | final class FailedNode[K, V](p: MainNode[K, V]) extends MainNode[K, V] { 379 | WRITE_PREV(p) 380 | 381 | def string(lev: Int) = throw new UnsupportedOperationException 382 | 383 | override def toString = "FailedNode(%s)".format(p) 384 | } 385 | 386 | 387 | trait KVNode[K, V] { 388 | def kvPair: (K, V) 389 | } 390 | 391 | 392 | final class SNode[K, V](final val k: K, final val v: V, final val hc: Int) 393 | extends BasicNode with KVNode[K, V] { 394 | final def copy = new SNode(k, v, hc) 395 | final def copyTombed = new TNode(k, v, hc) 396 | final def copyUntombed = new SNode(k, v, hc) 397 | final def kvPair = (k, v) 398 | final def string(lev: Int) = (" " * lev) + "SNode(%s, %s, %x)".format(k, v, hc) 399 | } 400 | 401 | 402 | final class TNode[K, V](final val k: K, final val v: V, final val hc: Int) 403 | extends MainNode[K, V] with KVNode[K, V] { 404 | final def copy = new TNode(k, v, hc) 405 | final def copyTombed = new TNode(k, v, hc) 406 | final def copyUntombed = new SNode(k, v, hc) 407 | final def kvPair = (k, v) 408 | final def string(lev: Int) = (" " * lev) + "TNode(%s, %s, %x, !)".format(k, v, hc) 409 | } 410 | 411 | 412 | final class LNode[K, V](final val listmap: ListMap[K, V]) 413 | extends MainNode[K, V] { 414 | def this(k: K, v: V) = this(ListMap(k -> v)) 415 | def this(k1: K, v1: V, k2: K, v2: V) = this(ListMap(k1 -> v1, k2 -> v2)) 416 | def inserted(k: K, v: V) = new LNode(listmap + ((k, v))) 417 | def removed(k: K) = { 418 | val updmap = listmap - k 419 | if (updmap.size > 1) new LNode(updmap) 420 | else { 421 | val (k, v) = updmap.iterator.next 422 | new TNode(k, v, ConcurrentTrie.computeHash(k)) // create it tombed so that it gets compressed on subsequent accesses 423 | } 424 | } 425 | def get(k: K) = listmap.get(k) 426 | def string(lev: Int) = (" " * lev) + "LNode(%s)".format(listmap.mkString(", ")) 427 | } 428 | 429 | 430 | final class CNode[K, V](final val bitmap: Int, final val array: Array[BasicNode], final val gen: Gen) 431 | extends MainNode[K, V] { 432 | 433 | final def updatedAt(pos: Int, nn: BasicNode, gen: Gen) = { 434 | val len = array.length 435 | val narr = new Array[BasicNode](len) 436 | Array.copy(array, 0, narr, 0, len) 437 | narr(pos) = nn 438 | new CNode[K, V](bitmap, narr, gen) 439 | } 440 | 441 | final def removedAt(pos: Int, flag: Int, gen: Gen) = { 442 | val arr = array 443 | val len = arr.length 444 | val narr = new Array[BasicNode](len - 1) 445 | Array.copy(arr, 0, narr, 0, pos) 446 | Array.copy(arr, pos + 1, narr, pos, len - pos - 1) 447 | new CNode[K, V](bitmap ^ flag, narr, gen) 448 | } 449 | 450 | final def insertedAt(pos: Int, flag: Int, nn: BasicNode, gen: Gen) = { 451 | val len = array.length 452 | val bmp = bitmap 453 | val narr = new Array[BasicNode](len + 1) 454 | Array.copy(array, 0, narr, 0, pos) 455 | narr(pos) = nn 456 | Array.copy(array, pos, narr, pos + 1, len - pos) 457 | new CNode[K, V](bmp | flag, narr, gen) 458 | } 459 | 460 | /** Returns a copy of this cnode such that all the i-nodes below it are copied 461 | * to the specified generation `ngen`. 462 | */ 463 | final def renewed(ngen: Gen, ct: ConcurrentTrie[K, V]) = { 464 | var i = 0 465 | val arr = array 466 | val len = arr.length 467 | val narr = new Array[BasicNode](len) 468 | while (i < len) { 469 | arr(i) match { 470 | case in: INode[K, V] => narr(i) = in.copyToGen(ngen, ct) 471 | case bn: BasicNode => narr(i) = bn 472 | } 473 | i += 1 474 | } 475 | new CNode[K, V](bitmap, narr, ngen) 476 | } 477 | 478 | private def resurrect(inode: INode[K, V], inodemain: AnyRef) = inodemain match { 479 | case tn: TNode[_, _] => tn.copyUntombed 480 | case _ => inode 481 | } 482 | 483 | final def toContracted(lev: Int) = if (array.length == 1 && lev > 0) array(0) match { 484 | case sn: SNode[K, V] => sn.copyTombed 485 | case _ => this 486 | } else this 487 | 488 | // - if the branching factor is 1 for this CNode, and the child 489 | // is a tombed SNode, returns its tombed version 490 | // - otherwise, if there is at least one non-null node below, 491 | // returns the version of this node with at least some null-inodes 492 | // removed (those existing when the op began) 493 | // - if there are only null-i-nodes below, returns null 494 | final def toCompressed(ct: ConcurrentTrie[K, V], lev: Int, gen: Gen) = { 495 | var bmp = bitmap 496 | var i = 0 497 | val arr = array 498 | val tmparray = new Array[BasicNode](arr.length) 499 | while (i < arr.length) { // construct new bitmap 500 | val sub = arr(i) 501 | sub match { 502 | case in: INode[K, V] => 503 | val inodemain = in.GCAS_READ(ct) 504 | assert(inodemain ne null) 505 | tmparray(i) = resurrect(in, inodemain) 506 | case sn: SNode[K, V] => 507 | tmparray(i) = sn 508 | } 509 | i += 1 510 | } 511 | 512 | new CNode[K, V](bmp, tmparray, gen).toContracted(lev) 513 | } 514 | 515 | private[ctries2] def string(lev: Int): String = "CNode %x\n%s".format(bitmap, array.map(_.string(lev + 1)).mkString("\n")) 516 | 517 | /* quiescently consistent - don't call concurrently to anything involving a GCAS!! */ 518 | protected def collectElems: Seq[(K, V)] = array flatMap { 519 | case sn: SNode[K, V] => Some(sn.kvPair) 520 | case in: INode[K, V] => in.mainnode match { 521 | case tn: TNode[K, V] => Some(tn.kvPair) 522 | case ln: LNode[K, V] => ln.listmap.toList 523 | case cn: CNode[K, V] => cn.collectElems 524 | } 525 | } 526 | 527 | protected def collectLocalElems: Seq[String] = array flatMap { 528 | case sn: SNode[K, V] => Some(sn.kvPair._2.toString) 529 | case in: INode[K, V] => Some(in.toString.drop(14) + "(" + in.gen + ")") 530 | } 531 | 532 | override def toString = { 533 | val elems = collectLocalElems 534 | "CNode(sz: %d; %s)".format(elems.size, elems.sorted.mkString(", ")) 535 | } 536 | } 537 | 538 | 539 | object CNode { 540 | 541 | def dual[K, V](x: SNode[K, V], xhc: Int, y: SNode[K, V], yhc: Int, lev: Int, gen: Gen): MainNode[K, V] = if (lev < 35) { 542 | val xidx = (xhc >>> lev) & 0x1f 543 | val yidx = (yhc >>> lev) & 0x1f 544 | val bmp = (1 << xidx) | (1 << yidx) 545 | if (xidx == yidx) { 546 | val subinode = new INode[K, V](gen)//(ConcurrentTrie.inodeupdater) 547 | subinode.mainnode = dual(x, xhc, y, yhc, lev + 5, gen) 548 | new CNode(bmp, Array(subinode), gen) 549 | } else { 550 | if (xidx < yidx) new CNode(bmp, Array(x, y), gen) 551 | else new CNode(bmp, Array(y, x), gen) 552 | } 553 | } else { 554 | new LNode(x.k, x.v, y.k, y.v) 555 | } 556 | 557 | } 558 | 559 | 560 | case class RDCSS_Descriptor[K, V](old: INode[K, V], expectedmain: MainNode[K, V], nv: INode[K, V]) { 561 | @volatile var committed = false 562 | } 563 | 564 | 565 | class ConcurrentTrie[K, V] private (r: AnyRef, rtupd: AtomicReferenceFieldUpdater[ConcurrentTrie[K, V], AnyRef]) 566 | extends ConcurrentMap[K, V] 567 | { 568 | import ConcurrentTrie.computeHash 569 | 570 | private val rootupdater = rtupd 571 | @volatile var root = r 572 | 573 | def this() = this( 574 | INode.newRootNode, 575 | AtomicReferenceFieldUpdater.newUpdater(classOf[ConcurrentTrie[K, V]], classOf[AnyRef], "root") 576 | ) 577 | 578 | /* internal methods */ 579 | 580 | @inline final def CAS_ROOT(ov: AnyRef, nv: AnyRef) = rootupdater.compareAndSet(this, ov, nv) 581 | 582 | @inline final def RDCSS_READ_ROOT(abort: Boolean = false): INode[K, V] = { 583 | val r = /*READ*/root 584 | r match { 585 | case in: INode[K, V] => in 586 | case desc: RDCSS_Descriptor[K, V] => RDCSS_Complete(abort) 587 | } 588 | } 589 | 590 | @tailrec private def RDCSS_Complete(abort: Boolean): INode[K, V] = { 591 | val v = /*READ*/root 592 | v match { 593 | case in: INode[K, V] => in 594 | case desc: RDCSS_Descriptor[K, V] => 595 | val RDCSS_Descriptor(ov, exp, nv) = desc 596 | if (abort) { 597 | if (CAS_ROOT(desc, ov)) ov 598 | else RDCSS_Complete(abort) 599 | } else { 600 | val oldmain = ov.GCAS_READ(this) 601 | if (oldmain eq exp) { 602 | if (CAS_ROOT(desc, nv)) { 603 | desc.committed = true 604 | nv 605 | } else RDCSS_Complete(abort) 606 | } else { 607 | if (CAS_ROOT(desc, ov)) ov 608 | else RDCSS_Complete(abort) 609 | } 610 | } 611 | } 612 | } 613 | 614 | private def RDCSS_ROOT(ov: INode[K, V], expectedmain: MainNode[K, V], nv: INode[K, V]): Boolean = { 615 | val desc = RDCSS_Descriptor(ov, expectedmain, nv) 616 | if (CAS_ROOT(ov, desc)) { 617 | RDCSS_Complete(false) 618 | /*READ*/desc.committed 619 | } else false 620 | } 621 | 622 | @tailrec private def inserthc(k: K, hc: Int, v: V) { 623 | val r = RDCSS_READ_ROOT() 624 | if (!r.rec_insert(k, v, hc, 0, null, r.gen, this)) inserthc(k, hc, v) 625 | } 626 | 627 | @tailrec private def insertifhc(k: K, hc: Int, v: V, cond: AnyRef): Option[V] = { 628 | val r = RDCSS_READ_ROOT() 629 | 630 | val ret = r.rec_insertif(k, v, hc, cond, 0, null, r.gen, this) 631 | if (ret eq null) insertifhc(k, hc, v, cond) 632 | else ret 633 | } 634 | 635 | @tailrec private def lookuphc(k: K, hc: Int): AnyRef = { 636 | val r = RDCSS_READ_ROOT() 637 | val res = r.rec_lookup(k, hc, 0, null, r.gen, this) 638 | if (res eq INodeBase.RESTART) lookuphc(k, hc) 639 | else res 640 | } 641 | 642 | /* 643 | //@tailrec 644 | private def lookuphc(k: K, hc: Int): AnyRef = { 645 | val r = RDCSS_READ_ROOT() 646 | try { 647 | r.rec_lookup(k, hc, 0, null, r.gen, this) 648 | } catch { 649 | case RestartException => 650 | lookuphc(k, hc) 651 | } 652 | } 653 | */ 654 | 655 | @tailrec private def removehc(k: K, v: V, hc: Int): Option[V] = { 656 | val r = RDCSS_READ_ROOT() 657 | val res = r.rec_remove(k, v, hc, 0, null, r.gen, this) 658 | if (res ne null) res 659 | else removehc(k, v, hc) 660 | } 661 | 662 | def string = RDCSS_READ_ROOT().string(0) 663 | 664 | /* public methods */ 665 | 666 | @inline final def isReadOnly = rootupdater eq null 667 | 668 | @inline final def nonReadOnly = rootupdater ne null 669 | 670 | @tailrec final def snapshot(): ConcurrentTrie[K, V] = { 671 | val r = RDCSS_READ_ROOT() 672 | val expmain = r.GCAS_READ(this) 673 | if (RDCSS_ROOT(r, expmain, r.copyToGen(new Gen, this))) new ConcurrentTrie(r.copyToGen(new Gen, this), rootupdater) 674 | else snapshot() 675 | } 676 | 677 | @tailrec final def readOnlySnapshot(): Map[K, V] = { 678 | val r = RDCSS_READ_ROOT() 679 | val expmain = r.GCAS_READ(this) 680 | if (RDCSS_ROOT(r, expmain, r.copyToGen(new Gen, this))) new ConcurrentTrie(r, null) 681 | else readOnlySnapshot() 682 | } 683 | 684 | @tailrec final override def clear() { 685 | val r = RDCSS_READ_ROOT() 686 | if (!RDCSS_ROOT(r, r.GCAS_READ(this), INode.newRootNode[K, V])) clear() 687 | } 688 | 689 | final def lookup(k: K): V = { 690 | val hc = computeHash(k) 691 | lookuphc(k, hc).asInstanceOf[V] 692 | } 693 | 694 | final override def apply(k: K): V = { 695 | val hc = computeHash(k) 696 | val res = lookuphc(k, hc) 697 | if (res eq null) throw new NoSuchElementException 698 | else res.asInstanceOf[V] 699 | } 700 | 701 | final def get(k: K): Option[V] = { 702 | val hc = computeHash(k) 703 | Option(lookuphc(k, hc)).asInstanceOf[Option[V]] 704 | } 705 | 706 | override def put(key: K, value: V): Option[V] = { 707 | val hc = computeHash(key) 708 | insertifhc(key, hc, value, null) 709 | } 710 | 711 | final override def update(k: K, v: V) { 712 | val hc = computeHash(k) 713 | inserthc(k, hc, v) 714 | } 715 | 716 | final def +=(kv: (K, V)) = { 717 | update(kv._1, kv._2) 718 | this 719 | } 720 | 721 | final override def remove(k: K): Option[V] = { 722 | val hc = computeHash(k) 723 | removehc(k, null.asInstanceOf[V], hc) 724 | } 725 | 726 | final def -=(k: K) = { 727 | remove(k) 728 | this 729 | } 730 | 731 | def putIfAbsent(k: K, v: V): Option[V] = { 732 | val hc = computeHash(k) 733 | insertifhc(k, hc, v, INode.KEY_ABSENT) 734 | } 735 | 736 | def remove(k: K, v: V): Boolean = { 737 | val hc = computeHash(k) 738 | removehc(k, v, hc).nonEmpty 739 | } 740 | 741 | def replace(k: K, oldvalue: V, newvalue: V): Boolean = { 742 | val hc = computeHash(k) 743 | insertifhc(k, hc, newvalue, oldvalue.asInstanceOf[AnyRef]).nonEmpty 744 | } 745 | 746 | def replace(k: K, v: V): Option[V] = { 747 | val hc = computeHash(k) 748 | insertifhc(k, hc, v, INode.KEY_PRESENT) 749 | } 750 | 751 | def iterator: Iterator[(K, V)] = 752 | if (nonReadOnly) readOnlySnapshot().iterator 753 | else new CtrieIterator(this) 754 | 755 | } 756 | 757 | 758 | object ConcurrentTrie { 759 | val inodeupdater = AtomicReferenceFieldUpdater.newUpdater(classOf[INodeBase[_, _]], classOf[MainNode[_, _]], "mainnode") 760 | 761 | @inline final def computeHash[K](k: K): Int = { 762 | k.hashCode 763 | } 764 | } 765 | 766 | 767 | class CtrieIterator[K, V](ct: ConcurrentTrie[K, V], mustInit: Boolean = true) extends Iterator[(K, V)] { 768 | var stack = new Array[Array[BasicNode]](7) 769 | var stackpos = new Array[Int](7) 770 | var depth = -1 771 | var subiter: Iterator[(K, V)] = null 772 | var current: KVNode[K, V] = null 773 | 774 | if (mustInit) initialize() 775 | 776 | def hasNext = (current ne null) || (subiter ne null) 777 | 778 | def next() = if (hasNext) { 779 | var r: (K, V) = null 780 | if (subiter ne null) { 781 | r = subiter.next() 782 | checkSubiter() 783 | } else { 784 | r = current.kvPair 785 | advance() 786 | } 787 | r 788 | } else Iterator.empty.next() 789 | 790 | private def readin(in: INode[K, V]) = in.GCAS_READ(ct) match { 791 | case cn: CNode[K, V] => 792 | depth += 1 793 | stack(depth) = cn.array 794 | stackpos(depth) = -1 795 | advance() 796 | case tn: TNode[K, V] => 797 | current = tn 798 | case ln: LNode[K, V] => 799 | subiter = ln.listmap.iterator 800 | checkSubiter() 801 | case null => 802 | current = null 803 | } 804 | 805 | @inline private def checkSubiter() = if (!subiter.hasNext) { 806 | subiter = null 807 | advance() 808 | } 809 | 810 | @inline private def initialize() { 811 | assert(ct.isReadOnly) 812 | 813 | val r = ct.RDCSS_READ_ROOT() 814 | readin(r) 815 | } 816 | 817 | def advance(): Unit = if (depth >= 0) { 818 | val npos = stackpos(depth) + 1 819 | if (npos < stack(depth).length) { 820 | stackpos(depth) = npos 821 | stack(depth)(npos) match { 822 | case sn: SNode[K, V] => 823 | current = sn 824 | case in: INode[K, V] => 825 | readin(in) 826 | } 827 | } else { 828 | depth -= 1 829 | advance() 830 | } 831 | } else current = null 832 | 833 | /** Returns a sequence of iterators over subsets of this iterator. 834 | * It's used to ease the implementation of splitters for a parallel version of the Ctrie. 835 | */ 836 | protected def subdivide: Seq[Iterator[(K, V)]] = if (subiter ne null) { 837 | // the case where an LNode is being iterated 838 | val it = subiter 839 | subiter = null 840 | advance() 841 | Seq(it, this) 842 | } else if (depth == -1) Seq(this) else { 843 | var d = 0 844 | while (d <= depth) { 845 | val rem = stack(d).length - 1 - stackpos(d) 846 | if (rem > 0) { 847 | val (arr1, arr2) = stack(d).drop(stackpos(d) + 1).splitAt(rem / 2) 848 | stack(d) = arr1 849 | stackpos(d) = -1 850 | val it = new CtrieIterator[K, V](ct, false) 851 | it.stack(0) = arr2 852 | it.stackpos(0) = -1 853 | it.depth = 0 854 | it.advance() // <-- fix it 855 | return Seq(this, it) 856 | } 857 | d += 1 858 | } 859 | Seq(this) 860 | } 861 | 862 | private def print { 863 | println("ctrie iterator") 864 | println(stackpos.mkString(",")) 865 | println("depth: " + depth) 866 | println("curr.: " + current) 867 | println(stack.mkString("\n")) 868 | } 869 | 870 | } 871 | 872 | 873 | object RestartException extends util.control.ControlThrowable 874 | 875 | 876 | final class Gen 877 | 878 | 879 | private[ctries2] object Debug { 880 | import collection._ 881 | 882 | lazy val logbuffer = new java.util.concurrent.ConcurrentLinkedQueue[AnyRef] 883 | 884 | def log(s: AnyRef) = logbuffer.add(s) 885 | 886 | def flush() { 887 | for (s <- JavaConversions.asScalaIterator(logbuffer.iterator())) Console.out.println(s.toString) 888 | logbuffer.clear() 889 | } 890 | 891 | def clear() { 892 | logbuffer.clear() 893 | } 894 | 895 | } 896 | 897 | 898 | 899 | 900 | 901 | 902 | 903 | 904 | 905 | 906 | 907 | -------------------------------------------------------------------------------- /src/test/scala/ctries/ctries.scala: -------------------------------------------------------------------------------- 1 | package ctries 2 | 3 | 4 | 5 | 6 | import org.scalatest._ 7 | import org.scalatest.matchers.ShouldMatchers 8 | 9 | 10 | /* disabled! 11 | case class Wrap(i: Int) { 12 | override def hashCode = i * 0x9e3775cd 13 | } 14 | 15 | 16 | class CtrieSpec extends WordSpec with ShouldMatchers { 17 | 18 | "A ctrie" should { 19 | 20 | "be created" in { 21 | val ct = new ConcurrentTrie 22 | ct 23 | println(ct.string) 24 | } 25 | 26 | "be inserted into" in { 27 | val ct = new ConcurrentTrie[Int, Int] 28 | ct.insert(5, 5) 29 | println(ct.string) 30 | assert(ct.lookupOpt(5) == Some(5)) 31 | } 32 | 33 | "be inserted two values" in { 34 | val ct = new ConcurrentTrie[Int, Int] 35 | ct.insert(0, 5) 36 | ct.insert(1, 10) 37 | println(ct.string) 38 | assert(ct.lookupOpt(0) == Some(5)) 39 | assert(ct.lookupOpt(1) == Some(10)) 40 | } 41 | 42 | "be inserted many values" in { 43 | val sz = 1408 44 | val ct = new ConcurrentTrie[Int, Int] 45 | for (i <- 0 until sz) ct.insert(i, i) 46 | // println(ct.string) 47 | for (i <- 0 until sz) assert(ct.lookupOpt(i) == Some(i)) 48 | } 49 | 50 | "be concurrently inserted non-overlapping values" in { 51 | val sz = 14080 52 | val ct = new ConcurrentTrie[Wrap, Int] 53 | val nump = Runtime.getRuntime.availableProcessors 54 | 55 | class Inserter(mult: Int) extends Thread { 56 | override def run { 57 | for (i <- 0 until sz) ct.insert(new Wrap(i + mult * sz), i + mult * sz) 58 | } 59 | } 60 | 61 | val threads = for (i <- 0 until nump) yield new Inserter(i) 62 | threads.foreach(_.start()) 63 | threads.foreach(_.join()) 64 | 65 | for (i <- 0 until (nump * sz)) assert(ct.lookupOpt(new Wrap(i)) == Some(i), (i, ct.lookupOpt(new Wrap(i)))) 66 | } 67 | 68 | "be concurrently inserted overlapping values" in { 69 | val sz = 25000 70 | val ct = new ConcurrentTrie[Wrap, Int] 71 | val nump = Runtime.getRuntime.availableProcessors 72 | 73 | class Inserter(mult: Int) extends Thread { 74 | override def run { 75 | for (i <- 0 until sz) { 76 | val x = (i + mult * (sz / nump)) % sz 77 | ct.insert(new Wrap(x), x) 78 | } 79 | } 80 | } 81 | 82 | val threads = for (i <- 0 until nump) yield new Inserter(i) 83 | threads.foreach(_.start()) 84 | threads.foreach(_.join()) 85 | 86 | for (i <- 0 until sz) assert(ct.lookupOpt(new Wrap(i)) == Some(i), (i, ct.lookupOpt(new Wrap(i)))) 87 | } 88 | 89 | "be inserted into and removed from" in { 90 | val sz = 148 91 | val ct = new ConcurrentTrie[Int, Int] 92 | 93 | for (i <- 0 until sz) ct.insert(i, i) 94 | 95 | for (i <- (0 until sz).reverse) { 96 | val removedvalue = ct.remove(i) 97 | assert(removedvalue == Some(i), (i, removedvalue, ct.string)) 98 | assert(ct.lookupOpt(i) == None, i) 99 | for (j <- 0 until i) assert(ct.lookupOpt(j) == Some(j), (j, ct.lookupOpt(j))) 100 | } 101 | 102 | println(ct.string) 103 | } 104 | 105 | "be inserted many values and removed from" in { 106 | val sz = 14800 107 | val ct = new ConcurrentTrie[Int, Int] 108 | 109 | for (i <- 0 until sz) ct.insert(i, i) 110 | 111 | for (i <- (0 until sz).reverse) { 112 | val removedvalue = ct.remove(i) 113 | assert(removedvalue == Some(i), (i, removedvalue)) 114 | assert(ct.lookupOpt(i) == None, i) 115 | } 116 | } 117 | 118 | "be alternately inserted into and removed from" in { 119 | val sz = 32000 120 | val ct = new ConcurrentTrie[Int, Int] 121 | 122 | for (i <- 0 until sz) { 123 | ct.insert(i, i) 124 | if (i % 3 == 0) assert(ct.remove(i) == Some(i)) 125 | } 126 | 127 | for (i <- 0 until sz) assert(ct.lookupOpt(i) == (if (i % 3 != 0) Some(i) else None)) 128 | } 129 | 130 | def removeNonOverlapping(perthread: Int, nump: Int): ConcurrentTrie[Wrap, Int] = { 131 | val ct = new ConcurrentTrie[Wrap, Int] 132 | val sz = perthread * nump 133 | for (i <- 0 until sz) ct.insert(new Wrap(i), i) 134 | for (i <- 0 until sz) assert(ct.lookupOpt(new Wrap(i)) == Some(i)) 135 | 136 | class Remover(mult: Int) extends Thread { 137 | override def run { 138 | for (i <- (mult * perthread) until (mult * perthread + perthread)) ct.remove(new Wrap(i)) 139 | } 140 | } 141 | 142 | val threads = for (i <- 0 until nump) yield new Remover(i) 143 | threads.foreach(_.start()) 144 | threads.foreach(_.join()) 145 | 146 | for (i <- 0 until sz) assert(ct.lookupOpt(new Wrap(i)) == None) 147 | ct 148 | } 149 | 150 | "be concurrently removed from with 128 non-overlapping values * procs, repeated 100 times" in { 151 | for (i <- 0 until 100) removeNonOverlapping(128, Runtime.getRuntime.availableProcessors) 152 | } 153 | 154 | "be concurrently removed from with 32k non-overlapping values * procs" in { 155 | val ct = removeNonOverlapping(32000, Runtime.getRuntime.availableProcessors) 156 | println(ct.string) 157 | } 158 | 159 | "be concurrently removed from with 256k non-overlapping values * procs" in { 160 | val ct = removeNonOverlapping(256000, Runtime.getRuntime.availableProcessors) 161 | println(ct.string) 162 | } 163 | 164 | "be concurrently removed from with 16k non-overlapping values * procs * 10" in { 165 | val ct = removeNonOverlapping(16000, 10 * Runtime.getRuntime.availableProcessors) 166 | println(ct.string) 167 | } 168 | 169 | "be concurrently removed from with overlapping values" in { 170 | val sz = 256000 171 | val ct = new ConcurrentTrie[Wrap, Int] 172 | val nump = Runtime.getRuntime.availableProcessors * 2 173 | for (i <- 0 until sz) ct.insert(new Wrap(i), i) 174 | for (i <- 0 until sz) assert(ct.lookupOpt(new Wrap(i)) == Some(i)) 175 | 176 | class Remover(mult: Int) extends Thread { 177 | override def run { 178 | for (i <- 0 until sz) { 179 | val x = (i + mult * (sz / nump)) % sz 180 | ct.remove(new Wrap(x)) 181 | } 182 | } 183 | } 184 | 185 | val threads = for (i <- 0 until nump) yield new Remover(i) 186 | threads.foreach(_.start()) 187 | threads.foreach(_.join()) 188 | 189 | for (i <- 0 until sz) assert(ct.lookupOpt(new Wrap(i)) == None) 190 | println(ct.string) 191 | } 192 | 193 | "be concurrently inserted into and removed from" in { 194 | val presz = 256000 195 | val chsz = 256000 196 | val totelems = presz + chsz 197 | val ct = new ConcurrentTrie[Wrap, Int] 198 | val nump = Runtime.getRuntime.availableProcessors 199 | for (i <- 0 until presz) ct.insert(new Wrap(i), i) 200 | 201 | class Inserter(mult: Int) extends Thread { 202 | override def run { 203 | for (i <- 0 until chsz) ct.insert(new Wrap(presz + i), i) 204 | } 205 | } 206 | 207 | class Remover(offset: Int, total: Int, mult: Int) extends Thread { 208 | override def run { 209 | for (i <- 0 until total) ct.remove(new Wrap(offset + i)) 210 | } 211 | } 212 | 213 | val inserters = for (i <- 0 until nump) yield new Inserter(i) 214 | val removers = for (i <- 0 until nump) yield new Remover(presz, chsz, i) 215 | val threads = inserters ++ removers 216 | threads.foreach(_.start()) 217 | threads.foreach(_.join()) 218 | 219 | for (i <- 0 until presz) assert(ct.lookupOpt(new Wrap(i)) == Some(i)) 220 | 221 | val cleaners = for (i <- 0 until nump) yield new Remover(0, totelems, i) 222 | cleaners.foreach(_.start()) 223 | cleaners.foreach(_.join()) 224 | 225 | for (i <- 0 until totelems) assert(ct.lookupOpt(new Wrap(i)) == None) 226 | println(ct.string) 227 | } 228 | 229 | } 230 | 231 | } 232 | */ 233 | -------------------------------------------------------------------------------- /src/test/scala/ctries2/DumbHash.scala: -------------------------------------------------------------------------------- 1 | package ctries2 2 | 3 | 4 | 5 | 6 | 7 | class DumbHash(val i: Int) { 8 | override def equals(other: Any) = other match { 9 | case that: DumbHash => that.i == this.i 10 | case _ => false 11 | } 12 | override def hashCode = i % 5 13 | override def toString = "DH(%s)".format(i) 14 | } 15 | -------------------------------------------------------------------------------- /src/test/scala/ctries2/Wrap.scala: -------------------------------------------------------------------------------- 1 | package ctries2 2 | 3 | 4 | 5 | 6 | 7 | case class Wrap(i: Int) { 8 | override def hashCode = i // * 0x9e3775cd 9 | } 10 | 11 | 12 | -------------------------------------------------------------------------------- /src/test/scala/ctries2/basics.scala: -------------------------------------------------------------------------------- 1 | package ctries2 2 | 3 | 4 | 5 | import org.scalacheck._ 6 | import Prop._ 7 | import org.scalacheck.Gen._ 8 | import collection._ 9 | 10 | 11 | 12 | object CtrieChecks extends Properties("Ctrie") { 13 | 14 | /* generators */ 15 | 16 | val sizes = choose(0, 200000) 17 | 18 | val threadCounts = choose(2, 16) 19 | 20 | val threadCountsAndSizes = for { 21 | p <- threadCounts 22 | sz <- sizes 23 | } yield (p, sz); 24 | 25 | 26 | /* helpers */ 27 | 28 | def inParallel[T](totalThreads: Int)(body: Int => T): Seq[T] = { 29 | val threads = for (idx <- 0 until totalThreads) yield new Thread { 30 | setName("ParThread-" + idx) 31 | private var res: T = _ 32 | override def run() { 33 | res = body(idx) 34 | } 35 | def result = { 36 | this.join() 37 | res 38 | } 39 | } 40 | 41 | threads foreach (_.start()) 42 | threads map (_.result) 43 | } 44 | 45 | def spawn[T](body: =>T): { def get: T } = { 46 | val t = new Thread { 47 | setName("SpawnThread") 48 | private var res: T = _ 49 | override def run() { 50 | res = body 51 | } 52 | def result = res 53 | } 54 | t.start() 55 | new { 56 | def get: T = { 57 | t.join() 58 | t.result 59 | } 60 | } 61 | } 62 | 63 | def elementRange(threadIdx: Int, totalThreads: Int, totalElems: Int): Range = { 64 | val sz = totalElems 65 | val idx = threadIdx 66 | val p = totalThreads 67 | val start = (sz / p) * idx + math.min(idx, sz % p) 68 | val elems = (sz / p) + (if (idx < sz % p) 1 else 0) 69 | val end = start + elems 70 | (start until end) 71 | } 72 | 73 | def hasGrown[K, V](last: Map[K, V], current: Map[K, V]) = { 74 | (last.size <= current.size) && { 75 | last forall { 76 | case (k, v) => current.get(k) == Some(v) 77 | } 78 | } 79 | } 80 | 81 | object err { 82 | var buffer = new StringBuilder 83 | def println(a: AnyRef) = buffer.append(a.toString).append("\n") 84 | def clear() = buffer.clear() 85 | def flush() = { 86 | Console.out.println(buffer) 87 | clear() 88 | } 89 | } 90 | 91 | 92 | /* properties */ 93 | 94 | property("concurrent growing snapshots") = forAll(threadCounts, sizes) { 95 | (numThreads, numElems) => 96 | val p = 3 //numThreads 97 | val sz = 102 //numElems 98 | val ct = new ConcurrentTrie[Wrap, Int] 99 | err.println("----------------------") 100 | Debug.clear() 101 | 102 | // checker 103 | val checker = spawn { 104 | def check(last: Map[Wrap, Int], iterationsLeft: Int): Boolean = { 105 | val current = ct.readOnlySnapshot() 106 | // err.println("new check! " + current.size + " / " + sz) 107 | if (!hasGrown(last, current)) false 108 | else if (current.size >= sz) true 109 | else if (iterationsLeft < 0) false 110 | else check(current, iterationsLeft - 1) 111 | } 112 | check(ct.readOnlySnapshot(), 500) 113 | } 114 | 115 | // fillers 116 | inParallel(p) { 117 | idx => 118 | elementRange(idx, p, sz) foreach (i => ct.update(Wrap(i), i)) 119 | } 120 | 121 | // wait for checker to finish 122 | val growing = true//checker.get 123 | 124 | val ok = growing && ((0 until sz) forall { 125 | case i => ct.get(Wrap(i)) == Some(i) 126 | }) 127 | 128 | if (!ok) { 129 | err.flush() 130 | Debug.flush() 131 | err.println("size: " + ct.size) 132 | err.println(ct.toList.map(_._2).sorted.zip(0 until sz).find(p => p._1 != p._2)) 133 | err.println(ct.RDCSS_READ_ROOT().mainnode) 134 | err.flush() 135 | } else err.clear() 136 | 137 | ok 138 | } 139 | 140 | property("update") = forAll(sizes) { 141 | (n: Int) => 142 | val ct = new ConcurrentTrie[Int, Int] 143 | for (i <- 0 until n) ct(i) = i 144 | (0 until n) forall { 145 | case i => ct(i) == i 146 | } 147 | } 148 | 149 | 150 | property("concurrent update") = forAll(threadCountsAndSizes) { 151 | case (p, sz) => 152 | val ct = new ConcurrentTrie[Wrap, Int] 153 | 154 | inParallel(p) { 155 | idx => 156 | for (i <- elementRange(idx, p, sz)) ct(Wrap(i)) = i 157 | } 158 | 159 | (0 until sz) forall { 160 | case i => ct(Wrap(i)) == i 161 | } 162 | } 163 | 164 | 165 | property("concurrent remove") = forAll(threadCounts, sizes) { 166 | (p, sz) => 167 | val ct = new ConcurrentTrie[Wrap, Int] 168 | for (i <- 0 until sz) ct(Wrap(i)) = i 169 | 170 | inParallel(p) { 171 | idx => 172 | for (i <- elementRange(idx, p, sz)) ct.remove(Wrap(i)) 173 | } 174 | 175 | (0 until sz) forall { 176 | case i => ct.get(Wrap(i)) == None 177 | } 178 | } 179 | 180 | 181 | property("concurrent putIfAbsent") = forAll(threadCounts, sizes) { 182 | (p, sz) => 183 | val ct = new ConcurrentTrie[Wrap, Int] 184 | 185 | val results = inParallel(p) { 186 | idx => 187 | elementRange(idx, p, sz) find (i => ct.putIfAbsent(Wrap(i), i) != None) 188 | } 189 | 190 | (results forall (_ == None)) && ((0 until sz) forall { 191 | case i => ct.get(Wrap(i)) == Some(i) 192 | }) 193 | } 194 | 195 | } 196 | 197 | 198 | 199 | 200 | 201 | 202 | 203 | 204 | 205 | 206 | -------------------------------------------------------------------------------- /src/test/scala/ctries2/concmap.scala: -------------------------------------------------------------------------------- 1 | package ctries2 2 | 3 | 4 | 5 | import org.scalatest._ 6 | import org.scalatest.matchers.ShouldMatchers 7 | 8 | 9 | 10 | class ConcurrentMapSpec extends WordSpec with ShouldMatchers { 11 | 12 | val initsz = 1500 13 | val secondsz = 2000 14 | 15 | "A ctrie2" should { 16 | 17 | "support put" in { 18 | val ct = new ConcurrentTrie[Wrap, Int] 19 | for (i <- 0 until initsz) assert(ct.put(new Wrap(i), i) == None) 20 | for (i <- 0 until initsz) assert(ct.put(new Wrap(i), -i) == Some(i)) 21 | } 22 | 23 | "support put if absent" in { 24 | val ct = new ConcurrentTrie[Wrap, Int] 25 | for (i <- 0 until initsz) ct.update(new Wrap(i), i) 26 | for (i <- 0 until initsz) assert(ct.putIfAbsent(new Wrap(i), -i) == Some(i)) 27 | for (i <- 0 until initsz) assert(ct.putIfAbsent(new Wrap(i), -i) == Some(i)) 28 | for (i <- initsz until secondsz) assert(ct.putIfAbsent(new Wrap(i), -i) == None) 29 | for (i <- initsz until secondsz) assert(ct.putIfAbsent(new Wrap(i), i) == Some(-i)) 30 | } 31 | 32 | "support remove if mapped to a specific value" in { 33 | val ct = new ConcurrentTrie[Wrap, Int] 34 | for (i <- 0 until initsz) ct.update(new Wrap(i), i) 35 | for (i <- 0 until initsz) assert(ct.remove(new Wrap(i), -i - 1) == false) 36 | for (i <- 0 until initsz) assert(ct.remove(new Wrap(i), i) == true) 37 | for (i <- 0 until initsz) assert(ct.remove(new Wrap(i), i) == false) 38 | } 39 | 40 | "support replace if mapped to a specific value" in { 41 | val ct = new ConcurrentTrie[Wrap, Int] 42 | for (i <- 0 until initsz) ct.update(new Wrap(i), i) 43 | for (i <- 0 until initsz) assert(ct.replace(new Wrap(i), -i - 1, -i - 2) == false) 44 | for (i <- 0 until initsz) assert(ct.replace(new Wrap(i), i, -i - 2) == true) 45 | for (i <- 0 until initsz) assert(ct.replace(new Wrap(i), i, -i - 2) == false) 46 | for (i <- initsz until secondsz) assert(ct.replace(new Wrap(i), i, 0) == false) 47 | } 48 | 49 | "support replace if present" in { 50 | val ct = new ConcurrentTrie[Wrap, Int] 51 | for (i <- 0 until initsz) ct.update(new Wrap(i), i) 52 | for (i <- 0 until initsz) assert(ct.replace(new Wrap(i), -i) == Some(i)) 53 | for (i <- 0 until initsz) assert(ct.replace(new Wrap(i), i) == Some(-i)) 54 | for (i <- initsz until secondsz) assert(ct.replace(new Wrap(i), i) == None) 55 | } 56 | 57 | def assertEqual(a: Any, b: Any) = { 58 | if (a != b) println(a, b) 59 | assert(a == b) 60 | } 61 | 62 | "support replace if mapped to a specific value, using several threads" in { 63 | val ct = new ConcurrentTrie[Wrap, Int] 64 | val sz = 155000 65 | for (i <- 0 until sz) ct.update(new Wrap(i), i) 66 | 67 | class Updater(index: Int, offs: Int) extends Thread { 68 | override def run() { 69 | var repeats = 0 70 | for (i <- 0 until sz) { 71 | val j = (offs + i) % sz 72 | var k = Int.MaxValue 73 | do { 74 | if (k != Int.MaxValue) repeats += 1 75 | k = ct.lookup(new Wrap(j)) 76 | } while (!ct.replace(new Wrap(j), k, -k)) 77 | } 78 | //println("Thread %d repeats: %d".format(index, repeats)) 79 | } 80 | } 81 | 82 | val threads = for (i <- 0 until 16) yield new Updater(i, sz / 32 * i) 83 | threads.foreach(_.start()) 84 | threads.foreach(_.join()) 85 | 86 | for (i <- 0 until sz) assertEqual(ct(new Wrap(i)), i) 87 | 88 | val threads2 = for (i <- 0 until 15) yield new Updater(i, sz / 32 * i) 89 | threads2.foreach(_.start()) 90 | threads2.foreach(_.join()) 91 | 92 | for (i <- 0 until sz) assertEqual(ct(new Wrap(i)), -i) 93 | } 94 | 95 | "support put if absent, several threads" in { 96 | val ct = new ConcurrentTrie[Wrap, Int] 97 | val sz = 210000 98 | 99 | class Updater(offs: Int) extends Thread { 100 | override def run() { 101 | for (i <- 0 until sz) { 102 | val j = (offs + i) % sz 103 | ct.putIfAbsent(new Wrap(j), j) 104 | assert(ct.lookup(new Wrap(j)) == j) 105 | } 106 | } 107 | } 108 | 109 | val threads = for (i <- 0 until 16) yield new Updater(sz / 32 * i) 110 | threads.foreach(_.start()) 111 | threads.foreach(_.join()) 112 | 113 | for (i <- 0 until sz) assert(ct(new Wrap(i)) == i) 114 | } 115 | 116 | "support remove if mapped to a specific value, several threads" in { 117 | val ct = new ConcurrentTrie[Wrap, Int] 118 | val sz = 155000 119 | for (i <- 0 until sz) ct.update(new Wrap(i), i) 120 | 121 | class Remover(offs: Int) extends Thread { 122 | override def run() { 123 | for (i <- 0 until sz) { 124 | val j = (offs + i) % sz 125 | ct.remove(new Wrap(j), j) 126 | assert(ct.get(new Wrap(j)) == None) 127 | } 128 | } 129 | } 130 | 131 | val threads = for (i <- 0 until 16) yield new Remover(sz / 32 * i) 132 | threads.foreach(_.start()) 133 | threads.foreach(_.join()) 134 | 135 | for (i <- 0 until sz) assert(ct.get(new Wrap(i)) == None) 136 | } 137 | 138 | "have all or none of the elements depending on the oddity" in { 139 | val ct = new ConcurrentTrie[Wrap, Int] 140 | val sz = 65000 141 | for (i <- 0 until sz) ct(new Wrap(i)) = i 142 | 143 | class Modifier(index: Int, offs: Int) extends Thread { 144 | override def run() { 145 | for (j <- 0 until sz) { 146 | val i = (offs + j) % sz 147 | var success = false 148 | do { 149 | if (ct.contains(new Wrap(i))) { 150 | success = ct.remove(new Wrap(i)) != None 151 | } else { 152 | success = ct.putIfAbsent(new Wrap(i), i) == None 153 | } 154 | } while (!success) 155 | } 156 | } 157 | } 158 | 159 | def modify(n: Int) = { 160 | val threads = for (i <- 0 until n) yield new Modifier(i, sz / n * i) 161 | threads.foreach(_.start()) 162 | threads.foreach(_.join()) 163 | } 164 | 165 | modify(16) 166 | for (i <- 0 until sz) assertEqual(ct.get(new Wrap(i)), Some(i)) 167 | modify(15) 168 | for (i <- 0 until sz) assertEqual(ct.get(new Wrap(i)), None) 169 | } 170 | 171 | } 172 | } 173 | -------------------------------------------------------------------------------- /src/test/scala/ctries2/ctries2.scala: -------------------------------------------------------------------------------- 1 | package ctries2 2 | 3 | 4 | 5 | 6 | import org.scalatest._ 7 | import org.scalatest.matchers.ShouldMatchers 8 | import annotation.tailrec 9 | 10 | 11 | 12 | class CtrieSpec extends WordSpec with ShouldMatchers { 13 | 14 | "A ctrie2" should { 15 | 16 | "be created" in { 17 | val ct = new ConcurrentTrie 18 | ct 19 | println(ct.string) 20 | } 21 | 22 | "be updated into" in { 23 | val ct = new ConcurrentTrie[Int, Int] 24 | ct.update(5, 5) 25 | println(ct.string) 26 | assert(ct.get(5) == Some(5)) 27 | } 28 | 29 | "be updated two values" in { 30 | val ct = new ConcurrentTrie[Int, Int] 31 | ct.update(0, 5) 32 | ct.update(1, 10) 33 | println(ct.string) 34 | assert(ct.get(0) == Some(5)) 35 | assert(ct.get(1) == Some(10)) 36 | } 37 | 38 | "be updated many values" in { 39 | val sz = 1408 40 | val ct = new ConcurrentTrie[Int, Int] 41 | for (i <- 0 until sz) ct.update(i, i) 42 | //println(ct.string) 43 | for (i <- 0 until sz) assert(ct.get(i) == Some(i)) 44 | } 45 | 46 | "be concurrently updated non-overlapping values" in { 47 | val sz = 14080 48 | val ct = new ConcurrentTrie[Wrap, Int] 49 | val nump = Runtime.getRuntime.availableProcessors 50 | 51 | class Updateer(mult: Int) extends Thread { 52 | override def run() { 53 | for (i <- 0 until sz) ct.update(new Wrap(i + mult * sz), i + mult * sz) 54 | } 55 | } 56 | 57 | val threads = for (i <- 0 until nump) yield new Updateer(i) 58 | threads.foreach(_.start()) 59 | threads.foreach(_.join()) 60 | 61 | for (i <- 0 until (nump * sz)) assert(ct.get(new Wrap(i)) == Some(i), (i, ct.get(new Wrap(i)))) 62 | } 63 | 64 | "be concurrently updated overlapping values" in { 65 | val sz = 125000 66 | val ct = new ConcurrentTrie[Wrap, Int] 67 | val nump = Runtime.getRuntime.availableProcessors 68 | 69 | class Updateer(mult: Int) extends Thread { 70 | override def run() { 71 | for (i <- 0 until sz) { 72 | val x = (i + mult * (sz / nump)) % sz 73 | ct.update(new Wrap(x), x) 74 | } 75 | } 76 | } 77 | 78 | val threads = for (i <- 0 until nump) yield new Updateer(i) 79 | threads.foreach(_.start()) 80 | threads.foreach(_.join()) 81 | 82 | for (i <- 0 until sz) assert(ct.get(new Wrap(i)) == Some(i), (i, ct.get(new Wrap(i)))) 83 | } 84 | 85 | "be updated into and removed from" in { 86 | val sz = 148 87 | val ct = new ConcurrentTrie[Int, Int] 88 | 89 | for (i <- 0 until sz) ct.update(i, i) 90 | 91 | for (i <- (0 until sz).reverse) { 92 | val removedvalue = ct.remove(i) 93 | assert(removedvalue == Some(i), (i, removedvalue, ct.string)) 94 | assert(ct.get(i) == None, i) 95 | for (j <- 0 until i) assert(ct.get(j) == Some(j), (j, ct.get(j))) 96 | } 97 | 98 | println(ct.string) 99 | } 100 | 101 | "be updated many values and removed from" in { 102 | val sz = 14800 103 | val ct = new ConcurrentTrie[Int, Int] 104 | 105 | for (i <- 0 until sz) ct.update(i, i) 106 | 107 | for (i <- (0 until sz).reverse) { 108 | val removedvalue = ct.remove(i) 109 | assert(removedvalue == Some(i), (i, removedvalue)) 110 | assert(ct.get(i) == None, i) 111 | //if (i < 40) println(ct.string) 112 | } 113 | } 114 | 115 | "be alternately updated into and removed from" in { 116 | val sz = 32000 117 | val ct = new ConcurrentTrie[Int, Int] 118 | 119 | for (i <- 0 until sz) { 120 | ct.update(i, i) 121 | if (i % 3 == 0) assert(ct.remove(i) == Some(i)) 122 | } 123 | 124 | for (i <- 0 until sz) assert(ct.get(i) == (if (i % 3 != 0) Some(i) else None)) 125 | } 126 | 127 | def removeNonOverlapping(perthread: Int, nump: Int): ConcurrentTrie[Wrap, Int] = { 128 | val ct = new ConcurrentTrie[Wrap, Int] 129 | val sz = perthread * (nump max 8) 130 | for (i <- 0 until sz) ct.update(new Wrap(i), i) 131 | for (i <- 0 until sz) assert(ct.get(new Wrap(i)) == Some(i)) 132 | 133 | class Remover(mult: Int) extends Thread { 134 | override def run() { 135 | for (i <- (mult * perthread) until (mult * perthread + perthread)) ct.remove(new Wrap(i)) 136 | } 137 | } 138 | 139 | val threads = for (i <- 0 until nump) yield new Remover(i) 140 | threads.foreach(_.start()) 141 | threads.foreach(_.join()) 142 | 143 | for (i <- 0 until sz) assert(ct.get(new Wrap(i)) == None) 144 | ct 145 | } 146 | 147 | "be concurrently removed from with 128 non-overlapping values * procs, repeated 100 times" in { 148 | for (i <- 0 until 100) removeNonOverlapping(128, Runtime.getRuntime.availableProcessors) 149 | } 150 | 151 | "be concurrently removed from with 32k non-overlapping values * procs" in { 152 | val ct = removeNonOverlapping(32000, Runtime.getRuntime.availableProcessors) 153 | println(ct.string) 154 | } 155 | 156 | "be concurrently removed from with 256k non-overlapping values * procs" in { 157 | val ct = removeNonOverlapping(256000, Runtime.getRuntime.availableProcessors) 158 | println(ct.string) 159 | } 160 | 161 | "be concurrently removed from with 16k non-overlapping values * procs * 10" in { 162 | val ct = removeNonOverlapping(16000, 10 * Runtime.getRuntime.availableProcessors) 163 | println(ct.string) 164 | } 165 | 166 | "be concurrently removed from with overlapping values" in { 167 | val sz = 256000 168 | val ct = new ConcurrentTrie[Wrap, Int] 169 | val nump = Runtime.getRuntime.availableProcessors * 2 170 | for (i <- 0 until sz) ct.update(new Wrap(i), i) 171 | for (i <- 0 until sz) assert(ct.get(new Wrap(i)) == Some(i)) 172 | 173 | class Remover(mult: Int) extends Thread { 174 | override def run() { 175 | for (i <- 0 until sz) { 176 | val x = (i + mult * (sz / nump)) % sz 177 | ct.remove(new Wrap(x)) 178 | } 179 | } 180 | } 181 | 182 | val threads = for (i <- 0 until nump) yield new Remover(i) 183 | threads.foreach(_.start()) 184 | threads.foreach(_.join()) 185 | 186 | for (i <- 0 until sz) assert(ct.get(new Wrap(i)) == None) 187 | println(ct.string) 188 | } 189 | 190 | "be concurrently updated into and removed from" in { 191 | val presz = 256000 192 | val chsz = 256000 193 | val totelems = presz + chsz 194 | val ct = new ConcurrentTrie[Wrap, Int] 195 | val nump = Runtime.getRuntime.availableProcessors 196 | for (i <- 0 until presz) ct.update(new Wrap(i), i) 197 | 198 | class Updater(mult: Int) extends Thread { 199 | override def run() { 200 | for (i <- 0 until chsz) ct.update(new Wrap(presz + i), i) 201 | } 202 | } 203 | 204 | class Remover(offset: Int, total: Int, mult: Int) extends Thread { 205 | override def run() { 206 | for (i <- 0 until total) ct.remove(new Wrap(offset + i)) 207 | } 208 | } 209 | 210 | val updaters = for (i <- 0 until nump) yield new Updater(i) 211 | val removers = for (i <- 0 until nump) yield new Remover(presz, chsz, i) 212 | val threads = updaters ++ removers 213 | threads.foreach(_.start()) 214 | threads.foreach(_.join()) 215 | 216 | for (i <- 0 until presz) assert(ct.get(new Wrap(i)) == Some(i)) 217 | 218 | val cleaners = for (i <- 0 until nump) yield new Remover(0, totelems, i) 219 | cleaners.foreach(_.start()) 220 | cleaners.foreach(_.join()) 221 | 222 | for (i <- 0 until totelems) assert(ct.get(new Wrap(i)) == None) 223 | println(ct.string) 224 | } 225 | 226 | "be concurrently emptied and then filled" in { 227 | val sz = 1 << 18 228 | val nump = 16 229 | val ct = new ConcurrentTrie[Wrap, Int] 230 | for (i <- 0 until sz) ct.update(new Wrap(i), i) 231 | 232 | class Worker(i: Int) extends Thread { 233 | override def run() { 234 | val start = i * sz / nump 235 | val end = start + sz / nump 236 | for (i <- start until end) ct.remove(new Wrap(i)) 237 | ct.put(new Wrap(-i), -i) 238 | 239 | @tailrec def barrier() { 240 | if ((0 until nump).exists(i => ct.get(new Wrap(-i)) == None)) barrier() 241 | } 242 | barrier() 243 | 244 | for (i <- 0 until sz) ct.put(new Wrap(i), -i) 245 | } 246 | } 247 | 248 | val workers = for (i <- 0 until nump) yield new Worker(i) 249 | workers.foreach(_.start()) 250 | workers.foreach(_.join()) 251 | 252 | for (i <- 0 until sz) assert(ct.get(new Wrap(i)) == Some(-i)) 253 | for (i <- 0 until nump) assert(ct.get(new Wrap(-i)) == Some(-i)) 254 | } 255 | 256 | "be concurrently looked up, emptied and filled" in { 257 | val R = 5 258 | val W = 10 259 | val filltimes = 10 260 | val readtimes = 100 261 | val sz = 145000 262 | val ct = new ConcurrentTrie[Wrap, Int] 263 | 264 | class Reader extends Thread { 265 | override def run() { 266 | for (k <- 0 until readtimes) { 267 | for (i <- 0 until sz) ct.get(new Wrap(i)) 268 | } 269 | } 270 | } 271 | 272 | class Filler extends Thread { 273 | override def run() { 274 | for (i <- 0 until sz) ct.put(new Wrap(i), i) 275 | } 276 | } 277 | 278 | class Remover extends Thread { 279 | override def run() { 280 | for (i <- 0 until sz) ct.remove(new Wrap(i)) 281 | } 282 | } 283 | 284 | def fill() { 285 | val fs = for (i <- 0 until W) yield new Filler 286 | fs.foreach(_.start()) 287 | fs.foreach(_.join()) 288 | } 289 | 290 | def empty() { 291 | val rs = for (i <- 0 until W) yield new Remover 292 | rs.foreach(_.start()) 293 | rs.foreach(_.join()) 294 | } 295 | 296 | def checkEmpty = for (i <- 0 until sz) assert(ct.get(new Wrap(i)) == None) 297 | 298 | fill() 299 | 300 | val readers = for (i <- 0 until R) yield new Reader 301 | readers.foreach(_.start()) 302 | 303 | for (i <- 0 until filltimes) { 304 | empty() 305 | fill() 306 | } 307 | empty() 308 | 309 | checkEmpty 310 | readers.foreach(_.join()) 311 | checkEmpty 312 | } 313 | 314 | } 315 | 316 | } 317 | 318 | 319 | 320 | 321 | 322 | 323 | 324 | 325 | 326 | 327 | 328 | 329 | 330 | 331 | 332 | 333 | 334 | 335 | 336 | 337 | 338 | 339 | 340 | 341 | 342 | 343 | 344 | 345 | -------------------------------------------------------------------------------- /src/test/scala/ctries2/iteratorspec.scala: -------------------------------------------------------------------------------- 1 | package ctries2 2 | 3 | 4 | 5 | import org.scalatest._ 6 | import org.scalatest.matchers.ShouldMatchers 7 | import collection._ 8 | 9 | 10 | 11 | class IteratorSpec extends WordSpec with ShouldMatchers { 12 | 13 | "A ctrie2 iterator" should { 14 | 15 | "work for an empty trie" in { 16 | val ct = new ConcurrentTrie 17 | val it = ct.iterator 18 | 19 | it.hasNext should equal (false) 20 | evaluating { it.next() } should produce [NoSuchElementException] 21 | } 22 | 23 | def nonEmptyIteratorCheck(sz: Int) { 24 | val ct = new ConcurrentTrie[Wrap, Int] 25 | for (i <- 0 until sz) ct.put(new Wrap(i), i) 26 | 27 | val it = ct.iterator 28 | val tracker = mutable.Map[Wrap, Int]() 29 | for (i <- 0 until sz) { 30 | assert(it.hasNext == true) 31 | tracker += it.next 32 | } 33 | 34 | it.hasNext should equal (false) 35 | evaluating { it.next() } should produce [NoSuchElementException] 36 | tracker.size should equal (sz) 37 | tracker should equal (ct) 38 | } 39 | 40 | "work for a 1 element trie" in { 41 | nonEmptyIteratorCheck(1) 42 | } 43 | 44 | "work for a 2 element trie" in { 45 | nonEmptyIteratorCheck(2) 46 | } 47 | 48 | "work for a 3 element trie" in { 49 | nonEmptyIteratorCheck(3) 50 | } 51 | 52 | "work for a 5 element trie" in { 53 | nonEmptyIteratorCheck(5) 54 | } 55 | 56 | "work for a 10 element trie" in { 57 | nonEmptyIteratorCheck(10) 58 | } 59 | 60 | "work for a 20 element trie" in { 61 | nonEmptyIteratorCheck(20) 62 | } 63 | 64 | "work for a 50 element trie" in { 65 | nonEmptyIteratorCheck(50) 66 | } 67 | 68 | "work for a 100 element trie" in { 69 | nonEmptyIteratorCheck(100) 70 | } 71 | 72 | "work for a 1k element trie" in { 73 | nonEmptyIteratorCheck(1000) 74 | } 75 | 76 | "work for a 5k element trie" in { 77 | nonEmptyIteratorCheck(5000) 78 | } 79 | 80 | "work for a 75k element trie" in { 81 | nonEmptyIteratorCheck(75000) 82 | } 83 | 84 | "work for a 250k element trie" in { 85 | nonEmptyIteratorCheck(500000) 86 | } 87 | 88 | def nonEmptyCollideCheck(sz: Int) { 89 | val ct = new ConcurrentTrie[DumbHash, Int] 90 | for (i <- 0 until sz) ct.put(new DumbHash(i), i) 91 | 92 | val it = ct.iterator 93 | val tracker = mutable.Map[DumbHash, Int]() 94 | for (i <- 0 until sz) { 95 | assert(it.hasNext == true) 96 | tracker += it.next 97 | } 98 | 99 | it.hasNext should equal (false) 100 | evaluating { it.next() } should produce [NoSuchElementException] 101 | tracker.size should equal (sz) 102 | tracker should equal (ct) 103 | } 104 | 105 | "work for colliding hashcodes, 2 element trie" in { 106 | nonEmptyCollideCheck(2) 107 | } 108 | 109 | "work for colliding hashcodes, 3 element trie" in { 110 | nonEmptyCollideCheck(3) 111 | } 112 | 113 | "work for colliding hashcodes, 5 element trie" in { 114 | nonEmptyCollideCheck(5) 115 | } 116 | 117 | "work for colliding hashcodes, 10 element trie" in { 118 | nonEmptyCollideCheck(10) 119 | } 120 | 121 | "work for colliding hashcodes, 100 element trie" in { 122 | nonEmptyCollideCheck(100) 123 | } 124 | 125 | "work for colliding hashcodes, 500 element trie" in { 126 | nonEmptyCollideCheck(500) 127 | } 128 | 129 | "work for colliding hashcodes, 5k element trie" in { 130 | nonEmptyCollideCheck(5000) 131 | } 132 | 133 | def assertEqual(a: Map[Wrap, Int], b: Map[Wrap, Int]) { 134 | if (a != b) { 135 | println(a.size + " vs " + b.size) 136 | // println(a) 137 | // println(b) 138 | // println(a.toSeq.sortBy((x: (Wrap, Int)) => x._1.i)) 139 | // println(b.toSeq.sortBy((x: (Wrap, Int)) => x._1.i)) 140 | } 141 | assert(a == b) 142 | } 143 | 144 | "be consistent when taken with concurrent modifications" in { 145 | val sz = 25000 146 | val W = 25 147 | val S = 10 148 | val checks = 5 149 | val ct = new ConcurrentTrie[Wrap, Int] 150 | for (i <- 0 until sz) ct.put(new Wrap(i), i) 151 | 152 | class Modifier extends Thread { 153 | override def run() { 154 | for (i <- 0 until sz) ct.putIfAbsent(new Wrap(i), i) match { 155 | case Some(_) => ct.remove(new Wrap(i)) 156 | case None => 157 | } 158 | } 159 | } 160 | 161 | def consistentIteration(ct: ConcurrentTrie[Wrap, Int], checks: Int) { 162 | class Iter extends Thread { 163 | override def run() { 164 | val snap = ct.readOnlySnapshot() 165 | val initial = mutable.Map[Wrap, Int]() 166 | for (kv <- snap) initial += kv 167 | 168 | for (i <- 0 until checks) { 169 | assertEqual(snap.iterator.toMap, initial) 170 | } 171 | } 172 | } 173 | 174 | val iter = new Iter 175 | iter.start() 176 | iter.join() 177 | } 178 | 179 | val threads = for (_ <- 0 until W) yield new Modifier 180 | threads.foreach(_.start()) 181 | for (_ <- 0 until S) consistentIteration(ct, checks) 182 | threads.foreach(_.join()) 183 | } 184 | 185 | "be consistent with a concurrent removal with a well defined order" in { 186 | val sz = 450000 187 | val sgroupsize = 40 188 | val sgroupnum = 20 189 | val removerslowdown = 50 190 | val ct = new ConcurrentTrie[Wrap, Int] 191 | for (i <- 0 until sz) ct.put(new Wrap(i), i) 192 | 193 | class Remover extends Thread { 194 | override def run() { 195 | for (i <- 0 until sz) { 196 | assert(ct.remove(new Wrap(i)) == Some(i)) 197 | for (i <- 0 until removerslowdown) ct.get(new Wrap(i)) // slow down, mate 198 | } 199 | //println("done removing") 200 | } 201 | } 202 | 203 | def consistentIteration(it: Iterator[(Wrap, Int)]) = { 204 | class Iter extends Thread { 205 | override def run() { 206 | val elems = it.toSeq 207 | if (elems.nonEmpty) { 208 | val minelem = elems.minBy((x: (Wrap, Int)) => x._1.i)._1.i 209 | assert(elems.forall(_._1.i >= minelem)) 210 | } 211 | } 212 | } 213 | new Iter 214 | } 215 | 216 | val remover = new Remover 217 | remover.start() 218 | for (_ <- 0 until sgroupnum) { 219 | val iters = for (_ <- 0 until sgroupsize) yield consistentIteration(ct.iterator) 220 | iters.foreach(_.start()) 221 | iters.foreach(_.join()) 222 | } 223 | //println("done with iterators") 224 | remover.join() 225 | } 226 | 227 | "be consistent with a concurrent insertion with a well defined order" in { 228 | val sz = 450000 229 | val sgroupsize = 30 230 | val sgroupnum = 30 231 | val inserterslowdown = 50 232 | val ct = new ConcurrentTrie[Wrap, Int] 233 | 234 | class Inserter extends Thread { 235 | override def run() { 236 | for (i <- 0 until sz) { 237 | assert(ct.put(new Wrap(i), i) == None) 238 | for (i <- 0 until inserterslowdown) ct.get(new Wrap(i)) // slow down, mate 239 | } 240 | //println("done inserting") 241 | } 242 | } 243 | 244 | def consistentIteration(it: Iterator[(Wrap, Int)]) = { 245 | class Iter extends Thread { 246 | override def run() { 247 | val elems = it.toSeq 248 | if (elems.nonEmpty) { 249 | val maxelem = elems.maxBy((x: (Wrap, Int)) => x._1.i)._1.i 250 | assert(elems.forall(_._1.i <= maxelem)) 251 | } 252 | } 253 | } 254 | new Iter 255 | } 256 | 257 | val inserter = new Inserter 258 | inserter.start() 259 | for (_ <- 0 until sgroupnum) { 260 | val iters = for (_ <- 0 until sgroupsize) yield consistentIteration(ct.iterator) 261 | iters.foreach(_.start()) 262 | iters.foreach(_.join()) 263 | } 264 | //println("done with iterators") 265 | inserter.join() 266 | } 267 | 268 | "work on a yet unevaluated snapshot" in { 269 | val sz = 50000 270 | val ct = new ConcurrentTrie[Wrap, Int] 271 | for (i <- 0 until sz) ct.update(new Wrap(i), i) 272 | 273 | val snap = ct.snapshot() 274 | val it = snap.iterator 275 | 276 | while (it.hasNext) it.next() 277 | } 278 | 279 | } 280 | 281 | } 282 | -------------------------------------------------------------------------------- /src/test/scala/ctries2/lnodes.scala: -------------------------------------------------------------------------------- 1 | package ctries2 2 | 3 | 4 | 5 | import org.scalatest._ 6 | import org.scalatest.matchers.ShouldMatchers 7 | 8 | 9 | 10 | class LNodeSpec extends WordSpec with ShouldMatchers { 11 | 12 | val initsz = 1500 13 | val secondsz = 1750 14 | 15 | "A ctrie2" should { 16 | 17 | "accept elements with the same hash codes" in { 18 | val ct = new ConcurrentTrie[DumbHash, Int] 19 | for (i <- 0 until initsz) ct.update(new DumbHash(i), i) 20 | } 21 | 22 | "lookup elements with the same hash codes" in { 23 | val ct = new ConcurrentTrie[DumbHash, Int] 24 | for (i <- 0 until initsz) ct.update(new DumbHash(i), i) 25 | for (i <- 0 until initsz) assert(ct.get(new DumbHash(i)) == Some(i)) 26 | for (i <- initsz until secondsz) assert(ct.get(new DumbHash(i)) == None) 27 | } 28 | 29 | "remove elements with the same hash codes" in { 30 | val ct = new ConcurrentTrie[DumbHash, Int] 31 | for (i <- 0 until initsz) ct.update(new DumbHash(i), i) 32 | for (i <- 0 until initsz) assert(ct.remove(new DumbHash(i)) == Some(i)) 33 | for (i <- 0 until initsz) assert(ct.get(new DumbHash(i)) == None) 34 | } 35 | 36 | "put elements with the same hash codes if absent" in { 37 | val ct = new ConcurrentTrie[DumbHash, Int] 38 | for (i <- 0 until initsz) ct.put(new DumbHash(i), i) 39 | for (i <- 0 until initsz) assert(ct.lookup(new DumbHash(i)) == i) 40 | for (i <- 0 until initsz) assert(ct.putIfAbsent(new DumbHash(i), i) == Some(i)) 41 | for (i <- initsz until secondsz) assert(ct.putIfAbsent(new DumbHash(i), i) == None) 42 | for (i <- initsz until secondsz) assert(ct.lookup(new DumbHash(i)) == i) 43 | } 44 | 45 | "replace elements with the same hash codes" in { 46 | val ct = new ConcurrentTrie[DumbHash, Int] 47 | for (i <- 0 until initsz) assert(ct.put(new DumbHash(i), i) == None) 48 | for (i <- 0 until initsz) assert(ct.lookup(new DumbHash(i)) == i) 49 | for (i <- 0 until initsz) assert(ct.replace(new DumbHash(i), -i) == Some(i)) 50 | for (i <- 0 until initsz) assert(ct.lookup(new DumbHash(i)) == -i) 51 | for (i <- 0 until initsz) assert(ct.replace(new DumbHash(i), -i, i) == true) 52 | } 53 | 54 | "remove elements with the same hash codes if mapped to a specific value" in { 55 | val ct = new ConcurrentTrie[DumbHash, Int] 56 | for (i <- 0 until initsz) assert(ct.put(new DumbHash(i), i) == None) 57 | for (i <- 0 until initsz) assert(ct.remove(new DumbHash(i), i) == true) 58 | } 59 | 60 | } 61 | 62 | } 63 | -------------------------------------------------------------------------------- /src/test/scala/ctries2/snapshots.scala: -------------------------------------------------------------------------------- 1 | package ctries2 2 | 3 | 4 | 5 | 6 | import org.scalatest._ 7 | import org.scalatest.matchers.ShouldMatchers 8 | import collection._ 9 | import annotation.tailrec 10 | 11 | 12 | 13 | class SnapshotSpec extends WordSpec with ShouldMatchers { 14 | 15 | "A ctrie2" should { 16 | 17 | "support snapshots" in { 18 | val ctn = new ConcurrentTrie 19 | ctn.snapshot() 20 | ctn.readOnlySnapshot() 21 | 22 | val ct = new ConcurrentTrie[Int, Int] 23 | for (i <- 0 until 100) ct.put(i, i) 24 | ct.snapshot() 25 | ct.readOnlySnapshot() 26 | } 27 | 28 | "empty 2 quiescent snapshots in isolation" in { 29 | val sz = 4000 30 | 31 | class Worker(trie: ConcurrentTrie[Wrap, Int]) extends Thread { 32 | override def run() { 33 | for (i <- 0 until sz) { 34 | assert(trie.remove(new Wrap(i)) == Some(i)) 35 | for (j <- 0 until sz) 36 | if (j <= i) assert(trie.get(new Wrap(j)) == None) 37 | else assert(trie.get(new Wrap(j)) == Some(j)) 38 | } 39 | } 40 | } 41 | 42 | val ct = new ConcurrentTrie[Wrap, Int] 43 | for (i <- 0 until sz) ct.put(new Wrap(i), i) 44 | val snapt = ct.snapshot() 45 | 46 | val original = new Worker(ct) 47 | val snapshot = new Worker(snapt) 48 | original.start() 49 | snapshot.start() 50 | original.join() 51 | snapshot.join() 52 | 53 | for (i <- 0 until sz) { 54 | assert(ct.get(new Wrap(i)) == None) 55 | assert(snapt.get(new Wrap(i)) == None) 56 | } 57 | } 58 | 59 | def consistentReadOnly(name: String, readonly: Map[Wrap, Int], sz: Int, N: Int) { 60 | @volatile var e: Exception = null 61 | 62 | // reads possible entries once and stores them 63 | // then reads all these N more times to check if the 64 | // state stayed the same 65 | class Reader(trie: Map[Wrap, Int]) extends Thread { 66 | setName("Reader " + name) 67 | 68 | override def run() = 69 | try check() 70 | catch { 71 | case ex: Exception => e = ex 72 | } 73 | 74 | def check() { 75 | val initial = mutable.Map[Wrap, Int]() 76 | for (i <- 0 until sz) trie.get(new Wrap(i)) match { 77 | case Some(i) => initial.put(new Wrap(i), i) 78 | case None => // do nothing 79 | } 80 | 81 | for (k <- 0 until N) { 82 | for (i <- 0 until sz) { 83 | val tres = trie.get(new Wrap(i)) 84 | val ires = initial.get(new Wrap(i)) 85 | if (tres != ires) println(i, "initially: " + ires, "traversal %d: %s".format(k, tres)) 86 | assert(tres == ires) 87 | } 88 | } 89 | } 90 | } 91 | 92 | val reader = new Reader(readonly) 93 | reader.start() 94 | reader.join() 95 | 96 | if (e ne null) { 97 | e.printStackTrace() 98 | throw e 99 | } 100 | } 101 | 102 | // traverses the trie `rep` times and modifies each entry 103 | class Modifier(trie: ConcurrentTrie[Wrap, Int], index: Int, rep: Int, sz: Int) extends Thread { 104 | setName("Modifier %d".format(index)) 105 | 106 | override def run() { 107 | for (k <- 0 until rep) { 108 | for (i <- 0 until sz) trie.putIfAbsent(new Wrap(i), i) match { 109 | case Some(_) => trie.remove(new Wrap(i)) 110 | case None => // do nothing 111 | } 112 | } 113 | } 114 | } 115 | 116 | // removes all the elements from the trie 117 | class Remover(trie: ConcurrentTrie[Wrap, Int], index: Int, totremovers: Int, sz: Int) extends Thread { 118 | setName("Remover %d".format(index)) 119 | 120 | override def run() { 121 | for (i <- 0 until sz) trie.remove(new Wrap((i + sz / totremovers * index) % sz)) 122 | } 123 | } 124 | 125 | "have a consistent quiescent read-only snapshot" in { 126 | val sz = 10000 127 | val N = 100 128 | val W = 10 129 | 130 | val ct = new ConcurrentTrie[Wrap, Int] 131 | for (i <- 0 until sz) ct(new Wrap(i)) = i 132 | val readonly = ct.readOnlySnapshot() 133 | val threads = for (i <- 0 until W) yield new Modifier(ct, i, N, sz) 134 | 135 | threads.foreach(_.start()) 136 | consistentReadOnly("qm", readonly, sz, N) 137 | threads.foreach(_.join()) 138 | } 139 | 140 | // now, we check non-quiescent snapshots, as these permit situations 141 | // where a thread is caught in the middle of the update when a snapshot is taken 142 | 143 | "have a consistent non-quiescent read-only snapshot, concurrent with removes only" in { 144 | val sz = 1250 145 | val W = 100 146 | val S = 5000 147 | 148 | val ct = new ConcurrentTrie[Wrap, Int] 149 | for (i <- 0 until sz) ct(new Wrap(i)) = i 150 | val threads = for (i <- 0 until W) yield new Remover(ct, i, W, sz) 151 | 152 | threads.foreach(_.start()) 153 | for (i <- 0 until S) consistentReadOnly("non-qr", ct.readOnlySnapshot(), sz, 5) 154 | threads.foreach(_.join()) 155 | } 156 | 157 | "have a consistent non-quiescent read-only snapshot, concurrent with modifications" in { 158 | val sz = 1000 159 | val N = 7000 160 | val W = 10 161 | val S = 7000 162 | 163 | val ct = new ConcurrentTrie[Wrap, Int] 164 | for (i <- 0 until sz) ct(new Wrap(i)) = i 165 | val threads = for (i <- 0 until W) yield new Modifier(ct, i, N, sz) 166 | 167 | threads.foreach(_.start()) 168 | for (i <- 0 until S) consistentReadOnly("non-qm", ct.readOnlySnapshot(), sz, 5) 169 | threads.foreach(_.join()) 170 | } 171 | 172 | def consistentNonReadOnly(name: String, trie: ConcurrentTrie[Wrap, Int], sz: Int, N: Int) { 173 | @volatile var e: Exception = null 174 | 175 | // reads possible entries once and stores them 176 | // then reads all these N more times to check if the 177 | // state stayed the same 178 | class Worker extends Thread { 179 | setName("Worker " + name) 180 | 181 | override def run() = 182 | try check() 183 | catch { 184 | case ex: Exception => e = ex 185 | } 186 | 187 | def check() { 188 | val initial = mutable.Map[Wrap, Int]() 189 | for (i <- 0 until sz) trie.get(new Wrap(i)) match { 190 | case Some(i) => initial.put(new Wrap(i), i) 191 | case None => // do nothing 192 | } 193 | 194 | for (k <- 0 until N) { 195 | // modify 196 | for ((key, value) <- initial) { 197 | val oldv = if (k % 2 == 0) value else -value 198 | val newv = -oldv 199 | trie.replace(key, oldv, newv) 200 | } 201 | 202 | // check 203 | for (i <- 0 until sz) if (initial.contains(new Wrap(i))) { 204 | val expected = if (k % 2 == 0) -i else i 205 | //println(trie.get(new Wrap(i))) 206 | assert(trie.get(new Wrap(i)) == Some(expected)) 207 | } else { 208 | assert(trie.get(new Wrap(i)) == None) 209 | } 210 | } 211 | } 212 | } 213 | 214 | val worker = new Worker 215 | worker.start() 216 | worker.join() 217 | 218 | if (e ne null) { 219 | e.printStackTrace() 220 | throw e 221 | } 222 | } 223 | 224 | "have a consistent non-quiescent snapshot, concurrent with modifications" in { 225 | val sz = 9000 226 | val N = 1000 227 | val W = 10 228 | val S = 400 229 | 230 | val ct = new ConcurrentTrie[Wrap, Int] 231 | for (i <- 0 until sz) ct(new Wrap(i)) = i 232 | val threads = for (i <- 0 until W) yield new Modifier(ct, i, N, sz) 233 | 234 | threads.foreach(_.start()) 235 | for (i <- 0 until S) { 236 | consistentReadOnly("non-qm", ct.snapshot(), sz, 5) 237 | consistentNonReadOnly("non-qsnap", ct.snapshot(), sz, 5) 238 | } 239 | threads.foreach(_.join()) 240 | } 241 | 242 | "work when many concurrent snapshots are taken, concurrent with modifications" in { 243 | val sz = 12000 244 | val W = 10 245 | val S = 10 246 | val modifytimes = 1200 247 | val snaptimes = 600 248 | val ct = new ConcurrentTrie[Wrap, Int] 249 | for (i <- 0 until sz) ct(new Wrap(i)) = i 250 | 251 | class Snapshooter extends Thread { 252 | setName("Snapshooter") 253 | override def run() { 254 | for (k <- 0 until snaptimes) { 255 | val snap = ct.snapshot() 256 | for (i <- 0 until sz) snap.remove(new Wrap(i)) 257 | for (i <- 0 until sz) assert(!snap.contains(new Wrap(i))) 258 | } 259 | } 260 | } 261 | 262 | val mods = for (i <- 0 until W) yield new Modifier(ct, i, modifytimes, sz) 263 | val shooters = for (i <- 0 until S) yield new Snapshooter 264 | val threads = mods ++ shooters 265 | threads.foreach(_.start()) 266 | threads.foreach(_.join()) 267 | } 268 | 269 | } 270 | 271 | } 272 | 273 | 274 | 275 | 276 | 277 | 278 | 279 | 280 | 281 | 282 | 283 | -------------------------------------------------------------------------------- /tools/bench-all: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | if [ "$1" == "-h" ]; then 4 | echo "Runs all benchmarks. Synthax: bench-all " 5 | else 6 | tools/bench-insert $@ 7 | tools/bench-lookup $@ 8 | tools/bench-remove $@ 9 | tools/bench-update $@ 10 | tools/bench-snapshot $@ 11 | tools/bench-iter $@ 12 | tools/bench-pagerank $@ 13 | fi 14 | -------------------------------------------------------------------------------- /tools/bench-insert: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | TASK="bench-batch --gnuplot=$1-insert -Dsz=$3 -Dpar=$2 ctries.MultiInsertCtrie2 ctries.MultiInsertSkipList ctries.MultiInsertCHM ctries.MultiInsertCliff" 3 | sbt "$TASK" 4 | -------------------------------------------------------------------------------- /tools/bench-iter: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | TASK="bench-batch --gnuplot=$1-iter -Dsz=$3 -Dpar=$2 ctries.IterationCtrie2 ctries.IterationSkipList ctries.IterationCHM ctries.IterationHashTrie ctries.IterationHashTable" 3 | sbt "$TASK" -------------------------------------------------------------------------------- /tools/bench-lookup: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | TASK="bench-batch --gnuplot=$1-lookup -Dsz=$3 -Dpar=$2 ctries.MultiLookupCtrie2 ctries.MultiLookupSkipList ctries.MultiLookupCHM ctries.MultiLookupCliff" 3 | sbt "$TASK" 4 | -------------------------------------------------------------------------------- /tools/bench-pagerank: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | TASK="bench-batch --gnuplot=$1-pagerank -Dsz=${11} -Dpar=$2 -Dpagegenerator=$8 -Dmaxlinks=$9 -Ddamping=${10} -Ddebug=false ctries.ParPageRank ctries.FilterPageRank" 3 | sbt "$TASK" 4 | -------------------------------------------------------------------------------- /tools/bench-remove: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | TASK="bench-batch --gnuplot=$1-remove -Dsz=$3 -Dpar=$2 ctries.MultiRemoveCtrie2 ctries.MultiRemoveSkipList ctries.MultiRemoveCHM ctries.MultiRemoveCliff" 3 | sbt "$TASK" 4 | -------------------------------------------------------------------------------- /tools/bench-snapshot: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | TASK="bench-batch --gnuplot=$1-snapshot -Dsz=$3 -Dpar=$2 ctries.MultiRemovingCtrie2 ctries.MultiRemovingCtrie2Snapshot" 3 | sbt "$TASK" -------------------------------------------------------------------------------- /tools/bench-snapshot-lookup: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | TASK="bench-batch --gnuplot=$1-snapshot-lookup -Dsz=$3 -Dpar=$2 ctries.MultiLookingCtrie2 ctries.MultiLookingCtrie2Snapshot" 3 | sbt "$TASK" 4 | -------------------------------------------------------------------------------- /tools/bench-update: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | TASK="bench-batch --gnuplot=$1-update -Dsz=$3 -Dpar=$2 -Dlookups=$4 -Dinserts=$5 -Dremoves=$6 -Dtotalops=$7 ctries.MultiUpdateCtrie2 ctries.MultiUpdateSkipList ctries.MultiUpdateCHM ctries.MultiUpdateCliff" 3 | sbt "$TASK" 4 | -------------------------------------------------------------------------------- /tools/toplot: -------------------------------------------------------------------------------- 1 | #!/usr/bin/perl 2 | 3 | # 4 | # Reads lines with numbers from the first file, computes the average and writes it to the second file. 5 | # 6 | 7 | 8 | open(HANDLE, $ARGV[0]); 9 | open(PLOT,">>$ARGV[1]") || die("Cannot Open File"); 10 | 11 | #printf("filename: $ARGV[0]\n"); 12 | 13 | while ($line = ) { 14 | # printf("line: $line\n"); 15 | $sum = 0; 16 | $num = 0; 17 | @rawdata = split(' ', $line); 18 | # printf("data: @rawdata\n"); 19 | foreach (@rawdata) { 20 | if ($num > 3) { 21 | $sum = $sum + $_; 22 | } 23 | $num = $num + 1; 24 | } 25 | # printf("$num, $sum\n"); 26 | $avg = 1.0 * $sum / ($num - 4); 27 | 28 | print PLOT " $avg "; 29 | } 30 | print PLOT "\n"; 31 | 32 | close(HANDLE); 33 | close(PLOT); 34 | 35 | 36 | 37 | -------------------------------------------------------------------------------- /tools/totikz: -------------------------------------------------------------------------------- 1 | #!/usr/bin/perl 2 | 3 | if ( @ARGV != 6 ) { 4 | print "Usage: totikz \n"; 5 | print "where: \n"; 6 | print " source - file with benchmark results \n"; 7 | print " destin - file to store the tikz graph into \n"; 8 | print " x-column - the index of the x-column, used for x-axis \n"; 9 | print " z-column - the index of the z-column, used for multiple graphs \n"; 10 | print " y-column - the index of the first y-column, used for curves in the graphs \n"; 11 | print " process - how to process the entry, one of: none, throughput \n"; 12 | exit; 13 | } 14 | 15 | 16 | open(SOURCE, $ARGV[0]); 17 | open(PLOT, ">$ARGV[1]") || die("Cannot Open File"); 18 | $x = $ARGV[2]; 19 | $z = $ARGV[3]; 20 | $y = $ARGV[4]; 21 | $proc = $ARGV[5]; 22 | %plots = (); 23 | $gcount = 0; 24 | 25 | 26 | sub process { 27 | print "$proc \n"; 28 | if ($proc eq "none") { return $_[0]; } 29 | elsif ($proc eq "throughput") { return 1000 / $_[0]; } 30 | else { return $_[0]; } 31 | } 32 | 33 | 34 | while ($line = ) { 35 | #print "$line"; 36 | @cells = split(/ | \n| /, $line); 37 | $len = @cells; 38 | #print "$len \n"; 39 | $zcol = int(@cells[$z]); 40 | $plots{$zcol} = () if !exists $plots{$zcol}; 41 | for ($n = $y; $n < $len; $n++) { 42 | $plots{$zcol}{$n} = () if !exists $plots{$zcol}{$n}; 43 | $plots{$zcol}{$n}{@cells[$x]} = process(@cells[$n]); 44 | print "$n: process(@cells[$n]) \n"; 45 | } 46 | } 47 | 48 | 49 | print PLOT "\\begin{figure}[h] \n"; 50 | print PLOT "\\begin{center} \n"; 51 | print PLOT "\\renewcommand{\\tabcolsep}{0cm} \n"; 52 | print PLOT "\\begin{tabular}{ c c c } \n"; 53 | print PLOT "\n"; 54 | foreach $graph (sort {$a <=> $b} (keys %plots)) { 55 | print "$graph: $plots{$graph} \n"; 56 | print PLOT "\\begin{tikzpicture}[scale=0.67] \n"; 57 | print PLOT "\\begin{axis}[height=5cm,width=5cm] \n"; 58 | foreach $plot (sort(keys %{$plots{$graph}})) { 59 | print " $plot: $plots{$graph}{$plot} \n"; 60 | $mnum = $plot - $y; 61 | if ($mnum == 0) { $mark = ""; } 62 | if ($mnum == 1) { $mark = ",mark=o"; } 63 | if ($mnum == 2) { $mark = ",mark=x"; } 64 | if ($mnum == 3) { $mark = ",mark=*"; } 65 | print "$mnum \n"; 66 | print PLOT "\\addplot[smooth$mark] \n"; 67 | print PLOT "plot coordinates { \n"; 68 | foreach $ntr (sort {$a <=> $b} (keys %{$plots{$graph}{$plot}})) { 69 | print " $ntr: $plots{$graph}{$plot}{$ntr} \n"; 70 | print PLOT "($ntr, $plots{$graph}{$plot}{$ntr}) \n"; 71 | } 72 | print PLOT "}; \n"; 73 | } 74 | print PLOT "\\end{axis} \n"; 75 | print PLOT "\\end{tikzpicture} \n"; 76 | print PLOT "\n"; 77 | $gcount++; 78 | if ($gcount % 3 != 0) { print PLOT "& \n"; } 79 | else { print PLOT "\\\\ \n"; } 80 | print PLOT "\n"; 81 | } 82 | print PLOT "\n"; 83 | print PLOT "\\end{tabular} \n"; 84 | print PLOT "\\end{center} \n"; 85 | print PLOT "\n"; 86 | print PLOT "\\caption{Benchmark} \n"; 87 | print PLOT "\\label{f-bench} \n"; 88 | print PLOT "\\end{figure} \n"; 89 | 90 | 91 | close(SOURCE); 92 | close(PLOT); 93 | --------------------------------------------------------------------------------