2 | halide-haskell 3 |
4 | 5 |
6 |
7 | 
This project is still under heavy development and might not be production-ready
8 | We encourage experimenting with it and reporting any issues you run into via 9 | [Github issues](https://github.com/twesterhout/halide-haskell/issues). 10 | 11 |
12 | 13 | [**Tutorials**](https://github.com/twesterhout/halide-haskell/tree/master/tutorials) | [**Documentation**](https://hackage.haskell.org/package/halide-haskell-0.0.1.0) | [**Showcases**](https://github.com/twesterhout/halide-haskell-examples) 14 | 15 | [](LICENSE) 16 | [](https://github.com/twesterhout/halide-haskell/actions/workflows/ci.yml) 17 | [](https://hackage.haskell.org/package/halide-haskell) 18 | 19 |
20 |
21 | This project is still under heavy development and might not be production-ready8 | We encourage experimenting with it and reporting any issues you run into via 9 | [Github issues](https://github.com/twesterhout/halide-haskell/issues). 10 | 11 |
12 | 13 | [**Tutorials**](https://github.com/twesterhout/halide-haskell/tree/master/tutorials) | [**Documentation**](https://hackage.haskell.org/package/halide-haskell-0.0.1.0) | [**Showcases**](https://github.com/twesterhout/halide-haskell-examples) 14 | 15 | [](LICENSE) 16 | [](https://github.com/twesterhout/halide-haskell/actions/workflows/ci.yml) 17 | [](https://hackage.haskell.org/package/halide-haskell) 18 | 19 |
| 24 | 25 | [Halide](https://halide-lang.org/) is a programming language designed to make 26 | it easier to write high-performance image and array processing code on modern 27 | machines. Rather than being a standalone programming language, Halide is 28 | embedded in C++. This means you write C++ code that builds an in-memory 29 | representation of a Halide pipeline using Halide's C++ API. You can then 30 | compile this representation to an object file, or JIT-compile it and run it in 31 | the same process. 32 | 33 | | 34 |
38 | This package provides Haskell bindings that allow to write Halide embedded in 39 | Haskell without C++ 😋. 40 |
41 | 42 | - [Example usage](#-example-usage) 43 | - [Installing](#-installing) 44 | - [Motivation](#-motivation) 45 | - [Contributing](#-contributing) 46 | 47 | 48 | ## 🚀 Example usage 49 | 50 | As a simple example, here's how you could implement array addition with halide-haskell: 51 | 52 | ```haskell 53 | {-# LANGUAGE AllowAmbiguousTypes, DataKinds, OverloadedStrings, ViewPatterns #-} 54 | import Language.Halide 55 | 56 | -- The algorithm 57 | mkArrayPlus = compile $ \(buffer "a" -> a) (buffer "b" -> b) -> do 58 | -- Create an index variable 59 | i <- mkVar "i" 60 | -- Define the resulting function. We call it "out". 61 | -- In pseudocode it's equivalent to the following: out[i] = a[i] + b[i] 62 | out <- define "out" i $ a ! i + b ! i 63 | -- Perform a fancy optimization and use SIMD: we split the loop over i into 64 | -- an inner and an outer loop and then vectorize the inner loop 65 | inner <- mkVar "inner" 66 | split TailAuto i (i, inner) 4 out >>= vectorize inner 67 | 68 | -- Example usage of our Halide pipeline 69 | main :: IO () 70 | main = do 71 | let a, b :: [Float] 72 | a = [1, 2, 3, 4, 5] 73 | b = [6, 7, 8, 9, 10] 74 | -- Compile the code 75 | arrayPlus <- mkArrayPlus 76 | -- We tell Halide to treat our list as a one-dimensional buffer 77 | withHalideBuffer @1 @Float a $ \a' -> 78 | withHalideBuffer b $ \b' -> 79 | -- allocate a temporary buffer for the output 80 | allocaCpuBuffer [length a] $ \out' -> do 81 | -- execute the kernel -- it is a normal function call! 82 | arrayPlus a' b' out' 83 | -- print the result 84 | print =<< peekToList out' 85 | ``` 86 | 87 | For more examples, have a look at the [tutorials](https://github.com/twesterhout/halide-haskell/tree/master/tutorials). 88 | 89 | ## 💻 Installing 90 | 91 | The library is avaiable on Hackage, so you can just add `halide-haskell` to the 92 | `build-depends` field in your Cabal file. However, halide-haskell depends on Halide, 93 | so you need to make sure it's available. 94 | 95 | ### ❄️ Nix 96 | 97 | No need to do anything 😃, Nix will make Halide avaiable automatically. 98 | 99 | If you just want to hack on the library, after `git clone`ing, type 100 | 101 | ```sh 102 | nix build 103 | ``` 104 | 105 | and to run an example, try 106 | 107 | ```sh 108 | nix run 109 | nix run .#ghc928-intel-ocl.halide-haskell # for Intel OpenCL support 110 | nix run .#ghc928-cuda.halide-haskell # for CUDA support 111 | nix run .#ghc945.halide-haskell # to build with GHC 9.4.5 instead 112 | nix run .#ghc945.halide-tutorial01 # to run the first tutorial 113 | ``` 114 | 115 | ### non-Nix 116 | 117 | This set up is not tested in CI... but 118 | 119 | - Download a pre-built Halide for your system from [their releases page](https://github.com/halide/Halide/releases/). 120 | - Pass `--extra-include-dirs` and `--extra-lib-dirs` options to Cabal on the command line or add them to your `cabal.project.local` file. 121 | 122 | ## 🤩 Motivation 123 | 124 | The availability of Deep Learning frameworks such as 125 | [PyTorch](https://pytorch.org/) or [JAX](https://github.com/google/jax) has 126 | revolutionized array processing, independently of whether one works on Machine 127 | Learning tasks or other numerical algorithms. The ecosystem in Haskell has been 128 | catching up as well, and there are now multiple good array 129 | libraries ([hmatrix](https://github.com/haskell-numerics/hmatrix), 130 | [massiv](https://github.com/lehins/massiv), 131 | [Accelerate](https://www.acceleratehs.org/), 132 | [arrayfire-haskell](https://github.com/arrayfire/arrayfire-haskell), 133 | [Hasktorch](https://github.com/hasktorch/hasktorch), are all high-quality 134 | libraries). To accommodate multiple domains, such libraries 135 | have to support hundreds if not thousands of operations (e.g. there are more 136 | than 3.5 thousand of so called [“native” functions in PyTorch](https://github.com/pytorch/pytorch/blob/6a09847c42bf7d33ba0aea5b083eebd846661ce1/aten/src/ATen/native/native_functions.yaml)), 137 | and this count does not include specializations for different device and data 138 | types). 139 | 140 | To overcome this difficulty, we propose to build a common extension mechanism 141 | for Haskell array libraries. The mechanism is based on embedding the 142 | [Halide](https://halide-lang.org/) language into Haskell that allows to 143 | just-in-time (JIT) compile computational kernels for various hardware. 144 | 145 | ### 🤨 Why not Accelerate? 146 | 147 | One might wonder, why write another package instead of relying on 148 | [Accelerate](https://www.acceleratehs.org/) for the JIT compilation of the 149 | kernels. Accelerate is a Haskell eDSL (embedded Domain Specific Language) for 150 | collective operations on dense multi-dimensional arrays. It relies on 151 | [LLVM](https://llvm.org/) to JIT compile the computational kernels for the 152 | target architecture including multicore CPUs and GPUs. Users have to rely on 153 | Accelerate to generate high-performance kernels and have no way to force some 154 | low-level optimizations. For example, [Trevor L. McDonell et 155 | al.](https://doi.org/10.1145/2887747.2804313) explain that the reason why 156 | hand-written [CUDA](https://www.nvidia.com/en-gb/geforce/technologies/cuda/) 157 | implementation of the [N-body 158 | problem](https://en.wikipedia.org/wiki/N-body_problem) outperforms Accelerate 159 | is the use of on-chip shared memory. Another example would be the matrix-matrix 160 | product where achieving maximal performance requires writing no fewer than six 161 | nested loops instead of the naive three ([ACM Trans. Math. Softw. 34, 3, 162 | Article 12 (May 2008)](https://doi.org/10.1145/1356052.1356053)). 163 | Accelerate has no way of knowing that such optimizations have to be applied and 164 | cannot perform them automatically, and this is precisely the gap that we are 165 | trying to fill by embedding Halide into Haskell. 166 | 167 | Halide is a C++ eDSL for high-performance image and array processing. Its core 168 | idea is to decouple the *algorithm* (i.e. what is computed) from the *schedule* 169 | (i.e. where and when it is computed). The eDSL allows to quickly prototype and 170 | test the algorithm and then move on to the optimization. Optimizations such as 171 | fusion, tiling, parallelism and vectorization can be freely explored without 172 | the risk of breaking the original algorithm definition. Schedulers can also be 173 | generated automatically by [advanced optimization 174 | algorithms](https://halide-lang.org/papers/autoscheduler2019.html) 175 | 176 | Halide provides a lower level interface than Accelerate and thus does not aim 177 | to replace it. Instead, Halide can be used to extend Accelerate, and later on 178 | one might even think about using Halide as a backend for Accelerate. 179 | 180 | ## 🔨 Contributing 181 | 182 | Currently, the best way to get started is to use Nix: 183 | 184 | ```sh 185 | nix develop 186 | ``` 187 | 188 | This will drop you into a shell with all the necessary tools to build the code 189 | such that you can do 190 | 191 | ```sh 192 | cabal build 193 | ``` 194 | 195 | and 196 | 197 | ```sh 198 | cabal test 199 | ``` 200 | -------------------------------------------------------------------------------- /test-readme/halide-readme.cabal: -------------------------------------------------------------------------------- 1 | cabal-version: 3.0 2 | name: halide-readme 3 | version: 0.0.1.0 4 | license: BSD-3-Clause 5 | author: twesterhout 6 | maintainer: 14264576+twesterhout@users.noreply.github.com 7 | build-type: Simple 8 | 9 | common setup 10 | ghc-options: -W -pgmL markdown-unlit 11 | default-language: GHC2021 12 | build-depends: base >=4.16.0.0 && <5 13 | , vector 14 | , halide-haskell 15 | 16 | executable halide-readme 17 | import: setup 18 | main-is: README.lhs 19 | hs-source-dirs: . 20 | 21 | test-suite halide-readme-test 22 | import: setup 23 | main-is: README.lhs 24 | type: exitcode-stdio-1.0 25 | -------------------------------------------------------------------------------- /test/Language/Halide/BoundaryConditionsSpec.hs: -------------------------------------------------------------------------------- 1 | module Language.Halide.BoundaryConditionsSpec (spec) where 2 | 3 | import Language.Halide 4 | import Test.Hspec 5 | 6 | spec :: Spec 7 | spec = do 8 | it "extends the domain by repeating the edges" $ do 9 | let mat :: [[Float]] 10 | mat = [[1, 2, 3], [4, 5, 6]] 11 | asBufferParam @2 @Float mat $ \mat' -> do 12 | [i, j] <- mapM mkVar ["i", "j"] 13 | extended <- repeatEdge mat' 14 | f <- define "f" (i, j) $ extended ! (i - 1, j - 1) 15 | realize f [4, 5] peekToList 16 | `shouldReturn` [[1, 1, 2, 3, 3], [1, 1, 2, 3, 3], [4, 4, 5, 6, 6], [4, 4, 5, 6, 6]] 17 | it "extends the domain by filling with a constant value" $ do 18 | let mat :: [[Float]] 19 | mat = [[1, 2, 3], [4, 5, 6]] 20 | asBufferParam @2 @Float mat $ \mat' -> do 21 | [i, j] <- mapM mkVar ["i", "j"] 22 | extended <- constantExterior 0 mat' 23 | f <- define "f" (i, j) $ extended ! (i - 1, j - 1) 24 | realize f [4, 5] peekToList 25 | `shouldReturn` [[0, 0, 0, 0, 0], [0, 1, 2, 3, 0], [0, 4, 5, 6, 0], [0, 0, 0, 0, 0]] 26 | -------------------------------------------------------------------------------- /test/Language/Halide/BufferSpec.hs: -------------------------------------------------------------------------------- 1 | {-# LANGUAGE OverloadedRecordDot #-} 2 | 3 | module Language.Halide.BufferSpec (spec) where 4 | 5 | import Data.Int (Int64) 6 | import Foreign.Ptr (nullPtr) 7 | import Language.Halide 8 | import Test.Hspec 9 | import Test.Hspec.QuickCheck 10 | import Test.QuickCheck 11 | 12 | newtype ListVector a = ListVector [a] 13 | deriving stock (Show) 14 | 15 | newtype ListMatrix a = ListMatrix [[a]] 16 | deriving stock (Show) 17 | 18 | newtype ListTensor3D a = ListTensor3D [[[a]]] 19 | deriving stock (Show) 20 | 21 | newtype ListTensor4D a = ListTensor4D [[[[a]]]] 22 | deriving stock (Show) 23 | 24 | instance Arbitrary a => Arbitrary (ListVector a) where 25 | arbitrary = ListVector <$> listOf arbitrary 26 | 27 | instance Arbitrary a => Arbitrary (ListMatrix a) where 28 | arbitrary = do 29 | d0 <- chooseInt (0, 50) 30 | d1 <- chooseInt (0, 50) 31 | ListMatrix <$> vectorOf d0 (vector d1) 32 | 33 | instance Arbitrary a => Arbitrary (ListTensor3D a) where 34 | arbitrary = do 35 | d0 <- chooseInt (0, 30) 36 | d1 <- chooseInt (0, 30) 37 | d2 <- chooseInt (0, 30) 38 | ListTensor3D <$> vectorOf d0 (vectorOf d1 (vector d2)) 39 | 40 | instance Arbitrary a => Arbitrary (ListTensor4D a) where 41 | arbitrary = do 42 | d0 <- chooseInt (0, 30) 43 | d1 <- chooseInt (0, 30) 44 | d2 <- chooseInt (0, 30) 45 | d3 <- chooseInt (0, 30) 46 | ListTensor4D <$> vectorOf d0 (vectorOf d1 (vectorOf d2 (vector d3))) 47 | 48 | spec :: Spec 49 | spec = do 50 | it "rowMajorStrides" $ do 51 | rowMajorStrides [1, 1, 1] `shouldBe` ([1, 1, 1] :: [Int]) 52 | rowMajorStrides [2, 1, 3] `shouldBe` ([3, 3, 1] :: [Int]) 53 | rowMajorStrides [3, 2] `shouldBe` ([2, 1] :: [Int]) 54 | rowMajorStrides [] `shouldBe` ([] :: [Int]) 55 | it "bufferFromPtrShapeStrides" $ do 56 | bufferFromPtrShapeStrides nullPtr [3, 2, 1] [1, 1, 1] (\(_ :: Ptr (HalideBuffer 2 Int32)) -> pure ()) 57 | `shouldThrow` anyErrorCall 58 | bufferFromPtrShapeStrides nullPtr [3] [1] (\(_ :: Ptr (HalideBuffer 2 Int32)) -> pure ()) 59 | `shouldThrow` anyErrorCall 60 | prop "works with [a]" $ \(ListVector xs :: ListVector Float) -> 61 | withHalideBuffer @1 @Float xs peekToList `shouldReturn` xs 62 | prop "works with [[a]]" $ \(ListMatrix xs :: ListMatrix Int64) -> 63 | withHalideBuffer @2 @Int64 xs peekToList `shouldReturn` xs 64 | prop "works with [[[a]]]" $ \(ListTensor3D xs :: ListTensor3D Double) -> 65 | withHalideBuffer @3 @Double xs peekToList `shouldReturn` xs 66 | modifyMaxSuccess (const 20) $ 67 | prop "works with [[[[a]]]]" $ \(ListTensor4D @Double xs) -> 68 | withHalideBuffer @4 @Double xs peekToList `shouldReturn` xs 69 | it "creates cropped buffers" $ do 70 | let mkFill2D (scalar @Float "value" -> value) = do 71 | [i, j] <- mapM mkVar ["i", "j"] 72 | define "fill" (i, j) value 73 | fill <- compile mkFill2D 74 | allocaCpuBuffer [4, 4] $ \buf -> do 75 | fill 0 buf 76 | peekToList buf `shouldReturn` [[0, 0, 0, 0], 77 | [0, 0, 0, 0], 78 | [0, 0, 0, 0], 79 | [0, 0, 0, 0]] 80 | withCropped buf 1 0 1 $ fill 1 81 | peekToList buf `shouldReturn` [[1, 0, 0, 0], 82 | [1, 0, 0, 0], 83 | [1, 0, 0, 0], 84 | [1, 0, 0, 0]] 85 | withCropped buf 0 1 2 $ fill 2 86 | peekToList buf `shouldReturn` [[1, 0, 0, 0], 87 | [2, 2, 2, 2], 88 | [2, 2, 2, 2], 89 | [1, 0, 0, 0]] 90 | -------------------------------------------------------------------------------- /test/Language/Halide/ExprSpec.hs: -------------------------------------------------------------------------------- 1 | module Language.Halide.ExprSpec (spec) where 2 | 3 | import Control.Monad (unless, when) 4 | import Data.Int 5 | import Data.Text (Text) 6 | import Data.Word 7 | import Language.Halide 8 | import Test.Hspec 9 | import Test.Hspec.QuickCheck 10 | import Test.QuickCheck 11 | import Type.Reflection 12 | import Utils 13 | 14 | isOverflowing :: Typeable a => (Integer -> Integer -> Integer) -> a -> a -> Bool 15 | isOverflowing op x y 16 | | Just HRefl <- eqTypeRep (typeOf x) (typeRep @Int32) = 17 | op (toInteger x) (toInteger y) > toInteger (maxBound @Int32) 18 | || op (toInteger x) (toInteger y) < toInteger (minBound @Int32) 19 | | Just HRefl <- eqTypeRep (typeOf x) (typeRep @Int64) = 20 | op (toInteger x) (toInteger y) > toInteger (maxBound @Int64) 21 | || op (toInteger x) (toInteger y) < toInteger (minBound @Int64) 22 | | otherwise = False 23 | 24 | -- infix 1 `evaluatesTo` 25 | 26 | -- evaluatesTo :: (Eq a, IsHalideType a) => Expr a -> a -> PropertyM IO () 27 | -- evaluatesTo expr expected = 28 | -- assert . (expected ==) =<< (run . evaluate) expr 29 | -- 30 | 31 | infix 1 `shouldEvaluateToApprox` 32 | 33 | shouldEvaluateToApprox :: (Ord a, IsHalideType a, HasEpsilon a, Show a) => Expr a -> a -> Expectation 34 | shouldEvaluateToApprox expr expected = do 35 | e <- evaluate expr 36 | e `shouldBeApprox` expected 37 | 38 | infix 1 `shouldEvaluateTo` 39 | 40 | shouldEvaluateTo :: (Eq a, IsHalideType a, Show a) => Expr a -> a -> Expectation 41 | shouldEvaluateTo expr expected = evaluate expr `shouldReturn` expected 42 | 43 | spec :: Spec 44 | spec = do 45 | describe "mkExpr" $ modifyMaxSuccess (const 10) $ do 46 | prop "Bool" $ \x -> 47 | mkExpr (x :: Bool) `shouldEvaluateTo` x 48 | 49 | describe "Num Expr" $ modifyMaxSuccess (const 10) $ do 50 | let whenNotOverflowing op x y check 51 | | isOverflowing op x y = pure () 52 | | otherwise = check 53 | p :: forall a. (IsHalideType a, Eq a, Num a, Typeable a, Show a) => a -> a -> Expectation 54 | p x y = do 55 | whenNotOverflowing (+) x y $ 56 | mkExpr x + mkExpr y `shouldEvaluateTo` x + y 57 | whenNotOverflowing (-) x y $ 58 | mkExpr x - mkExpr y `shouldEvaluateTo` x - y 59 | whenNotOverflowing (*) x y $ 60 | mkExpr x * mkExpr y `shouldEvaluateTo` x * y 61 | unless (x == -128) $ 62 | abs (mkExpr x) `shouldEvaluateTo` abs x 63 | negate (mkExpr x) `shouldEvaluateTo` negate x 64 | prop "Int8" $ p @Int8 65 | prop "Int16" $ p @Int16 66 | prop "Int32" $ p @Int32 67 | prop "Int64" $ p @Int64 68 | prop "Word8" $ p @Word8 69 | prop "Word16" $ p @Word16 70 | prop "Word32" $ p @Word32 71 | prop "Word64" $ p @Word64 72 | prop "Float" $ p @Float 73 | prop "Double" $ p @Double 74 | 75 | describe "Fractional Expr" $ modifyMaxSuccess (const 10) $ do 76 | let p :: forall a. (IsHalideType a, Eq a, Fractional a, Show a) => a -> a -> Expectation 77 | p x y = do 78 | unless (x == 0 && y == 0) $ 79 | mkExpr x / mkExpr y `shouldEvaluateTo` x / y 80 | prop "Float" $ property (p @Float) 81 | prop "Double" $ property (p @Double) 82 | 83 | describe "Floating Expr" $ modifyMaxSuccess (const 10) $ do 84 | let p :: forall a. (IsHalideType a, Ord a, Floating a, HasEpsilon a, Show a) => a -> Expectation 85 | p x = do 86 | let y = mkExpr x 87 | when (x > 0) $ do 88 | log y `shouldEvaluateToApprox` log x 89 | sqrt y `shouldEvaluateToApprox` sqrt x 90 | when (x < 50) $ 91 | exp y `shouldEvaluateToApprox` exp x 92 | sin y `shouldEvaluateToApprox` sin x 93 | cos y `shouldEvaluateToApprox` cos x 94 | tan y `shouldEvaluateToApprox` tan x 95 | when (-1 <= x && x <= 1) $ do 96 | asin y `shouldEvaluateToApprox` asin x 97 | acos y `shouldEvaluateToApprox` acos x 98 | atan y `shouldEvaluateToApprox` atan x 99 | when (abs x < 50) $ do 100 | sinh y `shouldEvaluateToApprox` sinh x 101 | cosh y `shouldEvaluateToApprox` cosh x 102 | tanh y `shouldEvaluateToApprox` tanh x 103 | asinh y `shouldEvaluateToApprox` asinh x 104 | when (x >= 1) $ 105 | acosh y `shouldEvaluateToApprox` acosh x 106 | when (-1 <= x && x <= 1) $ 107 | atanh y `shouldEvaluateToApprox` atanh x 108 | prop "Float" $ p @Float 109 | prop "Double" $ p @Double 110 | it "defines pi" $ do 111 | (pi :: Expr Float) `shouldEvaluateToApprox` pi 112 | (pi :: Expr Double) `shouldEvaluateToApprox` pi 113 | 114 | describe "printed" $ 115 | it "prints expressions when evaluated" $ do 116 | printed (1 :: Expr Int32) `shouldEvaluateTo` 1 117 | printed (1 :: Expr Int32) ("<- when" :: String) ("haha" :: String) `shouldEvaluateTo` 1 118 | let x :: Expr Float 119 | x = 1 120 | in printed (sin x) ("<- sin(" :: Text) x (")" :: Text) `shouldEvaluateToApprox` sin 1 121 | 122 | -- describe "Show" $ 123 | -- it "shows 123" $ do 124 | -- show (123 :: Expr Int32) `shouldBe` "123" 125 | describe "testWriteToStderr" $ do 126 | it "shows 123" $ do 127 | testWriteToStderr 128 | -------------------------------------------------------------------------------- /test/Language/Halide/FuncSpec.hs: -------------------------------------------------------------------------------- 1 | {-# LANGUAGE OverloadedLists #-} 2 | {-# LANGUAGE OverloadedRecordDot #-} 3 | {-# LANGUAGE QuasiQuotes #-} 4 | {-# LANGUAGE TemplateHaskell #-} 5 | {-# LANGUAGE ViewPatterns #-} 6 | 7 | module Language.Halide.FuncSpec (spec) where 8 | 9 | import Control.Monad.ST (RealWorld) 10 | import Data.Vector.Storable.Mutable qualified as SM 11 | import Language.Halide 12 | import Test.Hspec hiding (parallel) 13 | import Utils 14 | 15 | importHalide 16 | 17 | data Matrix v a = Matrix 18 | { matrixRows :: !Int 19 | , matrixCols :: !Int 20 | , matrixData :: !(v a) 21 | } 22 | deriving stock (Show, Eq) 23 | 24 | instance IsHalideType a => IsHalideBuffer (Matrix (SM.MVector RealWorld) a) 2 a where 25 | withHalideBufferImpl (Matrix n m v) f = 26 | SM.unsafeWith v $ \dataPtr -> 27 | bufferFromPtrShapeStrides dataPtr [n, m] [1, n] f 28 | 29 | spec :: Spec 30 | spec = do 31 | describe "indexing" $ do 32 | it "supports empty tuples" $ do 33 | let x = mkExpr (5 :: Double) 34 | f <- define "f" () $ x * x - 2 * x + 5 + 3 / x 35 | g <- define "g" () $ f ! () 36 | realize g [] peekScalar `shouldReturn` 20.6 37 | 38 | describe "vectorize" $ do 39 | it "vectorizes loops" $ do 40 | i <- mkVar "i" 41 | ii <- mkVar "inner" 42 | func <- define "func" i $ (3 * i + 1) * (i - 5) 43 | -- by default, nothing is vectorized 44 | prettyLoopNest func >>= \s -> do 45 | s `shouldNotContainText` "vectorized" 46 | s `shouldNotContainText` "[0, 3]" 47 | 48 | void $ 49 | split TailShiftInwards i (i, ii) 4 func 50 | >>= vectorize ii 51 | 52 | -- now, the inner loop is vectorized 53 | prettyLoopNest func >>= \s -> do 54 | s `shouldContainText` "vectorized i.inner" 55 | s `shouldContainText` "in [0, 3]" 56 | 57 | let n = 10 58 | realize func [n] peekToList 59 | `shouldReturn` [(3 * k + 1) * (k - 5) | k <- [0 .. fromIntegral n - 1]] 60 | 61 | describe "unroll" $ do 62 | it "unrolls loops" $ do 63 | i <- mkVar "i" 64 | ii <- mkVar "inner" 65 | func <- define "func" i $ (3 * i + 1) * (i - 5) 66 | -- by default, nothing is unrolled 67 | prettyLoopNest func >>= \s -> do 68 | s `shouldNotContainText` "unrolled" 69 | s `shouldNotContainText` "[0, 2]" 70 | 71 | void $ 72 | split TailGuardWithIf i (i, ii) 3 func 73 | >>= unroll ii 74 | 75 | -- now, the inner loop is unrolled 76 | prettyLoopNest func >>= \s -> do 77 | s `shouldContainText` "unrolled i.inner" 78 | s `shouldContainText` "in [0, 2]" 79 | 80 | let n = 17 81 | realize func [n] peekToList 82 | `shouldReturn` [(3 * k + 1) * (k - 5) | k <- [0 .. fromIntegral n - 1]] 83 | 84 | describe "reorder" $ do 85 | it "reorders loops" $ do 86 | x <- mkVar "x" 87 | y <- mkVar "y" 88 | z <- mkVar "z" 89 | func <- define "func" (x, y, z) $ x * (x + y) - 3 * z 90 | 91 | -- we have 92 | -- 93 | -- for z 94 | -- for y 95 | -- for x 96 | prettyLoopNest func >>= \s -> do 97 | s & "for z" `appearsBeforeText` "for y" 98 | s & "for y" `appearsBeforeText` "for x" 99 | 100 | void $ reorder [z, x, y] func 101 | 102 | -- now we expect 103 | -- 104 | -- for y 105 | -- for x 106 | -- for z 107 | prettyLoopNest func >>= \s -> do 108 | s & "for y" `appearsBeforeText` "for x" 109 | s & "for x" `appearsBeforeText` "for z" 110 | 111 | describe "split" $ do 112 | it "splits loops into sub-loops" $ do 113 | x <- mkVar "x" 114 | y <- mkVar "y" 115 | func <- define "func" (x, y) $ x * y 116 | 117 | -- we have 118 | -- 119 | -- for y 120 | -- for x 121 | prettyLoopNest func >>= \s -> do 122 | s & "for y" `appearsBeforeText` "for x" 123 | s `shouldNotContainText` "outer" 124 | s `shouldNotContainText` "inner" 125 | 126 | outer <- mkVar "outer" 127 | inner <- mkVar "inner" 128 | void $ split TailGuardWithIf x (outer, inner) 7 func 129 | 130 | -- now we expect 131 | -- 132 | -- for y 133 | -- for x.outer 134 | -- for x.inner 135 | prettyLoopNest func >>= \s -> do 136 | s & "for y" `appearsBeforeText` "for x.outer" 137 | s & "for x.outer" `appearsBeforeText` "for x.inner" 138 | 139 | describe "fuse" $ do 140 | it "merges sub-loops into one" $ do 141 | x <- mkVar "x" 142 | y <- mkVar "y" 143 | func <- define "func" (x, y) $ x * y 144 | 145 | -- we have 146 | -- 147 | -- for y 148 | -- for x 149 | prettyLoopNest func >>= \s -> do 150 | s `shouldNotContainText` "common" 151 | 152 | common <- mkVar "common" 153 | void $ fuse (x, y) common func 154 | 155 | -- now we expect 156 | -- 157 | -- for common 158 | prettyLoopNest func >>= \s -> do 159 | s `shouldNotContainText` "for x:" 160 | s `shouldNotContainText` "for y" 161 | s `shouldContainText` "for x.common" 162 | 163 | describe "parallel" $ do 164 | it "marks dimensions as parallel" $ do 165 | x <- mkVar "x" 166 | y <- mkVar "y" 167 | func <- define "func" (x, y) $ x * y 168 | 169 | prettyLoopNest func >>= \s -> 170 | s `shouldNotContainText` "parallel" 171 | 172 | void $ 173 | parallel x func 174 | >>= serial y 175 | 176 | prettyLoopNest func >>= \s -> 177 | s `shouldContainText` "parallel x" 178 | 179 | describe "gpuBlocks" $ do 180 | it "marks dimensions as corresponding to GPU blocks" $ do 181 | do 182 | x <- mkVar "x" 183 | y <- mkVar "y" 184 | func <- define "func" (x, y) $ x * y 185 | 186 | prettyLoopNest func >>= \s -> do 187 | s `shouldNotContainText` "gpu_block" 188 | s `shouldNotContainText` "Default_GPU" 189 | void $ gpuBlocks DeviceDefaultGPU (x, y) func 190 | prettyLoopNest func >>= \s -> do 191 | s `shouldContainText` "gpu_block yGenerated output
45 | 46 | ``` 47 | Creating initial loop nests... 48 | Injecting realization of { gradient } 49 | Skipping injecting memoization... 50 | Injecting tracing... 51 | Adding checks for parameters 52 | Computing bounds of each function's value 53 | Clamping unsafe data-dependent accesses 54 | Performing computation bounds inference... 55 | Removing extern loops... 56 | Performing sliding window optimization... 57 | Uniquifying variable names... 58 | Simplifying... 59 | Simplifying correlated differences... 60 | Performing allocation bounds inference... 61 | Adding checks for images 62 | Removing code that depends on undef values... 63 | Performing storage folding optimization... 64 | Injecting debug_to_file calls... 65 | Injecting prefetches... 66 | Discarding safe promises... 67 | Dynamically skipping stages... 68 | Forking asynchronous producers... 69 | Destructuring tuple-valued realizations... 70 | Bounding small realizations... 71 | Performing storage flattening... 72 | Adding atomic mutex allocation... 73 | Unpacking buffer arguments... 74 | Skipping rewriting memoized allocations... 75 | Simplifying... 76 | Reduce prefetch dimension... 77 | Simplifying correlated differences... 78 | Unrolling... 79 | Vectorizing... 80 | Detecting vector interleavings... 81 | Partitioning loops to simplify boundary conditions... 82 | Staging strided loads... 83 | Trimming loops to the region over which they do something... 84 | Rebasing loops to zero... 85 | Hoisting loop invariant if statements... 86 | Injecting early frees... 87 | Simplifying correlated differences... 88 | Bounding small allocations... 89 | Simplifying... 90 | Lowering unsafe promises... 91 | Flattening nested ramps... 92 | Removing dead allocations and moving loop invariant code... 93 | Finding intrinsics... 94 | Hoisting prefetches... 95 | Lowering after final simplification: 96 | assert(reinterpret
44 |
117 |
118 | This will print out all the times `gradient(x, y)` gets evaluated.
119 |
120 | Now that we can snoop on what Halide is doing, let's try our first scheduling
121 | primitive. We'll make a new version of gradient that processes each scanline in
122 | parallel.
123 |
124 | ```haskell
125 | parallelGradient <- define "parallelGradient" (x, y) $ x + y
126 | ```
127 |
128 | We'll also trace this function.
129 |
130 | ```haskell
131 | traceStores parallelGradient
132 | ```
133 |
134 | Things are the same so far. We've defined the algorithm, but haven't said
135 | anything about how to schedule it. In general, exploring different scheduling
136 | decisions doesn't change the code that describes the algorithm.
137 |
138 | Now we tell Halide to use a parallel for loop over the y coordinate. On Linux
139 | we run this using a thread pool and a task queue. On OS X we call into grand
140 | central dispatch, which does the same thing for us.
141 |
142 | ```haskell
143 | void $ parallel y parallelGradient
144 | ```
145 |
146 | This time the printfs should come out of order, because each scanline is
147 | potentially being processed in a different thread. The number of threads should
148 | adapt to your system, but on linux you can control it manually using the
149 | environment variable `HL_NUM_THREADS`.
150 |
151 | ```haskell
152 | putStrLn "Evaluating parallelGradient ..."
153 | realize parallelGradient [8, 8] . const $ pure ()
154 | ```
155 |
156 | Show output...
45 | 46 | ``` 47 | Begin pipeline gradient.0() 48 | Tag gradient.0() tag = "func_type_and_dim: 1 0 32 1 2 0 8 0 8" 49 | Store gradient.0(0, 0) = 0 50 | Store gradient.0(1, 0) = 1 51 | Store gradient.0(2, 0) = 2 52 | Store gradient.0(3, 0) = 3 53 | Store gradient.0(4, 0) = 4 54 | Store gradient.0(5, 0) = 5 55 | Store gradient.0(6, 0) = 6 56 | Store gradient.0(7, 0) = 7 57 | Store gradient.0(0, 1) = 1 58 | Store gradient.0(1, 1) = 2 59 | Store gradient.0(2, 1) = 3 60 | Store gradient.0(3, 1) = 4 61 | Store gradient.0(4, 1) = 5 62 | Store gradient.0(5, 1) = 6 63 | Store gradient.0(6, 1) = 7 64 | Store gradient.0(7, 1) = 8 65 | Store gradient.0(0, 2) = 2 66 | Store gradient.0(1, 2) = 3 67 | Store gradient.0(2, 2) = 4 68 | Store gradient.0(3, 2) = 5 69 | Store gradient.0(4, 2) = 6 70 | Store gradient.0(5, 2) = 7 71 | Store gradient.0(6, 2) = 8 72 | Store gradient.0(7, 2) = 9 73 | Store gradient.0(0, 3) = 3 74 | Store gradient.0(1, 3) = 4 75 | Store gradient.0(2, 3) = 5 76 | Store gradient.0(3, 3) = 6 77 | Store gradient.0(4, 3) = 7 78 | Store gradient.0(5, 3) = 8 79 | Store gradient.0(6, 3) = 9 80 | Store gradient.0(7, 3) = 10 81 | Store gradient.0(0, 4) = 4 82 | Store gradient.0(1, 4) = 5 83 | Store gradient.0(2, 4) = 6 84 | Store gradient.0(3, 4) = 7 85 | Store gradient.0(4, 4) = 8 86 | Store gradient.0(5, 4) = 9 87 | Store gradient.0(6, 4) = 10 88 | Store gradient.0(7, 4) = 11 89 | Store gradient.0(0, 5) = 5 90 | Store gradient.0(1, 5) = 6 91 | Store gradient.0(2, 5) = 7 92 | Store gradient.0(3, 5) = 8 93 | Store gradient.0(4, 5) = 9 94 | Store gradient.0(5, 5) = 10 95 | Store gradient.0(6, 5) = 11 96 | Store gradient.0(7, 5) = 12 97 | Store gradient.0(0, 6) = 6 98 | Store gradient.0(1, 6) = 7 99 | Store gradient.0(2, 6) = 8 100 | Store gradient.0(3, 6) = 9 101 | Store gradient.0(4, 6) = 10 102 | Store gradient.0(5, 6) = 11 103 | Store gradient.0(6, 6) = 12 104 | Store gradient.0(7, 6) = 13 105 | Store gradient.0(0, 7) = 7 106 | Store gradient.0(1, 7) = 8 107 | Store gradient.0(2, 7) = 9 108 | Store gradient.0(3, 7) = 10 109 | Store gradient.0(4, 7) = 11 110 | Store gradient.0(5, 7) = 12 111 | Store gradient.0(6, 7) = 13 112 | Store gradient.0(7, 7) = 14 113 | End pipeline gradient.0() 114 | ``` 115 | 116 |
157 |
231 |
232 |
233 | ### Printing individual Exprs.
234 |
235 | `traceStores` can only print the value of a `Func`. Sometimes you want to
236 | inspect the value of sub-expressions rather than the entire `Func`. The
237 | function `printed` can be wrapped around any `Expr` to print the value of that
238 | `Expr` every time it is evaluated.
239 |
240 | For example, say we have some `Func` that is the sum of two terms:
241 |
242 | ```haskell
243 | it "Prints individual Exprs" $ do
244 | x <- mkVar "x"
245 | y <- mkVar "y"
246 | f <- define "f" (x, y) $ sin (cast @Float x) + cos (cast @Float y)
247 | ```
248 |
249 | If we want to inspect just one of the terms, we can wrap it in a call to `printed`:
250 |
251 | ```haskell
252 | g <- define "g" (x, y) $ sin (cast @Float x) + printed (cos (cast @Float y))
253 | putStrLn "Evaluating sin(x) + cos(y), and just printing cos(y)..."
254 | realize g [4, 4] . const $ pure ()
255 | ```
256 |
257 | Show output...
158 | 159 | ``` 160 | Evaluating parallelGradient ... 161 | Begin pipeline parallelGradient.0() 162 | Tag parallelGradient.0() tag = "func_type_and_dim: 1 0 32 1 2 0 8 0 8" 163 | Store parallelGradient.0(0, 0) = 0 164 | Store parallelGradient.0(1, 0) = 1 165 | Store parallelGradient.0(2, 0) = 2 166 | Store parallelGradient.0(3, 0) = 3 167 | Store parallelGradient.0(4, 0) = 4 168 | Store parallelGradient.0(5, 0) = 5 169 | Store parallelGradient.0(6, 0) = 6 170 | Store parallelGradient.0(7, 0) = 7 171 | Store parallelGradient.0(0, 4) = 4 172 | Store parallelGradient.0(1, 4) = 5 173 | Store parallelGradient.0(2, 4) = 6 174 | Store parallelGradient.0(3, 4) = 7 175 | Store parallelGradient.0(4, 4) = 8 176 | Store parallelGradient.0(5, 4) = 9 177 | Store parallelGradient.0(6, 4) = 10 178 | Store parallelGradient.0(7, 4) = 11 179 | Store parallelGradient.0(0, 3) = 3 180 | Store parallelGradient.0(1, 3) = 4 181 | Store parallelGradient.0(2, 3) = 5 182 | Store parallelGradient.0(3, 3) = 6 183 | Store parallelGradient.0(4, 3) = 7 184 | Store parallelGradient.0(5, 3) = 8 185 | Store parallelGradient.0(6, 3) = 9 186 | Store parallelGradient.0(7, 3) = 10 187 | Store parallelGradient.0(0, 7) = 7 188 | Store parallelGradient.0(0, 5) = 5 189 | Store parallelGradient.0(0, 6) = 6 190 | Store parallelGradient.0(1, 7) = 8 191 | Store parallelGradient.0(1, 5) = 6 192 | Store parallelGradient.0(2, 5) = 7 193 | Store parallelGradient.0(1, 6) = 7 194 | Store parallelGradient.0(2, 7) = 9 195 | Store parallelGradient.0(3, 5) = 8 196 | Store parallelGradient.0(2, 6) = 8 197 | Store parallelGradient.0(3, 7) = 10 198 | Store parallelGradient.0(4, 5) = 9 199 | Store parallelGradient.0(4, 7) = 11 200 | Store parallelGradient.0(5, 5) = 10 201 | Store parallelGradient.0(3, 6) = 9 202 | Store parallelGradient.0(5, 7) = 12 203 | Store parallelGradient.0(6, 5) = 11 204 | Store parallelGradient.0(4, 6) = 10 205 | Store parallelGradient.0(6, 7) = 13 206 | Store parallelGradient.0(5, 6) = 11 207 | Store parallelGradient.0(7, 5) = 12 208 | Store parallelGradient.0(7, 7) = 14 209 | Store parallelGradient.0(6, 6) = 12 210 | Store parallelGradient.0(7, 6) = 13 211 | Store parallelGradient.0(0, 1) = 1 212 | Store parallelGradient.0(1, 1) = 2 213 | Store parallelGradient.0(2, 1) = 3 214 | Store parallelGradient.0(3, 1) = 4 215 | Store parallelGradient.0(4, 1) = 5 216 | Store parallelGradient.0(5, 1) = 6 217 | Store parallelGradient.0(6, 1) = 7 218 | Store parallelGradient.0(7, 1) = 8 219 | Store parallelGradient.0(0, 2) = 2 220 | Store parallelGradient.0(1, 2) = 3 221 | Store parallelGradient.0(2, 2) = 4 222 | Store parallelGradient.0(3, 2) = 5 223 | Store parallelGradient.0(4, 2) = 6 224 | Store parallelGradient.0(5, 2) = 7 225 | Store parallelGradient.0(6, 2) = 8 226 | Store parallelGradient.0(7, 2) = 9 227 | End pipeline parallelGradient.0() 228 | ``` 229 | 230 |
258 |
280 |
281 |
282 | ### Printing additional context
283 |
284 | `printed` can take multiple arguments. It prints all of them and evaluates to
285 | the first one. The arguments can be `Expr`s, `Text`, or `String`. This can be
286 | used to print additional context alongside the value:
287 |
288 | ```haskell
289 | it "Prints additional context" $ do
290 | x <- mkVar "x"
291 | y <- mkVar "y"
292 | let second = printed (cos (cast @Float y)) ("<- this is cos(" :: Text) y (") when x =" :: Text) x
293 | f <- define "f" (x, y) $ sin (cast @Float x) + second
294 | putStrLn "Evaluating sin(x) + cos(y), and printing cos(y) with more context..."
295 | realize f [4, 4] . const $ pure ()
296 | ```
297 |
298 | Show output...
259 | 260 | ``` 261 | 1.000000 262 | 1.000000 263 | 1.000000 264 | 1.000000 265 | 0.540302 266 | 0.540302 267 | 0.540302 268 | 0.540302 269 | -0.416147 270 | -0.416147 271 | -0.416147 272 | -0.416147 273 | -0.989992 274 | -0.989992 275 | -0.989992 276 | -0.989992 277 | ``` 278 | 279 |
299 |
321 |
322 |
323 | ### Conditional printing
324 |
325 | Both `printed` and `traceStores` can produce a lot of output. If you're looking for
326 | a rare event, or just want to see what happens at a single pixel, this amount
327 | of output can be difficult to dig through. Instead, the function `printedWhen` can
328 | be used to conditionally print an `Expr`. The first argument to `printedWhen`
329 | is an `Expr Bool`. If the `Expr` evaluates to `True`, it returns the second
330 | argument and prints all of the arguments. If the `Expr` evaluates to `False` it
331 | just returns the second argument and does not print.
332 |
333 | ```haskell
334 | it "Conditionally prints Exprs" $ do
335 | x <- mkVar "x"
336 | y <- mkVar "y"
337 | let e = printedWhen (eq x 37 `and` eq y 42) (cos (cast @Float y)) ("<- this is cos(y) at x, y == (37, 42)" :: Text)
338 | f <- define "f" (x, y) $ sin (cast @Float x) + e
339 | putStrLn $ "Evaluating sin(x) + cos(y), and printing cos(y) at a single pixel..."
340 | realize f [640, 480] . const $ pure ()
341 | ```
342 |
343 | Show output...
300 | 301 | ``` 302 | 1.000000 <- this is cos( 0 ) when x = 0 303 | 1.000000 <- this is cos( 0 ) when x = 1 304 | 1.000000 <- this is cos( 0 ) when x = 2 305 | 1.000000 <- this is cos( 0 ) when x = 3 306 | 0.540302 <- this is cos( 1 ) when x = 0 307 | 0.540302 <- this is cos( 1 ) when x = 1 308 | 0.540302 <- this is cos( 1 ) when x = 2 309 | 0.540302 <- this is cos( 1 ) when x = 3 310 | -0.416147 <- this is cos( 2 ) when x = 0 311 | -0.416147 <- this is cos( 2 ) when x = 1 312 | -0.416147 <- this is cos( 2 ) when x = 2 313 | -0.416147 <- this is cos( 2 ) when x = 3 314 | -0.989992 <- this is cos( 3 ) when x = 0 315 | -0.989992 <- this is cos( 3 ) when x = 1 316 | -0.989992 <- this is cos( 3 ) when x = 2 317 | -0.989992 <- this is cos( 3 ) when x = 3 318 | ``` 319 | 320 |
344 |
351 |
352 | `printedWhen` can also be used to check for values you're not expecting:
353 |
354 | ```haskell
355 | let e = cos (cast @Float y)
356 | e' = printedWhen (e `lt` 0) e ("cos(y) < 0 at y ==" :: Text) y
357 | g <- define "g" (x, y) $ sin (cast @Float x) + e'
358 | putStrLn $ "Evaluating sin(x) + cos(y), and printing whenever cos(y) < 0..."
359 | realize g [4, 4] . const $ pure ()
360 | ```
361 |
362 | Show output...
345 | 346 | ``` 347 | -0.399985 <- this is cos(y) at x, y == (37, 42) 348 | ``` 349 | 350 |
363 |
377 |
378 |
379 | ### Printing expressions at compile-time.
380 |
381 | The code below builds up a Halide expression across several lines of code. If
382 | you're programmatically constructing a complex expression, and you want to
383 | check the expression you've created is what you think it is, you can also print
384 | out the expression itself the `Show` typeclass:
385 |
386 | ```haskell
387 | it "Prints expressions at compile-time" $ do
388 | fizz <- mkVar "fizz"
389 | buzz <- mkVar "buzz"
390 | let combine e i
391 | | i `mod` 3 == 0 && i `mod` 5 == 0 = e + fizz * buzz
392 | | i `mod` 3 == 0 = e + fizz
393 | | i `mod` 5 == 0 = e + buzz
394 | | otherwise = e + mkExpr i
395 | let e = foldl combine 1 [2 .. 99]
396 | print e
397 | ```
398 |
399 | Show output...
364 | 365 | ``` 366 | -0.416147 cos(y) < 0 at y == 2 367 | -0.416147 cos(y) < 0 at y == 2 368 | -0.416147 cos(y) < 0 at y == 2 369 | -0.416147 cos(y) < 0 at y == 2 370 | -0.989992 cos(y) < 0 at y == 3 371 | -0.989992 cos(y) < 0 at y == 3 372 | -0.989992 cos(y) < 0 at y == 3 373 | -0.989992 cos(y) < 0 at y == 3 374 | ``` 375 | 376 |
400 |
407 |
--------------------------------------------------------------------------------
/tutorials/04-Debugging/halide-tutorial04.cabal:
--------------------------------------------------------------------------------
1 | cabal-version: 3.0
2 | name: halide-tutorial04
3 | version: 0.0.1.0
4 | license: BSD-3-Clause
5 | author: twesterhout
6 | maintainer: 14264576+twesterhout@users.noreply.github.com
7 | build-type: Simple
8 |
9 | common setup
10 | ghc-options: -W -pgmL markdown-unlit
11 | default-language: GHC2021
12 | build-depends:
13 | , base >=4.16.0.0 && <5
14 | , halide-haskell
15 | , hspec
16 | , text
17 | , vector
18 |
19 | -- build-tools-depends: markdown-unlit:exe:markdown-unlit
20 |
21 | executable halide-tutorial04
22 | import: setup
23 | main-is: README.lhs
24 | hs-source-dirs: .
25 |
26 | test-suite halide-tutorial04-test
27 | import: setup
28 | main-is: README.lhs
29 | type: exitcode-stdio-1.0
30 |
--------------------------------------------------------------------------------
/tutorials/05-Scheduling/LICENSE:
--------------------------------------------------------------------------------
1 | Copyright (c) 2023, twesterhout
2 |
3 | All rights reserved.
4 |
5 | Redistribution and use in source and binary forms, with or without
6 | modification, are permitted provided that the following conditions are met:
7 |
8 | * Redistributions of source code must retain the above copyright
9 | notice, this list of conditions and the following disclaimer.
10 |
11 | * Redistributions in binary form must reproduce the above
12 | copyright notice, this list of conditions and the following
13 | disclaimer in the documentation and/or other materials provided
14 | with the distribution.
15 |
16 | * Neither the name of twesterhout nor the names of other
17 | contributors may be used to endorse or promote products derived
18 | from this software without specific prior written permission.
19 |
20 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
21 | "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
22 | LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
23 | A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
24 | OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
25 | SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
26 | LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
27 | DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
28 | THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
29 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
30 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
31 |
--------------------------------------------------------------------------------
/tutorials/05-Scheduling/README.lhs:
--------------------------------------------------------------------------------
1 | README.md
--------------------------------------------------------------------------------
/tutorials/05-Scheduling/README.md:
--------------------------------------------------------------------------------
1 | # Vectorize, parallelize, unroll and tile your code
2 |
3 | This lesson demonstrates how to manipulate the order in which you evaluate
4 | pixels in a `Func`, including vectorization, parallelization, unrolling, and
5 | tiling.
6 |
7 | Because this README is also a literate Haskell file, we start with a few common imports.
8 |
9 | ```haskell
10 | {-# LANGUAGE OverloadedStrings, TypeApplications, DataKinds #-}
11 | import Control.Monad (void)
12 | import Data.Text (Text, pack)
13 | import Data.Text.IO (hPutStrLn)
14 | import System.IO (stderr)
15 | import Test.Hspec hiding (parallel)
16 |
17 | import Language.Halide
18 | ```
19 |
20 | We're going to define and schedule our gradient function in several different
21 | ways, and see what order pixels are computed in. For this, we define a helper
22 | function such that we don't have to constantly repeat the definition of it.
23 |
24 | ```haskell
25 | prepare :: IO (Expr Int32, Expr Int32, Func 'FuncTy 2 (Expr Int32))
26 | prepare = do
27 | x <- mkVar "x"
28 | y <- mkVar "y"
29 | gradient <- define "gradient" (x, y) $ x + y
30 | pure (x, y, gradient)
31 | ```
32 |
33 | We also define a helper function to evaluate a `Func` and collect the order of
34 | stores performed by the function. We will use it for testing.
35 |
36 | ```haskell
37 | getIterationOrder :: (KnownNat n, IsHalideType a) => [Int] -> Func t n (Expr a) -> IO [[Int]]
38 | getIterationOrder shape f =
39 | fmap fst $
40 | collectIterationOrder (TraceStore ==) f $ do
41 | _ <- traceStores f
42 | realize f shape (void . pure)
43 | ```
44 |
45 | Finally, we define a helper function to dump `Text` to stderr.
46 |
47 | ```haskell
48 | toStderr :: Text -> IO ()
49 | toStderr a = hPutStrLn stderr ("\n" <> a)
50 | ```
51 |
52 | ## Default
53 |
54 | By default, Halide traverses arrays in column-major order. That means that the
55 | loop over x happens first, and then the loop over y. This can be visualized
56 | using the `prettyLoopNest` function:
57 |
58 | ```haskell
59 | main :: IO ()
60 | main = hspec $ describe "Tutorial 5" $ do
61 | it "Has column-major ordering by default" $ do
62 | (_, _, gradient) <- prepare
63 | toStderr =<< prettyLoopNest gradient
64 | ```
65 |
66 | This produces:
67 |
68 | ```
69 | produce gradient:
70 | for y:
71 | for x:
72 | gradient(...) = ...
73 | ```
74 |
75 | Let's make sure that the function actually computes what we want:
76 |
77 | ```haskell
78 | realize gradient [4, 4] peekToList
79 | `shouldReturn` [[i + j | j <- [0 .. 3]] | i <- [0 .. 3]]
80 | ```
81 |
82 | `realize` function evaluates our pipeline over a rectangular domain -- in our
83 | case a 4x4 buffer, and `peekToList` then reads the contents of the buffer into
84 | a nested list.
85 |
86 | We check our assumption about the iteration order by building the expected
87 | iteration order using standard list comprehensions:
88 |
89 | ```haskell
90 | getIterationOrder [4, 4] gradient
91 | `shouldReturn` [[i, j] | j <- [0 .. 3], i <- [0 .. 3]]
92 | ```
93 |
94 | ## Reorder
95 |
96 | If we reorder `x` and `y`, we can walk along the rows instead. The `reorder` call takes the
97 | arguments of the func, and sets a new nesting order for the for loops that are generated.
98 | The arguments are specified from the innermost loop out, so the following puts `y` in
99 | the inner loop:
100 |
101 | ```haskell
102 | it "Reorders loops to have row-major order" $ do
103 | (x, y, gradient) <- prepare
104 | void $ reorder [y, x] gradient
105 | toStderr =<< prettyLoopNest gradient
106 | getIterationOrder [4, 4] gradient
107 | `shouldReturn` [[i, j] | i <- [0 .. 3], j <- [0 .. 3]]
108 | ```
109 |
110 | And the loop nest now reads
111 |
112 | ```
113 | produce gradient:
114 | for x:
115 | for y:
116 | gradient(...) = ...
117 | ```
118 |
119 | ## Split
120 |
121 | The most powerful primitive scheduling operation you can do to a var is to split it into
122 | inner and outer sub-variables:
123 |
124 | ```haskell
125 | it "Splits a variable into two" $ do
126 | (x, _, gradient) <- prepare
127 | inner <- mkVar "inner"
128 | outer <- mkVar "outer"
129 | void $ split TailAuto x (outer, inner) 2 gradient
130 | toStderr =<< prettyLoopNest gradient
131 | ```
132 |
133 | This breaks the loop over `x` into two nested loops: an outer one over outer, and an inner
134 | one over inner. We have also specified the "split factor". The inner loop runs from zero
135 | to the split factor. The outer loop runs from zero to the extent required of `x` (4 in this case)
136 | divided by the split factor. Within the loops, the old variable is defined to be
137 | `outer * factor + inner`. If the old loop started at a value other than zero, then that is
138 | also added within the loops.
139 |
140 | ```
141 | produce gradient:
142 | for y:
143 | for x.outer:
144 | for x.inner in [0, 1]:
145 | gradient(...) = ...
146 | ```
147 |
148 | Note that the order of evaluation of pixels didn't actually change! Splitting by itself does
149 | nothing, but it does open up all of the scheduling possibilities that we will explore below.
150 |
151 | ```haskell
152 | getIterationOrder [4, 4] gradient
153 | `shouldReturn` [[i, j] | j <- [0 .. 3], i <- [0 .. 3]]
154 | ```
155 |
156 | ## Fuse
157 |
158 | The opposite of splitting is fusing. Fusing two variables merges the two loops into
159 | a single for loop over the product of the extents. Fusing is less important than splitting,
160 | but it also sees use (as we'll see later in this lesson). Like splitting, fusing by itself
161 | doesn't change the order of evaluation.
162 |
163 | ```haskell
164 | it "Fuses two variables into one" $ do
165 | (x, y, gradient) <- prepare
166 | fused <- mkVar "fused"
167 | void $ fuse (x, y) fused gradient
168 | toStderr =<< prettyLoopNest gradient
169 |
170 | getIterationOrder [4, 4] gradient
171 | `shouldReturn` [[i, j] | j <- [0 .. 3], i <- [0 .. 3]]
172 | ```
173 |
174 | ```
175 | produce gradient:
176 | for x.fused:
177 | gradient(...) = ...
178 | ```
179 |
180 | ## Tile
181 |
182 | Now that we can both split and reorder, we can do tiled evaluation. Let's split both
183 | x and y by a factor of four, and then reorder the vars to express a tiled traversal:
184 |
185 | ```haskell
186 | it "Evaluates in tiles" $ do
187 | (x, y, gradient) <- prepare
188 | [xOuter, xInner, yOuter, yInner] <- mapM mkVar ["xOuter", "xInner", "yOuter", "yInner"]
189 | void $
190 | split TailAuto x (xOuter, xInner) 2 gradient
191 | >>= split TailAuto y (yOuter, yInner) 2
192 | >>= reorder [xInner, yInner, xOuter, yOuter]
193 | toStderr =<< prettyLoopNest gradient
194 | ```
195 |
196 | ```
197 | produce gradient:
198 | for y.yOuter:
199 | for x.xOuter:
200 | for y.yInner in [0, 1]:
201 | for x.xInner in [0, 1]:
202 | gradient(...) = ...
203 | ```
204 |
205 | A tiled traversal splits the domain into small rectangular tiles, and outermost iterates
206 | over the tiles, and within that iterates over the points within each tile. It can be good
207 | for performance if neighboring pixels use overlapping input data, for example in a blur.
208 |
209 | ```haskell
210 | getIterationOrder [4, 4] gradient
211 | `shouldReturn` [ [i + 2 * iTile, j + 2 * jTile]
212 | | jTile <- [0 .. 1]
213 | , iTile <- [0 .. 1]
214 | , j <- [0 .. 1]
215 | , i <- [0 .. 1]
216 | ]
217 | ```
218 |
219 | ## Vectorize
220 |
221 | The nice thing about splitting is that it guarantees the inner variable runs from zero
222 | to the split factor. Most of the time the split-factor will be a compile-time constant,
223 | so we can replace the loop over the inner variable with a single vectorized computation.
224 | This time we'll split by a factor of four, because on X86 we can use SSE to compute in
225 | 4-wide vectors.
226 |
227 |
228 | ```haskell
229 | it "Evaluates in vectors" $ do
230 | (x, _, gradient) <- prepare
231 | inner <- mkVar "inner"
232 | void $
233 | split TailAuto x (x, inner) 4 gradient
234 | >>= vectorize inner
235 | toStderr =<< prettyLoopNest gradient
236 | ```
237 |
238 | ```
239 | produce gradient:
240 | for y:
241 | for x.x:
242 | vectorized x.inner in [0, 3]:
243 | gradient(...) = ...
244 | ```
245 |
246 | We can check that the vectorization is reflected in the iteration order. We now expect coordinates to be a
247 | length 8 list instead of length 2. I.e. each element is [x1, x2, x3, x4, y1, y2, y3, y4]. This time we'll evaluate over an 8x4 box, so that we have more than one vector of work per scanline.
248 |
249 | ```haskell
250 | getIterationOrder [8, 4] gradient
251 | `shouldReturn` [ [4 * i .. 4 * i + 3] <> [j, j, j, j]
252 | | j <- [0 .. 3]
253 | , i <- [0 .. 1]
254 | ]
255 | ```
256 |
257 | ## Unroll
258 |
259 | If multiple pixels share overlapping data, it can make sense to unroll a computation
260 | so that shared values are only computed or loaded once. We do this similarly to how we
261 | expressed vectorizing. We split a dimension and then fully unroll the loop of the inner
262 | variable. Unrolling doesn't change the order in which things are evaluated.
263 |
264 | ```haskell
265 | it "Unrolls loops" $ do
266 | (x, _, gradient) <- prepare
267 | inner <- mkVar "inner"
268 | void $
269 | split TailAuto x (x, inner) 2 gradient
270 | >>= unroll inner
271 | toStderr =<< prettyLoopNest gradient
272 |
273 | getIterationOrder [4, 4] gradient
274 | `shouldReturn` [[i, j] | j <- [0 .. 3], i <- [0 .. 3]]
275 |
276 | ```
277 |
278 | ```
279 | produce gradient:
280 | for y:
281 | for x.x:
282 | unrolled x.inner in [0, 1]:
283 | gradient(...) = ...
284 | ```
285 |
286 | ## Split again
287 |
288 | Splitting guarantees that the inner loop runs from zero to the split factor, which is
289 | important for the uses we saw above. So what happens when the total extent we wish to
290 | evaluate x over isn't a multiple of the split factor? We'll split by a factor three,
291 | and we'll evaluate gradient over a 7x2 box instead of the 4x4 box we've been using.
292 |
293 | ```haskell
294 | it "Splits by factors that don't divide the extent" $ do
295 | (x, _, gradient) <- prepare
296 | inner <- mkVar "inner"
297 | void $ split TailAuto x (x, inner) 3 gradient
298 |
299 | order <- getIterationOrder [7, 4] gradient
300 | toStderr . pack . show $ order
301 | ```
302 |
303 | ```
304 | [[0,0],[1,0],[2,0],[3,0],[4,0],[5,0],[4,0],[5,0],[6,0],
305 | [0,1],[1,1],[2,1],[3,1],[4,1],[5,1],[4,1],[5,1],[6,1],
306 | [0,2],[1,2],[2,2],[3,2],[4,2],[5,2],[4,2],[5,2],[6,2],
307 | [0,3],[1,3],[2,3],[3,3],[4,3],[5,3],[4,3],[5,3],[6,3]]
308 | ```
309 |
310 | If you read the output, you'll see that some coordinates were evaluated more than once.
311 | That's generally OK, because pure Halide functions have no side-effects, so it's safe to
312 | evaluate the same point multiple times.
313 |
314 | The general rule is: If we require x from x_min to x_min + x_extent, and we split by a
315 | factor factor, then:
316 |
317 | x_outer runs from 0 to (x_extent + factor - 1)/factor
318 | x_inner runs from 0 to factor
319 | x = min(x_outer * factor, x_extent - factor) + x_inner + x_min
320 |
321 | In our example, x_min was 0, x_extent was 7, and factor was 3.
322 |
323 | ```haskell
324 | order
325 | `shouldBe` [ [iInner + Prelude.min (3 * iOuter) (7 - 3), j]
326 | | j <- [0 .. 3]
327 | , iOuter <- [0 .. 2]
328 | , iInner <- [0 .. 2]
329 | ]
330 | ```
331 |
332 |
333 | ## Parallelize
334 |
335 | We saw in the previous lesson that we can parallelize across a variable. Here we combine
336 | it with fusing and tiling to express a useful pattern - processing tiles in parallel.
337 |
338 | This is where fusing shines. Fusing helps when you want to parallelize across multiple
339 | dimensions without introducing nested parallelism. Nested parallelism (parallel for loops
340 | within parallel for loops) is supported by Halide, but often gives poor performance compared
341 | to fusing the parallel variables into a single parallel for loop.
342 |
343 | ```haskell
344 | it "Fuses, tiles, and parallelizes" $ do
345 | (x, y, gradient) <- prepare
346 | [xOuter, xInner, yOuter, yInner, tileIndex] <-
347 | mapM mkVar ["xOuter", "xInner", "yOuter", "yInner", "tileIndex"]
348 |
349 | void $
350 | split TailAuto x (xOuter, xInner) 4 gradient
351 | >>= split TailAuto y (yOuter, yInner) 4
352 | >>= reorder [xInner, yInner, xOuter, yOuter]
353 | >>= fuse (xOuter, yOuter) tileIndex
354 | >>= parallel tileIndex
355 |
356 | toStderr =<< prettyLoopNest gradient
357 | ```
358 |
359 | ```
360 | produce gradient:
361 | parallel x.xOuter.tileIndex:
362 | for y.yInner in [0, 3]:
363 | for x.xInner in [0, 3]:
364 | gradient(...) = ...
365 | ```
366 |
367 | ## All together now
368 |
369 | Are you ready? We're going to use all of the features above now.
370 |
371 | We'll process 64x64 tiles in parallel.
372 |
373 | ```haskell
374 | it "Puts it all together for ultimate performance!" $ do
375 | (x, y, gradient) <- prepare
376 | [xOuter, yOuter, xInner, yInner, tileIndex] <-
377 | mapM mkVar ["xOuter", "yOuter", "xInner", "yInner", "tileIndex"]
378 | void $
379 | split TailAuto x (xOuter, xInner) 64 gradient
380 | >>= split TailAuto y (yOuter, yInner) 64
381 | >>= reorder [xInner, yInner, xOuter, yOuter]
382 | >>= fuse (xOuter, yOuter) tileIndex
383 | >>= parallel tileIndex
384 | ```
385 |
386 | We'll compute two scanlines at once while we walk across each tile. We'll also vectorize in x.
387 | The easiest way to express this is to recursively tile again within each tile into 4x2 subtiles,
388 | then vectorize the subtiles across x and unroll them across y:
389 |
390 | ```haskell
391 | [xInnerOuter, yInnerOuter, xVectors, yPairs] <-
392 | mapM mkVar ["xInnerOuter", "yInnerOuter", "xVectors", "yPairs"]
393 | void $
394 | split TailAuto xInner (xInnerOuter, xVectors) 4 gradient
395 | >>= split TailAuto yInner (yInnerOuter, yPairs) 2
396 | >>= reorder [xVectors, yPairs, xInnerOuter, yInnerOuter]
397 | >>= vectorize xVectors
398 | >>= unroll yPairs
399 |
400 | toStderr =<< prettyLoopNest gradient
401 | ```
402 |
403 | ```
404 | produce gradient:
405 | parallel x.xOuter.tileIndex:
406 | for y.yInner.yInnerOuter in [0, 31]:
407 | for x.xInner.xInnerOuter in [0, 15]:
408 | unrolled y.yInner.yPairs in [0, 1]:
409 | vectorized x.xInner.xVectors in [0, 3]:
410 | gradient(...) = ...
411 | ```
412 |
413 | Let's check that the result is still as we expect
414 |
415 | ```haskell
416 | realize gradient [350, 250] peekToList
417 | `shouldReturn` [[i + j | j <- [0 .. 249]] | i <- [0 .. 349]]
418 | ```
419 |
--------------------------------------------------------------------------------
/tutorials/05-Scheduling/halide-tutorial05.cabal:
--------------------------------------------------------------------------------
1 | cabal-version: 3.0
2 | name: halide-tutorial05
3 | version: 0.0.1.0
4 | license: BSD-3-Clause
5 | license-file: LICENSE
6 | author: twesterhout
7 | maintainer: 14264576+twesterhout@users.noreply.github.com
8 | build-type: Simple
9 |
10 | common setup
11 | ghc-options: -W -pgmL markdown-unlit
12 | default-language: GHC2021
13 | build-depends:
14 | , base >=4.16.0.0 && <5
15 | , halide-haskell
16 | , hspec
17 | , text
18 | , vector
19 |
20 | -- build-tools-depends: markdown-unlit:exe:markdown-unlit
21 |
22 | executable halide-tutorial05
23 | import: setup
24 | main-is: README.lhs
25 | hs-source-dirs: .
26 |
27 | test-suite halide-tutorial05-test
28 | import: setup
29 | main-is: README.lhs
30 | type: exitcode-stdio-1.0
31 |
--------------------------------------------------------------------------------
/tutorials/05-Scheduling/test.hs:
--------------------------------------------------------------------------------
1 | #line 10 "all"
2 | {-# LANGUAGE OverloadedStrings, TypeApplications #-}
3 | import Test.Hspec
4 |
5 | import Language.Halide
6 | #line 21 "all"
7 | prepare :: IO (Expr Int32, Expr Int32, Func 'FuncTy 2 Int32)
8 | prepare = do
9 | x <- mkVar "x"
10 | y <- mkVar "y"
11 | gradient <- define "gradient" (x, y) $ x + y
12 | pure (x, y, gradient)
13 | #line 33 "all"
14 | getIterationOrder :: (KnownNat n, IsHalideType a) => [Int] -> Func t n a -> IO [[Int]]
15 | getIterationOrder shape f =
16 | fmap fst $
17 | collectIterationOrder (TraceStore ==) f $ do
18 | _ <- traceStores f
19 | realize f shape (void . pure)
20 | #line 53 "all"
21 | example01 :: Spec
22 | example01 = it "Has column-major ordering by default" $ do
23 | -- First we observe the default ordering
24 | (_, _, gradient) <- prepare
25 | -- Let's first make sure that the function actually computes what we want
26 | realize gradient [4, 4] peekToList
27 | `shouldReturn` [[i + j | j <- [0 .. 3]] | i <- [0 .. 3]]
28 | -- We check our assumption about the iteration order by building the expected
29 | -- iteration order using standard list comprehensions:
30 | getIterationOrder [4, 4] gradient
31 | `shouldReturn` [[i, j] | j <- [0 .. 3], i <- [0 .. 3]]
32 | #line 73 "all"
33 | main :: IO ()
34 | main = hspec $ specify "Tutorial 1" $ do
35 | x <- mkVar "x"
36 | y <- mkVar "y"
37 |
--------------------------------------------------------------------------------
/tutorials/README.md:
--------------------------------------------------------------------------------
1 | # Tutorials
2 |
3 | - [Getting started with `Func`s, `Var`s, and `Expr`s](./01-Basics)
4 | - [Inspecting the generated code](./03-Inspecting)
5 | - [Debugging with tracing, `printed`, and `printedWhen`](./04-Debugging)
6 | - [Vectorize, parallelize, unroll and tile your code](./05-Scheduling)
7 |
8 | For more, see the [tutorials on the C++ API of Halide](https://halide-lang.org/tutorials/).
9 |
--------------------------------------------------------------------------------