![\"Julia\"](\"images/logos/julia-logo.svg\"){kind=logo}
\n",
64 | " \n",
65 | " \n",
66 | " Example\n",
67 | " \n",
68 | "\n",
64 | " \n",
65 | " \n",
66 | " Example\n",
67 | " \n",
68 | "\n",
72 | " Oliver Schulz
\n",
73 | " \n",
74 | " Max Planck Institute for Physics
\n",
75 | " oschulz@mpp.mpg.de\n",
76 | " \n",
77 | "
\n",
79 | " ![](\"images/logos/mpg-logo.svg\"){kind=logo}
\n",
80 | " ![](\"images/logos/mpp-logo.svg\"){kind=logo}
\n",
81 | "
\n", 85 | " MPI for Physics, March 2021\n", 86 | "
" 87 | ] 88 | }, 89 | { 90 | "cell_type": "markdown", 91 | "metadata": {}, 92 | "source": [ 93 | "## Example: Trajectory of a ball" 94 | ] 95 | }, 96 | { 97 | "cell_type": "markdown", 98 | "metadata": {}, 99 | "source": [ 100 | "Let's analyze the slow-motion video of a bouncing ball:" 101 | ] 102 | }, 103 | { 104 | "cell_type": "code", 105 | "execution_count": null, 106 | "metadata": {}, 107 | "outputs": [], 108 | "source": [ 109 | "videofile = \"ball_throw.mp4\"" 110 | ] 111 | }, 112 | { 113 | "cell_type": "markdown", 114 | "metadata": {}, 115 | "source": [ 116 | "We need to load a few packages to deal with [videos](https://juliaio.github.io/VideoIO.jl), [images](https://github.com/JuliaImages/Images.jl), [colors](https://github.com/JuliaGraphics/Colors.jl) and [units](https://github.com/PainterQubits/Unitful.jl):" 117 | ] 118 | }, 119 | { 120 | "cell_type": "code", 121 | "execution_count": null, 122 | "metadata": {}, 123 | "outputs": [], 124 | "source": [ 125 | "using VideoIO, Images, Colors, Unitful" 126 | ] 127 | }, 128 | { 129 | "cell_type": "markdown", 130 | "metadata": {}, 131 | "source": [ 132 | "Let's get the duration of the video" 133 | ] 134 | }, 135 | { 136 | "cell_type": "code", 137 | "execution_count": null, 138 | "metadata": {}, 139 | "outputs": [], 140 | "source": [ 141 | "video_duration = VideoIO.get_duration(videofile) * u\"s\"" 142 | ] 143 | }, 144 | { 145 | "cell_type": "markdown", 146 | "metadata": {}, 147 | "source": [ 148 | "and open a video input stream:" 149 | ] 150 | }, 151 | { 152 | "cell_type": "code", 153 | "execution_count": null, 154 | "metadata": {}, 155 | "outputs": [], 156 | "source": [ 157 | "video_input = VideoIO.openvideo(videofile)" 158 | ] 159 | }, 160 | { 161 | "cell_type": "markdown", 162 | "metadata": {}, 163 | "source": [ 164 | "Get the video frame rate - `video_input.framerate` yields framerate * 1000 for some reason:" 165 | ] 166 | }, 167 | { 168 | "cell_type": "code", 169 | "execution_count": null, 170 | "metadata": {}, 171 | "outputs": [], 172 | "source": [ 173 | "fps = VideoIO.framerate(video_input)/1000 * u\"s^-1\"" 174 | ] 175 | }, 176 | { 177 | "cell_type": "markdown", 178 | "metadata": {}, 179 | "source": [ 180 | "Reading the whole video would use a lot of RAM. Let's write a function to read single frames at a given point in time:" 181 | ] 182 | }, 183 | { 184 | "cell_type": "code", 185 | "execution_count": null, 186 | "metadata": {}, 187 | "outputs": [], 188 | "source": [ 189 | "typeof(video_input)" 190 | ] 191 | }, 192 | { 193 | "cell_type": "code", 194 | "execution_count": null, 195 | "metadata": {}, 196 | "outputs": [], 197 | "source": [ 198 | "function read_frame(input::VideoIO.VideoReader, t::Number)\n", 199 | " t_in_s = float(ustrip(uconvert(u\"s\", t)))\n", 200 | " seek(input, t_in_s)\n", 201 | " read(input)\n", 202 | "end" 203 | ] 204 | }, 205 | { 206 | "cell_type": "markdown", 207 | "metadata": {}, 208 | "source": [ 209 | "Ok, now we can read a frame. We'll subsample it before display, to keep it small:" 210 | ] 211 | }, 212 | { 213 | "cell_type": "code", 214 | "execution_count": null, 215 | "metadata": {}, 216 | "outputs": [], 217 | "source": [ 218 | "frameimg = read_frame(video_input, 3u\"s\")\n", 219 | "frameimg[1:5:end,1:5:end]" 220 | ] 221 | }, 222 | { 223 | "cell_type": "markdown", 224 | "metadata": {}, 225 | "source": [ 226 | "Images in in Julia (more specifically, in [Images.jl](https://github.com/JuliaImages/Images.jl)) are simply arrays of color values:" 227 | ] 228 | }, 229 | { 230 | "cell_type": "code", 231 | "execution_count": null, 232 | "metadata": {}, 233 | "outputs": [], 234 | "source": [ 235 | "typeof(frameimg)" 236 | ] 237 | }, 238 | { 239 | "cell_type": "markdown", 240 | "metadata": {}, 241 | "source": [ 242 | "But oops, the video was recorded in portrait mode, so it appears rotated. Let's transpose the image array - this will also result in a mirrored image, but that doesn't matter here:" 243 | ] 244 | }, 245 | { 246 | "cell_type": "code", 247 | "execution_count": null, 248 | "metadata": {}, 249 | "outputs": [], 250 | "source": [ 251 | "frameimg[1:5:end,1:5:end]'" 252 | ] 253 | }, 254 | { 255 | "cell_type": "markdown", 256 | "metadata": {}, 257 | "source": [ 258 | "Nice, now let's load the plotting package Plots.jl" 259 | ] 260 | }, 261 | { 262 | "cell_type": "code", 263 | "execution_count": null, 264 | "metadata": {}, 265 | "outputs": [], 266 | "source": [ 267 | "using Plots; gr(format = :png)" 268 | ] 269 | }, 270 | { 271 | "cell_type": "markdown", 272 | "metadata": {}, 273 | "source": [ 274 | "and plot a frame every second:" 275 | ] 276 | }, 277 | { 278 | "cell_type": "code", 279 | "execution_count": null, 280 | "metadata": {}, 281 | "outputs": [], 282 | "source": [ 283 | "plot(plot.(broadcast(\n", 284 | " (input, t) -> read_frame(input, t)[1:5:end,1:5:end]',\n", 285 | " Ref(video_input),\n", 286 | " 0u\"s\":1u\"s\":video_duration\n", 287 | "), axis=nothing)...)" 288 | ] 289 | }, 290 | { 291 | "cell_type": "markdown", 292 | "metadata": {}, 293 | "source": [ 294 | "To develop a method that detects the ball, we'll need a frame with and another frame without the ball.\n", 295 | "\n", 296 | "We'll also need image coordinates, so we'll use [Plots.jl](https://github.com/JuliaPlots/Plots.jl) to plot the frames with a coordinate system. Let's load Plots and select the [GR.jl](https://github.com/jheinen/GR.jl) backend, which create plots via the [GR Framework](https://gr-framework.org/):" 297 | ] 298 | }, 299 | { 300 | "cell_type": "markdown", 301 | "metadata": {}, 302 | "source": [ 303 | "We won't flip/transpose the images this time, so that we don't confuse the images axes later on:" 304 | ] 305 | }, 306 | { 307 | "cell_type": "code", 308 | "execution_count": null, 309 | "metadata": {}, 310 | "outputs": [], 311 | "source": [ 312 | "background_frame, ball_frame = read_frame.(Ref(video_input), [0, 3]u\"s\")\n", 313 | "\n", 314 | "plot(\n", 315 | " plot(background_frame, xlabel = \"j\", ylabel = \"i\"),\n", 316 | " plot(ball_frame, xlabel = \"j\", ylabel = \"i\"),\n", 317 | ")" 318 | ] 319 | }, 320 | { 321 | "cell_type": "markdown", 322 | "metadata": {}, 323 | "source": [ 324 | "Note that Plots.jl plots images with matrix-like row/column direction.\n", 325 | "\n", 326 | "Each frame image/array is a 1080 x 1920 matrix, with indices 1:1080 for the rows and 1:1920 for the columns:" 327 | ] 328 | }, 329 | { 330 | "cell_type": "code", 331 | "execution_count": null, 332 | "metadata": {}, 333 | "outputs": [], 334 | "source": [ 335 | "size(background_frame)" 336 | ] 337 | }, 338 | { 339 | "cell_type": "code", 340 | "execution_count": null, 341 | "metadata": {}, 342 | "outputs": [], 343 | "source": [ 344 | "axes(background_frame)" 345 | ] 346 | }, 347 | { 348 | "cell_type": "markdown", 349 | "metadata": {}, 350 | "source": [ 351 | "To find the ball in the video, we need it's color. Let's zoom into `ball_frame`:" 352 | ] 353 | }, 354 | { 355 | "cell_type": "code", 356 | "execution_count": null, 357 | "metadata": {}, 358 | "outputs": [], 359 | "source": [ 360 | "plot(ball_frame[180:260,90:170], ratio = 1)" 361 | ] 362 | }, 363 | { 364 | "cell_type": "markdown", 365 | "metadata": {}, 366 | "source": [ 367 | "And zoom in some more, until only ball color is left:" 368 | ] 369 | }, 370 | { 371 | "cell_type": "code", 372 | "execution_count": null, 373 | "metadata": {}, 374 | "outputs": [], 375 | "source": [ 376 | "plot(ball_frame[202:238,112:148], ratio = 1)" 377 | ] 378 | }, 379 | { 380 | "cell_type": "markdown", 381 | "metadata": {}, 382 | "source": [ 383 | "We want the average color in that image region. To calculate means, we need the [Statistics](https://docs.julialang.org/en/v1/stdlib/Statistics) package, which is part of the Julia standard library:" 384 | ] 385 | }, 386 | { 387 | "cell_type": "code", 388 | "execution_count": null, 389 | "metadata": {}, 390 | "outputs": [], 391 | "source": [ 392 | "using Statistics" 393 | ] 394 | }, 395 | { 396 | "cell_type": "markdown", 397 | "metadata": {}, 398 | "source": [ 399 | "Since images are arrays, we can simply use the function `mean` to get the average color. We convert the color to the HSV color space, since it should be easiest to locate the ball based on color:" 400 | ] 401 | }, 402 | { 403 | "cell_type": "code", 404 | "execution_count": null, 405 | "metadata": {}, 406 | "outputs": [], 407 | "source": [ 408 | "ball_color = HSV{Float32}(mean(ball_frame[205:235,115:145]))" 409 | ] 410 | }, 411 | { 412 | "cell_type": "markdown", 413 | "metadata": {}, 414 | "source": [ 415 | "We define the distance between two colors based on the difference in hue and saturation:" 416 | ] 417 | }, 418 | { 419 | "cell_type": "code", 420 | "execution_count": null, 421 | "metadata": {}, 422 | "outputs": [], 423 | "source": [ 424 | "function color_dist(a::Color, b::Color)\n", 425 | " @fastmath begin\n", 426 | " ca = convert(HSV, a)\n", 427 | " cb = convert(HSV, b)\n", 428 | " sqrt((Float32(ca.h - cb.h)/360)^2 + Float32(ca.s - cb.s)^2)\n", 429 | " end\n", 430 | "end" 431 | ] 432 | }, 433 | { 434 | "cell_type": "markdown", 435 | "metadata": {}, 436 | "source": [ 437 | "Using `color_dist`, we can define the difference between two frames:" 438 | ] 439 | }, 440 | { 441 | "cell_type": "code", 442 | "execution_count": null, 443 | "metadata": {}, 444 | "outputs": [], 445 | "source": [ 446 | "framediff(f::Function, frame::AbstractArray, ref_frame::AbstractArray, ref_color::Color) =\n", 447 | " f.(color_dist.(frame, ball_color) .- color_dist.(ref_frame, ball_color))\n", 448 | "\n", 449 | "framediff(frame::AbstractArray, ref_frame::AbstractArray, ref_color::Color) =\n", 450 | " framediff(identity, frame, ref_frame, ref_color)" 451 | ] 452 | }, 453 | { 454 | "cell_type": "markdown", 455 | "metadata": {}, 456 | "source": [ 457 | "Let's see how this performs:" 458 | ] 459 | }, 460 | { 461 | "cell_type": "code", 462 | "execution_count": null, 463 | "metadata": {}, 464 | "outputs": [], 465 | "source": [ 466 | "typeof(similar(background_frame))" 467 | ] 468 | }, 469 | { 470 | "cell_type": "code", 471 | "execution_count": null, 472 | "metadata": {}, 473 | "outputs": [], 474 | "source": [ 475 | "heatmap(framediff(ball_frame, background_frame, ball_color))" 476 | ] 477 | }, 478 | { 479 | "cell_type": "markdown", 480 | "metadata": {}, 481 | "source": [ 482 | "Not bad - looks like a threshold of -0.4 might be a good choice to separate pixels belonging to the ball from pixels belonging to the background:" 483 | ] 484 | }, 485 | { 486 | "cell_type": "code", 487 | "execution_count": null, 488 | "metadata": {}, 489 | "outputs": [], 490 | "source": [ 491 | "heatmap(framediff(x -> x < -0.4, ball_frame, background_frame, ball_color))" 492 | ] 493 | }, 494 | { 495 | "cell_type": "markdown", 496 | "metadata": {}, 497 | "source": [ 498 | "That looks like a clean cut. Now all we need to do is to process the whole video. We generate the pixel masks on the fly, to avoid storing the whole video in RAM. Let's define a function for this, in case it needs to be re-run this a different reference color or threshold. Also, let's use multi-threading to process video frames in parallel:" 499 | ] 500 | }, 501 | { 502 | "cell_type": "code", 503 | "execution_count": null, 504 | "metadata": {}, 505 | "outputs": [], 506 | "source": [ 507 | "using Base.Threads\n", 508 | "\n", 509 | "function process_video(input::VideoIO.VideoReader, bg_frame::AbstractMatrix, fg_color::Color, threshold::Real)\n", 510 | " seek(input, 0.0)\n", 511 | "\n", 512 | " first_frame = read(input)\n", 513 | " result = [framediff(x -> x < threshold, first_frame, bg_frame, fg_color)]\n", 514 | " \n", 515 | " result_lock = ReentrantLock()\n", 516 | "\n", 517 | " input_channel = Channel{Tuple{Int, typeof(first_frame)}}(nthreads(), spawn = true) do ch\n", 518 | " i = length(result)\n", 519 | " while !eof(input)\n", 520 | " i += 1\n", 521 | " push!(ch, (i, read(input)))\n", 522 | " end\n", 523 | " end \n", 524 | " \n", 525 | " @sync for _ in 1:nthreads()\n", 526 | " @Base.Threads.spawn for (i, frame) in input_channel\n", 527 | " r = framediff(x -> x < threshold, frame, bg_frame, fg_color)\n", 528 | " lock(result_lock) do\n", 529 | " nframes = max(length(result), i)\n", 530 | " resize!(result, nframes)\n", 531 | " result[i] = r\n", 532 | " end\n", 533 | " end\n", 534 | " end\n", 535 | " \n", 536 | " @assert all(isassigned.(Ref(result), eachindex(result)))\n", 537 | " \n", 538 | " result\n", 539 | "end" 540 | ] 541 | }, 542 | { 543 | "cell_type": "code", 544 | "execution_count": null, 545 | "metadata": {}, 546 | "outputs": [], 547 | "source": [ 548 | "diffvideo = @time process_video(video_input, background_frame, ball_color, -0.4)\n", 549 | "typeof(diffvideo), length(diffvideo)" 550 | ] 551 | }, 552 | { 553 | "cell_type": "code", 554 | "execution_count": null, 555 | "metadata": {}, 556 | "outputs": [], 557 | "source": [ 558 | "heatmap(diffvideo[50])" 559 | ] 560 | }, 561 | { 562 | "cell_type": "markdown", 563 | "metadata": {}, 564 | "source": [ 565 | "We interpret each difference frame as a matrix of weights (0 or 1) and estimate the position of the ball as the weighted mean of image coordinates/indices. [StaticArrays.jl](https://github.com/JuliaArrays/StaticArrays.jl) will come in handy here to handle vectors of fixed size that can be stack-allocated:" 566 | ] 567 | }, 568 | { 569 | "cell_type": "code", 570 | "execution_count": null, 571 | "metadata": {}, 572 | "outputs": [], 573 | "source": [ 574 | "using StaticArrays" 575 | ] 576 | }, 577 | { 578 | "cell_type": "code", 579 | "execution_count": null, 580 | "metadata": {}, 581 | "outputs": [], 582 | "source": [ 583 | "function mean_pos(W::AbstractArray{T,N}) where {T,N}\n", 584 | " U = float(T)\n", 585 | " R = SVector{N,U}\n", 586 | " sum_pos::R = zero(R)\n", 587 | " sum_w::U = zero(U)\n", 588 | " @inbounds for idx in CartesianIndices(W)\n", 589 | " w = W[idx]\n", 590 | " sum_pos += SVector(Tuple(idx)) * w\n", 591 | " sum_w += w\n", 592 | " end\n", 593 | " sum_pos / sum_w\n", 594 | "end" 595 | ] 596 | }, 597 | { 598 | "cell_type": "markdown", 599 | "metadata": {}, 600 | "source": [ 601 | "Let's see if the is fast enough, using [BenchmarkTools.jl](https://github.com/JuliaCI/BenchmarkTools.jl):" 602 | ] 603 | }, 604 | { 605 | "cell_type": "code", 606 | "execution_count": null, 607 | "metadata": {}, 608 | "outputs": [], 609 | "source": [ 610 | "using BenchmarkTools" 611 | ] 612 | }, 613 | { 614 | "cell_type": "code", 615 | "execution_count": null, 616 | "metadata": {}, 617 | "outputs": [], 618 | "source": [ 619 | "@benchmark mean_pos($diffvideo[1])" 620 | ] 621 | }, 622 | { 623 | "cell_type": "markdown", 624 | "metadata": {}, 625 | "source": [ 626 | "That should do, speed-wise!\n", 627 | "\n", 628 | "StaticArrays.jl allows us to define custom field-vector types, we'll need something to represent 2D x/y vectors:" 629 | ] 630 | }, 631 | { 632 | "cell_type": "code", 633 | "execution_count": null, 634 | "metadata": {}, 635 | "outputs": [], 636 | "source": [ 637 | "struct Vec2D{T} <: FieldVector{2,T}\n", 638 | " x::T\n", 639 | " y::T\n", 640 | "end" 641 | ] 642 | }, 643 | { 644 | "cell_type": "markdown", 645 | "metadata": {}, 646 | "source": [ 647 | "Now we can reconstruct the ball positions, as a vector of `Vec2D`:" 648 | ] 649 | }, 650 | { 651 | "cell_type": "code", 652 | "execution_count": null, 653 | "metadata": {}, 654 | "outputs": [], 655 | "source": [ 656 | "Vec2D.(mean_pos.(diffvideo))" 657 | ] 658 | }, 659 | { 660 | "cell_type": "markdown", 661 | "metadata": {}, 662 | "source": [ 663 | "However, we'll also frequently want to access all `x` and `y` fields as separate vectors. [StructArrays.jl](https://github.com/JuliaArrays/StructArrays.jl) allows us to store this data as a [Structure of Arrays](https://en.wikipedia.org/wiki/AoS_and_SoA), with both AoS and SoA semantics:" 664 | ] 665 | }, 666 | { 667 | "cell_type": "code", 668 | "execution_count": null, 669 | "metadata": {}, 670 | "outputs": [], 671 | "source": [ 672 | "using StructArrays" 673 | ] 674 | }, 675 | { 676 | "cell_type": "code", 677 | "execution_count": null, 678 | "metadata": {}, 679 | "outputs": [], 680 | "source": [ 681 | "PV = StructArray(Vec2D.(mean_pos.(diffvideo)))\n", 682 | "typeof(PV.x), typeof(PV.y), typeof(PV[1])" 683 | ] 684 | }, 685 | { 686 | "cell_type": "markdown", 687 | "metadata": {}, 688 | "source": [ 689 | "Did we reconstruct the ball positions correctly?" 690 | ] 691 | }, 692 | { 693 | "cell_type": "code", 694 | "execution_count": null, 695 | "metadata": {}, 696 | "outputs": [], 697 | "source": [ 698 | "plot(PV.x, PV.y, yflip = true)" 699 | ] 700 | }, 701 | { 702 | "cell_type": "markdown", 703 | "metadata": {}, 704 | "source": [ 705 | "That looks promising. We also need a time axis, though - let's use a [`TypedTables.Table`](https://github.com/JuliaData/TypedTables.jl) to put it all together:" 706 | ] 707 | }, 708 | { 709 | "cell_type": "code", 710 | "execution_count": null, 711 | "metadata": {}, 712 | "outputs": [], 713 | "source": [ 714 | "using TypedTables" 715 | ] 716 | }, 717 | { 718 | "cell_type": "code", 719 | "execution_count": null, 720 | "metadata": {}, 721 | "outputs": [], 722 | "source": [ 723 | "realtime_framerate = 240\n", 724 | "raw_data = Table(xy = PV, t = (eachindex(PV) .- firstindex(PV)) / realtime_framerate)" 725 | ] 726 | }, 727 | { 728 | "cell_type": "markdown", 729 | "metadata": {}, 730 | "source": [ 731 | "Note: A [`DataFrames.DataFrame`](https://github.com/JuliaData/DataFrames.jl) would also do, we choose `TypedTables.Table` here for type stability.\n", 732 | "\n", 733 | "Let's pull in [Interact.jl](https://github.com/JuliaGizmos/Interact.jl) for interactive data exploration. We'll also use [Printf](https://docs.julialang.org/en/v1/stdlib/Printf/) from the Julia standard library for number formatting." 734 | ] 735 | }, 736 | { 737 | "cell_type": "code", 738 | "execution_count": null, 739 | "metadata": {}, 740 | "outputs": [], 741 | "source": [ 742 | "using Interact, Printf" 743 | ] 744 | }, 745 | { 746 | "cell_type": "code", 747 | "execution_count": null, 748 | "metadata": {}, 749 | "outputs": [], 750 | "source": [ 751 | "@manipulate for i in eachindex(diffvideo)\n", 752 | " white = eltype(background_frame)(colorant\"springgreen4\")\n", 753 | "\n", 754 | " plot(\n", 755 | " ((w,c) -> w ? white : c).(diffvideo[i], background_frame)',\n", 756 | " # ratio = 1\n", 757 | " )\n", 758 | "\n", 759 | " plot!(\n", 760 | " raw_data.xy.x, raw_data.xy.y,\n", 761 | " yflip = true, color = :blue, label = \"trajectory\"\n", 762 | " )\n", 763 | "\n", 764 | " scatter!(\n", 765 | " [raw_data.xy.x[i]], [raw_data.xy.y[i]],\n", 766 | " marker = (:xcross, :red),\n", 767 | " label = (@sprintf \"%.2f s\" raw_data.t[i])\n", 768 | " )\n", 769 | "end" 770 | ] 771 | }, 772 | { 773 | "cell_type": "markdown", 774 | "metadata": {}, 775 | "source": [ 776 | "In the following, we'll only analyse the fist arc of the trajectory:" 777 | ] 778 | }, 779 | { 780 | "cell_type": "code", 781 | "execution_count": null, 782 | "metadata": {}, 783 | "outputs": [], 784 | "source": [ 785 | "sel_idxs = 27:244" 786 | ] 787 | }, 788 | { 789 | "cell_type": "code", 790 | "execution_count": null, 791 | "metadata": {}, 792 | "outputs": [], 793 | "source": [ 794 | "# FileIO.save(\"background.png\", background_frame)" 795 | ] 796 | }, 797 | { 798 | "cell_type": "code", 799 | "execution_count": null, 800 | "metadata": {}, 801 | "outputs": [], 802 | "source": [ 803 | "raw_xy_shift = Vec2D(0, lastindex(axes(background_frame,2)))\n", 804 | "\n", 805 | "xy_cal_factor = 1.83 / 1559 * 1.72/1.82\n", 806 | "\n", 807 | "xy_cal = SMatrix{2,2}(\n", 808 | " xy_cal_factor, 0,\n", 809 | " 0, -xy_cal_factor\n", 810 | ")\n", 811 | "\n", 812 | "cal_data = Table(\n", 813 | " xy = StructArray(Vec2D.(Ref(xy_cal) .* (raw_data.xy .- Ref(raw_xy_shift)))),\n", 814 | " t = copy(collect(raw_data.t)),\n", 815 | ")\n", 816 | "\n", 817 | "# Fix missing frame:\n", 818 | "view(cal_data.t, 170:lastindex(cal_data.t)) .+= 1 / realtime_framerate\n", 819 | "\n", 820 | "sel_data = cal_data[sel_idxs]\n", 821 | "\n", 822 | "scatter(\n", 823 | " sel_data.xy.x, sel_data.xy.y,\n", 824 | " marker = (:circle, 2, :black, stroke(0)),\n", 825 | " xlabel = \"x [m]\", ylabel = \"y [m]\"\n", 826 | ")" 827 | ] 828 | }, 829 | { 830 | "cell_type": "code", 831 | "execution_count": null, 832 | "metadata": {}, 833 | "outputs": [], 834 | "source": [ 835 | "using CurveFit" 836 | ] 837 | }, 838 | { 839 | "cell_type": "code", 840 | "execution_count": null, 841 | "metadata": {}, 842 | "outputs": [], 843 | "source": [ 844 | "f = curve_fit(CurveFit.Polynomial, sel_data.t, sel_data.xy.y, 2)" 845 | ] 846 | }, 847 | { 848 | "cell_type": "code", 849 | "execution_count": null, 850 | "metadata": {}, 851 | "outputs": [], 852 | "source": [ 853 | "scatter(\n", 854 | " sel_data.xy.x, sel_data.xy.y,\n", 855 | " marker = (:circle, 2, :black, stroke(0)),\n", 856 | " xlabel = \"x [m]\", ylabel = \"y [m]\"\n", 857 | ")" 858 | ] 859 | }, 860 | { 861 | "cell_type": "code", 862 | "execution_count": null, 863 | "metadata": {}, 864 | "outputs": [], 865 | "source": [ 866 | "plot(sel_data.t, f.(sel_data.t), label = \"fit\")\n", 867 | "scatter!(\n", 868 | " sel_data.t, sel_data.xy.y,\n", 869 | " marker = (:circle, 2, :black, stroke(0)),\n", 870 | " xlabel = \"t [s]\", ylabel = \"y [m]\",\n", 871 | " label = \"data\"\n", 872 | ")" 873 | ] 874 | }, 875 | { 876 | "cell_type": "code", 877 | "execution_count": null, 878 | "metadata": {}, 879 | "outputs": [], 880 | "source": [ 881 | "g_curvefit = -2 * f.coeffs[3] * u\"m/s\"" 882 | ] 883 | }, 884 | { 885 | "cell_type": "markdown", 886 | "metadata": {}, 887 | "source": [ 888 | "### Bayesian inference of motion parameters" 889 | ] 890 | }, 891 | { 892 | "cell_type": "markdown", 893 | "metadata": {}, 894 | "source": [ 895 | "In the following, we'll need [LinearAlgebra](https://docs.julialang.org/en/v1/stdlib/LinearAlgebra/) from the Julia standard library, [OrdinaryDiffEq.jl](https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl) from the [Julia differential equations suite](https://docs.juliadiffeq.org/) and [ValueShapes.jl](https://github.com/oschulz/ValueShapes.jl):" 896 | ] 897 | }, 898 | { 899 | "cell_type": "code", 900 | "execution_count": null, 901 | "metadata": {}, 902 | "outputs": [], 903 | "source": [ 904 | "using LinearAlgebra, StaticArrays, OrdinaryDiffEq\n", 905 | "using Statistics, StatsBase, Distributions, ValueShapes, InverseFunctions, BAT" 906 | ] 907 | }, 908 | { 909 | "cell_type": "code", 910 | "execution_count": null, 911 | "metadata": {}, 912 | "outputs": [], 913 | "source": [ 914 | "function motion_eqn!(du::AbstractVector, u::AbstractVector, p::AbstractVector, t::Real)\n", 915 | " x, y, dx, dy = u\n", 916 | " ρ, A, m, g, C_w = p\n", 917 | "\n", 918 | " xy = SVector(x, y); d_xy = SVector(dx, dy)\n", 919 | " \n", 920 | " f_drag = - ρ * A * C_w * norm(d_xy) * d_xy / 2\n", 921 | " f_grav = m * SVector(zero(g), -g)\n", 922 | " dd_xy = (f_drag + f_grav) / m\n", 923 | "\n", 924 | " du .= (d_xy[1], d_xy[2], dd_xy[1], dd_xy[2])\n", 925 | " return du\n", 926 | "end\n", 927 | "\n", 928 | "function simulate_motion(v::NamedTuple, timesteps::AbstractVector = 0:0.05:1)\n", 929 | " u0 = [v.x, v.y, v.vx, v.vy]\n", 930 | " p = [v.ρ, v.A, v.m, v.g, v.C_w]\n", 931 | "\n", 932 | " odeprob = ODEProblem{true}(motion_eqn!, u0, (first(timesteps), last(timesteps)), p)\n", 933 | "\n", 934 | " sol = solve(odeprob, Tsit5(), saveat = timesteps)\n", 935 | " (x = sol[1,:], y = sol[2,:], t = timesteps)\n", 936 | "end\n", 937 | "\n", 938 | "\n", 939 | "likelihood = let data = (x = sel_data.xy.x, y = sel_data.xy.y, t = sel_data.t)\n", 940 | " v -> begin\n", 941 | " σ_x, σ_y = v.noise .^ 2\n", 942 | " sim_data = simulate_motion(v, data.t)\n", 943 | " (log = sum(logpdf.(Normal.(sim_data.x, σ_x), data.x)) + sum(logpdf.(Normal.(sim_data.y, σ_y), data.y)),)\n", 944 | " end\n", 945 | "end\n", 946 | "\n", 947 | "\n", 948 | "prior = NamedTupleDist(\n", 949 | " x = Normal(0, 1),\n", 950 | " y = Normal(1, 2),\n", 951 | " vx = Normal(1, 1),\n", 952 | " vy = Normal(2, 2),\n", 953 | " ρ = 1.209, # air density at 22°C and 1024 mbar, in kg/m^3\n", 954 | " A = pi * (60e-3/2)^2, # ball cross section area\n", 955 | " m = 7.1e-3, # mass of ball, in kg\n", 956 | " g = Weibull(250, 9.8), # 9.81\n", 957 | " C_w = Weibull(20, 0.5), # unitless, 0.47 would be a typical value for a sphere\n", 958 | " noise = [sqrt(0.01), sqrt(0.01)]\n", 959 | " #noise = [Weibull(1, 0.005), Weibull(1, 0.005)] # noise (stderr)\n", 960 | ")\n", 961 | "\n", 962 | "posterior = PosteriorDensity(likelihood, prior)\n", 963 | "\n", 964 | "logvalof(posterior)(rand(prior))" 965 | ] 966 | }, 967 | { 968 | "cell_type": "code", 969 | "execution_count": null, 970 | "metadata": {}, 971 | "outputs": [], 972 | "source": [ 973 | "@benchmark logvalof(posterior)(rand(prior))" 974 | ] 975 | }, 976 | { 977 | "cell_type": "code", 978 | "execution_count": null, 979 | "metadata": {}, 980 | "outputs": [], 981 | "source": [ 982 | "plt = scatter(\n", 983 | " sel_data.xy.x, sel_data.xy.y,\n", 984 | " marker = (:circle, 2, :black, stroke(0))\n", 985 | ")\n", 986 | "for xy in simulate_motion.(rand(prior, 100))\n", 987 | " plot!(xy.x, xy.y, color = :lightblue, legend = false)\n", 988 | "end\n", 989 | "plt" 990 | ] 991 | }, 992 | { 993 | "cell_type": "code", 994 | "execution_count": null, 995 | "metadata": {}, 996 | "outputs": [], 997 | "source": [ 998 | "v_guess = rand(prior)" 999 | ] 1000 | }, 1001 | { 1002 | "cell_type": "code", 1003 | "execution_count": null, 1004 | "metadata": {}, 1005 | "outputs": [], 1006 | "source": [ 1007 | "using ForwardDiff\n", 1008 | "let vs = varshape(prior)\n", 1009 | " ForwardDiff.gradient(v -> likelihood(vs(v)).log, inverse(vs)(v_guess))\n", 1010 | "end" 1011 | ] 1012 | }, 1013 | { 1014 | "cell_type": "markdown", 1015 | "metadata": {}, 1016 | "source": [ 1017 | "Simple maximum likelihood:" 1018 | ] 1019 | }, 1020 | { 1021 | "cell_type": "code", 1022 | "execution_count": null, 1023 | "metadata": {}, 1024 | "outputs": [], 1025 | "source": [ 1026 | "using Optim\n", 1027 | "let vs = varshape(prior)\n", 1028 | " r = Optim.optimize(v -> - likelihood(vs(v)).log, inverse(vs)(v_guess), Optim.LBFGS(); autodiff = :forward)\n", 1029 | " varshape(prior)(Optim.minimizer(r))\n", 1030 | "end" 1031 | ] 1032 | }, 1033 | { 1034 | "cell_type": "markdown", 1035 | "metadata": {}, 1036 | "source": [ 1037 | "Maximum posterior estimate:" 1038 | ] 1039 | }, 1040 | { 1041 | "cell_type": "code", 1042 | "execution_count": null, 1043 | "metadata": {}, 1044 | "outputs": [], 1045 | "source": [ 1046 | "findmode_ret = bat_findmode(posterior, MaxDensityLBFGS())\n", 1047 | "findmode_ret.info" 1048 | ] 1049 | }, 1050 | { 1051 | "cell_type": "code", 1052 | "execution_count": null, 1053 | "metadata": {}, 1054 | "outputs": [], 1055 | "source": [ 1056 | "mode_est = findmode_ret.result" 1057 | ] 1058 | }, 1059 | { 1060 | "cell_type": "code", 1061 | "execution_count": null, 1062 | "metadata": { 1063 | "scrolled": true 1064 | }, 1065 | "outputs": [], 1066 | "source": [ 1067 | "sim_data_bestfit = simulate_motion(mode_est, sel_data.t)\n", 1068 | "plot(\n", 1069 | " sim_data_bestfit.x, sim_data_bestfit.y,\n", 1070 | " #=marker = (:circle, 2, stroke(0)),=#\n", 1071 | " label = \"best fit\"\n", 1072 | ")\n", 1073 | "scatter!(\n", 1074 | " sel_data.xy.x, sel_data.xy.y,\n", 1075 | " marker = (:circle, 2, :black, stroke(0)),\n", 1076 | " xlabel = \"x [m]\", ylabel = \"y [m]\",\n", 1077 | " label = \"data\"\n", 1078 | ")" 1079 | ] 1080 | }, 1081 | { 1082 | "cell_type": "code", 1083 | "execution_count": null, 1084 | "metadata": {}, 1085 | "outputs": [], 1086 | "source": [ 1087 | "samling_output = @time bat_sample(posterior, MCMCSampling(mcalg = HamiltonianMC(), nchains = 4, nsteps = 10^4))\n", 1088 | "samples = samling_output.result;" 1089 | ] 1090 | }, 1091 | { 1092 | "cell_type": "code", 1093 | "execution_count": null, 1094 | "metadata": {}, 1095 | "outputs": [], 1096 | "source": [ 1097 | "plot(samples)" 1098 | ] 1099 | }, 1100 | { 1101 | "cell_type": "code", 1102 | "execution_count": null, 1103 | "metadata": {}, 1104 | "outputs": [], 1105 | "source": [ 1106 | "plot(samples, vsel = [:vx, :vy, :C_w, :g])" 1107 | ] 1108 | }, 1109 | { 1110 | "cell_type": "code", 1111 | "execution_count": null, 1112 | "metadata": {}, 1113 | "outputs": [], 1114 | "source": [ 1115 | "SampledDensity(posterior, samples)" 1116 | ] 1117 | }, 1118 | { 1119 | "cell_type": "code", 1120 | "execution_count": null, 1121 | "metadata": {}, 1122 | "outputs": [], 1123 | "source": [ 1124 | "mode_samples = bat_findmode(samples).result" 1125 | ] 1126 | }, 1127 | { 1128 | "cell_type": "code", 1129 | "execution_count": null, 1130 | "metadata": {}, 1131 | "outputs": [], 1132 | "source": [ 1133 | "mode_refined = bat_findmode(posterior, MaxDensityLBFGS(init = ExplicitInit([mode_samples]))).result" 1134 | ] 1135 | }, 1136 | { 1137 | "cell_type": "code", 1138 | "execution_count": null, 1139 | "metadata": {}, 1140 | "outputs": [], 1141 | "source": [ 1142 | "sim_data_bestfit = simulate_motion(mode_refined, sel_data.t)\n", 1143 | "plot(\n", 1144 | " sim_data_bestfit.x, sim_data_bestfit.y,\n", 1145 | " #=marker = (:circle, 2, stroke(0)),=#\n", 1146 | " label = \"best fit, refined\"\n", 1147 | ")\n", 1148 | "scatter!(\n", 1149 | " sel_data.xy.x, sel_data.xy.y,\n", 1150 | " marker = (:circle, 2, :black, stroke(0)),\n", 1151 | " xlabel = \"x [m]\", ylabel = \"y [m]\",\n", 1152 | " label = \"data\"\n", 1153 | ")" 1154 | ] 1155 | } 1156 | ], 1157 | "metadata": { 1158 | "@webio": { 1159 | "lastCommId": null, 1160 | "lastKernelId": null 1161 | }, 1162 | "celltoolbar": "Slideshow", 1163 | "hide_input": false, 1164 | "kernelspec": { 1165 | "display_name": "Julia 1.9.0", 1166 | "language": "julia", 1167 | "name": "julia-1.9" 1168 | }, 1169 | "language_info": { 1170 | "file_extension": ".jl", 1171 | "mimetype": "application/julia", 1172 | "name": "julia", 1173 | "version": "1.9.0" 1174 | }, 1175 | "livereveal": { 1176 | "start_slideshow_at": "selected", 1177 | "theme": "simple", 1178 | "transition": "none" 1179 | }, 1180 | "nbpresent": { 1181 | "slides": {}, 1182 | "themes": {} 1183 | }, 1184 | "toc": { 1185 | "base_numbering": 1, 1186 | "nav_menu": {}, 1187 | "number_sections": false, 1188 | "sideBar": true, 1189 | "skip_h1_title": false, 1190 | "title_cell": "Table of Contents", 1191 | "title_sidebar": "Contents", 1192 | "toc_cell": false, 1193 | "toc_position": {}, 1194 | "toc_section_display": true, 1195 | "toc_window_display": false 1196 | } 1197 | }, 1198 | "nbformat": 4, 1199 | "nbformat_minor": 2 1200 | } 1201 | -------------------------------------------------------------------------------- /julia-course-slides.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "metadata": { 7 | "slideshow": { 8 | "slide_type": "skip" 9 | } 10 | }, 11 | "outputs": [], 12 | "source": [ 13 | "# # Check multithreading config:\n", 14 | "Base.Threads.nthreads()" 15 | ] 16 | }, 17 | { 18 | "cell_type": "code", 19 | "execution_count": null, 20 | "metadata": { 21 | "slideshow": { 22 | "slide_type": "skip" 23 | } 24 | }, 25 | "outputs": [], 26 | "source": [ 27 | "# # Instantiate package environment for this notebook\n", 28 | "# using Pkg; pkg\"instantiate\"" 29 | ] 30 | }, 31 | { 32 | "cell_type": "code", 33 | "execution_count": null, 34 | "metadata": { 35 | "slideshow": { 36 | "slide_type": "skip" 37 | } 38 | }, 39 | "outputs": [], 40 | "source": [ 41 | "# # Check active package versions:\n", 42 | "# using Pkg; pkg\"status\"" 43 | ] 44 | }, 45 | { 46 | "cell_type": "markdown", 47 | "metadata": { 48 | "collapsed": true, 49 | "nbpresent": { 50 | "id": "7b7b849a-81ae-4985-92ec-78b66da75f5f" 51 | }, 52 | "slideshow": { 53 | "slide_type": "slide" 54 | } 55 | }, 56 | "source": [ 57 | "\n",
63 | " \n",
64 | "\n",
68 | " Oliver Schulz
\n",
69 | " \n",
70 | " Max Planck Institute for Physics
\n",
71 | " oschulz@mpp.mpg.de\n",
72 | " \n",
73 | "
\n",
75 | " \n",
76 | " \n",
77 | "
\n", 81 | " March 2023\n", 82 | "
" 83 | ] 84 | }, 85 | { 86 | "cell_type": "markdown", 87 | "metadata": { 88 | "slideshow": { 89 | "slide_type": "slide" 90 | } 91 | }, 92 | "source": [ 93 | "## Course material\n", 94 | "\n", 95 | "**Git-clone or download the course material:**\n", 96 | "\n", 97 | "## https://github.com/oschulz/julia-course" 98 | ] 99 | }, 100 | { 101 | "cell_type": "markdown", 102 | "metadata": { 103 | "slideshow": { 104 | "slide_type": "slide" 105 | } 106 | }, 107 | "source": [ 108 | "## Why Julia?" 109 | ] 110 | }, 111 | { 112 | "cell_type": "markdown", 113 | "metadata": { 114 | "slideshow": { 115 | "slide_type": "slide" 116 | } 117 | }, 118 | "source": [ 119 | "### Science needs code - but how to write it?\n", 120 | "\n", 121 | "* Choice of programming language(s) matter!\n", 122 | "\n", 123 | "* Need to balance:\n", 124 | " * Learning time\n", 125 | " * Productivity\n", 126 | " * Performance\n", 127 | "\n", 128 | "* Usually involves compromises" 129 | ] 130 | }, 131 | { 132 | "cell_type": "markdown", 133 | "metadata": { 134 | "slideshow": { 135 | "slide_type": "slide" 136 | } 137 | }, 138 | "source": [ 139 | "### Programming Language Options\n", 140 | "\n", 141 | "* C++:\n", 142 | " * Pro: Very fast (in expert hands)\n", 143 | " * Pro: Really cool new concepts (even literally) in C++11/14/17/...\n", 144 | " * Con: Complex, takes long time to learn and much longer to master\n", 145 | " * Con: Straightforward tasks often result in lengthy code\n", 146 | " * Con: No memory management (General protection faults) \n", 147 | " * Con: No universal package management\n", 148 | " * Con: Composability isn't great" 149 | ] 150 | }, 151 | { 152 | "cell_type": "markdown", 153 | "metadata": { 154 | "slideshow": { 155 | "slide_type": "slide" 156 | } 157 | }, 158 | "source": [ 159 | "### Programming Language Options\n", 160 | "\n", 161 | "* Python:\n", 162 | " * Pro: Broad user base, popular first programming language\n", 163 | " * Pro: Easy to learn, good standard library\n", 164 | " * Con: Can't write time-critical loops in Python, \n", 165 | " workarounds like Numba/Cython have\n", 166 | " [many limitations](http://www.stochasticlifestyle.com/why-numba-and-cython-are-not-substitutes-for-julia/), \n", 167 | " don't compose well\n", 168 | " * Con: Language itself fairly primitive, not very expressive\n", 169 | " * Con: Duck-Typing necessitates lots of test code\n", 170 | " * Con: No effective multi-threading\n", 171 | " * Con: Composability isn't great" 172 | ] 173 | }, 174 | { 175 | "cell_type": "markdown", 176 | "metadata": { 177 | "slideshow": { 178 | "slide_type": "slide" 179 | } 180 | }, 181 | "source": [ 182 | "### What else is there?\n", 183 | "\n", 184 | "* Fortran:\n", 185 | " * Pro: Math can be really fast\n", 186 | " * Con: Old language, few modern concepts\n", 187 | " * Con: Shrinking user base\n", 188 | " * Con: Composability isn't great\n", 189 | " * Do you *really* want to ...?\n", 190 | "\n", 191 | "\n", 192 | "* Scala, Go, Kotlin etc.:\n", 193 | " * Pro: Lots of individual strengths\n", 194 | " * Con: Math either fast *or* generic *or* or complicated\n", 195 | " * Con: Calling C, Fortran or Phython code often difficult\n", 196 | " * Con: Composability isn't great" 197 | ] 198 | }, 199 | { 200 | "cell_type": "markdown", 201 | "metadata": { 202 | "slideshow": { 203 | "slide_type": "slide" 204 | } 205 | }, 206 | "source": [ 207 | "### The 97 and the 3 Percent\n", 208 | "\n", 209 | "> We should forget about small efficiencies, say about 97% of the time: *premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%*.\n", 210 | "\n", 211 | "Donald E. Knuth\n", 212 | "\n", 213 | "* Some programming languages (e.g. Python) great for the 97% - but can't make the 3% fast.\n", 214 | "* Some other languages (e.g. C/C++, Fortran) can handle the 3% - but makes the 97% complicated." 215 | ] 216 | }, 217 | { 218 | "cell_type": "markdown", 219 | "metadata": { 220 | "slideshow": { 221 | "slide_type": "slide" 222 | } 223 | }, 224 | "source": [ 225 | "### The Two-language Problem\n", 226 | "\n", 227 | "* Common approach nowadays: \n", 228 | " Write time critical code in C/C++, rest in Python\n", 229 | "\n", 230 | "* Pro: End-user can code comfortably in Python, with good performance\n", 231 | "\n", 232 | "* Con: Complexity of C/C++ **plus** complexity of Python\n", 233 | "\n", 234 | "* Con: Need proficiency in **two** languages, barrier that prevents \n", 235 | " non-expert users from contributing to important parts of code\n", 236 | "\n", 237 | "* Con: Limits generic implementation of algorithms\n", 238 | "\n", 239 | "* Con: Severely limits metaprogramming, automatic differentiation, etc." 240 | ] 241 | }, 242 | { 243 | "cell_type": "markdown", 244 | "metadata": { 245 | "slideshow": { 246 | "slide_type": "slide" 247 | } 248 | }, 249 | "source": [ 250 | "## The Expression Problem\n", 251 | "\n", 252 | "> The expression problem is a new name for an old problem. The goal is to define a datatype by cases, where one can add new cases to the datatype and new functions over the datatype, without recompiling existing code, and while retaining static type safety (e.g., no casts).\n", 253 | "\n", 254 | "Philip Wadler\n", 255 | "\n", 256 | "* In other words: The capability to add both new subtypes and new functionality for a type defined in a package you don't own\n", 257 | "* Object oriented languages typically can't do this \n", 258 | " (Ruby has a dirty way, Scala a clean workaround)\n", 259 | "* If you have programming experience, you have felt this, even if you didn't name it\n", 260 | "* Result: Packages tend not to compose well\n" 261 | ] 262 | }, 263 | { 264 | "cell_type": "markdown", 265 | "metadata": { 266 | "slideshow": { 267 | "slide_type": "slide" 268 | } 269 | }, 270 | "source": [ 271 | "### We were looking for a language ...\n", 272 | "\n", 273 | "* as fast as C/C++/Fortran\n", 274 | "* as easy to learn and productive as Python\n", 275 | "* with a solution for the expression problem\n", 276 | "* with first class math support (vectors, matrices, etc.)\n", 277 | "* with true functional programming\n", 278 | "* with great Fortran/C/C++/Python integration\n", 279 | "* with true metaprogramming (like Lisp or Scala)\n", 280 | "* good at parallel and distributed programming\n", 281 | "* suitable for for interactive, small and large applications" 282 | ] 283 | }, 284 | { 285 | "cell_type": "markdown", 286 | "metadata": { 287 | "slideshow": { 288 | "slide_type": "slide" 289 | } 290 | }, 291 | "source": [ 292 | "### Julia\n", 293 | "\n", 294 | "* Designed for scientific/technical computing\n", 295 | "* Originated at MIT, first public version 2012\n", 296 | "* Covers the whole wish-list\n", 297 | "* Clear focus on user productivity and software quality\n", 298 | "* Rapid growth of user base and software packages\n", 299 | "* Current version: Julia v1.8 (v1.9 release candidate available)" 300 | ] 301 | }, 302 | { 303 | "cell_type": "markdown", 304 | "metadata": { 305 | "nbpresent": { 306 | "id": "b6763142-7446-43bf-a8a3-e7af798914d0" 307 | }, 308 | "slideshow": { 309 | "slide_type": "slide" 310 | } 311 | }, 312 | "source": [ 313 | "### Julia Language Properties\n", 314 | "\n", 315 | "* Fast: JAOT compilation to native CPU and GPU code\n", 316 | "* Multiple-dispatch (more powerful than object-oriented): \n", 317 | " solves the expression problem\n", 318 | "* Dynamically typed\n", 319 | "* Very powerful type system, types are first-class values\n", 320 | "* Functional programming and metaprogramming\n", 321 | "* First-class math support (like Fortran or Matlab)\n", 322 | "* ..." 323 | ] 324 | }, 325 | { 326 | "cell_type": "markdown", 327 | "metadata": { 328 | "slideshow": { 329 | "slide_type": "slide" 330 | } 331 | }, 332 | "source": [ 333 | "### Julia Language Properties, cont.\n", 334 | "\n", 335 | "* ...\n", 336 | "* Local and distributed code execution\n", 337 | "* State-of-the-art multi-threading: parallel code \n", 338 | " can call parallel code that can call parallel code, ..., \n", 339 | " without oversubscribing threads\n", 340 | "* Software package management: \n", 341 | " Trivial to create and install packages\n", 342 | "* Excellent REPL (console)\n", 343 | "* Easy to call Fortran, C/C++ and Python code" 344 | ] 345 | }, 346 | { 347 | "cell_type": "markdown", 348 | "metadata": { 349 | "slideshow": { 350 | "slide_type": "slide" 351 | } 352 | }, 353 | "source": [ 354 | "### Julia large-scale use case examples\n", 355 | "\n", 356 | "* Celeste: Variational Bayesian inference for astronomical images (doi:10.1214/19-AOAS1258), 1.54 petaflops using 1.3 million threads on 9,300 Knights Landing (KNL) nodes on Cori at NERSC\n", 357 | "\n", 358 | "* Clima: Full-earth climate simulation, https://clima.caltech.edu, large team, uses everything from MPI to GPUs\n", 359 | "\n", 360 | "* ...\n" 361 | ] 362 | }, 363 | { 364 | "cell_type": "markdown", 365 | "metadata": { 366 | "slideshow": { 367 | "slide_type": "slide" 368 | } 369 | }, 370 | "source": [ 371 | "### When (not) to use Julia\n", 372 | "\n", 373 | "* *Do* use Julia for computations, visualization, data processing ... pretty much anything scientific/technical\n", 374 | "\n", 375 | "* *Do not* use Julia for scripts what will only run for a second (code gen overhead), use Python or shell scripts\n", 376 | "\n", 377 | "* *Do not* use Julia for non-computing web apps, etc. (*at least not yet*), use Go or Node.js" 378 | ] 379 | }, 380 | { 381 | "cell_type": "markdown", 382 | "metadata": { 383 | "slideshow": { 384 | "slide_type": "slide" 385 | } 386 | }, 387 | "source": [ 388 | "## Julia 101" 389 | ] 390 | }, 391 | { 392 | "cell_type": "markdown", 393 | "metadata": { 394 | "slideshow": { 395 | "slide_type": "slide" 396 | } 397 | }, 398 | "source": [ 399 | "### Verbs and nouns - functions and types\n", 400 | "\n", 401 | "* Julia is not Java: Verbs aren't owned by nouns\n", 402 | "\n", 403 | "* Julia has: types, functions and methods\n", 404 | "\n", 405 | "* Methods belong to *functions*, not to types!" 406 | ] 407 | }, 408 | { 409 | "cell_type": "markdown", 410 | "metadata": { 411 | "slideshow": { 412 | "slide_type": "slide" 413 | } 414 | }, 415 | "source": [ 416 | "### Functions\n", 417 | "\n", 418 | "Short one-liner function:" 419 | ] 420 | }, 421 | { 422 | "cell_type": "code", 423 | "execution_count": null, 424 | "metadata": {}, 425 | "outputs": [], 426 | "source": [ 427 | "f(x) = x^2" 428 | ] 429 | }, 430 | { 431 | "cell_type": "code", 432 | "execution_count": null, 433 | "metadata": {}, 434 | "outputs": [], 435 | "source": [ 436 | "f(3)" 437 | ] 438 | }, 439 | { 440 | "cell_type": "markdown", 441 | "metadata": { 442 | "slideshow": { 443 | "slide_type": "slide" 444 | } 445 | }, 446 | "source": [ 447 | "Function that needs more than one line:" 448 | ] 449 | }, 450 | { 451 | "cell_type": "code", 452 | "execution_count": null, 453 | "metadata": {}, 454 | "outputs": [], 455 | "source": [ 456 | "function f(x)\n", 457 | " # ... something ...\n", 458 | " x^2\n", 459 | "end" 460 | ] 461 | }, 462 | { 463 | "cell_type": "markdown", 464 | "metadata": {}, 465 | "source": [ 466 | "is equivalent to" 467 | ] 468 | }, 469 | { 470 | "cell_type": "code", 471 | "execution_count": null, 472 | "metadata": {}, 473 | "outputs": [], 474 | "source": [ 475 | "function f(x)\n", 476 | " # ... something ...\n", 477 | " return x^2\n", 478 | "end" 479 | ] 480 | }, 481 | { 482 | "cell_type": "markdown", 483 | "metadata": {}, 484 | "source": [ 485 | "**Note:** `return` is optional, and often not used explicitly. Last expression in a function, block, etc. is automatically returned (like in Mathematica)." 486 | ] 487 | }, 488 | { 489 | "cell_type": "markdown", 490 | "metadata": { 491 | "slideshow": { 492 | "slide_type": "slide" 493 | } 494 | }, 495 | "source": [ 496 | "### Types\n", 497 | "\n", 498 | "An abstract type, must be empty:\n", 499 | "\n", 500 | "```julia\n", 501 | "abstract type MySuperType end\n", 502 | "```\n", 503 | "\n", 504 | "An immutable type, value of `i` can't change:\n", 505 | "\n", 506 | "```julia\n", 507 | "struct MySubType <: MySuperType\n", 508 | " i::Int\n", 509 | "end\n", 510 | "```\n", 511 | "\n", 512 | "A mutable type, value of `i` can change:\n", 513 | "\n", 514 | "```julia\n", 515 | "mutable struct MyMutableSubType <: MySuperType\n", 516 | " i::Int\n", 517 | "end\n", 518 | "```" 519 | ] 520 | }, 521 | { 522 | "cell_type": "markdown", 523 | "metadata": { 524 | "slideshow": { 525 | "slide_type": "slide" 526 | } 527 | }, 528 | "source": [ 529 | "### Type parameters\n", 530 | "\n", 531 | "Julia has a powerful parametric type system, based on set theory:\n", 532 | "\n", 533 | "```julia\n", 534 | "struct MyRealArray{T<:Real,N} <: AbstractArray{T,N}\n", 535 | " # ...\n", 536 | "end\n", 537 | "```\n", 538 | "\n", 539 | "defines an array type with real-valued elements.\n", 540 | "\n", 541 | "```julia\n", 542 | "foo(A::AbstractArray{<:Real}) = do_something_with(A)\n", 543 | "```\n", 544 | "\n", 545 | "is equivalent to\n", 546 | "\n", 547 | "```julia\n", 548 | "bar(A::AbstractArray{T,N}) where {T<:Real,N} = do_something_with(A)\n", 549 | "```\n", 550 | "\n", 551 | "defines a function covariant in the element type of `A`. Can also be contravariant:\n", 552 | "\n", 553 | "```julia\n", 554 | "baz(A::AbstractArray{>:Real}) = do_something_with(A)\n", 555 | "```" 556 | ] 557 | }, 558 | { 559 | "cell_type": "markdown", 560 | "metadata": { 561 | "slideshow": { 562 | "slide_type": "slide" 563 | } 564 | }, 565 | "source": [ 566 | "### Type aliases and union types\n", 567 | "\n", 568 | "Type aliases are just const values:\n", 569 | "\n", 570 | "```julia\n", 571 | "const Abstract2DArray{T} = AbstractArray{T,2}\n", 572 | "rand(2, 2) isa Abstract2DArray == true\n", 573 | "```\n", 574 | "\n", 575 | "Type unions are unions of set of types.\n", 576 | "\n", 577 | "```julia\n", 578 | "const RealVecOrMat{T} where {T<:Real} = Union{AbstractArray{T,1}, AbstractArray{T,2}}\n", 579 | "```\n", 580 | "\n", 581 | "is the union of a 1D and 2D array types with real-valued elements." 582 | ] 583 | }, 584 | { 585 | "cell_type": "markdown", 586 | "metadata": { 587 | "slideshow": { 588 | "slide_type": "slide" 589 | } 590 | }, 591 | "source": [ 592 | "### Syntax: Variables\n", 593 | "\n", 594 | "```julia\n", 595 | "# Global variables:\n", 596 | "const a = 42\n", 597 | "b = 24\n", 598 | "\n", 599 | "function foo(x)\n", 600 | " # Local variables:\n", 601 | "\n", 602 | " c = a * x\n", 603 | " d = b * x # Avoid, type of b can change!\n", 604 | " #...\n", 605 | "end\n", 606 | "```" 607 | ] 608 | }, 609 | { 610 | "cell_type": "markdown", 611 | "metadata": { 612 | "slideshow": { 613 | "slide_type": "slide" 614 | } 615 | }, 616 | "source": [ 617 | "### Loops\n", 618 | "\n", 619 | "For loop:\n", 620 | "\n", 621 | "```julia\n", 622 | "for i in something_iterable\n", 623 | " # ...\n", 624 | "end\n", 625 | "```\n", 626 | "\n", 627 | "`something_iterable` can be a range, an array, anything that implements the Julia [iterator API](https://docs.julialang.org/en/v1/manual/interfaces/).\n", 628 | "\n", 629 | "While loop:\n", 630 | "\n", 631 | "```julia\n", 632 | "while condition\n", 633 | " # do something\n", 634 | "end\n", 635 | "```" 636 | ] 637 | }, 638 | { 639 | "cell_type": "markdown", 640 | "metadata": { 641 | "slideshow": { 642 | "slide_type": "slide" 643 | } 644 | }, 645 | "source": [ 646 | "### Control flow\n", 647 | "\n", 648 | "If-else, evaluate only one branch:\n", 649 | "\n", 650 | "```julia\n", 651 | "if condition\n", 652 | " # do something\n", 653 | "elseif condition\n", 654 | " # do something else\n", 655 | "else\n", 656 | " # or something different\n", 657 | "end\n", 658 | "```\n", 659 | "\n", 660 | "Ternary operator, evaluate only one branch:\n", 661 | "\n", 662 | "```julia\n", 663 | "condition ? result_if_true : result_if_false\n", 664 | "```\n", 665 | "\n", 666 | "`ifelse`, evaluate both results but return only one:\n", 667 | "\n", 668 | "```julia\n", 669 | "ifelse(condition, result_if_true, result_if_false)\n", 670 | "```" 671 | ] 672 | }, 673 | { 674 | "cell_type": "markdown", 675 | "metadata": { 676 | "slideshow": { 677 | "slide_type": "slide" 678 | } 679 | }, 680 | "source": [ 681 | "## Blocks and scoping\n", 682 | "\n", 683 | "Begin/end-block:\n", 684 | "\n", 685 | "```julia\n", 686 | "begin\n", 687 | " # *Not* a new scope in here\n", 688 | " # ...\n", 689 | "end\n", 690 | "```\n", 691 | "\n", 692 | "Let-block:\n", 693 | "\n", 694 | "```julia\n", 695 | "b = 24\n", 696 | "\n", 697 | "let my_b = b\n", 698 | " # New scope in here.\n", 699 | " # If b is bound to new value, my_b won't change.\n", 700 | " # ...\n", 701 | "end\n", 702 | "```" 703 | ] 704 | }, 705 | { 706 | "cell_type": "markdown", 707 | "metadata": { 708 | "slideshow": { 709 | "slide_type": "slide" 710 | } 711 | }, 712 | "source": [ 713 | "## Arrays\n", 714 | "\n", 715 | "Vectors:\n", 716 | "\n", 717 | "```julia\n", 718 | "v = [1, 2, 3]\n", 719 | "\n", 720 | "v = rand(5)\n", 721 | "```\n", 722 | "\n", 723 | "Matrices:\n", 724 | "\n", 725 | "```julia\n", 726 | "A = [1 2; 3 4]\n", 727 | "\n", 728 | "A = rand(4, 5)\n", 729 | "```\n", 730 | "\n", 731 | "* Column-first memory layout!\n", 732 | "\n", 733 | "* Almost anything array-like is subtype of `AbstractArray`." 734 | ] 735 | }, 736 | { 737 | "cell_type": "markdown", 738 | "metadata": { 739 | "slideshow": { 740 | "slide_type": "slide" 741 | } 742 | }, 743 | "source": [ 744 | "### Array indexing\n", 745 | "\n", 746 | "Get `i`-th element of vector `v`:\n", 747 | "\n", 748 | "```julia\n", 749 | "v[i]\n", 750 | "```\n", 751 | "\n", 752 | "Most higher-dimensional array types support cartesian and linear indexing (usually faster):\n", 753 | "\n", 754 | "```julia\n", 755 | "A[i, j]\n", 756 | "A[lin_idx] \n", 757 | "```\n", 758 | "\n", 759 | "Use `eachindex(A)` to get indices of best type for given `A` (usually linear).\n", 760 | "\n", 761 | "\n", 762 | "In Julia, anything array-like can usually be an index as well\n", 763 | "\n", 764 | "```julia\n", 765 | "A[2:3, [1, 4, 5]]\n", 766 | "```" 767 | ] 768 | }, 769 | { 770 | "cell_type": "markdown", 771 | "metadata": { 772 | "slideshow": { 773 | "slide_type": "slide" 774 | } 775 | }, 776 | "source": [ 777 | "### Array comprehension and generators\n", 778 | "\n", 779 | "Returns an array:\n", 780 | "\n", 781 | "```\n", 782 | "[f(x) for x in some_collection]\n", 783 | "```\n", 784 | "\n", 785 | "Returns an iterable generator:\n", 786 | "\n", 787 | "```\n", 788 | "(f(x) for x in some_collection)\n", 789 | "```" 790 | ] 791 | }, 792 | { 793 | "cell_type": "markdown", 794 | "metadata": { 795 | "slideshow": { 796 | "slide_type": "slide" 797 | } 798 | }, 799 | "source": [ 800 | "### Hello World (and more) in Julia" 801 | ] 802 | }, 803 | { 804 | "cell_type": "code", 805 | "execution_count": null, 806 | "metadata": {}, 807 | "outputs": [], 808 | "source": [ 809 | "println(\"Hello, World!\")" 810 | ] 811 | }, 812 | { 813 | "cell_type": "markdown", 814 | "metadata": { 815 | "slideshow": { 816 | "slide_type": "fragment" 817 | } 818 | }, 819 | "source": [ 820 | "Let's define a function" 821 | ] 822 | }, 823 | { 824 | "cell_type": "code", 825 | "execution_count": null, 826 | "metadata": {}, 827 | "outputs": [], 828 | "source": [ 829 | "f(x, y) = x * y\n", 830 | "f(20, 2.1)" 831 | ] 832 | }, 833 | { 834 | "cell_type": "markdown", 835 | "metadata": {}, 836 | "source": [ 837 | "Multiplication is also defined for vectors, so this works, too:" 838 | ] 839 | }, 840 | { 841 | "cell_type": "code", 842 | "execution_count": null, 843 | "metadata": {}, 844 | "outputs": [], 845 | "source": [ 846 | "f(4.2, [1, 2, 3, 4])" 847 | ] 848 | }, 849 | { 850 | "cell_type": "markdown", 851 | "metadata": { 852 | "slideshow": { 853 | "slide_type": "slide" 854 | } 855 | }, 856 | "source": [ 857 | "### Let's Look Under the Hood" 858 | ] 859 | }, 860 | { 861 | "cell_type": "code", 862 | "execution_count": null, 863 | "metadata": {}, 864 | "outputs": [], 865 | "source": [ 866 | "@code_llvm debuginfo=:none f(20, 2.1)" 867 | ] 868 | }, 869 | { 870 | "cell_type": "code", 871 | "execution_count": null, 872 | "metadata": {}, 873 | "outputs": [], 874 | "source": [ 875 | "@code_native debuginfo=:none f(20, 2.1)" 876 | ] 877 | }, 878 | { 879 | "cell_type": "markdown", 880 | "metadata": { 881 | "slideshow": { 882 | "slide_type": "slide" 883 | } 884 | }, 885 | "source": [ 886 | "### Multiple Dispatch" 887 | ] 888 | }, 889 | { 890 | "cell_type": "code", 891 | "execution_count": null, 892 | "metadata": {}, 893 | "outputs": [], 894 | "source": [ 895 | "foo(x::Integer, y::Number) = x * y\n", 896 | "foo(x::Integer, y::AbstractString) = join(fill(y, x))" 897 | ] 898 | }, 899 | { 900 | "cell_type": "code", 901 | "execution_count": null, 902 | "metadata": {}, 903 | "outputs": [], 904 | "source": [ 905 | "foo(3, 4)" 906 | ] 907 | }, 908 | { 909 | "cell_type": "code", 910 | "execution_count": null, 911 | "metadata": {}, 912 | "outputs": [], 913 | "source": [ 914 | "foo(3, \"abc\")" 915 | ] 916 | }, 917 | { 918 | "cell_type": "code", 919 | "execution_count": null, 920 | "metadata": {}, 921 | "outputs": [], 922 | "source": [ 923 | "foo(4.5, 3)" 924 | ] 925 | }, 926 | { 927 | "cell_type": "markdown", 928 | "metadata": { 929 | "slideshow": { 930 | "slide_type": "slide" 931 | } 932 | }, 933 | "source": [ 934 | "### Functional Programming" 935 | ] 936 | }, 937 | { 938 | "cell_type": "code", 939 | "execution_count": null, 940 | "metadata": {}, 941 | "outputs": [], 942 | "source": [ 943 | "A = rand(10)" 944 | ] 945 | }, 946 | { 947 | "cell_type": "code", 948 | "execution_count": null, 949 | "metadata": {}, 950 | "outputs": [], 951 | "source": [ 952 | "idxs = findall(x -> 0.2 < x < 0.6, A)" 953 | ] 954 | }, 955 | { 956 | "cell_type": "code", 957 | "execution_count": null, 958 | "metadata": {}, 959 | "outputs": [], 960 | "source": [ 961 | "A[idxs]" 962 | ] 963 | }, 964 | { 965 | "cell_type": "markdown", 966 | "metadata": { 967 | "slideshow": { 968 | "slide_type": "slide" 969 | } 970 | }, 971 | "source": [ 972 | "Even types are first-class values:" 973 | ] 974 | }, 975 | { 976 | "cell_type": "code", 977 | "execution_count": null, 978 | "metadata": {}, 979 | "outputs": [], 980 | "source": [ 981 | "mytype = Number" 982 | ] 983 | }, 984 | { 985 | "cell_type": "code", 986 | "execution_count": null, 987 | "metadata": {}, 988 | "outputs": [], 989 | "source": [ 990 | "subtypes(mytype)" 991 | ] 992 | }, 993 | { 994 | "cell_type": "markdown", 995 | "metadata": {}, 996 | "source": [ 997 | "Julia type hierarchy extends all the way down to primitive types:" 998 | ] 999 | }, 1000 | { 1001 | "cell_type": "code", 1002 | "execution_count": null, 1003 | "metadata": {}, 1004 | "outputs": [], 1005 | "source": [ 1006 | "Float64 <: AbstractFloat <: Real <: Number <: Any" 1007 | ] 1008 | }, 1009 | { 1010 | "cell_type": "markdown", 1011 | "metadata": { 1012 | "slideshow": { 1013 | "slide_type": "slide" 1014 | } 1015 | }, 1016 | "source": [ 1017 | "### Broadcasting" 1018 | ] 1019 | }, 1020 | { 1021 | "cell_type": "code", 1022 | "execution_count": null, 1023 | "metadata": {}, 1024 | "outputs": [], 1025 | "source": [ 1026 | "A = [1.1, 2.2, 3.3]\n", 1027 | "B = [4.4, 5.5, 6.6]\n", 1028 | "broadcast((x, y) -> (x + y)^2, A, B)" 1029 | ] 1030 | }, 1031 | { 1032 | "cell_type": "markdown", 1033 | "metadata": {}, 1034 | "source": [ 1035 | "Shorter broadcast syntax:" 1036 | ] 1037 | }, 1038 | { 1039 | "cell_type": "code", 1040 | "execution_count": null, 1041 | "metadata": {}, 1042 | "outputs": [], 1043 | "source": [ 1044 | "(A .+ B) .^ 2" 1045 | ] 1046 | }, 1047 | { 1048 | "cell_type": "markdown", 1049 | "metadata": { 1050 | "slideshow": { 1051 | "slide_type": "slide" 1052 | } 1053 | }, 1054 | "source": [ 1055 | "#### Loop Fusion and SIMD Vectorization" 1056 | ] 1057 | }, 1058 | { 1059 | "cell_type": "code", 1060 | "execution_count": null, 1061 | "metadata": {}, 1062 | "outputs": [], 1063 | "source": [ 1064 | "foo(X, Y) = (X .+ Y) .^ 2\n", 1065 | "@code_llvm raw=false debuginfo=:none foo(A, B)" 1066 | ] 1067 | }, 1068 | { 1069 | "cell_type": "code", 1070 | "execution_count": null, 1071 | "metadata": { 1072 | "slideshow": { 1073 | "slide_type": "slide" 1074 | } 1075 | }, 1076 | "outputs": [], 1077 | "source": [ 1078 | "@code_native debuginfo=:none foo(A, B)" 1079 | ] 1080 | }, 1081 | { 1082 | "cell_type": "markdown", 1083 | "metadata": { 1084 | "slideshow": { 1085 | "slide_type": "slide" 1086 | } 1087 | }, 1088 | "source": [ 1089 | "### Package management\n", 1090 | "\n", 1091 | "* Julia probably has the best package management to date\n", 1092 | "\n", 1093 | "* Press \"]\" to enter package management console\n", 1094 | "\n", 1095 | "* Typically `add PACKAGE_NAME` is sufficient, can also do `add PACKAGE_NAME@VERSION`\n", 1096 | "\n", 1097 | "* To get an unreleased version, use `add PACKAGE_NAME#BRANCH_NAME`\n", 1098 | "\n", 1099 | "* Easy to start modifying a package via `dev PACKAGE_NAME`\n", 1100 | "\n", 1101 | "* Multiple package versions can be installed, selection via [Pkg.jl environments](https://julialang.github.io/Pkg.jl/v1/environments).\n", 1102 | "\n", 1103 | "* Also useful: `julia> using Pkg; pkg\"