├── .Rbuildignore ├── .gitignore ├── NEWS ├── NAMESPACE ├── man ├── bisectr.Rd ├── bisect_return_interactive.Rd ├── bisect_load_all.Rd ├── bisect_source.Rd ├── bisect_require.Rd ├── bisect_install.Rd └── bisect_runtest.Rd ├── bisectr.Rproj ├── DESCRIPTION ├── R └── bisect.r └── README.md /.Rbuildignore: -------------------------------------------------------------------------------- 1 | ^.*\.Rproj$ 2 | ^\.Rproj\.user$ 3 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .Rproj.user 2 | .Rhistory 3 | .RData 4 | -------------------------------------------------------------------------------- /NEWS: -------------------------------------------------------------------------------- 1 | bisectr 0.1.0 2 | ------------- 3 | 4 | * Add bisect_source() function. 5 | -------------------------------------------------------------------------------- /NAMESPACE: -------------------------------------------------------------------------------- 1 | # Generated by roxygen2 (4.1.0): do not edit by hand 2 | 3 | export(bisect_install) 4 | export(bisect_load_all) 5 | export(bisect_require) 6 | export(bisect_return_interactive) 7 | export(bisect_runtest) 8 | export(bisect_source) 9 | importFrom(devtools,dev_mode) 10 | importFrom(devtools,install) 11 | importFrom(devtools,load_all) 12 | -------------------------------------------------------------------------------- /man/bisectr.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2 (4.1.0): do not edit by hand 2 | % Please edit documentation in R/bisect.r 3 | \docType{package} 4 | \name{bisectr} 5 | \alias{bisectr} 6 | \alias{bisectr-package} 7 | \alias{package-bisectr} 8 | \title{bisectr package} 9 | \description{ 10 | This package is used for creating test scripts to find bad commits with 11 | git bisect. 12 | For example test scripts, see \url{https://github.com/wch/bisectr}. 13 | } 14 | 15 | -------------------------------------------------------------------------------- /bisectr.Rproj: -------------------------------------------------------------------------------- 1 | Version: 1.0 2 | 3 | RestoreWorkspace: No 4 | SaveWorkspace: No 5 | AlwaysSaveHistory: Default 6 | 7 | EnableCodeIndexing: Yes 8 | UseSpacesForTab: Yes 9 | NumSpacesForTab: 2 10 | Encoding: UTF-8 11 | 12 | RnwWeave: Sweave 13 | LaTeX: pdfLaTeX 14 | 15 | AutoAppendNewline: Yes 16 | StripTrailingWhitespace: Yes 17 | 18 | BuildType: Package 19 | PackageUseDevtools: Yes 20 | PackageInstallArgs: --no-multiarch --with-keep.source 21 | PackageRoxygenize: rd,collate,namespace 22 | -------------------------------------------------------------------------------- /DESCRIPTION: -------------------------------------------------------------------------------- 1 | Package: bisectr 2 | Title: Tools to Find Bad Commits with 'git bisect' 3 | Version: 0.2.0 4 | Author: Winston Chang 5 | Maintainer: Winston Chang 6 | Description: Tools to find bad commits with git bisect. See 7 | https://github.com/wch/bisectr for examples and test script templates. 8 | URL: https://github.com/wch/bisectr 9 | Depends: 10 | R (>= 2.14) 11 | Imports: 12 | devtools (>= 1.7.0) 13 | License: GPL-2 14 | Collate: 15 | 'bisect.r' 16 | -------------------------------------------------------------------------------- /man/bisect_return_interactive.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2 (4.1.0): do not edit by hand 2 | % Please edit documentation in R/bisect.r 3 | \name{bisect_return_interactive} 4 | \alias{bisect_return_interactive} 5 | \title{Prompt the user for an interactive good/bad/skip response and return 6 | the appropriate value (to be passed to \code{bisect_runtest}).} 7 | \usage{ 8 | bisect_return_interactive() 9 | } 10 | \description{ 11 | Prompt the user for an interactive good/bad/skip response and return 12 | the appropriate value (to be passed to \code{bisect_runtest}). 13 | } 14 | \seealso{ 15 | \code{\link{bisect_runtest}} 16 | 17 | \code{\link{bisect_load_all}} 18 | 19 | \code{\link{bisect_install}} 20 | 21 | \code{\link{bisect_source}} 22 | } 23 | 24 | -------------------------------------------------------------------------------- /man/bisect_load_all.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2 (4.1.0): do not edit by hand 2 | % Please edit documentation in R/bisect.r 3 | \name{bisect_load_all} 4 | \alias{bisect_load_all} 5 | \title{Like \code{load_all}, but for bisect tests.} 6 | \usage{ 7 | bisect_load_all(pkgdir = ".", on_error = "skip") 8 | } 9 | \arguments{ 10 | \item{pkgdir}{The directory to load from} 11 | 12 | \item{on_error}{What to do if loading throws an error (default is to mark this 13 | commit as "skip")} 14 | } 15 | \description{ 16 | If the package fails to load, the default is to mark this commit as skip. 17 | } 18 | \seealso{ 19 | \code{\link{bisect_source}} 20 | 21 | \code{\link{bisect_install}} 22 | 23 | \code{\link{bisect_runtest}} 24 | 25 | \code{\link{bisect_return_interactive}} 26 | } 27 | 28 | -------------------------------------------------------------------------------- /man/bisect_source.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2 (4.1.0): do not edit by hand 2 | % Please edit documentation in R/bisect.r 3 | \name{bisect_source} 4 | \alias{bisect_source} 5 | \title{Like \code{source}, but for bisect tests.} 6 | \usage{ 7 | bisect_source(file, ..., on_error = "skip") 8 | } 9 | \arguments{ 10 | \item{file}{The file to load} 11 | 12 | \item{...}{Other arguments to pass to \code{\link{source}}} 13 | 14 | \item{on_error}{What to do if loading throws an error (default is to mark this 15 | commit as "skip")} 16 | } 17 | \description{ 18 | If the file fails to load, the default is mark this commit as skip. 19 | } 20 | \seealso{ 21 | \code{\link{source}} 22 | 23 | \code{\link{bisect_load_all}} 24 | 25 | \code{\link{bisect_install}} 26 | 27 | \code{\link{bisect_runtest}} 28 | 29 | \code{\link{bisect_return_interactive}} 30 | } 31 | 32 | -------------------------------------------------------------------------------- /man/bisect_require.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2 (4.1.0): do not edit by hand 2 | % Please edit documentation in R/bisect.r 3 | \name{bisect_require} 4 | \alias{bisect_require} 5 | \title{Load a package like \code{require()}, for bisect tests.} 6 | \usage{ 7 | bisect_require(package, on_fail = "skip") 8 | } 9 | \arguments{ 10 | \item{package}{Name of package} 11 | 12 | \item{on_fail}{What to do if loading fails (default "skip")} 13 | } 14 | \description{ 15 | If the package fails to load, the default behavior is to mark this commit 16 | as skip. 17 | } 18 | \details{ 19 | This function is usually used together with \code{bisect_install}. 20 | } 21 | \seealso{ 22 | \code{\link{bisect_install}} 23 | 24 | \code{\link{bisect_load_all}} 25 | 26 | \code{\link{bisect_source}} 27 | 28 | \code{\link{bisect_runtest}} 29 | 30 | \code{\link{bisect_return_interactive}} 31 | } 32 | 33 | -------------------------------------------------------------------------------- /man/bisect_install.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2 (4.1.0): do not edit by hand 2 | % Please edit documentation in R/bisect.r 3 | \name{bisect_install} 4 | \alias{bisect_install} 5 | \title{Install a package from source, for bisect tests.} 6 | \usage{ 7 | bisect_install(pkgdir = ".", on_fail = "skip") 8 | } 9 | \arguments{ 10 | \item{pkgdir}{The directory to load from} 11 | 12 | \item{on_fail}{What to do if installation fails (default is to mark this 13 | commit as "skip")} 14 | } 15 | \description{ 16 | If the installation fails, the default behavior is to mark this commit 17 | as skip. 18 | } 19 | \details{ 20 | This function is usually used together with \code{bisect_require}. 21 | } 22 | \seealso{ 23 | \code{\link{bisect_require}} 24 | 25 | \code{\link{bisect_load_all}} 26 | 27 | \code{\link{bisect_source}} 28 | 29 | \code{\link{bisect_runtest}} 30 | 31 | \code{\link{bisect_return_interactive}} 32 | } 33 | 34 | -------------------------------------------------------------------------------- /man/bisect_runtest.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2 (4.1.0): do not edit by hand 2 | % Please edit documentation in R/bisect.r 3 | \name{bisect_runtest} 4 | \alias{bisect_runtest} 5 | \title{Run a test function for git bisect testing.} 6 | \usage{ 7 | bisect_runtest(fun, on_error = "skip", msg = "Running test...") 8 | } 9 | \arguments{ 10 | \item{fun}{The test function} 11 | 12 | \item{on_error}{What to do if running \code{fun} throws an error 13 | (default is to mark this commit as skip)} 14 | 15 | \item{msg}{A message to print to the console when running the test} 16 | } 17 | \description{ 18 | If the function \code{fun} returns \code{"good"} or \code{TRUE}, 19 | quit and return a code to mark this commit as good. 20 | If the function returns \code{"bad"} or \code{FALSE}, 21 | quit and return a code to mark this commit as bad. 22 | If the function returns \code{"skip"} or \code{NA}, 23 | quit and return a code to mark this commit as skip. 24 | If the function returns \code{"ignore"} or \code{NULL}, do nothing. 25 | } 26 | \details{ 27 | It is also important to set \code{on_error}. This tells it what to 28 | do when the test function throws an error. The default behavior is to 29 | mark this commit as skip. However, in some cases, it makes 30 | sense to mark this commit as bad if an error is thrown. 31 | } 32 | \seealso{ 33 | \code{\link{bisect_load_all}} 34 | 35 | \code{\link{bisect_install}} 36 | 37 | \code{\link{bisect_source}} 38 | 39 | \code{\link{bisect_return_interactive}} 40 | } 41 | 42 | -------------------------------------------------------------------------------- /R/bisect.r: -------------------------------------------------------------------------------- 1 | #' bisectr package 2 | #' 3 | #' This package is used for creating test scripts to find bad commits with 4 | #' git bisect. 5 | #' For example test scripts, see \url{https://github.com/wch/bisectr}. 6 | #' 7 | #' @name bisectr 8 | #' @docType package 9 | #' @aliases bisectr package-bisectr 10 | NULL 11 | 12 | 13 | #' Run a test function for git bisect testing. 14 | #' 15 | #' If the function \code{fun} returns \code{"good"} or \code{TRUE}, 16 | #' quit and return a code to mark this commit as good. 17 | #' If the function returns \code{"bad"} or \code{FALSE}, 18 | #' quit and return a code to mark this commit as bad. 19 | #' If the function returns \code{"skip"} or \code{NA}, 20 | #' quit and return a code to mark this commit as skip. 21 | #' If the function returns \code{"ignore"} or \code{NULL}, do nothing. 22 | #' 23 | #' It is also important to set \code{on_error}. This tells it what to 24 | #' do when the test function throws an error. The default behavior is to 25 | #' mark this commit as skip. However, in some cases, it makes 26 | #' sense to mark this commit as bad if an error is thrown. 27 | #' 28 | #' @seealso \code{\link{bisect_load_all}} 29 | #' @seealso \code{\link{bisect_install}} 30 | #' @seealso \code{\link{bisect_source}} 31 | #' @seealso \code{\link{bisect_return_interactive}} 32 | #' 33 | #' @param fun The test function 34 | #' @param on_error What to do if running \code{fun} throws an error 35 | #' (default is to mark this commit as skip) 36 | #' @param msg A message to print to the console when running the test 37 | #' @export 38 | bisect_runtest <- function(fun, on_error = "skip", msg = "Running test...") { 39 | 40 | # Check that fun is a function -- easy to accidentally pass myfun() 41 | # instead of myfun. 42 | if (!is.function(fun)) { 43 | stop("'fun' is not a function. Make sure to pass 'myfunction' and not 'myfunction()'") 44 | } 45 | 46 | message(msg) 47 | 48 | error_fun <- function(e) { 49 | message(e) 50 | message("\nError encountered in test.") 51 | return(on_error) 52 | } 53 | 54 | status <- tryCatch(fun(), error = error_fun) 55 | 56 | # The identical() bit is necessary so that NULL and NA comparisons work 57 | if (is.null(status) || identical(tolower(status), "ignore")) { 58 | # Return NULL, but don't print 59 | invisible(NULL) 60 | } else if (is.na(status) || identical(tolower(status), "skip")) { 61 | mark_commit_skip() 62 | } else if (identical(status, TRUE) || identical(status, "good")) { 63 | mark_commit_good() 64 | } else if (identical(status, FALSE) || identical(status, "bad")) { 65 | mark_commit_bad() 66 | } 67 | } 68 | 69 | 70 | #' Like \code{source}, but for bisect tests. 71 | #' 72 | #' If the file fails to load, the default is mark this commit as skip. 73 | #' 74 | #' @seealso \code{\link{source}} 75 | #' @seealso \code{\link{bisect_load_all}} 76 | #' @seealso \code{\link{bisect_install}} 77 | #' @seealso \code{\link{bisect_runtest}} 78 | #' @seealso \code{\link{bisect_return_interactive}} 79 | #' 80 | #' @param file The file to load 81 | #' @param ... Other arguments to pass to \code{\link{source}} 82 | #' @param on_error What to do if loading throws an error (default is to mark this 83 | #' commit as "skip") 84 | #' @export 85 | #' @importFrom devtools load_all 86 | bisect_source <- function(file, ..., on_error = "skip") { 87 | bisect_runtest(function() { 88 | source(file, ...) 89 | return("good") 90 | }, 91 | on_error = on_error, 92 | msg = paste("Sourcing file ", file)) 93 | } 94 | 95 | 96 | #' Like \code{load_all}, but for bisect tests. 97 | #' 98 | #' If the package fails to load, the default is to mark this commit as skip. 99 | #' 100 | #' @seealso \code{\link{bisect_source}} 101 | #' @seealso \code{\link{bisect_install}} 102 | #' @seealso \code{\link{bisect_runtest}} 103 | #' @seealso \code{\link{bisect_return_interactive}} 104 | #' 105 | #' @param pkgdir The directory to load from 106 | #' @param on_error What to do if loading throws an error (default is to mark this 107 | #' commit as "skip") 108 | #' @export 109 | #' @importFrom devtools load_all 110 | bisect_load_all <- function(pkgdir = ".", on_error = "skip") { 111 | bisect_runtest(function() { 112 | res <- load_all(pkgdir, reset = TRUE) 113 | # This is an imperfect check for whether the package was successfully 114 | # loaded, but devtools::load_all doesn't seem to return more useful 115 | # information. 116 | if (is.null(res$code)) 117 | return(FALSE) 118 | 119 | TRUE 120 | }, 121 | on_error = on_error, 122 | msg = paste("Loading package in directory", pkgdir) 123 | ) 124 | } 125 | 126 | 127 | #' Install a package from source, for bisect tests. 128 | #' 129 | #' If the installation fails, the default behavior is to mark this commit 130 | #' as skip. 131 | #' 132 | #' This function is usually used together with \code{bisect_require}. 133 | #' 134 | #' @seealso \code{\link{bisect_require}} 135 | #' @seealso \code{\link{bisect_load_all}} 136 | #' @seealso \code{\link{bisect_source}} 137 | #' @seealso \code{\link{bisect_runtest}} 138 | #' @seealso \code{\link{bisect_return_interactive}} 139 | #' 140 | #' @param pkgdir The directory to load from 141 | #' @param on_fail What to do if installation fails (default is to mark this 142 | #' commit as "skip") 143 | #' @export 144 | #' @importFrom devtools dev_mode 145 | #' @importFrom devtools install 146 | bisect_install <- function(pkgdir = ".", on_fail = "skip") { 147 | tempPkgdir <- normalizePath(file.path(tempdir(), "bisect-pkgs"), 148 | winslash = "/", mustWork = FALSE) 149 | dev_mode(TRUE, path = tempPkgdir) 150 | message("Temp package installation directory: ", tempPkgdir) 151 | 152 | # install() returns TRUE on success; in this case, we'll give a "ignore" code 153 | # so that the test script will continue. 154 | # When install() fails, it throws an error, in which case we'll pass along 155 | # the on_fail code. 156 | bisect_runtest(function() { 157 | install(pkgdir) 158 | return("ignore") 159 | }, 160 | on_error = on_fail, 161 | msg = paste("Installing package in directory", pkgdir) 162 | ) 163 | } 164 | 165 | 166 | #' Load a package like \code{require()}, for bisect tests. 167 | #' 168 | #' If the package fails to load, the default behavior is to mark this commit 169 | #' as skip. 170 | #' 171 | #' This function is usually used together with \code{bisect_install}. 172 | #' 173 | #' @seealso \code{\link{bisect_install}} 174 | #' @seealso \code{\link{bisect_load_all}} 175 | #' @seealso \code{\link{bisect_source}} 176 | #' @seealso \code{\link{bisect_runtest}} 177 | #' @seealso \code{\link{bisect_return_interactive}} 178 | #' 179 | #' @param package Name of package 180 | #' @param on_fail What to do if loading fails (default "skip") 181 | #' @export 182 | bisect_require <- function(package, on_fail = "skip") { 183 | 184 | package <- as.character(substitute(package)) 185 | 186 | # With require(), success returns TRUE and failure returns FALSE 187 | # but we need to pass different return values to bisect_runtest(). 188 | # If success loading, do nothing ("ignore"); if failure, return on_fail 189 | bisect_runtest(function() { 190 | if (require(package, character.only = TRUE)) 191 | return("ignore") 192 | else 193 | return(on_fail) 194 | }, 195 | msg = paste("Loading package", package) 196 | ) 197 | } 198 | 199 | 200 | #' Prompt the user for an interactive good/bad/skip response and return 201 | #' the appropriate value (to be passed to \code{bisect_runtest}). 202 | #' 203 | #' @seealso \code{\link{bisect_runtest}} 204 | #' @seealso \code{\link{bisect_load_all}} 205 | #' @seealso \code{\link{bisect_install}} 206 | #' @seealso \code{\link{bisect_source}} 207 | #' 208 | #' @export 209 | bisect_return_interactive <- function() { 210 | while (1) { 211 | message("Mark this commit [g]ood, [b]ad, or [s]kip? ", appendLF = FALSE) 212 | 213 | # Need to use "stdin" to get user input in a script -- stdin() doesn't work 214 | response <- scan("stdin", what = character(), n = 1, quiet = TRUE) 215 | 216 | if (identical(tolower(response), "g")) { 217 | return("good") 218 | } else if (identical(tolower(response), "b")) { 219 | return("bad") 220 | } else if (identical(tolower(response), "s")) { 221 | return("skip") 222 | } else { 223 | message(paste("Unknown response:", response)) 224 | } 225 | } 226 | } 227 | 228 | 229 | # =========================================================== 230 | # Functions to quit with return code for marking commits 231 | 232 | mark_commit_good <- function() { 233 | message("Returning code: good (0)\n") 234 | quit(status = 0) 235 | } 236 | 237 | mark_commit_bad <- function() { 238 | message("Returning code: bad (1)\n") 239 | quit(status = 1) 240 | } 241 | 242 | mark_commit_skip <- function() { 243 | message("Returning code: skip (125)\n") 244 | quit(status = 125) 245 | } 246 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # bisectr 2 | 3 | The bisectr package is used to find commits that introduce bugs in a project's git history. 4 | The first bad commit is one that introduces a bug; the commits before the first bad commit are good, and the commits after it are bad. 5 | 6 | Sometimes you can't test a commit because it's broken and you can't load the package at that commit. 7 | In these cases, you don't know if this commit is good or bad, relative to the bug you're looking for. 8 | Instead of marking these commits good or bad, they can be marked **skip**. 9 | 10 | ***** 11 | 12 | # Usage 13 | 14 | The bisectr package is to be used with R scripts that are called from the command line -- not from inside R, but the regular command shell. 15 | 16 | It provides a few wrapper functions that capture errors and return exit codes that can be used by `git bisect run`. All of the following functions can capture errors and return good/bad/skip exit codes for `git bisect run`. See the test templates below for example usage. 17 | 18 | * `bisect_load_all()`: like `load_all()` in devtools, this loads a package in a specified directory. 19 | * `bisect_source()`: like `source()`, this runs an R script. 20 | * `bisect_install()`: like `install()` in devtools, this installs a package in a specified directory. This function installs the package to a temporary directory that is forgotten after the R session, so that it won't interfere with an existing installation. 21 | * `bisect_require()`: like `require()` and `library()`, this loads an installed package. 22 | * `bisect_return_interactive()`: this is for interactive tests (such as graphical ones). It prompts the user to report whether to mark the test Good, Bad, or Skip, and returns the appropriate exit code for `git bisect run`. 23 | 24 | 25 | ## Running a test script 26 | 27 | The scripts can be run from the command line. 28 | They should be saved in the top level of your project directory, like `ggplot2/mytestscript.r` and made executable with `chmod 755 mytestscript.r` 29 | 30 | ``` 31 | # Go to the top level dir of your project 32 | cd ggplot2 33 | chmod 755 mytestscript.r 34 | 35 | git bisect reset 36 | git bisect start 37 | ``` 38 | 39 | First, check if the script works properly, and find the good and bad commits. 40 | This will check out the `master` branch and run the test script. 41 | It won't actually mark commits good or bad, but it will report whether the test result is **good**, **bad**, or **skip**. 42 | This should be a bad commit. 43 | 44 | ``` 45 | git checkout master 46 | ./mytestscript.r 47 | 48 | # If this is indeed bad, you can mark the commit bad with: 49 | git bisect bad 50 | ``` 51 | 52 | Then you need to find an older commit that is good 53 | In this case, we'll check out an older version of ggplot2, a commit that was tagged `ggplot2-0.9.0`. 54 | (You can use any valid git commit ref, like a SHA hash, or branch name.) 55 | 56 | ``` 57 | git checkout ggplot2-0.9.0 58 | ./mytestcript.r 59 | 60 | # If this is indeed good, you can mark it good with: 61 | git bisect good 62 | ``` 63 | 64 | Now that you've marked a bad and good commit, you can run the automated test, which will do a binary search on commits until it finds the guilty commit. 65 | At the end, this will report the first bad commit. 66 | 67 | ``` 68 | git bisect run ./mytestscript.r 69 | ``` 70 | 71 | 72 | ## Template scripts 73 | 74 | These template scripts should be saved in the top level of your project directory. 75 | For example, if the project is ggplot2, a scripts should be saved in `ggplot2/mytestscript.r`. 76 | 77 | ### Fully automated tests 78 | 79 | Here is a template script for a fully automated test. 80 | In this template replace the (trivial) test (`x==2`) with your test code. 81 | It has three possible return values. 82 | 83 | * If the test is successful, mark the commit as **good**. 84 | * If the test is unsuccessful, mark the commit as **bad**. 85 | * If the test throws an error, mark the commit as **skip**. 86 | 87 | ```R 88 | #!/usr/bin/Rscript 89 | 90 | # To run this script: 91 | # git bisect reset 92 | # git bisect start 93 | # git bisect run mytestscript.r 94 | 95 | cat("\n===== Running test script ======\n") 96 | library(bisectr) 97 | 98 | # This is the test function. It is defined here and run later. 99 | # A fully automated test 100 | testfun <- function() { 101 | 102 | # ... Put your test code here ... 103 | x <- 1 + 1 104 | 105 | # If some condition is met, return good; otherwise bad 106 | if (x == 2) 107 | return("good") 108 | else 109 | return("bad") 110 | } 111 | 112 | # Load package in current directory 113 | # If load error, mark "skip" 114 | bisect_load_all(".", on_error = "skip") 115 | # To source a script instead, use: 116 | # bisect_source("file.r", on_error="skip") 117 | 118 | 119 | # Run the test, and if error, mark skip 120 | bisect_runtest(testfun, on_error = "skip") 121 | ``` 122 | 123 | You can change the return codes for your purposes. For example, this may be what you want: 124 | 125 | * If the test code runs without throwing an error, mark the commit as **good**. 126 | * If the test code throws an error, marks the commit as **bad**. 127 | 128 | 129 | ```R 130 | #!/usr/bin/Rscript 131 | 132 | # To run this script: 133 | # git bisect reset 134 | # git bisect start 135 | # git bisect run mytestscript.r 136 | 137 | cat("\n===== Running test script ======\n") 138 | library(bisectr) 139 | 140 | # This is the test function. It is defined here and run later. 141 | # A fully automated test 142 | testfun <- function() { 143 | 144 | # ... Put your test code here ... 145 | 146 | # If we made it this far, that means we didn't throw an error, so return good 147 | return("good") 148 | } 149 | 150 | # If load error, mark "skip" 151 | bisect_load_all(".", on_error = "skip") 152 | 153 | # Run the test, and if error, mark bad 154 | bisect_runtest(testfun, on_error = "bad") 155 | ``` 156 | 157 | ### Interactive tests 158 | 159 | Sometimes it's difficult or impossible to fully automate a test. 160 | This is often true if the test involves graphical output -- you need to visually inspect the output to see if it looks right. 161 | You can interactively get return codes with the `bisect_return_interactive()` function. 162 | This will allow you to view the output and then type in g/b/s, to mark it good/bad/skip. 163 | 164 | This template will run a test that does the following. 165 | It will try to make a plot, but inspecting the plot will require you to visually inspect and decide whether it's good or bad. 166 | Here are the desired criteria: 167 | 168 | * If the test looks right, mark the commit as **good**. (requires interaction) 169 | * If the test looks wrong, mark the commit as **bad**. (requires interaction) 170 | * If the test throws an error, mark the commit as **skip**. (automatic) 171 | 172 | 173 | ```R 174 | #!/usr/bin/Rscript 175 | 176 | # To run this script: 177 | # git bisect reset 178 | # git bisect start ggplot2-0.9.1 c5b872e 179 | # git bisect run mytestscript.r 180 | 181 | cat("\n===== Running test script ======\n") 182 | library(bisectr) 183 | 184 | # This is the test function. It is defined here and run later. 185 | # A test that requires visual inspection and manual response 186 | testRunInteractive <- function() { 187 | 188 | # ... Test code here ... 189 | dat <- data.frame(x=LETTERS[1:3], 190 | y=c(10,5,8,7,9,4), 191 | g=c("a","a","a","b","b","b")) 192 | 193 | p <- ggplot(dat, aes(x=x, y=y, fill=g)) + 194 | geom_bar(position="dodge", fill="white", colour="black", stat="identity") 195 | # ... End test code ... 196 | 197 | # Need to manually open graphics window because we're in a script 198 | # Try different graphic devices, depending on platform 199 | # (Not sure about Windows) 200 | if (capabilities()['aqua']) { quartz() } 201 | else if (capabilities()['X11']) { x11() } 202 | 203 | # Must explicitly print ggplot object because we're in a script 204 | print(p) 205 | 206 | # On some platforms we need to wait a little bit to allow the plot to render 207 | Sys.sleep(0.75) 208 | 209 | # User must visually inspect and mark good/bad/skip 210 | bisect_return_interactive() 211 | } 212 | 213 | # If load error, mark SKIP 214 | bisect_load_all(".", on_error = "skip") 215 | 216 | # If error, mark SKIP 217 | bisect_runtest(testRunInteractive, on_error = "skip") 218 | ``` 219 | 220 | ***** 221 | 222 | # Notes 223 | 224 | ## Installing packages instead of using `load_all` 225 | 226 | Normally, you can use the `bisect_load_all()` function, which calls `devtoos::load_all()`. 227 | However, sometimes `load_all()` doesn't work right, and you have to install the package to run the test. 228 | In these cases, use `bisect_install()` and `bisect_require()` (note that installing a package takes more time than `load_all()`.) 229 | . 230 | This will install the package to a temporary directory, so there's no need to manually clean it up after testing. 231 | 232 | ```R 233 | # Instead of bisect_load_all(".") 234 | bisect_install(".", on_fail = "skip") 235 | bisect_require(mypackage, on_fail = "skip") 236 | ``` 237 | 238 | By default, `bisect_install()` will mark the commit as **skip** on a failure to install, but you can change this. 239 | For example, you can mark the commit as **bad** on failure to install: 240 | 241 | ```R 242 | bisect_install(".", on_fail="bad") 243 | ``` --------------------------------------------------------------------------------