├── .Rbuildignore ├── .github ├── .gitignore ├── ISSUE_TEMPLATE │ ├── bug.md │ ├── config.yml │ ├── feature.md │ ├── maintenance.md │ └── performance.md ├── PULL_REQUEST_TEMPLATE.md └── workflows │ ├── check.yaml │ ├── cover.yaml │ └── lint.yaml ├── .gitignore ├── .lintr ├── CONTRIBUTING.md ├── DESCRIPTION ├── LICENSE ├── LICENSE.md ├── NAMESPACE ├── NEWS.md ├── R ├── tar_git_checkout.R ├── tar_git_init.R ├── tar_git_log.R ├── tar_git_ok.R ├── tar_git_snapshot.R ├── tar_git_status.R ├── tar_git_status_code.R ├── tar_git_status_data.R ├── tar_git_status_targets.R ├── tar_package.R ├── utils_assert.R ├── utils_callr.R ├── utils_cli.R ├── utils_data.R ├── utils_git.R └── utils_logic.R ├── README.Rmd ├── README.md ├── _pkgdown.yml ├── codemeta.json ├── gittargets.Rproj ├── inst ├── CITATION └── WORDLIST ├── man ├── figures │ ├── figures.pptx │ ├── image-credit.md │ ├── logo-font.svg │ ├── logo-hexbin.png │ ├── logo-readme.png │ ├── logo.png │ └── logo.svg ├── gittargets-package.Rd ├── tar_git_checkout.Rd ├── tar_git_init.Rd ├── tar_git_log.Rd ├── tar_git_ok.Rd ├── tar_git_snapshot.Rd ├── tar_git_snapshot_menu.Rd ├── tar_git_status.Rd ├── tar_git_status_code.Rd ├── tar_git_status_data.Rd └── tar_git_status_targets.Rd ├── pkgdown └── favicon │ ├── apple-touch-icon-120x120.png │ ├── apple-touch-icon-152x152.png │ ├── apple-touch-icon-180x180.png │ ├── apple-touch-icon-60x60.png │ ├── apple-touch-icon-76x76.png │ ├── apple-touch-icon.png │ ├── favicon-16x16.png │ ├── favicon-32x32.png │ └── favicon.ico ├── tests ├── interactive │ └── test-tar_git_snapshot.R ├── testthat.R └── testthat │ ├── helper-git.R │ ├── helper-utils.R │ ├── test-tar_git_checkout.R │ ├── test-tar_git_init.R │ ├── test-tar_git_log.R │ ├── test-tar_git_ok.R │ ├── test-tar_git_snapshot.R │ ├── test-tar_git_status.R │ ├── test-tar_git_status_code.R │ ├── test-tar_git_status_data.R │ ├── test-tar_git_status_targets.R │ ├── test-utils_assert.R │ ├── test-utils_cli.R │ ├── test-utils_data.R │ ├── test-utils_git.R │ └── test-utils_logic.R └── vignettes ├── .gitignore ├── git.Rmd └── snapshot-model-git.png /.Rbuildignore: -------------------------------------------------------------------------------- 1 | ^.*\.css$ 2 | ^.*code-workspace 3 | ^.*\.gcda$ 4 | ^.*\.gcno$ 5 | ^.*\.js$ 6 | ^.*\.log$ 7 | ^.*\.out$ 8 | ^.*\.nfs.*$ 9 | ^.*\.Rout$ 10 | ^.*\.Rproj$ 11 | ^.*\.svg$ 12 | ^\.DS_Store 13 | ^\.future$ 14 | ^\.git$ 15 | ^\.github$ 16 | ^\.gitignore$ 17 | ^\.gittargets$ 18 | ^\.httr-oauth$ 19 | ^\.lintr$ 20 | ^\.Rapp.history$ 21 | ^\.Rbuildignore$ 22 | ^\.RData$ 23 | ^\.Rhistory$ 24 | ^\.Rproj\.user$ 25 | ^_pkgdown\.yml$ 26 | ^_targets$ 27 | ^_targets_r$ 28 | ^_targets\.R$ 29 | ^_targets_packages\.R$ 30 | ^_targets\.Rmd$ 31 | ^_targets\.yaml$ 32 | ^skeleton\.html$ 33 | ^docs$ 34 | ^CODE_OF_CONDUCT.md$ 35 | ^codemeta\.json$ 36 | ^CONTRIBUTING.md$ 37 | ^pkgdown$ 38 | ^README\.md$ 39 | ^README\.Rmd$ 40 | ^revdep$ 41 | ^LICENSE\.md$ 42 | ^paper\.md$ 43 | ^paper\.bib$ 44 | ^CODE_OF_CONDUCT\.md$ 45 | -------------------------------------------------------------------------------- /.github/.gitignore: -------------------------------------------------------------------------------- 1 | *.html 2 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/bug.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Bug 3 | about: Please do not submit a bug report unless your issue is a genuine bug in gittargets and not a known limitation, usage error, or issue from another package that gittargets depends on. 4 | title: "" 5 | labels: "type: bug" 6 | assignees: wlandau 7 | --- 8 | 9 | ## Prework 10 | 11 | * [ ] Read and agree to the [Contributor Code of Conduct](https://ropensci.org/code-of-conduct/) and [contributing guidelines](https://github.com/ropensci/gittargets/blob/main/CONTRIBUTING.md). 12 | * [ ] Confirm that your issue is a genuine bug in the `gittargets` package itself and not a user error, known limitation, or issue from another package that `gittargets` depends on. For example, if you get errors running `tar_make_clustermq()`, try isolating the problem in a reproducible example that runs `clustermq` and not `gittargets`. And for miscellaneous troubleshooting, please post to [discussions](https://github.com/ropensci/gittargets/discussions) instead of [issues](https://github.com/ropensci/gittargets/issues). 13 | * [ ] If there is [already a relevant issue](https://github.com/ropensci/gittargets/issues), whether open or closed, comment on the existing thread instead of posting a new issue. 14 | * [ ] Post a [minimal reproducible example](https://www.tidyverse.org/help/) like [this one](https://github.com/ropensci/targets/issues/256#issuecomment-754229683) so the maintainer can troubleshoot the problems you identify. A reproducible example is: 15 | * [ ] **Runnable**: post enough R code and data so any onlooker can create the error on their own computer. 16 | * [ ] **Minimal**: reduce runtime wherever possible and remove complicated details that are irrelevant to the issue at hand. 17 | * [ ] **Readable**: format your code according to the [tidyverse style guide](https://style.tidyverse.org/). 18 | 19 | ## Description 20 | 21 | Please describe the bug. 22 | 23 | ## Reproducible example 24 | 25 | * [ ] Post a [minimal reproducible example](https://www.tidyverse.org/help/) so the maintainer can troubleshoot the problems you identify. A reproducible example is: 26 | * [ ] **Runnable**: post enough R code and data so any onlooker can create the error on their own computer. 27 | * [ ] **Minimal**: reduce runtime wherever possible and remove complicated details that are irrelevant to the issue at hand. 28 | * [ ] **Readable**: format your code according to the [tidyverse style guide](https://style.tidyverse.org/). 29 | 30 | ## Expected result 31 | 32 | What should have happened? Please be as specific as possible. 33 | 34 | ## Diagnostic information 35 | 36 | * A [reproducible example](https://github.com/tidyverse/reprex). 37 | * Session info, available through `sessionInfo()` or [`reprex(si = TRUE)`](https://github.com/tidyverse/reprex). 38 | * A stack trace from `traceback()` or `rlang::trace_back()`. 39 | * The [SHA-1 hash](https://git-scm.com/book/en/v1/Getting-Started-Git-Basics#Git-Has-Integrity) of the GitHub commit of `gittargets` currently installed. `packageDescription("gittargets")$GithubSHA1` shows you this. 40 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/config.yml: -------------------------------------------------------------------------------- 1 | lank_issues_enabled: true 2 | contact_links: 3 | - name: General 4 | url: https://github.com/ropensci/gittargets/discussions/categories/general 5 | about: Chat about targets. 6 | - name: Help 7 | url: https://github.com/ropensci/gittargets/discussions/categories/help 8 | about: Ask questions and get help. 9 | - name: Ideas 10 | url: https://github.com/ropensci/gittargets/discussions/categories/ideas 11 | about: Brainstorm ideas for new features or use cases. 12 | - name: Show and tell 13 | url: https://github.com/ropensci/gittargets/discussions/categories/show-and-tell 14 | about: Show off something you have made. 15 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/feature.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: New feature 3 | about: Suggest a new feature. 4 | title: "" 5 | labels: "type: new feature" 6 | assignees: wlandau 7 | --- 8 | 9 | ## Prework 10 | 11 | * [ ] Read and agree to the [Contributor Code of Conduct](https://ropensci.org/code-of-conduct/) and [contributing guidelines](https://github.com/ropensci/gittargets/blob/main/CONTRIBUTING.md). 12 | * [ ] If there is [already a relevant issue](https://github.com/ropensci/gittargets/issues), whether open or closed, comment on the existing thread instead of posting a new issue. 13 | * [ ] New features take time and effort to create, and they take even more effort to maintain. So if the purpose of the feature is to resolve a struggle you are encountering personally, please consider first posting a "trouble" or "other" issue so we can discuss your use case and search for existing solutions first. 14 | * [ ] Format your code according to the [tidyverse style guide](https://style.tidyverse.org/). 15 | 16 | ## Proposal 17 | 18 | Please describe the new feature. If applicable, write a minimal example in R code or pseudo-code to show input, usage, and desired output. 19 | 20 | To help us read any code you include (optional) please try to follow the [tidyverse style guide](https://style.tidyverse.org/). The `style_text()` and `style_file()` functions from the [`styler`](https://github.com/r-lib/styler) package make it easier. 21 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/maintenance.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Maintenance 3 | about: "Something in gittargets needs work: updates, documentation, etc. Not a bug, performance issue, or new feature." 4 | title: "" 5 | labels: "type: maintenance" 6 | assignees: "" 7 | --- 8 | 9 | ## Prework 10 | 11 | * [ ] Read and agree to the [Contributor Code of Conduct](https://ropensci.org/code-of-conduct/) and [contributing guidelines](https://github.com/ropensci/gittargets/blob/main/CONTRIBUTING.md). 12 | * [ ] If there is [already a relevant issue](https://github.com/ropensci/gittargets/issues), whether open or closed, comment on the existing thread instead of posting a new issue. 13 | * [ ] For any problems you identify, post a [minimal reproducible example](https://www.tidyverse.org/help/) like [this one](https://github.com/ropensci/targets/issues/256#issuecomment-754229683) so the maintainer can troubleshoot. A reproducible example is: 14 | * [ ] **Runnable**: post enough R code and data so any onlooker can create the error on their own computer. 15 | * [ ] **Minimal**: reduce runtime wherever possible and remove complicated details that are irrelevant to the issue at hand. 16 | * [ ] **Readable**: format your code according to the [tidyverse style guide](https://style.tidyverse.org/). 17 | 18 | ## Description 19 | 20 | Please describe the issue. 21 | 22 | To help us read any code you include (optional) please try to follow the [tidyverse style guide](https://style.tidyverse.org/). The `style_text()` and `style_file()` functions from the [`styler`](https://github.com/r-lib/styler) package make it easier. 23 | 24 | ## Reproducible example 25 | 26 | * [ ] For any problems you identify, post a [minimal reproducible example](https://www.tidyverse.org/help/) so the maintainer can troubleshoot. A reproducible example is: 27 | * [ ] **Runnable**: post enough R code and data so any onlooker can create the error on their own computer. 28 | * [ ] **Minimal**: reduce runtime wherever possible and remove complicated details that are irrelevant to the issue at hand. 29 | * [ ] **Readable**: format your code according to the [tidyverse style guide](https://style.tidyverse.org/). 30 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/performance.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Performance 3 | about: "Runtime, memory, or storage inefficiency" 4 | title: "" 5 | labels: "topic: performance" 6 | assignees: wlandau 7 | 8 | --- 9 | 10 | ## Prework 11 | 12 | * [ ] Read and agree to the [Contributor Code of Conduct](https://ropensci.org/code-of-conduct/) and [contributing guidelines](https://github.com/ropensci/gittargets/blob/main/CONTRIBUTING.md). 13 | * [ ] If there is [already a relevant issue](https://github.com/ropensci/gittargets/issues), whether open or closed, comment on the existing thread instead of posting a new issue. 14 | * [ ] Post a [minimal reproducible example](https://www.tidyverse.org/help/) like [this one](https://github.com/ropensci/targets/issues/256#issuecomment-754229683) so the maintainer can troubleshoot the problems you identify. A reproducible example is: 15 | * [ ] **Runnable**: post enough R code and data so any onlooker can create the error on their own computer. 16 | * [ ] **Minimal**: reduce runtime wherever possible and remove complicated details that are irrelevant to the issue at hand. 17 | * [ ] **Readable**: format your code according to the [tidyverse style guide](https://style.tidyverse.org/). 18 | 19 | ## Description 20 | 21 | Please describe the performance issue. 22 | 23 | ## Reproducible example 24 | 25 | * [ ] Post a [minimal reproducible example](https://www.tidyverse.org/help/) so the maintainer can troubleshoot the problems you identify. A reproducible example is: 26 | * [ ] **Runnable**: post enough R code and data so any onlooker can create the error on their own computer. 27 | * [ ] **Minimal**: reduce runtime wherever possible and remove complicated details that are irrelevant to the issue at hand. 28 | * [ ] **Readable**: format your code according to the [tidyverse style guide](https://style.tidyverse.org/). 29 | 30 | ## Benchmarks 31 | 32 | How poorly does `gittargets` perform? To find out, we recommend you use the [`proffer`](https://github.com/r-prof/proffer) package and take screenshots of the results displayed in your browser. 33 | 34 | ```r 35 | library(gittargets) 36 | library(proffer) 37 | px <- pprof({ 38 | # All your gittargets code goes here. 39 | }) 40 | ``` 41 | -------------------------------------------------------------------------------- /.github/PULL_REQUEST_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | # Prework 2 | 3 | * [ ] I understand and agree to the [Contributor Code of Conduct](https://ropensci.org/code-of-conduct/). 4 | * [ ] I have already submitted a [discussion topic](https://github.com/ropensci/gittargets/discussions) or [issue](https://github.com/ropensci/gittargets/issues) to discuss my idea with the maintainer. 5 | 6 | # Related GitHub issues and pull requests 7 | 8 | * Ref: # 9 | 10 | # Summary 11 | 12 | Please explain the purpose and scope of your contribution. 13 | -------------------------------------------------------------------------------- /.github/workflows/check.yaml: -------------------------------------------------------------------------------- 1 | on: [push, pull_request] 2 | 3 | name: check 4 | 5 | jobs: 6 | check: 7 | runs-on: ${{ matrix.config.os }} 8 | 9 | name: ${{ matrix.config.os }} (${{ matrix.config.r }}) 10 | 11 | strategy: 12 | fail-fast: false 13 | matrix: 14 | config: 15 | - {os: macOS-latest, r: 'release'} 16 | - {os: windows-latest, r: 'release'} 17 | - {os: ubuntu-latest, r: 'devel', http-user-agent: 'release'} 18 | - {os: ubuntu-latest, r: 'release'} 19 | - {os: ubuntu-latest, r: 'oldrel-1'} 20 | 21 | env: 22 | RSPM: ${{ matrix.config.rspm }} 23 | GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }} 24 | R_KEEP_PKG_SOURCE: yes 25 | R_REMOTES_NO_ERRORS_FROM_WARNINGS: false 26 | 27 | steps: 28 | - uses: actions/checkout@v2 29 | 30 | - uses: r-lib/actions/setup-pandoc@v2 31 | 32 | - uses: r-lib/actions/setup-r@v2 33 | with: 34 | r-version: ${{ matrix.config.r }} 35 | http-user-agent: ${{ matrix.config.http-user-agent }} 36 | use-public-rspm: true 37 | 38 | - uses: r-lib/actions/setup-r-dependencies@v2 39 | with: 40 | extra-packages: any::rcmdcheck 41 | needs: check 42 | 43 | - name: Git config 44 | run: | 45 | git config --global user.email "actions@github.com" 46 | git config --global user.name "GitHub Actions" 47 | 48 | - uses: r-lib/actions/check-r-package@v2 49 | with: 50 | upload-snapshots: true 51 | 52 | - name: Upload check results 53 | if: failure() 54 | uses: actions/upload-artifact@master 55 | with: 56 | name: ${{ runner.os }}-r${{ matrix.config.r }}-results 57 | path: check 58 | -------------------------------------------------------------------------------- /.github/workflows/cover.yaml: -------------------------------------------------------------------------------- 1 | # Workflow derived from https://github.com/r-lib/actions/tree/v2/examples 2 | # Need help debugging build failures? Start at https://github.com/r-lib/actions#where-to-find-help 3 | on: [push, pull_request] 4 | 5 | name: cover 6 | 7 | jobs: 8 | cover: 9 | runs-on: ubuntu-latest 10 | env: 11 | GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }} 12 | 13 | steps: 14 | - uses: actions/checkout@v2 15 | 16 | - uses: r-lib/actions/setup-r@v2 17 | with: 18 | use-public-rspm: true 19 | 20 | - uses: r-lib/actions/setup-r-dependencies@v2 21 | with: 22 | extra-packages: any::covr 23 | needs: coverage 24 | 25 | - name: Git config 26 | run: | 27 | git config --global user.email "actions@github.com" 28 | git config --global user.name "GitHub Actions" 29 | 30 | - name: Test coverage 31 | run: covr::codecov(quiet = FALSE) 32 | shell: Rscript {0} 33 | -------------------------------------------------------------------------------- /.github/workflows/lint.yaml: -------------------------------------------------------------------------------- 1 | # Workflow derived from https://github.com/r-lib/actions/tree/v2/examples 2 | # Need help debugging build failures? Start at https://github.com/r-lib/actions#where-to-find-help 3 | on: [push, pull_request] 4 | 5 | name: lint 6 | 7 | jobs: 8 | lint: 9 | runs-on: ubuntu-latest 10 | env: 11 | GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }} 12 | steps: 13 | - uses: actions/checkout@v2 14 | 15 | - uses: r-lib/actions/setup-r@v2 16 | with: 17 | use-public-rspm: true 18 | 19 | - uses: r-lib/actions/setup-r-dependencies@v2 20 | with: 21 | extra-packages: any::lintr, local::. 22 | needs: lint 23 | 24 | - name: Lint 25 | run: lintr::lint_package() 26 | shell: Rscript {0} 27 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | *~ 2 | *.out 3 | *.Rout 4 | .gittargets 5 | .DS_Store 6 | .future 7 | .Rapp.history 8 | .RData 9 | .Rhistory 10 | .Rproj.user/ 11 | docs 12 | inst/doc 13 | vignettes/*.html 14 | README.html 15 | paper.html 16 | .Rproj.user 17 | _packages.R 18 | _targets 19 | _targets.R 20 | _targets.Rmd 21 | _targets_r 22 | _targets.yaml 23 | skeleton.html 24 | revdep 25 | -------------------------------------------------------------------------------- /.lintr: -------------------------------------------------------------------------------- 1 | linters: linters_with_defaults( 2 | object_length_linter = NULL, 3 | object_name_linter = NULL, 4 | object_usage_linter = NULL) 5 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing 2 | 3 | Development is a community effort, and we welcome participation. 4 | 5 | ## Code of Conduct 6 | 7 | Please note that the `gittargets` project is released with a [Contributor Code of Conduct](https://ropensci.org/code-of-conduct/). By contributing to this project, you agree to abide by its terms. 8 | 9 | ## Discussions 10 | 11 | At , you can post general questions, brainstorm ideas, and ask for help. 12 | 13 | ## Issues 14 | 15 | is for bug reports, performance issues, maintenance tasks, and feature requests. When you post, please abide by the following guidelines. 16 | 17 | * Before posting a new issue or discussion topic, please take a moment to search for existing similar threads in order to avoid duplication. 18 | * For bug reports: if you can, please install the latest GitHub version of `gittargets` (i.e. `remotes::install_github("ropensci/gittargets")`) and verify that the issue still persists. 19 | * Describe your issue in prose as clearly and concisely as possible. 20 | * For any problem you identify, post a [minimal reproducible example](https://www.tidyverse.org/help/) like [this one](https://github.com/ropensci/targets/issues/256#issuecomment-754229683) so the maintainer can troubleshoot. A reproducible example is: 21 | * **Runnable**: post enough R code and data so any onlooker can create the error on their own computer. 22 | * **Minimal**: reduce runtime wherever possible and remove complicated details that are irrelevant to the issue at hand. 23 | * **Readable**: format your code according to the [tidyverse style guide](https://style.tidyverse.org/). 24 | 25 | ## Development 26 | 27 | External code contributions are extremely helpful in the right circumstances. Here are the recommended steps. 28 | 29 | 1. Prior to contribution, please propose your idea in a discussion topic or issue thread so you and the maintainer can define the intent and scope of your work. 30 | 2. [Fork the repository](https://help.github.com/articles/fork-a-repo/). 31 | 3. Follow the [GitHub flow](https://guides.github.com/introduction/flow/index.html) to create a new branch, add commits, and open a pull request. 32 | 4. Discuss your code with the maintainer in the pull request thread. 33 | 5. If everything looks good, the maintainer will merge your code into the project. 34 | 35 | Please also follow these additional guidelines. 36 | 37 | * Respect the architecture and reasoning of the package. Depending on the scope of your work, you may want to read the design documents (package vignettes). 38 | * If possible, keep contributions small enough to easily review manually. It is okay to split up your work into multiple pull requests. 39 | * Format your code according to the [tidyverse style guide](https://style.tidyverse.org/) and check your formatting with the `lint_package()` function from the [`lintr`](https://github.com/jimhester/lintr) package. 40 | * For new features or functionality, add tests in `tests`. Tests that can be automated should go in `tests/testthat/`. Tests that cannot be automated should go in `tests/interactive/`. For features affecting performance, it is good practice to add profiling studies to `tests/performance/`. 41 | * Check code coverage with `covr::package_coverage()`. Automated tests should cover all the new or changed functionality in your pull request. 42 | * Run overall package checks with `devtools::check()` and `goodpractice::gp()` 43 | * Describe your contribution in the project's [`NEWS.md`](https://github.com/ropensci/gittargets/blob/main/NEWS.md) file. Be sure to mention relevent GitHub issue numbers and your GitHub name as done in existing news entries. 44 | * If you feel contribution is substantial enough for official author or contributor status, please add yourself to the `Authors@R` field of the [`DESCRIPTION`](https://github.com/ropensci/gittargets/blob/main/DESCRIPTION) file. 45 | -------------------------------------------------------------------------------- /DESCRIPTION: -------------------------------------------------------------------------------- 1 | Package: gittargets 2 | Title: Data Version Control for the Targets Package 3 | Description: In computationally demanding data analysis pipelines, 4 | the 'targets' R package (2021, ) maintains 5 | an up-to-date set of results while skipping tasks that do not need to rerun. 6 | This process increases speed and increases trust in the final end product. 7 | However, it also overwrites old output with new output, and past 8 | results disappear by default. To preserve historical output, the 'gittargets' 9 | package captures version-controlled snapshots of the data store, 10 | and each snapshot links to the underlying commit of the source code. 11 | That way, when the user rolls back the code to a previous branch or commit, 12 | 'gittargets' can recover the data contemporaneous with that commit so that 13 | all targets remain up to date. 14 | Version: 0.0.7.9000 15 | License: MIT + file LICENSE 16 | URL: https://docs.ropensci.org/gittargets/, https://github.com/ropensci/gittargets 17 | BugReports: https://github.com/ropensci/gittargets/issues 18 | Authors@R: c( 19 | person( 20 | given = c("William", "Michael"), 21 | family = "Landau", 22 | role = c("aut", "cre"), 23 | email = "will.landau.oss@gmail.com", 24 | comment = c(ORCID = "0000-0003-1878-3253") 25 | ), 26 | person( 27 | given = "Saras", 28 | family = "Windecker", 29 | role = "rev", 30 | email = "saras.windecker@gmail.com" 31 | ), 32 | person( 33 | given = "David", 34 | family = "Neuzerling", 35 | role = "rev", 36 | email = "david@neuzerling.com" 37 | ), 38 | person( 39 | family = "Eli Lilly and Company", 40 | role = "cph" 41 | )) 42 | Depends: 43 | R (>= 3.5.0) 44 | Imports: 45 | callr (>= 3.0.0), 46 | cli (>= 3.1.0), 47 | data.table (>= 1.12.8), 48 | gert (>= 1.0.0), 49 | processx (>= 3.0.0), 50 | stats, 51 | targets (>= 0.6.0), 52 | tibble (>= 3.0.0), 53 | utils, 54 | uuid (>= 0.1.4) 55 | Suggests: 56 | knitr (>= 1.30), 57 | markdown (>= 1.1), 58 | rmarkdown (>= 2.4), 59 | testthat (>= 3.0.0) 60 | SystemRequirements: Git (>= 2.0.0) 61 | Encoding: UTF-8 62 | Language: en-US 63 | VignetteBuilder: knitr 64 | Config/testthat/edition: 3 65 | Roxygen: list(markdown = TRUE) 66 | RoxygenNote: 7.2.3 67 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | YEAR: 2021 2 | COPYRIGHT HOLDER: Eli Lilly and Company 3 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | # MIT License 2 | 3 | Copyright (c) 2021 Eli Lilly and Company 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /NAMESPACE: -------------------------------------------------------------------------------- 1 | # Generated by roxygen2: do not edit by hand 2 | 3 | export(tar_git_checkout) 4 | export(tar_git_init) 5 | export(tar_git_log) 6 | export(tar_git_ok) 7 | export(tar_git_snapshot) 8 | export(tar_git_status) 9 | export(tar_git_status_code) 10 | export(tar_git_status_data) 11 | export(tar_git_status_targets) 12 | importFrom(callr,r) 13 | importFrom(cli,cli_alert_danger) 14 | importFrom(cli,cli_alert_info) 15 | importFrom(cli,cli_alert_success) 16 | importFrom(cli,cli_h1) 17 | importFrom(cli,col_br_white) 18 | importFrom(data.table,as.data.table) 19 | importFrom(gert,git_branch) 20 | importFrom(gert,git_branch_exists) 21 | importFrom(gert,git_branch_list) 22 | importFrom(gert,git_commit_info) 23 | importFrom(gert,git_log) 24 | importFrom(gert,git_status) 25 | importFrom(processx,run) 26 | importFrom(stats,complete.cases) 27 | importFrom(targets,tar_config_get) 28 | importFrom(targets,tar_config_set) 29 | importFrom(targets,tar_dir) 30 | importFrom(targets,tar_outdated) 31 | importFrom(targets,tar_throw_validate) 32 | importFrom(tibble,as_tibble) 33 | importFrom(tibble,tibble) 34 | importFrom(utils,capture.output) 35 | importFrom(utils,head) 36 | importFrom(utils,menu) 37 | importFrom(uuid,UUIDgenerate) 38 | -------------------------------------------------------------------------------- /NEWS.md: -------------------------------------------------------------------------------- 1 | # gittargets 0.0.7.9000 (development) 2 | 3 | 4 | 5 | # gittargets 0.0.7 6 | 7 | * Migrate tests to `targets` >= 1.3.2.9004 progress statuses ("completed" instead of "built", "dispatched" instead of "started"). 8 | 9 | # gittargets 0.0.6 10 | 11 | * Import `callr::r()`. 12 | 13 | # gittargets 0.0.5 14 | 15 | * Use `processx::run()` instead of `system2()` in `tar_git_ok()` and set `HOME` to `USERPROFILE` on Windows (#12, @psychelzh). 16 | * Handle errors invoking Git to get global user name and email. 17 | 18 | # gittargets 0.0.4 19 | 20 | * Compatibility with {targets} 0.13.0. 21 | 22 | # gittargets 0.0.3 23 | 24 | * Fix an example for CRAN. 25 | 26 | # gittargets 0.0.2 27 | 28 | * Hard reset after checkout in `tar_git_checkout()` in order to recover potentially deleted files (#11). 29 | 30 | # gittargets 0.0.1 31 | 32 | * Join rOpenSci. 33 | * Rewrite README to motivate the use case. 34 | * Remove workflow diagram. 35 | * Simplify snapshot model diagram. 36 | * Fix the documentation of the `ref` argument of `tar_git_checkout()`. 37 | * Add a section to the `git.Rmd` vignette on code merges. 38 | * Allow command line Git tests to run locally on Windows. 39 | * First version. 40 | -------------------------------------------------------------------------------- /R/tar_git_checkout.R: -------------------------------------------------------------------------------- 1 | #' @title Check out a snapshot of the data (Git) 2 | #' @export 3 | #' @family git 4 | #' @description Check out a snapshot of the data associated with 5 | #' a particular code commit (default: `HEAD`). 6 | #' @return Nothing (invisibly). 7 | #' @param ref Character of length 1. SHA1 hash, branch name, 8 | #' or other reference in the code repository 9 | #' that points to a code commit. (You can also identify the code 10 | #' commit by supplying a data branch of the form `code=`.) 11 | #' Defaults to `"HEAD"`, which points to the currently 12 | #' checked out code commit. 13 | #' 14 | #' Once the desired code commit is identified, 15 | #' `tar_git_snapshot()` checks out the latest corresponding data snapshot. 16 | #' There may be earlier data snapshots corresponding to this code commit, 17 | #' but `tar_git_snapshot()` only checks out the latest one. 18 | #' To check out an earlier superseded data snapshot, 19 | #' you will need to manually use command line Git in the data repository. 20 | #' 21 | #' If `tar_git_snapshot()` cannot find a data snapshot for the 22 | #' desired code commit, then it will throw an error. 23 | #' For a list of commits in the current code branch 24 | #' that have available data snapshots, see the `commit_code` 25 | #' column of the output of [tar_git_log()]. 26 | #' @inheritParams gert::git_branch_checkout 27 | #' @inheritParams tar_git_snapshot 28 | #' @examples 29 | #' if (Sys.getenv("TAR_EXAMPLES") == "true" && tar_git_ok(verbose = FALSE)) { 30 | #' targets::tar_dir({ # Containing code does not modify the user's filespace. 31 | #' # Work on an initial branch. 32 | #' targets::tar_script(tar_target(data, "old_data")) 33 | #' targets::tar_make() 34 | #' targets::tar_read(data) # "old_data" 35 | #' gert::git_init() 36 | #' gert::git_add("_targets.R") 37 | #' gert::git_commit("First commit") 38 | #' gert::git_branch_create("old_branch") 39 | #' tar_git_init() 40 | #' # Work on a new branch. 41 | #' tar_git_snapshot(status = FALSE, verbose = FALSE) 42 | #' targets::tar_script(tar_target(data, "new_data")) 43 | #' targets::tar_make() 44 | #' targets::tar_read(data) # "new_data" 45 | #' gert::git_branch_create("new_branch") 46 | #' gert::git_add("_targets.R") 47 | #' gert::git_commit("Second commit") 48 | #' tar_git_snapshot(status = FALSE, verbose = FALSE) 49 | #' # Go back to the old branch. 50 | #' gert::git_branch_checkout("old_branch") 51 | #' # The target is out of date because we only reverted the code. 52 | #' targets::tar_outdated() 53 | #' # But tar_git_checkout() lets us restore the old version of the data! 54 | #' tar_git_checkout() 55 | #' targets::tar_read(data) # "old_data" 56 | #' # Now, the target is up to date! And we did not even have to rerun it! 57 | #' targets::tar_outdated() 58 | #' }) 59 | #' } 60 | tar_git_checkout <- function( 61 | ref = "HEAD", 62 | code = getwd(), 63 | store = targets::tar_config_get("store"), 64 | force = FALSE, 65 | verbose = TRUE 66 | ) { 67 | tar_assert_file(code) 68 | tar_assert_file(store) 69 | targets::tar_assert_chr(ref) 70 | targets::tar_assert_scalar(ref) 71 | tar_git_assert_repo_code(code) 72 | tar_git_assert_commits_code(code) 73 | tar_git_assert_repo_data(store) 74 | tar_git_assert_commits_data(store) 75 | ref <- tar_git_commit_code(ref) 76 | commit <- gert::git_commit_info(repo = code, ref = ref)$id 77 | branch <- tar_git_branch_snapshot(commit) 78 | tar_git_assert_snapshot(branch = branch, store = store) 79 | tar_git_branch_checkout(branch = branch, repo = store, force = force) 80 | commit <- gert::git_commit_info(repo = store)$id 81 | message <- gert::git_commit_info(repo = store, ref = commit)$message 82 | message <- first_line(message) 83 | cli_success("Checked out data snapshot ", commit, ".", verbose = verbose) 84 | cli_info("Code commit: ", branch, verbose = verbose) 85 | cli_info("Message: ", first_line(message), verbose = verbose) 86 | cli_info("Resetting to HEAD of checked-out snapshot.", verbose = verbose) 87 | tar_git_reset_hard(repo = store) 88 | invisible() 89 | } 90 | -------------------------------------------------------------------------------- /R/tar_git_init.R: -------------------------------------------------------------------------------- 1 | #' @title Initialize a data repository (Git). 2 | #' @export 3 | #' @family git 4 | #' @description Initialize a Git repository for a `targets` data store. 5 | #' @details `tar_git_init()` also writes a `.gitattributes` file to the 6 | #' store to automatically track target output date with `git-lfs` 7 | #' if it is installed on your system. 8 | #' @section Stashing .gitignore: 9 | #' The `targets` package writes a `.gitignore` file to new data stores 10 | #' in order to prevent accidental commits to the code Git repository. 11 | #' Unfortunately, for `gittargets`, this automatic `.gitignore` file 12 | #' interferes with proper data versioning. So by default, `gittargets` 13 | #' temporarily stashes it to a hidden file called `.gittargets_gitignore` 14 | #' inside the data store. If your R program crashes while the stash 15 | #' is active, you can simply move it manually back to `.gitignore` 16 | #' or run `tar_git_status_data()` to restore the stash automatically 17 | #' if no `.gitignore` already exists. 18 | #' @inheritParams targets::tar_config_set 19 | #' @return `NULL` (invisibly). 20 | #' @param stash_gitignore Logical of length 1, whether to temporarily 21 | #' stash the `.gitignore` file of the data store. See the 22 | #' "Stashing .gitignore" section for details. 23 | #' @param git_lfs Logical, whether to automatically opt into Git LFS to track 24 | #' large files in `_targets/objects` more efficiently. If `TRUE` 25 | #' and Git LFS is installed, it should work automatically. If `FALSE`, 26 | #' you can always opt in later by running `git lfs track objects` 27 | #' inside the data store. 28 | #' @param verbose Logical of length 1, whether to print messages to the 29 | #' R console. 30 | #' @examples 31 | #' if (Sys.getenv("TAR_EXAMPLES") == "true" && tar_git_ok(verbose = FALSE)) { 32 | #' targets::tar_dir({ # Containing code does not modify the user's file space. 33 | #' targets::tar_script(tar_target(data, 1)) 34 | #' targets::tar_make() 35 | #' tar_git_init() 36 | #' }) 37 | #' } 38 | tar_git_init <- function( 39 | store = targets::tar_config_get("store"), 40 | stash_gitignore = TRUE, 41 | git_lfs = TRUE, 42 | verbose = TRUE 43 | ) { 44 | tar_assert_file(store) 45 | targets::tar_assert_lgl(verbose) 46 | targets::tar_assert_scalar(verbose) 47 | if (tar_git_repo_exists(store)) { 48 | cli_info("Data store Git repository already exists.") 49 | cli_info("Remove ", file.path(store, ".git"), " to start over.") 50 | return(invisible()) 51 | } 52 | if (stash_gitignore) { 53 | tar_git_gitignore_restore(repo = store) 54 | tar_git_gitignore_stash(repo = store) 55 | on.exit(tar_git_gitignore_unstash(repo = store)) 56 | } 57 | tar_git_init_repo(path = store) 58 | cli_success("Created data store Git repository", verbose = verbose) 59 | if (git_lfs) { 60 | tar_git_init_lfs(repo = store, verbose = verbose) 61 | } 62 | tar_git_init_stub(repo = store, verbose = verbose) 63 | tar_git_commit(message = "Stub commit", repo = store, spinner = FALSE) 64 | cli_success("Created stub commit without data.", verbose = verbose) 65 | cli_info( 66 | "Run tar_git_snapshot() to put the data files under version control.", 67 | verbose = verbose 68 | ) 69 | invisible() 70 | } 71 | 72 | tar_git_init_lfs <- function(repo, verbose) { 73 | gitattributes <- file.path(repo, ".gitattributes") 74 | lines <- c( 75 | "objects filter=lfs diff=lfs merge=lfs -text", 76 | "objects/* filter=lfs diff=lfs merge=lfs -text", 77 | "objects/** filter=lfs diff=lfs merge=lfs -text", 78 | "objects/**/* filter=lfs diff=lfs merge=lfs -text" 79 | ) 80 | write_new_lines(path = gitattributes, lines = lines) 81 | cli_success( 82 | "Wrote to ", 83 | gitattributes, 84 | " for git-lfs: {.url https://git-lfs.com}.", 85 | verbose = verbose 86 | ) 87 | tar_git_add(files = basename(gitattributes), repo = repo, spinner = FALSE) 88 | } 89 | 90 | tar_git_init_stub <- function(repo, verbose) { 91 | tar_git_stub_write(repo = repo) 92 | tar_git_add( 93 | files = basename(tar_git_stub_path(repo)), 94 | repo = repo, 95 | spinner = FALSE 96 | ) 97 | } 98 | -------------------------------------------------------------------------------- /R/tar_git_log.R: -------------------------------------------------------------------------------- 1 | #' @title Data snapshots of a code branch (Git) 2 | #' @export 3 | #' @family git 4 | #' @description Show all the data snapshots of a code branch. 5 | #' @details By design, `tar_git_log()` only queries a single 6 | #' code branch at a time. This allows `tar_git_log()` 7 | #' to report more detailed information about the snapshots 8 | #' of the given code branch. 9 | #' To query all data snapshots over all branches, simply run 10 | #' `gert::git_branch_list(local = TRUE, repo = "_targets")`. 11 | #' The valid snapshots show `"code="` in the `name` column, 12 | #' where `` is the Git commit hash of the code commit 13 | #' corresponding to the data snapshot. 14 | #' @return A data frame of information about 15 | #' data snapshots and code commits. 16 | #' @inheritParams gert::git_log 17 | #' @inheritParams tar_git_status 18 | #' @param branch Character of length 1, name of the code repository branch 19 | #' to query. Defaults to the currently checked-out code branch. 20 | #' @param max Positive numeric of length 1, maximum number of code commits 21 | #' to inspect for the given branch. 22 | #' @examples 23 | #' if (Sys.getenv("TAR_EXAMPLES") == "true" && tar_git_ok(verbose = FALSE)) { 24 | #' targets::tar_dir({ # Containing code does not modify the user's filespace. 25 | #' targets::tar_script(tar_target(data, 1)) 26 | #' targets::tar_make() 27 | #' gert::git_init() 28 | #' gert::git_add("_targets.R") 29 | #' gert::git_commit("First commit") 30 | #' tar_git_init() 31 | #' tar_git_snapshot(status = FALSE, verbose = FALSE) 32 | #' tar_git_log() 33 | #' }) 34 | #' } 35 | tar_git_log <- function( 36 | code = getwd(), 37 | store = targets::tar_config_get("store"), 38 | branch = gert::git_branch(repo = code), 39 | max = 100 40 | ) { 41 | tar_assert_file(code) 42 | tar_assert_file(store) 43 | targets::tar_assert_chr(branch) 44 | targets::tar_assert_scalar(branch) 45 | targets::tar_assert_dbl(max) 46 | targets::tar_assert_positive(max) 47 | targets::tar_assert_scalar(max) 48 | tar_assert_finite(max, msg = "max must be finite.") 49 | tar_git_assert_repo_code(code) 50 | tar_git_assert_commits_code(code) 51 | tar_git_assert_repo_data(store) 52 | tar_git_assert_commits_data(store) 53 | raw_code <- gert::git_log(ref = branch, max = as.integer(max), repo = code) 54 | raw_data <- gert::git_branch_list(local = TRUE, repo = store) 55 | log_code <- tibble::tibble( 56 | commit_code = raw_code$commit, 57 | time_code = raw_code$time, 58 | message_code = trimws(raw_code$message) 59 | ) 60 | log_data <- tibble::tibble( 61 | commit_code = tar_git_commit_code(raw_data$name), 62 | commit_data = raw_data$commit, 63 | time_data = raw_data$updated 64 | ) 65 | log <- inner_merge(x = log_data, y = log_code, by = "commit_code") 66 | log$message_data <- vapply( 67 | X = log$commit_data, 68 | FUN = tar_git_log_data_message, 69 | store = store, 70 | FUN.VALUE = character(1), 71 | USE.NAMES = FALSE 72 | ) 73 | cols <- c( 74 | "message_code", 75 | "message_data", 76 | "time_code", 77 | "time_data", 78 | "commit_code", 79 | "commit_data" 80 | ) 81 | log[, cols] 82 | } 83 | 84 | tar_git_log_data_message <- function(commit, store) { 85 | trimws(gert::git_commit_info(ref = commit, repo = store)$message) 86 | } 87 | -------------------------------------------------------------------------------- /R/tar_git_ok.R: -------------------------------------------------------------------------------- 1 | #' @title Check Git 2 | #' @export 3 | #' @family git 4 | #' @description Check if Git is installed and if `user.name` and `user.email` 5 | #' are configured globally. 6 | #' @details You can install Git from 7 | #' and configure your identity using the instructions at 8 | #' . 9 | #' You may find it convenient to run `gert::git_config_global()` 10 | #' with `name` equal to `user.name` and `user.email`. 11 | #' @return Logical of length 1, whether Git is installed and configured 12 | #' correctly. 13 | #' @param verbose Whether to print messages to the console. 14 | #' @examples 15 | #' tar_git_ok() 16 | tar_git_ok <- function(verbose = TRUE) { 17 | binary <- tar_git_binary() 18 | cli_success("Git binary: {.file ", binary, "}", verbose = verbose) 19 | success <- tryCatch({ 20 | user_name <- tar_git_config_global_user_name() 21 | user_email <- tar_git_config_global_user_email() 22 | TRUE 23 | }, error = function(condition) { 24 | cli_danger( 25 | "Error getting Git global user name and email:", 26 | verbose = verbose 27 | ) 28 | cli_danger(conditionMessage(condition), verbose = verbose) 29 | FALSE 30 | }) 31 | if (!success) { 32 | return(FALSE) 33 | } 34 | if_any( 35 | nzchar(user_name), 36 | cli_success( 37 | "Git config global user name: {.field ", 38 | user_name, 39 | "}", 40 | verbose = verbose 41 | ), 42 | cli_danger( 43 | "No Git config global user email. See details in ?tar_git_ok().", 44 | verbose = verbose 45 | ) 46 | ) 47 | if_any( 48 | nzchar(user_email), 49 | cli_success( 50 | "Git config global user email: {.email ", 51 | user_email, 52 | "}", 53 | verbose = verbose 54 | ), 55 | cli_danger( 56 | "No Git config global user email. See details in ?tar_git_ok().", 57 | verbose = verbose 58 | ) 59 | ) 60 | length(user_name) && 61 | nzchar(user_name) && 62 | length(user_email) && 63 | nzchar(user_email) 64 | } 65 | -------------------------------------------------------------------------------- /R/tar_git_snapshot.R: -------------------------------------------------------------------------------- 1 | #' @title Snapshot the data repository (Git). 2 | #' @export 3 | #' @family git 4 | #' @description Snapshot the Git data repository of a `targets` project. 5 | #' @details A Git-backed `gittargets` data snapshot is a special kind of 6 | #' Git commit. Every data commit is part of a branch specific to 7 | #' the current code commit. 8 | #' That way, when you switch branches or commits in the code, 9 | #' `tar_git_checkout()` checks out the latest data snapshot 10 | #' that matches the code in your workspace. 11 | #' That way, your targets can stay up to date even as you 12 | #' transition among multiple branches. 13 | #' @inheritSection tar_git_init Stashing .gitignore 14 | #' @inheritParams tar_git_status 15 | #' @param message Optional Git commit message of the data snapshot. 16 | #' If `NULL`, then the message is the Git commit message of the 17 | #' matching code commit. 18 | #' @param ref Character of length 1, reference 19 | #' (branch name, Git SHA1 hash, etc.) of the code commit 20 | #' that will map to the new data snapshot. Defaults to the commit 21 | #' checked out right now. 22 | #' @param status Logical of length 1, whether to print the project status 23 | #' with [tar_git_status()] and ask whether a snapshot should be created. 24 | #' @param verbose Logical of length 1, whether to print R console messages 25 | #' confirming that a snapshot was created. 26 | #' @param force Logical of length 1. Force checkout the data branch 27 | #' of an existing data snapshot of the current code commit? 28 | #' @param pack_refs Logical of length 1, whether to run `git pack-refs --all` 29 | #' in the data store after taking the snapshot. Packing references 30 | #' improves efficiency when the number of snapshots is large. 31 | #' Learn more at . 32 | #' @examples 33 | #' if (Sys.getenv("TAR_EXAMPLES") == "true" && tar_git_ok(verbose = FALSE)) { 34 | #' targets::tar_dir({ # Containing code does not modify the user's filespace. 35 | #' targets::tar_script(tar_target(data, 1)) 36 | #' targets::tar_make() 37 | #' gert::git_init() 38 | #' gert::git_add("_targets.R") 39 | #' gert::git_commit("First commit") 40 | #' tar_git_init() 41 | #' tar_git_snapshot(status = FALSE) 42 | #' }) 43 | #' } 44 | tar_git_snapshot <- function( 45 | message = NULL, 46 | ref = "HEAD", 47 | code = getwd(), 48 | script = targets::tar_config_get("script"), 49 | store = targets::tar_config_get("store"), 50 | stash_gitignore = TRUE, 51 | reporter = targets::tar_config_get("reporter_outdated"), 52 | envir = parent.frame(), 53 | callr_function = callr::r, 54 | callr_arguments = NULL, 55 | status = interactive(), 56 | force = FALSE, 57 | pack_refs = TRUE, 58 | verbose = TRUE 59 | ) { 60 | callr_arguments <- callr_arguments %|||% callr_args_default( 61 | callr_function = callr_function, 62 | reporter = reporter 63 | ) 64 | tar_assert_file(code) 65 | tar_assert_file(store) 66 | targets::tar_assert_lgl(status) 67 | targets::tar_assert_scalar(status) 68 | targets::tar_assert_lgl(force) 69 | targets::tar_assert_scalar(force) 70 | targets::tar_assert_lgl(pack_refs) 71 | targets::tar_assert_scalar(pack_refs) 72 | targets::tar_assert_lgl(verbose) 73 | targets::tar_assert_scalar(verbose) 74 | tar_git_assert_repo_code(code) 75 | tar_git_assert_commits_code(code) 76 | tar_git_assert_repo_data(store) 77 | log <- gert::git_log(repo = code, max = 1L) 78 | commit <- gert::git_commit_info(repo = code, ref = ref)$id 79 | branch <- tar_git_branch_snapshot(commit) 80 | code_message <- gert::git_commit_info(repo = code, ref = ref)$message 81 | # Covered in tests/interactive/test-tar_git_snapshot.R 82 | # nocov start 83 | if (status) { 84 | choice <- tar_git_snapshot_menu( 85 | commit = commit, 86 | message = code_message, 87 | code = code, 88 | script = script, 89 | store = store, 90 | stash_gitignore = stash_gitignore, 91 | reporter = reporter, 92 | envir = envir, 93 | callr_function = callr_function, 94 | callr_arguments = callr_arguments 95 | ) 96 | if (!identical(as.integer(choice), 1L)) { 97 | cli_info("Snapshot skipped.", verbose = verbose) 98 | return(invisible()) 99 | } 100 | } 101 | # nocov end 102 | if (stash_gitignore) { 103 | tar_git_gitignore_restore(repo = store) 104 | tar_git_gitignore_stash(repo = store) 105 | on.exit(tar_git_gitignore_unstash(repo = store)) 106 | } 107 | tar_git_stub_write(repo = store) 108 | if_any( 109 | gert::git_branch_exists(branch = branch, repo = store), 110 | tar_git_snapshot_branch_exists(branch = branch, verbose = verbose), 111 | tar_git_snapshot_branch_create( 112 | branch = branch, 113 | repo = store, 114 | verbose = verbose 115 | ) 116 | ) 117 | tar_git_branch_checkout(branch = branch, repo = store, force = force) 118 | cli_info("Staging data files.", verbose = verbose) 119 | tar_git_add(files = "*", repo = store) 120 | staged <- gert::git_status(staged = TRUE, repo = store) 121 | staged$staged <- NULL 122 | cli_success( 123 | sprintf("Staged %s files in the data store.", nrow(staged)), 124 | verbose = verbose 125 | ) 126 | cli_info("Committing data changes.", verbose = verbose) 127 | tar_git_commit_all(message = message %|||% code_message, repo = store) 128 | commit <- gert::git_commit_info(repo = store)$id 129 | cli_success( 130 | sprintf("Created new data snapshot %s.", commit), 131 | verbose = verbose 132 | ) 133 | if (pack_refs) { 134 | cli_info("Packing references.", verbose = verbose) 135 | tar_git_pack_refs(repo = store, spinner = verbose) 136 | } 137 | invisible() 138 | } 139 | 140 | tar_git_snapshot_branch_create <- function(branch, repo, verbose) { 141 | cli_info(sprintf("Creating data branch %s.", branch), verbose = verbose) 142 | tar_git_branch_create(branch = branch, repo = repo) 143 | } 144 | 145 | tar_git_snapshot_branch_exists <- function(branch, verbose) { 146 | cli_info( 147 | "Data snapshot already exists for the current code commit.", 148 | verbose = verbose 149 | ) 150 | cli_info( 151 | "The new data snapshot will supersede the old one in data branch:", 152 | verbose = verbose 153 | ) 154 | cli_indent("{.field ", branch, "}", verbose = verbose) 155 | } 156 | 157 | # Covered in tests/interactive/test-tar_git_snapshot.R 158 | # nocov start 159 | #' @title Data snapshot menu (Git) 160 | #' @keywords internal 161 | #' @description Check the project status and show an interactive menu 162 | #' for [tar_git_snapshot()]. 163 | #' @return Integer of length 1: `2L` if the user agrees to snapshot, 164 | #' `1L` if the user declines. 165 | #' @inheritParams tar_git_snapshot 166 | #' @param commit Character of length 1, Git SHA1 hash of the code commit 167 | #' that will correspond to the data snapshot (if created). 168 | #' @examples 169 | #' # See the examples of tar_git_snapshot(). 170 | tar_git_snapshot_menu <- function( 171 | commit, 172 | message, 173 | code, 174 | script, 175 | store, 176 | stash_gitignore, 177 | reporter, 178 | envir, 179 | callr_function, 180 | callr_arguments 181 | ) { 182 | tar_git_status( 183 | code = code, 184 | script = script, 185 | store = store, 186 | stash_gitignore = stash_gitignore, 187 | reporter = reporter, 188 | envir = envir, 189 | callr_function = callr_function, 190 | callr_arguments = callr_arguments 191 | ) 192 | cli::cli_h1("Snapshot the data?") 193 | line <- paste( 194 | "The new snapshot will be a data commit", 195 | "that maps to the following code commit:" 196 | ) 197 | cli_info(line) 198 | cli_indent(commit) 199 | cli_indent(first_line(message)) 200 | line <- paste( 201 | "Please make sure the code repo and", 202 | "{.pkg targets} pipeline are clean and up to date." 203 | ) 204 | cli_info(line) 205 | utils::menu(c("yes", "no")) 206 | } 207 | # nocov end 208 | -------------------------------------------------------------------------------- /R/tar_git_status.R: -------------------------------------------------------------------------------- 1 | #' @title Status of the project (Git) 2 | #' @export 3 | #' @family git 4 | #' @description Print the status of the code repository, 5 | #' the data repository, and the targets. 6 | #' @inheritSection tar_git_init Stashing .gitignore 7 | #' @return `NULL` (invisibly). Status information is printed 8 | #' to the R console. 9 | #' @inheritParams tar_git_status_code 10 | #' @inheritParams tar_git_status_data 11 | #' @inheritParams tar_git_status_targets 12 | #' @examples 13 | #' if (Sys.getenv("TAR_EXAMPLES") == "true" && tar_git_ok(verbose = FALSE)) { 14 | #' targets::tar_dir({ # Containing code does not modify the user's files pace. 15 | #' targets::tar_script(tar_target(data, 1)) 16 | #' targets::tar_make() 17 | #' list.files("_targets", all.files = TRUE) 18 | #' gert::git_init() 19 | #' tar_git_init() 20 | #' tar_git_status() 21 | #' }) 22 | #' } 23 | tar_git_status <- function( 24 | code = getwd(), 25 | script = targets::tar_config_get("script"), 26 | store = targets::tar_config_get("store"), 27 | stash_gitignore = TRUE, 28 | reporter = targets::tar_config_get("reporter_outdated"), 29 | envir = parent.frame(), 30 | callr_function = callr::r, 31 | callr_arguments = NULL 32 | ) { 33 | callr_arguments <- callr_arguments %|||% callr_args_default( 34 | callr_function = callr_function, 35 | reporter = reporter 36 | ) 37 | cli::cli_h1("Data Git status") 38 | status <- tar_git_status_data(store, stash_gitignore) 39 | if_any( 40 | is.null(status), 41 | tar_git_status_data_none(), 42 | tar_git_status_data_message(status) 43 | ) 44 | cli::cli_h1("Code Git status") 45 | status <- tar_git_status_code(code) 46 | if_any( 47 | is.null(status), 48 | tar_git_status_code_none(), 49 | tar_git_status_code_message(status) 50 | ) 51 | cli::cli_h1("Outdated targets") 52 | status <- tar_git_status_targets( 53 | script = script, 54 | store = store, 55 | reporter = reporter, 56 | envir = envir, 57 | callr_function = callr_function, 58 | callr_arguments = callr_arguments 59 | ) 60 | if_any( 61 | nrow(status), 62 | cli_df(status), 63 | cli_success("All targets are up to date.") 64 | ) 65 | invisible() 66 | } 67 | 68 | tar_git_status_code_message <- function(status) { 69 | if_any( 70 | nrow(status), 71 | cli_df(status), 72 | cli_success("Code repository is clean.") 73 | ) 74 | } 75 | 76 | tar_git_status_code_none <- function() { 77 | cli_danger("Code has no Git repository.") 78 | cli_warning("Create one with {.code gert::git_init()}).") 79 | } 80 | 81 | tar_git_status_data_message <- function(status) { 82 | if_any( 83 | nrow(status), 84 | cli_df(status), 85 | cli_success("Data repository is clean.") 86 | ) 87 | } 88 | 89 | tar_git_status_data_none <- function() { 90 | cli_danger("No Git repository for the data store.") 91 | cli_warning("Create one with {.code gittargets::tar_git_init()}.") 92 | } 93 | -------------------------------------------------------------------------------- /R/tar_git_status_code.R: -------------------------------------------------------------------------------- 1 | #' @title Status of the code repository (Git) 2 | #' @export 3 | #' @family git 4 | #' @description Show the Git status of the code repository. 5 | #' @return If the code repository exists, the return value is the data frame 6 | #' produced by `gert::git_status(repo = code)`. If the code has no Git 7 | #' repository, then the return value is `NULL`. 8 | #' @param code Character of length 1, directory path to the code repository, 9 | #' usually the root of the `targets` project. 10 | #' @examples 11 | #' if (Sys.getenv("TAR_EXAMPLES") == "true" && tar_git_ok(verbose = FALSE)) { 12 | #' targets::tar_dir({ # Containing code does not modify the user's file space. 13 | #' targets::tar_script(tar_target(data, 1)) 14 | #' targets::tar_make() 15 | #' list.files("_targets", all.files = TRUE) 16 | #' gert::git_init() 17 | #' tar_git_init() 18 | #' tar_git_status_code() 19 | #' }) 20 | #' } 21 | tar_git_status_code <- function(code = getwd()) { 22 | tar_assert_file(code) 23 | if_any( 24 | tar_git_repo_exists(repo = code), 25 | gert::git_status(repo = code), 26 | NULL 27 | ) 28 | } 29 | -------------------------------------------------------------------------------- /R/tar_git_status_data.R: -------------------------------------------------------------------------------- 1 | #' @title Status of the data repository (Git) 2 | #' @export 3 | #' @family git 4 | #' @description Show the Git status of the data repository. 5 | #' @inheritSection tar_git_init Stashing .gitignore 6 | #' @return If the data repository exists, the return value is the data frame 7 | #' produced by `gert::git_status(repo = store)`. If the data store has no Git 8 | #' repository, then the return value is `NULL`. 9 | #' @inheritParams tar_git_init 10 | #' @examples 11 | #' if (Sys.getenv("TAR_EXAMPLES") == "true" && tar_git_ok(verbose = FALSE)) { 12 | #' targets::tar_dir({ # Containing code does not modify the user's file space. 13 | #' targets::tar_script(tar_target(data, 1)) 14 | #' targets::tar_make() 15 | #' list.files("_targets", all.files = TRUE) 16 | #' gert::git_init() 17 | #' tar_git_init() 18 | #' tar_git_status_data() 19 | #' }) 20 | #' } 21 | tar_git_status_data <- function( 22 | store = targets::tar_config_get("store"), 23 | stash_gitignore = TRUE 24 | ) { 25 | tar_assert_file(store) 26 | if (stash_gitignore) { 27 | tar_git_gitignore_restore(repo = store) 28 | tar_git_gitignore_stash(repo = store) 29 | on.exit(tar_git_gitignore_unstash(repo = store)) 30 | } 31 | if_any( 32 | tar_git_repo_exists(repo = store), 33 | gert::git_status(repo = store), 34 | NULL 35 | ) 36 | } 37 | -------------------------------------------------------------------------------- /R/tar_git_status_targets.R: -------------------------------------------------------------------------------- 1 | #' @title Status of the targets (Git) 2 | #' @export 3 | #' @family git 4 | #' @description Show which targets are outdated. 5 | #' @details This function has prettier output than `targets::tar_outdated()`, 6 | #' and it mainly serves [tar_git_status()]. 7 | #' @return A `tibble` with the names of outdated targets. 8 | #' @inheritParams targets::tar_outdated 9 | #' @examples 10 | #' targets::tar_dir({ # Containing code does not modify the user's file space. 11 | #' targets::tar_script(tar_target(data, 1)) 12 | #' targets::tar_make() 13 | #' list.files("_targets", all.files = TRUE) 14 | #' tar_git_status_targets() 15 | #' }) 16 | tar_git_status_targets <- function( 17 | script = targets::tar_config_get("script"), 18 | store = targets::tar_config_get("store"), 19 | reporter = targets::tar_config_get("reporter_outdated"), 20 | envir = parent.frame(), 21 | callr_function = callr::r, 22 | callr_arguments = NULL 23 | ) { 24 | callr_arguments <- callr_arguments %|||% callr_args_default( 25 | callr_function = callr_function, 26 | reporter = reporter 27 | ) 28 | outdated <- targets::tar_outdated( 29 | reporter = reporter, 30 | envir = envir, 31 | script = script, 32 | store = store, 33 | callr_function = callr_function, 34 | callr_arguments = callr_arguments 35 | ) 36 | tibble::as_tibble(list(outdated = outdated)) 37 | } 38 | -------------------------------------------------------------------------------- /R/tar_package.R: -------------------------------------------------------------------------------- 1 | #' targets: Dynamic Function-Oriented Make-Like Declarative Pipelines for R 2 | #' @docType package 3 | #' @name gittargets-package 4 | #' @description In computationally demanding data analysis pipelines, 5 | #' the `targets` R package maintains an up-to-date set of results 6 | #' while skipping tasks that do not need to rerun. This process 7 | #' increases speed and increases trust in the final end product. 8 | #' However, it also overwrites old output with new output, 9 | #' and past results disappear by default. To preserve historical output, 10 | #' the `gittargets` package captures version-controlled snapshots 11 | #' of the data store, and each snapshot links to the underlying 12 | #' commit of the source code. That way, when the user rolls back 13 | #' the code to a previous branch or commit, `gittargets` can recover 14 | #' the data contemporaneous with that commit so that all targets 15 | #' remain up to date. 16 | #' @family help 17 | #' @importFrom callr r 18 | #' @importFrom cli cli_alert_danger cli_alert_info cli_alert_success cli_h1 19 | #' col_br_white 20 | #' @importFrom data.table as.data.table 21 | #' @importFrom gert git_branch git_branch_exists git_branch_list 22 | #' git_commit_info git_log git_status 23 | #' @importFrom processx run 24 | #' @importFrom stats complete.cases 25 | #' @importFrom targets tar_config_get tar_config_set tar_dir 26 | #' tar_outdated tar_throw_validate 27 | #' @importFrom tibble as_tibble tibble 28 | #' @importFrom utils capture.output head menu 29 | #' @importFrom uuid UUIDgenerate 30 | NULL 31 | -------------------------------------------------------------------------------- /R/utils_assert.R: -------------------------------------------------------------------------------- 1 | # TODO: import these function from `targets` 2 | # when the version supporting it is on CRAN. 3 | tar_assert_file <- function(x) { 4 | name <- deparse(substitute(x)) 5 | targets::tar_assert_chr(x, paste(name, "must be a character string.")) 6 | targets::tar_assert_scalar(x, paste(name, "must have length 1.")) 7 | targets::tar_assert_path(x) 8 | } 9 | 10 | tar_assert_finite <- function(x, msg = NULL) { 11 | name <- deparse(substitute(x)) 12 | default <- paste("all of", name, "must be finite") 13 | if (!all(is.finite(x))) { 14 | targets::tar_throw_validate(msg %|||% default) 15 | } 16 | } 17 | 18 | # gittargets-specific assertions: 19 | tar_git_assert_commits_code <- function(code) { 20 | no_commits <- is.null(gert::git_branch(repo = code)) || 21 | !nrow(gert::git_log(max = 1, repo = code)) 22 | if (no_commits) { 23 | msg <- paste( 24 | "The code repository has no commits.", 25 | "Create one with gert::git_add() and gert::git_commit()." 26 | ) 27 | targets::tar_throw_validate(msg) 28 | } 29 | } 30 | 31 | tar_git_assert_commits_data <- function(store) { 32 | no_commits <- is.null(gert::git_branch(repo = store)) || 33 | !nrow(gert::git_log(max = 1, repo = store)) 34 | if (no_commits) { 35 | msg <- paste( 36 | "The data repository has no commits.", 37 | "Create one with gittargets::tar_git_snapshot()." 38 | ) 39 | targets::tar_throw_validate(msg) 40 | } 41 | } 42 | 43 | tar_git_assert_repo_code <- function(code) { 44 | if (!tar_git_repo_exists(code)) { 45 | msg <- paste( 46 | "No Git repository for the code.", 47 | "Create one with gert::git_init()." 48 | ) 49 | targets::tar_throw_validate(msg) 50 | } 51 | } 52 | 53 | tar_git_assert_repo_data <- function(store) { 54 | if (!tar_git_repo_exists(store)) { 55 | msg <- paste( 56 | "No Git repository for the data store.", 57 | "Create one with gittargets::tar_git_init()." 58 | ) 59 | targets::tar_throw_validate(msg) 60 | } 61 | } 62 | 63 | tar_git_assert_snapshot <- function(branch, store) { 64 | if (!gert::git_branch_exists(branch = branch, repo = store, local = TRUE)) { 65 | msg <- paste0( 66 | "No data snapshot for code commit ", 67 | branch, 68 | ". Create one with gittargets::tar_git_snapshot()." 69 | ) 70 | targets::tar_throw_validate(msg) 71 | } 72 | } 73 | -------------------------------------------------------------------------------- /R/utils_callr.R: -------------------------------------------------------------------------------- 1 | callr_args_default <- function(callr_function, reporter = NULL) { 2 | if (is.null(callr_function)) { 3 | return(list()) 4 | } 5 | out <- list(spinner = !identical(reporter, "summary")) 6 | out[intersect(names(out), names(formals(callr_function)))] 7 | } 8 | -------------------------------------------------------------------------------- /R/utils_cli.R: -------------------------------------------------------------------------------- 1 | cli_danger <- function(..., verbose = TRUE) { 2 | if (verbose) { 3 | cli::cli_alert_danger(paste0(...)) 4 | } 5 | } 6 | 7 | cli_df <- function(x, verbose = TRUE) { 8 | if (verbose) { 9 | lines <- utils::capture.output(x, type = "output") 10 | message(cli::col_br_white(paste0(lines, "\n"))) 11 | } 12 | } 13 | 14 | cli_info <- function(..., verbose = TRUE) { 15 | if (verbose) { 16 | cli_blue_bullet(paste0(...), verbose = verbose) 17 | } 18 | } 19 | 20 | cli_success <- function(..., verbose = TRUE) { 21 | if (verbose) { 22 | cli::cli_alert_success(paste0(...)) 23 | } 24 | } 25 | 26 | cli_warning <- function(..., verbose = TRUE) { 27 | if (verbose) { 28 | cli::cli_alert_warning(paste0(...)) 29 | } 30 | } 31 | 32 | cli_blue_bullet <- function(..., verbose = TRUE) { 33 | if (verbose) { 34 | symbol <- cli::col_blue(cli::symbol$bullet) 35 | cli::cli_text(paste(symbol, paste0(...))) 36 | } 37 | } 38 | 39 | cli_indent <- function(..., verbose = TRUE) { 40 | if (verbose) { 41 | cli::cli_bullets(c(" " = paste0(...))) 42 | } 43 | } 44 | -------------------------------------------------------------------------------- /R/utils_data.R: -------------------------------------------------------------------------------- 1 | first_line <- function(x) { 2 | trimws(utils::head(unlist(strsplit(x, split = "\n")), n = 1L)) 3 | } 4 | 5 | inner_merge <- function(x, y, by) { 6 | out <- merge( 7 | x = data.table::as.data.table(x), 8 | y = data.table::as.data.table(y), 9 | by = by, 10 | all.x = FALSE, 11 | all.y = FALSE 12 | ) 13 | out <- as.list(out) 14 | names <- names(out) 15 | attributes(out) <- NULL 16 | names(out) <- names 17 | tibble::as_tibble(out) 18 | } 19 | 20 | write_new_lines <- function(lines, path) { 21 | old_lines <- if_any(file.exists(path), readLines(path), character(0)) 22 | all_lines <- union(x = old_lines, y = lines) 23 | writeLines(all_lines, path) 24 | } 25 | -------------------------------------------------------------------------------- /R/utils_git.R: -------------------------------------------------------------------------------- 1 | tar_git_add <- function(files, repo, spinner = TRUE) { 2 | processx::run( 3 | command = tar_git_binary(), 4 | args = c("add", files), 5 | wd = repo, 6 | echo = FALSE, 7 | spinner = spinner, 8 | env = tar_git_env() 9 | ) 10 | } 11 | 12 | tar_git_branch_checkout <- function(branch, repo, force) { 13 | args <- c("checkout", if_any(force, "--force", character(0)), branch) 14 | processx::run( 15 | command = tar_git_binary(), 16 | args = args, 17 | wd = repo, 18 | echo = FALSE, 19 | env = tar_git_env() 20 | ) 21 | } 22 | 23 | tar_git_branch_create <- function(branch, repo) { 24 | processx::run( 25 | command = tar_git_binary(), 26 | args = c("branch", branch), 27 | wd = repo, 28 | echo = FALSE, 29 | env = tar_git_env() 30 | ) 31 | } 32 | 33 | tar_git_commit <- function(message, repo, spinner = TRUE) { 34 | processx::run( 35 | command = tar_git_binary(), 36 | args = c("commit", "--message", message), 37 | wd = repo, 38 | echo = FALSE, 39 | spinner = spinner, 40 | env = tar_git_env() 41 | ) 42 | } 43 | 44 | tar_git_commit_all <- function(message, repo, spinner = TRUE) { 45 | processx::run( 46 | command = tar_git_binary(), 47 | args = c("commit", "--all", "--message", message), 48 | wd = repo, 49 | echo = FALSE, 50 | spinner = spinner, 51 | env = tar_git_env() 52 | ) 53 | } 54 | 55 | tar_git_init_repo <- function(path) { 56 | processx::run( 57 | command = tar_git_binary(), 58 | args = "init", 59 | wd = path, 60 | env = tar_git_env() 61 | ) 62 | } 63 | 64 | tar_git_pack_refs <- function(repo, spinner = TRUE) { 65 | processx::run( 66 | command = tar_git_binary(), 67 | args = c("pack-refs", "--all"), 68 | wd = repo, 69 | echo = FALSE, 70 | spinner = spinner, 71 | env = tar_git_env() 72 | ) 73 | } 74 | 75 | tar_git_reset_hard <- function(repo, spinner = TRUE) { 76 | processx::run( 77 | command = tar_git_binary(), 78 | args = c("reset", "--hard", "HEAD"), 79 | wd = repo, 80 | echo = FALSE, 81 | spinner = spinner, 82 | env = tar_git_env() 83 | ) 84 | } 85 | 86 | # Get a data snapshot branch name from a code commit hash. 87 | # The branch name refers to the code commit with "code=" 88 | # followed by the commit hash. 89 | tar_git_branch_snapshot <- function(commit) { 90 | sprintf("code=%s", commit) 91 | } 92 | 93 | # Get a code commit hash from a data snapshot branch name. 94 | tar_git_commit_code <- function(branch) { 95 | gsub(pattern = "^code=", replacement = "", x = branch) 96 | } 97 | 98 | tar_git_binary <- function() { 99 | out <- Sys.getenv("TAR_GIT", unset = Sys.which("git")) 100 | msg <- paste( 101 | "no existing Git installation found.", 102 | "Install from https://git-scm.com/downloads.", 103 | "If you already installed Git", 104 | "set the TAR_GIT environment variable to the path of the", 105 | "Git executable. Functions usethis::edit_r_environ() and", 106 | "Sys.setenv() can help.", 107 | "Sys.getenv(\"TAR_GIT\", unset = Sys.which(\"git\"))", 108 | "is currently", 109 | out 110 | ) 111 | targets::tar_assert_chr( 112 | out, 113 | "Sys.getenv(\"TAR_GIT\", unset = Sys.which(\"git\")) must be a character." 114 | ) 115 | targets::tar_assert_scalar( 116 | out, 117 | "Sys.getenv(\"TAR_GIT\", unset = Sys.which(\"git\")) must have length 1." 118 | ) 119 | targets::tar_assert_nzchar(out, msg = msg) 120 | targets::tar_assert_path(out, msg = msg) 121 | out 122 | } 123 | 124 | tar_git_repo_exists <- function(repo) { 125 | file.exists(file.path(repo, ".git")) 126 | } 127 | 128 | tar_git_gitignore_stash <- function(repo) { 129 | from <- file.path(repo, ".gitignore") 130 | to <- file.path(repo, ".gittargets_gitignore") 131 | if (file.exists(from)) { 132 | file.rename(from = from, to = to) 133 | writeLines(tar_git_gitignore_lines(), from) 134 | } 135 | invisible() 136 | } 137 | 138 | tar_git_gitignore_restore <- function(repo) { 139 | from <- file.path(repo, ".gittargets_gitignore") 140 | to <- file.path(repo, ".gitignore") 141 | has_lines <- file.exists(to) && 142 | identical(readLines(to), tar_git_gitignore_lines()) 143 | restore <- file.exists(from) && (!file.exists(to) || has_lines) 144 | if (restore) { 145 | file.rename(from = from, to = to) 146 | } 147 | invisible() 148 | } 149 | 150 | tar_git_gitignore_unstash <- function(repo) { 151 | from <- file.path(repo, ".gittargets_gitignore") 152 | to <- file.path(repo, ".gitignore") 153 | if (file.exists(from)) { 154 | file.rename(from = from, to = to) 155 | } 156 | invisible() 157 | } 158 | 159 | tar_git_gitignore_lines <- function() { 160 | c( 161 | "# Generated by gittargets: do not edit by hand", 162 | ".gitignore", 163 | ".gittargets_gitignore" 164 | ) 165 | } 166 | 167 | tar_git_stub_path <- function(repo) { 168 | file.path(repo, ".gittargets") 169 | } 170 | 171 | tar_git_stub_write <- function(repo) { 172 | path <- tar_git_stub_path(repo) 173 | uuid <- uuid::UUIDgenerate(use.time = NA, n = 1L) 174 | writeLines(uuid, path) 175 | } 176 | 177 | tar_git_config_global_user_name <- function() { 178 | out <- processx::run( 179 | command = tar_git_binary(), 180 | args = c("config", "--global", "user.name"), 181 | echo = FALSE, 182 | env = tar_git_env() 183 | ) 184 | trimws(out$stdout) 185 | } 186 | 187 | tar_git_config_global_user_email <- function() { 188 | out <- processx::run( 189 | command = tar_git_binary(), 190 | args = c("config", "--global", "user.email"), 191 | echo = FALSE, 192 | env = tar_git_env() 193 | ) 194 | trimws(out$stdout) 195 | } 196 | 197 | tar_git_env <- function() { 198 | if_any( 199 | identical(as.character(tolower(Sys.info()["sysname"])), "windows"), 200 | c("current", HOME = Sys.getenv("USERPROFILE")), 201 | NULL 202 | ) 203 | } 204 | -------------------------------------------------------------------------------- /R/utils_logic.R: -------------------------------------------------------------------------------- 1 | `%|||%` <- function(x, y) { 2 | if (is.null(x)) { 3 | y 4 | } else { 5 | x 6 | } 7 | } 8 | 9 | if_any <- function(condition, x, y) { 10 | if (any(condition)) { 11 | x 12 | } else { 13 | y 14 | } 15 | } 16 | -------------------------------------------------------------------------------- /README.Rmd: -------------------------------------------------------------------------------- 1 | --- 2 | output: github_document 3 | --- 4 | 5 | ```{r, include = FALSE} 6 | knitr::opts_chunk$set( 7 | collapse = TRUE, 8 | comment = "#>", 9 | fig.path = "man/figures/README-", 10 | out.width = "100%" 11 | ) 12 | ``` 13 | 14 | # gittargets 15 | 16 | [![ropensci](https://badges.ropensci.org/486_status.svg)](https://github.com/ropensci/software-review/issues/486) 17 | [![CRAN](https://www.r-pkg.org/badges/version/gittargets)](https://CRAN.R-project.org/package=gittargets) 18 | [![status](https://www.repostatus.org/badges/latest/active.svg)](https://www.repostatus.org/#active) 19 | [![check](https://github.com/ropensci/gittargets/workflows/check/badge.svg)](https://github.com/ropensci/gittargets/actions?query=workflow%3Acheck) 20 | [![codecov](https://codecov.io/gh/ropensci/gittargets/branch/main/graph/badge.svg?token=3T5DlLwUVl)](https://app.codecov.io/gh/ropensci/gittargets) 21 | [![lint](https://github.com/ropensci/gittargets/workflows/lint/badge.svg)](https://github.com/ropensci/gittargets/actions?query=workflow%3Alint) 22 | 23 | In computationally demanding data analysis pipelines, the [`targets`](https://docs.ropensci.org/targets/) R package maintains an up-to-date set of results while skipping tasks that do not need to rerun. This process increases speed and increases trust in the final end product. However, it also overwrites old output with new output, and past results disappear by default. To preserve historical output, the `gittargets` package captures version-controlled snapshots of the data store, and each snapshot links to the underlying commit of the source code. That way, when the user rolls back the code to a previous branch or commit, `gittargets` can recover the data contemporaneous with that commit so that all targets remain up to date. 24 | 25 | ## Prerequisites 26 | 27 | 1. Familiarity with the [R programming language](https://www.r-project.org/), covered in [R for Data Science](https://r4ds.had.co.nz/). 28 | 1. [Data science workflow management best practices](https://rstats.wtf/index.html). 29 | 1. [Git](https://git-scm.com), covered in [Happy Git and GitHub for the useR](https://happygitwithr.com). 30 | 1. [`targets`](https://docs.ropensci.org/targets/), which has resources on the [documentation website](https://docs.ropensci.org/targets/#how-to-get-started). 31 | 1. Familiarity with the [`targets` data store](https://books.ropensci.org/targets/data.html). 32 | 33 | ## Installation 34 | 35 | The package is available to install from any of the following sources. 36 | 37 | Type | Source | Command 38 | ---|---|--- 39 | Release | CRAN | `install.packages("gittargets")` 40 | Development | GitHub | `remotes::install_github("ropensci/gittargets")` 41 | Development | rOpenSci | `install.packages("gittargets", repos = "https://ropensci.r-universe.dev")` 42 | 43 | You will also need command line Git, available at .^[`gert` does not have these requirements, but `gittargets` does not exclusively rely on `gert` because `libgit2` does not automatically work with [Git LFS](https://git-lfs.com).] Please make sure Git is reachable from your system path environment variables. To control which Git executable `gittargets` uses, you may set the `TAR_GIT` environment variable with `usethis::edit_r_environ()` or `Sys.setenv()`. You will also need to configure your user name and user email at the global level using the instructions at (or `gert::git_config_global_set()`). Run `tar_git_ok()` to check installation and configuration. 44 | 45 | ```{r, eval = FALSE} 46 | tar_git_ok() 47 | #> ✓ Git binary: /path/to/git 48 | #> ✓ Git config global user name: your_user_name 49 | #> ✓ Git config global user email: your_email@example.com 50 | #> [1] TRUE 51 | ``` 52 | 53 | There are also backend-specific installation requirements and recommendations in the [package vignettes](https://docs.ropensci.org/gittargets/articles/index.html). 54 | 55 | ## Motivation 56 | 57 | Consider an example pipeline with source code in `_targets.R` and output in the [data store](https://books.ropensci.org/targets/data.html). 58 | 59 | ```{r, eval = FALSE} 60 | # _targets.R 61 | library(targets) 62 | list( 63 | tar_target(data, airquality), 64 | tar_target(model, lm(Ozone ~ Wind, data = data)) # Regress on wind speed. 65 | ) 66 | ``` 67 | 68 | Suppose you run the pipeline and confirm that all targets are up to date. 69 | 70 | ```{r, eval = FALSE} 71 | tar_make() 72 | #> • start target data 73 | #> • built target data 74 | #> • start target model 75 | #> • built target model 76 | #> • end pipeline 77 | ``` 78 | 79 | ```{r, eval = FALSE} 80 | tar_outdated() 81 | #> character(0) 82 | ``` 83 | 84 | It is good practice to track the source code in a [version control repository](https://git-scm.com) so you can revert to previous commits or branches. However, the [data store](https://books.ropensci.org/targets/data.html) is usually too large to keep in the same repository as the code, which typically lives in a cloud platform like [GitHub](https://github.com) where space and bandwidth are pricey. So when you check out an old commit or branch, you revert the code, but not the data. In other words, your targets are out of sync and out of date. 85 | 86 | ```{r, eval = FALSE} 87 | gert::git_branch_checkout(branch = "other-model") 88 | ``` 89 | 90 | ```{r, eval = FALSE} 91 | # _targets.R 92 | library(targets) 93 | list( 94 | tar_target(data, airquality), 95 | tar_target(model, lm(Ozone ~ Temp, data = data)) # Regress on temperature. 96 | ) 97 | ``` 98 | 99 | ```{r, eval = FALSE} 100 | tar_outdated() 101 | #> [1] "model" 102 | ``` 103 | 104 | ## Usage 105 | 106 | With `gittargets`, you can keep your targets up to date even as you check out code from different commits or branches. The specific steps depend on the data backend you choose, and each supported backend has a [package vignette](https://docs.ropensci.org/gittargets/articles/index.html) with a walkthrough. For example, the most important steps of the [Git data backend](https://docs.ropensci.org/gittargets/articles/git.html) are as follows. 107 | 108 | 1. Create the source code and run the pipeline at least once so the [data store](https://books.ropensci.org/targets/data.html) exists. 109 | 1. `tar_git_init()`: initialize a [Git](https://git-scm.com)/[Git LFS](https://git-lfs.com) repository for the [data store](https://books.ropensci.org/targets/data.html). 110 | 1. Bring the pipeline up to date (e.g. with [`tar_make()`](https://docs.ropensci.org/targets/reference/tar_make.html)) and commit any changes to the source code. 111 | 1. `tar_git_snapshot()`: create a data snapshot for the current code commit. 112 | 1. Develop the pipeline. Creating new code commits and code branches early and often, and create data snapshots at key strategic milestones. 113 | 1. `tar_git_checkout()`: revert the data to the appropriate prior snapshot. 114 | 115 | ## Performance 116 | 117 | `targets` generates a large amount of data in `_targets/objects/`, and data snapshots and checkouts may take a long time. To work around performance limitations, you may wish to only snapshot the data at the most important milestones of your project. Please refer to the [package vignettes](https://docs.ropensci.org/gittargets/articles/index.html) for specific recommendations on optimizing performance. 118 | 119 | ## Future directions 120 | 121 | The first data versioning system in `gittargets` uses [Git](https://git-scm.com), which is designed for source code and may not scale to enormous amounts of compressed data. Future releases of `gittargets` may explore alternative data backends more powerful than [Git LFS](https://git-lfs.com). 122 | 123 | ## Alternatives 124 | 125 | Newer versions of the `targets` package (>= 0.9.0) support continuous data versioning through cloud storage, e.g. [Amazon Web Services](https://aws.amazon.com) for [S3 buckets](https://aws.amazon.com/s3/) with [versioning enabled](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Versioning.html). In this approach, `targets` tracks the version ID of each [cloud-backed target](https://books.ropensci.org/targets/data.html#cloud-storage). That way, when the metadata file reverts to a prior version, the pipeline automatically uses prior versions of targets that were up to date at the time the metadata was written. This approach has two distinct advantages over `gittargets`: 126 | 127 | 1. Cloud storage reduces the burden of local storage for large data pipelines. 128 | 2. Target data is uploaded and tracked continuously, which means the user does not need to proactively take data snapshots. 129 | 130 | However, not all users have access to cloud services like [AWS](https://aws.amazon.com), not everyone is able or willing to pay the monetary costs of cloud storage for every single version of every single target, and uploads and downloads to and from the cloud may bottleneck some pipelines. `gittargets` fills this niche with a data versioning system that is 131 | 132 | 1. Entirely local, and 133 | 2. Entirely opt-in: users pick and choose when to register data snapshots, which consumes less storage than continuous snapshots or continuous cloud uploads to a versioned S3 bucket. 134 | 135 | ## Code of Conduct 136 | 137 | Please note that the `gittargets` project is released with a [Contributor Code of Conduct](https://ropensci.org/code-of-conduct/). By contributing to this project, you agree to abide by its terms. 138 | 139 | ## Citation 140 | 141 | ```{r, warning = FALSE} 142 | citation("gittargets") 143 | ``` 144 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | # gittargets 3 | 4 | [![ropensci](https://badges.ropensci.org/486_status.svg)](https://github.com/ropensci/software-review/issues/486) 5 | [![CRAN](https://www.r-pkg.org/badges/version/gittargets)](https://CRAN.R-project.org/package=gittargets) 6 | [![status](https://www.repostatus.org/badges/latest/active.svg)](https://www.repostatus.org/#active) 7 | [![check](https://github.com/ropensci/gittargets/workflows/check/badge.svg)](https://github.com/ropensci/gittargets/actions?query=workflow%3Acheck) 8 | [![codecov](https://codecov.io/gh/ropensci/gittargets/branch/main/graph/badge.svg?token=3T5DlLwUVl)](https://app.codecov.io/gh/ropensci/gittargets) 9 | [![lint](https://github.com/ropensci/gittargets/workflows/lint/badge.svg)](https://github.com/ropensci/gittargets/actions?query=workflow%3Alint) 10 | 11 | In computationally demanding data analysis pipelines, the 12 | [`targets`](https://docs.ropensci.org/targets/) R package maintains an 13 | up-to-date set of results while skipping tasks that do not need to 14 | rerun. This process increases speed and increases trust in the final end 15 | product. However, it also overwrites old output with new output, and 16 | past results disappear by default. To preserve historical output, the 17 | `gittargets` package captures version-controlled snapshots of the data 18 | store, and each snapshot links to the underlying commit of the source 19 | code. That way, when the user rolls back the code to a previous branch 20 | or commit, `gittargets` can recover the data contemporaneous with that 21 | commit so that all targets remain up to date. 22 | 23 | ## Prerequisites 24 | 25 | 1. Familiarity with the [R programming 26 | language](https://www.r-project.org/), covered in [R for Data 27 | Science](https://r4ds.had.co.nz/). 28 | 2. [Data science workflow management best 29 | practices](https://rstats.wtf/index.html). 30 | 3. [Git](https://git-scm.com), covered in [Happy Git and GitHub for the 31 | useR](https://happygitwithr.com). 32 | 4. [`targets`](https://docs.ropensci.org/targets/), which has resources 33 | on the [documentation 34 | website](https://docs.ropensci.org/targets/#how-to-get-started). 35 | 5. Familiarity with the [`targets` data 36 | store](https://books.ropensci.org/targets/data.html). 37 | 38 | ## Installation 39 | 40 | The package is available to install from any of the following sources. 41 | 42 | | Type | Source | Command | 43 | |-------------|----------|-----------------------------------------------------------------------------| 44 | | Release | CRAN | `install.packages("gittargets")` | 45 | | Development | GitHub | `remotes::install_github("ropensci/gittargets")` | 46 | | Development | rOpenSci | `install.packages("gittargets", repos = "https://ropensci.r-universe.dev")` | 47 | 48 | You will also need command line Git, available at 49 | .[^1] Please make sure Git is reachable 50 | from your system path environment variables. To control which Git 51 | executable `gittargets` uses, you may set the `TAR_GIT` environment 52 | variable with `usethis::edit_r_environ()` or `Sys.setenv()`. You will 53 | also need to configure your user name and user email at the global level 54 | using the instructions at 55 | 56 | (or `gert::git_config_global_set()`). Run `tar_git_ok()` to check 57 | installation and configuration. 58 | 59 | ``` r 60 | tar_git_ok() 61 | #> ✓ Git binary: /path/to/git 62 | #> ✓ Git config global user name: your_user_name 63 | #> ✓ Git config global user email: your_email@example.com 64 | #> [1] TRUE 65 | ``` 66 | 67 | There are also backend-specific installation requirements and 68 | recommendations in the [package 69 | vignettes](https://docs.ropensci.org/gittargets/articles/index.html). 70 | 71 | ## Motivation 72 | 73 | Consider an example pipeline with source code in `_targets.R` and output 74 | in the [data store](https://books.ropensci.org/targets/data.html). 75 | 76 | ``` r 77 | # _targets.R 78 | library(targets) 79 | list( 80 | tar_target(data, airquality), 81 | tar_target(model, lm(Ozone ~ Wind, data = data)) # Regress on wind speed. 82 | ) 83 | ``` 84 | 85 | Suppose you run the pipeline and confirm that all targets are up to 86 | date. 87 | 88 | ``` r 89 | tar_make() 90 | #> • start target data 91 | #> • built target data 92 | #> • start target model 93 | #> • built target model 94 | #> • end pipeline 95 | ``` 96 | 97 | ``` r 98 | tar_outdated() 99 | #> character(0) 100 | ``` 101 | 102 | It is good practice to track the source code in a [version control 103 | repository](https://git-scm.com) so you can revert to previous commits 104 | or branches. However, the [data 105 | store](https://books.ropensci.org/targets/data.html) is usually too 106 | large to keep in the same repository as the code, which typically lives 107 | in a cloud platform like [GitHub](https://github.com) where space and 108 | bandwidth are pricey. So when you check out an old commit or branch, you 109 | revert the code, but not the data. In other words, your targets are out 110 | of sync and out of date. 111 | 112 | ``` r 113 | gert::git_branch_checkout(branch = "other-model") 114 | ``` 115 | 116 | ``` r 117 | # _targets.R 118 | library(targets) 119 | list( 120 | tar_target(data, airquality), 121 | tar_target(model, lm(Ozone ~ Temp, data = data)) # Regress on temperature. 122 | ) 123 | ``` 124 | 125 | ``` r 126 | tar_outdated() 127 | #> [1] "model" 128 | ``` 129 | 130 | ## Usage 131 | 132 | With `gittargets`, you can keep your targets up to date even as you 133 | check out code from different commits or branches. The specific steps 134 | depend on the data backend you choose, and each supported backend has a 135 | [package 136 | vignette](https://docs.ropensci.org/gittargets/articles/index.html) with 137 | a walkthrough. For example, the most important steps of the [Git data 138 | backend](https://docs.ropensci.org/gittargets/articles/git.html) are as 139 | follows. 140 | 141 | 1. Create the source code and run the pipeline at least once so the 142 | [data store](https://books.ropensci.org/targets/data.html) exists. 143 | 2. `tar_git_init()`: initialize a [Git](https://git-scm.com)/[Git 144 | LFS](https://git-lfs.com) repository for the [data 145 | store](https://books.ropensci.org/targets/data.html). 146 | 3. Bring the pipeline up to date (e.g. with 147 | [`tar_make()`](https://docs.ropensci.org/targets/reference/tar_make.html)) 148 | and commit any changes to the source code. 149 | 4. `tar_git_snapshot()`: create a data snapshot for the current code 150 | commit. 151 | 5. Develop the pipeline. Creating new code commits and code branches 152 | early and often, and create data snapshots at key strategic 153 | milestones. 154 | 6. `tar_git_checkout()`: revert the data to the appropriate prior 155 | snapshot. 156 | 157 | ## Performance 158 | 159 | `targets` generates a large amount of data in `_targets/objects/`, and 160 | data snapshots and checkouts may take a long time. To work around 161 | performance limitations, you may wish to only snapshot the data at the 162 | most important milestones of your project. Please refer to the [package 163 | vignettes](https://docs.ropensci.org/gittargets/articles/index.html) for 164 | specific recommendations on optimizing performance. 165 | 166 | ## Future directions 167 | 168 | The first data versioning system in `gittargets` uses 169 | [Git](https://git-scm.com), which is designed for source code and may 170 | not scale to enormous amounts of compressed data. Future releases of 171 | `gittargets` may explore alternative data backends more powerful than 172 | [Git LFS](https://git-lfs.com). 173 | 174 | ## Alternatives 175 | 176 | Newer versions of the `targets` package (\>= 0.9.0) support continuous 177 | data versioning through cloud storage, e.g. [Amazon Web 178 | Services](https://aws.amazon.com) for [S3 179 | buckets](https://aws.amazon.com/s3/) with [versioning 180 | enabled](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Versioning.html). 181 | In this approach, `targets` tracks the version ID of each [cloud-backed 182 | target](https://books.ropensci.org/targets/data.html#cloud-storage). 183 | That way, when the metadata file reverts to a prior version, the 184 | pipeline automatically uses prior versions of targets that were up to 185 | date at the time the metadata was written. This approach has two 186 | distinct advantages over `gittargets`: 187 | 188 | 1. Cloud storage reduces the burden of local storage for large data 189 | pipelines. 190 | 2. Target data is uploaded and tracked continuously, which means the 191 | user does not need to proactively take data snapshots. 192 | 193 | However, not all users have access to cloud services like 194 | [AWS](https://aws.amazon.com), not everyone is able or willing to pay 195 | the monetary costs of cloud storage for every single version of every 196 | single target, and uploads and downloads to and from the cloud may 197 | bottleneck some pipelines. `gittargets` fills this niche with a data 198 | versioning system that is 199 | 200 | 1. Entirely local, and 201 | 2. Entirely opt-in: users pick and choose when to register data 202 | snapshots, which consumes less storage than continuous snapshots or 203 | continuous cloud uploads to a versioned S3 bucket. 204 | 205 | ## Code of Conduct 206 | 207 | Please note that the `gittargets` project is released with a 208 | [Contributor Code of Conduct](https://ropensci.org/code-of-conduct/). By 209 | contributing to this project, you agree to abide by its terms. 210 | 211 | ## Citation 212 | 213 | ``` r 214 | citation("gittargets") 215 | #> 216 | #> To cite gittargets in publications use: 217 | #> 218 | #> William Michael Landau (2021). gittargets: Version Control for the 219 | #> targets Package. https://docs.ropensci.org/gittargets/, 220 | #> https://github.com/ropensci/gittargets. 221 | #> 222 | #> A BibTeX entry for LaTeX users is 223 | #> 224 | #> @Manual{, 225 | #> title = {gittargets: Version Control for the Targets Package}, 226 | #> author = {William Michael Landau}, 227 | #> note = {https://docs.ropensci.org/gittargets/, https://github.com/ropensci/gittargets}, 228 | #> year = {2021}, 229 | #> } 230 | ``` 231 | 232 | [^1]: `gert` does not have these requirements, but `gittargets` does not 233 | exclusively rely on `gert` because `libgit2` does not automatically 234 | work with [Git LFS](https://git-lfs.com). 235 | -------------------------------------------------------------------------------- /_pkgdown.yml: -------------------------------------------------------------------------------- 1 | destination: docs/ 2 | navbar: 3 | components: 4 | targets: 5 | text: targets 6 | href: https://docs.ropensci.org/targets 7 | structure: 8 | left: [reference, targets, articles, news] 9 | right: [search, github, lightswitch] 10 | reference: 11 | - title: Help 12 | contents: 13 | - '`gittargets-package`' 14 | - title: Git data backend 15 | contents: 16 | - '`tar_git_checkout`' 17 | - '`tar_git_init`' 18 | - '`tar_git_log`' 19 | - '`tar_git_ok`' 20 | - '`tar_git_snapshot`' 21 | - '`tar_git_status`' 22 | - '`tar_git_status_code`' 23 | - '`tar_git_status_data`' 24 | - '`tar_git_status_targets`' 25 | -------------------------------------------------------------------------------- /codemeta.json: -------------------------------------------------------------------------------- 1 | { 2 | "@context": [ 3 | "https://doi.org/10.5063/schema/codemeta-2.0", 4 | "http://schema.org" 5 | ], 6 | "@type": "SoftwareSourceCode", 7 | "identifier": "gittargets", 8 | "description": "Pipelines with the 'targets' R package\n (2021, ) skip steps that\n are up to already date. Although this behavior reduces the runtime\n of subsequent runs, it comes at the cost of overwriting previous\n results. So if the pipeline source code is under version control,\n and if you revert to a previous commit or branch,\n the data will no longer be up to date with the code you\n just checked out. Ordinarily, you would need to rerun the\n pipeline in order to recover the targets you had before.\n However, 'gittargets' preserves historical output,\n creating version control snapshots of data store.\n Each data snapshot remembers the contemporaneous Git commit\n of the pipeline source code, so you can recover the right\n data when you navigate the Git history. In other words,\n 'gittargets' makes it possible to switch commits or branches\n without invalidating the pipeline. You can simply check out\n the up-to-date targets from the past instead of taking the\n time to recompute them from scratch.", 9 | "name": "gittargets: Data Version Control for the Targets Package", 10 | "codeRepository": "https://github.com/ropensci/gittargets", 11 | "relatedLink": [ 12 | "https://wlandau.github.io/gittargets/", 13 | "https://docs.ropensci.org/gittargets/" 14 | ], 15 | "issueTracker": "https://github.com/ropensci/gittargets/issues", 16 | "license": "https://spdx.org/licenses/MIT", 17 | "version": "0.0.0.9000", 18 | "programmingLanguage": { 19 | "@type": "ComputerLanguage", 20 | "name": "R", 21 | "url": "https://r-project.org" 22 | }, 23 | "runtimePlatform": "R version 4.1.0 (2021-05-18)", 24 | "author": [ 25 | { 26 | "@type": "Person", 27 | "givenName": ["William", "Michael"], 28 | "familyName": "Landau", 29 | "email": "will.landau@gmail.com", 30 | "@id": "https://orcid.org/0000-0003-1878-3253" 31 | } 32 | ], 33 | "contributor": {}, 34 | "copyrightHolder": [ 35 | { 36 | "@type": "Organization", 37 | "name": "Eli Lilly and Company" 38 | } 39 | ], 40 | "funder": {}, 41 | "maintainer": [ 42 | { 43 | "@type": "Person", 44 | "givenName": ["William", "Michael"], 45 | "familyName": "Landau", 46 | "email": "will.landau@gmail.com", 47 | "@id": "https://orcid.org/0000-0003-1878-3253" 48 | } 49 | ], 50 | "softwareSuggestions": [ 51 | { 52 | "@type": "SoftwareApplication", 53 | "identifier": "knitr", 54 | "name": "knitr", 55 | "version": ">= 1.30", 56 | "provider": { 57 | "@id": "https://cran.r-project.org", 58 | "@type": "Organization", 59 | "name": "Comprehensive R Archive Network (CRAN)", 60 | "url": "https://cran.r-project.org" 61 | }, 62 | "sameAs": "https://CRAN.R-project.org/package=knitr" 63 | }, 64 | { 65 | "@type": "SoftwareApplication", 66 | "identifier": "markdown", 67 | "name": "markdown", 68 | "version": ">= 1.1", 69 | "provider": { 70 | "@id": "https://cran.r-project.org", 71 | "@type": "Organization", 72 | "name": "Comprehensive R Archive Network (CRAN)", 73 | "url": "https://cran.r-project.org" 74 | }, 75 | "sameAs": "https://CRAN.R-project.org/package=markdown" 76 | }, 77 | { 78 | "@type": "SoftwareApplication", 79 | "identifier": "rmarkdown", 80 | "name": "rmarkdown", 81 | "version": ">= 2.4", 82 | "provider": { 83 | "@id": "https://cran.r-project.org", 84 | "@type": "Organization", 85 | "name": "Comprehensive R Archive Network (CRAN)", 86 | "url": "https://cran.r-project.org" 87 | }, 88 | "sameAs": "https://CRAN.R-project.org/package=rmarkdown" 89 | }, 90 | { 91 | "@type": "SoftwareApplication", 92 | "identifier": "testthat", 93 | "name": "testthat", 94 | "version": ">= 3.0.0", 95 | "provider": { 96 | "@id": "https://cran.r-project.org", 97 | "@type": "Organization", 98 | "name": "Comprehensive R Archive Network (CRAN)", 99 | "url": "https://cran.r-project.org" 100 | }, 101 | "sameAs": "https://CRAN.R-project.org/package=testthat" 102 | } 103 | ], 104 | "softwareRequirements": [ 105 | { 106 | "@type": "SoftwareApplication", 107 | "identifier": "R", 108 | "name": "R", 109 | "version": ">= 3.5.0" 110 | }, 111 | { 112 | "@type": "SoftwareApplication", 113 | "identifier": "cli", 114 | "name": "cli", 115 | "version": ">= 3.1.0", 116 | "provider": { 117 | "@id": "https://cran.r-project.org", 118 | "@type": "Organization", 119 | "name": "Comprehensive R Archive Network (CRAN)", 120 | "url": "https://cran.r-project.org" 121 | }, 122 | "sameAs": "https://CRAN.R-project.org/package=cli" 123 | }, 124 | { 125 | "@type": "SoftwareApplication", 126 | "identifier": "data.table", 127 | "name": "data.table", 128 | "version": ">= 1.12.8", 129 | "provider": { 130 | "@id": "https://cran.r-project.org", 131 | "@type": "Organization", 132 | "name": "Comprehensive R Archive Network (CRAN)", 133 | "url": "https://cran.r-project.org" 134 | }, 135 | "sameAs": "https://CRAN.R-project.org/package=data.table" 136 | }, 137 | { 138 | "@type": "SoftwareApplication", 139 | "identifier": "gert", 140 | "name": "gert", 141 | "version": ">= 1.0.0", 142 | "provider": { 143 | "@id": "https://cran.r-project.org", 144 | "@type": "Organization", 145 | "name": "Comprehensive R Archive Network (CRAN)", 146 | "url": "https://cran.r-project.org" 147 | }, 148 | "sameAs": "https://CRAN.R-project.org/package=gert" 149 | }, 150 | { 151 | "@type": "SoftwareApplication", 152 | "identifier": "processx", 153 | "name": "processx", 154 | "version": ">= 3.0.0", 155 | "provider": { 156 | "@id": "https://cran.r-project.org", 157 | "@type": "Organization", 158 | "name": "Comprehensive R Archive Network (CRAN)", 159 | "url": "https://cran.r-project.org" 160 | }, 161 | "sameAs": "https://CRAN.R-project.org/package=processx" 162 | }, 163 | { 164 | "@type": "SoftwareApplication", 165 | "identifier": "stats", 166 | "name": "stats" 167 | }, 168 | { 169 | "@type": "SoftwareApplication", 170 | "identifier": "targets", 171 | "name": "targets", 172 | "version": ">= 0.6.0", 173 | "provider": { 174 | "@id": "https://cran.r-project.org", 175 | "@type": "Organization", 176 | "name": "Comprehensive R Archive Network (CRAN)", 177 | "url": "https://cran.r-project.org" 178 | }, 179 | "sameAs": "https://CRAN.R-project.org/package=targets" 180 | }, 181 | { 182 | "@type": "SoftwareApplication", 183 | "identifier": "tibble", 184 | "name": "tibble", 185 | "version": ">= 3.0.0", 186 | "provider": { 187 | "@id": "https://cran.r-project.org", 188 | "@type": "Organization", 189 | "name": "Comprehensive R Archive Network (CRAN)", 190 | "url": "https://cran.r-project.org" 191 | }, 192 | "sameAs": "https://CRAN.R-project.org/package=tibble" 193 | }, 194 | { 195 | "@type": "SoftwareApplication", 196 | "identifier": "utils", 197 | "name": "utils" 198 | }, 199 | { 200 | "@type": "SoftwareApplication", 201 | "identifier": "uuid", 202 | "name": "uuid", 203 | "version": ">= 0.1.4", 204 | "provider": { 205 | "@id": "https://cran.r-project.org", 206 | "@type": "Organization", 207 | "name": "Comprehensive R Archive Network (CRAN)", 208 | "url": "https://cran.r-project.org" 209 | }, 210 | "sameAs": "https://CRAN.R-project.org/package=uuid" 211 | }, 212 | { 213 | "@type": "SoftwareApplication", 214 | "identifier": "https://sysreqs.r-hub.io/get/git" 215 | } 216 | ], 217 | "releaseNotes": "https://github.com/ropensci/gittargets/blob/master/NEWS.md", 218 | "readme": "https://github.com/ropensci/gittargets/blob/main/README.md", 219 | "fileSize": "368.889KB", 220 | "contIntegration": ["https://github.com/ropensci/gittargets/actions?query=workflow%3Acheck", "https://app.codecov.io/gh/ropensci/gittargets", "https://github.com/ropensci/gittargets/actions?query=workflow%3Alint"], 221 | "developmentStatus": "https://www.repostatus.org/#active", 222 | "keywords": [ 223 | "data-version-control", 224 | "data-versioning", 225 | "workflow", 226 | "data-science", 227 | "r", 228 | "reproducible-research", 229 | "rstats", 230 | "r-package", 231 | "reproducibility", 232 | "targets" 233 | ], 234 | "review": { 235 | "@type": "Review", 236 | "url": "https://github.com/ropensci/software-review/issues/486", 237 | "provider": "https://ropensci.org" 238 | }, 239 | "citation": [ 240 | { 241 | "@type": "SoftwareSourceCode", 242 | "datePublished": "2021", 243 | "author": [ 244 | { 245 | "@type": "Person", 246 | "givenName": ["William", "Michael"], 247 | "familyName": "Landau" 248 | } 249 | ], 250 | "name": "gittargets: Version Control for the Targets Package", 251 | "description": "https://docs.ropensci.org/gittargets/, https://github.com/ropensci/gittargets" 252 | } 253 | ] 254 | } 255 | -------------------------------------------------------------------------------- /gittargets.Rproj: -------------------------------------------------------------------------------- 1 | Version: 1.0 2 | 3 | RestoreWorkspace: Default 4 | SaveWorkspace: Default 5 | AlwaysSaveHistory: Default 6 | 7 | EnableCodeIndexing: Yes 8 | UseSpacesForTab: Yes 9 | NumSpacesForTab: 2 10 | Encoding: UTF-8 11 | 12 | RnwWeave: Sweave 13 | LaTeX: pdfLaTeX 14 | 15 | BuildType: Package 16 | PackageUseDevtools: Yes 17 | PackageInstallArgs: --no-multiarch --with-keep.source 18 | -------------------------------------------------------------------------------- /inst/CITATION: -------------------------------------------------------------------------------- 1 | citHeader("To cite gittargets in publications use:") 2 | 3 | bibentry( 4 | bibtype = "Manual", 5 | title = "gittargets: Version Control for the Targets Package", 6 | author = "William Michael Landau", 7 | note = "https://docs.ropensci.org/gittargets/, https://github.com/ropensci/gittargets", 8 | year = "2021", 9 | textVersion = paste( 10 | "William Michael Landau (2021).", 11 | "gittargets: Version Control for the", 12 | "targets Package. https://docs.ropensci.org/gittargets/,", 13 | "https://github.com/ropensci/gittargets." 14 | ) 15 | ) 16 | -------------------------------------------------------------------------------- /inst/WORDLIST: -------------------------------------------------------------------------------- 1 | Bitbucket 2 | codecov 3 | doi 4 | DVC 5 | gitignore 6 | GitLab 7 | hpc 8 | http 9 | joss 10 | LFS 11 | README 12 | reproducibly 13 | rOpenSci 14 | ropensci 15 | scm 16 | SHA 17 | SHA1 18 | snapshotted 19 | useR 20 | walkthrough 21 | writeable 22 | YAML 23 | reproducibility 24 | -------------------------------------------------------------------------------- /man/figures/figures.pptx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ropensci/gittargets/c678c41c0f2e4357e817eb61c3a35f1bdbc305f3/man/figures/figures.pptx -------------------------------------------------------------------------------- /man/figures/image-credit.md: -------------------------------------------------------------------------------- 1 | The image of the guitar is in the public domain (no copyright, unlimited commercial use): https://openclipart.org/detail/10117/guitar-1 2 | -------------------------------------------------------------------------------- /man/figures/logo-hexbin.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ropensci/gittargets/c678c41c0f2e4357e817eb61c3a35f1bdbc305f3/man/figures/logo-hexbin.png -------------------------------------------------------------------------------- /man/figures/logo-readme.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ropensci/gittargets/c678c41c0f2e4357e817eb61c3a35f1bdbc305f3/man/figures/logo-readme.png -------------------------------------------------------------------------------- /man/figures/logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ropensci/gittargets/c678c41c0f2e4357e817eb61c3a35f1bdbc305f3/man/figures/logo.png -------------------------------------------------------------------------------- /man/gittargets-package.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/tar_package.R 3 | \docType{package} 4 | \name{gittargets-package} 5 | \alias{gittargets-package} 6 | \title{targets: Dynamic Function-Oriented Make-Like Declarative Pipelines for R} 7 | \description{ 8 | In computationally demanding data analysis pipelines, 9 | the \code{targets} R package maintains an up-to-date set of results 10 | while skipping tasks that do not need to rerun. This process 11 | increases speed and increases trust in the final end product. 12 | However, it also overwrites old output with new output, 13 | and past results disappear by default. To preserve historical output, 14 | the \code{gittargets} package captures version-controlled snapshots 15 | of the data store, and each snapshot links to the underlying 16 | commit of the source code. That way, when the user rolls back 17 | the code to a previous branch or commit, \code{gittargets} can recover 18 | the data contemporaneous with that commit so that all targets 19 | remain up to date. 20 | } 21 | \concept{help} 22 | -------------------------------------------------------------------------------- /man/tar_git_checkout.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/tar_git_checkout.R 3 | \name{tar_git_checkout} 4 | \alias{tar_git_checkout} 5 | \title{Check out a snapshot of the data (Git)} 6 | \usage{ 7 | tar_git_checkout( 8 | ref = "HEAD", 9 | code = getwd(), 10 | store = targets::tar_config_get("store"), 11 | force = FALSE, 12 | verbose = TRUE 13 | ) 14 | } 15 | \arguments{ 16 | \item{ref}{Character of length 1. SHA1 hash, branch name, 17 | or other reference in the code repository 18 | that points to a code commit. (You can also identify the code 19 | commit by supplying a data branch of the form \verb{code=}.) 20 | Defaults to \code{"HEAD"}, which points to the currently 21 | checked out code commit. 22 | 23 | Once the desired code commit is identified, 24 | \code{tar_git_snapshot()} checks out the latest corresponding data snapshot. 25 | There may be earlier data snapshots corresponding to this code commit, 26 | but \code{tar_git_snapshot()} only checks out the latest one. 27 | To check out an earlier superseded data snapshot, 28 | you will need to manually use command line Git in the data repository. 29 | 30 | If \code{tar_git_snapshot()} cannot find a data snapshot for the 31 | desired code commit, then it will throw an error. 32 | For a list of commits in the current code branch 33 | that have available data snapshots, see the \code{commit_code} 34 | column of the output of \code{\link[=tar_git_log]{tar_git_log()}}.} 35 | 36 | \item{code}{Character of length 1, directory path to the code repository, 37 | usually the root of the \code{targets} project.} 38 | 39 | \item{store}{Character of length 1, path to the data store of the pipeline. 40 | If \code{NULL}, the \code{store} setting is left unchanged in the 41 | YAML configuration file (default: \verb{_targets.yaml}). 42 | Usually, the data store lives at \verb{_targets}. 43 | Set \code{store} to a custom directory 44 | to specify a path other than \verb{_targets/}. The path need not exist 45 | before the pipeline begins, and it need not end with "_targets", 46 | but it must be writeable. 47 | For optimal performance, choose a storage location 48 | with fast read/write access. 49 | If the argument \code{NULL}, the setting is not modified. 50 | Use \code{\link[targets:tar_config_unset]{tar_config_unset()}} to delete a setting.} 51 | 52 | \item{force}{ignore conflicts and overwrite modified files} 53 | 54 | \item{verbose}{Logical of length 1, whether to print R console messages 55 | confirming that a snapshot was created.} 56 | } 57 | \value{ 58 | Nothing (invisibly). 59 | } 60 | \description{ 61 | Check out a snapshot of the data associated with 62 | a particular code commit (default: \code{HEAD}). 63 | } 64 | \examples{ 65 | if (Sys.getenv("TAR_EXAMPLES") == "true" && tar_git_ok(verbose = FALSE)) { 66 | targets::tar_dir({ # Containing code does not modify the user's filespace. 67 | # Work on an initial branch. 68 | targets::tar_script(tar_target(data, "old_data")) 69 | targets::tar_make() 70 | targets::tar_read(data) # "old_data" 71 | gert::git_init() 72 | gert::git_add("_targets.R") 73 | gert::git_commit("First commit") 74 | gert::git_branch_create("old_branch") 75 | tar_git_init() 76 | # Work on a new branch. 77 | tar_git_snapshot(status = FALSE, verbose = FALSE) 78 | targets::tar_script(tar_target(data, "new_data")) 79 | targets::tar_make() 80 | targets::tar_read(data) # "new_data" 81 | gert::git_branch_create("new_branch") 82 | gert::git_add("_targets.R") 83 | gert::git_commit("Second commit") 84 | tar_git_snapshot(status = FALSE, verbose = FALSE) 85 | # Go back to the old branch. 86 | gert::git_branch_checkout("old_branch") 87 | # The target is out of date because we only reverted the code. 88 | targets::tar_outdated() 89 | # But tar_git_checkout() lets us restore the old version of the data! 90 | tar_git_checkout() 91 | targets::tar_read(data) # "old_data" 92 | # Now, the target is up to date! And we did not even have to rerun it! 93 | targets::tar_outdated() 94 | }) 95 | } 96 | } 97 | \seealso{ 98 | Other git: 99 | \code{\link{tar_git_init}()}, 100 | \code{\link{tar_git_log}()}, 101 | \code{\link{tar_git_ok}()}, 102 | \code{\link{tar_git_snapshot}()}, 103 | \code{\link{tar_git_status_code}()}, 104 | \code{\link{tar_git_status_data}()}, 105 | \code{\link{tar_git_status_targets}()}, 106 | \code{\link{tar_git_status}()} 107 | } 108 | \concept{git} 109 | -------------------------------------------------------------------------------- /man/tar_git_init.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/tar_git_init.R 3 | \name{tar_git_init} 4 | \alias{tar_git_init} 5 | \title{Initialize a data repository (Git).} 6 | \usage{ 7 | tar_git_init( 8 | store = targets::tar_config_get("store"), 9 | stash_gitignore = TRUE, 10 | git_lfs = TRUE, 11 | verbose = TRUE 12 | ) 13 | } 14 | \arguments{ 15 | \item{store}{Character of length 1, path to the data store of the pipeline. 16 | If \code{NULL}, the \code{store} setting is left unchanged in the 17 | YAML configuration file (default: \verb{_targets.yaml}). 18 | Usually, the data store lives at \verb{_targets}. 19 | Set \code{store} to a custom directory 20 | to specify a path other than \verb{_targets/}. The path need not exist 21 | before the pipeline begins, and it need not end with "_targets", 22 | but it must be writeable. 23 | For optimal performance, choose a storage location 24 | with fast read/write access. 25 | If the argument \code{NULL}, the setting is not modified. 26 | Use \code{\link[targets:tar_config_unset]{tar_config_unset()}} to delete a setting.} 27 | 28 | \item{stash_gitignore}{Logical of length 1, whether to temporarily 29 | stash the \code{.gitignore} file of the data store. See the 30 | "Stashing .gitignore" section for details.} 31 | 32 | \item{git_lfs}{Logical, whether to automatically opt into Git LFS to track 33 | large files in \verb{_targets/objects} more efficiently. If \code{TRUE} 34 | and Git LFS is installed, it should work automatically. If \code{FALSE}, 35 | you can always opt in later by running \verb{git lfs track objects} 36 | inside the data store.} 37 | 38 | \item{verbose}{Logical of length 1, whether to print messages to the 39 | R console.} 40 | } 41 | \value{ 42 | \code{NULL} (invisibly). 43 | } 44 | \description{ 45 | Initialize a Git repository for a \code{targets} data store. 46 | } 47 | \details{ 48 | \code{tar_git_init()} also writes a \code{.gitattributes} file to the 49 | store to automatically track target output date with \code{git-lfs} 50 | if it is installed on your system. 51 | } 52 | \section{Stashing .gitignore}{ 53 | 54 | The \code{targets} package writes a \code{.gitignore} file to new data stores 55 | in order to prevent accidental commits to the code Git repository. 56 | Unfortunately, for \code{gittargets}, this automatic \code{.gitignore} file 57 | interferes with proper data versioning. So by default, \code{gittargets} 58 | temporarily stashes it to a hidden file called \code{.gittargets_gitignore} 59 | inside the data store. If your R program crashes while the stash 60 | is active, you can simply move it manually back to \code{.gitignore} 61 | or run \code{tar_git_status_data()} to restore the stash automatically 62 | if no \code{.gitignore} already exists. 63 | } 64 | 65 | \examples{ 66 | if (Sys.getenv("TAR_EXAMPLES") == "true" && tar_git_ok(verbose = FALSE)) { 67 | targets::tar_dir({ # Containing code does not modify the user's file space. 68 | targets::tar_script(tar_target(data, 1)) 69 | targets::tar_make() 70 | tar_git_init() 71 | }) 72 | } 73 | } 74 | \seealso{ 75 | Other git: 76 | \code{\link{tar_git_checkout}()}, 77 | \code{\link{tar_git_log}()}, 78 | \code{\link{tar_git_ok}()}, 79 | \code{\link{tar_git_snapshot}()}, 80 | \code{\link{tar_git_status_code}()}, 81 | \code{\link{tar_git_status_data}()}, 82 | \code{\link{tar_git_status_targets}()}, 83 | \code{\link{tar_git_status}()} 84 | } 85 | \concept{git} 86 | -------------------------------------------------------------------------------- /man/tar_git_log.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/tar_git_log.R 3 | \name{tar_git_log} 4 | \alias{tar_git_log} 5 | \title{Data snapshots of a code branch (Git)} 6 | \usage{ 7 | tar_git_log( 8 | code = getwd(), 9 | store = targets::tar_config_get("store"), 10 | branch = gert::git_branch(repo = code), 11 | max = 100 12 | ) 13 | } 14 | \arguments{ 15 | \item{code}{Character of length 1, directory path to the code repository, 16 | usually the root of the \code{targets} project.} 17 | 18 | \item{store}{Character of length 1, path to the data store of the pipeline. 19 | If \code{NULL}, the \code{store} setting is left unchanged in the 20 | YAML configuration file (default: \verb{_targets.yaml}). 21 | Usually, the data store lives at \verb{_targets}. 22 | Set \code{store} to a custom directory 23 | to specify a path other than \verb{_targets/}. The path need not exist 24 | before the pipeline begins, and it need not end with "_targets", 25 | but it must be writeable. 26 | For optimal performance, choose a storage location 27 | with fast read/write access. 28 | If the argument \code{NULL}, the setting is not modified. 29 | Use \code{\link[targets:tar_config_unset]{tar_config_unset()}} to delete a setting.} 30 | 31 | \item{branch}{Character of length 1, name of the code repository branch 32 | to query. Defaults to the currently checked-out code branch.} 33 | 34 | \item{max}{Positive numeric of length 1, maximum number of code commits 35 | to inspect for the given branch.} 36 | } 37 | \value{ 38 | A data frame of information about 39 | data snapshots and code commits. 40 | } 41 | \description{ 42 | Show all the data snapshots of a code branch. 43 | } 44 | \details{ 45 | By design, \code{tar_git_log()} only queries a single 46 | code branch at a time. This allows \code{tar_git_log()} 47 | to report more detailed information about the snapshots 48 | of the given code branch. 49 | To query all data snapshots over all branches, simply run 50 | \code{gert::git_branch_list(local = TRUE, repo = "_targets")}. 51 | The valid snapshots show \code{"code="} in the \code{name} column, 52 | where \verb{} is the Git commit hash of the code commit 53 | corresponding to the data snapshot. 54 | } 55 | \examples{ 56 | if (Sys.getenv("TAR_EXAMPLES") == "true" && tar_git_ok(verbose = FALSE)) { 57 | targets::tar_dir({ # Containing code does not modify the user's filespace. 58 | targets::tar_script(tar_target(data, 1)) 59 | targets::tar_make() 60 | gert::git_init() 61 | gert::git_add("_targets.R") 62 | gert::git_commit("First commit") 63 | tar_git_init() 64 | tar_git_snapshot(status = FALSE, verbose = FALSE) 65 | tar_git_log() 66 | }) 67 | } 68 | } 69 | \seealso{ 70 | Other git: 71 | \code{\link{tar_git_checkout}()}, 72 | \code{\link{tar_git_init}()}, 73 | \code{\link{tar_git_ok}()}, 74 | \code{\link{tar_git_snapshot}()}, 75 | \code{\link{tar_git_status_code}()}, 76 | \code{\link{tar_git_status_data}()}, 77 | \code{\link{tar_git_status_targets}()}, 78 | \code{\link{tar_git_status}()} 79 | } 80 | \concept{git} 81 | -------------------------------------------------------------------------------- /man/tar_git_ok.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/tar_git_ok.R 3 | \name{tar_git_ok} 4 | \alias{tar_git_ok} 5 | \title{Check Git} 6 | \usage{ 7 | tar_git_ok(verbose = TRUE) 8 | } 9 | \arguments{ 10 | \item{verbose}{Whether to print messages to the console.} 11 | } 12 | \value{ 13 | Logical of length 1, whether Git is installed and configured 14 | correctly. 15 | } 16 | \description{ 17 | Check if Git is installed and if \code{user.name} and \code{user.email} 18 | are configured globally. 19 | } 20 | \details{ 21 | You can install Git from \url{https://git-scm.com/downloads/} 22 | and configure your identity using the instructions at 23 | \url{https://git-scm.com/book/en/v2/Getting-Started-First-Time-Git-Setup}. 24 | You may find it convenient to run \code{gert::git_config_global()} 25 | with \code{name} equal to \code{user.name} and \code{user.email}. 26 | } 27 | \examples{ 28 | tar_git_ok() 29 | } 30 | \seealso{ 31 | Other git: 32 | \code{\link{tar_git_checkout}()}, 33 | \code{\link{tar_git_init}()}, 34 | \code{\link{tar_git_log}()}, 35 | \code{\link{tar_git_snapshot}()}, 36 | \code{\link{tar_git_status_code}()}, 37 | \code{\link{tar_git_status_data}()}, 38 | \code{\link{tar_git_status_targets}()}, 39 | \code{\link{tar_git_status}()} 40 | } 41 | \concept{git} 42 | -------------------------------------------------------------------------------- /man/tar_git_snapshot.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/tar_git_snapshot.R 3 | \name{tar_git_snapshot} 4 | \alias{tar_git_snapshot} 5 | \title{Snapshot the data repository (Git).} 6 | \usage{ 7 | tar_git_snapshot( 8 | message = NULL, 9 | ref = "HEAD", 10 | code = getwd(), 11 | script = targets::tar_config_get("script"), 12 | store = targets::tar_config_get("store"), 13 | stash_gitignore = TRUE, 14 | reporter = targets::tar_config_get("reporter_outdated"), 15 | envir = parent.frame(), 16 | callr_function = callr::r, 17 | callr_arguments = NULL, 18 | status = interactive(), 19 | force = FALSE, 20 | pack_refs = TRUE, 21 | verbose = TRUE 22 | ) 23 | } 24 | \arguments{ 25 | \item{message}{Optional Git commit message of the data snapshot. 26 | If \code{NULL}, then the message is the Git commit message of the 27 | matching code commit.} 28 | 29 | \item{ref}{Character of length 1, reference 30 | (branch name, Git SHA1 hash, etc.) of the code commit 31 | that will map to the new data snapshot. Defaults to the commit 32 | checked out right now.} 33 | 34 | \item{code}{Character of length 1, directory path to the code repository, 35 | usually the root of the \code{targets} project.} 36 | 37 | \item{script}{Character of length 1, path to the 38 | target script file. Defaults to \code{tar_config_get("script")}, 39 | which in turn defaults to \verb{_targets.R}. When you set 40 | this argument, the value of \code{tar_config_get("script")} 41 | is temporarily changed for the current function call. 42 | See \code{\link[targets:tar_script]{tar_script()}}, 43 | \code{\link[targets:tar_config_get]{tar_config_get()}}, and \code{\link[targets:tar_config_set]{tar_config_set()}} for details 44 | about the target script file and how to set it 45 | persistently for a project.} 46 | 47 | \item{store}{Character of length 1, path to the data store of the pipeline. 48 | If \code{NULL}, the \code{store} setting is left unchanged in the 49 | YAML configuration file (default: \verb{_targets.yaml}). 50 | Usually, the data store lives at \verb{_targets}. 51 | Set \code{store} to a custom directory 52 | to specify a path other than \verb{_targets/}. The path need not exist 53 | before the pipeline begins, and it need not end with "_targets", 54 | but it must be writeable. 55 | For optimal performance, choose a storage location 56 | with fast read/write access. 57 | If the argument \code{NULL}, the setting is not modified. 58 | Use \code{\link[targets:tar_config_unset]{tar_config_unset()}} to delete a setting.} 59 | 60 | \item{stash_gitignore}{Logical of length 1, whether to temporarily 61 | stash the \code{.gitignore} file of the data store. See the 62 | "Stashing .gitignore" section for details.} 63 | 64 | \item{reporter}{Character of length 1, name of the reporter to user. 65 | Controls how messages are printed as targets are checked. Choices: 66 | \itemize{ 67 | \item \code{"silent"}: print nothing. 68 | \item \code{"forecast"}: print running totals of the checked and outdated 69 | targets found so far. 70 | }} 71 | 72 | \item{envir}{An environment, where to run the target R script 73 | (default: \verb{_targets.R}) if \code{callr_function} is \code{NULL}. 74 | Ignored if \code{callr_function} is anything other than \code{NULL}. 75 | \code{callr_function} should only be \code{NULL} for debugging and 76 | testing purposes, not for serious runs of a pipeline, etc. 77 | 78 | The \code{envir} argument of \code{\link[targets:tar_make]{tar_make()}} and related 79 | functions always overrides 80 | the current value of \code{tar_option_get("envir")} in the current R session 81 | just before running the target script file, 82 | so whenever you need to set an alternative \code{envir}, you should always set 83 | it with \code{tar_option_set()} from within the target script file. 84 | In other words, if you call \code{tar_option_set(envir = envir1)} in an 85 | interactive session and then 86 | \code{tar_make(envir = envir2, callr_function = NULL)}, 87 | then \code{envir2} will be used.} 88 | 89 | \item{callr_function}{A function from \code{callr} to start a fresh clean R 90 | process to do the work. Set to \code{NULL} to run in the current session 91 | instead of an external process (but restart your R session just before 92 | you do in order to clear debris out of the global environment). 93 | \code{callr_function} needs to be \code{NULL} for interactive debugging, 94 | e.g. \code{tar_option_set(debug = "your_target")}. 95 | However, \code{callr_function} should not be \code{NULL} for serious 96 | reproducible work.} 97 | 98 | \item{callr_arguments}{A list of arguments to \code{callr_function}.} 99 | 100 | \item{status}{Logical of length 1, whether to print the project status 101 | with \code{\link[=tar_git_status]{tar_git_status()}} and ask whether a snapshot should be created.} 102 | 103 | \item{force}{Logical of length 1. Force checkout the data branch 104 | of an existing data snapshot of the current code commit?} 105 | 106 | \item{pack_refs}{Logical of length 1, whether to run \verb{git pack-refs --all} 107 | in the data store after taking the snapshot. Packing references 108 | improves efficiency when the number of snapshots is large. 109 | Learn more at \url{https://git-scm.com/docs/git-pack-refs}.} 110 | 111 | \item{verbose}{Logical of length 1, whether to print R console messages 112 | confirming that a snapshot was created.} 113 | } 114 | \description{ 115 | Snapshot the Git data repository of a \code{targets} project. 116 | } 117 | \details{ 118 | A Git-backed \code{gittargets} data snapshot is a special kind of 119 | Git commit. Every data commit is part of a branch specific to 120 | the current code commit. 121 | That way, when you switch branches or commits in the code, 122 | \code{tar_git_checkout()} checks out the latest data snapshot 123 | that matches the code in your workspace. 124 | That way, your targets can stay up to date even as you 125 | transition among multiple branches. 126 | } 127 | \section{Stashing .gitignore}{ 128 | 129 | The \code{targets} package writes a \code{.gitignore} file to new data stores 130 | in order to prevent accidental commits to the code Git repository. 131 | Unfortunately, for \code{gittargets}, this automatic \code{.gitignore} file 132 | interferes with proper data versioning. So by default, \code{gittargets} 133 | temporarily stashes it to a hidden file called \code{.gittargets_gitignore} 134 | inside the data store. If your R program crashes while the stash 135 | is active, you can simply move it manually back to \code{.gitignore} 136 | or run \code{tar_git_status_data()} to restore the stash automatically 137 | if no \code{.gitignore} already exists. 138 | } 139 | 140 | \examples{ 141 | if (Sys.getenv("TAR_EXAMPLES") == "true" && tar_git_ok(verbose = FALSE)) { 142 | targets::tar_dir({ # Containing code does not modify the user's filespace. 143 | targets::tar_script(tar_target(data, 1)) 144 | targets::tar_make() 145 | gert::git_init() 146 | gert::git_add("_targets.R") 147 | gert::git_commit("First commit") 148 | tar_git_init() 149 | tar_git_snapshot(status = FALSE) 150 | }) 151 | } 152 | } 153 | \seealso{ 154 | Other git: 155 | \code{\link{tar_git_checkout}()}, 156 | \code{\link{tar_git_init}()}, 157 | \code{\link{tar_git_log}()}, 158 | \code{\link{tar_git_ok}()}, 159 | \code{\link{tar_git_status_code}()}, 160 | \code{\link{tar_git_status_data}()}, 161 | \code{\link{tar_git_status_targets}()}, 162 | \code{\link{tar_git_status}()} 163 | } 164 | \concept{git} 165 | -------------------------------------------------------------------------------- /man/tar_git_snapshot_menu.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/tar_git_snapshot.R 3 | \name{tar_git_snapshot_menu} 4 | \alias{tar_git_snapshot_menu} 5 | \title{Data snapshot menu (Git)} 6 | \usage{ 7 | tar_git_snapshot_menu( 8 | commit, 9 | message, 10 | code, 11 | script, 12 | store, 13 | stash_gitignore, 14 | reporter, 15 | envir, 16 | callr_function, 17 | callr_arguments 18 | ) 19 | } 20 | \arguments{ 21 | \item{commit}{Character of length 1, Git SHA1 hash of the code commit 22 | that will correspond to the data snapshot (if created).} 23 | 24 | \item{message}{Optional Git commit message of the data snapshot. 25 | If \code{NULL}, then the message is the Git commit message of the 26 | matching code commit.} 27 | 28 | \item{code}{Character of length 1, directory path to the code repository, 29 | usually the root of the \code{targets} project.} 30 | 31 | \item{script}{Character of length 1, path to the 32 | target script file. Defaults to \code{tar_config_get("script")}, 33 | which in turn defaults to \verb{_targets.R}. When you set 34 | this argument, the value of \code{tar_config_get("script")} 35 | is temporarily changed for the current function call. 36 | See \code{\link[targets:tar_script]{tar_script()}}, 37 | \code{\link[targets:tar_config_get]{tar_config_get()}}, and \code{\link[targets:tar_config_set]{tar_config_set()}} for details 38 | about the target script file and how to set it 39 | persistently for a project.} 40 | 41 | \item{store}{Character of length 1, path to the data store of the pipeline. 42 | If \code{NULL}, the \code{store} setting is left unchanged in the 43 | YAML configuration file (default: \verb{_targets.yaml}). 44 | Usually, the data store lives at \verb{_targets}. 45 | Set \code{store} to a custom directory 46 | to specify a path other than \verb{_targets/}. The path need not exist 47 | before the pipeline begins, and it need not end with "_targets", 48 | but it must be writeable. 49 | For optimal performance, choose a storage location 50 | with fast read/write access. 51 | If the argument \code{NULL}, the setting is not modified. 52 | Use \code{\link[targets:tar_config_unset]{tar_config_unset()}} to delete a setting.} 53 | 54 | \item{stash_gitignore}{Logical of length 1, whether to temporarily 55 | stash the \code{.gitignore} file of the data store. See the 56 | "Stashing .gitignore" section for details.} 57 | 58 | \item{reporter}{Character of length 1, name of the reporter to user. 59 | Controls how messages are printed as targets are checked. Choices: 60 | \itemize{ 61 | \item \code{"silent"}: print nothing. 62 | \item \code{"forecast"}: print running totals of the checked and outdated 63 | targets found so far. 64 | }} 65 | 66 | \item{envir}{An environment, where to run the target R script 67 | (default: \verb{_targets.R}) if \code{callr_function} is \code{NULL}. 68 | Ignored if \code{callr_function} is anything other than \code{NULL}. 69 | \code{callr_function} should only be \code{NULL} for debugging and 70 | testing purposes, not for serious runs of a pipeline, etc. 71 | 72 | The \code{envir} argument of \code{\link[targets:tar_make]{tar_make()}} and related 73 | functions always overrides 74 | the current value of \code{tar_option_get("envir")} in the current R session 75 | just before running the target script file, 76 | so whenever you need to set an alternative \code{envir}, you should always set 77 | it with \code{tar_option_set()} from within the target script file. 78 | In other words, if you call \code{tar_option_set(envir = envir1)} in an 79 | interactive session and then 80 | \code{tar_make(envir = envir2, callr_function = NULL)}, 81 | then \code{envir2} will be used.} 82 | 83 | \item{callr_function}{A function from \code{callr} to start a fresh clean R 84 | process to do the work. Set to \code{NULL} to run in the current session 85 | instead of an external process (but restart your R session just before 86 | you do in order to clear debris out of the global environment). 87 | \code{callr_function} needs to be \code{NULL} for interactive debugging, 88 | e.g. \code{tar_option_set(debug = "your_target")}. 89 | However, \code{callr_function} should not be \code{NULL} for serious 90 | reproducible work.} 91 | 92 | \item{callr_arguments}{A list of arguments to \code{callr_function}.} 93 | } 94 | \value{ 95 | Integer of length 1: \code{2L} if the user agrees to snapshot, 96 | \code{1L} if the user declines. 97 | } 98 | \description{ 99 | Check the project status and show an interactive menu 100 | for \code{\link[=tar_git_snapshot]{tar_git_snapshot()}}. 101 | } 102 | \examples{ 103 | # See the examples of tar_git_snapshot(). 104 | } 105 | \keyword{internal} 106 | -------------------------------------------------------------------------------- /man/tar_git_status.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/tar_git_status.R 3 | \name{tar_git_status} 4 | \alias{tar_git_status} 5 | \title{Status of the project (Git)} 6 | \usage{ 7 | tar_git_status( 8 | code = getwd(), 9 | script = targets::tar_config_get("script"), 10 | store = targets::tar_config_get("store"), 11 | stash_gitignore = TRUE, 12 | reporter = targets::tar_config_get("reporter_outdated"), 13 | envir = parent.frame(), 14 | callr_function = callr::r, 15 | callr_arguments = NULL 16 | ) 17 | } 18 | \arguments{ 19 | \item{code}{Character of length 1, directory path to the code repository, 20 | usually the root of the \code{targets} project.} 21 | 22 | \item{script}{Character of length 1, path to the 23 | target script file. Defaults to \code{tar_config_get("script")}, 24 | which in turn defaults to \verb{_targets.R}. When you set 25 | this argument, the value of \code{tar_config_get("script")} 26 | is temporarily changed for the current function call. 27 | See \code{\link[targets:tar_script]{tar_script()}}, 28 | \code{\link[targets:tar_config_get]{tar_config_get()}}, and \code{\link[targets:tar_config_set]{tar_config_set()}} for details 29 | about the target script file and how to set it 30 | persistently for a project.} 31 | 32 | \item{store}{Character of length 1, path to the data store of the pipeline. 33 | If \code{NULL}, the \code{store} setting is left unchanged in the 34 | YAML configuration file (default: \verb{_targets.yaml}). 35 | Usually, the data store lives at \verb{_targets}. 36 | Set \code{store} to a custom directory 37 | to specify a path other than \verb{_targets/}. The path need not exist 38 | before the pipeline begins, and it need not end with "_targets", 39 | but it must be writeable. 40 | For optimal performance, choose a storage location 41 | with fast read/write access. 42 | If the argument \code{NULL}, the setting is not modified. 43 | Use \code{\link[targets:tar_config_unset]{tar_config_unset()}} to delete a setting.} 44 | 45 | \item{stash_gitignore}{Logical of length 1, whether to temporarily 46 | stash the \code{.gitignore} file of the data store. See the 47 | "Stashing .gitignore" section for details.} 48 | 49 | \item{reporter}{Character of length 1, name of the reporter to user. 50 | Controls how messages are printed as targets are checked. Choices: 51 | \itemize{ 52 | \item \code{"silent"}: print nothing. 53 | \item \code{"forecast"}: print running totals of the checked and outdated 54 | targets found so far. 55 | }} 56 | 57 | \item{envir}{An environment, where to run the target R script 58 | (default: \verb{_targets.R}) if \code{callr_function} is \code{NULL}. 59 | Ignored if \code{callr_function} is anything other than \code{NULL}. 60 | \code{callr_function} should only be \code{NULL} for debugging and 61 | testing purposes, not for serious runs of a pipeline, etc. 62 | 63 | The \code{envir} argument of \code{\link[targets:tar_make]{tar_make()}} and related 64 | functions always overrides 65 | the current value of \code{tar_option_get("envir")} in the current R session 66 | just before running the target script file, 67 | so whenever you need to set an alternative \code{envir}, you should always set 68 | it with \code{tar_option_set()} from within the target script file. 69 | In other words, if you call \code{tar_option_set(envir = envir1)} in an 70 | interactive session and then 71 | \code{tar_make(envir = envir2, callr_function = NULL)}, 72 | then \code{envir2} will be used.} 73 | 74 | \item{callr_function}{A function from \code{callr} to start a fresh clean R 75 | process to do the work. Set to \code{NULL} to run in the current session 76 | instead of an external process (but restart your R session just before 77 | you do in order to clear debris out of the global environment). 78 | \code{callr_function} needs to be \code{NULL} for interactive debugging, 79 | e.g. \code{tar_option_set(debug = "your_target")}. 80 | However, \code{callr_function} should not be \code{NULL} for serious 81 | reproducible work.} 82 | 83 | \item{callr_arguments}{A list of arguments to \code{callr_function}.} 84 | } 85 | \value{ 86 | \code{NULL} (invisibly). Status information is printed 87 | to the R console. 88 | } 89 | \description{ 90 | Print the status of the code repository, 91 | the data repository, and the targets. 92 | } 93 | \section{Stashing .gitignore}{ 94 | 95 | The \code{targets} package writes a \code{.gitignore} file to new data stores 96 | in order to prevent accidental commits to the code Git repository. 97 | Unfortunately, for \code{gittargets}, this automatic \code{.gitignore} file 98 | interferes with proper data versioning. So by default, \code{gittargets} 99 | temporarily stashes it to a hidden file called \code{.gittargets_gitignore} 100 | inside the data store. If your R program crashes while the stash 101 | is active, you can simply move it manually back to \code{.gitignore} 102 | or run \code{tar_git_status_data()} to restore the stash automatically 103 | if no \code{.gitignore} already exists. 104 | } 105 | 106 | \examples{ 107 | if (Sys.getenv("TAR_EXAMPLES") == "true" && tar_git_ok(verbose = FALSE)) { 108 | targets::tar_dir({ # Containing code does not modify the user's files pace. 109 | targets::tar_script(tar_target(data, 1)) 110 | targets::tar_make() 111 | list.files("_targets", all.files = TRUE) 112 | gert::git_init() 113 | tar_git_init() 114 | tar_git_status() 115 | }) 116 | } 117 | } 118 | \seealso{ 119 | Other git: 120 | \code{\link{tar_git_checkout}()}, 121 | \code{\link{tar_git_init}()}, 122 | \code{\link{tar_git_log}()}, 123 | \code{\link{tar_git_ok}()}, 124 | \code{\link{tar_git_snapshot}()}, 125 | \code{\link{tar_git_status_code}()}, 126 | \code{\link{tar_git_status_data}()}, 127 | \code{\link{tar_git_status_targets}()} 128 | } 129 | \concept{git} 130 | -------------------------------------------------------------------------------- /man/tar_git_status_code.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/tar_git_status_code.R 3 | \name{tar_git_status_code} 4 | \alias{tar_git_status_code} 5 | \title{Status of the code repository (Git)} 6 | \usage{ 7 | tar_git_status_code(code = getwd()) 8 | } 9 | \arguments{ 10 | \item{code}{Character of length 1, directory path to the code repository, 11 | usually the root of the \code{targets} project.} 12 | } 13 | \value{ 14 | If the code repository exists, the return value is the data frame 15 | produced by \code{gert::git_status(repo = code)}. If the code has no Git 16 | repository, then the return value is \code{NULL}. 17 | } 18 | \description{ 19 | Show the Git status of the code repository. 20 | } 21 | \examples{ 22 | if (Sys.getenv("TAR_EXAMPLES") == "true" && tar_git_ok(verbose = FALSE)) { 23 | targets::tar_dir({ # Containing code does not modify the user's file space. 24 | targets::tar_script(tar_target(data, 1)) 25 | targets::tar_make() 26 | list.files("_targets", all.files = TRUE) 27 | gert::git_init() 28 | tar_git_init() 29 | tar_git_status_code() 30 | }) 31 | } 32 | } 33 | \seealso{ 34 | Other git: 35 | \code{\link{tar_git_checkout}()}, 36 | \code{\link{tar_git_init}()}, 37 | \code{\link{tar_git_log}()}, 38 | \code{\link{tar_git_ok}()}, 39 | \code{\link{tar_git_snapshot}()}, 40 | \code{\link{tar_git_status_data}()}, 41 | \code{\link{tar_git_status_targets}()}, 42 | \code{\link{tar_git_status}()} 43 | } 44 | \concept{git} 45 | -------------------------------------------------------------------------------- /man/tar_git_status_data.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/tar_git_status_data.R 3 | \name{tar_git_status_data} 4 | \alias{tar_git_status_data} 5 | \title{Status of the data repository (Git)} 6 | \usage{ 7 | tar_git_status_data( 8 | store = targets::tar_config_get("store"), 9 | stash_gitignore = TRUE 10 | ) 11 | } 12 | \arguments{ 13 | \item{store}{Character of length 1, path to the data store of the pipeline. 14 | If \code{NULL}, the \code{store} setting is left unchanged in the 15 | YAML configuration file (default: \verb{_targets.yaml}). 16 | Usually, the data store lives at \verb{_targets}. 17 | Set \code{store} to a custom directory 18 | to specify a path other than \verb{_targets/}. The path need not exist 19 | before the pipeline begins, and it need not end with "_targets", 20 | but it must be writeable. 21 | For optimal performance, choose a storage location 22 | with fast read/write access. 23 | If the argument \code{NULL}, the setting is not modified. 24 | Use \code{\link[targets:tar_config_unset]{tar_config_unset()}} to delete a setting.} 25 | 26 | \item{stash_gitignore}{Logical of length 1, whether to temporarily 27 | stash the \code{.gitignore} file of the data store. See the 28 | "Stashing .gitignore" section for details.} 29 | } 30 | \value{ 31 | If the data repository exists, the return value is the data frame 32 | produced by \code{gert::git_status(repo = store)}. If the data store has no Git 33 | repository, then the return value is \code{NULL}. 34 | } 35 | \description{ 36 | Show the Git status of the data repository. 37 | } 38 | \section{Stashing .gitignore}{ 39 | 40 | The \code{targets} package writes a \code{.gitignore} file to new data stores 41 | in order to prevent accidental commits to the code Git repository. 42 | Unfortunately, for \code{gittargets}, this automatic \code{.gitignore} file 43 | interferes with proper data versioning. So by default, \code{gittargets} 44 | temporarily stashes it to a hidden file called \code{.gittargets_gitignore} 45 | inside the data store. If your R program crashes while the stash 46 | is active, you can simply move it manually back to \code{.gitignore} 47 | or run \code{tar_git_status_data()} to restore the stash automatically 48 | if no \code{.gitignore} already exists. 49 | } 50 | 51 | \examples{ 52 | if (Sys.getenv("TAR_EXAMPLES") == "true" && tar_git_ok(verbose = FALSE)) { 53 | targets::tar_dir({ # Containing code does not modify the user's file space. 54 | targets::tar_script(tar_target(data, 1)) 55 | targets::tar_make() 56 | list.files("_targets", all.files = TRUE) 57 | gert::git_init() 58 | tar_git_init() 59 | tar_git_status_data() 60 | }) 61 | } 62 | } 63 | \seealso{ 64 | Other git: 65 | \code{\link{tar_git_checkout}()}, 66 | \code{\link{tar_git_init}()}, 67 | \code{\link{tar_git_log}()}, 68 | \code{\link{tar_git_ok}()}, 69 | \code{\link{tar_git_snapshot}()}, 70 | \code{\link{tar_git_status_code}()}, 71 | \code{\link{tar_git_status_targets}()}, 72 | \code{\link{tar_git_status}()} 73 | } 74 | \concept{git} 75 | -------------------------------------------------------------------------------- /man/tar_git_status_targets.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/tar_git_status_targets.R 3 | \name{tar_git_status_targets} 4 | \alias{tar_git_status_targets} 5 | \title{Status of the targets (Git)} 6 | \usage{ 7 | tar_git_status_targets( 8 | script = targets::tar_config_get("script"), 9 | store = targets::tar_config_get("store"), 10 | reporter = targets::tar_config_get("reporter_outdated"), 11 | envir = parent.frame(), 12 | callr_function = callr::r, 13 | callr_arguments = NULL 14 | ) 15 | } 16 | \arguments{ 17 | \item{script}{Character of length 1, path to the 18 | target script file. Defaults to \code{tar_config_get("script")}, 19 | which in turn defaults to \verb{_targets.R}. When you set 20 | this argument, the value of \code{tar_config_get("script")} 21 | is temporarily changed for the current function call. 22 | See \code{\link[targets:tar_script]{tar_script()}}, 23 | \code{\link[targets:tar_config_get]{tar_config_get()}}, and \code{\link[targets:tar_config_set]{tar_config_set()}} for details 24 | about the target script file and how to set it 25 | persistently for a project.} 26 | 27 | \item{store}{Character of length 1, path to the 28 | \code{targets} data store. Defaults to \code{tar_config_get("store")}, 29 | which in turn defaults to \verb{_targets/}. 30 | When you set this argument, the value of \code{tar_config_get("store")} 31 | is temporarily changed for the current function call. 32 | See \code{\link[targets:tar_config_get]{tar_config_get()}} and \code{\link[targets:tar_config_set]{tar_config_set()}} for details 33 | about how to set the data store path persistently 34 | for a project.} 35 | 36 | \item{reporter}{Character of length 1, name of the reporter to user. 37 | Controls how messages are printed as targets are checked. Choices: 38 | \itemize{ 39 | \item \code{"silent"}: print nothing. 40 | \item \code{"forecast"}: print running totals of the checked and outdated 41 | targets found so far. 42 | }} 43 | 44 | \item{envir}{An environment, where to run the target R script 45 | (default: \verb{_targets.R}) if \code{callr_function} is \code{NULL}. 46 | Ignored if \code{callr_function} is anything other than \code{NULL}. 47 | \code{callr_function} should only be \code{NULL} for debugging and 48 | testing purposes, not for serious runs of a pipeline, etc. 49 | 50 | The \code{envir} argument of \code{\link[targets:tar_make]{tar_make()}} and related 51 | functions always overrides 52 | the current value of \code{tar_option_get("envir")} in the current R session 53 | just before running the target script file, 54 | so whenever you need to set an alternative \code{envir}, you should always set 55 | it with \code{tar_option_set()} from within the target script file. 56 | In other words, if you call \code{tar_option_set(envir = envir1)} in an 57 | interactive session and then 58 | \code{tar_make(envir = envir2, callr_function = NULL)}, 59 | then \code{envir2} will be used.} 60 | 61 | \item{callr_function}{A function from \code{callr} to start a fresh clean R 62 | process to do the work. Set to \code{NULL} to run in the current session 63 | instead of an external process (but restart your R session just before 64 | you do in order to clear debris out of the global environment). 65 | \code{callr_function} needs to be \code{NULL} for interactive debugging, 66 | e.g. \code{tar_option_set(debug = "your_target")}. 67 | However, \code{callr_function} should not be \code{NULL} for serious 68 | reproducible work.} 69 | 70 | \item{callr_arguments}{A list of arguments to \code{callr_function}.} 71 | } 72 | \value{ 73 | A \code{tibble} with the names of outdated targets. 74 | } 75 | \description{ 76 | Show which targets are outdated. 77 | } 78 | \details{ 79 | This function has prettier output than \code{targets::tar_outdated()}, 80 | and it mainly serves \code{\link[=tar_git_status]{tar_git_status()}}. 81 | } 82 | \examples{ 83 | targets::tar_dir({ # Containing code does not modify the user's file space. 84 | targets::tar_script(tar_target(data, 1)) 85 | targets::tar_make() 86 | list.files("_targets", all.files = TRUE) 87 | tar_git_status_targets() 88 | }) 89 | } 90 | \seealso{ 91 | Other git: 92 | \code{\link{tar_git_checkout}()}, 93 | \code{\link{tar_git_init}()}, 94 | \code{\link{tar_git_log}()}, 95 | \code{\link{tar_git_ok}()}, 96 | \code{\link{tar_git_snapshot}()}, 97 | \code{\link{tar_git_status_code}()}, 98 | \code{\link{tar_git_status_data}()}, 99 | \code{\link{tar_git_status}()} 100 | } 101 | \concept{git} 102 | -------------------------------------------------------------------------------- /pkgdown/favicon/apple-touch-icon-120x120.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ropensci/gittargets/c678c41c0f2e4357e817eb61c3a35f1bdbc305f3/pkgdown/favicon/apple-touch-icon-120x120.png -------------------------------------------------------------------------------- /pkgdown/favicon/apple-touch-icon-152x152.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ropensci/gittargets/c678c41c0f2e4357e817eb61c3a35f1bdbc305f3/pkgdown/favicon/apple-touch-icon-152x152.png -------------------------------------------------------------------------------- /pkgdown/favicon/apple-touch-icon-180x180.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ropensci/gittargets/c678c41c0f2e4357e817eb61c3a35f1bdbc305f3/pkgdown/favicon/apple-touch-icon-180x180.png -------------------------------------------------------------------------------- /pkgdown/favicon/apple-touch-icon-60x60.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ropensci/gittargets/c678c41c0f2e4357e817eb61c3a35f1bdbc305f3/pkgdown/favicon/apple-touch-icon-60x60.png -------------------------------------------------------------------------------- /pkgdown/favicon/apple-touch-icon-76x76.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ropensci/gittargets/c678c41c0f2e4357e817eb61c3a35f1bdbc305f3/pkgdown/favicon/apple-touch-icon-76x76.png -------------------------------------------------------------------------------- /pkgdown/favicon/apple-touch-icon.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ropensci/gittargets/c678c41c0f2e4357e817eb61c3a35f1bdbc305f3/pkgdown/favicon/apple-touch-icon.png -------------------------------------------------------------------------------- /pkgdown/favicon/favicon-16x16.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ropensci/gittargets/c678c41c0f2e4357e817eb61c3a35f1bdbc305f3/pkgdown/favicon/favicon-16x16.png -------------------------------------------------------------------------------- /pkgdown/favicon/favicon-32x32.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ropensci/gittargets/c678c41c0f2e4357e817eb61c3a35f1bdbc305f3/pkgdown/favicon/favicon-32x32.png -------------------------------------------------------------------------------- /pkgdown/favicon/favicon.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ropensci/gittargets/c678c41c0f2e4357e817eb61c3a35f1bdbc305f3/pkgdown/favicon/favicon.ico -------------------------------------------------------------------------------- /tests/interactive/test-tar_git_snapshot.R: -------------------------------------------------------------------------------- 1 | targets::tar_test("tar_git_snapshot()", { 2 | git_setup_init() 3 | expect_gt(nrow(tar_git_status_data()), 0L) 4 | store <- targets::tar_config_get("store") 5 | gitignore <- file.path(store, ".gitignore") 6 | writeLines("*", gitignore) 7 | capture.output(tar_git_snapshot(status = TRUE)) # choose 2 8 | expect_equal(readLines(gitignore), "*") 9 | expect_gt(nrow(tar_git_status_data()), 0L) 10 | capture.output(tar_git_snapshot(status = TRUE)) # choose 1 11 | expect_equal(readLines(gitignore), "*") 12 | expect_equal(nrow(tar_git_status_data()), 0L) 13 | commit <- paste0("code=", gert::git_commit_info(repo = getwd())$id) 14 | branch <- gert::git_branch(repo = store) 15 | expect_equal(commit, branch) 16 | message_code <- gert::git_commit_info(repo = getwd())$message 17 | message_data <- gert::git_commit_info(repo = store)$message 18 | expect_equal(message_code, message_data) 19 | }) 20 | -------------------------------------------------------------------------------- /tests/testthat.R: -------------------------------------------------------------------------------- 1 | library(testthat) 2 | library(gittargets) 3 | 4 | test_check("gittargets") 5 | -------------------------------------------------------------------------------- /tests/testthat/helper-git.R: -------------------------------------------------------------------------------- 1 | git_setup_init <- function() { 2 | stopifnot(identical(Sys.getenv("TAR_TEST"), "true")) 3 | targets::tar_script(targets::tar_target(x, 1)) 4 | targets::tar_make(callr_function = NULL) 5 | gert::git_init() 6 | gert::git_add("_targets.R") 7 | gert::git_commit("First commit") 8 | tar_git_init() 9 | invisible() 10 | } 11 | 12 | skip_os_git <- function() { 13 | skip_on_cran() 14 | skip_on_os(os = "solaris") 15 | skip_if(!tar_git_ok(verbose = FALSE)) 16 | } 17 | -------------------------------------------------------------------------------- /tests/testthat/helper-utils.R: -------------------------------------------------------------------------------- 1 | targets_1.3.2.9004 <- function() { 2 | utils::compareVersion( 3 | a = as.character(packageVersion("targets")), 4 | b = "1.3.2.9004" 5 | ) > -1L 6 | } 7 | 8 | status_completed <- function() { 9 | if (targets_1.3.2.9004()) { 10 | "completed" 11 | } else { 12 | "built" 13 | } 14 | } 15 | 16 | status_dispatched <- function() { 17 | if (targets_1.3.2.9004()) { 18 | "dispatched" 19 | } else { 20 | "started" 21 | } 22 | } 23 | -------------------------------------------------------------------------------- /tests/testthat/test-tar_git_checkout.R: -------------------------------------------------------------------------------- 1 | targets::tar_test("tar_git_checkout()", { 2 | skip_os_git() 3 | # Work on an initial branch. 4 | targets::tar_script(tar_target(data, "old_data")) 5 | targets::tar_make(callr_function = NULL) 6 | expect_equal(targets::tar_progress(data)$progress, status_completed()) 7 | expect_equal(targets::tar_read(data), "old_data") 8 | gert::git_init() 9 | gert::git_add("_targets.R") 10 | gert::git_commit("First commit") 11 | gert::git_branch_create("old_branch") 12 | tar_git_init() 13 | # Work on a new branch. 14 | tar_git_snapshot(status = FALSE, verbose = FALSE) 15 | targets::tar_script(tar_target(data, "new_data")) 16 | targets::tar_make(callr_function = NULL) 17 | expect_equal(targets::tar_progress(data)$progress, status_completed()) 18 | expect_equal(targets::tar_read(data), "new_data") 19 | gert::git_branch_create("new_branch") 20 | gert::git_add("_targets.R") 21 | gert::git_commit("Second commit") 22 | tar_git_snapshot(status = FALSE, verbose = FALSE) 23 | # Go back to the old branch. 24 | gert::git_branch_checkout("old_branch") 25 | # The target is out of date because we only reverted the code. 26 | targets::tar_outdated(callr_function = NULL) 27 | # But tar_git_checkout() lets us restore the old version of the data! 28 | tar_git_checkout() 29 | # Now, the target is up to date! And we did not even have to rerun it! 30 | expect_equal(targets::tar_outdated(callr_function = NULL), character(0)) 31 | # Modified files get recovered on checkout. 32 | old <- readLines(file.path("_targets", "meta", "meta")) 33 | expect_equal(targets::tar_read(data), "old_data") 34 | writeLines("line", file.path("_targets", "meta", "meta")) 35 | expect_equal(readLines(file.path("_targets", "meta", "meta")), "line") 36 | tar_git_checkout() 37 | expect_equal(readLines(file.path("_targets", "meta", "meta")), old) 38 | expect_equal(targets::tar_outdated(callr_function = NULL), character(0)) 39 | # Deleted files get recovered on checkout. 40 | unlink(file.path("_targets", "meta", "meta")) 41 | expect_false(file.exists(file.path("_targets", "meta", "meta"))) 42 | tar_git_checkout() 43 | expect_equal(readLines(file.path("_targets", "meta", "meta")), old) 44 | expect_equal(targets::tar_outdated(callr_function = NULL), character(0)) 45 | }) 46 | -------------------------------------------------------------------------------- /tests/testthat/test-tar_git_init.R: -------------------------------------------------------------------------------- 1 | targets::tar_test("tar_git_init() without .gitignore", { 2 | skip_os_git() 3 | store <- targets::tar_config_get("store") 4 | dir.create(store) 5 | expect_false(file.exists(file.path(store, ".git"))) 6 | expect_false(file.exists(file.path(store, ".gitignore"))) 7 | dir <- getwd() 8 | tar_git_init(store = store) 9 | expect_equal(getwd(), dir) 10 | expect_true(file.exists(file.path(store, ".git"))) 11 | expect_false(file.exists(file.path(store, ".gitignore"))) 12 | }) 13 | 14 | targets::tar_test("tar_git_init() .gitattributes under vcs", { 15 | skip_os_git() 16 | store <- targets::tar_config_get("store") 17 | dir.create(store) 18 | expect_false(file.exists(file.path(store, ".gitattributes"))) 19 | dir <- getwd() 20 | tar_git_init(store = store) 21 | expect_true(file.exists(file.path(store, ".gitattributes"))) 22 | unlink(file.path(store, ".gitattributes")) 23 | status <- tar_git_status_data(store = store) 24 | expect_equal(status$file, ".gitattributes") 25 | expect_equal(status$status, "deleted") 26 | expect_equal(status$staged, FALSE) 27 | }) 28 | 29 | targets::tar_test("tar_git_init() with git_lfs = FALSE", { 30 | skip_os_git() 31 | store <- targets::tar_config_get("store") 32 | dir.create(store) 33 | expect_false(file.exists(file.path(store, ".gitattributes"))) 34 | dir <- getwd() 35 | tar_git_init(store = store, git_lfs = FALSE) 36 | expect_false(file.exists(file.path(store, ".gitattributes"))) 37 | }) 38 | 39 | targets::tar_test("tar_git_init() with .gitignore", { 40 | skip_os_git() 41 | store <- targets::tar_config_get("store") 42 | targets::tar_script(targets::tar_target(x, 1)) 43 | targets::tar_make(callr_function = NULL) 44 | writeLines("*", file.path(store, ".gitignore")) 45 | expect_false(file.exists(file.path(store, ".git"))) 46 | expect_true(file.exists(file.path(store, ".gitignore"))) 47 | dir <- getwd() 48 | tar_git_init(store = store) 49 | expect_equal(getwd(), dir) 50 | expect_true(file.exists(file.path(store, ".git"))) 51 | expect_true(file.exists(file.path(store, ".gitignore"))) 52 | }) 53 | 54 | targets::tar_test("tar_git_init() is idempotent", { 55 | skip_os_git() 56 | store <- targets::tar_config_get("store") 57 | dir.create(store) 58 | expect_false(file.exists(file.path(store, ".git"))) 59 | expect_false(file.exists(file.path(store, ".gitignore"))) 60 | dir <- getwd() 61 | tar_git_init(store = store) 62 | tar_git_init(store = store) 63 | expect_equal(getwd(), dir) 64 | expect_true(file.exists(file.path(store, ".git"))) 65 | expect_false(file.exists(file.path(store, ".gitignore"))) 66 | }) 67 | -------------------------------------------------------------------------------- /tests/testthat/test-tar_git_log.R: -------------------------------------------------------------------------------- 1 | targets::tar_test("tar_git_log()", { 2 | skip_os_git() 3 | git_setup_init() 4 | tar_git_snapshot(status = FALSE, verbose = FALSE, message = "First snapshot") 5 | out <- tar_git_log() 6 | cols <- c( 7 | "message_code", 8 | "message_data", 9 | "time_code", 10 | "time_data", 11 | "commit_code", 12 | "commit_data" 13 | ) 14 | expect_equal(sort(colnames(out)), sort(cols)) 15 | expect_equal(nrow(out), 1L) 16 | expect_false(any(is.na(out))) 17 | expect_equal(out$message_code, "First commit") 18 | expect_equal(out$message_data, "First snapshot") 19 | for (field in c("time_code", "time_data")) { 20 | expect_s3_class(out[[field]], "POSIXct") 21 | } 22 | for (field in c("commit_code", "commit_data")) { 23 | expect_true(is.character(out[[field]])) 24 | expect_true(nzchar(out[[field]])) 25 | } 26 | }) 27 | -------------------------------------------------------------------------------- /tests/testthat/test-tar_git_ok.R: -------------------------------------------------------------------------------- 1 | targets::tar_test("tar_git_ok()", { 2 | skip_os_git() 3 | expect_true(tar_git_ok()) 4 | expect_message(tar_git_ok(verbose = TRUE)) 5 | expect_silent(tar_git_ok(verbose = FALSE)) 6 | }) 7 | 8 | targets::tar_test("tar_git_ok()", { 9 | skip_os_git() 10 | old <- Sys.getenv("TAR_GIT", unset = Sys.which("git")) 11 | on.exit(Sys.setenv(TAR_GIT = old)) 12 | tmp <- tempfile() 13 | file.create(tmp) 14 | Sys.setenv(TAR_GIT = tmp) 15 | expect_false(tar_git_ok()) 16 | expect_message(tar_git_ok(verbose = TRUE)) 17 | expect_silent(tar_git_ok(verbose = FALSE)) 18 | }) 19 | -------------------------------------------------------------------------------- /tests/testthat/test-tar_git_snapshot.R: -------------------------------------------------------------------------------- 1 | targets::tar_test("tar_git_snapshot()", { 2 | skip_os_git() 3 | git_setup_init() 4 | expect_gt(nrow(tar_git_status_data()), 0L) 5 | store <- targets::tar_config_get("store") 6 | gitignore <- file.path(store, ".gitignore") 7 | writeLines("*", gitignore) 8 | tar_git_snapshot(status = FALSE) 9 | expect_equal(readLines(gitignore), "*") 10 | expect_equal(nrow(tar_git_status_data()), 0L) 11 | commit_code <- gert::git_commit_info(repo = getwd())$id 12 | commit_store <- gert::git_commit_info(repo = store)$id 13 | branch <- gert::git_branch(repo = store) 14 | expect_equal(tar_git_branch_snapshot(commit_code), branch) 15 | message_code <- gert::git_commit_info(repo = getwd())$message 16 | message_data <- gert::git_commit_info(repo = store)$message 17 | expect_equal(message_code, message_data) 18 | log <- gert::git_log(repo = store) 19 | expect_equal(nrow(log), 2L) 20 | expect_true(commit_store %in% log$commit) 21 | # second data snapshot for the same code commit 22 | commit_store_old <- commit_store 23 | tar_git_snapshot(status = FALSE) 24 | commit_store <- gert::git_commit_info(repo = store)$id 25 | commit_code <- gert::git_commit_info(repo = getwd())$id 26 | expect_false(identical(commit_store, commit_store_old)) 27 | branch <- gert::git_branch(repo = store) 28 | expect_equal(tar_git_branch_snapshot(commit_code), branch) 29 | message_code <- gert::git_commit_info(repo = getwd())$message 30 | message_data <- gert::git_commit_info(repo = store)$message 31 | expect_equal(message_code, message_data) 32 | log <- gert::git_log(repo = store) 33 | expect_true(commit_store %in% log$commit) 34 | log <- gert::git_log(repo = store) 35 | expect_equal(nrow(log), 3L) 36 | expect_true(commit_store %in% log$commit) 37 | expect_true(commit_store_old %in% log$commit) 38 | }) 39 | 40 | targets::tar_test("tar_git_snapshot() custom message", { 41 | skip_os_git() 42 | git_setup_init() 43 | expect_gt(nrow(tar_git_status_data()), 0L) 44 | store <- targets::tar_config_get("store") 45 | gitignore <- file.path(store, ".gitignore") 46 | writeLines("*", gitignore) 47 | tar_git_snapshot(message = "custom message", status = FALSE) 48 | message_code <- gert::git_commit_info(repo = getwd())$message 49 | message_data <- gert::git_commit_info(repo = store)$message 50 | expect_false(identical(message_code, message_data)) 51 | expect_equal(trimws(message_data), "custom message") 52 | }) 53 | -------------------------------------------------------------------------------- /tests/testthat/test-tar_git_status.R: -------------------------------------------------------------------------------- 1 | targets::tar_test("tar_git_status() git set up and targets up to date", { 2 | skip_os_git() 3 | git_setup_init() 4 | expect_message(tar_git_status(callr_function = NULL)) 5 | }) 6 | 7 | targets::tar_test("tar_git_status() git not set up and targets outdated", { 8 | skip_os_git() 9 | targets::tar_script(targets::tar_target(x, 1)) 10 | targets::tar_make(callr_function = NULL) 11 | targets::tar_invalidate(everything()) 12 | expect_message(tar_git_status(callr_function = NULL)) 13 | }) 14 | -------------------------------------------------------------------------------- /tests/testthat/test-tar_git_status_code.R: -------------------------------------------------------------------------------- 1 | targets::tar_test("tar_git_status_code() with no repo", { 2 | skip_os_git() 3 | expect_null(tar_git_status_code()) 4 | }) 5 | 6 | targets::tar_test("tar_git_status_code() with clean worktree", { 7 | skip_os_git() 8 | git_setup_init() 9 | store <- targets::tar_config_get("store") 10 | writeLines("*", file.path(store, ".gitignore")) 11 | out <- tar_git_status_code() 12 | expect_equal(nrow(out), 0L) 13 | }) 14 | 15 | targets::tar_test("tar_git_status_code() with unclean worktree", { 16 | skip_os_git() 17 | git_setup_init() 18 | unlink("_targets.R") 19 | out <- tar_git_status_code() 20 | expect_true(nrow(out) > 0L) 21 | }) 22 | -------------------------------------------------------------------------------- /tests/testthat/test-tar_git_status_data.R: -------------------------------------------------------------------------------- 1 | targets::tar_test("tar_git_status_data() with no repo", { 2 | skip_os_git() 3 | targets::tar_script(targets::tar_target(x, 1)) 4 | targets::tar_make(callr_function = NULL) 5 | expect_null(tar_git_status_data()) 6 | }) 7 | 8 | targets::tar_test("tar_git_status_data() with clean worktree", { 9 | skip_os_git() 10 | git_setup_init() 11 | out <- tar_git_status_data() 12 | store <- tar_config_get("store") 13 | unlink(file.path(store, out$file), recursive = TRUE) 14 | out <- tar_git_status_data() 15 | expect_equal(nrow(out), 0L) 16 | }) 17 | 18 | targets::tar_test("tar_git_status_data() with unclean worktree", { 19 | skip_os_git() 20 | git_setup_init() 21 | out <- tar_git_status_data() 22 | expect_true(nrow(out) > 0L) 23 | }) 24 | 25 | targets::tar_test("tar_git_status_data() without stashing .gitignore", { 26 | skip_os_git() 27 | git_setup_init() 28 | store <- tar_config_get("store") 29 | writeLines("*", file.path(store, ".gitignore")) 30 | out <- tar_git_status_data(stash_gitignore = FALSE) 31 | expect_equal(nrow(out), 0L) 32 | }) 33 | -------------------------------------------------------------------------------- /tests/testthat/test-tar_git_status_targets.R: -------------------------------------------------------------------------------- 1 | targets::tar_test("tar_git_status_targets() up-to-date targets", { 2 | skip_os_git() 3 | git_setup_init() 4 | expect_equal(nrow(tar_git_status_targets(callr_function = NULL)), 0L) 5 | }) 6 | 7 | targets::tar_test("tar_git_status_targets() outdated targets", { 8 | skip_os_git() 9 | git_setup_init() 10 | targets::tar_invalidate(everything()) 11 | expect_gt(nrow(tar_git_status_targets(callr_function = NULL)), 0L) 12 | }) 13 | -------------------------------------------------------------------------------- /tests/testthat/test-utils_assert.R: -------------------------------------------------------------------------------- 1 | # TODO: remove when it is safe to import 2 | # tar_assert_file() from `targets`. 3 | targets::tar_test("tar_assert_file()", { 4 | expect_error(tar_assert_file(0), class = "tar_condition_validate") 5 | expect_error(tar_assert_file("x"), class = "tar_condition_validate") 6 | file.create("x") 7 | expect_error(tar_assert_file(c("x", "y")), class = "tar_condition_validate") 8 | expect_silent(tar_assert_file("x")) 9 | }) 10 | 11 | targets::tar_test("tar_git_assert_commits_code()", { 12 | skip_os_git() 13 | gert::git_init() 14 | expect_error( 15 | tar_git_assert_commits_code(getwd()), 16 | class = "tar_condition_validate" 17 | ) 18 | git_setup_init() 19 | expect_silent(tar_git_assert_commits_code(getwd())) 20 | }) 21 | 22 | targets::tar_test("tar_git_assert_commits_data()", { 23 | skip_os_git() 24 | gert::git_init() 25 | expect_error( 26 | tar_git_assert_commits_data(getwd()), 27 | class = "tar_condition_validate" 28 | ) 29 | git_setup_init() 30 | expect_silent(tar_git_assert_commits_data(getwd())) 31 | }) 32 | 33 | targets::tar_test("tar_git_assert_repo_code()", { 34 | skip_os_git() 35 | expect_error( 36 | tar_git_assert_repo_code(getwd()), 37 | class = "tar_condition_validate" 38 | ) 39 | gert::git_init() 40 | expect_silent(tar_git_assert_repo_code(getwd())) 41 | }) 42 | 43 | targets::tar_test("tar_git_assert_repo_data()", { 44 | skip_os_git() 45 | expect_error( 46 | tar_git_assert_repo_data(getwd()), 47 | class = "tar_condition_validate" 48 | ) 49 | gert::git_init() 50 | expect_silent(tar_git_assert_repo_data(getwd())) 51 | }) 52 | 53 | targets::tar_test("tar_git_assert_repo_data() no branch", { 54 | skip_os_git() 55 | git_setup_init() 56 | store <- targets::tar_config_get("store") 57 | expect_error( 58 | tar_git_assert_snapshot(branch = "nope", store = store), 59 | class = "tar_condition_validate" 60 | ) 61 | tar_git_snapshot(status = FALSE, verbose = FALSE) 62 | branch <- tar_git_branch_snapshot(tar_git_log()$commit_code) 63 | expect_silent(tar_git_assert_snapshot(branch = branch, store = store)) 64 | }) 65 | 66 | targets::tar_test("tar_assert_finite", { 67 | expect_silent(tar_assert_finite(1)) 68 | expect_silent(tar_assert_finite(c(1, 2))) 69 | expect_error(tar_assert_finite(c(1, Inf)), class = "tar_condition_validate") 70 | }) 71 | -------------------------------------------------------------------------------- /tests/testthat/test-utils_cli.R: -------------------------------------------------------------------------------- 1 | targets::tar_test("cli_danger()", { 2 | expect_message(cli_danger("x", verbose = TRUE)) 3 | expect_silent(cli_danger("x", verbose = FALSE)) 4 | }) 5 | 6 | targets::tar_test("cli_info()", { 7 | expect_message(cli_info("x", verbose = TRUE)) 8 | expect_silent(cli_info("x", verbose = FALSE)) 9 | }) 10 | 11 | targets::tar_test("cli_success()", { 12 | expect_message(cli_success("x", verbose = TRUE)) 13 | expect_silent(cli_success("x", verbose = FALSE)) 14 | }) 15 | 16 | targets::tar_test("cli_warning()", { 17 | expect_message(cli_warning("x", verbose = TRUE)) 18 | expect_silent(cli_warning("x", verbose = FALSE)) 19 | }) 20 | 21 | targets::tar_test("cli_blue_bullet()", { 22 | expect_message(cli_blue_bullet("x")) 23 | }) 24 | 25 | targets::tar_test("cli_indent()", { 26 | expect_message(cli_indent("x")) 27 | }) 28 | -------------------------------------------------------------------------------- /tests/testthat/test-utils_data.R: -------------------------------------------------------------------------------- 1 | targets::tar_test("inner_merge()", { 2 | x <- tibble::tibble(by = letters[c(1, 1, 2, 2, 3, 3, 4)], x = seq_len(7)) 3 | y <- tibble::tibble(by = letters[c(1, 2, 3, 5)], y = LETTERS[c(1, 2, 3, 5)]) 4 | out <- inner_merge(x = x, y = y, by = "by") 5 | expect_equal(sort(colnames(out)), sort(c("by", "x", "y"))) 6 | out <- tibble::tibble(by = out$by, x = out$x, y = out$y) 7 | exp <- tibble::tibble( 8 | by = letters[c(1, 1, 2, 2, 3, 3)], 9 | x = seq_len(6), 10 | y = LETTERS[c(1, 1, 2, 2, 3, 3)] 11 | ) 12 | expect_equal(out, exp) 13 | }) 14 | 15 | targets::tar_test("write_new_lines()", { 16 | write_new_lines(lines = c("a", "b", "c"), path = "file.txt") 17 | expect_equal(readLines("file.txt"), c("a", "b", "c")) 18 | write_new_lines(lines = c("b", "d", "e"), path = "file.txt") 19 | expect_equal( 20 | sort(readLines("file.txt")), 21 | sort(c("a", "b", "c", "d", "e")) 22 | ) 23 | write_new_lines(lines = c("b", "d", "e"), path = "file.txt") 24 | expect_equal( 25 | sort(readLines("file.txt")), 26 | sort(c("a", "b", "c", "d", "e")) 27 | ) 28 | }) 29 | -------------------------------------------------------------------------------- /tests/testthat/test-utils_git.R: -------------------------------------------------------------------------------- 1 | targets::tar_test("tar_git_add()", { 2 | skip_os_git() 3 | repo <- tempfile() 4 | dir.create(repo) 5 | gert::git_init(path = repo) 6 | writeLines("contents", file.path(repo, "file")) 7 | tar_git_add(files = "file", repo = repo) 8 | status <- gert::git_status(repo = repo) 9 | expect_equal(status$file, "file") 10 | expect_equal(status$status, "new") 11 | expect_true(status$staged) 12 | }) 13 | 14 | targets::tar_test("tar_git_branch_checkout()", { 15 | skip_os_git() 16 | repo <- tempfile() 17 | dir.create(repo) 18 | gert::git_init(path = repo) 19 | writeLines("contents", file.path(repo, "file")) 20 | gert::git_add(files = "file", repo = repo) 21 | gert::git_commit(message = "msg", repo = repo) 22 | gert::git_branch_create(branch = "new_branch", repo = repo) 23 | tar_git_branch_checkout(branch = "new_branch", repo = repo, force = FALSE) 24 | expect_equal(gert::git_branch(repo = repo), "new_branch") 25 | }) 26 | 27 | targets::tar_test("tar_git_branch_create()", { 28 | skip_os_git() 29 | repo <- tempfile() 30 | dir.create(repo) 31 | gert::git_init(path = repo) 32 | writeLines("contents", file.path(repo, "file")) 33 | gert::git_add(files = "file", repo = repo) 34 | gert::git_commit(message = "msg", repo = repo) 35 | branch <- gert::git_branch(repo = repo) 36 | tar_git_branch_create(branch = "new_branch", repo = repo) 37 | expect_equal(gert::git_branch(repo = repo), branch) 38 | expect_true("new_branch" %in% gert::git_branch_list(repo = repo)$name) 39 | }) 40 | 41 | targets::tar_test("tar_git_commit()", { 42 | skip_os_git() 43 | repo <- tempfile() 44 | dir.create(repo) 45 | gert::git_init(path = repo) 46 | writeLines("contents", file.path(repo, "file")) 47 | gert::git_add(files = "file", repo = repo) 48 | tar_git_commit(message = "msg", repo = repo) 49 | status <- gert::git_status(repo = repo) 50 | expect_equal(nrow(status), 0L) 51 | log <- gert::git_log(repo = repo) 52 | expect_equal(trimws(log$message), "msg") 53 | }) 54 | 55 | targets::tar_test("tar_git_commit_all()", { 56 | skip_os_git() 57 | repo <- tempfile() 58 | dir.create(repo) 59 | gert::git_init(path = repo) 60 | writeLines("contents", file.path(repo, "file")) 61 | gert::git_add(files = "file", repo = repo) 62 | gert::git_commit(message = "First commit", repo = repo) 63 | writeLines("new_contents", file.path(repo, "file")) 64 | tar_git_commit_all(message = "msg2", repo = repo) 65 | status <- gert::git_status(repo = repo) 66 | expect_equal(nrow(status), 0L) 67 | log <- gert::git_log(repo = repo) 68 | expect_true("msg2" %in% trimws(log$message)) 69 | }) 70 | 71 | targets::tar_test("tar_git_init_repo()", { 72 | skip_os_git() 73 | repo <- tempfile() 74 | dir.create(repo) 75 | tar_git_init_repo(path = repo) 76 | expect_true(file.exists(file.path(repo, ".git"))) 77 | }) 78 | 79 | targets::tar_test("tar_git_reset_hard()", { 80 | skip_os_git() 81 | repo <- tempfile() 82 | dir.create(repo) 83 | gert::git_init(path = repo) 84 | writeLines("contents", file.path(repo, "file")) 85 | gert::git_add(files = "file", repo = repo) 86 | gert::git_commit(message = "First commit", repo = repo) 87 | writeLines("new_contents", file.path(repo, "file")) 88 | expect_equal(readLines(file.path(repo, "file")), "new_contents") 89 | tar_git_reset_hard(repo = repo) 90 | expect_equal(readLines(file.path(repo, "file")), "contents") 91 | }) 92 | 93 | targets::tar_test("data branch naming", { 94 | expect_equal(tar_git_branch_snapshot("x"), "code=x") 95 | expect_equal(tar_git_commit_code("code=x"), "x") 96 | }) 97 | 98 | targets::tar_test("tar_git_gitignore_unstash()", { 99 | tmp <- tempfile() 100 | dir.create(tmp) 101 | from <- file.path(tmp, ".gitignore") 102 | to <- file.path(tmp, ".gittargets_gitignore") 103 | writeLines("*", from) 104 | tar_git_gitignore_stash(tmp) 105 | expect_equal(readLines(from), tar_git_gitignore_lines()) 106 | expect_equal(readLines(to), "*") 107 | tar_git_gitignore_unstash(tmp) 108 | expect_false(file.exists(file.path(tmp, ".gittargets_gitignore"))) 109 | expect_equal(readLines(from), "*") 110 | }) 111 | 112 | targets::tar_test("tar_git_gitignore_restore(), stash .gitignore", { 113 | tmp <- tempfile() 114 | dir.create(tmp) 115 | from <- file.path(tmp, ".gitignore") 116 | to <- file.path(tmp, ".gittargets_gitignore") 117 | writeLines("*", from) 118 | tar_git_gitignore_stash(tmp) 119 | tar_git_gitignore_restore(tmp) 120 | expect_equal(readLines(from), "*") 121 | }) 122 | 123 | targets::tar_test("tar_git_gitignore_restore(), no .gitignore", { 124 | tmp <- tempfile() 125 | dir.create(tmp) 126 | from <- file.path(tmp, ".gitignore") 127 | to <- file.path(tmp, ".gittargets_gitignore") 128 | writeLines("*", from) 129 | tar_git_gitignore_stash(tmp) 130 | unlink(from) 131 | tar_git_gitignore_restore(tmp) 132 | expect_equal(readLines(from), "*") 133 | }) 134 | 135 | targets::tar_test("tar_git_binary()", { 136 | skip_os_git() 137 | old <- Sys.getenv("TAR_GIT") 138 | on.exit(Sys.setenv(TAR_GIT = old)) 139 | Sys.setenv(TAR_GIT = "nope") 140 | expect_error(tar_git_binary(), class = "tar_condition_validate") 141 | file.create("nope") 142 | expect_equal(tar_git_binary(), "nope") 143 | }) 144 | -------------------------------------------------------------------------------- /tests/testthat/test-utils_logic.R: -------------------------------------------------------------------------------- 1 | targets::tar_test("%|||%", { 2 | expect_equal("x" %|||% "y", "x") 3 | expect_equal(character(0) %|||% "y", character(0)) 4 | expect_equal(NULL %|||% "y", "y") 5 | }) 6 | 7 | targets::tar_test("if_any()", { 8 | expect_true(if_any(TRUE, TRUE, FALSE)) 9 | expect_false(if_any(FALSE, TRUE, FALSE)) 10 | }) 11 | -------------------------------------------------------------------------------- /vignettes/.gitignore: -------------------------------------------------------------------------------- 1 | *.html 2 | *.R 3 | -------------------------------------------------------------------------------- /vignettes/git.Rmd: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Tutorial: Git data backend" 3 | output: rmarkdown::html_vignette 4 | vignette: > 5 | %\VignetteIndexEntry{Tutorial: Git data backend} 6 | %\VignetteEngine{knitr::rmarkdown} 7 | %\VignetteEncoding{UTF-8} 8 | --- 9 | 10 | ```{r setup, include = FALSE} 11 | library(gert) 12 | library(gittargets) 13 | library(targets) 14 | tmp <- tempfile() 15 | dir.create(tmp) 16 | knitr::opts_knit$set(root.dir = tmp) 17 | knitr::opts_chunk$set( 18 | collapse = TRUE, 19 | comment = "#>", 20 | eval = identical(Sys.getenv("NOT_CRAN"), "true") && 21 | tar_git_ok(verbose = FALSE) 22 | ) 23 | ``` 24 | 25 | This tutorial shows how to use `gittargets` with the Git-based data versioning backend. Before proceeding, please read the [README](https://github.com/ropensci/gittargets/blob/main/README.md) file or [documentation website front page](https://docs.ropensci.org/gittargets/) for an overview of the package. 26 | 27 | ## Installation 28 | 29 | Please begin with the installation instructions on the [documentation website](https://docs.ropensci.org/gittargets/). In addition, if your `targets` pipeline generates large data files, consider installing [Git LFS](https://git-lfs.com). The Git data backend in `gittargets` automatically opts into [Git LFS](https://git-lfs.com), so you should not need to do any manual configuration to reap the performance benefits. 30 | 31 | ## Remotes 32 | 33 | This backend is uses local [Git](https://git-scm.com) only. It is possible to push the snapshotted [data store](https://books.ropensci.org/targets/data.html) to a cloud service like [GitHub](https://github.com), [GitLab](https://about.gitlab.com), or [Bitbucket](https://bitbucket.org), but this is the user's responsibility. Pipelines usually generate large data files, and [GitHub](https://github.com) and its peers have file size limitations. Also, `gittargets` automatically opts into [Git LFS](https://git-lfs.com) locally (unless `git_lfs` is `FALSE` in `tar_git_init()`), and [Git LFS](https://git-lfs.com) on the cloud is a paid service. 34 | 35 | ## Overall workflow 36 | 37 | The most important steps of the Git data backend are as follows. The rest of this vignette walks through these steps in greater depth. 38 | 39 | 1. Create the source code and run the pipeline at least once so the [data store](https://books.ropensci.org/targets/data.html) exists. 40 | 1. `tar_git_init()`: initialize a [Git](https://git-scm.com)/[Git-LFS](https://git-lfs.com) repository for the [data store](https://books.ropensci.org/targets/data.html). 41 | 1. Bring the pipeline up to date (e.g. with [`tar_make()`](https://docs.ropensci.org/targets/reference/tar_make.html)) and commit any changes to the source code. 42 | 1. `tar_git_snapshot()`: create a data snapshot for the current code commit. 43 | 1. Develop the pipeline. Creating new code commits and code branches early and often, and create data snapshots at key strategic milestones. 44 | 1. `tar_git_checkout()`: revert the data to the appropriate prior snapshot. 45 | 46 | ## Create code 47 | 48 | To begin development, we write `_targets.R` file for a [`targets`](https://docs.ropensci.org/targets/) pipeline. [`targets`](https://docs.ropensci.org/targets/) can handle large complex pipelines for [machine learning](https://github.com/wlandau/targets-keras), [Bayesian data analysis](https://github.com/wlandau/rmedicine2021-pipeline), and much more. However, this tutorial focuses on a much simpler pipeline for the sake of pedagogical simplicity. 49 | 50 | ```{r, eval = FALSE} 51 | # _targets.R 52 | library(targets) 53 | list( 54 | tar_target(data, datasets::airquality), 55 | tar_target(result, summary(data)) 56 | ) 57 | ``` 58 | 59 | ```{r, echo = FALSE} 60 | tar_script( 61 | list( 62 | tar_target(data, datasets::airquality), 63 | tar_target(result, summary(data)) 64 | ) 65 | ) 66 | ``` 67 | 68 | ## Run pipeline 69 | 70 | With our target script in hand, we run the pipeline.^[ describes heavy-duty alternatives to `tar_make()`.] 71 | 72 | ```{r} 73 | tar_make() 74 | ``` 75 | 76 | We inspect the output with `tar_read()`. 77 | 78 | ```{r} 79 | tar_read(result) 80 | ``` 81 | 82 | ## Commit code 83 | 84 | We usually iterate between writing code and running the pipeline until we have a decent set of results. After that, we commit the code to a [Git](https://git-scm.com/docs/git-pack-refs) repository, which may or may not live on [GitHub](https://github.com).^[Alternatives to GitHub include GitLab and Bitbucket.] [Happy Git with R](https://happygitwithr.com) is a great way to learn Git, and the [`gert`](https://docs.ropensci.org/gert/) package is a convenient way to interact with Git from R. 85 | 86 | ```{r, message = FALSE, output = FALSE, results = "hide"} 87 | library(gert) 88 | git_init() 89 | git_add("_targets.R") 90 | git_commit("Begin analyzing the airquality dataset") 91 | git_branch_create("airquality") 92 | ``` 93 | 94 | ## Snapshot data 95 | 96 | Before we snapshot the data, we should check that the code is up to date in the Git repository and the targets are up to date in the pipeline. The `tar_git_status()` function is an easy way to do this.^[Helper functions `tar_git_status_code()`, `tar_git_status_targets()`, and `tar_git_status_data()` each generate a piece of the `tar_git_status()` output.] 97 | 98 | ```{r} 99 | tar_git_status() 100 | ``` 101 | 102 | Our code and pipeline look ready for a data snapshot. First, we initialize the data repository with `tar_git_init()`. `tar_git_init()` writes a `.gitattributes` file in the data store that automatically opts into [Git LFS](https://git-lfs.com). If you have [Git LFS](https://git-lfs.com) but do not wish to use it, please remove the `.gitattributes` after calling `tar_git_init()`. 103 | 104 | ```{r} 105 | tar_git_init() 106 | ``` 107 | 108 | Then, we create our first data commit with `tar_git_snapshot()`.^[Ordinarily, `tar_git_snapshot()` shows runs `tar_git_status()` and prompts the user to confirm the snapshot. But in this example, we skip this step.] 109 | 110 | ```{r, eval = FALSE} 111 | tar_git_snapshot() 112 | ``` 113 | 114 | ```{r, echo = FALSE} 115 | tar_git_snapshot(status = FALSE) 116 | ``` 117 | 118 | ### Snapshot model 119 | 120 | ![](./snapshot-model-git.png) 121 | 122 | In the Git data backend, a data snapshot is a special kind of Git commit (gray boxes above). Each data commit is part of a data branch (vertical dashed lines above), and each data branch is specific to the current code commit (green and brown boxes above). In fact, each data branch name is of the form `"code="`, where `` is the Git SHA1 hash of the corresponding code commit. You can always create a data snapshot, but it will supersede any prior data snapshot you already have for the current code commit. To revert to a previous data snapshots for a given code snapshot, you will need to manually enter the repository and check out the relevant data commit. 123 | 124 | ## Repeat 125 | 126 | Development typically happens in cycles: develop the code, run the pipeline, commit the code, snapshot the data, and repeat. Not all code commits need a data snapshot, especially if the [`targets`](https://docs.ropensci.org/targets/) pipeline generates a lot of data. But even then, it is helpful to snapshot the data at key milestones, e.g. if an alternative research question comes up and it is desirable to create a new Git branch for the code. For example, suppose we wish to apply the same pipeline to a different dataset. The code changes: 127 | 128 | ```{r, eval = FALSE} 129 | # _targets.R 130 | library(targets) 131 | list( 132 | tar_target(data, datasets::UKgas), # different dataset 133 | tar_target(result, summary(data)) 134 | ) 135 | ``` 136 | 137 | ```{r, echo = FALSE} 138 | tar_script( 139 | list( 140 | tar_target(data, datasets::UKgas), 141 | tar_target(result, summary(data)) 142 | ) 143 | ) 144 | ``` 145 | 146 | We run the pipeline and inspect the new output. 147 | 148 | ```{r} 149 | tar_make() 150 | ``` 151 | 152 | ```{r} 153 | tar_read(result) 154 | ``` 155 | 156 | We put the code in a new Git branch. 157 | 158 | ```{r} 159 | git_branch_create("UKgas") 160 | git_add("_targets.R") 161 | git_commit("Switch to UKgas dataset") 162 | ``` 163 | 164 | Finally, we create a data snapshot for the new code commit. 165 | 166 | ```{r, eval = FALSE} 167 | tar_git_snapshot() 168 | ``` 169 | 170 | ```{r, echo = FALSE} 171 | tar_git_snapshot(status = FALSE) 172 | ``` 173 | 174 | ## View log 175 | 176 | Now, suppose we want to switch the project back to the original dataset (`airquality`). To transition completely, we need to revert both the code and the data. If we only revert the code, then the data store will sill reflect the `UKgas` dataset, and none of our targets will be up to date. At this point, it is a good time to pause and check the `gittargets` log to see which code commits have available data snapshots.^[If you chose not to call `tar_git_snapshot()` for some code commits, then not all your code commits will have available data snapshots.] 177 | 178 | ```{r} 179 | tar_git_log() 180 | ``` 181 | 182 | ## Check out code 183 | 184 | To check out the old `airquality` code, we use `gert::git_branch_checkout()`. 185 | 186 | ```{r, message = FALSE, output = FALSE, results = "hide"} 187 | git_branch_checkout("airquality") 188 | ``` 189 | 190 | But because we did not revert the data, our results still reflect the `UKgas` dataset. 191 | 192 | ```{r} 193 | tar_read(result) 194 | ``` 195 | 196 | Thus, all our targets are out of date. 197 | 198 | ```{r} 199 | tar_outdated() 200 | ``` 201 | 202 | ## Check out data 203 | 204 | To bring our targets back up to date with the `airquality` data, instead of beginning a potentially long computation with `tar_make()`, we can check out the data snapshot that matches our current code commit. 205 | 206 | ```{r} 207 | tar_git_checkout() 208 | ``` 209 | 210 | Now, our results reflect the `airquality` dataset we previously analyzed. 211 | 212 | ```{r} 213 | tar_read(result) 214 | ``` 215 | 216 | And all our targets are up to date. 217 | 218 | ```{r} 219 | tar_outdated() 220 | ``` 221 | 222 | ## Merges 223 | 224 | It is common to [merge](https://git-scm.com/docs/git-merge) code branches into one another. When this happens, a new merge commit is created in the code repository, and the data repository remains unchanged. In fact, the only change is that the code repository is now at a new commit that has no data snapshot yet. If you wish, simply create a new data snapshot with `tar_git_snapshot()`. If the code commit immediately prior had an up-to-date data snapshot of its own, then the new snapshot for the merge commit should cost little storage or runtime. 225 | 226 | ## Custom data files 227 | 228 | Only files inside the `targets` data store are tracked in a `gittargets` data snapshot. If your pipeline requires custom external files, you may put them in a folder called `_targets/user/`. That way, `gittargets` will automatically put them under data version control in the next snapshot. 229 | 230 | ## Performance 231 | 232 | If your `targets` pipeline generates large data files, consider installing [Git LFS](https://git-lfs.com). Once you install [Git LFS](https://git-lfs.com), it should just work on your project right out of the box because `tar_git_init()` writes the following to `_targets/.gitattributes`: 233 | 234 | ```{sh, eval = FALSE} 235 | objects filter=lfs diff=lfs merge=lfs -text 236 | ``` 237 | 238 | In addition, every data snapshot with `tar_git_snapshot()` creates a new Git branch. With thousands of commits and thus thousands of branches, performance may suffer unless you ensure `pack_refs` is `TRUE` in the function call (default).^[Alternatively, you can call `tar_git_snapshot(pack_refs = FALSE)` and then afterwards run `git pack-refs --all`](https://git-scm.com/docs/git-pack-refs) in the command line with your working directory inside `_targets/`.] 239 | -------------------------------------------------------------------------------- /vignettes/snapshot-model-git.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ropensci/gittargets/c678c41c0f2e4357e817eb61c3a35f1bdbc305f3/vignettes/snapshot-model-git.png --------------------------------------------------------------------------------