├── .editorconfig ├── .github └── workflows │ ├── README.md │ ├── pr-close-signal.yaml │ ├── pr-comment.yaml │ ├── pr-post-remove-branch.yaml │ ├── pr-preflight.yaml │ ├── pr-receive.yaml │ ├── sandpaper-main.yaml │ ├── sandpaper-version.txt │ ├── update-cache.yaml │ └── update-workflows.yaml ├── .gitignore ├── AUTHORS ├── CITATION ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── LICENSE.md ├── README.md ├── Topic_list_brainstorming.md ├── config.yaml ├── episodes ├── 01-introduction.md ├── 02-Wikidata_underlying_concepts.md ├── 03-intro_to_editing.md ├── 04-advanced_editing.md ├── 05-intro_to_querying.md ├── 06-bulk_actions.md └── fig │ ├── Data_Structure_Diagram.jpg │ ├── Knowledge-Graph-Example-Meaning-of-it-All-More.png │ ├── Knowledge-Graph-Example-Meaning-of-it-All.png │ ├── Mae_Jemison_Wikidata.png │ ├── RDF_subject_predicate_object.jpg │ ├── RDF_subject_predicate_object.svg │ ├── Relational_database_terms.svg.png │ ├── Wikidata_Item_Q2_Earth.png │ ├── Wikidata_Main_Page.png │ ├── episode_04_screenshot_01_latest_articles.png │ ├── episode_04_screenshot_02_choose_doi.png │ ├── episode_04_screenshot_03_paste_into_sourcemd.png │ ├── episode_04_screenshot_04_open_quickstatements.png │ ├── episode_04_screenshot_05_run_editing.png │ ├── episode_05_Barchart.png │ ├── episode_05_Bubblechart.png │ ├── episode_05_Graph.png │ ├── episode_05_Map.jpg │ ├── episode_05_Map_CloseUp.png │ ├── episode_05_Map_Toolkit.jpg │ ├── episode_5_Bar_Countries.png │ ├── episode_5_Bubble.png │ ├── episode_5_Graph.png │ ├── episode_5_Imagegrid.jpg │ ├── episode_5_Map_USA.png │ ├── episode_5_Map_WorldWide.png │ ├── episode_5_Table.png │ ├── episode_5_defaultview.png │ ├── episode_5_table.jpg │ ├── l03-Mae_Jemison_01_Enter_in_form.png │ ├── l03-Mae_Jemison_02_Newly_created_page.png │ ├── l03-Mae_Jemison_03_Official_Wikidata_entry_full.png │ ├── l03-Mae_Jemison_04_Enter_in_form_book.png │ ├── l03-Mae_Jemison_05_Newly_created_book.png │ ├── l03-Mae_Jemison_06_Form_adding_instance_of.png │ ├── l03-Mae_Jemison_07_Added_instance_of.png │ ├── l03-Mae_Jemison_08_Form_adding_publication_date.png │ ├── l03-Mae_Jemison_09_Added_publication_date.png │ ├── l03-Mae_Jemison_10_Added_authors.png │ ├── l03-Mae_Jemison_10_Form_adding_authors.png │ ├── l03-Mae_Jemison_11_Hoover_link.png │ ├── l03-Screenshot_test_instance_empty_create_page.png │ ├── l03-Screenshot_test_instance_front_page.png │ └── obsolete │ ├── Screenshot_2019-02-18 Surely You're Joking, Mr Feynman .png │ ├── Screenshot_Wikidata_Joking.png │ ├── Screenshot_Wikidata_Richard_Feynman.png │ ├── Screenshot_Wikidata_Richard_Feynman_excerpt.png │ ├── Screenshot_test_instance_Joking_add_statements_01.png │ ├── Screenshot_test_instance_Joking_add_statements_02.png │ ├── Screenshot_test_instance_Joking_add_statements_03.png │ ├── Screenshot_test_instance_Joking_add_statements_04.png │ ├── Screenshot_test_instance_Joking_add_statements_05.png │ ├── Screenshot_test_instance_Joking_add_statements_06.png │ ├── Screenshot_test_instance_Joking_add_statements_08.png │ ├── Screenshot_test_instance_Joking_add_statements_09.png │ ├── Screenshot_test_instance_Joking_add_statements_10.png │ ├── Screenshot_test_instance_Joking_add_statements_11.png │ ├── Screenshot_test_instance_Joking_add_statements_12.png │ ├── Screenshot_test_instance_Joking_fresh.png │ ├── Screenshot_test_instance_Richard_Feynman_fresh.png │ ├── Screenshot_test_instance_create_Feyman.png │ └── Screenshot_test_instance_create_Joking.png ├── index.md ├── instructors ├── design.md └── instructor-notes.md ├── learners ├── discuss.md ├── reference.md └── setup.md ├── meeting_notes.md ├── profiles └── learner-profiles.md └── site └── README.md /.editorconfig: -------------------------------------------------------------------------------- 1 | root = true 2 | 3 | [*] 4 | charset = utf-8 5 | insert_final_newline = true 6 | trim_trailing_whitespace = true 7 | 8 | [*.md] 9 | indent_size = 2 10 | indent_style = space 11 | max_line_length = 100 # Please keep this in sync with bin/lesson_check.py! 12 | trim_trailing_whitespace = false # keep trailing spaces in markdown - 2+ spaces are translated to a hard break (
) 13 | 14 | [*.r] 15 | max_line_length = 80 16 | 17 | [*.py] 18 | indent_size = 4 19 | indent_style = space 20 | max_line_length = 79 21 | 22 | [*.sh] 23 | end_of_line = lf 24 | 25 | [Makefile] 26 | indent_style = tab 27 | -------------------------------------------------------------------------------- /.github/workflows/README.md: -------------------------------------------------------------------------------- 1 | # Carpentries Workflows 2 | 3 | This directory contains workflows to be used for Lessons using the {sandpaper} 4 | lesson infrastructure. Two of these workflows require R (`sandpaper-main.yaml` 5 | and `pr-receive.yaml`) and the rest are bots to handle pull request management. 6 | 7 | These workflows will likely change as {sandpaper} evolves, so it is important to 8 | keep them up-to-date. To do this in your lesson you can do the following in your 9 | R console: 10 | 11 | ```r 12 | # Install/Update sandpaper 13 | options(repos = c(carpentries = "https://carpentries.r-universe.dev/", 14 | CRAN = "https://cloud.r-project.org")) 15 | install.packages("sandpaper") 16 | 17 | # update the workflows in your lesson 18 | library("sandpaper") 19 | update_github_workflows() 20 | ``` 21 | 22 | Inside this folder, you will find a file called `sandpaper-version.txt`, which 23 | will contain a version number for sandpaper. This will be used in the future to 24 | alert you if a workflow update is needed. 25 | 26 | What follows are the descriptions of the workflow files: 27 | 28 | ## Deployment 29 | 30 | ### 01 Build and Deploy (sandpaper-main.yaml) 31 | 32 | This is the main driver that will only act on the main branch of the repository. 33 | This workflow does the following: 34 | 35 | 1. checks out the lesson 36 | 2. provisions the following resources 37 | - R 38 | - pandoc 39 | - lesson infrastructure (stored in a cache) 40 | - lesson dependencies if needed (stored in a cache) 41 | 3. builds the lesson via `sandpaper:::ci_deploy()` 42 | 43 | #### Caching 44 | 45 | This workflow has two caches; one cache is for the lesson infrastructure and 46 | the other is for the lesson dependencies if the lesson contains rendered 47 | content. These caches are invalidated by new versions of the infrastructure and 48 | the `renv.lock` file, respectively. If there is a problem with the cache, 49 | manual invaliation is necessary. You will need maintain access to the repository 50 | and you can either go to the actions tab and [click on the caches button to find 51 | and invalidate the failing cache](https://github.blog/changelog/2022-10-20-manage-caches-in-your-actions-workflows-from-web-interface/) 52 | or by setting the `CACHE_VERSION` secret to the current date (which will 53 | invalidate all of the caches). 54 | 55 | ## Updates 56 | 57 | ### Setup Information 58 | 59 | These workflows run on a schedule and at the maintainer's request. Because they 60 | create pull requests that update workflows/require the downstream actions to run, 61 | they need a special repository/organization secret token called 62 | `SANDPAPER_WORKFLOW` and it must have the `public_repo` and `workflow` scope. 63 | 64 | This can be an individual user token, OR it can be a trusted bot account. If you 65 | have a repository in one of the official Carpentries accounts, then you do not 66 | need to worry about this token being present because the Carpentries Core Team 67 | will take care of supplying this token. 68 | 69 | If you want to use your personal account: you can go to 70 | 71 | to create a token. Once you have created your token, you should copy it to your 72 | clipboard and then go to your repository's settings > secrets > actions and 73 | create or edit the `SANDPAPER_WORKFLOW` secret, pasting in the generated token. 74 | 75 | If you do not specify your token correctly, the runs will not fail and they will 76 | give you instructions to provide the token for your repository. 77 | 78 | ### 02 Maintain: Update Workflow Files (update-workflow.yaml) 79 | 80 | The {sandpaper} repository was designed to do as much as possible to separate 81 | the tools from the content. For local builds, this is absolutely true, but 82 | there is a minor issue when it comes to workflow files: they must live inside 83 | the repository. 84 | 85 | This workflow ensures that the workflow files are up-to-date. The way it work is 86 | to download the update-workflows.sh script from GitHub and run it. The script 87 | will do the following: 88 | 89 | 1. check the recorded version of sandpaper against the current version on github 90 | 2. update the files if there is a difference in versions 91 | 92 | After the files are updated, if there are any changes, they are pushed to a 93 | branch called `update/workflows` and a pull request is created. Maintainers are 94 | encouraged to review the changes and accept the pull request if the outputs 95 | are okay. 96 | 97 | This update is run weekly or on demand. 98 | 99 | ### 03 Maintain: Update Package Cache (update-cache.yaml) 100 | 101 | For lessons that have generated content, we use {renv} to ensure that the output 102 | is stable. This is controlled by a single lockfile which documents the packages 103 | needed for the lesson and the version numbers. This workflow is skipped in 104 | lessons that do not have generated content. 105 | 106 | Because the lessons need to remain current with the package ecosystem, it's a 107 | good idea to make sure these packages can be updated periodically. The 108 | update cache workflow will do this by checking for updates, applying them in a 109 | branch called `updates/packages` and creating a pull request with _only the 110 | lockfile changed_. 111 | 112 | From here, the markdown documents will be rebuilt and you can inspect what has 113 | changed based on how the packages have updated. 114 | 115 | ## Pull Request and Review Management 116 | 117 | Because our lessons execute code, pull requests are a secruity risk for any 118 | lesson and thus have security measures associted with them. **Do not merge any 119 | pull requests that do not pass checks and do not have bots commented on them.** 120 | 121 | This series of workflows all go together and are described in the following 122 | diagram and the below sections: 123 | 124 | ![Graph representation of a pull request](https://carpentries.github.io/sandpaper/articles/img/pr-flow.dot.svg) 125 | 126 | ### Pre Flight Pull Request Validation (pr-preflight.yaml) 127 | 128 | This workflow runs every time a pull request is created and its purpose is to 129 | validate that the pull request is okay to run. This means the following things: 130 | 131 | 1. The pull request does not contain modified workflow files 132 | 2. If the pull request contains modified workflow files, it does not contain 133 | modified content files (such as a situation where @carpentries-bot will 134 | make an automated pull request) 135 | 3. The pull request does not contain an invalid commit hash (e.g. from a fork 136 | that was made before a lesson was transitioned from styles to use the 137 | workbench). 138 | 139 | Once the checks are finished, a comment is issued to the pull request, which 140 | will allow maintainers to determine if it is safe to run the 141 | "Receive Pull Request" workflow from new contributors. 142 | 143 | ### Receive Pull Request (pr-receive.yaml) 144 | 145 | **Note of caution:** This workflow runs arbitrary code by anyone who creates a 146 | pull request. GitHub has safeguarded the token used in this workflow to have no 147 | priviledges in the repository, but we have taken precautions to protect against 148 | spoofing. 149 | 150 | This workflow is triggered with every push to a pull request. If this workflow 151 | is already running and a new push is sent to the pull request, the workflow 152 | running from the previous push will be cancelled and a new workflow run will be 153 | started. 154 | 155 | The first step of this workflow is to check if it is valid (e.g. that no 156 | workflow files have been modified). If there are workflow files that have been 157 | modified, a comment is made that indicates that the workflow is not run. If 158 | both a workflow file and lesson content is modified, an error will occurr. 159 | 160 | The second step (if valid) is to build the generated content from the pull 161 | request. This builds the content and uploads three artifacts: 162 | 163 | 1. The pull request number (pr) 164 | 2. A summary of changes after the rendering process (diff) 165 | 3. The rendered files (build) 166 | 167 | Because this workflow builds generated content, it follows the same general 168 | process as the `sandpaper-main` workflow with the same caching mechanisms. 169 | 170 | The artifacts produced are used by the next workflow. 171 | 172 | ### Comment on Pull Request (pr-comment.yaml) 173 | 174 | This workflow is triggered if the `pr-receive.yaml` workflow is successful. 175 | The steps in this workflow are: 176 | 177 | 1. Test if the workflow is valid and comment the validity of the workflow to the 178 | pull request. 179 | 2. If it is valid: create an orphan branch with two commits: the current state 180 | of the repository and the proposed changes. 181 | 3. If it is valid: update the pull request comment with the summary of changes 182 | 183 | Importantly: if the pull request is invalid, the branch is not created so any 184 | malicious code is not published. 185 | 186 | From here, the maintainer can request changes from the author and eventually 187 | either merge or reject the PR. When this happens, if the PR was valid, the 188 | preview branch needs to be deleted. 189 | 190 | ### Send Close PR Signal (pr-close-signal.yaml) 191 | 192 | Triggered any time a pull request is closed. This emits an artifact that is the 193 | pull request number for the next action 194 | 195 | ### Remove Pull Request Branch (pr-post-remove-branch.yaml) 196 | 197 | Tiggered by `pr-close-signal.yaml`. This removes the temporary branch associated with 198 | the pull request (if it was created). 199 | -------------------------------------------------------------------------------- /.github/workflows/pr-close-signal.yaml: -------------------------------------------------------------------------------- 1 | name: "Bot: Send Close Pull Request Signal" 2 | 3 | on: 4 | pull_request: 5 | types: 6 | [closed] 7 | 8 | jobs: 9 | send-close-signal: 10 | name: "Send closing signal" 11 | runs-on: ubuntu-22.04 12 | if: ${{ github.event.action == 'closed' }} 13 | steps: 14 | - name: "Create PRtifact" 15 | run: | 16 | mkdir -p ./pr 17 | printf ${{ github.event.number }} > ./pr/NUM 18 | - name: Upload Diff 19 | uses: actions/upload-artifact@v4 20 | with: 21 | name: pr 22 | path: ./pr 23 | -------------------------------------------------------------------------------- /.github/workflows/pr-comment.yaml: -------------------------------------------------------------------------------- 1 | name: "Bot: Comment on the Pull Request" 2 | 3 | # read-write repo token 4 | # access to secrets 5 | on: 6 | workflow_run: 7 | workflows: ["Receive Pull Request"] 8 | types: 9 | - completed 10 | 11 | concurrency: 12 | group: pr-${{ github.event.workflow_run.pull_requests[0].number }} 13 | cancel-in-progress: true 14 | 15 | 16 | jobs: 17 | # Pull requests are valid if: 18 | # - they match the sha of the workflow run head commit 19 | # - they are open 20 | # - no .github files were committed 21 | test-pr: 22 | name: "Test if pull request is valid" 23 | runs-on: ubuntu-22.04 24 | if: > 25 | github.event.workflow_run.event == 'pull_request' && 26 | github.event.workflow_run.conclusion == 'success' 27 | outputs: 28 | is_valid: ${{ steps.check-pr.outputs.VALID }} 29 | payload: ${{ steps.check-pr.outputs.payload }} 30 | number: ${{ steps.get-pr.outputs.NUM }} 31 | msg: ${{ steps.check-pr.outputs.MSG }} 32 | steps: 33 | - name: 'Download PR artifact' 34 | id: dl 35 | uses: carpentries/actions/download-workflow-artifact@main 36 | with: 37 | run: ${{ github.event.workflow_run.id }} 38 | name: 'pr' 39 | 40 | - name: "Get PR Number" 41 | if: ${{ steps.dl.outputs.success == 'true' }} 42 | id: get-pr 43 | run: | 44 | unzip pr.zip 45 | echo "NUM=$(<./NR)" >> $GITHUB_OUTPUT 46 | 47 | - name: "Fail if PR number was not present" 48 | id: bad-pr 49 | if: ${{ steps.dl.outputs.success != 'true' }} 50 | run: | 51 | echo '::error::A pull request number was not recorded. The pull request that triggered this workflow is likely malicious.' 52 | exit 1 53 | - name: "Get Invalid Hashes File" 54 | id: hash 55 | run: | 56 | echo "json<> $GITHUB_OUTPUT 59 | - name: "Check PR" 60 | id: check-pr 61 | if: ${{ steps.dl.outputs.success == 'true' }} 62 | uses: carpentries/actions/check-valid-pr@main 63 | with: 64 | pr: ${{ steps.get-pr.outputs.NUM }} 65 | sha: ${{ github.event.workflow_run.head_sha }} 66 | headroom: 3 # if it's within the last three commits, we can keep going, because it's likely rapid-fire 67 | invalid: ${{ fromJSON(steps.hash.outputs.json)[github.repository] }} 68 | fail_on_error: true 69 | 70 | # Create an orphan branch on this repository with two commits 71 | # - the current HEAD of the md-outputs branch 72 | # - the output from running the current HEAD of the pull request through 73 | # the md generator 74 | create-branch: 75 | name: "Create Git Branch" 76 | needs: test-pr 77 | runs-on: ubuntu-22.04 78 | if: ${{ needs.test-pr.outputs.is_valid == 'true' }} 79 | env: 80 | NR: ${{ needs.test-pr.outputs.number }} 81 | permissions: 82 | contents: write 83 | steps: 84 | - name: 'Checkout md outputs' 85 | uses: actions/checkout@v4 86 | with: 87 | ref: md-outputs 88 | path: built 89 | fetch-depth: 1 90 | 91 | - name: 'Download built markdown' 92 | id: dl 93 | uses: carpentries/actions/download-workflow-artifact@main 94 | with: 95 | run: ${{ github.event.workflow_run.id }} 96 | name: 'built' 97 | 98 | - if: ${{ steps.dl.outputs.success == 'true' }} 99 | run: unzip built.zip 100 | 101 | - name: "Create orphan and push" 102 | if: ${{ steps.dl.outputs.success == 'true' }} 103 | run: | 104 | cd built/ 105 | git config --local user.email "actions@github.com" 106 | git config --local user.name "GitHub Actions" 107 | CURR_HEAD=$(git rev-parse HEAD) 108 | git checkout --orphan md-outputs-PR-${NR} 109 | git add -A 110 | git commit -m "source commit: ${CURR_HEAD}" 111 | ls -A | grep -v '^.git$' | xargs -I _ rm -r '_' 112 | cd .. 113 | unzip -o -d built built.zip 114 | cd built 115 | git add -A 116 | git commit --allow-empty -m "differences for PR #${NR}" 117 | git push -u --force --set-upstream origin md-outputs-PR-${NR} 118 | 119 | # Comment on the Pull Request with a link to the branch and the diff 120 | comment-pr: 121 | name: "Comment on Pull Request" 122 | needs: [test-pr, create-branch] 123 | runs-on: ubuntu-22.04 124 | if: ${{ needs.test-pr.outputs.is_valid == 'true' }} 125 | env: 126 | NR: ${{ needs.test-pr.outputs.number }} 127 | permissions: 128 | pull-requests: write 129 | steps: 130 | - name: 'Download comment artifact' 131 | id: dl 132 | uses: carpentries/actions/download-workflow-artifact@main 133 | with: 134 | run: ${{ github.event.workflow_run.id }} 135 | name: 'diff' 136 | 137 | - if: ${{ steps.dl.outputs.success == 'true' }} 138 | run: unzip ${{ github.workspace }}/diff.zip 139 | 140 | - name: "Comment on PR" 141 | id: comment-diff 142 | if: ${{ steps.dl.outputs.success == 'true' }} 143 | uses: carpentries/actions/comment-diff@main 144 | with: 145 | pr: ${{ env.NR }} 146 | path: ${{ github.workspace }}/diff.md 147 | 148 | # Comment if the PR is open and matches the SHA, but the workflow files have 149 | # changed 150 | comment-changed-workflow: 151 | name: "Comment if workflow files have changed" 152 | needs: test-pr 153 | runs-on: ubuntu-22.04 154 | if: ${{ always() && needs.test-pr.outputs.is_valid == 'false' }} 155 | env: 156 | NR: ${{ github.event.workflow_run.pull_requests[0].number }} 157 | body: ${{ needs.test-pr.outputs.msg }} 158 | permissions: 159 | pull-requests: write 160 | steps: 161 | - name: 'Check for spoofing' 162 | id: dl 163 | uses: carpentries/actions/download-workflow-artifact@main 164 | with: 165 | run: ${{ github.event.workflow_run.id }} 166 | name: 'built' 167 | 168 | - name: 'Alert if spoofed' 169 | id: spoof 170 | if: ${{ steps.dl.outputs.success == 'true' }} 171 | run: | 172 | echo 'body<> $GITHUB_ENV 173 | echo '' >> $GITHUB_ENV 174 | echo '## :x: DANGER :x:' >> $GITHUB_ENV 175 | echo 'This pull request has modified workflows that created output. Close this now.' >> $GITHUB_ENV 176 | echo '' >> $GITHUB_ENV 177 | echo 'EOF' >> $GITHUB_ENV 178 | 179 | - name: "Comment on PR" 180 | id: comment-diff 181 | uses: carpentries/actions/comment-diff@main 182 | with: 183 | pr: ${{ env.NR }} 184 | body: ${{ env.body }} 185 | -------------------------------------------------------------------------------- /.github/workflows/pr-post-remove-branch.yaml: -------------------------------------------------------------------------------- 1 | name: "Bot: Remove Temporary PR Branch" 2 | 3 | on: 4 | workflow_run: 5 | workflows: ["Bot: Send Close Pull Request Signal"] 6 | types: 7 | - completed 8 | 9 | jobs: 10 | delete: 11 | name: "Delete branch from Pull Request" 12 | runs-on: ubuntu-22.04 13 | if: > 14 | github.event.workflow_run.event == 'pull_request' && 15 | github.event.workflow_run.conclusion == 'success' 16 | permissions: 17 | contents: write 18 | steps: 19 | - name: 'Download artifact' 20 | uses: carpentries/actions/download-workflow-artifact@main 21 | with: 22 | run: ${{ github.event.workflow_run.id }} 23 | name: pr 24 | - name: "Get PR Number" 25 | id: get-pr 26 | run: | 27 | unzip pr.zip 28 | echo "NUM=$(<./NUM)" >> $GITHUB_OUTPUT 29 | - name: 'Remove branch' 30 | uses: carpentries/actions/remove-branch@main 31 | with: 32 | pr: ${{ steps.get-pr.outputs.NUM }} 33 | -------------------------------------------------------------------------------- /.github/workflows/pr-preflight.yaml: -------------------------------------------------------------------------------- 1 | name: "Pull Request Preflight Check" 2 | 3 | on: 4 | pull_request_target: 5 | branches: 6 | ["main"] 7 | types: 8 | ["opened", "synchronize", "reopened"] 9 | 10 | jobs: 11 | test-pr: 12 | name: "Test if pull request is valid" 13 | if: ${{ github.event.action != 'closed' }} 14 | runs-on: ubuntu-22.04 15 | outputs: 16 | is_valid: ${{ steps.check-pr.outputs.VALID }} 17 | permissions: 18 | pull-requests: write 19 | steps: 20 | - name: "Get Invalid Hashes File" 21 | id: hash 22 | run: | 23 | echo "json<> $GITHUB_OUTPUT 26 | - name: "Check PR" 27 | id: check-pr 28 | uses: carpentries/actions/check-valid-pr@main 29 | with: 30 | pr: ${{ github.event.number }} 31 | invalid: ${{ fromJSON(steps.hash.outputs.json)[github.repository] }} 32 | fail_on_error: true 33 | - name: "Comment result of validation" 34 | id: comment-diff 35 | if: ${{ always() }} 36 | uses: carpentries/actions/comment-diff@main 37 | with: 38 | pr: ${{ github.event.number }} 39 | body: ${{ steps.check-pr.outputs.MSG }} 40 | -------------------------------------------------------------------------------- /.github/workflows/pr-receive.yaml: -------------------------------------------------------------------------------- 1 | name: "Receive Pull Request" 2 | 3 | on: 4 | pull_request: 5 | types: 6 | [opened, synchronize, reopened] 7 | 8 | concurrency: 9 | group: ${{ github.ref }} 10 | cancel-in-progress: true 11 | 12 | jobs: 13 | test-pr: 14 | name: "Record PR number" 15 | if: ${{ github.event.action != 'closed' }} 16 | runs-on: ubuntu-22.04 17 | outputs: 18 | is_valid: ${{ steps.check-pr.outputs.VALID }} 19 | steps: 20 | - name: "Record PR number" 21 | id: record 22 | if: ${{ always() }} 23 | run: | 24 | echo ${{ github.event.number }} > ${{ github.workspace }}/NR # 2022-03-02: artifact name fixed to be NR 25 | - name: "Upload PR number" 26 | id: upload 27 | if: ${{ always() }} 28 | uses: actions/upload-artifact@v4 29 | with: 30 | name: pr 31 | path: ${{ github.workspace }}/NR 32 | - name: "Get Invalid Hashes File" 33 | id: hash 34 | run: | 35 | echo "json<> $GITHUB_OUTPUT 38 | - name: "echo output" 39 | run: | 40 | echo "${{ steps.hash.outputs.json }}" 41 | - name: "Check PR" 42 | id: check-pr 43 | uses: carpentries/actions/check-valid-pr@main 44 | with: 45 | pr: ${{ github.event.number }} 46 | invalid: ${{ fromJSON(steps.hash.outputs.json)[github.repository] }} 47 | 48 | build-md-source: 49 | name: "Build markdown source files if valid" 50 | needs: test-pr 51 | runs-on: ubuntu-22.04 52 | if: ${{ needs.test-pr.outputs.is_valid == 'true' }} 53 | env: 54 | GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }} 55 | RENV_PATHS_ROOT: ~/.local/share/renv/ 56 | CHIVE: ${{ github.workspace }}/site/chive 57 | PR: ${{ github.workspace }}/site/pr 58 | MD: ${{ github.workspace }}/site/built 59 | steps: 60 | - name: "Check Out Main Branch" 61 | uses: actions/checkout@v4 62 | 63 | - name: "Check Out Staging Branch" 64 | uses: actions/checkout@v4 65 | with: 66 | ref: md-outputs 67 | path: ${{ env.MD }} 68 | 69 | - name: "Set up R" 70 | uses: r-lib/actions/setup-r@v2 71 | with: 72 | use-public-rspm: true 73 | install-r: false 74 | 75 | - name: "Set up Pandoc" 76 | uses: r-lib/actions/setup-pandoc@v2 77 | 78 | - name: "Setup Lesson Engine" 79 | uses: carpentries/actions/setup-sandpaper@main 80 | with: 81 | cache-version: ${{ secrets.CACHE_VERSION }} 82 | 83 | - name: "Setup Package Cache" 84 | uses: carpentries/actions/setup-lesson-deps@main 85 | with: 86 | cache-version: ${{ secrets.CACHE_VERSION }} 87 | 88 | - name: "Validate and Build Markdown" 89 | id: build-site 90 | run: | 91 | sandpaper::package_cache_trigger(TRUE) 92 | sandpaper::validate_lesson(path = '${{ github.workspace }}') 93 | sandpaper:::build_markdown(path = '${{ github.workspace }}', quiet = FALSE) 94 | shell: Rscript {0} 95 | 96 | - name: "Generate Artifacts" 97 | id: generate-artifacts 98 | run: | 99 | sandpaper:::ci_bundle_pr_artifacts( 100 | repo = '${{ github.repository }}', 101 | pr_number = '${{ github.event.number }}', 102 | path_md = '${{ env.MD }}', 103 | path_pr = '${{ env.PR }}', 104 | path_archive = '${{ env.CHIVE }}', 105 | branch = 'md-outputs' 106 | ) 107 | shell: Rscript {0} 108 | 109 | - name: "Upload PR" 110 | uses: actions/upload-artifact@v4 111 | with: 112 | name: pr 113 | path: ${{ env.PR }} 114 | overwrite: true 115 | 116 | - name: "Upload Diff" 117 | uses: actions/upload-artifact@v4 118 | with: 119 | name: diff 120 | path: ${{ env.CHIVE }} 121 | retention-days: 1 122 | 123 | - name: "Upload Build" 124 | uses: actions/upload-artifact@v4 125 | with: 126 | name: built 127 | path: ${{ env.MD }} 128 | retention-days: 1 129 | 130 | - name: "Teardown" 131 | run: sandpaper::reset_site() 132 | shell: Rscript {0} 133 | -------------------------------------------------------------------------------- /.github/workflows/sandpaper-main.yaml: -------------------------------------------------------------------------------- 1 | name: "01 Build and Deploy Site" 2 | 3 | on: 4 | push: 5 | branches: 6 | - main 7 | - master 8 | schedule: 9 | - cron: '0 0 * * 2' 10 | workflow_dispatch: 11 | inputs: 12 | name: 13 | description: 'Who triggered this build?' 14 | required: true 15 | default: 'Maintainer (via GitHub)' 16 | reset: 17 | description: 'Reset cached markdown files' 18 | required: false 19 | default: false 20 | type: boolean 21 | jobs: 22 | full-build: 23 | name: "Build Full Site" 24 | 25 | # 2024-10-01: ubuntu-latest is now 24.04 and R is not installed by default in the runner image 26 | # pin to 22.04 for now 27 | runs-on: ubuntu-22.04 28 | permissions: 29 | checks: write 30 | contents: write 31 | pages: write 32 | env: 33 | GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }} 34 | RENV_PATHS_ROOT: ~/.local/share/renv/ 35 | steps: 36 | 37 | - name: "Checkout Lesson" 38 | uses: actions/checkout@v4 39 | 40 | - name: "Set up R" 41 | uses: r-lib/actions/setup-r@v2 42 | with: 43 | use-public-rspm: true 44 | install-r: false 45 | 46 | - name: "Set up Pandoc" 47 | uses: r-lib/actions/setup-pandoc@v2 48 | 49 | - name: "Setup Lesson Engine" 50 | uses: carpentries/actions/setup-sandpaper@main 51 | with: 52 | cache-version: ${{ secrets.CACHE_VERSION }} 53 | 54 | - name: "Setup Package Cache" 55 | uses: carpentries/actions/setup-lesson-deps@main 56 | with: 57 | cache-version: ${{ secrets.CACHE_VERSION }} 58 | 59 | - name: "Deploy Site" 60 | run: | 61 | reset <- "${{ github.event.inputs.reset }}" == "true" 62 | sandpaper::package_cache_trigger(TRUE) 63 | sandpaper:::ci_deploy(reset = reset) 64 | shell: Rscript {0} 65 | -------------------------------------------------------------------------------- /.github/workflows/sandpaper-version.txt: -------------------------------------------------------------------------------- 1 | 0.16.12 2 | -------------------------------------------------------------------------------- /.github/workflows/update-cache.yaml: -------------------------------------------------------------------------------- 1 | name: "03 Maintain: Update Package Cache" 2 | 3 | on: 4 | workflow_dispatch: 5 | inputs: 6 | name: 7 | description: 'Who triggered this build (enter github username to tag yourself)?' 8 | required: true 9 | default: 'monthly run' 10 | schedule: 11 | # Run every tuesday 12 | - cron: '0 0 * * 2' 13 | 14 | jobs: 15 | preflight: 16 | name: "Preflight Check" 17 | runs-on: ubuntu-22.04 18 | outputs: 19 | ok: ${{ steps.check.outputs.ok }} 20 | steps: 21 | - id: check 22 | run: | 23 | if [[ ${{ github.event_name }} == 'workflow_dispatch' ]]; then 24 | echo "ok=true" >> $GITHUB_OUTPUT 25 | echo "Running on request" 26 | # using single brackets here to avoid 08 being interpreted as octal 27 | # https://github.com/carpentries/sandpaper/issues/250 28 | elif [ `date +%d` -le 7 ]; then 29 | # If the Tuesday lands in the first week of the month, run it 30 | echo "ok=true" >> $GITHUB_OUTPUT 31 | echo "Running on schedule" 32 | else 33 | echo "ok=false" >> $GITHUB_OUTPUT 34 | echo "Not Running Today" 35 | fi 36 | 37 | check_renv: 38 | name: "Check if We Need {renv}" 39 | runs-on: ubuntu-22.04 40 | needs: preflight 41 | if: ${{ needs.preflight.outputs.ok == 'true'}} 42 | outputs: 43 | needed: ${{ steps.renv.outputs.exists }} 44 | steps: 45 | - name: "Checkout Lesson" 46 | uses: actions/checkout@v4 47 | - id: renv 48 | run: | 49 | if [[ -d renv ]]; then 50 | echo "exists=true" >> $GITHUB_OUTPUT 51 | fi 52 | 53 | check_token: 54 | name: "Check SANDPAPER_WORKFLOW token" 55 | runs-on: ubuntu-22.04 56 | needs: check_renv 57 | if: ${{ needs.check_renv.outputs.needed == 'true' }} 58 | outputs: 59 | workflow: ${{ steps.validate.outputs.wf }} 60 | repo: ${{ steps.validate.outputs.repo }} 61 | steps: 62 | - name: "validate token" 63 | id: validate 64 | uses: carpentries/actions/check-valid-credentials@main 65 | with: 66 | token: ${{ secrets.SANDPAPER_WORKFLOW }} 67 | 68 | update_cache: 69 | name: "Update Package Cache" 70 | needs: check_token 71 | if: ${{ needs.check_token.outputs.repo== 'true' }} 72 | runs-on: ubuntu-22.04 73 | env: 74 | GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }} 75 | RENV_PATHS_ROOT: ~/.local/share/renv/ 76 | steps: 77 | 78 | - name: "Checkout Lesson" 79 | uses: actions/checkout@v4 80 | 81 | - name: "Set up R" 82 | uses: r-lib/actions/setup-r@v2 83 | with: 84 | use-public-rspm: true 85 | install-r: false 86 | 87 | - name: "Update {renv} deps and determine if a PR is needed" 88 | id: update 89 | uses: carpentries/actions/update-lockfile@main 90 | with: 91 | cache-version: ${{ secrets.CACHE_VERSION }} 92 | 93 | - name: Create Pull Request 94 | id: cpr 95 | if: ${{ steps.update.outputs.n > 0 }} 96 | uses: carpentries/create-pull-request@main 97 | with: 98 | token: ${{ secrets.SANDPAPER_WORKFLOW }} 99 | delete-branch: true 100 | branch: "update/packages" 101 | commit-message: "[actions] update ${{ steps.update.outputs.n }} packages" 102 | title: "Update ${{ steps.update.outputs.n }} packages" 103 | body: | 104 | :robot: This is an automated build 105 | 106 | This will update ${{ steps.update.outputs.n }} packages in your lesson with the following versions: 107 | 108 | ``` 109 | ${{ steps.update.outputs.report }} 110 | ``` 111 | 112 | :stopwatch: In a few minutes, a comment will appear that will show you how the output has changed based on these updates. 113 | 114 | If you want to inspect these changes locally, you can use the following code to check out a new branch: 115 | 116 | ```bash 117 | git fetch origin update/packages 118 | git checkout update/packages 119 | ``` 120 | 121 | - Auto-generated by [create-pull-request][1] on ${{ steps.update.outputs.date }} 122 | 123 | [1]: https://github.com/carpentries/create-pull-request/tree/main 124 | labels: "type: package cache" 125 | draft: false 126 | -------------------------------------------------------------------------------- /.github/workflows/update-workflows.yaml: -------------------------------------------------------------------------------- 1 | name: "02 Maintain: Update Workflow Files" 2 | 3 | on: 4 | workflow_dispatch: 5 | inputs: 6 | name: 7 | description: 'Who triggered this build (enter github username to tag yourself)?' 8 | required: true 9 | default: 'weekly run' 10 | clean: 11 | description: 'Workflow files/file extensions to clean (no wildcards, enter "" for none)' 12 | required: false 13 | default: '.yaml' 14 | schedule: 15 | # Run every Tuesday 16 | - cron: '0 0 * * 2' 17 | 18 | jobs: 19 | check_token: 20 | name: "Check SANDPAPER_WORKFLOW token" 21 | runs-on: ubuntu-22.04 22 | outputs: 23 | workflow: ${{ steps.validate.outputs.wf }} 24 | repo: ${{ steps.validate.outputs.repo }} 25 | steps: 26 | - name: "validate token" 27 | id: validate 28 | uses: carpentries/actions/check-valid-credentials@main 29 | with: 30 | token: ${{ secrets.SANDPAPER_WORKFLOW }} 31 | 32 | update_workflow: 33 | name: "Update Workflow" 34 | runs-on: ubuntu-22.04 35 | needs: check_token 36 | if: ${{ needs.check_token.outputs.workflow == 'true' }} 37 | steps: 38 | - name: "Checkout Repository" 39 | uses: actions/checkout@v4 40 | 41 | - name: Update Workflows 42 | id: update 43 | uses: carpentries/actions/update-workflows@main 44 | with: 45 | clean: ${{ github.event.inputs.clean }} 46 | 47 | - name: Create Pull Request 48 | id: cpr 49 | if: "${{ steps.update.outputs.new }}" 50 | uses: carpentries/create-pull-request@main 51 | with: 52 | token: ${{ secrets.SANDPAPER_WORKFLOW }} 53 | delete-branch: true 54 | branch: "update/workflows" 55 | commit-message: "[actions] update sandpaper workflow to version ${{ steps.update.outputs.new }}" 56 | title: "Update Workflows to Version ${{ steps.update.outputs.new }}" 57 | body: | 58 | :robot: This is an automated build 59 | 60 | Update Workflows from sandpaper version ${{ steps.update.outputs.old }} -> ${{ steps.update.outputs.new }} 61 | 62 | - Auto-generated by [create-pull-request][1] on ${{ steps.update.outputs.date }} 63 | 64 | [1]: https://github.com/carpentries/create-pull-request/tree/main 65 | labels: "type: template and tools" 66 | draft: false 67 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # sandpaper files 2 | episodes/*html 3 | site/* 4 | !site/README.md 5 | 6 | # History files 7 | .Rhistory 8 | .Rapp.history 9 | # Session Data files 10 | .RData 11 | # User-specific files 12 | .Ruserdata 13 | # Example code in package build process 14 | *-Ex.R 15 | # Output files from R CMD build 16 | /*.tar.gz 17 | # Output files from R CMD check 18 | /*.Rcheck/ 19 | # RStudio files 20 | .Rproj.user/ 21 | # produced vignettes 22 | vignettes/*.html 23 | vignettes/*.pdf 24 | # OAuth2 token, see https://github.com/hadley/httr/releases/tag/v0.3 25 | .httr-oauth 26 | # knitr and R markdown default cache directories 27 | *_cache/ 28 | /cache/ 29 | # Temporary files created by R markdown 30 | *.utf8.md 31 | *.knit.md 32 | # R Environment Variables 33 | .Renviron 34 | # pkgdown site 35 | docs/ 36 | # translation temp files 37 | po/*~ 38 | # renv detritus 39 | renv/sandbox/ 40 | *.pyc 41 | *~ 42 | .DS_Store 43 | .ipynb_checkpoints 44 | .sass-cache 45 | .jekyll-cache/ 46 | .jekyll-metadata 47 | __pycache__ 48 | _site 49 | .Rproj.user 50 | .bundle/ 51 | .vendor/ 52 | vendor/ 53 | .docker-vendor/ 54 | Gemfile.lock 55 | .*history 56 | -------------------------------------------------------------------------------- /AUTHORS: -------------------------------------------------------------------------------- 1 | Muhammad Elhossary, elhossary@zbmed.de 2 | Celia Emmelhainz, emmelhainz@berkeley.edu 3 | Konrad Förstner, foerstner@zbmed.de 4 | Xiaoli Li, xlli@ucdavis.edu 5 | Daniel Mietchen, dm7gn@virginia.edu 6 | Rabea Müller, muellerr@zbmed.de 7 | Mairelys Lemus-Rojas, mlemusrojas@gmail.com 8 | Till Sauerwein, sauerwein@zbmed.de 9 | Philipp Zumstein, zumstein@bib.uni-mannheim.de 10 | Aschraf Aouina 11 | -------------------------------------------------------------------------------- /CITATION: -------------------------------------------------------------------------------- 1 | FIXME: describe how to cite this lesson. -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Contributor Code of Conduct" 3 | --- 4 | 5 | As contributors and maintainers of this project, 6 | we pledge to follow the [The Carpentries Code of Conduct][coc]. 7 | 8 | Instances of abusive, harassing, or otherwise unacceptable behavior 9 | may be reported by following our [reporting guidelines][coc-reporting]. 10 | 11 | 12 | [coc-reporting]: https://docs.carpentries.org/topic_folders/policies/incident-reporting.html 13 | [coc]: https://docs.carpentries.org/topic_folders/policies/code-of-conduct.html 14 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | ## Contributing 2 | 3 | [The Carpentries][cp-site] ([Software Carpentry][swc-site], [Data 4 | Carpentry][dc-site], and [Library Carpentry][lc-site]) are open source 5 | projects, and we welcome contributions of all kinds: new lessons, fixes to 6 | existing material, bug reports, and reviews of proposed changes are all 7 | welcome. 8 | 9 | ### Contributor Agreement 10 | 11 | By contributing, you agree that we may redistribute your work under [our 12 | license](LICENSE.md). In exchange, we will address your issues and/or assess 13 | your change proposal as promptly as we can, and help you become a member of our 14 | community. Everyone involved in [The Carpentries][cp-site] agrees to abide by 15 | our [code of conduct](CODE_OF_CONDUCT.md). 16 | 17 | ### How to Contribute 18 | 19 | The easiest way to get started is to file an issue to tell us about a spelling 20 | mistake, some awkward wording, or a factual error. This is a good way to 21 | introduce yourself and to meet some of our community members. 22 | 23 | 1. If you do not have a [GitHub][github] account, you can [send us comments by 24 | email][contact]. However, we will be able to respond more quickly if you use 25 | one of the other methods described below. 26 | 27 | 2. If you have a [GitHub][github] account, or are willing to [create 28 | one][github-join], but do not know how to use Git, you can report problems 29 | or suggest improvements by [creating an issue][repo-issues]. This allows us 30 | to assign the item to someone and to respond to it in a threaded discussion. 31 | 32 | 3. If you are comfortable with Git, and would like to add or change material, 33 | you can submit a pull request (PR). Instructions for doing this are 34 | [included below](#using-github). For inspiration about changes that need to 35 | be made, check out the [list of open issues][issues] across the Carpentries. 36 | 37 | Note: if you want to build the website locally, please refer to [The Workbench 38 | documentation][template-doc]. 39 | 40 | ### Where to Contribute 41 | 42 | 1. If you wish to change this lesson, add issues and pull requests here. 43 | 2. If you wish to change the template used for workshop websites, please refer 44 | to [The Workbench documentation][template-doc]. 45 | 46 | 47 | ### What to Contribute 48 | 49 | There are many ways to contribute, from writing new exercises and improving 50 | existing ones to updating or filling in the documentation and submitting [bug 51 | reports][issues] about things that do not work, are not clear, or are missing. 52 | If you are looking for ideas, please see [the list of issues for this 53 | repository][repo-issues], or the issues for [Data Carpentry][dc-issues], 54 | [Library Carpentry][lc-issues], and [Software Carpentry][swc-issues] projects. 55 | 56 | Comments on issues and reviews of pull requests are just as welcome: we are 57 | smarter together than we are on our own. **Reviews from novices and newcomers 58 | are particularly valuable**: it's easy for people who have been using these 59 | lessons for a while to forget how impenetrable some of this material can be, so 60 | fresh eyes are always welcome. 61 | 62 | ### What *Not* to Contribute 63 | 64 | Our lessons already contain more material than we can cover in a typical 65 | workshop, so we are usually *not* looking for more concepts or tools to add to 66 | them. As a rule, if you want to introduce a new idea, you must (a) estimate how 67 | long it will take to teach and (b) explain what you would take out to make room 68 | for it. The first encourages contributors to be honest about requirements; the 69 | second, to think hard about priorities. 70 | 71 | We are also not looking for exercises or other material that only run on one 72 | platform. Our workshops typically contain a mixture of Windows, macOS, and 73 | Linux users; in order to be usable, our lessons must run equally well on all 74 | three. 75 | 76 | ### Using GitHub 77 | 78 | If you choose to contribute via GitHub, you may want to look at [How to 79 | Contribute to an Open Source Project on GitHub](https://egghead.io/courses/how-to-contribute-to-an-open-source-project-on-github). In brief, we 80 | use [GitHub flow][github-flow] to manage changes: 81 | 82 | 1. Create a new branch in your desktop copy of this repository for each 83 | significant change. 84 | 2. Commit the change in that branch. 85 | 3. Push that branch to your fork of this repository on GitHub. 86 | 4. Submit a pull request from that branch to the [upstream repository][repo]. 87 | 5. If you receive feedback, make changes on your desktop and push to your 88 | branch on GitHub: the pull request will update automatically. 89 | 90 | NB: The published copy of the lesson is usually in the `main` branch. 91 | 92 | Each lesson has a team of maintainers who review issues and pull requests or 93 | encourage others to do so. The maintainers are community volunteers, and have 94 | final say over what gets merged into the lesson. 95 | 96 | ### Other Resources 97 | 98 | The Carpentries is a global organisation with volunteers and learners all over 99 | the world. We share values of inclusivity and a passion for sharing knowledge, 100 | teaching and learning. There are several ways to connect with The Carpentries 101 | community listed at including via social 102 | media, slack, newsletters, and email lists. You can also [reach us by 103 | email][contact]. 104 | 105 | [repo]: [https://example.com/FIXME](https://github.com/LibraryCarpentry/lc-wikidata) 106 | [repo-issues]: [https://example.com/FIXME/issues](https://github.com/LibraryCarpentry/lc-wikidata/issues) 107 | [contact]: mailto:team@carpentries.org 108 | [cp-site]: https://carpentries.org/ 109 | [dc-issues]: https://github.com/issues?q=user%3Adatacarpentry 110 | [dc-lessons]: https://datacarpentry.org/lessons/ 111 | [dc-site]: https://datacarpentry.org/ 112 | [discuss-list]: https://lists.software-carpentry.org/listinfo/discuss 113 | [github]: https://github.com 114 | [github-flow]: https://guides.github.com/introduction/flow/ 115 | [github-join]: https://github.com/join 116 | [how-contribute]: https://egghead.io/courses/how-to-contribute-to-an-open-source-project-on-github 117 | [issues]: https://carpentries.org/help-wanted-issues/ 118 | [lc-issues]: https://github.com/issues?q=user%3ALibraryCarpentry 119 | [swc-issues]: https://github.com/issues?q=user%3Aswcarpentry 120 | [swc-lessons]: https://software-carpentry.org/lessons/ 121 | [swc-site]: https://software-carpentry.org/ 122 | [lc-site]: https://librarycarpentry.org/ 123 | [template-doc]: https://carpentries.github.io/workbench/ 124 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Licenses" 3 | --- 4 | 5 | ## Instructional Material 6 | 7 | All Carpentries (Software Carpentry, Data Carpentry, and Library Carpentry) 8 | instructional material is made available under the [Creative Commons 9 | Attribution license][cc-by-human]. The following is a human-readable summary of 10 | (and not a substitute for) the [full legal text of the CC BY 4.0 11 | license][cc-by-legal]. 12 | 13 | You are free: 14 | 15 | - to **Share**---copy and redistribute the material in any medium or format 16 | - to **Adapt**---remix, transform, and build upon the material 17 | 18 | for any purpose, even commercially. 19 | 20 | The licensor cannot revoke these freedoms as long as you follow the license 21 | terms. 22 | 23 | Under the following terms: 24 | 25 | - **Attribution**---You must give appropriate credit (mentioning that your work 26 | is derived from work that is Copyright (c) The Carpentries and, where 27 | practical, linking to ), provide a [link to the 28 | license][cc-by-human], and indicate if changes were made. You may do so in 29 | any reasonable manner, but not in any way that suggests the licensor endorses 30 | you or your use. 31 | 32 | - **No additional restrictions**---You may not apply legal terms or 33 | technological measures that legally restrict others from doing anything the 34 | license permits. With the understanding that: 35 | 36 | Notices: 37 | 38 | * You do not have to comply with the license for elements of the material in 39 | the public domain or where your use is permitted by an applicable exception 40 | or limitation. 41 | * No warranties are given. The license may not give you all of the permissions 42 | necessary for your intended use. For example, other rights such as publicity, 43 | privacy, or moral rights may limit how you use the material. 44 | 45 | ## Software 46 | 47 | Except where otherwise noted, the example programs and other software provided 48 | by The Carpentries are made available under the [OSI][osi]-approved [MIT 49 | license][mit-license]. 50 | 51 | Permission is hereby granted, free of charge, to any person obtaining a copy of 52 | this software and associated documentation files (the "Software"), to deal in 53 | the Software without restriction, including without limitation the rights to 54 | use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies 55 | of the Software, and to permit persons to whom the Software is furnished to do 56 | so, subject to the following conditions: 57 | 58 | The above copyright notice and this permission notice shall be included in all 59 | copies or substantial portions of the Software. 60 | 61 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 62 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 63 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 64 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 65 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 66 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 67 | SOFTWARE. 68 | 69 | ## Trademark 70 | 71 | "The Carpentries", "Software Carpentry", "Data Carpentry", and "Library 72 | Carpentry" and their respective logos are registered trademarks of [Community 73 | Initiatives][ci]. 74 | 75 | [cc-by-human]: https://creativecommons.org/licenses/by/4.0/ 76 | [cc-by-legal]: https://creativecommons.org/licenses/by/4.0/legalcode 77 | [mit-license]: https://opensource.org/licenses/mit-license.html 78 | [ci]: https://communityin.org/ 79 | [osi]: https://opensource.org 80 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Maintainers for Library Carpentry: Wikidata 2 | 3 | - [Till Sauerwein](https://github.com/Tillsa) 4 | - [Muhammad Elhossary](https://github.com/elhossary) 5 | - [Konrad Förstner](https://github.com/konrad) 6 | - [Rabea Müller](https://github.com/RabeaMue) 7 | 8 | Lesson Maintainers communication is via the [team site](https://github.com/orgs/LibraryCarpentry/teams/lc-wikidata). 9 | 10 | ## Library Carpentry 11 | 12 | [Library Carpentry](https://librarycarpentry.org) is a software and data skills training programme for people working in library- and information-related roles. It builds on the work of [Software Carpentry](https://software-carpentry.org/) and [Data Carpentry](https://www.datacarpentry.org/). Library Carpentry is an official Lesson Program of [The Carpentries](https://carpentries.org/). 13 | 14 | ## License 15 | 16 | All Software, Data, and Library Carpentry instructional material is made available under the [Creative Commons Attribution 17 | license](https://github.com/LibraryCarpentry/lc-wikidata/blob/gh-pages/LICENSE.md). 18 | 19 | ## Contributing 20 | 21 | There are many ways to discuss and contribute to Library Carpentry lessons. Visit the lesson [discussion page](https://librarycarpentry.org/lc-wikidata/discuss/index.html) to learn more. Also see [Contributing](https://github.com/LibraryCarpentry/lc-wikidata/blob/gh-pages/CONTRIBUTING.md). 22 | 23 | ## Code of Conduct 24 | 25 | All participants should agree to abide by The Carpentries [Code of Conduct](https://docs.carpentries.org/topic_folders/policies/code-of-conduct.html). 26 | 27 | ## Authors 28 | 29 | Library Carpentry is authored and maintained through issues, commits, and pull requests from the community. 30 | 31 | ## Citation 32 | 33 | Cite as: 34 | 35 | Library Carpentry: Wikidata. September 2019. [https://librarycarpentry.org/lc-wikidata](https://librarycarpentry.org/lc-wikidata). 36 | 37 | ## Checking and Previewing the Lesson 38 | 39 | To check and preview a lesson locally, see [http://carpentries.github.io/lesson-example/07-checking/index.html](https://carpentries.github.io/lesson-example/07-checking/index.html). 40 | 41 | 42 | -------------------------------------------------------------------------------- /Topic_list_brainstorming.md: -------------------------------------------------------------------------------- 1 | 1. Concepts foundation 2 | - Structural Databases vs Semantic Data 3 | - Wikidata 4 | - Wikibase 5 | - LOD 6 | - FAIR 7 | 2. Introducing Wikidata visual interface 8 | 3. Brief overview of RDF / SPARQL language 9 | 4. Introducing Simple Wikidata practical SPARQL Queries and Wikidata Query Service. 10 | 5. Advanced Queries 11 | 6. Adding bulk data via Bots / Automated adding of bulk data 12 | 13 | Suggestions for excecises: 14 | - "Plot the amount of female mayors of european capitals over time (1900s till today)" 15 | - [Wikidata curricula/Activities](https://www.wikidata.org/wiki/Wikidata:Wikidata_curricula/Activities) 16 | -------------------------------------------------------------------------------- /config.yaml: -------------------------------------------------------------------------------- 1 | #------------------------------------------------------------ 2 | # Values for this lesson. 3 | #------------------------------------------------------------ 4 | 5 | # Which carpentry is this (swc, dc, lc, or cp)? 6 | # swc: Software Carpentry 7 | # dc: Data Carpentry 8 | # lc: Library Carpentry 9 | # cp: Carpentries (to use for instructor training for instance) 10 | # incubator: The Carpentries Incubator 11 | carpentry: 'lc' 12 | 13 | # Overall title for pages. 14 | title: 'Library Carpentry Wikidata' 15 | 16 | # Date the lesson was created (YYYY-MM-DD, this is empty by default) 17 | created: '2018-09-17' 18 | 19 | # Comma-separated list of keywords for the lesson 20 | keywords: 'software, data, lesson, The Carpentries' 21 | 22 | # Life cycle stage of the lesson 23 | # possible values: pre-alpha, alpha, beta, stable 24 | life_cycle: 'beta' 25 | 26 | # License of the lesson materials (recommended CC-BY 4.0) 27 | license: 'CC-BY 4.0' 28 | 29 | # Link to the source repository for this lesson 30 | source: 'https://github.com/librarycarpentry/lc-wikidata' 31 | 32 | # Default branch of your lesson 33 | branch: 'main' 34 | 35 | # Who to contact if there are any issues 36 | contact: 'team@carpentries.org' 37 | 38 | # Navigation ------------------------------------------------ 39 | # 40 | # Use the following menu items to specify the order of 41 | # individual pages in each dropdown section. Leave blank to 42 | # include all pages in the folder. 43 | # 44 | # Example ------------- 45 | # 46 | # episodes: 47 | # - introduction.md 48 | # - first-steps.md 49 | # 50 | # learners: 51 | # - setup.md 52 | # 53 | # instructors: 54 | # - instructor-notes.md 55 | # 56 | # profiles: 57 | # - one-learner.md 58 | # - another-learner.md 59 | 60 | # Order of episodes in your lesson 61 | episodes: 62 | - 01-introduction.md 63 | - 02-Wikidata_underlying_concepts.md 64 | - 03-intro_to_editing.md 65 | - 04-advanced_editing.md 66 | - 05-intro_to_querying.md 67 | - 06-bulk_actions.md 68 | 69 | # Information for Learners 70 | learners: 71 | 72 | # Information for Instructors 73 | instructors: 74 | 75 | # Learner Profiles 76 | profiles: 77 | 78 | # Customisation --------------------------------------------- 79 | # 80 | # This space below is where custom yaml items (e.g. pinning 81 | # sandpaper and varnish versions) should live 82 | 83 | 84 | url: 'https://librarycarpentry.github.io/lc-wikidata' 85 | analytics: carpentries 86 | lang: en 87 | -------------------------------------------------------------------------------- /episodes/01-introduction.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: What is Wikidata? 3 | teaching: 20 4 | exercises: 10 5 | --- 6 | 7 | ::::::::::::::::::::::::::::::::::::::: objectives 8 | 9 | - Feel comfortable describing Wikidata to colleagues. 10 | - Learn about Wikimedia projects (e.g. Wikipedia, WikiCommons) and Wikidata is related to them. 11 | - Know why linked open data is important in my work as a cataloging or teaching librarian. 12 | - Know able identify components of a Wikidata item page, how Wikidata is organized and how to navigate in it. 13 | 14 | :::::::::::::::::::::::::::::::::::::::::::::::::: 15 | 16 | :::::::::::::::::::::::::::::::::::::::: questions 17 | 18 | - What are Items and Statements? 19 | - How does the Wikidata interface look like? 20 | - How is Wikidata linked to other Wiki projects? 21 | 22 | :::::::::::::::::::::::::::::::::::::::::::::::::: 23 | 24 | ## What is Wikidata? 25 | 26 | Wikidata's [description](https://www.wikidata.org/wiki/Wikidata:Main_Page) explains that "Wikidata is a free and open knowledge base that can be read and edited by both humans and machines." Wikidata functions as the central database for a variety of Wiki projects, including Wikipedia, Wiktionary, and Wikisource, among others. 27 | 28 | Most users will be familiar with Wikpedia, which [describes itself](https://en.wikipedia.org/wiki/Wikipedia:Introduction) as "a free encyclopedia, written collaboratively by the people who use it. It is a special type of website designed to make collaboration easy, called a wiki. Many people are constantly improving Wikipedia, making thousands of changes per hour. All of these changes are recorded in article histories and recent changes." 29 | 30 | Wikidata contains various data types (e.g. text, images, quantities, coordinates, geographic shapes, dates). The data can be queried via a query interface called SPARQL, which we will cover later in this lesson. Data is published under the Creative Commons Public Domain 1.0 license. It can be modified, copied, and distributed without permission. 31 | 32 | Wikidata also contains authority files, bibliographic data, and other content normally managed in library databases. 33 | 34 | Importantly, Wikidata can be [interlinked](https://www.wikidata.org/wiki/Wikidata:Data_access#How_can_I_get_data_out_of_Wikidata?) to other open data sets on the linked data web. 35 | 36 | ## Motivation and "Why should I use Wikidata" 37 | 1. Knowledge Integration and Access: Wikidata offers an open and structured way to interlink various identifiers (like ORCID, GND, or VIAF). This is essential for librarians who manage resources and need to ensure that different systems and databases can communicate with each other seamlessly. 38 | 39 | 2. Authority Control: Librarians often work with authority files like GND or VIAF to ensure consistent naming conventions. Wikidata helps to map and retrieve these identifiers, making cataloging more efficient. 40 | 41 | 3. Real-World Use Cases: 42 | 43 | - **Scholia**: A tool built on top of Wikidata that visualizes scholarly profiles and research outputs, showing the impact of Wikidata in academic and research contexts. Librarians can showcase Scholia as a tangible example of how data in Wikidata is used for research and scholarship. 44 | - **Crosswalks between systems**: Wikidata’s ability to link various identifiers (e.g., connecting ORCID to GND or VIAF) is beneficial for cross-referencing and data cleaning in library management systems. 45 | 4. Global and Collaborative Nature: Wikidata is a collaborative platform where librarians can contribute and maintain data, ensuring that their records stay relevant and up to date within a global information network. 46 | 47 | ## 1\.1 Intro interface 48 | 49 | - Let's try this out in the next section of this lesson and see if we as humans can simply read the data on Wikidata: 50 | 51 | - Explore a Wikidata Item page: 52 | 53 | - Start by going to the [Wikidata Main Page](https://www.wikidata.org/wiki/Wikidata:Main_Page) by typing "www.wikidata.org" into your browser. This is what you should see: 54 | 55 | ![](fig/Wikidata_Main_Page.png){alt='Wikidata\_Main\_Page'} 56 | *Screenshot of [Wikidata Main Page](https://www.wikidata.org/wiki/Wikidata:Main_Page)* 57 | 58 | - Now go to the search bar in the top right corner and enter "british library". This will give you a list with search results. Click the entry that says: "British Library (Q23308) national library of the United Kingdom". Now you should see the british library's item page: 59 | [https://www.wikidata.org/wiki/Q23308](https://www.wikidata.org/wiki/Q23308) 60 | 61 | - Let us explore the item British Library (Q23308). The top part of the item page serves for identifying the item. It has: 62 | 63 | - unique identifier (Q + a number) 64 | - label 65 | - description 66 | - aliases 67 | 68 | - The bottom part is the "statement" section, that adds statements to the item. A statement has: 69 | 70 | - property (P + a number) 71 | - value 72 | - qualifier (optional) 73 | - references (optional) 74 | - is a so called triple which will be explained later 75 | - As you can see a property can have multiple values for one property; for example "member of"; can be further specified by qualifiers (not showen on item British Library). 76 | 77 | - All these new definitions like statements, qualifiers and so on can be confusing. If you are not sure you can check this overview graphic [https://upload.wikimedia.org/wikipedia/commons/a/ae/Datamodel\_in\_Wikidata.svg](https://upload.wikimedia.org/wikipedia/commons/a/ae/Datamodel_in_Wikidata.svg): 78 | 79 | ![](https://upload.wikimedia.org/wikipedia/commons/a/ae/Datamodel_in_Wikidata.svg){alt='Datamodel in Wikidata'} 80 | 81 | - Usually pages can be edited by anyone; click the pen on the upper-right; Q23308 - British Library is semi-protected; don't worry if you made a mistake, you can always go back in history 82 | 83 | - "View history" - more later 84 | - "Log in" and other things for registered users 85 | 86 | - All structured data is under the creative Creative Commons CC0 License: 87 | "The person who associated a work with this deed has dedicated the work to the public domain by waiving all of his or her rights to the work worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law. You can copy, modify, distribute and perform the work, even for commercial purposes, all without asking permission." 88 | from [https://creativecommons.org/publicdomain/zero/1.0/](https://creativecommons.org/publicdomain/zero/1.0/) 89 | 90 | - Further Link describing Wikidata in one page (visual) 91 | 92 | - [https://commons.wikimedia.org/wiki/File:Wikidata-in-brief-1.0.pdf](https://commons.wikimedia.org/wiki/File:Wikidata-in-brief-1.0.pdf) 93 | 94 | ## 1\.2 Play games to open 95 | 96 | - Visit the Wikipedia page of the city you were born in two languages 97 | of you choice (you can choose different language version in the left 98 | side of a Wikipedia page) and look the size of the population. Are 99 | the numbers the same in the different language? Visit the item in 100 | Wikidata. 101 | 102 | ## 1\.3 Wikidata Item Eastereggs 103 | 104 | - [Q1 - Universe](https://www.wikidata.org/wiki/Q1) 105 | - [Q2 - Earth](https://www.wikidata.org/wiki/Q2) 106 | - [Q3 - Life](https://www.wikidata.org/wiki/Q3) 107 | - [Q4 - Death](https://www.wikidata.org/wiki/Q4) 108 | - [Q5 - Human](https://www.wikidata.org/wiki/Q5) 109 | - [Q13 - Fear of the number 13](https://www.wikidata.org/wiki/Q13) 110 | - [Q24 - Jack Bauer](https://www.wikidata.org/wiki/Q24) 111 | - [Q42 - Douglas Adams (The Hitchhiker's Guide to the Galaxy)](https://www.wikidata.org/wiki/Q42) 112 | - [Q404 - HTTP 404](https://www.wikidata.org/wiki/Q404) 113 | - [Q666 - Number of the beast](https://www.wikidata.org/wiki/Q666) 114 | - [Q12345 - Count von Count, Character on Sesame Street](https://www.wikidata.org/wiki/Q12345) 115 | 116 | ## 1\.4 Linking Wikidata to other Wiki resources 117 | 118 | - Link from Wikipedia to Wikidata 119 | - e.g. [https://en.wikipedia.org/wiki/On\_the\_Origin\_of\_Species](https://en.wikipedia.org/wiki/On_the_Origin_of_Species) 120 | - \=> Follow the link "Wikidata item" on the left side under "tools" 121 | - \=> [https://www.wikidata.org/wiki/Q20124](https://www.wikidata.org/wiki/Q20124) 122 | - \=> the Wikipedia article is linked on the Wikidata's item page. You can find it on the right side. 123 | - \=> link to WikiCommons and WikiSource 124 | 125 | :::::::::::::::::::::::::::::::::::::::: keypoints 126 | 127 | - Item 128 | - Statement 129 | 130 | :::::::::::::::::::::::::::::::::::::::::::::::::: 131 | 132 | 133 | -------------------------------------------------------------------------------- /episodes/02-Wikidata_underlying_concepts.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Underlying concepts of Wikidata 3 | teaching: 0 4 | exercises: 0 5 | --- 6 | 7 | ::::::::::::::::::::::::::::::::::::::: objectives 8 | 9 | - Know what a triple is, and relate structure of a Wikidata statement to traditional metadata field structure 10 | - Know how linked data can create more context for patrons/users in library catalogs 11 | - Know how linked data can improve recall in library catalogs? (TODO: Check if we want to address this here). 12 | 13 | :::::::::::::::::::::::::::::::::::::::::::::::::: 14 | 15 | :::::::::::::::::::::::::::::::::::::::: questions 16 | 17 | - What is a RDF triple? 18 | - What are the underlying components of RDF? 19 | 20 | :::::::::::::::::::::::::::::::::::::::::::::::::: 21 | 22 | ## 2\.1 Concepts foundations: ways of storing data. 23 | 24 | There are many types of databases, the most common types are: 25 | 26 | ### 2\.1.1 Relational databases: 27 | 28 | A relational database is a set of formally described related tables from which data can be accessed or reassembled. This model organizes data into one or more tables (or "relations") of columns and rows, with a unique key identifying each row. each table/relation represents one "entity type" and these entities are connected via constrained relationships. This model is fully structured and mostly uses SQL (Structured Query Language) to retrive and manuplate data. 29 | Examples: 30 | 31 | ![](fig/Relational_database_terms.svg.png){alt='relational database'} 32 | 33 | ### 2\.1.2 Graph / Semantic databases 34 | 35 | Semantic web is an extension of the World Wide Web standards, which promote common data formats and exchange protocols on the Web, most fundamentally the Resource Description Framework (RDF) is used to store data. Most RDF fundamentally uses SPARQL (Simple Protocol and Rdf Query Language) to read stored data while relational databases uses SQL (Structured Query Language) to do so. In SQL relational database terms, RDF data can also be considered or viewed as a table with only three columns – the subject column, the predicate column, and the object column. 36 | 37 | ![](fig/Data_Structure_Diagram.jpg){alt='data structure diagram'} 38 | 39 | ## 2\.2 Concepts foundations (RDF and RDF triples) 40 | 41 | - The RDF is a conceptual data model, It is based on the idea of making statements about resources in expressions of the form (subject–predicate–object), known as triples. 42 | 43 | - The subject denotes the resource, and the predicate denotes traits or aspects of the resource, and expresses a relationship between the subject and the object, for example: John-is-a person, John-born in-1980, John-works as-Engineer 44 | 45 | - RDF data are stored on containers known as triplestores. 46 | 47 | - [https://en.wikipedia.org/wiki/Semantic\_triple](https://en.wikipedia.org/wiki/Semantic_triple) 48 | 49 | ![](fig/RDF_subject_predicate_object.jpg){alt='RDF Tripe'} 50 | 51 | ## 2\.3 Underlying components 52 | 53 | - Items 54 | Items represents subjects such Douglas Adams and have identifiers that starts with letter "Q" like: Q42 for Douglas Adams. 55 | Each item must have a name in one or more langauges, optionally have alternative names and descrition. 56 | - Properties 57 | Properties represents attributes of the subject such occupation and have identifiers that starts with letter "P" like: P106 for Occupation. 58 | - Claims 59 | Claims are the triples, which combine the formation of Item and Property and a value such: 60 | Douglas Adams (Q42) - occupation (P106) - comedian. 61 | Note: value can be already stored in wikidata, therefore the bot assigns the Q number of the value instead. 62 | - Statement 63 | A Claim is a part of a statement, a statement also includes: References, Ranks, and Qualifiers. 64 | - References 65 | Used to store the source of the claim, using properties, such stated in, qoute, and etc. 66 | - Ranks 67 | A useful component to mark outdated claims. 68 | - Qualifiers 69 | Qualifiers are besicly properties but on claims rather than items. 70 | 71 | ::::::::::::::::::::::::::::::::::::::: challenge 72 | 73 | ## Is data stored in the RDF triple format part of your work as a librarian? 74 | 75 | Take some time to think about if data stored in the RDF triple format 76 | is part of your work as a librarian. Can you give an example in the format of an RDF triplet? 77 | 78 | ::::::::::::::: solution 79 | 80 | ## Solution 81 | 82 | *TO DO*: PLEASE ADD A REAL LIFE EXAMPLE 83 | 84 | 85 | 86 | ::::::::::::::::::::::::: 87 | 88 | :::::::::::::::::::::::::::::::::::::::::::::::::: 89 | 90 | ::::::::::::::::::::::::::::::::::::::: challenge 91 | 92 | ## Point out one RDF triple on the Wikidata item page of former astronaut Mae Jemison. 93 | 94 | Got to the Wikidata page of Mae Jemison and point out one RDF triple. 95 | An RDF triplet consists of a subject, a predicate and an object. 96 | Can you assign the three corresponding Wikidata terms? 97 | 98 | ::::::::::::::: solution 99 | 100 | ## Solution 101 | 102 | Got to Wikidata and either search for "Mae Jemison" or enter the ID *Q34091*. 103 | In the picture below the statement "Mae C. Jemison - part of - NASA Astronaut Group 12" is an RDF triple. 104 | ![](fig/Mae_Jemison_Wikidata.png){alt='Wikidata\_Main\_Page'} 105 | *Screenshot of [Wikidata Main Page](https://www.wikidata.org/wiki/Q34091)* 106 | 107 | ::::::::::::::::::::::::: 108 | 109 | :::::::::::::::::::::::::::::::::::::::::::::::::: 110 | 111 | ## 2\.4 Scholia - a webserive with Wikidata as underlying database 112 | 113 | - Introduction with [The Linked Open Data Cloud](https://www.lod-cloud.net/) 114 | - the structure enables queries 115 | - reference to DBPedia 116 | - you can build your own web services with Wikidata as database > [Scholia](https://scholia.toolforge.org/) 117 | - e.g. search for Alex Bateman 118 | 119 | ## 2\.5 Wikidata one pager 120 | 121 | - [https://commons.wikimedia.org/wiki/File:Wikidata-in-brief-1.0.pdf](https://commons.wikimedia.org/wiki/File:Wikidata-in-brief-1.0.pdf) 122 | 123 | ## 2\.6 How Wikidata compares with other data sets 124 | 125 | - [https://meta.wikimedia.org/wiki/Wikidata/Notes/DBpedia\_and\_Wikidata](https://meta.wikimedia.org/wiki/Wikidata/Notes/DBpedia_and_Wikidata) 126 | - [https://lod-cloud.net/](https://lod-cloud.net/) 127 | 128 | FIXME 129 | 130 | 131 | 132 | :::::::::::::::::::::::::::::::::::::::: keypoints 133 | 134 | - First key point. (FIXME) 135 | 136 | :::::::::::::::::::::::::::::::::::::::::::::::::: 137 | 138 | 139 | -------------------------------------------------------------------------------- /episodes/03-intro_to_editing.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Introduction to editing 3 | teaching: 0 4 | exercises: 0 5 | --- 6 | 7 | ::::::::::::::::::::::::::::::::::::::: objectives 8 | 9 | - Be able to create and edit a Wikidata entry. 10 | - Know properties and relations, and where to find lists of approved properties and relations. 11 | - Be able to add new statements that link to other items. 12 | - Be aware of property constraints. 13 | - Know community norms around Wikidata and why they are important. 14 | - Be able to add references appropriately. 15 | - Know what identifiers are and how to add them to a statement. 16 | - Know different stable identifiers (e.g. ORCID for authors, DOI for works) and why makes sense to use them as properties. 17 | - Know the correct use of properties. 18 | 19 | :::::::::::::::::::::::::::::::::::::::::::::::::: 20 | 21 | :::::::::::::::::::::::::::::::::::::::: questions 22 | 23 | - How to create and edit a Wikidata item? 24 | 25 | :::::::::::::::::::::::::::::::::::::::::::::::::: 26 | 27 | 28 | 29 | ## 3\.1 Introduction 30 | 31 | Here we will work in the test instance of Wikidata so you will not break anything. Also keep in mind that the editing history is kept in Wikidata so error can also be easily fixed there. The test instance is cleaned regularly. You can quickly figure out if you are on the Wikidata instance (colored logo) or the test version (black-and-white only). 32 | 33 | ## 3\.2 Create new items 34 | 35 | In the following we will create new items. In order to avoid to fill Wikidata with test entries, we will use the test instance ([https://test.wikidata.org/](https://test.wikidata.org/)) and not the official, production version ([https://wikidata.org/](https://wikidata.org/)). 36 | 37 | Go to the test instance at [https://test.wikidata.org/](https://test.wikidata.org/) 38 | 39 | Click "[Create a new Item](https://test.wikidata.org/wiki/Special:NewItem)" link on the left site. You will see a form that looks like this: 40 | 41 | ![](fig/l03-Screenshot_test_instance_front_page.png){alt='Front Page of the test instance'} 42 | 43 | - Please fill the form. You can now add an entry about anything you want like a book, a research article or and author. We will generate an entry of [Mae Jemison](https://en.wikipedia.org/wiki/Mae_Jemison) an American engineer, physician and NASA astronaut. You can also add yourself (if you feel famous enough). We choose "en" int the *Language* drop-down menue, write "Mae Jemison" in the *Label* field, "an American engineer, physician and NASA astronaut." in the *Description* field and "Mae Carol Jemison" in the *Aliases* field. 44 | 45 | ![](fig/l03-Screenshot_test_instance_empty_create_page.png){alt='Empty create form of the test instance'} 46 | 47 | ![](fig/l03-Mae_Jemison_01_Enter_in_form.png){alt='Empty create form of the test instance'} 48 | 49 | - Once we are done we click click "Create". You should see you newly created article. The URL, the adress shown in your web browser, should contain "Q" and some number that is unique for this entry at the end. 50 | 51 | ![](fig/l03-Mae_Jemison_02_Newly_created_page.png){alt='Freshly created Item of Mae Jemison'} 52 | 53 | You can compare the entry that you have generated on the test instance with the current version of the item in Wikidata ([Q34091](https://www.wikidata.org/wiki/Q34091)). 54 | 55 | ![](fig/l03-Mae_Jemison_03_Official_Wikidata_entry_full.png){alt='Wikidata Jemison'} 56 | 57 | ## 3\.3 Add Statements - birth reference 58 | 59 | Why Wikidata uses references: 60 | Like in Wikipedia it is important that content can be verified by others to make sure it is correct and comes from a reliable source of information, such as a book, scientific publication, or newspaper article. 61 | A Reference (or source) is used to point to specific sources that back up a claim in Wikidata. A reference can be a link to a URL or an item; for example, a book. Wikidata does not aim to answer the question of whether a statement is correct, but only whether the statement appears in a reference. 62 | 63 | Task: 64 | 65 | - Support a statement by reference 66 | 67 | - Add the birth date (October 17, 1956) of Mae Jemison as a statement using property P569 "date of birth" to the "Mae X Jemison" item you created above. 68 | - Afterwards add a reference to the satement with the following url as the source: [https://www.biography.com/astronaut/mae-c-jemison](https://www.biography.com/astronaut/mae-c-jemison) 69 | 70 | ## 3\.4 Add Statements - Add ID to Mae Jemison 71 | 72 | Task: 73 | Support a person by it's IDs. Give the participtants the identifiers and source page for an ID and let them add it on the Mae Jemison item on the test instance of Wikidata: 74 | 75 | - VIAF ID 76 | - identifier: 33699121 77 | - source page: [https://viaf.org/viaf/33699121/](https://viaf.org/viaf/33699121/) 78 | - Library of Congress authority ID 79 | - identfier: n95004729 80 | - source page: [http://id.loc.gov/authorities/names/n95004729.html](https://id.loc.gov/authorities/names/n95004729.html) 81 | - IMDb ID 82 | - identifier: nm0420648 83 | - source page: [https://www.imdb.com/name/nm0420648/](https://www.imdb.com/name/nm0420648/) 84 | Site note: 85 | - ORCID is an often used ID, in this case Mae Jemison doesn't have one, but it's good to mention ORCID anyway. 86 | 87 | ## 3\.5 Norms and good practices 88 | 89 | - Customization of languages for user interface 90 | - Wikidata "item" vs. "article" vs. "entry" 91 | - Policies for 92 | - [Labels](https://www.wikidata.org/wiki/Special:MyLanguage/Help:Label) 93 | - [Descriptions](https://www.wikidata.org/wiki/Special:MyLanguage/Help:Description) 94 | - Books 95 | - [Data model](https://www.wikidata.org/wiki/Wikidata:WikiProject_Books) 96 | - [Inventaire](https://inventaire.io/welcome) 97 | 98 | :::::::::::::::::::::::::::::::::::::::: keypoints 99 | 100 | - create new items 101 | - add new statements 102 | 103 | :::::::::::::::::::::::::::::::::::::::::::::::::: 104 | 105 | 106 | -------------------------------------------------------------------------------- /episodes/04-advanced_editing.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Advanced editing 3 | teaching: 0 4 | exercises: 0 5 | --- 6 | 7 | ::::::::::::::::::::::::::::::::::::::: objectives 8 | 9 | - Be familiar with some tools for editing, e.g. TABernacle, Wikidata Games, QuickStatements, Source MetaData or Author Disambiguator/Author resolver. 10 | 11 | :::::::::::::::::::::::::::::::::::::::::::::::::: 12 | 13 | :::::::::::::::::::::::::::::::::::::::: questions 14 | 15 | - How to automatically add statements with sourcemd and quickstatements? 16 | 17 | :::::::::::::::::::::::::::::::::::::::::::::::::: 18 | 19 | ## 4\.1 Disclaimer 20 | 21 | The tools are under heavy development and due to that they might change or don't work as expected. If that happens just move on to the next episode. 22 | 23 | ## 4\.2 Introduction 24 | 25 | So now we will work in the productive version. We will use DOI to automatically put an article into Wikidata via sourcemd. If you are familiar in Life Science you can use our example with PubMed for finding DOIs of new article, optional you can choose an journal related to your scientific field. 26 | Sourcemd gets it metadata from [Crossref](https://de.wikipedia.org/wiki/Crossref), also look to [sourcemd:instructions](https://www.wikidata.org/wiki/Wikidata:SourceMD/instructions) 27 | 28 | Potential open access journal: 29 | 30 | - [PLOS One](https://plos.org/) 31 | - [Beilstein Journal of Organic Chemistry (BJOC)](https://www.beilstein-journals.org/bjoc/articles/16/138) 32 | 33 | ## 4\.3 Adding statements via sourcemd and quickstatements 34 | 35 | Go to [pubmed](https://pubmed.ncbi.nlm.nih.gov/), scroll down to "latest literature" and select an article: 36 | ![](fig/episode_04_screenshot_01_latest_articles.png){alt='latest\_articles'} 37 | 38 | Save the DOI, PMID or PMCID of the article: 39 | 40 | ![](fig/episode_04_screenshot_02_choose_doi.png){alt='choose\_doi'} 41 | 42 | Go to [sourcemd](https://sourcemd.toolforge.org/orcidator_old.php) and paste the DOI or PMID into the search field: 43 | 44 | ![](fig/episode_04_screenshot_03_paste_into_sourcemd.png){alt='paste\_into\_sourcemd'} 45 | 46 | Click on "check source". Now you can see automatically generated statements including meta data of the article like author names or date of publication. Click on "Open in QuickStatements". 47 | 48 | ![](fig/episode_04_screenshot_04_open_quickstatements.png){alt='open quickstatements'} 49 | 50 | A new window with QuickStatements will pop up. Now you'll get an overview of the new item and its statements. Confirm the changes by hitting the the "run" button: 51 | 52 | ![](fig/episode_04_screenshot_05_run_editing.png){alt='run\_editing'} 53 | 54 | ## 4\.4 (OPTIONAL) Converting "author strings" to "author" 55 | 56 | [Find Author Strings](https://github.com/foerstner-lab/Publication_list/blob/master/Query_Strings_for_Wikidata.md) 57 | [Author Disambiguator](https://author-disambiguator.toolforge.org/) 58 | 59 | 60 | 61 | :::::::::::::::::::::::::::::::::::::::: keypoints 62 | 63 | - First key point. (FIXME) 64 | 65 | :::::::::::::::::::::::::::::::::::::::::::::::::: 66 | 67 | 68 | -------------------------------------------------------------------------------- /episodes/05-intro_to_querying.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Introduction to querying 3 | teaching: 0 4 | exercises: 0 5 | --- 6 | 7 | ::::::::::::::::::::::::::::::::::::::: objectives 8 | 9 | - Know what a query language is, and how SPARQL differs SQL. 10 | - Be able to use SPARQL to query Wikidata. 11 | - Potentially be able to use a tool like [TABernacle](https://tools.wmflabs.org/tabernacle/#/) to edit based on a query. 12 | - Have a cursory knowledge of the plethora of Wikidata querying tools and how they can be used by librarians. 13 | - Know the purpose and usefulness of maintenance queries for identifying missing information. 14 | - Be able to create maintenance queries. 15 | 16 | :::::::::::::::::::::::::::::::::::::::::::::::::: 17 | 18 | :::::::::::::::::::::::::::::::::::::::: questions 19 | 20 | - What is SPARQL? 21 | - How to use SPARQL to query Wikidata? 22 | - How to use Wikidata querying tools? 23 | 24 | :::::::::::::::::::::::::::::::::::::::::::::::::: 25 | 26 | FIXME 27 | 28 | 29 | 30 | There are different ways to query information in Wikidata. The simplest way 31 | is to search for an entry in Wikidata and looking up all information for that entry, 32 | e.g. [search for Richard Feynmann](https://www.wikidata.org/w/index.php?search=Richard+Feynman). 33 | This search looks by default in the Q-pages as well as the P-pages. 34 | However, we can restrict a search for a property by only looking in the P-pages, 35 | e.g. if we want to look whether there is property for the ISBN we can 36 | [restrict that search to properties only](https://www.wikidata.org/w/index.php?search=isbn&title=Special%3ASearch&profile=advanced&fulltext=1&advancedSearch-current=%7B%22namespaces%22%3A%5B120%5D%7D&ns120=1). 37 | Moreover, for a given entry there is always the possibility to see other pages 38 | which links to that (e.g. using it as an object), e.g. all pages linking 39 | to Richard Feynman: [https://www.wikidata.org/wiki/Special:WhatLinksHere/Q39246](https://www.wikidata.org/wiki/Special:WhatLinksHere/Q39246) 40 | 41 | That is not much different from other searches you may be familiar with. However, the 42 | real potential of Wikidata as a huge knowledge graph, can be experienced through 43 | more advanced querying with the Wikidata query service where the queries have to entered 44 | in SPARQL. 45 | 46 | % To discover Wikidata objects nearby there is the nearby search: 47 | % [https://www.wikidata.org/wiki/Special:Nearby](https://www.wikidata.org/wiki/Special:Nearby) 48 | 49 | ## 5\.1 What is SPARQL? 50 | 51 | SPARQL is a query language for RDF data and is a W3C recommendations since 2008. 52 | The data has to be stored as triples where the object of one triple can be 53 | the subject of another triple. Thus, one can think about a huge knowledge graph, 54 | where the nodes are connected by the predicates with other nodes. For example here 55 | we see all the information about the book "The Meaning of It All" from Wikidata as 56 | a graph: 57 | 58 | ![](fig/Knowledge-Graph-Example-Meaning-of-it-All.png){alt='Example for the knowledge graph spanned by one Wikidata item'} 59 | % source: [http://tinyurl.com/y267yz5q](https://tinyurl.com/y267yz5q) 60 | 61 | However, this is only the graph spanned by one item and its connected entries, 62 | which then itself also have more connections, e.g. we can open some links from 63 | the author Richard Feynman: 64 | 65 | ![](fig/Knowledge-Graph-Example-Meaning-of-it-All-More.png){alt='Example for the knowledge graph spanned by one Wikidata item and more details about Feynman'} 66 | % click on that node in the above query 67 | 68 | For querying data now in this knowledge graph with SPARQL we define some graph patterns 69 | which we want to search. The simplest form is a triple where we replace one of the 70 | components with a variable, which is indicated by a string starting with a question mark: 71 | 72 | - Query for the publisher: `{ wd:Q7750812 wdt:P123 ?publisher . }` 73 | - Query for the connection: `{ wd:Q7750812 ?property wd:Q353060 . }` 74 | - Query for the publications from Addison-Wesley: `{ ?book wdt:P123 wd:Q353060 . }` 75 | 76 | ## 5\.2 Wikidata Query Service 77 | 78 | The Wikidata query service can be found at [https://query.wikidata.org/](https://query.wikidata.org/). There is the 79 | main window on the right to formulate your query in SPARQL. On the left there is the 80 | query helper and at the bottom the result will show up. 81 | 82 | We will only cover here `SELECT`\-statements and start by typing 83 | 84 | ``` 85 | SELECT * WHERE { 86 | 87 | } 88 | ``` 89 | 90 | *Hint* It is enough to start typing "SELECT" and then use the auto-completion with 91 | Ctrl+Space. % TODO what is this for on a Mac? 92 | 93 | Inside the parenthesis you can then place the statements describing the graph pattern 94 | you are looking for. 95 | 96 | ::::::::::::::::::::::::::::::::::::::: challenge 97 | 98 | ## Exercise: Your first SPARQL query 99 | 100 | Write your first SPARQL query for the publisher of the above mentioned book 101 | by copying the part from above point inside a SELECT-statement. 102 | 103 | 104 | :::::::::::::::::::::::::::::::::::::::::::::::::: 105 | 106 | ::::::::::::::: solution 107 | 108 | ## Solution 109 | 110 | ``` 111 | SELECT * WHERE { 112 | wd:Q7750812 wdt:P123 ?publisher . 113 | } 114 | ``` 115 | 116 | ::::::::::::::::::::::::: 117 | 118 | ### Showing labels to Q-numbers 119 | 120 | ### Namespaces and Prefixes 121 | 122 | Prefixes are short abbrevations in the Wikidata Query Service. Some prefixes in Wikidata are: wd, wdt, p, ps, bd, etc. 123 | 124 | Example: 125 | 126 | ``` 127 | SELECT ?item ?itemLabel 128 | WHERE 129 | { 130 | ?item wdt:P50 wd:Q23434. 131 | SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". } 132 | } 133 | ``` 134 | 135 | Items should be prefixed with wd: and properties with wdt: . 136 | 137 | Namespaces in Wikidata are: 138 | 139 | - Main namespace 140 | - Property 141 | - Wikidata: it is for information and discussions about Wikidata itself. 142 | etc. 143 | 144 | ### More conditions 145 | 146 | - publications from Addison-Wesley vs. books from Addison-Wesley vs. books authored by Richard Feynman from Addison-Wesley 147 | - LIMIT 148 | - ORDER 149 | - FILTER 150 | - OPTIONAL 151 | 152 | ### How to visualize your query 153 | 154 | #### Manual visualization in the results window: 155 | 156 | Start by opening the results window. In the results pane, click the "Table" button and choose the type of visualization you want. This way, you can try different ways of visualizing your data without having to change the query code. 157 | 158 | ![](fig/episode_5_table.jpg){alt='Wikidata Query Service results window'} 159 | 160 | 161 | #### Automated visualization with #defaultView: 162 | For an easy start, add the #defaultView: snippet at the beginning of your query. This method ensures that your results will be automatically visualized in a predefined style. This will save you time without having to manually adjust the result window after each query. This method is useful for queries where you already know which visualization types you want to use. 163 | 164 | ![](fig/episode_5_defaultview.png){alt='Extract from the SPARQL editor'} 165 | 166 | ### Exercises 167 | 168 | ## 5\.3 Try examples 169 | 170 | #### Enough theory! it's time to get hands-on. Let's start with a simple example. Literally everyone likes cats, right? So lets search for them in the Wikidata database. 171 | 172 | **Cats example** 173 | 174 | ``` 175 | SELECT ?item ?itemLabel 176 | WHERE 177 | { 178 | ?item wdt:P31 wd:Q146. # Must be of a cat 179 | SERVICE wikibase:label { bd:serviceParam wikibase:language "en,[AUTO_LANGUAGE]". } # Helps get the label in your language, if not, then en language 180 | } 181 | ``` 182 | 183 | Alright! So we found all the cats. But I am pretty sure that I have seen pictures of cats somewhere in Wikipedia or Wikidata. Is there a way to display the images in the Wikidata query service? The simple answer is yes! Let me show you how to do it. 184 | 185 | **Cats pictures** 186 | 187 | In the first step, we searched for cats. It is also possible to search for images in Wikidata, if they are available. The Wikidata query service offers several types of visualization. The image grid is a good way to display images. 188 | 189 | 190 | ``` 191 | #defaultView:ImageGrid 192 | #Normally, the default output is a table, but with the defaultView we can directly specify that the results should be displayed in a grid 193 | 194 | SELECT ?item ?itemLabel ?itemPic 195 | #Show me the item, label and the picture of it 196 | #The result list will look like this (wd:Q123185365/senior cats/ commons:Оредеж, Железнодорожная 9, кот (cropped).jpg) 197 | 198 | WHERE 199 | { 200 | ?item wdt:P31 wd:Q146. #The item of this search is a cat 201 | ?item wdt:P18 ?itemPic. #Show me only cats with pictures. If you want to include very cats in your search, you need to place the Option{} function in front 202 | 203 | SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". } 204 | #Helps to get the label in English. If not, yours will be selected automatically 205 | } 206 | 207 | ``` 208 | ![](fig/episode_5_Imagegrid.jpg){alt='Example of displaying cats in grid format'} 209 | 210 | Wow that was quit a lot of Code, hasnt it. lets break it down so you understand the synatx better. 211 | 212 | Lets move on to another example 213 | 214 | **Worldmap of libraries** 215 | 216 | ``` 217 | #defaultView:Map 218 | #Display the results as a Map 219 | 220 | SELECT distinct * 221 | #Display all available geographic coordinates 222 | 223 | WHERE { 224 | 225 | ?item wdt:P31 wd:Q7075; #Define the item as a library ";"(and) define the geographic coordinates of item as ?geo 226 | wdt:P625 ?geo. 227 | 228 | } 229 | ``` 230 | ![](fig/episode_5_Map_WorldWide.png){alt='Extract from the SPARQL editor'} 231 | 232 | **Map of libraries in the USA** 233 | 234 | ``` 235 | #defaultView:Map 236 | #Display the results as a Map 237 | 238 | 239 | SELECT ?itemLabel ?geo 240 | #Display the label and the geographical coordinates of the item 241 | 242 | WHERE { 243 | 244 | ?item wdt:P31 wd:Q7075. #Define the item as a library 245 | ?item wdt:P625 ?geo. #bind the geographic coordinates as ?geo 246 | ?item wdt:P17 ?country. #bind the country of item as ?country 247 | 248 | FILTER(?country = wd:Q30) #Use Filter to set Country to wd:Q30(U.S.A) 249 | 250 | 251 | SERVICE wikibase:label { bd:serviceParam wikibase:language "en,[AUTO_LANGUAGE]". } 252 | #Helps to get the label in English. If not, yours will be selected automatically 253 | } 254 | ``` 255 | ![](fig/episode_5_Map_USA.png){alt='Extract from the SPARQL editor'} 256 | 257 | 258 | 259 | **Map visualization toolbox for libraries** 260 | 261 | ``` 262 | #defaultView:Map 263 | #Display the results as a Map 264 | 265 | #the main query----------------------------------------------------------------------- 266 | SELECT DISTINCT ?itemLabel ?geo #(?itemLabel AS ?layer) 267 | #Show the label and geographic coordinates of the item 268 | #Remove the „#“ before „(?itemLabel AS ?layer)“ to filter the result list. 269 | 270 | 271 | WHERE { 272 | ?item wdt:P31 wd:Q7075; #Define the item as a library 273 | wdt:P625 ?geo. #Get the Coordinates of the library 274 | 275 | 276 | #Advanced Query Options----------------------------------------------- 277 | #Bind the attributes you need to filter them later. 278 | ?item wdt:P17 ?country. #Bind the country of item to ?country 279 | ?item wdt:P131 ?stateORcity. #Bind the territory of item to ?stateORcity 280 | 281 | 282 | #Filter--------------------------------------------------------------- 283 | #Select only one filter at a time to filter between country, state, and city. 284 | 285 | #FILTER(?country = wd:Q30) #Set ?country to U.S.A(wd:Q30) 286 | #FILTER(?stateORcity = wd:Q1261) #Set ?StateOrCity to Colorado(wd:Q1261) 287 | #FILTER(?stateORcity = wd:Q16554) #Set ?StateOrCity to Denver(wd:Q16554 288 | #-------------------------------------------------------------------- 289 | 290 | SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". } 291 | #Helps to get the label in English. If not, yours will be SELECTed automatically 292 | } 293 | 294 | ``` 295 | ![](fig/episode_05_Map_Toolkit.jpg){alt='Extract from the SPARQL editor'} 296 | 297 | 298 | **Count of libraries per Country** 299 | 300 | ``` 301 | #defaultView:BarChart 302 | #Display the results as a Bar Chart 303 | 304 | SELECT distinct ?country ?countryLabel (COUNT(?item) as ?Count) 305 | #Show me the genre, the genre label, and count the available books as a new label ?bookCount 306 | #The result list will look like this (wd:Q213051/Non-Fiction/252) 307 | 308 | WHERE { 309 | ?item wdt:P31 wd:Q7075; #Define the item as a library ";"(and) define the country of item as ?country 310 | wdt:P17 ?country. 311 | 312 | 313 | SERVICE wikibase:label { bd:serviceParam wikibase:language "en,[AUTO_LANGUAGE]". } 314 | #Helps to get the label in English. If not, yours will be selected automatically 315 | 316 | } 317 | Group by ?geo ?geoLabel 318 | #Aggregate with the group function 319 | 320 | Order by DESC(?Count) 321 | #Order result list by variable ?count in descending order 322 | 323 | LIMIT 10 324 | #Limit the shown results down to 10. 325 | 326 | ``` 327 | 328 | ![](fig/episode_5_Bar_Countries.png){alt='Extract from the SPARQL editor'} 329 | 330 | **Books weight by genre** 331 | Number of available books weighted by genre. 332 | ``` 333 | #defaultView:BubbleChart 334 | #Display the results as a Bubble Chart 335 | 336 | SELECT ?genre ?genreLabel (COUNT(?book) as ?bookCount) 337 | #Show me the genre, the genre label, and count the available books as a new label ?bookCount 338 | #The result list will look like this (wd:Q213051/Non-Fiction/252) 339 | 340 | WHERE 341 | { 342 | ?book wdt:P31 wd:Q571. #Searched item is a book 343 | ?book wdt:P136 ?genre. #Get the attribute genre form item 344 | 345 | SERVICE wikibase:label { bd:serviceParam wikibase:language "en,[AUTO_LANGUAGE]". } 346 | #Helps to get the label in English. If not, yours will be selected automatically 347 | } 348 | 349 | GROUP BY ?genre ?genreLabel 350 | #Aggregate with the group function 351 | 352 | LIMIT 15 353 | #Limit the shown results down to 15. 354 | ``` 355 | 356 | ![](fig/episode_5_Bubble.png){alt='Bubblechart of books weight by genre'} 357 | 358 | 359 | **Map of NFDI Consortia in Germany** 360 | 361 | ``` 362 | #defaultView:Map 363 | #Display the results as a Map 364 | 365 | SELECT DISTINCT ?affiliateLabel ?affiliatepicture ?coordinates ?NFDILabel (?NFDILabel AS ?layer) 366 | #Show me the label, image, and coordinates of the affiliate parties. 367 | #Show me the NFDI label as well, using the NFDILabel as a layer filter. 368 | #The layer filter lets you choose which NFDI to display on the map. 369 | #The result list will look like this (Deutsche Nationalbibliothek/commons:DNB.svg/NFDI4Culture/Point(8.683333333 50.131111111)) 370 | 371 | WHERE 372 | { 373 | ?NFDI wdt:P31 wd:Q98270496. #Searched item is an accepted NFDI 374 | ?NFDI wdt:P1416 ?affiliate. #Get the affiliates of the accepted NFDI. 375 | ?affiliate wdt:P625 ?coordinates. #Get the coordinates of the affiliate parties 376 | 377 | OPTIONAL { ?affiliate wdt:P17 ?country } #Get attribute country if available 378 | OPTIONAL { ?affiliate wdt:P154 ?affiliatepicture } #Get attribute picture if available 379 | 380 | FILTER(?country = wd:Q183) #Use Filter to set country to wd:Q183(Germany) 381 | 382 | 383 | SERVICE wikibase:label { bd:serviceParam wikibase:language "en,[AUTO_LANGUAGE]" . } 384 | #Helps to get the label in English. If not, yours will be selected automatically 385 | } 386 | ``` 387 | ![](fig/episode_05_Map.jpg){alt='Map of NFDI Consortia in Germany'} 388 | 389 | 390 | ![](fig/episode_05_Map_CloseUp.png){alt='Close up look'} 391 | 392 | 393 | **Number of participants in NFDI consortia** 394 | 395 | ``` 396 | #Number of participants in NFDI consortia 397 | #defaultView:BarChart 398 | #Use the bar chart as the visualization type and give me the results immediately in the form of a bar chart. 399 | 400 | SELECT DISTINCT ?NFDIKLabel (COUNT(DISTINCT ?participants ) as ?participantsCount) 401 | #Give me the unique (no double entries) names of the accepted NFDIK consortia. 402 | #Count the participants using the COUNT function and return the number of participants as a new variable ?participantsCount. 403 | 404 | 405 | WHERE 406 | { 407 | ?NFDIK wdt:P31 wd:Q98270496. #Give me all accepted NFDI consortia. 408 | ?NFDIK wdt:P710 ?participants. #Show me the all participants of this NFDI consortium. 409 | 410 | #Attention: Not all consortia are listed here, but only those that have an entry participants in Wikidata. 411 | #Participants can be researchers, research institutions, universities, companies and many more. 412 | 413 | SERVICE wikibase:label { bd:serviceParam wikibase:language "en,[AUTO_LANGUAGE]" . } 414 | #Helps to get the label in English. If not, yours will be selected automatically 415 | } 416 | 417 | GROUP BY ?NFDIKLabel 418 | #Group by NFDIK 419 | 420 | HAVING (?participantsCount > 4) 421 | #Show me only NFDI consortia that have more than four registered participants. 422 | 423 | ``` 424 | 425 | ![](fig/episode_05_Barchart.png){alt='Number of participants in NFDI consortia visualized as bar chart'} 426 | 427 | **NFDI Consortia in Berlin, Germany** 428 | 429 | ``` 430 | #defaultView:Graph 431 | #Use the graph as the visualization type 432 | 433 | 434 | SELECT ?affiliate ?affiliateLabel ?affiliatepicture ?NFDIK ?NFDIKLabel ?NFDIKpicture 435 | #Give me the label and the pictures of the affiliated parties of the accepted consortia of the NFDIK. 436 | #Give me label and pictures of accepted NFDIK consortia. 437 | 438 | 439 | WHERE 440 | { 441 | ?NFDIK wdt:P31 wd:Q98270496. #Get me all accepted NFDI consortia. 442 | ?NFDIK wdt:P1416 ?affiliate. #Get the affiliates of the accepted NFDI. 443 | ?affiliate wdt:P131 ?location. #Provide me with the location of the affiliated parties. 444 | 445 | 446 | FILTER(?location = wd:Q64) #Set the location to Berlin. 447 | 448 | OPTIONAL { ?affiliate wdt:P154 ?affiliatepicture } #Give me the pictures of the affilated partner, if available. 449 | OPTIONAL { ?NFDIK wdt:P154 ?NFDIKpicture } #Give me the pictures of the NFDI, if available. 450 | 451 | SERVICE wikibase:label { bd:serviceParam wikibase:language "en,[AUTO_LANGUAGE]" . } 452 | #Helps to get the label in English. If not, yours will be selected automatically 453 | 454 | } 455 | 456 | 457 | ``` 458 | 459 | ![](fig/episode_5_Graph.png){alt='relationships between NFDI Consortia in Berlin, Germany'} 460 | 461 | 462 | **scholarly articles by Alex Bateman** 463 | 464 | ``` 465 | SELECT ?item ?itemLabel ?journalLabel 466 | WHERE 467 | { 468 | ?item wdt:P31 wd:Q13442814. 469 | ?item wdt:P50 wd:Q18921408. 470 | ?item wdt:P1433 ?journal. 471 | SERVICE wikibase:label { bd:serviceParam wikibase:language "en,[AUTO_LANGUAGE]". } 472 | } 473 | ``` 474 | 475 | **Russian poets** 476 | 477 | ``` 478 | SELECT ?item ?itemLabel ?place ?placeLabel ?coord 479 | WHERE 480 | { 481 | ?item wdt:P31 wd:Q5. 482 | ?item wdt:P106 wd:Q49757. 483 | ?item wdt:P19 ?place. 484 | ?place wdt:P17 wd:Q159. 485 | ?place wdt:P625 ?coord 486 | 487 | SERVICE wikibase:label { bd:serviceParam wikibase:language "en,[AUTO_LANGUAGE]". } 488 | } 489 | ``` 490 | 491 | **chemicals example** 492 | 493 | ``` 494 | SELECT ?item ?itemLabel WHERE { 495 | 496 | ?item wdt:P31 wd:Q11173, wd:Q12140. 497 | 498 | SERVICE wikibase:label { bd:serviceParam wikibase:language "en,[AUTO_LANGUAGE]". } 499 | } 500 | ``` 501 | 502 | ``` 503 | SELECT ?item ?itemLabel ?struc ?formula 504 | 505 | WHERE { 506 | 507 | ?item wdt:P31 wd:Q11173, wd:Q12140. 508 | ?item wdt:P117 ?struc. 509 | ?item wdt:P274 ?formula 510 | SERVICE wikibase:label { bd:serviceParam wikibase:language "en,[AUTO_LANGUAGE]". } 511 | 512 | } 513 | ``` 514 | 515 | ``` 516 | SELECT ?item ?itemLabel ?formula ?mass ?struc 517 | 518 | WHERE { 519 | 520 | ?item wdt:P31 wd:Q11173, wd:Q12140. 521 | ?item wdt:P117 ?struc. 522 | ?item wdt:P274 ?formula. 523 | ?item wdt:P2067 ?mass. 524 | SERVICE wikibase:label { bd:serviceParam wikibase:language "en,[AUTO_LANGUAGE]". } 525 | 526 | } 527 | 528 | ORDER BY DESC(?mass) 529 | LIMIT 10 530 | ``` 531 | 532 | **People born in Berlin filtered by year 1970** 533 | 534 | ``` 535 | SELECT ?item ?itemLabel ?dob 536 | WHERE 537 | { 538 | ?item wdt:P31 wd:Q5. 539 | ?item wdt:P19 wd:Q64. 540 | ?item wdt:P569 ?dob. 541 | 542 | FILTER(YEAR(?dob) = 1970) 543 | 544 | SERVICE wikibase:label { bd:serviceParam wikibase:language "en,[AUTO_LANGUAGE]". } 545 | } 546 | ``` 547 | 548 | ## 5\.4 More Advanced queries 549 | 550 | further links 551 | 552 | ``` 553 | https://commons.wikimedia.org/wiki/File:Wikidata_Query_Service_in_Brief.pdf 554 | https://www.uni-mannheim.de/media/Einrichtungen/dws/Files_Teaching/Semantic_Web_Technologies/SWT05-SPARQL-v1.pdf 555 | https://www.wikidata.org/wiki/Wikidata:SPARQL_tutorial 556 | ``` 557 | 558 | :::::::::::::::::::::::::::::::::::::::: keypoints 559 | 560 | - First key point. (FIXME) 561 | 562 | :::::::::::::::::::::::::::::::::::::::::::::::::: 563 | 564 | 565 | -------------------------------------------------------------------------------- /episodes/06-bulk_actions.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Advanced bulk updating, bots 3 | teaching: 0 4 | exercises: 0 5 | --- 6 | 7 | ::::::::::::::::::::::::::::::::::::::: objectives 8 | 9 | - Know how to run a bulk import into Wikidata. 10 | - Be able to create items and/or claims using quickstatements. 11 | - Be familiar with the tools used for bulk edits and imports. 12 | - Be able to articulate how bulk import tools can apply to cataloging and digital science/archive projects. 13 | - Know how to write effective queries in terms of performance. (TODO: may refine or improve or delete) 14 | 15 | :::::::::::::::::::::::::::::::::::::::::::::::::: 16 | 17 | :::::::::::::::::::::::::::::::::::::::: questions 18 | 19 | - How do you bulk upload to Wikidata? 20 | 21 | :::::::::::::::::::::::::::::::::::::::::::::::::: 22 | 23 | FIXME 24 | 25 | 26 | 27 | ## Learning outcomes 28 | 29 | - Understand how to run a bulk import into Wikidata 30 | 31 | - practice using quickstatements? (module 3 already includes QuickStatements) 32 | 33 | - Be familiar with the tools used for bulk edits and imports 34 | 35 | - Tools for bulk upload: - Quick statements ([https://www.wikidata.org/wiki/Help:QuickStatements](https://www.wikidata.org/wiki/Help:QuickStatements)) 36 | 37 | - Connect bulk import possibilities to cataloging and digital science/archive projects? 38 | 39 | - Understand how to write a good queries in terms of performance 40 | 41 | - 42 | ## 6\.1 Bulk uploads/harvests (lead to OpenRefine modules) 43 | 44 | Test test test 45 | 46 | ## 6\.2 Bulk edits 47 | 48 | ## 6\.3 Bulk creation/harvesting 49 | 50 | ## 6\.4 Performance 51 | 52 | - [https://www.wikidata.org/wiki/Wikidata:SPARQL\_query\_service/query\_optimization](https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service/query_optimization) 53 | 54 | :::::::::::::::::::::::::::::::::::::::: keypoints 55 | 56 | - First key point. (FIXME) 57 | 58 | :::::::::::::::::::::::::::::::::::::::::::::::::: 59 | 60 | 61 | -------------------------------------------------------------------------------- /episodes/fig/Data_Structure_Diagram.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/Data_Structure_Diagram.jpg -------------------------------------------------------------------------------- /episodes/fig/Knowledge-Graph-Example-Meaning-of-it-All-More.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/Knowledge-Graph-Example-Meaning-of-it-All-More.png -------------------------------------------------------------------------------- /episodes/fig/Knowledge-Graph-Example-Meaning-of-it-All.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/Knowledge-Graph-Example-Meaning-of-it-All.png -------------------------------------------------------------------------------- /episodes/fig/Mae_Jemison_Wikidata.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/Mae_Jemison_Wikidata.png -------------------------------------------------------------------------------- /episodes/fig/RDF_subject_predicate_object.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/RDF_subject_predicate_object.jpg -------------------------------------------------------------------------------- /episodes/fig/RDF_subject_predicate_object.svg: -------------------------------------------------------------------------------- 1 | 2 | 17 | 19 | 27 | 32 | 33 | 41 | 46 | 47 | 53 | 54 | 73 | 75 | 76 | 78 | image/svg+xml 79 | 81 | 82 | 83 | 84 | 85 | 89 | 93 | Subject 104 | 108 | Object 119 | predicate 130 | 131 | 132 | -------------------------------------------------------------------------------- /episodes/fig/Relational_database_terms.svg.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/Relational_database_terms.svg.png -------------------------------------------------------------------------------- /episodes/fig/Wikidata_Item_Q2_Earth.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/Wikidata_Item_Q2_Earth.png -------------------------------------------------------------------------------- /episodes/fig/Wikidata_Main_Page.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/Wikidata_Main_Page.png -------------------------------------------------------------------------------- /episodes/fig/episode_04_screenshot_01_latest_articles.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/episode_04_screenshot_01_latest_articles.png -------------------------------------------------------------------------------- /episodes/fig/episode_04_screenshot_02_choose_doi.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/episode_04_screenshot_02_choose_doi.png -------------------------------------------------------------------------------- /episodes/fig/episode_04_screenshot_03_paste_into_sourcemd.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/episode_04_screenshot_03_paste_into_sourcemd.png -------------------------------------------------------------------------------- /episodes/fig/episode_04_screenshot_04_open_quickstatements.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/episode_04_screenshot_04_open_quickstatements.png -------------------------------------------------------------------------------- /episodes/fig/episode_04_screenshot_05_run_editing.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/episode_04_screenshot_05_run_editing.png -------------------------------------------------------------------------------- /episodes/fig/episode_05_Barchart.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/episode_05_Barchart.png -------------------------------------------------------------------------------- /episodes/fig/episode_05_Bubblechart.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/episode_05_Bubblechart.png -------------------------------------------------------------------------------- /episodes/fig/episode_05_Graph.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/episode_05_Graph.png -------------------------------------------------------------------------------- /episodes/fig/episode_05_Map.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/episode_05_Map.jpg -------------------------------------------------------------------------------- /episodes/fig/episode_05_Map_CloseUp.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/episode_05_Map_CloseUp.png -------------------------------------------------------------------------------- /episodes/fig/episode_05_Map_Toolkit.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/episode_05_Map_Toolkit.jpg -------------------------------------------------------------------------------- /episodes/fig/episode_5_Bar_Countries.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/episode_5_Bar_Countries.png -------------------------------------------------------------------------------- /episodes/fig/episode_5_Bubble.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/episode_5_Bubble.png -------------------------------------------------------------------------------- /episodes/fig/episode_5_Graph.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/episode_5_Graph.png -------------------------------------------------------------------------------- /episodes/fig/episode_5_Imagegrid.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/episode_5_Imagegrid.jpg -------------------------------------------------------------------------------- /episodes/fig/episode_5_Map_USA.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/episode_5_Map_USA.png -------------------------------------------------------------------------------- /episodes/fig/episode_5_Map_WorldWide.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/episode_5_Map_WorldWide.png -------------------------------------------------------------------------------- /episodes/fig/episode_5_Table.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/episode_5_Table.png -------------------------------------------------------------------------------- /episodes/fig/episode_5_defaultview.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/episode_5_defaultview.png -------------------------------------------------------------------------------- /episodes/fig/episode_5_table.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/episode_5_table.jpg -------------------------------------------------------------------------------- /episodes/fig/l03-Mae_Jemison_01_Enter_in_form.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/l03-Mae_Jemison_01_Enter_in_form.png -------------------------------------------------------------------------------- /episodes/fig/l03-Mae_Jemison_02_Newly_created_page.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/l03-Mae_Jemison_02_Newly_created_page.png -------------------------------------------------------------------------------- /episodes/fig/l03-Mae_Jemison_03_Official_Wikidata_entry_full.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/l03-Mae_Jemison_03_Official_Wikidata_entry_full.png -------------------------------------------------------------------------------- /episodes/fig/l03-Mae_Jemison_04_Enter_in_form_book.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/l03-Mae_Jemison_04_Enter_in_form_book.png -------------------------------------------------------------------------------- /episodes/fig/l03-Mae_Jemison_05_Newly_created_book.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/l03-Mae_Jemison_05_Newly_created_book.png -------------------------------------------------------------------------------- /episodes/fig/l03-Mae_Jemison_06_Form_adding_instance_of.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/l03-Mae_Jemison_06_Form_adding_instance_of.png -------------------------------------------------------------------------------- /episodes/fig/l03-Mae_Jemison_07_Added_instance_of.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/l03-Mae_Jemison_07_Added_instance_of.png -------------------------------------------------------------------------------- /episodes/fig/l03-Mae_Jemison_08_Form_adding_publication_date.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/l03-Mae_Jemison_08_Form_adding_publication_date.png -------------------------------------------------------------------------------- /episodes/fig/l03-Mae_Jemison_09_Added_publication_date.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/l03-Mae_Jemison_09_Added_publication_date.png -------------------------------------------------------------------------------- /episodes/fig/l03-Mae_Jemison_10_Added_authors.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/l03-Mae_Jemison_10_Added_authors.png -------------------------------------------------------------------------------- /episodes/fig/l03-Mae_Jemison_10_Form_adding_authors.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/l03-Mae_Jemison_10_Form_adding_authors.png -------------------------------------------------------------------------------- /episodes/fig/l03-Mae_Jemison_11_Hoover_link.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/l03-Mae_Jemison_11_Hoover_link.png -------------------------------------------------------------------------------- /episodes/fig/l03-Screenshot_test_instance_empty_create_page.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/l03-Screenshot_test_instance_empty_create_page.png -------------------------------------------------------------------------------- /episodes/fig/l03-Screenshot_test_instance_front_page.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/l03-Screenshot_test_instance_front_page.png -------------------------------------------------------------------------------- /episodes/fig/obsolete/Screenshot_2019-02-18 Surely You're Joking, Mr Feynman .png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/obsolete/Screenshot_2019-02-18 Surely You're Joking, Mr Feynman .png -------------------------------------------------------------------------------- /episodes/fig/obsolete/Screenshot_Wikidata_Joking.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/obsolete/Screenshot_Wikidata_Joking.png -------------------------------------------------------------------------------- /episodes/fig/obsolete/Screenshot_Wikidata_Richard_Feynman.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/obsolete/Screenshot_Wikidata_Richard_Feynman.png -------------------------------------------------------------------------------- /episodes/fig/obsolete/Screenshot_Wikidata_Richard_Feynman_excerpt.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/obsolete/Screenshot_Wikidata_Richard_Feynman_excerpt.png -------------------------------------------------------------------------------- /episodes/fig/obsolete/Screenshot_test_instance_Joking_add_statements_01.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/obsolete/Screenshot_test_instance_Joking_add_statements_01.png -------------------------------------------------------------------------------- /episodes/fig/obsolete/Screenshot_test_instance_Joking_add_statements_02.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/obsolete/Screenshot_test_instance_Joking_add_statements_02.png -------------------------------------------------------------------------------- /episodes/fig/obsolete/Screenshot_test_instance_Joking_add_statements_03.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/obsolete/Screenshot_test_instance_Joking_add_statements_03.png -------------------------------------------------------------------------------- /episodes/fig/obsolete/Screenshot_test_instance_Joking_add_statements_04.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/obsolete/Screenshot_test_instance_Joking_add_statements_04.png -------------------------------------------------------------------------------- /episodes/fig/obsolete/Screenshot_test_instance_Joking_add_statements_05.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/obsolete/Screenshot_test_instance_Joking_add_statements_05.png -------------------------------------------------------------------------------- /episodes/fig/obsolete/Screenshot_test_instance_Joking_add_statements_06.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/obsolete/Screenshot_test_instance_Joking_add_statements_06.png -------------------------------------------------------------------------------- /episodes/fig/obsolete/Screenshot_test_instance_Joking_add_statements_08.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/obsolete/Screenshot_test_instance_Joking_add_statements_08.png -------------------------------------------------------------------------------- /episodes/fig/obsolete/Screenshot_test_instance_Joking_add_statements_09.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/obsolete/Screenshot_test_instance_Joking_add_statements_09.png -------------------------------------------------------------------------------- /episodes/fig/obsolete/Screenshot_test_instance_Joking_add_statements_10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/obsolete/Screenshot_test_instance_Joking_add_statements_10.png -------------------------------------------------------------------------------- /episodes/fig/obsolete/Screenshot_test_instance_Joking_add_statements_11.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/obsolete/Screenshot_test_instance_Joking_add_statements_11.png -------------------------------------------------------------------------------- /episodes/fig/obsolete/Screenshot_test_instance_Joking_add_statements_12.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/obsolete/Screenshot_test_instance_Joking_add_statements_12.png -------------------------------------------------------------------------------- /episodes/fig/obsolete/Screenshot_test_instance_Joking_fresh.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/obsolete/Screenshot_test_instance_Joking_fresh.png -------------------------------------------------------------------------------- /episodes/fig/obsolete/Screenshot_test_instance_Richard_Feynman_fresh.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/obsolete/Screenshot_test_instance_Richard_Feynman_fresh.png -------------------------------------------------------------------------------- /episodes/fig/obsolete/Screenshot_test_instance_create_Feyman.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/obsolete/Screenshot_test_instance_create_Feyman.png -------------------------------------------------------------------------------- /episodes/fig/obsolete/Screenshot_test_instance_create_Joking.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LibraryCarpentry/lc-wikidata/4bb8c8b5725f7475ea6bfe7fe1f60c09ef7345cf/episodes/fig/obsolete/Screenshot_test_instance_create_Joking.png -------------------------------------------------------------------------------- /index.md: -------------------------------------------------------------------------------- 1 | --- 2 | permalink: index.html 3 | site: sandpaper::sandpaper_site 4 | --- 5 | 6 | This Library Carpentry lesson introduces librarians to Wikidata. 7 | At the conclusion of the lesson you will: know what the Wikidata interface looks like; 8 | know how Wikidata is linked to other Wiki projects; 9 | know the underlying concepts of Wikidata; 10 | create and develop Wikidata items yourself; 11 | add references to Wikidata; 12 | create a search query in Wikidata using the query language SPARQL; 13 | understand how to perform a mass import into Wikidata. 14 | 15 | ::::::::::::::::::::::::::::::::::::::::: callout 16 | 17 | ## Under Design 18 | 19 | **This lesson is currently in its early design stage; 20 | please check the repository issues and pull requests 21 | to see what we have so far. 22 | Contributions are very welcome** 23 | 24 | 25 | :::::::::::::::::::::::::::::::::::::::::::::::::: 26 | 27 | :::::::::::::::::::::::::::::::::::::::::: prereq 28 | 29 | ## Prerequisites 30 | 31 | 1. Learners need a proper internet connection. 32 | 33 | 2. There is no need for pre installations. 34 | 35 | 36 | :::::::::::::::::::::::::::::::::::::::::::::::::: 37 | 38 | 39 | 40 | 41 | -------------------------------------------------------------------------------- /instructors/design.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Lesson Design 3 | --- 4 | 5 | ## Design decision :bulb: 6 | 7 | Sometime there is more than one way to do something. Here are some 8 | reason why we did certain things in a certain manner. 9 | 10 | ## Write what you decide :pencil2: 11 | 12 | In lesson development, many decisions are made within group discussions, sprints or conferences. In order to be able to track these, it is useful to store a meeting\_notes.md in the repo. 13 | This md file documents all discussions that have been held and all decisions that have been made, to make the development easier to understand for newcomers. To make such a meeting\_notes.md file was also one decision at a certain time and now is also tracked! :+1: 14 | 15 | ## Edit first vs. create first 16 | 17 | In order to lower the hurdle for first edit the idea was suggested on 18 | the WikiCite 2019 brain storming that at first a exiting article 19 | should be edited and then later a new one should be created. This 20 | comes with the disadvantage, that either the participants work on 21 | rather different items (e.g. find something via the random item link) 22 | or the instructor has to pick/create for each participant a suiting 23 | item. Due to this we decided to rather start with the creation of new 24 | items in the Wikidata test instance (which needs only 4 fields to be 25 | filled out). 26 | 27 | ## Keep lessons as simple as possible 28 | 29 | ## Pacing and overall schedule 30 | 31 | - Pacing: The modularity of the lesson allows instructors to select the amount of content 32 | appropriate to the needs of the learners. 33 | - Each module has 15 minutes of stretch content which may be optionally included. 34 | - There is an additional 30 minutes built into the startup time to accomodate delays. 35 | 36 | ### Possible paths through the lesson 37 | 38 | #### Only teaching the basics 39 | 40 | - Startup 41 | - Episode 1: What is Wikidata? 42 | - Epidose 2: Underlying concepts of Wikidata 43 | - Episode 3: Introduction into editing 44 | - Episode 5: Introduction to querying 45 | 46 | #### Full day option 47 | 48 | - Startup + What is Wikidata + Underlying concepts + Intro into editing + advanced editing + lunch + intro to querying + advanced bulk updating, bots 49 | 50 | ## Modules and suggested timing 51 | 52 | ### Startup (30 min) 53 | 54 | - 9:00 Late buffer 55 | - 9:15 Welcome 56 | - Code of conduct 57 | - Room logistics 58 | - Install/setup 59 | - Introducing to eachother 60 | 61 | ### Wikidata-Hands-on-Introduction Part 1 (2 hours) 62 | 63 | - 9:45 Episode 1 (30 min) 64 | - What is Wikidata? 65 | - Introduction into interface 66 | - Wikidata Game 67 | - How is Wikidata linked to other Wiki projects? 68 | - 10:15: Episode 2 (30 min) 69 | - Concepts foundations 70 | - RDF triple in work of a librarian 71 | - RDF triple example 72 | - How Wikidata compares with other data sets 73 | - 10:45: Coffee (15 min) 74 | - 11:00: Episode 3 (30 min) 75 | - Intro 76 | - Create a new item 77 | - Add new statements 78 | - Questions 79 | - 11:30: Episode 4 (45 min) 80 | - Add references 81 | - Citation link in Wikidata 82 | - Norms 83 | - Finding stable identifiers 84 | - Use cases for good practice 85 | 86 | ### Wikidata-Hands-on-Introduction Part 2 (1 1/2 hour) 87 | 88 | - 12:15 Episode 5 (45 min) 89 | - What is SPARQL? 90 | - How to query 91 | - Query on example 92 | - 13:00 Episode 6 (45 min) 93 | - Intro 94 | - Bulk upload/harvests 95 | - Bulk edits 96 | - Bulk creation/harvesting 97 | - Performance 98 | 99 | ### Wrap-Up and Feedback (30 min) 100 | 101 | 102 | -------------------------------------------------------------------------------- /instructors/instructor-notes.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Instructor Notes 3 | --- 4 | 5 | ## Editing Wikidata items with learners 6 | 7 | If you want your learners to edit Wikidata items in the real instance, note that participants need to be registrated in Wikidata for four days to be able to edit wikidata items. If they are not registered they can use the [wikidata test instance](https://test.wikidata.org/wiki/Wikidata:Main_Page) 8 | 9 | ## Organize your lesson with your instructors 10 | 11 | If you do the lesson together with other instructors, it can be helpful to divide the individual sections of the lesson among themselves and thus create a schedule. This could look like this: 12 | 13 | | Episode | Time | Content | Method | Teacher | 14 | | -------- | ------ | ------------------------------------------ | ------------- | ------- | 15 | | 1 | 10 min | What is Wikidata? | Vortrag | Konrad | 16 | | 1 | 10 min | Introduction into Interface | Live "Coding" | Rabea | 17 | | 1 | 10 min | Wikidata Game | Spiel | Rabea | 18 | | 1 | 2 min | Linking Wikidata with other Wiki resources | Demonstration | Rabea | 19 | 20 | Here is the template for your schedule: 21 | 22 | | Episode | Time | Content | Method | Teacher | 23 | | -------- | ------ | ------------------------------------------ | ------------- | ------- | 24 | | 1 | 10 min | What is Wikidata | Presentation | | 25 | | 1 | 10 min | Introduction into Interface | Live Coding | | 26 | | 1 | 10 min | Wikidata Game | Game | | 27 | | 1 | 2 min | Linking Wikidata with other Wiki resources | Demonstration | | 28 | | 2 | 5 min | Concepts foundations | Presentation | | 29 | | 2 | 10 min | RDF triple in work of a librarian | Exercise | | 30 | | 2 | 5 min | RDF Triple Mae Jemison | Exercise | | 31 | | 2 | 10 min | How Wikidata compares with other data sets | Presentation | | 32 | | 3 | 2 min | Intro | Presentation | | 33 | | 3 | 10 min | Create a new item | Live Coding | | 34 | | 3 | 10 min | Add new statements | Live Coding | | 35 | | 3 | 10 min | Questions | Discussion | | 36 | | 4 | 2 min | Intro | Presentation | | 37 | | 4 | | Add references | Live Coding | | 38 | | 4 | | Citations link in Wikidata | | | 39 | | 4 | | Norms | Presentation | | 40 | | 4 | | Finding stable identifiers | Live Coding | | 41 | | 4 | | Use cases for good practice | | | 42 | | 5 | 10 min | What is SPARQL? | Presentation | | 43 | | 5 | 10 min | How to query | Live Coding | | 44 | | 5 | 10 min | Query on "The Meaning of It All" | Exercise | | 45 | | 6 | | Intro | Presentation | | 46 | | 6 | | Bulk upload/harvests | | | 47 | | 6 | | Bulk edits | | | 48 | | 6 | | Bulk creation/harvesting | | | 49 | | 6 | | Performance | | | 50 | | Feedback | 20 min | One up one down | Discussion | | 51 | 52 | 53 | -------------------------------------------------------------------------------- /learners/discuss.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Discussion 3 | --- 4 | 5 | There are many ways to discuss Library Carpentry lessons: 6 | 7 | - Join our [Slack organisation](https://slack-invite.carpentries.org/) and #libraries channel. 8 | - Stay in touch with our [Topicbox Group](https://carpentries.topicbox.com/groups/discuss-library-carpentry). 9 | - Follow updates on [Twitter](https://twitter.com/LibCarpentry). 10 | - Make a suggestion or correct an error by [raising an Issue](https://github.com/LibraryCarpentry/lc-open-refine/issues). 11 | -------------------------------------------------------------------------------- /learners/reference.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: 'FIXME' 3 | --- 4 | 5 | ## Glossary 6 | 7 | ## Glossary 8 | 9 | FIXME 10 | 11 | ## References 12 | 13 | - Bianchini, C., \& Sardo, L. (2022). [Wikidata: a new perspective towards universal bibliographic control](https://www.jlis.it/index.php/jlis/article/view/439) 14 | . JLIS.It, 13(1), 291–311. 15 | 16 | 17 | 18 | 19 | -------------------------------------------------------------------------------- /learners/setup.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Setup 3 | --- 4 | 5 | - Proper internet connection required 6 | - A web browser 7 | 8 | 9 | 10 | 11 | -------------------------------------------------------------------------------- /meeting_notes.md: -------------------------------------------------------------------------------- 1 | ## CC@Home: Library Carpentry Wikidata lesson sprint 2 | ## 2020-07-14, 8:00 UTC 3 | 4 | ### Roll call + Check in: 5 | 6 | Niklas zimmer / niklas.zimmer@uct.ac.za / he, him, his / @niklaszimmer 7 | Wangui Mwangi / mwangiwangui25@gmail.com/ she and her/ @essmarie_1 8 | Serah Rono / serah@carpentries.org / she, her, hers / @serahrono 9 | Till Sauerwein / sauerwein@zbmed.de / he / @TillSauerwein 10 | Rabea Müller / mueller@zbmed.de / she/her / @MueRabea 11 | Konrad Förstner / foerstner@zbmed.de, him/her | @konradfoerstner 12 | Chris Erdmann / erdmannc@renci.org, he, him, his / @libcce 13 | 14 | ### Notes: 15 | Episodes 4 and 6 need to be worked on 16 | Episodes 1,2,3 and 5 are almost done. Proofreading? 17 | 18 | - Add Episode/Set-Up disclaimer that attendees need to make a Wikidata account at least 4(!!) Days in advance. Editing on Wikidata is limited for new user accounts. Caution: Multiple accounts might not be possilbe to be created from the same IP 19 | 20 | ### History: 21 | discussions around this lesson started in november 2018 in Berkeley at a Wikicite conference 22 | the lesson was then set up, but active work started once Rabea came on board 23 | it is now in a 'ready to use' state 24 | 25 | ### Quick overview 26 | Human and machine readable knowledge base 27 | coming from Wikimedia universe, all content is open (CC-BY, CC-0 license) that is editable 28 | editing and reuse is relatively easy for all, by design 29 | unmaintained versions of wikipedia, i.e. those run by smaller communities (languages not used by many), ---- (ask Konrad to expound here) 30 | 31 | ### TODO suggestions: 32 | Niklas 33 | Maps out of copyright, video, audio … unpublished material, possibly with links to published materials or other kinds of ‘public’ ‘products’ 34 | Also as a registry of identifiers for authors across the institution, so we don’t keep duplicating ‘person’ in our various databases 35 | One example where we focus on building academic profiles: Zimmer, Niklas; Muftic, Sanjin; Slingsby, Thomas (2020): Setting up and linking your academic profiles online. University of Cape Town. Presentation. https://doi.org/10.25375/uct.9725393.v1 36 | - add motivation and why to use wikidata as an librarian in episode 37 | - add registration for participants two weeks earlier 38 | - WikiProject: Women in Redhttps://blog.wikimedia.org/2018/03/29/increasing-visibility-women-with-wikidata/ 39 | - add as game later on (episode 5, after introduction into querying) 40 | -source for getting DOIS 41 | - find your own preferred example or you use or example 42 | - PLOS ONE as example 43 | - using test instance for editing Mae Jemison 44 | - use the initials of the participants to keep them different 45 | - and then also use for reference example 46 | - give link to the participants and tell them to add birth with reference 47 | - https://www.britannica.com/biography/Mae-Jemison for searching on reference 48 | # **2019-04-01 Library Carpentry - Wikidata Lesson Sprint** 49 | 50 | * **Aim of the Sprint:** 51 | * Further develop the lesson in detail 52 | * The previous work of the **last session** that took place on 2019-03-12 can be found at https://hackmd.io/T9E1Q-QZRdu-pN8ubyi4-w?both#. 53 | The work included developping the LC wikidata lesson GitHub page: https://github.com/LibraryCarpentry/lc-wikidata 54 | 55 | 56 | * **Participants** (please add your name): 57 | * Konrad Förstner (foerstner@zbmed.de) 58 | * Till Sauerwein (sauerwein@zbmed.de) 59 | * Muhammad Elhossary (elhossary@zbmed.de) 60 | * Alexandra Provo (alexandra.provo@nyu.edu) 61 | * Mairelys Lemus-Rojas (mlemusro@iupui.edu) 62 | 63 | * **Data model - Marie Curie** 64 | Spanish - https://commons.wikimedia.org/wiki/File:Datamodel_in_Wikidata_Marie_Curie_es.svg 65 | English - https://commons.wikimedia.org/wiki/File:Datamodel_in_Wikidata_Marie_Curie_en.svg 66 | 67 | # **2019-01-21 Library Carpentry - Wikidata Lesson Sprint** 68 | 69 | * **Participants** (please add your name): 70 | * Konrad Förstner (foerstner@zbmed.de) 71 | * Till Sauerwein (sauerwein@zbmed.de) 72 | * Muhammad Elhossary (elhossary@zbmed.de) 73 | 74 | * We will work separately as small teams on the 6 modules below. The modules were first transcribed at https://www.wikidata.org/wiki/Wikidata:Wikidata_curricula#Wikidata_for_librarians. 75 | 76 | An extended version with learning outcomes can be found further below. 77 | 78 | * Please **assign yourself to one of the 6 modules** below by adding your name after the module headline and if neccesary a link to a zoom/hangouts/skype/* meeting so that others can join your work group. 79 | An extended version with learning outcomes can be found further below. 80 | 1. What is Wikidata? 81 | 2. Structure/Conceptual foundation/Mechanics 82 | 3. Intro to editing 83 | 4. Creating new items 84 | 5. Intro to querying 85 | 6. Advanced 86 | 87 | ### Extended version with learning outcomes 88 | (taken from https://etherpad.wikimedia.org/p/WikiCite18Day3Tamalpais3B): 89 | 90 | # Todo: 91 | * Add participants of the WikiCite session to https://github.com/LibraryCarpentry/lc-wikidata/blob/gh-pages/AUTHORS after asking for their permission (Konrad) 92 | * Describe pro and cons of first editing an existing item vs. creating a new item (Till) 93 | * creating a new item and editing it: 94 | * pros: 95 | * more stable solution 96 | * not overwelming by too much information 97 | * same starting point for everbody 98 | * Instructor does not have to prepare 99 | * cons: 100 | * editing an already existing item: 101 | * cons: 102 | * can be overwhelming if there are already a lot of items 103 | # Current working version 104 | 105 | ## Module 0: Requirements ## 106 | - Proper internet connection 107 | 108 | ## Module 1: What is Wikidata? ## 109 | 110 | ### Learning outcomes ### 111 | * Feel comfortable describing Wikidata to colleagues 112 | * Understand how Wikidata relates to other Wikimedia projects 113 | * Understand why linked open data is important in my work as a cataloging or teaching librarian 114 | 115 | ### 1.1 Intro interface ### 116 | 117 | - Homepage - Screenshot and link to page (Participants should visit the page: https://www.wikidata.org/wiki/Wikidata:Main_Page) 118 | - Earth (Q2) - Screenshot and link to page (Participants should visit the page https://www.wikidata.org/wiki/Q2) 119 | - Description of Earth (Q2) as an example for an entry 120 | - Title - Earth (Q2) 121 | - What does "Q2"? 122 | - What is an item https://www.wikidata.org/wiki/Help:Items 123 | - Has 124 | - unique identifier (Q + a number) 125 | - label 126 | - description 127 | - aliases 128 | - "Statement" section 129 | - What is a statement? 130 | - Has 131 | - property (P + a number) 132 | - value 133 | - qualifier (optional) 134 | - references (optional) 135 | - is a so called triple which will be explained later 136 | - As you can see a property can have multiple statements for one property; for example "Population"; they are specified by "qualifiers" like point in time; an can references e.g. population at the 5 January 2016 has the reference http://www.worldometers.info/world-population/ as source. 137 | - Usually pages can be edited by anyonce; click the pen on the upper-right; Q2 - Earth is semi-protected (as it is very important); don't worry you made a mistake, you can alway go back in history 138 | - "View history" - more later 139 | - "Log in" and other things for registered users 140 | - CC0 141 | - Further Links 142 | - https://commons.wikimedia.org/wiki/File:Wikidata-in-brief-1.0.pdf 143 | 144 | - https://www.wikidata.org/wiki/Wikidata:Introduction#/media/File:Datamodel_in_Wikidata.svg 145 | - 146 | 147 | ### 1.2 Play games to open ### 148 | 149 | - Visit random items: https://www.wikidata.org/wiki/Special:Random 150 | - TODO - more games needed 151 | 152 | ### 1.3 Relationship with other wiki projects ### 153 | - Link from Wikipedia to Wikidata 154 | - e.g. https://en.wikipedia.org/wiki/On_the_Origin_of_Species 155 | - => Follow the link "Wikidata item" 156 | - => https://www.wikidata.org/wiki/Q20124 157 | - => link to WikiCommons and WikiSource 158 | - Link items to other wikis either added by humans or bot 159 | 160 | ## Module 2: Structure/Conceptual foundation/Mechanics ## 161 | 162 | DISCUSSION: 163 | * There are already clashes with the previous module 164 | * Option 1 set focus or on the usage of it for librarians 165 | * Option 2: merge this completely into Module 1 166 | * Option 3: Put to the end (later module) 167 | 168 | ### Learning outcomes ### 169 | * Identify components of a Wikidata item page (DONE ABOVE) 170 | * Know where to find information on navigating Wikidata (AIM NOT CLEAR) 171 | * Define what's different about open data and why it's important (DIFFERENT TO WHAT?) 172 | * Understand what a triple is, and relate structure of a Wikidata statement to traditional metadata field structure 173 | * How linked data can create more context for patrons/users in library catalogs 174 | * How linked data can improve recall in library catalogs 175 | 176 | 177 | ### 2.1 RDF triples 178 | 179 | - RDF vs. relational database 180 | 181 | - https://en.wikipedia.org/wiki/Semantic_triple 182 | 183 | ### 2.2 Semantic web/structured data 184 | 185 | - https://lod-cloud.net/ 186 | 187 | 188 | ### 2.3 Wikidata one pager 189 | - https://commons.wikimedia.org/wiki/File:Wikidata-in-brief-1.0.pdf 190 | 191 | ### 2.4 How Wikidata compares with other data sets 192 | 193 | ## Module 3: Intro to editing ## 194 | 195 | ### Learning outcomes ### 196 | * Be able to create and edit a Wikidata entry 197 | * Understand properties and relations, and where to find lists of approved properties and relations 198 | * Be able to add new statements that link to other items 199 | * Understand property constraints 200 | 201 | 202 | ### 3.1 Create and edit an new item in the test instance ### 203 | 204 | Here we will work in the test instance of Wikidata so you will not break anything. Also keep in mind that the editing history is kept in Wikidata so error can also be easily fixed there: 205 | 206 | #### 3.1.1 Create a new item #### 207 | 208 | - Goto https://test.wikidata.org/ 209 | - Click "Create a new Item" 210 | - Fill the form 211 | - Language: 212 | - Label 213 | - Description 214 | - Aliases 215 | - Click "Create" 216 | 217 | #### 3.1.2 Add statements #### 218 | 219 | - Data types: 220 | - String 221 | - Properties 222 | - Quantity 223 | - Time 224 | - URL 225 | - And many more https://www.wikidata.org/wiki/Help:Data_type 226 | 227 | - Click add statement 228 | - Fill a property and a values 229 | 230 | - Some suggestions for statements: 231 | - "instance of" (P82) - "book", "scholarly article" 232 | - "date of publication" (P151) 233 | 234 | Have a look again at Origin of Species to get inspiration: https://www.wikidata.org/wiki/Q20124 235 | 236 | ### 3.4 Community norms [examples of how other libraries have used?] ### 237 | 238 | ## Module 4: Advanced editing ## 239 | 240 | ### Learning outcomes ### 241 | 242 | * Use some tools for editing, e.g. TABernacle, Wikidata Games, QuickStatements, Source MetaData or Author Disambiguator/Author resolver 243 | * Understand community norms around WikiData and why they're important 244 | * Be confortable with collaborative, decentralized data creation e.g. carpentry participants should be able to identify gaps in Wikidata that could be filled not just by the participants themselves (and at the event) but by their community more broadly, or specific actors within it 245 | * Add references appropriately 246 | * Be able to compare process of editing to process of editing/adding metadata in existing library information systems (ILS, IR)...? 247 | * Understanding Identifiers 248 | * Connect to how this improves things for our patrons/users... 249 | 250 | ### 4.1 References ### 251 | 252 | - Support a statement by reference 253 | - TODO - find example; Backup solution use the "origin of species example" 254 | 255 | ### 4.2 Citations link in wikidata ### 256 | 257 | Example of a scholorly article with citations: https://www.wikidata.org/wiki/Q52641751 258 | 259 | - some notes regarding the "cites" statement 260 | 261 | ### 4.3 Norms ### 262 | 263 | ### 4.4 Finding stable identifiers (???) ### 264 | 265 | ## Module 5: Introduction to querying ## 266 | 267 | ### Learning outcomes ### 268 | * Understand what a query language is, and how SPARQL differs from, say, SQL 269 | * Be able to use SPARQL to query WikiData 270 | * Potentially be able to use a tool like TABernacle (sp?) to edit based on a query? 271 | * Have a cursory knowledge of the plethora of Wikidata querying tools (and how they can be used by librarians) 272 | * Understand purpose and potential usefulness of maintenance queries for identifying missing information 273 | * Practice creating maintenance queries 274 | 275 | ### 5.1 Wikidata Query Service (SPARQL) ### 276 | 277 | * Query syntax 278 | * What are Prefixes? 279 | 280 | 281 | ### 5.2 Try examples (research published that week?) ### 282 | 283 | 284 | ### further links ### 285 | - https://commons.wikimedia.org/wiki/File:Wikidata_Query_Service_in_Brief.pdf 286 | 287 | ## Module 6: Advanced bulk updating, bots ## 288 | 289 | ### Learning outcomes ### 290 | * Understand how to run a bulk import into Wikidata 291 | * practice using quickstatements? (module 3 already includes QuickStatements) 292 | * Be familiar with the tools used for bulk edits and imports 293 | * Tools for bulk upload: - Quick statements (https://www.wikidata.org/wiki/Help:QuickStatements) 294 | 295 | * Connect bulk import possibilities to cataloging and digital science/archive projects? 296 | 297 | * Understand how to write a good queries in terms of performance 298 | 299 | -------------------------------------------------------------------------------- /profiles/learner-profiles.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: FIXME 3 | --- 4 | 5 | This is a placeholder file. Please add content here. 6 | -------------------------------------------------------------------------------- /site/README.md: -------------------------------------------------------------------------------- 1 | This directory contains rendered lesson materials. Please do not edit files 2 | here. 3 | --------------------------------------------------------------------------------