├── .Rbuildignore ├── .gitattributes ├── .github ├── CONTRIBUTING.md ├── FUNDING.yml └── workflows │ ├── R-CMD-check.yaml │ ├── pkgdown.yaml │ ├── pr-commands.yaml │ └── test-coverage.yaml ├── .gitignore ├── DESCRIPTION ├── LICENSE ├── NAMESPACE ├── NEWS.md ├── R ├── data.R ├── modelStudio.R ├── ms_merge_observations.R ├── ms_options.R ├── ms_update_observations.R ├── ms_update_options.R └── prepare.R ├── README.md ├── codecov.yml ├── data-raw ├── 2015.csv ├── 2016.csv ├── 2017.csv ├── 2018.csv ├── 2019.csv ├── LICENSE.txt └── use_data.R ├── data ├── happiness_test.rda └── happiness_train.rda ├── inst ├── CITATION ├── WORDLIST └── d3js │ ├── d3-interpolate-path.min.js │ ├── d3-simple-slider.min.js │ ├── d3-tip.js │ ├── generatePlots.js │ ├── generateTooltipHtml.js │ ├── hackHead.js │ ├── modelStudio.css │ ├── modelStudio.js │ └── myTools.js ├── man ├── figures │ ├── cheatsheet.png │ ├── controls.png │ ├── demo_big.gif │ ├── demo_small.gif │ ├── logo.gif │ ├── logo.png │ ├── long.gif │ ├── misc.png │ └── short.gif ├── happiness_train.Rd ├── modelStudio.Rd ├── ms_merge_observations.Rd ├── ms_options.Rd ├── ms_update_observations.Rd └── ms_update_options.Rd ├── misc ├── MLinPL2019_modelStudio_poster.pdf ├── paper.bib ├── paper.html └── paper.md ├── pkgdown ├── _pkgdown.yml └── favicon │ ├── CNAME │ ├── apple-touch-icon-120x120.png │ ├── apple-touch-icon-152x152.png │ ├── apple-touch-icon-180x180.png │ ├── apple-touch-icon-60x60.png │ ├── apple-touch-icon-76x76.png │ ├── apple-touch-icon.png │ ├── caret.html │ ├── demo.html │ ├── favicon-16x16.png │ ├── favicon-32x32.png │ ├── favicon.ico │ ├── h2o.html │ ├── keras.html │ ├── lightgbm.html │ ├── mlr.html │ ├── mlr3.html │ ├── parsnip.html │ ├── scikitlearn.html │ ├── tidymodels.html │ └── xgboost.html ├── tests ├── spelling.R ├── testthat.R └── testthat │ ├── test_2_0.R │ ├── test_2_1.R │ ├── test_3_0.R │ ├── test_3_1.R │ ├── test_modelStudio.R │ ├── test_ms_options.R │ ├── test_ms_update_observations.R │ ├── test_ms_update_options.R │ ├── test_objects.R │ └── test_warnings_and_errors.R └── vignettes ├── ms-perks-features.Rmd ├── ms-r-python-examples.Rmd └── ms-rmarkdown.Rmd /.Rbuildignore: -------------------------------------------------------------------------------- 1 | ^.*\.Rproj$ 2 | ^\.Rproj\.user$ 3 | ^arepo.*$ 4 | ^misc.*$ 5 | ^.travis.yml$ 6 | ^cheatsheet$ 7 | ^binder.*$ 8 | ^images.*$ 9 | ^jupyter-notebooks.*$ 10 | ^docs.*$ 11 | ^.*README.*$ 12 | ^_pkgdown\.yml$ 13 | ^codecov\.yml$ 14 | modelStudio_my_test.R 15 | ^CRAN-RELEASE$ 16 | ^cran-comments\.md$ 17 | ^pkgdown.*$ 18 | ^LICENSE$ 19 | ^CONTRIBUTING.md 20 | ^\.github$ 21 | ^.*\.gif 22 | ^.*\.png 23 | ^pkgdown$ 24 | ^\.ipynb_checkpoints$ 25 | [.]pkl$ 26 | [.]pickle$ 27 | [.]ipynb$ 28 | ^data-raw$ 29 | -------------------------------------------------------------------------------- /.gitattributes: -------------------------------------------------------------------------------- 1 | inst/d3js/modelStudio.js linguist-vendored=false 2 | 3 | tests/testthat/test_warnings_and_errors.R linguist-documentation 4 | 5 | docs/* linguist-documentation 6 | man/* linguist-documentation 7 | misc/* linguist-documentation 8 | pkgdown/* linguist-documentation 9 | pkgdown/favicon/*.html linguist-documentation 10 | 11 | **.html linguist-documentation 12 | **.htm linguist-documentation 13 | -------------------------------------------------------------------------------- /.github/CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contribute to modelStudio 2 | 3 | #### **Did you find a bug?** 4 | 5 | Please follow these rules when reporting bugs: 6 | 7 | * Install the latest version of [modelStudio from GitHub](https://github.com/ModelOriented/modelStudio) and check whether the problem still occurs. 8 | * Install the latest version of the main dependencies [DALEX from Github](https://github.com/ModelOriented/DALEX), 9 | [ingredients from GitHub](https://github.com/ModelOriented/ingredients) and 10 | [iBreakDown from GitHub](https://github.com/ModelOriented/iBreakDown). 11 | 12 | * **Check, if the bug was not already reported** by searching through [Issues](https://github.com/ModelOriented/modelStudio/issues). 13 | 14 | * If you're unable to find an open issue addressing the problem, [open a new one](https://github.com/ModelOriented/modelStudio/issues/new). 15 | Be sure to include a **title**, a clear **description** and a **sample code** demonstrating the problem. 16 | 17 | 18 | #### **Did you fix a bug?** 19 | 20 | * Ensure that changes meet the requirements of [The tidyverse style guide](http://style.tidyverse.org) and relevant tests are added. 21 | 22 | * Open a new GitHub pull request with the solution. 23 | 24 | * Ensure that the PR clearly describes the problem and solution. Include the relevant issue number if applicable. 25 | 26 | 27 | #### **Do you intend to add a new feature or change an existing one?** 28 | 29 | * Suggest your change in the [Issues](https://github.com/ModelOriented/modelStudio/issues) and start writing your code :smile:. 30 | 31 | 32 | Thanks! 33 | 34 | Hubert Baniecki and the ModelOriented team 35 | -------------------------------------------------------------------------------- /.github/FUNDING.yml: -------------------------------------------------------------------------------- 1 | # These are supported funding model platforms 2 | 3 | github: hbaniecki 4 | -------------------------------------------------------------------------------- /.github/workflows/R-CMD-check.yaml: -------------------------------------------------------------------------------- 1 | # Workflow derived from https://github.com/r-lib/actions/tree/v2/examples 2 | # Need help debugging build failures? Start at https://github.com/r-lib/actions#where-to-find-help 3 | # 4 | # NOTE: This workflow is overkill for most R packages and 5 | # check-standard.yaml is likely a better choice. 6 | # usethis::use_github_action("check-standard") will install it. 7 | on: 8 | push: 9 | branches: [main, master] 10 | pull_request: 11 | branches: [main, master] 12 | 13 | name: R-CMD-check 14 | 15 | jobs: 16 | R-CMD-check: 17 | runs-on: ${{ matrix.config.os }} 18 | 19 | name: ${{ matrix.config.os }} (${{ matrix.config.r }}) 20 | 21 | strategy: 22 | fail-fast: false 23 | matrix: 24 | config: 25 | - {os: macos-latest, r: 'release'} 26 | 27 | - {os: windows-latest, r: 'release'} 28 | # Use 3.6 to trigger usage of RTools35 29 | - {os: windows-latest, r: '3.6'} 30 | # use 4.1 to check with rtools40's older compiler 31 | - {os: windows-latest, r: '4.1'} 32 | 33 | - {os: ubuntu-latest, r: 'devel', http-user-agent: 'release'} 34 | - {os: ubuntu-latest, r: 'release'} 35 | - {os: ubuntu-latest, r: 'oldrel-1'} 36 | - {os: ubuntu-latest, r: 'oldrel-2'} 37 | # - {os: ubuntu-latest, r: 'oldrel-3'} 38 | # - {os: ubuntu-latest, r: 'oldrel-4'} 39 | 40 | env: 41 | GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }} 42 | R_KEEP_PKG_SOURCE: yes 43 | 44 | steps: 45 | - uses: actions/checkout@v3 46 | 47 | - uses: r-lib/actions/setup-pandoc@v2 48 | 49 | - uses: r-lib/actions/setup-r@v2 50 | with: 51 | r-version: ${{ matrix.config.r }} 52 | http-user-agent: ${{ matrix.config.http-user-agent }} 53 | use-public-rspm: true 54 | 55 | - uses: r-lib/actions/setup-r-dependencies@v2 56 | with: 57 | extra-packages: any::rcmdcheck 58 | needs: check 59 | 60 | - uses: r-lib/actions/check-r-package@v2 61 | with: 62 | upload-snapshots: true 63 | -------------------------------------------------------------------------------- /.github/workflows/pkgdown.yaml: -------------------------------------------------------------------------------- 1 | # Workflow derived from https://github.com/r-lib/actions/tree/v2/examples 2 | # Need help debugging build failures? Start at https://github.com/r-lib/actions#where-to-find-help 3 | on: 4 | push: 5 | branches: [main, master] 6 | pull_request: 7 | branches: [main, master] 8 | release: 9 | types: [published] 10 | workflow_dispatch: 11 | 12 | name: pkgdown 13 | 14 | jobs: 15 | pkgdown: 16 | runs-on: ubuntu-latest 17 | # Only restrict concurrency for non-PR jobs 18 | concurrency: 19 | group: pkgdown-${{ github.event_name != 'pull_request' || github.run_id }} 20 | env: 21 | GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }} 22 | permissions: 23 | contents: write 24 | steps: 25 | - uses: actions/checkout@v3 26 | 27 | - uses: r-lib/actions/setup-pandoc@v2 28 | 29 | - uses: r-lib/actions/setup-r@v2 30 | with: 31 | use-public-rspm: true 32 | 33 | - uses: r-lib/actions/setup-r-dependencies@v2 34 | with: 35 | extra-packages: any::pkgdown, local::. 36 | needs: website 37 | 38 | - name: Install dependencies 39 | run: | 40 | install.packages("remotes") 41 | remotes::install_github("ModelOriented/DrWhyTemplate") 42 | install.packages("textshaping") 43 | shell: Rscript {0} 44 | 45 | - name: Build site 46 | run: pkgdown::build_site_github_pages(new_process = FALSE, install = FALSE) 47 | shell: Rscript {0} 48 | 49 | - name: Deploy to GitHub pages 🚀 50 | if: github.event_name != 'pull_request' 51 | uses: JamesIves/github-pages-deploy-action@v4.4.1 52 | with: 53 | clean: false 54 | branch: gh-pages 55 | folder: docs 56 | -------------------------------------------------------------------------------- /.github/workflows/pr-commands.yaml: -------------------------------------------------------------------------------- 1 | # Workflow derived from https://github.com/r-lib/actions/tree/v2/examples 2 | # Need help debugging build failures? Start at https://github.com/r-lib/actions#where-to-find-help 3 | on: 4 | issue_comment: 5 | types: [created] 6 | 7 | name: Commands 8 | 9 | jobs: 10 | document: 11 | if: ${{ github.event.issue.pull_request && (github.event.comment.author_association == 'MEMBER' || github.event.comment.author_association == 'OWNER') && startsWith(github.event.comment.body, '/document') }} 12 | name: document 13 | runs-on: ubuntu-latest 14 | env: 15 | GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }} 16 | steps: 17 | - uses: actions/checkout@v3 18 | 19 | - uses: r-lib/actions/pr-fetch@v2 20 | with: 21 | repo-token: ${{ secrets.GITHUB_TOKEN }} 22 | 23 | - uses: r-lib/actions/setup-r@v2 24 | with: 25 | use-public-rspm: true 26 | 27 | - uses: r-lib/actions/setup-r-dependencies@v2 28 | with: 29 | extra-packages: any::roxygen2 30 | needs: pr-document 31 | 32 | - name: Document 33 | run: roxygen2::roxygenise() 34 | shell: Rscript {0} 35 | 36 | - name: commit 37 | run: | 38 | git config --local user.name "$GITHUB_ACTOR" 39 | git config --local user.email "$GITHUB_ACTOR@users.noreply.github.com" 40 | git add man/\* NAMESPACE 41 | git commit -m 'Document' 42 | 43 | - uses: r-lib/actions/pr-push@v2 44 | with: 45 | repo-token: ${{ secrets.GITHUB_TOKEN }} 46 | 47 | style: 48 | if: ${{ github.event.issue.pull_request && (github.event.comment.author_association == 'MEMBER' || github.event.comment.author_association == 'OWNER') && startsWith(github.event.comment.body, '/style') }} 49 | name: style 50 | runs-on: ubuntu-latest 51 | env: 52 | GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }} 53 | steps: 54 | - uses: actions/checkout@v3 55 | 56 | - uses: r-lib/actions/pr-fetch@v2 57 | with: 58 | repo-token: ${{ secrets.GITHUB_TOKEN }} 59 | 60 | - uses: r-lib/actions/setup-r@v2 61 | 62 | - name: Install dependencies 63 | run: install.packages("styler") 64 | shell: Rscript {0} 65 | 66 | - name: Style 67 | run: styler::style_pkg() 68 | shell: Rscript {0} 69 | 70 | - name: commit 71 | run: | 72 | git config --local user.name "$GITHUB_ACTOR" 73 | git config --local user.email "$GITHUB_ACTOR@users.noreply.github.com" 74 | git add \*.R 75 | git commit -m 'Style' 76 | 77 | - uses: r-lib/actions/pr-push@v2 78 | with: 79 | repo-token: ${{ secrets.GITHUB_TOKEN }} 80 | -------------------------------------------------------------------------------- /.github/workflows/test-coverage.yaml: -------------------------------------------------------------------------------- 1 | # Workflow derived from https://github.com/r-lib/actions/tree/v2/examples 2 | # Need help debugging build failures? Start at https://github.com/r-lib/actions#where-to-find-help 3 | on: 4 | push: 5 | branches: [main, master] 6 | pull_request: 7 | branches: [main, master] 8 | 9 | name: test-coverage 10 | 11 | jobs: 12 | test-coverage: 13 | runs-on: ubuntu-latest 14 | env: 15 | GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }} 16 | 17 | steps: 18 | - uses: actions/checkout@v3 19 | 20 | - uses: r-lib/actions/setup-r@v2 21 | with: 22 | use-public-rspm: true 23 | 24 | - uses: r-lib/actions/setup-r-dependencies@v2 25 | with: 26 | extra-packages: any::covr 27 | needs: coverage 28 | 29 | - name: Test coverage 30 | run: | 31 | covr::codecov( 32 | quiet = FALSE, 33 | clean = FALSE, 34 | install_path = file.path(Sys.getenv("RUNNER_TEMP"), "package") 35 | ) 36 | shell: Rscript {0} 37 | 38 | - name: Show testthat output 39 | if: always() 40 | run: | 41 | ## -------------------------------------------------------------------- 42 | find ${{ runner.temp }}/package -name 'testthat.Rout*' -exec cat '{}' \; || true 43 | shell: bash 44 | 45 | - name: Upload test results 46 | if: failure() 47 | uses: actions/upload-artifact@v3 48 | with: 49 | name: coverage-test-failures 50 | path: ${{ runner.temp }}/package 51 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | my_tests.R 2 | 3 | # CRAN 4 | CRAN-RELEASE 5 | cran-comments.md 6 | 7 | # History files 8 | .Rhistory 9 | .Rapp.history 10 | 11 | # Session Data files 12 | .RData 13 | 14 | # Example code in package build process 15 | *-Ex.R 16 | 17 | # Output files from R CMD build 18 | /*.tar.gz 19 | 20 | # Output files from R CMD check 21 | /*.Rcheck/ 22 | 23 | # RStudio files 24 | .Rproj.user/ 25 | 26 | # produced vignettes 27 | vignettes/*.html 28 | vignettes/*.pdf 29 | 30 | # OAuth2 token, see https://github.com/hadley/httr/releases/tag/v0.3 31 | .httr-oauth 32 | 33 | # knitr and R markdown default cache directories 34 | /*_cache/ 35 | /cache/ 36 | 37 | # Temporary files created by R markdown 38 | *.utf8.md 39 | *.knit.md 40 | 41 | # Shiny token, see https://shiny.rstudio.com/articles/shinyapps.html 42 | rsconnect/ 43 | 44 | modelStudio.Rproj 45 | #*.html 46 | 47 | docs/* 48 | 49 | *.pickle 50 | 51 | .ipynb_checkpoints 52 | */.ipynb_checkpoints/* 53 | **.ipynb 54 | **.pickle 55 | -------------------------------------------------------------------------------- /DESCRIPTION: -------------------------------------------------------------------------------- 1 | Package: modelStudio 2 | Title: Interactive Studio for Explanatory Model Analysis 3 | Version: 3.1.2.9000 4 | Authors@R: 5 | c(person("Hubert", "Baniecki", role = c("aut", "cre"), 6 | email = "hbaniecki@gmail.com", 7 | comment = c(ORCID = "0000-0001-6661-5364")), 8 | person("Przemyslaw", "Biecek", role = c("aut"), 9 | comment = c(ORCID = "0000-0001-8423-1823")), 10 | person("Piotr", "Piatyszek", role = c("ctb"))) 11 | Description: Automate the explanatory analysis of machine learning predictive 12 | models. Generate advanced interactive model explanations in the form of 13 | a serverless HTML site with only one line of code. This tool is 14 | model-agnostic, therefore compatible with most of the black-box predictive 15 | models and frameworks. The main function computes various (instance and 16 | model-level) explanations and produces a customisable dashboard, which 17 | consists of multiple panels for plots with their short descriptions. It is 18 | possible to easily save the dashboard and share it with others. 'modelStudio' 19 | facilitates the process of Interactive Explanatory Model Analysis introduced 20 | in Baniecki et al. (2023) . 21 | Depends: R (>= 3.6) 22 | License: GPL-3 23 | Encoding: UTF-8 24 | RoxygenNote: 7.1.2 25 | Imports: 26 | DALEX (>= 2.2.1), 27 | ingredients (>= 2.2.0), 28 | iBreakDown (>= 2.0.1), 29 | r2d3, 30 | jsonlite, 31 | progress, 32 | digest 33 | Suggests: 34 | parallelMap, 35 | ranger, 36 | xgboost, 37 | knitr, 38 | rmarkdown, 39 | testthat, 40 | spelling 41 | VignetteBuilder: knitr 42 | URL: https://modelstudio.drwhy.ai, https://github.com/ModelOriented/modelStudio 43 | BugReports: https://github.com/ModelOriented/modelStudio/issues 44 | Language: en-US 45 | LazyData: true 46 | -------------------------------------------------------------------------------- /NAMESPACE: -------------------------------------------------------------------------------- 1 | # Generated by roxygen2: do not edit by hand 2 | 3 | S3method(modelStudio,dalex._explainer.object.Explainer) 4 | S3method(modelStudio,explainer) 5 | S3method(modelStudio,python.builtin.object) 6 | export(modelStudio) 7 | export(ms_merge_observations) 8 | export(ms_options) 9 | export(ms_update_observations) 10 | export(ms_update_options) 11 | import(progress) 12 | importFrom(grDevices,nclass.Sturges) 13 | importFrom(stats,IQR) 14 | importFrom(stats,aggregate) 15 | importFrom(stats,median) 16 | importFrom(stats,na.omit) 17 | importFrom(stats,predict) 18 | importFrom(stats,quantile) 19 | importFrom(utils,head) 20 | importFrom(utils,packageVersion) 21 | importFrom(utils,tail) 22 | -------------------------------------------------------------------------------- /NEWS.md: -------------------------------------------------------------------------------- 1 | # modelStudio (development) 2 | * ... 3 | 4 | # modelStudio 3.1.2 5 | * added new parameter to `modelStudio()`: `open_plots = c("fi")`, which is a vector listing plots to be initially opened (and on which positions) [(#112)](https://github.com/ModelOriented/modelStudio/issues/112) 6 | * fixed future warning with `DALEX::loss_default()` since `DALEX >=2.5.0` 7 | 8 | # modelStudio 3.1.0 9 | * changed y-axis variable labels in `SV` to the same as in `BD` 10 | * added new parameter to `modelStudio()`: `max_features_fi = max_features`, which allows displaying a distinctive number of features in `FI` plot (other than in `BD` and `SV`) 11 | * added new options to `ms_options()`: `**_axis_title`, which allow changing plot-specific axis title (default varies) 12 | 13 | # modelStudio 3.0.0 14 | * **BREAKING CHANGES**: 15 | * this version requires `R >=3.6`, `DALEX >=2.2.1`, `ingredients >=2.2.0` and `iBreakDown >=2.0.1` 16 | * the deprecated alias `modelStudioOptions()` is removed from this version of the package; after being deprecated for over a year since **v1.1.0**. Use the recommended `ms_options()` instead. 17 | * added new parameter to `modelStudio()`: `N_sv = 3*N`, which by default decreases the number of observations used for the calculation of `Shapley Values` (rows in `data`) 18 | * `margin_left = NULL` by default and it is adjusted based on the length of variable names 19 | * the first plot opened in the dashboard is now `FI` instead of `BD` by default 20 | * added the `verbose` parameter to `modelStudio()` as an alias to `show_info` [(#101)](https://github.com/ModelOriented/modelStudio/issues/101) 21 | * added new `ms_merge_observations()` function that merges local explanation of observations from multiple `modelStudio` objects [(#102)](https://github.com/ModelOriented/modelStudio/issues/102) 22 | 23 | # modelStudio 2.1.2 24 | * fixed an error in `modelStudio()` when data had only one variable [(#99)](https://github.com/ModelOriented/modelStudio/issues/99) 25 | 26 | # modelStudio 2.1.1 27 | * fix CRAN checks 28 | 29 | # modelStudio 2.1.0 30 | * **DEFAULTS CHANGES**: if `new_observation = NULL` then choose `new_observation_n = 3` observations, evenly spread by the order of `y_hat`. This shall always include the observations, which ids are `which.min(y_hat)` and `which.max(y_hat)`. Additionally, improve the observation dropdown text in dashboard. [(#94)](https://github.com/ModelOriented/modelStudio/issues/94) 31 | * updated the progress printing 32 | * this version requires `DALEX v2.0.1` 33 | * added new options to `ms_options`: `ms_subtitle`, `ms_margin_top` and `ms_margin_bottom` 34 | * added new parameters to `modelStudio()`: `N_fi = 10*N` and `B_fi = B` 35 | * added new `license` parameter to `modelStudio()` which allows to specify the connection for `readLines()` (e.g. `'LICENSE'`) which will add file contents into the HTML output as a comment 36 | 37 | # modelStudio 2.0.0 38 | * this version requires `DALEX v2.0`, `ingredients v2.0` and `iBreakDown v1.3.1` 39 | * The dashboard gathers useful, but not sensitive, information about how it is being used (e.g. computation length, package version, dashboard dimensions). This is for the development purposes only and can be blocked by setting `telemetry` to `FALSE`. 40 | * add support for `modelStudio` in Shiny [(#77)](https://github.com/ModelOriented/modelStudio/issues/77) 41 | using new `widget_id` argument 42 | * modelStudio now works with `NA` in `data` [(#71)](https://github.com/ModelOriented/modelStudio/issues/71) 43 | * CP, PD and AD plots are now calculated with `variable_splits_type='uniform'` and CP plots are now calculated with `variable_splits_with_obs=TRUE` [(#74)](https://github.com/ModelOriented/modelStudio/issues/74) 44 | * By default the `loss_function` in FI is now different for each `explainer$model_info$type` [(#73)](https://github.com/ModelOriented/modelStudio/issues/73) 45 | * fixed a bug where passing additional parameters in `...` would cause an error 46 | * added a `max_vars` alias for the `max_features` parameter 47 | * added median line to the boxplots in FI and SV plots, added boxplots to TV categorical plots (regression) 48 | * TV plot uses boxplots and barplot when the target `y` has only two unique 49 | values (classification) [(#76)](https://github.com/ModelOriented/modelStudio/issues/76) 50 | * added more checks for input 51 | * added the Residuals vs Feature plot (RV) [(#84)](https://github.com/ModelOriented/modelStudio/issues/84) 52 | * added model performance measures to the footnote [#(85)](https://github.com/ModelOriented/modelStudio/issues/85) 53 | 54 | # modelStudio 1.2.0 55 | * remove redundant documentation resources so that the package weights less 56 | * add a second dropdown list for variable change 57 | * fix check class warning 58 | * add `stringsAsFactors=TRUE` where `data.frame` is used 59 | 60 | # modelStudio 1.1.0 61 | * rename `modelStudioOptions()` to `ms_options()` 62 | * add new `ms_update_options()` function that updates the options of a `modelStudio` object 63 | * add new `ms_update_observations()` function that updates the observations of a `modelStudio` object 64 | * lower `B` default value from `15` to `10` and `N` default value from `400` to `300` 65 | * `feature_importance` is now calculated on `10*N` sampled rows from the data 66 | * use `ranger` instead of `randomForest` everywhere 67 | * remove unnecessary imports, update the documentation 68 | * added `auto_unbox = TRUE` to `jsonlite::toJSON` and changed the `.js` code to comply 69 | * add new class `"modelStudio"` to the `modelStudio()` output 70 | 71 | # modelStudio 1.0.2 72 | * fix `FD` plot on matrix-like data 73 | * center `modelStudio` position in HTML 74 | * `modelStudio` now renders properly from `.Rmd` to `.html` 75 | 76 | # modelStudio 1.0.1 77 | * fix `devel-fedora` tests for cran 78 | 79 | # modelStudio 1.0.0 80 | * stable release after fixing minor issues 81 | * comply with `R v4.0` changes 82 | * add support for matrix-like `data` with `xgboost` working example 83 | * boxplot whiskers end in `max(min, q1 - 1.5*iqr)` and `min(max, q3 + 1.5*iqr)` 84 | * upgrade `show_info` with `progress` package 85 | 86 | # modelStudio 0.3.0 87 | * **`modelStudio()` now only works on `explainer` class object made with `DALEX::explain()`** 88 | * this version requires `iBreakDown v1.0.0` and `ingredients v1.0.0` 89 | * change `cummulative` to `cumulative` in code (#49) 90 | * change `dependency` to `dependence` in code (#52) 91 | * update package title and description 92 | * change LICENSE to GPL-3 (#55) 93 | * add boxplots to `SV` plot (#50) 94 | * add `eda` argument to `modelStudio()` 95 | * add `show_boxplot` argument to `modelStudioOptions()` 96 | 97 | # modelStudio 0.2.1 98 | * fix `TV` plot (X had columns sorted while y was the same) 99 | * add `ms_title` argument to `modelStudioOptions()` (#46) 100 | * `modelStudio` footer is generated faster 101 | 102 | # modelStudio 0.2.0 103 | * new plot: `Target vs Feature [EDA]` (#38) 104 | * new plot: `Average Target vs Feature [EDA]` (#41) 105 | * add boxplots to `FI` plot [ingredients/72](https://github.com/ModelOriented/ingredients/issues/72) 106 | * add `new_observation_y` argument to `modelStudio()` (#39) 107 | * pass `...` to `prepare_*` functions (e.g. allows to round numbers) 108 | * add `margin_ytitle` argument to `modelStudioOptions()` 109 | * by default: observations to calculate local explanations are taken at random from the `data` (#25) 110 | * by default: first plot is selected as `BD` and second is `clicked` (#37) 111 | * nicer histogram when few unique values 112 | * all categorical plots now have the same bar order [ingredients/82](https://github.com/ModelOriented/ingredients/issues/82) 113 | * `try-catch` blocks added - errors in `ingredients` or `iBreakDown` functions are 114 | now treated as warnings and do not stop the `modelStudio` computation (#35) 115 | * `show_info` adds messages saying what is currently calculated (#40) 116 | * add `spellcheck` to tests (#36) 117 | * Travis-CI now checks OSX 118 | 119 | # modelStudio 0.1.9 120 | * this version requires `DALEX v0.4.9` and `ingredients v0.4.0` 121 | 122 | # modelStudio 0.1.8 123 | * lower `B` default value from `25` to `15`, `N` default value from `500` to `400` 124 | 125 | # modelStudio 0.1.7 126 | * fix tests for CRAN 127 | 128 | # modelStudio 0.1.6 129 | Many minor changes stated in #20, most notably: 130 | * rename `x` parameter to `object` in `modelStudio()` 131 | * rename `getOptions()` to `modelStudioOptions()` 132 | * add `viewer` parameter to `modelStudio()` 133 | * add suppressWarnings(ingredients::describe) 134 | * upgrade documentation, examples and vignette 135 | 136 | # modelStudio 0.1.5 137 | * add description to all plots besides AD and FD 138 | * add footer to `modelStudio` 139 | * change `only_numerical` to `variable_type` in `ingredients` functions 140 | 141 | # modelStudio 0.1.4 142 | * add support for parallel computation with `parallelMap` 143 | * more `modelStudio` customization with `options` parameter 144 | * add `getOptions()` function 145 | * remove plot subtitles by default 146 | 147 | # modelStudio 0.1.3 148 | * remove file paths of dependencies from html file 149 | * add animations to line plots 150 | * change .js dependencies to min.js 151 | 152 | # modelStudio 0.1.2 153 | * major .js code refactoring 154 | * add proper exit plot buttons 155 | * import Fira Sans font 156 | * `modelStudio` does not reload on resize 157 | * rewrite d3-tip code, fix placement, add pointer 158 | 159 | # modelStudio 0.1.1 160 | * add demo, cheatsheet, animated instructions, more tests and examples 161 | * `description` won't show if there are less than 4 features in the model 162 | 163 | # modelStudio 0.1.0 164 | * first stable version of the `modelStudio` package 165 | * `modelStudio()` function implemented 166 | -------------------------------------------------------------------------------- /R/data.R: -------------------------------------------------------------------------------- 1 | #' World Happiness Report 2 | #' 3 | #' Datasets \code{happiness_train} and \code{happiness_test} are real data from the 4 | #' World Happiness Reports. Happiness is scored according to economic production, 5 | #' social support, etc. \code{happiness_train} accumulates the data from years 2015-2018, 6 | #' while \code{happiness_test} is the data from the year 2019, which imitates the 7 | #' out-of-time validation. 8 | #' 9 | #' Source: \href{https://www.kaggle.com/unsdsn/world-happiness}{World Happiness Report at Kaggle.com} 10 | #' 11 | #' The following columns: GDP per Capita, Social Support, Life Expectancy, 12 | #' Freedom, Generosity, Corruption describe the extent to which these factors 13 | #' contribute in evaluating the happiness in each country. Variables: 14 | #' 15 | #' \itemize{ 16 | #' \item \strong{score} - target variable, continuous value between 0 and 10 (regression) 17 | #' \item gdp_per_capita 18 | #' \item social_support 19 | #' \item healthy_life_expectancy 20 | #' \item freedom_life_choices 21 | #' \item generosity 22 | #' \item perceptions_of_corruption 23 | #' } 24 | #' 25 | #' @aliases happiness_train happiness_test 26 | #' @docType data 27 | #' @name happiness_train 28 | #' @usage data(happiness_train); data(happiness_test) 29 | #' @format \code{happiness_train}: a data frame with 625 rows and 7 columns, \code{happiness_test}: a data frame with 156 rows and 7 columns 30 | NULL -------------------------------------------------------------------------------- /R/ms_merge_observations.R: -------------------------------------------------------------------------------- 1 | #' @title Merge the observations of modelStudio objects 2 | #' 3 | #' @description 4 | #' This function merges local explanations from multiple \code{modelStudio} objects into one. 5 | #' 6 | #' @param ... \code{modelStudio} objects created with \code{modelStudio()}. 7 | #' 8 | #' @return An object of the \code{r2d3, htmlwidget, modelStudio} class. 9 | #' 10 | #' @references 11 | #' 12 | #' \itemize{ 13 | #' \item The input object is implemented in \href{https://modeloriented.github.io/DALEX/}{\bold{DALEX}} 14 | #' \item Feature Importance, Ceteris Paribus, Partial Dependence and Accumulated Dependence explanations 15 | #' are implemented in \href{https://modeloriented.github.io/ingredients/}{\bold{ingredients}} 16 | #' \item Break Down and Shapley Values explanations are implemented in 17 | #' \href{https://modeloriented.github.io/iBreakDown/}{\bold{iBreakDown}} 18 | #' } 19 | #' 20 | #' @seealso 21 | #' Vignettes: \href{https://modelstudio.drwhy.ai/articles/ms-r-python-examples.html}{\bold{modelStudio - R & Python examples}} 22 | #' and \href{https://modelstudio.drwhy.ai/articles/ms-perks-features.html}{\bold{modelStudio - perks and features}} 23 | #' 24 | #' @examples 25 | #' \donttest{ 26 | #' library("DALEX") 27 | #' library("modelStudio") 28 | #' 29 | #' # fit a model 30 | #' model_happiness <- glm(score ~., data = happiness_train) 31 | #' 32 | #' # create an explainer for the model 33 | #' explainer_happiness <- explain(model_happiness, 34 | #' data = happiness_test, 35 | #' y = happiness_test$score) 36 | #' 37 | #' # make studios for the model 38 | #' ms1 <- modelStudio(explainer_happiness, 39 | #' N = 200, B = 5) 40 | #' 41 | #' ms2 <- modelStudio(explainer_happiness, 42 | #' new_observation = head(happiness_test, 3), 43 | #' N = 200, B = 5) 44 | #' 45 | #' # merge 46 | #' ms <- ms_merge_observations(ms1, ms2) 47 | #' ms 48 | #' } 49 | #' 50 | #' @export 51 | #' @rdname ms_merge_observations 52 | ms_merge_observations <- function(...) { 53 | 54 | #:# extract data 55 | obs_list <- list() 56 | var_list <- list() 57 | dropdown_df <- list() 58 | for (object in list(...)) { 59 | stopifnot("modelStudio" %in% class(object)) 60 | temp <- jsonlite::fromJSON(object$x$data, simplifyVector = FALSE) 61 | obs_list <- c(obs_list, temp[[1]]) 62 | var_list <- c(var_list, object$x$options$variable_names) 63 | dropdown_df <- rbind( 64 | dropdown_df, 65 | jsonlite::fromJSON(object$x$options$drop_down_data, 66 | simplifyVector = FALSE, 67 | simplifyDataFrame = TRUE) 68 | ) 69 | } 70 | 71 | #:# create new data 72 | temp <- jsonlite::toJSON(list(obs_list, temp[[2]], temp[[3]], 73 | temp[[4]], temp[[5]], temp[[6]]), 74 | auto_unbox = TRUE) 75 | widget_id <- paste0("widget-", digest::digest(temp)) 76 | 77 | #:# extract old options and update them 78 | new_options <- object$x$options 79 | new_options$widget_id <- widget_id 80 | new_options$variable_names <- unique(var_list) 81 | new_options$footer_text <- paste0("Site built with modelStudio v", 82 | as.character(packageVersion("modelStudio")), 83 | " on ", 84 | format(Sys.time(), usetz = FALSE)) 85 | new_options$drop_down_data <- jsonlite::toJSON(dropdown_df) 86 | 87 | options("r2d3.shadow" = FALSE) # set this option to avoid using shadow-root 88 | 89 | model_studio <- r2d3::r2d3( 90 | data = temp, 91 | script = system.file("d3js/modelStudio.js", package = "modelStudio"), 92 | dependencies = list( 93 | system.file("d3js/hackHead.js", package = "modelStudio"), 94 | system.file("d3js/myTools.js", package = "modelStudio"), 95 | system.file("d3js/d3-tip.js", package = "modelStudio"), 96 | system.file("d3js/d3-simple-slider.min.js", package = "modelStudio"), 97 | system.file("d3js/d3-interpolate-path.min.js", package = "modelStudio"), 98 | system.file("d3js/generatePlots.js", package = "modelStudio"), 99 | system.file("d3js/generateTooltipHtml.js", package = "modelStudio") 100 | ), 101 | css = system.file("d3js/modelStudio.css", package = "modelStudio"), 102 | options = new_options, 103 | d3_version = "4", 104 | sizing = object$sizingPolicy, 105 | elementId = widget_id, 106 | width = new_options$facet_dim[2]*(new_options$w + new_options$margin_left + new_options$margin_right), 107 | height = 100 + new_options$facet_dim[1]*(new_options$h + new_options$margin_top + new_options$margin_bottom) 108 | ) 109 | 110 | model_studio$x$script <- remove_file_paths(model_studio$x$script, "js") 111 | model_studio$x$style <- remove_file_paths(model_studio$x$style, "css") 112 | 113 | class(model_studio) <- c(class(model_studio), "modelStudio") 114 | 115 | model_studio 116 | } 117 | -------------------------------------------------------------------------------- /R/ms_options.R: -------------------------------------------------------------------------------- 1 | #' @title Modify default options and pass them to modelStudio 2 | #' 3 | #' @description 4 | #' This function returns default options for \code{\link{modelStudio}}. 5 | #' It is possible to modify values of this list and pass it to the \code{options} 6 | #' parameter in the main function. \strong{WARNING: Editing default options may cause 7 | #' unintended behavior.} 8 | #' 9 | #' @param ... Options to change in the form \code{option_name = value}. 10 | #' 11 | #' @return \code{list} of options for \code{modelStudio}. 12 | #' 13 | #' @section Options: 14 | #' \subsection{Main options:}{ 15 | #' \describe{ 16 | #' \item{scale_plot}{\code{TRUE} Makes every plot the same height, ignores \code{bar_width}.} 17 | #' \item{show_boxplot}{\code{TRUE} Display boxplots in Feature Importance and Shapley Values plots.} 18 | #' \item{show_subtitle}{\code{TRUE} Should the subtitle be displayed?} 19 | #' \item{subtitle}{\code{label} parameter from \code{explainer}.} 20 | #' \item{ms_title}{Title of the dashboard.} 21 | #' \item{ms_subtitle}{Subtitle of the dashboard (makes space between the title and line).} 22 | #' \item{ms_margin_*}{Dashboard margins. Change \code{margin_top} for more \code{ms_subtitle} space.} 23 | #' \item{margin_*}{Plot margins. Change \code{margin_left} for longer/shorter axis labels.} 24 | #' \item{w}{\code{420} in px. Inner plot width.} 25 | #' \item{h}{\code{280} in px. Inner plot height.} 26 | #' \item{bar_width}{\code{16} in px. Default width of bars for all plots, 27 | #' ignored when \code{scale_plot = TRUE}.} 28 | #' \item{line_size}{\code{2} in px. Default width of lines for all plots.} 29 | #' \item{point_size}{\code{3} in px. Default point radius for all plots.} 30 | #' \item{[bar,line,point]_color}{\code{[#46bac2,#46bac2,#371ea3]}} 31 | #' \item{positive_color}{\code{#8bdcbe} for Break Down and Shapley Values bars.} 32 | #' \item{negative_color}{\code{#f05a71} for Break Down and Shapley Values bars.} 33 | #' \item{default_color}{\code{#371ea3} for Break Down bar and highlighted line.} 34 | #' } 35 | #' } 36 | #' \subsection{Plot-specific options:}{ 37 | #' \code{**} is a two letter code unique to each plot, might be 38 | #' one of \code{[bd,sv,cp,fi,pd,ad,rv,fd,tv,at]}.\cr 39 | #' 40 | #' \describe{ 41 | #' \item{**_title}{Plot-specific title. Default varies.} 42 | #' \item{**_subtitle}{Plot-specific subtitle. Default is \code{subtitle}.} 43 | #' \item{**_axis_title}{Plot-specific axis title. Default varies.} 44 | #' \item{**_bar_width}{Plot-specific width of bars. Default is \code{bar_width}, 45 | #' ignored when \code{scale_plot = TRUE}.} 46 | #' \item{**_line_size}{Plot-specific width of lines. Default is \code{line_size}.} 47 | #' \item{**_point_size}{Plot-specific point radius. Default is \code{point_size}.} 48 | #' \item{**_*_color}{Plot-specific \code{[bar,line,point]} color. Default is \code{[bar,line,point]_color}.} 49 | #' } 50 | #' } 51 | #' 52 | #' @references 53 | #' 54 | #' \itemize{ 55 | #' \item The input object is implemented in \href{https://modeloriented.github.io/DALEX/}{\bold{DALEX}} 56 | #' \item Feature Importance, Ceteris Paribus, Partial Dependence and Accumulated Dependence explanations 57 | #' are implemented in \href{https://modeloriented.github.io/ingredients/}{\bold{ingredients}} 58 | #' \item Break Down and Shapley Values explanations are implemented in 59 | #' \href{https://modeloriented.github.io/iBreakDown/}{\bold{iBreakDown}} 60 | #' } 61 | #' 62 | #' @seealso 63 | #' Vignettes: \href{https://modelstudio.drwhy.ai/articles/ms-r-python-examples.html}{\bold{modelStudio - R & Python examples}} 64 | #' and \href{https://modelstudio.drwhy.ai/articles/ms-perks-features.html}{\bold{modelStudio - perks and features}} 65 | #' 66 | #' @examples 67 | #' library("DALEX") 68 | #' library("modelStudio") 69 | #' 70 | #' # fit a model 71 | #' model_apartments <- glm(m2.price ~. , data = apartments) 72 | #' 73 | #' # create an explainer for the model 74 | #' explainer_apartments <- explain(model_apartments, 75 | #' data = apartments, 76 | #' y = apartments$m2.price) 77 | #' 78 | #' # pick observations 79 | #' new_observation <- apartments[1:2,] 80 | #' rownames(new_observation) <- c("ap1","ap2") 81 | #' 82 | #' # modify default options 83 | #' new_options <- ms_options( 84 | #' show_subtitle = TRUE, 85 | #' bd_subtitle = "Hello World", 86 | #' line_size = 5, 87 | #' point_size = 9, 88 | #' line_color = "pink", 89 | #' point_color = "purple", 90 | #' bd_positive_color = "yellow", 91 | #' bd_negative_color = "orange" 92 | #' ) 93 | #' 94 | #' # make a studio for the model 95 | #' modelStudio(explainer_apartments, 96 | #' new_observation, 97 | #' options = new_options, 98 | #' N = 200, B = 5) # faster example 99 | #' 100 | #' @export 101 | #' @rdname ms_options 102 | ms_options <- function(...) { 103 | 104 | # prepare default options 105 | default_options <- list( 106 | scale_plot = TRUE, 107 | show_boxplot = TRUE, 108 | show_subtitle = FALSE, 109 | subtitle = NULL, 110 | ms_title = NULL, 111 | ms_subtitle = NULL, 112 | ms_margin_top = 50, 113 | ms_margin_bottom = 50, 114 | margin_top = 50, 115 | margin_right = 20, 116 | margin_bottom = 70, 117 | margin_left = NULL, # 105, 118 | margin_inner = 40, 119 | margin_small = 5, 120 | margin_big = 10, 121 | margin_ytitle = 40, 122 | w = 420, 123 | h = 280, 124 | bar_width = 16, 125 | line_size = 2, 126 | point_size = 2, 127 | bar_color = "#46bac2", 128 | line_color = "#46bac2", 129 | point_color = "#46bac2", 130 | positive_color = "#8bdcbe", 131 | negative_color = "#f05a71", 132 | default_color = "#371ea3", 133 | bd_title = "Break Down", 134 | bd_subtitle = NULL, 135 | bd_axis_title = "contribution", 136 | bd_bar_width = NULL, 137 | bd_positive_color = NULL, 138 | bd_negative_color = NULL, 139 | bd_default_color = NULL, 140 | sv_title = "Shapley Values", 141 | sv_subtitle = NULL, 142 | sv_axis_title = "contribution", 143 | sv_bar_width = NULL, 144 | sv_positive_color = NULL, 145 | sv_negative_color = NULL, 146 | sv_default_color = NULL, 147 | cp_title = "Ceteris Paribus", 148 | cp_subtitle = NULL, 149 | cp_axis_title = "prediction", 150 | cp_bar_width = NULL, 151 | cp_line_size = NULL, 152 | cp_point_size = 3, 153 | cp_bar_color = NULL, 154 | cp_line_color = NULL, 155 | cp_point_color = "#371ea3", 156 | fi_title = "Feature Importance", 157 | fi_subtitle = NULL, 158 | fi_axis_title = NULL, 159 | fi_bar_width = NULL, 160 | fi_bar_color = NULL, 161 | pd_title = "Partial Dependence", 162 | pd_subtitle = NULL, 163 | pd_axis_title = "average prediction", 164 | pd_bar_width = NULL, 165 | pd_line_size = NULL, 166 | pd_bar_color = NULL, 167 | pd_line_color = NULL, 168 | ad_title = "Accumulated Dependence", 169 | ad_subtitle = NULL, 170 | ad_axis_title = "accumulated prediction", 171 | ad_bar_width = NULL, 172 | ad_line_size = NULL, 173 | ad_bar_color = NULL, 174 | ad_line_color = NULL, 175 | rv_title = "Residuals vs Feature", 176 | rv_subtitle = NULL, 177 | rv_axis_title = "residuals", 178 | rv_point_size = NULL, 179 | rv_point_color = NULL, 180 | fd_title = "Feature Distribution", 181 | fd_subtitle = NULL, 182 | fd_axis_title = "count", 183 | fd_bar_width = NULL, 184 | fd_bar_color = NULL, 185 | tv_title = "Target vs Feature", 186 | tv_subtitle = NULL, 187 | tv_axis_title = "target", 188 | tv_point_size = NULL, 189 | tv_point_color = NULL, 190 | at_title = "Average Target vs Feature", 191 | at_subtitle = NULL, 192 | at_axis_title = "average target", 193 | at_bar_width = NULL, 194 | at_line_size = NULL, 195 | at_point_size = 3, 196 | at_bar_color = NULL, 197 | at_line_color = NULL, 198 | at_point_color = "#371ea3", 199 | showcase_name = NULL 200 | ) 201 | 202 | # input new options 203 | default_options[names(list(...))] <- list(...) 204 | 205 | default_options 206 | } 207 | 208 | #' deprecated since v1.1 (May 2020) 209 | #' removed in v2.2 (July 2021) 210 | #' @export 211 | #' @rdname ms_options 212 | # modelStudioOptions <- function(...) { 213 | # warning("The 'modelStudioOptions()' function is deprecated; use 'ms_options()' instead.") 214 | # ret <- ms_options(...) 215 | # ret 216 | # } 217 | -------------------------------------------------------------------------------- /R/ms_update_observations.R: -------------------------------------------------------------------------------- 1 | #' @title Update the observations of a modelStudio object 2 | #' 3 | #' @description 4 | #' This function calculates local explanations on new observations and adds them 5 | #' to the \code{modelStudio} object. 6 | #' 7 | #' @param object A \code{modelStudio} created with \code{modelStudio()}. 8 | #' @param explainer An \code{explainer} created with \code{DALEX::explain()}. 9 | #' @param new_observation New observations with columns that correspond to variables used in the model. 10 | #' @param new_observation_y True label for \code{new_observation} (optional). 11 | #' @param max_features Maximum number of features to be included in BD and SV plots. 12 | #' Default is \code{10}. 13 | #' @param B Number of permutation rounds used for calculation of SV and FI. 14 | #' Default is \code{10}. 15 | #' See \href{https://modelstudio.drwhy.ai/articles/ms-perks-features.html#more-calculations-means-more-time}{\bold{vignette}} 16 | #' @param show_info Verbose a progress on the console. Default is \code{TRUE}. 17 | #' @param parallel Speed up the computation using \code{parallelMap::parallelMap()}. 18 | #' See \href{https://modelstudio.drwhy.ai/articles/ms-perks-features.html#parallel-computation}{\bold{vignette}}. 19 | #' This might interfere with showing progress using \code{show_info}. 20 | #' @param widget_id Use an explicit element ID for the widget (rather than an automatically generated one). 21 | #' Useful e.g. when using \code{modelStudio} with Shiny. 22 | #' See \href{https://modelstudio.drwhy.ai/articles/ms-perks-features.html#shiny-1}{\bold{vignette}}. 23 | #' @param overwrite Overwrite existing observations and their explanations. 24 | #' Default is \code{FALSE} which means add new observations to the existing ones. 25 | #' @param ... Other parameters. 26 | #' 27 | #' @return An object of the \code{r2d3, htmlwidget, modelStudio} class. 28 | #' 29 | #' @references 30 | #' 31 | #' \itemize{ 32 | #' \item The input object is implemented in \href{https://modeloriented.github.io/DALEX/}{\bold{DALEX}} 33 | #' \item Feature Importance, Ceteris Paribus, Partial Dependence and Accumulated Dependence explanations 34 | #' are implemented in \href{https://modeloriented.github.io/ingredients/}{\bold{ingredients}} 35 | #' \item Break Down and Shapley Values explanations are implemented in 36 | #' \href{https://modeloriented.github.io/iBreakDown/}{\bold{iBreakDown}} 37 | #' } 38 | #' 39 | #' @seealso 40 | #' Vignettes: \href{https://modelstudio.drwhy.ai/articles/ms-r-python-examples.html}{\bold{modelStudio - R & Python examples}} 41 | #' and \href{https://modelstudio.drwhy.ai/articles/ms-perks-features.html}{\bold{modelStudio - perks and features}} 42 | #' 43 | #' @examples 44 | #' library("DALEX") 45 | #' library("modelStudio") 46 | #' 47 | #' # fit a model 48 | #' model_titanic <- glm(survived ~., data = titanic_imputed, family = "binomial") 49 | #' 50 | #' # create an explainer for the model 51 | #' explainer_titanic <- explain(model_titanic, 52 | #' data = titanic_imputed, 53 | #' y = titanic_imputed$survived) 54 | #' 55 | #' # make a studio for the model 56 | #' ms <- modelStudio(explainer_titanic, 57 | #' N = 200, B = 5) # faster example 58 | #' 59 | #' \donttest{ 60 | #' 61 | #' # add new observations 62 | #' ms <- ms_update_observations(ms, 63 | #' explainer_titanic, 64 | #' new_observation = titanic_imputed[100:101,], 65 | #' new_observation_y = titanic_imputed$survived[100:101]) 66 | #' ms 67 | #' 68 | #' 69 | #' 70 | #' # overwrite the observations with new ones 71 | #' ms <- ms_update_observations(ms, 72 | #' explainer_titanic, 73 | #' new_observation = titanic_imputed[100:101,], 74 | #' overwrite = TRUE) 75 | #' ms 76 | #' 77 | #' } 78 | #' 79 | #' @export 80 | #' @rdname ms_update_observations 81 | ms_update_observations <- function(object, 82 | explainer, 83 | new_observation = NULL, 84 | new_observation_y = NULL, 85 | max_features = 10, 86 | B = 10, 87 | show_info = TRUE, 88 | parallel = FALSE, 89 | widget_id = NULL, 90 | overwrite = FALSE, 91 | ...) { 92 | 93 | stopifnot("modelStudio" %in% class(object)) 94 | stopifnot("explainer" %in% class(explainer)) 95 | 96 | model <- explainer$model 97 | data <- explainer$data 98 | y <- explainer$y 99 | predict_function <- explainer$predict_function 100 | label <- explainer$label 101 | 102 | # extract old options 103 | options <- object$x$options 104 | 105 | #:# checks 106 | if (is.null(rownames(data))) { 107 | rownames(data) <- 1:nrow(data) 108 | } 109 | 110 | if (is.null(new_observation)) { 111 | if (show_info) message("`new_observation` argument is NULL.\n", 112 | "Observations needed to calculate local explanations are taken at random from the data.\n") 113 | new_observation <- ingredients::select_sample(data, 3) 114 | 115 | } else if (is.null(dim(new_observation))) { 116 | warning("`new_observation` argument is not a data.frame nor a matrix, coerced to data.frame\n") 117 | new_observation <- as.data.frame(new_observation) 118 | 119 | } else if (is.null(rownames(new_observation))) { 120 | rownames(new_observation) <- 1:nrow(new_observation) 121 | } 122 | 123 | check_single_prediction <- try(predict_function(model, new_observation[1,, drop = FALSE]), silent = TRUE) 124 | if ("try-error" %in% class(check_single_prediction)) { 125 | stop("`explainer$predict_function` returns an error when executed on `new_observation[1,, drop = FALSE]` \n") 126 | } 127 | #:# 128 | 129 | ## get proper names of features that arent target 130 | is_y <- is_y_in_data(data, y) 131 | potential_variable_names <- names(is_y[!is_y]) 132 | variable_names <- intersect(potential_variable_names, colnames(new_observation)) 133 | ## get rid of target in data 134 | data <- data[,!is_y] 135 | 136 | obs_count <- dim(new_observation)[1] 137 | obs_data <- new_observation 138 | obs_list <- list() 139 | 140 | ## later update progress bar after all explanation functions 141 | if (show_info) { 142 | pb <- progress_bar$new( 143 | format = " Calculating :what \n Elapsed time: :elapsedfull ETA::eta", # :percent [:bar] 144 | total = (3*B + 2 + 1)*obs_count, 145 | show_after = 0 146 | ) 147 | pb$tick(0, tokens = list(what = "...")) 148 | } 149 | 150 | if (parallel) { 151 | parallelMap::parallelStart() 152 | parallelMap::parallelLibrary(packages = loadedNamespaces()) 153 | 154 | f <- function(i, model, data, predict_function, label, B, show_boxplot, ...) { 155 | new_observation <- obs_data[i,, drop = FALSE] 156 | 157 | bd <- calculate( 158 | iBreakDown::local_attributions( 159 | model, data, predict_function, new_observation, label = label), 160 | paste0("iBreakDown::local_attributions (", i, ")"), show_info, pb, 2) 161 | sv <- calculate( 162 | iBreakDown::shap( 163 | model, data, predict_function, new_observation, label = label, B = B), 164 | paste0("iBreakDown::shap (", i, ")"), show_info, pb, 3*B) 165 | cp <- calculate( 166 | ingredients::ceteris_paribus( 167 | model, data, predict_function, new_observation, label = label), 168 | paste0("ingredients::ceteris_paribus (", i, ")"), show_info, pb, 1) 169 | 170 | bd_data <- prepare_break_down(bd, max_features, ...) 171 | sv_data <- prepare_shapley_values(sv, max_features, show_boxplot, ...) 172 | cp_data <- prepare_ceteris_paribus(cp, variables = variable_names) 173 | 174 | list(bd_data, cp_data, sv_data) 175 | } 176 | 177 | obs_list <- parallelMap::parallelMap(f, 1:obs_count, 178 | more.args = list( 179 | model = model, 180 | data = data, 181 | predict_function = predict_function, 182 | label = label, 183 | B = B, 184 | show_boxplot = options$show_boxplot, 185 | ... 186 | )) 187 | 188 | parallelMap::parallelStop() 189 | 190 | } else { 191 | ## count once per observation 192 | for(i in 1:obs_count) { 193 | new_observation <- obs_data[i,, drop = FALSE] 194 | 195 | bd <- calculate( 196 | iBreakDown::local_attributions( 197 | model, data, predict_function, new_observation, label = label), 198 | paste0("iBreakDown::local_attributions (", i, ")"), show_info, pb, 2) 199 | sv <- calculate( 200 | iBreakDown::shap( 201 | model, data, predict_function, new_observation, label = label, B = B), 202 | paste0("iBreakDown::shap (", i, ")"), show_info, pb, 3*B) 203 | cp <- calculate( 204 | ingredients::ceteris_paribus( 205 | model, data, predict_function, new_observation, label = label), 206 | paste0("ingredients::ceteris_paribus (", i, ")"), show_info, pb, 1) 207 | 208 | bd_data <- prepare_break_down(bd, max_features, ...) 209 | sv_data <- prepare_shapley_values(sv, max_features, options$show_boxplot, ...) 210 | cp_data <- prepare_ceteris_paribus(cp, variables = variable_names) 211 | 212 | obs_list[[i]] <- list(bd_data, cp_data, sv_data) 213 | } 214 | } 215 | 216 | names(obs_list) <- rownames(obs_data) 217 | 218 | #:# prepare new observation data for drop down 219 | between <- " - " 220 | if (is.null(new_observation_y)) new_observation_y <- between <- "" 221 | drop_down_data <- as.data.frame(cbind(rownames(obs_data), 222 | paste0(rownames(obs_data), between, new_observation_y))) 223 | colnames(drop_down_data) <- c("id", "text") 224 | 225 | #:# extract old data 226 | old_data <- jsonlite::fromJSON(object$x$data, simplifyVector = FALSE) 227 | 228 | if (!overwrite) { 229 | #:# extract old drop down and merge with new one 230 | old_drop_down_data <- jsonlite::fromJSON(options$drop_down_data, 231 | simplifyVector = FALSE, simplifyDataFrame = TRUE) 232 | drop_down_data <- rbind(old_drop_down_data, drop_down_data) 233 | 234 | #:# update new data 235 | obs_list <- c(old_data[[1]], obs_list) 236 | if (length(unique(names(obs_list))) != length(obs_list)) { 237 | warning("new_observation ids overlap with existing data, using unique ids") 238 | obs_list <- obs_list[unique(names(obs_list))] 239 | drop_down_data <- drop_down_data[!duplicated(drop_down_data$id),] 240 | } 241 | } 242 | 243 | #:# input new data 244 | temp <- jsonlite::toJSON(list(obs_list, old_data[[2]], old_data[[3]], 245 | old_data[[4]], old_data[[5]], old_data[[6]]), 246 | auto_unbox = TRUE) 247 | widget_id <- ifelse(!is.null(widget_id), 248 | widget_id, 249 | paste0("widget-", digest::digest(temp))) 250 | 251 | #:# extract old options and update them 252 | new_options <- options 253 | new_options$widget_id <- widget_id 254 | new_options$variable_names <- variable_names 255 | new_options$footer_text <- paste0("Site built with modelStudio v", 256 | as.character(packageVersion("modelStudio")), 257 | " on ", 258 | format(Sys.time(), usetz = FALSE)) 259 | new_options$drop_down_data <- jsonlite::toJSON(drop_down_data) 260 | 261 | options("r2d3.shadow" = FALSE) # set this option to avoid using shadow-root 262 | 263 | model_studio <- r2d3::r2d3( 264 | data = temp, 265 | script = system.file("d3js/modelStudio.js", package = "modelStudio"), 266 | dependencies = list( 267 | system.file("d3js/hackHead.js", package = "modelStudio"), 268 | system.file("d3js/myTools.js", package = "modelStudio"), 269 | system.file("d3js/d3-tip.js", package = "modelStudio"), 270 | system.file("d3js/d3-simple-slider.min.js", package = "modelStudio"), 271 | system.file("d3js/d3-interpolate-path.min.js", package = "modelStudio"), 272 | system.file("d3js/generatePlots.js", package = "modelStudio"), 273 | system.file("d3js/generateTooltipHtml.js", package = "modelStudio") 274 | ), 275 | css = system.file("d3js/modelStudio.css", package = "modelStudio"), 276 | options = new_options, 277 | d3_version = "4", 278 | sizing = object$sizingPolicy, 279 | elementId = widget_id, 280 | width = new_options$facet_dim[2]*(new_options$w + new_options$margin_left + new_options$margin_right), 281 | height = 100 + new_options$facet_dim[1]*(new_options$h + new_options$margin_top + new_options$margin_bottom) 282 | ) 283 | 284 | model_studio$x$script <- remove_file_paths(model_studio$x$script, "js") 285 | model_studio$x$style <- remove_file_paths(model_studio$x$style, "css") 286 | 287 | class(model_studio) <- c(class(model_studio), "modelStudio") 288 | 289 | model_studio 290 | } 291 | -------------------------------------------------------------------------------- /R/ms_update_options.R: -------------------------------------------------------------------------------- 1 | #' @title Update the options of a modelStudio object 2 | #' 3 | #' @description 4 | #' This function updates the options of a \code{\link{modelStudio}} object. 5 | #' \strong{WARNING: Editing default options may cause unintended behavior.} 6 | #' 7 | #' @param object A \code{modelStudio} created with \code{modelStudio()}. 8 | #' @param ... Options to change in the form \code{option_name = value}, 9 | #' e.g. \code{time = 0}, \code{facet_dim = c(1,2)}. 10 | #' 11 | #' @return An object of the \code{r2d3, htmlwidget, modelStudio} class. 12 | #' 13 | #' @inheritSection ms_options Options 14 | #' 15 | #' @references 16 | #' 17 | #' \itemize{ 18 | #' \item The input object is implemented in \href{https://modeloriented.github.io/DALEX/}{\bold{DALEX}} 19 | #' \item Feature Importance, Ceteris Paribus, Partial Dependence and Accumulated Dependence explanations 20 | #' are implemented in \href{https://modeloriented.github.io/ingredients/}{\bold{ingredients}} 21 | #' \item Break Down and Shapley Values explanations are implemented in 22 | #' \href{https://modeloriented.github.io/iBreakDown/}{\bold{iBreakDown}} 23 | #' } 24 | #' 25 | #' @seealso 26 | #' Vignettes: \href{https://modelstudio.drwhy.ai/articles/ms-r-python-examples.html}{\bold{modelStudio - R & Python examples}} 27 | #' and \href{https://modelstudio.drwhy.ai/articles/ms-perks-features.html}{\bold{modelStudio - perks and features}} 28 | #' 29 | #' @examples 30 | #' library("DALEX") 31 | #' library("modelStudio") 32 | #' 33 | #' # fit a model 34 | #' model_titanic <- glm(survived ~., data = titanic_imputed, family = "binomial") 35 | #' 36 | #' # create an explainer for the model 37 | #' explainer_titanic <- explain(model_titanic, 38 | #' data = titanic_imputed, 39 | #' y = titanic_imputed$survived) 40 | #' 41 | #' # make a studio for the model 42 | #' ms <- modelStudio(explainer_titanic, 43 | #' N = 200, B = 5) # faster example 44 | #' 45 | #' # update the options 46 | #' new_ms <- ms_update_options(ms, 47 | #' time = 0, 48 | #' facet_dim = c(1,2), 49 | #' margin_left = 150) 50 | #' new_ms 51 | #' 52 | #' @export 53 | #' @rdname ms_update_options 54 | ms_update_options <- function(object, ...) { 55 | 56 | stopifnot("modelStudio" %in% class(object)) 57 | 58 | # extract old options 59 | old_options <- object$x$options 60 | # input new options 61 | old_options[names(list(...))] <- list(...) 62 | new_options <- old_options 63 | 64 | # update footer text 65 | new_options$footer_text <- paste0("Site built with modelStudio v", packageVersion("modelStudio"), 66 | " on ", format(Sys.time(), usetz = FALSE)) 67 | 68 | options("r2d3.shadow" = FALSE) # set this option to avoid using shadow-root 69 | 70 | model_studio <- r2d3::r2d3( 71 | data = object$x$data, 72 | script = system.file("d3js/modelStudio.js", package = "modelStudio"), 73 | dependencies = list( 74 | system.file("d3js/hackHead.js", package = "modelStudio"), 75 | system.file("d3js/myTools.js", package = "modelStudio"), 76 | system.file("d3js/d3-tip.js", package = "modelStudio"), 77 | system.file("d3js/d3-simple-slider.min.js", package = "modelStudio"), 78 | system.file("d3js/d3-interpolate-path.min.js", package = "modelStudio"), 79 | system.file("d3js/generatePlots.js", package = "modelStudio"), 80 | system.file("d3js/generateTooltipHtml.js", package = "modelStudio") 81 | ), 82 | css = system.file("d3js/modelStudio.css", package = "modelStudio"), 83 | options = new_options, 84 | d3_version = "4", 85 | sizing = object$sizingPolicy, 86 | elementId = object$elementId, 87 | width = new_options$facet_dim[2]*(new_options$w + new_options$margin_left + new_options$margin_right), 88 | height = 100 + new_options$facet_dim[1]*(new_options$h + new_options$margin_top + new_options$margin_bottom) 89 | ) 90 | 91 | model_studio$x$script <- remove_file_paths(model_studio$x$script, "js") 92 | model_studio$x$style <- remove_file_paths(model_studio$x$style, "css") 93 | 94 | class(model_studio) <- c(class(model_studio), "modelStudio") 95 | 96 | model_studio 97 | } 98 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Interactive Studio for Explanatory Model Analysis 2 | 3 |

4 | 5 |

6 | 7 | [![CRAN_Status_Badge](http://www.r-pkg.org/badges/version/modelStudio)](https://cran.r-project.org/package=modelStudio) 8 | [![R build status](https://github.com/ModelOriented/modelStudio/workflows/R-CMD-check/badge.svg)](https://github.com/ModelOriented/modelStudio/actions?query=workflow%3AR-CMD-check) 9 | [![Codecov test coverage](https://codecov.io/gh/ModelOriented/modelStudio/branch/master/graph/badge.svg)](https://app.codecov.io/gh/ModelOriented/modelStudio?branch=master) 10 | [![](https://cranlogs.r-pkg.org/badges/grand-total/modelStudio)](https://cranlogs.r-pkg.org/badges/grand-total/modelStudio) 11 | [![JOSS-status](https://joss.theoj.org/papers/9eec8c9d1969fbd44b3ea438a74af911/status.svg)](https://joss.theoj.org/papers/9eec8c9d1969fbd44b3ea438a74af911) 12 | 13 | ## Overview 14 | 15 | The `modelStudio` package **automates the explanatory analysis of machine learning predictive models**. It generates advanced interactive model explanations in the form of a **serverless HTML site** with only one line of code. This tool is model-agnostic, therefore compatible with most of the black-box predictive models and frameworks (e.g. `mlr/mlr3`, `xgboost`, `caret`, `h2o`, `parsnip`, `tidymodels`, `scikit-learn`, `lightgbm`, `keras/tensorflow`). 16 | 17 | The main `modelStudio()` function computes various (instance and model-level) explanations and produces a **customisable dashboard**, which consists of multiple panels for plots with their short descriptions. It is possible to easily **save** the dashboard and **share** it with others. Tools for [Explanatory Model Analysis](https://ema.drwhy.ai) unite with tools for Exploratory Data Analysis to give a broad overview of the model behavior. 18 | 19 |

20 | explain COVID-19   21 | R & Python examples   22 | More resources   23 | Interactive EMA   24 |

25 | 26 | [![](man/figures/demo_small.gif)](https://modelstudio.drwhy.ai/demo.html) 27 | 28 | The `modelStudio` package is a part of the [**DrWhy.AI**](http://drwhy.ai) universe. 29 | 30 | ## Installation 31 | 32 | ```r 33 | # Install from CRAN: 34 | install.packages("modelStudio") 35 | 36 | # Install the development version from GitHub: 37 | devtools::install_github("ModelOriented/modelStudio") 38 | ``` 39 | 40 | ## Simple demo 41 | 42 | ```r 43 | library("DALEX") 44 | library("ranger") 45 | library("modelStudio") 46 | 47 | # fit a model 48 | model <- ranger(score ~., data = happiness_train) 49 | 50 | # create an explainer for the model 51 | explainer <- explain(model, 52 | data = happiness_test, 53 | y = happiness_test$score, 54 | label = "Random Forest") 55 | 56 | # make a studio for the model 57 | modelStudio(explainer) 58 | ``` 59 | 60 | [Save the output](https://modelstudio.drwhy.ai/#save--share) in the form of a HTML file - [**Demo Dashboard**](https://modelstudio.drwhy.ai/demo.html). 61 | 62 | [![](man/figures/demo_big.gif)](https://modelstudio.drwhy.ai/demo.html) 63 | 64 | ## R & Python examples [more](https://modelstudio.drwhy.ai/articles/ms-r-python-examples.html) 65 | 66 | ------------------------------- 67 | 68 | The `modelStudio()` function uses `DALEX` explainers created with `DALEX::explain()` or `DALEXtra::explain_*()`. 69 | 70 | ```r 71 | # packages for the explainer objects 72 | install.packages("DALEX") 73 | install.packages("DALEXtra") 74 | ``` 75 | 76 | ### mlr [dashboard](https://modelstudio.drwhy.ai/mlr.html) 77 | 78 | Make a studio for the regression `ranger` model on the `apartments` data. 79 | 80 |
81 | code 82 | 83 | ```r 84 | # load packages and data 85 | library(mlr) 86 | library(DALEXtra) 87 | library(modelStudio) 88 | 89 | data <- DALEX::apartments 90 | 91 | # split the data 92 | index <- sample(1:nrow(data), 0.7*nrow(data)) 93 | train <- data[index,] 94 | test <- data[-index,] 95 | 96 | # fit a model 97 | task <- makeRegrTask(id = "apartments", data = train, target = "m2.price") 98 | learner <- makeLearner("regr.ranger", predict.type = "response") 99 | model <- train(learner, task) 100 | 101 | # create an explainer for the model 102 | explainer <- explain_mlr(model, 103 | data = test, 104 | y = test$m2.price, 105 | label = "mlr") 106 | 107 | # pick observations 108 | new_observation <- test[1:2,] 109 | rownames(new_observation) <- c("id1", "id2") 110 | 111 | # make a studio for the model 112 | modelStudio(explainer, new_observation) 113 | ``` 114 | 115 |
116 | 117 | ### xgboost [dashboard](https://modelstudio.drwhy.ai/xgboost.html) 118 | 119 | Make a studio for the classification `xgboost` model on the `titanic` data. 120 | 121 |
122 | code 123 | 124 | ```r 125 | # load packages and data 126 | library(xgboost) 127 | library(DALEX) 128 | library(modelStudio) 129 | 130 | data <- DALEX::titanic_imputed 131 | 132 | # split the data 133 | index <- sample(1:nrow(data), 0.7*nrow(data)) 134 | train <- data[index,] 135 | test <- data[-index,] 136 | 137 | train_matrix <- model.matrix(survived ~.-1, train) 138 | test_matrix <- model.matrix(survived ~.-1, test) 139 | 140 | # fit a model 141 | xgb_matrix <- xgb.DMatrix(train_matrix, label = train$survived) 142 | params <- list(max_depth = 3, objective = "binary:logistic", eval_metric = "auc") 143 | model <- xgb.train(params, xgb_matrix, nrounds = 500) 144 | 145 | # create an explainer for the model 146 | explainer <- explain(model, 147 | data = test_matrix, 148 | y = test$survived, 149 | type = "classification", 150 | label = "xgboost") 151 | 152 | # pick observations 153 | new_observation <- test_matrix[1:2, , drop=FALSE] 154 | rownames(new_observation) <- c("id1", "id2") 155 | 156 | # make a studio for the model 157 | modelStudio(explainer, new_observation) 158 | ``` 159 | 160 |
161 | 162 | ------------------------- 163 | 164 | The `modelStudio()` function uses `dalex` explainers created with `dalex.Explainer()`. 165 | 166 | ```console 167 | :: package for the Explainer object 168 | pip install dalex -U 169 | ``` 170 | 171 | Use `pickle` Python module and `reticulate` R package to easily make a studio for a model. 172 | 173 | ```r 174 | # package for pickle load 175 | install.packages("reticulate") 176 | ``` 177 | 178 | ### scikit-learn [dashboard](https://modelstudio.drwhy.ai/scikitlearn.html) 179 | 180 | Make a studio for the regression `Pipeline SVR` model on the `fifa` data. 181 | 182 |
183 | code 184 | 185 | First, use `dalex` in Python: 186 | 187 | ```python 188 | # load packages and data 189 | import dalex as dx 190 | from sklearn.model_selection import train_test_split 191 | from sklearn.pipeline import Pipeline 192 | from sklearn.preprocessing import StandardScaler 193 | from sklearn.svm import SVR 194 | from numpy import log 195 | 196 | data = dx.datasets.load_fifa() 197 | X = data.drop(columns=['overall', 'potential', 'value_eur', 'wage_eur', 'nationality'], axis=1) 198 | y = log(data.value_eur) 199 | 200 | # split the data 201 | X_train, X_test, y_train, y_test = train_test_split(X, y) 202 | 203 | # fit a pipeline model 204 | model = Pipeline([('scale', StandardScaler()), ('svm', SVR())]) 205 | model.fit(X_train, y_train) 206 | 207 | # create an explainer for the model 208 | explainer = dx.Explainer(model, data=X_test, y=y_test, label='scikit-learn') 209 | 210 | # pack the explainer into a pickle file 211 | explainer.dump(open('explainer_scikitlearn.pickle', 'wb')) 212 | ``` 213 | 214 | Then, use `modelStudio` in R: 215 | 216 | ```r 217 | # load the explainer from the pickle file 218 | library(reticulate) 219 | explainer <- py_load_object("explainer_scikitlearn.pickle", pickle = "pickle") 220 | 221 | # make a studio for the model 222 | library(modelStudio) 223 | modelStudio(explainer, B = 5) 224 | ``` 225 | 226 |
227 | 228 | ### lightgbm [dashboard](https://modelstudio.drwhy.ai/lightgbm.html) 229 | 230 | Make a studio for the classification `Pipeline LGBMClassifier` model on the `titanic` data. 231 | 232 |
233 | code 234 | 235 | First, use `dalex` in Python: 236 | 237 | ```python 238 | # load packages and data 239 | import dalex as dx 240 | from sklearn.model_selection import train_test_split 241 | from sklearn.pipeline import Pipeline 242 | from sklearn.preprocessing import StandardScaler, OneHotEncoder 243 | from sklearn.impute import SimpleImputer 244 | from sklearn.compose import ColumnTransformer 245 | from lightgbm import LGBMClassifier 246 | 247 | data = dx.datasets.load_titanic() 248 | X = data.drop(columns='survived') 249 | y = data.survived 250 | 251 | # split the data 252 | X_train, X_test, y_train, y_test = train_test_split(X, y) 253 | 254 | # fit a pipeline model 255 | numerical_features = ['age', 'fare', 'sibsp', 'parch'] 256 | numerical_transformer = Pipeline( 257 | steps=[ 258 | ('imputer', SimpleImputer(strategy='median')), 259 | ('scaler', StandardScaler()) 260 | ] 261 | ) 262 | categorical_features = ['gender', 'class', 'embarked'] 263 | categorical_transformer = Pipeline( 264 | steps=[ 265 | ('imputer', SimpleImputer(strategy='constant', fill_value='missing')), 266 | ('onehot', OneHotEncoder(handle_unknown='ignore')) 267 | ] 268 | ) 269 | 270 | preprocessor = ColumnTransformer( 271 | transformers=[ 272 | ('num', numerical_transformer, numerical_features), 273 | ('cat', categorical_transformer, categorical_features) 274 | ] 275 | ) 276 | 277 | classifier = LGBMClassifier(n_estimators=300) 278 | 279 | model = Pipeline( 280 | steps=[ 281 | ('preprocessor', preprocessor), 282 | ('classifier', classifier) 283 | ] 284 | ) 285 | model.fit(X_train, y_train) 286 | 287 | # create an explainer for the model 288 | explainer = dx.Explainer(model, data=X_test, y=y_test, label='lightgbm') 289 | 290 | # pack the explainer into a pickle file 291 | explainer.dump(open('explainer_lightgbm.pickle', 'wb')) 292 | ``` 293 | 294 | Then, use `modelStudio` in R: 295 | 296 | ```r 297 | # load the explainer from the pickle file 298 | library(reticulate) 299 | explainer <- py_load_object("explainer_lightgbm.pickle", pickle = "pickle") 300 | 301 | # make a studio for the model 302 | library(modelStudio) 303 | modelStudio(explainer) 304 | ``` 305 | 306 |
307 | 308 | ------------------------------- 309 | 310 | ## Save & share 311 | 312 | Save `modelStudio` as a HTML file using buttons on the top of the RStudio Viewer 313 | or with [`r2d3::save_d3_html()`](https://rstudio.github.io/r2d3/articles/publishing.html#save-as-html). 314 | 315 |

316 | 317 |

318 | 319 | ## Citations 320 | 321 | If you use `modelStudio`, please cite our [JOSS article](https://joss.theoj.org/papers/10.21105/joss.01798): 322 | 323 | ``` 324 | @article{baniecki2019modelstudio, 325 | title = {{modelStudio: Interactive Studio with Explanations for ML Predictive Models}}, 326 | author = {Hubert Baniecki and Przemyslaw Biecek}, 327 | journal = {Journal of Open Source Software}, 328 | year = {2019}, 329 | volume = {4}, 330 | number = {43}, 331 | pages = {1798}, 332 | url = {https://doi.org/10.21105/joss.01798} 333 | } 334 | ``` 335 | 336 | For a description and evaluation of the Interactive EMA process, refer to our [DAMI article](https://doi.org/10.1007/s10618-023-00924-w): 337 | 338 | ``` 339 | @article{baniecki2023grammar, 340 | title = {The grammar of interactive explanatory model analysis}, 341 | author = {Hubert Baniecki and Dariusz Parzych and Przemyslaw Biecek}, 342 | journal = {Data Mining and Knowledge Discovery}, 343 | year = {2023}, 344 | pages = {1--37}, 345 | url = {https://doi.org/10.1007/s10618-023-00924-w} 346 | } 347 | ``` 348 | 349 | ## More resources 350 | 351 | - Introduction to the plots: [Explanatory Model Analysis: Explore, Explain, and Examine Predictive Models](https://ema.drwhy.ai) 352 | 353 | - Vignettes: [perks and features](https://modelstudio.drwhy.ai/articles/ms-perks-features.html), [R & Python examples](https://modelstudio.drwhy.ai/articles/ms-r-python-examples.html), [modelStudio in R Markdown HTML](https://modelstudio.drwhy.ai/articles/ms-rmarkdown.html) 354 | 355 | - Changelog: [NEWS](https://modelstudio.drwhy.ai/news/index.html) 356 | 357 | - Conference poster: [ML in PL 2019](https://github.com/ModelOriented/modelStudio/blob/master/misc/MLinPL2019_modelStudio_poster.pdf) 358 | 359 | 360 | ## Acknowledgments 361 | 362 | Work on this package was financially supported by the National Science Centre (Poland) grant `2016/21/B/ST6/02176` and National Centre for Research and Development grant `POIR.01.01.01-00-0328/17`. 363 | -------------------------------------------------------------------------------- /codecov.yml: -------------------------------------------------------------------------------- 1 | comment: false 2 | 3 | coverage: 4 | status: 5 | project: 6 | default: 7 | target: auto 8 | threshold: 1% 9 | informational: true 10 | patch: 11 | default: 12 | target: auto 13 | threshold: 1% 14 | informational: true 15 | -------------------------------------------------------------------------------- /data-raw/2018.csv: -------------------------------------------------------------------------------- 1 | Overall rank,Country or region,Score,GDP per capita,Social support,Healthy life expectancy,Freedom to make life choices,Generosity,Perceptions of corruption 2 | 1,Finland,7.632,1.305,1.592,0.874,0.681,0.202,0.393 3 | 2,Norway,7.594,1.456,1.582,0.861,0.686,0.286,0.340 4 | 3,Denmark,7.555,1.351,1.590,0.868,0.683,0.284,0.408 5 | 4,Iceland,7.495,1.343,1.644,0.914,0.677,0.353,0.138 6 | 5,Switzerland,7.487,1.420,1.549,0.927,0.660,0.256,0.357 7 | 6,Netherlands,7.441,1.361,1.488,0.878,0.638,0.333,0.295 8 | 7,Canada,7.328,1.330,1.532,0.896,0.653,0.321,0.291 9 | 8,New Zealand,7.324,1.268,1.601,0.876,0.669,0.365,0.389 10 | 9,Sweden,7.314,1.355,1.501,0.913,0.659,0.285,0.383 11 | 10,Australia,7.272,1.340,1.573,0.910,0.647,0.361,0.302 12 | 11,United Kingdom,7.190,1.244,1.433,0.888,0.464,0.262,0.082 13 | 12,Austria,7.139,1.341,1.504,0.891,0.617,0.242,0.224 14 | 13,Costa Rica,7.072,1.010,1.459,0.817,0.632,0.143,0.101 15 | 14,Ireland,6.977,1.448,1.583,0.876,0.614,0.307,0.306 16 | 15,Germany,6.965,1.340,1.474,0.861,0.586,0.273,0.280 17 | 16,Belgium,6.927,1.324,1.483,0.894,0.583,0.188,0.240 18 | 17,Luxembourg,6.910,1.576,1.520,0.896,0.632,0.196,0.321 19 | 18,United States,6.886,1.398,1.471,0.819,0.547,0.291,0.133 20 | 19,Israel,6.814,1.301,1.559,0.883,0.533,0.354,0.272 21 | 20,United Arab Emirates,6.774,2.096,0.776,0.670,0.284,0.186,N/A 22 | 21,Czech Republic,6.711,1.233,1.489,0.854,0.543,0.064,0.034 23 | 22,Malta,6.627,1.270,1.525,0.884,0.645,0.376,0.142 24 | 23,France,6.489,1.293,1.466,0.908,0.520,0.098,0.176 25 | 24,Mexico,6.488,1.038,1.252,0.761,0.479,0.069,0.095 26 | 25,Chile,6.476,1.131,1.331,0.808,0.431,0.197,0.061 27 | 26,Taiwan,6.441,1.365,1.436,0.857,0.418,0.151,0.078 28 | 27,Panama,6.430,1.112,1.438,0.759,0.597,0.125,0.063 29 | 28,Brazil,6.419,0.986,1.474,0.675,0.493,0.110,0.088 30 | 29,Argentina,6.388,1.073,1.468,0.744,0.570,0.062,0.054 31 | 30,Guatemala,6.382,0.781,1.268,0.608,0.604,0.179,0.071 32 | 31,Uruguay,6.379,1.093,1.459,0.771,0.625,0.130,0.155 33 | 32,Qatar,6.374,1.649,1.303,0.748,0.654,0.256,0.171 34 | 33,Saudi Arabia,6.371,1.379,1.331,0.633,0.509,0.098,0.127 35 | 34,Singapore,6.343,1.529,1.451,1.008,0.631,0.261,0.457 36 | 35,Malaysia,6.322,1.161,1.258,0.669,0.356,0.311,0.059 37 | 36,Spain,6.310,1.251,1.538,0.965,0.449,0.142,0.074 38 | 37,Colombia,6.260,0.960,1.439,0.635,0.531,0.099,0.039 39 | 38,Trinidad & Tobago,6.192,1.223,1.492,0.564,0.575,0.171,0.019 40 | 39,Slovakia,6.173,1.210,1.537,0.776,0.354,0.118,0.014 41 | 40,El Salvador,6.167,0.806,1.231,0.639,0.461,0.065,0.082 42 | 41,Nicaragua,6.141,0.668,1.319,0.700,0.527,0.208,0.128 43 | 42,Poland,6.123,1.176,1.448,0.781,0.546,0.108,0.064 44 | 43,Bahrain,6.105,1.338,1.366,0.698,0.594,0.243,0.123 45 | 44,Uzbekistan,6.096,0.719,1.584,0.605,0.724,0.328,0.259 46 | 45,Kuwait,6.083,1.474,1.301,0.675,0.554,0.167,0.106 47 | 46,Thailand,6.072,1.016,1.417,0.707,0.637,0.364,0.029 48 | 47,Italy,6.000,1.264,1.501,0.946,0.281,0.137,0.028 49 | 48,Ecuador,5.973,0.889,1.330,0.736,0.556,0.114,0.120 50 | 49,Belize,5.956,0.807,1.101,0.474,0.593,0.183,0.089 51 | 50,Lithuania,5.952,1.197,1.527,0.716,0.350,0.026,0.006 52 | 51,Slovenia,5.948,1.219,1.506,0.856,0.633,0.160,0.051 53 | 52,Romania,5.945,1.116,1.219,0.726,0.528,0.088,0.001 54 | 53,Latvia,5.933,1.148,1.454,0.671,0.363,0.092,0.066 55 | 54,Japan,5.915,1.294,1.462,0.988,0.553,0.079,0.150 56 | 55,Mauritius,5.891,1.090,1.387,0.684,0.584,0.245,0.050 57 | 56,Jamaica,5.890,0.819,1.493,0.693,0.575,0.096,0.031 58 | 57,South Korea,5.875,1.266,1.204,0.955,0.244,0.175,0.051 59 | 58,Northern Cyprus,5.835,1.229,1.211,0.909,0.495,0.179,0.154 60 | 59,Russia,5.810,1.151,1.479,0.599,0.399,0.065,0.025 61 | 60,Kazakhstan,5.790,1.143,1.516,0.631,0.454,0.148,0.121 62 | 61,Cyprus,5.762,1.229,1.191,0.909,0.423,0.202,0.035 63 | 62,Bolivia,5.752,0.751,1.223,0.508,0.606,0.141,0.054 64 | 63,Estonia,5.739,1.200,1.532,0.737,0.553,0.086,0.174 65 | 64,Paraguay,5.681,0.835,1.522,0.615,0.541,0.162,0.074 66 | 65,Peru,5.663,0.934,1.249,0.674,0.530,0.092,0.034 67 | 66,Kosovo,5.662,0.855,1.230,0.578,0.448,0.274,0.023 68 | 67,Moldova,5.640,0.657,1.301,0.620,0.232,0.171,0.000 69 | 68,Turkmenistan,5.636,1.016,1.533,0.517,0.417,0.199,0.037 70 | 69,Hungary,5.620,1.171,1.401,0.732,0.259,0.061,0.022 71 | 70,Libya,5.566,0.985,1.350,0.553,0.496,0.116,0.148 72 | 71,Philippines,5.524,0.775,1.312,0.513,0.643,0.120,0.105 73 | 72,Honduras,5.504,0.620,1.205,0.622,0.459,0.197,0.074 74 | 73,Belarus,5.483,1.039,1.498,0.700,0.307,0.101,0.154 75 | 74,Turkey,5.483,1.148,1.380,0.686,0.324,0.106,0.109 76 | 75,Pakistan,5.472,0.652,0.810,0.424,0.334,0.216,0.113 77 | 76,Hong Kong,5.430,1.405,1.290,1.030,0.524,0.246,0.291 78 | 77,Portugal,5.410,1.188,1.429,0.884,0.562,0.055,0.017 79 | 78,Serbia,5.398,0.975,1.369,0.685,0.288,0.134,0.043 80 | 79,Greece,5.358,1.154,1.202,0.879,0.131,0.000,0.044 81 | 80,Lebanon,5.358,0.965,1.179,0.785,0.503,0.214,0.136 82 | 81,Montenegro,5.347,1.017,1.279,0.729,0.259,0.111,0.081 83 | 82,Croatia,5.321,1.115,1.161,0.737,0.380,0.120,0.039 84 | 83,Dominican Republic,5.302,0.982,1.441,0.614,0.578,0.120,0.106 85 | 84,Algeria,5.295,0.979,1.154,0.687,0.077,0.055,0.135 86 | 85,Morocco,5.254,0.779,0.797,0.669,0.460,0.026,0.074 87 | 86,China,5.246,0.989,1.142,0.799,0.597,0.029,0.103 88 | 87,Azerbaijan,5.201,1.024,1.161,0.603,0.430,0.031,0.176 89 | 88,Tajikistan,5.199,0.474,1.166,0.598,0.292,0.187,0.034 90 | 89,Macedonia,5.185,0.959,1.239,0.691,0.394,0.173,0.052 91 | 90,Jordan,5.161,0.822,1.265,0.645,0.468,0.130,0.134 92 | 91,Nigeria,5.155,0.689,1.172,0.048,0.462,0.201,0.032 93 | 92,Kyrgyzstan,5.131,0.530,1.416,0.594,0.540,0.281,0.035 94 | 93,Bosnia and Herzegovina,5.129,0.915,1.078,0.758,0.280,0.216,0.000 95 | 94,Mongolia,5.125,0.914,1.517,0.575,0.395,0.253,0.032 96 | 95,Vietnam,5.103,0.715,1.365,0.702,0.618,0.177,0.079 97 | 96,Indonesia,5.093,0.899,1.215,0.522,0.538,0.484,0.018 98 | 97,Bhutan,5.082,0.796,1.335,0.527,0.541,0.364,0.171 99 | 98,Somalia,4.982,0.000,0.712,0.115,0.674,0.238,0.282 100 | 99,Cameroon,4.975,0.535,0.891,0.182,0.454,0.183,0.043 101 | 100,Bulgaria,4.933,1.054,1.515,0.712,0.359,0.064,0.009 102 | 101,Nepal,4.880,0.425,1.228,0.539,0.526,0.302,0.078 103 | 102,Venezuela,4.806,0.996,1.469,0.657,0.133,0.056,0.052 104 | 103,Gabon,4.758,1.036,1.164,0.404,0.356,0.032,0.052 105 | 104,Palestinian Territories,4.743,0.642,1.217,0.602,0.266,0.086,0.076 106 | 105,South Africa,4.724,0.940,1.410,0.330,0.516,0.103,0.056 107 | 106,Iran,4.707,1.059,0.771,0.691,0.459,0.282,0.129 108 | 107,Ivory Coast,4.671,0.541,0.872,0.080,0.467,0.146,0.103 109 | 108,Ghana,4.657,0.592,0.896,0.337,0.499,0.212,0.029 110 | 109,Senegal,4.631,0.429,1.117,0.433,0.406,0.138,0.082 111 | 110,Laos,4.623,0.720,1.034,0.441,0.626,0.230,0.174 112 | 111,Tunisia,4.592,0.900,0.906,0.690,0.271,0.040,0.063 113 | 112,Albania,4.586,0.916,0.817,0.790,0.419,0.149,0.032 114 | 113,Sierra Leone,4.571,0.256,0.813,0.000,0.355,0.238,0.053 115 | 114,Congo (Brazzaville),4.559,0.682,0.811,0.343,0.514,0.091,0.077 116 | 115,Bangladesh,4.500,0.532,0.850,0.579,0.580,0.153,0.144 117 | 116,Sri Lanka,4.471,0.918,1.314,0.672,0.585,0.307,0.050 118 | 117,Iraq,4.456,1.010,0.971,0.536,0.304,0.148,0.095 119 | 118,Mali,4.447,0.370,1.233,0.152,0.367,0.139,0.056 120 | 119,Namibia,4.441,0.874,1.281,0.365,0.519,0.051,0.064 121 | 120,Cambodia,4.433,0.549,1.088,0.457,0.696,0.256,0.065 122 | 121,Burkina Faso,4.424,0.314,1.097,0.254,0.312,0.175,0.128 123 | 122,Egypt,4.419,0.885,1.025,0.553,0.312,0.092,0.107 124 | 123,Mozambique,4.417,0.198,0.902,0.173,0.531,0.206,0.158 125 | 124,Kenya,4.410,0.493,1.048,0.454,0.504,0.352,0.055 126 | 125,Zambia,4.377,0.562,1.047,0.295,0.503,0.221,0.082 127 | 126,Mauritania,4.356,0.557,1.245,0.292,0.129,0.134,0.093 128 | 127,Ethiopia,4.350,0.308,0.950,0.391,0.452,0.220,0.146 129 | 128,Georgia,4.340,0.853,0.592,0.643,0.375,0.038,0.215 130 | 129,Armenia,4.321,0.816,0.990,0.666,0.260,0.077,0.028 131 | 130,Myanmar,4.308,0.682,1.174,0.429,0.580,0.598,0.178 132 | 131,Chad,4.301,0.358,0.907,0.053,0.189,0.181,0.060 133 | 132,Congo (Kinshasa),4.245,0.069,1.136,0.204,0.312,0.197,0.052 134 | 133,India,4.190,0.721,0.747,0.485,0.539,0.172,0.093 135 | 134,Niger,4.166,0.131,0.867,0.221,0.390,0.175,0.099 136 | 135,Uganda,4.161,0.322,1.090,0.237,0.450,0.259,0.061 137 | 136,Benin,4.141,0.378,0.372,0.240,0.440,0.163,0.067 138 | 137,Sudan,4.139,0.605,1.240,0.312,0.016,0.134,0.082 139 | 138,Ukraine,4.103,0.793,1.413,0.609,0.163,0.187,0.011 140 | 139,Togo,3.999,0.259,0.474,0.253,0.434,0.158,0.101 141 | 140,Guinea,3.964,0.344,0.792,0.211,0.394,0.185,0.094 142 | 141,Lesotho,3.808,0.472,1.215,0.079,0.423,0.116,0.112 143 | 142,Angola,3.795,0.730,1.125,0.269,0.000,0.079,0.061 144 | 143,Madagascar,3.774,0.262,0.908,0.402,0.221,0.155,0.049 145 | 144,Zimbabwe,3.692,0.357,1.094,0.248,0.406,0.132,0.099 146 | 145,Afghanistan,3.632,0.332,0.537,0.255,0.085,0.191,0.036 147 | 146,Botswana,3.590,1.017,1.174,0.417,0.557,0.042,0.092 148 | 147,Malawi,3.587,0.186,0.541,0.306,0.531,0.210,0.080 149 | 148,Haiti,3.582,0.315,0.714,0.289,0.025,0.392,0.104 150 | 149,Liberia,3.495,0.076,0.858,0.267,0.419,0.206,0.030 151 | 150,Syria,3.462,0.689,0.382,0.539,0.088,0.376,0.144 152 | 151,Rwanda,3.408,0.332,0.896,0.400,0.636,0.200,0.444 153 | 152,Yemen,3.355,0.442,1.073,0.343,0.244,0.083,0.064 154 | 153,Tanzania,3.303,0.455,0.991,0.381,0.481,0.270,0.097 155 | 154,South Sudan,3.254,0.337,0.608,0.177,0.112,0.224,0.106 156 | 155,Central African Republic,3.083,0.024,0.000,0.010,0.305,0.218,0.038 157 | 156,Burundi,2.905,0.091,0.627,0.145,0.065,0.149,0.076 158 | -------------------------------------------------------------------------------- /data-raw/2019.csv: -------------------------------------------------------------------------------- 1 | Overall rank,Country or region,Score,GDP per capita,Social support,Healthy life expectancy,Freedom to make life choices,Generosity,Perceptions of corruption 2 | 1,Finland,7.769,1.340,1.587,0.986,0.596,0.153,0.393 3 | 2,Denmark,7.600,1.383,1.573,0.996,0.592,0.252,0.410 4 | 3,Norway,7.554,1.488,1.582,1.028,0.603,0.271,0.341 5 | 4,Iceland,7.494,1.380,1.624,1.026,0.591,0.354,0.118 6 | 5,Netherlands,7.488,1.396,1.522,0.999,0.557,0.322,0.298 7 | 6,Switzerland,7.480,1.452,1.526,1.052,0.572,0.263,0.343 8 | 7,Sweden,7.343,1.387,1.487,1.009,0.574,0.267,0.373 9 | 8,New Zealand,7.307,1.303,1.557,1.026,0.585,0.330,0.380 10 | 9,Canada,7.278,1.365,1.505,1.039,0.584,0.285,0.308 11 | 10,Austria,7.246,1.376,1.475,1.016,0.532,0.244,0.226 12 | 11,Australia,7.228,1.372,1.548,1.036,0.557,0.332,0.290 13 | 12,Costa Rica,7.167,1.034,1.441,0.963,0.558,0.144,0.093 14 | 13,Israel,7.139,1.276,1.455,1.029,0.371,0.261,0.082 15 | 14,Luxembourg,7.090,1.609,1.479,1.012,0.526,0.194,0.316 16 | 15,United Kingdom,7.054,1.333,1.538,0.996,0.450,0.348,0.278 17 | 16,Ireland,7.021,1.499,1.553,0.999,0.516,0.298,0.310 18 | 17,Germany,6.985,1.373,1.454,0.987,0.495,0.261,0.265 19 | 18,Belgium,6.923,1.356,1.504,0.986,0.473,0.160,0.210 20 | 19,United States,6.892,1.433,1.457,0.874,0.454,0.280,0.128 21 | 20,Czech Republic,6.852,1.269,1.487,0.920,0.457,0.046,0.036 22 | 21,United Arab Emirates,6.825,1.503,1.310,0.825,0.598,0.262,0.182 23 | 22,Malta,6.726,1.300,1.520,0.999,0.564,0.375,0.151 24 | 23,Mexico,6.595,1.070,1.323,0.861,0.433,0.074,0.073 25 | 24,France,6.592,1.324,1.472,1.045,0.436,0.111,0.183 26 | 25,Taiwan,6.446,1.368,1.430,0.914,0.351,0.242,0.097 27 | 26,Chile,6.444,1.159,1.369,0.920,0.357,0.187,0.056 28 | 27,Guatemala,6.436,0.800,1.269,0.746,0.535,0.175,0.078 29 | 28,Saudi Arabia,6.375,1.403,1.357,0.795,0.439,0.080,0.132 30 | 29,Qatar,6.374,1.684,1.313,0.871,0.555,0.220,0.167 31 | 30,Spain,6.354,1.286,1.484,1.062,0.362,0.153,0.079 32 | 31,Panama,6.321,1.149,1.442,0.910,0.516,0.109,0.054 33 | 32,Brazil,6.300,1.004,1.439,0.802,0.390,0.099,0.086 34 | 33,Uruguay,6.293,1.124,1.465,0.891,0.523,0.127,0.150 35 | 34,Singapore,6.262,1.572,1.463,1.141,0.556,0.271,0.453 36 | 35,El Salvador,6.253,0.794,1.242,0.789,0.430,0.093,0.074 37 | 36,Italy,6.223,1.294,1.488,1.039,0.231,0.158,0.030 38 | 37,Bahrain,6.199,1.362,1.368,0.871,0.536,0.255,0.110 39 | 38,Slovakia,6.198,1.246,1.504,0.881,0.334,0.121,0.014 40 | 39,Trinidad & Tobago,6.192,1.231,1.477,0.713,0.489,0.185,0.016 41 | 40,Poland,6.182,1.206,1.438,0.884,0.483,0.117,0.050 42 | 41,Uzbekistan,6.174,0.745,1.529,0.756,0.631,0.322,0.240 43 | 42,Lithuania,6.149,1.238,1.515,0.818,0.291,0.043,0.042 44 | 43,Colombia,6.125,0.985,1.410,0.841,0.470,0.099,0.034 45 | 44,Slovenia,6.118,1.258,1.523,0.953,0.564,0.144,0.057 46 | 45,Nicaragua,6.105,0.694,1.325,0.835,0.435,0.200,0.127 47 | 46,Kosovo,6.100,0.882,1.232,0.758,0.489,0.262,0.006 48 | 47,Argentina,6.086,1.092,1.432,0.881,0.471,0.066,0.050 49 | 48,Romania,6.070,1.162,1.232,0.825,0.462,0.083,0.005 50 | 49,Cyprus,6.046,1.263,1.223,1.042,0.406,0.190,0.041 51 | 50,Ecuador,6.028,0.912,1.312,0.868,0.498,0.126,0.087 52 | 51,Kuwait,6.021,1.500,1.319,0.808,0.493,0.142,0.097 53 | 52,Thailand,6.008,1.050,1.409,0.828,0.557,0.359,0.028 54 | 53,Latvia,5.940,1.187,1.465,0.812,0.264,0.075,0.064 55 | 54,South Korea,5.895,1.301,1.219,1.036,0.159,0.175,0.056 56 | 55,Estonia,5.893,1.237,1.528,0.874,0.495,0.103,0.161 57 | 56,Jamaica,5.890,0.831,1.478,0.831,0.490,0.107,0.028 58 | 57,Mauritius,5.888,1.120,1.402,0.798,0.498,0.215,0.060 59 | 58,Japan,5.886,1.327,1.419,1.088,0.445,0.069,0.140 60 | 59,Honduras,5.860,0.642,1.236,0.828,0.507,0.246,0.078 61 | 60,Kazakhstan,5.809,1.173,1.508,0.729,0.410,0.146,0.096 62 | 61,Bolivia,5.779,0.776,1.209,0.706,0.511,0.137,0.064 63 | 62,Hungary,5.758,1.201,1.410,0.828,0.199,0.081,0.020 64 | 63,Paraguay,5.743,0.855,1.475,0.777,0.514,0.184,0.080 65 | 64,Northern Cyprus,5.718,1.263,1.252,1.042,0.417,0.191,0.162 66 | 65,Peru,5.697,0.960,1.274,0.854,0.455,0.083,0.027 67 | 66,Portugal,5.693,1.221,1.431,0.999,0.508,0.047,0.025 68 | 67,Pakistan,5.653,0.677,0.886,0.535,0.313,0.220,0.098 69 | 68,Russia,5.648,1.183,1.452,0.726,0.334,0.082,0.031 70 | 69,Philippines,5.631,0.807,1.293,0.657,0.558,0.117,0.107 71 | 70,Serbia,5.603,1.004,1.383,0.854,0.282,0.137,0.039 72 | 71,Moldova,5.529,0.685,1.328,0.739,0.245,0.181,0.000 73 | 72,Libya,5.525,1.044,1.303,0.673,0.416,0.133,0.152 74 | 73,Montenegro,5.523,1.051,1.361,0.871,0.197,0.142,0.080 75 | 74,Tajikistan,5.467,0.493,1.098,0.718,0.389,0.230,0.144 76 | 75,Croatia,5.432,1.155,1.266,0.914,0.296,0.119,0.022 77 | 76,Hong Kong,5.430,1.438,1.277,1.122,0.440,0.258,0.287 78 | 77,Dominican Republic,5.425,1.015,1.401,0.779,0.497,0.113,0.101 79 | 78,Bosnia and Herzegovina,5.386,0.945,1.212,0.845,0.212,0.263,0.006 80 | 79,Turkey,5.373,1.183,1.360,0.808,0.195,0.083,0.106 81 | 80,Malaysia,5.339,1.221,1.171,0.828,0.508,0.260,0.024 82 | 81,Belarus,5.323,1.067,1.465,0.789,0.235,0.094,0.142 83 | 82,Greece,5.287,1.181,1.156,0.999,0.067,0.000,0.034 84 | 83,Mongolia,5.285,0.948,1.531,0.667,0.317,0.235,0.038 85 | 84,North Macedonia,5.274,0.983,1.294,0.838,0.345,0.185,0.034 86 | 85,Nigeria,5.265,0.696,1.111,0.245,0.426,0.215,0.041 87 | 86,Kyrgyzstan,5.261,0.551,1.438,0.723,0.508,0.300,0.023 88 | 87,Turkmenistan,5.247,1.052,1.538,0.657,0.394,0.244,0.028 89 | 88,Algeria,5.211,1.002,1.160,0.785,0.086,0.073,0.114 90 | 89,Morocco,5.208,0.801,0.782,0.782,0.418,0.036,0.076 91 | 90,Azerbaijan,5.208,1.043,1.147,0.769,0.351,0.035,0.182 92 | 91,Lebanon,5.197,0.987,1.224,0.815,0.216,0.166,0.027 93 | 92,Indonesia,5.192,0.931,1.203,0.660,0.491,0.498,0.028 94 | 93,China,5.191,1.029,1.125,0.893,0.521,0.058,0.100 95 | 94,Vietnam,5.175,0.741,1.346,0.851,0.543,0.147,0.073 96 | 95,Bhutan,5.082,0.813,1.321,0.604,0.457,0.370,0.167 97 | 96,Cameroon,5.044,0.549,0.910,0.331,0.381,0.187,0.037 98 | 97,Bulgaria,5.011,1.092,1.513,0.815,0.311,0.081,0.004 99 | 98,Ghana,4.996,0.611,0.868,0.486,0.381,0.245,0.040 100 | 99,Ivory Coast,4.944,0.569,0.808,0.232,0.352,0.154,0.090 101 | 100,Nepal,4.913,0.446,1.226,0.677,0.439,0.285,0.089 102 | 101,Jordan,4.906,0.837,1.225,0.815,0.383,0.110,0.130 103 | 102,Benin,4.883,0.393,0.437,0.397,0.349,0.175,0.082 104 | 103,Congo (Brazzaville),4.812,0.673,0.799,0.508,0.372,0.105,0.093 105 | 104,Gabon,4.799,1.057,1.183,0.571,0.295,0.043,0.055 106 | 105,Laos,4.796,0.764,1.030,0.551,0.547,0.266,0.164 107 | 106,South Africa,4.722,0.960,1.351,0.469,0.389,0.130,0.055 108 | 107,Albania,4.719,0.947,0.848,0.874,0.383,0.178,0.027 109 | 108,Venezuela,4.707,0.960,1.427,0.805,0.154,0.064,0.047 110 | 109,Cambodia,4.700,0.574,1.122,0.637,0.609,0.232,0.062 111 | 110,Palestinian Territories,4.696,0.657,1.247,0.672,0.225,0.103,0.066 112 | 111,Senegal,4.681,0.450,1.134,0.571,0.292,0.153,0.072 113 | 112,Somalia,4.668,0.000,0.698,0.268,0.559,0.243,0.270 114 | 113,Namibia,4.639,0.879,1.313,0.477,0.401,0.070,0.056 115 | 114,Niger,4.628,0.138,0.774,0.366,0.318,0.188,0.102 116 | 115,Burkina Faso,4.587,0.331,1.056,0.380,0.255,0.177,0.113 117 | 116,Armenia,4.559,0.850,1.055,0.815,0.283,0.095,0.064 118 | 117,Iran,4.548,1.100,0.842,0.785,0.305,0.270,0.125 119 | 118,Guinea,4.534,0.380,0.829,0.375,0.332,0.207,0.086 120 | 119,Georgia,4.519,0.886,0.666,0.752,0.346,0.043,0.164 121 | 120,Gambia,4.516,0.308,0.939,0.428,0.382,0.269,0.167 122 | 121,Kenya,4.509,0.512,0.983,0.581,0.431,0.372,0.053 123 | 122,Mauritania,4.490,0.570,1.167,0.489,0.066,0.106,0.088 124 | 123,Mozambique,4.466,0.204,0.986,0.390,0.494,0.197,0.138 125 | 124,Tunisia,4.461,0.921,1.000,0.815,0.167,0.059,0.055 126 | 125,Bangladesh,4.456,0.562,0.928,0.723,0.527,0.166,0.143 127 | 126,Iraq,4.437,1.043,0.980,0.574,0.241,0.148,0.089 128 | 127,Congo (Kinshasa),4.418,0.094,1.125,0.357,0.269,0.212,0.053 129 | 128,Mali,4.390,0.385,1.105,0.308,0.327,0.153,0.052 130 | 129,Sierra Leone,4.374,0.268,0.841,0.242,0.309,0.252,0.045 131 | 130,Sri Lanka,4.366,0.949,1.265,0.831,0.470,0.244,0.047 132 | 131,Myanmar,4.360,0.710,1.181,0.555,0.525,0.566,0.172 133 | 132,Chad,4.350,0.350,0.766,0.192,0.174,0.198,0.078 134 | 133,Ukraine,4.332,0.820,1.390,0.739,0.178,0.187,0.010 135 | 134,Ethiopia,4.286,0.336,1.033,0.532,0.344,0.209,0.100 136 | 135,Swaziland,4.212,0.811,1.149,0.000,0.313,0.074,0.135 137 | 136,Uganda,4.189,0.332,1.069,0.443,0.356,0.252,0.060 138 | 137,Egypt,4.166,0.913,1.039,0.644,0.241,0.076,0.067 139 | 138,Zambia,4.107,0.578,1.058,0.426,0.431,0.247,0.087 140 | 139,Togo,4.085,0.275,0.572,0.410,0.293,0.177,0.085 141 | 140,India,4.015,0.755,0.765,0.588,0.498,0.200,0.085 142 | 141,Liberia,3.975,0.073,0.922,0.443,0.370,0.233,0.033 143 | 142,Comoros,3.973,0.274,0.757,0.505,0.142,0.275,0.078 144 | 143,Madagascar,3.933,0.274,0.916,0.555,0.148,0.169,0.041 145 | 144,Lesotho,3.802,0.489,1.169,0.168,0.359,0.107,0.093 146 | 145,Burundi,3.775,0.046,0.447,0.380,0.220,0.176,0.180 147 | 146,Zimbabwe,3.663,0.366,1.114,0.433,0.361,0.151,0.089 148 | 147,Haiti,3.597,0.323,0.688,0.449,0.026,0.419,0.110 149 | 148,Botswana,3.488,1.041,1.145,0.538,0.455,0.025,0.100 150 | 149,Syria,3.462,0.619,0.378,0.440,0.013,0.331,0.141 151 | 150,Malawi,3.410,0.191,0.560,0.495,0.443,0.218,0.089 152 | 151,Yemen,3.380,0.287,1.163,0.463,0.143,0.108,0.077 153 | 152,Rwanda,3.334,0.359,0.711,0.614,0.555,0.217,0.411 154 | 153,Tanzania,3.231,0.476,0.885,0.499,0.417,0.276,0.147 155 | 154,Afghanistan,3.203,0.350,0.517,0.361,0.000,0.158,0.025 156 | 155,Central African Republic,3.083,0.026,0.000,0.105,0.225,0.235,0.035 157 | 156,South Sudan,2.853,0.306,0.575,0.295,0.010,0.202,0.091 158 | -------------------------------------------------------------------------------- /data-raw/LICENSE.txt: -------------------------------------------------------------------------------- 1 | LICENSE: CC0: Public Domain 2 | SOURCE: https://www.kaggle.com/unsdsn/world-happiness -------------------------------------------------------------------------------- /data-raw/use_data.R: -------------------------------------------------------------------------------- 1 | df2015 <- read.csv("2015.csv") 2 | df2016 <- read.csv("2016.csv") 3 | df2017 <- read.csv("2017.csv") 4 | df2018 <- read.csv("2018.csv") 5 | df2019 <- read.csv("2019.csv") 6 | 7 | colnames(df2015) 8 | colnames(df2016) 9 | colnames(df2017) 10 | colnames(df2018) 11 | colnames(df2019) 12 | 13 | library(dplyr) 14 | library(DataExplorer) 15 | 16 | colnames(df2015) <- c("country", "region", "rank", "score", "misc1", "gdp_per_capita", "social_support", "healthy_life_expectancy", "freedom_life_choices", "perceptions_of_corruption", "generosity", "misc3") 17 | rownames(df2015) <- paste0(df2015$country, "_2015") 18 | df2015_clean <- df2015 %>% select("score", "gdp_per_capita", "social_support", "healthy_life_expectancy", "freedom_life_choices", "generosity", "perceptions_of_corruption") 19 | plot_histogram(df2015_clean) 20 | 21 | colnames(df2016) <- c("country", "region", "rank", "score", "misc1", "misc2", "gdp_per_capita", "social_support", "healthy_life_expectancy", "freedom_life_choices", "generosity", "perceptions_of_corruption", "misc3") 22 | rownames(df2016) <- paste0(df2016$country, "_2016") 23 | df2016_clean <- df2016 %>% select("score", "gdp_per_capita", "social_support", "healthy_life_expectancy", "freedom_life_choices", "generosity", "perceptions_of_corruption") 24 | plot_histogram(df2016_clean) 25 | 26 | colnames(df2017) <- c("country", "rank", "score", "misc1", "misc2", "gdp_per_capita", "social_support", "healthy_life_expectancy", "freedom_life_choices", "generosity", "perceptions_of_corruption", "misc3") 27 | rownames(df2017) <- paste0(df2017$country, "_2017") 28 | df2017_clean <- df2017 %>% select("score", "gdp_per_capita", "social_support", "healthy_life_expectancy", "freedom_life_choices", "generosity", "perceptions_of_corruption") 29 | plot_histogram(df2017_clean) 30 | 31 | colnames(df2018) <- c("rank", "country", "score", "gdp_per_capita", "social_support", "healthy_life_expectancy", "freedom_life_choices", "generosity", "perceptions_of_corruption") 32 | df2018 <- df2018[df2018$perceptions_of_corruption != "N/A", ] 33 | df2018$perceptions_of_corruption <- as.numeric(df2018$perceptions_of_corruption) 34 | rownames(df2018) <- paste0(df2018$country, "_2018") 35 | df2018_clean <- df2018 %>% select("score", "gdp_per_capita", "social_support", "healthy_life_expectancy", "freedom_life_choices", "generosity", "perceptions_of_corruption") 36 | plot_histogram(df2018_clean) 37 | 38 | colnames(df2019) <- c("rank", "country", "score", "gdp_per_capita", "social_support", "healthy_life_expectancy", "freedom_life_choices", "generosity", "perceptions_of_corruption") 39 | rownames(df2019) <- paste0(df2019$country) 40 | df2019_clean <- df2019 %>% select("score", "gdp_per_capita", "social_support", "healthy_life_expectancy", "freedom_life_choices", "generosity", "perceptions_of_corruption") 41 | plot_histogram(df2019_clean) 42 | 43 | happiness_train <- rbind(df2015_clean, df2016_clean, df2017_clean, df2018_clean) 44 | happiness_test <- df2019_clean 45 | 46 | usethis::use_data(happiness_train, overwrite = TRUE) 47 | usethis::use_data(happiness_test, overwrite = TRUE) -------------------------------------------------------------------------------- /data/happiness_test.rda: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ModelOriented/modelStudio/62d23a802f88c5efc98b285f08b4ccf97049a63e/data/happiness_test.rda -------------------------------------------------------------------------------- /data/happiness_train.rda: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ModelOriented/modelStudio/62d23a802f88c5efc98b285f08b4ccf97049a63e/data/happiness_train.rda -------------------------------------------------------------------------------- /inst/CITATION: -------------------------------------------------------------------------------- 1 | bibentry(key = "modelStudio", 2 | bibtype = "Article", 3 | author = c( 4 | person("Hubert", "Baniecki"), 5 | person("Przemyslaw", "Biecek") 6 | ), 7 | title = "{modelStudio}: Interactive Studio with Explanations for {ML} Predictive Models", 8 | doi = "10.21105/joss.01798", 9 | url = "https://doi.org/10.21105/joss.01798", 10 | year = "2019", 11 | month = "Nov", 12 | volume = "4", 13 | number = "43", 14 | pages = "1798", 15 | publisher = "The Open Journal", 16 | journal = "Journal of Open Source Software" 17 | ) 18 | 19 | -------------------------------------------------------------------------------- /inst/WORDLIST: -------------------------------------------------------------------------------- 1 | behaviour 2 | cheatsheet 3 | CheatSheet 4 | DALEX 5 | DALEXtra 6 | DrWhy 7 | eda 8 | EDA 9 | explainer 10 | explainers 11 | eXtrAI 12 | FeatureDistribution 13 | Fira 14 | iBreakDown 15 | JOSS 16 | js 17 | mlr 18 | OSX 19 | px 20 | RStudio 21 | scikit 22 | serverless 23 | SHAP 24 | suppressWarnings 25 | updateModelStudio 26 | xgboost 27 | tensorflow 28 | Shapley 29 | cran 30 | CRAN 31 | MLPClassifier 32 | keras 33 | lightGBM 34 | customizable 35 | customisable 36 | BD 37 | FI 38 | SV 39 | -------------------------------------------------------------------------------- /inst/d3js/d3-interpolate-path.min.js: -------------------------------------------------------------------------------- 1 | /* https://github.com/pbeshai/d3-interpolate-path Copyright 2016, Peter Beshai */ 2 | !function(t,e){"object"==typeof exports&&"undefined"!=typeof module?e(exports):"function"==typeof define&&define.amd?define(["exports"],e):e((t=t||self).d3=t.d3||{})}(this,function(t){"use strict";function i(){return(i=Object.assign||function(t){for(var e=1;ed.length?d=n(d,b,e):b.lengthM&&(r+=M),t instanceof Date?new Date(r):r}if(z){var n=e.scan(z.map(function(e){return Math.abs(t-e)}));return z[n]}return t}function N(t,e){(k[0]!==t[0]||k.length>1&&k[1]!==t[1])&&(k=t,e&&P.call("onchange",I,1===t.length?t[0]:t),J())}function S(t,e){R&&((e=void 0!==e&&e)?(R.selectAll(".parameter-value").data(t).transition().ease(l.easeQuadOut).duration(s).attr("transform",function(t){return U(h(t))}).select(".handle").attr("aria-valuenow",function(t){return t}),O&&C.transition().ease(l.easeQuadOut).duration(s).attr(H+"1",1===k.length?h.range()[0]-B*c:h(t[0])).attr(H+"2",1===k.length?h(t[0]):h(t[1]))):(R.selectAll(".parameter-value").data(t).attr("transform",function(t){return U(h(t))}).select(".handle").attr("aria-valuenow",function(t){return t}),O&&C.attr(H+"1",1===k.length?h.range()[0]-B*c:h(t[0])).attr(H+"2",1===k.length?h(t[0]):h(t[1]))),b&&G.text(V(t[0])))}return h&&(y=[e.min(h.domain()),e.max(h.domain())],t===d||t===m?w=e.max(h.range())-e.min(h.range()):A=e.max(h.range())-e.min(h.range()),h=h.clamp(!0)),I.min=function(t){return arguments.length?(y[0]=t,I):y[0]},I.max=function(t){return arguments.length?(y[1]=t,I):y[1]},I.domain=function(t){return arguments.length?(y=t,I):y},I.width=function(t){return arguments.length?(w=t,I):w},I.height=function(t){return arguments.length?(A=t,I):A},I.tickFormat=function(t){return arguments.length?(L=t,I):L},I.displayFormat=function(t){return arguments.length?(V=t,I):V},I.ticks=function(t){return arguments.length?(F=t,I):F},I.value=function(t){if(!arguments.length)return 1===k.length?k[0]:k;var e=Array.isArray(t)?t:[t];e.sort(function(t,e){return t-e});var a=e.map(h).map(T).map(h.invert).map(K);return S(a,!0),N(a,!0),I},I.silentValue=function(t){if(!arguments.length)return 1===k.length?k[0]:k;var e=Array.isArray(t)?t:[t];e.sort(function(t,e){return t-e});var a=e.map(h).map(T).map(h.invert).map(K);return S(a,!1),N(a,!1),I},I.default=function(t){if(!arguments.length)return 1===x.length?x[0]:x;var e=Array.isArray(t)?t:[t];return e.sort(function(t,e){return t-e}),x=e,k=e,I},I.step=function(t){return arguments.length?(M=t,I):M},I.tickValues=function(t){return arguments.length?(q=t,I):q},I.marks=function(t){return arguments.length?(z=t,I):z},I.handle=function(t){return arguments.length?(D=t,I):D},I.displayValue=function(t){return arguments.length?(b=t,I):b},I.fill=function(t){return arguments.length?(O=t,I):O},I.on=function(){var t=P.on.apply(P,arguments);return t===P?I:t},I}t.sliderHorizontal=function(t){return h(m,t)},t.sliderVertical=function(t){return h(p,t)},t.sliderTop=function(t){return h(d,t)},t.sliderRight=function(t){return h(f,t)},t.sliderBottom=function(t){return h(m,t)},t.sliderLeft=function(t){return h(p,t)},Object.defineProperty(t,"__esModule",{value:!0})}); 3 | -------------------------------------------------------------------------------- /inst/d3js/d3-tip.js: -------------------------------------------------------------------------------- 1 | /* d3.tip Copyright (c) 2013 Justin Palmer Tooltips for d3.js SVG visualizations */ 2 | /// MADE SOME CHANGES 3 | 4 | d3.functor = function functor(v) { 5 | return typeof v === "function" ? v : function() { 6 | return v; 7 | }; 8 | }; 9 | 10 | d3.tip = function(widget_id) { 11 | 12 | var direction = d3_tip_direction, 13 | offset = d3_tip_offset, 14 | html = d3_tip_html, 15 | node = initNode(), 16 | svg = null, 17 | point = null, 18 | target = null, 19 | widget_id = widget_id; 20 | 21 | function tip(vis) { 22 | svg = getSVGNode(vis) 23 | point = svg.createSVGPoint() 24 | document.getElementById(widget_id).appendChild(node) 25 | } 26 | 27 | // Public - show the tooltip on the screen 28 | // 29 | // Returns a tip 30 | tip.show = function() { 31 | var args = Array.prototype.slice.call(arguments) 32 | if(args[args.length - 1] instanceof SVGElement) target = args.pop() 33 | 34 | var content = html.apply(this, args), 35 | poffset = offset.apply(this, args), 36 | dir = direction.apply(this, args), 37 | nodel = getNodeEl(), 38 | i = directions.length, 39 | coords, 40 | scrollTop = document.documentElement.scrollTop || document.body.scrollTop, 41 | scrollLeft = document.documentElement.scrollLeft || document.body.scrollLeft 42 | 43 | /// unclass all directions 44 | while(i--) nodel.classed(directions[i], false) 45 | 46 | ////////////////////////////////::::::::://////////////////////////////// 47 | // make sure that tip is pointing right direction (not outside of svg) \\ 48 | // 'n' means that tooltip will go north from pointer 49 | 50 | // do not move this code V 51 | nodel.html(content) 52 | .style('position', 'fixed') 53 | .style('opacity', .8) 54 | .style('pointer-events', 'all') 55 | // do not move this code ^ 56 | 57 | var tdir = dir; 58 | 59 | var divDim = node.getBoundingClientRect(), 60 | svgDim = svg.getBBox(); 61 | 62 | // 20 is for 2x r2d3 margin and 7 was added empiricaly 63 | var sh = 20 + svgDim.height;// + 7; // this is modelStudio plotHeight 64 | var sw = 20 + svgDim.width; 65 | var dh = divDim.height; 66 | var dw = divDim.width; 67 | var py = d3.event.pageY; 68 | var px = d3.event.pageX; 69 | 70 | // by default put tooltip 'n' 71 | var ttop = py - dh - 5; 72 | var tleft = px - dw/2; 73 | var tdir = "n"; 74 | var tpdd = '6px 6px 13px 6px'; 75 | 76 | if (px - dw/2 < 10) { 77 | tleft = px + 5; 78 | tdir = "ne"; 79 | tpdd = '6px 6px 11px 11px'; 80 | if (py - dh/2 < 10) { 81 | ttop = py - dh/2; 82 | tdir = "e"; 83 | tpdd = '6px 6px 6px 13px'; 84 | } else if (py - dh < 10) { 85 | ttop = py + 5; 86 | tdir = "se"; 87 | tpdd = '11px 6px 6px 11px'; 88 | } 89 | } else if (px + dw/2 > sw - 10) { 90 | tleft = px - dw - 5; 91 | ttop = py - dh - 5; 92 | tdir = "nw"; 93 | tpdd = '6px 11px 11px 6px'; 94 | if (py - dh/2 < 10) { 95 | ttop = py - dh/2; 96 | tdir = "w" 97 | tpdd = '6px 13px 6px 6px'; 98 | } else if (py - dh < 10) { 99 | ttop = py + 5; 100 | tdir = "sw"; 101 | tpdd = '11px 11px 6px 6px'; 102 | } 103 | // because description is to long FIXME/TODO: more cases 104 | if (tleft < 5) { 105 | ttop = py + 5; 106 | tleft = px - dw/2; 107 | tdir = "s"; 108 | tpdd = '13px 6px 6px 6px'; 109 | } 110 | } else if (py - dh < 10) { 111 | ttop = py + 5; 112 | tleft = px - dw/2; 113 | tdir = "s"; 114 | tpdd = '13px 6px 6px 6px'; 115 | } 116 | 117 | nodel.classed(tdir, true) 118 | .style('top', (ttop - scrollTop) + 'px') 119 | .style('left', (tleft - scrollLeft) + 'px') 120 | .style('padding', tpdd); 121 | 122 | ////////////////////////////////:::::::::///////////////////////////////// 123 | 124 | //safeguard 125 | if (dw == 0) { 126 | return tip.hide() 127 | } else { 128 | return tip 129 | } 130 | } 131 | 132 | // Public - hide the tooltip 133 | // 134 | // Returns a tip 135 | tip.hide = function() { 136 | var nodel = getNodeEl() 137 | nodel 138 | .style('opacity', 0) 139 | .style('pointer-events', 'none') 140 | return tip 141 | } 142 | 143 | // Public: Proxy attr calls to the d3 tip container. Sets or gets attribute value. 144 | // 145 | // n - name of the attribute 146 | // v - value of the attribute 147 | // 148 | // Returns tip or attribute value 149 | tip.attr = function(n, v) { 150 | if (arguments.length < 2 && typeof n === 'string') { 151 | return getNodeEl().attr(n) 152 | } else { 153 | var args = Array.prototype.slice.call(arguments) 154 | d3.selection.prototype.attr.apply(getNodeEl(), args) 155 | } 156 | 157 | return tip 158 | } 159 | 160 | // Public: Proxy style calls to the d3 tip container. Sets or gets a style value. 161 | // 162 | // n - name of the property 163 | // v - value of the property 164 | // 165 | // Returns tip or style property value 166 | tip.style = function(n, v) { 167 | // debugger; 168 | if (arguments.length < 2 && typeof n === 'string') { 169 | return getNodeEl().style(n) 170 | } else { 171 | var args = Array.prototype.slice.call(arguments); 172 | if (args.length === 1) { 173 | var styles = args[0]; 174 | Object.keys(styles).forEach(function(key) { 175 | return d3.selection.prototype.style.apply(getNodeEl(), [key, styles[key]]); 176 | }); 177 | } 178 | } 179 | 180 | return tip 181 | } 182 | 183 | // Public: Set or get the direction of the tooltip 184 | // 185 | // v - One of n(north), s(south), e(east), or w(west), nw(northwest), 186 | // sw(southwest), ne(northeast) or se(southeast) 187 | // 188 | // Returns tip or direction 189 | tip.direction = function(v) { 190 | if (!arguments.length) return direction 191 | direction = v == null ? v : d3.functor(v) 192 | 193 | return tip 194 | } 195 | 196 | // Public: Sets or gets the offset of the tip 197 | // 198 | // v - Array of [x, y] offset 199 | // 200 | // Returns offset or 201 | tip.offset = function(v) { 202 | if (!arguments.length) return offset 203 | offset = v == null ? v : d3.functor(v) 204 | 205 | return tip 206 | } 207 | 208 | // Public: sets or gets the html value of the tooltip 209 | // 210 | // v - String value of the tip 211 | // 212 | // Returns html value or tip 213 | tip.html = function(v) { 214 | if (!arguments.length) return html 215 | html = v == null ? v : d3.functor(v) 216 | 217 | return tip 218 | } 219 | 220 | // Public: destroys the tooltip and removes it from the DOM 221 | // 222 | // Returns a tip 223 | tip.destroy = function() { 224 | if(node) { 225 | getNodeEl().remove(); 226 | node = null; 227 | } 228 | return tip; 229 | } 230 | 231 | function d3_tip_direction() { return 'n' } 232 | function d3_tip_offset() { return [0, 0] } 233 | function d3_tip_html() { return ' ' } 234 | 235 | var direction_callbacks = { 236 | n: direction_n, 237 | s: direction_s, 238 | e: direction_e, 239 | w: direction_w, 240 | nw: direction_nw, 241 | ne: direction_ne, 242 | sw: direction_sw, 243 | se: direction_se 244 | }; 245 | 246 | var directions = Object.keys(direction_callbacks); 247 | 248 | function direction_n() { 249 | var bbox = getScreenBBox() 250 | return { 251 | top: bbox.n.y - node.offsetHeight, 252 | left: bbox.n.x - node.offsetWidth / 2 253 | } 254 | } 255 | 256 | function direction_s() { 257 | var bbox = getScreenBBox() 258 | return { 259 | top: bbox.s.y, 260 | left: bbox.s.x - node.offsetWidth / 2 261 | } 262 | } 263 | 264 | function direction_e() { 265 | var bbox = getScreenBBox() 266 | return { 267 | top: bbox.e.y - node.offsetHeight / 2, 268 | left: bbox.e.x 269 | } 270 | } 271 | 272 | function direction_w() { 273 | var bbox = getScreenBBox() 274 | return { 275 | top: bbox.w.y - node.offsetHeight / 2, 276 | left: bbox.w.x - node.offsetWidth 277 | } 278 | } 279 | 280 | function direction_nw() { 281 | var bbox = getScreenBBox() 282 | return { 283 | top: bbox.nw.y - node.offsetHeight, 284 | left: bbox.nw.x - node.offsetWidth 285 | } 286 | } 287 | 288 | function direction_ne() { 289 | var bbox = getScreenBBox() 290 | return { 291 | top: bbox.ne.y - node.offsetHeight, 292 | left: bbox.ne.x 293 | } 294 | } 295 | 296 | function direction_sw() { 297 | var bbox = getScreenBBox() 298 | return { 299 | top: bbox.sw.y, 300 | left: bbox.sw.x - node.offsetWidth 301 | } 302 | } 303 | 304 | function direction_se() { 305 | var bbox = getScreenBBox() 306 | return { 307 | top: bbox.se.y, 308 | left: bbox.e.x 309 | } 310 | } 311 | 312 | function initNode() { 313 | var node = d3.select(document.createElement('div')) 314 | node 315 | .style('position', 'fixed') 316 | .style('top', 0) 317 | .style('opacity', 0) 318 | .style('pointer-events', 'none') 319 | .style('box-sizing', 'border-box') 320 | .style('line-height', 1.1) 321 | .style('background', "#000000") 322 | .style('color', '#fff') 323 | .style('font-size', '14px') 324 | .style('font-family', "'Roboto Condensed', sans-serif"); 325 | 326 | return node.node() 327 | } 328 | 329 | 330 | function getSVGNode(el) { 331 | el = el.node() 332 | if(el.tagName.toLowerCase() === 'svg') 333 | return el 334 | 335 | return el.ownerSVGElement 336 | } 337 | 338 | function getNodeEl() { 339 | if(node === null) { 340 | node = initNode(); 341 | // re-add node to DOM 342 | document.getElementById(widget_id).appendChild(node); 343 | }; 344 | return d3.select(node); 345 | } 346 | 347 | // Returns an Object {n, s, e, w, nw, sw, ne, se} 348 | function getScreenBBox() { 349 | var targetel = target || d3.event.target; 350 | 351 | while ('undefined' === typeof targetel.getScreenCTM && 'undefined' === targetel.parentNode) { 352 | targetel = targetel.parentNode; 353 | } 354 | 355 | var bbox = {}, 356 | matrix = targetel.getScreenCTM(), 357 | tbbox = targetel.getBBox(), 358 | width = tbbox.width, 359 | height = tbbox.height, 360 | x = tbbox.x, 361 | y = tbbox.y 362 | 363 | point.x = x 364 | point.y = y 365 | bbox.nw = point.matrixTransform(matrix) 366 | point.x += width 367 | bbox.ne = point.matrixTransform(matrix) 368 | point.y += height 369 | bbox.se = point.matrixTransform(matrix) 370 | point.x -= width 371 | bbox.sw = point.matrixTransform(matrix) 372 | point.y -= height / 2 373 | bbox.w = point.matrixTransform(matrix) 374 | point.x += width 375 | bbox.e = point.matrixTransform(matrix) 376 | point.x -= width / 2 377 | point.y -= height / 2 378 | bbox.n = point.matrixTransform(matrix) 379 | point.y += height 380 | bbox.s = point.matrixTransform(matrix) 381 | 382 | return bbox 383 | } 384 | 385 | return tip 386 | }; 387 | -------------------------------------------------------------------------------- /inst/d3js/generateTooltipHtml.js: -------------------------------------------------------------------------------- 1 | //:\\\ here are functions for tooltips HTML //:\\\ 2 | 3 | function bdTooltipHtml(d) { 4 | var temp = "
"; 5 | temp += d.tooltipText; 6 | temp += "
"; 7 | return temp; 8 | } 9 | 10 | function cpStaticTooltipHtml(d) { 11 | // function formats tooltip text 12 | var temp = ""; 13 | for (var [k, v] of Object.entries(d)) { 14 | if (k === "yhat") { 15 | k = "prediction"; 16 | temp += "
" + k + ": " + v + "
"; 17 | temp += "
"; 18 | } else { 19 | temp += "
" + k + ": " + v + "
"; 20 | } 21 | } 22 | return temp; 23 | } 24 | 25 | function cpChangedTooltipHtml(d, addData) { 26 | // function formats tooltip text with update in red 27 | var temp = "
"; 28 | for (var [k, v] of Object.entries(addData)) { 29 | if (k === "yhat") { 30 | temp += "prediction:
"; 31 | temp += "- before" + ": " + v + "
"; 32 | temp += "- after" + ": " + "" + d.yhat + "
"; 33 | temp += "
"; 34 | } else if (k === d.vname) { 35 | temp += k + ": " + "" + d.xhat + "
"; 36 | } else { 37 | temp += k + ": " + v + "
"; 38 | } 39 | } 40 | temp += "
"; 41 | return temp; 42 | } 43 | 44 | function fiStaticTooltipHtml(d) { 45 | let sign; 46 | if (d.dropout_loss > d.full_model) sign = "+"; else sign = ""; 47 | var temp = "
" + "model loss after feature " + d.variable 48 | + "
" + "
" + 49 | " is permuted: " + Math.round(d.dropout_loss * 1000)/1000 50 | + "
" + "
" + "drop-out loss change: " + 51 | sign + Math.round((d.dropout_loss-d.full_model)*1000)/1000; 52 | return temp; 53 | } 54 | 55 | function pdStaticTooltipHtml(d, variableName, yMean) { 56 | // function formats tooltip text 57 | var temp = ""; 58 | for (var [k, v] of Object.entries(d)) { 59 | switch(k){ 60 | case "xhat": 61 | temp += "
" + variableName + ": " + v + "
"; 62 | break; 63 | case "yhat": 64 | temp += "
" + "average prediction" + ": " + v + "
"; 65 | break; 66 | case "vname": 67 | break; 68 | default: 69 | temp += "
" + k + ": " + v + "
"; 70 | break; 71 | } 72 | } 73 | 74 | temp += "
" + 75 | "mean observation prediction:" + 76 | "
" + yMean + "
"; 77 | return temp; 78 | } 79 | 80 | function adStaticTooltipHtml(d, variableName, yMean) { 81 | // function formats tooltip text 82 | var temp = ""; 83 | for (var [k, v] of Object.entries(d)) { 84 | switch(k){ 85 | case "xhat": 86 | temp += "
" + variableName + ": " + v + "
"; 87 | break; 88 | case "yhat": 89 | temp += "
" + "accumulated prediction" + ": " + v + "
"; 90 | break; 91 | case "vname": 92 | break; 93 | default: 94 | temp += "
" + k + ": " + v + "
"; 95 | break; 96 | } 97 | } 98 | 99 | temp += "
" + 100 | "mean observation prediction:" + 101 | "
" + yMean + "
"; 102 | return temp; 103 | } 104 | 105 | function descTooltipHtml(d) { 106 | var temp = "
"; 107 | temp += d.text; 108 | temp += "
"; 109 | return temp; 110 | } 111 | -------------------------------------------------------------------------------- /inst/d3js/hackHead.js: -------------------------------------------------------------------------------- 1 | // SPECIAL CAUTION NEEDED WHEN EDITING THIS FILE; MAY CAUSE FATAL ERROR 2 | 3 | function addCssToDocument(css) { 4 | // this function adds custom css to document head 5 | var style = document.createElement('style'); 6 | style.innerText = css; 7 | document.head.appendChild(style); 8 | } 9 | 10 | // load tip:after related css (for triangle) 11 | addCssToDocument("d3-tip:after{box-sizing:border-box;display:inline;font-size:10px;width:inherit;"+ 12 | "height:inherit;line-height:1;color:rgba(0,0,0,0.8);content:'\\25BC';position:absolute;text-align: center;}"+ 13 | ".d3-tip.n:after{content:'\\25BC';position:absolute;margin:0 0 0 0;top:100%;left:50%;text-align:center;transform:translate(-7px,-15px)}"+ 14 | ".d3-tip.e:after{content:'\\25C0';position:absolute;margin:0 0 0 0;top:50%;left:0%;text-align:center;transform:translate(0px,-7px)}"+ 15 | ".d3-tip.s:after{content:'\\25B2';position:absolute;margin:0 0 0 0;top:0%;left:50%;text-align:center;transform:translate(-7px,0px)}"+ 16 | ".d3-tip.w:after{content:'\\25B6';position:absolute;margin:0 0 0 0;top:50%;left:100%;text-align:center;transform:translate(-15px,-7px)}"+ 17 | ".d3-tip.se:after{content:'\\25E4';position:absolute;margin:0 0 0 0;top:0%;left:0%;text-align:center;transform:translate(1px,1px)}"+ 18 | ".d3-tip.ne:after{content:'\\25E3';position:absolute;margin:0 0 0 0;top:100%;left:0%;text-align:center;transform:translate(1px,-15px)}"+ 19 | ".d3-tip.sw:after{content:'\\25E5';position:absolute;margin:0 0 0 0;top:0%;left:100%;text-align:center;transform:translate(-15px,1px)}"+ 20 | ".d3-tip.nw:after{content:'\\25E2';position:absolute;margin:0 0 0 0;top:100%;left:100%;text-align:center;transform:translate(-14px,-16px)};"); 21 | 22 | // load Fira Sans fonts from googleapis 23 | addCssToDocument("@font-face{font-family:'Fira Sans';font-style:normal;font-weight:400;font-display:swap;src:local('Fira Sans Regular'),local('FiraSans-Regular'),url(https://fonts.gstatic.com/s/firasans/v10/va9E4kDNxMZdWfMOD5VvmYjLeTY.woff2) format('woff2');unicode-range:U+0100-024F,U+0259,U+1E00-1EFF,U+2020,U+20A0-20AB,U+20AD-20CF,U+2113,U+2C60-2C7F,U+A720-A7FF}@font-face{font-family:'Fira Sans';font-style:normal;font-weight:400;font-display:swap;src:local('Fira Sans Regular'),local('FiraSans-Regular'),url(https://fonts.gstatic.com/s/firasans/v10/va9E4kDNxMZdWfMOD5Vvl4jL.woff2) format('woff2');unicode-range:U+0000-00FF,U+0131,U+0152-0153,U+02BB-02BC,U+02C6,U+02DA,U+02DC,U+2000-206F,U+2074,U+20AC,U+2122,U+2191,U+2193,U+2212,U+2215,U+FEFF,U+FFFD}@font-face{font-family:'Fira Sans';font-style:normal;font-weight:500;font-display:swap;src:local('Fira Sans Medium'),local('FiraSans-Medium'),url(https://fonts.gstatic.com/s/firasans/v10/va9B4kDNxMZdWfMOD5VnZKveSBf6TF0.woff2) format('woff2');unicode-range:U+0100-024F,U+0259,U+1E00-1EFF,U+2020,U+20A0-20AB,U+20AD-20CF,U+2113,U+2C60-2C7F,U+A720-A7FF}@font-face{font-family:'Fira Sans';font-style:normal;font-weight:500;font-display:swap;src:local('Fira Sans Medium'),local('FiraSans-Medium'),url(https://fonts.gstatic.com/s/firasans/v10/va9B4kDNxMZdWfMOD5VnZKveRhf6.woff2) format('woff2');unicode-range:U+0000-00FF,U+0131,U+0152-0153,U+02BB-02BC,U+02C6,U+02DA,U+02DC,U+2000-206F,U+2074,U+20AC,U+2122,U+2191,U+2193,U+2212,U+2215,U+FEFF,U+FFFD}@font-face{font-family:'Fira Sans';font-style:normal;font-weight:600;font-display:swap;src:local('Fira Sans SemiBold'),local('FiraSans-SemiBold'),url(https://fonts.gstatic.com/s/firasans/v10/va9B4kDNxMZdWfMOD5VnSKzeSBf6TF0.woff2) format('woff2');unicode-range:U+0100-024F,U+0259,U+1E00-1EFF,U+2020,U+20A0-20AB,U+20AD-20CF,U+2113,U+2C60-2C7F,U+A720-A7FF}@font-face{font-family:'Fira Sans';font-style:normal;font-weight:600;font-display:swap;src:local('Fira Sans SemiBold'),local('FiraSans-SemiBold'),url(https://fonts.gstatic.com/s/firasans/v10/va9B4kDNxMZdWfMOD5VnSKzeRhf6.woff2) format('woff2');unicode-range:U+0000-00FF,U+0131,U+0152-0153,U+02BB-02BC,U+02C6,U+02DA,U+02DC,U+2000-206F,U+2074,U+20AC,U+2122,U+2191,U+2193,U+2212,U+2215,U+FEFF,U+FFFD}@font-face{font-family:'Fira Sans';font-style:normal;font-weight:700;font-display:swap;src:local('Fira Sans Bold'),local('FiraSans-Bold'),url(https://fonts.gstatic.com/s/firasans/v10/va9B4kDNxMZdWfMOD5VnLK3eSBf6TF0.woff2) format('woff2');unicode-range:U+0100-024F,U+0259,U+1E00-1EFF,U+2020,U+20A0-20AB,U+20AD-20CF,U+2113,U+2C60-2C7F,U+A720-A7FF}@font-face{font-family:'Fira Sans';font-style:normal;font-weight:700;font-display:swap;src:local('Fira Sans Bold'),local('FiraSans-Bold'),url(https://fonts.gstatic.com/s/firasans/v10/va9B4kDNxMZdWfMOD5VnLK3eRhf6.woff2) format('woff2');unicode-range:U+0000-00FF,U+0131,U+0152-0153,U+02BB-02BC,U+02C6,U+02DA,U+02DC,U+2000-206F,U+2074,U+20AC,U+2122,U+2191,U+2193,U+2212,U+2215,U+FEFF,U+FFFD}"); 24 | 25 | /*function ga() { 26 | var script = document.createElement('script'); 27 | script.setAttribute('src', 'https://www.googletagmanager.com/gtag/js?id=UA-160640242-1'); 28 | script.setAttribute('async', ''); 29 | document.head.appendChild(script); 30 | 31 | var script2 = document.createElement('script'); 32 | script2.innerText = "window.dataLayer = window.dataLayer || [];" + 33 | "function gtag(){dataLayer.push(arguments);}" + 34 | "gtag('js', new Date()); gtag('config', 'UA-160640242-1');"; 35 | document.head.appendChild(script2); 36 | }*/ 37 | 38 | -------------------------------------------------------------------------------- /inst/d3js/modelStudio.css: -------------------------------------------------------------------------------- 1 | text { 2 | font-family:'Fira Sans'; 3 | font-style:normal; 4 | fill: #371ea3; 5 | -webkit-user-select: none; 6 | -moz-user-select: none; 7 | -ms-user-select: none; 8 | -o-user-select: none; 9 | user-select: none; 10 | } 11 | 12 | .bigTitle { 13 | font-family:'Fira Sans'; 14 | font-style:normal; 15 | font-weight: 700; 16 | font-size: 18px; 17 | text-align: left; 18 | } 19 | 20 | .smallTitle { 21 | font-family:'Fira Sans'; 22 | font-style:normal; 23 | font-weight: 700; 24 | font-size: 13px; 25 | text-align: left; 26 | } 27 | 28 | .axisTitle { 29 | font-family:'Fira Sans'; 30 | font-style:normal; 31 | font-weight: 700; 32 | font-size: 13px; 33 | text-align: center; 34 | } 35 | 36 | .axisLabel { 37 | font-family:'Fira Sans'; 38 | font-style:normal; 39 | font-weight: 500; 40 | font-size: 11px; 41 | } 42 | 43 | .axisLabel .domain { 44 | stroke: #371ea3; 45 | } 46 | 47 | .legendTitle { 48 | font-family:'Fira Sans'; 49 | font-style: normal; 50 | font-weight: 500; 51 | font-size: 13px; 52 | text-align: left; 53 | } 54 | 55 | .legendLabel { 56 | font-family:'Fira Sans'; 57 | font-style: normal; 58 | font-weight: 500; 59 | font-size: 11px; 60 | text-align: left; 61 | } 62 | 63 | .legendBox { 64 | fill: #ceced9; 65 | opacity: 0.5; 66 | } 67 | 68 | .interceptLine { 69 | stroke: #371ea3; 70 | stroke-width: 1px; 71 | stroke-opacity: 1; 72 | stroke-linecap: "butt"; 73 | } 74 | 75 | .dotLine { 76 | fill: none; 77 | stroke: #371ea3; 78 | stroke-width: 0.7px; 79 | } 80 | 81 | .grid .tick line { 82 | /*stroke: #f0f0f4;*/ 83 | stroke: #ceced9; 84 | opacity: 0.5; 85 | stroke-width: 1px 86 | } 87 | 88 | .mainTitle { 89 | font-family:'Fira Sans'; 90 | font-style: normal; 91 | font-size: 25px; 92 | font-weight: 500; 93 | } 94 | 95 | .subTitle { 96 | font-family:'Fira Sans'; 97 | font-style: normal; 98 | font-size: 18px; 99 | font-weight: 500; 100 | dominant-baseline: hanging; 101 | } 102 | 103 | .mainLine { 104 | stroke: #371ea3; 105 | stroke-width: 1.5px; 106 | stroke-opacity: 1; 107 | stroke-linecap: butt; 108 | } 109 | 110 | .footerTitle { 111 | font-family:'Fira Sans'; 112 | font-style: normal; 113 | font-size: 12px; 114 | font-weight: 400; 115 | } 116 | 117 | .footerLine { 118 | stroke: #371ea3; 119 | stroke-width: 1px; 120 | stroke-opacity: 0.5; 121 | stroke-linecap: butt; 122 | } 123 | 124 | 125 | .enterChoiceButton { 126 | fill: #ceced9; 127 | opacity: 0.5; 128 | stroke:#371ea3; 129 | stroke-width:2; 130 | } 131 | 132 | .exitChoiceButton { 133 | fill: #FFFFFF; 134 | stroke:#371ea3; 135 | stroke-width:2; 136 | } 137 | 138 | .descriptionBox { 139 | fill: #ceced9; 140 | opacity: 0.5; 141 | } 142 | 143 | .descriptionLabel { 144 | font-family:'Fira Sans'; 145 | font-style: normal; 146 | fill: #371ea3; 147 | font-weight: 700; 148 | font-size: 14px; 149 | } 150 | -------------------------------------------------------------------------------- /inst/d3js/myTools.js: -------------------------------------------------------------------------------- 1 | //:\\\ usefull functions //:\\\ 2 | 3 | function getColors(n, type) { 4 | // get drWhy palette 5 | 6 | var temp = ["#8bdcbe", "#f05a71", "#371ea3", "#46bac2", "#ae2c87", "#ffa58c", "#4378bf"]; 7 | var ret = []; 8 | 9 | if (type == "bar") { 10 | switch (n) { 11 | case 1: 12 | return ["#46bac2"]; 13 | case 2: 14 | return ["#46bac2", "#4378bf"]; 15 | case 3: 16 | return ["#8bdcbe", "#4378bf", "#46bac2"]; 17 | case 4: 18 | return ["#46bac2", "#371ea3", "#8bdcbe", "#4378bf"]; 19 | case 5: 20 | return ["#8bdcbe", "#f05a71", "#371ea3", "#46bac2", "#ffa58c"]; 21 | case 6: 22 | return ["#8bdcbe", "#f05a71", "#371ea3", "#46bac2", "#ae2c87", "#ffa58c"]; 23 | case 7: 24 | return temp; 25 | default: 26 | for (var i = 0; i <= n%7; i++) { 27 | ret = ret.concat(temp); 28 | } 29 | return ret; 30 | } 31 | } else if (type == "line") { 32 | switch (n) { 33 | case 1: 34 | return ["#46bac2"]; 35 | case 2: 36 | return ["#8bdcbe", "#4378bf"]; 37 | case 3: 38 | return ["#8bdcbe", "#f05a71", "#4378bf"]; 39 | case 4: 40 | return ["#8bdcbe", "#f05a71", "#4378bf", "#ffa58c"]; 41 | case 5: 42 | return ["#8bdcbe", "#f05a71", "#4378bf", "#ae2c87", "#ffa58c"]; 43 | case 6: 44 | return ["#8bdcbe", "#f05a71", "#46bac2", "#ae2c87", "#ffa58c", "#4378bf"]; 45 | case 7: 46 | return temp; 47 | default: 48 | for (var j = 0; j <= n%7; j++) { 49 | ret = ret.concat(temp); 50 | } 51 | return ret; 52 | } 53 | } else if (type == "point") { 54 | switch (n) { 55 | default: 56 | return ["#371ea3", "#46bac2", "#ceced9"]; 57 | } 58 | } else if (type == "breakDown") { 59 | switch (n) { 60 | default: 61 | return ["#8bdcbe", "#f05a71", "#371ea3"]; 62 | } 63 | } 64 | } 65 | 66 | function getTickValues(domain) { 67 | // find 5 nice ticks with max and min - do better than d3 68 | 69 | var tickValues = d3.ticks(domain[0], domain[1],5); 70 | 71 | switch (tickValues.length) { 72 | case 3: 73 | tickValues.unshift(domain[0]); 74 | tickValues.push(domain[1]); 75 | break; 76 | 77 | case 4: 78 | if(Math.abs(domain[0] - tickValues[0]) < Math.abs(domain[1] - tickValues[3])){ 79 | tickValues.shift(); 80 | tickValues.unshift(domain[0]); 81 | tickValues.push(domain[1]); 82 | } else { 83 | tickValues.pop(); 84 | tickValues.push(domain[1]); 85 | tickValues.unshift(domain[0]); 86 | } 87 | break; 88 | 89 | case 5: 90 | tickValues.pop(); 91 | tickValues.shift(); 92 | tickValues.push(domain[1]); 93 | tickValues.unshift(domain[0]); 94 | break; 95 | 96 | case 6: 97 | if(Math.abs(domain[0] - tickValues[0]) < Math.abs(domain[1] - tickValues[5])){ 98 | tickValues.pop(); 99 | tickValues.shift(); 100 | tickValues.shift(); 101 | tickValues.push(domain[1]); 102 | tickValues.unshift(domain[0]); 103 | } else { 104 | tickValues.pop(); 105 | tickValues.pop(); 106 | tickValues.shift(); 107 | tickValues.push(domain[1]); 108 | tickValues.unshift(domain[0]); 109 | } 110 | break; 111 | 112 | case 7: 113 | tickValues.pop(); 114 | tickValues.pop(); 115 | tickValues.shift(); 116 | tickValues.shift(); 117 | tickValues.push(domain[1]); 118 | tickValues.unshift(domain[0]); 119 | break; 120 | 121 | case 8: 122 | if(Math.abs(domain[0] - tickValues[0]) < Math.abs(domain[1] - tickValues[7])){ 123 | tickValues.pop(); 124 | tickValues.pop(); 125 | tickValues.shift(); 126 | tickValues.shift(); 127 | tickValues.shift(); 128 | tickValues.push(domain[1]); 129 | tickValues.unshift(domain[0]); 130 | } else { 131 | tickValues.pop(); 132 | tickValues.pop(); 133 | tickValues.pop(); 134 | tickValues.shift(); 135 | tickValues.shift(); 136 | tickValues.push(domain[1]); 137 | tickValues.unshift(domain[0]); 138 | } 139 | break; 140 | } 141 | 142 | return tickValues; 143 | } 144 | 145 | function calculateTextWidth(text) { 146 | // calculate max width of 11px text array 147 | 148 | var temp = svg.selectAll() 149 | .data(text) 150 | .enter(); 151 | 152 | var textWidth = []; 153 | 154 | temp.append("text") 155 | .attr("class", "toRemove") 156 | .text(function(d) { return d;}) 157 | .style("font-size", "11px") 158 | .style('font-family', 'Fira Sans, sans-serif') 159 | .each(function(d,i) { 160 | var thisWidth = this.getComputedTextLength(); 161 | textWidth.push(thisWidth); 162 | }); 163 | 164 | svg.selectAll('.toRemove').remove(); 165 | temp.remove(); 166 | 167 | var maxLength = d3.max(textWidth); 168 | 169 | return maxLength; 170 | } 171 | 172 | function getTextWidth(text, fontSize, fontFace) { 173 | // calculate width of single text 174 | 175 | var canvas = document.createElement('canvas'); 176 | var context = canvas.getContext('2d'); 177 | context.font = fontSize + 'px ' + fontFace; 178 | return context.measureText(text).width; 179 | } 180 | 181 | function getMaxTextWidth(textArray, fontSize, fontFace) { 182 | let maxTextWidth = 0; 183 | for (let i = 0; i < textArray.length; i++) { 184 | let textWidth = getTextWidth(textArray[i], fontSize, fontFace) 185 | if (textWidth > maxTextWidth) { 186 | maxTextWidth = textWidth 187 | } 188 | } 189 | return maxTextWidth 190 | } 191 | 192 | function wrapText(text, width) { 193 | // this function wraps text 194 | text.each(function () { 195 | var text = d3.select(this).style('font-family', 'Fira Sans, sans-serif'), 196 | words = text.text().split(/\s+/).reverse(), 197 | word, 198 | line = [], 199 | lineNumber = 0, 200 | lineHeight = 1.05, // ems 201 | x = text.attr("x"), 202 | y = text.attr("y"), 203 | dy = 0, //parseFloat(text.attr("dy")), 204 | tspan = text.text(null) 205 | .append("tspan") 206 | .attr("x", x) 207 | .attr("y", y) 208 | .attr("dy", dy + "em"); 209 | while (word = words.pop()) { 210 | line.push(word); 211 | tspan.text(line.join(" ")); 212 | if (tspan.node().getComputedTextLength() > width) { 213 | line.pop(); 214 | tspan.text(line.join(" ")); 215 | line = [word]; 216 | tspan = text.append("tspan") 217 | .attr("x", x) 218 | .attr("y", y) 219 | .attr("dy", ++lineNumber * lineHeight + dy + "em") 220 | .text(word); 221 | } 222 | } 223 | }); 224 | } 225 | 226 | function wrapHtmlOutput(text) { 227 | var output = ""; 228 | text.replace(/\(?[A-Z][^\.]+[\.!\?]\)?/g, function (sentence) { 229 | output += (sentence + '
'); 230 | }); 231 | return output; 232 | } 233 | -------------------------------------------------------------------------------- /man/figures/cheatsheet.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ModelOriented/modelStudio/62d23a802f88c5efc98b285f08b4ccf97049a63e/man/figures/cheatsheet.png -------------------------------------------------------------------------------- /man/figures/controls.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ModelOriented/modelStudio/62d23a802f88c5efc98b285f08b4ccf97049a63e/man/figures/controls.png -------------------------------------------------------------------------------- /man/figures/demo_big.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ModelOriented/modelStudio/62d23a802f88c5efc98b285f08b4ccf97049a63e/man/figures/demo_big.gif -------------------------------------------------------------------------------- /man/figures/demo_small.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ModelOriented/modelStudio/62d23a802f88c5efc98b285f08b4ccf97049a63e/man/figures/demo_small.gif -------------------------------------------------------------------------------- /man/figures/logo.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ModelOriented/modelStudio/62d23a802f88c5efc98b285f08b4ccf97049a63e/man/figures/logo.gif -------------------------------------------------------------------------------- /man/figures/logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ModelOriented/modelStudio/62d23a802f88c5efc98b285f08b4ccf97049a63e/man/figures/logo.png -------------------------------------------------------------------------------- /man/figures/long.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ModelOriented/modelStudio/62d23a802f88c5efc98b285f08b4ccf97049a63e/man/figures/long.gif -------------------------------------------------------------------------------- /man/figures/misc.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ModelOriented/modelStudio/62d23a802f88c5efc98b285f08b4ccf97049a63e/man/figures/misc.png -------------------------------------------------------------------------------- /man/figures/short.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ModelOriented/modelStudio/62d23a802f88c5efc98b285f08b4ccf97049a63e/man/figures/short.gif -------------------------------------------------------------------------------- /man/happiness_train.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/data.R 3 | \docType{data} 4 | \name{happiness_train} 5 | \alias{happiness_train} 6 | \alias{happiness_test} 7 | \title{World Happiness Report} 8 | \format{ 9 | \code{happiness_train}: a data frame with 625 rows and 7 columns, \code{happiness_test}: a data frame with 156 rows and 7 columns 10 | } 11 | \usage{ 12 | data(happiness_train); data(happiness_test) 13 | } 14 | \description{ 15 | Datasets \code{happiness_train} and \code{happiness_test} are real data from the 16 | World Happiness Reports. Happiness is scored according to economic production, 17 | social support, etc. \code{happiness_train} accumulates the data from years 2015-2018, 18 | while \code{happiness_test} is the data from the year 2019, which imitates the 19 | out-of-time validation. 20 | } 21 | \details{ 22 | Source: \href{https://www.kaggle.com/unsdsn/world-happiness}{World Happiness Report at Kaggle.com} 23 | 24 | The following columns: GDP per Capita, Social Support, Life Expectancy, 25 | Freedom, Generosity, Corruption describe the extent to which these factors 26 | contribute in evaluating the happiness in each country. Variables: 27 | 28 | \itemize{ 29 | \item \strong{score} - target variable, continuous value between 0 and 10 (regression) 30 | \item gdp_per_capita 31 | \item social_support 32 | \item healthy_life_expectancy 33 | \item freedom_life_choices 34 | \item generosity 35 | \item perceptions_of_corruption 36 | } 37 | } 38 | -------------------------------------------------------------------------------- /man/modelStudio.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/modelStudio.R 3 | \name{modelStudio} 4 | \alias{modelStudio} 5 | \alias{modelStudio.explainer} 6 | \title{Interactive Studio for Explanatory Model Analysis} 7 | \usage{ 8 | modelStudio(explainer, ...) 9 | 10 | \method{modelStudio}{explainer}( 11 | explainer, 12 | new_observation = NULL, 13 | new_observation_y = NULL, 14 | new_observation_n = 3, 15 | facet_dim = c(2, 2), 16 | time = 500, 17 | max_features = 10, 18 | max_features_fi = NULL, 19 | N = 300, 20 | N_fi = N * 10, 21 | N_sv = N * 3, 22 | B = 10, 23 | B_fi = B, 24 | eda = TRUE, 25 | open_plots = c("fi"), 26 | show_info = TRUE, 27 | parallel = FALSE, 28 | options = ms_options(), 29 | viewer = "external", 30 | widget_id = NULL, 31 | license = NULL, 32 | telemetry = TRUE, 33 | max_vars = NULL, 34 | verbose = NULL, 35 | ... 36 | ) 37 | } 38 | \arguments{ 39 | \item{explainer}{An \code{explainer} created with \code{DALEX::explain()}.} 40 | 41 | \item{...}{Other parameters.} 42 | 43 | \item{new_observation}{New observations with columns that correspond to variables used in the model.} 44 | 45 | \item{new_observation_y}{True label for \code{new_observation} (optional).} 46 | 47 | \item{new_observation_n}{Number of observations to be taken from the \code{explainer$data} if \code{new_observation = NULL}. 48 | See \href{https://modelstudio.drwhy.ai/articles/ms-perks-features.html#instance-explanations}{\bold{vignette}}} 49 | 50 | \item{facet_dim}{Dimensions of the grid. Default is \code{c(2,2)}.} 51 | 52 | \item{time}{Time in ms. Set the animation length. Default is \code{500}.} 53 | 54 | \item{max_features}{Maximum number of features to be included in BD, SV, and FI plots. 55 | Default is \code{10}.} 56 | 57 | \item{max_features_fi}{Maximum number of features to be included in FI plot. Default is \code{max_features}.} 58 | 59 | \item{N}{Number of observations used for the calculation of PD and AD. Default is \code{300}. 60 | See \href{https://modelstudio.drwhy.ai/articles/ms-perks-features.html#more-calculations-means-more-time}{\bold{vignette}}} 61 | 62 | \item{N_fi}{Number of observations used for the calculation of FI. Default is \code{10*N}.} 63 | 64 | \item{N_sv}{Number of observations used for the calculation of SV. Default is \code{3*N}.} 65 | 66 | \item{B}{Number of permutation rounds used for calculation of SV. Default is \code{10}. 67 | See \href{https://modelstudio.drwhy.ai/articles/ms-perks-features.html#more-calculations-means-more-time}{\bold{vignette}}} 68 | 69 | \item{B_fi}{Number of permutation rounds used for calculation of FI. Default is \code{B}.} 70 | 71 | \item{eda}{Compute EDA plots and Residuals vs Feature plot, which adds the data to the dashboard. Default is \code{TRUE}.} 72 | 73 | \item{open_plots}{A vector listing plots to be initially opened (and on which positions). Default is \code{c("fi")}.} 74 | 75 | \item{show_info}{Verbose a progress on the console. Default is \code{TRUE}.} 76 | 77 | \item{parallel}{Speed up the computation using \code{parallelMap::parallelMap()}. 78 | See \href{https://modelstudio.drwhy.ai/articles/ms-perks-features.html#parallel-computation}{\bold{vignette}}. 79 | This might interfere with showing progress using \code{show_info}.} 80 | 81 | \item{options}{Customize \code{modelStudio}. See \code{\link{ms_options}} and 82 | \href{https://modelstudio.drwhy.ai/articles/ms-perks-features.html#additional-options-1}{\bold{vignette}}.} 83 | 84 | \item{viewer}{Default is \code{external} to display in an external RStudio window. 85 | Use \code{browser} to display in an external browser or 86 | \code{internal} to use the RStudio internal viewer pane for output.} 87 | 88 | \item{widget_id}{Use an explicit element ID for the widget (rather than an automatically generated one). 89 | Useful e.g. when using \code{modelStudio} with Shiny. 90 | See \href{https://modelstudio.drwhy.ai/articles/ms-perks-features.html#shiny-1}{\bold{vignette}}.} 91 | 92 | \item{license}{Path to the file containing the license (\code{con} parameter passed to \code{readLines()}). 93 | It can be used e.g. to include the license for \code{explainer$data} as a comment in the source of \code{.html} output file.} 94 | 95 | \item{telemetry}{The dashboard gathers useful, but not sensitive, information about how it is being used (e.g. computation length, 96 | package version, dashboard dimensions). This is for the development purposes only and can be blocked by setting \code{telemetry} to \code{FALSE}.} 97 | 98 | \item{max_vars}{An alias for \code{max_features}. If provided, it will override the value.} 99 | 100 | \item{verbose}{An alias for \code{show_info}. If provided, it will override the value.} 101 | } 102 | \value{ 103 | An object of the \code{r2d3, htmlwidget, modelStudio} class. 104 | } 105 | \description{ 106 | This function computes various (instance and dataset level) model explanations and 107 | produces a customisable dashboard, which consists of multiple panels for plots with their 108 | short descriptions. Easily save the dashboard and share it with others. Tools for 109 | \href{https://ema.drwhy.ai/}{Explanatory Model Analysis} unite with tools for 110 | Exploratory Data Analysis to give a broad overview of the model behavior. 111 | 112 | The extensive documentation covers: 113 | 114 | \itemize{ 115 | \item Function parameters description - 116 | \href{https://modelstudio.drwhy.ai/articles/ms-perks-features.html}{\bold{perks and features}} 117 | \item Framework and model compatibility - 118 | \href{https://modelstudio.drwhy.ai/articles/ms-r-python-examples.html}{\bold{R & Python examples}} 119 | \item Theoretical introduction to the plots - 120 | \href{https://ema.drwhy.ai/}{Explanatory Model Analysis: Explore, Explain, and Examine Predictive Models} 121 | } 122 | 123 | Displayed variable can be changed by clicking on the bars of plots or with the first dropdown list, 124 | and observation can be changed with the second dropdown list. 125 | The dashboard gathers useful, but not sensitive, information about how it is being used (e.g. computation length, 126 | package version, dashboard dimensions). This is for the development purposes only and can be blocked 127 | by setting \code{telemetry} to \code{FALSE}. 128 | } 129 | \examples{ 130 | library("DALEX") 131 | library("modelStudio") 132 | 133 | #:# ex1 classification on 'titanic' data 134 | 135 | # fit a model 136 | model_titanic <- glm(survived ~., data = titanic_imputed, family = "binomial") 137 | 138 | # create an explainer for the model 139 | explainer_titanic <- explain(model_titanic, 140 | data = titanic_imputed, 141 | y = titanic_imputed$survived, 142 | label = "Titanic GLM") 143 | 144 | # pick observations 145 | new_observations <- titanic_imputed[1:2,] 146 | rownames(new_observations) <- c("Lucas","James") 147 | 148 | # make a studio for the model 149 | modelStudio(explainer_titanic, 150 | new_observations, 151 | N = 200, B = 5) # faster example 152 | 153 | \donttest{ 154 | 155 | #:# ex2 regression on 'apartments' data 156 | if (requireNamespace("ranger", quietly=TRUE)) { 157 | library("ranger") 158 | model_apartments <- ranger(m2.price ~. ,data = apartments) 159 | 160 | explainer_apartments <- explain(model_apartments, 161 | data = apartments, 162 | y = apartments$m2.price) 163 | 164 | new_apartments <- apartments[1:2,] 165 | rownames(new_apartments) <- c("ap1","ap2") 166 | 167 | # change dashboard dimensions and animation length 168 | modelStudio(explainer_apartments, 169 | new_apartments, 170 | facet_dim = c(2, 3), 171 | time = 800) 172 | 173 | # add information about true labels 174 | modelStudio(explainer_apartments, 175 | new_apartments, 176 | new_observation_y = new_apartments$m2.price) 177 | 178 | # don't compute EDA plots 179 | modelStudio(explainer_apartments, 180 | eda = FALSE) 181 | } 182 | 183 | #:# ex3 xgboost model on 'HR' dataset 184 | if (requireNamespace("xgboost", quietly=TRUE)) { 185 | library("xgboost") 186 | HR_matrix <- model.matrix(status == "fired" ~ . -1, HR) 187 | 188 | # fit a model 189 | xgb_matrix <- xgb.DMatrix(HR_matrix, label = HR$status == "fired") 190 | params <- list(max_depth = 3, objective = "binary:logistic", eval_metric = "auc") 191 | model_HR <- xgb.train(params, xgb_matrix, nrounds = 300) 192 | 193 | # create an explainer for the model 194 | explainer_HR <- explain(model_HR, 195 | data = HR_matrix, 196 | y = HR$status == "fired", 197 | type = "classification", 198 | label = "xgboost") 199 | 200 | # pick observations 201 | new_observation <- HR_matrix[1:2, , drop=FALSE] 202 | rownames(new_observation) <- c("id1", "id2") 203 | 204 | # make a studio for the model 205 | modelStudio(explainer_HR, 206 | new_observation) 207 | } 208 | } 209 | 210 | } 211 | \references{ 212 | \itemize{ 213 | \item The input object is implemented in \href{https://modeloriented.github.io/DALEX/}{\bold{DALEX}} 214 | \item Feature Importance, Ceteris Paribus, Partial Dependence and Accumulated Dependence explanations 215 | are implemented in \href{https://modeloriented.github.io/ingredients/}{\bold{ingredients}} 216 | \item Break Down and Shapley Values explanations are implemented in 217 | \href{https://modeloriented.github.io/iBreakDown/}{\bold{iBreakDown}} 218 | } 219 | } 220 | \seealso{ 221 | Vignettes: \href{https://modelstudio.drwhy.ai/articles/ms-r-python-examples.html}{\bold{modelStudio - R & Python examples}} 222 | and \href{https://modelstudio.drwhy.ai/articles/ms-perks-features.html}{\bold{modelStudio - perks and features}} 223 | } 224 | -------------------------------------------------------------------------------- /man/ms_merge_observations.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/ms_merge_observations.R 3 | \name{ms_merge_observations} 4 | \alias{ms_merge_observations} 5 | \title{Merge the observations of modelStudio objects} 6 | \usage{ 7 | ms_merge_observations(...) 8 | } 9 | \arguments{ 10 | \item{...}{\code{modelStudio} objects created with \code{modelStudio()}.} 11 | } 12 | \value{ 13 | An object of the \code{r2d3, htmlwidget, modelStudio} class. 14 | } 15 | \description{ 16 | This function merges local explanations from multiple \code{modelStudio} objects into one. 17 | } 18 | \examples{ 19 | \donttest{ 20 | library("DALEX") 21 | library("modelStudio") 22 | 23 | # fit a model 24 | model_happiness <- glm(score ~., data = happiness_train) 25 | 26 | # create an explainer for the model 27 | explainer_happiness <- explain(model_happiness, 28 | data = happiness_test, 29 | y = happiness_test$score) 30 | 31 | # make studios for the model 32 | ms1 <- modelStudio(explainer_happiness, 33 | N = 200, B = 5) 34 | 35 | ms2 <- modelStudio(explainer_happiness, 36 | new_observation = head(happiness_test, 3), 37 | N = 200, B = 5) 38 | 39 | # merge 40 | ms <- ms_merge_observations(ms1, ms2) 41 | ms 42 | } 43 | 44 | } 45 | \references{ 46 | \itemize{ 47 | \item The input object is implemented in \href{https://modeloriented.github.io/DALEX/}{\bold{DALEX}} 48 | \item Feature Importance, Ceteris Paribus, Partial Dependence and Accumulated Dependence explanations 49 | are implemented in \href{https://modeloriented.github.io/ingredients/}{\bold{ingredients}} 50 | \item Break Down and Shapley Values explanations are implemented in 51 | \href{https://modeloriented.github.io/iBreakDown/}{\bold{iBreakDown}} 52 | } 53 | } 54 | \seealso{ 55 | Vignettes: \href{https://modelstudio.drwhy.ai/articles/ms-r-python-examples.html}{\bold{modelStudio - R & Python examples}} 56 | and \href{https://modelstudio.drwhy.ai/articles/ms-perks-features.html}{\bold{modelStudio - perks and features}} 57 | } 58 | -------------------------------------------------------------------------------- /man/ms_options.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/ms_options.R 3 | \name{ms_options} 4 | \alias{ms_options} 5 | \title{Modify default options and pass them to modelStudio} 6 | \usage{ 7 | ms_options(...) 8 | } 9 | \arguments{ 10 | \item{...}{Options to change in the form \code{option_name = value}.} 11 | } 12 | \value{ 13 | \code{list} of options for \code{modelStudio}. 14 | } 15 | \description{ 16 | This function returns default options for \code{\link{modelStudio}}. 17 | It is possible to modify values of this list and pass it to the \code{options} 18 | parameter in the main function. \strong{WARNING: Editing default options may cause 19 | unintended behavior.} 20 | } 21 | \section{Options}{ 22 | 23 | \subsection{Main options:}{ 24 | \describe{ 25 | \item{scale_plot}{\code{TRUE} Makes every plot the same height, ignores \code{bar_width}.} 26 | \item{show_boxplot}{\code{TRUE} Display boxplots in Feature Importance and Shapley Values plots.} 27 | \item{show_subtitle}{\code{TRUE} Should the subtitle be displayed?} 28 | \item{subtitle}{\code{label} parameter from \code{explainer}.} 29 | \item{ms_title}{Title of the dashboard.} 30 | \item{ms_subtitle}{Subtitle of the dashboard (makes space between the title and line).} 31 | \item{ms_margin_*}{Dashboard margins. Change \code{margin_top} for more \code{ms_subtitle} space.} 32 | \item{margin_*}{Plot margins. Change \code{margin_left} for longer/shorter axis labels.} 33 | \item{w}{\code{420} in px. Inner plot width.} 34 | \item{h}{\code{280} in px. Inner plot height.} 35 | \item{bar_width}{\code{16} in px. Default width of bars for all plots, 36 | ignored when \code{scale_plot = TRUE}.} 37 | \item{line_size}{\code{2} in px. Default width of lines for all plots.} 38 | \item{point_size}{\code{3} in px. Default point radius for all plots.} 39 | \item{[bar,line,point]_color}{\code{[#46bac2,#46bac2,#371ea3]}} 40 | \item{positive_color}{\code{#8bdcbe} for Break Down and Shapley Values bars.} 41 | \item{negative_color}{\code{#f05a71} for Break Down and Shapley Values bars.} 42 | \item{default_color}{\code{#371ea3} for Break Down bar and highlighted line.} 43 | } 44 | } 45 | \subsection{Plot-specific options:}{ 46 | \code{**} is a two letter code unique to each plot, might be 47 | one of \code{[bd,sv,cp,fi,pd,ad,rv,fd,tv,at]}.\cr 48 | 49 | \describe{ 50 | \item{**_title}{Plot-specific title. Default varies.} 51 | \item{**_subtitle}{Plot-specific subtitle. Default is \code{subtitle}.} 52 | \item{**_axis_title}{Plot-specific axis title. Default varies.} 53 | \item{**_bar_width}{Plot-specific width of bars. Default is \code{bar_width}, 54 | ignored when \code{scale_plot = TRUE}.} 55 | \item{**_line_size}{Plot-specific width of lines. Default is \code{line_size}.} 56 | \item{**_point_size}{Plot-specific point radius. Default is \code{point_size}.} 57 | \item{**_*_color}{Plot-specific \code{[bar,line,point]} color. Default is \code{[bar,line,point]_color}.} 58 | } 59 | } 60 | } 61 | 62 | \examples{ 63 | library("DALEX") 64 | library("modelStudio") 65 | 66 | # fit a model 67 | model_apartments <- glm(m2.price ~. , data = apartments) 68 | 69 | # create an explainer for the model 70 | explainer_apartments <- explain(model_apartments, 71 | data = apartments, 72 | y = apartments$m2.price) 73 | 74 | # pick observations 75 | new_observation <- apartments[1:2,] 76 | rownames(new_observation) <- c("ap1","ap2") 77 | 78 | # modify default options 79 | new_options <- ms_options( 80 | show_subtitle = TRUE, 81 | bd_subtitle = "Hello World", 82 | line_size = 5, 83 | point_size = 9, 84 | line_color = "pink", 85 | point_color = "purple", 86 | bd_positive_color = "yellow", 87 | bd_negative_color = "orange" 88 | ) 89 | 90 | # make a studio for the model 91 | modelStudio(explainer_apartments, 92 | new_observation, 93 | options = new_options, 94 | N = 200, B = 5) # faster example 95 | 96 | } 97 | \references{ 98 | \itemize{ 99 | \item The input object is implemented in \href{https://modeloriented.github.io/DALEX/}{\bold{DALEX}} 100 | \item Feature Importance, Ceteris Paribus, Partial Dependence and Accumulated Dependence explanations 101 | are implemented in \href{https://modeloriented.github.io/ingredients/}{\bold{ingredients}} 102 | \item Break Down and Shapley Values explanations are implemented in 103 | \href{https://modeloriented.github.io/iBreakDown/}{\bold{iBreakDown}} 104 | } 105 | } 106 | \seealso{ 107 | Vignettes: \href{https://modelstudio.drwhy.ai/articles/ms-r-python-examples.html}{\bold{modelStudio - R & Python examples}} 108 | and \href{https://modelstudio.drwhy.ai/articles/ms-perks-features.html}{\bold{modelStudio - perks and features}} 109 | } 110 | -------------------------------------------------------------------------------- /man/ms_update_observations.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/ms_update_observations.R 3 | \name{ms_update_observations} 4 | \alias{ms_update_observations} 5 | \title{Update the observations of a modelStudio object} 6 | \usage{ 7 | ms_update_observations( 8 | object, 9 | explainer, 10 | new_observation = NULL, 11 | new_observation_y = NULL, 12 | max_features = 10, 13 | B = 10, 14 | show_info = TRUE, 15 | parallel = FALSE, 16 | widget_id = NULL, 17 | overwrite = FALSE, 18 | ... 19 | ) 20 | } 21 | \arguments{ 22 | \item{object}{A \code{modelStudio} created with \code{modelStudio()}.} 23 | 24 | \item{explainer}{An \code{explainer} created with \code{DALEX::explain()}.} 25 | 26 | \item{new_observation}{New observations with columns that correspond to variables used in the model.} 27 | 28 | \item{new_observation_y}{True label for \code{new_observation} (optional).} 29 | 30 | \item{max_features}{Maximum number of features to be included in BD and SV plots. 31 | Default is \code{10}.} 32 | 33 | \item{B}{Number of permutation rounds used for calculation of SV and FI. 34 | Default is \code{10}. 35 | See \href{https://modelstudio.drwhy.ai/articles/ms-perks-features.html#more-calculations-means-more-time}{\bold{vignette}}} 36 | 37 | \item{show_info}{Verbose a progress on the console. Default is \code{TRUE}.} 38 | 39 | \item{parallel}{Speed up the computation using \code{parallelMap::parallelMap()}. 40 | See \href{https://modelstudio.drwhy.ai/articles/ms-perks-features.html#parallel-computation}{\bold{vignette}}. 41 | This might interfere with showing progress using \code{show_info}.} 42 | 43 | \item{widget_id}{Use an explicit element ID for the widget (rather than an automatically generated one). 44 | Useful e.g. when using \code{modelStudio} with Shiny. 45 | See \href{https://modelstudio.drwhy.ai/articles/ms-perks-features.html#shiny-1}{\bold{vignette}}.} 46 | 47 | \item{overwrite}{Overwrite existing observations and their explanations. 48 | Default is \code{FALSE} which means add new observations to the existing ones.} 49 | 50 | \item{...}{Other parameters.} 51 | } 52 | \value{ 53 | An object of the \code{r2d3, htmlwidget, modelStudio} class. 54 | } 55 | \description{ 56 | This function calculates local explanations on new observations and adds them 57 | to the \code{modelStudio} object. 58 | } 59 | \examples{ 60 | library("DALEX") 61 | library("modelStudio") 62 | 63 | # fit a model 64 | model_titanic <- glm(survived ~., data = titanic_imputed, family = "binomial") 65 | 66 | # create an explainer for the model 67 | explainer_titanic <- explain(model_titanic, 68 | data = titanic_imputed, 69 | y = titanic_imputed$survived) 70 | 71 | # make a studio for the model 72 | ms <- modelStudio(explainer_titanic, 73 | N = 200, B = 5) # faster example 74 | 75 | \donttest{ 76 | 77 | # add new observations 78 | ms <- ms_update_observations(ms, 79 | explainer_titanic, 80 | new_observation = titanic_imputed[100:101,], 81 | new_observation_y = titanic_imputed$survived[100:101]) 82 | ms 83 | 84 | 85 | 86 | # overwrite the observations with new ones 87 | ms <- ms_update_observations(ms, 88 | explainer_titanic, 89 | new_observation = titanic_imputed[100:101,], 90 | overwrite = TRUE) 91 | ms 92 | 93 | } 94 | 95 | } 96 | \references{ 97 | \itemize{ 98 | \item The input object is implemented in \href{https://modeloriented.github.io/DALEX/}{\bold{DALEX}} 99 | \item Feature Importance, Ceteris Paribus, Partial Dependence and Accumulated Dependence explanations 100 | are implemented in \href{https://modeloriented.github.io/ingredients/}{\bold{ingredients}} 101 | \item Break Down and Shapley Values explanations are implemented in 102 | \href{https://modeloriented.github.io/iBreakDown/}{\bold{iBreakDown}} 103 | } 104 | } 105 | \seealso{ 106 | Vignettes: \href{https://modelstudio.drwhy.ai/articles/ms-r-python-examples.html}{\bold{modelStudio - R & Python examples}} 107 | and \href{https://modelstudio.drwhy.ai/articles/ms-perks-features.html}{\bold{modelStudio - perks and features}} 108 | } 109 | -------------------------------------------------------------------------------- /man/ms_update_options.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/ms_update_options.R 3 | \name{ms_update_options} 4 | \alias{ms_update_options} 5 | \title{Update the options of a modelStudio object} 6 | \usage{ 7 | ms_update_options(object, ...) 8 | } 9 | \arguments{ 10 | \item{object}{A \code{modelStudio} created with \code{modelStudio()}.} 11 | 12 | \item{...}{Options to change in the form \code{option_name = value}, 13 | e.g. \code{time = 0}, \code{facet_dim = c(1,2)}.} 14 | } 15 | \value{ 16 | An object of the \code{r2d3, htmlwidget, modelStudio} class. 17 | } 18 | \description{ 19 | This function updates the options of a \code{\link{modelStudio}} object. 20 | \strong{WARNING: Editing default options may cause unintended behavior.} 21 | } 22 | \section{Options}{ 23 | 24 | \subsection{Main options:}{ 25 | \describe{ 26 | \item{scale_plot}{\code{TRUE} Makes every plot the same height, ignores \code{bar_width}.} 27 | \item{show_boxplot}{\code{TRUE} Display boxplots in Feature Importance and Shapley Values plots.} 28 | \item{show_subtitle}{\code{TRUE} Should the subtitle be displayed?} 29 | \item{subtitle}{\code{label} parameter from \code{explainer}.} 30 | \item{ms_title}{Title of the dashboard.} 31 | \item{ms_subtitle}{Subtitle of the dashboard (makes space between the title and line).} 32 | \item{ms_margin_*}{Dashboard margins. Change \code{margin_top} for more \code{ms_subtitle} space.} 33 | \item{margin_*}{Plot margins. Change \code{margin_left} for longer/shorter axis labels.} 34 | \item{w}{\code{420} in px. Inner plot width.} 35 | \item{h}{\code{280} in px. Inner plot height.} 36 | \item{bar_width}{\code{16} in px. Default width of bars for all plots, 37 | ignored when \code{scale_plot = TRUE}.} 38 | \item{line_size}{\code{2} in px. Default width of lines for all plots.} 39 | \item{point_size}{\code{3} in px. Default point radius for all plots.} 40 | \item{[bar,line,point]_color}{\code{[#46bac2,#46bac2,#371ea3]}} 41 | \item{positive_color}{\code{#8bdcbe} for Break Down and Shapley Values bars.} 42 | \item{negative_color}{\code{#f05a71} for Break Down and Shapley Values bars.} 43 | \item{default_color}{\code{#371ea3} for Break Down bar and highlighted line.} 44 | } 45 | } 46 | \subsection{Plot-specific options:}{ 47 | \code{**} is a two letter code unique to each plot, might be 48 | one of \code{[bd,sv,cp,fi,pd,ad,rv,fd,tv,at]}.\cr 49 | 50 | \describe{ 51 | \item{**_title}{Plot-specific title. Default varies.} 52 | \item{**_subtitle}{Plot-specific subtitle. Default is \code{subtitle}.} 53 | \item{**_axis_title}{Plot-specific axis title. Default varies.} 54 | \item{**_bar_width}{Plot-specific width of bars. Default is \code{bar_width}, 55 | ignored when \code{scale_plot = TRUE}.} 56 | \item{**_line_size}{Plot-specific width of lines. Default is \code{line_size}.} 57 | \item{**_point_size}{Plot-specific point radius. Default is \code{point_size}.} 58 | \item{**_*_color}{Plot-specific \code{[bar,line,point]} color. Default is \code{[bar,line,point]_color}.} 59 | } 60 | } 61 | } 62 | 63 | \examples{ 64 | library("DALEX") 65 | library("modelStudio") 66 | 67 | # fit a model 68 | model_titanic <- glm(survived ~., data = titanic_imputed, family = "binomial") 69 | 70 | # create an explainer for the model 71 | explainer_titanic <- explain(model_titanic, 72 | data = titanic_imputed, 73 | y = titanic_imputed$survived) 74 | 75 | # make a studio for the model 76 | ms <- modelStudio(explainer_titanic, 77 | N = 200, B = 5) # faster example 78 | 79 | # update the options 80 | new_ms <- ms_update_options(ms, 81 | time = 0, 82 | facet_dim = c(1,2), 83 | margin_left = 150) 84 | new_ms 85 | 86 | } 87 | \references{ 88 | \itemize{ 89 | \item The input object is implemented in \href{https://modeloriented.github.io/DALEX/}{\bold{DALEX}} 90 | \item Feature Importance, Ceteris Paribus, Partial Dependence and Accumulated Dependence explanations 91 | are implemented in \href{https://modeloriented.github.io/ingredients/}{\bold{ingredients}} 92 | \item Break Down and Shapley Values explanations are implemented in 93 | \href{https://modeloriented.github.io/iBreakDown/}{\bold{iBreakDown}} 94 | } 95 | } 96 | \seealso{ 97 | Vignettes: \href{https://modelstudio.drwhy.ai/articles/ms-r-python-examples.html}{\bold{modelStudio - R & Python examples}} 98 | and \href{https://modelstudio.drwhy.ai/articles/ms-perks-features.html}{\bold{modelStudio - perks and features}} 99 | } 100 | -------------------------------------------------------------------------------- /misc/MLinPL2019_modelStudio_poster.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ModelOriented/modelStudio/62d23a802f88c5efc98b285f08b4ccf97049a63e/misc/MLinPL2019_modelStudio_poster.pdf -------------------------------------------------------------------------------- /misc/paper.bib: -------------------------------------------------------------------------------- 1 | @inproceedings{lime, 2 | author = {Marco Tulio Ribeiro and 3 | Sameer Singh and 4 | Carlos Guestrin}, 5 | title = {"Why Should {I} Trust You?": Explaining the Predictions of Any Classifier}, 6 | booktitle = {Proceedings of the 22nd {ACM} {SIGKDD} International Conference on 7 | Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 8 | 13-17, 2016}, 9 | pages = {1135--1144}, 10 | year = {2016}, 11 | doi = {10.18653/v1/n16-3020} 12 | } 13 | 14 | @article{DALEX, 15 | author = {{Biecek}, Przemysław}, 16 | title = "{DALEX: explainers for complex predictive models}", 17 | url = {http://arxiv.org/abs/1806.08915}, 18 | eprint = {1806.08915}, 19 | primaryClass = "stat.ML", 20 | keywords = {Statistics - Machine Learning, Computer Science - Artificial Intelligence, Computer Science - Machine Learning, Statistics - Applications}, 21 | year = 2018, 22 | adsurl = {http://adsabs.harvard.edu/abs/2018arXiv180608915B}, 23 | adsnote = {Provided by the SAO/NASA Astrophysics Data System} 24 | } 25 | 26 | @incollection{NIPS2017_7062, 27 | title = {A Unified Approach to Interpreting Model Predictions}, 28 | author = {Lundberg, Scott M and Lee, Su-In}, 29 | booktitle = {Advances in Neural Information Processing Systems 30}, 30 | editor = {I. Guyon and U. V. Luxburg and S. Bengio and H. Wallach and R. Fergus and S. Vishwanathan and R. Garnett}, 31 | pages = {4765--4774}, 32 | year = {2017}, 33 | publisher = {Curran Associates, Inc.}, 34 | url = {http://papers.nips.cc/paper/7062-a-unified-approach-to-interpreting-model-predictions.pdf} 35 | } 36 | 37 | @book{Gill2017MachineLI, 38 | title={Machine Learning Interpretability with {H2O} Driverless {AI}}, 39 | author={Patrick Hall and Navdeep Gill and Megan Kurka and Wen Phan}, 40 | year={2017}, 41 | month=feb, 42 | edition=1, 43 | editor={Angela Bartz}, 44 | url={http://docs.h2o.ai}, 45 | publisher = {{H2O.ai, Inc.}}, 46 | address={Mountain View, CA}, 47 | } 48 | 49 | @ARTICLE{2018arXiv180401955S, 50 | author = {Mateusz Staniak and Przemysław Biecek}, 51 | title = {{Explanations of Model Predictions with live and breakDown Packages}}, 52 | year = {2018}, 53 | journal = {{The R Journal}}, 54 | doi = {10.32614/RJ-2018-072}, 55 | pages = {395--409}, 56 | volume = {10}, 57 | number = {2} 58 | } 59 | 60 | @article{Truong2019TowardsAM, 61 | title={Towards Automated Machine Learning: Evaluation and Comparison of AutoML Approaches and Tools}, 62 | author={Anh Truong and Austin Walters and Jeremy Goodsitt and Keegan Hines and C. Bayan Bruss and Reza Farivar}, 63 | journal={ArXiv}, 64 | year={2019}, 65 | volume={abs/1908.05557} 66 | } 67 | 68 | @Manual{R, 69 | title = {R: A Language and Environment for Statistical Computing}, 70 | author = {{R Core Team}}, 71 | organization = {R Foundation for Statistical Computing}, 72 | address = {Vienna, Austria}, 73 | year = {2019}, 74 | url = {https://www.R-project.org/}, 75 | } 76 | 77 | @article{bostock2016d3, 78 | title={D3.js-data-driven documents (2016)}, 79 | author={Bostock, Mike}, 80 | journal={URL: https://d3js.org}, 81 | year={2016} 82 | } 83 | 84 | @article{Romaszko2019, 85 | doi = {10.21105/joss.01444}, 86 | url = {https://doi.org/10.21105/joss.01444}, 87 | year = {2019}, 88 | month = jun, 89 | publisher = {The Open Journal}, 90 | volume = {4}, 91 | number = {38}, 92 | pages = {1444}, 93 | author = {Kamil Romaszko and Magda Tatarynowicz and Mateusz Urba{\'{n}}ski and Przemys{\l}aw Biecek}, 94 | title = {{modelDown}: automated website generator with interpretable documentation for predictive machine learning models}, 95 | journal = {Journal of Open Source Software} 96 | } 97 | 98 | @article{gosiewska2019ibreakdown, 99 | title={iBreakDown: Uncertainty of Model Explanations for Non-additive Predictive Models}, 100 | author={Gosiewska, Alicja and Biecek, Przemyslaw}, 101 | journal={arXiv preprint arXiv:1903.11420}, 102 | year={2019} 103 | } 104 | 105 | @Manual{iBreakDown, 106 | title = {iBreakDown: Model Agnostic Instance Level Variable Attributions}, 107 | author = {Przemyslaw Biecek and Alicja Gosiewska and Hubert Baniecki and Adam Izdebski}, 108 | year = {2019}, 109 | note = {R package version 0.9.9}, 110 | url = {https://CRAN.R-project.org/package=iBreakDown}, 111 | } 112 | 113 | @Manual{ingredients, 114 | title = {ingredients: Effects and Importances of Model Ingredients}, 115 | author = {Przemyslaw Biecek and Hubert Baniecki and Adam Izdebski and Katarzyna Pekala}, 116 | year = {2019}, 117 | note = {https://ModelOriented.github.io/ingredients/, 118 | https://github.com/ModelOriented/ingredients}, 119 | url = {http://CRAN.R-project.org/package=ingredients} 120 | } 121 | 122 | @article{iml, 123 | title = {{iml: An R package for Interpretable Machine Learning}}, 124 | author = {Christoph Molnar and Giuseppe Casalicchio and Bernd Bischl}, 125 | year = {2018}, 126 | journal = {{Journal of Open Source Software}}, 127 | volume = {3}, 128 | number = {26}, 129 | page = {786}, 130 | doi = {10.21105/joss.00786} 131 | } 132 | 133 | @Manual{tfexplain, 134 | title = {Interpretability Methods for tf.keras models with Tensorflow 2.0}, 135 | author = {Raphael Meudec}, 136 | year = {2019}, 137 | note = {https://tf-explain.readthedocs.io}, 138 | url = {https://tf-explain.readthedocs.io} 139 | } 140 | 141 | @Manual{sklearnexplain, 142 | title = {sklearn explain}, 143 | author = {Antoine Carme}, 144 | year = {2019}, 145 | note = {https://github.com/antoinecarme/sklearn_explain}, 146 | url = {https://github.com/antoinecarme/sklearn_explain} 147 | } 148 | 149 | @Manual{InterpretML, 150 | title = {InterpretML}, 151 | author = {Samuel Jenkins and Harsha Nori and Paul Koch and Rich Caruana}, 152 | year = {2019}, 153 | note = {https://github.com/microsoft/interpret}, 154 | url = {https://github.com/microsoft/interpret} 155 | } 156 | 157 | @ARTICLE{2019arXiv190402101S, 158 | author = {{Staniak}, Mateusz and {Biecek}, Przemyslaw}, 159 | title = "{The Landscape of R Packages for Automated Exploratory Data Analysis}", 160 | journal = {arXiv e-prints}, 161 | keywords = {Statistics - Computation, Computer Science - Machine Learning, Statistics - Machine Learning}, 162 | year = "2019", 163 | month = "Mar", 164 | eid = {arXiv:1904.02101}, 165 | pages = {arXiv:1904.02101}, 166 | archivePrefix = {arXiv}, 167 | eprint = {1904.02101}, 168 | primaryClass = {stat.CO}, 169 | adsurl = {https://ui.adsabs.harvard.edu/abs/2019arXiv190402101S}, 170 | adsnote = {Provided by the SAO/NASA Astrophysics Data System} 171 | } 172 | 173 | @Manual{ceterisParibus, 174 | title = {ceterisParibus: Ceteris Paribus Profiles}, 175 | author = {Przemyslaw Biecek}, 176 | year = {2019}, 177 | note = {R package version 0.3.1}, 178 | url = {https://CRAN.R-project.org/package=ceterisParibus} 179 | } 180 | 181 | @article{fisher2018all, 182 | title={All models are wrong but many are useful: Variable importance for black-box, proprietary, or misspecified prediction models, using model class reliance}, 183 | author={Fisher, Aaron and Rudin, Cynthia and Dominici, Francesca}, 184 | journal={arXiv preprint arXiv:1801.01489}, 185 | year={2018} 186 | } 187 | 188 | @article{RJ-2017-016, 189 | author = {Greenwell, Brandon M.}, 190 | title = {{pdp: An R Package for Constructing Partial Dependence Plots}}, 191 | year = {2017}, 192 | journal = {{The R Journal}}, 193 | url = {http://journal.r-project.org/archive/2017/RJ-2017-016/index.html}, 194 | pages = {421--436}, 195 | volume = {9}, 196 | number = {1}, 197 | doi = {10.32614/rj-2017-016} 198 | } 199 | 200 | @article{apley2016visualizing, 201 | title={Visualizing the effects of predictor variables in black box supervised learning models}, 202 | author={Apley, Daniel W}, 203 | journal={arXiv preprint arXiv:1612.08468}, 204 | year={2016} 205 | } 206 | 207 | @Manual{pmeed, 208 | title = {Predictive Models: Explore, Explain, and Debug}, 209 | author = {Przemyslaw Biecek and Tomasz Burzykowski}, 210 | year = {2019}, 211 | note = {https://pbiecek.github.io/PM_VEE/}, 212 | url = {https://pbiecek.github.io/PM_VEE/} 213 | } 214 | 215 | -------------------------------------------------------------------------------- /misc/paper.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: 'modelStudio: Interactive Studio with Explanations for ML Predictive Models' 3 | authors: 4 | - affiliation: 1 5 | name: Hubert Baniecki 6 | orcid: 0000-0001-6661-5364 7 | - affiliation: 1 8 | name: Przemyslaw Biecek 9 | orcid: 0000-0001-8423-1823 10 | date: "22 October 2019" 11 | bibliography: paper.bib 12 | tags: 13 | - automated data analysis 14 | - model visualization 15 | - explainable artificial intelligence 16 | - predictive modeling 17 | - interpretable machine learning 18 | affiliations: 19 | - index: 1 20 | name: Faculty of Mathematics and Information Science, Warsaw University of Technology 21 | --- 22 | 23 | # Introduction 24 | 25 | Machine learning predictive models are widely used in many areas of 26 | business and research. Their rising popularity is due to them being 27 | effective but often leads to problems with explaining their prediction. 28 | This has led to development of many Interpretable Machine Learning 29 | tools, e.g., `DALEX` [@DALEX] R package, `lime` [@lime] and 30 | `shap` [@NIPS2017_7062] Python packages and `H2o.ai Driverless AI` [@Gill2017MachineLI]. 31 | 32 | Nowadays, we can see a huge demand for automation in many areas. This is 33 | how Automated Machine Learning and Automated Exploratory Data Analysis 34 | came to existence. AutoML [@Truong2019TowardsAM] and AutoEDA [@2018arXiv180401955S] 35 | tools not only speed up the model development process 36 | but also often lead to new discoveries or higher quality of models. 37 | 38 | Explaining predictive models might be a time consuming and tedious task. 39 | Libraries for interpretable machine learning [@DALEX; @iml; @tfexplain; @sklearnexplain; @InterpretML] require high programing skills and endless exploration of different aspects of a predictive model. 40 | 41 | There are tools for automation of the XAI process like `modelDown` [@Romaszko2019] which produces static HTML site to compare and explain various models. Unfortunately, such tools are focused on global level explanations and deliver monotonous experience. 42 | 43 | # The `modelStudio` package 44 | 45 | The `modelStudio` R package automates the process of model exploration. It generates advanced interactive and animated model explanations in the form of a serverless HTML site. It combines **R** [@R] with **D3.js** [@bostock2016d3] to produce plots and descriptions for various local and global explanations. Tools for model exploration unite with tools for EDA to give a broad overview of the model behaviour. 46 | 47 | The usage of `modelStudio` is meant to be intuitive and simple. The computation time needed to produce the output might not be short though. 48 | The main goal of this tool is to make model explaining more automated and achieve higher quality explanations by juxtaposition of complementary aspects of a model. 49 | 50 | Comparing instance level explanations and model level 51 | explanations side by side adds wider context and allows for deeper understanding. 52 | `modelStudio` helps to study relations between various methods for model explanation like *Break Down*, *SHAP*, *Partial Dependency Plots*, *Feature Importance*, and others. 53 | 54 | 55 | 56 | # Example 57 | 58 | The package `modelStudio` is available on [CRAN](https://CRAN.R-project.org/package=modelStudio). It can be installed 59 | using the `install.packages('modelStudio')` command. This package is based 60 | on `DALEX` explainers created with `DALEX::explain()`. Below there is a basic code 61 | example, which produces [demo](https://modeloriented.github.io/modelStudio/demo.html). 62 | 63 | ``` r 64 | library("modelStudio") 65 | 66 | # Create a model 67 | model <- glm(survived ~., data = DALEX::titanic_imputed, family = "binomial") 68 | 69 | # Wrap it into an explainer 70 | explainer <- DALEX::explain(model, data = DALEX::titanic_imputed[,-8], 71 | y = DALEX::titanic_imputed[,8], label = "glm") 72 | 73 | # Pick some data points 74 | new_observations <- DALEX::titanic_imputed[1:4,] 75 | rownames(new_observations) <- c("Lucas", "James", "Thomas", "Nancy") 76 | 77 | # Make a studio for the model 78 | modelStudio(explainer, new_observations) 79 | ``` 80 | 81 | ![Exemplary HTML output layout.](images/demo.png) 82 | 83 | # Key Features 84 | 85 | The generated HTML site has many interactive features. One can choose 86 | which plots are displayed on the grid and change them at any given 87 | moment by clicking the X symbol. A drop down list may be used to pick the 88 | observation that will be considered for local explanation plots. One may manipulate plots having a variable-based dimension by selecting corresponding bars on the other plots. Mousing over the D symbol displays a description of the plot. Finally, mousing over lines and bars displays the tooltip. 89 | 90 | ![1. Open in browser or save as HTML document or PNG image 2. Click on bars to choose which feature will be used for other plots 3. Mouse over the D symbol to display a description of the plot and click X to close the plot 4. Choose which observation will be used for local explanations 5. Mouse over lines and bars to display the tooltip 6. Click on the text to choose the plot 7. Interact with other elements like a slider](images/cheatsheetcut.png) 91 | 92 | # Explanations 93 | 94 | Seven possible plots to choose from are implemented. There are three 95 | local explanation plots, three global explanation plots and a feature density plot. 96 | 97 | **Local explanations** are designed to better understand model behaviour 98 | around a single observation. 99 | 100 | - **Break Down** plot and **SHAP Values** [@NIPS2017_7062] 101 | plot present variable contributions to a model prediction [@gosiewska2019ibreakdown]. 102 | Both of them come from the `iBreakDown` [@iBreakDown] R package. 103 | 104 | - **Ceteris Paribus** plot presents model responses around a single 105 | point in the feature space [@ceterisParibus]. 106 | 107 | **Global explanations** are designed to allow for better understanding of how the model 108 | works in general, for some population of interest. 109 | 110 | - **Feature Importance** plot presents permutation based feature 111 | importance [@fisher2018all]. 112 | 113 | - **Partial Dependency** plot presents averages from N number of 114 | Ceteris Paribus Profiles [@RJ-2017-016]. 115 | 116 | - **Accumulated Dependency** plot presents accumulated local changes 117 | in Ceteris Paribus Profiles [@apley2016visualizing]. 118 | 119 | Detailed overview of these methods can be found in “Predictive Models: 120 | Explore, Explain, and Debug” [@pmeed]. The last explanations are implemented 121 | in the `ingredients` [@ingredients] R package. 122 | 123 | # Conclusions 124 | 125 | The `modelStudio` package is easy to use and its output is intuitive to 126 | explore. Automation is convenient and interactivity adds an another 127 | dimension to visualisations. All of this enhance explanation of machine 128 | learning predictive models. More features and examples can be found in 129 | the vignette: [modelStudio - perks and features](https://modeloriented.github.io/modelStudio/articles/vignette_modelStudio.html) 130 | and on [GitHub](https://github.com/ModelOriented/modelStudio). 131 | 132 | # Acknowledgments 133 | 134 | Work on this package was financially supported by the ‘NCN Opus grant 2016/21/B/ST6/02176’. 135 | 136 | # References 137 | -------------------------------------------------------------------------------- /pkgdown/_pkgdown.yml: -------------------------------------------------------------------------------- 1 | template: 2 | package: DrWhyTemplate 3 | default_assets: false 4 | params: 5 | ganalytics: UA-5650686-14 6 | noindex: true 7 | -------------------------------------------------------------------------------- /pkgdown/favicon/CNAME: -------------------------------------------------------------------------------- 1 | modelstudio.drwhy.ai 2 | -------------------------------------------------------------------------------- /pkgdown/favicon/apple-touch-icon-120x120.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ModelOriented/modelStudio/62d23a802f88c5efc98b285f08b4ccf97049a63e/pkgdown/favicon/apple-touch-icon-120x120.png -------------------------------------------------------------------------------- /pkgdown/favicon/apple-touch-icon-152x152.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ModelOriented/modelStudio/62d23a802f88c5efc98b285f08b4ccf97049a63e/pkgdown/favicon/apple-touch-icon-152x152.png -------------------------------------------------------------------------------- /pkgdown/favicon/apple-touch-icon-180x180.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ModelOriented/modelStudio/62d23a802f88c5efc98b285f08b4ccf97049a63e/pkgdown/favicon/apple-touch-icon-180x180.png -------------------------------------------------------------------------------- /pkgdown/favicon/apple-touch-icon-60x60.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ModelOriented/modelStudio/62d23a802f88c5efc98b285f08b4ccf97049a63e/pkgdown/favicon/apple-touch-icon-60x60.png -------------------------------------------------------------------------------- /pkgdown/favicon/apple-touch-icon-76x76.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ModelOriented/modelStudio/62d23a802f88c5efc98b285f08b4ccf97049a63e/pkgdown/favicon/apple-touch-icon-76x76.png -------------------------------------------------------------------------------- /pkgdown/favicon/apple-touch-icon.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ModelOriented/modelStudio/62d23a802f88c5efc98b285f08b4ccf97049a63e/pkgdown/favicon/apple-touch-icon.png -------------------------------------------------------------------------------- /pkgdown/favicon/favicon-16x16.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ModelOriented/modelStudio/62d23a802f88c5efc98b285f08b4ccf97049a63e/pkgdown/favicon/favicon-16x16.png -------------------------------------------------------------------------------- /pkgdown/favicon/favicon-32x32.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ModelOriented/modelStudio/62d23a802f88c5efc98b285f08b4ccf97049a63e/pkgdown/favicon/favicon-32x32.png -------------------------------------------------------------------------------- /pkgdown/favicon/favicon.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ModelOriented/modelStudio/62d23a802f88c5efc98b285f08b4ccf97049a63e/pkgdown/favicon/favicon.ico -------------------------------------------------------------------------------- /tests/spelling.R: -------------------------------------------------------------------------------- 1 | if(requireNamespace('spelling', quietly = TRUE)) 2 | spelling::spell_check_test(vignettes = TRUE, error = FALSE, 3 | skip_on_cran = TRUE) 4 | -------------------------------------------------------------------------------- /tests/testthat.R: -------------------------------------------------------------------------------- 1 | library("testthat") 2 | library("modelStudio") 3 | library("DALEX") 4 | requireNamespace("ranger", quietly=TRUE) 5 | requireNamespace("xgboost", quietly=TRUE) 6 | 7 | test_check("modelStudio") 8 | -------------------------------------------------------------------------------- /tests/testthat/test_2_0.R: -------------------------------------------------------------------------------- 1 | context("Test v2.0.0") 2 | 3 | set.seed(1313) 4 | v <- FALSE 5 | 6 | if (requireNamespace("ranger", quietly=TRUE)) { 7 | data_multiclass <- DALEX::HR[1:100,] 8 | model_multiclass <- ranger::ranger(status ~. , 9 | data = data_multiclass, 10 | probability = TRUE) 11 | exp_multiclass <- DALEX::explain(model_multiclass, 12 | data = data_multiclass, 13 | y = data_multiclass$status, 14 | verbose = v) 15 | 16 | data_fifa <- DALEX::fifa[1:100, 1:10] 17 | model_fifa <- ranger::ranger(value_eur ~. , 18 | data_fifa) 19 | exp_fifa <- DALEX::explain(model_fifa, 20 | data = data_fifa, 21 | y = data_fifa$value_eur, 22 | verbose = v) 23 | 24 | #:# errors #:# 25 | exp_nodata <- DALEX::explain(model_fifa, verbose = v) 26 | exp_noy <- DALEX::explain(model_fifa, data = data_fifa, verbose = v) 27 | 28 | testthat::expect_error(modelStudio::modelStudio(exp_multiclass)) 29 | testthat::expect_error(modelStudio::modelStudio(exp_nodata)) 30 | testthat::expect_error(modelStudio::modelStudio(exp_noy)) 31 | 32 | #:# new functionalities #:# 33 | case1 <- testthat::expect_silent( 34 | modelStudio::modelStudio(exp_fifa, data_fifa[1,], 35 | widget_id = "MS", telemetry = F, 36 | N = 5, B = 2, show_info = v)) 37 | 38 | testthat::expect_true(case1$elementId == 'MS') 39 | testthat::expect_true(!isTRUE(case1$x$options$telemetry)) 40 | 41 | case2 <- testthat::expect_silent( 42 | modelStudio::modelStudio(exp_fifa, data_fifa[1:2,], 43 | max_vars = 5, rounding_funtion = signif, 44 | digits = 3, 45 | N = 5, B = 2, show_info = v)) 46 | testthat::expect_true( 47 | all(c("mse:", "rmse:", "r2:", "mad:") %in% 48 | strsplit(case2$x$options$measure_text, split=" ")[[1]])) 49 | 50 | #:# test NA #:# 51 | 52 | data_na <- DALEX::fifa[1:100, 2:10] 53 | model_na <- ranger::ranger(value_eur~., data_na) 54 | data_na[1:10,] <- NA 55 | 56 | pf <- function(model, data) { 57 | data <- impute(data) 58 | predict(model, data)$predictions 59 | } 60 | 61 | impute <- function(x, val = 0) { 62 | for (i in 1:dim(x)[1]) { 63 | for (j in 1:dim(x)[2]) { 64 | if (is.na(x[i, j])) x[i, j] <- val 65 | } 66 | } 67 | x 68 | } 69 | 70 | exp_na <- DALEX::explain(model_na, 71 | data = data_na, 72 | y = data_na$value_eur, 73 | predict_function = pf, 74 | verbose = v) 75 | 76 | case3 <- testthat::expect_silent( 77 | modelStudio::modelStudio(exp_na, data_na[11,], 78 | N=5, B=2, show_info = v) 79 | ) 80 | 81 | testthat::expect_true(!case3$x$options$is_target_binary) 82 | } 83 | 84 | #:# other #:# 85 | 86 | model_bin <- glm(survived ~., data = DALEX::titanic_imputed, family = "binomial") 87 | exp_bin <- DALEX::explain(model_bin, 88 | data = DALEX::titanic_imputed, 89 | y = DALEX::titanic_imputed$survived, 90 | verbose=v) 91 | case4 <- testthat::expect_silent( 92 | modelStudio::modelStudio(exp_bin, DALEX::titanic_imputed[3:4, ], 93 | loss_function = DALEX::loss_sum_of_squares, 94 | variable_splits_with_obs = FALSE, variable_splits_type='quantiles', 95 | N = 5, B = 2, show_info = v) 96 | ) 97 | 98 | testthat::expect_true(case4$x$options$is_target_binary) -------------------------------------------------------------------------------- /tests/testthat/test_2_1.R: -------------------------------------------------------------------------------- 1 | context("Test v2.1.0") 2 | 3 | set.seed(1313) 4 | v <- FALSE 5 | 6 | model <- glm(survived ~., data = DALEX::titanic_imputed, family = "binomial") 7 | 8 | explainer <- DALEX::explain(model, 9 | data = DALEX::titanic_imputed, 10 | y = DALEX::titanic_imputed$survived, 11 | label = "Titanic GLM", 12 | verbose=v) 13 | 14 | testthat::test_that("ms_options parameters", { 15 | ms1 <- testthat::expect_silent( 16 | modelStudio::modelStudio(explainer, 17 | N = 10, B = 2, 18 | options = modelStudio::ms_options( 19 | ms_subtitle = "Nice model", 20 | ms_margin_top = 80, 21 | ms_margin_bottom = 80 22 | ), 23 | show_info = v) 24 | ) 25 | testthat::expect_is(ms1, "r2d3") 26 | }) 27 | 28 | testthat::test_that("N_fi, B_fi parameters", { 29 | ms2 <- testthat::expect_silent( 30 | modelStudio::modelStudio(explainer, 31 | N_fi = 200, B_fi = 3, 32 | show_info = v) 33 | ) 34 | testthat::expect_is(ms2, "r2d3") 35 | }) 36 | 37 | testthat::test_that("N = NULL", { 38 | ms3 <- testthat::expect_silent( 39 | modelStudio::modelStudio(explainer, 40 | N_fi = NULL, B = 2, 41 | show_info = v) 42 | ) 43 | testthat::expect_is(ms3, "r2d3") 44 | }) 45 | -------------------------------------------------------------------------------- /tests/testthat/test_3_0.R: -------------------------------------------------------------------------------- 1 | context("Test v3.0.0") 2 | 3 | set.seed(1313) 4 | v <- FALSE 5 | 6 | model <- glm(survived ~., data = DALEX::titanic_imputed, family = "binomial") 7 | 8 | explainer <- DALEX::explain(model, 9 | data = DALEX::titanic_imputed, 10 | y = DALEX::titanic_imputed$survived, 11 | verbose = v) 12 | 13 | testthat::test_that("N_sv, verbose parameters", { 14 | ms1 <- testthat::expect_silent( 15 | modelStudio::modelStudio(explainer, 16 | N_sv = 200, B = 2, 17 | verbose = v) 18 | ) 19 | testthat::expect_is(ms1, "r2d3") 20 | }) 21 | -------------------------------------------------------------------------------- /tests/testthat/test_3_1.R: -------------------------------------------------------------------------------- 1 | context("Test v3.1.1") 2 | 3 | set.seed(1313) 4 | v <- FALSE 5 | 6 | model <- glm(survived ~., data = DALEX::titanic_imputed, family = "binomial") 7 | 8 | explainer <- DALEX::explain(model, 9 | data = DALEX::titanic_imputed, 10 | y = DALEX::titanic_imputed$survived, 11 | verbose = v) 12 | 13 | testthat::test_that("max_features_fi, **_axis_title", { 14 | ms <- testthat::expect_silent( 15 | modelStudio::modelStudio(explainer, 16 | B = 2, 17 | max_features_fi = 2, 18 | max_features = 2, 19 | options = ms_options(cp_axis_title = "pred", 20 | bd_axis_title = "attribution"), 21 | verbose = v) 22 | ) 23 | testthat::expect_is(ms, "r2d3") 24 | }) 25 | 26 | testthat::test_that("open_plots", { 27 | ms <- testthat::expect_silent( 28 | modelStudio::modelStudio(explainer, 29 | B = 2, 30 | open_plots = c("fi", "bd", "rv"), 31 | verbose = v) 32 | ) 33 | testthat::expect_is(ms, "r2d3") 34 | 35 | testthat::expect_error( 36 | modelStudio::modelStudio(explainer, 37 | B = 2, 38 | open_plots = c("bd", "test"), 39 | verbose = v) 40 | ) 41 | testthat::expect_error( 42 | modelStudio::modelStudio(explainer, 43 | B = 2, 44 | facet_dim = c(2, 1), 45 | open_plots = c("pd", "ad", "cp"), 46 | verbose = v) 47 | ) 48 | testthat::expect_silent( 49 | modelStudio::modelStudio(explainer, 50 | B = 2, 51 | facet_dim = c(2, 1), 52 | open_plots = c("SV", "FI"), 53 | verbose = v) 54 | ) 55 | }) -------------------------------------------------------------------------------- /tests/testthat/test_modelStudio.R: -------------------------------------------------------------------------------- 1 | context("Check modelStudio() function") 2 | 3 | source("test_objects.R") 4 | 5 | N = 10 6 | B = 2 7 | 8 | ms1 <- testthat::expect_silent(modelStudio::modelStudio(explain_glm, 9 | new_observation = titanic_test[1,-9], 10 | show_info = v)) 11 | 12 | ms4 <- testthat::expect_silent(modelStudio::modelStudio(explain_glm_numerical, 13 | new_observation = titanic_test[1:2, c(2,6,7,8)], 14 | N = N, B = B, 15 | show_info = v)) 16 | 17 | ms5 <- testthat::expect_silent(modelStudio::modelStudio(explain_glm_not_numerical, 18 | new_observation = titanic_test[1:2, c(1,3,4,5)], 19 | N = N, B = B, 20 | show_info = v)) 21 | 22 | ms6 <- testthat::expect_silent(modelStudio::modelStudio(explain_model_small, 23 | new_observation = titanic_test[1:2, c(1,2)], 24 | N = N, B = B, 25 | show_info = v)) 26 | 27 | ms_readme <- testthat::expect_silent(modelStudio::modelStudio(explain_titanic_glm, 28 | new_observations, 29 | facet_dim = c(2,2), 30 | N = N, B = B, time = 0, 31 | show_info = v)) 32 | 33 | ms_rf_apartments <- testthat::expect_silent(modelStudio::modelStudio(explain_rf, 34 | new_observation = apartments[1:2,-1], 35 | N = N, B = B, facet_dim = c(3,3), 36 | time = 50, max_features = 4, 37 | show_info = v)) 38 | 39 | both_without_target <- testthat::expect_silent(modelStudio::modelStudio(explain_both_without_target, 40 | new_observation = nx, 41 | N = N, 42 | B = B, 43 | show_info = v)) 44 | 45 | both_full <- testthat::expect_silent(modelStudio::modelStudio(explain_both_full, 46 | new_observation = nz, 47 | N = N, 48 | B = B, 49 | show_info = v)) 50 | 51 | obs_without_target_data_full <- testthat::expect_silent(modelStudio::modelStudio(explain_obs_without_target_data_full, 52 | new_observation = nx, 53 | N = N, 54 | B = B, 55 | show_info = v)) 56 | 57 | obs_full_data_without_target <- testthat::expect_silent(modelStudio::modelStudio(explain_obs_full_data_without_target, 58 | new_observation = nz, 59 | N = N, 60 | B = B, 61 | show_info = v)) 62 | 63 | ms_big <- testthat::expect_silent(modelStudio::modelStudio(explain_artifficial, 64 | new_observation = artifficial[1:2,], 65 | N = N, B = B, 66 | facet_dim = c(3,3), 67 | show_info = v)) 68 | 69 | ms_parallel <- modelStudio::modelStudio(explain_glm, new_observation = titanic_test[1:2,-9], 70 | N = N, B = B, parallel = TRUE, 71 | show_info = v) 72 | 73 | ms_parallel_rf <- modelStudio::modelStudio(explain_rf, new_observation = apartments[1:5,-1], 74 | N = N, B = B, parallel = TRUE, 75 | show_info = v) 76 | 77 | # tests 78 | 79 | testthat::test_that("explainer test", { 80 | testthat::expect_is(ms1, "r2d3") 81 | }) 82 | 83 | testthat::test_that("only_numerical", { 84 | testthat::expect_is(ms4, "r2d3") 85 | }) 86 | 87 | testthat::test_that("only_not_numerical", { 88 | testthat::expect_is(ms5, "r2d3") 89 | }) 90 | 91 | testthat::test_that("description test, 2 features", { 92 | testthat::expect_is(ms6, "r2d3") 93 | }) 94 | 95 | testthat::test_that("README DEMO", { 96 | testthat::expect_is(ms_readme, "r2d3") 97 | }) 98 | 99 | testthat::test_that("ranger apartments", { 100 | testthat::expect_is(ms_rf_apartments, "r2d3") 101 | }) 102 | 103 | testthat::test_that("test various possibilities of data and new obs", { 104 | testthat::expect_is(both_full, "r2d3") 105 | testthat::expect_is(both_without_target, "r2d3") 106 | testthat::expect_is(obs_without_target_data_full, "r2d3") 107 | testthat::expect_is(obs_full_data_without_target, "r2d3") 108 | }) 109 | 110 | testthat::test_that("more than 10 features", { 111 | testthat::expect_is(ms_big, "r2d3") 112 | }) 113 | 114 | testthat::test_that("parallel", { 115 | testthat::expect_is(ms_parallel, "r2d3") 116 | }) 117 | 118 | if (requireNamespace("ranger", quietly=TRUE)) { 119 | testthat::test_that("parallel rf", { 120 | testthat::expect_is(ms_parallel_rf, "r2d3") 121 | }) 122 | } 123 | 124 | testthat::test_that("show_info_and_new_observation_y", { 125 | testthat::expect_is(modelStudio::modelStudio(explain_glm, 126 | titanic_test[1:2,-9], 127 | titanic_test[1:2, 9], 128 | N = N, B = B, show_info = TRUE), "r2d3") 129 | }) 130 | 131 | testthat::test_that("eda = FALSE", { 132 | testthat::expect_is(modelStudio::modelStudio(explain_glm, eda = FALSE, 133 | N = N, B = B, show_info = v), "r2d3") 134 | }) 135 | 136 | if (requireNamespace("xgboost", quietly=TRUE)) { 137 | testthat::test_that("xgboost matrix", { 138 | testthat::expect_is(modelStudio::modelStudio(explainer_xgb, 139 | N = N, B = B, show_info = v), "r2d3") 140 | }) 141 | } 142 | -------------------------------------------------------------------------------- /tests/testthat/test_ms_options.R: -------------------------------------------------------------------------------- 1 | context("Check options parameter and ms_options() function") 2 | 3 | source("test_objects.R") 4 | 5 | op <- modelStudio::ms_options() 6 | 7 | testthat::test_that("ms_options()", { 8 | testthat::expect_is(op, "list") 9 | testthat::expect_true(length(op) > 30) 10 | }) 11 | 12 | new_options <- modelStudio::ms_options( 13 | scale_plot = FALSE, 14 | show_subtitle = TRUE, 15 | subtitle = "hello", 16 | margin_top = 51, 17 | margin_right = 21, 18 | margin_bottom = 71, 19 | margin_left = 106, 20 | margin_inner = 410, 21 | margin_small = 6, 22 | margin_big = 11, 23 | w = 421, 24 | h = 281, 25 | bar_width = 17, 26 | line_size = 3, 27 | point_size = 4, 28 | bar_color = "red", 29 | line_color = "orange", 30 | point_color = "red", 31 | positive_color = "yellow", 32 | negative_color = "black", 33 | default_color = "green", 34 | bd_title = "Break Down2", 35 | bd_subtitle = "Break Down3", 36 | bd_bar_width = 6, 37 | bd_positive_color = NULL, 38 | bd_negative_color = NULL, 39 | bd_default_color = NULL, 40 | sv_title = "Shapley Values2", 41 | sv_subtitle = "Shapley Values3", 42 | sv_bar_width = 6, 43 | sv_positive_color = NULL, 44 | sv_negative_color = NULL, 45 | sv_default_color = NULL, 46 | cp_title = "Ceteris Paribus2", 47 | cp_subtitle = "Ceteris Paribus3", 48 | cp_bar_width = NULL, 49 | cp_line_size = NULL, 50 | cp_point_size = NULL, 51 | cp_bar_color = "black", 52 | cp_line_color = "black", 53 | cp_point_color = "black", 54 | fi_title = "Feature Importance2", 55 | fi_subtitle = "Feature Importance3", 56 | fi_bar_width = 6, 57 | fi_bar_color = NULL, 58 | pd_title = "Partial Dependency2", 59 | pd_subtitle = "Partial Dependency3", 60 | pd_bar_width = 6, 61 | pd_line_size = 8, 62 | pd_bar_color = NULL, 63 | pd_line_color = NULL, 64 | ad_title = "Accumulated Dependency2", 65 | ad_subtitle = "Accumulated Dependency3", 66 | ad_bar_width = 6, 67 | ad_line_size = 8, 68 | ad_bar_color = NULL, 69 | ad_line_color = NULL, 70 | fd_title = "Feature Distribution2", 71 | fd_subtitle = "Feature Distribution3", 72 | fd_bar_width = 6, 73 | fd_bar_color = NULL 74 | ) 75 | 76 | ms <- modelStudio::modelStudio(explain_rf, apartments[1:2,], 77 | facet_dim = c(2,3), N = 5, B = 2, show_info = v, 78 | options = new_options) 79 | 80 | testthat::test_that("options parameter", { 81 | testthat::expect_is(ms, "r2d3") 82 | }) 83 | -------------------------------------------------------------------------------- /tests/testthat/test_ms_update_observations.R: -------------------------------------------------------------------------------- 1 | context("Check ms_update_observations() function") 2 | 3 | source("test_objects.R") 4 | 5 | if (requireNamespace("ranger", quietly=TRUE)) { 6 | ms <- modelStudio::modelStudio(explain_rf, apartments[1:2,], N = 5, B = 2, show_info = v) 7 | 8 | testthat::test_that("modelStudio class", { 9 | testthat::expect_is(ms, "modelStudio") 10 | testthat::expect_silent(ms) 11 | }) 12 | 13 | testthat::test_that("ms_update_observations", { 14 | testthat::expect_silent(new_ms1 <- modelStudio::ms_update_observations(ms, explain_rf, B = 2, 15 | show_info = v)) 16 | testthat::expect_is(new_ms1, "modelStudio") 17 | 18 | testthat::expect_silent(new_ms2 <- modelStudio::ms_update_observations(ms, explain_rf, B = 2, 19 | show_info = v, 20 | new_observation = apartments[100:101,], 21 | overwrite = FALSE)) 22 | testthat::expect_is(new_ms2, "modelStudio") 23 | 24 | testthat::expect_silent(new_ms3 <- modelStudio::ms_update_observations(ms, explain_rf, B = 2, 25 | show_info = v, 26 | new_observation = apartments[1:2,], 27 | overwrite = TRUE)) 28 | testthat::expect_is(new_ms3, "modelStudio") 29 | 30 | testthat::test_that("ms_merge_observations", { 31 | testthat::expect_silent(merged_ms <- ms_merge_observations(new_ms1, new_ms2, new_ms3)) 32 | testthat::expect_is(merged_ms, "modelStudio") 33 | }) 34 | }) 35 | } 36 | -------------------------------------------------------------------------------- /tests/testthat/test_ms_update_options.R: -------------------------------------------------------------------------------- 1 | context("Check ms_update_options() function") 2 | 3 | source("test_objects.R") 4 | 5 | if (requireNamespace("ranger", quietly=TRUE)) { 6 | ms <- modelStudio::modelStudio(explain_rf, apartments[1:2,], 7 | facet_dim = c(2,3), N = 5, B = 2, show_info = v) 8 | 9 | testthat::test_that("modelStudio class", { 10 | testthat::expect_is(ms, "modelStudio") 11 | testthat::expect_silent(modelStudio::modelStudio(explain_rf, apartments[1:2,], 12 | facet_dim = c(2,3), N = 5, B = 2, show_info = v)) 13 | }) 14 | 15 | new_ms <- modelStudio::ms_update_options(ms, facet_dim = c(1,2), time = 0, 16 | margin_left = 100) 17 | 18 | testthat::test_that("ms_update_options", { 19 | testthat::expect_is(new_ms, "modelStudio") 20 | testthat::expect_equal(new_ms$x$options$time, 0) 21 | testthat::expect_equal(new_ms$x$options$margin_left, 100) 22 | testthat::expect_equal(new_ms$x$options$facet_dim, c(1,2)) 23 | }) 24 | 25 | testthat::test_that("ms_update_options run", { 26 | testthat::expect_silent(modelStudio::ms_update_options(ms, facet_dim = c(1,2), time = 0, 27 | margin_left = 100, show_info = v)) 28 | }) 29 | } 30 | -------------------------------------------------------------------------------- /tests/testthat/test_objects.R: -------------------------------------------------------------------------------- 1 | context("Objects for tests") 2 | 3 | titanic <- na.omit(DALEX::titanic) 4 | apartments <- DALEX::apartments 5 | set.seed(1313) 6 | v <- FALSE 7 | 8 | ### README DEMO 9 | 10 | # Create a model 11 | model_titanic_glm <- glm(survived ~., 12 | data = DALEX::titanic_imputed, 13 | family = "binomial") 14 | 15 | # Wrap it into an explainer 16 | explain_titanic_glm <- DALEX::explain(model_titanic_glm, 17 | data = DALEX::titanic_imputed[,-8], 18 | y = DALEX::titanic_imputed[,8], 19 | label = "Titanic GLM", 20 | verbose = v) 21 | 22 | # Pick some data points 23 | new_observations <- DALEX::titanic_imputed[1:4,] 24 | rownames(new_observations) <- c("Lucas", "James", "Thomas", "Nancy") 25 | 26 | 27 | ### glm + titanic 28 | 29 | titanic_test <- titanic[sample(1:nrow(titanic), 500),] 30 | 31 | model_glm <- glm(survived == "yes" ~., 32 | data = titanic_test, family = "binomial") 33 | 34 | explain_glm <- DALEX::explain(model_glm, 35 | data = titanic_test[,-9], 36 | y = titanic_test$survived == "yes", 37 | label = "glm", 38 | verbose = v) 39 | 40 | glm_numerical <- glm(survived == "yes" ~ age + fare + sibsp + parch, 41 | data = titanic_test[, c(2,6,7,8,9)], 42 | family = "binomial") 43 | 44 | explain_glm_numerical <- DALEX::explain(glm_numerical, 45 | data = titanic_test[, c(2,6,7,8)], 46 | y = titanic_test$survived == "yes", 47 | verbose = v) 48 | 49 | glm_not_numerical <- glm(survived == "yes" ~ gender + class + embarked + country, 50 | data = titanic_test[, c(1,3,4,5,9)], 51 | family = "binomial") 52 | 53 | explain_glm_not_numerical <- DALEX::explain(glm_not_numerical, 54 | data = titanic_test[, c(1,3,4,5)], 55 | y = titanic_test$survived == "yes", 56 | verbose = v) 57 | 58 | model_small <- glm(survived == "yes" ~ age + gender, 59 | data = titanic_test[, c(1,2,9)], 60 | family = "binomial") 61 | 62 | explain_model_small <- DALEX::explain(model_small, 63 | data = titanic_test[, c(1,2)], 64 | y = titanic_test$survived == "yes", 65 | verbose = v) 66 | 67 | 68 | ### ranger + apartments 69 | 70 | if (requireNamespace("ranger", quietly=TRUE)) { 71 | model_rf <- ranger::ranger(m2.price ~. , data = apartments) 72 | explain_rf <- DALEX::explain(model_rf, 73 | data = apartments, 74 | y = apartments$m2.price, 75 | verbose = v) 76 | } 77 | 78 | 79 | ### data/new_observation permutations 80 | 81 | titanic_small <- DALEX::titanic_imputed[1:500,] 82 | x <- titanic_small[,-8] 83 | nx <- titanic_small[1,-8] 84 | 85 | z <- titanic_small[,] 86 | nz <- titanic_small[1,] 87 | 88 | w <- titanic_small[,1:3] 89 | nw <- titanic_small[1,1:3] 90 | 91 | y <- titanic_small[,8] 92 | 93 | explain_both_without_target <- DALEX::explain(model_titanic_glm, 94 | data = x, 95 | y = y, 96 | verbose = v) 97 | 98 | explain_both_full <- DALEX::explain(model_titanic_glm, 99 | data = z, 100 | y = y, 101 | verbose = v) 102 | 103 | explain_obs_without_target_data_full <- DALEX::explain(model_titanic_glm, 104 | data = z, 105 | y = y, 106 | verbose = v) 107 | 108 | explain_obs_full_data_without_target <- DALEX::explain(model_titanic_glm, 109 | data = x, 110 | y = y, 111 | verbose = v) 112 | ### more than 10 features 113 | 114 | n <- 50 115 | artifficial <- data.frame(x1 = rnorm(n), 116 | x2 = rnorm(n), 117 | x3 = rnorm(n), 118 | x4 = rnorm(n), 119 | x5 = rnorm(n), 120 | x6 = rnorm(n), 121 | x7 = rnorm(n), 122 | x8 = rnorm(n), 123 | x9 = runif(n), 124 | x10 = runif(n), 125 | x11 = runif(n), 126 | y = rbinom(n, 1, prob = 0.4)) 127 | 128 | model_artifficial <- glm(y ~., 129 | data = artifficial, 130 | family = "binomial") 131 | 132 | explain_artifficial <- DALEX::explain(model_artifficial, 133 | data = artifficial[,-12], 134 | y = artifficial[,12], 135 | verbose = v) 136 | 137 | if (requireNamespace("xgboost", quietly=TRUE)) { 138 | ### xgboost matrix 139 | model_matrix_train <- model.matrix(status == "fired" ~ . -1, DALEX::HR) 140 | data_train <- xgboost::xgb.DMatrix(model_matrix_train, label = DALEX::HR$status == "fired") 141 | 142 | param <- list(max_depth = 2, eta = 1, nthread = 2, 143 | objective = "binary:logistic", eval_metric = "auc") 144 | HR_xgb_model <- xgboost::xgb.train(param, data_train, nrounds = 50) 145 | 146 | explainer_xgb <- DALEX::explain(HR_xgb_model, data = model_matrix_train, 147 | y = DALEX::HR$status == "fired", label = "xgboost", 148 | verbose = v) 149 | } 150 | -------------------------------------------------------------------------------- /tests/testthat/test_warnings_and_errors.R: -------------------------------------------------------------------------------- 1 | context("Check functions for warnings and errors") 2 | 3 | source("test_objects.R") 4 | 5 | testthat::test_that("new_observation as list", { 6 | testthat::expect_warning( 7 | ms <- modelStudio::modelStudio(explain_glm, 8 | new_observation = as.list(titanic_test[1,-9]), 9 | show_info = v, B = 3) 10 | ) 11 | }) 12 | 13 | if (requireNamespace("xgboost", quietly=TRUE)) { 14 | testthat::test_that("check_single_prediction error", { 15 | testthat::expect_warning( 16 | testthat::expect_error( 17 | ms <- modelStudio::modelStudio(explainer_xgb, 18 | new_observation = model_matrix_train[1,], 19 | show_info = v, B = 3) 20 | ) 21 | ) 22 | }) 23 | } 24 | 25 | # deprecated since v1.1 (May 2020) 26 | # removed in v2.2 (July 2021) 27 | # testthat::test_that("deprecated modelStudioOptions", { 28 | # testthat::expect_warning( 29 | # ms <- modelStudio::modelStudioOptions() 30 | # ) 31 | # }) 32 | testthat::test_that("removed modelStudioOptions", { 33 | testthat::expect_error( 34 | ms <- modelStudio::modelStudioOptions() 35 | ) 36 | }) 37 | 38 | 39 | if (requireNamespace("ranger", quietly=TRUE)) { 40 | ms <- modelStudio::modelStudio(explain_rf, apartments[1:2,], N = 5, B = 2, show_info = v) 41 | new_ms <- modelStudio::ms_update_observations(ms, explain_rf, B = 2, show_info = v, 42 | new_observation = apartments[3,], 43 | new_observation_y = apartments$m2.price[3]) 44 | 45 | testthat::test_that("duplicated ids", { 46 | testthat::expect_is(new_ms, "modelStudio") 47 | testthat::expect_warning( 48 | ms <- modelStudio::ms_update_observations(ms, explain_rf, B = 2, show_info = v, 49 | new_observation = apartments[1,], 50 | new_observation_y = apartments$m2.price[1]) 51 | ) 52 | }) 53 | 54 | } 55 | -------------------------------------------------------------------------------- /vignettes/ms-perks-features.Rmd: -------------------------------------------------------------------------------- 1 | --- 2 | title: "modelStudio - perks and features" 3 | author: "Hubert Baniecki" 4 | date: "`r Sys.Date()`" 5 | output: rmarkdown::html_vignette 6 | vignette: > 7 | %\VignetteIndexEntry{modelStudio - perks and features} 8 | %\VignetteEngine{knitr::rmarkdown} 9 | %\VignetteEncoding{UTF-8} 10 | --- 11 | 12 | ```{r setup, include = FALSE} 13 | knitr::opts_chunk$set( 14 | collapse = FALSE, 15 | comment = "#>", 16 | warning = FALSE, 17 | message = FALSE 18 | ) 19 | ``` 20 | 21 | The `modelStudio()` function computes various (instance and dataset level) model explanations 22 | and produces a customisable dashboard, which consists of multiple panels for plots with their 23 | short descriptions. Easily save the dashboard and share it with others. Tools for 24 | [Explanatory Model Analysis](https://ema.drwhy.ai/) unite with tools for Exploratory Data Analysis 25 | to give a broad overview of the model behavior. 26 | 27 | Let's use `HR` dataset to explore `modelStudio` parameters: 28 | 29 | ```{r results="hide"} 30 | train <- DALEX::HR 31 | train$fired <- as.factor(ifelse(train$status == "fired", 1, 0)) 32 | train$status <- NULL 33 | 34 | head(train) 35 | ``` 36 | 37 | ```{r echo = FALSE, fig.align='center'} 38 | knitr::kable(head(train), digits = 2, caption = "DALEX::HR dataset") 39 | ``` 40 | 41 | Prepare `HR_test` data and a `ranger` model for the explainer: 42 | 43 | ```{r results="hide", eval = FALSE} 44 | # fit a ranger model 45 | library("ranger") 46 | model <- ranger(fired ~., data = train, probability = TRUE) 47 | 48 | # prepare validation dataset 49 | test <- DALEX::HR_test[1:1000,] 50 | test$fired <- ifelse(test$status == "fired", 1, 0) 51 | test$status <- NULL 52 | 53 | # create an explainer for the model 54 | explainer <- DALEX::explain(model, 55 | data = test, 56 | y = test$fired) 57 | 58 | # start modelStudio 59 | library("modelStudio") 60 | ``` 61 | 62 | ------------------------------------------------------------------- 63 | 64 | ## modelStudio parameters 65 | 66 | ### instance explanations 67 | 68 | Pass data points to the `new_observation` parameter for instance explanations 69 | such as [Break Down](https://ema.drwhy.ai/breakDown.html), 70 | [Shapley Values](https://ema.drwhy.ai/shapley.html) and 71 | [Ceteris Paribus](https://ema.drwhy.ai/ceterisParibus.html) Profiles. 72 | Use `new_observation_y` to show their true labels. 73 | 74 | ```{r eval = FALSE} 75 | new_observation <- test[1:3,] 76 | rownames(new_observation) <- c("John Snow", "Arya Stark", "Samwell Tarly") 77 | true_labels <- test[1:3,]$fired 78 | 79 | modelStudio(explainer, 80 | new_observation = new_observation, 81 | new_observation_y = true_labels) 82 | ``` 83 | 84 | If `new_observation = NULL`, then choose `new_observation_n` observations, evenly spread by the order of `y_hat`. This shall always include the observations, which ids are `which.min(y_hat)` and `which.max(y_hat)`. 85 | 86 | ```{r eval = FALSE} 87 | modelStudio(explainer, new_observation_n = 5) # default is 3 88 | ``` 89 | 90 | ### grid size 91 | 92 | Achieve bigger or smaller `modelStudio` grid with `facet_dim` parameter. 93 | 94 | ```{r eval = FALSE} 95 | # small dashboard with 2 panels 96 | modelStudio(explainer, 97 | facet_dim = c(1,2)) 98 | 99 | # large dashboard with 9 panels 100 | modelStudio(explainer, 101 | facet_dim = c(3,3)) 102 | ``` 103 | 104 | ### animations 105 | 106 | Manipulate `time` parameter to set animation length. Value 0 will make 107 | them invisible. 108 | 109 | ```{r eval = FALSE} 110 | # slow down animations 111 | modelStudio(explainer, 112 | time = 1000) 113 | 114 | # turn off animations 115 | modelStudio(explainer, 116 | time = 0) 117 | ``` 118 | 119 | ### more calculations means more time 120 | 121 | - `N` is a number of observations used for calculation of 122 | [Partial Dependence](https://ema.drwhy.ai/partialDependenceProfiles.html) 123 | and [Accumulated Dependence](https://ema.drwhy.ai/accumulatedLocalProfiles.html) Profiles (default is `300`). 124 | - `N_fi` is a number of observations used for calculation of 125 | [Feature Importance](https://ema.drwhy.ai/featureImportance.html) (default is `N*10`). 126 | - `N_sv` is a number of observations used for calculation of 127 | [Shapley Values](https://ema.drwhy.ai/shapley.html) (default is `N*3`). 128 | - `B` is a number of permutation rounds used for calculation of 129 | [Shapley Values](https://ema.drwhy.ai/shapley.html) (default is `10`). 130 | - `B_fi` is a number of permutation rounds used for calculation of 131 | [Feature Importance](https://ema.drwhy.ai/featureImportance.html) (default is `B`). 132 | 133 | Decrease `N` and `B` parameters to lower the computation time or increase 134 | them to get more accurate empirical results. 135 | 136 | ```{r eval = FALSE} 137 | # faster, less precise 138 | modelStudio(explainer, 139 | N = 200, B = 5) 140 | 141 | # slower, more precise 142 | modelStudio(explainer, 143 | N = 500, B = 15) 144 | ``` 145 | 146 | ### no EDA mode 147 | 148 | Don't compute the EDA plots if they are not needed. Set the `eda` parameter to `FALSE`. 149 | 150 | ```{r eval = FALSE} 151 | modelStudio(explainer, 152 | eda = FALSE) 153 | ``` 154 | 155 | ### progress bar 156 | 157 | Hide computation progress bar messages with `show_info` parameter. 158 | 159 | ```{r eval = FALSE} 160 | modelStudio(explainer, 161 | show_info = FALSE) 162 | ``` 163 | 164 | ### viewer or browser? 165 | 166 | Change `viewer` parameter to set where to display `modelStudio`. 167 | [Best described in `r2d3` documentation](https://rstudio.github.io/r2d3/articles/visualization_options.html#viewer). 168 | 169 | ```{r eval = FALSE} 170 | modelStudio(explainer, 171 | viewer = "browser") 172 | ``` 173 | 174 | ------------------------------------------------------------------- 175 | 176 | ## parallel computation 177 | 178 | Speed up `modelStudio` computation by setting `parallel` parameter to `TRUE`. 179 | It uses [`parallelMap`](https://www.rdocumentation.org/packages/parallelMap) package 180 | to calculate local explainers faster. It is really useful when using `modelStudio` with 181 | complicated models, vast datasets or **many observations are being processed**. 182 | 183 | All options can be set outside of the function call. 184 | [How to use parallelMap](https://github.com/mlr-org/parallelMap#being-lazy-configuration). 185 | 186 | ```{r eval = FALSE} 187 | # set up the cluster 188 | options( 189 | parallelMap.default.mode = "socket", 190 | parallelMap.default.cpus = 4, 191 | parallelMap.default.show.info = FALSE 192 | ) 193 | 194 | # calculations of local explanations will be distributed into 4 cores 195 | modelStudio(explainer, 196 | new_observation = test[1:16,], 197 | parallel = TRUE) 198 | ``` 199 | 200 | -------------------------------------------------------------------- 201 | 202 | ## additional options 203 | 204 | Customize some of the `modelStudio` looks by overwriting default options returned 205 | by the `ms_options()` function. 206 | [Full list of options](https://modelstudio.drwhy.ai/reference/ms_options.html). 207 | 208 | ```{r eval = FALSE} 209 | # set additional graphical parameters 210 | new_options <- ms_options( 211 | show_subtitle = TRUE, 212 | bd_subtitle = "Hello World", 213 | line_size = 5, 214 | point_size = 9, 215 | line_color = "pink", 216 | point_color = "purple", 217 | bd_positive_color = "yellow", 218 | bd_negative_color = "orange" 219 | ) 220 | 221 | modelStudio(explainer, 222 | options = new_options) 223 | ``` 224 | 225 | All visual options can be changed after the calculations using `ms_update_options()`. 226 | 227 | ```{r eval = FALSE} 228 | old_ms <- modelStudio(explainer) 229 | old_ms 230 | 231 | # update the options 232 | new_ms <- ms_update_options(old_ms, 233 | time = 0, 234 | facet_dim = c(1,2), 235 | margin_left = 150) 236 | new_ms 237 | ``` 238 | 239 | ------------------------------------------------------------------- 240 | 241 | ## update observations 242 | 243 | Use `ms_update_observations()` to add more observations with their local explanations to the `modelStudio`. 244 | 245 | ```{r eval = FALSE} 246 | old_ms <- modelStudio(explainer) 247 | old_ms 248 | 249 | # add new observations 250 | plus_ms <- ms_update_observations(old_ms, 251 | explainer, 252 | new_observation = test[101:102,]) 253 | plus_ms 254 | 255 | # overwrite old observations 256 | new_ms <- ms_update_observations(old_ms, 257 | explainer, 258 | new_observation = test[103:104,], 259 | overwrite = TRUE) 260 | new_ms 261 | ``` 262 | 263 | ------------------------------------------------------------------- 264 | 265 | ## Shiny 266 | 267 | Use the `widget_id` argument and `r2d3` package to render the `modelStudio` output in Shiny. 268 | See [Using r2d3 with Shiny](https://rstudio.github.io/r2d3/articles/shiny.html) and consider 269 | the following example: 270 | 271 | ```{r eval = FALSE} 272 | library(shiny) 273 | library(r2d3) 274 | 275 | 276 | ui <- fluidPage( 277 | textInput("text", h3("Text input"), 278 | value = "Enter text..."), 279 | uiOutput('dashboard') 280 | ) 281 | 282 | server <- function(input, output) { 283 | #:# id of div where modelStudio will appear 284 | WIDGET_ID = 'MODELSTUDIO' 285 | 286 | #:# create modelStudio 287 | library(modelStudio) 288 | library(DALEX) 289 | model <- glm(survived ~., data = titanic_imputed, family = "binomial") 290 | explainer <- DALEX::explain(model, 291 | data = titanic_imputed, 292 | y = titanic_imputed$survived, 293 | label = "Titanic GLM", 294 | verbose = FALSE) 295 | ms <- modelStudio(explainer, 296 | widget_id = WIDGET_ID, #:# use the widget_id 297 | show_info = FALSE) 298 | ms$elementId <- NULL #:# remove elementId to stop the warning 299 | 300 | #:# basic render d3 output 301 | output[[WIDGET_ID]] <- renderD3({ 302 | ms 303 | }) 304 | 305 | #:# use render ui to set proper width and height 306 | output$dashboard <- renderUI({ 307 | d3Output(WIDGET_ID, width=ms$width, height=ms$height) 308 | }) 309 | } 310 | 311 | shinyApp(ui = ui, server = server) 312 | ``` 313 | 314 | ------------------------------------------------------------------- 315 | 316 | ## DALEXtra 317 | 318 | Use `explain_*()` functions from the [DALEXtra](https://github.com/ModelOriented/DALEXtra/) 319 | package to explain various models. 320 | 321 | Bellow basic example of making `modelStudio` for a `mlr` model using `explain_mlr()`. 322 | 323 | ```{r eval = FALSE} 324 | library(DALEXtra) 325 | library(mlr) 326 | 327 | # fit a model 328 | task <- makeClassifTask(id = "task", data = train, target = "fired") 329 | learner <- makeLearner("classif.ranger", predict.type = "prob") 330 | model <- train(learner, task) 331 | 332 | # create an explainer for the model 333 | explainer_mlr <- explain_mlr(model, 334 | data = test, 335 | y = test$fired, 336 | label = "mlr") 337 | 338 | # make a studio for the model 339 | modelStudio(explainer_mlr) 340 | ``` 341 | 342 | ------------------------------------------------------------------- 343 | 344 | ## References 345 | 346 | * Theoretical introduction to the plots: [Explanatory Model Analysis. Explore, Explain, and Examine Predictive Models.](https://ema.drwhy.ai/) 347 | * The input object is implemented in [DALEX](https://modeloriented.github.io/DALEX/) 348 | * Feature Importance, Ceteris Paribus, Partial Dependence and Accumulated Dependence explanations 349 | are implemented in [ingredients](https://modeloriented.github.io/ingredients/) 350 | * Break Down and Shapley Values explanations are implemented in [iBreakDown](https://modeloriented.github.io/iBreakDown/) 351 | -------------------------------------------------------------------------------- /vignettes/ms-rmarkdown.Rmd: -------------------------------------------------------------------------------- 1 | --- 2 | title: "modelStudio in R Markdown HTML" 3 | author: "Hubert Baniecki" 4 | date: "`r Sys.Date()`" 5 | output: rmarkdown::html_vignette 6 | vignette: > 7 | %\VignetteIndexEntry{modelStudio in R Markdown HTML} 8 | %\VignetteEngine{knitr::rmarkdown} 9 | %\VignetteEncoding{UTF-8} 10 | --- 11 | 12 | 13 | 19 | 20 | 21 | ```{r setup, include=FALSE} 22 | knitr::opts_chunk$set(echo = TRUE) 23 | ``` 24 | 25 | To properly generate `modelStudio` in R Markdown HTML, add additional CSS into the `.Rmd` file. 26 | 27 | One can either make the space wider, e.g.: 28 | 29 | ``` 30 | 41 | ``` 42 | 43 | Or/and move the dashboard to the left, e.g.: 44 | 45 | ``` 46 | 54 | ``` 55 | 56 | Then, proceed to generate the `modelStudio`: 57 | 58 | ```{r warning = FALSE, message = FALSE} 59 | library("DALEX") 60 | library("ranger") 61 | library("modelStudio") 62 | 63 | # fit a model 64 | model <- ranger(score ~., data = happiness_train) 65 | 66 | # create an explainer for the model 67 | explainer <- explain(model, 68 | data = happiness_test, 69 | y = happiness_test$score, 70 | label = "Random Forest", 71 | verbose = FALSE) 72 | 73 | # make a studio for the model 74 | modelStudio(explainer) 75 | ``` --------------------------------------------------------------------------------