├── docs ├── .nojekyll ├── collaborative-coding │ ├── index.md │ └── shared-repl-server.md ├── reference │ └── index.md ├── assets │ ├── favicon.ico │ ├── favicon-16x16.png │ ├── favicon-32x32.png │ ├── apple-touch-icon.png │ ├── android-chrome-192x192.png │ ├── android-chrome-512x512.png │ ├── images │ │ ├── practicalli-logo.png │ │ └── social │ │ │ └── README.md │ ├── stylesheets │ │ ├── extra.css │ │ └── practicalli-light.css │ ├── favicon.svg │ └── practicalli-logo.svg ├── r │ └── index.md ├── introduction │ ├── concepts │ │ └── index.md │ ├── clojure-libraries.md │ ├── data-science.md │ ├── data-science-books.md │ ├── online-courses.md │ ├── what-is-clojure.md │ ├── persona.md │ ├── contributing.md │ ├── repl-workflow.md │ └── writing-tips.md ├── data-transformation │ ├── index.md │ └── tablecloth │ │ └── index.md ├── visualisation │ └── index.md ├── integration │ └── python │ │ └── index.md ├── artificial-intelligence │ └── index.md ├── data-mining │ ├── webscraping │ │ ├── enlive.md │ │ ├── table-data.md │ │ └── index.md │ └── index.md ├── notebooks │ ├── clerk │ │ └── index.md │ ├── next-journal │ │ └── index.md │ ├── index.md │ └── notespace │ │ ├── index.md │ │ └── configure-notespace-project.md ├── mathematics │ ├── linear-algebra │ │ └── index.md │ └── index.md ├── install │ └── index.md ├── index.md └── natural-language │ └── index.md ├── includes └── abbreviations.md ├── .github ├── CODEOWNERS ├── config │ ├── secretlintrc.json │ ├── markdown-link-check.json │ ├── gitleaks.toml │ ├── lychee.toml │ ├── megalinter.yaml │ └── markdown-lint.jsonc ├── pull_request_template.md └── workflows │ ├── changelog-check.yaml │ ├── scheduled-version-check.yaml │ ├── publish-book.yaml │ ├── scheduled-stale-check.yaml │ └── megalinter.yaml ├── CHANGELOG.md ├── overrides ├── main.html ├── partials │ ├── source.html │ ├── palette.html │ └── header.html └── 404.html ├── .gitattributes ├── .gitignore ├── SUMMARY.md ├── Makefile ├── README.md ├── mkdocs.yml └── LICENSE /docs/.nojekyll: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /docs/collaborative-coding/index.md: -------------------------------------------------------------------------------- 1 | # Collaborative Coding 2 | 3 | 4 | * Shared REPL Server 5 | -------------------------------------------------------------------------------- /includes/abbreviations.md: -------------------------------------------------------------------------------- 1 | *[HTML]: Hyper Text Markup Language 2 | *[W3C]: World Wide Web Consortium 3 | -------------------------------------------------------------------------------- /docs/reference/index.md: -------------------------------------------------------------------------------- 1 | # Reference 2 | 3 | Terminology, useful background information and external links. 4 | -------------------------------------------------------------------------------- /docs/assets/favicon.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/practicalli/clojure-data-science/main/docs/assets/favicon.ico -------------------------------------------------------------------------------- /docs/assets/favicon-16x16.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/practicalli/clojure-data-science/main/docs/assets/favicon-16x16.png -------------------------------------------------------------------------------- /docs/assets/favicon-32x32.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/practicalli/clojure-data-science/main/docs/assets/favicon-32x32.png -------------------------------------------------------------------------------- /docs/assets/apple-touch-icon.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/practicalli/clojure-data-science/main/docs/assets/apple-touch-icon.png -------------------------------------------------------------------------------- /docs/r/index.md: -------------------------------------------------------------------------------- 1 | # R 2 | 3 | R is a programming language... 4 | 5 | and something Pirates say, especially on International Pirates day. 6 | -------------------------------------------------------------------------------- /docs/assets/android-chrome-192x192.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/practicalli/clojure-data-science/main/docs/assets/android-chrome-192x192.png -------------------------------------------------------------------------------- /docs/assets/android-chrome-512x512.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/practicalli/clojure-data-science/main/docs/assets/android-chrome-512x512.png -------------------------------------------------------------------------------- /docs/assets/images/practicalli-logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/practicalli/clojure-data-science/main/docs/assets/images/practicalli-logo.png -------------------------------------------------------------------------------- /docs/introduction/concepts/index.md: -------------------------------------------------------------------------------- 1 | # Concepts 2 | 3 | Data engineering gathers and organises information in a form that is suitable for data science 4 | -------------------------------------------------------------------------------- /.github/CODEOWNERS: -------------------------------------------------------------------------------- 1 | # Codeowners 2 | 3 | # Default owner accounts for the current repository 4 | # Automatically added as a reviewr to all pull requests (not including drafts) 5 | 6 | * @practicalli-johnny 7 | -------------------------------------------------------------------------------- /docs/data-transformation/index.md: -------------------------------------------------------------------------------- 1 | # Data Transformation 2 | 3 | - Clojure core functions for general data transformation 4 | - [TableCloth](tablecloth/index.md) - create table based data structures for data science models 5 | -------------------------------------------------------------------------------- /docs/visualisation/index.md: -------------------------------------------------------------------------------- 1 | # Visualisation 2 | 3 | When data is visualised effectively then a meaningful story can be told, explaining the ramifications of the source data. 4 | 5 | 6 | - telling stories 7 | - plotting graphs 8 | - Oz 9 | - Hanami 10 | -------------------------------------------------------------------------------- /docs/data-transformation/tablecloth/index.md: -------------------------------------------------------------------------------- 1 | ## Data Transformation - TableCloth 2 | 3 | 4 | TableCloth - an abstractions library for tech.ml which is optomised to working with data in table form. Data science projects often work with large data sets in table format. 5 | -------------------------------------------------------------------------------- /docs/integration/python/index.md: -------------------------------------------------------------------------------- 1 | # Integration: Python 2 | 3 | 4 | 5 | ## References 6 | * [Clojure Interop With Python NLP Libraries](http://gigasquidsoftware.com/blog/2020/01/24/clojure-interop-with-python-nlp-libraries/) 7 | * [Parens for Polyglog](https://nextjournal.com/kommen/parens-for-polyglot) 8 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # Changelog 2 | 3 | ## Unreleased 4 | ### Added 5 | - dev: add scheduled stale issue & pull request check (monthly) 6 | 7 | ### Changed 8 | - ci: spell lychee & repository trufflehog linters warn only (false positives) 9 | - dev: GitHub workflows checkout action v4 with sparce-checkout 10 | -------------------------------------------------------------------------------- /docs/artificial-intelligence/index.md: -------------------------------------------------------------------------------- 1 | # Artificial Intelligence 2 | 3 | Neural networks are a collection of predicate functions that are assembled into a network 4 | 5 | AI is the future we are trying to build 6 | machine learning, neural networks, natural language programming are the practical tools and approaches that we are applying to solve AI 7 | -------------------------------------------------------------------------------- /overrides/main.html: -------------------------------------------------------------------------------- 1 | 7 | 8 | 9 | {% extends "base.html" %} 10 | 11 | 12 | {% block announce %} 13 | 14 | ⚠ Practicalli Clojure is simply a brain dump of ideas 15 | 16 | {% endblock %} 17 | -------------------------------------------------------------------------------- /.gitattributes: -------------------------------------------------------------------------------- 1 | # Configure Languages for GitHub repository using Linguist 2 | 3 | # Markdown & Make detection, 4 | # exclude HTML, CSS & JavaScript 5 | docs/** linguist-detectable 6 | *.md linguist-detectable=true 7 | make linguist-detectable=true 8 | *.css linguist-detectable=false 9 | *.js linguist-detectable=false 10 | *.html linguist-detectable=false 11 | -------------------------------------------------------------------------------- /.github/config/secretlintrc.json: -------------------------------------------------------------------------------- 1 | { 2 | "rules": [ 3 | { 4 | "id": "@secretlint/secretlint-rule-basicauth", 5 | "options": { 6 | "allows": [ 7 | "hostname.domain.com", 8 | "jdbc:postgresql://:port/?user=&password=", 9 | "postgres://postgres://username:password@hostname.domain.com:1234/database-name" 10 | ] 11 | } 12 | } 13 | ] 14 | } 15 | -------------------------------------------------------------------------------- /docs/assets/images/social/README.md: -------------------------------------------------------------------------------- 1 | # Social Cards 2 | 3 | Social Cards are visual previews of the website that are included when sending links via social media platforms. 4 | 5 | Material for MkDocs is [configured to generate beautiful social cards automatically](https://squidfunk.github.io/mkdocs-material/setup/setting-up-social-cards/), using the colors, fonts and logos defined in `mkdocs.yml` 6 | 7 | Generated images are stored in this directory. 8 | -------------------------------------------------------------------------------- /overrides/partials/source.html: -------------------------------------------------------------------------------- 1 | 2 |
3 | {% set icon = config.theme.icon.repo or "fontawesome/brands/git-alt" %} 4 | {% include ".icons/" ~ icon ~ ".svg" %} 5 |
6 |
7 | {{ config.repo_name }} 8 |
9 |
10 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Exclude all files from root directory 2 | /* 3 | 4 | # ------------------------ 5 | # Common project files 6 | !CHANGELOG.md 7 | !README.md 8 | !LICENSE 9 | 10 | # ------------------------ 11 | # Include MkDocs files 12 | !docs/ 13 | !includes/ 14 | !overrides/ 15 | !mkdocs.yml 16 | 17 | # ------------------------ 18 | # Project automation 19 | !Makefile 20 | 21 | # ------------------------ 22 | # Version Control 23 | !.gitignore 24 | !.gitattributes 25 | !.github/ 26 | 27 | -------------------------------------------------------------------------------- /docs/data-mining/webscraping/enlive.md: -------------------------------------------------------------------------------- 1 | # Webscraping with Enlive 2 | 3 | quickly selecting elements based on CSS and/or Xpath 4 | 5 | https://clojureverse.org/t/best-library-for-querying-html/1103 6 | 7 | ```clojure 8 | (require '[net.cgrand.enlive-html :as enlive]) 9 | 10 | (let [doc (enlive/html-snippet "
Hello, world
") 11 | em (enlive/select doc [:.hello :em])] 12 | (enlive/texts em)) 13 | ``` 14 | 15 | ## Including Enlive 16 | 17 | ## Searching for tags 18 | -------------------------------------------------------------------------------- /docs/introduction/clojure-libraries.md: -------------------------------------------------------------------------------- 1 | ## Clojure libraries for data science 2 | 3 | - Tablecloth 4 | - dtype-next 5 | - tech.viz 6 | - Oz 7 | - Hanami 8 | 9 | 10 | ## Integration with data science tools 11 | 12 | - [:fontawesome-brands-github: libpython-clj](https://github.com/clj-python/libpython-clj){target=_blank} - Deep Clojure/Python Integration 13 | - [:fontawesome-brands-youtube: Extending Clojure with Python - Chris Nuernberger](https://www.youtube.com/watch?v=vQPW16_jixs){target=_blank} 14 | -------------------------------------------------------------------------------- /.github/config/markdown-link-check.json: -------------------------------------------------------------------------------- 1 | { 2 | "ignorePatterns": [ 3 | { 4 | "pattern": "^http://localhost" 5 | }, 6 | { 7 | "pattern": "^mailto:*" 8 | }, 9 | { 10 | "pattern": "^#*" 11 | }, 12 | { 13 | "pattern": "^https://127.0.0.0/" 14 | } 15 | ], 16 | "timeout": "20s", 17 | "retryOn429": true, 18 | "retryCount": 5, 19 | "fallbackRetryDelay": "30s", 20 | "aliveStatusCodes": [ 21 | 200, 22 | 206 23 | ] 24 | } 25 | -------------------------------------------------------------------------------- /docs/notebooks/clerk/index.md: -------------------------------------------------------------------------------- 1 | # Notebooks: Clerk 2 | 3 | The soon to be released Clerk project is a notebook that is designed with local development as its main focus. 4 | 5 | 6 | 7 | 8 | ## ClojureD presentation 9 | 10 | 11 | 12 | 13 | ## References 14 | * Clerk author [Martin Kavalar](https://twitter.com/mkvlr) 15 | -------------------------------------------------------------------------------- /docs/data-mining/index.md: -------------------------------------------------------------------------------- 1 | # Data Mining 2 | 3 | The data required is often not provided in an easily consumable form, referred to as unstructured data. 4 | 5 | Data mining techniques like web-scraping and Extra-Transform-Load (ETL) tools can turn unstructured data into something useful. 6 | 7 | > The clojure.core library has a wide range of functions for transforming data into the right shape, as the Clojure language is a data structure. 8 | 9 | - scraping unstructured data sources 10 | - consuming open API data sources 11 | - highlighting reputable and high quality data sources 12 | 13 | -------------------------------------------------------------------------------- /.github/config/gitleaks.toml: -------------------------------------------------------------------------------- 1 | title = "gitleaks config" 2 | 3 | [allowlist] 4 | description = "global allow lists" 5 | paths = [ 6 | '''gitleaks.toml''', 7 | '''(.*?)(jpg|gif|doc|docx|zip|xls|pdf|bin|svg|socket)$''', 8 | '''(go.mod|go.sum)$''', 9 | '''gradle.lockfile''', 10 | '''node_modules''', 11 | '''package-lock.json''', 12 | '''pnpm-lock.yaml''', 13 | '''Database.refactorlog''', 14 | '''vendor''', 15 | ] 16 | 17 | [[rules]] 18 | description = "AWS Example API Key" 19 | id = "aws-example-api-key" 20 | regex = '''AKIAIOSFODNN7EXAMPLE''' 21 | keywords = [ 22 | "awstoken", 23 | ] 24 | -------------------------------------------------------------------------------- /docs/introduction/data-science.md: -------------------------------------------------------------------------------- 1 | # Data Science 2 | 3 | - Statistics and probability 4 | - Linear Algebra 5 | - Mathematical Optimisation 6 | - Calculus 7 | 8 | 9 | ## Future topics 10 | 11 | - Trend analysis 12 | - Principle component analysis 13 | - Basic Modelling (for machine learning) 14 | - Time series [tidyverts](https://tidyverts.org/) 15 | - Data generation - examples of how to generate sample data 16 | - Data sources - what are good data sources and what concepts are they good for exploring 17 | - Identify and simply explain data science concepts and provide understanding / examples of how they can be valuable 18 | -------------------------------------------------------------------------------- /docs/introduction/data-science-books.md: -------------------------------------------------------------------------------- 1 | # Data Science books 2 | 3 | There are many great data science books and Practicalli will curate a selection of books that are considered investing time into. 4 | 5 | ## Clojure specific data science books 6 | 7 | * [Linear Algebra for Programmers]](https://aiprobook.com/numerical-linear-algebra-for-programmers/) 8 | * [Deep Learning for Programmers](https://aiprobook.com/deep-learning-for-programmers/) 9 | * [Clojure for Data Science](https://www.packtpub.com/product/clojure-for-data-science/9781784397180) 10 | 11 | 12 | ## General Data Science books 13 | * [Python Data Science Handbook](https://jakevdp.github.io/PythonDataScienceHandbook/) 14 | * [R for Data Science](https://r4ds.had.co.nz/) 15 | -------------------------------------------------------------------------------- /docs/introduction/online-courses.md: -------------------------------------------------------------------------------- 1 | # Online Data Science Courses 2 | 3 | 4 | | Course | Math Level | Language | Tools | Summary | 5 | |--------------------------------------------------------------|-------------|----------|--------------------------------------------------------|-------------------------------| 6 | | [Practical Deep Learning for Coders](https://course.fast.ai) | High School | Python | [Jupiter notebook](https://github.com/fastai/fastbook) | AI Applications Without a PhD | 7 | | | | | | | 8 | -------------------------------------------------------------------------------- /docs/introduction/what-is-clojure.md: -------------------------------------------------------------------------------- 1 | # What is Clojure? 2 | 3 | Clojure is a general purpose programming language and is used for creating any applications, tools, services across any industry. 4 | 5 | Clojure is a hosted language, so it can run on a Java Virtual machine (especially for high volume data processing). Clojure can also run on JavaScript engines found in web browsers and nodejs. 6 | 7 | Clojure has a very simple syntax and is simple to write and maintain. 8 | 9 | Clojure is very data centric and the standard library has over 500 functions for manipulating and transforming data, so is ideal for data science tasks. 10 | 11 | [Practicalli Clojure will help you get started with Clojure](https://practicalli.github.io/clojure/) and help you create a Clojure development environment with a [Clojure aware editor](http://practicalli.github.io/clojure/clojure-editors/). 12 | -------------------------------------------------------------------------------- /docs/mathematics/linear-algebra/index.md: -------------------------------------------------------------------------------- 1 | # Mathematics: Linear Algebra 2 | 3 | Linear Algebra comprises of mathematical techniques that help efficiently calculate sums for large data sets 4 | 5 | * dot product 6 | * vectors 7 | * matrix transformation 8 | 9 | 10 | ## References 11 | * [Wikipedia: Linear Algebra](https://en.wikipedia.org/wiki/Linear_algebra) 12 | * [Khan Academy](https://www.khanacademy.org/math/linear-algebra) 13 | * [MIT Open Courseware: Linear Algebra](https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/) 14 | * [YouTube: Essence of Linear Algebra](https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab) 15 | * [UC Davis: Linear Algebra](https://www.math.ucdavis.edu/~linear/linear-guest.pdf) 16 | * FreeCodeCamp: Linear Algebra full college course [part 1](https://www.youtube.com/watch?v=JnTa9XtvmfI) and [part 2](https://www.youtube.com/watch?v=DJ6YwBN7Ya8) 17 | -------------------------------------------------------------------------------- /docs/notebooks/next-journal/index.md: -------------------------------------------------------------------------------- 1 | # Notbooks: NextJournal 2 | 3 | [NextJournal](https://nextjournal.com/) is a hosted Journal with wide support for programming languages (Clojure, Python, Scala, etc.). 4 | 5 | The [NextJournal](https://nextjournal.com/) platform has been developed with Clojure. 6 | 7 | Enjoy Version Control, Simple Sharing and Collaboration. Set Up Your Data Science Project Within Minutes. Simple Collaboration. Cloud Computing. Features: Automatic Provisioning & Shutdown, Data & Secrets Management, Rapid And Full Reusability. 8 | 9 | Runs anything you can put into a Docker container. Improve your workflow with polyglot notebooks, automatic versioning and real-time collaboration. Save time and money with on-demand provisioning, including GPU support. 10 | 11 | 12 | ![NextJournal Screenshot](https://cdn.nextjournal.com/data/QmWXcRowpRPVGXyeUCnF98EF19FcmEEQmwXUzXYy77J29b?filename=science.png&content-type=image/png) 13 | -------------------------------------------------------------------------------- /docs/data-mining/webscraping/table-data.md: -------------------------------------------------------------------------------- 1 | # Webscraping HTML table data 2 | 3 | Data is often published using HTML tables, as its a very simple language that non-developers can easily pick up. 4 | 5 | 6 | 7 | ## Data in a pre tag 8 | Using pre to create a table in a web page is similar to using a CSV except whitespace is used instead of commas to demarcated the individual elements. 9 | 10 | Data can be put into a vector, optionally grouping each line as a vector within a vector using ``partition`. 11 | 12 | If data should be associated with a name, then use the `map` function to convert the 13 | 14 | 15 | #### Example data 16 | Tables without headings 17 | * [Riverflow reconstructions for England and Wales](https://crudata.uea.ac.uk/cru/data/riverflow/) - University of Eas Anglia Climactic Research Unit 18 | 19 | Tables with headings and comments 20 | * [Cloud Cover by Country](https://crudata.uea.ac.uk/cru/data/hrg/cru_ts_4.01/crucy.1709191757.v4.01/countries/cld/) - University of Eas Anglia Climactic Research Unit 21 | -------------------------------------------------------------------------------- /.github/config/lychee.toml: -------------------------------------------------------------------------------- 1 | 2 | # ---------------------------------------- 3 | # Base URL or website root directory to check relative URLs. 4 | base = "https://practical.li/clojure-data-science" 5 | 6 | # Only test links with the given schemes (e.g. https). 7 | # Omit to check links with any other scheme. 8 | # At the moment, we support http, https, file, and mailto. 9 | scheme = ["https"] 10 | 11 | # ---------------------------------------- 12 | # Exclusions 13 | 14 | # Exclude URLs and mail addresses from checking (supports regex). 15 | exclude = ['^https://127.0.0.0', '^https://www\.linkedin\.com', '^https://web\.archive\.org/web/'] 16 | 17 | # Exclude these filesystem paths from getting checked. 18 | exclude_path = ["mkdocs.yml", "overrides", "includes", ".github", ".git"] 19 | 20 | # Exclude all private IPs from checking. 21 | # Equivalent to setting `exclude_private`, `exclude_link_local`, and 22 | # `exclude_loopback` to true. 23 | exclude_all_private = true 24 | 25 | # Check mail addresses 26 | include_mail = false 27 | # ---------------------------------------- 28 | -------------------------------------------------------------------------------- /overrides/404.html: -------------------------------------------------------------------------------- 1 | 4 | 5 | 6 | {% extends "main.html" %} 7 | 8 | 9 | {% block content %} 10 |

This is not the page you are looking for

11 | 12 |

13 | Sorry we have arrived at a page that does not exist... 14 |

15 | 16 |

17 | Practicalli Clojure Data Science is very early stages 18 |

19 | 20 |

21 | Use the Search bar at the top of the page or left navigation to find the relevant content. 22 |

23 | 38 | 39 | {% endblock %} 40 | -------------------------------------------------------------------------------- /overrides/partials/palette.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 |
6 | {% for option in config.theme.palette %} 7 | {% set scheme = option.scheme | d("default", true) %} 8 | {% set primary = option.primary | d("indigo", true) %} 9 | {% set accent = option.accent | d("indigo", true) %} 10 | 25 | {% if option.toggle %} 26 | 34 | {% endif %} 35 | {% endfor %} 36 |
37 | -------------------------------------------------------------------------------- /.github/pull_request_template.md: -------------------------------------------------------------------------------- 1 | :memo: Description 2 | 3 | 4 | :white_check_mark: Checklist 5 | 6 | - [ ] Commits should be cryptographically signed (SSH or GPG) 7 | 8 | 9 | ## Practicalli Guidelines 10 | 11 | Please follow these guidelines when submitting a pull request 12 | 13 | - refer to all relevant issues, using `#` followed by the issue number (or paste full link to the issue) 14 | - PR should contain the smallest possible change 15 | - PR should contain a very specific change 16 | - PR should contain only a single commit (squash your commits locally if required) 17 | - Avoid multiple changes across multiple files (raise an issue so we can discuss) 18 | - Avoid a long list of spelling or grammar corrections. These take too long to review and cherry pick. 19 | 20 | ## Submitting articles 21 | 22 | [Create an issue using the article template](https://github.com/practicalli/blog-content/issues/new?assignees=&labels=article&template=article.md&title=Suggested+article+title), 23 | providing as much detail as possible. 24 | 25 | ## Website design 26 | 27 | Suggestions about website design changes are most welcome, especially in terms of usability and accessibility. 28 | 29 | Please raise an issue so we can discuss changes first, especially changes related to aesthetics. 30 | 31 | ## Review process 32 | 33 | All pull requests are reviewed by @practicalli-johnny and feedback provided, usually the same day but please be patient. 34 | -------------------------------------------------------------------------------- /.github/workflows/changelog-check.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | # Check CHANGELOG.md file updated for every pull request 3 | 4 | name: Changelog Check 5 | on: 6 | pull_request: 7 | paths-ignore: 8 | - "README.md" 9 | types: [opened, synchronize, reopened, ready_for_review, labeled, unlabeled] 10 | 11 | jobs: 12 | changelog: 13 | name: Changelog Update Check 14 | runs-on: ubuntu-latest 15 | steps: 16 | - run: echo "🚀 Job automatically triggered by ${{ github.event_name }}" 17 | - run: echo "🐧 Job running on ${{ runner.os }} server" 18 | - run: echo "🐙 Using ${{ github.ref }} branch from ${{ github.repository }} repository" 19 | 20 | # Git Checkout 21 | - name: Checkout Code 22 | uses: actions/checkout@v4 23 | with: 24 | fetch-depth: 0 25 | sparse-checkout: | 26 | docs 27 | overrides 28 | .github 29 | CHANGELOG.md 30 | - run: echo "🐙 Sparse Checkout of ${{ github.repository }} repository to the CI runner." 31 | 32 | # Changelog Enforcer 33 | - name: Changelog Enforcer 34 | uses: dangoslen/changelog-enforcer@v3 35 | with: 36 | changeLogPath: "CHANGELOG.md" 37 | skipLabels: "skip-changelog-check" 38 | 39 | # Summary and status 40 | - run: echo "🎨 Changelog Enforcer quality checks completed" 41 | - run: echo "🍏 Job status is ${{ job.status }}." 42 | -------------------------------------------------------------------------------- /docs/notebooks/index.md: -------------------------------------------------------------------------------- 1 | # Clojure Data Science tooling: Notebooks 2 | 3 | Notebooks are interactive documents that include executable code and its results. They are a form of literate programming. 4 | 5 | For over 30 years data scientists have used notebooks for experiments, exploration of data and communicating insights to other data scientist and businesses. 6 | 7 | Software engineers can also use notebooks for development, providing fast feedback and a consistent way to visualise data. 8 | 9 | Open source notebooks include Jupyter (formerly known as iPython) and Apache Zeppelin. 10 | 11 | Hosted offerings include Deepnote, Noteable, Databricks Collaborative Notebooks, Google Colab, Jovian.ml, among others. 12 | 13 | Notebooks are programming language specific or support code for multiple languages like Polynote. 14 | 15 | ## Clojure Notebooks 16 | * NextJournal - hosted platform, written in Clojure, supports multiple programming languages 17 | * Notespace - 18 | * Clerk - Clojure based notebook focused on local development (not yet released) 19 | * Pink Gorilla 20 | 21 | 22 | > #### Hint::Wolfram Mathematica - first computational notebook 23 | > On June 23 1988 Stephan Wolfram first released [Wolfram Mathematica](https://en.wikipedia.org/wiki/Wolfram_Mathematica) that allow machine learning, statistics, symbolic computation, manipulating matrices, plotting functions and various types of data, implementation of algorithms, creation of user interfaces, and interfacing with programs written in other programming languages. 24 | -------------------------------------------------------------------------------- /docs/install/index.md: -------------------------------------------------------------------------------- 1 | # Install 2 | 3 | Clojure CLI is used to run Clojure code and is extended by a wide range of community tools. 4 | 5 | A [Clojure aware editor](https://practical.li/clojure/clojure-editors/) is highly recommended. Practicalli uses the following 6 | 7 | - [Spacemacs](https://practical.li/spacemacs/) - an extensive and easy to use Emacs configuration 8 | - [Neovim and Conjure](https://practical.li/neovim/) - a lightweight and highly extensible modal editing experience 9 | 10 | Practialli Clojure provides [an overview of Clojure editors available](https://practical.li/clojure/clojure-editors/) 11 | 12 | 13 | ## Install Clojure CLI 14 | 15 | ![Clojure logo](https://raw.githubusercontent.com/practicalli/graphic-design/live/logos/clojure-logo.svg){align=right loading=lazy style="height:64px;width:64px"} 16 | 17 | [Practicalli Clojure Install Guide](https://practical.li/clojure/install/){target=_blank .md-button} 18 | 19 | Clojure CLI provide a simple and configurable way to: 20 | 21 | * Run Clojure programs and tools 22 | * Run an interactive REPL (Read-Eval-Print Loop) and evaluate Clojure expressions, usually with a [Clojure aware editor](https://practical.li/clojure/clojure-editors/) 23 | * Managing dependencies (via tools.deps) from Maven and Git repositories 24 | 25 | Using community tools on top of Clojure CLI tools provides tasks to create, develop, build and deploy Clojure applications and services 26 | 27 | Follow the [Practicalli Clojure Install Guide](https://practical.li/clojure/install/) to install Clojure and supporting tools. 28 | -------------------------------------------------------------------------------- /.github/workflows/scheduled-version-check.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | # ------------------------------------------ 3 | # Scheduled check of versions 4 | # - use as non-urgent report on versions 5 | # - Uses POSIX Cron syntax 6 | # - Minute [0,59] 7 | # - Hour [0,23] 8 | # - Day of the month [1,31] 9 | # - Month of the year [1,12] 10 | # - Day of the week ([0,6] with 0=Sunday) 11 | # 12 | # Using liquidz/anta to check: 13 | # - GitHub workflows 14 | # - deps.edn 15 | # ------------------------------------------ 16 | 17 | name: "Scheduled Version Check" 18 | on: 19 | schedule: 20 | # - cron: "0 4 * * *" # at 04:04:04 ever day 21 | # - cron: "0 4 * * 5" # at 04:04:04 ever Friday 22 | - cron: "0 4 1 * *" # at 04:04:04 on first day of month 23 | workflow_dispatch: # Run manually via GitHub Actions Workflow page 24 | 25 | jobs: 26 | scheduled-version-check: 27 | name: "Scheduled Version Check" 28 | runs-on: ubuntu-latest 29 | steps: 30 | - run: echo "🚀 Job automatically triggered by ${{ github.event_name }}" 31 | - run: echo "🐧 Job running on ${{ runner.os }} server" 32 | - run: echo "🐙 Using ${{ github.ref }} branch from ${{ github.repository }} repository" 33 | 34 | - name: Checkout Code 35 | uses: actions/checkout@v4 36 | with: 37 | fetch-depth: 0 38 | sparse-checkout: | 39 | .github 40 | - run: echo "🐙 ${{ github.repository }} repository sparse-checkout to the CI runner." 41 | - name: "Antq Check versions" 42 | uses: liquidz/antq-action@main 43 | with: 44 | excludes: "" 45 | skips: "boot clojure-cli pom shadow-cljs leiningen" 46 | 47 | # Summary 48 | - run: echo "🎨 library versions checked with liquidz/antq" 49 | - run: echo "🍏 Job status is ${{ job.status }}." 50 | -------------------------------------------------------------------------------- /.github/workflows/publish-book.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | name: Publish Book 3 | on: 4 | # Manually trigger workflow 5 | workflow_dispatch: 6 | 7 | # Run work flow conditional on linter workflow success 8 | workflow_run: 9 | workflows: 10 | - "MegaLinter" 11 | paths: 12 | - "docs/**" 13 | - "includes/**" 14 | - "overrides/**" 15 | - "mkdocs.yaml" 16 | branches: 17 | - main 18 | types: 19 | - completed 20 | 21 | permissions: 22 | contents: write 23 | 24 | jobs: 25 | publish-book: 26 | name: MkDocs Publish 27 | runs-on: ubuntu-latest 28 | steps: 29 | - run: echo "🚀 Job automatically triggered by ${{ github.event_name }}" 30 | - run: echo "🐧 Job running on ${{ runner.os }} server" 31 | - run: echo "🐙 Using ${{ github.ref }} branch from ${{ github.repository }} repository" 32 | 33 | - name: Checkout Code 34 | uses: actions/checkout@v4 35 | with: 36 | fetch-depth: 0 37 | sparse-checkout: | 38 | docs 39 | includes 40 | overrides 41 | - run: echo "🐙 ${{ github.repository }} repository sparse-checkout to the CI runner." 42 | 43 | - name: Setup Python 44 | uses: actions/setup-python@v5 45 | with: 46 | python-version: 3.x 47 | 48 | - name: Cache 49 | uses: actions/cache@v4 50 | with: 51 | key: ${{ github.ref }} 52 | path: .cache 53 | 54 | - run: pip install mkdocs-material mkdocs-callouts mkdocs-glightbox mkdocs-git-revision-date-localized-plugin mkdocs-redirects pillow cairosvg 55 | - run: mkdocs gh-deploy --force 56 | - run: echo "🐙 ." 57 | 58 | # Summary and status 59 | - run: echo "🎨 MkDocs Publish Book workflow completed" 60 | - run: echo "🍏 Job status is ${{ job.status }}." 61 | -------------------------------------------------------------------------------- /docs/notebooks/notespace/index.md: -------------------------------------------------------------------------------- 1 | # Notespace - data science journal 2 | [Notespace](https://github.com/scicloj/notespace) generates a data science journal from a Clojure namespace, to give a simple to use literate programming experience. 3 | 4 | The [notespace introduction](https://scicloj.github.io/notespace/doc/notespace/v3-experiment1-test/index.html) shows how to create a Clojure namespace into a notespace. Clojure values, generative functions, markdown and graphics are defined using `notespace.kinds` as meta-data on Clojure expressions in the namespace. 5 | 6 | Run notespace listening to all changes, or just show a particular view (useful for large notespace) or even just update a line. 7 | 8 | 9 | ## Example project with notespace 10 | Use the [example Notespace project](https://github.com/practicalli/scicloj-notespace-simple-demo) which contains simple examples and also launches an empty notespace webpage and Portal data inspector window on REPL startup. 11 | 12 | Start a REPL for the project with Leiningen or Clojure CLI tools (version 1.10.1.697 or greater and aliases from [practicalli/clojure-deps-edn](https://github.com/practicalli/clojure-deps-edn)) 13 | 14 | ``` 15 | lein repl 16 | 17 | clojure -M:env/dev:inspect/portal 18 | ``` 19 | 20 | Wait a few seconds for the Portal and Notespace pages to load into your browser. 21 | 22 | Evaluate the `(notespace/listen)` expression and the current namespace is loaded into the Notespace page. If the Notespace page stops updating, evaluate this expression again. 23 | 24 | Wrap values and function calls with the `tap>` function to see the values in the Portal window. 25 | 26 | 27 | > #### Hint::Emacs support - Clojure CLI tools 28 | > the [example Notespace project](https://github.com/practicalli/scicloj-notespace-simple-demo) contains a `.dir-locals.el` file for Clojure CLI tools, using the `:env/dev` and `:inspect/portal-cli` aliases from [practicalli/clojure-deps-edn](https://github.com/practicalli/clojure-deps-edn) user level configuration. 29 | > 30 | > When running `cider-jack-in` the aliases are automatically added to jack-in startup command. 31 | -------------------------------------------------------------------------------- /.github/workflows/scheduled-stale-check.yaml: -------------------------------------------------------------------------------- 1 | # ---------------------------------------- 2 | # Scheduled stale issue & pull request check 3 | # 4 | # Adds 'stale' label after a set piece of time, 5 | # then closes stale issues & pull requests a short period after 6 | # 7 | # Using "Close Stale Issues" action 8 | # https://github.com/marketplace/actions/close-stale-issues 9 | # ---------------------------------------- 10 | 11 | name: 'Scheduled stale check' 12 | on: 13 | workflow_dispatch: 14 | schedule: 15 | - cron: "0 1 1 * *" # at 01:00 on first day of month 16 | 17 | jobs: 18 | stale: 19 | runs-on: ubuntu-latest 20 | steps: 21 | - run: echo "🚀 Job automatically triggered by ${{ github.event_name }}" 22 | - run: echo "🐧 Job running on ${{ runner.os }} server" 23 | - run: echo "🐙 Using ${{ github.ref }} branch from ${{ github.repository }} repository" 24 | 25 | - uses: actions/stale@v9 26 | with: 27 | stale-issue-message: 'After 30 days with no activity, the issue was automatically marked stale. Remove stale label or add a comment to prevent the issue being closed in 5 days.' 28 | stale-pr-message: 'After 45 days with no activity, the Pull Request was automatically marked stale. Remove stale label or comment to prevent the PR being closed in 10 days.' 29 | close-issue-message: 'This issue was closed because it has been stalled for 5 days with no activity.' 30 | close-pr-message: 'This PR was closed because it has been stalled for 10 days with no activity.' 31 | days-before-issue-stale: 30 32 | days-before-pr-stale: 45 33 | days-before-issue-close: 5 34 | days-before-pr-close: 10 35 | start-date: '2025-04-05T00:00:00Z' # only affect issues/PRs from date created (ISO 8601 or RFC 2822 format) 36 | any-of-labels: 'future,keep' # labels to keep 37 | exempt-issue-assignees: 'practicalli-johnny' 38 | exempt-pr-assignees: 'practicalli-johnny' 39 | 40 | # Summary 41 | - run: echo "🎨 Issues & Pull Request checked with actions/stale" 42 | - run: echo "🍏 Job status is ${{ job.status }}." 43 | -------------------------------------------------------------------------------- /docs/mathematics/index.md: -------------------------------------------------------------------------------- 1 | # Math Essentials 2 | 3 | Data Science is an application of several fields of mathematics: 4 | 5 | - Statistics & Probability (process & interpret data) 6 | - Linear Algebra 7 | - Mathematical Optimisation (solving puzzles by finding optimal solution) 8 | - Calculus 9 | 10 | Mathematics uses these fields via experimental design (e.g. A/B testing), data processing and modelling. 11 | 12 | !!! QUOTE 13 | Without mathematics there is no data science... only data - John Stevenson (and probably many others before) 14 | 15 |

16 | 17 |

18 | 19 | 20 | ## Video Tutorials 21 | 22 | [:fontawesome-brands-youtube: Animated Mathematic Theory Explanations](https://www.youtube.com/@3blue1brown){target=_blank .md-button} 23 | 24 | ??? WARNING "Videos and Channels currently being reviewed by Practicalli" 25 | 26 | - [Data Science introduction](https://www.youtube.com/watch?v=ua-CiDNNj30) - Free Code Camp 27 | - [:fontawesome-brands-youtube: Damsel in Data](https://www.youtube.com/@damselindata) 28 | - [:fontawesome-brands-youtube: How to learn math for data science (the minimize effort maximize outcome way)](https://www.youtube.com/watch?v=5wMl5FM2swo) 29 | - [:fontawesome-brands-youtube: Maths For Data Science](https://www.youtube.com/playlist?list=PLwYvzp9lBJVmv6yLK1hDH_Hqosl-lzGOl) - Geek's Lesson 30 | - [:fontawesome-brands-youtube: Thu Vu data analytics](https://www.youtube.com/@Thuvu5) 31 | - [:fontawesome-brands-youtube: Mathematics, statistics for data science and machine learning](https://www.youtube.com/playlist?list=PLeo1K3hjS3uuKaU2nBDwr6zrSOTzNCs0l) 32 | - [Sundas Khalid Data Science, Tech and Career Tips](https://www.youtube.com/@SundasKhalid) 33 | 34 | 35 | - [All Machine Learning Models Explained in 5 Minutes](https://www.youtube.com/watch?v=yN7ypxC7838) 36 | 37 | 38 | 39 | ## Resources 40 | 41 | - [Math for Clojurists](https://alanmarazzi.gitlab.io/blog/posts/2020-3-23-math-for-clojurists/) 42 | -------------------------------------------------------------------------------- /.github/workflows/megalinter.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | # MegaLinter GitHub Action configuration file 3 | # More info at https://megalinter.io 4 | # All variables described in https://megalinter.io/latest/configuration/ 5 | 6 | name: MegaLinter 7 | on: 8 | workflow_dispatch: 9 | pull_request: 10 | branches: [main] 11 | push: 12 | branches: [main] 13 | 14 | # Run Linters in parallel 15 | # Cancel running job if new job is triggered 16 | concurrency: 17 | group: "${{ github.ref }}-${{ github.workflow }}" 18 | cancel-in-progress: true 19 | 20 | jobs: 21 | megalinter: 22 | name: MegaLinter 23 | runs-on: ubuntu-latest 24 | steps: 25 | - run: echo "🚀 Job automatically triggered by ${{ github.event_name }}" 26 | - run: echo "🐧 Job running on ${{ runner.os }} server" 27 | - run: echo "🐙 Using ${{ github.ref }} branch from ${{ github.repository }} repository" 28 | 29 | # Git Checkout 30 | - name: Checkout Code 31 | uses: actions/checkout@v4 32 | with: 33 | fetch-depth: 0 34 | sparse-checkout: | 35 | docs 36 | overrides 37 | .github 38 | - run: echo "🐙 Sparse Checkout of ${{ github.repository }} repository to the CI runner." 39 | 40 | # MegaLinter Configuration 41 | - name: MegaLinter Run 42 | id: ml 43 | ## latest release of major version 44 | uses: oxsecurity/megalinter/flavors/documentation@v8 45 | env: 46 | # ADD CUSTOM ENV VARIABLES OR DEFINE IN MEGALINTER_CONFIG file 47 | MEGALINTER_CONFIG: .github/config/megalinter.yaml 48 | 49 | GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}" # report individual linter status 50 | # Validate all source when push on main, else just the git diff with live. 51 | VALIDATE_ALL_CODEBASE: >- 52 | ${{ github.event_name == 'push' && github.ref == 'refs/heads/main'}} 53 | 54 | # Upload MegaLinter artifacts 55 | - name: Archive production artifacts 56 | if: ${{ success() }} || ${{ failure() }} 57 | uses: actions/upload-artifact@v4 58 | with: 59 | name: MegaLinter reports 60 | path: | 61 | megalinter-reports 62 | mega-linter.log 63 | 64 | # Summary and status 65 | - run: echo "🎨 MegaLinter quality checks completed" 66 | - run: echo "🍏 Job status is ${{ job.status }}." 67 | -------------------------------------------------------------------------------- /SUMMARY.md: -------------------------------------------------------------------------------- 1 | # Summary 2 | 3 | * [Overview](overview.md) 4 | * [Persona](persona/index.md) 5 | * [What is Clojure?](what-is-clojure.md) 6 | * [Recommended books](data-science-books.md) 7 | * [Contributing](contributing.md) 8 | 9 | ## Tooling 10 | 11 | * Literate programming 12 | 13 | * [Notebooks](notebooks/index.md) 14 | * [notespace](notebooks/notespace/index.md) 15 | * [Configure notespace](notebooks/notespace/configure-notespace-project.md) 16 | 17 | * [Clerk](notebooks/clerk/index.md) 18 | * [Next Journal](notebooks/next-journal/index.md) 19 | * [Pink Gorilla](notebooks/pink-gorilla/index.md) 20 | 21 | 22 | 23 | ## Data mining 24 | 25 | * [Data Mining](data-mining/index.md) 26 | * [Web scraping](data-mining/webscraping/index.md) 27 | * [Web Tags to Clojure](data-mining/webscraping/enlive.md) 28 | * [Table data](data-mining/webscraping/table-data.md) 29 | 30 | 31 | 32 | 33 | 34 | ## Data Wrangling 35 | 36 | * [tablecloth](tablecloth/index.md) 37 | 38 | ## Visualisation 39 | 40 | * [Story telling](visualization/story-telling/index.md) 41 | * [Oz](visualization/oz/index.md) 42 | * [Create Project](visualization/oz/create-project.md) 43 | * [Oz](visualization/oz/clojure-spec.md) 44 | * [Generate Static site](visualization/oz/clojure-spec.md) 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | ## Mathematics 54 | 55 | * [Linear Algebra](mathematics/linear-algebra/index.md) 56 | 57 | 58 | 59 | 60 | 61 | 62 | ### Integration 63 | 64 | * [Python](integration/python/index.md) 65 | * [libpython-clj](https://github.com/clj-python/libpython-clj) 66 | 67 | 68 | ## Collaborative Programming 69 | * [Using a shared server](collaborative-coding/shared-server.md) 70 | 71 | 72 | ## Reference 73 | * [Reference](reference/index.md) 74 | * [Math for Clojure developers](reference/math/math-for-clojure-developers.md) 75 | 76 | 77 | * [Data Files](reference/data-files.md) 78 | 79 | 80 | 81 | -------------------------------------------------------------------------------- /docs/data-mining/webscraping/index.md: -------------------------------------------------------------------------------- 1 | # Web Scraping unstructured data 2 | 3 | 4 | ## Complexity of JavaScript websites 5 | 6 | * Sparkledriver 7 | 8 | 9 | 10 | ## Tools 11 | * [Enlive](https://github.com/cgrand/enlive) 12 | * [Scoopi](https://github.com/maithilish/scoopi-scraper) - a tool to extract and transform data from web pages 13 | * [demeter](https://github.com/gtebbutt/demeter) - fast, concurrent web scraper with headless JavaScript execution 14 | * [web-scraper](https://github.com/ka2u/web-scraper) - library with fairly good JavaScript support 15 | * [Web Page Summarizer](https://github.com/DFW-Clojure/web-page-summarizer) - gui for getting web pages and summarizing them. Demonstrates enlive and compojure 16 | * [Scraper](https://github.com/cmiles74/scraper) - JavaFX web engine and WebKit 17 | * [Abrade](https://github.com/weavejester/abrade) - scraping web sites, even ones that heavily rely on Javascript. The Java HtmlUnit library is used under the hood 18 | * [Etaoin](https://github.com/igrishaev/etaoin) - Clojure implementation of webdriver protocol 19 | 20 | 21 | ## Example projects 22 | * [parkrun-app](https://github.com/ldnclj/parkrun-app) - enlive 23 | * [clj-scraper](https://github.com/0rca/clj-scraper) - enlive, http-kit, core.async 24 | * [ldnpyvideo](https://github.com/necaris/ldnpyvideo) - Scraper (from pyvideo.org) and web site for London PyCon video meetup 25 | * [nba-scraper](https://github.com/yoyostevo/nba-scraper) - scraping NBA boxscore data from ESPN 26 | * [Clojure web scraping with Enlive](https://ericsomdahl.github.io/posts/2015-03-07-gevents1.html) 27 | 28 | 29 | ## References 30 | * [Practicalli: Web Scraping with Clojure - Hacker news](http://practical.li/blog/posts/web-scraping-with-clojure-hacking-hacker-news/) 31 | * [Clojure Data Analysis Cookbook: Scraping data from tables in web pages](https://subscription.packtpub.com/book/application_development/9781784390297/1/ch01lvl1sec15/scraping-data-from-tables-in-web-pages) 32 | * [ClojureVerse: First time webscraper, could you give any pointers?](https://clojureverse.org/t/first-time-webscraper-could-you-give-any-pointers/7663/6) 33 | * [Web Scraping with Clojure](http://masnun.com/2016/03/20/web-scraping-with-clojure.html) - http-kit and Enlive 34 | * [How to Scrape Modern Websites Without Headless Browsers - Python](https://betterprogramming.pub/how-to-scrape-modern-websites-without-headless-browsers-d871bbd1119e) 35 | 36 | 37 | > #### Hint::Be Respectful of data sources 38 | > Avoid high number of requests to websites with unstructured data, they are unlikely to have much capacity to serve requests. Consider downloading the content locally to minimise the requests to the website. 39 | -------------------------------------------------------------------------------- /docs/introduction/persona.md: -------------------------------------------------------------------------------- 1 | # Persona 2 | 3 | Persona help identify commonality of experience and approaches for people in the realm of data science and software engineering. 4 | 5 | 6 | ## Jane - mid-level Clojure developer 7 | 8 | Jane is a Clojure developer with a couple of years commercial experience 9 | Jane has no data science experience 10 | She has a project to create a dashboard to visualise covid19 data for her company 11 | And has to find useful tools and data sources to build the dashboard 12 | 13 | An expanded version of the Jane Clojure developer persona: 14 | 15 | Jane is a Clojure developer with a couple of years commercial experience and a year prior to that as a hobby 16 | Jane has no data science experience but has seen TED talks and tutorials on visualising data 17 | Jane has a project to create a dashboard to visualise covid19 data for her company 18 | And has to find useful tools and data sources to build the dashboard 19 | Jane visits the Government website for the country she resides in and finds various data sets in JSON and CSV formats 20 | Jane choose to use Clojure CLI tools for the Clojure project to make use of the community tooling that helps visualise data as she is transforming it (Portal, Reveal) 21 | Jane uses Spacemacs (Emacs / Cider) as its is a tool she is familiar with and Cider has simple to use debug and data inspector tools. Jane started with LightTable but that project is no longer active. She has also tried VS Code and Calva, but Cider is more stable and mature, has more features and better structural editing support. 22 | The data wrangling libraries Jane is considering are clojure.data.json and clojure.data.csv as they are already included in Clojure 23 | Jane has heard of other libraries for manipulating data, but doesnt know if they are relevant or easy to use 24 | Jane will try using Oz for visualisations as she likes the idea of Vega & Vega lite as a data language, because it reminds her of the approach taken with Clojure itself. 25 | 26 | 27 | ## Jada - experienced data scientist 28 | Jada 29 | - statisticians 30 | - machine learning 31 | - visualisation 32 | - computer vision 33 | - ML Ops 34 | - Probabilistic ??? 35 | 36 | 37 | 38 | Jamie is a analyst using data science for the last year as a company doing X 39 | and has completed X data science course 40 | and works with several more experienced data scientist who are using some python and R to build data models. 41 | 42 | 43 | Jenny is new to coding and has an interest in data science and machine learning and has no real experience in any of these. 44 | doesnt know what she doesnt know 45 | doesnt know where to start looking 46 | 47 | 48 | Jennifer has several years experience of R and Python (and occasionally wrangled a few C++ libraries) 49 | Jennifer is curious about Clojure because ??? 50 | - listened to a talk on Simple made Easy and wishes that R and Python would make some of the work simpler 51 | - 52 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | # ------------------------------------------ 2 | # Practicalli: Makefile 3 | # 4 | # Consistent set of targets to support local book development 5 | # ------------------------------------------ 6 | 7 | # .PHONY: ensures target used rather than matching file name 8 | # https://makefiletutorial.com/#phony 9 | .PHONY: all clean docs lint pre-commit-check test 10 | 11 | # ------- Makefile Variables --------- # 12 | # run help if no target specified 13 | .DEFAULT_GOAL := help 14 | 15 | # Column the target description is printed from 16 | HELP-DESCRIPTION-SPACING := 24 17 | 18 | # Tool Commands 19 | MEGALINTER_RUNNER := npx mega-linter-runner --flavor documentation --env "'MEGALINTER_CONFIG=.github/config/megalinter.yaml'" --env "'VALIDATE_ALL_CODEBASE=true'" --remove-container 20 | MKDOCS_SERVER := mkdocs serve --dev-addr localhost:7777 21 | 22 | # Makefile file and directory name wildcard 23 | EDN-FILES := $(wildcard *.edn) 24 | # ------------------------------------ # 25 | 26 | # ------ Quality Checks ------------ # 27 | pre-commit-check: lint 28 | 29 | lint: ## Run MegaLinter with custom configuration (node.js required) 30 | $(info --------- MegaLinter Runner ---------) 31 | $(MEGALINTER_RUNNER) 32 | 33 | lint-fix: ## Run MegaLinter with custom configuration (node.js required) 34 | $(info --------- MegaLinter Runner ---------) 35 | $(MEGALINTER_RUNNER) --fix 36 | 37 | lint-clean: ## Clean MegaLinter report information 38 | $(info --------- MegaLinter Clean Reports ---------) 39 | - rm -rf ./megalinter-reports 40 | 41 | megalinter-upgrade: ## Upgrade MegaLinter config to latest version 42 | $(info --------- MegaLinter Upgrade Config ---------) 43 | npx mega-linter-runner@latest --upgrade 44 | # ------------------------------------ # 45 | 46 | # --- Documentation Generation ------ # 47 | python-venv: ## Enable Python Virtual Environment for MkDocs 48 | $(info --------- Mkdocs Local Server ---------) 49 | source ~/.local/venv/bin/activate 50 | 51 | docs: ## Build and run mkdocs in local server (python venv) 52 | $(info --------- Mkdocs Local Server ---------) 53 | source ~/.local/venv/bin/activate && $(MKDOCS_SERVER) 54 | 55 | docs-changed: ## Build only changed files and run mkdocs in local server (python venv) 56 | $(info --------- Mkdocs Local Server ---------) 57 | source ~/.local/venv/bin/activate && $(MKDOCS_SERVER) --dirtyreload 58 | 59 | docs-build: ## Build mkdocs (python venv) 60 | $(info --------- Mkdocs Local Server ---------) 61 | source ~/.local/venv/bin/activate && mkdocs build 62 | # ------------------------------------ # 63 | 64 | # ------------ Help ------------------ # 65 | # Source: https://nedbatchelder.com/blog/201804/makefile_help_target.html 66 | 67 | help: ## Describe available tasks in Makefile 68 | @grep '^[a-zA-Z]' $(MAKEFILE_LIST) | \ 69 | sort | \ 70 | awk -F ':.*?## ' 'NF==2 {printf "\033[36m %-$(HELP-DESCRIPTION-SPACING)s\033[0m %s\n", $$1, $$2}' 71 | # ------------------------------------ # 72 | -------------------------------------------------------------------------------- /overrides/partials/header.html: -------------------------------------------------------------------------------- 1 | 2 | {% set class = "md-header" %} {% if "navigation.tabs.sticky" in features %} {% 3 | set class = class ~ " md-header--shadow md-header--lifted" %} {% elif 4 | "navigation.tabs" not in features %} {% set class = class ~ " md-header--shadow" 5 | %} {% endif %} 6 | 7 | 8 |
9 | 74 | 75 | 76 | {% if "navigation.tabs.sticky" in features %} {% if "navigation.tabs" in 77 | features %} {% include "partials/tabs.html" %} {% endif %} {% endif %} 78 |
79 | -------------------------------------------------------------------------------- /.github/config/megalinter.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | # Configuration file for MegaLinter 3 | # 4 | # General configuration: 5 | # https://megalinter.io/latest/configuration/ 6 | # 7 | # Specific Linters: 8 | # https://megalinter.io/latest/supported-linters/ 9 | 10 | # ------------------------ 11 | # Validate all files if true 12 | # or new / edited files if false 13 | VALIDATE_ALL_CODEBASE: false 14 | 15 | # ------------------------ 16 | # Linters 17 | 18 | # Run linters in parallel 19 | PARALLEL: true 20 | 21 | # ENABLE specific linters, all other linters automatically disabled 22 | ENABLE: 23 | # - CLOJURE 24 | - CREDENTIALS 25 | - DOCKERFILE 26 | - MAKEFILE 27 | - MARKDOWN 28 | - GIT 29 | - SPELL 30 | - YAML 31 | - REPOSITORY 32 | 33 | # Linter specific configuration 34 | 35 | # CLOJURE_CLJ_KONDO_CONFIG_FILE: ".github/config/clj-kondo-ci-config.edn" 36 | # CLOJURE_CLJ_KONDO_ARGUMENTS: "--lint deps.edn" 37 | # CLOJURE_CLJ_KONDO_FILTER_REGEX_EXCLUDE: "dev|develop" 38 | # CLOJURE_CLJ_KONDO_FILTER_REGEX_EXCLUDE: "resources" 39 | 40 | # CREDENTIALS_SECRETLINT_DISABLE_ERRORS: true 41 | CREDENTIALS_SECRETLINT_CONFIG_FILE: ".github/config/secretlintrc.json" 42 | 43 | MARKDOWN_MARKDOWNLINT_CONFIG_FILE: ".github/config/markdown-lint.jsonc" 44 | MARKDOWN_MARKDOWNLINT_FILTER_REGEX_EXCLUDE: ".github/pull_request_template.md|CHANGELOG.md|README.md|GLOSSARY.md|java-17-flags.md|abbreviations.md" 45 | # MARKDOWN_MARKDOWNLINT_DISABLE_ERRORS: true 46 | MARKDOWN_MARKDOWN_LINK_CHECK_CONFIG_FILE: ".github/config/markdown-link-check.json" 47 | # MARKDOWN_MARKDOWN_LINK_CHECK_CLI_LINT_MODE: "project" 48 | # MARKDOWN_MARKDOWN_LINK_CHECK_DISABLE_ERRORS: false 49 | MARKDOWN_REMARK_LINT_DISABLE_ERRORS: true 50 | # MARKDOWN_MARKDOWN_TABLE_FORMATTER_DISABLE_ERRORS: false 51 | 52 | REPOSITORY_GITLEAKS_CONFIG_FILE: ".github/config/gitleaks.toml" 53 | REPOSITORY_TRUFFLEHOG_DISABLE_ERRORS: true # Errors only as warnings 54 | 55 | # SPELL_CSPELL_DISABLE_ERRORS: true 56 | SPELL_MISSPELL_DISABLE_ERRORS: true 57 | SPELL_LYCHEE_CONFIG_FILE: ".github/config/lychee.toml" 58 | SPELL_LYCHEE_DISABLE_ERRORS: true # Errors are only warnings 59 | 60 | # YAML_PRETTIER_FILTER_REGEX_EXCLUDE: (docs/) 61 | # YAML_YAMLLINT_FILTER_REGEX_EXCLUDE: (docs/) 62 | 63 | # Explicitly disable linters to ensure they are never run 64 | # DISABLE: 65 | # - COPYPASTE # checks for excessive copy-pastes 66 | # - SPELL # spell checking - often creates many false positives 67 | # - CSS # 68 | 69 | # Disable linter features 70 | DISABLE_LINTERS: 71 | - YAML_PRETTIER # draconian format rules 72 | - SPELL_CSPELL # many clojure references causing false positives 73 | - YAML_YAMLLINT # vague error mesages, investigation required 74 | - REPOSITORY_GIT_DIFF # warnings about LF to CRLF 75 | - REPOSITORY_SECRETLINT # reporting errors in its own config file 76 | # - REPOSITORY_DEVSKIM # unnecessary URL TLS checks 77 | - REPOSITORY_CHECKOV # fails on root user in Dockerfile 78 | - REPOSITORY_SECRETLINT 79 | 80 | # Ignore all errors and return without error status 81 | # DISABLE_ERRORS: true 82 | 83 | # ------------------------ 84 | 85 | # ------------------------ 86 | # Reporting 87 | 88 | # Activate sources reporter 89 | UPDATED_SOURCES_REPORTER: false 90 | 91 | # Show Linter timings in summary table at end of run 92 | SHOW_ELAPSED_TIME: true 93 | 94 | # Upload reports to file.io 95 | FILEIO_REPORTER: false 96 | # ------------------------ 97 | 98 | # ------------------------ 99 | # Over-ride errors 100 | 101 | # detect errors but do not block CI passing 102 | # DISABLE_ERRORS: true 103 | # ------------------------ 104 | -------------------------------------------------------------------------------- /docs/assets/stylesheets/extra.css: -------------------------------------------------------------------------------- 1 | [data-md-color-scheme="default"] { 2 | --md-default-bg-color: hsla(208, 100%, 96%, 0.94); 3 | --md-code-bg-color: hsla(208, 80%, 88%, 0.64); 4 | --md-code-hl-color: hsla(208, 88%, 80%, 0.92); 5 | --md-admonition-bg-color: hsla(208, 80%, 92%, 0.92); 6 | --md-typeset-kbd-color: hsla(208, 100%, 98%, 0.98); 7 | } 8 | 9 | /* Custom Admonitions */ 10 | 11 | 12 | :root { 13 | /* Clojure Idiom*/ 14 | --md-admonition-icon--clojure-idiom: url(https://raw.githubusercontent.com/practicalli/graphic-design/c40cc063cc5bb07525b524d8a3d638e2f42bc38a/logos/clojure-logo-bullet.svg); 15 | 16 | /* Round corners */ 17 | --base-border-radius: 0.5rem; 18 | } 19 | 20 | /*Admonitions colors*/ 21 | .md-typeset .admonition.clojure-idiom, 22 | .md-typeset details.clojure-idiom { 23 | border-color: rgb(43, 155, 70); 24 | } 25 | .md-typeset .clojure-idiom > .admonition-title, 26 | .md-typeset .clojure-idiom > summary { 27 | background-color: rgba(43, 155, 70, 0.1); 28 | } 29 | .md-typeset .clojure-idiom > .admonition-title::before, 30 | .md-typeset .clojure-idiom > summary::before { 31 | background-color: rgb(250, 250, 250); 32 | background-image: var(--md-admonition-icon--clojure-idiom); 33 | -webkit-mask-image: var(--md-admonition-icon--clojure-idiom); 34 | mask-image: var(--md-admonition-icon--clojure-idiom); 35 | } 36 | 37 | 38 | /* Change font family of filename present on top of code block. */ 39 | .highlight span.filename { 40 | border-bottom: none; 41 | border-radius: var(--base-border-radius); 42 | display: inline; 43 | font-family: var(--md-code-font-family); 44 | border-bottom-left-radius: 0; 45 | border-bottom-right-radius: 0; 46 | margin-bottom: 5px; 47 | text-align: center; 48 | } 49 | .highlight span.filename + pre > code { 50 | border-radius: var(--base-border-radius); 51 | border-top-left-radius: 0; 52 | } 53 | .md-typeset pre > code { 54 | border-radius: var(--base-border-radius); 55 | } 56 | 57 | /* Grid Cards */ 58 | .md-typeset .grid.cards > ul > li { 59 | border-radius: var(--base-border-radius); 60 | } 61 | .md-typeset .grid.cards > ul > li:hover { 62 | box-shadow: 0 0 0.2rem #ffffff40; 63 | } 64 | 65 | /* Markdown Button */ 66 | .md-typeset .md-button { 67 | border-radius: var(--base-border-radius); 68 | } 69 | 70 | /* Critic, Mark */ 71 | ins.critic, 72 | del.critic { 73 | text-decoration: none; 74 | } 75 | 76 | .md-typeset .critic, 77 | .md-typeset mark { 78 | border-radius: 0.2rem; 79 | padding: 0 0.2rem; 80 | } 81 | 82 | .md-typeset mark { 83 | box-shadow: 0 0 0 0.1rem var(--md-typeset-mark-color); 84 | } 85 | 86 | .md-typeset ins.critic { 87 | box-shadow: 0 0 0 0.1rem var(--md-typeset-ins-color); 88 | } 89 | 90 | .md-typeset del.critic { 91 | box-shadow: 0 0 0 0.1rem var(--md-typeset-del-color); 92 | } 93 | 94 | /* Forms */ 95 | .md-search__form { 96 | border-radius: var(--base-border-radius); 97 | } 98 | 99 | [data-md-toggle="search"]:checked ~ .md-header .md-search__form { 100 | border-top-right-radius: var(--base-border-radius); 101 | border-top-left-radius: var(--base-border-radius); 102 | } 103 | 104 | [dir="ltr"] .md-search__output { 105 | border-bottom-right-radius: var(--base-border-radius); 106 | border-bottom-left-radius: var(--base-border-radius); 107 | } 108 | 109 | /* Blog - index.md */ 110 | .md-post--excerpt { 111 | background-color: var(--md-accent-fg-color--transparent); 112 | box-shadow: 0 0 0 1rem var(--md-accent-fg-color--transparent); 113 | border-radius: var(--base-border-radius); 114 | } 115 | 116 | /* Table */ 117 | .md-typeset table:not([class]) { 118 | border-radius: var(--base-border-radius); 119 | } 120 | -------------------------------------------------------------------------------- /docs/notebooks/notespace/configure-notespace-project.md: -------------------------------------------------------------------------------- 1 | ## Run Notespace on REPL startup 2 | [Example Notespace project](https://github.com/practicalli/scicloj-notespace-simple-demo) is a project with simple examples and also launches an empty notespace webpage on REPL startup. 3 | 4 | {% tabs deps="Clojure CLI tools", lein="Leiningnen" %} 5 | 6 | {% content "deps" %} 7 | Use a deps.edn file that includes the `scicloj/notespace` library 8 | 9 | 10 | ## Creating a new project 11 | Use Clojure CLI tools ([:project/new](http://practicalli.github.io/clojure/clojure-tools/projects/create.html)) to create a project. 12 | 13 | ``` 14 | clojure -X:project/new :template app :name practicalli/notespace-demo 15 | ``` 16 | 17 | Edit the `deps.edn` file 18 | 19 | Add `scicloj/notespace` library as dependency 20 | 21 | Use [practicalli/clojure-deps-edn](http://practicalli.github.io/clojure/clojure-tools/install/community-tools.html) user configuration or add an `:env/dev` and `:inspect/portal-clj` aliases to the project `deps.edn` file 22 | 23 | ``` 24 | {:paths ["src" "resources"] 25 | :deps 26 | {org.clojure/clojure {:mvn/version "1.10.1"} 27 | scicloj/notespace {:mvn/version "3-alpha3-SNAPSHOT"}} 28 | :aliases 29 | {:env/dev {:extra-paths ["dev"]} 30 | :inspect/portal-cli {:extra-deps {djblue/portal {:mvn/version "0.6.4"}}}}} 31 | ``` 32 | 33 | 34 | {% content "lein" %} 35 | ## Creating a new project 36 | Use Leinigen to create a new project 37 | 38 | ``` 39 | lein new app practicalli/notespace-demo 40 | ``` 41 | 42 | Edit the project.clj file 43 | 44 | Add notespace as a dependency to the project configuration 45 | 46 | add a `:dev` profile with `:resource-paths` which adds the `dev` directory to class path. Leiningen uses the `:dev` profile unless another profile is specified. 47 | 48 | ``` 49 | (defproject practicalli/notespace-demo "0.1.0-SNAPSHOT" 50 | :description "Notespace journal - simple demo" 51 | :url "https://practicalli.github.io/data-science/notebooks/notespace/" 52 | :license {:name "Creative Commons Attribution Share-Alike 4.0 International" 53 | :url "https://creativecommons.org"} 54 | :dependencies [[org.clojure/clojure "1.10.1"] 55 | [scicloj/notespace "3-alpha3"]] 56 | :main ^:skip-aot practicalli.notespace-demo 57 | :target-path "target/%s" 58 | :profiles {:dev {:resource-paths ["dev"] 59 | :dependencies [[djblue/portal "0.6.4"]]} 60 | :uberjar {:aot :all}}) 61 | ``` 62 | 63 | {% endtabs %} 64 | 65 | 66 | ## Add namespace to start notespace with REPL 67 | Create a `dev/user.clj` source code file 68 | 69 | Add a `user` namespace definition that requires notespace and portal 70 | 71 | Add a call to start notespace 72 | 73 | Add a call to start Portal and connect it to the REPL as a tap> source 74 | 75 | Add notespace and portal helper functions 76 | 77 | ``` 78 | (ns user 79 | (:require [notespace.api :as notespace] 80 | [portal.api :as inspect])) 81 | 82 | ;; Start Notespace 83 | (notespace/init-with-browser) 84 | 85 | ;; Start Portal 86 | ;; Open a portal inspector window using default nord theme 87 | (inspect/open {:portal.colors/theme :portal.colors/solarized-light}) 88 | 89 | ;; Add portal as a tap> target 90 | (inspect/tap) 91 | 92 | 93 | (comment 94 | 95 | ;; Notespace helpers 96 | ;;;;;;;;;;;;;;;;;;;;;; 97 | 98 | ;; Initialise notespace, headless or open browser 99 | (notespace/init) 100 | (notespace/init-with-browser) 101 | 102 | 103 | ;; Portal helpers 104 | ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; 105 | 106 | ;; Open Portal window with nord theme (default) 107 | (inspect/open) 108 | 109 | ;; Open Portal window with solarized-dark theme 110 | (inspect/open {:portal.colors/theme :portal.colors/solarized-dark}) 111 | 112 | ;; Open Portal window with solarized-light theme 113 | (inspect/open {:portal.colors/theme :portal.colors/solarized-light}) 114 | 115 | ;; Clear all values in the portal inspector window 116 | (inspect/clear) 117 | 118 | ;; Close the portal window 119 | (inspect/close) 120 | 121 | ) ;; End of rich comment block 122 | 123 | ``` 124 | 125 | 126 | 127 | 128 | 129 | ## Run a REPL for the project 130 | 131 | {% tabs depsstart="Clojure CLI tools", leinstart="Leiningnen" %} 132 | 133 | {% content "depsstart" %} 134 | Open a terminal and run a REPL using the `:env/dev` and `:inspect/portal-cli` aliases 135 | 136 | ``` 137 | clojure -M:env/dev:inspect/portal-cli 138 | ``` 139 | 140 | 141 | {% content "leinstart" %} 142 | Open a terminal and start a REPL with Leinigen, using the `:dev` profile by default 143 | 144 | ``` 145 | lein repl 146 | 147 | ``` 148 | 149 | 150 | {% endtabs %} 151 | -------------------------------------------------------------------------------- /docs/assets/favicon.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | image/svg+xml 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | I 19 | P 20 | 9 21 | 22 | 23 | 24 | 25 | -------------------------------------------------------------------------------- /docs/assets/stylesheets/practicalli-light.css: -------------------------------------------------------------------------------- 1 | // ---------------------------------------------------------------------------- 2 | // Rules 3 | // ---------------------------------------------------------------------------- 4 | 5 | // Color variables 6 | :root { 7 | @extend %root; 8 | 9 | // Primary color shades 10 | --md-primary-fg-color: hsla(#{hex2hsl($clr-indigo-500)}, 1); 11 | --md-primary-fg-color--light: hsla(#{hex2hsl($clr-indigo-400)}, 1); 12 | --md-primary-fg-color--dark: hsla(#{hex2hsl($clr-indigo-700)}, 1); 13 | --md-primary-bg-color: hsla(0, 0%, 100%, 1); 14 | --md-primary-bg-color--light: hsla(0, 0%, 100%, 0.7); 15 | 16 | // Accent color shades 17 | --md-accent-fg-color: hsla(#{hex2hsl($clr-indigo-a200)}, 1); 18 | --md-accent-fg-color--transparent: hsla(#{hex2hsl($clr-indigo-a200)}, 0.1); 19 | --md-accent-bg-color: hsla(0, 0%, 100%, 1); 20 | --md-accent-bg-color--light: hsla(0, 0%, 100%, 0.7); 21 | } 22 | 23 | // ---------------------------------------------------------------------------- 24 | 25 | // Allow to explicitly use color schemes in nested content 26 | [data-md-color-scheme="practicalli"] { 27 | @extend %root; 28 | } 29 | 30 | // ---------------------------------------------------------------------------- 31 | // Placeholders 32 | // ---------------------------------------------------------------------------- 33 | 34 | // Default theme, i.e. light mode 35 | %root { 36 | 37 | // Default color shades 38 | --md-default-fg-color: hsla(0, 0%, 0%, 0.87); 39 | --md-default-fg-color--light: hsla(0, 0%, 0%, 0.54); 40 | --md-default-fg-color--lighter: hsla(0, 0%, 0%, 0.32); 41 | --md-default-fg-color--lightest: hsla(0, 0%, 0%, 0.07); 42 | --md-default-bg-color: hsla(53, 90%, 90%, 1); 43 | --md-default-bg-color--light: hsla(53, 90%, 90%, 0.7); 44 | --md-default-bg-color--lighter: hsla(53, 90%, 90%, 0.3); 45 | --md-default-bg-color--lightest: hsla(53, 90%, 90%, 0.12); 46 | 47 | // Code color shades 48 | --md-code-fg-color: hsla(200, 18%, 26%, 1); 49 | --md-code-bg-color: hsla(0, 0%, 96%, 1); 50 | 51 | // Code highlighting color shades 52 | --md-code-hl-color: hsla(#{hex2hsl($clr-yellow-a200)}, 0.5); 53 | --md-code-hl-number-color: hsla(0, 67%, 50%, 1); 54 | --md-code-hl-special-color: hsla(340, 83%, 47%, 1); 55 | --md-code-hl-function-color: hsla(291, 45%, 50%, 1); 56 | --md-code-hl-constant-color: hsla(250, 63%, 60%, 1); 57 | --md-code-hl-keyword-color: hsla(219, 54%, 51%, 1); 58 | --md-code-hl-string-color: hsla(150, 63%, 30%, 1); 59 | --md-code-hl-name-color: var(--md-code-fg-color); 60 | --md-code-hl-operator-color: var(--md-default-fg-color--light); 61 | --md-code-hl-punctuation-color: var(--md-default-fg-color--light); 62 | --md-code-hl-comment-color: var(--md-default-fg-color--light); 63 | --md-code-hl-generic-color: var(--md-default-fg-color--light); 64 | --md-code-hl-variable-color: var(--md-default-fg-color--light); 65 | 66 | // Typeset color shades 67 | --md-typeset-color: var(--md-default-fg-color); 68 | 69 | // Typeset `a` color shades 70 | --md-typeset-a-color: var(--md-primary-fg-color); 71 | 72 | // Typeset `mark` color shades 73 | --md-typeset-mark-color: hsla(#{hex2hsl($clr-yellow-a200)}, 0.5); 74 | 75 | // Typeset `del` and `ins` color shades 76 | --md-typeset-del-color: hsla(6, 90%, 60%, 0.15); 77 | --md-typeset-ins-color: hsla(150, 90%, 44%, 0.15); 78 | 79 | // Typeset `kbd` color shades 80 | --md-typeset-kbd-color: hsla(0, 0%, 98%, 1); 81 | --md-typeset-kbd-accent-color: hsla(0, 100%, 100%, 1); 82 | --md-typeset-kbd-border-color: hsla(0, 0%, 72%, 1); 83 | 84 | // Typeset `table` color shades 85 | --md-typeset-table-color: hsla(0, 0%, 0%, 0.12); 86 | 87 | // Admonition color shades 88 | --md-admonition-fg-color: var(--md-default-fg-color); 89 | --md-admonition-bg-color: var(--md-default-bg-color); 90 | 91 | // Footer color shades 92 | --md-footer-fg-color: hsla(0, 0%, 100%, 1); 93 | --md-footer-fg-color--light: hsla(0, 0%, 100%, 0.7); 94 | --md-footer-fg-color--lighter: hsla(0, 0%, 100%, 0.3); 95 | --md-footer-bg-color: hsla(0, 0%, 0%, 0.87); 96 | --md-footer-bg-color--dark: hsla(0, 0%, 0%, 0.32); 97 | 98 | // Shadow depth 1 99 | --md-shadow-z1: 100 | 0 #{px2rem(4px)} #{px2rem(10px)} hsla(0, 0%, 0%, 0.05), 101 | 0 0 #{px2rem(1px)} hsla(0, 0%, 0%, 0.1); 102 | 103 | // Shadow depth 2 104 | --md-shadow-z2: 105 | 0 #{px2rem(4px)} #{px2rem(10px)} hsla(0, 0%, 0%, 0.1), 106 | 0 0 #{px2rem(1px)} hsla(0, 0%, 0%, 0.25); 107 | 108 | // Shadow depth 3 109 | --md-shadow-z3: 110 | 0 #{px2rem(4px)} #{px2rem(10px)} hsla(0, 0%, 0%, 0.2), 111 | 0 0 #{px2rem(1px)} hsla(0, 0%, 0%, 0.35); 112 | } 113 | -------------------------------------------------------------------------------- /docs/introduction/contributing.md: -------------------------------------------------------------------------------- 1 | # Contributing to Practicalli 2 | 3 | Practicalli books are written in markdown and use MkDocs to generate the published website via a GitHub workflow. MkDocs can also run a local server using the `make docs` target from the `Makefile` 4 | 5 | By submitting content ideas and corrections you are agreeing they can be used in this book under the [Creative Commons Attribution ShareAlike 4.0 International license](https://creativecommons.org/licenses/by-sa/4.0/){target=_blank}. Attribution will be detailed via [GitHub contributors](https://github.com/practicalli/clojure-data-science/graphs/contributors){target=_blank}. 6 | 7 | All content and interaction with any persons or systems must be done so with respect and within the Practicalli Code of Conduct. 8 | 9 | ## Book status 10 | 11 | [![MegaLinter](https://github.com/practicalli/clojure-data-science/actions/workflows/megalinter.yaml/badge.svg)](https://github.com/practicalli/clojure-data-science/actions/workflows/megalinter.yaml)[![Publish Book](https://github.com/practicalli/clojure-data-science/actions/workflows/publish-book.yaml/badge.svg)](https://github.com/practicalli/clojure-data-science/actions/workflows/publish-book.yaml){target=_blank} 12 | [![pages-build-deployment](https://github.com/practicalli/clojure-data-science/actions/workflows/pages/pages-build-deployment/badge.svg)](https://github.com/practicalli/clojure-data-science/actions/workflows/pages/pages-build-deployment){target=_blank} 13 | 14 | [![Ideas & Issues](https://img.shields.io/github/issues/practicalli/clojure-data-science?label=content%20ideas%20and%20issues&logoColor=green&style=for-the-badge)](https://github.com/practicalli/clojure-practicalli-content/issues){target=_blank} 15 | [![Pull requests](https://img.shields.io/github/issues-pr/practicalli/clojure-data-science?style=for-the-badge)](https://github.com/practicalli/clojure-practicalli-content/pulls){target=_blank} 16 | 17 | ![GitHub commit activity](https://img.shields.io/github/commit-activity/m/practicalli/clojure-data-science?style=for-the-badge) 18 | ![GitHub contributors](https://img.shields.io/github/contributors/practicalli/clojure-data-science?style=for-the-badge&label=github%20contributors) 19 | 20 | ## Submit and issue or idea 21 | 22 | If something doesnt seem quite right or something is missing from the book, please [raise an issue via the GitHub repository](https://github.com/practicalli/clojure-data-science/issues){target=_blank} explaining in as much detail as you can. 23 | 24 | Raising an issue before creating a pull request will save you and the maintainer time. 25 | 26 | ## Considering a Pull request? 27 | 28 | !!! INFO "Pull Request Commits must be cryptographically signed" 29 | All commits contributed to Practicalli must be signed via a legitimate SSH or GPG key to avoid the risk of commit spoofing. 30 | 31 | [Configure commit signing with SSH key - Practicalli Engineering](https://practical.li/engineering-playbook/source-control/git-configuration/#commit-signing-with-ssh-key){target=_blank .md-button} 32 | 33 | All pull requests must include an entry in CHANGELOG.md or will not be merged. A changelog entry allows the community to follow the changes to the book. 34 | 35 | Each pull request will have a number of CI workflows run against the contribution, checking the format of the content and if a changelog entry has been provided. 36 | 37 | Please keep pull requests small and focused, as they are much quicker to review and easier to accept. Ideally PR's should be for a specific page or at most a section. 38 | 39 | A PR with a list of changes across different sections will be closed without merging as these take considerable time to review. 40 | 41 | Issues such as grammar improvements are typically a sign of a rushed section that requires a rewrite, so a pull request to fix a typeographic error will probably not be merged. Raise an issue, or post a thread in the [:globe_with_meridians: Clojurians Slack #practicall channel](https://clojurians.slack.com/messages/practicalli) 42 | 43 | ## Thank you to everyone that has contributed 44 | 45 | * [SciCloj community](https://scicloj.github.io/){target=_blank .md-button} and Daniel Slutsky 46 | * Dave Leepmann and [Applied](http://www.appliedscience.studio/articles/covid19.html{target=_blank .md-button}) 47 | * Clojure Data Science community on Zulip 48 | 49 | A huge thank you to Rich Hickey and the team at Cognitect for creating and continually guiding the Clojure language. Special thank you to Alex Miller who has provided excellent advice on working with Clojure and the CLI tooling. 50 | 51 | The Clojure community has been highly supportive of everyone using Clojure and I'd like to thank everyone for the feedback and contributions. I would also like to thank everyone that has joined in with the [London Clojurins community](https://www.meetup.com/London-Clojurians/){target=_blank}, [ClojureBridgeLondon](https://clojurebridgelondon.github.io/){target=_blank}, [Clojurians Slack community](http://clojurians.net/){target=_blank}, [Clojurians Zulip](https://clojurians.zulipchat.com/){target=_blank} community and [Clojureverse community](https://clojureverse.org/){target=_blank}. 52 | 53 | Thank you to everyone who sponsors the Practicalli websites and videos and for the [Clojurists Together sponsorship](https://www.clojuriststogether.org/){target=_blank}, it helps me continue the work at a much faster pace. 54 | 55 | Special thanks to [Bruce Durling](https://twitter.com/otfrom){target=_blank} for getting me into Cloure in the first place. 56 | 57 | ![GitHub contributors](https://img.shields.io/github/contributors/practicalli/clojure-data-science?style=for-the-badge&label=github%20contributors) 58 | -------------------------------------------------------------------------------- /docs/index.md: -------------------------------------------------------------------------------- 1 | # Practicalli Clojure Data Science 2 | 3 | ![Practicalli Data Science book banner](https://raw.githubusercontent.com/practicalli/graphic-design/live/book-covers/practicalli-clojure-data-science-book-banner-alpha.png) 4 | 5 | !!! WARNING "Alpha stage book" 6 | Currently the book is a collection of various external references with a small amount of commercial experience captured. 7 | 8 | 9 | A guide to writing Clojure services and using tools to support a range of data science activities. 10 | 11 | Topics will include data mining, data transformation and visualisation tools, techniques and libraries. Guides will take a story telling approach using visualisation tools to ensure a meaningful context in which design decisions are taken. 12 | 13 | Data Science is a vast subject area so this guide will recommend numerous resources that go deeper into specific areas. 14 | 15 | !!! QUOTE "John Stevenson, Practicalli" 16 | Without good data, there is no science 17 | 18 | 19 | ## Data Engineering 20 | 21 | Data Science relies on effecive data engineering workflow 22 | 23 | ![Data Engineering workflow - aquire generate clean explore transform](https://github.com/practicalli/graphic-design/blob/live/data-science/data-workflow-aquire-clean-explore-transform.png?raw=true){loading=lazy} 24 | 25 | 26 | !!! HINT "Dont skimp on the data" 27 | Effecive models require a good history of data to work with and evolve the capabilities of a model. 28 | 29 | If there is data to collect, then it should be collected by default and only dropped if proved to be of no value over (a long period of) time. 30 | 31 | Not capturing data from the ealiest possible time can greatly limit or at least delay the creation of more effective models. 32 | 33 | 34 | 35 | > This guide does not claim to make any attempt to teach you how to be a professional data scientist. 36 | 37 | 38 | [:fontawesome-solid-book-open: Data Mining](data-mining/){.md-button} 39 | 40 | [:fontawesome-solid-book-open: Data Transformation](data-transformation/){.md-button} 41 | 42 | [:fontawesome-solid-book-open: Story telling with data](visualisation/){.md-button} 43 | 44 | 45 | ## Practicalli projects 46 | 47 | [:fontawesome-brands-github: Simple Covid19 dashboard](https://github.com/practicalli/covid19-dashboard) - import data from CSV, simple data wrangling and creating a website dashboard using Oz for a vega-lite based visualisation and CSS for highlighting specific data points 48 | 49 | ![Practicalli Covid 19 Dashboard](https://github.com/practicalli/covid19-dashboard/raw/master/resources/covid19-tracker-oz-dashboard.png) 50 | 51 | [:fontawesome-brands-github: Oz Visualisation examples](https://github.com/practicalli/oz-visualisations) - learn some of the capabilities of Oz and demonstrate how to create a range of visualisations 52 | 53 | ![Practicalli Oz visualisation examples](https://github.com/practicalli/oz-visualisations/raw/master/resources/public/oz-visualizing-the-weather.png) 54 | 55 | - Ascii visualisation 56 | 57 | - Visualising survey data 58 | 59 | 60 | ## SciCloj community 61 | 62 | A community of people using Clojure and other tools to create amazing things in the realm of science 63 | 64 | - [Homepage](https://scicloj.github.io/){target=_blank} 65 | - [:fontawesome-brands-youtube: SciCloj YouTube channel](https://www.youtube.com/channel/UCaoZzhNzq-H7YiQczXKuXuw/){target=_blank} 66 | - [GitHub repository](https://github.com/scicloj/){target=_blank} 67 | - [SciCloj discussions](https://scicloj.github.io/pages/chat_streams/){target=_blank} on [Zulip Clojurians](http://clojurians.zulipchat.com/){target=_blank} 68 | 69 | 70 | ## Resources 71 | 72 | - [Interactive Programming for Artificial Intelligence](https://aiprobook.com/){target=_blank} - learn artificial intelligence programming skills 73 | - [dragan.rocks - articles on data science with Clojure](https://dragan.rocks/){target=_blank} 74 | - [clojuredatascience.com](http://clojuredatascience.com/){target=_blank} - Henry Garner 75 | - [:fontawesome-brands-youtube: Kaggle CareerCon 2019](https://www.youtube.com/playlist?list=PLqFaTIg4myu-npFrYu6cO7h7AI6bkcOlL) 76 | 77 | ## Navigate the book 78 | 79 | Use the mouse or built-in key bindings to navigate the pages of the book 80 | 81 | - ++p++ , ++comma++ : go to previous page 82 | - ++n++ , ++period++ : go to next page 83 | 84 | Use the search box to quickly find a specific topic 85 | 86 | - ++f++ , ++s++ , ++slash++ : open search dialog 87 | - ++arrow-down++ , ++arrow-up++ : select next / previous result 88 | - ++esc++ , ++tab++ : close search dialog 89 | - ++enter++ : follow selected result 90 | 91 | 92 | ## Sponsor Practicalli 93 | 94 | [![Sponsor practicalli-johnny](https://raw.githubusercontent.com/practicalli/graphic-design/live/buttons/practicalli-github-sponsors-button.png){ align=left loading=lazy }](https://github.com/sponsors/practicalli-johnny/) 95 | 96 | All sponsorship funds are used to support the continued development of [Practicalli series of books and videos](https://practical.li/){target=_blank}, although most work is done at personal cost and time. 97 | 98 | Thanks to [Cognitect](https://www.cognitect.com/){target=_blank}, [Nubank](https://nubank.com.br/){target=_blank} and a wide range of other [sponsors](https://github.com/sponsors/practicalli-johnny#sponsors){target=_blank} from the Clojure community for your continued support 99 | 100 | 101 | ## Creative commons license 102 | 103 |
104 | Creative Commons License 105 | This work is licensed under a Creative Commons Attribution 4.0 ShareAlike License (including images & stylesheets). 106 |
107 | -------------------------------------------------------------------------------- /docs/assets/practicalli-logo.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | image/svg+xml 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | I 31 | P 32 | 9 33 | 34 | 35 | 36 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Data Science with Clojure 2 | 3 | ```none 4 | ██████╗ ██████╗ █████╗ ██████╗████████╗██╗ ██████╗ █████╗ ██╗ ██╗ ██╗ 5 | ██╔══██╗██╔══██╗██╔══██╗██╔════╝╚══██╔══╝██║██╔════╝██╔══██╗██║ ██║ ██║ 6 | ██████╔╝██████╔╝███████║██║ ██║ ██║██║ ███████║██║ ██║ ██║ 7 | ██╔═══╝ ██╔══██╗██╔══██║██║ ██║ ██║██║ ██╔══██║██║ ██║ ██║ 8 | ██║ ██║ ██║██║ ██║╚██████╗ ██║ ██║╚██████╗██║ ██║███████╗███████╗██║ 9 | ╚═╝ ╚═╝ ╚═╝╚═╝ ╚═╝ ╚═════╝ ╚═╝ ╚═╝ ╚═════╝╚═╝ ╚═╝╚══════╝╚══════╝╚═╝ 10 | ``` 11 | 12 | ## Book status 13 | 14 | [![MegaLinter](https://github.com/practicalli/clojure-data-science/actions/workflows/megalinter.yaml/badge.svg)](https://github.com/practicalli/clojure-data-science/actions/workflows/megalinter.yaml) 15 | [![Publish Book](https://github.com/practicalli/clojure-data-science/actions/workflows/publish-book.yaml/badge.svg)](https://github.com/practicalli/clojur-data-sciencee/actions/workflows/publish-book.yaml) 16 | [![pages-build-deployment](https://github.com/practicalli/clojur-data-sciencee/actions/workflows/pages/pages-build-deployment/badge.svg)](https://github.com/practicalli/clojur-data-sciencee/actions/workflows/pages/pages-build-deployment) 17 | 18 | [![Ideas & Issues](https://img.shields.io/github/issues/practicalli/clojure-data-science?label=content%20ideas%20and%20issues&logoColor=green&style=for-the-badge)](https://github.com/practicalli/clojure-data-science/issues) 19 | [![Pull requests](https://img.shields.io/github/issues-pr/practicalli/clojure-data-science?style=for-the-badge)](https://github.com/practicalli/clojure-data-science/pulls) 20 | 21 | ![GitHub commit activity](https://img.shields.io/github/commit-activity/m/practicalli/clojure-practicalli-content?style=for-the-badge) 22 | ![GitHub contributors](https://img.shields.io/github/contributors/practicalli/clojure?style=for-the-badge&label=github%20contributors) 23 | 24 | ## Creative commons license 25 | 26 |
27 | Creative Commons License 28 | This work is licensed under a Creative Commons Attribution 4.0 ShareAlike License (including images & stylesheets). 29 |
30 | 31 | ## Overview 32 | 33 | A guide to writing services and applications in the context of data science 34 | 35 | Introduces libraries and tools useful for creating those applications 36 | 37 | Visualization of data 38 | 39 | Integration with other data science tooling 40 | 41 | Reference to useful mathematical techniques 42 | 43 | This guide does not claim to make any attempt to teach you how to be a professional data scientist. 44 | 45 | ![Practicalli Data Science book banner](https://raw.githubusercontent.com/practicalli/graphic-design/live/book-covers/practicalli-clojure-data-science-book-banner-alpha.png) 46 | 47 | > Alpha stage book - very early concept capturing experiences of data science work from the last 3 commercial engagments by the Practicalli team 48 | 49 | 50 | A practical guide to data mining, data wrangling and visualisation tools, techniques and libraries. Guides will take a story telling approach to ensure a meaningful context in which design decisions are taken. 51 | 52 | A starting point for Clojure developers on their journey into Data Science. As Data Science is such a vast subject area, no one resource can cover such diversity, so this guide will recommend numerous resources that go far deeper into specific areas. 53 | 54 | 55 | ## Target Audience 56 | 57 | This guide is aimed at software engineers interested in data science or working as data analysts or with data science teams. No data science experience is expected and the authors of this guide are not data science experts. 58 | 59 | Mathematics concepts underpin much of data science and will be explained as they are used, or recommend resources linked to. Mathematics experience should also notbe required. 60 | 61 | ## Contributing 62 | 63 | Issues and pull requests are most welcome. Please detail issues as much as you can. Pull requests are simpler to work with when they are specific to a page or at most a section. The smaller the change the quicker it is to review and merge. 64 | 65 | Please [see the detailed contributing section of the book](contributing.html) before raising an issue or pull request 66 | 67 | * [Current Issues](https://github.com/practicalli/clojure-data-science/issues) 68 | * [Current pull requests](https://github.com/practicalli/clojure-data-science/pulls) 69 | 70 | [Practicalli Clojure CLI Config](clojure/clojure-cli/practicalli-config.md) provides a user level configuration used in this guide and issues and pull requests can also be made there. 71 | 72 | By submitting content ideas and corrections you are agreeing they can be used in this workshop under the [Creative Commons Attribution ShareAlike 4.0 International license](https://creativecommons.org/licenses/by-sa/4.0/). Attribution will be detailed via [GitHub contributors](https://github.com/practicalli/clojure-data-science/graphs/contributors). 73 | 74 | 75 | ## Sponsor Practicalli 76 | 77 | [![Sponsor Practicalli via GitHub](https://raw.githubusercontent.com/practicalli/graphic-design/live/buttons/practicalli-github-sponsors-button.png)](https://github.com/sponsors/practicalli-johnny/) 78 | 79 | All sponsorship funds are used to support the continued development of [Practicalli series of books and videos](https://practical.li/), although most work is done at personal cost and time. 80 | 81 | Thanks to [Cognitect](https://www.cognitect.com/), [Nubank](https://nubank.com.br/) and a wide range of other [sponsors](https://github.com/sponsors/practicalli-johnny#sponsors) for your continued support 82 | 83 | 84 | ## Star History 85 | 86 | [![Star History Chart](https://api.star-history.com/svg?repos=practicalli/clojure-data-science&type=Date)](https://star-history.com/#practicalli/clojure-data-science&Date) 87 | 88 | 89 | ## GitHub Actions 90 | 91 | The megalinter GitHub actions will run when a pull request is created,checking basic markdown syntax. 92 | 93 | A review of the change will be carried out by the Practicalli team and the PR merged if the change is acceptable. 94 | 95 | The Publish Book GitHub action will run when PR's are merged into main (or the Practicalli team pushes changes to the default branch). 96 | 97 | Publish book workflow installs Material for MkDocs version 9 98 | 99 | 100 | ## Local development 101 | 102 | Install mkdocs version 9 using the Python pip package manager 103 | 104 | ```bash 105 | pip install mkdocs-material=="9.*" 106 | ``` 107 | 108 | Install the plugins used by the Practicalli site using Pip (these are also installed in the GitHub Action workflow) 109 | 110 | ```bash 111 | pip3 install mkdocs-material mkdocs-callouts mkdocs-glightbox mkdocs-git-revision-date-localized-plugin mkdocs-redirects pillow cairosvg 112 | ``` 113 | 114 | > pillow and cairosvg python packages are required for [Social Cards](https://squidfunk.github.io/mkdocs-material/setup/setting-up-social-cards/) 115 | 116 | Fork the GitHub repository and clone that fork to your computer, 117 | 118 | ```bash 119 | git clone https://github.com//.git 120 | 121 | ``` 122 | 123 | Run a local server from the root of the cloned project 124 | 125 | ```bash 126 | mkdocs serve 127 | ``` 128 | 129 | The website will open at 130 | -------------------------------------------------------------------------------- /mkdocs.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Practicalli Clojure Data Science 3 | site_name: Practicalli Clojure Data Science 4 | site_url: https://practical.li/clojure-data-science 5 | site_description: Practical guide to Clojure tools and services to support Data Science 6 | site_author: Practicalli 7 | site_org: https://practical.li 8 | copyright: Copyright © 2023 Practicali CC BY-SA 4.0 9 | repo_url: https://github.com/practicalli/clojure-data-science/ 10 | edit_uri: https://github.com/practicalli/clojure-data-science/edit/main/docs/ 11 | 12 | # Deployment 13 | # remote_name: origin 14 | remote_branch: gh-pages # deployment branch 15 | 16 | # Theme and styling 17 | theme: 18 | name: material 19 | logo: assets/images/practicalli-logo.png 20 | favicon: assets/favicon.svg 21 | features: 22 | - announce.dismiss 23 | - content.action.edit 24 | - content.action.view 25 | - content.code.annotate 26 | - content.code.copy 27 | - content.tabs.link 28 | - navigation.footer 29 | - navigation.indexes # Nav sections can have files 30 | - navigation.instant # Avoid page reloading for internal links 31 | - navigation.top 32 | - navigation.tracking # Update URL with sub-heading anchor names 33 | palette: 34 | # Palette toggle for light mode 35 | - media: "(prefers-color-scheme: light)" 36 | scheme: default 37 | primary: blue 38 | accent: teal 39 | toggle: 40 | icon: material/brightness-7 41 | name: Switch to dark mode 42 | # Palette toggle for dark mode 43 | - media: "(prefers-color-scheme: dark)" 44 | scheme: slate 45 | primary: blue 46 | accent: teal 47 | toggle: 48 | icon: material/brightness-4 49 | name: Switch to light mode 50 | # Override theme 51 | custom_dir: overrides 52 | 53 | extra_css: 54 | - assets/stylesheets/extra.css 55 | 56 | ## Additional styling 57 | markdown_extensions: 58 | - admonition 59 | - pymdownx.details 60 | - pymdownx.superfences 61 | - attr_list 62 | - md_in_html # Grids 63 | - footnotes # footnotes and abbreviations 64 | - pymdownx.emoji: 65 | emoji_index: !!python/name:material.extensions.emoji.twemoji 66 | emoji_generator: !!python/name:material.extensions.emoji.to_svg 67 | - pymdownx.highlight: 68 | anchor_linenums: true 69 | - pymdownx.inlinehilite 70 | - pymdownx.snippets: 71 | url_download: true 72 | - pymdownx.superfences: 73 | custom_fences: 74 | - name: mermaid 75 | class: mermaid 76 | format: !!python/name:pymdownx.superfences.fence_code_format 77 | - pymdownx.tabbed: 78 | alternate_style: true 79 | - pymdownx.keys # keyboard keys 80 | - pymdownx.magiclink 81 | - def_list # lists 82 | - pymdownx.tasklist: 83 | custom_checkbox: true # checkboxes 84 | - toc: 85 | permalink: λ︎ 86 | 87 | ## Plugins 88 | plugins: 89 | # Explicitly add search plugin when defining plugins in this configuration file 90 | - search 91 | - callouts 92 | - glightbox # Image aligning 93 | - git-revision-date-localized: # Update and Creation date of each page 94 | # enable_creation_date: true 95 | fallback_to_build_date: true 96 | 97 | # Generate Social Cards via CI only 98 | # in assets/images/social 99 | - social: 100 | cards: !ENV [MKDOCS_SOCIAL_CARDS_GENERATE, true] 101 | 102 | # Redirect pages when moved or changed 103 | - redirects: 104 | redirect_maps: 105 | repl-driven-development.md: introduction/repl-workflow.md 106 | clojure-cli/data-browsers/index.md: data-inspector/index.md 107 | clojure-editors/editor-install-guides/vspacecode-calva.md: clojure-editors/index.md 108 | clojure-editors/editor-user-guides/vspacecode-calva.md: clojure-editors/index.md 109 | 110 | # Footer / Social Media 111 | extra: 112 | analytics: 113 | provider: google 114 | property: G-FHWHDMDEJ4 115 | social: 116 | - icon: material/web 117 | link: https://practical.li/ 118 | - icon: fontawesome/brands/linkedin 119 | link: https://www.linkedin.com/in/jr0cket/ 120 | - icon: fontawesome/brands/slack 121 | link: https://clojurians.slack.com/messages/practicalli 122 | - icon: fontawesome/brands/twitter 123 | link: https://twitter.com/practical_li 124 | - icon: fontawesome/brands/github 125 | link: https://github.com/practicalli 126 | - icon: fontawesome/brands/docker 127 | link: https://hub.docker.com/u/practicalli 128 | 129 | # Navigation 130 | nav: 131 | - Introduction: 132 | - index.md 133 | 134 | - Persona: introduction/persona.md 135 | - What is Clojure?: introduction/what-is-clojure.md 136 | - Recommended books: introduction/data-science-books.md 137 | - REPL Workflow: introduction/repl-workflow.md 138 | - Concepts: 139 | - introduction/concepts/index.md 140 | - Contributing: introduction/contributing.md 141 | - Writing Tips: introduction/writing-tips.md 142 | - Install: 143 | - install/index.md 144 | ## Tooling 145 | - Collaborative Coding: 146 | - collaborative-coding/index.md 147 | - Shared Server: collaborative-coding/shared-repl-server.md 148 | ### - Literate programming: 149 | # - literate-programming/index.md 150 | 151 | - Notebooks: 152 | - notebooks/index.md 153 | - notespace: 154 | - notebooks/notespace/index.md 155 | - Configure project: notebooks/notespace/configure-notespace-project.md 156 | - Clerk: 157 | - notebooks/clerk/index.md 158 | - Next Journal: 159 | - notebooks/next-journal/index.md 160 | # - Pink Gorilla: 161 | # - notebooks/pink-gorilla/index.md 162 | - Data Mining: 163 | - data-mining/index.md 164 | - Web scraping: 165 | - data-mining/webscraping/index.md 166 | - HTML Tags: data-mining/webscraping/enlive.md 167 | - Table data: data-mining/webscraping/table-data.md 168 | # - CSV transforms: 169 | # - data-mining/comman-separate-values/index.md 170 | # - clojure.data.csv: data-mining/comman-separate-values/clojure-data-csv.md 171 | # - semantic-csv: data-mining/comman-separate-values/semantic-csv.md 172 | # - JSON transforms: data-mining/json/index.md 173 | - Data Transformation: 174 | - data-transformation/index.md 175 | - Tablecloth: data-transformation/tablecloth/index.md 176 | - Visualisation: 177 | - visualisation/index.md 178 | # - Story telling: visualization/story-telling/index.md 179 | # - Oz: 180 | # - visualization/oz/index.md 181 | # - Create Project: visualization/oz/create-project.md 182 | # - Clojure Spec: visualization/oz/clojure-spec.md 183 | # - Generate Static site: visualization/oz/clojure-spec.md 184 | # - ggplot: 185 | # - visualization/ggplot/index.md 186 | - R: 187 | - r/index.md 188 | # - Incantor: r/incantor.md 189 | # - Python Integration: 190 | # - python-integration/index.md 191 | # - libpython-clj: python-integration/libpython-clj 192 | - Mathematics: 193 | - mathematics/index.md 194 | - Linear Algebra: 195 | - mathematics/linear-algebra/index.md 196 | - Natural Language: 197 | - natural-language/index.md 198 | - Artificial Intelligence: 199 | - artificial-intelligence/index.md 200 | - Reference: 201 | - reference/index.md 202 | # - Data Files: reference/data-files.md 203 | -------------------------------------------------------------------------------- /docs/collaborative-coding/shared-repl-server.md: -------------------------------------------------------------------------------- 1 | # Shared REPL server 2 | 3 | ??? TODO "Create a remote serve" 4 | Define how to create a suitable remote server on AWS, Google, etc. 5 | 6 | !!! WARNING "Switch off Server when done" 7 | Dont forget to switch the server off when not in use to avoid additional costs 8 | 9 | 10 | ## Install Clojure 11 | 12 | Use the Clojure.org Getting Started instructions to download Clojure for Linux and run the install script. This installs `clojure` command system wide (`/usr/local/bin/`) along with the `clj` wapper that runs a terminal REPL using `rlwrap` (TODO: check rlwrap is installed). 13 | 14 | Use a range of community tools by installing the practicalli/clojure-deps-edn configuration (first deleting the `~/.clojure` directory if it exists - create if you run `clojure` or `clj` for the first time) 15 | 16 | ```bash 17 | git clone https://github.com/practicalli/clojure-deps-edn.git ~/.config/clojure/ 18 | ``` 19 | 20 | Created a new project on the remote server using a [Clojure project templates](https://practical.li/clojure/clojure-cli/projects/create-from-template/). For example a simple command line application 21 | 22 | ```bash 23 | clojure -X:project/create :template app :name practicalli/science-is-fun 24 | ``` 25 | 26 | ## Run REPL with nREPL server 27 | 28 | Start a REPL process with an nREPL server to connect a [Clojure aware editor](https://practical.li/clojure/clojure-editors/). 29 | 30 | The REPL process can be started with a rich terminal UI if actively using the REPL prompt. Or as a simple prompt with only command history or even as a headless process. 31 | 32 | === "Rich Terminal UI" 33 | Run a feature rich terminal REPL using the `:repl/rebel` alias from [Practicalli Clojure CLI Config](https://practical.li/clojure/clojure-cli/practicalli-config/) 34 | ```shell 35 | clojure -M:repl/rebel 36 | ``` 37 | 38 | === "Headless REPL process" 39 | Run a non-interactive 'headless' REPL process if all the interaction is to be done via a [Clojure aware editor](https://practical.li/clojure/clojure-editors/) 40 | 41 | Use the `:repl/headless` alias from [Practicalli Clojure CLI Config](https://practical.li/clojure/clojure-cli/practicalli-config/) 42 | ```shell 43 | clojure -M:repl/headless 44 | ``` 45 | 46 | === "Simple Built-in REPL" 47 | Run the REPL process with the simple prompt provided with Clojure CLI if resources are very constrained. The `clj` wrapper requires `rlwrap` binary to be available on the operating system execution path 48 | ```shell 49 | clj 50 | ``` 51 | 52 | 53 | ## Configure SSH connection 54 | 55 | Generate a permissions file, `.pem` from the server (TODO: how to generate .pem files) and save it to `~/.ssh/` directory (or your preferred location). 56 | 57 | Edit the `~/.ssh/config` file (create the file if needed) and add a host configuration for the remote server. 58 | 59 | Add the IP address of the remote server to `HostName`. 60 | 61 | Use the `LocalForward` for the port used by the nREPL server. 62 | 63 | ```bash 64 | Host remote-clojure-server 65 | HostName 99.99.99.99 66 | IdentityFile ~/.ssh/remote-server.pem 67 | User ubuntu 68 | PasswordAuthentication yes 69 | LocalForward 1234 localhost:1234 70 | Port 22 71 | ``` 72 | 73 | 74 | ## Start REPL with Editor support 75 | 76 | The majority of [Clojure aware editors](https://practicalli.github.io/clojure/clojure-editors/) can connect to an external REPL using the nREPL protocol. Emacs CIDER, VSCode Calva and NeoVim Conjure all use nREPL and the Cider middleware. 77 | 78 | The `:middleware/cider-clj` will inject the nREPL and Cider middleware libraries. 79 | 80 | SSH into the remote server. 81 | 82 | Change to a Clojure project 83 | 84 | Start the REPL using the cider middleware on the same port as defined in `LocalForward` in the SSH configuration, using the `-p` to set the port number. 85 | 86 | ```bash 87 | clojure -M:middleware/cider-clj -p 1234 88 | ``` 89 | 90 | 91 | ## Configure Emacs 92 | 93 | Emacs can be used to connect to a running Clojure project that has been run with the nREPL and Cider middleware, e.g. `:middleware/cider-clj` from [`practicalli/clojure-deps-edn`](http://practicalli.github.io/clojure/clojure-tools/install/community-tools.html) 94 | 95 | Edit your Emacs `~/.emacs.d/init.el` file and add the following configuration. 96 | 97 | For Spacemacs, edit `~/.spacemacs` and add the following code to `dotspacemacs/user-config` 98 | 99 | ```emacs 100 | (setq nrepl-use-ssh-fallback-for-remote-hosts t) 101 | ``` 102 | 103 | 104 | ## NeoVim Conjure on remote server 105 | 106 | NeoVim and Conjure are installed on the remote server using the [Practicalli install guide](https://practicalli.github.io/clojure/clojure-editors/editor-install-guides/neovim-conjure.html). 107 | 108 | Conjure will automatically connect to a running REPL 109 | 110 | !!! HINT "Conjure uses .nrepl-port" 111 | Conjure looks for the `.nrepl-port` in the root of the project. If this file is not created when running the REPL, create the file and use the same value as `LocalForward` from the SSH configuration. 112 | 113 | 114 | ## Apache HTTPd server reverse proxy 115 | 116 | Use a reverse proxy if you wish to expose visualizations via a browser, such as Oz or notespace. 117 | 118 | ssh into the remote server with an account that has sudo privileges 119 | 120 | Enable the reverse proxy module `mod_proxy_http` which has a dependency on `mod_proxy` and is also enabled automatically. 121 | 122 | ```bash 123 | sudo a2enmod proxy_http 124 | ``` 125 | 126 | ??? HINT "Modules are part of the Ubuntu Apache install" 127 | Proxy modules are already part of the Apache2 package install on Ubuntu, so additional package install is not required 128 | 129 | Create a new configuration by copying the `/etc/apache2/sites-available/000-default.conf` to `/etc/apache2/sites-available/reverse-proxy.conf`. This ensures you have a working backup if the configuration breaks or you wish to quickly switch back to a non-proxy configuration. 130 | 131 | Edit `/etc/apache2/sites-available/reverse-proxy.conf` and add the following configuration inside the `VirtualHost` directive 132 | ```bash 133 | ProxyRequests Off 134 | ProxyPass "/" "http://localhost:8080/" 135 | ProxyPassReverse "/" "http://localhost:8080/" 136 | ``` 137 | 138 | `sudo a2ensite reverse-proxy` to enable the reverse-proxy configuration, which adds a symbolic link in the `/etc/apache2/sites-enabled/` directory (all configurations in this directory are included via `/etc/apache2/apache2.conf`) 139 | 140 | Disable the other sites so that they do not over-ride the reverse proxy configuraion. This simply removes the symbolic link from sites-enabled directory, their configuration is still in the sites-available directory. 141 | 142 | ```bash 143 | sudo a2dissite scicloj 144 | sudo a2dissite 000-default 145 | ``` 146 | 147 | `sudo systemctl restart apache2` to restart apache http server with the reverse proxy configuration 148 | 149 | 150 | To use a number of projects or visualisation tools, update the `/etc/apache2/sites-available/reverse-proxy.conf` and add more `ProxyPass` and `ProxyReverse` directives 151 | 152 | ```bash 153 | ProxyPass "/oz" "http://localhost:8080/" 154 | ProxyPassReverse "/oz" "http://localhost:8080/" 155 | 156 | ProxyPass "/notespace" "http://localhost:8181/" 157 | ProxyPassReverse "/notespace" "http://localhost:8181/" 158 | ``` 159 | 160 | !!! WARNING "Oz only working on `/`" 161 | Using the reverse proxy configuration so far, Oz will only render the server page and not the views. It is assumed that this is because of web sockets and will need to enable the mod proxy module for websockets too. 162 | 163 | 164 | ## References 165 | 166 | * https://httpd.apache.org/docs/2.4/mod/mod_proxy_wstunnel.html 167 | * https://www.serverlab.ca/tutorials/linux/web-servers-linux/how-to-reverse-proxy-websockets-with-apache-2-4/ 168 | * https://websiteforstudents.com/configure-reverse-proxies-using-apache2-http-server-on-ubuntu-18-04/ 169 | -------------------------------------------------------------------------------- /.github/config/markdown-lint.jsonc: -------------------------------------------------------------------------------- 1 | // Example markdownlint configuration with all properties set to their default value 2 | { 3 | 4 | // Default state for all rules 5 | "default": true, 6 | 7 | // Path to configuration file to extend 8 | "extends": null, 9 | 10 | // MD001/heading-increment/header-increment - Heading levels should only increment by one level at a time 11 | "MD001": true, 12 | 13 | // MD002/first-heading-h1/first-header-h1 - First heading should be a top-level heading 14 | "MD002": { 15 | // Heading level 16 | "level": 1 17 | }, 18 | 19 | // MD003/heading-style/header-style - Heading style 20 | "MD003": { 21 | // Heading style 22 | "style": "consistent" 23 | }, 24 | 25 | // MD004/ul-style - Unordered list style 26 | "MD004": { 27 | // List style 28 | "style": "consistent" 29 | }, 30 | 31 | // MD005/list-indent - Inconsistent indentation for list items at the same level 32 | "MD005": true, 33 | 34 | // MD006/ul-start-left - Consider starting bulleted lists at the beginning of the line 35 | "MD006": true, 36 | 37 | // MD007/ul-indent - Unordered list indentation 38 | "MD007": { 39 | // Spaces for indent 40 | "indent": 2, 41 | // Whether to indent the first level of the list 42 | "start_indented": false, 43 | // Spaces for first level indent (when start_indented is set) 44 | "start_indent": 2 45 | }, 46 | 47 | // MD009/no-trailing-spaces - Trailing spaces 48 | "MD009": { 49 | // Spaces for line break 50 | "br_spaces": 2, 51 | // Allow spaces for empty lines in list items 52 | "list_item_empty_lines": false, 53 | // Include unnecessary breaks 54 | "strict": true 55 | }, 56 | 57 | // MD010/no-hard-tabs - Hard tabs 58 | "MD010": { 59 | // Include code blocks 60 | "code_blocks": true, 61 | // Fenced code languages to ignore 62 | "ignore_code_languages": [], 63 | // Number of spaces for each hard tab 64 | "spaces_per_tab": 1 65 | }, 66 | 67 | // MD011/no-reversed-links - Reversed link syntax 68 | "MD011": true, 69 | 70 | // MD012/no-multiple-blanks - Multiple consecutive blank lines 71 | "MD012": { 72 | // Consecutive blank lines 73 | "maximum": 2 74 | }, 75 | 76 | // MD013/line-length - Line length 77 | "MD013": { 78 | // Number of characters 79 | "line_length": 420, 80 | // Number of characters for headings 81 | "heading_line_length": 90, 82 | // Number of characters for code blocks 83 | "code_block_line_length": 420, 84 | // Include code blocks 85 | "code_blocks": true, 86 | // Include tables 87 | "tables": true, 88 | // Include headings 89 | "headings": true, 90 | // Include headings 91 | "headers": true, 92 | // Strict length checking 93 | "strict": false, 94 | // Stern length checking 95 | "stern": true 96 | }, 97 | 98 | // MD014/commands-show-output - Dollar signs used before commands without showing output 99 | "MD014": true, 100 | 101 | // MD018/no-missing-space-atx - No space after hash on atx style heading 102 | "MD018": true, 103 | 104 | // MD019/no-multiple-space-atx - Multiple spaces after hash on atx style heading 105 | "MD019": true, 106 | 107 | // MD020/no-missing-space-closed-atx - No space inside hashes on closed atx style heading 108 | "MD020": true, 109 | 110 | // MD021/no-multiple-space-closed-atx - Multiple spaces inside hashes on closed atx style heading 111 | "MD021": true, 112 | 113 | // MD022/blanks-around-headings/blanks-around-headers - Headings should be surrounded by blank lines 114 | "MD022": { 115 | // Blank lines above heading 116 | "lines_above": 1, 117 | // Blank lines below heading 118 | "lines_below": 1 119 | }, 120 | 121 | // MD023/heading-start-left/header-start-left - Headings must start at the beginning of the line 122 | "MD023": true, 123 | 124 | // MD024/no-duplicate-heading/no-duplicate-header - Multiple headings with the same content 125 | "MD024": { 126 | // Only check sibling headings 127 | "allow_different_nesting": false, 128 | // Only check sibling headings 129 | "siblings_only": false 130 | }, 131 | 132 | // MD025/single-title/single-h1 - Multiple top-level headings in the same document 133 | "MD025": { 134 | // Heading level 135 | "level": 1, 136 | // RegExp for matching title in front matter 137 | "front_matter_title": "^\\s*title\\s*[:=]" 138 | }, 139 | 140 | // MD026/no-trailing-punctuation - Trailing punctuation in heading 141 | "MD026": { 142 | // Punctuation characters not allowed at end of headings 143 | "punctuation": ".,;:!。,;:!" 144 | }, 145 | 146 | // MD027/no-multiple-space-blockquote - Multiple spaces after blockquote symbol 147 | "MD027": true, 148 | 149 | // MD028/no-blanks-blockquote - Blank line inside blockquote 150 | "MD028": true, 151 | 152 | // MD029/ol-prefix - Ordered list item prefix 153 | "MD029": { 154 | // List style 155 | "style": "one_or_ordered" 156 | }, 157 | 158 | // MD030/list-marker-space - Spaces after list markers 159 | "MD030": { 160 | // Spaces for single-line unordered list items 161 | "ul_single": 1, 162 | // Spaces for single-line ordered list items 163 | "ol_single": 1, 164 | // Spaces for multi-line unordered list items 165 | "ul_multi": 1, 166 | // Spaces for multi-line ordered list items 167 | "ol_multi": 1 168 | }, 169 | 170 | // MD031/blanks-around-fences - Fenced code blocks should be surrounded by blank lines 171 | "MD031": { 172 | // Include list items 173 | "list_items": true 174 | }, 175 | 176 | // MD032/blanks-around-lists - Lists should be surrounded by blank lines 177 | "MD032": true, 178 | 179 | // MD033/no-inline-html - Inline HTML 180 | "MD033": { 181 | // Allowed elements 182 | "allowed_elements": ["a", "iframe", "img", "p", "div"] 183 | }, 184 | 185 | // MD034/no-bare-urls - Bare URL used 186 | "MD034": true, 187 | 188 | // MD035/hr-style - Horizontal rule style 189 | "MD035": { 190 | // Horizontal rule style 191 | "style": "consistent" 192 | }, 193 | 194 | // MD036/no-emphasis-as-heading/no-emphasis-as-header - Emphasis used instead of a heading 195 | "MD036": { 196 | // Punctuation characters 197 | "punctuation": ".,;:!?。,;:!?" 198 | }, 199 | 200 | // MD037/no-space-in-emphasis - Spaces inside emphasis markers 201 | "MD037": true, 202 | 203 | // MD038/no-space-in-code - Spaces inside code span elements 204 | "MD038": true, 205 | 206 | // MD039/no-space-in-links - Spaces inside link text 207 | "MD039": true, 208 | 209 | // MD040/fenced-code-language - Fenced code blocks should have a language specified 210 | "MD040": { 211 | // List of languages 212 | "allowed_languages": [], 213 | // Require language only 214 | "language_only": false 215 | }, 216 | 217 | // MD041/first-line-heading/first-line-h1 - First line in a file should be a top-level heading 218 | "MD041": { 219 | // Heading level 220 | "level": 1, 221 | // RegExp for matching title in front matter 222 | "front_matter_title": "^\\s*title\\s*[:=]" 223 | }, 224 | 225 | // MD042/no-empty-links - No empty links 226 | "MD042": true, 227 | 228 | // MD043/required-headings/required-headers - Required heading structure 229 | "MD043": {}, 230 | 231 | // MD044/proper-names - Proper names should have the correct capitalization 232 | "MD044": { 233 | // List of proper names 234 | "names": [], 235 | // Include code blocks 236 | "code_blocks": true, 237 | // Include HTML elements 238 | "html_elements": true 239 | }, 240 | 241 | // MD045/no-alt-text - Images should have alternate text (alt text) 242 | "MD045": true, 243 | 244 | // MD046/code-block-style - Code block style 245 | "MD046": { 246 | // Block style 247 | "style": "consistent" 248 | }, 249 | 250 | // MD047/single-trailing-newline - Files should end with a single newline character 251 | "MD047": true, 252 | 253 | // MD048/code-fence-style - Code fence style 254 | "MD048": { 255 | // Code fence style 256 | "style": "consistent" 257 | }, 258 | 259 | // MD049/emphasis-style - Emphasis style should be consistent 260 | "MD049": { 261 | // Emphasis style should be consistent 262 | "style": "consistent" 263 | }, 264 | 265 | // MD050/strong-style - Strong style should be consistent 266 | "MD050": { 267 | // Strong style should be consistent 268 | "style": "consistent" 269 | }, 270 | 271 | // MD051/link-fragments - Link fragments should be valid 272 | "MD051": true, 273 | 274 | // MD052/reference-links-images - Reference links and images should use a label that is defined 275 | "MD052": true, 276 | 277 | // MD053/link-image-reference-definitions - Link and image reference definitions should be needed 278 | "MD053": { 279 | // Ignored definitions 280 | "ignored_definitions": ["//"] 281 | } 282 | } 283 | -------------------------------------------------------------------------------- /docs/natural-language/index.md: -------------------------------------------------------------------------------- 1 | # Natural Language Processing Introduction 2 | 3 | Natural language processing comprises of a set of computational techniques to understand natural languages such as English, Spanish, Chinese, etc. 4 | 5 | Natural Language Processing (NLP) consists of a set of algorithms and tools so that machines can make sense of data available in natural (human) languages such as English, German, French, etc. Although there are traces of NLP research since a long time ago, the concept got well defined in the 1950s, with the breakthrough research work of Alan Turing and Noam Chomsky. 6 | 7 | The primary objectives are: 8 | 9 | * Understand and implement NLP techniques for sentiment classification, information retrieval (search engines) and topic modeling. 10 | 11 | * Understand and implement NLP techniques for uncovering text syntax and structure. That is, predicting part-of-speech tags, parse tree structure, named-entities like people and places, etc. 12 | 13 | * Understand and implement NLP techniques for some non-traditional topics such as language identification, spelling correction, and creating word clouds. 14 | 15 | * Understand and implement deep learning methods for NLP (also called Deep NLP), and apply them to text generation and language translation. These methods represent the state-of-the-art for advanced tasks such as language translation, question answering, speech recognition and music composition and power systems like the Google Assistant and Amazon Alexa. 16 | 17 | 18 | ## What is Natural Language? 19 | 20 | Natural language refers to the medium in which humans communicate with each other. This could be in the form of writings (text) for example emails, articles, news, blogs, bank documents, etc or speech for example lectures, speeches, audio calls, etc. There are trillions of web pages full of natural text, so imagine the scale of data available today. 21 | 22 | NLP algorithms often model the hierarchical structure of natural language i.e. characters form words, words form phrases, phrases form sentences, sentences form paragraphs, and paragraphs form documents. 23 | 24 | ## Applications of NLP 25 | 26 | Common NLP applications and use cases: 27 | 28 | Document Classification deals with classifying textual documents and assigning it one or multiple categories. Example applications include, classifying news articles into categories like sports, politics, business, technology, etc, or segregating different types of invoices and sales deeds in a large company. 29 | 30 | Document Clustering is used to find similar documents and segregate them to form groups. Documents that are closely related will be part of the same group. Example applications include finding similar questions that have already posted in a forum, or finding new published medical research related to a patient's symptoms. 31 | 32 | Sentiment Analysis is used to classify text for different sentiments ranging from negative to neutral to positive. This is commonly used to understand customer opinions from product reviews or their posts on social media. 33 | 34 | Document Summarization helps to extract the most important and central ideas in a document. For example, one could train a model to summarize a 3000 word article to 200 words. This allows the reader to save time and get the gist, and is often useful for news, research papers, etc. 35 | 36 | Named Entity Recognition, also known as entity extraction, identifies named entities and classifies them into categories such as person, organization, location, etc. Such a system can be used by stock investors to follow news corresponding to companies they have invested in, or to get news relating to your favorite sports players and teams from across various news sources. 37 | 38 | Question Answering systems are intelligent systems that generate responses to the questions being asked by the user. Such systems often use facts and rules stored in their knowledge base. Many conversational AI and personal assistant solutions (for example Amazon Alexa) are able to perform question answering. 39 | 40 | Machine Translation is the task of automatically translating from one natural language to another. This is the task Google translate is performing when you visit a website that is written in a language you do not understand. 41 | 42 | ## Components of NLP 43 | 44 | There are 2 main components of natural language processing - natural language understanding, and natural language generation. 45 | 46 | 47 | ### Natural Language Understanding (NLU) 48 | NLU enables machines to understand the intent or the meaning of the text. The different levels of analysis that NLU requires are as follows: 49 | 50 | Morphological Analysis is the analysis of the structure of individual words. A morpheme is defined as the "minimal unit of meaning". For example, the words "care", "cares", "caring", "careless", "careful", "uncaring" are different forms of the same word, with stem "care". Also note that the structure of a word could include a prefix ("un-" in "uncaring") or a suffix ("-less" in "careless"). 51 | 52 | Syntactic Analysis (also called parsing) involves analysis of words in the sentence for grammar. For example, syntactic analysis is used by Microsoft Word, Google Docs, etc to highlight phrases with incorrect grammar. 53 | 54 | Semantic Analysis uses morphological and syntactic knowledge to understand the meaning, intent and purpose of the text as a whole. For example, consider the two sentences "I went to the market in my shorts" vs "I went to the market in the city". In terms of grammar (syntax) the two sentences are equivalent. However, based on semantics (meaning), we know that "in my shorts" refers to "I", whereas "in the city" refers to the "market". Semantic analysis is necessary since grammar leaves a lot of ambiguity, and we implicitly rely on a shared understanding of the world to communicate with each other. 55 | 56 | Discourse Analysis is a more advanced stage of NLU where we perform syntactic or semantic analysis in a longer piece of text. That is, the analysis is performed over a paragraph or an entire document, as opposed to a single sentence. 57 | 58 | 59 | ### Natural Language Generation (NLG) 60 | 61 | Once the machine understands the natural language, NLG is used to respond in natural language, or to produce written text. Recent applications include chat-bots and personal assistants like Alexa and Siri. In general, NLG systems are more complex than NLU. Some approaches to NLG are: 62 | 63 | Content Determination involves deciding what information we need to convey in the generated text. There are pre-built schemas or templates to specify the content. Using knowledge-based rules and pattern detection, the words in these templates are predicted. For example, when Google Assistant is asked the age of a person, it responds "{person_name} is {age} years old. {He/She} was born on {date}." 64 | 65 | Planning / Micro-planning involves finding, mixing and merging different sentence representations into more concise representation. For example, if the predicted response for an input is a set of 3 sentences - "Larry is feeling sleepy", "Larry is drinking coffee" and "Coffee is hot", a better concise representation would be "Since Larry is sleepy, he is drinking hot coffee". 66 | 67 | Deep Learning is a subfield within machine learning which has proven very successful in applications that require language generation, such as translation and question answering. The last section of this course covers Deep Learning methods for NLP. Whereas the above two approaches involve lots of hand-crafted rules, deep learning can be used in an 'end-to-end' fashion. 68 | 69 | ## Challenges in NLP 70 | 71 | The properties and nature of natural language bring many challenges to NLP. A few of these are discussed below: 72 | 73 | Assumed knowledge / common sense: By far the biggest challenge in NLP is that anytime we communicate, we have an inherent assumption about the knowledge the reader has. Even though this knowledge might seem very basic for a human, a computer does not have access to such knowledge. When we say "Fox jumped over the fence" vs "Fox jumped over the building" vs "Fox jumped over the leaf", we know that the first is sensible, second is not possible, and third is too insignificant (and hence weird) to mention. All of this depends on our understanding of a fox, a fence, a leaf, etc. 74 | 75 | Volume: A huge amount of text is available online, and in documents within organizations. This large amount of data is a great bonus for machine learning approaches to NLP, but at the same time, it requires our solutions to be computationally efficient. 76 | 77 | Variation: Natural language is infinitely wide in terms of vocabulary, dialects, etc. Understanding all these different types of languages and variations in dialects makes NLP much harder than having one standardized language across the globe. 78 | 79 | Complex: The way concepts link together to convey meaning is often quite complex for an algorithm to understand. For example, a large percentage of words have different meaning depending on the context in which they are used. Words like "set" and "run" have more than 300 definitions depending on context (homonyms). Consider the word "having" in "Adam is having coffee" vs "Adam is having a good time". 80 | 81 | Grammar is vague: Sometimes grammatical rules are not well defined, and often, there are exceptions to grammatical rules. Grammar also leaves ambiguity in possible meanings. We saw this in the "I went to the market" example above. Another example is, "Robert really likes Michael. He is such a charming personality." Based on grammar alone, it is not possible to know whether "he" refers to "Robert" or "Michael". 82 | 83 | Expressions: Emotions or sarcasm in written text have proven quite difficult for an algorithm to understand. The literal meaning for "I work 40 hours a week to be this poor" is very different from the intended meaning. 84 | 85 | ## Approaches to NLP 86 | Broadly there are 3 different ways to approach any problem in NLP. 87 | 88 | Computational Linguistics 89 | It is the mathematical or scientific study of language. In this approach, classical linguists devised various language rules for grammar, part-of-speech, etc. The NLP algorithms are based on discrete mathematics such as automata, probability, and co-occurrence statistics of words. Most of the theories given by Noam Chomsky were based on computational linguistics. 90 | 91 | ## Machine Learning 92 | In machine learning approach, the text data is converted to a numerical vector. These vectors could be based on the frequency of words or how they co-occur with neighbouring words. One such meaningful representation is tf-idf, which we'll see in the next tutorial. 93 | 94 | Once the text is transformed to vectors, machine learning algorithms such as Naive Bayes', Support Vector Machine, etc. can be used to classify text data or perform other tasks. 95 | 96 | Machine learning methods for NLP have now become so popular that they have mostly taken over computational linguistics. The fact that substantial amount of text is now available to train accurate models is an important contributing factor to this development. The other factor for inclination towards machine learning is that rule-based systems are difficult to scale because the exclusive hand-crafted rules are required for each natural language, while machine learning can be adopted easily across different languages. 97 | 98 | ## Deep Learning 99 | Deep learning is a subfield of machine learning focused on training models that are more complex and flexible than traditional machine learning models. These methods have achieved state-of-the-art results in problems such as translation and question answering. The increase in processing power and availability of large amounts of textual data has enabled us to train deep neural networks and achieve high accuracy. 100 | 101 | ## Conclusion 102 | In the upcoming tutorials, we will encounter many of the applications, approaches and challenges described in this article. Wishing you a happy journey in exploring natural language processing. :) 103 | 104 | -------------------------------------------------------------------------------- /docs/introduction/repl-workflow.md: -------------------------------------------------------------------------------- 1 | # REPL Driven Development 2 | 3 | ![Clojure repl driven development](https://raw.githubusercontent.com/practicalli/graphic-design/live/clojure/clojure-repl-workflow-concept.png){loading=lazy} 4 | 5 | !!! Quote "Always be REPL'ing" 6 | Coding without a REPL feels limiting. The REPL provides fast feedback from code as its crafted, testing assumptions and design choices every step of the journey to a solution 7 | - John Stevenson, Practical.li 8 | 9 | Clojure is a powerful, fun and highly productive language for developing applications and services. 10 | The clear language design is supported by a powerful development environment known as the REPL (read, evaluate, print, loop). The REPL gives you instant feedback on what your code does and enables you to test either a single expression or run the whole application (including tests). 11 | 12 | **REPL driven development is the foundation of working with Clojure effectively** 13 | 14 | An effective Clojure workflow begins by running a REPL process. Clojure expressions are written and evaluated immediately to provide instant feedback. The REPL feedback helps test the assumptions that are driving the design choices. 15 | 16 | * Read - code is read by the Clojure reader, passing any macros to the macro reader which converts those macros into Clojure code. 17 | * Evaluate - code is compiled into the host language (e.g. Java bytecode) and executed 18 | * Print - results of the code are displayed, either in the REPL or as part of the application. 19 | * Loop - the REPL is a continuous process that evaluates code, either a single expression or the whole application. 20 | 21 | Design decisions and valuable data from REPL experiments can be codified as [specifications](#data-and-function-specifications) and [unit tests](#test-driven-development-and-repl-driven-development) 22 | 23 | !!! HINT "Practicalli REPL Reloaded Workflow" 24 | The principles of REPL driven development are implemented in practice using the [Practicalli REPL Reloaded Workflow and supporting tooling](https://practical.li/clojure/clojure-cli/repl-reloaded/){target=_blank}. This workflow uses Portal to inspect all evaluation results and log events, hot-load libraries into the running REPL process and reloads namespaces to support major refactor changes. 25 | 26 | ## Evaluating source code 27 | 28 | ![Clojure repl driven development using Clojure aware editor](https://raw.githubusercontent.com/practicalli/graphic-design/live/clojure/clojure-repl-driven-development-clojure-aware-editor.png){align=right loading=lazy} 29 | 30 | A REPL connected editor is the primary tool for evaluating Clojure code from source code files, displaying the results inline. 31 | 32 | Source code is automatically evaluated in its respective namespace, removing the need to change namespaces in the REPL with (`in-ns`) or use fully qualified names to call functions. 33 | 34 |

35 | 36 |

37 | 38 | ??? HINT "Evaluate Clojure in a Terminal UI REPL" 39 | Entering expressions at the REPL prompt evaluates the expression immediately, returning the result directly underneath 40 | ![Clojure Terminal UI REPL](https://raw.githubusercontent.com/practicalli/graphic-design/live/clojure/rebel/clojure-repl-rebel-eval-map-function-dark.png#only-dark){loading=lazy} 41 | ![Clojure Terminal UI REPL](https://raw.githubusercontent.com/practicalli/graphic-design/live/clojure/rebel/clojure-repl-rebel-eval-map-function-light.png#only-light){loading=lazy} 42 | 43 | ## Rich Comment blocks - living documentation 44 | 45 | The `(comment ,,,)` function wraps code that is only run directly by the developer using a [Clojure aware editor](https://practical.li/clojure/clojure-editors/){target=_blank}. 46 | 47 | Expressions in rich comment blocks can represent how to use the functions that make up the namespace API. For example, starting/restarting the system, updating the database, etc. Expressions provide examples of calling functions with typical arguments and make a project more accessible and easier to work with. 48 | 49 | !!! EXAMPLE "Clojure Rich Comment to manage a service" 50 | ```clojure 51 | (ns practicalli.gameboard.service) 52 | 53 | (defn app-server-start [port] ,,,) 54 | (defn app-server-start [] ,,,) 55 | (defn app-server-restart [] ,,,) 56 | 57 | (defn -main 58 | "Start the service using system components" 59 | [& options] ,,,) 60 | 61 | (comment 62 | (-main) 63 | (app-server-start 8888) 64 | (app-server-stop) 65 | (app-server-restart 8888) 66 | 67 | (System/getenv "PORT") 68 | (def environment (System/getenv)) 69 | (def system-properties (System/getProperties)) 70 | ) ; End of rich comment block 71 | ``` 72 | 73 | Rich comment blocks are very useful for rapidly iterating over different design decisions by including the same function but with different implementations. Hide [clj-kondo linter](https://practical.li/clojure/clojure-cli/install/code-analysis.html){target=_blank} warnings for redefined vars (`def`, `defn`) when using this approach. 74 | 75 | ```clojure 76 | ;; Rich comment block with redefined vars ignored 77 | #_{:clj-kondo/ignore [:redefined-var]} 78 | (comment 79 | (defn value-added-tax [] 80 | ;; algorithm design - first try) 81 | 82 | (defn value-added-tax [] 83 | ;; algorithm design - second try) 84 | 85 | ) ;; End of rich comment block 86 | ``` 87 | 88 | The "Rich" in the name is an honourary mention to Rich Hickey, the author and benevolent dictator of Clojure design. 89 | 90 | ## Design Journal 91 | 92 | A journal of design decisions makes the code easier to understand and maintain. Code examples of design decisions and alternative design discussions are captured, reducing the time spent revisiting those discussions. 93 | 94 | Journals simplify the developer on-boarding processes as the journey through design decisions are already documented. 95 | 96 | A Design Journal is usually created in a separate namespace, although it may start as a rich comment at the bottom of a namespace. 97 | 98 | A journal should cover the following aspects 99 | 100 | * Relevant expressions use to test assumptions about design options. 101 | * Examples of design choices not taken and discussions why (saves repeating the same design discussions) 102 | * Expressions that can be evaluated to explain how a function or parts of a function work 103 | 104 | The design journal can be used to create meaningful documentation for the project very easily and should prevent time spent on repeating the same conversations. 105 | 106 | !!! HINT "Example design journal" 107 | [Design journal for TicTacToe game using Reagent, ClojureScript and Scalable Vector Graphics](https://github.com/practicalli-john/tictactoe-reagent/blob/master/src/tictactoe_reagent/core.cljs#L124){target=_blank} 108 | 109 | ## Viewing data structures 110 | 111 | Pretty print shows the structure of results from function calls in a human-friendly form, making it easier for a developer to parse and more likely to notice incorrect results. 112 | 113 | Tools to view and navigate code 114 | 115 | * [:fontawesome-solid-book-open: Cider inspector](https://practical.li/spacemacs/evaluating-clojure/inspect/){target=_blank} is an effective way to navigate nested data and page through large data sets. 116 | * [:fontawesome-solid-book-open: Portal Inspector](https://practical.li/clojure/clojure-tools/data-inspector/portal){target=_blank} to visualise many kinds of data in many different forms. 117 | 118 | ![Portal - view and navigate Clojure data and event logs](https://raw.githubusercontent.com/practicalli/graphic-design/live/portal/portal-data-browser-example.png) 119 | 120 | 121 | ## Code Style and idiomatic Clojure 122 | 123 | Clojure aware editors should automatically apply formatting that follows the [:globe_with_meridians: Clojure Style guide](https://github.com/bbatsov/clojure-style-guide){target=_blank}. 124 | 125 | Live linting with [clj-kondo](:fontawesome-brands-github: } suggests common idioms and highlights a wide range of syntax errors as code is written, minimizing bugs and therefore speeding up the development process. 126 | 127 | ![Clojure code static analysis for live linting](https://raw.githubusercontent.com/practicalli/graphic-design/live/spacemacs/screenshots/spacemacs-clojure-live-linting-flycheck-errors-light.png#only-light) 128 | ![Clojure code static analysis for live linting](https://raw.githubusercontent.com/practicalli/graphic-design/live/spacemacs/screenshots/spacemacs-clojure-live-linting-flycheck-errors-dark.png#only-dark) 129 | 130 | !!! INFO "Clojure LSP is build on top of clj-kondo" 131 | [:fontawesome-solid-book-open: Clojure LSP](https://practical.li/clojure/clojure-editors/clojure-lsp/){target=_blank} uses clj-kondo static analysis to provide a standard set of development tools (format, refactor, auto-complete, syntax highlighting, syntax & idiom warnings, code navigation, etc). 132 | 133 | Clojure LSP can be used with any Clojure aware editor that provides an LSP client, e.g. [:fontawesome-solid-book-open: Spacemacs](https://practical.li/spacemacs/install-spacemacs/clojure-lsp/){target=_blank}, [:fontawesome-solid-book-open: Doom Emacs](https://practical.li/doom-emacs/install/clojure-configuration/#clojure-cli){target=_blank}, [:fontawesome-solid-book-open: Neovim](https://practical.li/neovim/repl-driven-development/){target=_blank}, VSCode. 134 | 135 | !!! INFO "Clojure Style Guide" 136 | The [:globe_with_meridians: Clojure Style guide](https://github.com/bbatsov/clojure-style-guide){target=_blank} provides examples of common formatting approaches, although the development team should decide which of these to adopt. Emacs `clojure-mode` will automatically format code and so will Clojure LSP (via cljfmt). These tools are configurable and should be tailored to the teams standard. 137 | 138 | ## Data and Function specifications 139 | 140 | [:fontawesome-solid-book-open: Clojure spec](https://practical.li/clojure/clojure-spec/){target=_blank} is used to define a contract on incoming and outgoing data, to ensure it is of the correct form. 141 | 142 | As data structures are identified in REPL experiments, create data specification to validate the keys and value types of that data. 143 | 144 | ```clojure 145 | ;; --------------------------------------------------- 146 | ;; Address specifications 147 | (spec/def ::house-number string?) 148 | (spec/def ::street string?) 149 | (spec/def ::postal-code string?) 150 | (spec/def ::city string?) 151 | (spec/def ::country string?) 152 | (spec/def ::additional string?) 153 | 154 | (spec/def ::address ; Composite data specification 155 | (spec/keys 156 | :req-un [::street ::postal-code ::city ::country] 157 | :opt-un [::house-number ::additional])) 158 | ;; --------------------------------------------------- 159 | ``` 160 | 161 | As the public API is designed, specifications for each functions arguments are added to validate the correct data is used when calling those functions. 162 | 163 | [:fontawesome-solid-book-open: Generative testing](https://practical.li/clojure/clojure-spec/generative-testing/){target=_blank} provides a far greater scope of test values used incorporated into unit tests. Data uses clojure.spec to randomly generate data for testing on each test run. 164 | 165 | ## Test Driven Development and REPL Driven Development 166 | 167 | ![Clojure REPL driven development (RDD) and Test Driven Development (TDD)](https://raw.githubusercontent.com/practicalli/graphic-design/live/clojure/repl-tdd-flow.png){align=right loading=lazy} 168 | 169 | Test Driven Development (TDD) and REPL Driven Development (RDD) complement each other as they both encourage incremental changes and continuous feedback. 170 | 171 | > Test Driven Development fits well with Hammock Time, as good design comes from deep thought 172 | 173 | * RDD enables rapid design experiments so different approaches can easily and quickly be evaluated . 174 | * TDD focuses the results of the REPL experiments into design decisions, codified as unit tests. These tests guide the correctness of specific implementations and provide critical feedback when changes break that design. 175 | 176 | [:fontawesome-solid-book-open: Unit tests](https://practical.li/clojure/testing/unit-testing/){target=_blank} should support the public API of each namespace in a project to help prevent regressions in the code. Its far more efficient in terms of thinking time to define unit tests as the design starts to stabilize than as an after thought. 177 | 178 | `clojure.test` library is part of the Clojure standard library that provides a simple way to start writing unit tests. 179 | 180 | [:fontawesome-solid-book-open: Clojure spec](https://practical.li/clojure/clojure-spec/){target=_blank} can also be used for generative testing, providing far greater scope in values used when running unit tests. Specifications can be defined for values and functions. 181 | 182 | Clojure has a number of [:fontawesome-solid-book-open: test runners](https://practical.li/clojure/testing/test-runners/){target=_blank} available. Kaocha is a test runner that will run unit tests and function specification checks. 183 | 184 | !!! Hint "Automate local test runner" 185 | Use [:fontawesome-solid-book-open: kaocha test runner](https://practical.li/clojure/testing/test-runners/kaocha-test-runner/){target=_blank} in watch mode to run tests and specification check automatically (when changes are saved) 186 | ```bash 187 | clojure -X:test/watch 188 | ``` 189 | 190 | ## Continuous Integration and Deployment 191 | 192 | Add a [:fontawesome-solid-book-open: continuous integration service](https://practical.li/clojure/continuous-integration/){target=_blank} to run tests and builds code on every shared commit. Spin up testable review deployments when commits pushed to a pull request branch, before pushing commits to the main deployment branch, creating an effective pipeline to gain further feedback. 193 | 194 | * [:globe_with_meridians: CircleCI](https://practical.li/clojure/continuous-integration/circle-ci/){target=_blank} provides a simple to use service that supports Clojure projects. 195 | * [:globe_with_meridians: GitHub Workflows](https://docs.github.com/en/actions/using-workflows){target=_blank} and [GitHub actions marketplace](https://github.com/marketplace?type=actions){target=_blank} to quickly build a tailored continuous integration service, e.g. [Setup Clojure GitHub Action](https://github.com/marketplace/actions/setup-clojure){target=_blank}. 196 | * [:globe_with_meridians: GitLab CI](https://docs.gitlab.com/ee/ci/introduction/index.html){target=_blank} 197 | 198 | ![Continuous Integration](https://raw.githubusercontent.com/practicalli/graphic-design/live/continuous-integration/continuous-integration-overview.svg) 199 | 200 | ## Live Coding with Data - Stuart Halloway 201 | 202 | There are few novel features of programming languages, but each combination has different properties. The combination of dynamic, hosted, functional and extended Lisp in Clojure gives developers the tools for making effective programs. The ways in which Clojure's unique combination of features can yield a highly effective development process. 203 | 204 | Over more than a decade we have developed an effective approach to writing code in Clojure whose power comes from composing many of its key features. As different as Clojure programs are from e.g. Java programs, so to can and should be the development experience. You are not in Kansas anymore! 205 | 206 | This talk presents a demonstration of the leverage you can get when writing programs in Clojure, with examples, based on my experiences as a core developer of Clojure and Datomic. 207 | 208 |

209 | 210 |

211 | -------------------------------------------------------------------------------- /docs/introduction/writing-tips.md: -------------------------------------------------------------------------------- 1 | # Writing tips for MkDocs 2 | 3 | Making the docs more engaging using the [mkdocs-material theme reference guide](https://squidfunk.github.io/mkdocs-material/reference/){target=_blank} 4 | 5 | ??? INFO "Configuring Colors" 6 | [Material for MkDocs - Changing the colors](https://squidfunk.github.io/mkdocs-material/setup/changing-the-colors/){target=_blank} lists the primary and accent colors available. 7 | 8 | [HSL Color Picker](https://hslpicker.com/) for codes to modify the theme style, overriding colors in `docs/assets/stylesheets/extra.css` 9 | 10 | 11 | ## Hypertext links 12 | 13 | Links open in the same browser window/tab by default. 14 | 15 | Add `{target=_blank}` to the end of a link to configure opening in a new tab 16 | 17 | ```markdown 18 | [link text](url){target=_blank} 19 | ``` 20 | 21 | ## Buttons 22 | 23 | Convert any link into a button by adding `{.md-button}` class names to end of the markdown for a link, which uses `.md-button-primary` by default. Include `target=_blank` for buttons with links to external sites. 24 | 25 | ``` 26 | [link text](http://practical.li/blog){.md-button target=_blank} 27 | ``` 28 | 29 | Or specify a different class 30 | 31 | ``` 32 | [link text](http://practical.li/blog){.md-button .md-button-primary} 33 | ``` 34 | 35 | Add an icon to the button 36 | 37 | [:fontawesome-brands-github: Practicalli Issues](http://practical.li/blog){ .md-button .md-button-primary } 38 | [:octicons-heart-fill-24: Practicalli Blog](http://practical.li/blog){ .md-button .md-button-primary } 39 | 40 | ```markdown 41 | [:fontawesome-brands-github: Practicalli Issues](http://practical.li/blog){ .md-button .md-button-primary } 42 | [:octicons-heart-fill-24: Practicalli Blog](http://practical.li/blog){ .md-button .md-button-primary } 43 | ``` 44 | 45 | [Search all supported icons](https://squidfunk.github.io/mkdocs-material/reference/icons-emojis/){target=_blank .md-button} 46 | 47 | ## YouTube video 48 | 49 | Use an iframe element to include a YouTube video, wrapping in a paragraph tag with center alignment to place the video in a centered horizontal position 50 | 51 | ```html 52 |

53 | 54 |

55 | ``` 56 | 57 | > mkdocs material does not have direct support for adding a YouTube video via markdown. 58 | 59 | 60 | ## Admonitions 61 | 62 | [Supported admonition types](https://squidfunk.github.io/mkdocs-material/reference/admonitions/#supported-types) 63 | 64 | !!! NOTE 65 | Use `!!!` followed by `NOTE` 66 | 67 | !!! NOTE "Adding a title" 68 | Use `!!!` followed by `NOTE` and a `"title in double quotes"` 69 | 70 | !!! NOTE "" 71 | Shh, no title bar just the text... 72 | Use `!!!` followed by `NOTE` and a `""` empty double quotes 73 | 74 | !!! ABSTRACT 75 | Use `!!!` followed by `ABSTRACT` 76 | 77 | !!! INFO 78 | Use `!!!` followed by `INFO` 79 | 80 | !!! TIP 81 | Use `!!!` followed by `TIP` 82 | 83 | !!! SUCCESS 84 | Use `!!!` followed by `SUCCESS` 85 | 86 | !!! QUESTION 87 | Use `!!!` followed by `QUESTION` 88 | 89 | !!! WARNING 90 | Use `!!!` followed by `WARNING` 91 | 92 | !!! FAILURE 93 | Use `!!!` followed by `FAILURE` 94 | 95 | !!! DANGER 96 | Use `!!!` followed by `DANGER` 97 | 98 | !!! BUG 99 | Use `!!!` followed by `BUG` 100 | 101 | !!! EXAMPLE 102 | Use `!!!` followed by `EXAMPLE` 103 | 104 | !!! QUOTE 105 | Use `!!!` followed by `QUOTE` 106 | 107 | 108 | 109 | ### Collapsing admonitions 110 | 111 | ??? NOTE 112 | Collapse those admonitions using `???` instead of `!!!` 113 | 114 | ??? NOTE "Replace with a title" 115 | Use `???` followed by `NOTE` and a `"title in double quotes"` 116 | 117 | 118 | ???+ NOTE "Expanded by default" 119 | Use `???+`, note the `+` character, followed by `NOTE` and a `"title in double quotes"` 120 | 121 | 122 | ### Inline blocks 123 | 124 | Inline blocks of text to make a very specific callout within text 125 | 126 | !!! info inline 127 | 128 | Lorem ipsum dolor sit amet, consectetur 129 | adipiscing elit. Nulla et euismod nulla. 130 | Curabitur feugiat, tortor non consequat 131 | finibus, justo purus auctor massa, nec 132 | semper lorem quam in massa. 133 | 134 | 135 | Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla et euismod nulla. Curabitur feugiat, tortor non consequat finibus, justo purus auctor massa, nec semper lorem quam in massa. 136 | 137 | Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla et euismod nulla. Curabitur feugiat, tortor non consequat finibus, justo purus auctor massa, nec semper lorem quam in massa. 138 | 139 | Adding something to then end of text is probably my favourite 140 | 141 | !!! info inline end 142 | 143 | Lorem ipsum dolor sit amet, consectetur 144 | adipiscing elit. Nulla et euismod nulla. 145 | Curabitur feugiat, tortor non consequat 146 | finibus, justo purus auctor massa, nec 147 | semper lorem quam in massa. 148 | 149 | 150 | Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla et euismod nulla. Curabitur feugiat, tortor non consequat finibus, justo purus auctor massa, nec semper lorem quam in massa. 151 | 152 | Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla et euismod nulla. Curabitur feugiat, tortor non consequat finibus, justo purus auctor massa, nec semper lorem quam in massa. 153 | 154 | 155 | 156 | ## Code blocks 157 | 158 | Code blocks include a copy icon automatically 159 | 160 | Syntax highlighting in code blocks 161 | 162 | ```clojure 163 | (defn my-function ; Write a simple function 164 | "With a lovely doc-string" 165 | [arguments] 166 | (map inc [1 2 3])) 167 | ``` 168 | 169 | Give the code block a title using `title=""` after the backtics and language name 170 | 171 | 172 | ```clojure title="src/practicalli/gameboard.clj" 173 | (defn my-function 174 | "With a lovely doc-string" 175 | [arguments] 176 | (map inc [1 2 3])) 177 | ``` 178 | 179 | We all like line numbers, especially when you can set the starting line 180 | 181 | ```clojure linenums="42" title="src/practicalli/gameboard.clj" 182 | (defn my-function 183 | "With a lovely doc-string" 184 | [arguments] 185 | (map inc [1 2 3])) 186 | ``` 187 | 188 | Add `linenums=42` to start line numbers from 42 onward 189 | 190 | ``` 191 | clojure linenums="42" title="src/practicalli/gameboard.clj" 192 | ``` 193 | 194 | 195 | ### Annotations 196 | 197 | Annotations in a code block help to highlight important aspects. Use the comment character for the language followed by a space and a number in brackets 198 | 199 | For example, in a shell code block, use `# (1)` where 1 is the number of the annotation 200 | 201 | Use a number after the code block to add the text for the annotation, e.g. `1.`. Ensure there is a space between the code block and the annotation text. 202 | 203 | ```shell 204 | ls -la $HOME/Downloads # (1) 205 | ``` 206 | 207 | 1. :woman_raising_hand: I'm a code annotation! I can contain `code`, __formatted text__, images, ... basically anything that can be written in Markdown. 208 | 209 | 210 | Code blocks with annotation, add `!` after the annotation number to suppress the `#` character 211 | 212 | ```clojure 213 | (defn helper-function 214 | "Doc-string with description of function purpose" ; (1)! 215 | [data] 216 | (merge {:fish 1} data) 217 | ) 218 | ``` 219 | 220 | 1. Always include a doc-string in every function to describe the purpose of that function, identifying why it was added and what its value is. 221 | 222 | 223 | GitHub action example with multiple annotations 224 | 225 | ``` yaml 226 | name: ci # (1)! 227 | on: 228 | push: 229 | branches: 230 | - master # (2)! 231 | - main 232 | permissions: 233 | contents: write 234 | jobs: 235 | deploy: 236 | runs-on: ubuntu-latest 237 | steps: 238 | - uses: actions/checkout@v3 239 | - uses: actions/setup-python@v4 240 | with: 241 | python-version: 3.x 242 | - run: pip install mkdocs-material # (3)! 243 | - run: mkdocs gh-deploy --force 244 | ``` 245 | 246 | 1. You can change the name to your liking. 247 | 248 | 2. At some point, GitHub renamed `master` to `main`. If your default branch 249 | is named `master`, you can safely remove `main`, vice versa. 250 | 251 | 3. This is the place to install further [MkDocs plugins] or Markdown 252 | extensions with `pip` to be used during the build: 253 | 254 | ``` sh 255 | pip install \ 256 | mkdocs-material \ 257 | mkdocs-awesome-pages-plugin \ 258 | ... 259 | ``` 260 | 261 | 262 | ### Highlight lines in code blocks 263 | 264 | Add highlight line meta data to a code block after the opening backticks and code block language. 265 | 266 | `hl_lines="2"` highlights line 2 in the codeblock 267 | 268 | ```clojure hl_lines="4 5 6" 269 | (defn my-function 270 | "With a lovely doc-string" 271 | [arguments] 272 | (map 273 | inc 274 | [1 2 3])) 275 | ``` 276 | 277 | ### Embed external files 278 | 279 | `--8<--` in a code block inserts code from a source code file or other text file 280 | 281 | Specify a local file from the root of the book project (the directory containing mkdocs.yml) 282 | 283 | ??? EXAMPLE "Scheduled Version Check GitHub Workflow from source code file" 284 | ```yaml title="scheduled version check" 285 | --8<-- ".github/workflows/scheduled-version-check.yaml" 286 | ``` 287 | 288 | ??? EXAMPLE "Practicalli Project Templates" 289 | ```markdown title="Emacs project configuration - .dir-locals.el" 290 | --8<-- "https://raw.githubusercontent.com/practicalli/project-templates/main/.dir-locals.el" 291 | ``` 292 | 293 | !!! HINT "Code example reuse" 294 | Use an embedded local or external file (URL) when the same content is required in more than one place in the book. 295 | 296 | An effective way of sharing code and configuration mutliple times in a book or across multiple books. 297 | 298 | ## Content tabs 299 | 300 | Create in page tabs that can also be 301 | 302 | Setting up a project 303 | 304 | === "Clojure CLI" 305 | ```shell 306 | clojure -T:project/new :template app :name practicalli/gameboard 307 | ``` 308 | 309 | 310 | === "Leiningen" 311 | ```shell 312 | lein new app practicalli/gameboard 313 | ``` 314 | 315 | 316 | Or nest the content tabs in an admonition 317 | 318 | 319 | !!! INFO "Run a terminal REPL" 320 | 321 | === "Clojure CLI" 322 | ```shell 323 | clojure -T:repl/rebel 324 | ``` 325 | 326 | 327 | === "Leiningen" 328 | ```shell 329 | lein repl 330 | ``` 331 | 332 | 333 | ## Diagrams 334 | 335 | Neat flow diagrams 336 | 337 | [Diagrams - Material for MkDocs](https://squidfunk.github.io/mkdocs-material/reference/diagrams/){target=_blank .md-button} 338 | 339 | ``` mermaid 340 | graph LR 341 | A[Start] --> B{Error?}; 342 | B -->|Yes| C[Hmm...]; 343 | C --> D[Debug]; 344 | D --> B; 345 | B ---->|No| E[Yay!]; 346 | ``` 347 | 348 | 349 | UML Sequence Diagrams 350 | 351 | ``` mermaid 352 | sequenceDiagram 353 | Alice->>John: Hello John, how are you? 354 | loop Healthcheck 355 | John->>John: Fight against hypochondria 356 | end 357 | Note right of John: Rational thoughts! 358 | John-->>Alice: Great! 359 | John->>Bob: How about you? 360 | Bob-->>John: Jolly good! 361 | ``` 362 | 363 | state transition diagrams 364 | 365 | ``` mermaid 366 | stateDiagram-v2 367 | state fork_state <> 368 | [*] --> fork_state 369 | fork_state --> State2 370 | fork_state --> State3 371 | 372 | state join_state <> 373 | State2 --> join_state 374 | State3 --> join_state 375 | join_state --> State4 376 | State4 --> [*] 377 | ``` 378 | 379 | Class diagrams - not needed for Clojure 380 | 381 | Entity relationship diagrams are handy though 382 | 383 | ``` mermaid 384 | erDiagram 385 | CUSTOMER ||--o{ ORDER : places 386 | ORDER ||--|{ LINE-ITEM : contains 387 | LINE-ITEM { 388 | customer-name string 389 | unit-price int 390 | } 391 | CUSTOMER }|..|{ DELIVERY-ADDRESS : uses 392 | ``` 393 | 394 | 395 | ## Keyboard keys 396 | 397 | Represent key bindings with [Keyboard keys](https://facelessuser.github.io/pymdown-extensions/extensions/keys/#extendingmodifying-key-map-index){target=_blank}. Each number and alphabet character has their own key. 398 | 399 | * ++1++ `++1++` for numbers 400 | * ++"l"++ `++"l"++` for lowercase character 401 | * ++u++ `++u++` for uppercase character or `++"U"++` for consistency 402 | 403 | [Punctionation keys](https://facelessuser.github.io/pymdown-extensions/extensions/keys/#punctuation-keys){target=_blank} use their name 404 | 405 | * ++spc++ `++spc++` 406 | * ++comma++ `++comma++` 407 | * ++arrow-left++ `++arrow-left++` 408 | 409 | For key sequences, place a space between each keyboard character 410 | 411 | * ++spc++ ++"g"++ ++"s"++ `++spc++ ++"g"++ ++"s"++` 412 | 413 | For key combinations, use join they key identifies with a `+` 414 | 415 | * ++meta+x++ `++meta+x++` 416 | * ++ctrl+alt+del++ `++ctrl+alt+del++` 417 | 418 | [MkDocs keyboard keys reference](https://facelessuser.github.io/pymdown-extensions/extensions/keys/#extendingmodifying-key-map-index){target=_blank .md-button} 419 | 420 | ## Images 421 | 422 | Markdown images can be appended with material tags to set the size of the image, whether to appear on light or dark theme and support lazy image loading in browsers 423 | 424 | === "Size" 425 | `{style="height:150px;width:150px"}` specifies the image size 426 | ```markdown 427 | ![Kitty Logo](https://raw.githubusercontent.com/practicalli/graphic-design/live/icons/kitty-light.png#only-dark){style="height:150px;width:150px"} 428 | ``` 429 | 430 | ![Kitty Logo](https://raw.githubusercontent.com/practicalli/graphic-design/live/icons/kitty-light.png#only-dark){style="height:150px;width:150px"} 431 | 432 | 433 | === "Lazy Loading" 434 | 435 | `{loading=lazy}` specifies an image should lazily load in the browser 436 | ```markdown 437 | ![Kitty Logo](https://raw.githubusercontent.com/practicalli/graphic-design/live/icons/kitty-light.png){loading=lazy} 438 | ``` 439 | 440 | === "Align" 441 | 442 | `{aligh=left}` or `{aligh=right}` specifies the page alignment of an image. 443 | ```markdown 444 | ![Kitty Logo](https://raw.githubusercontent.com/practicalli/graphic-design/live/icons/kitty-light.png#only-dark){align=right} 445 | ![Kitty Logo](https://raw.githubusercontent.com/practicalli/graphic-design/live/icons/kitty-dark.png#only-light){align=right} 446 | ``` 447 | 448 | ![Kitty Logo](https://raw.githubusercontent.com/practicalli/graphic-design/live/icons/kitty-light.png){align=left style="height:64px;width:64px"} 449 | ![Kitty Logo](https://raw.githubusercontent.com/practicalli/graphic-design/live/icons/kitty-dark.png){align=right style="height:64px;width:64px"} 450 | Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla et euismod nulla. Curabitur feugiat, tortor non consequat finibus, justo purus auctor massa, nec semper lorem quam in massa. 451 | 452 | 453 | === "Theme Specific" 454 | 455 | `![Kitty Logo](image/kitty-light.png#only-dark)` or `![Kitty Logo](image/kitty-light.png#only-light)` specifies the theme the image should be shown, allowing different versions of images to be shown based on the theme. 456 | ```markdown 457 | ![Kitty Logo](https://raw.githubusercontent.com/practicalli/graphic-design/live/icons/kitty-light.png#only-dark){style="height:150px;width:150px"} 458 | ![Kitty Logo](https://raw.githubusercontent.com/practicalli/graphic-design/live/icons/kitty-dark.png#only-light){style="height:150px;width:150px"} 459 | ``` 460 | Use the theme toggle in the top nav bar to see the icon change between light and dark. 461 | ![Kitty Logo](https://raw.githubusercontent.com/practicalli/graphic-design/live/icons/kitty-light.png#only-dark){style="height:150px;width:150px"} 462 | ![Kitty Logo](https://raw.githubusercontent.com/practicalli/graphic-design/live/icons/kitty-dark.png#only-light){style="height:150px;width:150px"} 463 | 464 | > Requires the [color pallet toggle](https://squidfunk.github.io/mkdocs-material/setup/changing-the-colors/#color-palette-toggle) 465 | 466 | 467 | === "All Image Attributes" 468 | Alight right, lazy load and set image to 150x150 469 | 470 | ```markdown 471 | ![Kitty Logo](https://raw.githubusercontent.com/practicalli/graphic-design/live/icons/kitty-light.png#only-dark){align=right loading=lazy style="height:64px;width:64px"} 472 | ![Kitty Logo](https://raw.githubusercontent.com/practicalli/graphic-design/live/icons/kitty-dark.png#only-light){align=right loading=lazy style="height:64px;width:64px"} 473 | ``` 474 | 475 | ![Kitty Logo](https://raw.githubusercontent.com/practicalli/graphic-design/live/icons/kitty-light.png#only-dark){align=right loading=lazy style="height:64px;width:64px"} 476 | ![Kitty Logo](https://raw.githubusercontent.com/practicalli/graphic-design/live/icons/kitty-dark.png#only-light){align=left loading=lazy style="height:64px;width:64px"} 477 | Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla et euismod nulla. Curabitur feugiat, tortor non consequat finibus, justo purus auctor massa, nec semper lorem quam in massa. 478 | 479 | 480 | ## Lists 481 | 482 | Task lists 483 | 484 | - [x] Lorem ipsum dolor sit amet, consectetur adipiscing elit 485 | - [ ] Vestibulum convallis sit amet nisi a tincidunt 486 | * [x] In hac habitasse platea dictumst 487 | * [x] In scelerisque nibh non dolor mollis congue sed et metus 488 | * [ ] Praesent sed risus massa 489 | - [ ] Aenean pretium efficitur erat, donec pharetra, ligula non scelerisque 490 | 491 | !!! EXAMPLE "Task List example" 492 | ```markdown 493 | - [x] Lorem ipsum dolor sit amet, consectetur adipiscing elit 494 | - [ ] Vestibulum convallis sit amet nisi a tincidunt 495 | * [x] In hac habitasse platea dictumst 496 | * [x] In scelerisque nibh non dolor mollis congue sed et metus 497 | * [ ] Praesent sed risus massa 498 | - [ ] Aenean pretium efficitur erat, donec pharetra, ligula non scelerisque 499 | ``` 500 | 501 | ## Tooltips 502 | 503 | The humble tool tip 504 | 505 | [Hover me](https://example.com "I'm a tooltip!") 506 | 507 | with references 508 | 509 | [Hover me][example] 510 | 511 | [example]: https://example.com "I'm a tooltip!" 512 | 513 | 514 | Icon tool tip with a title 515 | 516 | :material-information-outline:{ title="Important information" } 517 | 518 | 519 | ### Abreviations 520 | 521 | The HTML specification is maintained by the W3C. 522 | 523 | *[HTML]: Hyper Text Markup Language 524 | *[W3C]: World Wide Web Consortium 525 | 526 | 527 | 528 | ## Magic links 529 | 530 | [MagicLink](https://facelessuser.github.io/pymdown-extensions/extensions/magiclink/) can auto-link HTML, FTP, and email links. It can auto-convert repository links (GitHub, GitLab, and Bitbucket) and display them in a more concise, shorthand format. 531 | 532 | [Email Practicalli](mailto:info@) 533 | 534 | [Practicalli Neovim](https://github.com/practicalli/neovim) 535 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Attribution-ShareAlike 4.0 International 2 | 3 | ======================================================================= 4 | 5 | Creative Commons Corporation ("Creative Commons") is not a law firm and 6 | does not provide legal services or legal advice. Distribution of 7 | Creative Commons public licenses does not create a lawyer-client or 8 | other relationship. Creative Commons makes its licenses and related 9 | information available on an "as-is" basis. Creative Commons gives no 10 | warranties regarding its licenses, any material licensed under their 11 | terms and conditions, or any related information. Creative Commons 12 | disclaims all liability for damages resulting from their use to the 13 | fullest extent possible. 14 | 15 | Using Creative Commons Public Licenses 16 | 17 | Creative Commons public licenses provide a standard set of terms and 18 | conditions that creators and other rights holders may use to share 19 | original works of authorship and other material subject to copyright 20 | and certain other rights specified in the public license below. The 21 | following considerations are for informational purposes only, are not 22 | exhaustive, and do not form part of our licenses. 23 | 24 | Considerations for licensors: Our public licenses are 25 | intended for use by those authorized to give the public 26 | permission to use material in ways otherwise restricted by 27 | copyright and certain other rights. Our licenses are 28 | irrevocable. Licensors should read and understand the terms 29 | and conditions of the license they choose before applying it. 30 | Licensors should also secure all rights necessary before 31 | applying our licenses so that the public can reuse the 32 | material as expected. Licensors should clearly mark any 33 | material not subject to the license. This includes other CC- 34 | licensed material, or material used under an exception or 35 | limitation to copyright. More considerations for licensors: 36 | wiki.creativecommons.org/Considerations_for_licensors 37 | 38 | Considerations for the public: By using one of our public 39 | licenses, a licensor grants the public permission to use the 40 | licensed material under specified terms and conditions. If 41 | the licensor's permission is not necessary for any reason--for 42 | example, because of any applicable exception or limitation to 43 | copyright--then that use is not regulated by the license. Our 44 | licenses grant only permissions under copyright and certain 45 | other rights that a licensor has authority to grant. Use of 46 | the licensed material may still be restricted for other 47 | reasons, including because others have copyright or other 48 | rights in the material. A licensor may make special requests, 49 | such as asking that all changes be marked or described. 50 | Although not required by our licenses, you are encouraged to 51 | respect those requests where reasonable. More_considerations 52 | for the public: 53 | wiki.creativecommons.org/Considerations_for_licensees 54 | 55 | ======================================================================= 56 | 57 | Creative Commons Attribution-ShareAlike 4.0 International Public 58 | License 59 | 60 | By exercising the Licensed Rights (defined below), You accept and agree 61 | to be bound by the terms and conditions of this Creative Commons 62 | Attribution-ShareAlike 4.0 International Public License ("Public 63 | License"). To the extent this Public License may be interpreted as a 64 | contract, You are granted the Licensed Rights in consideration of Your 65 | acceptance of these terms and conditions, and the Licensor grants You 66 | such rights in consideration of benefits the Licensor receives from 67 | making the Licensed Material available under these terms and 68 | conditions. 69 | 70 | 71 | Section 1 -- Definitions. 72 | 73 | a. Adapted Material means material subject to Copyright and Similar 74 | Rights that is derived from or based upon the Licensed Material 75 | and in which the Licensed Material is translated, altered, 76 | arranged, transformed, or otherwise modified in a manner requiring 77 | permission under the Copyright and Similar Rights held by the 78 | Licensor. For purposes of this Public License, where the Licensed 79 | Material is a musical work, performance, or sound recording, 80 | Adapted Material is always produced where the Licensed Material is 81 | synched in timed relation with a moving image. 82 | 83 | b. Adapter's License means the license You apply to Your Copyright 84 | and Similar Rights in Your contributions to Adapted Material in 85 | accordance with the terms and conditions of this Public License. 86 | 87 | c. BY-SA Compatible License means a license listed at 88 | creativecommons.org/compatiblelicenses, approved by Creative 89 | Commons as essentially the equivalent of this Public License. 90 | 91 | d. Copyright and Similar Rights means copyright and/or similar rights 92 | closely related to copyright including, without limitation, 93 | performance, broadcast, sound recording, and Sui Generis Database 94 | Rights, without regard to how the rights are labeled or 95 | categorized. For purposes of this Public License, the rights 96 | specified in Section 2(b)(1)-(2) are not Copyright and Similar 97 | Rights. 98 | 99 | e. Effective Technological Measures means those measures that, in the 100 | absence of proper authority, may not be circumvented under laws 101 | fulfilling obligations under Article 11 of the WIPO Copyright 102 | Treaty adopted on December 20, 1996, and/or similar international 103 | agreements. 104 | 105 | f. Exceptions and Limitations means fair use, fair dealing, and/or 106 | any other exception or limitation to Copyright and Similar Rights 107 | that applies to Your use of the Licensed Material. 108 | 109 | g. License Elements means the license attributes listed in the name 110 | of a Creative Commons Public License. The License Elements of this 111 | Public License are Attribution and ShareAlike. 112 | 113 | h. Licensed Material means the artistic or literary work, database, 114 | or other material to which the Licensor applied this Public 115 | License. 116 | 117 | i. Licensed Rights means the rights granted to You subject to the 118 | terms and conditions of this Public License, which are limited to 119 | all Copyright and Similar Rights that apply to Your use of the 120 | Licensed Material and that the Licensor has authority to license. 121 | 122 | j. Licensor means the individual(s) or entity(ies) granting rights 123 | under this Public License. 124 | 125 | k. Share means to provide material to the public by any means or 126 | process that requires permission under the Licensed Rights, such 127 | as reproduction, public display, public performance, distribution, 128 | dissemination, communication, or importation, and to make material 129 | available to the public including in ways that members of the 130 | public may access the material from a place and at a time 131 | individually chosen by them. 132 | 133 | l. Sui Generis Database Rights means rights other than copyright 134 | resulting from Directive 96/9/EC of the European Parliament and of 135 | the Council of 11 March 1996 on the legal protection of databases, 136 | as amended and/or succeeded, as well as other essentially 137 | equivalent rights anywhere in the world. 138 | 139 | m. You means the individual or entity exercising the Licensed Rights 140 | under this Public License. Your has a corresponding meaning. 141 | 142 | 143 | Section 2 -- Scope. 144 | 145 | a. License grant. 146 | 147 | 1. Subject to the terms and conditions of this Public License, 148 | the Licensor hereby grants You a worldwide, royalty-free, 149 | non-sublicensable, non-exclusive, irrevocable license to 150 | exercise the Licensed Rights in the Licensed Material to: 151 | 152 | a. reproduce and Share the Licensed Material, in whole or 153 | in part; and 154 | 155 | b. produce, reproduce, and Share Adapted Material. 156 | 157 | 2. Exceptions and Limitations. For the avoidance of doubt, where 158 | Exceptions and Limitations apply to Your use, this Public 159 | License does not apply, and You do not need to comply with 160 | its terms and conditions. 161 | 162 | 3. Term. The term of this Public License is specified in Section 163 | 6(a). 164 | 165 | 4. Media and formats; technical modifications allowed. The 166 | Licensor authorizes You to exercise the Licensed Rights in 167 | all media and formats whether now known or hereafter created, 168 | and to make technical modifications necessary to do so. The 169 | Licensor waives and/or agrees not to assert any right or 170 | authority to forbid You from making technical modifications 171 | necessary to exercise the Licensed Rights, including 172 | technical modifications necessary to circumvent Effective 173 | Technological Measures. For purposes of this Public License, 174 | simply making modifications authorized by this Section 2(a) 175 | (4) never produces Adapted Material. 176 | 177 | 5. Downstream recipients. 178 | 179 | a. Offer from the Licensor -- Licensed Material. Every 180 | recipient of the Licensed Material automatically 181 | receives an offer from the Licensor to exercise the 182 | Licensed Rights under the terms and conditions of this 183 | Public License. 184 | 185 | b. Additional offer from the Licensor -- Adapted Material. 186 | Every recipient of Adapted Material from You 187 | automatically receives an offer from the Licensor to 188 | exercise the Licensed Rights in the Adapted Material 189 | under the conditions of the Adapter's License You apply. 190 | 191 | c. No downstream restrictions. You may not offer or impose 192 | any additional or different terms or conditions on, or 193 | apply any Effective Technological Measures to, the 194 | Licensed Material if doing so restricts exercise of the 195 | Licensed Rights by any recipient of the Licensed 196 | Material. 197 | 198 | 6. No endorsement. Nothing in this Public License constitutes or 199 | may be construed as permission to assert or imply that You 200 | are, or that Your use of the Licensed Material is, connected 201 | with, or sponsored, endorsed, or granted official status by, 202 | the Licensor or others designated to receive attribution as 203 | provided in Section 3(a)(1)(A)(i). 204 | 205 | b. Other rights. 206 | 207 | 1. Moral rights, such as the right of integrity, are not 208 | licensed under this Public License, nor are publicity, 209 | privacy, and/or other similar personality rights; however, to 210 | the extent possible, the Licensor waives and/or agrees not to 211 | assert any such rights held by the Licensor to the limited 212 | extent necessary to allow You to exercise the Licensed 213 | Rights, but not otherwise. 214 | 215 | 2. Patent and trademark rights are not licensed under this 216 | Public License. 217 | 218 | 3. To the extent possible, the Licensor waives any right to 219 | collect royalties from You for the exercise of the Licensed 220 | Rights, whether directly or through a collecting society 221 | under any voluntary or waivable statutory or compulsory 222 | licensing scheme. In all other cases the Licensor expressly 223 | reserves any right to collect such royalties. 224 | 225 | 226 | Section 3 -- License Conditions. 227 | 228 | Your exercise of the Licensed Rights is expressly made subject to the 229 | following conditions. 230 | 231 | a. Attribution. 232 | 233 | 1. If You Share the Licensed Material (including in modified 234 | form), You must: 235 | 236 | a. retain the following if it is supplied by the Licensor 237 | with the Licensed Material: 238 | 239 | i. identification of the creator(s) of the Licensed 240 | Material and any others designated to receive 241 | attribution, in any reasonable manner requested by 242 | the Licensor (including by pseudonym if 243 | designated); 244 | 245 | ii. a copyright notice; 246 | 247 | iii. a notice that refers to this Public License; 248 | 249 | iv. a notice that refers to the disclaimer of 250 | warranties; 251 | 252 | v. a URI or hyperlink to the Licensed Material to the 253 | extent reasonably practicable; 254 | 255 | b. indicate if You modified the Licensed Material and 256 | retain an indication of any previous modifications; and 257 | 258 | c. indicate the Licensed Material is licensed under this 259 | Public License, and include the text of, or the URI or 260 | hyperlink to, this Public License. 261 | 262 | 2. You may satisfy the conditions in Section 3(a)(1) in any 263 | reasonable manner based on the medium, means, and context in 264 | which You Share the Licensed Material. For example, it may be 265 | reasonable to satisfy the conditions by providing a URI or 266 | hyperlink to a resource that includes the required 267 | information. 268 | 269 | 3. If requested by the Licensor, You must remove any of the 270 | information required by Section 3(a)(1)(A) to the extent 271 | reasonably practicable. 272 | 273 | b. ShareAlike. 274 | 275 | In addition to the conditions in Section 3(a), if You Share 276 | Adapted Material You produce, the following conditions also apply. 277 | 278 | 1. The Adapter's License You apply must be a Creative Commons 279 | license with the same License Elements, this version or 280 | later, or a BY-SA Compatible License. 281 | 282 | 2. You must include the text of, or the URI or hyperlink to, the 283 | Adapter's License You apply. You may satisfy this condition 284 | in any reasonable manner based on the medium, means, and 285 | context in which You Share Adapted Material. 286 | 287 | 3. You may not offer or impose any additional or different terms 288 | or conditions on, or apply any Effective Technological 289 | Measures to, Adapted Material that restrict exercise of the 290 | rights granted under the Adapter's License You apply. 291 | 292 | 293 | Section 4 -- Sui Generis Database Rights. 294 | 295 | Where the Licensed Rights include Sui Generis Database Rights that 296 | apply to Your use of the Licensed Material: 297 | 298 | a. for the avoidance of doubt, Section 2(a)(1) grants You the right 299 | to extract, reuse, reproduce, and Share all or a substantial 300 | portion of the contents of the database; 301 | 302 | b. if You include all or a substantial portion of the database 303 | contents in a database in which You have Sui Generis Database 304 | Rights, then the database in which You have Sui Generis Database 305 | Rights (but not its individual contents) is Adapted Material, 306 | 307 | including for purposes of Section 3(b); and 308 | c. You must comply with the conditions in Section 3(a) if You Share 309 | all or a substantial portion of the contents of the database. 310 | 311 | For the avoidance of doubt, this Section 4 supplements and does not 312 | replace Your obligations under this Public License where the Licensed 313 | Rights include other Copyright and Similar Rights. 314 | 315 | 316 | Section 5 -- Disclaimer of Warranties and Limitation of Liability. 317 | 318 | a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE 319 | EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS 320 | AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF 321 | ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS, 322 | IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION, 323 | WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR 324 | PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS, 325 | ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT 326 | KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT 327 | ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU. 328 | 329 | b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE 330 | TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION, 331 | NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT, 332 | INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES, 333 | COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR 334 | USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN 335 | ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR 336 | DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR 337 | IN PART, THIS LIMITATION MAY NOT APPLY TO YOU. 338 | 339 | c. The disclaimer of warranties and limitation of liability provided 340 | above shall be interpreted in a manner that, to the extent 341 | possible, most closely approximates an absolute disclaimer and 342 | waiver of all liability. 343 | 344 | 345 | Section 6 -- Term and Termination. 346 | 347 | a. This Public License applies for the term of the Copyright and 348 | Similar Rights licensed here. However, if You fail to comply with 349 | this Public License, then Your rights under this Public License 350 | terminate automatically. 351 | 352 | b. Where Your right to use the Licensed Material has terminated under 353 | Section 6(a), it reinstates: 354 | 355 | 1. automatically as of the date the violation is cured, provided 356 | it is cured within 30 days of Your discovery of the 357 | violation; or 358 | 359 | 2. upon express reinstatement by the Licensor. 360 | 361 | For the avoidance of doubt, this Section 6(b) does not affect any 362 | right the Licensor may have to seek remedies for Your violations 363 | of this Public License. 364 | 365 | c. For the avoidance of doubt, the Licensor may also offer the 366 | Licensed Material under separate terms or conditions or stop 367 | distributing the Licensed Material at any time; however, doing so 368 | will not terminate this Public License. 369 | 370 | d. Sections 1, 5, 6, 7, and 8 survive termination of this Public 371 | License. 372 | 373 | 374 | Section 7 -- Other Terms and Conditions. 375 | 376 | a. The Licensor shall not be bound by any additional or different 377 | terms or conditions communicated by You unless expressly agreed. 378 | 379 | b. Any arrangements, understandings, or agreements regarding the 380 | Licensed Material not stated herein are separate from and 381 | independent of the terms and conditions of this Public License. 382 | 383 | 384 | Section 8 -- Interpretation. 385 | 386 | a. For the avoidance of doubt, this Public License does not, and 387 | shall not be interpreted to, reduce, limit, restrict, or impose 388 | conditions on any use of the Licensed Material that could lawfully 389 | be made without permission under this Public License. 390 | 391 | b. To the extent possible, if any provision of this Public License is 392 | deemed unenforceable, it shall be automatically reformed to the 393 | minimum extent necessary to make it enforceable. If the provision 394 | cannot be reformed, it shall be severed from this Public License 395 | without affecting the enforceability of the remaining terms and 396 | conditions. 397 | 398 | c. No term or condition of this Public License will be waived and no 399 | failure to comply consented to unless expressly agreed to by the 400 | Licensor. 401 | 402 | d. Nothing in this Public License constitutes or may be interpreted 403 | as a limitation upon, or waiver of, any privileges and immunities 404 | that apply to the Licensor or You, including from the legal 405 | processes of any jurisdiction or authority. 406 | 407 | 408 | ======================================================================= 409 | 410 | Creative Commons is not a party to its public 411 | licenses. Notwithstanding, Creative Commons may elect to apply one of 412 | its public licenses to material it publishes and in those instances 413 | will be considered the “Licensor.” The text of the Creative Commons 414 | public licenses is dedicated to the public domain under the CC0 Public 415 | Domain Dedication. Except for the limited purpose of indicating that 416 | material is shared under a Creative Commons public license or as 417 | otherwise permitted by the Creative Commons policies published at 418 | creativecommons.org/policies, Creative Commons does not authorize the 419 | use of the trademark "Creative Commons" or any other trademark or logo 420 | of Creative Commons without its prior written consent including, 421 | without limitation, in connection with any unauthorized modifications 422 | to any of its public licenses or any other arrangements, 423 | understandings, or agreements concerning use of licensed material. For 424 | the avoidance of doubt, this paragraph does not form part of the 425 | public licenses. 426 | 427 | Creative Commons may be contacted at creativecommons.org. 428 | --------------------------------------------------------------------------------