├── data ├── fonts │ └── .gitkeep ├── themes │ └── .gitkeep └── page_sharer.toml ├── static └── img │ ├── .gitkeep │ ├── image_test.png │ └── mse_v_logistic_loss.png ├── .gitignore ├── view.sh ├── content ├── authors │ └── admin │ │ ├── avatar.jpg │ │ ├── avatar_old.jpg │ │ └── _index.md ├── home │ ├── index.md │ ├── about.md │ ├── contact.md │ ├── people.md │ ├── tags.md │ ├── skills.md │ ├── experience.md │ ├── accomplishments.md │ ├── featured.md │ ├── posts.md │ ├── slider.md │ ├── talks.md │ ├── publications.md │ ├── hero.md │ ├── projects.md │ └── demo.md ├── post │ ├── _index.md │ ├── transformers_introduction.md │ ├── practice_questions.md │ └── linear-regression-basics.md ├── talk │ └── _index.md └── publication │ └── _index.md ├── .gitmodules ├── _redirects ├── academic.Rproj ├── .editorconfig ├── netlify.toml ├── config └── _default │ ├── languages.toml │ ├── menus.toml │ ├── config.toml │ └── params.toml ├── LICENSE.md ├── scripts └── init_kickstart.sh ├── update_academic.sh └── README.md /data/fonts/.gitkeep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /data/themes/.gitkeep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /static/img/.gitkeep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | public/ 2 | .idea/ 3 | -------------------------------------------------------------------------------- /view.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | hugo --i18n-warnings server 4 | -------------------------------------------------------------------------------- /static/img/image_test.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hatemr/academic-kickstart/master/static/img/image_test.png -------------------------------------------------------------------------------- /content/authors/admin/avatar.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hatemr/academic-kickstart/master/content/authors/admin/avatar.jpg -------------------------------------------------------------------------------- /.gitmodules: -------------------------------------------------------------------------------- 1 | [submodule "themes/academic"] 2 | path = themes/academic 3 | url = https://github.com/gcushen/hugo-academic.git 4 | -------------------------------------------------------------------------------- /static/img/mse_v_logistic_loss.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hatemr/academic-kickstart/master/static/img/mse_v_logistic_loss.png -------------------------------------------------------------------------------- /content/authors/admin/avatar_old.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hatemr/academic-kickstart/master/content/authors/admin/avatar_old.jpg -------------------------------------------------------------------------------- /content/home/index.md: -------------------------------------------------------------------------------- 1 | +++ 2 | # Homepage 3 | type = "widget_page" 4 | headless = true # Homepage is headless, other widget pages are not. 5 | +++ 6 | -------------------------------------------------------------------------------- /content/post/_index.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Posts 3 | 4 | # View. 5 | # 1 = List 6 | # 2 = Compact 7 | # 3 = Card 8 | view: 2 9 | 10 | # Optional header image (relative to `static/img/` folder). 11 | header: 12 | caption: "" 13 | image: "" 14 | --- 15 | -------------------------------------------------------------------------------- /content/talk/_index.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Recent & Upcoming Talks 3 | 4 | # View. 5 | # 1 = List 6 | # 2 = Compact 7 | # 3 = Card 8 | view: 2 9 | 10 | # Optional header image (relative to `static/img/` folder). 11 | header: 12 | caption: "" 13 | image: "" 14 | --- 15 | -------------------------------------------------------------------------------- /content/publication/_index.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Publications 3 | 4 | # View. 5 | # 1 = List 6 | # 2 = Compact 7 | # 3 = Card 8 | # 4 = Citation 9 | view: 4 10 | 11 | # Optional header image (relative to `static/img/` folder). 12 | header: 13 | caption: "" 14 | image: "" 15 | --- 16 | -------------------------------------------------------------------------------- /_redirects: -------------------------------------------------------------------------------- 1 | # *** Warning! Your custom domain is not using HTTPS yet *** 2 | # These rules will change if you change your site’s custom domains or HTTPS settings 3 | 4 | # Redirect default Netlify subdomain to primary domain 5 | http://optimistic-kepler-6c1e46.netlify.com/* http://robert-hatem.com/:splat 301! 6 | -------------------------------------------------------------------------------- /academic.Rproj: -------------------------------------------------------------------------------- 1 | Version: 1.0 2 | 3 | RestoreWorkspace: Default 4 | SaveWorkspace: Default 5 | AlwaysSaveHistory: Default 6 | 7 | EnableCodeIndexing: Yes 8 | UseSpacesForTab: Yes 9 | NumSpacesForTab: 2 10 | Encoding: UTF-8 11 | 12 | RnwWeave: Sweave 13 | LaTeX: pdfLaTeX 14 | 15 | AutoAppendNewline: Yes 16 | StripTrailingWhitespace: Yes 17 | -------------------------------------------------------------------------------- /.editorconfig: -------------------------------------------------------------------------------- 1 | # editorconfig.org 2 | 3 | root = true 4 | 5 | [*] 6 | charset = utf-8 7 | end_of_line = lf 8 | indent_size = 2 9 | indent_style = space 10 | insert_final_newline = true 11 | trim_trailing_whitespace = true 12 | 13 | [*.toml] 14 | max_line_length = 100 15 | 16 | [*.md] 17 | trim_trailing_whitespace = false 18 | 19 | [layouts/shortcodes/*.html] 20 | insert_final_newline = false 21 | -------------------------------------------------------------------------------- /content/home/about.md: -------------------------------------------------------------------------------- 1 | +++ 2 | # About widget. 3 | widget = "about" # See https://sourcethemes.com/academic/docs/page-builder/ 4 | headless = true # This file represents a page section. 5 | active = true # Activate this widget? true/false 6 | weight = 20 # Order that this section will appear in. 7 | 8 | title = "Biography" 9 | 10 | # Choose the user profile to display 11 | # This should be the username of a profile in your `content/authors/` folder. 12 | # See https://sourcethemes.com/academic/docs/get-started/#introduce-yourself 13 | author = "admin" 14 | +++ 15 | -------------------------------------------------------------------------------- /content/home/contact.md: -------------------------------------------------------------------------------- 1 | +++ 2 | # Contact widget. 3 | widget = "contact" # See https://sourcethemes.com/academic/docs/page-builder/ 4 | headless = true # This file represents a page section. 5 | active = false # Activate this widget? true/false 6 | weight = 130 # Order that this section will appear. 7 | 8 | title = "Contact" 9 | subtitle = "" 10 | 11 | # Automatically link email and phone? 12 | autolink = true 13 | 14 | # Email form provider 15 | # 0: Disable email form 16 | # 1: Netlify (requires that the site is hosted by Netlify) 17 | # 2: formspree.io 18 | email_form = 0 19 | +++ 20 | 21 | -------------------------------------------------------------------------------- /content/home/people.md: -------------------------------------------------------------------------------- 1 | +++ 2 | # People widget. 3 | widget = "people" # See https://sourcethemes.com/academic/docs/page-builder/ 4 | headless = true # This file represents a page section. 5 | active = false # Activate this widget? true/false 6 | weight = 68 # Order that this section will appear. 7 | 8 | title = "People" 9 | subtitle = "" 10 | 11 | # List user groups to display. 12 | # Edit each user's `user_groups` to add them to one or more of these groups. 13 | user_groups = ["Principal Investigators", 14 | "Researchers", 15 | "Grad Students", 16 | "Administration", 17 | "Visitors", 18 | "Alumni"] 19 | +++ 20 | -------------------------------------------------------------------------------- /content/home/tags.md: -------------------------------------------------------------------------------- 1 | +++ 2 | # Tag Cloud widget. 3 | widget = "tag_cloud" # See https://sourcethemes.com/academic/docs/page-builder/ 4 | headless = true # This file represents a page section. 5 | active = false # Activate this widget? true/false 6 | weight = 120 # Order that this section will appear. 7 | 8 | title = "Popular Topics" 9 | subtitle = "" 10 | 11 | [content] 12 | # Choose the taxonomy from `config.toml` to display (e.g. tags, categories) 13 | taxonomy = "tags" 14 | 15 | # Choose how many tags you would like to display (0 = all tags) 16 | count = 20 17 | 18 | [design] 19 | # Minimum and maximum font sizes (1.0 = 100%). 20 | font_size_min = 0.7 21 | font_size_max = 2.0 22 | +++ 23 | -------------------------------------------------------------------------------- /netlify.toml: -------------------------------------------------------------------------------- 1 | [build] 2 | command = "hugo --gc --minify -b $URL" 3 | publish = "public" 4 | 5 | [build.environment] 6 | HUGO_VERSION = "0.58.1" 7 | HUGO_ENABLEGITINFO = "true" 8 | 9 | [context.production.environment] 10 | HUGO_ENV = "production" 11 | 12 | [context.deploy-preview] 13 | command = "hugo --gc --minify --buildFuture -b $DEPLOY_PRIME_URL" 14 | 15 | [context.branch-deploy] 16 | command = "hugo --gc --minify -b $DEPLOY_PRIME_URL" 17 | 18 | [[headers]] 19 | for = "*.webmanifest" 20 | [headers.values] 21 | Content-Type = "application/manifest+json; charset=UTF-8" 22 | 23 | [[headers]] 24 | for = "index.xml" 25 | [headers.values] 26 | Content-Type = "application/rss+xml" 27 | -------------------------------------------------------------------------------- /config/_default/languages.toml: -------------------------------------------------------------------------------- 1 | # Languages 2 | # Create a `[X]` block for each language you want, where X is the language ID. 3 | # Refer to https://sourcethemes.com/academic/docs/language/ 4 | 5 | # Configure the English version of the site. 6 | [en] 7 | languageCode = "en-us" 8 | # contentDir = "content/en" # Uncomment for multi-lingual sites, and move English content into `en` sub-folder. 9 | 10 | # Uncomment the lines below to configure your website in a second language. 11 | #[zh] 12 | # languageCode = "zh-Hans" 13 | # contentDir = "content/zh" 14 | # title = "Chinese website title..." 15 | # [zh.params] 16 | # description = "Site description in Chinese..." 17 | # [[zh.menu.main]] 18 | # name = "Wo" 19 | # url = "#about" 20 | # weight = 1 21 | -------------------------------------------------------------------------------- /content/home/skills.md: -------------------------------------------------------------------------------- 1 | +++ 2 | # A Skills section created with the Featurette widget. 3 | widget = "featurette" # See https://sourcethemes.com/academic/docs/page-builder/ 4 | headless = true # This file represents a page section. 5 | active = false # Activate this widget? true/false 6 | weight = 30 # Order that this section will appear. 7 | 8 | title = "Skills" 9 | subtitle = "" 10 | 11 | # Showcase personal skills or business features. 12 | # 13 | # Add/remove as many `[[feature]]` blocks below as you like. 14 | # 15 | # For available icons, see: https://sourcethemes.com/academic/docs/widgets/#icons 16 | 17 | [[feature]] 18 | icon = "r-project" 19 | icon_pack = "fab" 20 | name = "R" 21 | description = "90%" 22 | 23 | [[feature]] 24 | icon = "chart-line" 25 | icon_pack = "fas" 26 | name = "Statistics" 27 | description = "100%" 28 | 29 | [[feature]] 30 | icon = "camera-retro" 31 | icon_pack = "fas" 32 | name = "Photography" 33 | description = "10%" 34 | 35 | +++ 36 | -------------------------------------------------------------------------------- /config/_default/menus.toml: -------------------------------------------------------------------------------- 1 | # Navigation Links 2 | # To link a homepage widget, specify the URL as a hash `#` followed by the filename of the 3 | # desired widget in your `content/home/` folder. 4 | # The weight parameter defines the order that the links will appear in. 5 | 6 | [[main]] 7 | name = "Home" 8 | url = "#about" 9 | weight = 10 10 | 11 | #[[main]] 12 | # name = "Posts" 13 | # url = "#posts" 14 | # weight = 20 15 | 16 | #[[main]] 17 | # name = "Projects" 18 | # url = "#projects" 19 | # weight = 30 20 | 21 | [[About]] 22 | name = "About" 23 | url = "#about" 24 | weight = 30 25 | 26 | #[[main]] 27 | # name = "Publications" 28 | # url = "#featured" 29 | # weight = 40 30 | 31 | #[[main]] 32 | # name = "Contact" 33 | # url = "#contact" 34 | # weight = 60 35 | 36 | # Link to a PDF of your resume/CV from the menu. 37 | # To enable, copy your resume/CV to `static/files/cv.pdf` and uncomment the lines below. 38 | # [[main]] 39 | # name = "CV" 40 | # url = "files/cv.pdf" 41 | # weight = 70 42 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2016-present George Cushen 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /content/home/experience.md: -------------------------------------------------------------------------------- 1 | +++ 2 | # Experience widget. 3 | widget = "experience" # See https://sourcethemes.com/academic/docs/page-builder/ 4 | headless = true # This file represents a page section. 5 | active = false # Activate this widget? true/false 6 | weight = 40 # Order that this section will appear. 7 | 8 | title = "Experience" 9 | subtitle = "" 10 | 11 | # Date format for experience 12 | # Refer to https://sourcethemes.com/academic/docs/customization/#date-format 13 | date_format = "Jan 2006" 14 | 15 | # Experiences. 16 | # Add/remove as many `[[experience]]` blocks below as you like. 17 | # Required fields are `title`, `company`, and `date_start`. 18 | # Leave `date_end` empty if it's your current employer. 19 | # Begin/end multi-line descriptions with 3 quotes `"""`. 20 | [[experience]] 21 | title = "CEO" 22 | company = "GenCoin" 23 | company_url = "" 24 | location = "California" 25 | date_start = "2017-01-01" 26 | date_end = "" 27 | description = """ 28 | Responsibilities include: 29 | 30 | * Analysing 31 | * Modelling 32 | * Deploying 33 | """ 34 | 35 | [[experience]] 36 | title = "Professor" 37 | company = "University X" 38 | company_url = "" 39 | location = "California" 40 | date_start = "2016-01-01" 41 | date_end = "2016-12-31" 42 | description = """Taught electronic engineering and researched semiconductor physics.""" 43 | 44 | +++ 45 | -------------------------------------------------------------------------------- /scripts/init_kickstart.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # WARNING: this will reset the project to the Kickstart template! 4 | 5 | # Update Academic 6 | cd .. 7 | source update_academic.sh 8 | cd scripts/ 9 | 10 | ################################################# 11 | 12 | # Install demo config 13 | rsync -av ../themes/academic/exampleSite/config/ ../config/ 14 | 15 | # Install demo user 16 | rsync -av ../themes/academic/exampleSite/content/authors/ ../content/authors/ 17 | 18 | # Install page sharer button customizer 19 | rsync -av ../themes/academic/data/page_sharer.toml ../data/ 20 | 21 | # Install an example instance of each widget type 22 | rsync -av --exclude gallery/ ../themes/academic/exampleSite/content/home/ ../content/home/ 23 | 24 | # Install indices 25 | rsync -av ../themes/academic/exampleSite/content/post/_index.md ../content/post/_index.md 26 | rsync -av ../themes/academic/exampleSite/content/publication/_index.md ../content/publication/_index.md 27 | rsync -av ../themes/academic/exampleSite/content/talk/_index.md ../content/talk/_index.md 28 | 29 | # Skip static dir - do not import the demo's media library 30 | 31 | ################################################# 32 | 33 | # Post processing 34 | 35 | # Deactivate Hero 36 | sed -i '' -e 's/active = true/active = false/' ../content/home/hero.md 37 | 38 | # Manual Steps: 39 | # - content/home/project.md: Re-comment out project filters 40 | # - content/home/demo.md: Re-modify content & set gradient background instead of image 41 | # - content/home/hero.md: Clear `hero_media` value & set gradient background instead of image 42 | -------------------------------------------------------------------------------- /content/home/accomplishments.md: -------------------------------------------------------------------------------- 1 | +++ 2 | # Accomplishments widget. 3 | widget = "accomplishments" # See https://sourcethemes.com/academic/docs/page-builder/ 4 | headless = true # This file represents a page section. 5 | active = false # Activate this widget? true/false 6 | weight = 50 # Order that this section will appear. 7 | 8 | title = "Accomplish­ments" 9 | subtitle = "" 10 | 11 | # Date format 12 | # Refer to https://sourcethemes.com/academic/docs/customization/#date-format 13 | date_format = "Jan 2006" 14 | 15 | # Accomplishments. 16 | # Add/remove as many `[[item]]` blocks below as you like. 17 | # `title`, `organization` and `date_start` are the required parameters. 18 | # Leave other parameters empty if not required. 19 | # Begin/end multi-line descriptions with 3 quotes `"""`. 20 | 21 | [[item]] 22 | organization = "Coursera" 23 | organization_url = "https://www.coursera.org" 24 | title = "Neural Networks and Deep Learning" 25 | url = "" 26 | certificate_url = "https://www.coursera.org" 27 | date_start = "2018-10-01" 28 | date_end = "" 29 | description = "" 30 | 31 | [[item]] 32 | organization = "edX" 33 | organization_url = "https://www.edx.org" 34 | title = "Blockchain Fundamentals" 35 | url = "https://www.edx.org/professional-certificate/uc-berkeleyx-blockchain-fundamentals" 36 | certificate_url = "https://www.edx.org" 37 | date_start = "2018-03-01" 38 | date_end = "" 39 | description = "Formulated informed blockchain models, hypotheses, and use cases." 40 | 41 | [[item]] 42 | organization = "DataCamp" 43 | organization_url = "https://www.datacamp.com" 44 | title = "Object-Oriented Programming in R: S3 and R6 Course" 45 | url = "" 46 | certificate_url = "https://www.datacamp.com" 47 | date_start = "2017-07-01" 48 | date_end = "2017-12-21" 49 | description = "" 50 | 51 | +++ 52 | -------------------------------------------------------------------------------- /update_academic.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Source Themes Academic: Theme updater 4 | # Checks for available updates and then asks to install any updates. 5 | # https://sourcethemes.com/academic/ 6 | # 7 | # Command: bash ./update_academic.sh 8 | 9 | # Check for prerequisites. 10 | if [ ! -d .git ]; then 11 | echo "ERROR: This tool is for Git repositories only." 12 | exit 1; 13 | fi 14 | 15 | # Function to update Academic 16 | function install_update () { 17 | # Apply any updates 18 | git submodule update --remote --merge 19 | 20 | # - Update Netlify.toml with required Hugo version 21 | if [ -f ./netlify.toml ]; then 22 | version=$(sed -n 's/^min_version = //p' themes/academic/theme.toml) 23 | sed -i '' -e "s/HUGO_VERSION = .*/HUGO_VERSION = $version/g" ./netlify.toml 24 | fi 25 | 26 | echo 27 | echo "View the release notes at: https://sourcethemes.com/academic/updates" 28 | echo "If there are breaking changes, the config and/or front matter of content" \ 29 | "may need upgrading by following the steps in the release notes." 30 | } 31 | 32 | # Display currently installed version (although could be between versions if updated to master rather than tag) 33 | version=$(sed -n 's/^version = "//p' themes/academic/data/academic.toml) 34 | echo -e "Source Themes Academic v$version\n" 35 | 36 | # Display available updates 37 | echo -e "Checking for updates...\n" 38 | cd themes/academic 39 | git fetch 40 | git log --pretty=oneline --abbrev-commit --decorate HEAD..origin/master 41 | cd ../../ 42 | 43 | title="Do you wish to install the above updates?" 44 | prompt="Choose an option and press Enter:" 45 | options=("Yes" "No") 46 | 47 | echo "$title" 48 | PS3="$prompt " 49 | select opt in "${options[@]}"; do 50 | case $opt in 51 | Yes ) install_update; break;; 52 | No ) break;; 53 | * ) break;; 54 | esac 55 | done 56 | -------------------------------------------------------------------------------- /content/home/featured.md: -------------------------------------------------------------------------------- 1 | +++ 2 | # A Featured Publications section created with the Featured Content widget. 3 | # This section displays publications from `content/publication/` which have 4 | # `featured = true` in their front matter. 5 | 6 | widget = "featured" # See https://sourcethemes.com/academic/docs/page-builder/ 7 | headless = true # This file represents a page section. 8 | active = false # Activate this widget? true/false 9 | weight = 80 # Order that this section will appear. 10 | 11 | title = "Featured Publications" 12 | subtitle = "" 13 | 14 | [content] 15 | # Page type to display. E.g. post, talk, or publication. 16 | page_type = "publication" 17 | 18 | # Choose how much pages you would like to display (0 = all pages) 19 | count = 0 20 | 21 | # Page order. Descending (desc) or ascending (asc) date. 22 | order = "desc" 23 | 24 | # Filter posts by a taxonomy term. 25 | [content.filters] 26 | tag = "" 27 | category = "" 28 | publication_type = "" 29 | 30 | [design] 31 | # Toggle between the various page layout types. 32 | # 1 = List 33 | # 2 = Compact 34 | # 3 = Card 35 | # 4 = Citation (publication only) 36 | view = 3 37 | 38 | [design.background] 39 | # Apply a background color, gradient, or image. 40 | # Uncomment (by removing `#`) an option to apply it. 41 | # Choose a light or dark text color by setting `text_color_light`. 42 | # Any HTML color name or Hex value is valid. 43 | 44 | # Background color. 45 | # color = "navy" 46 | 47 | # Background gradient. 48 | # gradient_start = "DeepSkyBlue" 49 | # gradient_end = "SkyBlue" 50 | 51 | # Background image. 52 | # image = "background.jpg" # Name of image in `static/img/`. 53 | # image_darken = 0.6 # Darken the image? Range 0-1 where 0 is transparent and 1 is opaque. 54 | 55 | # Text color (true=light or false=dark). 56 | # text_color_light = true 57 | 58 | [advanced] 59 | # Custom CSS. 60 | css_style = "" 61 | 62 | # CSS class. 63 | css_class = "" 64 | +++ 65 | -------------------------------------------------------------------------------- /content/home/posts.md: -------------------------------------------------------------------------------- 1 | +++ 2 | # A Recent Blog Posts section created with the Pages widget. 3 | # This section displays recent blog posts from `content/post/`. 4 | 5 | widget = "pages" # See https://sourcethemes.com/academic/docs/page-builder/ 6 | headless = true # This file represents a page section. 7 | active = true # Activate this widget? true/false 8 | weight = 60 # Order that this section will appear. 9 | 10 | title = "Recent Posts" 11 | subtitle = "" 12 | 13 | [content] 14 | # Page type to display. E.g. post, talk, or publication. 15 | page_type = "post" 16 | 17 | # Choose how much pages you would like to display (0 = all pages) 18 | count = 3 19 | 20 | # Choose how many pages you would like to offset by 21 | offset = 0 22 | 23 | # Page order. Descending (desc) or ascending (asc) date. 24 | order = "desc" 25 | 26 | # Filter posts by a taxonomy term. 27 | [content.filters] 28 | tag = "" 29 | category = "" 30 | publication_type = "" 31 | exclude_featured = false 32 | 33 | [design] 34 | # Toggle between the various page layout types. 35 | # 1 = List 36 | # 2 = Compact 37 | # 3 = Card 38 | # 4 = Citation (publication only) 39 | view = 2 40 | 41 | [design.background] 42 | # Apply a background color, gradient, or image. 43 | # Uncomment (by removing `#`) an option to apply it. 44 | # Choose a light or dark text color by setting `text_color_light`. 45 | # Any HTML color name or Hex value is valid. 46 | 47 | # Background color. 48 | # color = "navy" 49 | 50 | # Background gradient. 51 | # gradient_start = "DeepSkyBlue" 52 | # gradient_end = "SkyBlue" 53 | 54 | # Background image. 55 | # image = "background.jpg" # Name of image in `static/img/`. 56 | # image_darken = 0.6 # Darken the image? Range 0-1 where 0 is transparent and 1 is opaque. 57 | 58 | # Text color (true=light or false=dark). 59 | # text_color_light = true 60 | 61 | [advanced] 62 | # Custom CSS. 63 | css_style = "" 64 | 65 | # CSS class. 66 | css_class = "" 67 | +++ 68 | -------------------------------------------------------------------------------- /content/home/slider.md: -------------------------------------------------------------------------------- 1 | +++ 2 | # Slider widget. 3 | widget = "slider" # See https://sourcethemes.com/academic/docs/page-builder/ 4 | headless = true # This file represents a page section. 5 | active = false # Activate this widget? true/false 6 | weight = 1 # Order that this section will appear. 7 | 8 | # Slide interval. 9 | # Use `false` to disable animation or enter a time in ms, e.g. `5000` (5s). 10 | interval = false 11 | 12 | # Slide height (optional). 13 | # E.g. `500px` for 500 pixels or `calc(100vh - 70px)` for full screen. 14 | height = "" 15 | 16 | # Slides. 17 | # Duplicate an `[[item]]` block to add more slides. 18 | [[item]] 19 | title = "Hello" 20 | content = "I am center aligned :smile:" 21 | align = "center" # Choose `center`, `left`, or `right`. 22 | 23 | # Overlay a color or image (optional). 24 | # Deactivate an option by commenting out the line, prefixing it with `#`. 25 | overlay_color = "#666" # An HTML color value. 26 | overlay_img = "headers/bubbles-wide.jpg" # Image path relative to your `static/img/` folder. 27 | overlay_filter = 0.5 # Darken the image. Value in range 0-1. 28 | 29 | # Call to action button (optional). 30 | # Activate the button by specifying a URL and button label below. 31 | # Deactivate by commenting out parameters, prefixing lines with `#`. 32 | cta_label = "Get Academic" 33 | cta_url = "https://sourcethemes.com/academic/" 34 | cta_icon_pack = "fas" 35 | cta_icon = "graduation-cap" 36 | 37 | [[item]] 38 | title = "Left" 39 | content = "I am left aligned :smile:" 40 | align = "left" 41 | 42 | overlay_color = "#555" # An HTML color value. 43 | overlay_img = "" # Image path relative to your `static/img/` folder. 44 | overlay_filter = 0.5 # Darken the image. Value in range 0-1. 45 | 46 | [[item]] 47 | title = "Right" 48 | content = "I am right aligned :smile:" 49 | align = "right" 50 | 51 | overlay_color = "#333" # An HTML color value. 52 | overlay_img = "" # Image path relative to your `static/img/` folder. 53 | overlay_filter = 0.5 # Darken the image. Value in range 0-1. 54 | +++ 55 | -------------------------------------------------------------------------------- /content/home/talks.md: -------------------------------------------------------------------------------- 1 | +++ 2 | # A Recent and Upcoming Talks section created with the Pages widget. 3 | # This section displays recent talks from `content/talk/`. 4 | 5 | widget = "pages" # See https://sourcethemes.com/academic/docs/page-builder/ 6 | headless = true # This file represents a page section. 7 | active = false # Activate this widget? true/false 8 | weight = 70 # Order that this section will appear. 9 | 10 | title = "Recent & Upcoming Talks" 11 | subtitle = "" 12 | 13 | [content] 14 | # Page type to display. E.g. post, talk, or publication. 15 | page_type = "talk" 16 | 17 | # Choose how much pages you would like to display (0 = all pages) 18 | count = 5 19 | 20 | # Choose how many pages you would like to offset by 21 | offset = 0 22 | 23 | # Page order. Descending (desc) or ascending (asc) date. 24 | order = "desc" 25 | 26 | # Filter posts by a taxonomy term. 27 | [content.filters] 28 | tag = "" 29 | category = "" 30 | publication_type = "" 31 | exclude_featured = false 32 | exclude_past = false 33 | exclude_future = false 34 | 35 | [design] 36 | # Toggle between the various page layout types. 37 | # 1 = List 38 | # 2 = Compact 39 | # 3 = Card 40 | # 4 = Citation (publication only) 41 | view = 2 42 | 43 | [design.background] 44 | # Apply a background color, gradient, or image. 45 | # Uncomment (by removing `#`) an option to apply it. 46 | # Choose a light or dark text color by setting `text_color_light`. 47 | # Any HTML color name or Hex value is valid. 48 | 49 | # Background color. 50 | # color = "navy" 51 | 52 | # Background gradient. 53 | # gradient_start = "DeepSkyBlue" 54 | # gradient_end = "SkyBlue" 55 | 56 | # Background image. 57 | # image = "background.jpg" # Name of image in `static/img/`. 58 | # image_darken = 0.6 # Darken the image? Range 0-1 where 0 is transparent and 1 is opaque. 59 | 60 | # Text color (true=light or false=dark). 61 | # text_color_light = true 62 | 63 | [advanced] 64 | # Custom CSS. 65 | css_style = "" 66 | 67 | # CSS class. 68 | css_class = "" 69 | +++ 70 | -------------------------------------------------------------------------------- /data/page_sharer.toml: -------------------------------------------------------------------------------- 1 | # Page Sharer 2 | # Documentation: https://sourcethemes.com/academic/docs/customization/#page-sharer 3 | 4 | [[buttons]] 5 | id = "twitter" 6 | url = "https://twitter.com/intent/tweet?url={url}&text={title}" 7 | title = "Twitter" 8 | icon_pack = "fab" 9 | icon = "twitter" 10 | enable = true 11 | 12 | [[buttons]] 13 | id = "facebook" 14 | url = "https://www.facebook.com/sharer.php?u={url}&t={title}" 15 | title = "Facebook" 16 | icon_pack = "fab" 17 | icon = "facebook-f" 18 | enable = true 19 | 20 | [[buttons]] 21 | id = "email" 22 | url = "mailto:?subject={title}&body={url}" 23 | title = "Email" 24 | icon_pack = "fas" 25 | icon = "envelope" 26 | enable = true 27 | 28 | [[buttons]] 29 | id = "linkedin" 30 | url = "https://www.linkedin.com/shareArticle?url={url}&title={title}" 31 | title = "LinkedIn" 32 | icon_pack = "fab" 33 | icon = "linkedin-in" 34 | enable = true 35 | 36 | [[buttons]] 37 | id = "whatsapp" 38 | url = "https://web.whatsapp.com/send?text={title}%20{url}" 39 | title = "WhatsApp" 40 | icon_pack = "fab" 41 | icon = "whatsapp" 42 | enable = true 43 | 44 | [[buttons]] 45 | id = "weibo" 46 | url = "https://service.weibo.com/share/share.php?url={url}&title={title}" 47 | title = "Weibo" 48 | icon_pack = "fab" 49 | icon = "weibo" 50 | enable = true 51 | 52 | [[buttons]] 53 | id = "reddit" 54 | url = "https://reddit.com/submit?url={url}&title={title}" 55 | title = "Reddit" 56 | icon_pack = "fab" 57 | icon = "reddit-alien" 58 | enable = false 59 | 60 | [[buttons]] 61 | id = "pinterest" 62 | url = "https://pinterest.com/pin/create/link/?url={url}&description={title}" 63 | title = "Pinterest" 64 | icon_pack = "fab" 65 | icon = "pinterest" 66 | enable = false 67 | 68 | [[buttons]] 69 | id = "xing" 70 | url = "https://www.xing.com/spi/shares/new?url={url}&title={title}" 71 | title = "Xing" 72 | icon_pack = "fab" 73 | icon = "xing" 74 | enable = false 75 | 76 | [[buttons]] 77 | id = "tumblr" 78 | url = "https://www.tumblr.com/widgets/share/tool?canonicalUrl={url}&title={title}" 79 | title = "Tumblr" 80 | icon_pack = "fab" 81 | icon = "tumblr" 82 | enable = false 83 | -------------------------------------------------------------------------------- /content/home/publications.md: -------------------------------------------------------------------------------- 1 | +++ 2 | # A Recent Publications section created with the Pages widget. 3 | # This section displays recent blog posts from `content/publication/`. 4 | 5 | widget = "pages" # See https://sourcethemes.com/academic/docs/page-builder/ 6 | headless = true # This file represents a page section. 7 | active = false # Activate this widget? true/false 8 | weight = 90 # Order that this section will appear. 9 | 10 | title = "Recent Publications" 11 | subtitle = "" 12 | 13 | [content] 14 | # Page type to display. E.g. post, talk, or publication. 15 | page_type = "publication" 16 | 17 | # Choose how much pages you would like to display (0 = all pages) 18 | count = 5 19 | 20 | # Choose how many pages you would like to offset by 21 | offset = 0 22 | 23 | # Page order. Descending (desc) or ascending (asc) date. 24 | order = "desc" 25 | 26 | # Filter posts by a taxonomy term. 27 | [content.filters] 28 | tag = "" 29 | category = "" 30 | publication_type = "" 31 | exclude_featured = false 32 | 33 | [design] 34 | # Toggle between the various page layout types. 35 | # 1 = List 36 | # 2 = Compact 37 | # 3 = Card 38 | # 4 = Citation (publication only) 39 | view = 2 40 | 41 | [design.background] 42 | # Apply a background color, gradient, or image. 43 | # Uncomment (by removing `#`) an option to apply it. 44 | # Choose a light or dark text color by setting `text_color_light`. 45 | # Any HTML color name or Hex value is valid. 46 | 47 | # Background color. 48 | # color = "navy" 49 | 50 | # Background gradient. 51 | # gradient_start = "DeepSkyBlue" 52 | # gradient_end = "SkyBlue" 53 | 54 | # Background image. 55 | # image = "background.jpg" # Name of image in `static/img/`. 56 | # image_darken = 0.6 # Darken the image? Range 0-1 where 0 is transparent and 1 is opaque. 57 | 58 | # Text color (true=light or false=dark). 59 | # text_color_light = true 60 | 61 | [advanced] 62 | # Custom CSS. 63 | css_style = "" 64 | 65 | # CSS class. 66 | css_class = "" 67 | +++ 68 | 69 | {{% alert note %}} 70 | Quickly discover relevant content by [filtering publications]({{< ref "/publication/_index.md" >}}). 71 | {{% /alert %}} 72 | -------------------------------------------------------------------------------- /content/home/hero.md: -------------------------------------------------------------------------------- 1 | +++ 2 | # Hero widget. 3 | widget = "hero" # See https://sourcethemes.com/academic/docs/page-builder/ 4 | headless = true # This file represents a page section. 5 | active = false # Activate this widget? true/false 6 | weight = 10 # Order that this section will appear. 7 | 8 | title = "Academic" 9 | 10 | # Hero image (optional). Enter filename of an image in the `static/img/` folder. 11 | hero_media = "hero-academic.png" 12 | 13 | [design.background] 14 | # Apply a background color, gradient, or image. 15 | # Uncomment (by removing `#`) an option to apply it. 16 | # Choose a light or dark text color by setting `text_color_light`. 17 | # Any HTML color name or Hex value is valid. 18 | 19 | # Background color. 20 | # color = "navy" 21 | 22 | # Background gradient. 23 | gradient_start = "#4bb4e3" 24 | gradient_end = "#2b94c3" 25 | 26 | # Background image. 27 | # image = "" # Name of image in `static/img/`. 28 | # image_darken = 0.6 # Darken the image? Range 0-1 where 0 is transparent and 1 is opaque. 29 | 30 | # Text color (true=light or false=dark). 31 | text_color_light = true 32 | 33 | # Call to action links (optional). 34 | # Display link(s) by specifying a URL and label below. Icon is optional for `[cta]`. 35 | # Remove a link/note by deleting a cta/note block. 36 | [cta] 37 | url = "https://sourcethemes.com/academic/docs/install/" 38 | label = "Get Started" 39 | icon_pack = "fas" 40 | icon = "download" 41 | 42 | [cta_alt] 43 | url = "https://sourcethemes.com/academic/" 44 | label = "View Documentation" 45 | 46 | # Note. An optional note to show underneath the links. 47 | [cta_note] 48 | label = 'Latest release' 49 | +++ 50 | 51 | **The Best Way to Create the Website You Want from Markdown (or Jupyter/RStudio)** 52 | 53 | Build **Anything** with Widgets 54 | 55 | Star 56 | -------------------------------------------------------------------------------- /content/home/projects.md: -------------------------------------------------------------------------------- 1 | +++ 2 | # A Projects section created with the Portfolio widget. 3 | widget = "portfolio" # See https://sourcethemes.com/academic/docs/page-builder/ 4 | headless = true # This file represents a page section. 5 | active = false # Activate this widget? true/false 6 | weight = 65 # Order that this section will appear. 7 | 8 | title = "Projects" 9 | subtitle = "" 10 | 11 | [content] 12 | # Page type to display. E.g. project. 13 | page_type = "project" 14 | 15 | # Filter toolbar (optional). 16 | # Add or remove as many filters (`[[content.filter_button]]` instances) as you like. 17 | # To show all items, set `tag` to "*". 18 | # To filter by a specific tag, set `tag` to an existing tag name. 19 | # To remove toolbar, delete/comment all instances of `[[content.filter_button]]` below. 20 | 21 | # Default filter index (e.g. 0 corresponds to the first `[[filter_button]]` instance below). 22 | filter_default = 0 23 | 24 | # [[content.filter_button]] 25 | # name = "All" 26 | # tag = "*" 27 | 28 | # [[content.filter_button]] 29 | # name = "Deep Learning" 30 | # tag = "Deep Learning" 31 | 32 | # [[content.filter_button]] 33 | # name = "Other" 34 | # tag = "Demo" 35 | 36 | [design] 37 | # Choose how many columns the section has. Valid values: 1 or 2. 38 | columns = "2" 39 | 40 | # Toggle between the various page layout types. 41 | # 1 = List 42 | # 2 = Compact 43 | # 3 = Card 44 | # 5 = Showcase 45 | view = 3 46 | 47 | # For Showcase view, flip alternate rows? 48 | flip_alt_rows = false 49 | 50 | [design.background] 51 | # Apply a background color, gradient, or image. 52 | # Uncomment (by removing `#`) an option to apply it. 53 | # Choose a light or dark text color by setting `text_color_light`. 54 | # Any HTML color name or Hex value is valid. 55 | 56 | # Background color. 57 | # color = "navy" 58 | 59 | # Background gradient. 60 | # gradient_start = "DeepSkyBlue" 61 | # gradient_end = "SkyBlue" 62 | 63 | # Background image. 64 | # image = "background.jpg" # Name of image in `static/img/`. 65 | # image_darken = 0.6 # Darken the image? Range 0-1 where 0 is transparent and 1 is opaque. 66 | 67 | # Text color (true=light or false=dark). 68 | # text_color_light = true 69 | 70 | [advanced] 71 | # Custom CSS. 72 | css_style = "" 73 | 74 | # CSS class. 75 | css_class = "" 76 | +++ 77 | 78 | -------------------------------------------------------------------------------- /content/authors/admin/_index.md: -------------------------------------------------------------------------------- 1 | --- 2 | # Display name 3 | name: Robert Hatem 4 | 5 | # Username (this should match the folder name) 6 | authors: 7 | - admin 8 | 9 | # Is this the primary user of the site? 10 | superuser: true 11 | 12 | # Role/position 13 | role: Data Scientist 14 | 15 | # Organizations/Affiliations 16 | #organizations: 17 | #- name: #Stanford University 18 | # url: "" 19 | 20 | # Short bio (displayed in user profile at end of posts) 21 | bio: Data scientist interested in machine learning and natural language processing. 22 | 23 | interests: 24 | - Machine Learning 25 | - Natural Langauge Processing 26 | - Causal Inference 27 | 28 | education: 29 | courses: 30 | - course: MS in Fin. Mathematics 31 | institution: University of Chicago 32 | year: 2019 33 | - course: BSc in Physics 34 | institution: Boston College 35 | year: 2015 36 | 37 | # Social/Academic Networking 38 | # For available icons, see: https://sourcethemes.com/academic/docs/widgets/#icons 39 | # For an email link, use "fas" icon pack, "envelope" icon, and a link in the 40 | # form "mailto:your-email@example.com" or "#contact" for contact widget. 41 | social: 42 | - icon: envelope 43 | icon_pack: fas 44 | link: "mailto:rhatem@uchicago.edu" #'#contact' 45 | #- icon: twitter 46 | # icon_pack: fab 47 | # link: https://twitter.com/RobertEHatem 48 | #- icon: google-scholar 49 | # icon_pack: ai 50 | # link: https://scholar.google.co.uk/citations?user=sIwtMXoAAAAJ 51 | - icon: github 52 | icon_pack: fab 53 | link: https://github.com/hatemr 54 | 55 | - icon: linkedin 56 | icon_pack: fab 57 | link: https://www.linkedin.com/in/robert-e-hatem/ 58 | 59 | # Link to a PDF of your resume/CV from the About widget. 60 | # To enable, copy your resume/CV to `static/files/cv.pdf` and uncomment the lines below. 61 | # - icon: cv 62 | # icon_pack: ai 63 | # link: files/cv.pdf 64 | 65 | # Enter email to display Gravatar (if Gravatar enabled in Config) 66 | email: "rhatem@uchicago.edu" 67 | 68 | # Organizational groups that you belong to (for People widget) 69 | # Set this to `[]` or comment out if you are not using People widget. 70 | user_groups: 71 | - Researchers 72 | - Visitors 73 | --- 74 | 75 | I am a master's student in Fin. Mathematics at the University of Chicago. I am interested in machine learning and natural language processing. 76 | 77 | See my other site [hatemr.github.io](https://hatemr.github.io) while I migrate websites. -------------------------------------------------------------------------------- /config/_default/config.toml: -------------------------------------------------------------------------------- 1 | # Configuration of Academic 2 | # Documentation: https://sourcethemes.com/academic/ 3 | # 4 | # This file is formatted using TOML syntax - learn more at https://learnxinyminutes.com/docs/toml/ 5 | # Each configuration section is defined by a name in square brackets (e.g. `[outputs]`). 6 | 7 | # Title of your site 8 | title = "Robert Hatem" 9 | 10 | # The URL of your site. 11 | # End your URL with a `/` trailing slash, e.g. `https://example.com/`. 12 | baseurl = "/" 13 | 14 | # Enter a copyright notice to display in the site footer. 15 | # To display a copyright symbol, type `©`. For current year, type `{year}`. 16 | copyright = "" 17 | 18 | # Enable analytics by entering your Google Analytics tracking ID 19 | googleAnalytics = "" 20 | 21 | ############################ 22 | ## Advanced options below ## 23 | ############################ 24 | 25 | # Name of Academic theme folder in `themes/`. 26 | theme = "academic" 27 | 28 | # Get last modified date for content from Git? 29 | enableGitInfo = false 30 | 31 | # Default language to use (if you setup multilingual support) 32 | defaultContentLanguage = "en" 33 | hasCJKLanguage = false # Set `true` for Chinese/Japanese/Korean languages. 34 | defaultContentLanguageInSubdir = false 35 | removePathAccents = true # Workaround for https://github.com/gohugoio/hugo/issues/5687 36 | 37 | paginate = 10 # Number of items per page in paginated lists. 38 | enableEmoji = true 39 | footnotereturnlinkcontents = "^" 40 | ignoreFiles = ["\\.ipynb$", ".ipynb_checkpoints$", "\\.Rmd$", "\\.Rmarkdown$", "_files$", "_cache$"] 41 | 42 | [outputs] 43 | home = [ "HTML", "RSS", "JSON", "WebAppManifest" ] 44 | section = [ "HTML", "RSS" ] 45 | 46 | [mediaTypes."application/manifest+json"] 47 | suffixes = ["webmanifest"] 48 | 49 | [outputFormats.WebAppManifest] 50 | mediaType = "application/manifest+json" 51 | rel = "manifest" 52 | 53 | # Configure BlackFriday Markdown rendering. 54 | # See: https://gohugo.io/getting-started/configuration/#configure-blackfriday 55 | [blackfriday] 56 | hrefTargetBlank = true # `true` opens external links in a new tab. See https://github.com/gohugoio/hugo/issues/2424 57 | angledQuotes = false 58 | latexDashes = true 59 | extensions = ["backslashLineBreak"] 60 | 61 | [imaging] 62 | resampleFilter = "lanczos" 63 | quality = 90 64 | anchor = "smart" # Anchor for cropping. Options include Smart and Center. 65 | 66 | # Taxonomies. 67 | [taxonomies] 68 | tag = "tags" 69 | category = "categories" 70 | publication_type = "publication_types" 71 | author = "authors" -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # [Academic Kickstart](https://sourcethemes.com/academic/) 2 | 3 | **Academic** makes it easy to create a beautiful website for free using Markdown, Jupyter, or RStudio. Customize anything on your site with widgets, themes, and language packs. [Check out the latest demo](https://academic-demo.netlify.com/) of what you'll get in less than 10 minutes, or [view the showcase](https://sourcethemes.com/academic/#expo). 4 | 5 | **Academic Kickstart** provides a minimal template to kickstart your new website. 6 | 7 | - [**Get Started**](#install) 8 | - [View the documentation](https://sourcethemes.com/academic/docs/) 9 | - [Ask a question](http://discuss.gohugo.io/) 10 | - [Request a feature or report a bug](https://github.com/gcushen/hugo-academic/issues) 11 | - Updating? View the [Update Guide](https://sourcethemes.com/academic/docs/update/) and [Release Notes](https://sourcethemes.com/academic/updates/) 12 | - Support development of Academic: 13 | - [Donate a coffee](https://paypal.me/cushen) 14 | - [Become a backer on Patreon](https://www.patreon.com/cushen) 15 | - [Decorate your laptop or journal with an Academic sticker](https://www.redbubble.com/people/neutreno/works/34387919-academic) 16 | - [Wear the T-shirt](https://academic.threadless.com/) 17 | 18 | [![Screenshot](https://raw.githubusercontent.com/gcushen/hugo-academic/master/academic.png)](https://github.com/gcushen/hugo-academic/) 19 | 20 | ## Install 21 | 22 | You can choose from one of the following four methods to install: 23 | 24 | * [**one-click install using your web browser (recommended)**](https://sourcethemes.com/academic/docs/install/#install-with-web-browser) 25 | * [install on your computer using **Git** with the Command Prompt/Terminal app](https://sourcethemes.com/academic/docs/install/#install-with-git) 26 | * [install on your computer by downloading the **ZIP files**](https://sourcethemes.com/academic/docs/install/#install-with-zip) 27 | * [install on your computer with **RStudio**](https://sourcethemes.com/academic/docs/install/#install-with-rstudio) 28 | 29 | Then [personalize your new site](https://sourcethemes.com/academic/docs/get-started/). 30 | 31 | ## Ecosystem 32 | 33 | * **[Academic Admin](https://github.com/sourcethemes/academic-admin):** An admin tool to import publications from BibTeX or import assets for an offline site 34 | * **[Academic Scripts](https://github.com/sourcethemes/academic-scripts):** Scripts to help migrate content to new versions of Academic 35 | 36 | ## License 37 | 38 | Copyright 2017-present [George Cushen](https://georgecushen.com). 39 | 40 | Released under the [MIT](https://github.com/sourcethemes/academic-kickstart/blob/master/LICENSE.md) license. 41 | 42 | [![Analytics](https://ga-beacon.appspot.com/UA-78646709-2/academic-kickstart/readme?pixel)](https://github.com/igrigorik/ga-beacon) 43 | -------------------------------------------------------------------------------- /content/post/transformers_introduction.md: -------------------------------------------------------------------------------- 1 | +++ 2 | title = "Transformers Introduction" 3 | date = 2019-11-03 #T13:23:10+01:00 4 | draft = false 5 | #tags = ["Transformers"] 6 | categories = [] 7 | #markup: mmark 8 | summary = "Notes on Transformers" 9 | #disable_comments: true 10 | +++ 11 | 12 | ## Transformers 13 | Transformers are models which use attention to speed up training. While other models use attention, transformers discard the recurrent and convolution used in other architectures. 14 | 15 | ## Self-attention 16 | Turn every word into a linear combination of each words' _value vector_ ($V$). The weights in the linear combination come inner products of word pairs' _query vector_ ($Q$) and _key vector_ ($K$). These three matrices $Q, K, V$ are parameters learned during training. 17 | 18 | 1. Compute three vectors from X, whose rows are word vectors, like word2vec: 19 | * Query: $X W^Q = Q$ 20 | * Key: $X W^K = K$ 21 | * Value: $X W^V = V$ 22 | * These three vectors are all N-by-$d_k$, where $N$=number of words. 23 | 2. Calculate a score for each input word with all other words. 24 | * Dot product of the word's _query vector_ with the other word's _key vector_. 25 | * $Q K^T$ 26 | 3. Normalize by the dimension of the _key vector_: 27 | * $\frac{Q K^T}{\sqrt{d_K}}$ 28 | 4. Apply __softmax__ to rows so each words' scores are positive and sum to 1. 29 | * $softmax\(\frac{Q K^T}{\sqrt{d_K}}\)$ 30 | 5. For each word, take a linear combination of the words (rows in the value matrix V). Words that have higher weights from the softmax will receive more weight: 31 | * $Z = softmax\(\frac{Q K^T}{\sqrt{d_K}}\) V$ 32 | * Each row of the new matrix is a weighted sum of the rows of V. This is because left-multiplicaiton of matrices is a linear combination or rows. 33 | * The left matrix forces the V matrix to focus on certain words and not others. 34 | 35 | The resulting matrix $Z$ has for each word a weight sum of the words' values from $V$, where the weights are computed using inner products from $Q$ and $K$. 36 | 37 | ## Multi-headed attention 38 | Now let's use mutiple sets of Query/Key/Value matrices, one for each _head_. 39 | 40 | * Before: $(W^Q, W^K, W^V)$ 41 | * After: $\{(W_0^Q, W_0^K, W_0^V),..,(W_7^Q, W_7^K, W_7^V)\}$ 42 | 43 | That gives us eight different $Z$ matrics: 44 | 45 | * $Z_0,...,Z_7$ 46 | 47 | Now, concatenate all the matrices, and multiply by an additional weights matrix $W^0 \(8d_k \times d_W \)$. 48 | 49 | * $Z_c=[Z_0,...,Z_8] \\ \(N \times 8d_k)\)$ 50 | * $Z = Z_cW^0$ 51 | 52 | This output We can summarize the calculation as follows. 53 | 54 | $$ 55 | Z = Z_c W^0 = [Z_1,...,Z_7] 56 | $$ 57 | 58 | where 59 | 60 | $$Z_i = softmax\(\frac{Q_i K_i^T}{\sqrt{d_k}}\) V_i$$ 61 | 62 | The weight matrices $\(W_i^Q, W_i^K, W_i^V\)$ are initialized randomly and learned during training. They project input embeddings into different representation subspaces. 63 | 64 | 65 | ### References 66 | * [The Illustrated Transformer](http://jalammar.github.io/illustrated-transformer/) 67 | * [Attention Is All You Need](https://arxiv.org/pdf/1706.03762.pdf) -------------------------------------------------------------------------------- /content/home/demo.md: -------------------------------------------------------------------------------- 1 | +++ 2 | # A Demo section created with the Blank widget. 3 | # Any elements can be added in the body: https://sourcethemes.com/academic/docs/writing-markdown-latex/ 4 | # Add more sections by duplicating this file and customizing to your requirements. 5 | 6 | widget = "blank" # See https://sourcethemes.com/academic/docs/page-builder/ 7 | headless = true # This file represents a page section. 8 | active = false # Activate this widget? true/false 9 | weight = 15 # Order that this section will appear. 10 | 11 | title = "Academic Kickstart" 12 | subtitle = "" 13 | 14 | [design] 15 | # Choose how many columns the section has. Valid values: 1 or 2. 16 | columns = "1" 17 | 18 | [design.background] 19 | # Apply a background color, gradient, or image. 20 | # Uncomment (by removing `#`) an option to apply it. 21 | # Choose a light or dark text color by setting `text_color_light`. 22 | # Any HTML color name or Hex value is valid. 23 | 24 | # Background color. 25 | # color = "navy" 26 | 27 | # Background gradient. 28 | gradient_start = "DarkGreen" 29 | gradient_end = "ForestGreen" 30 | 31 | # Background image. 32 | # image = "image.jpg" # Name of image in `static/img/`. 33 | # image_darken = 0.6 # Darken the image? Range 0-1 where 0 is transparent and 1 is opaque. 34 | 35 | # Text color (true=light or false=dark). 36 | text_color_light = true 37 | 38 | [design.spacing] 39 | # Customize the section spacing. Order is top, right, bottom, left. 40 | padding = ["20px", "0", "20px", "0"] 41 | 42 | [advanced] 43 | # Custom CSS. 44 | css_style = "" 45 | 46 | # CSS class. 47 | css_class = "" 48 | +++ 49 | 50 | Welcome to the **Academic Kickstart** template! 51 | 52 | Follow our [Getting Started](https://sourcethemes.com/academic/docs/get-started/) and [Page Builder](https://sourcethemes.com/academic/docs/widgets/) guides to easily personalize the template and then [add your own content](https://sourcethemes.com/academic/docs/managing-content/). 53 | 54 | For inspiration, check out [the Markdown files](https://sourcethemes.com/academic/docs/install/#demo-content) which power the [personal demo](https://academic-demo.netlify.com/). The easiest way to publish your new site to the internet is with [Netlify](https://sourcethemes.com/academic/docs/deployment/). 55 | 56 | - [View the documentation](https://sourcethemes.com/academic/docs/) 57 | - [Ask a question](http://discuss.gohugo.io/) 58 | - [Request a feature or report a bug](https://github.com/gcushen/hugo-academic/issues) 59 | - Updating? View the [Update Guide](https://sourcethemes.com/academic/docs/update/) and [Release Notes](https://sourcethemes.com/academic/updates/) 60 | - Support development of Academic: 61 | - [Donate a coffee](https://paypal.me/cushen) 62 | - [Become a backer on Patreon](https://www.patreon.com/cushen) 63 | - [Decorate your laptop or journal with an Academic sticker](https://www.redbubble.com/people/neutreno/works/34387919-academic) 64 | - [Wear the T-shirt](https://academic.threadless.com/) 65 | 66 | {{% alert note %}} 67 | This homepage section is an example of adding [elements](https://sourcethemes.com/academic/docs/writing-markdown-latex/) to the [*Blank* widget](https://sourcethemes.com/academic/docs/widgets/). 68 | 69 | Backgrounds can be applied to any section. Here, the *background* option is set give a *color gradient*. 70 | 71 | **To remove this section, delete `content/home/demo.md`.** 72 | {{% /alert %}} 73 | -------------------------------------------------------------------------------- /content/post/practice_questions.md: -------------------------------------------------------------------------------- 1 | +++ 2 | title = "ML Practice Questions" 3 | date = 2019-09-27 #T13:23:10+01:00 4 | draft = false 5 | #tags = ["Getting started"] 6 | categories = [] 7 | #markup: mmark 8 | summary = "Practice Questions on Machine Learning" 9 | #disable_comments: true 10 | +++ 11 | 12 | ## Linear Regression 13 | 14 | 1. What is linear regression? 15 | * We model a continuous target variable as a linear combination of predictor variables. The "true" relationship need not be linear, we just treat the relationship as linear to be able to predict stuff. 16 | 17 | 2. How do you train a linear regression model? 18 | * After assuming the linear relationship, mx+b, you define a loss funciton as the mean squared error. Then you initilize the parameters, compute the loss, and iteratively take steps in the weight space in the direction which reduces the loss the most. 19 | * This direction is defined by the gradient. The gradient points uphill in the steapest direction, so to minimize you go the opposite direction, which is downhill. 20 | * The gradient is a derivative in multidimensional space. For $mx+b$, there are two parameters so the gradient is a 2d vector. 21 | * You update the parameters by a step size times the gradient $w^{t} = w^{t-1} - \eta \nabla_{w}L$. 22 | 23 | 3. How do you change linear regression if the target variable is binary? 24 | * Since the target variable is 0 or 1, you squeeze the prediction to the range 0-1. You squeeze by applying using the logistic function $ \frac{1}{1+d^{-1}} $ to the $w^{T}x + b$ term. Finally, you assume this value represents the probability of the target variable being 1. This is called logistic regression. 25 | 26 | 4. What else changes in logistic regression? The training? MSE? 27 | * Since we are now modeling the probability of $y=1$ as a binomial variable where the probability parameter is a linear combination of the predictor variables. When we maximize joint probability, over all samples, we compute the log-loss the function look different from mse: 28 | 29 | $$\sum_i y_i \log \sigma(w \cdot x+b)+(1-y_i)\log(1-\sigma(w \cdot x+b))$$ 30 | 31 | 5. Why don't we use the MSE for the loss for classification, practically speaking? 32 | * The "logistic" loss is convex and easier to compute. We prefer to minimize _accuracy loss_ (i.e. "0/1" loss), but this loss is non-convex and thus hard to minimize. 33 | * Since we can't use the _accuracy loss_, which we really want, we must use a _surrogate_ loss function which approximates the 1/0 loss as closely as possible. The logistic loss is closer to the _accuracy loss_ than the MSE, as shown below [^1]: 34 | 35 | ![mse v logistic loss](/img/mse_v_logistic_loss.png) 36 | 37 | Finally, note that logistic regression uses the logistic loss; training a logistic regression with the MSE loss wouldn't be logistic regression anymore. It would be just a classifier. 38 | 39 | ## PCA and Decision Trees 40 | 1. What is PCA? 41 | * PCA takes a data matrix and creates a new set of variables that are linear combinations of the original variables. The new variables, or _principal components_ are constructed in such a way that the first principal component has the largest variance possible, followed by the second principal component, and so on. The principal components are all orthogonal to each other. The principal components can be computed by finding the eigenvectors of the data covariance matrix. PCA is mostly useful for dimensionality reduction; reducing the number of variables to a feasible number, while maintaining as much variation in the data as possible. 42 | * Finding the new axes is the key part of PCA. Once you find the new axes, you can convert new data into the new axes. 43 | * In T-SNE, we apply the intuition that points close together in 3d should be close in 2d also. We build a neural network to map higher-dimensional points to lower-dimensional space. Minimize the KL divergence of probabilities of points being neighbors. 44 | 45 | 2. What are decision trees? 46 | * Decision trees split a feature space into non-overlapping regions, then predicts the average of target values in that region. Decision treess split the feature space by iteratively choosing a variable and value to split the predictions on; the variable and value are chosen according to whichever combination reduces an “impurity measure” the most at each step. Impurity is usually measure as entropy; the extent to which the two groups from the split have mostly one class, instead of a mix of two. The algorithm will continue to split the space until a stopping condition is reached, such as there being a small enough number of observations in the resulting leaves, or the depth of the tree reaches a maximum. 47 | 48 | ## SVM, Adaboost, and gradient boosting 49 | 1. What are SVMs? 50 | * SVMs are binary classifiers which try to find a hyperplane that splits the two classes in their feature space. Observations lying above the hyperplane are predicted as 1, observations below are predicted as -1. If the data are indeed linearly separable, then the hyperplane is chosen by maximizing the _margin_; the distance of the closest point to the hyperplane. No data points fall within the _margin_. 51 | 52 | * The data are often not linearly separable, so the model is modified to allow observations to violate the margin and the hyperplane. The number and severity of violations allowed is controlled with a tuning parameter, which acts as a "budget" for the amount of violations. When the budget is small, the model has low bias and high variance. High budget means high bias but low variance. 53 | 54 | * For non-linear class boundaries, we could expand the feature space using functions, like polynomials, of the predictors. In the enlarged feature space, the decision boundary that results is linear. However, with many new features, the computations could be unmanageable. The support vector machine allows us to enlarge the feature space in a way that leads to efficient computations. 55 | 56 | * The SVM is an extension of the support vector classifier that results from enlarging the feature space in a specific way, using _kernels_. Kernels are a generalization of the inner product, and they measure the similarity of two observations. Some popular kernels are the polynomial and radial kernels. Kernels are appealing because taking a kernel in the orginal feature space is less expensive than computing an inner product in the expanded feature space. 57 | 58 | 2. What is Adaboost? 59 | * Adaboost combine multiple 'base' classifiers to form a model whose performance can be significantly better than that of any of the base classifiers. Overall performance can be good even when performance of each base classifier is only slightly better than random guessing. The base classifiers are trained in sequence, using weighted versions of the data with higher weight going to observations that were classified wrong in the previous classifier in the sequence. Once all the classifiers have been trained, their predictions are combined through a weighted majority voting scheme. 60 | 61 | ## Transformers 62 | 1. What is attention? 63 | * In sequential models like recurrent neural networks, the final hidden state may have forgotten information from earlier in the sequence. Therefore, the model takes a weighted sum of the previous hidden states. This gives a vector called the _context vector_. 64 | 2. What are transformers? 65 | * Transformers are distinct from RNNs or CNNs. They rely on attention mechanims, hence the founding paper's title _Attention Is All You Need_. Transformers compute attention differently from recurrent neural networks. Transformers represent each word as a linear combination of other word vectors in a value matrix $V$, where the weights are computed based on similarity to the other words. 66 | 67 | [^1]: [What are the main reasons not to use MSE as a cost functoin for Logistic Regression?](https://www.quora.com/What-are-the-main-reasons-not-to-use-MSE-as-a-cost-function-for-Logistic-Regression) -------------------------------------------------------------------------------- /config/_default/params.toml: -------------------------------------------------------------------------------- 1 | # SITE SETUP 2 | # Documentation: https://sourcethemes.com/academic/ 3 | 4 | ############################ 5 | ## Theme 6 | ############################ 7 | 8 | # Choose a theme. 9 | # Latest themes (may require updating): https://sourcethemes.com/academic/themes/ 10 | # Browse built-in themes in `themes/academic/data/themes/` 11 | # Browse user installed themes in `data/themes/` 12 | theme = "minimal" 13 | 14 | #color_theme = "minimal" # I think it's theme, not color_theme 15 | 16 | # Enable users to switch between day and night mode? 17 | day_night = true 18 | 19 | # Override the theme's font set (optional). 20 | # Latest font sets (may require updating): https://sourcethemes.com/academic/themes/ 21 | # Browse built-in font sets in `themes/academic/data/fonts/` 22 | # Browse user installed font sets in `data/fonts/` 23 | font = "" 24 | 25 | # Choose a font size. 26 | # Sizes: XS (extra small), S (small), M (medium), L (large - DEFAULT), XL (extra large) 27 | font_size = "L" 28 | 29 | ############################ 30 | ## Basic Info 31 | ############################ 32 | 33 | # Website type 34 | # Improve how search engines understand your site. 35 | # For personal sites, choose "Person". 36 | # For organizations and projects, choose from https://schema.org/Organization#subtypes 37 | # E.g. Person, Organization, LocalBusiness, Project, EducationalOrganization 38 | site_type = "Person" 39 | 40 | # Local business type (optional) 41 | # If you entered "LocalBusiness" above, choose the type of business from https://schema.org/LocalBusiness#subtypes 42 | local_business_type = "" 43 | 44 | # Organization name (optional) 45 | # Enter an organization or project name. Defaults to the site title from `config.toml`. 46 | org_name = "" 47 | 48 | # Description for social sharing and search engines. If undefined, superuser role is used in place. 49 | description = "" 50 | 51 | # Display a logo in navigation bar rather than title (optional). 52 | # To enable, place an image in `static/img/` and reference its filename below. To disable, set the value to "". 53 | logo = "" 54 | 55 | ############################ 56 | ## Site Features 57 | ############################ 58 | 59 | # Enable source code highlighting? true/false 60 | # Documentation: https://sourcethemes.com/academic/docs/writing-markdown-latex/#highlighting-options 61 | highlight = true 62 | highlight_languages = ["r"] # Add support for highlighting additional languages 63 | # highlight_style = "github" # For supported styles, see https://cdnjs.com/libraries/highlight.js/ 64 | 65 | # Enable LaTeX math rendering? true/false 66 | # If false, you can enable math on a per page basis as needed. 67 | math = true 68 | 69 | # Enable diagram rendering? true/false 70 | # If false, you can enable diagrams on a per page basis as needed. 71 | diagram = false 72 | 73 | # Privacy pack 74 | # Show a cookie consent message to visitors 75 | # Anonymize IP in Google Analytics (if enabled) 76 | privacy_pack = false 77 | 78 | # Enable visitors to edit pages? 79 | # `repo` defines the repository URL. `editable` defines which page types can be edited. 80 | edit_page = {repo_url = "https://github.com/gcushen/hugo-academic", repo_branch = "master", editable = {docs = true, page = false, post = false}} 81 | 82 | ############################ 83 | ## Contact details 84 | ## 85 | ## These details power the Contact widget (if enabled). 86 | ## Additionally, for organizations, these details may be used to enrich search engine results. 87 | ############################ 88 | 89 | # Enter contact details (optional). To hide a field, clear it to "". 90 | email = "test@example.org" 91 | phone = "888 888 88 88" 92 | 93 | # Address 94 | # For country_code, use the 2-letter ISO code (see https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2 ) 95 | address = {street = "450 Serra Mall", city = "Stanford", region = "CA", postcode = "94305", country = "United States", country_code = "US"} 96 | 97 | # Geographic coordinates 98 | # To get your coordinates, right-click on Google Maps and choose "What's here?". The coords will show up at the bottom. 99 | coordinates = {latitude = "37.4275", longitude = "-122.1697"} 100 | 101 | # Directions for visitors to locate you. 102 | directions = "Enter Building 1 and take the stairs to Office 200 on Floor 2" 103 | 104 | # Office hours 105 | # A list of your office hours. To remove, set to an empty list `[]`. 106 | office_hours = ["Monday 10:00 to 13:00", "Wednesday 09:00 to 10:00"] 107 | 108 | # Enter an optional link for booking appointments (e.g. calendly.com). 109 | appointment_url = "https://calendly.com" 110 | 111 | # Contact links 112 | # Set to `[]` to disable, or comment out unwanted lines with a hash `#`. 113 | contact_links = [ 114 | {icon = "twitter", icon_pack = "fab", name = "DM Me", link = "https://twitter.com/Twitter"}, 115 | {icon = "skype", icon_pack = "fab", name = "Skype Me", link = "skype:echo123?call"}, 116 | {icon = "keybase", icon_pack = "fab", name = "Chat on Keybase", link = "https://keybase.io/"}, 117 | {icon = "comments", icon_pack = "fas", name = "Discuss on Forum", link = "https://discourse.gohugo.io"}, 118 | # {icon = "telegram", icon_pack = "fab", name = "Telegram Me", link = "https://telegram.me/@Telegram"}, 119 | ] 120 | 121 | ############################ 122 | ## Social 123 | ############################ 124 | 125 | # Default image for social sharing and search engines. Place image in `static/img/` folder and specify image name here. 126 | sharing_image = "" 127 | 128 | # Twitter username (without @). Used when a visitor shares your site on Twitter. 129 | twitter = "" 130 | 131 | ############################ 132 | ## Regional Settings 133 | ############################ 134 | 135 | # Date and time format (refer to https://sourcethemes.com/academic/docs/customization/#date-format ) 136 | # Examples: "Mon, Jan 2, 2006" or "2006-01-02" 137 | date_format = "Jan 2, 2006" 138 | # Examples: "3:04 pm" or "15:04" 139 | time_format = "3:04 PM" 140 | 141 | # List delimiter (separates parts of an address) 142 | # Popular separators are ", " (comma), "
" (new line), " " (space) 143 | list_delimiter = ", " 144 | 145 | ############################ 146 | ## Advanced 147 | ############################ 148 | 149 | # Get user avatars from Gravatar.com? (true/false) 150 | gravatar = false 151 | 152 | # Align the main menu to the right of the page? (true/false) 153 | menu_align_right = false 154 | 155 | # Show estimated reading time for posts? (true/false) 156 | reading_time = true 157 | 158 | # Display next/previous section pager? (true/false) 159 | section_pager = false 160 | docs_section_pager = true # Display pager in Docs layout (e.g. tutorials)? 161 | 162 | # Enable in-built social sharing buttons? (true/false) 163 | sharing = true 164 | 165 | # Load CSS and JS plugins 166 | # E.g. To load `/assets/css/custom.css`, set `plugins_css = ["custom"]`. 167 | # E.g. To load `/assets/js/custom.js`, set `plugins_js = ["custom"]`. 168 | plugins_css = [] 169 | plugins_js = [] 170 | 171 | # Configuration of publication pages. 172 | [publications] 173 | # Date format (refer to https://sourcethemes.com/academic/docs/customization/#date-format ) 174 | # Examples: "Mon, Jan 2, 2006" or "2006-01-02" 175 | date_format = "January 2006" 176 | 177 | # Citation style ("apa" or "mla") 178 | citation_style = "apa" 179 | 180 | # Configuration of project pages. 181 | [projects] 182 | # Views for associated content. 183 | # 1: List 184 | # 2: Compact 185 | # 3: Card 186 | # 4: Citation (publications only) 187 | post_view = 2 188 | publication_view = 2 189 | talk_view = 2 190 | 191 | ############################ 192 | ## Comments 193 | ############################ 194 | [comments] 195 | # Comment provider: 196 | # 0: Disabled 197 | # 1: Disqus (https://disqus.com) 198 | # 2: Commento (https://commento.io) 199 | engine = 0 200 | 201 | # Which page types are commentable? 202 | commentable = {page = true, post = true, docs = true, project = true, publication = true, talk = true} 203 | 204 | # Configuration of Disqus. 205 | [comments.disqus] 206 | shortname = "" # Paste the shortname from your Disqus dashboard. 207 | show_count = true # Show comment count in page header? (true/false) 208 | 209 | ############################ 210 | ## Search 211 | ############################ 212 | [search] 213 | # Search provider: 214 | # 0: No search engine 215 | # 1: Academic (built-in) 216 | # 2: Algolia (https://www.algolia.com) 217 | engine = 1 218 | 219 | # Configuration of Algolia search engine. 220 | # Paste the values from your Algolia dashboard. 221 | [search.algolia] 222 | app_id = "" 223 | api_key = "" 224 | index_name = "" 225 | show_logo = false 226 | 227 | ############################ 228 | ## Maps 229 | ############################ 230 | [map] 231 | # To show your address on a map in the Contact widget, enter your latitude and longitude (above) 232 | # and choose a map provider below. 233 | # 234 | # To use Google Maps, set `engine` to 1 and enter your API key that can be obtained here: 235 | # https://developers.google.com/maps/documentation/javascript/get-api-key 236 | # To use OpenStreetMap tiles, set `engine` to 2. 237 | # To use OpenStreetMap on a high traffic site, set `engine` to 3 and enter your API key that can be obtained here: 238 | # https://www.mapbox.com/studio/account/tokens 239 | # 240 | # Map provider: 241 | # 0: No map 242 | # 1: Google Maps 243 | # 2: OpenStreetMap (Mapnik) 244 | # 3: OpenStreetMap (Mapbox) 245 | engine = 2 246 | api_key = "" 247 | zoom = 15 248 | -------------------------------------------------------------------------------- /content/post/linear-regression-basics.md: -------------------------------------------------------------------------------- 1 | +++ 2 | title = "Linear Regression Regfresher" 3 | date = 2018-04-28 #T13:23:10+01:00 4 | draft = false 5 | #tags = ["Getting started"] 6 | categories = [] 7 | #markup: mmark 8 | summary = "Refresher on the basics of linear regression" 9 | #disable_comments: true 10 | +++ 11 | 12 | This article introduces linear regression from an elementary point of view. Returning to the theoretical foundations helps since linear regression serves as a building block for more advanced techniques. 13 | 14 | This tutorial relies heavily on Cosma Shalizi's lectures on modern regression[^1], which are insightful, witty, and a delight to read. I recommend anyone who is curious to check them out. 15 | 16 | ## Predicting a Random Variable 17 | 18 | Suppose we have a variable $Y$ that we want to predict. $Y$ is a random variable whose value we don't know, but we must make a guess of it. The only information that we have is $Y$'s distribution - we have a sense of which values $Y$ are highly probable and which are less probable. Using the distribution, we can use a method that gives us the guess which is 'best' in some sense, even though we do not know the actual value of $Y$. 19 | 20 | Let's call our prediction $m$. I use lower case because $m$ is a value - it's not a random variable. Our goal is to choose $m$ that is "best". 21 | 22 | ### Mean Sqauared Error 23 | 24 | Let's define what we mean by 'best.' Ideally, we want our prediction $m$ to be close to the true value $Y$. Let's choose to measure how good our prediction is with the squared error $(Y - m)^2$. Other measures could be used, but using the squared error is traditional so we use it here. 25 | 26 | [comment]: <> (This is a comment, it will not be included in the HTML file. So you can't use inspect to see it) 27 | 28 | Also, the squared error ignores the direction of the error - if we always overpredict, or always underpredict, the metric will not distinguish the two because of the 'squared' part. This feature is nice because errors of opposite signs can cancel each other out, leading to an overconfident picture of our estimate. 29 | 30 | Since $Y$ is a random variable, so is the squared error. Therefore let's look at the value 31 | 32 | $$\mathbb{E}\left[(Y - m)^2\right]$$ 33 | 34 | We call this value the *mean squared error*, $MSE(m)$, and we want to choose our prediction $m$ which minimizes it. Note that the $MSE$ is a function of $m$; by choosing the 'best' prediction, we will obtain the smallest $MSE$ possible. 35 | 36 | Now we must choose a value for our prediction $m$. If we don't know anything about $Y$ except its distribution, what should our one-number guess be? 37 | 38 | ### Our prediction $m$ 39 | 40 | Intuitively, we might choose our prediction value to be the median of the distribution; it's a valid summary of the entire distribution, and maybe even a good summary. Alternatively, we could choose the mean because it's also a good summary. We would probably avoid setting $m$ to the maximum or minimum value because they are poor representations of the distribution as a whole. 41 | 42 | To help our analysis, let's expand the $MSE$ into two terms: 43 | 44 | $$MSE(m) = \mathbb{E}[(Y - m)^2] = (\mathbb{E}[Y-m])^2 + Var[Y-m]$$ 45 | 46 | This is the *bias-variance* decomposition, one of the most important ideas in statistics. The first term, $(\mathbb{E}[Y-m])^2$, is the **bias** squared and it measures the expected error of using $m$ to estimate $Y$. Ideally, our estimate $m$ would have zero bias - the prediction expected to equal the true value of $Y$, but this won't always be true. 47 | 48 | The **variance** term is $Var[Y-m] = Var[Y]$, which measures how spread out the distribution of $Y$ is. If $Y$ usually takes on values centered near one specific value, the variance will be low. It says no matter what we used as an estimate, if $Y$ is narrowly concentrated then our prediction will be better overall. The variance term does not depend on our estimate $m$, so it won't help us minimize the $MSE$. If we make some predictions, then increase $Var[Y]$, then some of $Y$'s distribution will move away from the predictions resulting in increased error. All else equal, having small $Var[Y]$ helps lower our mean squared error. 49 | 50 | To choose our prediction, let's see which $m$ minimizes the squared error $MSE$. As we normally do in calculus, to find a minimum we will find the derivative of $MSE(m)$, set it to zero, then solve for $m$: 51 | 52 | $$ 53 | \begin{align} 54 | \frac{dMSE(m)}{dm} &= \frac{d}{dm} \left (Var[Y]+(\mathbb{E}[Y]-m)^2 \right) \\ 55 | &= 2 \left( \mathbb{E}[Y]-m \right) \left(\frac{d\mathbb{E}[Y]}{dm} - \frac{dm}{dm} \right) \\ 56 | &= -2 \left( \mathbb{E}[Y]-m \right) \\ 57 | &= 0 58 | \end{align} 59 | $$ 60 | 61 | ### Result 62 | 63 | Setting $-2 \left( \mathbb{E}[Y]-m \right)=0$ gives the result: $\mathbb{E}[Y] = \mu$, where we have replaced $m$ with its optimal value denoted $\mu$. This says that to minimize mean squared error, we should always predict the mean of $Y$. This of course assumes we can find the mean of $Y$, that is, we assume we know the distribution of $Y$ (or least its moments). 64 | 65 | This sounds reasonable - if we want to predict a value of $Y$ and we have its distribution but no other information, we should predict the mean of $Y$. Since the mean is a common summary statistic of $Y$, it's not surprising that using it as a prediction is optimal. 66 | 67 | ## Predicting One Variable from Another Variable 68 | 69 | Let's make the situation more interesting. Suppose we also have another random variable $X$ which is related to $Y$, and therefore can help us predict $Y$. That is, $X$ are $Y$ may be dependent: $p(Y \| X) \neq p(Y)$. Therefore, knowing $X$ could help us predict $Y$. Therefore, our prediction will be $m(X)$ where $m$ is now a function of $X$. 70 | 71 | Now the $MSE$ is given by $\mathbb{E}[(Y - m(X))^2]$, and our goal, again, is to find the estimate $m(X)$ that minimizes the $MSE$. We can use the result we found previously by noticing that now $Y$ is distributed as $p(Y \| X)$ instead of $p(Y)$ as used before. Therefore, instead of predicting $m=\mathbb{E}[Y]$, we simply predict $\mu(x) = \mathbb{E}[Y \| X=x]$. This is still the mean of $Y$, but now under a new distribution of $Y$. The prediction $m$ changed to $\mu(x)$ to recognize that our prediction now uses the known $x$ value to help predict $Y$, so the prediction is a function of $x$. 72 | 73 | As in the one-variable case, if we knew the distribution $p(Y \| X=x)$, we would just take the mean and use that as our prediction. However, we don't know the whole distribution $p(Y \| X=x)$; moreover, all we need is its mean $=\mathbb{E}[Y \| X=x]=\mu(x)$, which we call the **regression function**. 74 | 75 | Note that for any two random variables, which have a joint distribution, the conditional expectation of one on the other can, in general, be a function of the second variable. For example, if $X_1$ and $X_2$ are bivariate normal, then $\mathbb{E}[X_1 \| X_2 = x_2]=\mu_1+\rho\frac{\sigma_1}{\sigma_2}(x_2-\mu_2) = f(x_2)$; the conditional expectation is a function of $x_2$. 76 | 77 | ### An Approximation 78 | 79 | The actual relationship between $X$ and $Y$ could be very complicated, that is, the regression function $\mu(x)= \mathbb{E}[Y \| X=x]$ is unknown and potentially complex. 80 | 81 | Since we don't know this complex function $\mu(x)$, which may not even have a nice mathematical form, we can choose to approximate it as a linear function of $X$: 82 | 83 | $$\mu(x) = \mathbb{E}[Y|X=x] = \beta_0 + \beta_1x$$ 84 | 85 | It is usually a vast generalization to say that $\mathbb{E}[Y \| X=x]$ is a linear function of $X$. However, using such simple approximations allows for a starting point and often captures enough of the relationship to make useful models. It also does not assume the true relationship is linear - it is just approximating the true relationship with a linear one. 86 | 87 | ### Deriving the coefficients 88 | 89 | Now the regression function $\mu(x)$ is defined by the coefficients $\beta_0$ and $\beta_1$. We will adjust the coefficients to find the function $\mu(x)$ which minimizes the $MSE$, which is a function of the coefficients: 90 | 91 | $$ 92 | \begin{aligned}[t] 93 | MSE(\beta_0, \beta_1) &= \mathbb{E}\left[\left(Y - \left(\beta_0 + \beta_1X \right)\right)^2\right] \\ 94 | &= \mathbb{E}[Y^2] - 2\beta_0\mathbb{E}[Y] - 2\beta_1\mathbb{E}[XY] + \mathbb{E}\left[\left(\beta_0+\beta_1X\right)^2 \right] \\ 95 | &= \mathbb{E}[Y^2] - 2\beta_0\mathbb{E}[Y] - 2\beta_1\left(Cov[X,Y] + \mathbb{E}[X]\mathbb{E}[Y]\right) + \beta_0^2 + 2\beta_1\mathbb{E}[X] + \beta_1^2\mathbb{E}[X^2] \\ 96 | &= \mathbb{E}[Y^2] - 2\beta_0\mathbb{E}[Y] - 2\beta_1(Cov[X,Y] + 2\beta_1\mathbb{E}[X]\mathbb{E}[Y] + \beta_0^2 + 2\beta_0\beta_1\mathbb{E}[X] + \beta_1^2Var[X] + \beta_1^2(\mathbb{E}[X])^2 97 | \end{aligned} 98 | $$ 99 | 100 | As before, we find the values of $\beta_0$ and $\beta_1$ which minimize the $MSE(\beta_0, \beta_1)$ by setting the derivatives equal to zero and solving for the betas: 101 | 102 | 103 | $$\frac{d(MSE)}{d\beta_0} = 0 = -2\mathbb{E}[Y] + 2\beta_0 + 2\beta_1\mathbb{E}[X]$$ 104 | 105 | $$\frac{d(MSE)}{d\beta_1} = 0 = -2Cov[X,Y] - 2\mathbb{E}[X]\mathbb{E}[Y] + 2\beta_0\mathbb{E}[X] + 2\beta_1Var[X] + 2\beta_1(\mathbb{E}[X])^2$$ 106 | 107 | These are two equations with two unknowns, so they can be solved for $\beta_0$ and $\beta_1$. After some algebra, the result is: 108 | 109 | $$\beta_0 = \mathbb{E}[Y] - \beta_1\mathbb{E}[X]$$ 110 | $$\beta_1 = \frac{Cov(X,Y)}{Var(X)}$$ 111 | 112 | Note that if $X$ and $Y$ are centered, that is they have mean zero, then the intercept $\beta_0$ is zero and line goes through the origin. For $\beta_1$, the more that $X$ and $Y$ vary together (tend to move when the other one moves), the steeper the line. As the variance of $X$ increases, $\beta_1$ goes to zero, regardless of its covariance with $Y$. 113 | 114 | The result is the line $\beta_0 + \beta_1x$ is the **optimal regression line** or the **optimal linear predictor**. This line, defined by $\beta_0$ and $\beta_1$ is the **best** line we can choose that approximates the true regression function, according to the $MSE$. 115 | 116 | ### Take-aways 117 | 1. We did not assume that the relationship between $X$ and $Y$ is actually linear. What we did was derive the optimal linear approximation to the true relationship, whatever that relationship might be (e.g. $\mu(x)=e^x$ or $\sin x$). This approximation is not necessarily a good one, and it may well be an awful one. 118 | 119 | 2. We did not assume anything about the marginal distributions of $X$ or $Y$ (just that the expectations exist). The optimal linear predictor doesn't require any stronger assumptions; $X$ and $Y$ can be any shaped distribution and these coefficents would still give the optimal regression line. 120 | 121 | 3. We made no assumptions on the specific distributions of the variables, $p(X)$ and $p(Y)$, nor about the joint distribution $p(X,Y)$. These assumptions can help make stronger inferences later on, but for now they are unnecessary for our purposes of predicting $Y$ using $X$ under a linear approximation. 122 | 123 | 4. In general, changing the distribution of $X$ will change the optimal regression line because both the $Cov(X,Y)$ and $Var(X)$ will change (and usually not cancel out each other's change). 124 | 125 | 5. There is no assumption that X *causes* Y; they just have non-zero covariance so one can be useful for predicting the other. 126 | 127 | Note that to find the *optimal linear predictor*, we will need some information on our random variables; specifically, $\mathbb{E}[X]$, $\mathbb{E}[Y]$, $Cov[X,Y], Var[X]$. Since we don't know these a priori, we will need to estimate them from noisy data, making this task a *statistical* problem. 128 | 129 | This thoery of optimal linear predictor will be used later when we introduce a more specific type of linear predictor - the *simple linear regression model*. 130 | 131 | ### Model specification 132 | In a statistical model, we treat the variables as random varianbles. The *specification* of a statistical model says what the random variables are, and lays down more or less narrow restrictions on their distributions and how they relate to each other. We will discuss the specification of the *simple linear regression model* next. 133 | 134 | ### Simple Linear Regression[^2] 135 | 136 | Now let's introduce the **simple linear regression model**, the "most basic of models that's actually useful for anything." Suppose we're trying to predict $Y$ from $X$. Here are the mode's assumptions: 137 | 138 | #### Assumptions 139 | 1. The distribution of $X$ is arbitrary, possibly even deterministic. 140 | 2. If $X=x$, then $Y=\beta_0 + \beta_1x + \epsilon$ for some constants ("cefficient", "parameters") $\beta_0$ and $\beta_1$, and some random noise variable $\epsilon$ (random variable) 141 | 3. $\epsilon$ is a random variable with unspecified distribution, but has $\mathbb{E}\left[\epsilon \| X=x\right]=0$ (no matter what $x$ is), $Var\left[\epsilon \| X=x\right]=\sigma^2$ (no matter what $x$ is). 142 | 4. $\epsilon$ is uncorrelated across observations 143 | 144 | All of these assumptions will need to be checked if the model is to be used. It is possible to use stronger assumptions and draw more powerful inferences, but these assumptions are strong enough already to start on inferece. 145 | 146 | ### Note on p-values 147 | With the added assumption that $\epsilon$ follows a normal distribution, we can derive the sampling distributions of the coefficients. They come out to be Gaussian and we can easily measure their p-value to test the null hypothesis that the coefficients are 0. This step is out-of-scope for this tutorial but it's worth mentioning here. 148 | 149 | P-values on the coefficients are tricky to interpret and many people have trouble with them. The term **statistical significance** has a specific definition and is not the same, nor does it imply, scientific or practical significance. Shalizi says it best: 150 | 151 | > Statistical significance is a weird mixture of how big the coefficient is, how big a sample we've got, how much noise there is around the regression line, and how spread out the data is along the $x$ axis. This has so little to do with "significance" in ordinary language that it's pretty unfortunate we're stuck with the word; if the Ancestors had decided to say "statistically detectable" or "statistically distinguishable from 0", we might have avoided a lot of confusion.[^3] 152 | 153 | When we reject the hypothesis that $\beta_1=0$, what we're saying is "It's really implausibly hard to fit this data with a flat line, as opposed to one with a slope." It prefer the term **statistically detectable** since it makes it clear that we can detect even very small coefficients. 154 | 155 | ## Summary 156 | 157 | In conclusion, this tutorial showed that the optimal regression line of $Y$ on $X$ is defined by the coefficients $\beta_0$ and $\beta_1$ that we derived. These coefficients made no assumptions on the distributions of $X$ and $Y$, nor did we assume that the true relationship between $X$ and $Y$ is linear -- we only used the linear function as an approximation to the true relationship. The next steps would be to formalize the simple linear model by adding an error term, estimate the coefficeints from data and derive their distributions, expecations, and variances, assume that the error terms is normally distibuted, and show this assumption implies the least squares estimate gives the same result as the maximum likelihood estimate. If you want to find more information, or just want a good read on linear regression, I recommend continuing with Shalizi's [lectures](http://www.stat.cmu.edu/~cshalizi/mreg/15/). Thanks for reading. 158 | 159 | 160 | ### References: 161 | 162 | [^1]: [Lecture 1: Optimal Prediction](http://www.stat.cmu.edu/~cshalizi/mreg/15/lectures/01/lecture-01.pdf) 163 | 164 | [^2]: [Lecture 3: Introducing Statistical Modeling](http://www.stat.cmu.edu/~cshalizi/mreg/15/lectures/03/lecture-03.pdf) 165 | 166 | [^3]: [Lecture 8: Inference on Parameters](http://www.stat.cmu.edu/~cshalizi/mreg/15/lectures/08/lecture-08.pdf) --------------------------------------------------------------------------------