├── .github ├── ISSUE_TEMPLATE │ ├── bug_report.md │ └── feature-requests-and-questions.md └── workflows │ ├── build-and-release.yml │ └── pr-validation.yml ├── .gitignore ├── .prettierrc ├── .vscode └── launch.json ├── CHANGELOG.md ├── CONTRIBUTING.md ├── LICENSE ├── README.md ├── README_zh.md ├── SECURITY.md ├── docs ├── architecture-design.md ├── customization-guide.md ├── developer-guide.md ├── headless-usage-guide.md ├── how-to-customize-refresh-schedule.md ├── how-to-deploy-to-other-hosts.md ├── media │ └── osmosfeed-square-badge.svg └── osmosfeed-yaml-reference.md ├── package-lock.json ├── package.json └── packages ├── cli ├── .gitignore ├── main.js ├── package.json ├── scripts │ ├── build.js │ └── jsconfig.json ├── src │ ├── lib │ │ ├── __tests__ │ │ │ ├── normalize-feed.test.ts │ │ │ ├── normalize-url.test.ts │ │ │ └── time.test.ts │ │ ├── compile-templates.ts │ │ ├── copy-static.ts │ │ ├── discover-files.ts │ │ ├── enrich.ts │ │ ├── get-cache.ts │ │ ├── get-config.ts │ │ ├── get-copy-static-plan.ts │ │ ├── get-snippets.ts │ │ ├── get-template-data.ts │ │ ├── normalize-feed.ts │ │ ├── normalize-url.ts │ │ ├── path-constants.ts │ │ ├── render-atom.ts │ │ ├── render-user-snippets.ts │ │ ├── set-cache.ts │ │ ├── time.ts │ │ └── write-files.ts │ ├── main.ts │ ├── system-static │ │ ├── favicon.ico │ │ ├── index.css │ │ └── index.js │ ├── system-templates │ │ └── index.hbs │ └── utils │ │ ├── download.ts │ │ ├── ensure-string-content.ts │ │ ├── entry-dir.ts │ │ ├── file-request.ts │ │ ├── fs.ts │ │ ├── get-first-non-empty-array.ts │ │ ├── get-first-non-null-item.ts │ │ ├── html-to-text.ts │ │ ├── is-not-null.ts │ │ ├── sanitize-html.ts │ │ ├── trim-with-threshold.ts │ │ ├── url.ts │ │ └── version.ts └── tsconfig.json ├── sandbox ├── .gitignore ├── osmosfeed.yaml └── package.json └── test ├── feed.xml ├── fixture ├── runner.js └── utils.js ├── jsconfig.json ├── mock-server ├── feeds │ └── rss-2-empty.xml └── serve-static.js ├── package.json ├── scenarios ├── default-empty │ ├── .gitignore │ ├── osmosfeed.yaml │ ├── package.json │ └── snapshots │ │ └── cache.json ├── with-user-assets │ ├── .gitignore │ ├── osmosfeed.yaml │ ├── package.json │ ├── snapshots │ │ └── cache.json │ └── static │ │ ├── favicon.ico │ │ ├── index.js │ │ └── robots.txt ├── with-user-snippets │ ├── .gitignore │ ├── includes │ │ ├── after-body-begin.html │ │ ├── before-body-end.html │ │ └── before-head-end.html │ ├── osmosfeed.yaml │ ├── package.json │ └── snapshots │ │ └── cache.json └── with-user-templates │ ├── .gitignore │ ├── includes │ └── index.hbs │ ├── osmosfeed.yaml │ ├── package.json │ └── snapshots │ └── cache.json └── test.js /.github/ISSUE_TEMPLATE/bug_report.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Bug report 3 | about: Create a report to help us improve 4 | title: '' 5 | labels: '' 6 | assignees: '' 7 | 8 | --- 9 | 10 | **Describe the bug** 11 | A clear and concise description of what the bug is. 12 | 13 | **Bug area (choose one)** 14 | - [ ] Site builder: anything broken during in the GitHub action 15 | - [ ] Reader UI: anything broken while you are reading the feed 16 | 17 | **To reproduce the bug** 18 | - Share your repo (if visible to public) 19 | - Share the GitHub Action history (if visible to public) 20 | - Share the reader UI site (if visible to public) 21 | 22 | **Expected behavior** 23 | A clear and concise description of what you expected to happen. 24 | 25 | **Screenshots** 26 | If applicable, add screenshots to help explain your problem. 27 | 28 | **Additional context** 29 | Add any other context about the problem here. 30 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/feature-requests-and-questions.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Feature requests and questions 3 | about: ⛔ Please use GitHub Discussions instead. Visit https://github.com/osmoscraft/osmosfeed/discussions 4 | title: '' 5 | labels: '' 6 | assignees: '' 7 | 8 | --- 9 | 10 | Apologies. I really hate to send you to another place but there is no easier way to guide you toward GitHub Discussions. Please visit: https://github.com/osmoscraft/osmosfeed/discussions). 11 | 12 | 1. To submit a feature requests, create a Discussion in the **💡 Idea** category. 13 | 2. To ask a question, create a Discussion in the **🙏 Q&A** category. 14 | 15 | Thank you in advance for your contribution! 16 | -------------------------------------------------------------------------------- /.github/workflows/build-and-release.yml: -------------------------------------------------------------------------------- 1 | name: Build and release 2 | 3 | on: 4 | push: 5 | branches: 6 | - master 7 | tags: 8 | - "v*" 9 | 10 | jobs: 11 | build-test-and-publish: 12 | runs-on: ubuntu-latest 13 | 14 | steps: 15 | - uses: actions/checkout@v2 16 | - name: Setup Node.js environment 17 | uses: actions/setup-node@v2 18 | with: 19 | node-version: 16.x 20 | registry-url: https://registry.npmjs.org/ 21 | - name: Install cli package 22 | run: npm install 23 | working-directory: packages/cli 24 | - name: Build cli package 25 | run: npm run build 26 | working-directory: packages/cli 27 | - name: Install root 28 | run: npm install 29 | - name: Test 30 | run: npm run test 31 | 32 | # Conditionally run if the build is triggered by a tagged push 33 | - name: Publish to npm 34 | if: startsWith(github.ref, 'refs/tags/v') 35 | run: npm publish --access=public 36 | working-directory: packages/cli 37 | env: 38 | NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }} 39 | - name: Create Release 40 | if: startsWith(github.ref, 'refs/tags/v') 41 | id: create_release 42 | uses: actions/create-release@v1 43 | env: 44 | GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # This token is provided by Actions, you do not need to create your own token 45 | with: 46 | tag_name: ${{ github.ref }} 47 | release_name: ${{ github.ref }} 48 | draft: true 49 | prerelease: false 50 | -------------------------------------------------------------------------------- /.github/workflows/pr-validation.yml: -------------------------------------------------------------------------------- 1 | name: PR validation 2 | 3 | on: pull_request 4 | 5 | jobs: 6 | build-and-test: 7 | runs-on: ubuntu-latest 8 | 9 | steps: 10 | - uses: actions/checkout@v2 11 | - name: Setup Node.js environment 12 | uses: actions/setup-node@v2 13 | with: 14 | node-version: 16.x 15 | registry-url: https://registry.npmjs.org/ 16 | - name: Install cli package 17 | run: npm install 18 | working-directory: packages/cli 19 | - name: Build cli package 20 | run: npm run build 21 | working-directory: packages/cli 22 | - name: Install root 23 | run: npm install 24 | - name: Test 25 | run: npm run test 26 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Depenendies 2 | node_modules 3 | 4 | # Build output 5 | bin 6 | 7 | # Directories that build generates. A test run will generate it but don't check in 8 | public -------------------------------------------------------------------------------- /.prettierrc: -------------------------------------------------------------------------------- 1 | 2 | { 3 | "printWidth": 120 4 | } -------------------------------------------------------------------------------- /.vscode/launch.json: -------------------------------------------------------------------------------- 1 | { 2 | "version": "0.2.0", 3 | "configurations": [ 4 | { 5 | "name": "Debug with sandbox", 6 | "request": "launch", 7 | "runtimeArgs": ["run-script", "start"], 8 | "runtimeVersion": "16", 9 | "runtimeExecutable": "npm", 10 | "skipFiles": ["/**"], 11 | "type": "pwa-node" 12 | }, 13 | { 14 | "name": "Debug tests", 15 | "request": "launch", 16 | "runtimeArgs": ["run-script", "test"], 17 | "runtimeVersion": "16", 18 | "runtimeExecutable": "npm", 19 | "skipFiles": ["/**"], 20 | "type": "pwa-node" 21 | } 22 | ] 23 | } 24 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # v1.15.1 2 | 3 | - Fixed: Favicon used as article image 4 | 5 | # v1.15.0 6 | 7 | - Added: Resolve relative URLs in item link and media 8 | - Added: Item thumbnail fallback to channel thumbnail 9 | - Chore: Dependency updates 10 | - Thank you: @cpwnd 11 | 12 | # v1.14.4 13 | 14 | - Fixed: Several style issues 15 | 16 | # v1.14.2 17 | 18 | - New: Adjust article grouping based on timezone. You need to add `timezone` in `osmosfeed.yml` for accurate grouping. [See details in documentation](./docs/osmosfeed-yaml-reference.md). 19 | - New: Card title now toggles card content 20 | - New: all toggles on the UI are persisted with local storage. You can use it to track read/unread status within a single browser. 21 | - New: Build timestamp now links to the GitHub Action run 22 | - Changed: Sources are sorted based on publish time rather than alphabetical order 23 | - Changed: Style adjusted for easier reading 24 | - Fixed: Horizontal overflow on Safari 25 | - Fixed: HTML syntax error in default template 26 | 27 | # v1.13.0 28 | 29 | - New: Thumbnail image display for each article 30 | - New: Improved color, spacing, and typography for easier reading 31 | - New: Click the date to toggle all accordions on that day. Ctrl + Click to toggle all accordions on the entire site. 32 | - Changed: All sections are expanded by default 33 | - Changed: Switched to vitest for easier testing 34 | - Fixed: Long string (e.g. URL) caused horizontal overflow 35 | - Thank you: @naari3, @tianheg 36 | 37 | # v1.12.1 38 | 39 | - Fixed: Future articles appeared in the feed 40 | - Fixed: Security vulnerability from axios 41 | - Thank you: @naari3 42 | 43 | # v1.12.0 44 | 45 | - New: When source download fails, fall back to cache and continue rest of the build. 46 | - Chore: Dependency updates 47 | - Thank you: @sokomin, @molakirlee, @philippnagel 48 | 49 | # v1.11.3 50 | 51 | - Fixed: Broken links in documentation 52 | - Chore: Dependency updates 53 | - Thank you: @hirdot 54 | 55 | # v1.11.2 56 | 57 | - Fixed: unicode characters in URL caused download error 58 | - Fixed: Project root npm install error 59 | - Added: Unit test library 60 | - Chore: Dependency updates 61 | - Thank you: @CaptNo1 62 | 63 | # v1.11.1 64 | 65 | - Fixed: Unexpected nested element in `author` field crashes the parser 66 | 67 | # v1.11.0 68 | 69 | - Changed: Default HTML template meta tag adjusted to match convention 70 | - Changed: `sources` in template data is now ordered by last update timestamp (as opposed to alphabetical) 71 | 72 | # v1.10.2 73 | 74 | - Added: testing and CI/CD workflows 75 | 76 | # v1.9.0 77 | 78 | - Added: experimental support for podcast sources. You can render an audio control to play the mp3 file and display iTunes episode duration. Caveats: 79 | - 1. This feature is currently only available through custom template. 80 | - 2. iTunes has no strict format requirement for `duration`. The unit is most likely seconds. A future version may expose a human readable string to the template. 81 | - 3. No image support yet. Some shows might have disabled CORS or use a relative URL for image, which forces osmosfeed to download the image during build. A future version may support it. 82 | - 4. You can't republish the podcast in the feed output. Supporting this goes beyond the scope of the project at the moment. 83 | - Fixed: the `articles` in template data were not sorted. 84 | - Thank you: @Zenigata 85 | 86 | # v1.8.1 87 | 88 | - Fixed: `UnhandledPromiseRejectionWarning: Error: At least one option must be a string` 89 | 90 | # v1.8.0 91 | 92 | - Changed: Smart summary truncation. When there is no dedicated `summary` (only Atom feed has it, and many sites don't use it) from the source, we check if the content/description (Both Atom and RSS) field is long enough to be full text. If so, it will be truncated into a "pseudo" summary. If not, we assume the source used the `content` field as summary, and it will be displayed in full length. 93 | - Thank you: @LooperXX. 94 | 95 | # v1.7.3 96 | 97 | - This release has no user facing changes. It is created for recording keeping and won't be published. 98 | - Chore: Refactoring initialization and filesystem io. 99 | - Chore: Removed dependency `fs-extra`. 100 | 101 | # v1.7.2 102 | 103 | - ⚠ Deprecated: To accommodate the monorepo setup, the node version in all templates/demos/examples has changed from 14 to 16. This is not a breaking change as our compile target is still node 14, and node 16 is backward compatible. We still recommend you start using `node-version: "16"` in your `.github/workflows/update-feed.yaml` today to get the performance/security improvements from the newer node. [See example](https://github.com/osmoscraft/osmosfeed-template/blob/main/.github/workflows/update-feed.yaml#L22). 104 | - Chore: Reorganize to monorepo with npm workspace. 105 | 106 | # v1.7.1 107 | 108 | - Fixed: Static files were unnecessarily copied for custom templates. 109 | 110 | # v1.7.0 111 | 112 | - Added: Templating system. [Demo](https://osmoscraft.github.io/osmosfeed-examples/articles-unstyled/) | [Source](https://github.com/osmoscraft/osmosfeed-examples/tree/main/examples/articles-unstyled) | [Docs](https://github.com/osmoscraft/osmosfeed/blob/master/docs/customization-guide.md#template-customization-guide) 113 | - Chore: Rendering logic refactoring. Adopting functional paradigm. 114 | - Thank you @tianheg, @onnyyonn 115 | 116 | # v1.6.0 117 | 118 | - Added: Link from the title of a subscription to its website. 119 | - Added: Store `feedUrl` and `siteUrl` for each source in cache. 120 | - Added: Semantic element (`