├── README.md ├── docs.jsonl └── scripts ├── reorder.sh └── test.sh /README.md: -------------------------------------------------------------------------------- 1 | # Cursor Built-In Documentation 2 | 3 | This repository contains the official list of built-in documentation sources available in [Cursor](https://cursor.com/), the AI code editor. 4 | 5 |

6 | 7 |
8 | 9 |

10 | 11 | ## What is this? 12 | 13 | When using Cursor, you can reference documentation for libraries, frameworks, and languages using the `@docs` functionality. For example, typing `@React` will provide context about React's documentation. 14 | 15 | Learn more about using @docs, and how to add your own documentation sources in Cursor at our [documentation](https://docs.cursor.com/context/@-symbols/@-docs). 16 | 17 | Documentation is automatically updated in the editor every week, on Sunday. 18 | 19 | While you can add custom documentation sources to your local Cursor installation, this repository controls what documentation is available out-of-the-box for all users. 20 | 21 | ## What Qualifies for Built-in Docs? 22 | 23 | We aim to include documentation for tools and technologies that are: 24 | 25 | 1. **Widely Used**: Technologies with significant adoption in the developer community (e.g. React, Python, AWS services) 26 | 2. **Well-Documented**: Sources must have comprehensive, maintained documentation with a stable URL structure 27 | 3. **Developer-Focused**: Tools and frameworks used directly by developers in their workflow 28 | 4. **Production-Ready**: No beta/alpha versions or experimental tools 29 | 5. **English Language**: Documentation must be primarily in English 30 | 6. **Latest Versions**: Only the most recent stable version of libraries will be included, with rare exceptions for widely-used legacy versions (e.g. Python 2) 31 | 32 | Some examples that would NOT qualify: 33 | - Personal project documentation 34 | - Company-specific internal tools 35 | - Outdated or unmaintained projects 36 | - Documentation that requires authentication 37 | - Unofficial documentation sources 38 | 39 | ## Contributing 40 | 41 | We welcome contributions to expand our built-in documentation or fix issues with existing sources! 42 | 43 | To add a new documentation source: 44 | 45 | 1. Add a single new entry to [`docs.jsonl`](docs.jsonl) with: 46 | - Project name 47 | - Starting URL for our crawler 48 | - Prefix URL that all crawled pages should be under 49 | 2. Run `./scripts/reorder.sh` to sort the entries 50 | 3. Submit a PR with: 51 | - Only ONE new documentation source (for additions) 52 | - A comment explaining what you're adding/fixing and why 53 | - Confirmation that you've run the reorder script 54 | 55 | **Note:** When adding new sources, PRs with multiple documentation additions will be closed. Please submit separate PRs for each new source. PRs fixing existing documentation sources can include multiple fixes. 56 | 57 | Once approved, we'll crawl the documentation and make it available to all Cursor users via the `@ProjectName` command. 58 | -------------------------------------------------------------------------------- /docs.jsonl: -------------------------------------------------------------------------------- 1 | { "name": "ASP.NET", "crawlerStart": "https://learn.microsoft.com/en-us/aspnet/core/?view=aspnetcore-7.0", "crawlerPrefix": "https://learn.microsoft.com/en-us/aspnet/core/"} 2 | { "name": "AWS Amplify", "crawlerStart": "https://docs.amplify.aws/", "crawlerPrefix": "https://docs.amplify.aws/"} 3 | { "name": "AWS CLI", "crawlerStart": "https://docs.aws.amazon.com/cli/latest/reference/", "crawlerPrefix": "https://docs.aws.amazon.com/cli/latest/reference/"} 4 | { "name": "AWS DynamoDB", "crawlerStart": "https://docs.aws.amazon.com/dynamodb/", "crawlerPrefix": "https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/"} 5 | { "name": "AWS ECS", "crawlerStart": "https://docs.aws.amazon.com/ecs/", "crawlerPrefix": "https://docs.aws.amazon.com/ecs/latest/developerguide/"} 6 | { "name": "AWS Lambda", "crawlerStart": "https://docs.aws.amazon.com/lambda/index.html", "crawlerPrefix": "https://docs.aws.amazon.com/lambda/latest/dg/"} 7 | { "name": "AWS RDS", "crawlerStart": "https://docs.aws.amazon.com/rds/", "crawlerPrefix": "https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/"} 8 | { "name": "Amazon EC2", "crawlerStart": "https://docs.aws.amazon.com/ec2/index.html", "crawlerPrefix": "https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/"} 9 | { "name": "Amazon S3", "crawlerStart": "https://docs.aws.amazon.com/s3/index.html", "crawlerPrefix": "https://docs.aws.amazon.com/AmazonS3/latest/userguide/"} 10 | { "name": "Android SDK", "crawlerStart": "https://developer.android.com/docs", "crawlerPrefix": "https://developer.android.com/"} 11 | { "name": "Angular", "crawlerStart": "https://angular.dev/overview", "crawlerPrefix": "https://angular.dev/"} 12 | { "name": "Ansible", "crawlerStart": "https://docs.ansible.com/ansible/latest/index.html", "crawlerPrefix": "https://docs.ansible.com/ansible/latest/"} 13 | { "name": "Ant Design", "crawlerStart": "https://ant.design/docs/react/introduce", "crawlerPrefix": "https://ant.design/docs/react/"} 14 | { "name": "Apache Airflow", "crawlerStart": "https://airflow.apache.org/docs/apache-airflow/stable/index.html", "crawlerPrefix": "https://airflow.apache.org/docs/apache-airflow/stable/"} 15 | { "name": "Apollo GraphQL", "crawlerStart": "https://www.apollographql.com/docs/", "crawlerPrefix": "https://www.apollographql.com/docs/"} 16 | { "name": "Apple Developer Documentation", "crawlerStart": "https://developer.apple.com/documentation/", "crawlerPrefix": "https://developer.apple.com/documentation/"} 17 | { "name": "Astro", "crawlerStart": "https://docs.astro.build/en/", "crawlerPrefix": "https://docs.astro.build/en/"} 18 | { "name": "Auth0", "crawlerStart": "https://auth0.com/docs", "crawlerPrefix": "https://auth0.com/docs"} 19 | { "name": "Azure Pipelines", "crawlerStart": "https://docs.microsoft.com/en-us/azure/devops/pipelines/?view=azure-devops", "crawlerPrefix": "https://docs.microsoft.com/en-us/azure/devops/pipelines/"} 20 | { "name": "Bash", "crawlerStart": "https://www.gnu.org/software/bash/manual/bash.html", "crawlerPrefix": "https://www.gnu.org/software/bash/manual/"} 21 | { "name": "BeautifulSoup", "crawlerStart": "https://www.crummy.com/software/BeautifulSoup/bs4/doc/", "crawlerPrefix": "https://www.crummy.com/software/BeautifulSoup/bs4/doc/"} 22 | { "name": "Boto3", "crawlerStart": "https://boto3.amazonaws.com/v1/documentation/api/latest/index.html", "crawlerPrefix": "https://boto3.amazonaws.com/v1/documentation/api/latest/"} 23 | { "name": "Bun", "crawlerStart": "https://bun.sh/docs", "crawlerPrefix": "https://bun.sh/docs"} 24 | { "name": "C#", "crawlerStart": "https://learn.microsoft.com/en-us/dotnet/csharp/", "crawlerPrefix": "https://learn.microsoft.com/en-us/dotnet/csharp/"} 25 | { "name": "CSS", "crawlerStart": "https://developer.mozilla.org/en-US/docs/Web/CSS", "crawlerPrefix": "https://developer.mozilla.org/en-US/docs/Web/CSS"} 26 | { "name": "CUDA", "crawlerStart": "https://docs.nvidia.com/cuda/index.html", "crawlerPrefix": "https://docs.nvidia.com/cuda/"} 27 | { "name": "Cheerio", "crawlerStart": "https://cheerio.js.org/docs/intro", "crawlerPrefix": "https://cheerio.js.org/docs/"} 28 | { "name": "CircleCI", "crawlerStart": "https://circleci.com/docs/", "crawlerPrefix": "https://circleci.com/docs/"} 29 | { "name": "Clerk", "crawlerStart": "https://clerk.com/docs", "crawlerPrefix": "https://clerk.com/docs"} 30 | { "name": "Cloudflare", "crawlerStart": "https://developers.cloudflare.com/", "crawlerPrefix": "https://developers.cloudflare.com/"} 31 | { "name": "CodeMirror", "crawlerStart": "https://codemirror.net/docs/", "crawlerPrefix": "https://codemirror.net/docs/"} 32 | { "name": "Cursor", "crawlerStart": "https://docs.cursor.com/", "crawlerPrefix": "https://docs.cursor.com/"} 33 | { "name": "Cypress", "crawlerStart": "https://docs.cypress.io/guides/overview/why-cypress", "crawlerPrefix": "https://docs.cypress.io/guides/"} 34 | { "name": "D3", "crawlerStart": "https://d3js.org/getting-started", "crawlerPrefix": "https://d3js.org/"} 35 | { "name": "Datadog", "crawlerStart": "https://docs.datadoghq.com/", "crawlerPrefix": "https://docs.datadoghq.com/"} 36 | { "name": "Deno", "crawlerStart": "https://deno.land/manual@v1.35.0/introduction", "crawlerPrefix": "https://deno.land/manual@v1.35.0/"} 37 | { "name": "DigitalOcean", "crawlerStart": "https://docs.digitalocean.com/", "crawlerPrefix": "https://docs.digitalocean.com/"} 38 | { "name": "Discord API", "crawlerStart": "https://discord.com/developers/docs/intro", "crawlerPrefix": "https://discord.com/developers/docs"} 39 | { "name": "Django", "crawlerStart": "https://docs.djangoproject.com/en/4.2/", "crawlerPrefix": "https://docs.djangoproject.com/en/4.2/"} 40 | { "name": "Django Rest Framework", "crawlerStart": "https://www.django-rest-framework.org/api-guide/requests/", "crawlerPrefix": "https://www.django-rest-framework.org/api-guide"} 41 | { "name": "Docker", "crawlerStart": "https://docs.docker.com/", "crawlerPrefix": "https://docs.docker.com/"} 42 | { "name": "Drizzle", "crawlerStart": "https://orm.drizzle.team/docs/overview", "crawlerPrefix": "https://orm.drizzle.team/docs/overview"} 43 | { "name": "DuckDB", "crawlerStart": "https://duckdb.org/docs/sitemap", "crawlerPrefix": "https://duckdb.org/docs/stable"} 44 | { "name": "ELK Stack", "crawlerStart": "https://www.elastic.co/guide/en/elastic-stack/current/index.html", "crawlerPrefix": "https://www.elastic.co/guide/en/elastic-stack/current/"} 45 | { "name": "ESBuild", "crawlerStart": "https://esbuild.github.io/api/", "crawlerPrefix": "https://esbuild.github.io/api/"} 46 | { "name": "ESLint", "crawlerStart": "https://eslint.org/docs/latest/", "crawlerPrefix": "https://eslint.org/docs/latest/"} 47 | { "name": "Elasticsearch", "crawlerStart": "https://www.elastic.co/guide/en/enterprise-search/current/index.html", "crawlerPrefix": "https://www.elastic.co/guide/en/enterprise-search/current/"} 48 | { "name": "Electron", "crawlerStart": "https://www.electronjs.org/docs/latest/", "crawlerPrefix": "https://www.electronjs.org/docs/latest/"} 49 | { "name": "Emacs", "crawlerStart": "https://www.gnu.org/software/emacs/manual/html_node/emacs/", "crawlerPrefix": "https://www.gnu.org/software/emacs/manual/html_node/emacs/"} 50 | { "name": "Expo", "crawlerStart": "https://docs.expo.dev/", "crawlerPrefix": "https://docs.expo.dev/"} 51 | { "name": "Express", "crawlerStart": "https://expressjs.com/en/5x/api.html", "crawlerPrefix": "https://expressjs.com/en/5x/"} 52 | { "name": "FFmpeg", "crawlerStart": "https://ffmpeg.org/ffmpeg.html", "crawlerPrefix": "https://ffmpeg.org/ffmpeg.html"} 53 | { "name": "Fabric.js", "crawlerStart": "http://fabricjs.com/docs/", "crawlerPrefix": "http://fabricjs.com/docs/"} 54 | { "name": "Fal.ai", "crawlerStart": "https://docs.fal.ai/quick-start", "crawlerPrefix": "https://docs.fal.ai/"} 55 | { "name": "FastAPI", "crawlerStart": "https://fastapi.tiangolo.com/tutorial/", "crawlerPrefix": "https://fastapi.tiangolo.com/tutorial/"} 56 | { "name": "Firebase", "crawlerStart": "https://firebase.google.com/docs", "crawlerPrefix": "https://firebase.google.com/docs"} 57 | { "name": "Flask", "crawlerStart": "https://flask.palletsprojects.com/en/2.3.x/", "crawlerPrefix": "https://flask.palletsprojects.com/en/2.3.x/"} 58 | { "name": "Flutter", "crawlerStart": "https://docs.flutter.dev/", "crawlerPrefix": "https://docs.flutter.dev/"} 59 | { "name": "FontAwesome", "crawlerStart": "https://fontawesome.com/docs/web/", "crawlerPrefix": "https://fontawesome.com/docs/web/"} 60 | { "name": "GCP CLI", "crawlerStart": "https://cloud.google.com/sdk/docs", "crawlerPrefix": "https://cloud.google.com/sdk/docs"} 61 | { "name": "Git", "crawlerStart": "https://git-scm.com/docs", "crawlerPrefix": "https://git-scm.com/docs"} 62 | { "name": "GitHub Actions", "crawlerStart": "https://docs.github.com/en/actions", "crawlerPrefix": "https://docs.github.com/en/actions"} 63 | { "name": "GitLab CI", "crawlerStart": "https://docs.gitlab.com/ee/ci/", "crawlerPrefix": "https://docs.gitlab.com/ee/ci/"} 64 | { "name": "Go", "crawlerStart": "https://go.dev/doc", "crawlerPrefix": "https://go.dev/doc"} 65 | { "name": "Godot", "crawlerStart": "https://docs.godotengine.org/en/stable/", "crawlerPrefix": "https://docs.godotengine.org/en/stable/"} 66 | { "name": "Google Maps JS API", "crawlerStart": "https://developers.google.com/maps/documentation/javascript", "crawlerPrefix": "https://developers.google.com/maps/documentation/javascript"} 67 | { "name": "Gradle", "crawlerStart": "https://docs.gradle.org/current/userguide/userguide.html", "crawlerPrefix": "https://docs.gradle.org/current/userguide/"} 68 | { "name": "Grafana", "crawlerStart": "https://grafana.com/docs/grafana/latest/", "crawlerPrefix": "https://grafana.com/docs/grafana/latest/"} 69 | { "name": "GraphQL", "crawlerStart": "https://graphql.org/learn/", "crawlerPrefix": "https://graphql.org/learn/"} 70 | { "name": "HTML", "crawlerStart": "https://developer.mozilla.org/en-US/docs/Web/HTML", "crawlerPrefix": "https://developer.mozilla.org/en-US/docs/Web/HTML"} 71 | { "name": "Heroku", "crawlerStart": "https://devcenter.heroku.com/categories/reference", "crawlerPrefix": "https://devcenter.heroku.com/"} 72 | { "name": "Hono", "crawlerStart": "https://hono.dev/docs/", "crawlerPrefix": "https://hono.dev/docs/"} 73 | { "name": "Insomnia", "crawlerStart": "https://docs.insomnia.rest/", "crawlerPrefix": "https://docs.insomnia.rest/"} 74 | { "name": "Ionic", "crawlerStart": "https://ionicframework.com/docs", "crawlerPrefix": "https://ionicframework.com/docs"} 75 | { "name": "JAX", "crawlerStart": "https://jax.readthedocs.io/en/latest/", "crawlerPrefix": "https://jax.readthedocs.io/"} 76 | { "name": "JUnit 5", "crawlerStart": "https://junit.org/junit5/docs/current/user-guide/", "crawlerPrefix": "https://junit.org/junit5/docs/current/user-guide/"} 77 | { "name": "Java", "crawlerStart": "https://docs.oracle.com/javase/8/docs/api/", "crawlerPrefix": "https://docs.oracle.com/javase/8/docs/api/"} 78 | { "name": "Jenkins", "crawlerStart": "https://www.jenkins.io/doc/", "crawlerPrefix": "https://www.jenkins.io/doc/"} 79 | { "name": "Jest", "crawlerStart": "https://jestjs.io/docs/getting-started", "crawlerPrefix": "https://jestjs.io/docs/getting-started"} 80 | { "name": "Jquery", "crawlerStart": "https://api.jqueryui.com/1.12/", "crawlerPrefix": "https://api.jqueryui.com/1.12/"} 81 | { "name": "Keras", "crawlerStart": "https://keras.io/api/", "crawlerPrefix": "https://keras.io/api/"} 82 | { "name": "Kubernetes", "crawlerStart": "https://kubernetes.io/docs/", "crawlerPrefix": "https://kubernetes.io/docs/"} 83 | { "name": "LLVM", "crawlerStart": "https://llvm.org/docs/", "crawlerPrefix": "https://llvm.org/docs/"} 84 | { "name": "Langchain", "crawlerStart": "https://python.langchain.com/docs/", "crawlerPrefix": "https://python.langchain.com/docs/"} 85 | { "name": "Langchain-JS", "crawlerStart": "https://js.langchain.com/docs/", "crawlerPrefix": "https://js.langchain.com/docs/"} 86 | { "name": "Laravel", "crawlerStart": "https://laravel.com/docs/10.x", "crawlerPrefix": "https://laravel.com/docs/10.x"} 87 | { "name": "Linux Man Pages", "crawlerStart": "https://man7.org/linux/man-pages/dir_all_alphabetic.html", "crawlerPrefix": "https://man7.org/linux/man-pages/"} 88 | { "name": "MCP", "crawlerStart": "https://modelcontextprotocol.io/introduction", "crawlerPrefix": "https://modelcontextprotocol.io/"} 89 | { "name": "MLX", "crawlerStart": "https://ml-explore.github.io/mlx/build/html/", "crawlerPrefix": "https://ml-explore.github.io/mlx/build/html/"} 90 | { "name": "Material UI", "crawlerStart": "https://mui.com/material-ui/getting-started/", "crawlerPrefix": "https://mui.com/material-ui/"} 91 | { "name": "Matplotlib", "crawlerStart": "https://matplotlib.org/stable/api/", "crawlerPrefix": "https://matplotlib.org/stable/api/"} 92 | { "name": "Maven", "crawlerStart": "https://maven.apache.org/guides/", "crawlerPrefix": "https://maven.apache.org/guides/"} 93 | { "name": "Microsoft Teams", "crawlerStart": "https://learn.microsoft.com/en-us/microsoftteams/platform/", "crawlerPrefix": "https://learn.microsoft.com/en-us/microsoftteams/platform/"} 94 | { "name": "Mockito", "crawlerStart": "https://javadoc.io/doc/org.mockito/mockito-core/latest/index.html", "crawlerPrefix": "https://javadoc.io/doc/org.mockito/mockito-core/latest/"} 95 | { "name": "MongoDB", "crawlerStart": "https://www.mongodb.com/docs/manual/", "crawlerPrefix": "https://www.mongodb.com/docs/manual/"} 96 | { "name": "MySQL", "crawlerStart": "https://dev.mysql.com/doc/", "crawlerPrefix": "https://dev.mysql.com/doc/"} 97 | { "name": "Mypy", "crawlerStart": "https://mypy.readthedocs.io/en/stable/", "crawlerPrefix": "https://mypy.readthedocs.io/en/stable/"} 98 | { "name": "NLTK", "crawlerStart": "https://www.nltk.org/", "crawlerPrefix": "https://www.nltk.org/"} 99 | { "name": "Neo4j", "crawlerStart": "https://neo4j.com/docs/", "crawlerPrefix": "https://neo4j.com/docs/"} 100 | { "name": "NestJS", "crawlerStart": "https://docs.nestjs.com/", "crawlerPrefix": "https://docs.nestjs.com/"} 101 | { "name": "Netlify", "crawlerStart": "https://docs.netlify.com/", "crawlerPrefix": "https://docs.netlify.com/"} 102 | { "name": "NextJS", "crawlerStart": "https://nextjs.org/docs", "crawlerPrefix": "https://nextjs.org/docs"} 103 | { "name": "Nginx", "crawlerStart": "http://nginx.org/en/docs/", "crawlerPrefix": "http://nginx.org/en/docs/"} 104 | { "name": "NodeJS", "crawlerStart": "https://nodejs.org/api/", "crawlerPrefix": "https://nodejs.org/api/"} 105 | { "name": "Notion", "crawlerStart": "https://developers.notion.com/reference/", "crawlerPrefix": "https://developers.notion.com/reference/"} 106 | { "name": "NumPy", "crawlerStart": "https://numpy.org/doc/stable/", "crawlerPrefix": "https://numpy.org/doc/stable/"} 107 | { "name": "Nuxt", "crawlerStart": "https://nuxt.com/docs", "crawlerPrefix": "https://nuxt.com/docs"} 108 | { "name": "Nx", "crawlerStart": "https://nx.dev/getting-started/intro", "crawlerPrefix": "https://nx.dev/"} 109 | { "name": "OpenAI", "crawlerStart": "https://platform.openai.com/docs/", "crawlerPrefix": "https://platform.openai.com/docs/"} 110 | { "name": "OpenCV", "crawlerStart": "https://docs.opencv.org/4.x/", "crawlerPrefix": "https://docs.opencv.org/4.x/"} 111 | { "name": "PHP", "crawlerStart": "https://www.php.net/manual/en/", "crawlerPrefix": "https://www.php.net/manual/en/"} 112 | { "name": "POSIX.1-2024", "crawlerStart": "https://pubs.opengroup.org/onlinepubs/9799919799/idx/index.html", "crawlerPrefix": "https://pubs.opengroup.org/onlinepubs/9799919799/"} 113 | { "name": "Pandas", "crawlerStart": "https://pandas.pydata.org/docs/", "crawlerPrefix": "https://pandas.pydata.org/docs/"} 114 | { "name": "Pinia", "crawlerStart": "https://pinia.vuejs.org/core-concepts/", "crawlerPrefix": "https://pinia.vuejs.org/"} 115 | { "name": "Playwright", "crawlerStart": "https://playwright.dev/docs/intro", "crawlerPrefix": "https://playwright.dev/docs/"} 116 | { "name": "Pnpm", "crawlerStart": "https://pnpm.io/", "crawlerPrefix": "https://pnpm.io/"} 117 | { "name": "Polaris", "crawlerStart": "https://polaris.shopify.com/", "crawlerPrefix": "https://polaris.shopify.com/"} 118 | { "name": "Polars", "crawlerStart": "https://docs.pola.rs/", "crawlerPrefix": "https://docs.pola.rs/"} 119 | { "name": "PostgreSQL", "crawlerStart": "https://www.postgresql.org/docs/current/", "crawlerPrefix": "https://www.postgresql.org/docs/current/"} 120 | { "name": "Postman", "crawlerStart": "https://learning.postman.com/docs/", "crawlerPrefix": "https://learning.postman.com/docs/"} 121 | { "name": "Prisma", "crawlerStart": "https://www.prisma.io/docs", "crawlerPrefix": "https://www.prisma.io/docs"} 122 | { "name": "Pulumi", "crawlerStart": "https://www.pulumi.com/docs/", "crawlerPrefix": "https://www.pulumi.com/docs/"} 123 | { "name": "Puppeteer", "crawlerStart": "https://pptr.dev/", "crawlerPrefix": "https://pptr.dev/"} 124 | { "name": "PyTorch", "crawlerStart": "https://pytorch.org/docs/stable/", "crawlerPrefix": "https://pytorch.org/docs/stable/"} 125 | { "name": "Pydantic", "crawlerStart": "https://docs.pydantic.dev/latest/", "crawlerPrefix": "https://docs.pydantic.dev/latest/"} 126 | { "name": "Pytest", "crawlerStart": "https://docs.pytest.org/en/stable/", "crawlerPrefix": "https://docs.pytest.org/en/stable/"} 127 | { "name": "Python", "crawlerStart": "https://docs.python.org/3/", "crawlerPrefix": "https://docs.python.org/3/"} 128 | { "name": "ROS", "crawlerStart": "https://docs.ros.org/en/rolling/", "crawlerPrefix": "https://docs.ros.org/en/rolling/"} 129 | { "name": "Railway", "crawlerStart": "https://docs.railway.app/", "crawlerPrefix": "https://docs.railway.app/"} 130 | { "name": "React", "crawlerStart": "https://react.dev/reference/react", "crawlerPrefix": "https://react.dev/reference/"} 131 | { "name": "Redis", "crawlerStart": "https://redis.io/docs/", "crawlerPrefix": "https://redis.io/docs/"} 132 | { "name": "Regex", "crawlerStart": "https://www.regular-expressions.info/", "crawlerPrefix": "https://www.regular-expressions.info/"} 133 | { "name": "Remix", "crawlerStart": "https://remix.run/docs/en/main", "crawlerPrefix": "https://remix.run/docs/"} 134 | { "name": "Ruby", "crawlerStart": "https://docs.ruby-lang.org/en/master/", "crawlerPrefix": "https://docs.ruby-lang.org/en/"} 135 | { "name": "Ruff", "crawlerStart": "https://docs.astral.sh/ruff/", "crawlerPrefix": "https://docs.astral.sh/ruff/"} 136 | { "name": "Rust", "crawlerStart": "https://doc.rust-lang.org/book/", "crawlerPrefix": "https://doc.rust-lang.org/book/"} 137 | { "name": "Rust Stdlib", "crawlerStart": "https://doc.rust-lang.org/std/", "crawlerPrefix": "https://doc.rust-lang.org/std/"} 138 | { "name": "SQLite", "crawlerStart": "https://www.sqlite.org/docs.html", "crawlerPrefix": "https://www.sqlite.org/"} 139 | { "name": "Scikit-learn", "crawlerStart": "https://scikit-learn.org/stable/", "crawlerPrefix": "https://scikit-learn.org/stable/"} 140 | { "name": "Selenium", "crawlerStart": "https://www.selenium.dev/documentation/", "crawlerPrefix": "https://www.selenium.dev/documentation/"} 141 | { "name": "Sentry", "crawlerStart": "https://docs.sentry.io/", "crawlerPrefix": "https://docs.sentry.io/"} 142 | { "name": "Shadcn", "crawlerStart": "https://ui.shadcn.com/docs", "crawlerPrefix": "https://ui.shadcn.com/docs"} 143 | { "name": "Socket.io", "crawlerStart": "https://socket.io/docs/v4/", "crawlerPrefix": "https://socket.io/docs/v4/"} 144 | { "name": "Solidity", "crawlerStart": "https://docs.soliditylang.org/en/latest/", "crawlerPrefix": "https://docs.soliditylang.org/en/latest/"} 145 | { "name": "Spring", "crawlerStart": "https://docs.spring.io/spring-framework/reference/", "crawlerPrefix": "https://docs.spring.io/spring-framework/reference/"} 146 | { "name": "Stripe", "crawlerStart": "https://stripe.com/docs", "crawlerPrefix": "https://stripe.com/docs"} 147 | { "name": "Supabase", "crawlerStart": "https://supabase.com/docs", "crawlerPrefix": "https://supabase.com/docs"} 148 | { "name": "Svelte", "crawlerStart": "https://svelte.dev/docs", "crawlerPrefix": "https://svelte.dev/docs"} 149 | { "name": "Symfony", "crawlerStart": "https://symfony.com/doc/current/", "crawlerPrefix": "https://symfony.com/doc/"} 150 | { "name": "Tailwind", "crawlerStart": "https://tailwindcss.com/docs", "crawlerPrefix": "https://tailwindcss.com/docs"} 151 | { "name": "Terraform", "crawlerStart": "https://developer.hashicorp.com/terraform/docs", "crawlerPrefix": "https://developer.hashicorp.com/terraform/docs"} 152 | { "name": "Three.js", "crawlerStart": "https://threejs.org/docs/", "crawlerPrefix": "https://threejs.org/docs/"} 153 | { "name": "Tinygrad", "crawlerStart": "https://docs.tinygrad.org", "crawlerPrefix": "https://docs.tinygrad.org"} 154 | { "name": "Tokio.rs", "crawlerStart": "https://docs.rs/tokio/latest/tokio/", "crawlerPrefix": "https://docs.rs/tokio/latest/tokio/"} 155 | { "name": "Transformers", "crawlerStart": "https://huggingface.co/docs/transformers/index", "crawlerPrefix": "https://huggingface.co/docs/transformers/"} 156 | { "name": "TypeScript", "crawlerStart": "https://www.typescriptlang.org/docs/", "crawlerPrefix": "https://www.typescriptlang.org/docs/"} 157 | { "name": "Unity", "crawlerStart": "https://docs.unity3d.com/Manual/", "crawlerPrefix": "https://docs.unity3d.com/Manual/"} 158 | { "name": "Unreal Engine", "crawlerStart": "https://docs.unrealengine.com/5.0/en-US/", "crawlerPrefix": "https://docs.unrealengine.com/5.0/en-US/"} 159 | { "name": "Uv", "crawlerStart": "https://docs.astral.sh/uv/", "crawlerPrefix": "https://docs.astral.sh/uv/"} 160 | { "name": "Vercel", "crawlerStart": "https://vercel.com/docs", "crawlerPrefix": "https://vercel.com/docs"} 161 | { "name": "Vim", "crawlerStart": "https://vimhelp.org/", "crawlerPrefix": "https://vimhelp.org/"} 162 | { "name": "Vite", "crawlerStart": "https://vitejs.dev/guide/", "crawlerPrefix": "https://vitejs.dev/guide/"} 163 | { "name": "Vitest", "crawlerStart": "https://vitest.dev/guide/", "crawlerPrefix": "https://vitest.dev/guide/"} 164 | { "name": "Vue", "crawlerStart": "https://vuejs.org/guide/introduction.html", "crawlerPrefix": "https://vuejs.org/guide/"} 165 | { "name": "Webpack", "crawlerStart": "https://webpack.js.org/concepts/", "crawlerPrefix": "https://webpack.js.org/concepts/"} 166 | { "name": "XGBoost", "crawlerStart": "https://xgboost.readthedocs.io/en/stable/", "crawlerPrefix": "https://xgboost.readthedocs.io/en/stable/"} 167 | { "name": "Zod", "crawlerStart": "https://zod.dev/", "crawlerPrefix": "https://zod.dev/"} 168 | { "name": "Zsh", "crawlerStart": "https://zsh.sourceforge.io/Doc/", "crawlerPrefix": "https://zsh.sourceforge.io/Doc/"} 169 | { "name": "help", "crawlerStart": "https://docs.cursor.com/get-started/welcome", "crawlerPrefix": "https://docs.cursor.com/"} -------------------------------------------------------------------------------- /scripts/reorder.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Get the directory where the script is located 4 | SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" 5 | 6 | # Sort JSON entries by name, remove duplicates, ensure one item per line 7 | jq -c '.' "$SCRIPT_DIR/../docs.jsonl" | \ 8 | jq -s 'sort_by(.name) | unique_by(.name) | .[]' | \ 9 | tr -d '\n' | sed 's/}{/}\n{/g' > "$SCRIPT_DIR/../docs.jsonl.tmp" 10 | 11 | mv "$SCRIPT_DIR/../docs.jsonl.tmp" "$SCRIPT_DIR/../docs.jsonl" 12 | 13 | # Count items after sorting 14 | TOTAL=$(wc -l < "$SCRIPT_DIR/../docs.jsonl" | tr -d ' ') 15 | 16 | echo -e "\n\033[32m✓ Successfully reordered $TOTAL items\033[0m\n" -------------------------------------------------------------------------------- /scripts/test.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Get the directory where the script is located 4 | SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" 5 | 6 | # Extract URLs and test each one 7 | while read -r line; do 8 | # Extract URL and line number 9 | url=$(echo "$line" | cut -d: -f2- | sed -E 's/.*"crawlerStart": "([^"]+)".*/\1/') 10 | line_num=$(echo "$line" | cut -d: -f1) 11 | 12 | # Get HTTP status code and redirect location for URL 13 | response=$(curl -s -L -o /dev/null -w "%{http_code} %{url_effective}" "$url") 14 | status=$(echo "$response" | cut -d' ' -f1) 15 | redirect_url=$(echo "$response" | cut -d' ' -f2-) 16 | 17 | # Check if status code indicates success (2xx), redirect (3xx), or failure 18 | if [[ $status =~ ^2 ]]; then 19 | echo -e "\033[32m✓ $url ($status)\033[0m" 20 | elif [[ $status =~ ^3 ]]; then 21 | echo -e "\033[33m→ $url redirects to $redirect_url ($status)\033[0m" 22 | # Update docs.jsonl with redirect destination 23 | sed -i '' "${line_num}s|\"crawlerStart\": \"$url\"|\"crawlerStart\": \"$redirect_url\"|" "$SCRIPT_DIR/../docs.jsonl" 24 | elif [[ $status == "404" ]]; then 25 | echo -e "\033[31m✗ $url (404) - removing entry\033[0m" 26 | # Remove the line from docs.jsonl 27 | sed -i '' "${line_num}d" "$SCRIPT_DIR/../docs.jsonl" 28 | else 29 | echo -e "\033[31m✗ $url ($status)\033[0m" 30 | fi 31 | done < <(grep -n "crawlerStart" "$SCRIPT_DIR/../docs.jsonl") 32 | --------------------------------------------------------------------------------