├── requirements.txt ├── .github └── dependabot.yml ├── LICENSE ├── CONTRIBUTING.md ├── .gitignore ├── README.md └── cxl_feature_tracker.py /requirements.txt: -------------------------------------------------------------------------------- 1 | # Requests library for making HTTP requests 2 | requests>=2.25.1 -------------------------------------------------------------------------------- /.github/dependabot.yml: -------------------------------------------------------------------------------- 1 | # To get started with Dependabot version updates, you'll need to specify which 2 | # package ecosystems to update and where the package manifests are located. 3 | # Please see the documentation for all configuration options: 4 | # https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file 5 | 6 | version: 2 7 | updates: 8 | - package-ecosystem: "pip" # See documentation for possible values 9 | directory: "/" # Location of package manifests 10 | schedule: 11 | interval: "weekly" 12 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2024 Steve Scargall 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing to Linux Kernel CXL Feature Tracker 2 | 3 | Thank you for your interest in contributing to the Linux Kernel CXL Feature Tracker! We welcome contributions from everyone, and we hope our guidelines make the process as smooth as possible for everyone involved. 4 | 5 | ## How to Contribute 6 | 7 | Contributing to this project can be done in several ways. Below are some guidelines to help you get started: 8 | 9 | ## Reporting Issues 10 | 11 | If you find a bug or have a suggestion for improving the documentation or code: 12 | 13 | - Check the Issue Tracker to see if your issue has already been reported. 14 | - If not, create a new issue with a detailed description: 15 | - Explain the expected behavior and the actual behavior. 16 | - Provide steps to reproduce the issue. 17 | - Mention the version of the software you are using (if applicable). 18 | - Include the output from the script, if applicable. 19 | 20 | ## Submitting Changes or New Features 21 | 22 | If you'd like to submit a change to the repository: 23 | 24 | 1. Fork the repository. 25 | 2. Clone your fork: 26 | ```bash 27 | git clone https://github.com/sscargal/linux-cxl-tracker 28 | ``` 29 | 3. Create a new branch for your changes: 30 | ```.bash 31 | git checkout -b feature-branch-name 32 | ``` 33 | 4. Make your changes and commit them using a signed commit. To create a certificate for signed commits and configure `git`, follow the instructions to install and configure [GitSign GitStore](https://docs.sigstore.dev/signing/gitsign/), then: 34 | ```bash 35 | git commit -am "Add some feature" -S 36 | ``` 37 | 38 | 5. Push the changes to your fork: 39 | ```bash 40 | git push origin feature-branch-name 41 | ``` 42 | 6. Submit a pull request from your feature branch to the original repo's main branch. 43 | 44 | ## Pull Request Guidelines 45 | 46 | Ensure your code adheres to the existing style so that it's as readable and maintainable as possible. 47 | 48 | Update the README.md with details of changes, including new environment variables, exposed ports, useful file locations, and container parameters. 49 | 50 | Increase the version numbers in any examples files and the README.md to the new version that this Pull Request would represent. 51 | 52 | You may merge the Pull Request in once you have the sign-off of two other developers, or if you do not have permission to do that, you may request the second reviewer to merge it for you. 53 | 54 | ## Development Setup 55 | 56 | Here's a quick setup guide if you want to run the script locally: 57 | 58 | 1. Install Python 3.x. 59 | 2. Create a Virtual Environment. It's a good practice to use a virtual environment for Python projects to manage dependencies separately from the global Python environment. You can create a virtual environment using `venv` or `conda`: 60 | 61 | Using `venv` 62 | ```bash 63 | python -m venv env 64 | source env/bin/activate # On Windows use env\Scripts\activate 65 | ``` 66 | Using `Conda` 67 | ```bash 68 | conda create --name myenv python=3.8 69 | conda activate myenv 70 | ``` 71 | 3. Install the required dependencies: 72 | ```bash 73 | pip install -r requirements.txt 74 | ``` 75 | 4. Run the script with a test command to ensure it's working: 76 | ```bash 77 | ./cxl_feature_tracker.py --start-version v5.8 --end-version v5.9 --verbose 78 | ``` 79 | 80 | ## Community 81 | 82 | We want to maintain a welcoming and respectful community. Whether you are a newcomer or a long-time contributor, we encourage you to follow the Community Code of Conduct. 83 | 84 | Thank you for contributing to the Linux Kernel CXL Feature Tracker. We look forward to your contributions! -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | .eggs/ 17 | lib/ 18 | lib64/ 19 | parts/ 20 | sdist/ 21 | var/ 22 | wheels/ 23 | share/python-wheels/ 24 | *.egg-info/ 25 | .installed.cfg 26 | *.egg 27 | MANIFEST 28 | 29 | # PyInstaller 30 | # Usually these files are written by a python script from a template 31 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 32 | *.manifest 33 | *.spec 34 | 35 | # Installer logs 36 | pip-log.txt 37 | pip-delete-this-directory.txt 38 | 39 | # Unit test / coverage reports 40 | htmlcov/ 41 | .tox/ 42 | .nox/ 43 | .coverage 44 | .coverage.* 45 | .cache 46 | nosetests.xml 47 | coverage.xml 48 | *.cover 49 | *.py,cover 50 | .hypothesis/ 51 | .pytest_cache/ 52 | cover/ 53 | 54 | # Translations 55 | *.mo 56 | *.pot 57 | 58 | # Django stuff: 59 | *.log 60 | local_settings.py 61 | db.sqlite3 62 | db.sqlite3-journal 63 | 64 | # Flask stuff: 65 | instance/ 66 | .webassets-cache 67 | 68 | # Scrapy stuff: 69 | .scrapy 70 | 71 | # Sphinx documentation 72 | docs/_build/ 73 | 74 | # PyBuilder 75 | .pybuilder/ 76 | target/ 77 | 78 | # Jupyter Notebook 79 | .ipynb_checkpoints 80 | 81 | # IPython 82 | profile_default/ 83 | ipython_config.py 84 | 85 | # pyenv 86 | # For a library or package, you might want to ignore these files since the code is 87 | # intended to run in multiple environments; otherwise, check them in: 88 | # .python-version 89 | 90 | # pipenv 91 | # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. 92 | # However, in case of collaboration, if having platform-specific dependencies or dependencies 93 | # having no cross-platform support, pipenv may install dependencies that don't work, or not 94 | # install all needed dependencies. 95 | #Pipfile.lock 96 | 97 | # poetry 98 | # Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control. 99 | # This is especially recommended for binary packages to ensure reproducibility, and is more 100 | # commonly ignored for libraries. 101 | # https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control 102 | #poetry.lock 103 | 104 | # pdm 105 | # Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control. 106 | #pdm.lock 107 | # pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it 108 | # in version control. 109 | # https://pdm.fming.dev/#use-with-ide 110 | .pdm.toml 111 | 112 | # PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm 113 | __pypackages__/ 114 | 115 | # Celery stuff 116 | celerybeat-schedule 117 | celerybeat.pid 118 | 119 | # SageMath parsed files 120 | *.sage.py 121 | 122 | # Environments 123 | .env 124 | .venv 125 | env/ 126 | venv/ 127 | ENV/ 128 | env.bak/ 129 | venv.bak/ 130 | 131 | # Spyder project settings 132 | .spyderproject 133 | .spyproject 134 | 135 | # Rope project settings 136 | .ropeproject 137 | 138 | # mkdocs documentation 139 | /site 140 | 141 | # mypy 142 | .mypy_cache/ 143 | .dmypy.json 144 | dmypy.json 145 | 146 | # Pyre type checker 147 | .pyre/ 148 | 149 | # pytype static type analyzer 150 | .pytype/ 151 | 152 | # Cython debug symbols 153 | cython_debug/ 154 | 155 | # PyCharm 156 | # JetBrains specific template is maintained in a separate JetBrains.gitignore that can 157 | # be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore 158 | # and can be added to the global gitignore or merged into this file. For a more nuclear 159 | # option (not recommended) you can uncomment the following to ignore the entire idea folder. 160 | #.idea/ 161 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Linux Kernel CXL Feature Tracker 2 | 3 | ## Description 4 | 5 | This Python script tracks changes related to Compute Express Link (CXL) in the Linux kernel. It fetches commit messages from the Linux GitHub repository that are pertinent to CXL, specifically focusing on the drivers/cxl and drivers/dax directories. Users can specify the range of kernel versions to check, control the verbosity of the output, and decide on the output format and destination. 6 | 7 | ## Features 8 | 9 | - **Graceful Interruption**: The script can be safely interrupted at any time by pressing Ctrl+C. It will stop processing and exit gracefully. 10 | 11 | ## Installation 12 | 13 | ### Prerequisites 14 | 15 | - Python 3.6 or higher 16 | - Python requests library 17 | - `venv` or `conda` Python Virtual Environment (Optional) 18 | 19 | ### Setup 20 | 21 | Here's a quick setup guide if you want to run the script locally: 22 | 23 | 1. Install Python 3.x. 24 | 2. Create a Virtual Environment. It's a good practice to use a virtual environment for Python projects to manage dependencies separately from the global Python environment. You can create a virtual environment using `venv` or `conda`: 25 | 26 | Using `venv` 27 | ```bash 28 | python -m venv env 29 | source env/bin/activate # On Windows use env\Scripts\activate 30 | ``` 31 | Using `Conda` 32 | ```bash 33 | conda create --name myenv python=3.8 34 | conda activate myenv 35 | ``` 36 | 3. Install the required dependencies: 37 | ```bash 38 | pip install -r requirements.txt 39 | ``` 40 | 41 | ### Download 42 | 43 | Clone this repository 44 | 45 | ```bash 46 | git clone https://github.com/sscargal/linux-cxl-tracker 47 | ``` 48 | 49 | or download the script directly: 50 | 51 | ```bash 52 | git clone https://github.com/sscargal/linux-cxl-tracker/linux-cxl-tracker.git 53 | ``` 54 | 55 | ## Usage 56 | 57 | Command-Line Options 58 | 59 | - `--ghtoken`: Specifies the GitHub API token for authenticated requests (optional but recommended to avoid rate limits). 60 | - `--start-version`: Specifies the starting kernel version to track changes from. 61 | - `--end-version`: Specifies the ending kernel version to track changes to. 62 | - `--output`: Specifies the filename where the output should be written. If not provided, output is printed to the terminal. 63 | - `--format`: Specifies the format of the output file. Options include txt, md, json. Default is terminal output. 64 | - `--verbose`: When set, the script will display detailed commit messages. By default, only commit titles are shown. 65 | 66 | ## Examples 67 | 68 | Track Changes with Default Options: 69 | ```bash 70 | ./cxl_feature_tracker.py --start-version v5.8 --end-version v5.9 71 | ``` 72 | 73 | Track Changes with Verbose Output: 74 | ```bash 75 | ./cxl_feature_tracker.py --start-version v5.8 --end-version v5.9 --verbose 76 | ``` 77 | 78 | Track Changes and Write to a Text File: 79 | ```bash 80 | ./cxl_feature_tracker.py --start-version v5.8 --end-version v5.9 --output changes.txt --format txt 81 | ``` 82 | 83 | Track Changes and Write to a Markdown File with Verbose Output: 84 | ```bash 85 | ./cxl_feature_tracker.py --start-version v5.8 --end-version v5.9 --output changes.md --format md --verbose 86 | ``` 87 | 88 | Track Changes Using GitHub Token: 89 | GitHub API has a rate limit, especially for unauthenticated requests (default). For more frequent usage or large data, use authentication by adding a token in the request headers. 90 | ```bash 91 | ./cxl_feature_tracker.py --ghtoken YOUR_GITHUB_TOKEN --start-version v5.8 --end-version v5.9 92 | ``` 93 | 94 | Track Changes and Write to a JSON File: 95 | ```bash 96 | ./cxl_feature_tracker.py --start-version v5.8 --end-version v5.9 --output changes.json --format json 97 | ``` 98 | 99 | ## Contributing 100 | 101 | Contributions are welcome! Please fork the repository and submit a pull request with your improvements. See [CONTRIBUTING](CONTRIBUTING.md) for more information and instructions. 102 | 103 | ## License 104 | 105 | This project is open source and available under the MIT License. -------------------------------------------------------------------------------- /cxl_feature_tracker.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | import requests 3 | import argparse 4 | import json 5 | 6 | def get_tags(token): 7 | """ 8 | Fetches all tags from the Linux kernel GitHub repository and filters out release candidate tags. 9 | """ 10 | url = "https://api.github.com/repos/torvalds/linux/tags" 11 | headers = {'Authorization': f'token {token}'} if token else {} 12 | tags = [] 13 | 14 | while url: 15 | try: 16 | response = requests.get(url, headers=headers) 17 | if response.status_code == 200: 18 | tags.extend([tag['name'] for tag in response.json()]) 19 | if 'next' in response.links: 20 | url = response.links['next']['url'] 21 | else: 22 | url = None 23 | else: 24 | print(f"Failed to fetch tags: {response.status_code} {response.reason}") 25 | return [] 26 | except requests.RequestException as e: 27 | print(f"Error fetching tags from GitHub: {e}") 28 | return [] 29 | 30 | stable_tags = sorted([tag for tag in tags if "-rc" not in tag], key=lambda x: x.strip('v')) 31 | return stable_tags 32 | 33 | def get_commits(from_tag, to_tag, token): 34 | """ 35 | Fetches commits related to CXL from specified directories between two tags and includes URL links to each commit. 36 | """ 37 | commits = [] 38 | for path in ["drivers/cxl", "drivers/dax"]: 39 | url = f"https://api.github.com/repos/torvalds/linux/commits?sha={to_tag}&path={path}" 40 | headers = {'Authorization': f'token {token}'} if token else {} 41 | while url: 42 | try: 43 | response = requests.get(url, headers=headers) 44 | if response.status_code == 200: 45 | page_commits = response.json() 46 | if 'next' in response.links: 47 | url = response.links['next']['url'] 48 | else: 49 | url = None 50 | for commit in page_commits: 51 | if 'commit' in commit: 52 | commit_message = commit['commit']['message'].split('\n')[0] # Extract only the title 53 | commit_url = commit['html_url'] # URL to the commit on GitHub 54 | commits.append((commit_message, commit_url)) 55 | else: 56 | print(f"Failed to fetch commits: {response.status_code} {response.reason} (URL: {url})") 57 | break 58 | except requests.RequestException as e: 59 | print(f"Error fetching commits from GitHub: {e}") 60 | break 61 | return commits 62 | 63 | def write_output(commits, output, format): 64 | """ 65 | Writes commits to a file or prints to the terminal based on the user's choice of format, includes markdown link format. 66 | """ 67 | try: 68 | if format == 'txt': 69 | with open(output, 'w') as file: 70 | for commit in commits: 71 | file.write(commit[0] + '\n') # Write only the commit message 72 | elif format == 'md': 73 | with open(output, 'w') as file: 74 | for commit_message, commit_url in commits: 75 | file.write(f"- [{commit_message}]({commit_url})\n") # Format as markdown link 76 | elif format == 'json': 77 | with open(output, 'w') as file: 78 | json.dump(commits, file, indent=4) # Dump the entire list as JSON 79 | except IOError as e: 80 | print(f"Error writing to file {output}: {e}") 81 | 82 | def main(args): 83 | """ 84 | Main function to process the command-line arguments and initiate fetching and output of CXL changes. 85 | """ 86 | try: 87 | token = args.ghtoken # GitHub API token 88 | from_version = args.start_version # Starting kernel version 89 | to_version = args.end_version # Ending kernel version 90 | tags = get_tags(token) # List of tags from the Linux kernel repository 91 | 92 | # If the user only wants to list tags, print them and exit 93 | if args.list_tags: 94 | if tags: 95 | if args.format == 'json': 96 | print(json.dumps(tags, indent=4)) 97 | else: 98 | print("\n".join(tags)) 99 | else: 100 | print("No tags found.") 101 | exit(0) 102 | 103 | # If no tags are available, exit the program 104 | if not tags: 105 | print("No tags available, exiting.") 106 | return 107 | 108 | # Validate the specified versions match the tags from the GitHub project 109 | def validate_version(version, tags): 110 | # Ensure the version starts with 'v' and exists in the list of tags 111 | if not version.startswith('v'): 112 | version = 'v' + version 113 | if version not in tags: 114 | raise ValueError(f"Invalid version '{version}'.") 115 | return version 116 | 117 | # Validate the specified versions and set defaults if not provided 118 | try: 119 | if from_version: 120 | from_version = validate_version(from_version, tags) 121 | if to_version: 122 | to_version = validate_version(to_version, tags) 123 | except ValueError as e: 124 | print(f"Error: {e}. Please ensure the specified versions exist in the repository tags.") 125 | return 126 | 127 | 128 | # If no versions are specified, default to the last two tags (latest and latest-1) 129 | if not from_version or not to_version: 130 | from_version = tags[-2] 131 | to_version = tags[-1] 132 | 133 | # Fetch commits related to CXL between the specified versions 134 | commits = get_commits(from_version, to_version, token) 135 | 136 | # Output the commits based on the user's choice of format 137 | if commits: 138 | if args.output: 139 | # If output file is specified, write to file according to the format 140 | write_output(commits, args.output, args.format) 141 | else: 142 | # No output file specified 143 | print(f"\nCXL related changes from Kernel {from_version} to {to_version}:") 144 | if args.format == 'md': 145 | # Output Markdown to STDOUT if format is 'md' 146 | for commit_message, commit_url in commits: 147 | print(f"- [{commit_message}]({commit_url})") 148 | elif args.format == 'json': 149 | # Output JSON to STDOUT if format is 'json' 150 | print(json.dumps(commits, indent=4)) 151 | elif args.verbose: 152 | # If verbose is specified, output detailed text to STDOUT 153 | for commit_message, commit_url in commits: 154 | print(f"- {commit_message} ({commit_url})") 155 | else: 156 | # Default behavior, output only commit messages to STDOUT 157 | print("- " + "\n- ".join(commit_message for commit_message, commit_url in commits)) 158 | else: 159 | print("No CXL related changes found.") 160 | except KeyboardInterrupt: 161 | print("Process interrupted by user.") 162 | sys.exit(1) 163 | 164 | if __name__ == "__main__": 165 | # Parse command-line arguments 166 | parser = argparse.ArgumentParser(description="Track CXL feature changes in the Linux kernel.") 167 | parser.add_argument('--ghtoken', type=str, help='GitHub API token for authenticated requests') 168 | parser.add_argument('--start-version', type=str, help='Starting kernel version') 169 | parser.add_argument('--end-version', type=str, help='Ending kernel version') 170 | parser.add_argument('--output', type=str, help='Output file name (optional)') 171 | parser.add_argument('--format', choices=['txt', 'md', 'json'], default='default', help='Output format: txt, md, json, or default (terminal)') 172 | parser.add_argument('--verbose', action='store_true', help='Display detailed commit messages instead of just titles') 173 | parser.add_argument('--list-tags', action='store_true', help='List all tags from the repository') 174 | args = parser.parse_args() 175 | 176 | # Call the main function with the parsed arguments 177 | main(args) 178 | 179 | --------------------------------------------------------------------------------