├── .env ├── README.md ├── mail2github.py └── requirements.txt /.env: -------------------------------------------------------------------------------- 1 | # .env File Example 2 | IMAP_SERVER=imap.example.com 3 | EMAIL_ACCOUNT=your.email@example.com 4 | EMAIL_PASSWORD=your password 5 | GITHUB_TOKEN=your-github-token 6 | DEFAULT_GITHUB_REPO_NAME=username/reponame 7 | DEFAULT_BRANCH=main 8 | EMAIL_SENDER_WHITELIST=email1@mydomain.com,email2@mydomain.com 9 | REPO_WHITELIST=mygithubhandle/repo1,mygithubhandle/repo2 10 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # mail2github 2 | Send email and create files in Github with content from the mail. 3 | 4 | Useful i.e. to archive your chatgpt answers on github wirhout the need to install anything on the client. 5 | 6 | ## Why email? 7 | - You can use the email as access constraint with SPF and DKIM 8 | - Many email providers have a Virus protection 9 | - You can use seen and unseen to activate or deactivate import in github 10 | - It is available nearly everywhere. No app needed 11 | 12 | ## How to start? 13 | 1. Get an email you can send things to. Keep the email adress private and secure. 14 | 2. Get connection parameters for the email provider (imap server name etc.) 15 | 3. Create a github repo 16 | 4. Get the github api token (repo permissions are enough) 17 | 5. Fill the .env file 18 | 6. Use pip to install requirements.txt 19 | 7. Start a cron mith mail2github.py 20 | 8. Send an email (for security reason from the target email adress itself). Your sender email should send SPF and DKIM 21 | 22 | This will read the emails and commit the email body to the github repo provided. There are contol markers you can use in the email subject: 23 | 24 | ## E-Mail subject 25 | 26 | The following control markers can be used in the email subject to specify metadata for creating or updating files in the GitHub repository. These control markers are optional, and you can combine them flexibly: 27 | 28 | 1. **[commit_msg:]** 29 | - Example: `[commit_msg:Initial Commit]` 30 | - This control marker allows you to specify the commit message used for the Git commit. If no message is provided, a default message will be used: "Automatically generated change". 31 | 32 | 2. **[branch:]** 33 | - Example: `[branch:feature/update-file]` 34 | - This control marker lets you define the target branch where the changes will be committed. If no branch is specified, the default branch (e.g., `main`) will be used. 35 | 36 | 3. **[author:]** 37 | - Example: `[author:John Doe]` 38 | - This control marker allows you to specify the name of the author making the change. Currently, this is only for documentation purposes and is not directly used in the code, but it may be useful for future features. 39 | 40 | 4. **[repo:]** 41 | - Example: `[repo:username/another-repository]` 42 | - This control marker allows you to specify the GitHub repository where the file should be uploaded. If no repository is specified, the default repository defined in the environment variables will be used. 43 | 44 | 5. **[tag:]** 45 | - Example: `[tag:v1.0.0]` 46 | - This control marker allows you to set a tag for the current commit. A Git tag will be created with the given name after the commit, which can be useful for versioning and releases. 47 | 48 | 6. **Path and Filename** 49 | - Example: `foldername/filename.txt` 50 | - This is the path within the repository and the filename of the file. This is the only required field in the subject. If no path is specified, the file will be placed in the root folder of the repository. 51 | 52 | ### Examples of Complete Email Subjects: 53 | 54 | 1. **Only Path and Filename (Mandatory)** 55 | - `Folder1/file.txt` 56 | - This places `file.txt` in the `Folder1` subfolder. 57 | 58 | 2. **Path and Filename with Commit Message and Branch** 59 | - `[commit_msg:Added new feature] [branch:feature/branch-name] Folder1/file.txt` 60 | - This places `file.txt` in the branch `feature/branch-name` and uses the commit message "Added new feature". 61 | 62 | 3. **Filename without Path (Root of the Repository)** 63 | - `[branch:main] file.txt` 64 | - This places `file.txt` in the root of the repository in the `main` branch. 65 | 66 | 4. **Path and Filename with All Options** 67 | - `[commit_msg:Bug fix] [branch:hotfix] [author:John Doe] [repo:username/project-repo] [tag:v1.0.1] Folder2/bugfix.txt` 68 | - This places `bugfix.txt` in the `Folder2` subfolder of the repository `username/project-repo`, in the branch `hotfix`, with the commit message "Bug fix" and the author "John Doe", and creates the tag `v1.0.1`. 69 | 70 | The control markers are all optional, except for the filename (and optionally the path). If you choose the root of the repository as the target, you only need to provide the filename. The control markers give you the flexibility to define the desired parameters for making changes and keep control over the Git operations. 71 | -------------------------------------------------------------------------------- /mail2github.py: -------------------------------------------------------------------------------- 1 | import imaplib 2 | import email 3 | from email.header import decode_header 4 | import os 5 | import re 6 | import datetime 7 | import logging 8 | from logging.handlers import RotatingFileHandler 9 | from github import Github 10 | from dotenv import load_dotenv 11 | import dns.resolver 12 | import spf 13 | import dkim 14 | 15 | # Load environment variables from the .env file 16 | load_dotenv() 17 | 18 | # Logging configuration 19 | log_dir = os.path.join(os.path.dirname(__file__), 'log') 20 | os.makedirs(log_dir, exist_ok=True) 21 | log_file_path = os.path.join(log_dir, 'email_to_github.log') 22 | handler = RotatingFileHandler(log_file_path, maxBytes=5*1024*1024, backupCount=5) 23 | logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s', handlers=[ 24 | handler, 25 | logging.StreamHandler() 26 | ]) 27 | 28 | # Define constants 29 | IMAP_SERVER = os.getenv("IMAP_SERVER") 30 | EMAIL_ACCOUNT = os.getenv("EMAIL_ACCOUNT") 31 | EMAIL_PASSWORD = os.getenv("EMAIL_PASSWORD") 32 | GITHUB_TOKEN = os.getenv("GITHUB_TOKEN") 33 | DEFAULT_GITHUB_REPO_NAME = os.getenv("DEFAULT_GITHUB_REPO_NAME") 34 | DEFAULT_BRANCH = os.getenv("DEFAULT_BRANCH", "main") 35 | WHITELIST = os.getenv("EMAIL_SENDER_WHITELIST").split(',') 36 | REPO_WHITELIST = os.getenv("REPO_WHITELIST").split(',') 37 | 38 | # Establish IMAP connection 39 | def connect_to_email(): 40 | mail = imaplib.IMAP4_SSL(IMAP_SERVER) 41 | mail.login(EMAIL_ACCOUNT, EMAIL_PASSWORD) 42 | return mail 43 | 44 | # Check for unread emails 45 | def check_unread_emails(mail): 46 | mail.select("inbox") 47 | status, messages = mail.search(None, '(UNSEEN)') 48 | if status != "OK": 49 | logging.info("No unread messages found.") 50 | return [] 51 | return messages[0].split() 52 | 53 | # Verify sender against whitelist 54 | def is_sender_allowed(msg): 55 | sender = email.utils.parseaddr(msg["From"])[1] 56 | if sender in WHITELIST: 57 | return True 58 | else: 59 | logging.warning(f"Unauthorized email sender: {sender}. Ignoring email.") 60 | return False 61 | 62 | # Verify SPF record 63 | def verify_spf(sender_ip, domain): 64 | try: 65 | result, explanation = spf.check2(sender_ip, domain, EMAIL_ACCOUNT) 66 | if result == 'pass': 67 | return True 68 | elif result == 'softfail': 69 | logging.warning(f"SPF softfail for domain {domain}. Proceeding with caution: {explanation}") 70 | return True # Optional: We can decide to proceed with caution 71 | else: 72 | logging.warning(f"SPF verification failed for domain {domain} with result: {result}. {explanation}") 73 | return False 74 | except Exception as e: 75 | logging.error(f"Error during SPF verification: {e}") 76 | return False 77 | 78 | # Verify DKIM signature 79 | def verify_dkim(raw_email): 80 | try: 81 | if dkim.verify(raw_email): 82 | return True 83 | else: 84 | logging.warning("DKIM verification failed.") 85 | return False 86 | except Exception as e: 87 | logging.error(f"Error during DKIM verification: {e}") 88 | return False 89 | 90 | # Process email and extract content 91 | def process_email(mail, email_id): 92 | status, data = mail.fetch(email_id, '(RFC822)') 93 | if status != "OK": 94 | logging.error("Error fetching the email.") 95 | return None, None, None, None, None, None, None 96 | 97 | msg = email.message_from_bytes(data[0][1]) 98 | raw_email = data[0][1] 99 | 100 | # Check if the sender is allowed 101 | sender = email.utils.parseaddr(msg["From"])[1] 102 | if not is_sender_allowed(msg): 103 | return None, None, None, None, None, None, None 104 | 105 | # Get sender IP and domain for SPF verification 106 | received_headers = msg.get_all('Received') 107 | if received_headers: 108 | sender_ip = re.findall(r'\[([0-9\.]+)\]', received_headers[-1]) 109 | sender_ip = sender_ip[0] if sender_ip else None 110 | else: 111 | sender_ip = None 112 | 113 | domain = email.utils.parseaddr(msg["From"])[1].split('@')[-1] 114 | 115 | # Robust SPF and DKIM verification 116 | spf_pass = False 117 | dkim_pass = False 118 | 119 | # SPF verification 120 | if sender_ip: 121 | spf_pass = verify_spf(sender_ip, domain) 122 | 123 | # DKIM verification 124 | dkim_pass = verify_dkim(raw_email) 125 | 126 | # Log results and determine if we proceed 127 | if not spf_pass and not dkim_pass: 128 | logging.error(f"Both SPF and DKIM verification failed for sender '{sender}'. Ignoring email.") 129 | return None, None, None, None, None, None, None 130 | elif spf_pass and not dkim_pass: 131 | logging.warning(f"SPF verification passed, but DKIM verification failed for sender '{sender}'. Proceeding with caution.") 132 | elif not spf_pass and dkim_pass: 133 | logging.warning(f"DKIM verification passed, but SPF verification failed for sender '{sender}'. Proceeding with caution.") 134 | else: 135 | logging.info(f"Both SPF and DKIM verification passed for sender '{sender}'.") 136 | 137 | subject, encoding = decode_header(msg["Subject"])[0] 138 | if isinstance(subject, bytes): 139 | subject = subject.decode(encoding if encoding else 'utf-8') 140 | 141 | # Extract control markers from the subject 142 | commit_msg = "Automatically generated change" 143 | branch = DEFAULT_BRANCH 144 | author = "Unknown" 145 | repo_name = DEFAULT_GITHUB_REPO_NAME 146 | tag_name = None 147 | 148 | # Updated regex to allow more flexible subjects and support tags 149 | match = re.match(r"(?:\[commit_msg:(?P.*?)\])?\s*(?:\[branch:(?P.*?)\])?\s*(?:\[author:(?P.*?)\])?\s*(?:\[repo:(?P.*?)\])?\s*(?:\[tag:(?P.*?)\])?\s*(?P.+\..+)$", subject) 150 | if not match: 151 | logging.error(f"Subject not in expected format for sender '{sender}'.") 152 | return None, None, None, None, None, None, None 153 | 154 | if match.group("commit_msg"): 155 | commit_msg = match.group("commit_msg").strip() 156 | if match.group("branch"): 157 | branch = match.group("branch").strip() 158 | if match.group("author"): 159 | author = match.group("author").strip() 160 | if match.group("repo"): 161 | repo_name = match.group("repo").strip() 162 | if match.group("tag"): 163 | tag_name = match.group("tag").strip() 164 | filename = match.group("filename").strip() 165 | 166 | # Verify if the repository is whitelisted 167 | if repo_name not in REPO_WHITELIST: 168 | logging.error(f"Repository '{repo_name}' is not whitelisted for sender '{sender}'. Ignoring email.") 169 | return None, None, None, None, None, None, None 170 | 171 | # Log the processing details 172 | logging.info(f"Processing email from '{sender}' for repository '{repo_name}', branch '{branch}', and filename '{filename}'.") 173 | 174 | # Extract message body 175 | body = "" 176 | if msg.is_multipart(): 177 | for part in msg.walk(): 178 | content_type = part.get_content_type() 179 | content_disposition = str(part.get("Content-Disposition")) 180 | 181 | if content_type == "text/plain" and "attachment" not in content_disposition: 182 | body = part.get_payload(decode=True).decode() 183 | break 184 | else: 185 | body = msg.get_payload(decode=True).decode() 186 | 187 | return (sender, filename, body, commit_msg, branch, repo_name, tag_name) 188 | 189 | # Create or update file in GitHub repository 190 | def write_to_github_repo(sender, path, filename, content, commit_msg, branch, repo_name, tag_name): 191 | g = Github(GITHUB_TOKEN) 192 | repo = g.get_repo(repo_name) 193 | 194 | # Ensure branch exists 195 | try: 196 | repo.get_branch(branch) 197 | except Exception: 198 | source = repo.get_branch(DEFAULT_BRANCH) 199 | repo.create_git_ref(ref=f"refs/heads/{branch}", sha=source.commit.sha) 200 | 201 | # Define full path for the file in GitHub 202 | full_path = filename if not path else f"{path}/{filename}" 203 | 204 | # Ensure path exists 205 | if path: 206 | try: 207 | repo.get_contents(path, ref=branch) 208 | except Exception as e: 209 | # Path does not exist, create it recursively 210 | parts = path.split('/') 211 | current_path = "" 212 | for part in parts: 213 | current_path = f"{current_path}/{part}" if current_path else part 214 | try: 215 | repo.get_contents(current_path, ref=branch) 216 | except Exception: 217 | repo.create_file(f"{current_path}/.gitkeep", f"Create directory {current_path}", "", branch=branch) 218 | 219 | try: 220 | # Check if the file already exists to create a new version 221 | contents = repo.get_contents(full_path, ref=branch) 222 | # Update the file to create a new version 223 | repo.update_file(contents.path, commit_msg, content, contents.sha, branch=branch) 224 | logging.info(f"File '{filename}' updated in repository '{repo_name}' on branch '{branch}' by sender '{sender}'. Commit message: '{commit_msg}'") 225 | except Exception as e: 226 | # File does not exist, create it 227 | repo.create_file(full_path, commit_msg, content, branch=branch) 228 | logging.info(f"File '{filename}' created in repository '{repo_name}' on branch '{branch}' by sender '{sender}'. Commit message: '{commit_msg}'") 229 | 230 | # Create a tag if specified 231 | if tag_name: 232 | try: 233 | repo.create_git_tag_and_release(tag=tag_name, tag_message=f"Tag {tag_name}", release_name=tag_name, release_message=commit_msg, object=repo.get_branch(branch).commit.sha, type="commit") 234 | logging.info(f"Tag '{tag_name}' created and added to the commit in repository '{repo_name}' by sender '{sender}'.") 235 | except Exception as e: 236 | logging.error(f"Error creating tag '{tag_name}' in repository '{repo_name}' by sender '{sender}': {e}") 237 | 238 | # Main program 239 | def main(): 240 | mail = connect_to_email() 241 | email_ids = check_unread_emails(mail) 242 | 243 | for e_id in email_ids: 244 | sender, filename, body, commit_msg, branch, repo_name, tag_name = process_email(mail, e_id) 245 | if filename and body: 246 | write_to_github_repo(sender, None, filename, body, commit_msg, branch, repo_name, tag_name) 247 | 248 | mail.logout() 249 | 250 | if __name__ == "__main__": 251 | main() 252 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | certifi==2024.8.30 2 | cffi==1.17.1 3 | charset-normalizer==3.4.0 4 | cryptography==43.0.3 5 | Deprecated==1.2.14 6 | dkimpy==1.1.8 7 | dnspython==2.7.0 8 | idna==3.10 9 | imaplib2==3.6 10 | pycparser==2.22 11 | PyGithub==2.4.0 12 | PyJWT==2.9.0 13 | PyNaCl==1.5.0 14 | pyspf==2.0.14 15 | python-dotenv==1.0.1 16 | requests==2.32.3 17 | typing_extensions==4.12.2 18 | urllib3==2.2.3 19 | wrapt==1.16.0 20 | --------------------------------------------------------------------------------