├── .gitignore ├── .travis.yml ├── LICENSE ├── README.md ├── gmailfilterrecipes ├── __init__.py ├── client │ ├── __init__.py │ ├── index.py │ └── static │ │ ├── coffee │ │ └── recipes.coffee │ │ └── html │ │ └── index.html ├── jsonschemas.py ├── tests │ ├── __init__.py │ ├── available-filters.yml │ ├── data │ │ ├── filters.xml │ │ └── user_recipe_set.yml │ └── tests.py └── xmlgeneration.py ├── gmailfilterxml ├── __init__.py ├── api.py ├── tests │ ├── __init__.py │ ├── many-filters.xml │ ├── single-filter.xml │ └── tests.py └── xmlschemas.py ├── recipesets └── default.yml ├── requirements.txt ├── run_server.py ├── settings.ini └── utils ├── __init__.py └── unittest.py /.gitignore: -------------------------------------------------------------------------------- 1 | *.pyc 2 | .idea/ 3 | 4 | *.js 5 | *.js.map 6 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: python 2 | python: 3 | - "2.7" 4 | install: "pip install -r requirements.txt" 5 | script: 6 | - "python -m unittest gmailfilterxml.tests gmailfilterrecipes.tests" 7 | - "python -m doctest README.md" 8 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2016 Dimagi 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Gmail Filters 2 | 3 | Python library and simple web app for creating GMail filters. 4 | 5 | See [GMail help](https://support.google.com/mail/answer/6579) for info on importing filters into GMail. 6 | 7 | ## Install 8 | 9 | To install requirements: 10 | 11 | ```bash 12 | pip install -r requirements.txt 13 | npm install -g coffee-script 14 | ``` 15 | 16 | ## Compile Coffee 17 | 18 | ```bash 19 | coffee --compile gmailfilterrecipes/client/static/coffee/*.coffee 20 | ``` 21 | 22 | 23 | ## Run 24 | 25 | Just run 26 | 27 | ```bash 28 | python run_server.py 29 | ``` 30 | 31 | and the web frontend should be running at http://localhost:8080/. 32 | 33 | ## Configure your recipeset 34 | 35 | Each organization or team within an organization will have a different set of filters that are relevant to them; 36 | each member of that team may then want to further tweak the base set to conform to their particular needs. For example, 37 | everyone may want a basic filter labeling emails that contain their name, but each person's name is different, so each person's 38 | filter must be tweaked accordingly. 39 | 40 | In this framework, the shared filter descriptions are codified as "recipe sets", which are then displayed to the individual through 41 | a web interface, so that they may customize it to themself by simply filling out a web form. 42 | 43 | An example recipe set (which is used by the dev team at Dimagi) is included in this repo at `recipesets/default.yml`. 44 | To use your own instead, simply change the `gmailfilterrecipes.filters_yml` option in `settings.ini` to point to your own. 45 | 46 | A recipeset yaml file has two toplevel components, `options` and `recipes`: `options` let you specify filter options that can be shared 47 | across all your recipes, which are then defined under `recipes`. A recipe looks somethin like this: 48 | 49 | ```yaml 50 | - 51 | label: Mails directly to me 52 | custom_options: 53 | - 54 | key: names 55 | label: "List your email addresses" 56 | type: list 57 | match: 58 | to: "{{ names|join(' OR ') }}" 59 | filters: 60 | - label: =^.^= 61 | shouldNeverSpam: true 62 | shouldAlwaysMarkAsImportant: true 63 | ``` 64 | 65 | Each recipe specifies a row in the web form that will be shown to the individual. In this case, that row would be labelled 66 | "Mails directly to me", they would require the individual to specify a list of email addresses (could also be just a single one), 67 | and would create a filter that matches on those emails, applies a gmail label named "=^.^=", keeps it from going to spam, 68 | and marks it as important. 69 | 70 | ## Library Usage 71 | 72 | If you aren't interested in the front end, you can use `gmailfilterxml` 73 | as a library for reading and writing gmail filter xml files. 74 | 75 | To create a simple gmail filter: 76 | 77 | ```python 78 | >>> from gmailfilterxml import GmailFilterSet, GmailFilter 79 | >>> import datetime 80 | >>> 81 | >>> 82 | >>> filter_set = GmailFilterSet( 83 | ... author_name='Danny Roberts', 84 | ... author_email='droberts@dimagi.com', 85 | ... updated_timestamp=datetime.datetime(2014, 9, 19, 17, 40, 28), 86 | ... filters=[ 87 | ... GmailFilter( 88 | ... id='1286460749536', 89 | ... from_='noreply@github.com', 90 | ... label='github', 91 | ... shouldArchive=True, 92 | ... ) 93 | ... ] 94 | ... ) 95 | >>> print filter_set.to_xml(pretty=True), 96 | 97 | 98 | Mail Filters 99 | tag:mail.google.com,2008:filters:1286460749536 100 | 2014-09-19T17:40:28Z 101 | 102 | Danny Roberts 103 | droberts@dimagi.com 104 | 105 | 106 | 107 | Mail Filter 108 | tag:mail.google.com,2008:filter:1286460749536 109 | 2014-09-19T17:40:28Z 110 | 111 | 112 | 113 | 114 | 115 | 116 | >>> 117 | ``` 118 | -------------------------------------------------------------------------------- /gmailfilterrecipes/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dimagi/gmail-filters/bea5a547a35c786b155bcddfa40843680ad8bbe0/gmailfilterrecipes/__init__.py -------------------------------------------------------------------------------- /gmailfilterrecipes/client/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dimagi/gmail-filters/bea5a547a35c786b155bcddfa40843680ad8bbe0/gmailfilterrecipes/client/__init__.py -------------------------------------------------------------------------------- /gmailfilterrecipes/client/index.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | import json 3 | import bottle 4 | import yaml 5 | from gmailfilterrecipes.jsonschemas import UserRecipeSet, RecipeSet 6 | from gmailfilterrecipes.xmlgeneration import generate_gmail_fitler_set 7 | 8 | app = bottle.default_app() 9 | 10 | app.config.load_config('settings.ini') 11 | 12 | @bottle.route('/') 13 | def index(): 14 | return bottle.static_file('index.html', 15 | root='gmailfilterrecipes/client/static/html/') 16 | 17 | 18 | @bottle.route('/filters.json') 19 | def filters_json(): 20 | filters_yml = app.config.get('gmailfilterrecipes.filters_yml') 21 | with open(filters_yml) as f: 22 | filters = yaml.load(f) 23 | user_recipe_set = UserRecipeSet.from_recipe_set(RecipeSet.wrap(filters)) 24 | return json.dumps(user_recipe_set.to_json()) 25 | 26 | 27 | @bottle.post('/filters.xml') 28 | def filters_xml(): 29 | bottle.response.headers.update({ 30 | 'Content-Type': 'application/xml; charset="utf-8"', 31 | 'Content-Disposition': 'attachment; filename="gmailFilters.xml"', 32 | }) 33 | 34 | recipe_set_json = ( 35 | bottle.request.json 36 | or json.loads(bottle.request.POST.get('recipeSet')) 37 | ) 38 | user_recipe_set = UserRecipeSet.wrap(recipe_set_json) 39 | gmail_filter_set = generate_gmail_fitler_set(user_recipe_set) 40 | 41 | return gmail_filter_set.to_xml(pretty=True) 42 | 43 | 44 | @bottle.route('/static/') 45 | def static(path): 46 | return bottle.static_file(path, root='gmailfilterrecipes/client/static') 47 | -------------------------------------------------------------------------------- /gmailfilterrecipes/client/static/coffee/recipes.coffee: -------------------------------------------------------------------------------- 1 | recipesApp = angular.module('recipesApp', []) 2 | 3 | recipesApp.controller 'RecipesController', ($scope, $http, $window) -> 4 | $scope.recipeSetJson = '' 5 | $scope.show = false 6 | 7 | $http.get('/filters.json').success (data) -> 8 | $scope.recipeSet = data 9 | $scope.recipeSetJson = angular.toJson data, true 10 | 11 | $scope.filtersXml = '' 12 | $scope.refreshFiltersXml = -> 13 | $http.post('/filters.xml', $scope.recipeSet).success (data) -> 14 | $scope.filtersXml = data 15 | $scope.downloadFiltersXml = -> 16 | $scope.filtersXml = '' 17 | $window.post('/filters.xml?', $scope.recipeSet) 18 | $scope.updateForm = -> 19 | $scope.recipeSet = JSON.parse $scope.recipeSetJson 20 | $scope.updateRecipe = -> 21 | $scope.recipeSetJson = angular.toJson $scope.recipeSet, true 22 | $scope.showRecipe = -> 23 | $scope.show = true 24 | $scope.selectAll = (value) -> 25 | # value should be true or false 26 | for recipe in $scope.recipeSet.recipes 27 | recipe.selected = value -------------------------------------------------------------------------------- /gmailfilterrecipes/client/static/html/index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |
9 |
10 |

Gmail Filters

11 |
12 |
13 |
14 | Show Recipe Source 15 |
16 |
17 | 18 |
19 | 20 |
21 |
22 |
23 |
24 | 27 | 30 |
31 |
32 |
33 |
34 |
35 |
36 |
37 | 38 | 39 | 47 | 48 | 49 | 50 | 51 | 52 | 55 | 56 | 75 | 76 | 77 | 78 | 79 | 80 | 81 |
40 | Select 41 |
42 | 43 | 44 |
45 | 46 | 		RecipeOptions
53 | 54 | {{ recipe.label }} 57 |
    58 |
  • 59 | 60 | 62 | 63 | 64 | 66 | 67 | 68 | 69 | 70 | 71 | 72 |
    • 73 |
74 |
82 |
83 |
84 |
85 | 86 |
87 | 88 | 89 |
90 |
91 |
92 | 
{{ filtersXml }}
93 |
94 | 95 | 96 | 97 | 98 | -------------------------------------------------------------------------------- /gmailfilterrecipes/jsonschemas.py: -------------------------------------------------------------------------------- 1 | import jsonobject 2 | from jsonobject.base_properties import DefaultProperty 3 | import gmailfilterxml 4 | 5 | 6 | class Base(jsonobject.JsonObject): 7 | _allow_dynamic_properties = False 8 | 9 | 10 | FILTER_KEY_TO_TYPE = { 11 | name: py_type 12 | for name, py_type in gmailfilterxml.PROPERTIES 13 | } 14 | 15 | 16 | def validate_filter(filter_dict): 17 | for key, value in filter_dict.items(): 18 | assert key in FILTER_KEY_TO_TYPE, (key, FILTER_KEY_TO_TYPE.keys()) 19 | assert isinstance(value, FILTER_KEY_TO_TYPE[key]), ( 20 | value, FILTER_KEY_TO_TYPE[key] 21 | ) 22 | 23 | 24 | def validate_filters(filter_dict_list): 25 | for filter_dict in filter_dict_list: 26 | validate_filter(filter_dict) 27 | 28 | 29 | def _id_prefix_valid(value): 30 | assert len(value) == 6 31 | assert all(c in '0123456789' for c in value) 32 | 33 | 34 | class RecipeOption(Base): 35 | key = jsonobject.StringProperty(required=True) 36 | type = jsonobject.StringProperty(required=True, 37 | choices=['list', 'bool', 'inverted-bool']) 38 | label = jsonobject.StringProperty(required=True) 39 | filters = jsonobject.ListProperty(dict, validators=validate_filters) 40 | 41 | 42 | class Recipe(Base): 43 | label = jsonobject.StringProperty(required=True) 44 | id = jsonobject.StringProperty(validators=_id_prefix_valid) 45 | options = jsonobject.ListProperty(unicode) 46 | custom_options = jsonobject.ListProperty(RecipeOption) 47 | match = jsonobject.DictProperty(validators=validate_filter) 48 | filters = jsonobject.ListProperty(dict, validators=validate_filters) 49 | 50 | 51 | class RecipeSet(Base): 52 | id_prefix = jsonobject.StringProperty(validators=_id_prefix_valid) 53 | options = jsonobject.ListProperty(RecipeOption) 54 | recipes = jsonobject.ListProperty(Recipe) 55 | 56 | 57 | # User* are the above classes with user's responses 58 | 59 | class UserRecipeOption(RecipeOption): 60 | value = DefaultProperty() 61 | 62 | 63 | class UserRecipe(Recipe): 64 | selected = jsonobject.BooleanProperty() 65 | options = jsonobject.ListProperty(UserRecipeOption) 66 | custom_options = None 67 | 68 | 69 | class UserRecipeSet(RecipeSet): 70 | options = None 71 | recipes = jsonobject.ListProperty(UserRecipe) 72 | 73 | @classmethod 74 | def from_recipe_set(cls, recipe_set): 75 | options_by_key = {option.key: option for option in recipe_set.options} 76 | 77 | def get_default_value(type): 78 | return {'inverted-bool': lambda: True, 'list': list, 'bool': lambda: False}[type]() 79 | return UserRecipeSet( 80 | id_prefix=recipe_set.id_prefix, 81 | recipes=[ 82 | UserRecipe( 83 | selected=True, 84 | label=recipe.label, 85 | id=recipe.id or "{0:06d}".format(i), 86 | options=[ 87 | UserRecipeOption(value=get_default_value(option.type), 88 | **option.to_json()) 89 | for option in ( 90 | recipe.custom_options 91 | + [options_by_key[key] for key in recipe.options] 92 | ) 93 | ], 94 | match=recipe.match, 95 | filters=recipe.filters, 96 | ) for i, recipe in enumerate(recipe_set.recipes) 97 | ], 98 | ) 99 | -------------------------------------------------------------------------------- /gmailfilterrecipes/tests/__init__.py: -------------------------------------------------------------------------------- 1 | from .tests import * 2 | -------------------------------------------------------------------------------- /gmailfilterrecipes/tests/available-filters.yml: -------------------------------------------------------------------------------- 1 | # Every gmail filter has its own 13-character numerical id. 2 | # In order to support a download-edit-reupload workflow, 3 | # it's important that the same conceptual filters always use the same ids. 4 | # Thus, we use the following scheme to construct an id 5 | # <6-digit id_prefix> + <6-digit recipe id> + 6 | # 7 | # When the program receives a downloaded xml file, 8 | # all filters starting with id_prefix 9 | # are presumed to be managed by the program. 10 | # Since I don't think gmail filter uplaod supports deleting filters, 11 | # filters the program believes should be deleted on update 12 | # will be written out as empty filters, i.e. filters that perform no action 13 | # that way, they user can manually delete them if they wish. 14 | # 15 | # I just did a preliminary check, and it seems that there's... 16 | # ...no way to even _replace_ a given filter through a gmail upload. 17 | # Not sure if I'm missing some semantics here, since that seems really odd, 18 | # and renders this whole id thing useless. 19 | 20 | id_prefix: '346244' 21 | options: 22 | # you can list options here that you want to be globally available 23 | # for any of the filter "recipes" below to reference 24 | # (this does not mean they actually show up on every filter) 25 | - 26 | key: keywords 27 | label: "What keywords do you want to see in your inbox?" 28 | type: list 29 | - 30 | key: archive 31 | label: Do you want to see these in your inbox? 32 | # inverted bool means checked => false, unchecked => true 33 | type: inverted-bool 34 | filters: 35 | # when a recipe uses an option that has a 'filters' property 36 | # these filters are added to the recipe's filters 37 | # (in the case of a bool, only if the bool is true) 38 | - 39 | to: -me 40 | shouldArchive: true 41 | recipes: 42 | # each one of these represents a "filter" as it's presented to the user 43 | # but they may create any number of actual gmail filters 44 | # in order to get that single conceptual job done 45 | - 46 | label: Timecards 47 | id: '000001' 48 | custom_options: 49 | - 50 | key: names 51 | label: "Whose timecards do you want to see? (List first or last names as they appear in fogbugz)" 52 | type: list 53 | # all the filter properties included in match 54 | # are applied to every single one of the filters below 55 | match: 56 | from: reports@dimagi.com 57 | filters: 58 | # one gmail filter is created per item in this list 59 | - label: Timecard 60 | - 61 | subject: '{% for name in names %}-"{{ name }}" {% endfor %}' 62 | shouldArchive: true 63 | - 64 | label: Pingdom 65 | id: '000002' 66 | options: 67 | - archive 68 | match: 69 | from: alert@pingdom.com 70 | filters: 71 | - label: Pingdom 72 | - 73 | label: FogBugz Job Applicants 74 | id: '000003' 75 | options: 76 | - archive 77 | match: 78 | to: jobslisteners@dimagi.com 79 | filters: 80 | - label: Job Applicants 81 | - 82 | label: Uservoice 83 | id: '000004' 84 | options: 85 | - archive 86 | match: 87 | to: dev+uservoice@dimagi.com 88 | filters: 89 | - label: UserVoice 90 | - 91 | label: Server Emails 92 | id: '000005' 93 | options: 94 | - archive 95 | match: 96 | to: commcarehq-ops@dimagi.com 97 | filters: 98 | - label: Server 99 | - 100 | label: Travis Notifications 101 | id: '000006' 102 | options: 103 | - archive 104 | match: 105 | from: notifications@travis-ci.org 106 | filters: 107 | - label: Travis 108 | - 109 | label: Office 110 | id: '000007' 111 | match: 112 | to: office@dimagi.com 113 | filters: 114 | - label: Office 115 | - 116 | label: CommCare Users 117 | id: '000008' 118 | options: 119 | - archive 120 | match: 121 | to: commcare-users@googlegroups.com 122 | filters: 123 | - label: CommCare Users 124 | - 125 | label: From a Dimagi person 126 | id: '000009' 127 | match: 128 | from: "*@dimagi.com -commcarehq -fogbugz -devops -notifications -ops" 129 | filters: 130 | - label: ♥︎ 131 | - 132 | label: Project Reports 133 | id: '000010' 134 | match: 135 | to: project-reports@dimagi.com 136 | filters: 137 | - label: Project Reports 138 | - 139 | label: Information 140 | id: '000011' 141 | options: 142 | - keywords 143 | match: 144 | to: "information@dimagi.com OR info@dimagi.com" 145 | filters: 146 | - label: Information 147 | - doesNotHaveTheWord: '{% for keyword in keywords %}"{{ keyword }}" {% endfor %}' 148 | - 149 | label: Allstaff 150 | id: '000012' 151 | match: 152 | to: allstaff@dimagi.com 153 | filters: 154 | - label: Allstaff 155 | -------------------------------------------------------------------------------- /gmailfilterrecipes/tests/data/filters.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | Mail Filters 4 | tag:mail.google.com,2008:filters:3462440000010,3462440000011,3462440000020,3462440000021,3462440000030,3462440000031,3462440000040,3462440000041,3462440000050,3462440000051,3462440000060,3462440000061,3462440000070,3462440000080,3462440000081,3462440000090,3462440000100,3462440000110,3462440000111,3462440000120 5 | 2014-10-27T02:20:58Z 6 | 7 | 8 | 9 | 10 | 11 | 12 | Mail Filter 13 | tag:mail.google.com,2008:filter:3462440000010 14 | 2014-10-27T02:20:58Z 15 | 16 | 17 | 18 | 19 | 20 | 21 | Mail Filter 22 | tag:mail.google.com,2008:filter:3462440000011 23 | 2014-10-27T02:20:58Z 24 | 25 | 26 | 27 | 28 | 29 | 30 | Mail Filter 31 | tag:mail.google.com,2008:filter:3462440000020 32 | 2014-10-27T02:20:58Z 33 | 34 | 35 | 36 | 37 | 38 | 39 | Mail Filter 40 | tag:mail.google.com,2008:filter:3462440000021 41 | 2014-10-27T02:20:58Z 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | Mail Filter 50 | tag:mail.google.com,2008:filter:3462440000030 51 | 2014-10-27T02:20:58Z 52 | 53 | 54 | 55 | 56 | 57 | 58 | Mail Filter 59 | tag:mail.google.com,2008:filter:3462440000031 60 | 2014-10-27T02:20:58Z 61 | 62 | 63 | 64 | 65 | 66 | 67 | Mail Filter 68 | tag:mail.google.com,2008:filter:3462440000040 69 | 2014-10-27T02:20:58Z 70 | 71 | 72 | 73 | 74 | 75 | 76 | Mail Filter 77 | tag:mail.google.com,2008:filter:3462440000041 78 | 2014-10-27T02:20:58Z 79 | 80 | 81 | 82 | 83 | 84 | 85 | Mail Filter 86 | tag:mail.google.com,2008:filter:3462440000050 87 | 2014-10-27T02:20:58Z 88 | 89 | 90 | 91 | 92 | 93 | 94 | Mail Filter 95 | tag:mail.google.com,2008:filter:3462440000051 96 | 2014-10-27T02:20:58Z 97 | 98 | 99 | 100 | 101 | 102 | 103 | Mail Filter 104 | tag:mail.google.com,2008:filter:3462440000060 105 | 2014-10-27T02:20:58Z 106 | 107 | 108 | 109 | 110 | 111 | 112 | Mail Filter 113 | tag:mail.google.com,2008:filter:3462440000061 114 | 2014-10-27T02:20:58Z 115 | 116 | 117 | 118 | 119 | 120 | 121 | 122 | Mail Filter 123 | tag:mail.google.com,2008:filter:3462440000070 124 | 2014-10-27T02:20:58Z 125 | 126 | 127 | 128 | 129 | 130 | 131 | Mail Filter 132 | tag:mail.google.com,2008:filter:3462440000080 133 | 2014-10-27T02:20:58Z 134 | 135 | 136 | 137 | 138 | 139 | 140 | Mail Filter 141 | tag:mail.google.com,2008:filter:3462440000081 142 | 2014-10-27T02:20:58Z 143 | 144 | 145 | 146 | 147 | 148 | 149 | Mail Filter 150 | tag:mail.google.com,2008:filter:3462440000090 151 | 2014-10-27T02:20:58Z 152 | 153 | 154 | 155 | 156 | 157 | 158 | Mail Filter 159 | tag:mail.google.com,2008:filter:3462440000100 160 | 2014-10-27T02:20:58Z 161 | 162 | 163 | 164 | 165 | 166 | 167 | Mail Filter 168 | tag:mail.google.com,2008:filter:3462440000110 169 | 2014-10-27T02:20:58Z 170 | 171 | 172 | 173 | 174 | 175 | 176 | Mail Filter 177 | tag:mail.google.com,2008:filter:3462440000111 178 | 2014-10-27T02:20:58Z 179 | 180 | 181 | 182 | 183 | 184 | Mail Filter 185 | tag:mail.google.com,2008:filter:3462440000120 186 | 2014-10-27T02:20:58Z 187 | 188 | 189 | 190 | 191 | 192 | -------------------------------------------------------------------------------- /gmailfilterrecipes/tests/data/user_recipe_set.yml: -------------------------------------------------------------------------------- 1 | id_prefix: '346244' 2 | options: [] 3 | recipes: 4 | - custom_options: [] 5 | filters: 6 | - {label: Timecard} 7 | - {shouldArchive: true, subject: '{% for name in names %}-"{{ name }}" {% endfor %}'} 8 | id: '000001' 9 | label: Timecards 10 | match: {from: reports@dimagi.com} 11 | options: 12 | - filters: [] 13 | key: names 14 | label: Whose timecards do you want to see? (List first or last names as they appear in fogbugz) 15 | type: list 16 | value: [] 17 | selected: true 18 | - custom_options: [] 19 | filters: 20 | - {label: Pingdom} 21 | id: '000002' 22 | label: Pingdom 23 | match: {from: alert@pingdom.com} 24 | options: 25 | - filters: 26 | - {shouldArchive: true, to: -me} 27 | key: archive 28 | label: Do you want to see these in your inbox? 29 | type: inverted-bool 30 | value: true 31 | selected: true 32 | - custom_options: [] 33 | filters: 34 | - {label: Job Applicants} 35 | id: '000003' 36 | label: FogBugz Job Applicants 37 | match: {to: jobslisteners@dimagi.com} 38 | options: 39 | - filters: 40 | - {shouldArchive: true, to: -me} 41 | key: archive 42 | label: Do you want to see these in your inbox? 43 | type: inverted-bool 44 | value: true 45 | selected: true 46 | - custom_options: [] 47 | filters: 48 | - {label: UserVoice} 49 | id: '000004' 50 | label: Uservoice 51 | match: {to: dev+uservoice@dimagi.com} 52 | options: 53 | - filters: 54 | - {shouldArchive: true, to: -me} 55 | key: archive 56 | label: Do you want to see these in your inbox? 57 | type: inverted-bool 58 | value: true 59 | selected: true 60 | - custom_options: [] 61 | filters: 62 | - {label: Server} 63 | id: '000005' 64 | label: Server Emails 65 | match: {to: commcarehq-ops@dimagi.com} 66 | options: 67 | - filters: 68 | - {shouldArchive: true, to: -me} 69 | key: archive 70 | label: Do you want to see these in your inbox? 71 | type: inverted-bool 72 | value: true 73 | selected: true 74 | - custom_options: [] 75 | filters: 76 | - {label: Travis} 77 | id: '000006' 78 | label: Travis Notifications 79 | match: {from: notifications@travis-ci.org} 80 | options: 81 | - filters: 82 | - {shouldArchive: true, to: -me} 83 | key: archive 84 | label: Do you want to see these in your inbox? 85 | type: inverted-bool 86 | value: true 87 | selected: true 88 | - custom_options: [] 89 | filters: 90 | - {label: Office} 91 | id: '000007' 92 | label: Office 93 | match: {to: office@dimagi.com} 94 | options: [] 95 | selected: true 96 | - custom_options: [] 97 | filters: 98 | - {label: CommCare Users} 99 | id: 000008 100 | label: CommCare Users 101 | match: {to: commcare-users@googlegroups.com} 102 | options: 103 | - filters: 104 | - {shouldArchive: true, to: -me} 105 | key: archive 106 | label: Do you want to see these in your inbox? 107 | type: inverted-bool 108 | value: true 109 | selected: true 110 | - custom_options: [] 111 | filters: 112 | - {label: "\u2665\uFE0E"} 113 | id: 000009 114 | label: From a Dimagi person 115 | match: {from: '*@dimagi.com -commcarehq -fogbugz -devops -notifications -ops'} 116 | options: [] 117 | selected: true 118 | - custom_options: [] 119 | filters: 120 | - {label: Project Reports} 121 | id: '000010' 122 | label: Project Reports 123 | match: {to: project-reports@dimagi.com} 124 | options: [] 125 | selected: true 126 | - custom_options: [] 127 | filters: 128 | - {label: Information} 129 | - {doesNotHaveTheWord: '{% for keyword in keywords %}"{{ keyword }}" {% endfor %}'} 130 | id: '000011' 131 | label: Information 132 | match: {to: information@dimagi.com OR info@dimagi.com} 133 | options: [] 134 | selected: true 135 | - custom_options: [] 136 | filters: 137 | - {label: Allstaff} 138 | id: '000012' 139 | label: Allstaff 140 | match: {to: allstaff@dimagi.com} 141 | options: [] 142 | selected: true 143 | -------------------------------------------------------------------------------- /gmailfilterrecipes/tests/tests.py: -------------------------------------------------------------------------------- 1 | import datetime 2 | import yaml 3 | import os 4 | from unittest import TestCase 5 | from gmailfilterrecipes import jsonschemas 6 | from gmailfilterrecipes.jsonschemas import ( 7 | UserRecipe, 8 | UserRecipeOption, 9 | UserRecipeSet, 10 | ) 11 | from gmailfilterrecipes.xmlgeneration import ( 12 | generate_gmail_filters, 13 | generate_gmail_fitler_set, 14 | ) 15 | from gmailfilterxml import GmailFilter 16 | from utils.unittest import XmlTest 17 | 18 | 19 | class TestJsonSchemasTest(TestCase): 20 | def test(self): 21 | with open(os.path.join(os.path.dirname(__file__), 22 | 'available-filters.yml')) as f: 23 | recipe_set = yaml.load(f) 24 | self.assertIsInstance(recipe_set, dict) 25 | self.assertDictEqual( 26 | jsonschemas.RecipeSet.wrap(recipe_set).to_json(), recipe_set) 27 | 28 | 29 | class XmlGenerationTest(XmlTest): 30 | 31 | maxDiff = None 32 | 33 | @classmethod 34 | def setUpClass(cls): 35 | cls.archive_option = UserRecipeOption(yaml.load(""" 36 | key: archive 37 | label: Do you want to see these in your inbox? 38 | type: inverted-bool 39 | filters: 40 | - to: -me 41 | shouldArchive: yes 42 | value: yes 43 | """)) 44 | cls.id_prefix = '346244' 45 | 46 | def _test(self, recipe, expected_gmail_filters): 47 | recipe = UserRecipe.wrap(recipe) 48 | expected_gmail_filters = [GmailFilter(**kwargs) 49 | for kwargs in expected_gmail_filters] 50 | self.assertListEqual( 51 | generate_gmail_filters(recipe, id_prefix=self.id_prefix), 52 | expected_gmail_filters, 53 | ) 54 | 55 | def test_label_and_archive(self): 56 | recipe = yaml.load(""" 57 | label: Uservoice 58 | id: '000004' 59 | options: [] 60 | match: 61 | to: dev+uservoice@dimagi.com 62 | filters: 63 | - label: UserVoice 64 | """) 65 | recipe['options'].append(self.archive_option.to_json()) 66 | expected_gmail_filters = yaml.load(""" 67 | - id: '3462440000040' 68 | to: dev+uservoice@dimagi.com 69 | label: UserVoice 70 | 71 | - id: '3462440000041' 72 | to: -me AND dev+uservoice@dimagi.com 73 | shouldArchive: yes 74 | """) 75 | self._test(recipe, expected_gmail_filters,) 76 | 77 | def test_timecards(self): 78 | recipe = yaml.load(""" 79 | label: Timecards 80 | id: '000001' 81 | options: 82 | - key: names 83 | label: "Whose timecards do you want to see?" 84 | type: list 85 | value: ['sheffels', 'danny'] 86 | match: 87 | from: reports@dimagi.com 88 | filters: 89 | - label: Timecard 90 | - subject: '{% for name in names %}-"{{ name }}" {% endfor %}' 91 | shouldArchive: true 92 | """) 93 | expected_gmail_filters = yaml.load(""" 94 | - id: '3462440000010' 95 | from: reports@dimagi.com 96 | label: Timecard 97 | - id: '3462440000011' 98 | from: reports@dimagi.com 99 | subject: '-"sheffels" -"danny" ' 100 | shouldArchive: yes 101 | """) 102 | self._test(recipe, expected_gmail_filters) 103 | 104 | def test_generate_gmail_filter_set(self): 105 | with open(os.path.join(os.path.dirname(__file__), 106 | 'data', 'user_recipe_set.yml')) as f: 107 | filter_set = generate_gmail_fitler_set( 108 | UserRecipeSet.wrap(yaml.load(f)), 109 | updated_timestamp=datetime.datetime(2014, 10, 27, 2, 20, 58) 110 | ) 111 | with open(os.path.join(os.path.dirname(__file__), 112 | 'data', 'filters.xml')) as f: 113 | expected_xml = f.read() 114 | 115 | self.assertXmlEqual(filter_set.to_xml(pretty=True), expected_xml) 116 | -------------------------------------------------------------------------------- /gmailfilterrecipes/xmlgeneration.py: -------------------------------------------------------------------------------- 1 | from collections import defaultdict 2 | import datetime 3 | import jinja2 4 | from gmailfilterrecipes.jsonschemas import FILTER_KEY_TO_TYPE 5 | from gmailfilterxml import GmailFilter, GmailFilterSet 6 | 7 | 8 | def generate_gmail_fitler_set(user_recipe_set, author_name=None, 9 | author_email=None, updated_timestamp=None): 10 | gmail_filters = [] 11 | for recipe in user_recipe_set.recipes: 12 | if recipe.selected: 13 | gmail_filters.extend( 14 | generate_gmail_filters(recipe, user_recipe_set.id_prefix)) 15 | return GmailFilterSet( 16 | author_name=author_name, 17 | author_email=author_email, 18 | updated_timestamp=updated_timestamp or datetime.datetime.utcnow(), 19 | filters=gmail_filters, 20 | ) 21 | 22 | 23 | def generate_gmail_filters(user_recipe, id_prefix): 24 | """ 25 | Args: 26 | recipe (jsonschema.Recipe) 27 | options (list of jsonschema.RecipeOption) 28 | option_values (dict of string => string) 29 | """ 30 | 31 | match = user_recipe.match 32 | filters = list(user_recipe.filters) 33 | 34 | for option in user_recipe.options: 35 | if option.filters and option.value: 36 | filters.extend(option.filters) 37 | 38 | option_values = {option.key: option.value 39 | for option in user_recipe.options} 40 | 41 | gmail_filters = [] 42 | for i, filter in enumerate(filters): 43 | id = '{}{}{}'.format(id_prefix, user_recipe.id, i) 44 | kwargs = defaultdict(list) 45 | for key, value in filter.items() + match.items(): 46 | if isinstance(value, basestring): 47 | # it's a jinja2 template 48 | value = jinja2.Template(value).render(**option_values) 49 | kwargs[key].append(value) 50 | for key, value in kwargs.items(): 51 | if FILTER_KEY_TO_TYPE[key] == basestring: 52 | kwargs[key] = ' AND '.join(value) 53 | elif FILTER_KEY_TO_TYPE[key] == bool: 54 | assert len(kwargs[key]) == 1, kwargs[key] 55 | kwargs[key], = value 56 | else: 57 | assert False 58 | gmail_filters.append(GmailFilter(id=id, **kwargs)) 59 | return gmail_filters 60 | -------------------------------------------------------------------------------- /gmailfilterxml/__init__.py: -------------------------------------------------------------------------------- 1 | from api import * 2 | 3 | __all__ = ['GmailFilterSet', 'GmailFilter'] 4 | -------------------------------------------------------------------------------- /gmailfilterxml/api.py: -------------------------------------------------------------------------------- 1 | from . import xmlschemas 2 | 3 | PROPERTIES = ( 4 | ('from', basestring), 5 | ('subject', basestring), 6 | ('hasTheWord', basestring), 7 | ('to', basestring), 8 | ('doesNotHaveTheWord', basestring), 9 | ('label', basestring), 10 | ('shouldArchive', bool), 11 | ('shouldMarkAsRead', bool), 12 | ('shouldNeverSpam', bool), 13 | ('shouldAlwaysMarkAsImportant', bool), 14 | ('shouldNeverMarkAsImportant', bool), 15 | ) 16 | 17 | 18 | class GmailFilterSet(object): 19 | def __init__(self, author_name, author_email, updated_timestamp, 20 | filters=None): 21 | self.author_name = author_name 22 | self.author_email = author_email 23 | self.updated_timestamp = updated_timestamp 24 | self.filters = filters if filters is not None else [] 25 | 26 | def to_xml(self, **kwargs): 27 | 28 | def yield_properties(g): 29 | for name, py_type in PROPERTIES: 30 | value = getattr(g, name) 31 | if value: 32 | if py_type is bool: 33 | yield name, 'true' 34 | else: 35 | yield name, value 36 | 37 | entries = [ 38 | xmlschemas.Entry( 39 | id=gmail_filter.id, 40 | updated=self.updated_timestamp, 41 | properties=[xmlschemas.EntryProperty(name=name, value=value) 42 | for name, value in yield_properties(gmail_filter)] 43 | ) 44 | for gmail_filter in self.filters 45 | ] 46 | 47 | feed = xmlschemas.Feed( 48 | author_name=self.author_name or '', 49 | author_email=self.author_email or '', 50 | updated=self.updated_timestamp, 51 | ids=[gmail_filter.id for gmail_filter in self.filters], 52 | entries=entries, 53 | ) 54 | 55 | return feed.serializeDocument(**kwargs) 56 | 57 | 58 | class GmailFilter(object): 59 | def __init__(self, id=None, **kwargs): 60 | # deal with from_ => from 61 | kwargs = {key.rstrip('_'): value for key, value in kwargs.items()} 62 | unknown_kwargs = ( 63 | set(kwargs.keys()) 64 | - {name for name, _ in PROPERTIES} 65 | ) 66 | if unknown_kwargs: 67 | raise TypeError( 68 | "__init__() got an unexpected keyword argument '{}'" 69 | .format(list(unknown_kwargs)[0]) 70 | ) 71 | xmlschemas.validate_entry_id(id) 72 | self.id = id 73 | 74 | for name, py_type in PROPERTIES: 75 | if py_type is bool: 76 | kwargs[name] = kwargs.get(name) or False 77 | assert isinstance(kwargs[name], bool) 78 | setattr(self, name, kwargs.pop(name, None)) 79 | 80 | def __eq__(self, other): 81 | return self.id == other.id and all( 82 | getattr(self, name, None) == getattr(other, name, None) 83 | for name, _ in PROPERTIES 84 | ) 85 | 86 | def __repr__(self): 87 | return '{}(id={}, {})'.format( 88 | self.__class__.__name__, 89 | repr(self.id), 90 | ', '.join( 91 | '{}={}'.format(name, repr(getattr(self, name, None))) 92 | for name, _ in PROPERTIES 93 | if getattr(self, name, None) is not None 94 | ) 95 | ) 96 | -------------------------------------------------------------------------------- /gmailfilterxml/tests/__init__.py: -------------------------------------------------------------------------------- 1 | from .tests import * 2 | -------------------------------------------------------------------------------- /gmailfilterxml/tests/many-filters.xml: -------------------------------------------------------------------------------- 1 | 2 | Mail Filters 3 | tag:mail.google.com,2008:filters:1286460749536,1301418362291,1311264218436,1328530638969,1328531606815,1328531987415,1334695361272,1339518042838,1344957779523,1345135109593,1348163295797,1357167883807,1361290450763,1361375386891,1364330212484,1368101513770,1369199766942,1369201335559,1369230533920,1383134580199,1383135189830,1383135315107,1383136604908,1383213181442,1383554713304,1383554745854,1383565608544,1383580144651,1383653236554,1383734177270,1384155338534,1384398254855,1386595581624,1386595911921,1386596327622,1386691201549,1386691261354,1386691261371,1386691981113,1386692761541,1386692761577,1386693193648,1387315396215,1393441905180,1397149236854,1398354077759,1399296559142,1400187629377,1401981814313,1402415738400,1406134816759,1406137622806,1406405464337,1407252794687,1407252853043,1407856891904,1408972823205,1409677450717,1409677572066,1410187839011,1410643590000,1410721335116,1410721365117,1410743795723 4 | 2014-09-19T17:40:28Z 5 | 6 | Danny Roberts 7 | droberts@dimagi.com 8 | 9 | 10 | 11 | Mail Filter 12 | tag:mail.google.com,2008:filter:1286460749536 13 | 2014-09-19T17:40:28Z 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | Mail Filter 22 | tag:mail.google.com,2008:filter:1301418362291 23 | 2014-09-19T17:40:28Z 24 | 25 | 26 | 27 | 28 | 29 | 30 | Mail Filter 31 | tag:mail.google.com,2008:filter:1311264218436 32 | 2014-09-19T17:40:28Z 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | Mail Filter 41 | tag:mail.google.com,2008:filter:1328530638969 42 | 2014-09-19T17:40:28Z 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | Mail Filter 51 | tag:mail.google.com,2008:filter:1328531606815 52 | 2014-09-19T17:40:28Z 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | Mail Filter 61 | tag:mail.google.com,2008:filter:1328531987415 62 | 2014-09-19T17:40:28Z 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | Mail Filter 72 | tag:mail.google.com,2008:filter:1334695361272 73 | 2014-09-19T17:40:28Z 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | Mail Filter 82 | tag:mail.google.com,2008:filter:1339518042838 83 | 2014-09-19T17:40:28Z 84 | 85 | 86 | 87 | 88 | 89 | 90 | 91 | Mail Filter 92 | tag:mail.google.com,2008:filter:1344957779523 93 | 2014-09-19T17:40:28Z 94 | 95 | 96 | 97 | 98 | 99 | 100 | 101 | Mail Filter 102 | tag:mail.google.com,2008:filter:1345135109593 103 | 2014-09-19T17:40:28Z 104 | 105 | 106 | 107 | 108 | 109 | 110 | Mail Filter 111 | tag:mail.google.com,2008:filter:1348163295797 112 | 2014-09-19T17:40:28Z 113 | 114 | 115 | 116 | 117 | 118 | 119 | Mail Filter 120 | tag:mail.google.com,2008:filter:1357167883807 121 | 2014-09-19T17:40:28Z 122 | 123 | 124 | 125 | 126 | 127 | 128 | Mail Filter 129 | tag:mail.google.com,2008:filter:1361290450763 130 | 2014-09-19T17:40:28Z 131 | 132 | 133 | 134 | 135 | 136 | 137 | 138 | 139 | Mail Filter 140 | tag:mail.google.com,2008:filter:1361375386891 141 | 2014-09-19T17:40:28Z 142 | 143 | 144 | 145 | 146 | 147 | 148 | Mail Filter 149 | tag:mail.google.com,2008:filter:1364330212484 150 | 2014-09-19T17:40:28Z 151 | 152 | 153 | 154 | 155 | 156 | 157 | 158 | Mail Filter 159 | tag:mail.google.com,2008:filter:1368101513770 160 | 2014-09-19T17:40:28Z 161 | 162 | 163 | 164 | 165 | 166 | 167 | Mail Filter 168 | tag:mail.google.com,2008:filter:1369199766942 169 | 2014-09-19T17:40:28Z 170 | 171 | 172 | 173 | 174 | 175 | 176 | 177 | 178 | Mail Filter 179 | tag:mail.google.com,2008:filter:1369201335559 180 | 2014-09-19T17:40:28Z 181 | 182 | 183 | 184 | 185 | 186 | 187 | 188 | Mail Filter 189 | tag:mail.google.com,2008:filter:1369230533920 190 | 2014-09-19T17:40:28Z 191 | 192 | 193 | 194 | 195 | 196 | 197 | 198 | 199 | Mail Filter 200 | tag:mail.google.com,2008:filter:1383134580199 201 | 2014-09-19T17:40:28Z 202 | 203 | 204 | 205 | 206 | 207 | 208 | 209 | 210 | Mail Filter 211 | tag:mail.google.com,2008:filter:1383135189830 212 | 2014-09-19T17:40:28Z 213 | 214 | 215 | 216 | 217 | 218 | 219 | 220 | Mail Filter 221 | tag:mail.google.com,2008:filter:1383135315107 222 | 2014-09-19T17:40:28Z 223 | 224 | 225 | 226 | 227 | 228 | 229 | Mail Filter 230 | tag:mail.google.com,2008:filter:1383136604908 231 | 2014-09-19T17:40:28Z 232 | 233 | 234 | 235 | 236 | 237 | 238 | 239 | 240 | 241 | Mail Filter 242 | tag:mail.google.com,2008:filter:1383213181442 243 | 2014-09-19T17:40:28Z 244 | 245 | 246 | 247 | 248 | 249 | 250 | 251 | 252 | Mail Filter 253 | tag:mail.google.com,2008:filter:1383554713304 254 | 2014-09-19T17:40:28Z 255 | 256 | 257 | 258 | 259 | 260 | 261 | 262 | 263 | Mail Filter 264 | tag:mail.google.com,2008:filter:1383554745854 265 | 2014-09-19T17:40:28Z 266 | 267 | 268 | 269 | 270 | 271 | 272 | 273 | Mail Filter 274 | tag:mail.google.com,2008:filter:1383565608544 275 | 2014-09-19T17:40:28Z 276 | 277 | 278 | 279 | 280 | 281 | 282 | 283 | 284 | Mail Filter 285 | tag:mail.google.com,2008:filter:1383580144651 286 | 2014-09-19T17:40:28Z 287 | 288 | 289 | 290 | 291 | 292 | 293 | 294 | Mail Filter 295 | tag:mail.google.com,2008:filter:1383653236554 296 | 2014-09-19T17:40:28Z 297 | 298 | 299 | 300 | 301 | 302 | 303 | 304 | 305 | Mail Filter 306 | tag:mail.google.com,2008:filter:1383734177270 307 | 2014-09-19T17:40:28Z 308 | 309 | 310 | 311 | 312 | 313 | 314 | 315 | Mail Filter 316 | tag:mail.google.com,2008:filter:1384155338534 317 | 2014-09-19T17:40:28Z 318 | 319 | 320 | 321 | 322 | 323 | 324 | 325 | 326 | Mail Filter 327 | tag:mail.google.com,2008:filter:1384398254855 328 | 2014-09-19T17:40:28Z 329 | 330 | 331 | 332 | 333 | 334 | 335 | 336 | 337 | Mail Filter 338 | tag:mail.google.com,2008:filter:1386595581624 339 | 2014-09-19T17:40:28Z 340 | 341 | 342 | 343 | 344 | 345 | 346 | 347 | Mail Filter 348 | tag:mail.google.com,2008:filter:1386595911921 349 | 2014-09-19T17:40:28Z 350 | 351 | 352 | 353 | 354 | 355 | 356 | 357 | 358 | Mail Filter 359 | tag:mail.google.com,2008:filter:1386596327622 360 | 2014-09-19T17:40:28Z 361 | 362 | 363 | 364 | 365 | 366 | 367 | 368 | 369 | Mail Filter 370 | tag:mail.google.com,2008:filter:1386691201549 371 | 2014-09-19T17:40:28Z 372 | 373 | 374 | 375 | 376 | 377 | 378 | 379 | 380 | 381 | Mail Filter 382 | tag:mail.google.com,2008:filter:1386691261354 383 | 2014-09-19T17:40:28Z 384 | 385 | 386 | 387 | 388 | 389 | 390 | 391 | Mail Filter 392 | tag:mail.google.com,2008:filter:1386691261371 393 | 2014-09-19T17:40:28Z 394 | 395 | 396 | 397 | 398 | 399 | 400 | 401 | 402 | Mail Filter 403 | tag:mail.google.com,2008:filter:1386691981113 404 | 2014-09-19T17:40:28Z 405 | 406 | 407 | 408 | 409 | 410 | 411 | 412 | 413 | 414 | Mail Filter 415 | tag:mail.google.com,2008:filter:1386692761541 416 | 2014-09-19T17:40:28Z 417 | 418 | 419 | 420 | 421 | 422 | 423 | 424 | Mail Filter 425 | tag:mail.google.com,2008:filter:1386692761577 426 | 2014-09-19T17:40:28Z 427 | 428 | 429 | 430 | 431 | 432 | 433 | 434 | 435 | Mail Filter 436 | tag:mail.google.com,2008:filter:1386693193648 437 | 2014-09-19T17:40:28Z 438 | 439 | 440 | 441 | 442 | 443 | 444 | 445 | Mail Filter 446 | tag:mail.google.com,2008:filter:1387315396215 447 | 2014-09-19T17:40:28Z 448 | 449 | 450 | 451 | 452 | 453 | 454 | 455 | Mail Filter 456 | tag:mail.google.com,2008:filter:1393441905180 457 | 2014-09-19T17:40:28Z 458 | 459 | 460 | 461 | 462 | 463 | 464 | 465 | 466 | 467 | 468 | Mail Filter 469 | tag:mail.google.com,2008:filter:1397149236854 470 | 2014-09-19T17:40:28Z 471 | 472 | 473 | 474 | 475 | 476 | 477 | 478 | 479 | Mail Filter 480 | tag:mail.google.com,2008:filter:1398354077759 481 | 2014-09-19T17:40:28Z 482 | 483 | 484 | 485 | 486 | 487 | 488 | 489 | 490 | 491 | 492 | Mail Filter 493 | tag:mail.google.com,2008:filter:1399296559142 494 | 2014-09-19T17:40:28Z 495 | 496 | 497 | 498 | 499 | 500 | 501 | 502 | 503 | 504 | Mail Filter 505 | tag:mail.google.com,2008:filter:1400187629377 506 | 2014-09-19T17:40:28Z 507 | 508 | 509 | 510 | 511 | 512 | 513 | 514 | 515 | 516 | Mail Filter 517 | tag:mail.google.com,2008:filter:1401981814313 518 | 2014-09-19T17:40:28Z 519 | 520 | 521 | 522 | 523 | 524 | 525 | 526 | 527 | 528 | Mail Filter 529 | tag:mail.google.com,2008:filter:1402415738400 530 | 2014-09-19T17:40:28Z 531 | 532 | 533 | 534 | 535 | 536 | 537 | 538 | 539 | Mail Filter 540 | tag:mail.google.com,2008:filter:1406134816759 541 | 2014-09-19T17:40:28Z 542 | 543 | 544 | 545 | 546 | 547 | 548 | 549 | 550 | 551 | 552 | 553 | Mail Filter 554 | tag:mail.google.com,2008:filter:1406137622806 555 | 2014-09-19T17:40:28Z 556 | 557 | 558 | 559 | 560 | 561 | 562 | 563 | 564 | 565 | 566 | 567 | Mail Filter 568 | tag:mail.google.com,2008:filter:1406405464337 569 | 2014-09-19T17:40:28Z 570 | 571 | 572 | 573 | 574 | 575 | 576 | 577 | 578 | 579 | 580 | Mail Filter 581 | tag:mail.google.com,2008:filter:1407252794687 582 | 2014-09-19T17:40:28Z 583 | 584 | 585 | 586 | 587 | 588 | 589 | 590 | 591 | 592 | 593 | Mail Filter 594 | tag:mail.google.com,2008:filter:1407252853043 595 | 2014-09-19T17:40:28Z 596 | 597 | 598 | 599 | 600 | 601 | 602 | 603 | 604 | 605 | Mail Filter 606 | tag:mail.google.com,2008:filter:1407856891904 607 | 2014-09-19T17:40:28Z 608 | 609 | 610 | 611 | 612 | 613 | 614 | 615 | 616 | Mail Filter 617 | tag:mail.google.com,2008:filter:1408972823205 618 | 2014-09-19T17:40:28Z 619 | 620 | 621 | 622 | 623 | 624 | 625 | 626 | 627 | 628 | Mail Filter 629 | tag:mail.google.com,2008:filter:1409677450717 630 | 2014-09-19T17:40:28Z 631 | 632 | 633 | 634 | 635 | 636 | 637 | 638 | 639 | Mail Filter 640 | tag:mail.google.com,2008:filter:1409677572066 641 | 2014-09-19T17:40:28Z 642 | 643 | 644 | 645 | 646 | 647 | 648 | 649 | 650 | Mail Filter 651 | tag:mail.google.com,2008:filter:1410187839011 652 | 2014-09-19T17:40:28Z 653 | 654 | 655 | 656 | 657 | 658 | 659 | 660 | 661 | 662 | Mail Filter 663 | tag:mail.google.com,2008:filter:1410643590000 664 | 2014-09-19T17:40:28Z 665 | 666 | 667 | 668 | 669 | 670 | 671 | 672 | 673 | 674 | Mail Filter 675 | tag:mail.google.com,2008:filter:1410721335116 676 | 2014-09-19T17:40:28Z 677 | 678 | 679 | 680 | 681 | 682 | 683 | 684 | 685 | 686 | Mail Filter 687 | tag:mail.google.com,2008:filter:1410721365117 688 | 2014-09-19T17:40:28Z 689 | 690 | 691 | 692 | 693 | 694 | 695 | 696 | 697 | 698 | 699 | 700 | 701 | Mail Filter 702 | tag:mail.google.com,2008:filter:1410743795723 703 | 2014-09-19T17:40:28Z 704 | 705 | 706 | 707 | 708 | 709 | 710 | 711 | 712 | -------------------------------------------------------------------------------- /gmailfilterxml/tests/single-filter.xml: -------------------------------------------------------------------------------- 1 | 2 | Mail Filters 3 | tag:mail.google.com,2008:filters:1286460749536 4 | 2014-09-19T17:40:28Z 5 | 6 | Danny Roberts 7 | droberts@dimagi.com 8 | 9 | 10 | 11 | Mail Filter 12 | tag:mail.google.com,2008:filter:1286460749536 13 | 2014-09-19T17:40:28Z 14 | 15 | 16 | 17 | 18 | 19 | -------------------------------------------------------------------------------- /gmailfilterxml/tests/tests.py: -------------------------------------------------------------------------------- 1 | import os 2 | import unittest 3 | import datetime 4 | from gmailfilterxml.api import GmailFilterSet, GmailFilter 5 | from gmailfilterxml.xmlschemas import Feed, Entry, EntryProperty 6 | from eulxml.xmlmap import load_xmlobject_from_string 7 | from utils.unittest import XmlTest 8 | 9 | 10 | class SingleFilterTest(XmlTest): 11 | def setUp(self): 12 | with open(os.path.join(os.path.dirname(__file__), 'single-filter.xml')) as f: 13 | self.expected_xml = f.read() 14 | 15 | def test_api(self): 16 | filter_set = GmailFilterSet( 17 | author_name='Danny Roberts', 18 | author_email='droberts@dimagi.com', 19 | updated_timestamp=datetime.datetime(2014, 9, 19, 17, 40, 28), 20 | filters=[ 21 | GmailFilter( 22 | id='1286460749536', 23 | from_='noreply@github.com', 24 | label='github', 25 | shouldArchive=True, 26 | ) 27 | ] 28 | ) 29 | self.assertXmlEqual(filter_set.to_xml(), self.expected_xml) 30 | 31 | def test_schema(self): 32 | updated = datetime.datetime(2014, 9, 19, 17, 40, 28) 33 | 34 | entries = [ 35 | Entry( 36 | id='1286460749536', 37 | updated=updated, 38 | properties=[ 39 | EntryProperty(name='from', value='noreply@github.com'), 40 | EntryProperty(name='label', value='github'), 41 | EntryProperty(name='shouldArchive', value='true'), 42 | ] 43 | ) 44 | ] 45 | feed = Feed( 46 | author_name='Danny Roberts', 47 | author_email='droberts@dimagi.com', 48 | updated=updated, 49 | ids=['1286460749536'], 50 | entries=entries 51 | ) 52 | self.assertXmlEqual(feed.serializeDocument(), self.expected_xml) 53 | 54 | def test_multiple_entries_init(self): 55 | """ 56 | This is a regression test. 57 | There used to be a problem with initializing a feed 58 | with multiple entries 59 | 60 | """ 61 | updated = datetime.datetime(2014, 9, 19, 17, 40, 28) 62 | entries = [ 63 | Entry( 64 | id='1286460749536', 65 | updated=updated, 66 | properties=[ 67 | EntryProperty(name='from', value='noreply@github.com'), 68 | EntryProperty(name='label', value='github'), 69 | EntryProperty(name='shouldArchive', value='true'), 70 | ] 71 | ), 72 | Entry( 73 | id='1286460749537', 74 | updated=updated, 75 | properties=[ 76 | EntryProperty(name='from', value='noreply@github.com'), 77 | EntryProperty(name='label', value='github'), 78 | EntryProperty(name='shouldArchive', value='true'), 79 | ] 80 | ), 81 | ] 82 | feed = Feed( 83 | author_name='Danny Roberts', 84 | author_email='droberts@dimagi.com', 85 | updated=updated, 86 | ids=['1286460749536', '1286460749537'], 87 | entries=entries, 88 | ) 89 | self.assertEqual(feed.entries, entries) 90 | 91 | 92 | class FilterValidationTest(unittest.TestCase): 93 | def test(self): 94 | with self.assertRaises(TypeError): 95 | GmailFilter(id='1234567890123', xyz=123) 96 | with self.assertRaises(TypeError): 97 | GmailFilter(xyz=123) 98 | with self.assertRaises(AssertionError): 99 | GmailFilter(from_='github@dimagi.com') 100 | GmailFilter(id='1234567890123', from_='github@dimagi.com') 101 | 102 | 103 | class ReadXMLFileTest(XmlTest): 104 | @classmethod 105 | def setUpClass(self): 106 | with open(os.path.join(os.path.dirname(__file__), 'many-filters.xml')) as f: 107 | self.many_filters_xml = f.read() 108 | with open(os.path.join(os.path.dirname(__file__), 'single-filter.xml')) as f: 109 | self.single_filter_xml = f.read() 110 | 111 | def _test_round_trip(self, xml): 112 | feed = load_xmlobject_from_string(xml, Feed) 113 | self.assertXmlEqual(feed.serializeDocument(), xml) 114 | 115 | def test_many_filters_round_trip(self): 116 | self._test_round_trip(self.many_filters_xml) 117 | 118 | def test_single_filter_round_trip(self): 119 | self._test_round_trip(self.single_filter_xml) 120 | -------------------------------------------------------------------------------- /gmailfilterxml/xmlschemas.py: -------------------------------------------------------------------------------- 1 | import functools 2 | from eulxml.xmlmap import ( 3 | XmlObject, 4 | OrderedXmlObject, 5 | StringField, 6 | NodeListField, 7 | ) 8 | from eulxml.xmlmap.fields import SingleNodeManager, Field 9 | 10 | 11 | NS = { 12 | 'apps': 'http://schemas.google.com/apps/2006', 13 | 'atom': 'http://www.w3.org/2005/Atom' 14 | } 15 | 16 | 17 | def validate_entry_id(id): 18 | assert isinstance(id, basestring) 19 | assert len(id) == 13 20 | assert all(c in '0123456789' for c in id) 21 | return id 22 | 23 | 24 | class EntryIdMapper(object): 25 | PREFIX = 'tag:mail.google.com,2008:filter:' 26 | 27 | def to_xml(self, value): 28 | if value is None: 29 | return None 30 | id = validate_entry_id(value) 31 | return '{}{}'.format(self.PREFIX, id) 32 | 33 | def to_python(self, value): 34 | assert value.startswith(self.PREFIX) 35 | id = value[len(self.PREFIX):] 36 | return validate_entry_id(id) 37 | 38 | 39 | class EntryIdListMapper(object): 40 | PREFIX = 'tag:mail.google.com,2008:filters:' 41 | 42 | def to_xml(self, value): 43 | if value is None: 44 | return None 45 | ids = [validate_entry_id(id) for id in value] 46 | return '{}{}'.format(self.PREFIX, ','.join(ids)) 47 | 48 | def to_python(self, value): 49 | assert value.startswith(self.PREFIX) 50 | ids = value[len(self.PREFIX):].split(',') 51 | return [validate_entry_id(id) for id in ids] 52 | 53 | 54 | class UTCDateTimeMapper(object): 55 | def to_xml(self, value): 56 | if value is None: 57 | return None 58 | return value.strftime("%Y-%m-%dT%H:%M:%SZ") 59 | 60 | def to_python(self, value): 61 | import datetime 62 | return datetime.datetime.strptime(value, "%Y-%m-%dT%H:%M:%SZ") 63 | 64 | 65 | SingleNodeField = functools.partial(Field, manager=SingleNodeManager()) 66 | EntryIdField = functools.partial(SingleNodeField, mapper=EntryIdMapper()) 67 | EntryIdListField = functools.partial(SingleNodeField, 68 | mapper=EntryIdListMapper()) 69 | UTCDateTimeField = functools.partial(SingleNodeField, 70 | mapper=UTCDateTimeMapper()) 71 | 72 | class EntryProperty(XmlObject): 73 | 74 | ROOT_NAME = 'property' 75 | ROOT_NS = NS['apps'] 76 | 77 | name = StringField('@name') 78 | value = StringField('@value') 79 | 80 | 81 | class Entry(OrderedXmlObject): 82 | ROOT_NAME = 'entry' 83 | ROOT_NAMESPACES = NS 84 | ROOT_NS = NS['atom'] 85 | ORDER = ('category_term', 'title', 'id', 'updated', 'content', 86 | 'properties') 87 | 88 | category_term = StringField('atom:category/@term') 89 | title = StringField('atom:title') 90 | id = EntryIdField('atom:id') 91 | updated = UTCDateTimeField('atom:updated') 92 | content = StringField('atom:content') 93 | properties = NodeListField('apps:property', EntryProperty) 94 | 95 | def __init__(self, node=None, context=None, **kwargs): 96 | if node is None: 97 | if 'category_term' not in kwargs: 98 | kwargs['category_term'] = 'filter' 99 | if 'title' not in kwargs: 100 | kwargs['title'] = 'Mail Filter' 101 | if 'content' not in kwargs: 102 | kwargs['content'] = '' 103 | super(Entry, self).__init__(node, context, **kwargs) 104 | 105 | 106 | class Feed(OrderedXmlObject): 107 | 108 | ROOT_NAME = 'feed' 109 | ROOT_NS = NS['atom'] 110 | ROOT_NAMESPACES = NS 111 | ORDER = ('title', 'ids', 'updated', 'author_name', 'author_email', 112 | 'entries') 113 | 114 | title = StringField('atom:title') 115 | ids = EntryIdListField('atom:id') 116 | updated = UTCDateTimeField('atom:updated') 117 | author_name = StringField('atom:author/atom:name') 118 | author_email = StringField('atom:author/atom:email') 119 | entries = NodeListField('atom:entry', Entry) 120 | 121 | def __init__(self, node=None, context=None, **kwargs): 122 | if node is None and 'title' not in kwargs: 123 | kwargs['title'] = 'Mail Filters' 124 | super(Feed, self).__init__(node, context, **kwargs) 125 | -------------------------------------------------------------------------------- /recipesets/default.yml: -------------------------------------------------------------------------------- 1 | id_prefix: '346244' 2 | options: 3 | - 4 | key: keywords 5 | label: "What keywords do you want to see in your inbox?" 6 | type: list 7 | - 8 | key: archive 9 | label: Do you want to see these in your inbox? 10 | type: inverted-bool 11 | filters: 12 | - 13 | to: -me 14 | shouldArchive: true 15 | - 16 | key: mark_read 17 | label: Mark as READ? 18 | type: bool 19 | filters: 20 | - shouldMarkAsRead: true 21 | recipes: 22 | - 23 | label: Mails directly to me 24 | custom_options: 25 | - 26 | key: names 27 | label: "List your email addresses" 28 | type: list 29 | match: 30 | to: "{{ names|join(' OR ') }}" 31 | filters: 32 | - label: =^.^= 33 | shouldNeverSpam: true 34 | shouldAlwaysMarkAsImportant: true 35 | - 36 | label: Timecards 37 | custom_options: 38 | - 39 | key: names 40 | label: "Whose timecards do you want to see? (List first or last names as they appear in fogbugz)" 41 | type: list 42 | match: 43 | from: reports@dimagi.com 44 | filters: 45 | - label: Timecard 46 | - 47 | subject: '{% for name in names %}-"{{ name }}" {% endfor %}' 48 | shouldArchive: true 49 | - 50 | label: Pingdom 51 | options: 52 | - archive 53 | - mark_read 54 | match: 55 | from: alert@pingdom.com 56 | filters: 57 | - label: Pingdom 58 | - 59 | label: Fogbugz Cases 60 | custom_options: 61 | - 62 | key: names 63 | label: "Your full name as it appears in FogBugz" 64 | type: list 65 | match: 66 | from: 'fogbugz-noreply@dimagi.com OR fogbugz+noreply@dimagi.com' 67 | filters: 68 | - label: FogBugz 69 | - label: My Cases 70 | hasTheWord: "'{{ names|join('\\' OR \\'') }}'" 71 | - hasTheWord: "resolved -('{{ names|join('\\' OR \\'') }}')" 72 | shouldArchive: true 73 | shouldMarkAsRead: true 74 | - hasTheWord: "-('{{ names|join('\\' OR \\'') }}')" 75 | shouldNeverMarkAsImportant: true 76 | - 77 | label: Job Applicants 78 | options: 79 | - archive 80 | match: 81 | to: jobslisteners@dimagi.com 82 | filters: 83 | - label: Job Applicants 84 | - 85 | label: Uservoice 86 | options: 87 | - archive 88 | match: 89 | to: dev+uservoice@dimagi.com 90 | filters: 91 | - label: UserVoice 92 | - 93 | label: Server Emails 94 | options: 95 | - archive 96 | - mark_read 97 | match: 98 | to: commcarehq-ops@dimagi.com 99 | filters: 100 | - label: Server 101 | - 102 | label: Travis Notifications 103 | match: 104 | from: notifications@travis-ci.org 105 | filters: 106 | - label: Travis 107 | - subject: '[Passed]' 108 | shouldNeverMarkAsImportant: true 109 | - subject: '[Failed] OR [Still Failing] OR [Fixed]' 110 | shouldAlwaysMarkAsImportant: true 111 | - 112 | label: Boston Office 113 | match: 114 | to: office@dimagi.com 115 | filters: 116 | - label: Office 117 | - 118 | label: Cape Town Office 119 | match: 120 | to: capetown@dimagi.com 121 | filters: 122 | - label: Office 123 | - 124 | label: CommCare Users 125 | options: 126 | - archive 127 | match: 128 | to: commcare-users@googlegroups.com 129 | filters: 130 | - label: CommCare Users 131 | - 132 | label: From a Dimagi person 133 | match: 134 | from: "*@dimagi.com -commcarehq -fogbugz -devops -notifications -ops" 135 | filters: 136 | - label: ♥︎ 137 | - 138 | label: Project Reports 139 | match: 140 | to: project-reports@dimagi.com 141 | filters: 142 | - label: Project Reports 143 | - 144 | label: Field Reports 145 | match: 146 | to: field-reports@dimagi.com 147 | filters: 148 | - label: Field Reports 149 | - 150 | label: Field Dev 151 | match: 152 | to: field-dev@googlegroups.com 153 | filters: 154 | - label: Field Dev 155 | - 156 | label: DevOps 157 | options: 158 | - archive 159 | match: 160 | hasTheWord: "list:'devops.dimagi.com'" 161 | filters: 162 | - label: DevOps 163 | - 164 | label: Trello 165 | match: 166 | from: do-not-reply@trello.com 167 | filters: 168 | - label: Trello 169 | - 170 | label: GitHub 171 | match: 172 | from: notifications@github.com 173 | filters: 174 | - label: GitHub 175 | to: me 176 | shouldAlwaysMarkAsImportant: true 177 | - to: -me 178 | shouldNeverMarkAsImportant: true 179 | - 180 | label: Ignore Dimagimon 181 | filters: 182 | - from: dimagimon 183 | to: -me 184 | shouldArchive: true 185 | - 186 | label: Information 187 | options: 188 | - keywords 189 | match: 190 | to: "information@dimagi.com OR info@dimagi.com" 191 | filters: 192 | - label: Information 193 | - doesNotHaveTheWord: '{% for keyword in keywords %}"{{ keyword }}" {% endfor %}' 194 | - 195 | label: Allstaff 196 | match: 197 | to: allstaff@dimagi.com 198 | filters: 199 | - label: Allstaff 200 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | bottle 2 | # eulxml==0.21.1 3 | git+git://github.com/dannyroberts/eulxml.git@ac205937#egg=eulxml 4 | jinja2 5 | jsonobject==0.6.0b2 6 | pyyaml 7 | -------------------------------------------------------------------------------- /run_server.py: -------------------------------------------------------------------------------- 1 | import bottle 2 | import gmailfilterrecipes.client.index 3 | 4 | if __name__ == '__main__': 5 | bottle.run(debug=True, reloader=True) 6 | -------------------------------------------------------------------------------- /settings.ini: -------------------------------------------------------------------------------- 1 | [gmailfilterrecipes] 2 | filters_yml = recipesets/default.yml 3 | -------------------------------------------------------------------------------- /utils/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dimagi/gmail-filters/bea5a547a35c786b155bcddfa40843680ad8bbe0/utils/__init__.py -------------------------------------------------------------------------------- /utils/unittest.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | import unittest 3 | from lxml.doctestcompare import LXMLOutputChecker 4 | from doctest import Example 5 | 6 | 7 | class XmlTest(unittest.TestCase): 8 | """http://stackoverflow.com/a/7060342""" 9 | def assertXmlEqual(self, got, want): 10 | checker = LXMLOutputChecker() 11 | if not checker.check_output(want, got, 0): 12 | message = checker.output_difference(Example("", want), got, 0) 13 | raise AssertionError(message) 14 | --------------------------------------------------------------------------------