 }}){kind=logo}
10 | {{ body }}
9 |Your unique link is {{ url_for('benchmarks', _external=True, user=email_hash) }}?utm_source=activation_email&utm_medium=email.
5 | {% endblock %} -------------------------------------------------------------------------------- /app/static/scss/pages/_index.scss: -------------------------------------------------------------------------------- 1 | #index { 2 | .flow-overview { 3 | margin-left: 1.5rem; 4 | margin-right: 1.5rem; 5 | } 6 | 7 | .index-icon { 8 | height: $index-svg-height; 9 | fill: $crimson 10 | } 11 | 12 | .col-md-1 { 13 | height: $index-svg-height; 14 | } 15 | 16 | .arrow { 17 | fill: $medium-dark-grey; 18 | } 19 | 20 | .main-svg { 21 | max-width: 100%; 22 | } 23 | } 24 | -------------------------------------------------------------------------------- /app/static/scss/pages/_org-form.scss: -------------------------------------------------------------------------------- 1 | #financial-classification-wrapper.invalid:after, 2 | #coverage-scope-wrapper.invalid:after, 3 | #coverage-focus-wrapper.invalid:after, 4 | #platform-wrapper.invalid:after, 5 | #employee-range-wrapper.invalid:after, 6 | #budget-wrapper.invalid:after { 7 | content: 'Please select a value.'; 8 | } 9 | 10 | #other-affiliation-name-wrapper.invalid:after { 11 | content: 'Please enter a value.'; 12 | } -------------------------------------------------------------------------------- /app/static/scss/components/_structure.scss: -------------------------------------------------------------------------------- 1 | html, body { 2 | height: 100%; 3 | } 4 | 5 | body { 6 | display: flex; 7 | flex-direction: column; 8 | } 9 | 10 | .has-bottom-margin { 11 | margin-bottom: $std-bottom-margin; 12 | } 13 | 14 | .content-block { 15 | margin-bottom: $content-block-margin-bottom; 16 | } 17 | 18 | .bordered { 19 | padding: 2.5rem; 20 | border: 0.5px solid $medium-dark-grey; 21 | border-radius: $border-radius; 22 | } -------------------------------------------------------------------------------- /app/templates/select-list.html: -------------------------------------------------------------------------------- 1 | {% extends "base.html" %} 2 | 3 | {% block content %} 4 || List | 9 |Subscribers | 10 |Analysis Time | 11 |12 | |
|---|
The Shorenstein Center on Media, Politics and Public Policy
John F. Kennedy School of Government, Harvard University
124 Mt. Auburn Street, 2nd Floor, South Elevators
Cambridge, MA 02138
contact@emailbenchmarking.com
9 |
10 |
{{ title }}
12 | {% block content %}{% endblock %} 13 | 14 | -------------------------------------------------------------------------------- /.coveragerc: -------------------------------------------------------------------------------- 1 | # .coveragerc to control coverage.py 2 | [run] 3 | branch = True 4 | 5 | [report] 6 | # Regexes for lines to exclude from consideration 7 | exclude_lines = 8 | # Have to re-enable the standard pragma 9 | pragma: no cover 10 | 11 | # Don't complain about missing debug-only code: 12 | def __repr__ 13 | if self\.debug 14 | 15 | # Don't complain if tests don't hit defensive assertion code: 16 | raise AssertionError 17 | raise NotImplementedError 18 | 19 | # Don't complain if non-runnable code isn't run: 20 | if 0: 21 | if __name__ == .__main__.: 22 | 23 | ignore_errors = True 24 | 25 | [html] 26 | directory = coverage_html_report -------------------------------------------------------------------------------- /app/static/scss/main.scss: -------------------------------------------------------------------------------- 1 | @import "etc/fonts"; 2 | @import "etc/variables"; 3 | @import "bootstrap/override-variables"; 4 | 5 | @import "bootstrap/bootstrap-components"; 6 | @import "bootstrap/custom-modal"; 7 | @import "bootstrap/custom-toggle"; 8 | 9 | @import "components/nav"; 10 | @import "components/footer"; 11 | 12 | @import "components/structure"; 13 | @import "components/forms"; 14 | @import "components/switches"; 15 | @import "components/tables"; 16 | @import "components/shared"; 17 | @import "components/popovers"; 18 | 19 | @import "pages/index"; 20 | @import "pages/basic-form"; 21 | @import "pages/org-form"; 22 | @import "pages/api-key-form"; 23 | @import "pages/get-lists"; -------------------------------------------------------------------------------- /tests/integration/conftest.py: -------------------------------------------------------------------------------- 1 | import pytest 2 | from celery import current_app 3 | from app import app, db 4 | 5 | @pytest.fixture 6 | def test_app(): 7 | """Sets up a test app.""" 8 | app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///:memory:' 9 | app.config['TESTING'] = True 10 | app.config['WTF_CSRF_ENABLED'] = False 11 | app.config['NO_EMAIL'] = True 12 | app.config['SES_DEFAULT_EMAIL_SOURCE'] = 'testing@testing.com' 13 | current_app.conf.update(CELERY_ALWAYS_EAGER=True) 14 | with app.app_context(): 15 | db.create_all() 16 | yield app 17 | db.drop_all() 18 | 19 | @pytest.fixture 20 | def client(test_app): 21 | """Sets up a test client.""" 22 | client = test_app.test_client() 23 | yield client 24 | -------------------------------------------------------------------------------- /package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "benchmarks", 3 | "version": "3.1.0", 4 | "description": "Analytics for MailChimp Lists", 5 | "main": "app.py", 6 | "scripts": { 7 | "gulp": "./node_modules/.bin/gulp", 8 | "lint": "./node_modules/.bin/gulp lint" 9 | }, 10 | "author": "William Hakim", 11 | "license": "MIT", 12 | "devDependencies": { 13 | "bootstrap": "^4.3.0", 14 | "gulp": "^4.0.0", 15 | "gulp-autoprefixer": "^6.0.0", 16 | "gulp-cli": "^2.0.1", 17 | "gulp-concat": "^2.6.1", 18 | "gulp-eslint": "^5.0.0", 19 | "gulp-rename": "^1.4.0", 20 | "gulp-sass": "^4.0.2", 21 | "gulp-terser": "^1.1.7", 22 | "gulp-util": "^3.0.8" 23 | }, 24 | "dependencies": { 25 | "electron": "^4.0.4", 26 | "orca": "^1.2.1" 27 | } 28 | } 29 | -------------------------------------------------------------------------------- /app/templates/error-email.html: -------------------------------------------------------------------------------- 1 | {% extends "email-base.html" %} 2 | 3 | {% block content %} 4 |
5 | Error Details:
6 | {% for k, v in error_details.items() %}
7 | {{ k }}: {{ v }}
8 | {% endfor %}
9 |
Please try your request again, and consider whether your organization's ongoing API usage may be close to saturating MailChimp's connection limits. If problems persist, contact the Shorenstein Center.
11 | {% endblock %} -------------------------------------------------------------------------------- /app/templates/admin.html: -------------------------------------------------------------------------------- 1 | {% extends "base.html" %} 2 | 3 | {% block content %} 4 || {{ col }} | 9 | {% endfor %} 10 ||
|---|---|
| 22 | 23 | 24 | 25 | 26 | | 27 | {% else %} 28 |{{ attr[1] }} | 29 | {% endif %} 30 | {% endfor %} 31 |
It depends whether your organization is already in our system.
33 |If it is, we will send you a unique access link for the tool within 1-2 business days. Otherwise, we will collect some additional information from you on the next page in order to verify that you represent a valid news organization.
34 |By requesting access, you agree to our Terms of Service. 35 |
The Shorenstein Center at the Harvard Kennedy School produces original research on sustainable business models for news organizations in the digital age. The center works closely with legacy and emerging news organizations to put theories into practice—creating a cycle of research, implementation, and learning.
9 |This email benchmarking tool grew out of the Single Subject News Research Project at the Shorenstein Center, funded by the Knight Foundation. The Single Subject News research project seeks to answer the question – how can nonprofit, single-subject news sites grow and engage their online audiences? The email newsletter is one tactic we are continuously testing as an audience growth and monetization tool with our cohort of participating newsrooms.
10 |This benchmarking tool leverages our research into using data science to analyze email newsletters and allows news organizations to analyze their own MailChimp data without requiring any advanced technical knowledge. The goal of the tool is to encourage newsrooms to focus on the newsletter metrics that matter, including ones that focus on list composition, activity and quality of users.
11 |Once you've signed up, our staff will vet your organization (to ensure that you are, in fact, representing a news organization) and send you a unique access link. Then, simply enter your API key, select the data you'd like analyzed, and we'll send you a comprehensive email report containing visualizations, recommendations, and comparisons to the other participating media organizations included in our secure database.
12 |We store three kinds of data in our secure database: information about you (the user of this tool), information about the organization you represent and information about any MailChimp list(s) you ask us to analyze.
9 |We store the information that you provide about yourself (your name, email address, etc.) in order to ensure that you are indeed affiliated with the organization you registered with. We will not publically report this information.
10 |We store information about your organization (organization size, budget, coverage scope, etc.) in order to provide you with more personalized metrics, such as benchmarks of organizations "like yours." We also reserve the right to publish anonymized aggregate data on what types of organizations are using our service. We will not publically report your organization's information. 11 |
Finally, if you choose not to opt-out, we also store information about the MailChimp list(s) that you ask us to analyze. This information consists of summary statistics we generate through a number of API calls and calculations (the same statistics we use to generate the charts in the report email we send you), as well as the API key you provide to us and the unique MailChimp ID of the list you ask us to analyze. We will also use these summary statistics to help calculate a aggregate statistics for other users of this tool. We do not store any raw data about your list, nor do we access any personally-identifying information about your list members. When you enter your API key, you may choose to uncheck both checkboxes containing "Store this API key." If you do, we will not store any information at all about your MailChimp list. This does, however, prevent us from caching and updating your data in the background (allowing you to instantly receive an up-to-date report on your list, or a scheduled monthly report). It also means you will not be contributing your data to an aggregate pool which helps other tool users as well as our research team.
12 |If you would like us to delete your data, including summary statistics about your MailChimp lists, please contact us. Removal requests will be processed within 14 days.
13 |The following is a breakdown of organizations in our database. All data is self-reported. 10 |
The following statistics derive from those users who contributed their anonymized list data to our aggregate statistics (see our Privacy Policy for more information).
20 |Please fill out this form. Be sure to let us know whether you'd like an answer emailed directly to you or appended to this page.
27 |Last updated: August 15, 2018
9 |Please read these Terms of Service ("Terms", "Terms of Service") carefully before using the emailbenchmarking.com website (the "Service") operated by the Shorenstein Center at the Harvard Kennedy School ("the Shorenstein Center", also "us", "we", or "our").
10 |Your access to and use of the Service is conditioned on your acceptance of and compliance with these Terms. These Terms apply to all visitors, users and others who access or use the Service.
11 |By accessing or using the Service you agree to be bound by these Terms. If you disagree with any part of the terms then you may not access the Service.
12 |In order to use our Service, you must (a) be at least eighteen years of age, and legally able to enter into contracts; (b) complete the registration process; and (c) agree to our Privacy Policy. 14 |
Our Service may contain links to third-party web sites or services that are not owned or controlled by the Shorenstein Center.
16 |The Shorenstein Center has no control over, and assumes no responsibility for, the content, privacy policies, or practices of any third party web sites or services. You further acknowledge and agree that the Shorenstein Center shall not be responsible or liable, directly or indirectly, for any damage or loss caused or alleged to be caused by or in connection with use of or reliance on any such content, goods or services available on or through any such web sites or services.
17 |We strongly advise you to read the terms and conditions and privacy policies of any third-party web sites or services that you visit.
18 |These Terms shall be governed and construed in accordance with the laws of Massachusetts, United States, without regard to its conflict of law provisions.
20 |Our failure to enforce any right or provision of these Terms will not be considered a waiver of those rights. If any provision of these Terms is held to be invalid or unenforceable by a court, the remaining provisions of these Terms will remain in effect. These Terms constitute the entire agreement between us regarding our Service, and supersede and replace any prior agreements we might have between us regarding the Service.
21 |We reserve the right, at our sole discretion, to modify or replace these Terms at any time. If a revision is material we will try to provide at least 30 days notice prior to any new terms taking effect. What constitutes a material change will be determined at our sole discretion.
23 |By continuing to access or use our Service after those revisions become effective, you agree to be bound by the revised terms. If you do not agree to the new terms, please stop using the Service.
24 |If you have any questions about these Terms, please contact us.
26 |We collect this data in order to offer a benchmark tailed to your organization. As more users of the benchmarking tool tag these categories, we'll continue to build features that allow users to sort the email benchmarking database by those fields.
99 |We will use your API key to access MailChimp and request list data. We perform a number of calculations on this raw data, and use the calculation results to generate the email report we send to you. We do not store any raw data, nor do we request any personally-idenitifying information about your list members. You may choose how we store your API key and the calculation results: 41 |
If you check either box, we will securely store your API key. You will be able to generate new, up-to-date reports instantly.
42 |You can choose to contribute your data to a "Database Average" or to receive an up-to-date email report every month.
43 |If you uncheck both boxes, we will send you a report and then discard your list data. New reports will not be instantly generated.
44 |For more information about how we use your data, see our Privacy Policy.
45 | }}){kind=logo} |
7 | Check out our new email newsletters playbook! | 8 |
11 |  }})
12 |
Chart A compares the total number of current subscribers on your list to the mean number of current subscribers across all lists we're tracking in our database.
14 |
 }})
17 | Our Tips |
18 |
| The Shorenstein Center’s Email Newsletter Playbook contains a section on email acquisition tactics, such as signup field/signup lightbox placement and off-platform acquisition strategies. That being said, all subscribers are not created equal. We encourage you to challenge your team to think about your list in terms of the current status of every email address you’ve ever acquired (see Chart B, below). | 21 |
| Want to know whom you're being benchmarked against? Check out our FAQ! | 26 |
29 |  }})
30 |
Chart B breaks down the total number of unique email addresses in the entire list into percentages. In this case, the entire list refers to all email addresses ever acquired, both currently and formerly subscribed. MailChimp has four possible values for list member status:
32 |For more information about how MailChimp breaks down your list, refer to the following support material: MailChimp: About Your Contacts.
34 ||
37 | Our Tips |
38 |
| You should pay particular attention this chart if you see large percentages of "pending" or "unsubscribed" email addresses. Pendings can be caused by a number of factors, including confusing language around confirmation for double opt-in, bot traffic, or deliverability. Consider using a service like NeverBounce to test which of your pending users are valid and active email addresses, and consider reaching out to your unsubscribers with a survey request to learn more about why they didn’t love your email product. | 41 |
45 |  }})
46 |
Chart C shows your List Open Rate. MailChimp calculates List Open Rate by taking the mean of your past Campaign Open Rates over the life of your list. Each Campaign Open Rate is calculated by dividing the number of recipients who opened the campaign email by the number of emails delivered. While List and Campaign Open Rates are the traditional way of looking at your email performance, these metrics lose a large part of the story. As with list size (Chart A, above), a better way to look at your List Open Rate is through a distribution of your subscribers' individual unique open rates (see Chart D, below).
48 |
49 |  }})
50 |
Chart D shows the distribution of open rates among current subscribers on your list. (MailChimp calculates each user's open rate by dividing the total number of emails a user has opened by the total number of emails successfully delivered to him or her.) This histogram is created through binning, which groups together consecutive continuous numbers into discrete bins. The x axis shows the range that each bin contains. As an example, the leftmost bin contains subscribers with an open rate between 0% and 10%. The rightmost bin contains subscribers with an open rate between 90% and 100%. The y axis shows the number of current subscribers who fall into each bin. Open rates generally trend downward before upticking between 80-100%. For a more comprehensive look at typical distributions, refer to Section 3.4 of our Research Guide.
52 |
53 |  }})
54 |
Chart E shows your most engaged subscribers: those who open between 80% and 100% of your emails.
56 ||
59 | Our Tips |
60 |
| We recommend surveying these users. Who are they? What are their habits? What else do they like to read? What do they want out of an email product? What might compel them to support or contribute to your work? Then, develop a few user personas. You should use this data to target your acquisition efforts towards subscribers who are likely to be be highly engaged. Alternatively, try to figure out what separates users in this category from semi-engaged subscribers (those who open ~30-80% of your emails). Might they be better served with a different newsletter or product from your newsroom? | 63 |
66 |  }})
67 |
Chart F shows your current subscribers who haven't opened one of your emails within the past 365 days. Inactive subscribers can make it harder to understand your list dynamics as well as affect your email deliverability (i.e. increase the probability that your emails are relegated to spam).
69 ||
72 | Our Tips |
73 |
| We recommend that newsrooms routinely and manually remove inactive users from their list after sending them a re-engagement campaign. This is also called "cleaning" your list. If you have never cleaned your list, we strongly recommend that you set aside some time to do so. Then, monitor to see if and when the inactive or pending segment grows. We generally recommend a list clean twice a year (around every six months). To learn more, read the leaky bucket analogy. | 76 |
Dataset average represents the mean of all lists in our dataset. · We update cached data at least once every 30 days. · We acknowledge the frequency with which emails are sent can affect metrics, with daily sends having lower engagement than weekly or monthly sends. We do not take this into account at the time of this analysis.
To learn more about analyzing the health of your email list using data science tools, please see our Research Guide as well as our comprehensive Playbook for email newsletters.
For more information on how we use and store your data, check out our Privacy Policy.
If you would no longer like to hear from us, please email us at contact@emailbenchmarking.com.
79 |