6 |
7 |
--------------------------------------------------------------------------------
/.idea/vcs.xml:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 |
--------------------------------------------------------------------------------
/.travis.yml:
--------------------------------------------------------------------------------
1 | # Config file for automatic testing at travis-ci.org
2 |
3 | language: python
4 | python:
5 | - 3.7
6 | - 3.6
7 |
8 | # Command to install dependencies, e.g. pip install -r requirements.txt --use-mirrors
9 | install: pip install -U tox-travis
10 |
11 | # Command to run tests, e.g. python setup.py test
12 | script: tox
13 |
14 | # Assuming you have installed the travis-ci CLI tool, after you
15 | # create the Github repo and add it to Travis, run the
16 | # following command to finish PyPI deployment setup:
17 | # $ travis encrypt --add deploy.password
18 | deploy:
19 | provider: pypi
20 | distributions: sdist bdist_wheel
21 | user: RBStephenson
22 | password:
23 | secure: PLEASE_REPLACE_ME
24 | on:
25 | tags: true
26 | repo: RBStephenson/ai_news_writer
27 | python: 3.7
28 |
--------------------------------------------------------------------------------
/AUTHORS.rst:
--------------------------------------------------------------------------------
1 | =======
2 | Credits
3 | =======
4 |
5 | Development Lead
6 | ----------------
7 |
8 | * R. Brent Stephenson
9 |
10 | Contributors
11 | ------------
12 |
13 | None yet. Why not be the first?
14 |
--------------------------------------------------------------------------------
/CONTRIBUTING.rst:
--------------------------------------------------------------------------------
1 | .. highlight:: shell
2 |
3 | ============
4 | Contributing
5 | ============
6 |
7 | Contributions are welcome, and they are greatly appreciated! Every little bit
8 | helps, and credit will always be given.
9 |
10 | You can contribute in many ways:
11 |
12 | Types of Contributions
13 | ----------------------
14 |
15 | Report Bugs
16 | ~~~~~~~~~~~
17 |
18 | Report bugs at https://github.com/RBStephensno/ai_news_writer/issues.
19 |
20 | If you are reporting a bug, please include:
21 |
22 | * Your operating system name and version.
23 | * Any details about your local setup that might be helpful in troubleshooting.
24 | * Detailed steps to reproduce the bug.
25 |
26 | Fix Bugs
27 | ~~~~~~~~
28 |
29 | Look through the GitHub issues for bugs. Anything tagged with "bug" and "help
30 | wanted" is open to whoever wants to implement it.
31 |
32 | Implement Features
33 | ~~~~~~~~~~~~~~~~~~
34 |
35 | Look through the GitHub issues for features. Anything tagged with "enhancement"
36 | and "help wanted" is open to whoever wants to implement it.
37 |
38 | Write Documentation
39 | ~~~~~~~~~~~~~~~~~~~
40 |
41 | ai_news_writer could always use more documentation, whether as part of the
42 | official ai_news_writer docs, in docstrings, or even on the web in blog posts,
43 | articles, and such.
44 |
45 | Submit Feedback
46 | ~~~~~~~~~~~~~~~
47 |
48 | The best way to send feedback is to file an issue at https://github.com/RBStephensno/ai_news_writer/issues.
49 |
50 | If you are proposing a feature:
51 |
52 | * Explain in detail how it would work.
53 | * Keep the scope as narrow as possible, to make it easier to implement.
54 | * Remember that this is a volunteer-driven project, and that contributions
55 | are welcome :)
56 |
57 | Get Started!
58 | ------------
59 |
60 | Ready to contribute? Here's how to set up `ai_news_writer` for local development.
61 |
62 | 1. Fork the `ai_news_writer` repo on GitHub.
63 | 2. Clone your fork locally::
64 |
65 | $ git clone git@github.com:your_name_here/ai_news_writer.git
66 |
67 | 3. Install your local copy into a virtualenv. Assuming you have virtualenvwrapper installed, this is how you set up your fork for local development::
68 |
69 | $ mkvirtualenv ai_news_writer
70 | $ cd ai_news_writer/
71 | $ python setup.py develop
72 |
73 | 4. Create a branch for local development::
74 |
75 | $ git checkout -b name-of-your-bugfix-or-feature
76 |
77 | Now you can make your changes locally.
78 |
79 | 5. When you're done making changes, check that your changes pass flake8 and the
80 | tests, including testing other Python versions with tox::
81 |
82 | $ flake8 ai_news_writer tests
83 | $ python setup.py test or pytest
84 | $ tox
85 |
86 | To get flake8 and tox, just pip install them into your virtualenv.
87 |
88 | 6. Commit your changes and push your branch to GitHub::
89 |
90 | $ git add .
91 | $ git commit -m "Your detailed description of your changes."
92 | $ git push origin name-of-your-bugfix-or-feature
93 |
94 | 7. Submit a pull request through the GitHub website.
95 |
96 | Pull Request Guidelines
97 | -----------------------
98 |
99 | Before you submit a pull request, check that it meets these guidelines:
100 |
101 | 1. The pull request should include tests.
102 | 2. If the pull request adds functionality, the docs should be updated. Put
103 | your new functionality into a function with a docstring, and add the
104 | feature to the list in README.rst.
105 | 3. The pull request should work for Python 2.7, 3.5, 3.6 and 3.7, and for PyPy. Check
106 | https://travis-ci.org/RBStephensno/ai_news_writer/pull_requests
107 | and make sure that the tests pass for all supported Python versions.
108 |
109 | Tips
110 | ----
111 |
112 | To run a subset of tests::
113 |
114 | $ pytest tests.test_ai_news_writer
115 |
116 |
117 | Deploying
118 | ---------
119 |
120 | A reminder for the maintainers on how to deploy.
121 | Make sure all your changes are committed (including an entry in HISTORY.rst).
122 | Then run::
123 |
124 | $ bump2version patch # possible: major / minor / patch
125 | $ git push
126 | $ git push --tags
127 |
128 | Travis will then deploy to PyPI if tests pass.
129 |
--------------------------------------------------------------------------------
/HISTORY.rst:
--------------------------------------------------------------------------------
1 | =======
2 | History
3 | =======
4 |
5 | 0.1.2 (2019-11-01)
6 | ------------------
7 | * Added new RSS News Feed class
8 | * Added unit tests for RSS News Feed
9 | * Added RSS News Feed List feature
10 | * Added RSS News Feed step definitions
11 |
12 | Reconsidering
13 | =============
14 | After adding `pytest-bdd` I became overwhelmed with trying to write the actual test cases for some reason. I
15 | have decided to write regular unit tests for the time being and take a look at BDD at a later date. I don't
16 | want all progress to halt while I try to figure out BDD. I am keeping everything in place up to this point
17 | and will make the build pass, other than that BDD is on hold.
18 |
19 | * Started using `pytest-bdd` for feature and test development
20 | * Updated readme file to include some more details and ideas of what I want to accomplish with this project.
21 |
22 | 0.1.1 (2019-10-19)
23 | ------------------
24 |
25 | * Corrected issues with the readme file
26 | * Several minor updates for build configurations
27 | * Bumped the version number to test out bump2version
28 | * Removed PyPI link from readme because I don't plan on publishing this to PyPI
29 |
30 | 0.1.0 (2019-10-18)
31 | ------------------
32 |
33 | * First release on PyPI.
34 |
--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
1 | MIT License
2 |
3 | Copyright (c) 2019, R. Brent Stephenson
4 |
5 | Permission is hereby granted, free of charge, to any person obtaining a copy
6 | of this software and associated documentation files (the "Software"), to deal
7 | in the Software without restriction, including without limitation the rights
8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 |
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 |
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 |
23 |
--------------------------------------------------------------------------------
/MANIFEST.in:
--------------------------------------------------------------------------------
1 | include AUTHORS.rst
2 | include CONTRIBUTING.rst
3 | include HISTORY.rst
4 | include LICENSE
5 | include README.rst
6 |
7 | recursive-include tests *
8 | recursive-exclude * __pycache__
9 | recursive-exclude * *.py[co]
10 |
11 | recursive-include docs *.rst conf.py Makefile make.bat *.jpg *.png *.gif
12 |
--------------------------------------------------------------------------------
/Makefile:
--------------------------------------------------------------------------------
1 | .PHONY: clean clean-test clean-pyc clean-build docs help
2 | .DEFAULT_GOAL := help
3 |
4 | define BROWSER_PYSCRIPT
5 | import os, webbrowser, sys
6 |
7 | try:
8 | from urllib import pathname2url
9 | except:
10 | from urllib.request import pathname2url
11 |
12 | webbrowser.open("file://" + pathname2url(os.path.abspath(sys.argv[1])))
13 | endef
14 | export BROWSER_PYSCRIPT
15 |
16 | define PRINT_HELP_PYSCRIPT
17 | import re, sys
18 |
19 | for line in sys.stdin:
20 | match = re.match(r'^([a-zA-Z_-]+):.*?## (.*)$$', line)
21 | if match:
22 | target, help = match.groups()
23 | print("%-20s %s" % (target, help))
24 | endef
25 | export PRINT_HELP_PYSCRIPT
26 |
27 | BROWSER := python -c "$$BROWSER_PYSCRIPT"
28 |
29 | help:
30 | @python -c "$$PRINT_HELP_PYSCRIPT" < $(MAKEFILE_LIST)
31 |
32 | clean: clean-build clean-pyc clean-test ## remove all build, test, coverage and Python artifacts
33 |
34 | clean-build: ## remove build artifacts
35 | rm -fr build/
36 | rm -fr dist/
37 | rm -fr .eggs/
38 | find . -name '*.egg-info' -exec rm -fr {} +
39 | find . -name '*.egg' -exec rm -f {} +
40 |
41 | clean-pyc: ## remove Python file artifacts
42 | find . -name '*.pyc' -exec rm -f {} +
43 | find . -name '*.pyo' -exec rm -f {} +
44 | find . -name '*~' -exec rm -f {} +
45 | find . -name '__pycache__' -exec rm -fr {} +
46 |
47 | clean-test: ## remove test and coverage artifacts
48 | rm -fr .tox/
49 | rm -f .coverage
50 | rm -fr htmlcov/
51 | rm -fr .pytest_cache
52 |
53 | lint: ## check style with flake8
54 | flake8 ai_news_writer tests
55 |
56 | test: ## run tests quickly with the default Python
57 | pytest
58 |
59 | test-all: ## run tests on every Python version with tox
60 | tox
61 |
62 | coverage: ## check code coverage quickly with the default Python
63 | coverage run --source ai_news_writer -m pytest
64 | coverage report -m
65 | coverage html
66 | $(BROWSER) htmlcov/index.html
67 |
68 | docs: ## generate Sphinx HTML documentation, including API docs
69 | rm -f docs/ai_news_writer.rst
70 | rm -f docs/modules.rst
71 | sphinx-apidoc -o docs/ ai_news_writer
72 | $(MAKE) -C docs clean
73 | $(MAKE) -C docs html
74 | $(BROWSER) docs/_build/html/index.html
75 |
76 | servedocs: docs ## compile the docs watching for changes
77 | watchmedo shell-command -p '*.rst' -c '$(MAKE) -C docs html' -R -D .
78 |
79 | release: dist ## package and upload a release
80 | twine upload dist/*
81 |
82 | dist: clean ## builds source and wheel package
83 | python setup.py sdist
84 | python setup.py bdist_wheel
85 | ls -l dist
86 |
87 | install: clean ## install the package to the active Python's site-packages
88 | python setup.py install
89 |
--------------------------------------------------------------------------------
/README.rst:
--------------------------------------------------------------------------------
1 | ==============
2 | ai_news_writer
3 | ==============
4 |
5 |
6 | .. image:: https://img.shields.io/travis/RBStephenson/ai_news_writer.svg
7 | :target: https://travis-ci.org/RBStephenson/ai_news_writer
8 |
9 | .. image:: https://readthedocs.org/projects/ai-news-writer/badge/?version=latest
10 | :target: https://ai-news-writer.readthedocs.io/en/latest/?badge=latest
11 | :alt: Documentation Status
12 |
13 |
14 |
15 |
16 | The primary purpose of this project is to create a news aggregator that gathers headlines and summaries announcements,
17 | news, and trending information about artificial intelligence and other cool technology stuff. After it has collected
18 | the articles and announcements, I would like it to generate a summary giving the highlights of that week's news.
19 | Ideally, after it has collected and summarized the information, I would like to generate a script based on the
20 | summaries and hopefully do a weekly online show discussing the happenings and other current events.
21 |
22 | Additionally, this is an exercise in for me to learn Cookiecutter, Git, ReadTheDocs, and Travis.CI. I have primarily
23 | worked with TFS, SVN, and Jenkins for CI. This will be a *big* learning experience for me. I'm ready to bang my head
24 | against my desk and learn everything I can.
25 |
26 | I am also taking this opportunity to learn `pytest` and `pytest-bdd` to help me take the time a plan out what I want
27 | to do.
28 |
29 | And finally, I have decided that I will also build a `django` front-end for this project. This will allow me to maintain
30 | the RSS feed collection, gather articles, and generate my output (scripts). I'm really jumping in this whole hog and
31 | trying to see what's going to stick. In any case, this will be fun and a great learning experience.
32 |
33 | If you would like to contribute this project, please feel free and have fun!
34 |
35 |
36 | * Free software: MIT license
37 | * Documentation: https://ai-news-writer.readthedocs.io.
38 |
39 |
40 | Features
41 | --------
42 |
43 | * RSS Aggregator - Collecting news and other information on a configured topic
44 | * Article Summarizing - Analyzes the content of each of the articles and summarizes the articles
45 | * Content Generation - Generate basic script based on the gathered articles and processed summaries
46 |
47 |
48 | Credits
49 | -------
50 |
51 | This package was created with Cookiecutter_ and the `audreyr/cookiecutter-pypackage`_ project template.
52 |
53 | .. _Cookiecutter: https://github.com/audreyr/cookiecutter
54 | .. _`audreyr/cookiecutter-pypackage`: https://github.com/audreyr/cookiecutter-pypackage
55 |
--------------------------------------------------------------------------------
/ai_news_writer/__init__.py:
--------------------------------------------------------------------------------
1 | # -*- coding: utf-8 -*-
2 |
3 | """Top-level package for ai_news_writer."""
4 |
5 | __author__ = """R. Brent Stephenson"""
6 | __email__ = 'brent.stephenson@brenttheprogrammer.com'
7 | __version__ = '0.1.1'
8 |
--------------------------------------------------------------------------------
/ai_news_writer/ai_news_writer.py:
--------------------------------------------------------------------------------
1 | # -*- coding: utf-8 -*-
2 |
3 | """Main module."""
4 |
5 | import feedparser
6 |
7 | ai_news_feeds = [
8 | {"name": "AI Trends", "url": "https://www.aitrends.com/feed/"},
9 | {"name": "Science Daily", "url": "https://www.sciencedaily.com/rss/computers_math/artificial_intelligence.xml"},
10 | {"name": "MIT News - Artificial Intelligence", "url": "http://news.mit.edu/rss/topic/artificial-intelligence2"},
11 | {"name": "r/artificial", "url": "https://www.reddit.com/r/artificial/.rss"},
12 | {"name": "Chatbots Magazine - Medium", "url": "https://chatbotsmagazine.com/feed"},
13 | {"name": "Towards Data Science - Medium", "url": "https://towardsdatascience.com/feed"},
14 | {"name": "Chatbots Life - Medium", "url": "https://chatbotslife.com/feed"},
15 | {"name": "AWS Machine Learning Blog", "url": "https://aws.amazon.com/blogs/machine-learning/feed/"},
16 | {"name": "Artificial Intelligence - IBM Developer",
17 | "url": "https://developer.ibm.com/patterns/category/artificial-intelligence/feed/"},
18 | {"name": "Lex Fridman - Artificial Intelligence (AI)", "url": "https://lexfridman.com/category/ai/feed/"},
19 | {"name": "r/singularity", "url": "https://www.reddit.com/r/singularity/.rss?format=xml"},
20 | {"name": "Archie.AI - Medium", "url": "https://medium.com/feed/archieai"},
21 | {"name": "The Official NVIDIA Blog", "url": "http://feeds.feedburner.com/nvidiablog"},
22 | {"name": "OpenAI Blog", "url": "https://openai.com/blog/rss/"},
23 | {"name": "VentureBeat", "url": "http://feeds.feedburner.com/venturebeat/SZYF?format=xml"},
24 | ]
25 |
26 |
27 | def parse_feed(name, url):
28 | output = feedparser.parse(url_file_stream_or_string=url)
29 | return output
30 |
--------------------------------------------------------------------------------
/ai_news_writer/cli.py:
--------------------------------------------------------------------------------
1 | # -*- coding: utf-8 -*-
2 |
3 | """Console script for ai_news_writer."""
4 | import argparse
5 | import sys
6 |
7 |
8 | def main():
9 | """Console script for ai_news_writer."""
10 | parser = argparse.ArgumentParser()
11 | parser.add_argument('_', nargs='*')
12 | args = parser.parse_args()
13 |
14 | print("Arguments: " + str(args._))
15 | print("Replace this message by putting your code into "
16 | "ai_news_writer.cli.main")
17 | return 0
18 |
19 |
20 | if __name__ == "__main__":
21 | sys.exit(main()) # pragma: no cover
22 |
--------------------------------------------------------------------------------
/ai_news_writer/rss_news_feed.py:
--------------------------------------------------------------------------------
1 | class RSSNewsFeed(object):
2 | def __init__(self):
3 | self.news_feeds = []
4 |
5 | def add_item(self, news_feed, feed_name, category):
6 | self.news_feeds.append({"url": news_feed, "name": feed_name, "category": category})
7 |
8 | def remove_item(self, url):
9 | # I'm not sure I like this method or not, which makes me think I need to
10 | # change the storage method too. I know that I want to eventually store
11 | # this in a database, so I might investigate that. Originally, I wanted
12 | # to use a pandas DataFrame so I could pass that to SQLAlchemy and
13 | # handle everything that way. I will settle for this right now, just to
14 | # keep it as simple as possible.
15 | self.news_feeds = [x for x in self.news_feeds if x['url'] != url]
16 |
17 | def update_item(self, key, url=None, name=None, category=None):
18 | # This method is a real kludge and I don't like it at all. I know that I could be doing something
19 | # much more efficient Python-wise, however, I don't know what that is right now.
20 | org_item = self.get_item(key)
21 | self.remove_item(key)
22 |
23 | new_item = {"url": url, "name": name, "category": category}
24 | if new_item['url'] is None:
25 | new_item['url'] = org_item['url']
26 |
27 | if new_item['name'] is None:
28 | new_item['name'] = org_item['name']
29 |
30 | if new_item['category'] is None:
31 | new_item['category'] = org_item['category']
32 |
33 | self.news_feeds.append(new_item)
34 |
35 | def get_item(self, key):
36 | return [item for item in self.news_feeds if item['url'] == key][0]
37 |
--------------------------------------------------------------------------------
/docs/Makefile:
--------------------------------------------------------------------------------
1 | # Minimal makefile for Sphinx documentation
2 | #
3 |
4 | # You can set these variables from the command line.
5 | SPHINXOPTS =
6 | SPHINXBUILD = python -msphinx
7 | SPHINXPROJ = ai_news_writer
8 | SOURCEDIR = .
9 | BUILDDIR = _build
10 |
11 | # Put it first so that "make" without argument is like "make help".
12 | help:
13 | @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
14 |
15 | .PHONY: help Makefile
16 |
17 | # Catch-all target: route all unknown targets to Sphinx using the new
18 | # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
19 | %: Makefile
20 | @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
21 |
--------------------------------------------------------------------------------
/docs/authors.rst:
--------------------------------------------------------------------------------
1 | .. include:: ../AUTHORS.rst
2 |
--------------------------------------------------------------------------------
/docs/conf.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python
2 | # -*- coding: utf-8 -*-
3 | #
4 | # ai_news_writer documentation build configuration file, created by
5 | # sphinx-quickstart on Fri Jun 9 13:47:02 2017.
6 | #
7 | # This file is execfile()d with the current directory set to its
8 | # containing dir.
9 | #
10 | # Note that not all possible configuration values are present in this
11 | # autogenerated file.
12 | #
13 | # All configuration values have a default; values that are commented out
14 | # serve to show the default.
15 |
16 | # If extensions (or modules to document with autodoc) are in another
17 | # directory, add these directories to sys.path here. If the directory is
18 | # relative to the documentation root, use os.path.abspath to make it
19 | # absolute, like shown here.
20 | #
21 | import os
22 | import sys
23 | sys.path.insert(0, os.path.abspath('..'))
24 |
25 | import ai_news_writer
26 |
27 | # -- General configuration ---------------------------------------------
28 |
29 | # If your documentation needs a minimal Sphinx version, state it here.
30 | #
31 | # needs_sphinx = '1.0'
32 |
33 | # Add any Sphinx extension module names here, as strings. They can be
34 | # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
35 | extensions = ['sphinx.ext.autodoc', 'sphinx.ext.viewcode']
36 |
37 | # Add any paths that contain templates here, relative to this directory.
38 | templates_path = ['_templates']
39 |
40 | # The suffix(es) of source filenames.
41 | # You can specify multiple suffix as a list of string:
42 | #
43 | # source_suffix = ['.rst', '.md']
44 | source_suffix = '.rst'
45 |
46 | # The master toctree document.
47 | master_doc = 'index'
48 |
49 | # General information about the project.
50 | project = u'ai_news_writer'
51 | copyright = u"2019, R. Brent Stephenson"
52 | author = u"R. Brent Stephenson"
53 |
54 | # The version info for the project you're documenting, acts as replacement
55 | # for |version| and |release|, also used in various other places throughout
56 | # the built documents.
57 | #
58 | # The short X.Y version.
59 | version = ai_news_writer.__version__
60 | # The full version, including alpha/beta/rc tags.
61 | release = ai_news_writer.__version__
62 |
63 | # The language for content autogenerated by Sphinx. Refer to documentation
64 | # for a list of supported languages.
65 | #
66 | # This is also used if you do content translation via gettext catalogs.
67 | # Usually you set "language" from the command line for these cases.
68 | language = None
69 |
70 | # List of patterns, relative to source directory, that match files and
71 | # directories to ignore when looking for source files.
72 | # This patterns also effect to html_static_path and html_extra_path
73 | exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
74 |
75 | # The name of the Pygments (syntax highlighting) style to use.
76 | pygments_style = 'sphinx'
77 |
78 | # If true, `todo` and `todoList` produce output, else they produce nothing.
79 | todo_include_todos = False
80 |
81 |
82 | # -- Options for HTML output -------------------------------------------
83 |
84 | # The theme to use for HTML and HTML Help pages. See the documentation for
85 | # a list of builtin themes.
86 | #
87 | html_theme = 'alabaster'
88 |
89 | # Theme options are theme-specific and customize the look and feel of a
90 | # theme further. For a list of options available for each theme, see the
91 | # documentation.
92 | #
93 | # html_theme_options = {}
94 |
95 | # Add any paths that contain custom static files (such as style sheets) here,
96 | # relative to this directory. They are copied after the builtin static files,
97 | # so a file named "default.css" will overwrite the builtin "default.css".
98 | html_static_path = ['_static']
99 |
100 |
101 | # -- Options for HTMLHelp output ---------------------------------------
102 |
103 | # Output file base name for HTML help builder.
104 | htmlhelp_basename = 'ai_news_writerdoc'
105 |
106 |
107 | # -- Options for LaTeX output ------------------------------------------
108 |
109 | latex_elements = {
110 | # The paper size ('letterpaper' or 'a4paper').
111 | #
112 | # 'papersize': 'letterpaper',
113 |
114 | # The font size ('10pt', '11pt' or '12pt').
115 | #
116 | # 'pointsize': '10pt',
117 |
118 | # Additional stuff for the LaTeX preamble.
119 | #
120 | # 'preamble': '',
121 |
122 | # Latex figure (float) alignment
123 | #
124 | # 'figure_align': 'htbp',
125 | }
126 |
127 | # Grouping the document tree into LaTeX files. List of tuples
128 | # (source start file, target name, title, author, documentclass
129 | # [howto, manual, or own class]).
130 | latex_documents = [
131 | (master_doc, 'ai_news_writer.tex',
132 | u'ai_news_writer Documentation',
133 | u'R. Brent Stephenson', 'manual'),
134 | ]
135 |
136 |
137 | # -- Options for manual page output ------------------------------------
138 |
139 | # One entry per manual page. List of tuples
140 | # (source start file, name, description, authors, manual section).
141 | man_pages = [
142 | (master_doc, 'ai_news_writer',
143 | u'ai_news_writer Documentation',
144 | [author], 1)
145 | ]
146 |
147 |
148 | # -- Options for Texinfo output ----------------------------------------
149 |
150 | # Grouping the document tree into Texinfo files. List of tuples
151 | # (source start file, target name, title, author,
152 | # dir menu entry, description, category)
153 | texinfo_documents = [
154 | (master_doc, 'ai_news_writer',
155 | u'ai_news_writer Documentation',
156 | author,
157 | 'ai_news_writer',
158 | 'One line description of project.',
159 | 'Miscellaneous'),
160 | ]
161 |
162 |
163 |
164 |
--------------------------------------------------------------------------------
/docs/contributing.rst:
--------------------------------------------------------------------------------
1 | .. include:: ../CONTRIBUTING.rst
2 |
--------------------------------------------------------------------------------
/docs/history.rst:
--------------------------------------------------------------------------------
1 | .. include:: ../HISTORY.rst
2 |
--------------------------------------------------------------------------------
/docs/index.rst:
--------------------------------------------------------------------------------
1 | Welcome to ai_news_writer's documentation!
2 | ======================================
3 |
4 | .. toctree::
5 | :maxdepth: 2
6 | :caption: Contents:
7 |
8 | readme
9 | installation
10 | usage
11 | modules
12 | contributing
13 | authors
14 | history
15 |
16 | Indices and tables
17 | ==================
18 | * :ref:`genindex`
19 | * :ref:`modindex`
20 | * :ref:`search`
21 |
--------------------------------------------------------------------------------
/docs/installation.rst:
--------------------------------------------------------------------------------
1 | .. highlight:: shell
2 |
3 | ============
4 | Installation
5 | ============
6 |
7 |
8 | Stable release
9 | --------------
10 |
11 | To install ai_news_writer, run this command in your terminal:
12 |
13 | .. code-block:: console
14 |
15 | $ pip install ai_news_writer
16 |
17 | This is the preferred method to install ai_news_writer, as it will always install the most recent stable release.
18 |
19 | If you don't have `pip`_ installed, this `Python installation guide`_ can guide
20 | you through the process.
21 |
22 | .. _pip: https://pip.pypa.io
23 | .. _Python installation guide: http://docs.python-guide.org/en/latest/starting/installation/
24 |
25 |
26 | From sources
27 | ------------
28 |
29 | The sources for ai_news_writer can be downloaded from the `Github repo`_.
30 |
31 | You can either clone the public repository:
32 |
33 | .. code-block:: console
34 |
35 | $ git clone git://github.com/RBStephensno/ai_news_writer
36 |
37 | Or download the `tarball`_:
38 |
39 | .. code-block:: console
40 |
41 | $ curl -OJL https://github.com/RBStephensno/ai_news_writer/tarball/master
42 |
43 | Once you have a copy of the source, you can install it with:
44 |
45 | .. code-block:: console
46 |
47 | $ python setup.py install
48 |
49 |
50 | .. _Github repo: https://github.com/RBStephensno/ai_news_writer
51 | .. _tarball: https://github.com/RBStephensno/ai_news_writer/tarball/master
52 |
--------------------------------------------------------------------------------
/docs/make.bat:
--------------------------------------------------------------------------------
1 | @ECHO OFF
2 |
3 | pushd %~dp0
4 |
5 | REM Command file for Sphinx documentation
6 |
7 | if "%SPHINXBUILD%" == "" (
8 | set SPHINXBUILD=python -msphinx
9 | )
10 | set SOURCEDIR=.
11 | set BUILDDIR=_build
12 | set SPHINXPROJ=ai_news_writer
13 |
14 | if "%1" == "" goto help
15 |
16 | %SPHINXBUILD% >NUL 2>NUL
17 | if errorlevel 9009 (
18 | echo.
19 | echo.The Sphinx module was not found. Make sure you have Sphinx installed,
20 | echo.then set the SPHINXBUILD environment variable to point to the full
21 | echo.path of the 'sphinx-build' executable. Alternatively you may add the
22 | echo.Sphinx directory to PATH.
23 | echo.
24 | echo.If you don't have Sphinx installed, grab it from
25 | echo.http://sphinx-doc.org/
26 | exit /b 1
27 | )
28 |
29 | %SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%
30 | goto end
31 |
32 | :help
33 | %SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%
34 |
35 | :end
36 | popd
37 |
--------------------------------------------------------------------------------
/docs/readme.rst:
--------------------------------------------------------------------------------
1 | .. include:: ../README.rst
2 |
--------------------------------------------------------------------------------
/docs/usage.rst:
--------------------------------------------------------------------------------
1 | =====
2 | Usage
3 | =====
4 |
5 | To use ai_news_writer in a project::
6 |
7 | import ai_news_writer
8 |
--------------------------------------------------------------------------------
/requirements_dev.txt:
--------------------------------------------------------------------------------
1 | pip==19.2.3
2 | bump2version==0.5.11
3 | wheel==0.33.6
4 | watchdog==0.9.0
5 | flake8==3.7.8
6 | tox==3.14.0
7 | coverage==4.5.4
8 | Sphinx==1.8.5
9 | twine==1.14.0
10 |
11 | pytest==4.6.5
12 | pytest-runner==5.1
--------------------------------------------------------------------------------
/setup.cfg:
--------------------------------------------------------------------------------
1 | [bumpversion]
2 | current_version = 0.1.1
3 | commit = True
4 | tag = True
5 |
6 | [bumpversion:file:setup.py]
7 | search = version='{current_version}'
8 | replace = version='{new_version}'
9 |
10 | [bumpversion:file:ai_news_writer/__init__.py]
11 | search = __version__ = '{current_version}'
12 | replace = __version__ = '{new_version}'
13 |
14 | [bdist_wheel]
15 | universal = 1
16 |
17 | [flake8]
18 | exclude = docs
19 |
20 | [aliases]
21 | test = pytest
22 |
23 | [tool:pytest]
24 | collect_ignore = ['setup.py']
25 |
26 |
--------------------------------------------------------------------------------
/setup.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python
2 | # -*- coding: utf-8 -*-
3 |
4 | """The setup script."""
5 |
6 | from setuptools import setup, find_packages
7 |
8 | with open('README.rst') as readme_file:
9 | readme = readme_file.read()
10 |
11 | with open('HISTORY.rst') as history_file:
12 | history = history_file.read()
13 |
14 | requirements = ['feedparser']
15 |
16 | setup_requirements = ['pytest-runner', ]
17 |
18 | test_requirements = ['pytest>=3', ]
19 |
20 | setup(
21 | author="R. Brent Stephenson",
22 | author_email='brent.stephenson@brenttheprogrammer.com',
23 | # python_requires='>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*',
24 | python_requires='>=3.6, <3.8',
25 | classifiers=[
26 | 'Development Status :: 2 - Pre-Alpha',
27 | 'Intended Audience :: Developers',
28 | 'License :: OSI Approved :: MIT License',
29 | 'Natural Language :: English',
30 | # "Programming Language :: Python :: 2",
31 | # 'Programming Language :: Python :: 2.7',
32 | # 'Programming Language :: Python :: 3',
33 | # 'Programming Language :: Python :: 3.5',
34 | 'Programming Language :: Python :: 3.6',
35 | 'Programming Language :: Python :: 3.7',
36 | ],
37 | description="Python Boilerplate contains all the boilerplate you need to create a Python package.",
38 | entry_points={
39 | 'console_scripts': [
40 | 'ai_news_writer=ai_news_writer.cli:main',
41 | ],
42 | },
43 | install_requires=requirements,
44 | license="MIT license",
45 | long_description=readme + '\n\n' + history,
46 | include_package_data=True,
47 | keywords='ai_news_writer',
48 | name='ai_news_writer',
49 | packages=find_packages(include=['ai_news_writer', 'ai_news_writer.*']),
50 | setup_requires=setup_requirements,
51 | test_suite='tests',
52 | tests_require=test_requirements,
53 | url='https://github.com/RBStephenson/ai_news_writer',
54 | version='0.1.1',
55 | zip_safe=False,
56 | )
57 |
--------------------------------------------------------------------------------
/tests/__init__.py:
--------------------------------------------------------------------------------
1 | # -*- coding: utf-8 -*-
2 |
3 | """Unit test package for ai_news_writer."""
4 |
--------------------------------------------------------------------------------
/tests/features/rss_news_feeds.feature:
--------------------------------------------------------------------------------
1 | ## Created by brent at 11/1/2019
2 | #
3 | ## This python module is going to be used in a Natural Language Processing application. Right now, I am planning
4 | ## on creating a Django front end for this application. The Django front-end will allow us to manage and maintain
5 | ## the RSS Feed collection we use to build our scripts and summaries as well as keep an archive all of the stories,
6 | ## articles, generated summaries, and scripts.
7 | #
8 | ## This first set of tests is specifically for creating and maintaining our RSS feed collection. My idea is that
9 | ## I will use BDD (pytest-bdd) for development and feature design (see below tests). I am also following along
10 | ## With the pytest practices defined in https://www.obeythetestinggoat.com/ (great resource and I highly recommend it).
11 | #
12 | ## As a content creator, I want to be able to find RSS News Feeds that are specifically about AI (and all related
13 | ## disciplines.
14 | #
15 | ## Once I have found an RSS feed, I want to be able to add that to a collection of RSS feeds so that I can scrape
16 | ## articles from the feed.
17 | #
18 | ## I will need to be able to store the RSS Feed's name, URL, and it's category (AI, Computer, Tech, etc).
19 | #
20 | ## After adding an RSS Feed to the application, I will need to be able to edit the name, url and category and save
21 | ## those changes
22 | #
23 | ## After adding an RSS Feed to the application, I may need to delete or remove the RSS Feed from the application's
24 | ## collection
25 | #
26 | ## After adding an RSS Feed to the application, I may need to disable the feed without deleting it.
27 | #
28 | #@rss @news_feeds
29 | #Feature: RSS News Feed List
30 | # As a content creator,
31 | # I want to be able to create a collection of RSS News Feeds
32 | # So I can use them as source material for content generation
33 | #
34 | # Scenario:
35 | # Given I am a content creator
36 | # And I have an RSS news feed
37 | # When I execute the save command
38 | # Then I should not get an error
39 | # And I should see the RSS news feed added to the RSS News Feed List
40 | #
41 | # Scenario:
42 | # Given I am a content creator
43 | # And I want to remove an RSS news feed
44 | # When I select an RSS news feed
45 | # And execute the delete command
46 | # Then I should not get an error
47 | # And the RSS news feed should be removed from the RSS News Feed List
48 | #
49 | # Scenario:
50 | # Given I am a content creator
51 | # And I want to change information for an RSS News Feed
52 | # When I select the RSS News feed from the list
53 | # And supply updated information
54 | # When I execute the update command
55 | # Then I should not get an error
56 | # And I should see the changes in the RSS News Feed List
57 | #
58 | #
59 |
--------------------------------------------------------------------------------
/tests/step_defs/rss_news_feeds.py:
--------------------------------------------------------------------------------
1 | # from pytest_bdd import given, when, then
2 | #
3 | #
4 | # @given("I am a content creator")
5 | # def step_impl():
6 | # raise NotImplementedError(u'STEP: Given I am a content creator')
7 | #
8 | #
9 | # @given("I have an RSS news feed")
10 | # def step_impl():
11 | # raise NotImplementedError(u'STEP: And I have an RSS news feed')
12 | #
13 | #
14 | # @when("I execute the save command")
15 | # def step_impl():
16 | # raise NotImplementedError(u'STEP: When I execute the save command')
17 | #
18 | #
19 | # @then("I should not get an error")
20 | # def step_impl():
21 | # raise NotImplementedError(u'STEP: Then I should not get an error')
22 | #
23 | #
24 | # @given("I should see the RSS news feed added to the RSS News Feed List")
25 | # def step_impl():
26 | # raise NotImplementedError(u'STEP: And I should see the RSS news feed added to the RSS News Feed List')
27 | #
28 | #
29 | # @given("I want to remove an RSS news feed")
30 | # def step_impl():
31 | # raise NotImplementedError(u'STEP: And I want to remove an RSS news feed')
32 | #
33 | #
34 | # @when("I select an RSS news feed")
35 | # def step_impl():
36 | # raise NotImplementedError(u'STEP: When I select an RSS news feed')
37 | #
38 | #
39 | # @given("execute the delete command")
40 | # def step_impl():
41 | # raise NotImplementedError(u'STEP: And execute the delete command')
42 | #
43 | #
44 | # @given("the RSS news feed should be removed from the RSS News Feed List")
45 | # def step_impl():
46 | # raise NotImplementedError(u'STEP: And the RSS news feed should be removed from the RSS News Feed List')
47 | #
48 | #
49 | # @given("I want to change information for an RSS News Feed")
50 | # def step_impl():
51 | # raise NotImplementedError(u'STEP: And I want to change information for an RSS News Feed')
52 | #
53 | #
54 | # @when("I select the RSS News feed from the list")
55 | # def step_impl():
56 | # raise NotImplementedError(u'STEP: When I select the RSS News feed from the list')
57 | #
58 | #
59 | # @given("supply updated information")
60 | # def step_impl():
61 | # raise NotImplementedError(u'STEP: And supply updated information')
62 | #
63 | #
64 | # @when("I execute the update command")
65 | # def step_impl():
66 | # raise NotImplementedError(u'STEP: When I execute the update command')
67 | #
68 | #
69 | # @given("I should see the changes in the RSS News Feed List")
70 | # def step_impl():
71 | # raise NotImplementedError(u'STEP: And I should see the changes in the RSS News Feed List')
72 |
--------------------------------------------------------------------------------
/tests/test_ai_news_writer.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python
2 | # -*- coding: utf-8 -*-
3 |
4 | """Tests for `ai_news_writer` package."""
5 |
6 | import pytest
7 |
8 | from ai_news_writer import ai_news_writer
9 |
10 |
11 | @pytest.fixture
12 | def ai_news_feeds():
13 | ai_feeds = ai_news_writer.ai_news_feeds
14 | return ai_feeds
15 |
16 |
17 | @pytest.fixture
18 | def parse_feed():
19 | feed = ai_news_writer.ai_news_feeds[0]
20 | return ai_news_writer.parse_feed(feed['name'], feed['url'])
21 |
22 |
23 | def test_content(ai_news_feeds):
24 | """Sample pytest test function with the pytest fixture as an argument."""
25 | # from bs4 import BeautifulSoup
26 | # assert 'GitHub' in BeautifulSoup(response.content).title.string
27 | assert len(ai_news_feeds) > 0
28 |
29 |
30 | def test_feed_parse(parse_feed):
31 | print(parse_feed.feed.title)
32 | print(parse_feed.feed.link)
33 | print(parse_feed.feed.description)
34 | print(len(parse_feed.entries))
35 |
36 | if len(parse_feed.entries) > 0:
37 | for entry in parse_feed.entries:
38 | print(entry.title)
39 | print(entry.link)
40 | print(entry.id)
41 | print(entry.published)
42 | print(entry.summary)
43 | print(entry.content)
44 |
--------------------------------------------------------------------------------
/tests/test_rss_news_feeds.py:
--------------------------------------------------------------------------------
1 | import unittest
2 |
3 | from ai_news_writer import rss_news_feed
4 |
5 |
6 | class RSSNewsFeedsUnitTests(unittest.TestCase):
7 | def test_add_rss_news_feed(self):
8 | obj_under_test = rss_news_feed.RSSNewsFeed()
9 | obj_under_test.add_item('http://www.mynewsfeed.com', 'My News Feed', 'Test Category')
10 | self.assertTrue(obj_under_test.news_feeds[0]['url'] == 'http://www.mynewsfeed.com')
11 | print(obj_under_test.news_feeds)
12 |
13 | def test_update_rss_news_feed(self):
14 | obj_under_test = rss_news_feed.RSSNewsFeed()
15 | obj_under_test.add_item('http://www.mynewsfeed.com', 'My News Feed', 'Test Category')
16 | obj_under_test.update_item(key='http://www.mynewsfeed.com', url='http://www.mynewsfeed2.com')
17 | self.assertTrue(obj_under_test.news_feeds[0]['url'] != 'http://www.mynewsfeed.com')
18 | self.assertTrue(obj_under_test.news_feeds[0]['url'] == 'http://www.mynewsfeed2.com')
19 | print(obj_under_test.news_feeds)
20 |
21 | def test_delete_rss_news_feed(self):
22 | obj_under_test = rss_news_feed.RSSNewsFeed()
23 | obj_under_test.add_item('http://www.mynewsfeed.com', 'My News Feed', 'Test Category')
24 | self.assertTrue(obj_under_test.news_feeds[0]['url'] == 'http://www.mynewsfeed.com')
25 | obj_under_test.remove_item('http://www.mynewsfeed.com')
26 | self.assertTrue(len(obj_under_test.news_feeds) == 0)
27 | print(obj_under_test.news_feeds)
28 |
29 | def test_get_rss_news_feed_collection(self):
30 | obj_under_test = rss_news_feed.RSSNewsFeed()
31 | obj_under_test.add_item('http://www.mynewsfeed.com', 'My News Feed', 'Test Category')
32 | obj_under_test.add_item('http://www.notmynewsfeed.com', 'Not My News Feed', 'Test Category')
33 | obj_under_test.add_item('http://www.anothernewsfeed.com', 'Another News Feed', 'Test Category')
34 |
35 | expected_item = obj_under_test.news_feeds[1]
36 | actual_item = obj_under_test.get_item('http://www.notmynewsfeed.com')
37 |
38 | self.assertEqual(expected_item, actual_item)
39 |
40 |
41 | if __name__ == '__main__':
42 | unittest.main()
43 |
--------------------------------------------------------------------------------
/tox.ini:
--------------------------------------------------------------------------------
1 | [tox]
2 | envlist = py36, py37 flake8
3 |
4 | [travis]
5 | python =
6 | 3.7: py37
7 | 3.6: py36
8 | ; 3.5: py35
9 | ; 2.7: py27
10 |
11 | [testenv:flake8]
12 | basepython = python
13 | deps = flake8
14 | commands = flake8 ai_news_writer
15 |
16 | [testenv]
17 | setenv =
18 | PYTHONPATH = {toxinidir}
19 | deps =
20 | -r{toxinidir}/requirements_dev.txt
21 | ; If you want to make tox run the tests with the same versions, create a
22 | ; requirements.txt with the pinned versions and uncomment the following line:
23 | ; -r{toxinidir}/requirements.txt
24 | commands =
25 | pip install -U pip
26 | pytest --basetemp={envtmpdir}
27 |
28 |
--------------------------------------------------------------------------------