├── setup.cfg
├── tests
├── __init__.py
└── test_pyfpds.py
├── pyfpds
├── pyfpds.py
└── __init__.py
├── site
├── img
│ └── favicon.ico
├── fonts
│ ├── fontawesome-webfont.eot
│ ├── fontawesome-webfont.ttf
│ └── fontawesome-webfont.woff
├── css
│ ├── theme_extra.css
│ ├── highlight.css
│ └── theme.css
├── license
│ └── highlight.js
│ │ └── LICENSE
├── js
│ ├── theme.js
│ └── highlight.pack.js
└── index.html
├── mkdocs.yml
├── tox.ini
├── requirements.txt
├── example.py
├── CONTRIBUTING.md
├── .gitignore
├── LICENSE
├── Makefile
├── README.md
├── setup.py
└── docs
└── index.md
/setup.cfg:
--------------------------------------------------------------------------------
1 | [wheel]
2 | universal = 1
--------------------------------------------------------------------------------
/tests/__init__.py:
--------------------------------------------------------------------------------
1 | # -*- coding: utf-8 -*-
--------------------------------------------------------------------------------
/pyfpds/pyfpds.py:
--------------------------------------------------------------------------------
1 | # -*- coding: utf-8 -*-
2 |
--------------------------------------------------------------------------------
/site/img/favicon.ico:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/18F/pyfpds/HEAD/site/img/favicon.ico
--------------------------------------------------------------------------------
/mkdocs.yml:
--------------------------------------------------------------------------------
1 | site_name: Pyfpds
2 | pages:
3 | - [index.md, Pyfpds]
4 | theme: readthedocs
5 |
--------------------------------------------------------------------------------
/site/fonts/fontawesome-webfont.eot:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/18F/pyfpds/HEAD/site/fonts/fontawesome-webfont.eot
--------------------------------------------------------------------------------
/site/fonts/fontawesome-webfont.ttf:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/18F/pyfpds/HEAD/site/fonts/fontawesome-webfont.ttf
--------------------------------------------------------------------------------
/site/fonts/fontawesome-webfont.woff:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/18F/pyfpds/HEAD/site/fonts/fontawesome-webfont.woff
--------------------------------------------------------------------------------
/tox.ini:
--------------------------------------------------------------------------------
1 | [tox]
2 | envlist = py26, py27, py33, py34
3 |
4 | [testenv]
5 | setenv =
6 | PYTHONPATH = {toxinidir}:{toxinidir}/pyfpds
7 | commands = python setup.py test
8 | deps =
9 | -r{toxinidir}/requirements.txt
--------------------------------------------------------------------------------
/requirements.txt:
--------------------------------------------------------------------------------
1 | Jinja2==2.7.3
2 | Markdown==2.4.1
3 | MarkupSafe==0.23
4 | PyYAML==3.11
5 | argh==0.26.1
6 | #ghp-import==0.4.1
7 | #mkdocs==0.11.1
8 | pathtools==0.1.2
9 | pkginfo==1.1
10 | requests==2.3.0
11 | twine==1.3.1
12 | watchdog==0.8.2
13 | xmltodict==0.9.0
14 |
--------------------------------------------------------------------------------
/example.py:
--------------------------------------------------------------------------------
1 | from pyfpds import Contracts
2 |
3 | c = Contracts()
4 |
5 | #filter by a specific contract ID number
6 | records = c.get(piid="FA865014M5002")
7 |
8 | r = records[0]['content']['award']
9 |
10 | #pretty print the first record
11 | c.pretty_print(r)
12 |
13 | print("Length: {0}".format(len(records)))
14 |
--------------------------------------------------------------------------------
/CONTRIBUTING.md:
--------------------------------------------------------------------------------
1 | ## Public domain
2 |
3 | This project is in the public domain within the United States, and
4 | copyright and related rights in the work worldwide are waived through
5 | the [CC0 1.0 Universal public domain dedication](https://creativecommons.org/publicdomain/zero/1.0/).
6 |
7 | All contributions to this project will be released under the CC0
8 | dedication. By submitting a pull request, you are agreeing to comply
9 | with this waiver of copyright interest.
10 |
--------------------------------------------------------------------------------
/tests/test_pyfpds.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python
2 | # -*- coding: utf-8 -*-
3 |
4 | """
5 | test_pyfpds
6 | ----------------------------------
7 |
8 | Tests for `pyfpds` module.
9 | """
10 |
11 | import unittest
12 |
13 | from pyfpds import pyfpds
14 |
15 |
16 | class TestPyfpds(unittest.TestCase):
17 |
18 | def setUp(self):
19 | pass
20 |
21 | def test_something(self):
22 | pass
23 |
24 | def tearDown(self):
25 | pass
26 |
27 | if __name__ == '__main__':
28 | unittest.main()
--------------------------------------------------------------------------------
/.gitignore:
--------------------------------------------------------------------------------
1 | *.py[cod]
2 | .editorconfig
3 | .travis.yml
4 |
5 | # C extensions
6 | *.so
7 |
8 | # Packages
9 | *.egg
10 | *.egg-info
11 | dist
12 | build
13 | eggs
14 | parts
15 | bin
16 | var
17 | sdist
18 | develop-eggs
19 | .installed.cfg
20 | lib
21 | lib64
22 |
23 | # Installer logs
24 | pip-log.txt
25 |
26 | # Unit test / coverage reports
27 | .coverage
28 | .tox
29 | nosetests.xml
30 | htmlcov
31 |
32 | # Translations
33 | *.mo
34 |
35 | # Mr Developer
36 | .mr.developer.cfg
37 | .project
38 | .pydevproject
39 |
40 | # Complexity
41 | output/*.html
42 | output/*/index.html
43 |
44 | # Sphinx
45 | docs/_build
46 |
47 | Eclipse IDE
48 | .project
49 | .pydevproject
50 | .settings/
51 |
52 |
--------------------------------------------------------------------------------
/site/css/theme_extra.css:
--------------------------------------------------------------------------------
1 | /*
2 | * Sphinx doesn't have support for section dividers like we do in
3 | * MkDocs, this styles the section titles in the nav
4 | *
5 | * https://github.com/tomchristie/mkdocs/issues/175
6 | */
7 | .wy-menu-vertical span {
8 | line-height: 18px;
9 | padding: 0.4045em 0.809em;
10 | display: block;
11 | position: relative;
12 | font-size: 90%;
13 | color: #838383;
14 | }
15 |
16 | /*
17 | * Long navigations run off the bottom of the screen as the nav
18 | * area doesn't scroll.
19 | *
20 | * https://github.com/tomchristie/mkdocs/pull/202
21 | */
22 | .wy-nav-side {
23 | height: 100%;
24 | overflow-y: auto;
25 | }
26 |
27 | /*
28 | * Fix wrapping in the code highlighting
29 | *
30 | * https://github.com/tomchristie/mkdocs/issues/233
31 | */
32 | code {
33 | white-space: pre;
34 | }
35 |
--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
1 | As a work of the United States Government, this project is in the
2 | public domain within the United States.
3 |
4 | Additionally, we waive copyright and related rights in the work
5 | worldwide through the CC0 1.0 Universal public domain dedication.
6 |
7 | ## CC0 1.0 Universal Summary
8 |
9 | This is a human-readable summary of the [Legal Code (read the full text)](https://creativecommons.org/publicdomain/zero/1.0/legalcode).
10 |
11 | ### No Copyright
12 |
13 | The person who associated a work with this deed has dedicated the work to
14 | the public domain by waiving all of his or her rights to the work worldwide
15 | under copyright law, including all related and neighboring rights, to the
16 | extent allowed by law.
17 |
18 | You can copy, modify, distribute and perform the work, even for commercial
19 | purposes, all without asking permission.
20 |
21 | ### Other Information
22 |
23 | In no way are the patent or trademark rights of any person affected by CC0,
24 | nor are the rights that other persons may have in the work or in how the
25 | work is used, such as publicity or privacy rights.
26 |
27 | Unless expressly stated otherwise, the person who associated a work with
28 | this deed makes no warranties about the work, and disclaims liability for
29 | all uses of the work, to the fullest extent permitted by applicable law.
30 | When using or citing the work, you should not imply endorsement by the
31 | author or the affirmer.
32 |
--------------------------------------------------------------------------------
/Makefile:
--------------------------------------------------------------------------------
1 | .PHONY: clean-pyc clean-build docs clean
2 |
3 | help:
4 | @echo "clean-build - remove build artifacts"
5 | @echo "clean-pyc - remove Python file artifacts"
6 | @echo "lint - check style with flake8"
7 | @echo "test - run tests quickly with the default Python"
8 | @echo "test-all - run tests on every Python version with tox"
9 | @echo "coverage - check code coverage quickly with the default Python"
10 | @echo "docs - generate Sphinx HTML documentation, including API docs"
11 | @echo "release - package and upload a release"
12 | @echo "dist - package"
13 |
14 | clean: clean-build clean-pyc
15 | rm -fr htmlcov/
16 |
17 | clean-build:
18 | rm -fr build/
19 | rm -fr dist/
20 | rm -fr *.egg-info
21 |
22 | clean-pyc:
23 | find . -name '*.pyc' -exec rm -f {} +
24 | find . -name '*.pyo' -exec rm -f {} +
25 | find . -name '*~' -exec rm -f {} +
26 |
27 | lint:
28 | flake8 pyfpds tests
29 |
30 | test:
31 | python setup.py test
32 |
33 | test-all:
34 | tox
35 |
36 | coverage:
37 | coverage run --source pyfpds setup.py test
38 | coverage report -m
39 | coverage html
40 | open htmlcov/index.html
41 |
42 | docs:
43 | rm -f docs/pyfpds.rst
44 | rm -f docs/modules.rst
45 | sphinx-apidoc -o docs/ pyfpds
46 | $(MAKE) -C docs clean
47 | $(MAKE) -C docs html
48 | open docs/_build/html/index.html
49 |
50 | release: clean
51 | python setup.py sdist upload
52 | python setup.py bdist_wheel upload
53 |
54 | dist: clean
55 | python setup.py sdist
56 | python setup.py bdist_wheel
57 | ls -l dist
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # pyfpds
2 | pyfpds is a basic python wrapper for accessing federal contracting data in the Federal Procurement Data System (FPDS). The only programmatic access to this data is via an ATOM feed that limits each request to 10 records. This can be quite frustrating if you want more than 10 records! This library will grab up to any number of records requested (the default being 100) and compile them into one data structure. Subsequently, the performance is not exemplary, as much of the processing time is spent in http transport. However, if you have the time, it makes life a bit easier :). Additionally, the ATOM feed does not support sorting. If you want to get complete data sorted by a field, you essentially have to pull down all records and sort them in python.
3 |
4 | This project was created to assist with ETL tasks that are part of the [Mirage project](https://github.com/18F/mirage). If you have features, open an issue, or consider contributing to the project yourself (see CONTRIBUTING).
5 |
6 | ### Requirements
7 | This project supports python 2.7+. To install the dependencies, use pip:
8 | ``` pip install -r requirements.txt ```
9 |
10 |
11 | ### Some helpful links:
12 | * [FPDS ATOM feed FAQ](http://beta.fpdsng.com/wiki/index.php/ATOM_Feed_FAQ)
13 | * [FPDS ATOM feed query fields and example usage](https://www.fpds.gov/wiki/index.php/Atom_Feed_Usage)
14 | * [FPDS search](https://www.fpds.gov/fpdsng_cms/index.php/en/)
15 | * [Full documentation for pyfpds](https://pyfpds.readthedocs.org)
16 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
--------------------------------------------------------------------------------
/site/license/highlight.js/LICENSE:
--------------------------------------------------------------------------------
1 | Copyright (c) 2006, Ivan Sagalaev
2 | All rights reserved.
3 | Redistribution and use in source and binary forms, with or without
4 | modification, are permitted provided that the following conditions are met:
5 |
6 | * Redistributions of source code must retain the above copyright
7 | notice, this list of conditions and the following disclaimer.
8 | * Redistributions in binary form must reproduce the above copyright
9 | notice, this list of conditions and the following disclaimer in the
10 | documentation and/or other materials provided with the distribution.
11 | * Neither the name of highlight.js nor the names of its contributors
12 | may be used to endorse or promote products derived from this software
13 | without specific prior written permission.
14 |
15 | THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND ANY
16 | EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
17 | WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
18 | DISCLAIMED. IN NO EVENT SHALL THE REGENTS AND CONTRIBUTORS BE LIABLE FOR ANY
19 | DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
20 | (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
21 | LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
22 | ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
23 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
24 | SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
25 |
--------------------------------------------------------------------------------
/setup.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python
2 | # -*- coding: utf-8 -*-
3 |
4 |
5 | try:
6 | from setuptools import setup
7 | except ImportError:
8 | from distutils.core import setup
9 |
10 |
11 | readme = open('README.md').read()
12 |
13 | requirements = [
14 | # TODO: put package requirements here
15 | 'xmltodict',
16 | 'requests',
17 | ]
18 |
19 | test_requirements = [
20 | # TODO: put package test requirements here
21 | ]
22 |
23 | setup(
24 | name='pyfpds',
25 | version='0.1.0',
26 | description='pyfpds is a python wrapper for accessing federal contracting data in the Federal Procurement Data System (FPDS)',
27 | long_description=readme,
28 | author='Kaitlin Devine',
29 | author_email='kaitlin.devine@gsa.gov',
30 | url='https://github.com/kaitlin/pyfpds',
31 | packages=[
32 | 'pyfpds',
33 | ],
34 | package_dir={'pyfpds':
35 | 'pyfpds'},
36 | include_package_data=True,
37 | install_requires=requirements,
38 | license="BSD",
39 | zip_safe=False,
40 | keywords='pyfpds',
41 | classifiers=[
42 | 'Development Status :: 2 - Pre-Alpha',
43 | 'Intended Audience :: Developers',
44 | 'License :: OSI Approved :: BSD License',
45 | 'Natural Language :: English',
46 | "Programming Language :: Python :: 2",
47 | 'Programming Language :: Python :: 2.6',
48 | 'Programming Language :: Python :: 2.7',
49 | 'Programming Language :: Python :: 3',
50 | 'Programming Language :: Python :: 3.3',
51 | 'Programming Language :: Python :: 3.4',
52 | ],
53 | test_suite='tests',
54 | tests_require=test_requirements
55 | )
56 |
--------------------------------------------------------------------------------
/site/js/theme.js:
--------------------------------------------------------------------------------
1 | $( document ).ready(function() {
2 | // Shift nav in mobile when clicking the menu.
3 | $(document).on('click', "[data-toggle='wy-nav-top']", function() {
4 | $("[data-toggle='wy-nav-shift']").toggleClass("shift");
5 | $("[data-toggle='rst-versions']").toggleClass("shift");
6 | });
7 | // Close menu when you click a link.
8 | $(document).on('click', ".wy-menu-vertical .current ul li a", function() {
9 | $("[data-toggle='wy-nav-shift']").removeClass("shift");
10 | $("[data-toggle='rst-versions']").toggleClass("shift");
11 | });
12 | $(document).on('click', "[data-toggle='rst-current-version']", function() {
13 | $("[data-toggle='rst-versions']").toggleClass("shift-up");
14 | });
15 | // Make tables responsive
16 | $("table.docutils:not(.field-list)").wrap("
pyfpds is a basic python wrapper for accessing federal contracting data in the Federal Procurement Data System (FPDS). The only programmatic access to this data is via an ATOM feed that limits each request to 10 records. This can be quite frustrating if you want more than 10 records! This library will grab up to any number of records requested (the default being 100) and compile them into one data structure. Subsequently, the performance is not exemplary, as much of the processing time is spent in http transport. However, if you have the time, it makes life a bit easier :). Additionally, the ATOM feed does not support sorting. If you want to get complete data sorted by a field, you essentially have to pull down all records and sort them in python.
75 |
This project was created to assist with ETL tasks that are part of the Mirage project. If you have suggestions for features, open an issue, or consider contributing to the project yourself (see CONTRIBUTING).
76 |
Requirements
77 |
This project supports python 2.7+. To install the dependencies, use pip:
78 | pip install -r requirements.txt
The Contracts object can be used to fetch records filtered by several attributes using keyword arguments. The result returned is a list of OrderedDict objects.
88 |
Example:
89 |
from pyfpds import Contracts
90 |
91 | c = Contracts()
92 |
93 | #filter by a specific contract ID number
94 | records = c.get(piid="FA865014M5002")
95 |
96 | r = records[0]['content']['award']
97 |
98 | #pretty print the first record
99 | c.pretty_print(r)
100 |