├── AUTHORS.rst ├── CONTRIBUTING.rst ├── HISTORY.rst ├── LICENSE ├── MANIFEST.in ├── PKG-INFO ├── README.rst ├── docs ├── Makefile ├── make.bat └── source │ ├── appendix.rst │ ├── authors.rst │ ├── conf.py │ ├── contributing.rst │ ├── data.rst │ ├── history.rst │ ├── index.rst │ ├── installation.rst │ ├── readme.rst │ └── usage.rst ├── factorset ├── CONFIG.ini ├── Run │ ├── __init__.py │ └── data_fetch.py ├── Util │ ├── __init__.py │ ├── configutil.py │ ├── finance.py │ └── test.py ├── __init__.py ├── data │ ├── ArcticParser.py │ ├── CSVParser.py │ ├── FundCrawler.py │ ├── FundDict.py │ ├── OtherData.py │ ├── Proxy_start.py │ ├── StockSaver.py │ ├── __init__.py │ └── allAShare.csv └── factors │ ├── Accruals2price.py │ ├── AssetTurnover.py │ ├── Beta.py │ ├── CATurnover.py │ ├── CurrentRatio.py │ ├── EP_LYR.py │ ├── EP_TTM.py │ ├── GPOA.py │ ├── GrossMarginTTM.py │ ├── InterestCover.py │ ├── LDebt2TA.py │ ├── Momentum.py │ ├── NATurnover.py │ ├── QuickRatio.py │ ├── ROIC.py │ ├── RoeGrowth1.py │ ├── RoeGrowth2.py │ ├── TA2TL.py │ ├── UnleverBeta.py │ └── __init__.py ├── requirements_docs.txt ├── setup.cfg ├── setup.py └── tests └── test_factorset.py /AUTHORS.rst: -------------------------------------------------------------------------------- 1 | ======= 2 | Credits 3 | ======= 4 | 5 | Development Lead 6 | ---------------- 7 | 8 | * wingturb 9 | * Code37 <495673131@qq.com> 10 | * jiamicu 11 | * Mutex86 12 | * Andyliwr <541989418@qq.com> 13 | * Notmeor <531356207@qq.com> 14 | 15 | Contributors 16 | ------------ 17 | 18 | None yet. Why not be the first? 19 | -------------------------------------------------------------------------------- /CONTRIBUTING.rst: -------------------------------------------------------------------------------- 1 | .. highlight:: shell 2 | 3 | ============ 4 | Contributing 5 | ============ 6 | 7 | Contributions are welcome, and they are greatly appreciated! Every little bit 8 | helps, and credit will always be given. 9 | 10 | You can contribute in many ways: 11 | 12 | Types of Contributions 13 | ---------------------- 14 | 15 | Report Bugs 16 | ~~~~~~~~~~~ 17 | 18 | Report bugs at https://github.com/quantasset/factorset/issues. 19 | 20 | If you are reporting a bug, please include: 21 | 22 | * Your operating system name and version. 23 | * Any details about your local setup that might be helpful in troubleshooting. 24 | * Detailed steps to reproduce the bug. 25 | 26 | Fix Bugs 27 | ~~~~~~~~ 28 | 29 | Look through the GitHub issues for bugs. Anything tagged with "bug" and "help 30 | wanted" is open to whoever wants to implement it. 31 | 32 | Implement Features 33 | ~~~~~~~~~~~~~~~~~~ 34 | 35 | Look through the GitHub issues for features. Anything tagged with "enhancement" 36 | and "help wanted" is open to whoever wants to implement it. 37 | 38 | Write Documentation 39 | ~~~~~~~~~~~~~~~~~~~ 40 | 41 | factorset could always use more documentation, whether as part of the 42 | official factorset docs, in docstrings, or even on the web in blog posts, 43 | articles, and such. 44 | 45 | Submit Feedback 46 | ~~~~~~~~~~~~~~~ 47 | 48 | The best way to send feedback is to file an issue at https://github.com/quantasset/factorset/issues. 49 | 50 | If you are proposing a feature: 51 | 52 | * Explain in detail how it would work. 53 | * Keep the scope as narrow as possible, to make it easier to implement. 54 | * Remember that this is a volunteer-driven project, and that contributions 55 | are welcome :) 56 | 57 | Get Started! 58 | ------------ 59 | 60 | Ready to contribute? Here's how to set up `factorset` for local development. 61 | 62 | 1. Fork the `factorset` repo on GitHub. 63 | 2. Clone your fork locally:: 64 | 65 | $ git clone git@github.com:your_name_here/factorset.git 66 | 67 | 3. Install your local copy into a virtualenv. Assuming you have virtualenvwrapper installed, this is how you set up your fork for local development:: 68 | 69 | $ mkvirtualenv factorset 70 | $ cd factorset/ 71 | $ python setup.py develop 72 | 73 | 4. Create a branch for local development:: 74 | 75 | $ git checkout -b name-of-your-bugfix-or-feature 76 | 77 | Now you can make your changes locally. 78 | 79 | 5. When you're done making changes, check that your changes pass flake8 and the 80 | tests, including testing other Python versions with tox:: 81 | 82 | $ flake8 factorset tests 83 | $ python setup.py test or py.test 84 | $ tox 85 | 86 | To get flake8 and tox, just pip install them into your virtualenv. 87 | 88 | 6. Commit your changes and push your branch to GitHub:: 89 | 90 | $ git add . 91 | $ git commit -m "Your detailed description of your changes." 92 | $ git push origin name-of-your-bugfix-or-feature 93 | 94 | 7. Submit a pull request through the GitHub website. 95 | 96 | Pull Request Guidelines 97 | ----------------------- 98 | 99 | Before you submit a pull request, check that it meets these guidelines: 100 | 101 | 1. The pull request should include tests. 102 | 2. If the pull request adds functionality, the docs should be updated. Put 103 | your new functionality into a function with a docstring, and add the 104 | feature to the list in README.rst. 105 | 106 | 107 | Tips 108 | ---- 109 | 110 | To run a subset of tests:: 111 | 112 | $ py.test tests.test_factorset 113 | 114 | 115 | Deploying 116 | --------- 117 | 118 | A reminder for the maintainers on how to deploy. 119 | Make sure all your changes are committed (including an entry in HISTORY.rst). 120 | Then run:: 121 | 122 | $ bumpversion patch # possible: major / minor / patch 123 | $ git push 124 | $ git push --tags 125 | 126 | Travis will then deploy to PyPI if tests pass. 127 | -------------------------------------------------------------------------------- /HISTORY.rst: -------------------------------------------------------------------------------- 1 | ======= 2 | History 3 | ======= 4 | 5 | 6 | 0.0.2 (2018-05-04) 7 | ------------------ 8 | 9 | * Add Factors module with 20 sample factors. 10 | * Imporved data collection module. 11 | 12 | 0.0.1 (2018-04-19) 13 | ------------------ 14 | 15 | * First release on PyPI. 16 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2018, Wenchao Zhang 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | 23 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include AUTHORS.rst 2 | include CONTRIBUTING.rst 3 | include HISTORY.rst 4 | include LICENSE 5 | include README.rst 6 | 7 | recursive-include tests * 8 | recursive-include factorset * 9 | recursive-exclude * __pycache__ 10 | recursive-exclude * *.py[co] 11 | 12 | recursive-include docs *.rst conf.py Makefile make.bat *.jpg *.png *.gif 13 | -------------------------------------------------------------------------------- /PKG-INFO: -------------------------------------------------------------------------------- 1 | Metadata-Version: 1.1 2 | Name: factorset 3 | Version: 0.0.2 4 | Summary: A Python package for computing chinese financial factors. 5 | Home-page: https://github.com/quantasset/factorset 6 | Author: Wenchao Zhang 7 | Author-email: 495673131@qq.com 8 | License: MIT license 9 | Description: ========= 10 | factorset 11 | ========= 12 | 13 | |pypi| 14 | |version status| 15 | |travis status| 16 | |Docs| 17 | 18 | Factorset is a financial factors construction factory of Chinese A-share. 19 | 提供中国A股市场因子集合，包含各类常用及特异因子计算方法，持续更新中。 提供轻量级因子计算框架，高可扩展。持续更新中。 20 | 21 | 22 | - `Documentation `_ 23 | - Want to Contribute? See our `Development Guidelines `_ 24 | 25 | Features 26 | ======== 27 | 28 | - **Ease of Use:** factorset tries to get out of your way so that you can 29 | focus on algorithm development. See below for a code example. 30 | - **"Batteries Included":** many common statistics like 31 | moving average and linear regression can be readily accessed from 32 | within a user-written algorithm. 33 | 34 | Installation 35 | ============ 36 | 37 | Installing With ``pip`` 38 | ----------------------- 39 | 40 | Assuming you have all required (see note below) non-Python dependencies, you 41 | can install factorset with ``pip`` via: 42 | 43 | .. code-block:: bash 44 | 45 | $ pip install factorset 46 | 47 | **Note:** Installing factorset via ``pip`` is slightly more involved than the 48 | average Python package. Simply running ``pip install factorset`` will likely 49 | fail if you've never installed any scientific Python packages before. 50 | 51 | The ``FundCrawler`` class in factorset depends on `proxy_pool `_, a powerful proxy crawler. 52 | 53 | 54 | Quickstart 55 | ========== 56 | 57 | The following code implements a simple dual moving average algorithm. 58 | 59 | .. code:: python 60 | 61 | import pandas as pd 62 | import tushare as ts 63 | from factorset.factors import BaseFactor 64 | from factorset.data.OtherData import code_to_symbol, shift_date, market_value 65 | from factorset.data import CSVParser as cp 66 | from factorset.Util.finance import ttmContinues 67 | 68 | class NewFactor(BaseFactor): 69 | """ 70 | NewFactor = Calculate Method 71 | """ 72 | def __init__(self, factor_name='NewFactor', tickers='000016.SH', data_source='', factor_parameters={}, save_dir=None): 73 | # Initialize super class. 74 | super(NewFactor, self).__init__(factor_name=factor_name, tickers=tickers, 75 | factor_parameters=factor_parameters, 76 | data_source=data_source, save_dir=save_dir) 77 | 78 | def prepare_data(self, begin_date, end_date): 79 | shifted_begin_date = shift_date(begin_date, 500) 80 | # motherNetProfit 40 81 | inst = cp.concat_fund(self.data_source, self.tickers, 'IS').loc[shifted_begin_date:end_date,['ticker', 40]] 82 | inst['release_date'] = inst.index 83 | inst['report_date'] = inst.index 84 | # cash_flows_yield 133 85 | cf = cp.concat_fund(self.data_source, self.tickers, 'CF').loc[shifted_begin_date:end_date,['ticker', 133]] 86 | cf['release_date'] = cf.index 87 | cf['report_date'] = cf.index 88 | 89 | self.accrual_df = cf.merge(inst, on=['ticker', 'release_date', 'report_date']) 90 | self.accrual_df['accr'] = self.accrual_df[40] - self.accrual_df[133] 91 | 92 | cash_flow_ls = [] 93 | for ticker in self.accrual_df['ticker'].unique(): 94 | try: # 财务数据不足4条会有异常 95 | reven_df = ttmContinues(self.accrual_df[self.accrual_df['ticker'] == ticker], 'accr') 96 | reven_df['ticker'] = ticker 97 | except: 98 | continue 99 | cash_flow_ls.append(reven_df) 100 | 101 | self.accrual_ttm = pd.concat(cash_flow_ls) 102 | # 总市值 103 | # Tushare的市值数据只有17年-now 104 | df = market_value(self.data_source + '\\other\\otherdata.csv', self.tickers) 105 | self.mkt_value = df.drop(['price', 'totals'], axis=1) 106 | 107 | def generate_factor(self, trading_day): 108 | accr_df = self.accrual_ttm[self.accrual_ttm['datetime'] <= trading_day] 109 | accr_df = accr_df.sort_values(by=['datetime', 'report_date'], ascending=[False, False]) 110 | accr_se = accr_df.groupby('ticker')['accr_TTM'].apply(lambda x: x.iloc[0]) # 取最近1年的财报 111 | 112 | today_mkt_value = self.mkt_value.loc[trading_day] 113 | mkt_value = today_mkt_value.set_index('ticker')['mkt_value'] 114 | ret_se = accr_se / mkt_value 115 | return ret_se.dropna() 116 | 117 | 118 | if __name__ == '__main__': 119 | from_dt = '2017-07-15' 120 | to_dt = '2018-04-09' 121 | 122 | # 取沪深300 123 | hs300 = ts.get_hs300s() 124 | hs300.code = hs300.code.apply(code_to_symbol) 125 | 126 | NewFactor = NewFactor( 127 | factor_name='NewFactor', 128 | factor_parameters={}, 129 | tickers=hs300.code.tolist(), 130 | save_dir='', 131 | data_source='.\data', 132 | ) 133 | 134 | NewFactor.generate_factor_and_store(from_dt, to_dt) 135 | print('因子构建完成，并已成功入库!') 136 | 137 | 138 | You can find other factors in the ``factorset/factors`` directory. 139 | 140 | Questions? 141 | ========== 142 | 143 | If you find a bug, feel free to `open an issue `_ and fill out the issue template. 144 | 145 | Contributing 146 | ============ 147 | 148 | All contributions, bug reports, bug fixes, documentation improvements, enhancements, and ideas are welcome. 149 | 150 | 151 | .. |pypi| image:: https://img.shields.io/pypi/v/factorset.svg 152 | :target: https://pypi.python.org/pypi/factorset 153 | .. |version status| image:: https://img.shields.io/pypi/pyversions/factorset.svg 154 | :target: https://pypi.python.org/pypi/factorset 155 | .. |Docs| image:: https://readthedocs.org/projects/factorset/badge/?version=latest 156 | :target: https://factorset.readthedocs.io/en/latest/?badge=latest 157 | :alt: Documentation Status 158 | .. |travis status| image:: https://travis-ci.org/quantasset/factorset.png?branch=master 159 | :target: https://travis-ci.org/quantasset/factorset 160 | 161 | 162 | 163 | ======= 164 | History 165 | ======= 166 | 167 | 168 | 0.0.2 (2018-05-04) 169 | ------------------ 170 | 171 | * Add Factors module with 20 sample factors. 172 | * Imporved data collection module. 173 | 174 | 0.0.1 (2018-04-19) 175 | ------------------ 176 | 177 | * First release on PyPI. 178 | 179 | Keywords: factorset 180 | Platform: UNKNOWN 181 | Classifier: Development Status :: 2 - Pre-Alpha 182 | Classifier: Intended Audience :: Developers 183 | Classifier: License :: OSI Approved :: MIT License 184 | Classifier: Natural Language :: English 185 | Classifier: Programming Language :: Python :: 3.5 186 | Classifier: Programming Language :: Python :: 3.6 187 | Classifier: Topic :: Scientific/Engineering 188 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | ========= 2 | factorset 3 | ========= 4 | 5 | |pypi| 6 | |version status| 7 | |travis status| 8 | |Docs| 9 | 10 | Factorset is a financial factors construction factory of Chinese A-share. 11 | 提供中国A股市场因子集合，包含各类常用及特异因子计算方法，持续更新中。 提供轻量级因子计算框架，高可扩展。持续更新中。 12 | 13 | - `Documentation `_ 14 | - Want to Contribute? See our `Contributing Guidelines `_ 15 | 16 | Features 17 | ~~~~~~~~ 18 | 19 | - **Ease of Use:** factorset tries to get out of your way so that you can 20 | focus on algorithm development. See below for a code example. 21 | 22 | Installation 23 | ~~~~~~~~~~~~ 24 | 25 | Installing With ``pip`` 26 | ----------------------- 27 | 28 | Assuming you have all required (see note below) non-Python dependencies, you 29 | can install factorset with ``pip`` via: 30 | 31 | .. code-block:: bash 32 | 33 | $ pip install factorset 34 | 35 | **Note:** Installing factorset via ``pip`` is slightly more involved than the 36 | average Python package. Simply running ``pip install factorset`` will likely 37 | fail if you've never installed any scientific Python packages before. 38 | 39 | The ``FundCrawler`` class in factorset depends on `proxy_pool `_, a powerful proxy crawler. 40 | 41 | 42 | Quickstart 43 | ~~~~~~~~~~ 44 | 45 | The following code generate a ``EP_TTM`` factor class. 46 | 47 | .. code-block:: python 48 | 49 | import os 50 | import pandas as pd 51 | import tushare as ts 52 | from factorset.factors import BaseFactor 53 | from factorset.data.OtherData import code_to_symbol, shift_date, market_value 54 | from factorset.data import CSVParser as cp 55 | from factorset.Util.finance import ttmContinues 56 | 57 | class EP_TTM(BaseFactor): 58 | """ 59 | :名称: 过去滚动4个季度（12月）市盈率的倒数 60 | :计算方法: EP_TTM = 净利润（不含少数股东权益）_TTM /总市值 61 | :应用: 市盈率越低，代表投资者能够以相对较低价格购入股票。 62 | 63 | """ 64 | 65 | def __init__(self, factor_name='EP_TTM', tickers='000016.SH', data_source='', factor_parameters={}, save_dir=None): 66 | # Initialize super class. 67 | super(EP_TTM, self).__init__(factor_name=factor_name, tickers=tickers, 68 | factor_parameters=factor_parameters, 69 | data_source=data_source, save_dir=save_dir) 70 | 71 | def prepare_data(self, begin_date, end_date): 72 | """ 73 | 数据预处理 74 | """ 75 | 76 | shifted_begin_date = shift_date(begin_date, 500) # 向前取500个交易日 77 | 78 | # 取利润表中“归属于母公司股东的净利润”项目，项目名称及数字详见FundDict 79 | inst = cp.concat_fund(self.data_source, self.tickers, 'IS').loc[shifted_begin_date:end_date,['ticker', 40]] 80 | inst['motherNetProfit'] = inst[40] 81 | inst.drop(40, axis=1, inplace=True) 82 | 83 | # ttm算法需要“财报发布日”与“财报报告日”两个日期作为参数 84 | inst['release_date'] = inst.index 85 | inst['report_date'] = inst.index 86 | 87 | # 净利润ttm 88 | profitTTM_ls = [] 89 | for ticker in inst['ticker'].unique(): 90 | try: # 财务数据不足4条会有异常 91 | reven_df = ttmContinues(inst[inst['ticker'] == ticker], 'motherNetProfit') 92 | reven_df['ticker'] = ticker 93 | except: 94 | continue 95 | profitTTM_ls.append(reven_df) 96 | self.profitTTM = pd.concat(profitTTM_ls) 97 | 98 | # 取“OtherData”中总市值数据 99 | # Tushare的市值数据只有17年6月->now 100 | df = market_value(self.data_source + '\\other\\otherdata.csv', self.tickers) 101 | self.mkt_value = df.drop(['price', 'totals'], axis=1) 102 | 103 | def generate_factor(self, trading_day): 104 | # generate_factor会遍历交易日区间, 即生成每个交易日所有股票的因子值 105 | earings_df = self.profitTTM[self.profitTTM['datetime'] <= trading_day] 106 | earings_df = earings_df.sort_values(by=['datetime', 'report_date'], ascending=[False, False]) 107 | 108 | # 取最近1期ttm数据 109 | earings_df = earings_df.groupby('ticker').apply(lambda x: x.head(1)) 110 | 111 | # 取当前交易日市值数据 112 | today_mkt_value = self.mkt_value.loc[trading_day] 113 | 114 | ret_df = earings_df.merge(today_mkt_value, on='ticker', how='inner') 115 | ret_df['EP_TTM'] = ret_df['motherNetProfit_TTM'] / ret_df['mkt_value'] 116 | return ret_df.set_index('ticker')['EP_TTM'] 117 | 118 | if __name__ == '__main__': 119 | from_dt = '2017-07-15' 120 | to_dt = '2018-04-09' 121 | 122 | # 取沪深300 123 | hs300 = ts.get_hs300s() 124 | hs300.code = hs300.code.apply(code_to_symbol) 125 | 126 | EP_TTM = EP_TTM( 127 | factor_name='EP_TTM', 128 | factor_parameters={}, 129 | tickers=hs300.code.tolist(), 130 | save_dir='', 131 | data_source=os.path.abspath('.'), 132 | ) 133 | 134 | EP_TTM.generate_factor_and_store(from_dt, to_dt) 135 | print('因子构建完成，并已成功入库!') 136 | 137 | You can find other factors in the ``factorset/factors`` directory. 138 | 139 | Questions? 140 | ~~~~~~~~~~ 141 | 142 | If you find a bug, feel free to `open an issue `_ and fill out the issue template. 143 | 144 | Contributing 145 | ~~~~~~~~~~~~ 146 | 147 | All contributions, bug reports, bug fixes, documentation improvements, enhancements, and ideas are welcome. 148 | 149 | 150 | .. |pypi| image:: https://img.shields.io/pypi/v/factorset.svg 151 | :target: https://pypi.python.org/pypi/factorset 152 | .. |version status| image:: https://img.shields.io/pypi/pyversions/factorset.svg 153 | :target: https://pypi.python.org/pypi/factorset 154 | .. |Docs| image:: https://readthedocs.org/projects/factorset/badge/?version=latest 155 | :target: https://factorset.readthedocs.io/en/latest/?badge=latest 156 | :alt: Documentation Status 157 | .. |travis status| image:: https://travis-ci.org/quantasset/factorset.png?branch=master 158 | :target: https://travis-ci.org/quantasset/factorset 159 | 160 | -------------------------------------------------------------------------------- /docs/Makefile: -------------------------------------------------------------------------------- 1 | # Minimal makefile for Sphinx documentation 2 | # 3 | 4 | # You can set these variables from the command line. 5 | SPHINXOPTS = 6 | SPHINXBUILD = sphinx-build 7 | SPHINXPROJ = factorset 8 | SOURCEDIR = source 9 | BUILDDIR = build 10 | 11 | # Put it first so that "make" without argument is like "make help". 12 | help: 13 | @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) 14 | 15 | .PHONY: help Makefile 16 | 17 | # Catch-all target: route all unknown targets to Sphinx using the new 18 | # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). 19 | %: Makefile 20 | @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) -------------------------------------------------------------------------------- /docs/make.bat: -------------------------------------------------------------------------------- 1 | @ECHO OFF 2 | 3 | pushd %~dp0 4 | 5 | REM Command file for Sphinx documentation 6 | 7 | if "%SPHINXBUILD%" == "" ( 8 | set SPHINXBUILD=sphinx-build 9 | ) 10 | set SOURCEDIR=source 11 | set BUILDDIR=build 12 | set SPHINXPROJ=factorset 13 | 14 | if "%1" == "" goto help 15 | 16 | %SPHINXBUILD% >NUL 2>NUL 17 | if errorlevel 9009 ( 18 | echo. 19 | echo.The 'sphinx-build' command was not found. Make sure you have Sphinx 20 | echo.installed, then set the SPHINXBUILD environment variable to point 21 | echo.to the full path of the 'sphinx-build' executable. Alternatively you 22 | echo.may add the Sphinx directory to PATH. 23 | echo. 24 | echo.If you don't have Sphinx installed, grab it from 25 | echo.http://sphinx-doc.org/ 26 | exit /b 1 27 | ) 28 | 29 | %SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% 30 | goto end 31 | 32 | :help 33 | %SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% 34 | 35 | :end 36 | popd 37 | -------------------------------------------------------------------------------- /docs/source/appendix.rst: -------------------------------------------------------------------------------- 1 | API Reference 2 | ------------- 3 | 4 | Data Fetchers 5 | ~~~~~~~~~~~~~ 6 | 7 | Stock Data 8 | `````````` 9 | .. autofunction:: factorset.Run.data_fetch.data_fetch 10 | 11 | .. autofunction:: factorset.data.StockSaver.write_all_stock 12 | 13 | .. autofunction:: factorset.data.StockSaver.save_index 14 | 15 | Other Data 16 | `````````` 17 | .. autofunction:: factorset.data.OtherData.market_value 18 | 19 | .. autofunction:: factorset.data.OtherData.tradecal 20 | 21 | Fundamental Data 22 | ```````````````` 23 | .. autoclass:: factorset.data.FundCrawler.FundCrawler 24 | :members: 25 | :undoc-members: 26 | 27 | Data Reader 28 | ~~~~~~~~~~~ 29 | The following methods are available for use in the ``prepare_data`` (recommended), ``generate_factor`` API functions. 30 | 31 | Stock Data 32 | `````````` 33 | .. autofunction:: factorset.data.CSVParser.all_stock_symbol 34 | 35 | .. autofunction:: factorset.data.CSVParser.read_stock 36 | 37 | .. autofunction:: factorset.data.CSVParser.concat_stock 38 | 39 | .. autofunction:: factorset.data.CSVParser.concat_all_stock 40 | 41 | .. autofunction:: factorset.data.CSVParser.hconcat_stock_series 42 | 43 | 44 | Other Data 45 | `````````` 46 | .. autofunction:: factorset.data.OtherData.write_new_stocks 47 | 48 | .. autofunction:: factorset.data.OtherData.write_all_date 49 | 50 | Fundamental Data 51 | ```````````````` 52 | .. autofunction:: factorset.data.CSVParser.all_fund_symbol 53 | 54 | .. autofunction:: factorset.data.CSVParser.read_fund 55 | 56 | .. autofunction:: factorset.data.CSVParser.fund_collist 57 | 58 | .. autofunction:: factorset.data.CSVParser.concat_fund 59 | 60 | Data Util 61 | ~~~~~~~~~ 62 | .. autofunction:: factorset.data.OtherData.code_to_symbol 63 | 64 | .. autofunction:: factorset.data.OtherData.shift_date 65 | 66 | .. autofunction:: factorset.Util.finance.ttmContinues 67 | 68 | .. autofunction:: factorset.Util.finance.ttmDiscrete 69 | 70 | 71 | 72 | 73 | -------------------------------------------------------------------------------- /docs/source/authors.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../../AUTHORS.rst 2 | -------------------------------------------------------------------------------- /docs/source/conf.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # 3 | # Configuration file for the Sphinx documentation builder. 4 | # 5 | # This file does only contain a selection of the most common options. For a 6 | # full list see the documentation: 7 | # http://www.sphinx-doc.org/en/master/config 8 | 9 | # -- Path setup -------------------------------------------------------------- 10 | 11 | # If extensions (or modules to document with autodoc) are in another directory, 12 | # add these directories to sys.path here. If the directory is relative to the 13 | # documentation root, use os.path.abspath to make it absolute, like shown here. 14 | # 15 | import os 16 | import sys 17 | 18 | # sys.path.append(os.path.join(os.path.dirname(__name__), '..')) 19 | # sys.path.insert(0, os.path.abspath('../..')) 20 | sys.path.insert(0, os.path.abspath('..').strip('docs')) 21 | # sys.path.insert(0, os.path.abspath('./..')) 22 | 23 | import factorset 24 | 25 | # -- Project information ----------------------------------------------------- 26 | 27 | project = 'factorset' 28 | copyright = '2018, Wenchao Zhang' 29 | author = 'Wenchao Zhang' 30 | 31 | # The short X.Y version 32 | version = factorset.__version__ 33 | # The full version, including alpha/beta/rc tags 34 | release = factorset.__version__ 35 | 36 | 37 | # -- General configuration --------------------------------------------------- 38 | 39 | # If your documentation needs a minimal Sphinx version, state it here. 40 | # 41 | # needs_sphinx = '1.0' 42 | 43 | # Add any Sphinx extension module names here, as strings. They can be 44 | # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom 45 | # ones. 46 | extensions = [ 47 | 'sphinx.ext.autodoc', 48 | 'sphinx.ext.doctest', 49 | 'sphinx.ext.todo', 50 | 'sphinx.ext.coverage', 51 | 'sphinx.ext.mathjax', 52 | 'sphinx.ext.ifconfig', 53 | 'sphinx.ext.viewcode', 54 | ] 55 | autodoc_mock_imports = ['aiohttp','async_timeout','tushare', 'tushare.stock'] 56 | 57 | # -- Docstrings 58 | extensions += ['sphinx.ext.napoleon'] 59 | numpydoc_show_class_members = False 60 | 61 | # Add any paths that contain templates here, relative to this directory. 62 | templates_path = ['_templates'] 63 | 64 | # The suffix(es) of source filenames. 65 | # You can specify multiple suffix as a list of string: 66 | # 67 | # source_suffix = ['.rst', '.md'] 68 | source_suffix = '.rst' 69 | 70 | # The master toctree document. 71 | master_doc = 'index' 72 | 73 | # The language for content autogenerated by Sphinx. Refer to documentation 74 | # for a list of supported languages. 75 | # 76 | # This is also used if you do content translation via gettext catalogs. 77 | # Usually you set "language" from the command line for these cases. 78 | language = 'zh_CN' 79 | 80 | # List of patterns, relative to source directory, that match files and 81 | # directories to ignore when looking for source files. 82 | # This pattern also affects html_static_path and html_extra_path . 83 | exclude_patterns = [] 84 | 85 | # The theme to use for HTML and HTML Help pages. See the documentation for 86 | # a list of builtin themes. 87 | # Use the Zipline's solution 88 | on_rtd = os.environ.get('READTHEDOCS') == 'True' 89 | if not on_rtd: # only import and set the theme if we're building docs locally 90 | try: 91 | import sphinx_rtd_theme 92 | except ImportError: 93 | html_theme = 'default' 94 | html_theme_path = [] 95 | else: 96 | html_theme = 'sphinx_rtd_theme' 97 | html_theme_path = [sphinx_rtd_theme.get_html_theme_path()] 98 | 99 | # html_theme = 'sphinx_rtd_theme' 100 | 101 | # The name of the Pygments (syntax highlighting) style to use. 102 | # highlight_language = 'python' 103 | pygments_style = 'default' 104 | # pygments_style = 'sphinx' 105 | 106 | 107 | # -- Options for HTML output ------------------------------------------------- 108 | 109 | # source_encoding = 'utf-8' 110 | # Theme options are theme-specific and customize the look and feel of a theme 111 | # further. For a list of options available for each theme, see the 112 | # documentation. 113 | # 114 | # html_theme_options = {} 115 | 116 | # Add any paths that contain custom static files (such as style sheets) here, 117 | # relative to this directory. They are copied after the builtin static files, 118 | # so a file named "default.css" will overwrite the builtin "default.css". 119 | html_static_path = ['_static'] 120 | 121 | # Custom sidebar templates, must be a dictionary that maps document names 122 | # to template names. 123 | # 124 | # The default sidebars (for documents that don't match any pattern) are 125 | # defined by theme itself. Builtin themes are using these templates by 126 | # default: ``['localtoc.html', 'relations.html', 'sourcelink.html', 127 | # 'searchbox.html']``. 128 | # 129 | # html_sidebars = {} 130 | 131 | 132 | # -- Options for HTMLHelp output --------------------------------------------- 133 | 134 | # Output file base name for HTML help builder. 135 | htmlhelp_basename = 'factorsetdoc' 136 | 137 | 138 | # -- Options for LaTeX output ------------------------------------------------ 139 | if on_rtd: 140 | latex_elements = { 141 | # The paper size ('letterpaper' or 'a4paper'). 142 | #'papersize': 'letterpaper', 143 | # The font size ('10pt', '11pt' or '12pt'). 144 | #'pointsize': '10pt', 145 | # Additional stuff for the LaTeX preamble. 146 | 'preamble': r''' 147 | \hypersetup{unicode=true} 148 | \usepackage{CJKutf8} 149 | \DeclareUnicodeCharacter{00A0}{\nobreakspace} 150 | \DeclareUnicodeCharacter{2203}{\ensuremath{\exists}} 151 | \DeclareUnicodeCharacter{2200}{\ensuremath{\forall}} 152 | \DeclareUnicodeCharacter{2286}{\ensuremath{\subseteq}} 153 | \DeclareUnicodeCharacter{2713}{x} 154 | \DeclareUnicodeCharacter{27FA}{\ensuremath{\Longleftrightarrow}} 155 | \DeclareUnicodeCharacter{221A}{\ensuremath{\sqrt{}}} 156 | \DeclareUnicodeCharacter{221B}{\ensuremath{\sqrt[3]{}}} 157 | \DeclareUnicodeCharacter{2295}{\ensuremath{\oplus}} 158 | \DeclareUnicodeCharacter{2297}{\ensuremath{\otimes}} 159 | \begin{CJK}{UTF8}{gbsn} 160 | \AtEndDocument{\end{CJK}} 161 | ''', 162 | } 163 | else: 164 | latex_engine = 'xelatex' 165 | latex_elements = { 166 | # The paper size ('letterpaper' or 'a4paper'). 167 | # 168 | # 'papersize': 'letterpaper', 169 | 170 | # The font size ('10pt', '11pt' or '12pt'). 171 | # 172 | # 'pointsize': '10pt', 173 | 174 | 'babel': r'\usepackage[english]{babel}', # not supporting Chinese 175 | '' 176 | 'inputenc':'', # conflict with xeCJK 177 | 'utf8extra':'', # using inputenc 178 | 179 | # Additional stuff for the LaTeX preamble. 180 | 'preamble': r''' 181 | \usepackage{xeCJK} 182 | \usepackage{indentfirst} 183 | ''', 184 | # Latex figure (float) alignment 185 | # 186 | # 'figure_align': 'htbp', 187 | } 188 | 189 | # Grouping the document tree into LaTeX files. List of tuples 190 | # (source start file, target name, title, 191 | # author, documentclass [howto, manual, or own class]). 192 | latex_documents = [ 193 | (master_doc, 'factorset.tex', 'factorset Documentation', 194 | 'Wenchao Zhang', 'manual'), 195 | ] 196 | 197 | 198 | # -- Options for manual page output ------------------------------------------ 199 | 200 | # One entry per manual page. List of tuples 201 | # (source start file, name, description, authors, manual section). 202 | man_pages = [ 203 | (master_doc, 'factorset', 'factorset Documentation', 204 | [author], 1) 205 | ] 206 | 207 | 208 | # -- Options for Texinfo output ---------------------------------------------- 209 | 210 | # Grouping the document tree into Texinfo files. List of tuples 211 | # (source start file, target name, title, author, 212 | # dir menu entry, description, category) 213 | texinfo_documents = [ 214 | (master_doc, 'factorset', 'factorset Documentation', 215 | author, 'factorset', 'One line description of project.', 216 | 'Miscellaneous'), 217 | ] 218 | 219 | 220 | # -- Extension configuration ------------------------------------------------- 221 | 222 | # -- Options for todo extension ---------------------------------------------- 223 | 224 | # If true, `todo` and `todoList` produce output, else they produce nothing. 225 | todo_include_todos = True 226 | -------------------------------------------------------------------------------- /docs/source/contributing.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../../CONTRIBUTING.rst 2 | -------------------------------------------------------------------------------- /docs/source/history.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../../HISTORY.rst 2 | -------------------------------------------------------------------------------- /docs/source/index.rst: -------------------------------------------------------------------------------- 1 | .. factorset documentation master file, created by 2 | sphinx-quickstart on Mon May 14 15:05:37 2018. 3 | You can adapt this file completely to your liking, but it should at least 4 | contain the root `toctree` directive. 5 | 6 | .. include:: ../../README.rst 7 | 8 | .. toctree:: 9 | :maxdepth: 3 10 | :numbered: 2 11 | :caption: DOCUMENTATION INDEX 12 | 13 | installation 14 | data 15 | usage 16 | appendix 17 | contributing 18 | authors 19 | history 20 | 21 | Indices and tables 22 | ================== 23 | 24 | * :ref:`genindex` 25 | * :ref:`modindex` 26 | * :ref:`search` 27 | -------------------------------------------------------------------------------- /docs/source/installation.rst: -------------------------------------------------------------------------------- 1 | .. highlight:: shell 2 | 3 | ============ 4 | Installation 5 | ============ 6 | 7 | 8 | Stable release 9 | -------------- 10 | 11 | To install factorset, run this command in your terminal: 12 | 13 | .. code-block:: console 14 | 15 | $ pip install factorset 16 | 17 | This is the preferred method to install factorset, as it will always install the most recent stable release. 18 | 19 | If you don't have `pip`_ installed, this `Python installation guide`_ can guide 20 | you through the process. 21 | 22 | .. _pip: https://pip.pypa.io 23 | .. _Python installation guide: http://docs.python-guide.org/en/latest/starting/installation/ 24 | 25 | 26 | From sources 27 | ------------ 28 | 29 | The sources for factorset can be downloaded from the `Github repo`_. 30 | 31 | You can either clone the public repository: 32 | 33 | .. code-block:: console 34 | 35 | $ git clone git://github.com/quantasset/factorset 36 | 37 | Or download the `tarball`_: 38 | 39 | .. code-block:: console 40 | 41 | $ curl -OL https://github.com/quantasset/factorset/tarball/master 42 | 43 | Once you have a copy of the source, you can install it with: 44 | 45 | .. code-block:: console 46 | 47 | $ python setup.py install 48 | 49 | 50 | .. _Github repo: https://github.com/quantasset/factorset 51 | .. _tarball: https://github.com/quantasset/factorset/tarball/master 52 | -------------------------------------------------------------------------------- /docs/source/readme.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../../README.rst 2 | -------------------------------------------------------------------------------- /docs/source/usage.rst: -------------------------------------------------------------------------------- 1 | =============== 2 | Factorset Usage 3 | =============== 4 | 5 | Module contents 6 | ~~~~~~~~~~~~~~~ 7 | In all listed functions, the ``self`` argument is implicitly the 8 | currently-executing :class:`~factorset.factors.BaseFactor` instance. 9 | 10 | .. autoclass:: factorset.factors.BaseFactor 11 | :members: 12 | :undoc-members: 13 | 14 | Generate Factors 15 | ~~~~~~~~~~~~~~~~ 16 | To use factorset in a project, we need to inherit the :class:`~factorset.factors.BaseFactor` for the New Factor: 17 | 18 | .. code-block:: python 19 | 20 | import os 21 | import pandas as pd 22 | import tushare as ts 23 | from factorset.factors import BaseFactor 24 | from factorset.data.OtherData import code_to_symbol, shift_date, market_value 25 | from factorset.data import CSVParser as cp 26 | from factorset.Util.finance import ttmContinues 27 | 28 | class NewFactor(BaseFactor): 29 | """ 30 | :Name: NewFactor 31 | :Cal_Alg: NewFactor = blablabla 32 | :App: NewFactor can never blow up your account! 33 | 34 | """ 35 | 36 | def __init__(self, factor_name='NewFactor', tickers='000016.SH', data_source='', factor_parameters={}, save_dir=None): 37 | # Initialize super class. 38 | super(NewFactor, self).__init__(factor_name=factor_name, tickers=tickers, 39 | factor_parameters=factor_parameters, 40 | data_source=data_source, save_dir=save_dir) 41 | 42 | def prepare_data(self, begin_date, end_date): 43 | 44 | self.data = cp.choose_a_dataset() 45 | 46 | def generate_factor(self, trading_day): 47 | 48 | factor_at_trading_day = awesome_func(self.data) 49 | 50 | return factor_at_trading_day 51 | 52 | if __name__ == '__main__': 53 | from_dt = '2017-07-15' 54 | to_dt = '2018-04-09' 55 | 56 | # 取沪深300 57 | hs300 = ts.get_hs300s() 58 | hs300.code = hs300.code.apply(code_to_symbol) 59 | 60 | NewFactor = NewFactor( 61 | factor_name='NewFactor', 62 | factor_parameters={}, 63 | tickers=hs300.code.tolist(), 64 | save_dir='', 65 | data_source=os.path.abspath('.'), 66 | ) 67 | 68 | NewFactor.generate_factor_and_store(from_dt, to_dt) 69 | print('因子构建完成，并已成功入库!') 70 | 71 | 72 | .. note:: The ``data_source`` can be designated when using Arctic or MongoDB. 73 | 74 | Factors Set 75 | ~~~~~~~~~~~ 76 | 77 | Accruals2price 78 | -------------- 79 | 80 | .. autoclass:: factorset.factors.Accruals2price.Accruals2price 81 | :members: 82 | :undoc-members: 83 | :show-inheritance: 84 | 85 | AssetTurnover 86 | ------------- 87 | 88 | .. autoclass:: factorset.factors.AssetTurnover.AssetTurnover 89 | :members: 90 | :undoc-members: 91 | :show-inheritance: 92 | 93 | Beta 94 | ---- 95 | 96 | .. autoclass:: factorset.factors.Beta.Beta 97 | :members: 98 | :undoc-members: 99 | :show-inheritance: 100 | 101 | CATurnover 102 | ---------- 103 | 104 | .. autoclass:: factorset.factors.CATurnover.CATurnover 105 | :members: 106 | :undoc-members: 107 | :show-inheritance: 108 | 109 | CurrentRatio 110 | ------------ 111 | 112 | .. autoclass:: factorset.factors.CurrentRatio.CurrentRatio 113 | :members: 114 | :undoc-members: 115 | :show-inheritance: 116 | 117 | EP\_LYR 118 | ------- 119 | 120 | .. autoclass:: factorset.factors.EP_LYR.EP_LYR 121 | :members: 122 | :undoc-members: 123 | :show-inheritance: 124 | 125 | EP\_TTM 126 | ------- 127 | 128 | .. autoclass:: factorset.factors.EP_TTM.EP_TTM 129 | :members: 130 | :undoc-members: 131 | :show-inheritance: 132 | 133 | GPOA 134 | ---- 135 | 136 | .. autoclass:: factorset.factors.GPOA.GPOA 137 | :members: 138 | :undoc-members: 139 | :show-inheritance: 140 | 141 | GrossMarginTTM 142 | -------------- 143 | 144 | .. autoclass:: factorset.factors.GrossMarginTTM.GrossMarginTTM 145 | :members: 146 | :undoc-members: 147 | :show-inheritance: 148 | 149 | InterestCover 150 | ------------- 151 | 152 | .. autoclass:: factorset.factors.InterestCover.InterestCover 153 | :members: 154 | :undoc-members: 155 | :show-inheritance: 156 | 157 | LDebt2TA 158 | -------- 159 | 160 | .. autoclass:: factorset.factors.LDebt2TA.LDebt2TA 161 | :members: 162 | :undoc-members: 163 | :show-inheritance: 164 | 165 | Momentum 166 | -------- 167 | 168 | .. autoclass:: factorset.factors.Momentum.Momentum 169 | :members: 170 | :undoc-members: 171 | :show-inheritance: 172 | 173 | NATurnover 174 | ---------- 175 | 176 | .. autoclass:: factorset.factors.NATurnover.NATurnover 177 | :members: 178 | :undoc-members: 179 | :show-inheritance: 180 | 181 | QuickRatio 182 | ---------- 183 | 184 | .. autoclass:: factorset.factors.QuickRatio.QuickRatio 185 | :members: 186 | :undoc-members: 187 | :show-inheritance: 188 | 189 | ROIC 190 | ---- 191 | 192 | .. autoclass:: factorset.factors.ROIC.ROIC 193 | :members: 194 | :undoc-members: 195 | :show-inheritance: 196 | 197 | RoeGrowth1 198 | ---------- 199 | 200 | .. autoclass:: factorset.factors.RoeGrowth1.RoeGrowth1 201 | :members: 202 | :undoc-members: 203 | :show-inheritance: 204 | 205 | RoeGrowth2 206 | ---------- 207 | 208 | .. autoclass:: factorset.factors.RoeGrowth2.RoeGrowth2 209 | :members: 210 | :undoc-members: 211 | :show-inheritance: 212 | 213 | TA2TL 214 | ----- 215 | 216 | .. autoclass:: factorset.factors.TA2TL.TA2TL 217 | :members: 218 | :undoc-members: 219 | :show-inheritance: 220 | 221 | UnleverBeta 222 | ----------- 223 | 224 | .. autoclass:: factorset.factors.UnleverBeta.UnleverBeta 225 | :members: 226 | :undoc-members: 227 | :show-inheritance: 228 | 229 | -------------------------------------------------------------------------------- /factorset/CONFIG.ini: -------------------------------------------------------------------------------- 1 | # -*- coding:utf-8 -*- 2 | [TARGET] 3 | ;all = all 4 | all = 000001.SZ, 000002.SZ 5 | ;all = hs300 6 | 7 | [OPTIONS] 8 | MONGO = False 9 | CSV = True 10 | ;succ and fail list of symbol 11 | SFL = True 12 | 13 | [STORE] 14 | hqdir = ./hq 15 | ;Only fund crawler will use proxy_pool 16 | funddir = ./fund 17 | otherdir = ./other 18 | 19 | [ARCTIC] 20 | ;host = 127.0.0.1:27017 21 | host = localhost 22 | 23 | [READ] 24 | proxypool = 192.168.0.247:5010 25 | ;proxy_pool testing ip, only for test! 26 | ;https://github.com/jhao104/proxy_pool 27 | ;proxypool = 123.207.35.36:5010 28 | ;proxypool = 127.0.0.1:5010 29 | proxymin = 5 30 | encode = gbk 31 | 32 | -------------------------------------------------------------------------------- /factorset/Run/__init__.py: -------------------------------------------------------------------------------- 1 | # -*- coding:utf-8 -*- 2 | """ 3 | @author:code37 4 | @file:__init__.py 5 | @time:2018/5/79:13 6 | """ -------------------------------------------------------------------------------- /factorset/Run/data_fetch.py: -------------------------------------------------------------------------------- 1 | # -*- coding:utf-8 -*- 2 | """ 3 | @author:code37 4 | @file:data_fetch.py 5 | @time:2018/5/79:13 6 | """ 7 | 8 | from factorset.data import StockSaver, OtherData 9 | from factorset.data.FundCrawler import FundCrawler 10 | from factorset.data.OtherData import code_to_symbol 11 | from factorset.Util.configutil import GetConfig 12 | from factorset import data 13 | import pandas as pd 14 | import tushare as ts 15 | import time 16 | import requests 17 | 18 | def get_proxy(url): 19 | return requests.get("http://{}/get/".format(url)).content 20 | 21 | def get_all_proxy(url): 22 | return requests.get("http://{}/get_all/".format(url)).json() 23 | 24 | def data_fetch(): 25 | """ 26 | 从config中读取配置，爬取行情，基本面，及其他数据。 27 | """ 28 | gc = GetConfig() 29 | if gc.target == 'all': 30 | target = pd.read_csv(data.__file__.strip(data.__file__.split('\\')[-1])+'allAShare.csv') 31 | target = target['0'] 32 | elif gc.target == 'hs300': 33 | hs300 = ts.get_hs300s() 34 | hs300.code = hs300.code.apply(code_to_symbol) 35 | target = hs300.code.tolist() 36 | else: 37 | if isinstance(gc.target, str): 38 | target = gc.target.split(', ') 39 | assert isinstance(target, list) 40 | 41 | # Arctic Start 42 | if gc.MONGO: 43 | from arctic import Arctic 44 | a = Arctic(gc.ahost) 45 | a.initialize_library('Ashare') 46 | lib_stock = a['Ashare'] 47 | else: 48 | lib_stock = None 49 | 50 | # Stock & index 51 | print("Start Fetching Stock & Index Data!") 52 | StockSaver.write_all_stock(target, lib_stock) 53 | try: 54 | StockSaver.save_index('000905') 55 | time.sleep(0.1) 56 | StockSaver.save_index('000300') 57 | except IOError as e: 58 | print(e) 59 | print("Finish Fetching Stock & Index Data!") 60 | 61 | # Other data 62 | print("Start Fetching Other Data!") 63 | OtherData.write_all_date(OtherData.tradecal()) 64 | OtherData.write_new_stocks() 65 | print("Finish Fetching Other Data!") 66 | 67 | # Fundamental data 68 | while 1: 69 | print("Start Fetching Fundamental Data!") 70 | if len(get_all_proxy(gc.proxypool)) >= gc.proxymin: 71 | a = FundCrawler('BS') 72 | a.main(target, num=5) 73 | b = FundCrawler('IS') 74 | b.main(target, num=5) 75 | c = FundCrawler('CF') 76 | c.main(target, num=5) 77 | print("Finish Fetching Fundamental Data!") 78 | 79 | break 80 | else: 81 | print("Proxy pool is not ready! We only have {} proxies!".format(len(get_all_proxy(gc.proxypool)))) 82 | time.sleep(5) 83 | 84 | 85 | 86 | -------------------------------------------------------------------------------- /factorset/Util/__init__.py: -------------------------------------------------------------------------------- 1 | # -*- coding:utf-8 -*- 2 | """ 3 | @author:code37 4 | @file:__init__.py 5 | @time:2018/4/2116:35 6 | """ -------------------------------------------------------------------------------- /factorset/Util/configutil.py: -------------------------------------------------------------------------------- 1 | # -*- coding:utf-8 -*- 2 | """ 3 | @author:code37 4 | @file:configutil.py 5 | @time:2018/5/79:37 6 | """ 7 | import os 8 | from configparser import ConfigParser 9 | 10 | # 参考 proxy_pool.Util.GetConfig 11 | 12 | class ConfigParse(ConfigParser): 13 | """ 14 | rewrite ConfigParser, for support upper option 15 | """ 16 | 17 | def __init__(self): 18 | ConfigParser.__init__(self) 19 | 20 | def optionxform(self, optionstr): 21 | return optionstr 22 | 23 | class LazyProperty(object): 24 | """ 25 | LazyProperty 26 | https://blog.csdn.net/handsomekang/article/details/39933553 27 | 参考 proxy_pool.Util.GetConfig 28 | """ 29 | 30 | def __init__(self, func): 31 | self.func = func 32 | 33 | def __get__(self, instance, owner): 34 | if instance is None: 35 | return self 36 | else: 37 | value = self.func(instance) 38 | setattr(instance, self.func.__name__, value) 39 | return value 40 | 41 | class GetConfig(object): 42 | """ 43 | To get config from config.ini 44 | 参考 proxy_pool.Util.GetConfig 45 | """ 46 | 47 | def __init__(self): 48 | self.pwd = os.path.split(os.path.realpath(__file__))[0] 49 | self.config_path = os.path.join(os.path.split(self.pwd)[0], 'CONFIG.ini') 50 | self.config_file = ConfigParse() 51 | self.config_file.read(self.config_path) 52 | 53 | @LazyProperty 54 | def target(self): 55 | return self.config_file.get('TARGET', 'all') 56 | 57 | @LazyProperty 58 | def hq_dir(self): 59 | return self.config_file.get('STORE', 'hqdir') 60 | 61 | @LazyProperty 62 | def fund_dir(self): 63 | return self.config_file.get('STORE', 'funddir') 64 | 65 | @LazyProperty 66 | def other_dir(self): 67 | return self.config_file.get('STORE', 'otherdir') 68 | 69 | @LazyProperty 70 | def ahost(self): 71 | return self.config_file.get('ARCTIC', 'host') 72 | 73 | @LazyProperty 74 | def MONGO(self): 75 | return self.config_file.getboolean('OPTIONS', 'MONGO') 76 | 77 | @LazyProperty 78 | def CSV(self): 79 | return self.config_file.getboolean('OPTIONS', 'CSV') 80 | 81 | @LazyProperty 82 | def SFL(self): 83 | return self.config_file.getboolean('OPTIONS', 'SFL') 84 | 85 | @LazyProperty 86 | def options(self): 87 | return self.config_file.options('OPTIONS') 88 | 89 | @LazyProperty 90 | def proxymin(self): 91 | return self.config_file.getint('READ','proxymin') 92 | 93 | @LazyProperty 94 | def encode(self): 95 | return self.config_file.get('READ','encode') 96 | 97 | @LazyProperty 98 | def proxypool(self): 99 | return self.config_file.get('READ', 'proxypool') 100 | 101 | if __name__ == '__main__': 102 | gc = GetConfig() 103 | print(gc.target) 104 | print(gc.hq_dir) 105 | print(gc.fund_dir) 106 | print(gc.other_dir) 107 | print(gc.ahost) 108 | print(gc.options) 109 | print(gc.encode) 110 | print(gc.proxymin) 111 | print(gc.MONGO) 112 | print(gc.CSV) 113 | print(gc.SFL) 114 | print(gc.proxypool) 115 | -------------------------------------------------------------------------------- /factorset/Util/finance.py: -------------------------------------------------------------------------------- 1 | # -*- coding:utf-8 -*- 2 | """ 3 | @author:code37 4 | @file:finance.py 5 | @time:2018/4/279:46 6 | """ 7 | import pandas as pd 8 | from datetime import datetime 9 | 10 | def ttmContinues(report_df, label): 11 | """ 12 | Compute Trailing Twelve Months for multiple indicator. 13 | 14 | computation rules: 15 | #. ttm indicator is computed on announcement date. 16 | #. on given release_date, use the latest report_date and the previous report year for computation. 17 | #. if any report period is missing, use weighted method. 18 | #. if two reports (usually first-quoter and annual) are released together, only keep latest 19 | 20 | :param report_df: must have 'report_date', 'release_date', and