├── .gitignore ├── .gitmodules ├── .readthedocs.yml ├── CONTRIBUTING.md ├── README.md ├── docs ├── ACM.pdf ├── LPN_AI_symposium_handbook_poster.pdf ├── Makefile ├── TensorLayer ACM MM.pdf ├── TensorLayer_London_PhD_Network_20180212.pdf ├── _static │ └── fix_rtd.css ├── conf.py ├── images │ ├── RL_Group_QR.jpeg │ ├── donghao.jpeg │ ├── moyuanhan.jpeg │ ├── yangguang.jpeg │ └── yusimiao.jpeg ├── index.rst ├── make.bat ├── modules │ ├── activation.rst │ ├── array_ops.rst │ ├── cli.rst │ ├── cost.rst │ ├── db.rst │ ├── distributed.rst │ ├── files.rst │ ├── iterate.rst │ ├── layers.rst │ ├── models.rst │ ├── nlp.rst │ ├── ops.rst │ ├── optimizers │ ├── optimizers.rst │ ├── prepro.rst │ ├── rein.rst │ ├── utils.rst │ └── visualize.rst ├── robots.txt ├── user │ ├── contributing.rst │ ├── development.rst │ ├── example.rst │ ├── get_involved.rst │ ├── get_start_advance.rst │ ├── get_start_model.rst │ ├── installation.rst │ ├── more.rst │ ├── my_figs │ │ ├── basic_seq2seq.png │ │ ├── book_cover.jpeg │ │ ├── book_description.jpeg │ │ ├── img_tensorlayer.png │ │ ├── img_tlayer.png │ │ ├── img_tlayer_big.png │ │ ├── img_tunelayer.png │ │ ├── karpathy_rnn.jpeg │ │ ├── mnist.jpeg │ │ ├── pong_game.jpeg │ │ ├── tl_transparent_logo.png │ │ ├── tsne.png │ │ └── word2vec_basic.pdf │ └── tutorial.rst └── wechat_group.md ├── img ├── img_qq.jpeg ├── img_qq_old.png ├── img_tensorflow.png ├── img_tensorlayer.png ├── img_tlayer1.png └── tl_transparent_logo.png ├── requirements ├── requirements.txt ├── requirements_contrib_loggers.txt ├── requirements_db.txt ├── requirements_dev.txt ├── requirements_doc.txt ├── requirements_extra.txt ├── requirements_test.txt ├── requirements_tf_cpu.txt └── requirements_tf_gpu.txt ├── setup.py ├── tensorlayer └── tl /.gitignore: -------------------------------------------------------------------------------- 1 | docs/_build 2 | tensorlayer/__pycache__ 3 | tensorlayer/*pyc 4 | tensorlayer/layers/*pyc 5 | .DS_Store 6 | tensorlayer/cli/__pycache__ 7 | .idea/ 8 | env/ 9 | venv/ 10 | ENV/ 11 | VENV/ 12 | tensorlayer.egg-info/ 13 | -------------------------------------------------------------------------------- /.gitmodules: -------------------------------------------------------------------------------- 1 | [submodule "tensorlayer-master"] 2 | path = tensorlayer-master 3 | url = https://github.com/tensorlayer/tensorlayer.git 4 | -------------------------------------------------------------------------------- /.readthedocs.yml: -------------------------------------------------------------------------------- 1 | # https://docs.readthedocs.io/en/latest/yaml-config.html 2 | 3 | build: 4 | image: latest # For python 3.6 5 | 6 | submodules: 7 | include: all 8 | recursive: true 9 | 10 | formats: 11 | - epub 12 | - pdf 13 | 14 | python: 15 | version: 3.6 16 | 17 | requirements_file: 18 | requirements/requirements_doc.txt 19 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # TensorLayer Contributor Guideline 2 | 3 | We use `git-submodule` to keep sync with tensorlayer master repository. 4 | 5 | Run the following commands to upgrade submodule to a specific tag: 6 | 7 | ```bash 8 | git clone git@github.com:tensorlayer/tensorlayer-chinese.git --recursive 9 | cd tensorlayer-chinese/tensorlayer-master 10 | git pull origin master 11 | git checkout 1.9.0 # or the latest tag you want to sync to 12 | cd .. 13 | git add -A 14 | git commit -m 'upgrade submodule to xxx' 15 | ``` 16 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 10 | 11 |
12 | 13 |
14 | 15 | 16 | [![Chinese Documentation](https://img.shields.io/badge/documentation-中文-blue.svg)](https://tensorlayercn.readthedocs.io/) 17 | [![Chinese Book](https://img.shields.io/badge/book-中文-blue.svg)](http://www.broadview.com.cn/book/5059/) 18 | [![Downloads](http://pepy.tech/badge/tensorlayer)](http://pepy.tech/project/tensorlayer) 19 | 20 | TensorLayer是一个崭新的基于[TensorFlow](https://www.tensorflow.org)的深度学习与增强学习开发库。它提供大量的神经网络相关组件和函数来帮助顶尖研究人员和一线工程师解决复杂的人工智能问题。 21 | 本仓库维护了TensorLayer的[中文文档](http://tensorlayercn.readthedocs.io/zh/latest)。相对于[英文文档]((https://github.com/tensorlayer/tensorlayer))，中文文档会有一定滞后性。 22 | 23 | TensorLayer拥有大量的华人使用者和开发者。官方的中文社群包括了[QQ群](img/img_qq.jpeg)和[微信群](https://github.com/shorxp/tensorlayer-chinese/blob/master/docs/wechat_group.md)进行广泛的讨论。官方英文渠道是[Slack](https://join.slack.com/t/tensorlayer/shared_invite/enQtMjUyMjczMzU2Njg4LWI0MWU0MDFkOWY2YjQ4YjVhMzI5M2VlZmE4YTNhNGY1NjZhMzUwMmQ2MTc0YWRjMjQzMjdjMTg2MWQ2ZWJhYzc)。申请加入这些社群时，请star[官方项目](https://github.com/tensorlayer/tensorlayer)，并告知github用户名。 24 | -------------------------------------------------------------------------------- /docs/ACM.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/docs/ACM.pdf -------------------------------------------------------------------------------- /docs/LPN_AI_symposium_handbook_poster.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/docs/LPN_AI_symposium_handbook_poster.pdf -------------------------------------------------------------------------------- /docs/Makefile: -------------------------------------------------------------------------------- 1 | # Makefile for Sphinx documentation 2 | # 3 | 4 | # You can set these variables from the command line. 5 | SPHINXOPTS = 6 | SPHINXBUILD = sphinx-build 7 | PAPER = 8 | BUILDDIR = _build 9 | 10 | # Internal variables. 11 | PAPEROPT_a4 = -D latex_paper_size=a4 12 | PAPEROPT_letter = -D latex_paper_size=letter 13 | ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . 14 | # the i18n builder cannot share the environment and doctrees with the others 15 | I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . 16 | 17 | .PHONY: help 18 | help: 19 | @echo "Please use \`make ' where is one of" 20 | @echo " html to make standalone HTML files" 21 | @echo " dirhtml to make HTML files named index.html in directories" 22 | @echo " singlehtml to make a single large HTML file" 23 | @echo " pickle to make pickle files" 24 | @echo " json to make JSON files" 25 | @echo " htmlhelp to make HTML files and a HTML help project" 26 | @echo " qthelp to make HTML files and a qthelp project" 27 | @echo " applehelp to make an Apple Help Book" 28 | @echo " devhelp to make HTML files and a Devhelp project" 29 | @echo " epub to make an epub" 30 | @echo " epub3 to make an epub3" 31 | @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" 32 | @echo " latexpdf to make LaTeX files and run them through pdflatex" 33 | @echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx" 34 | @echo " text to make text files" 35 | @echo " man to make manual pages" 36 | @echo " texinfo to make Texinfo files" 37 | @echo " info to make Texinfo files and run them through makeinfo" 38 | @echo " gettext to make PO message catalogs" 39 | @echo " changes to make an overview of all changed/added/deprecated items" 40 | @echo " xml to make Docutils-native XML files" 41 | @echo " pseudoxml to make pseudoxml-XML files for display purposes" 42 | @echo " linkcheck to check all external links for integrity" 43 | @echo " doctest to run all doctests embedded in the documentation (if enabled)" 44 | @echo " coverage to run coverage check of the documentation (if enabled)" 45 | @echo " dummy to check syntax errors of document sources" 46 | 47 | .PHONY: clean 48 | clean: 49 | rm -rf $(BUILDDIR)/* 50 | 51 | .PHONY: html 52 | html: 53 | $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html 54 | @echo 55 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." 56 | 57 | .PHONY: dirhtml 58 | dirhtml: 59 | $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml 60 | @echo 61 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." 62 | 63 | .PHONY: singlehtml 64 | singlehtml: 65 | $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml 66 | @echo 67 | @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." 68 | 69 | .PHONY: pickle 70 | pickle: 71 | $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle 72 | @echo 73 | @echo "Build finished; now you can process the pickle files." 74 | 75 | .PHONY: json 76 | json: 77 | $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json 78 | @echo 79 | @echo "Build finished; now you can process the JSON files." 80 | 81 | .PHONY: htmlhelp 82 | htmlhelp: 83 | $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp 84 | @echo 85 | @echo "Build finished; now you can run HTML Help Workshop with the" \ 86 | ".hhp project file in $(BUILDDIR)/htmlhelp." 87 | 88 | .PHONY: qthelp 89 | qthelp: 90 | $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp 91 | @echo 92 | @echo "Build finished; now you can run "qcollectiongenerator" with the" \ 93 | ".qhcp project file in $(BUILDDIR)/qthelp, like this:" 94 | @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/TLayer.qhcp" 95 | @echo "To view the help file:" 96 | @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/TLayer.qhc" 97 | 98 | .PHONY: applehelp 99 | applehelp: 100 | $(SPHINXBUILD) -b applehelp $(ALLSPHINXOPTS) $(BUILDDIR)/applehelp 101 | @echo 102 | @echo "Build finished. The help book is in $(BUILDDIR)/applehelp." 103 | @echo "N.B. You won't be able to view it unless you put it in" \ 104 | "~/Library/Documentation/Help or install it in your application" \ 105 | "bundle." 106 | 107 | .PHONY: devhelp 108 | devhelp: 109 | $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp 110 | @echo 111 | @echo "Build finished." 112 | @echo "To view the help file:" 113 | @echo "# mkdir -p $$HOME/.local/share/devhelp/TLayer" 114 | @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/TLayer" 115 | @echo "# devhelp" 116 | 117 | .PHONY: epub 118 | epub: 119 | $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub 120 | @echo 121 | @echo "Build finished. The epub file is in $(BUILDDIR)/epub." 122 | 123 | .PHONY: epub3 124 | epub3: 125 | $(SPHINXBUILD) -b epub3 $(ALLSPHINXOPTS) $(BUILDDIR)/epub3 126 | @echo 127 | @echo "Build finished. The epub3 file is in $(BUILDDIR)/epub3." 128 | 129 | .PHONY: latex 130 | latex: 131 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 132 | @echo 133 | @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." 134 | @echo "Run \`make' in that directory to run these through (pdf)latex" \ 135 | "(use \`make latexpdf' here to do that automatically)." 136 | 137 | .PHONY: latexpdf 138 | latexpdf: 139 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 140 | @echo "Running LaTeX files through pdflatex..." 141 | $(MAKE) -C $(BUILDDIR)/latex all-pdf 142 | @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." 143 | 144 | .PHONY: latexpdfja 145 | latexpdfja: 146 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 147 | @echo "Running LaTeX files through platex and dvipdfmx..." 148 | $(MAKE) -C $(BUILDDIR)/latex all-pdf-ja 149 | @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." 150 | 151 | .PHONY: text 152 | text: 153 | $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text 154 | @echo 155 | @echo "Build finished. The text files are in $(BUILDDIR)/text." 156 | 157 | .PHONY: man 158 | man: 159 | $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man 160 | @echo 161 | @echo "Build finished. The manual pages are in $(BUILDDIR)/man." 162 | 163 | .PHONY: texinfo 164 | texinfo: 165 | $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo 166 | @echo 167 | @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." 168 | @echo "Run \`make' in that directory to run these through makeinfo" \ 169 | "(use \`make info' here to do that automatically)." 170 | 171 | .PHONY: info 172 | info: 173 | $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo 174 | @echo "Running Texinfo files through makeinfo..." 175 | make -C $(BUILDDIR)/texinfo info 176 | @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." 177 | 178 | .PHONY: gettext 179 | gettext: 180 | $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale 181 | @echo 182 | @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." 183 | 184 | .PHONY: changes 185 | changes: 186 | $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes 187 | @echo 188 | @echo "The overview file is in $(BUILDDIR)/changes." 189 | 190 | .PHONY: linkcheck 191 | linkcheck: 192 | $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck 193 | @echo 194 | @echo "Link check complete; look for any errors in the above output " \ 195 | "or in $(BUILDDIR)/linkcheck/output.txt." 196 | 197 | .PHONY: doctest 198 | doctest: 199 | $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest 200 | @echo "Testing of doctests in the sources finished, look at the " \ 201 | "results in $(BUILDDIR)/doctest/output.txt." 202 | 203 | .PHONY: coverage 204 | coverage: 205 | $(SPHINXBUILD) -b coverage $(ALLSPHINXOPTS) $(BUILDDIR)/coverage 206 | @echo "Testing of coverage in the sources finished, look at the " \ 207 | "results in $(BUILDDIR)/coverage/python.txt." 208 | 209 | .PHONY: xml 210 | xml: 211 | $(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml 212 | @echo 213 | @echo "Build finished. The XML files are in $(BUILDDIR)/xml." 214 | 215 | .PHONY: pseudoxml 216 | pseudoxml: 217 | $(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml 218 | @echo 219 | @echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml." 220 | 221 | .PHONY: dummy 222 | dummy: 223 | $(SPHINXBUILD) -b dummy $(ALLSPHINXOPTS) $(BUILDDIR)/dummy 224 | @echo 225 | @echo "Build finished. Dummy builder generates no files." 226 | 227 | .PHONY: xelatexpdf 228 | xelatexpdf: 229 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 230 | @echo "Running LaTeX files through xelatex..." 231 | sed -i s/pdflatex/xelatex/ $(BUILDDIR)/latex/Makefile 232 | $(MAKE) -C $(BUILDDIR)/latex all-pdf 233 | @echo "xelatex finished; the PDF files are in $(BUILDDIR)/latex." 234 | 235 | -------------------------------------------------------------------------------- /docs/TensorLayer ACM MM.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/docs/TensorLayer ACM MM.pdf -------------------------------------------------------------------------------- /docs/TensorLayer_London_PhD_Network_20180212.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/docs/TensorLayer_London_PhD_Network_20180212.pdf -------------------------------------------------------------------------------- /docs/_static/fix_rtd.css: -------------------------------------------------------------------------------- 1 | /* work around https://github.com/snide/sphinx_rtd_theme/issues/149 */ 2 | .rst-content table.field-list .field-body { 3 | padding-top: 8px; 4 | } 5 | 6 | /*.section #basic-2-flip-flop-synchronizer{ 7 | text-align:justify; 8 | }*/ 9 | -------------------------------------------------------------------------------- /docs/conf.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | # 4 | # TensorLayer documentation build configuration file, created by 5 | # sphinx-quickstart on Tue Aug 2 15:30:55 2016. 6 | # 7 | # This file is execfile()d with the current directory set to its 8 | # containing dir. 9 | # 10 | # Note that not all possible configuration values are present in this 11 | # autogenerated file. 12 | # 13 | # All configuration values have a default; values that are commented out 14 | # serve to show the default. 15 | 16 | # If extensions (or modules to document with autodoc) are in another directory, 17 | # add these directories to sys.path here. If the directory is relative to the 18 | # documentation root, use os.path.abspath to make it absolute, like shown here. 19 | # 20 | import os 21 | import sys 22 | import datetime 23 | sys.path.insert(0, os.path.abspath("../")) # Important 24 | sys.path.insert(0, os.path.abspath( 25 | os.path.join("..", "tensorlayer"))) # Important 26 | 27 | from package_info import __shortversion__ 28 | from package_info import __version__ 29 | 30 | # -- General configuration ------------------------------------------------ 31 | 32 | # If your documentation needs a minimal Sphinx version, state it here. 33 | # 34 | # needs_sphinx = '1.0' 35 | 36 | # Add any Sphinx extension module names here, as strings. They can be 37 | # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom 38 | # ones. 39 | 40 | # extensions = [ 41 | # 'sphinx.ext.coverage', 42 | # 'sphinx.ext.githubpages', 43 | # 'numpydoc', 44 | # ] 45 | 46 | extensions = [ 47 | 'sphinx.ext.autodoc', 48 | 'sphinx.ext.autosummary', 49 | 'sphinx.ext.doctest', 50 | 'sphinx.ext.ifconfig', 51 | 'sphinx.ext.inheritance_diagram', 52 | 'sphinx.ext.intersphinx', 53 | 'sphinx.ext.mathjax', 54 | 'sphinx.ext.napoleon', 55 | 'sphinx.ext.todo', 56 | 'sphinx.ext.viewcode', 57 | ] 58 | 59 | autodoc_mock_imports = [ 60 | 'cv2', 61 | 'gridfs', 62 | 'horovod', 63 | 'hyperdash', 64 | 'imageio', 65 | 'lxml', 66 | 'matplotlib', 67 | 'nltk', 68 | 'numpy', 69 | 'PIL', 70 | 'progressbar', 71 | 'pymongo', 72 | 'scipy', 73 | 'skimage', 74 | 'sklearn', 75 | 'tensorflow', 76 | 'tqdm', 77 | 'h5py', 78 | 79 | # TL C++ Packages 80 | 'tensorlayer.third_party.roi_pooling.roi_pooling.roi_pooling_ops', 81 | ] 82 | 83 | # Add any paths that contain templates here, relative to this directory. 84 | templates_path = ['_templates'] 85 | 86 | # The suffix(es) of source filenames. 87 | # You can specify multiple suffix as a list of string: 88 | # 89 | # source_suffix = ['.rst', '.md'] 90 | source_suffix = '.rst' 91 | 92 | # The encoding of source files. 93 | # 94 | # source_encoding = 'utf-8-sig' 95 | 96 | # The master toctree document. 97 | master_doc = 'index' 98 | 99 | # General information about the project. 100 | project = u'TensorLayer 中文版' 101 | copyright = '2016~%s, TensorLayer Contributors' % ( 102 | str(datetime.datetime.now().year)) 103 | author = 'TensorLayer Contributors' 104 | 105 | # The version info for the project you're documenting, acts as replacement for 106 | # |version| and |release|, also used in various other places throughout the 107 | # built documents. 108 | # 109 | # The short X.Y version. 110 | version = __shortversion__ 111 | # The full version, including alpha/beta/rc tags. 112 | release = __version__ 113 | 114 | # The language for content autogenerated by Sphinx. Refer to documentation 115 | # for a list of supported languages. 116 | # 117 | # This is also used if you do content translation via gettext catalogs. 118 | # Usually you set "language" from the command line for these cases. 119 | language = None 120 | 121 | # There are two options for replacing |today|: either, you set today to some 122 | # non-false value, then it is used: 123 | # 124 | # today = '' 125 | # 126 | # Else, today_fmt is used as the format for a strftime call. 127 | # 128 | # today_fmt = '%B %d, %Y' 129 | 130 | # List of patterns, relative to source directory, that match files and 131 | # directories to ignore when looking for source files. 132 | # This patterns also effect to html_static_path and html_extra_path 133 | exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] 134 | 135 | # The reST default role (used for this markup: `text`) to use for all 136 | # documents. 137 | # 138 | # default_role = None 139 | 140 | # If true, '()' will be appended to :func: etc. cross-reference text. 141 | # 142 | # add_function_parentheses = True 143 | 144 | # If true, the current module name will be prepended to all description 145 | # unit titles (such as .. function::). 146 | # 147 | # add_module_names = True 148 | 149 | # If true, sectionauthor and moduleauthor directives will be shown in the 150 | # output. They are ignored by default. 151 | # 152 | # show_authors = False 153 | 154 | # The name of the Pygments (syntax highlighting) style to use. 155 | pygments_style = 'sphinx' 156 | 157 | # A list of ignored prefixes for module index sorting. 158 | # modindex_common_prefix = [] 159 | 160 | # If true, keep warnings as "system message" paragraphs in the built documents. 161 | # keep_warnings = False 162 | 163 | # If true, `todo` and `todoList` produce output, else they produce nothing. 164 | todo_include_todos = False 165 | 166 | 167 | # -- Options for HTML output ---------------------------------------------- 168 | 169 | # The theme to use for HTML and HTML Help pages. See the documentation for 170 | # a list of builtin themes. 171 | # 172 | # html_theme = 'alabaster' 173 | 174 | # Theme options are theme-specific and customize the look and feel of a theme 175 | # further. For a list of options available for each theme, see the 176 | # documentation. 177 | # 178 | # html_theme_options = {} 179 | 180 | # Add any paths that contain custom themes here, relative to this directory. 181 | # html_theme_path = [] 182 | 183 | # The name for this set of Sphinx documents. 184 | # " v documentation" by default. 185 | # 186 | # html_title = 'TensorLayer' 187 | 188 | # A shorter title for the navigation bar. Default is the same as html_title. 189 | # 190 | # html_short_title = None 191 | 192 | # The name of an image file (relative to this directory) to place at the top 193 | # of the sidebar. 194 | # 195 | # html_logo = None 196 | 197 | # The name of an image file (relative to this directory) to use as a favicon of 198 | # the docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 199 | # pixels large. 200 | # 201 | # html_favicon = None 202 | 203 | # Add any paths that contain custom static files (such as style sheets) here, 204 | # relative to this directory. They are copied after the builtin static files, 205 | # so a file named "default.css" will overwrite the builtin "default.css". 206 | html_static_path = [] 207 | 208 | # Add any extra paths that contain custom files (such as robots.txt or 209 | # .htaccess) here, relative to this directory. These files are copied 210 | # directly to the root of the documentation. 211 | # 212 | # html_extra_path = [] 213 | 214 | # If not None, a 'Last updated on:' timestamp is inserted at every page 215 | # bottom, using the given strftime format. 216 | # The empty string is equivalent to '%b %d, %Y'. 217 | # 218 | # html_last_updated_fmt = None 219 | 220 | # If true, SmartyPants will be used to convert quotes and dashes to 221 | # typographically correct entities. 222 | # 223 | # html_use_smartypants = True 224 | 225 | # Custom sidebar templates, maps document names to template names. 226 | # 227 | # html_sidebars = {} 228 | 229 | # Additional templates that should be rendered to pages, maps page names to 230 | # template names. 231 | # 232 | # html_additional_pages = {} 233 | 234 | # If false, no module index is generated. 235 | # 236 | # html_domain_indices = True 237 | 238 | # If false, no index is generated. 239 | # 240 | # html_use_index = True 241 | 242 | # If true, the index is split into individual pages for each letter. 243 | # 244 | # html_split_index = False 245 | 246 | # If true, links to the reST sources are added to the pages. 247 | # 248 | # html_show_sourcelink = True 249 | 250 | # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. 251 | # 252 | # html_show_sphinx = True 253 | 254 | # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. 255 | # 256 | # html_show_copyright = True 257 | 258 | # If true, an OpenSearch description file will be output, and all pages will 259 | # contain a tag referring to it. The value of this option must be the 260 | # base URL from which the finished HTML is served. 261 | # 262 | # html_use_opensearch = '' 263 | 264 | # This is the file name suffix for HTML files (e.g. ".xhtml"). 265 | # html_file_suffix = None 266 | 267 | # Language to be used for generating the HTML full-text search index. 268 | # Sphinx supports the following languages: 269 | # 'da', 'de', 'en', 'es', 'fi', 'fr', 'h', 'it', 'ja' 270 | # 'nl', 'no', 'pt', 'ro', 'r', 'sv', 'tr', 'zh' 271 | # 272 | # html_search_language = 'en' 273 | 274 | # A dictionary with options for the search language support, empty by default. 275 | # 'ja' uses this config value. 276 | # 'zh' user can custom change `jieba` dictionary path. 277 | # 278 | # html_search_options = {'type': 'default'} 279 | 280 | # The name of a javascript file (relative to the configuration directory) that 281 | # implements a search results scorer. If empty, the default will be used. 282 | # 283 | # html_search_scorer = 'scorer.js' 284 | 285 | # Output file base name for HTML help builder. 286 | htmlhelp_basename = 'TensorLayerdoc' 287 | 288 | # -- Options for LaTeX output --------------------------------------------- 289 | 290 | latex_elements = { 291 | # The paper size ('letterpaper' or 'a4paper'). 292 | # 293 | # 'papersize': 'letterpaper', 294 | 295 | # The font size ('10pt', '11pt' or '12pt'). 296 | # 297 | # 'pointsize': '10pt', 298 | 299 | # Additional stuff for the LaTeX preamble. 300 | # 301 | # 'preamble': '', 302 | 303 | # Latex figure (float) alignment 304 | # 305 | # 'figure_align': 'htbp', 306 | } 307 | 308 | # Grouping the document tree into LaTeX files. List of tuples 309 | # (source start file, target name, title, 310 | # author, documentclass [howto, manual, or own class]). 311 | latex_documents = [ 312 | (master_doc, 'TensorLayer.tex', 'TensorLayer Documentation', 313 | 'TensorLayer contributors', 'manual'), 314 | ] 315 | 316 | # The name of an image file (relative to this directory) to place at the top of 317 | # the title page. 318 | # 319 | # latex_logo = None 320 | 321 | # For "manual" documents, if this is true, then toplevel headings are parts, 322 | # not chapters. 323 | # 324 | # latex_use_parts = False 325 | 326 | # If true, show page references after internal links. 327 | # 328 | # latex_show_pagerefs = False 329 | 330 | # If true, show URL addresses after external links. 331 | # 332 | # latex_show_urls = False 333 | 334 | # Documents to append as an appendix to all manuals. 335 | # 336 | # latex_appendices = [] 337 | 338 | # If false, no module index is generated. 339 | # 340 | # latex_domain_indices = True 341 | 342 | 343 | # -- Options for manual page output --------------------------------------- 344 | 345 | # One entry per manual page. List of tuples 346 | # (source start file, name, description, authors, manual section). 347 | man_pages = [ 348 | (master_doc, 'tensorlayer', 'TensorLayer Documentation', 349 | [author], 1) 350 | ] 351 | 352 | # If true, show URL addresses after external links. 353 | # 354 | # man_show_urls = False 355 | 356 | 357 | # -- Options for Texinfo output ------------------------------------------- 358 | 359 | # Grouping the document tree into Texinfo files. List of tuples 360 | # (source start file, target name, title, author, 361 | # dir menu entry, description, category) 362 | texinfo_documents = [ 363 | (master_doc, 'TensorLayer', 'TensorLayer Documentation', 364 | author, 'TensorLayer', 'Deep learning and Reinforcement learning library for Researchers and Engineers.', 365 | 'Miscellaneous'), 366 | ] 367 | 368 | # Documents to append as an appendix to all manuals. 369 | # 370 | # texinfo_appendices = [] 371 | 372 | # If false, no module index is generated. 373 | # 374 | # texinfo_domain_indices = True 375 | 376 | # How to display URL addresses: 'footnote', 'no', or 'inline'. 377 | # 378 | # texinfo_show_urls = 'footnote' 379 | 380 | # If true, do not generate a @detailmenu in the "Top" node's menu. 381 | # 382 | # texinfo_no_detailmenu = False 383 | 384 | 385 | # -- Options for Epub output ---------------------------------------------- 386 | 387 | # Bibliographic Dublin Core info. 388 | epub_title = project 389 | epub_author = author 390 | epub_publisher = author 391 | epub_copyright = copyright 392 | 393 | # The basename for the epub file. It defaults to the project name. 394 | # epub_basename = project 395 | 396 | # The HTML theme for the epub output. Since the default themes are not 397 | # optimized for small screen space, using the same theme for HTML and epub 398 | # output is usually not wise. This defaults to 'epub', a theme designed to save 399 | # visual space. 400 | # 401 | # epub_theme = 'epub' 402 | 403 | # The language of the text. It defaults to the language option 404 | # or 'en' if the language is not set. 405 | # 406 | # epub_language = '' 407 | 408 | # The scheme of the identifier. Typical schemes are ISBN or URL. 409 | # epub_scheme = '' 410 | 411 | # The unique identifier of the text. This can be a ISBN number 412 | # or the project homepage. 413 | # 414 | # epub_identifier = '' 415 | 416 | # A unique identification for the text. 417 | # 418 | # epub_uid = '' 419 | 420 | # A tuple containing the cover image and cover page html template filenames. 421 | # 422 | # epub_cover = () 423 | 424 | # A sequence of (type, uri, title) tuples for the guide element of content.opf. 425 | # 426 | # epub_guide = () 427 | 428 | # HTML files that should be inserted before the pages created by sphinx. 429 | # The format is a list of tuples containing the path and title. 430 | # 431 | # epub_pre_files = [] 432 | 433 | # HTML files that should be inserted after the pages created by sphinx. 434 | # The format is a list of tuples containing the path and title. 435 | # 436 | # epub_post_files = [] 437 | 438 | # A list of files that should not be packed into the epub file. 439 | epub_exclude_files = ['search.html'] 440 | 441 | # The depth of the table of contents in toc.ncx. 442 | # 443 | # epub_tocdepth = 3 444 | 445 | # Allow duplicate toc entries. 446 | # 447 | # epub_tocdup = True 448 | 449 | # Choose between 'default' and 'includehidden'. 450 | # 451 | # epub_tocscope = 'default' 452 | 453 | # Fix unsupported image types using the Pillow. 454 | # 455 | # epub_fix_images = False 456 | 457 | # Scale large images. 458 | # 459 | # epub_max_image_width = 0 460 | 461 | # How to display URL addresses: 'footnote', 'no', or 'inline'. 462 | # 463 | # epub_show_urls = 'inline' 464 | 465 | # If false, no index is generated. 466 | # 467 | # epub_use_index = True 468 | 469 | pygments_style = 'sphinx' 470 | 471 | html_theme = "sphinx_rtd_theme" 472 | 473 | html_theme_path = [] 474 | 475 | # Fix Chinese display issues 476 | on_rtd = os.environ.get('READTHEDOCS', None) == 'True' 477 | if on_rtd: 478 | latex_elements = { 479 | # The paper size ('letterpaper' or 'a4paper'). 480 | # 'papersize': 'letterpaper', 481 | # The font size ('10pt', '11pt' or '12pt'). 482 | # 'pointsize': '10pt', 483 | # Additional stuff for the LaTeX preamble. 484 | 'preamble': r''' 485 | \hypersetup{unicode=true} 486 | \usepackage{CJKutf8} 487 | \DeclareUnicodeCharacter{00A0}{\nobreakspace} 488 | \DeclareUnicodeCharacter{2203}{\ensuremath{\exists}} 489 | \DeclareUnicodeCharacter{2200}{\ensuremath{\forall}} 490 | \DeclareUnicodeCharacter{2286}{\ensuremath{\subseteq}} 491 | \DeclareUnicodeCharacter{2713}{x} 492 | \DeclareUnicodeCharacter{27FA}{\ensuremath{\Longleftrightarrow}} 493 | \DeclareUnicodeCharacter{221A}{\ensuremath{\sqrt{}}} 494 | \DeclareUnicodeCharacter{221B}{\ensuremath{\sqrt[3]{}}} 495 | \DeclareUnicodeCharacter{2295}{\ensuremath{\oplus}} 496 | \DeclareUnicodeCharacter{2297}{\ensuremath{\otimes}} 497 | \begin{CJK}{UTF8}{gbsn} 498 | \AtEndDocument{\end{CJK}} 499 | ''', 500 | } 501 | else: 502 | latex_elements = { 503 | 'papersize': 'a4paper', 504 | 'utf8extra': '', 505 | 'inputenc': '', 506 | 'babel': r'''\usepackage[english]{babel}''', 507 | 'preamble': r''' 508 | \usepackage{ctex} 509 | ''', 510 | } 511 | -------------------------------------------------------------------------------- /docs/images/RL_Group_QR.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/docs/images/RL_Group_QR.jpeg -------------------------------------------------------------------------------- /docs/images/donghao.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/docs/images/donghao.jpeg -------------------------------------------------------------------------------- /docs/images/moyuanhan.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/docs/images/moyuanhan.jpeg -------------------------------------------------------------------------------- /docs/images/yangguang.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/docs/images/yangguang.jpeg -------------------------------------------------------------------------------- /docs/images/yusimiao.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/docs/images/yusimiao.jpeg -------------------------------------------------------------------------------- /docs/index.rst: -------------------------------------------------------------------------------- 1 | TensorLayer 中文文档 2 | ============================== 3 | 4 | 5 | .. image:: user/my_figs/tl_transparent_logo.png 6 | :scale: 25 % 7 | :align: center 8 | :target: https://github.com/tensorlayer/tensorlayer 9 | 10 | **好消息** 我们获得了 `ACM Multimedia (MM) `_ **年度最佳开源软件奖**。 11 | 12 | **如果中文文档有更新错误，请暂时阅读** `英文文档 `_ 13 | 14 | TensorLayer 是为研究人员和工程师设计的一款基于Google TensorFlow开发的深度学习与强化学习库。 15 | 它提供高级别的（Higher-Level）深度学习API，这样不仅可以加快研究人员的实验速度，也能够减少工程师在实际开发当中的重复工作。 TensorLayer非常易于修改和扩展，这使它可以同时用于机器学习的研究与应用。 16 | 此外，TensorLayer 提供了大量示例和教程来帮助初学者理解深度学习，并提供大量的官方例子程序方便开发者快速找到适合自己项目的例子。 17 | 更多细节请点击 `这里 `_ 。 18 | 19 | 这篇文档不仅仅是为了描述如何使用这个库也是一个遍历不同类型的神经网络， 20 | 深度强化学习和自然语言处理等内容的教程。 21 | 此外，TensorLayer的Tutorial包含了所有TensorFlow官方深度学习教程的模块化实现，因此你可以对照TensorFlow深度学习教程来学习 `[英文] `_ `[极客学院中文翻译] `_ 22 | 23 | .. note:: 24 | 我们建议你在 `Github `_ 上star和watch `官方项目 `_ ，这样当官方有更新时，你会立即知道。本文档为 `官方RTD文档 `_ 的翻译版，更新速度会比英文原版慢，若你的英文还行，我们建议你直接阅读 `官方RTD文档 `_。 25 | 26 | 如果你阅读在线中文文档时有什么问题，你可以在github上下载这个项目， 27 | 然后去 ``/docs/cn/_build/html/index.html`` 阅读离线中文文档。 28 | 或者在 `Read the docs `_ 中阅读官方原文档。 29 | 30 | 用户指南 31 | ----------- 32 | 33 | TensorLayer用户指南说明了如何去安装TensorFlow,CUDA和cuDNN， 34 | 然后如何用TensorLayer建立和训练神经网络和如何作为（一个开发者支持这个库。） 35 | 36 | .. toctree:: 37 | :maxdepth: 2 38 | 39 | user/installation 40 | user/example 41 | user/get_involved 42 | 43 | .. toctree:: 44 | :maxdepth: 2 45 | :caption: 使用说明 46 | 47 | user/get_start_model 48 | user/get_start_advance 49 | 50 | API目录 51 | ---------- 52 | 53 | 如果你正在寻找某个特殊的函数，类或者方法，这一列文档就是为你准备的。 54 | 55 | .. toctree:: 56 | :maxdepth: 2 57 | :caption: 稳定模块 58 | 59 | modules/activation 60 | modules/array_ops 61 | modules/cost 62 | modules/files 63 | modules/iterate 64 | modules/layers 65 | modules/models 66 | modules/nlp 67 | modules/optimizers 68 | modules/prepro 69 | modules/rein 70 | modules/utils 71 | modules/visualize 72 | 73 | .. toctree:: 74 | :maxdepth: 2 75 | :caption: 开发中模块 76 | 77 | modules/db 78 | modules/optimizers 79 | modules/distributed 80 | 81 | 命令行界面 82 | ---------------------- 83 | 84 | TensorLayer提供简单易用的命令行工具 `tl` 来执行一些常用的任务。 85 | 86 | .. toctree:: 87 | :maxdepth: 2 88 | 89 | modules/cli 90 | 91 | 92 | 索引与附录 93 | ================== 94 | 95 | * :ref:`genindex` 96 | * :ref:`modindex` 97 | * :ref:`search` 98 | 99 | .. _GitHub: https://github.com/zsdonghao/tensorlayer 100 | -------------------------------------------------------------------------------- /docs/make.bat: -------------------------------------------------------------------------------- 1 | @ECHO OFF 2 | 3 | REM Command file for Sphinx documentation 4 | 5 | if "%SPHINXBUILD%" == "" ( 6 | set SPHINXBUILD=sphinx-build 7 | ) 8 | set BUILDDIR=_build 9 | set ALLSPHINXOPTS=-d %BUILDDIR%/doctrees %SPHINXOPTS% . 10 | set I18NSPHINXOPTS=%SPHINXOPTS% . 11 | if NOT "%PAPER%" == "" ( 12 | set ALLSPHINXOPTS=-D latex_paper_size=%PAPER% %ALLSPHINXOPTS% 13 | set I18NSPHINXOPTS=-D latex_paper_size=%PAPER% %I18NSPHINXOPTS% 14 | ) 15 | 16 | if "%1" == "" goto help 17 | 18 | if "%1" == "help" ( 19 | :help 20 | echo.Please use `make ^` where ^ is one of 21 | echo. html to make standalone HTML files 22 | echo. dirhtml to make HTML files named index.html in directories 23 | echo. singlehtml to make a single large HTML file 24 | echo. pickle to make pickle files 25 | echo. json to make JSON files 26 | echo. htmlhelp to make HTML files and a HTML help project 27 | echo. qthelp to make HTML files and a qthelp project 28 | echo. devhelp to make HTML files and a Devhelp project 29 | echo. epub to make an epub 30 | echo. epub3 to make an epub3 31 | echo. latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter 32 | echo. text to make text files 33 | echo. man to make manual pages 34 | echo. texinfo to make Texinfo files 35 | echo. gettext to make PO message catalogs 36 | echo. changes to make an overview over all changed/added/deprecated items 37 | echo. xml to make Docutils-native XML files 38 | echo. pseudoxml to make pseudoxml-XML files for display purposes 39 | echo. linkcheck to check all external links for integrity 40 | echo. doctest to run all doctests embedded in the documentation if enabled 41 | echo. coverage to run coverage check of the documentation if enabled 42 | echo. dummy to check syntax errors of document sources 43 | goto end 44 | ) 45 | 46 | if "%1" == "clean" ( 47 | for /d %%i in (%BUILDDIR%\*) do rmdir /q /s %%i 48 | del /q /s %BUILDDIR%\* 49 | goto end 50 | ) 51 | 52 | 53 | REM Check if sphinx-build is available and fallback to Python version if any 54 | %SPHINXBUILD% 1>NUL 2>NUL 55 | if errorlevel 9009 goto sphinx_python 56 | goto sphinx_ok 57 | 58 | :sphinx_python 59 | 60 | set SPHINXBUILD=python -m sphinx.__init__ 61 | %SPHINXBUILD% 2> nul 62 | if errorlevel 9009 ( 63 | echo. 64 | echo.The 'sphinx-build' command was not found. Make sure you have Sphinx 65 | echo.installed, then set the SPHINXBUILD environment variable to point 66 | echo.to the full path of the 'sphinx-build' executable. Alternatively you 67 | echo.may add the Sphinx directory to PATH. 68 | echo. 69 | echo.If you don't have Sphinx installed, grab it from 70 | echo.http://sphinx-doc.org/ 71 | exit /b 1 72 | ) 73 | 74 | :sphinx_ok 75 | 76 | 77 | if "%1" == "html" ( 78 | %SPHINXBUILD% -b html %ALLSPHINXOPTS% %BUILDDIR%/html 79 | if errorlevel 1 exit /b 1 80 | echo. 81 | echo.Build finished. The HTML pages are in %BUILDDIR%/html. 82 | goto end 83 | ) 84 | 85 | if "%1" == "dirhtml" ( 86 | %SPHINXBUILD% -b dirhtml %ALLSPHINXOPTS% %BUILDDIR%/dirhtml 87 | if errorlevel 1 exit /b 1 88 | echo. 89 | echo.Build finished. The HTML pages are in %BUILDDIR%/dirhtml. 90 | goto end 91 | ) 92 | 93 | if "%1" == "singlehtml" ( 94 | %SPHINXBUILD% -b singlehtml %ALLSPHINXOPTS% %BUILDDIR%/singlehtml 95 | if errorlevel 1 exit /b 1 96 | echo. 97 | echo.Build finished. The HTML pages are in %BUILDDIR%/singlehtml. 98 | goto end 99 | ) 100 | 101 | if "%1" == "pickle" ( 102 | %SPHINXBUILD% -b pickle %ALLSPHINXOPTS% %BUILDDIR%/pickle 103 | if errorlevel 1 exit /b 1 104 | echo. 105 | echo.Build finished; now you can process the pickle files. 106 | goto end 107 | ) 108 | 109 | if "%1" == "json" ( 110 | %SPHINXBUILD% -b json %ALLSPHINXOPTS% %BUILDDIR%/json 111 | if errorlevel 1 exit /b 1 112 | echo. 113 | echo.Build finished; now you can process the JSON files. 114 | goto end 115 | ) 116 | 117 | if "%1" == "htmlhelp" ( 118 | %SPHINXBUILD% -b htmlhelp %ALLSPHINXOPTS% %BUILDDIR%/htmlhelp 119 | if errorlevel 1 exit /b 1 120 | echo. 121 | echo.Build finished; now you can run HTML Help Workshop with the ^ 122 | .hhp project file in %BUILDDIR%/htmlhelp. 123 | goto end 124 | ) 125 | 126 | if "%1" == "qthelp" ( 127 | %SPHINXBUILD% -b qthelp %ALLSPHINXOPTS% %BUILDDIR%/qthelp 128 | if errorlevel 1 exit /b 1 129 | echo. 130 | echo.Build finished; now you can run "qcollectiongenerator" with the ^ 131 | .qhcp project file in %BUILDDIR%/qthelp, like this: 132 | echo.^> qcollectiongenerator %BUILDDIR%\qthelp\TLayer.qhcp 133 | echo.To view the help file: 134 | echo.^> assistant -collectionFile %BUILDDIR%\qthelp\TLayer.ghc 135 | goto end 136 | ) 137 | 138 | if "%1" == "devhelp" ( 139 | %SPHINXBUILD% -b devhelp %ALLSPHINXOPTS% %BUILDDIR%/devhelp 140 | if errorlevel 1 exit /b 1 141 | echo. 142 | echo.Build finished. 143 | goto end 144 | ) 145 | 146 | if "%1" == "epub" ( 147 | %SPHINXBUILD% -b epub %ALLSPHINXOPTS% %BUILDDIR%/epub 148 | if errorlevel 1 exit /b 1 149 | echo. 150 | echo.Build finished. The epub file is in %BUILDDIR%/epub. 151 | goto end 152 | ) 153 | 154 | if "%1" == "epub3" ( 155 | %SPHINXBUILD% -b epub3 %ALLSPHINXOPTS% %BUILDDIR%/epub3 156 | if errorlevel 1 exit /b 1 157 | echo. 158 | echo.Build finished. The epub3 file is in %BUILDDIR%/epub3. 159 | goto end 160 | ) 161 | 162 | if "%1" == "latex" ( 163 | %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex 164 | if errorlevel 1 exit /b 1 165 | echo. 166 | echo.Build finished; the LaTeX files are in %BUILDDIR%/latex. 167 | goto end 168 | ) 169 | 170 | if "%1" == "latexpdf" ( 171 | %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex 172 | cd %BUILDDIR%/latex 173 | make all-pdf 174 | cd %~dp0 175 | echo. 176 | echo.Build finished; the PDF files are in %BUILDDIR%/latex. 177 | goto end 178 | ) 179 | 180 | if "%1" == "latexpdfja" ( 181 | %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex 182 | cd %BUILDDIR%/latex 183 | make all-pdf-ja 184 | cd %~dp0 185 | echo. 186 | echo.Build finished; the PDF files are in %BUILDDIR%/latex. 187 | goto end 188 | ) 189 | 190 | if "%1" == "text" ( 191 | %SPHINXBUILD% -b text %ALLSPHINXOPTS% %BUILDDIR%/text 192 | if errorlevel 1 exit /b 1 193 | echo. 194 | echo.Build finished. The text files are in %BUILDDIR%/text. 195 | goto end 196 | ) 197 | 198 | if "%1" == "man" ( 199 | %SPHINXBUILD% -b man %ALLSPHINXOPTS% %BUILDDIR%/man 200 | if errorlevel 1 exit /b 1 201 | echo. 202 | echo.Build finished. The manual pages are in %BUILDDIR%/man. 203 | goto end 204 | ) 205 | 206 | if "%1" == "texinfo" ( 207 | %SPHINXBUILD% -b texinfo %ALLSPHINXOPTS% %BUILDDIR%/texinfo 208 | if errorlevel 1 exit /b 1 209 | echo. 210 | echo.Build finished. The Texinfo files are in %BUILDDIR%/texinfo. 211 | goto end 212 | ) 213 | 214 | if "%1" == "gettext" ( 215 | %SPHINXBUILD% -b gettext %I18NSPHINXOPTS% %BUILDDIR%/locale 216 | if errorlevel 1 exit /b 1 217 | echo. 218 | echo.Build finished. The message catalogs are in %BUILDDIR%/locale. 219 | goto end 220 | ) 221 | 222 | if "%1" == "changes" ( 223 | %SPHINXBUILD% -b changes %ALLSPHINXOPTS% %BUILDDIR%/changes 224 | if errorlevel 1 exit /b 1 225 | echo. 226 | echo.The overview file is in %BUILDDIR%/changes. 227 | goto end 228 | ) 229 | 230 | if "%1" == "linkcheck" ( 231 | %SPHINXBUILD% -b linkcheck %ALLSPHINXOPTS% %BUILDDIR%/linkcheck 232 | if errorlevel 1 exit /b 1 233 | echo. 234 | echo.Link check complete; look for any errors in the above output ^ 235 | or in %BUILDDIR%/linkcheck/output.txt. 236 | goto end 237 | ) 238 | 239 | if "%1" == "doctest" ( 240 | %SPHINXBUILD% -b doctest %ALLSPHINXOPTS% %BUILDDIR%/doctest 241 | if errorlevel 1 exit /b 1 242 | echo. 243 | echo.Testing of doctests in the sources finished, look at the ^ 244 | results in %BUILDDIR%/doctest/output.txt. 245 | goto end 246 | ) 247 | 248 | if "%1" == "coverage" ( 249 | %SPHINXBUILD% -b coverage %ALLSPHINXOPTS% %BUILDDIR%/coverage 250 | if errorlevel 1 exit /b 1 251 | echo. 252 | echo.Testing of coverage in the sources finished, look at the ^ 253 | results in %BUILDDIR%/coverage/python.txt. 254 | goto end 255 | ) 256 | 257 | if "%1" == "xml" ( 258 | %SPHINXBUILD% -b xml %ALLSPHINXOPTS% %BUILDDIR%/xml 259 | if errorlevel 1 exit /b 1 260 | echo. 261 | echo.Build finished. The XML files are in %BUILDDIR%/xml. 262 | goto end 263 | ) 264 | 265 | if "%1" == "pseudoxml" ( 266 | %SPHINXBUILD% -b pseudoxml %ALLSPHINXOPTS% %BUILDDIR%/pseudoxml 267 | if errorlevel 1 exit /b 1 268 | echo. 269 | echo.Build finished. The pseudo-XML files are in %BUILDDIR%/pseudoxml. 270 | goto end 271 | ) 272 | 273 | if "%1" == "dummy" ( 274 | %SPHINXBUILD% -b dummy %ALLSPHINXOPTS% %BUILDDIR%/dummy 275 | if errorlevel 1 exit /b 1 276 | echo. 277 | echo.Build finished. Dummy builder generates no files. 278 | goto end 279 | ) 280 | 281 | :end 282 | -------------------------------------------------------------------------------- /docs/modules/activation.rst: -------------------------------------------------------------------------------- 1 | API - 激活函数 2 | ============================== 3 | 4 | 为了尽可能地保持TensorLayer的简洁性，我们最小化激活函数的数量，因此我们鼓励用户直接使用 5 | TensorFlow官方的函数，比如 6 | ``tf.nn.relu``, ``tf.nn.relu6``, ``tf.nn.elu``, ``tf.nn.softplus``, 7 | ``tf.nn.softsign`` 等等。更多TensorFlow官方激活函数请看 8 | `这里 `_. 9 | 10 | 11 | 自定义激活函数 12 | --------------------------- 13 | 14 | 在TensorLayer中创造自定义激活函数非常简单。 15 | 16 | 下面的例子实现了把输入乘以2。对于更加复杂的激活函数，你需要用到TensorFlow的API。 17 | 18 | .. code-block:: python 19 | 20 | def double_activation(x): 21 | return x * 2 22 | 23 | .. automodule:: tensorlayer.activation 24 | 25 | .. autosummary:: 26 | 27 | leaky_relu 28 | leaky_relu6 29 | leaky_twice_relu6 30 | ramp 31 | swish 32 | sign 33 | hard_tanh 34 | pixel_wise_softmax 35 | mish 36 | 37 | 38 | Ramp 39 | ------ 40 | .. autofunction:: ramp 41 | 42 | Leaky ReLU 43 | ------------ 44 | .. autofunction:: leaky_relu 45 | 46 | Leaky ReLU6 47 | ------------ 48 | .. autofunction:: leaky_relu6 49 | 50 | Twice Leaky ReLU6 51 | ----------------- 52 | .. autofunction:: leaky_twice_relu6 53 | 54 | Swish 55 | ------------ 56 | .. autofunction:: swish 57 | 58 | Sign 59 | --------------------- 60 | .. autofunction:: sign 61 | 62 | Hard Tanh 63 | --------------------- 64 | .. autofunction:: hard_tanh 65 | 66 | Pixel-wise softmax 67 | -------------------- 68 | .. autofunction:: pixel_wise_softmax 69 | 70 | mish 71 | ------------- 72 | .. autofunciton:: mish 73 | 74 | 带有参数的激活函数 75 | ------------------------------ 76 | 请见神经网络层。 77 | -------------------------------------------------------------------------------- /docs/modules/array_ops.rst: -------------------------------------------------------------------------------- 1 | API - Array 操作 2 | ====================== 3 | 4 | .. automodule:: tensorlayer.array_ops 5 | 6 | .. autosummary:: 7 | 8 | alphas 9 | alphas_like 10 | 11 | Tensorflow Tensor 操作 12 | ---------------------------- 13 | 14 | tl.alphas 15 | ^^^^^^^^^ 16 | .. autofunction:: alphas 17 | 18 | tl.alphas_like 19 | ^^^^^^^^^^^^^^ 20 | .. autofunction:: alphas_like 21 | -------------------------------------------------------------------------------- /docs/modules/cli.rst: -------------------------------------------------------------------------------- 1 | CLI - 命令行界面 2 | ============================== 3 | 4 | .. automodule:: tensorlayer.cli 5 | 6 | .. automodule:: tensorlayer.cli.train 7 | -------------------------------------------------------------------------------- /docs/modules/cost.rst: -------------------------------------------------------------------------------- 1 | API - 损失函数 2 | ======================= 3 | 4 | 为了尽可能地保持TensorLayer的简洁性，我们最小化损失函数的数量。 5 | 因此我们鼓励直接使用TensorFlow官方的函数，比如你可以通过 ``tf.nn.l2_loss``, 6 | ``tf.contrib.layers.l1_regularizer``, ``tf.contrib.layers.l2_regularizer`` and 7 | ``tf.contrib.layers.sum_regularizer`` 来实现L1, L2 和 sum 规则化， 参考 `TensorFlow API `_。 8 | 9 | 10 | 自定义损失函数 11 | ----------------------- 12 | 13 | TensorLayer提供一个简单的方法来创建您自己的损失函数。 14 | 下面以多层神经网络(MLP)为例： 15 | 16 | .. code-block:: python 17 | 18 | network = tl.InputLayer(x, name='input_layer') 19 | network = tl.DropoutLayer(network, keep=0.8, name='drop1') 20 | network = tl.DenseLayer(network, n_units=800, act = tf.nn.relu, name='relu1') 21 | network = tl.DropoutLayer(network, keep=0.5, name='drop2') 22 | network = tl.DenseLayer(network, n_units=800, act = tf.nn.relu, name='relu2') 23 | network = tl.DropoutLayer(network, keep=0.5, name='drop3') 24 | network = tl.DenseLayer(network, n_units=10, act = tl.activation.identity, name='output_layer') 25 | 26 | 那么其模型参数为 ``[W1, b1, W2, b2, W_out, b_out]``， 27 | 这时，你可以像下面的例子那样实现对前两个weights矩阵的L2规则化。 28 | 29 | .. code-block:: python 30 | 31 | cost = tl.cost.cross_entropy(y, y_, name = 'cost') 32 | cost = cost + tf.contrib.layers.l2_regularizer(0.001)(network.all_params[0]) + tf.contrib.layers.l2_regularizer(0.001)(network.all_params[2]) 33 | 34 | 此外，TensorLayer 提供了通过给定名称，很方便地获取参数列表的方法，所以您可以如下对某些参数执行L2规则化。 35 | 36 | .. code-block:: python 37 | 38 | l2 = 0 39 | for w in tl.layers.get_variables_with_name('W_conv2d', train_only=True, printable=False):#[-3:]: 40 | l2 += tf.contrib.layers.l2_regularizer(1e-4)(w) 41 | cost = tl.cost.cross_entropy(y, y_, name = 'cost') + l2 42 | 43 | 44 | 45 | 权值的正则化 46 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 47 | 48 | 在初始化变量之后，网络参数的信息可以使用 ``network.print.params()`` 来获得。 49 | 50 | .. code-block:: python 51 | 52 | sess.run(tf.initialize_all_variables()) 53 | network.print_params() 54 | 55 | .. code-block:: text 56 | 57 | param 0: (784, 800) (mean: -0.000000, median: 0.000004 std: 0.035524) 58 | param 1: (800,) (mean: 0.000000, median: 0.000000 std: 0.000000) 59 | param 2: (800, 800) (mean: 0.000029, median: 0.000031 std: 0.035378) 60 | param 3: (800,) (mean: 0.000000, median: 0.000000 std: 0.000000) 61 | param 4: (800, 10) (mean: 0.000673, median: 0.000763 std: 0.049373) 62 | param 5: (10,) (mean: 0.000000, median: 0.000000 std: 0.000000) 63 | num of params: 1276810 64 | 65 | 网络的输出是 ``network.outputs`` ，那么交叉熵的可以被如下定义。 66 | 另外，要正则化权重， ``network.all_params`` 要包含网络的所有参数。 67 | 在这种情况下根据 ``network.print_params()`` 所展示的参数 0,1,...,5的值, ``network.all_params = [W1, b1, W2, b2, Wout, bout]`` 68 | 然后对W1和W2的最大范数正则化可以按如下进行： 69 | 70 | .. code-block:: python 71 | 72 | y = network.outputs 73 | # Alternatively, you can use tl.cost.cross_entropy(y, y_, name = 'cost') instead. 74 | cross_entropy = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(y, y_)) 75 | cost = cross_entropy 76 | cost = cost + tl.cost.maxnorm_regularizer(1.0)(network.all_params[0]) + 77 | tl.cost.maxnorm_regularizer(1.0)(network.all_params[2]) 78 | 79 | 另外，所有的TensorFlow的正则化函数，像 ``tf.contrib.layers.l2_regularizer`` 在TensorLayer中也能使用。 80 | 81 | 激活输出(Activation outputs)的规则化 82 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 83 | 84 | 实例方法 ``network.print_layers()`` 整齐地打印不同层的所有输出。 85 | 为了实现对激活输出的正则化，您可以使用 ``network.all_layers`` ，它包含了不同层的所有输出。 86 | 如果您想对第一层隐藏层的激活输出使用L1惩罚，仅仅需要添加 87 | ``tf.contrib.layers.l2_regularizer(lambda_l1)(network.all_layers[1])`` 到成本函数中。 88 | 89 | .. code-block:: python 90 | 91 | network.print_layers() 92 | 93 | .. code-block:: text 94 | 95 | layer 0: Tensor("dropout/mul_1:0", shape=(?, 784), dtype=float32) 96 | layer 1: Tensor("Relu:0", shape=(?, 800), dtype=float32) 97 | layer 2: Tensor("dropout_1/mul_1:0", shape=(?, 800), dtype=float32) 98 | layer 3: Tensor("Relu_1:0", shape=(?, 800), dtype=float32) 99 | layer 4: Tensor("dropout_2/mul_1:0", shape=(?, 800), dtype=float32) 100 | layer 5: Tensor("add_2:0", shape=(?, 10), dtype=float32) 101 | 102 | .. automodule:: tensorlayer.cost 103 | 104 | .. autosummary:: 105 | 106 | cross_entropy 107 | sigmoid_cross_entropy 108 | binary_cross_entropy 109 | mean_squared_error 110 | normalized_mean_square_error 111 | absolute_difference_error 112 | dice_coe 113 | dice_hard_coe 114 | iou_coe 115 | cross_entropy_seq 116 | cross_entropy_seq_with_mask 117 | cosine_similarity 118 | li_regularizer 119 | lo_regularizer 120 | maxnorm_regularizer 121 | maxnorm_o_regularizer 122 | maxnorm_i_regularizer 123 | 124 | Softmax cross entropy 125 | ---------------------- 126 | .. autofunction:: cross_entropy 127 | 128 | Sigmoid cross entropy 129 | ---------------------- 130 | .. autofunction:: sigmoid_cross_entropy 131 | 132 | Binary cross entropy 133 | ------------------------- 134 | .. autofunction:: binary_cross_entropy 135 | 136 | Mean squared error (L2) 137 | ----------------------- 138 | .. autofunction:: mean_squared_error 139 | 140 | Normalized mean square error 141 | --------------------------------- 142 | .. autofunction:: normalized_mean_square_error 143 | 144 | Absolute difference error (L1) 145 | --------------------------------- 146 | .. autofunction:: absolute_difference_error 147 | 148 | Dice coefficient 149 | ------------------- 150 | .. autofunction:: dice_coe 151 | 152 | Hard Dice coefficient 153 | ------------------------- 154 | .. autofunction:: dice_hard_coe 155 | 156 | IOU coefficient 157 | ----------------- 158 | .. autofunction:: iou_coe 159 | 160 | Cross entropy for sequence 161 | ----------------------------- 162 | .. autofunction:: cross_entropy_seq 163 | 164 | Cross entropy with mask for sequence 165 | --------------------------------------- 166 | .. autofunction:: cross_entropy_seq_with_mask 167 | 168 | Cosine similarity 169 | ------------------- 170 | .. autofunction:: cosine_similarity 171 | 172 | 173 | 规则化函数 174 | -------------------------- 175 | 176 | 更多 ``tf.nn.l2_loss``, ``tf.contrib.layers.l1_regularizer``, ``tf.contrib.layers.l2_regularizer`` 与 177 | ``tf.contrib.layers.sum_regularizer``, 请见 `TensorFlow API `_. 178 | 179 | Maxnorm 180 | ^^^^^^^^^^ 181 | .. autofunction:: maxnorm_regularizer 182 | 183 | Special 184 | ^^^^^^^^^^ 185 | .. autofunction:: li_regularizer 186 | .. autofunction:: lo_regularizer 187 | .. autofunction:: maxnorm_o_regularizer 188 | .. autofunction:: maxnorm_i_regularizer 189 | -------------------------------------------------------------------------------- /docs/modules/db.rst: -------------------------------------------------------------------------------- 1 | API - 数据库 2 | ====================== 3 | 4 | Alpha 版本的数据管理系统已经发布，详情请见 `英文文档 `_ . 5 | -------------------------------------------------------------------------------- /docs/modules/distributed.rst: -------------------------------------------------------------------------------- 1 | API - 分布式 2 | ====================== 3 | 4 | 分布式训练的帮助sessions和方法，请参考 `mnist例子 `_。 5 | 6 | .. automodule:: tensorlayer.distributed 7 | 8 | .. autosummary:: 9 | 10 | TaskSpecDef 11 | TaskSpec 12 | DistributedSession 13 | StopAtTimeHook 14 | LoadCheckpoint 15 | 16 | 分布式训练 17 | ---------------------- 18 | 19 | TaskSpecDef 20 | ^^^^^^^^^^^^^^^^^^^^^^ 21 | 22 | .. autofunction:: TaskSpecDef 23 | 24 | Create TaskSpecDef from environment variables 25 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 26 | 27 | .. autofunction:: TaskSpec 28 | 29 | Distributed Session object 30 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 31 | 32 | .. autofunction:: DistributedSession 33 | 34 | Data sharding 35 | ^^^^^^^^^^^^^^^^^^^^^^ 36 | 37 | 我们希望把数据分开很多块，放到每一个训练服务器上，而不是把整个数据放到所有的服务器上。 38 | TensorFlow >= 1.4 提供了一些帮助类（helper classes）来支持数据分区功能（data sharding）: `Datasets `_ 39 | 40 | 值得注意的是，在数据切分时，数据打乱非常重要，这些操作在建立shards的时候自动完成： 41 | 42 | .. code-block:: python 43 | 44 | from tensorflow.contrib.data import TextLineDataset 45 | from tensorflow.contrib.data import Dataset 46 | 47 | task_spec = TaskSpec() 48 | files_dataset = Dataset.list_files(files_pattern) 49 | dataset = TextLineDataset(files_dataset) 50 | dataset = dataset.map(your_python_map_function, num_threads=4) 51 | if task_spec is not None: 52 | dataset = dataset.shard(task_spec.num_workers, task_spec.shard_index) 53 | dataset = dataset.shuffle(buffer_size) 54 | dataset = dataset.batch(batch_size) 55 | dataset = dataset.repeat(num_epochs) 56 | iterator = dataset.make_one_shot_iterator() 57 | next_element = iterator.get_next() 58 | with tf.device(task_spec.device_fn()): 59 | tensors = create_graph(next_element) 60 | with tl.DistributedSession(task_spec=task_spec, 61 | checkpoint_dir='/tmp/ckpt') as session: 62 | while not session.should_stop(): 63 | session.run(tensors) 64 | 65 | 66 | Logging 67 | ^^^^^^^^^^^^^^^^^^^^^^ 68 | 69 | 我们可以使用task_spec来对主服务器（master server）做日志记录： 70 | 71 | .. code-block:: python 72 | 73 | while not session.should_stop(): 74 | should_log = task_spec.is_master() and your_conditions 75 | if should_log: 76 | results = session.run(tensors_with_log_info) 77 | logging.info(...) 78 | else: 79 | results = session.run(tensors) 80 | 81 | Continuous evaluation 82 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 83 | 84 | 我们可以使用其中一台子服务器（worker）来一直对保存下来对checkpoint做评估： 85 | 86 | .. code-block:: python 87 | 88 | import tensorflow as tf 89 | from tensorflow.python.training import session_run_hook 90 | from tensorflow.python.training.monitored_session import SingularMonitoredSession 91 | 92 | class Evaluator(session_run_hook.SessionRunHook): 93 | def __init__(self, checkpoints_path, output_path): 94 | self.checkpoints_path = checkpoints_path 95 | self.summary_writer = tf.summary.FileWriter(output_path) 96 | self.lastest_checkpoint = '' 97 | 98 | def after_create_session(self, session, coord): 99 | checkpoint = tf.train.latest_checkpoint(self.checkpoints_path) 100 | # wait until a new check point is available 101 | while self.lastest_checkpoint == checkpoint: 102 | time.sleep(30) 103 | checkpoint = tf.train.latest_checkpoint(self.checkpoints_path) 104 | self.saver.restore(session, checkpoint) 105 | self.lastest_checkpoint = checkpoint 106 | 107 | def end(self, session): 108 | super(Evaluator, self).end(session) 109 | # save summaries 110 | step = int(self.lastest_checkpoint.split('-')[-1]) 111 | self.summary_writer.add_summary(self.summary, step) 112 | 113 | def _create_graph(): 114 | # your code to create the graph with the dataset 115 | 116 | def run_evaluation(): 117 | with tf.Graph().as_default(): 118 | summary_tensors = create_graph() 119 | self.saver = tf.train.Saver(var_list=tf_variables.trainable_variables()) 120 | hooks = self.create_hooks() 121 | hooks.append(self) 122 | if self.max_time_secs and self.max_time_secs > 0: 123 | hooks.append(StopAtTimeHook(self.max_time_secs)) 124 | # this evaluation runs indefinitely, until the process is killed 125 | while True: 126 | with SingularMonitoredSession(hooks=[self]) as session: 127 | try: 128 | while not sess.should_stop(): 129 | self.summary = session.run(summary_tensors) 130 | except OutOfRangeError: 131 | pass 132 | # end of evaluation 133 | 134 | task_spec = TaskSpec().user_last_worker_as_evaluator() 135 | if task_spec.is_evaluator(): 136 | Evaluator().run_evaluation() 137 | else: 138 | # run normal training 139 | 140 | 141 | 142 | Session Hooks 143 | ---------------------- 144 | 145 | TensorFlow提供了一些 `Session Hooks `_ 146 | 来对sessions做操作，我们在这里加更多的helper来实现更多的常规操作。 147 | 148 | Stop after maximum time 149 | ^^^^^^^^^^^^^^^^^^^^^^ 150 | 151 | .. autofunction:: StopAtTimeHook 152 | 153 | Initialize network with checkpoint 154 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 155 | 156 | .. autofunction:: LoadCheckpoint 157 | -------------------------------------------------------------------------------- /docs/modules/files.rst: -------------------------------------------------------------------------------- 1 | API - 文件 2 | ======================== 3 | 4 | .. automodule:: tensorlayer.files 5 | 6 | .. autosummary:: 7 | 8 | load_mnist_dataset 9 | load_fashion_mnist_dataset 10 | load_cifar10_dataset 11 | load_cropped_svhn 12 | load_ptb_dataset 13 | load_matt_mahoney_text8_dataset 14 | load_imdb_dataset 15 | load_nietzsche_dataset 16 | load_wmt_en_fr_dataset 17 | load_flickr25k_dataset 18 | load_flickr1M_dataset 19 | load_cyclegan_dataset 20 | load_celebA_dataset 21 | load_voc_dataset 22 | load_mpii_pose_dataset 23 | download_file_from_google_drive 24 | 25 | save_npz 26 | load_npz 27 | 28 | load_and_assign_npz 29 | save_npz_dict 30 | load_and_assign_npz_dict 31 | save_ckpt 32 | load_ckpt 33 | 34 | save_any_to_npy 35 | load_npy_to_any 36 | 37 | file_exists 38 | folder_exists 39 | del_file 40 | del_folder 41 | read_file 42 | load_file_list 43 | load_folder_list 44 | exists_or_mkdir 45 | maybe_download_and_extract 46 | 47 | natural_keys 48 | 49 | npz_to_W_pdf 50 | 51 | 下载数据集 52 | ------------------------ 53 | 54 | MNIST 55 | ^^^^^^^ 56 | .. autofunction:: load_mnist_dataset 57 | 58 | Fashion-MNIST 59 | ^^^^^^^^^^^^^^^^ 60 | .. autofunction:: load_fashion_mnist_dataset 61 | 62 | CIFAR-10 63 | ^^^^^^^^^^^^ 64 | .. autofunction:: load_cifar10_dataset 65 | 66 | SVHN 67 | ^^^^^^^^^^^^ 68 | .. autofunction:: load_cropped_svhn 69 | 70 | Penn TreeBank (PTB) 71 | ^^^^^^^^^^^^^^^^^^^^^ 72 | .. autofunction:: load_ptb_dataset 73 | 74 | Matt Mahoney's text8 75 | ^^^^^^^^^^^^^^^^^^^^^^^ 76 | .. autofunction:: load_matt_mahoney_text8_dataset 77 | 78 | IMBD 79 | ^^^^^^^ 80 | .. autofunction:: load_imdb_dataset 81 | 82 | Nietzsche 83 | ^^^^^^^^^^^^^^ 84 | .. autofunction:: load_nietzsche_dataset 85 | 86 | WMT'15 Website 的英文译法文数据 87 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 88 | 89 | .. autofunction:: load_wmt_en_fr_dataset 90 | 91 | Flickr25k 92 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 93 | .. autofunction:: load_flickr25k_dataset 94 | 95 | Flickr1M 96 | ^^^^^^^^^^^^^^^^^^^^^^^^^ 97 | .. autofunction:: load_flickr1M_dataset 98 | 99 | CycleGAN 100 | ^^^^^^^^^^^^^^^^^^^^^^^^^ 101 | .. autofunction:: load_cyclegan_dataset 102 | 103 | CelebA 104 | ^^^^^^^^^ 105 | .. autofunction:: load_celebA_dataset 106 | 107 | VOC 2007/2012 108 | ^^^^^^^^^^^^^^^^ 109 | .. autofunction:: load_voc_dataset 110 | 111 | MPII 112 | ^^^^^^ 113 | .. autofunction:: load_mpii_pose_dataset 114 | 115 | Google Drive 116 | ^^^^^^^^^^^^^^^^ 117 | .. autofunction:: download_file_from_google_drive 118 | 119 | 保存与加载模型 120 | ---------------------- 121 | 122 | 以列表保存模型到 .npz 123 | ^^^^^^^^^^^^^^^^^^^^^^^^ 124 | .. autofunction:: save_npz 125 | 126 | 从save_npz加载模型参数列表 127 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 128 | .. autofunction:: load_npz 129 | 130 | 131 | 从.npz中加载参数并导入模型 132 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 133 | .. autofunction:: load_and_assign_npz 134 | 135 | 136 | 以字典保存模型到 .npz 137 | ^^^^^^^^^^^^^^^^^^^^^^^^ 138 | .. autofunction:: save_npz_dict 139 | 140 | 从save_npz_dict加载模型参数列表 141 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 142 | .. autofunction:: load_and_assign_npz_dict 143 | 144 | 145 | 以列表保存模型到 .ckpt 146 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 147 | .. autofunction:: save_ckpt 148 | 149 | 从.ckpt中加载参数并导入模型 150 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 151 | .. autofunction:: load_ckpt 152 | 153 | 154 | 155 | 156 | 157 | 保存与加载数据 158 | ------------------------ 159 | 160 | 保持数据到.npy文件 161 | ^^^^^^^^^^^^^^^^^^^^^ 162 | .. autofunction:: save_any_to_npy 163 | 164 | 从.npy文件加载数据 165 | ^^^^^^^^^^^^^^^^^^^^^^ 166 | .. autofunction:: load_npy_to_any 167 | 168 | 169 | 170 | 文件夹/文件相关函数 171 | ----------------------------- 172 | 173 | 判断文件存在 174 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 175 | .. autofunction:: file_exists 176 | 177 | 判断文件夹存在 178 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 179 | .. autofunction:: folder_exists 180 | 181 | 删除文件 182 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 183 | .. autofunction:: del_file 184 | 185 | 删除文件夹 186 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 187 | .. autofunction:: del_folder 188 | 189 | 读取文件 190 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 191 | .. autofunction:: read_file 192 | 193 | 从文件夹中读取文件名列表 194 | ^^^^^^^^^^^^^^^^^^^^^^^^ 195 | .. autofunction:: load_file_list 196 | 197 | 从文件夹中读取文件夹列表 198 | ^^^^^^^^^^^^^^^^^^^^^^^^ 199 | .. autofunction:: load_folder_list 200 | 201 | 查看或建立文件夹 202 | ^^^^^^^^^^^^^^^^^^ 203 | .. autofunction:: exists_or_mkdir 204 | 205 | 下载或解压 206 | ^^^^^^^^^^^^^^^^^^^^^^^^^ 207 | .. autofunction:: maybe_download_and_extract 208 | 209 | 210 | 排序 211 | ------------ 212 | 213 | 字符串按数字排序 214 | ^^^^^^^^^^^^^^^^^^^ 215 | .. autofunction:: natural_keys 216 | 217 | 218 | 可视化 npz 文件 219 | ---------------------- 220 | .. autofunction:: npz_to_W_pdf 221 | -------------------------------------------------------------------------------- /docs/modules/iterate.rst: -------------------------------------------------------------------------------- 1 | API - 迭代函数 2 | ========================== 3 | 4 | 数据迭代。 5 | 6 | 7 | .. automodule:: tensorlayer.iterate 8 | 9 | .. autosummary:: 10 | 11 | minibatches 12 | seq_minibatches 13 | seq_minibatches2 14 | ptb_iterator 15 | 16 | 17 | 非时间序列 18 | -------------------- 19 | 20 | .. autofunction:: minibatches 21 | 22 | 时间序列 23 | -------------------- 24 | 25 | Sequence iteration 1 26 | ^^^^^^^^^^^^^^^^^^^^^^^ 27 | .. autofunction:: seq_minibatches 28 | 29 | Sequence iteration 2 30 | ^^^^^^^^^^^^^^^^^^^^^^^ 31 | .. autofunction:: seq_minibatches2 32 | 33 | PTB dataset iteration 34 | ^^^^^^^^^^^^^^^^^^^^^^^^ 35 | .. autofunction:: ptb_iterator 36 | -------------------------------------------------------------------------------- /docs/modules/layers.rst: -------------------------------------------------------------------------------- 1 | API - 神经网络层 2 | ========================= 3 | 4 | 5 | .. ----------------------------------------------------------- 6 | .. Layer List 7 | .. ----------------------------------------------------------- 8 | 9 | 神经网络层列表 10 | --------------------------------- 11 | 12 | .. automodule:: tensorlayer.layers 13 | 14 | .. autosummary:: 15 | 16 | Layer 17 | 18 | Input 19 | 20 | OneHot 21 | Word2vecEmbedding 22 | Embedding 23 | AverageEmbedding 24 | 25 | Dense 26 | Dropout 27 | GaussianNoise 28 | DropconnectDense 29 | 30 | UpSampling2d 31 | DownSampling2d 32 | 33 | Conv1d 34 | Conv2d 35 | Conv3d 36 | DeConv2d 37 | DeConv3d 38 | DepthwiseConv2d 39 | SeparableConv1d 40 | SeparableConv2d 41 | DeformableConv2d 42 | GroupConv2d 43 | 44 | PadLayer 45 | PoolLayer 46 | ZeroPad1d 47 | ZeroPad2d 48 | ZeroPad3d 49 | MaxPool1d 50 | MeanPool1d 51 | MaxPool2d 52 | MeanPool2d 53 | MaxPool3d 54 | MeanPool3d 55 | GlobalMaxPool1d 56 | GlobalMeanPool1d 57 | GlobalMaxPool2d 58 | GlobalMeanPool2d 59 | GlobalMaxPool3d 60 | GlobalMeanPool3d 61 | CornerPool2d 62 | 63 | SubpixelConv1d 64 | SubpixelConv2d 65 | 66 | SpatialTransformer2dAffine 67 | transformer 68 | batch_transformer 69 | 70 | BatchNorm 71 | BatchNorm1d 72 | BatchNorm2d 73 | BatchNorm3d 74 | LocalResponseNorm 75 | InstanceNorm 76 | InstanceNorm1d 77 | InstanceNorm2d 78 | InstanceNorm3d 79 | LayerNorm 80 | GroupNorm 81 | SwitchNorm 82 | 83 | RNN 84 | SimpleRNN 85 | GRURNN 86 | LSTMRNN 87 | BiRNN 88 | 89 | retrieve_seq_length_op 90 | retrieve_seq_length_op2 91 | retrieve_seq_length_op3 92 | target_mask_op 93 | 94 | Flatten 95 | Reshape 96 | Transpose 97 | Shuffle 98 | 99 | Lambda 100 | 101 | Concat 102 | Elementwise 103 | ElementwiseLambda 104 | 105 | ExpandDims 106 | Tile 107 | 108 | Stack 109 | UnStack 110 | 111 | Sign 112 | Scale 113 | BinaryDense 114 | BinaryConv2d 115 | TernaryDense 116 | TernaryConv2d 117 | DorefaDense 118 | DorefaConv2d 119 | QuantizedDense 120 | QuantizedDenseWithBN 121 | QuantizedConv2d 122 | QuantizedConv2dWithBN 123 | 124 | PRelu 125 | PRelu6 126 | PTRelu6 127 | 128 | flatten_reshape 129 | initialize_rnn_state 130 | list_remove_repeat 131 | 132 | .. ----------------------------------------------------------- 133 | .. Basic Layers 134 | .. ----------------------------------------------------------- 135 | 136 | 层基础类 137 | ----------- 138 | 139 | .. autoclass:: Layer 140 | 141 | .. ----------------------------------------------------------- 142 | .. Input Layer 143 | .. ----------------------------------------------------------- 144 | 145 | 输入层 146 | --------------- 147 | 148 | 普通输入层 149 | ^^^^^^^^^^^^^^^^ 150 | .. autofunction:: Input 151 | 152 | .. ----------------------------------------------------------- 153 | .. Embedding Layers 154 | .. ----------------------------------------------------------- 155 | 156 | One-hot 输入层 157 | ^^^^^^^^^^^^^^^^^^^^ 158 | .. autoclass:: OneHot 159 | 160 | Word2Vec Embedding 输入层 161 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 162 | .. autoclass:: Word2vecEmbedding 163 | 164 | Embedding 输入层 165 | ^^^^^^^^^^^^^^^^^^^^^^^ 166 | .. autoclass:: Embedding 167 | 168 | Average Embedding 输入层 169 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 170 | .. autoclass:: AverageEmbedding 171 | 172 | .. ----------------------------------------------------------- 173 | .. Activation Layers 174 | .. ----------------------------------------------------------- 175 | 176 | 177 | 有参数激活函数层 178 | --------------------------- 179 | 180 | PReLU 层 181 | ^^^^^^^^^^^^^^^^^ 182 | .. autoclass:: PRelu 183 | 184 | 185 | PReLU6 层 186 | ^^^^^^^^^^^^^^^^^^ 187 | .. autoclass:: PRelu6 188 | 189 | 190 | PTReLU6 层 191 | ^^^^^^^^^^^^^^^^^^^ 192 | .. autoclass:: PTRelu6 193 | 194 | 195 | .. ----------------------------------------------------------- 196 | .. Convolutional Layers 197 | .. ----------------------------------------------------------- 198 | 199 | 卷积层 200 | --------------------- 201 | 202 | 卷积层 203 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 204 | 205 | Conv1d 206 | """"""""""""""""""""" 207 | .. autoclass:: Conv1d 208 | 209 | Conv2d 210 | """"""""""""""""""""" 211 | .. autoclass:: Conv2d 212 | 213 | Conv3d 214 | """"""""""""""""""""" 215 | .. autoclass:: Conv3d 216 | 217 | 反卷积层 218 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 219 | 220 | DeConv2d 221 | """"""""""""""""""""" 222 | .. autoclass:: DeConv2d 223 | 224 | DeConv3d 225 | """"""""""""""""""""" 226 | .. autoclass:: DeConv3d 227 | 228 | 229 | Deformable 卷积层 230 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 231 | 232 | DeformableConv2d 233 | """"""""""""""""""""" 234 | .. autoclass:: DeformableConv2d 235 | 236 | 237 | Depthwise 卷积层 238 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 239 | 240 | DepthwiseConv2d 241 | """"""""""""""""""""" 242 | .. autoclass:: DepthwiseConv2d 243 | 244 | 245 | Group 卷积层 246 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 247 | 248 | GroupConv2d 249 | """"""""""""""""""""" 250 | .. autoclass:: GroupConv2d 251 | 252 | 253 | Separable 卷积层 254 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 255 | 256 | SeparableConv1d 257 | """"""""""""""""""""" 258 | .. autoclass:: SeparableConv1d 259 | 260 | SeparableConv2d 261 | """"""""""""""""""""" 262 | .. autoclass:: SeparableConv2d 263 | 264 | 265 | SubPixel 卷积层 266 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 267 | 268 | SubpixelConv1d 269 | """"""""""""""""""""" 270 | .. autoclass:: SubpixelConv1d 271 | 272 | SubpixelConv2d 273 | """"""""""""""""""""" 274 | .. autoclass:: SubpixelConv2d 275 | 276 | 277 | .. ----------------------------------------------------------- 278 | .. Dense Layers 279 | .. ----------------------------------------------------------- 280 | 281 | 全连接层 282 | ------------- 283 | 284 | 全连接层 285 | 286 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 287 | .. autoclass:: Dense 288 | 289 | Drop Connection 全连接层 290 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 291 | .. autoclass:: DropconnectDense 292 | 293 | 294 | .. ----------------------------------------------------------- 295 | .. Dropout Layer 296 | .. ----------------------------------------------------------- 297 | 298 | Dropout 层 299 | ------------------- 300 | .. autoclass:: Dropout 301 | 302 | .. ----------------------------------------------------------- 303 | .. Extend Layers 304 | .. ----------------------------------------------------------- 305 | 306 | 拓展层 307 | ------------------- 308 | 309 | Expand Dims 层 310 | ^^^^^^^^^^^^^^^^^^^^ 311 | .. autoclass:: ExpandDims 312 | 313 | 314 | Tile 层 315 | ^^^^^^^^^^^^^^^^^^^^ 316 | .. autoclass:: Tile 317 | 318 | .. ----------------------------------------------------------- 319 | .. Image Resampling Layers 320 | .. ----------------------------------------------------------- 321 | 322 | 图像重采样层 323 | ------------------------- 324 | 325 | 2D 上采样层 326 | ^^^^^^^^^^^^^^^^^^^^^^^ 327 | .. autoclass:: UpSampling2d 328 | 329 | 2D 下采样层 330 | ^^^^^^^^^^^^^^^^^^^^^^^ 331 | .. autoclass:: DownSampling2d 332 | 333 | .. ----------------------------------------------------------- 334 | .. Lambda Layer 335 | .. ----------------------------------------------------------- 336 | 337 | Lambda 层 338 | --------------- 339 | 340 | 普通 Lambda 层 341 | ^^^^^^^^^^^^^^^^^^^ 342 | .. autoclass:: Lambda 343 | 344 | 逐点 Lambda 层 345 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 346 | .. autoclass:: ElementwiseLambda 347 | 348 | .. ----------------------------------------------------------- 349 | .. Merge Layer 350 | .. ----------------------------------------------------------- 351 | 352 | 合并层 353 | --------------- 354 | 355 | 合并连接层 356 | ^^^^^^^^^^^^^^^^^^^ 357 | .. autoclass:: Concat 358 | 359 | 逐点合并层 360 | ^^^^^^^^^^^^^^^^^^^ 361 | .. autoclass:: Elementwise 362 | 363 | .. ----------------------------------------------------------- 364 | .. Noise Layers 365 | .. ----------------------------------------------------------- 366 | 367 | 噪声层 368 | --------------- 369 | .. autoclass:: GaussianNoise 370 | 371 | .. ----------------------------------------------------------- 372 | .. Normalization Layers 373 | .. ----------------------------------------------------------- 374 | 375 | 标准化层 376 | -------------------- 377 | 378 | Batch 标准化层 379 | ^^^^^^^^^^^^^^^^^^^^^^ 380 | .. autoclass:: BatchNorm 381 | 382 | Batch1d 标准化层 383 | ^^^^^^^^^^^^^^^^^^^^^^ 384 | .. autoclass:: BatchNorm1d 385 | 386 | Batch2d 标准化层 387 | ^^^^^^^^^^^^^^^^^^^^^^ 388 | .. autoclass:: BatchNorm2d 389 | 390 | Batch3d 标准化层 391 | ^^^^^^^^^^^^^^^^^^^^^^ 392 | .. autoclass:: BatchNorm3d 393 | 394 | Local Response 标准化层 395 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 396 | .. autoclass:: LocalResponseNorm 397 | 398 | Instance 标准化层 399 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 400 | .. autoclass:: InstanceNorm 401 | 402 | Instance1d 标准化层 403 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 404 | .. autoclass:: InstanceNorm1d 405 | 406 | Instance2d 标准化层 407 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 408 | .. autoclass:: InstanceNorm2d 409 | 410 | Instance3d 标准化层 411 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 412 | .. autoclass:: InstanceNorm3d 413 | 414 | Layer 标准化层 415 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 416 | .. autoclass:: LayerNorm 417 | 418 | Group 标准化层 419 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 420 | .. autoclass:: GroupNorm 421 | 422 | Switch 标准化层 423 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 424 | .. autoclass:: SwitchNorm 425 | 426 | .. ----------------------------------------------------------- 427 | .. Padding Layers 428 | .. ----------------------------------------------------------- 429 | 430 | 填充层 431 | ------------------------ 432 | 433 | 填充层 (底层 API) 434 | ^^^^^^^^^^^^^^^^^^^^^^^^^ 435 | 436 | .. autoclass:: PadLayer 437 | 438 | 1D Zero 填充层 439 | ^^^^^^^^^^^^^^^^^^^ 440 | .. autoclass:: ZeroPad1d 441 | 442 | 2D Zero 填充层 443 | ^^^^^^^^^^^^^^^^^^^ 444 | .. autoclass:: ZeroPad2d 445 | 446 | 3D Zero 填充层 447 | ^^^^^^^^^^^^^^^^^^^ 448 | .. autoclass:: ZeroPad3d 449 | 450 | .. ----------------------------------------------------------- 451 | .. Pooling Layers 452 | .. ----------------------------------------------------------- 453 | 454 | 455 | 池化层 456 | ------------------------ 457 | 458 | 池化层 (底层 API) 459 | ^^^^^^^^^^^^^^^^^^^^^^^^^ 460 | 461 | .. autoclass:: PoolLayer 462 | 463 | 1D Max 池化层 464 | ^^^^^^^^^^^^^^^^^^^ 465 | .. autoclass:: MaxPool1d 466 | 467 | 1D Mean 池化层 468 | ^^^^^^^^^^^^^^^^^^^ 469 | .. autoclass:: MeanPool1d 470 | 471 | 2D Max 池化层 472 | ^^^^^^^^^^^^^^^^^^^ 473 | .. autoclass:: MaxPool2d 474 | 475 | 2D Mean 池化层 476 | ^^^^^^^^^^^^^^^^^^^ 477 | .. autoclass:: MeanPool2d 478 | 479 | 3D Max 池化层 480 | ^^^^^^^^^^^^^^^^^^^ 481 | .. autoclass:: MaxPool3d 482 | 483 | 3D Mean 池化层 484 | ^^^^^^^^^^^^^^^^^^^ 485 | .. autoclass:: MeanPool3d 486 | 487 | 1D Global Max 池化层 488 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 489 | .. autoclass:: GlobalMaxPool1d 490 | 491 | 1D Global Mean 池化层 492 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 493 | .. autoclass:: GlobalMeanPool1d 494 | 495 | 2D Global Max 池化层 496 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 497 | .. autoclass:: GlobalMaxPool2d 498 | 499 | 2D Global Mean 池化层 500 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 501 | .. autoclass:: GlobalMeanPool2d 502 | 503 | 3D Global Max 池化层 504 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 505 | .. autoclass:: GlobalMaxPool3d 506 | 507 | 3D Global Mean 池化层 508 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 509 | .. autoclass:: GlobalMeanPool3d 510 | 511 | 2D Corner 池化层 512 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 513 | .. autoclass:: CornerPool2d 514 | 515 | .. ----------------------------------------------------------- 516 | .. Quantized Layers 517 | .. ----------------------------------------------------------- 518 | 519 | 量化网络层 520 | ------------------ 521 | 522 | 这些层目前还是用矩阵实现的运算，未来我们将提供 bit-count 操作，以实现加速。 523 | 524 | Sign 525 | ^^^^^^^^^^^^^^ 526 | .. autoclass:: Sign 527 | 528 | Scale 529 | ^^^^^^^^^^^^^^ 530 | .. autoclass:: Scale 531 | 532 | Binary 全连接层 533 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 534 | .. autoclass:: BinaryDense 535 | 536 | Binary (De)卷积层 537 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 538 | 539 | BinaryConv2d 540 | """"""""""""""""""""" 541 | .. autoclass:: BinaryConv2d 542 | 543 | Ternary 全连接层 544 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 545 | 546 | TernaryDense 547 | """"""""""""""""""""" 548 | .. autoclass:: TernaryDense 549 | 550 | Ternary 卷积层 551 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 552 | 553 | TernaryConv2d 554 | """"""""""""""""""""" 555 | .. autoclass:: TernaryConv2d 556 | 557 | DoReFa 卷积层 558 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 559 | 560 | DorefaConv2d 561 | """"""""""""""""""""" 562 | .. autoclass:: DorefaConv2d 563 | 564 | DoReFa 卷积层 565 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 566 | 567 | DorefaConv2d 568 | """"""""""""""""""""" 569 | .. autoclass:: DorefaConv2d 570 | 571 | Quantization 全连接层 572 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 573 | 574 | QuantizedDense 575 | """"""""""""""""""""" 576 | .. autoclass:: QuantizedDense 577 | 578 | QuantizedDenseWithBN 全连接层+批标准化 579 | """""""""""""""""""""""""""""""""""" 580 | .. autoclass:: QuantizedDenseWithBN 581 | 582 | Quantization 卷积层 583 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 584 | 585 | Quantization 586 | """"""""""""""""""""" 587 | .. autoclass:: QuantizedConv2d 588 | 589 | QuantizedConv2dWithBN 590 | """"""""""""""""""""" 591 | .. autoclass:: QuantizedConv2dWithBN 592 | 593 | 594 | .. ----------------------------------------------------------- 595 | .. Recurrent Layers 596 | .. ----------------------------------------------------------- 597 | 598 | 循环层 599 | --------------------- 600 | 601 | 普通循环层 602 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 603 | 604 | RNN 层 605 | """""""""""""""""""""""""" 606 | .. autoclass:: RNN 607 | 608 | 609 | 基本RNN层 （使用简单循环单元） 610 | """"""""""""""""""""""""""""""" 611 | .. autoclass:: SimpleRNN 612 | 613 | 614 | 基于GRU的RNN层（使用GRU循环单元） 615 | """""""""""""""""""""""""""""""" 616 | .. autoclass:: GRURNN 617 | 618 | 基于LSTM的RNN层（使用LSTM循环单元） 619 | """""""""""""""""""""""""""""""" 620 | .. autoclass:: LSTMRNN 621 | 622 | Bidirectional 层 623 | """"""""""""""""""""""""""""""""" 624 | .. autoclass:: BiRNN 625 | 626 | 627 | 计算步长函数 628 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 629 | 630 | 方法 1 631 | """""""""""""""""""""""""" 632 | .. autofunction:: retrieve_seq_length_op 633 | 634 | 方法 2 635 | """""""""""""""""""""""""" 636 | .. autofunction:: retrieve_seq_length_op2 637 | 638 | 方法 3 639 | """""""""""""""""""""""""" 640 | .. autofunction:: retrieve_seq_length_op3 641 | 642 | 643 | 方法 4 644 | """"""""""""""""""""""""""" 645 | .. autofunction:: target_mask_op 646 | 647 | 648 | .. ----------------------------------------------------------- 649 | .. Shape Layers 650 | .. ----------------------------------------------------------- 651 | 652 | 形状修改层 653 | ------------ 654 | 655 | Flatten 层 656 | ^^^^^^^^^^^^^^^ 657 | .. autoclass:: Flatten 658 | 659 | Reshape 层 660 | ^^^^^^^^^^^^^^^ 661 | .. autoclass:: Reshape 662 | 663 | Transpose 层 664 | ^^^^^^^^^^^^^^^^^ 665 | .. autoclass:: Transpose 666 | 667 | Shuffle 层 668 | ^^^^^^^^^^^^^^^^^ 669 | .. autoclass:: Shuffle 670 | 671 | .. ----------------------------------------------------------- 672 | .. Spatial Transformer Layers 673 | .. ----------------------------------------------------------- 674 | 675 | 空间变换层 676 | ----------------------- 677 | 678 | 2D Affine Transformation 层 679 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 680 | .. autoclass:: SpatialTransformer2dAffine 681 | 682 | 2D Affine Transformation 函数 683 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 684 | .. autofunction:: transformer 685 | 686 | Batch 2D Affine Transformation 函数 687 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 688 | .. autofunction:: batch_transformer 689 | 690 | .. ----------------------------------------------------------- 691 | .. Stack Layers 692 | .. ----------------------------------------------------------- 693 | 694 | 堆叠层 695 | ------------- 696 | 697 | 堆叠层 698 | ^^^^^^^^^^^^^^ 699 | .. autoclass:: Stack 700 | 701 | 反堆叠层 702 | ^^^^^^^^^^^^^^^ 703 | .. autoclass:: UnStack 704 | 705 | 706 | .. ----------------------------------------------------------- 707 | .. Helper Functions 708 | .. ----------------------------------------------------------- 709 | 710 | 帮助函数 711 | ------------------------ 712 | 713 | Flatten 函数 714 | ^^^^^^^^^^^^^^^^^ 715 | .. autofunction:: flatten_reshape 716 | 717 | 初始化 循环层 状态 718 | ^^^^^^^^^^^^^^^^^^^^^^^^^ 719 | .. autofunction:: initialize_rnn_state 720 | 721 | 去除列表中重复元素 722 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 723 | .. autofunction:: list_remove_repeat 724 | 725 | -------------------------------------------------------------------------------- /docs/modules/models.rst: -------------------------------------------------------------------------------- 1 | API - 预训练模型 2 | ================================ 3 | 4 | TensorLayer 提供了一些预训练模型，通过这套API，您可以非常容易地使用整个或者部分网络。 5 | 6 | .. automodule:: tensorlayer.models 7 | 8 | .. autosummary:: 9 | 10 | Model 11 | 12 | VGG16 13 | VGG19 14 | SqueezeNetV1 15 | MobileNetV1 16 | 17 | Seq2seq 18 | Seq2seqLuongAttention 19 | 20 | 21 | 模型基类 22 | ------------------------- 23 | 24 | .. autoclass:: Model 25 | 26 | 27 | VGG16 28 | ---------------------- 29 | 30 | .. autofunction:: VGG16 31 | 32 | VGG19 33 | ---------------------- 34 | 35 | .. autofunction:: VGG19 36 | 37 | SqueezeNetV1 38 | ---------------- 39 | .. autofunction:: SqueezeNetV1 40 | 41 | MobileNetV1 42 | ---------------- 43 | 44 | .. autofunction:: MobileNetV1 45 | 46 | 47 | Seq2seq 48 | -------------------------- 49 | 50 | .. autoclass:: Seq2seq 51 | 52 | 使用Luong注意力机制的Seq2seq 53 | ------------------------------- 54 | 55 | .. autoclass:: Seq2seqLuongAttention 56 | -------------------------------------------------------------------------------- /docs/modules/nlp.rst: -------------------------------------------------------------------------------- 1 | API - 自然语言处理 2 | ====================== 3 | 4 | 自然语言处理与词向量。 5 | 6 | .. automodule:: tensorlayer.nlp 7 | 8 | .. autosummary:: 9 | 10 | generate_skip_gram_batch 11 | 12 | sample 13 | sample_top 14 | 15 | SimpleVocabulary 16 | Vocabulary 17 | process_sentence 18 | create_vocab 19 | 20 | simple_read_words 21 | read_words 22 | read_analogies_file 23 | build_vocab 24 | build_reverse_dictionary 25 | build_words_dataset 26 | save_vocab 27 | 28 | words_to_word_ids 29 | word_ids_to_words 30 | 31 | basic_tokenizer 32 | create_vocabulary 33 | initialize_vocabulary 34 | sentence_to_token_ids 35 | data_to_token_ids 36 | 37 | moses_multi_bleu 38 | 39 | 训练嵌入矩阵的迭代函数 40 | ------------------------------ 41 | 42 | .. autofunction:: generate_skip_gram_batch 43 | 44 | 45 | 抽样方法 46 | ------------------- 47 | 48 | 简单抽样 49 | ^^^^^^^^^^^^^^^^^^^ 50 | .. autofunction:: sample 51 | 52 | 从top k中抽样 53 | ^^^^^^^^^^^^^^^^^^^^ 54 | .. autofunction:: sample_top 55 | 56 | 57 | 词的向量表示 58 | ------------------------------- 59 | 60 | 词汇类 (class) 61 | ^^^^^^^^^^^^^^^^^ 62 | 63 | Simple vocabulary class 64 | ^^^^^^^^^^^^^^^^^^^^^^^^^ 65 | .. autoclass:: SimpleVocabulary 66 | 67 | Vocabulary class 68 | ^^^^^^^^^^^^^^^^^^^^^^^^^ 69 | .. autoclass:: Vocabulary 70 | 71 | Process sentence 72 | ^^^^^^^^^^^^^^^^^^^^^^^^ 73 | .. autofunction:: process_sentence 74 | 75 | Create vocabulary 76 | ^^^^^^^^^^^^^^^^^^^^^^^^ 77 | .. autofunction:: create_vocab 78 | 79 | 80 | 从文件中读取文本 81 | ---------------- 82 | 83 | Simple read file 84 | ^^^^^^^^^^^^^^^^^^ 85 | .. autofunction:: simple_read_words 86 | 87 | Read file 88 | ^^^^^^^^^^^^^^^^^^ 89 | .. autofunction:: read_words 90 | 91 | 从文件中读取类比题目 92 | -------------------------------------------------------- 93 | 94 | .. autofunction:: read_analogies_file 95 | 96 | 建立词汇表、文本与ID转换字典及文本ID化 97 | -------------------------------------------------------- 98 | 99 | 为单词到ID建立字典 100 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 101 | .. autofunction:: build_vocab 102 | 103 | 为ID到单词建立字典 104 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 105 | .. autofunction:: build_reverse_dictionary 106 | 107 | 建立字典，统计表等 108 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 109 | .. autofunction:: build_words_dataset 110 | 111 | 保存词汇表 112 | ^^^^^^^^^^^^^^^^^^^^^^ 113 | .. autofunction:: save_vocab 114 | 115 | 文本转ID，ID转文本 116 | -------------------------------------------------------- 117 | These functions can be done by ``Vocabulary`` class. 118 | 119 | 单词到ID 120 | ^^^^^^^^^^ 121 | .. autofunction:: words_to_word_ids 122 | 123 | ID到单词 124 | ^^^^^^^^^^^ 125 | .. autofunction:: word_ids_to_words 126 | 127 | 机器翻译相关函数 128 | --------------------------- 129 | 130 | 文本ID化 131 | ^^^^^^^^^^^^^^^^ 132 | .. autofunction:: basic_tokenizer 133 | 134 | 建立或读取词汇表 135 | ^^^^^^^^^^^^^^^^^^^^^^^ 136 | .. autofunction:: create_vocabulary 137 | .. autofunction:: initialize_vocabulary 138 | 139 | 文本转ID，ID转文本 140 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 141 | .. autofunction:: sentence_to_token_ids 142 | .. autofunction:: data_to_token_ids 143 | 144 | 145 | 衡量指标 146 | --------------------------- 147 | 148 | BLEU 149 | ^^^^^^^^^^^^^^^^^^^ 150 | .. autofunction:: moses_multi_bleu 151 | -------------------------------------------------------------------------------- /docs/modules/ops.rst: -------------------------------------------------------------------------------- 1 | API - 操作系统管理 2 | ====================== 3 | 4 | 系统操作，更多函数可在 `TensorFlow API `_ 中找到。 5 | 6 | .. automodule:: tensorlayer.ops 7 | 8 | .. autosummary:: 9 | 10 | exit_tf 11 | open_tb 12 | clear_all 13 | set_gpu_fraction 14 | disable_print 15 | enable_print 16 | suppress_stdout 17 | get_site_packages_directory 18 | empty_trash 19 | 20 | TensorFlow 操作函数 21 | --------------------------- 22 | 23 | 中断 Nvidia 进程 24 | ^^^^^^^^^^^^^^^^^ 25 | .. autofunction:: exit_tf 26 | 27 | 打开 TensorBoard 28 | ^^^^^^^^^^^^^^^^^^^ 29 | .. autofunction:: open_tb 30 | 31 | 删除 placeholder 32 | ^^^^^^^^^^^^^^^^^^ 33 | .. autofunction:: clear_all 34 | 35 | GPU 配置函数 36 | --------------------------- 37 | .. autofunction:: set_gpu_fraction 38 | 39 | 命令窗口显示 40 | ------------------ 41 | 42 | 禁止 print 43 | ^^^^^^^^^^^^ 44 | .. autofunction:: disable_print 45 | 46 | 允许 print 47 | ^^^^^^^^^^^^ 48 | .. autofunction:: enable_print 49 | 50 | 临时禁止 print 51 | ^^^^^^^^^^^^^^^^ 52 | .. autofunction:: suppress_stdout 53 | 54 | Site packages 信息 55 | -------------------- 56 | 57 | .. autofunction:: get_site_packages_directory 58 | 59 | 垃圾管理 60 | -------------------- 61 | .. autofunction:: empty_trash 62 | -------------------------------------------------------------------------------- /docs/modules/optimizers: -------------------------------------------------------------------------------- 1 | API - 优化器 2 | ================ 3 | 4 | .. automodule:: tensorlayer.optimizers 5 | 6 | 优化器列表 7 | --------------- 8 | 9 | .. autosummary:: 10 | 11 | AMSGrad 12 | 13 | AMSGrad 优化器 14 | ----------------- 15 | .. autoclass:: AMSGrad 16 | :members: 17 | -------------------------------------------------------------------------------- /docs/modules/optimizers.rst: -------------------------------------------------------------------------------- 1 | API - 优化器 2 | ================ 3 | 4 | .. automodule:: tensorlayer.optimizers 5 | 6 | 我们提供和TensorFlow兼容的新型优化器API，以节省您的开发时间。 7 | 8 | 优化器预览表 9 | --------------- 10 | 11 | .. autosummary:: 12 | 13 | AMSGrad 14 | 15 | AMSGrad 优化器 16 | ----------------- 17 | .. autoclass:: AMSGrad 18 | :members: 19 | -------------------------------------------------------------------------------- /docs/modules/prepro.rst: -------------------------------------------------------------------------------- 1 | API - 数据预处理 2 | ============================= 3 | 4 | 我们提供大量的数据增强及处理方法，若需要对图像进行仿射变换（Affine Transformation），请参考 `Python Can Be Fast `__ 的方法，并结合`tf.py_function`一起使用。 5 | 6 | .. automodule:: tensorlayer.prepro 7 | 8 | .. autosummary:: 9 | 10 | threading_data 11 | 12 | rotation 13 | rotation_multi 14 | crop 15 | crop_multi 16 | flip_axis 17 | flip_axis_multi 18 | shift 19 | shift_multi 20 | 21 | shear 22 | shear_multi 23 | shear2 24 | shear_multi2 25 | swirl 26 | swirl_multi 27 | elastic_transform 28 | elastic_transform_multi 29 | 30 | zoom 31 | zoom_multi 32 | 33 | brightness 34 | brightness_multi 35 | 36 | illumination 37 | 38 | rgb_to_hsv 39 | hsv_to_rgb 40 | adjust_hue 41 | 42 | imresize 43 | 44 | pixel_value_scale 45 | 46 | samplewise_norm 47 | featurewise_norm 48 | 49 | channel_shift 50 | channel_shift_multi 51 | 52 | drop 53 | 54 | transform_matrix_offset_center 55 | apply_transform 56 | projective_transform_by_points 57 | 58 | array_to_img 59 | 60 | find_contours 61 | pt2map 62 | binary_dilation 63 | dilation 64 | binary_erosion 65 | erosion 66 | 67 | 68 | obj_box_coord_rescale 69 | obj_box_coords_rescale 70 | obj_box_coord_scale_to_pixelunit 71 | obj_box_coord_centroid_to_upleft_butright 72 | obj_box_coord_upleft_butright_to_centroid 73 | obj_box_coord_centroid_to_upleft 74 | obj_box_coord_upleft_to_centroid 75 | 76 | parse_darknet_ann_str_to_list 77 | parse_darknet_ann_list_to_cls_box 78 | 79 | obj_box_horizontal_flip 80 | obj_box_imresize 81 | obj_box_crop 82 | obj_box_shift 83 | obj_box_zoom 84 | 85 | keypoint_random_crop 86 | keypoint_random_crop2 87 | keypoint_random_rotate 88 | keypoint_random_flip 89 | keypoint_random_resize 90 | keypoint_random_resize_shortestedge 91 | 92 | pad_sequences 93 | remove_pad_sequences 94 | process_sequences 95 | sequences_add_start_id 96 | sequences_add_end_id 97 | sequences_add_end_id_after_pad 98 | sequences_get_mask 99 | 100 | 101 | 并行 Threading 102 | ------------------ 103 | 104 | 对于当前的版本，我们建议使用`tf.data`和`tf.py_function`来实现训练数据处理，`threading_data`只适合需要处理一次的数据，请参考我们Github中CIFAR10的例子。 105 | 106 | .. autofunction:: threading_data 107 | 108 | 109 | 图像 110 | ----------- 111 | 112 | - 这些函数只对一个图像做处理， 使用 ``threading_data`` 函数来实现多线程处理，请参考 ``tutorial_image_preprocess.py`` 。 113 | - 所有函数都有一个 ``is_random`` 。 114 | - 所有结尾是 `multi` 的函数通常用于图像分隔，因为输入和输出的图像必需是匹配的。 115 | 116 | 旋转 117 | ^^^^^^^^^ 118 | .. autofunction:: rotation 119 | .. autofunction:: rotation_multi 120 | 121 | 裁剪 122 | ^^^^^^^^^ 123 | .. autofunction:: crop 124 | .. autofunction:: crop_multi 125 | 126 | 翻转 127 | ^^^^^^^^^ 128 | .. autofunction:: flip_axis 129 | .. autofunction:: flip_axis_multi 130 | 131 | 位移 132 | ^^^^^^^^^ 133 | .. autofunction:: shift 134 | .. autofunction:: shift_multi 135 | 136 | 切变 137 | ^^^^^^^^^ 138 | .. autofunction:: shear 139 | .. autofunction:: shear_multi 140 | 141 | 切变 V2 142 | ^^^^^^^^^ 143 | .. autofunction:: shear2 144 | .. autofunction:: shear_multi2 145 | 146 | 漩涡 147 | ^^^^^^^^^ 148 | .. autofunction:: swirl 149 | .. autofunction:: swirl_multi 150 | 151 | 局部扭曲(Elastic transform) 152 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 153 | 154 | .. autofunction:: elastic_transform 155 | .. autofunction:: elastic_transform_multi 156 | 157 | 缩放 158 | ^^^^^^^^^ 159 | .. autofunction:: zoom 160 | .. autofunction:: zoom_multi 161 | 162 | 亮度 163 | ^^^^^^^^^^^^ 164 | .. autofunction:: brightness 165 | .. autofunction:: brightness_multi 166 | 167 | 亮度, 饱和度, 对比度 168 | ^^^^^^^^^^^^^^^^^^^^ 169 | .. autofunction:: illumination 170 | 171 | 172 | RGB 转 HSV 173 | ^^^^^^^^^^^^^^ 174 | .. autofunction:: rgb_to_hsv 175 | 176 | HSV 转 RGB 177 | ^^^^^^^^^^^^^^ 178 | .. autofunction:: hsv_to_rgb 179 | 180 | 调整色调（Hue） 181 | ^^^^^^^^^^^^^^^^^^ 182 | .. autofunction:: adjust_hue 183 | 184 | 185 | 调整大小 186 | ^^^^^^^^^^^^ 187 | .. autofunction:: imresize 188 | 189 | 像素值缩放 190 | ^^^^^^^^^^^^^^^^^^^^ 191 | .. autofunction:: pixel_value_scale 192 | 193 | 正规化 194 | ^^^^^^^^^^^^^^^ 195 | .. autofunction:: samplewise_norm 196 | .. autofunction:: featurewise_norm 197 | 198 | 通道位移 199 | ^^^^^^^^^^^^^^ 200 | .. autofunction:: channel_shift 201 | .. autofunction:: channel_shift_multi 202 | 203 | 噪声 204 | ^^^^^^^^^^^^^^ 205 | .. autofunction:: drop 206 | 207 | 矩阵圆心转换到图中央 208 | ^^^^^^^^^^^^^^^^^^^^ 209 | .. autofunction:: transform_matrix_offset_center 210 | 211 | 基于矩阵的仿射变换 212 | ^^^^^^^^^^^^^^^^^^^^^ 213 | .. autofunction:: apply_transform 214 | 215 | 基于坐标点的的投影变换 216 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 217 | .. autofunction:: projective_transform_by_points 218 | 219 | 220 | Numpy 与 PIL 221 | ^^^^^^^^^^^^^^ 222 | .. autofunction:: array_to_img 223 | 224 | 找轮廓 225 | ^^^^^^^^ 226 | .. autofunction:: find_contours 227 | 228 | 一列点到图 229 | ^^^^^^^^^^^^^^^^^ 230 | .. autofunction:: pt2map 231 | 232 | 二值膨胀 233 | ^^^^^^^^^^^^^^^^^ 234 | .. autofunction:: binary_dilation 235 | 236 | 灰度膨胀 237 | ^^^^^^^^^^^^^^^^^^^^ 238 | .. autofunction:: dilation 239 | 240 | 二值腐蚀 241 | ^^^^^^^^^^^^^^^^^^^^ 242 | .. autofunction:: binary_erosion 243 | 244 | 灰度腐蚀 245 | ^^^^^^^^^^^^^^^^^^^^ 246 | .. autofunction:: erosion 247 | 248 | 249 | 250 | 目标检测 251 | ------------------- 252 | 253 | 教程-图像增强 254 | ^^^^^^^^^^^^^^^^^^^^^^^ 255 | 256 | 您好，这是基于VOC数据集的一个图像增强例子，请阅读这篇 `知乎文章 `_ 。 257 | 258 | .. code-block:: python 259 | 260 | import tensorlayer as tl 261 | 262 | ## 下载 VOC 2012 数据集 263 | imgs_file_list, _, _, _, classes, _, _,\ 264 | _, objs_info_list, _ = tl.files.load_voc_dataset(dataset="2012") 265 | 266 | ## 图片标记预处理为列表形式 267 | ann_list = [] 268 | for info in objs_info_list: 269 | ann = tl.prepro.parse_darknet_ann_str_to_list(info) 270 | c, b = tl.prepro.parse_darknet_ann_list_to_cls_box(ann) 271 | ann_list.append([c, b]) 272 | 273 | # 读取一张图片，并保存 274 | idx = 2 # 可自行选择图片 275 | image = tl.vis.read_image(imgs_file_list[idx]) 276 | tl.vis.draw_boxes_and_labels_to_image(image, ann_list[idx][0], 277 | ann_list[idx][1], [], classes, True, save_name='_im_original.png') 278 | 279 | # 左右翻转 280 | im_flip, coords = tl.prepro.obj_box_horizontal_flip(image, 281 | ann_list[idx][1], is_rescale=True, is_center=True, is_random=False) 282 | tl.vis.draw_boxes_and_labels_to_image(im_flip, ann_list[idx][0], 283 | coords, [], classes, True, save_name='_im_flip.png') 284 | 285 | # 调整图片大小 286 | im_resize, coords = tl.prepro.obj_box_imresize(image, 287 | coords=ann_list[idx][1], size=[300, 200], is_rescale=True) 288 | tl.vis.draw_boxes_and_labels_to_image(im_resize, ann_list[idx][0], 289 | coords, [], classes, True, save_name='_im_resize.png') 290 | 291 | # 裁剪 292 | im_crop, clas, coords = tl.prepro.obj_box_crop(image, ann_list[idx][0], 293 | ann_list[idx][1], wrg=200, hrg=200, 294 | is_rescale=True, is_center=True, is_random=False) 295 | tl.vis.draw_boxes_and_labels_to_image(im_crop, clas, coords, [], 296 | classes, True, save_name='_im_crop.png') 297 | 298 | # 位移 299 | im_shfit, clas, coords = tl.prepro.obj_box_shift(image, ann_list[idx][0], 300 | ann_list[idx][1], wrg=0.1, hrg=0.1, 301 | is_rescale=True, is_center=True, is_random=False) 302 | tl.vis.draw_boxes_and_labels_to_image(im_shfit, clas, coords, [], 303 | classes, True, save_name='_im_shift.png') 304 | 305 | # 高宽缩放 306 | im_zoom, clas, coords = tl.prepro.obj_box_zoom(image, ann_list[idx][0], 307 | ann_list[idx][1], zoom_range=(1.3, 0.7), 308 | is_rescale=True, is_center=True, is_random=False) 309 | tl.vis.draw_boxes_and_labels_to_image(im_zoom, clas, coords, [], 310 | classes, True, save_name='_im_zoom.png') 311 | 312 | 实际中，你可能希望如下使用多线程方式来处理一个batch的数据。 313 | 314 | .. code-block:: python 315 | 316 | import tensorlayer as tl 317 | import random 318 | 319 | batch_size = 64 320 | im_size = [416, 416] 321 | n_data = len(imgs_file_list) 322 | jitter = 0.2 323 | def _data_pre_aug_fn(data): 324 | im, ann = data 325 | clas, coords = ann 326 | ## 随机改变图片亮度、对比度和饱和度 327 | im = tl.prepro.illumination(im, gamma=(0.5, 1.5), 328 | contrast=(0.5, 1.5), saturation=(0.5, 1.5), is_random=True) 329 | ## 随机左右翻转 330 | im, coords = tl.prepro.obj_box_horizontal_flip(im, coords, 331 | is_rescale=True, is_center=True, is_random=True) 332 | ## 随机调整大小并裁剪出指定大小的图片，这同时达到了随机缩放的效果 333 | tmp0 = random.randint(1, int(im_size[0]*jitter)) 334 | tmp1 = random.randint(1, int(im_size[1]*jitter)) 335 | im, coords = tl.prepro.obj_box_imresize(im, coords, 336 | [im_size[0]+tmp0, im_size[1]+tmp1], is_rescale=True, 337 | interp='bicubic') 338 | im, clas, coords = tl.prepro.obj_box_crop(im, clas, coords, 339 | wrg=im_size[1], hrg=im_size[0], is_rescale=True, 340 | is_center=True, is_random=True) 341 | ## 把数值范围从 [0, 255] 转到 [-1, 1] (可选) 342 | im = im / 127.5 - 1 343 | return im, [clas, coords] 344 | 345 | # 随机读取一个batch的图片及其标记 346 | idexs = tl.utils.get_random_int(min=0, max=n_data-1, number=batch_size) 347 | b_im_path = [imgs_file_list[i] for i in idexs] 348 | b_images = tl.prepro.threading_data(b_im_path, fn=tl.vis.read_image) 349 | b_ann = [ann_list[i] for i in idexs] 350 | 351 | # 多线程处理 352 | data = tl.prepro.threading_data([_ for _ in zip(b_images, b_ann)], 353 | _data_pre_aug_fn) 354 | b_images2 = [d[0] for d in data] 355 | b_ann = [d[1] for d in data] 356 | 357 | # 保存每一组图片以供体会 358 | for i in range(len(b_images)): 359 | tl.vis.draw_boxes_and_labels_to_image(b_images[i], 360 | ann_list[idexs[i]][0], ann_list[idexs[i]][1], [], 361 | classes, True, save_name='_bbox_vis_%d_original.png' % i) 362 | tl.vis.draw_boxes_and_labels_to_image((b_images2[i]+1)*127.5, 363 | b_ann[i][0], b_ann[i][1], [], classes, True, 364 | save_name='_bbox_vis_%d.png' % i) 365 | 366 | 367 | 坐标-像素单位到比例单位 368 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 369 | .. autofunction:: obj_box_coord_rescale 370 | 371 | 坐标-像素单位到比例单位 （多个坐标） 372 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 373 | .. autofunction:: obj_box_coords_rescale 374 | 375 | 坐标-比例单位到像素单位 376 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 377 | .. autofunction:: obj_box_coord_scale_to_pixelunit 378 | 379 | 坐标-[x_center, x_center, w, h]到左上-右下单位 380 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 381 | .. autofunction:: obj_box_coord_centroid_to_upleft_butright 382 | 383 | 坐标-左上-右下单位到[x_center, x_center, w, h] 384 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 385 | .. autofunction:: obj_box_coord_upleft_butright_to_centroid 386 | 387 | 坐标-[x_center, x_center, w, h]到左上-高宽单位 388 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 389 | .. autofunction:: obj_box_coord_centroid_to_upleft 390 | 391 | 坐标-左上-高宽单位到[x_center, x_center, w, h] 392 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 393 | .. autofunction:: obj_box_coord_upleft_to_centroid 394 | 395 | Darknet格式-字符转列表 396 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 397 | .. autofunction:: parse_darknet_ann_str_to_list 398 | 399 | Darknet格式-分开列表的类别和坐标 400 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 401 | .. autofunction:: parse_darknet_ann_list_to_cls_box 402 | 403 | 图像-翻转 404 | ^^^^^^^^^^^^^^^^^^^^^^^^^ 405 | .. autofunction:: obj_box_horizontal_flip 406 | 407 | 图像-调整大小 408 | ^^^^^^^^^^^^^^^^^^^^^^^^^ 409 | .. autofunction:: obj_box_imresize 410 | 411 | 图像-裁剪 412 | ^^^^^^^^^^^^^^^^^^^^^^^^^ 413 | .. autofunction:: obj_box_crop 414 | 415 | 图像-位移 416 | ^^^^^^^^^^^^^^^^^^^^^^^^^ 417 | .. autofunction:: obj_box_shift 418 | 419 | 图像-缩放 420 | ^^^^^^^^^^^^^^^^^^^^^^^^^ 421 | .. autofunction:: obj_box_zoom 422 | 423 | 424 | 特征点 425 | ------------ 426 | 427 | 图像-裁剪 428 | ^^^^^^^^^^^ 429 | .. autofunction:: keypoint_random_crop 430 | 431 | .. autofunction:: keypoint_random_crop2 432 | 433 | 图像-旋转 434 | ^^^^^^^^^ 435 | .. autofunction:: keypoint_random_rotate 436 | 437 | 图像-翻转 438 | ^^^^^^^^^^^^ 439 | .. autofunction:: keypoint_random_flip 440 | 441 | 图像-缩放 442 | ^^^^^^^^^^^^ 443 | .. autofunction:: keypoint_random_resize 444 | 445 | 图像-缩放 最短边 446 | ^^^^^^^^^^^^^^^^ 447 | .. autofunction:: keypoint_random_resize_shortestedge 448 | 449 | 450 | 451 | 序列 452 | --------- 453 | 454 | 更多相关函数，请见 ``tensorlayer.nlp`` 。 455 | 456 | Padding 457 | ^^^^^^^^^^ 458 | .. autofunction:: pad_sequences 459 | 460 | 461 | Remove Padding 462 | ^^^^^^^^^^^^^^^^^ 463 | .. autofunction:: remove_pad_sequences 464 | 465 | Process 466 | ^^^^^^^^^ 467 | .. autofunction:: process_sequences 468 | 469 | Add Start ID 470 | ^^^^^^^^^^^^^^^ 471 | .. autofunction:: sequences_add_start_id 472 | 473 | 474 | Add End ID 475 | ^^^^^^^^^^^^^^^ 476 | .. autofunction:: sequences_add_end_id 477 | 478 | Add End ID after pad 479 | ^^^^^^^^^^^^^^^^^^^^^^^ 480 | .. autofunction:: sequences_add_end_id_after_pad 481 | 482 | 483 | Get Mask 484 | ^^^^^^^^^ 485 | .. autofunction:: sequences_get_mask 486 | -------------------------------------------------------------------------------- /docs/modules/rein.rst: -------------------------------------------------------------------------------- 1 | API - 强化学习 2 | ============================== 3 | 4 | 强化学习（增强学习）相关函数。 5 | 6 | .. automodule:: tensorlayer.rein 7 | 8 | .. autosummary:: 9 | 10 | discount_episode_rewards 11 | cross_entropy_reward_loss 12 | log_weight 13 | choice_action_by_probs 14 | 15 | 16 | 奖励函数 17 | --------------------- 18 | .. autofunction:: discount_episode_rewards 19 | 20 | 损失函数 21 | --------------------- 22 | 23 | Weighted Cross Entropy 24 | ^^^^^^^^^^^^^^^^^^^^^^^^ 25 | .. autofunction:: cross_entropy_reward_loss 26 | 27 | Log weight 28 | ^^^^^^^^^^^^^^ 29 | .. autofunction:: log_weight 30 | 31 | 采样选择函数 32 | --------------------- 33 | .. autofunction:: choice_action_by_probs 34 | -------------------------------------------------------------------------------- /docs/modules/utils.rst: -------------------------------------------------------------------------------- 1 | API - 实用函数 2 | ======================== 3 | 4 | .. automodule:: tensorlayer.utils 5 | 6 | .. autosummary:: 7 | 8 | fit 9 | test 10 | predict 11 | evaluation 12 | class_balancing_oversample 13 | get_random_int 14 | dict_to_one 15 | list_string_to_dict 16 | flatten_list 17 | 18 | 19 | 训练、测试及预测 20 | ----------------------- 21 | 22 | 训练 23 | ^^^^^^ 24 | .. autofunction:: fit 25 | 26 | 测试 27 | ^^^^^^^ 28 | .. autofunction:: test 29 | 30 | 预测 31 | ^^^^^^ 32 | .. autofunction:: predict 33 | 34 | 评估函数 35 | --------------------- 36 | .. autofunction:: evaluation 37 | 38 | 39 | 类平衡函数(class balancing) 40 | ---------------------------- 41 | .. autofunction:: class_balancing_oversample 42 | 43 | 44 | 随机函数 45 | ---------------------------- 46 | .. autofunction:: get_random_int 47 | 48 | 字典与列表 49 | -------------------- 50 | 51 | 设字典内容全为一 52 | ^^^^^^^^^^^^^^^^ 53 | .. autofunction:: dict_to_one 54 | 55 | 一列字符转字典 56 | ^^^^^^^^^^^^^^^^ 57 | .. autofunction:: list_string_to_dict 58 | 59 | 拉平列表 60 | ^^^^^^^^^^ 61 | .. autofunction:: flatten_list 62 | -------------------------------------------------------------------------------- /docs/modules/visualize.rst: -------------------------------------------------------------------------------- 1 | API - 可视化 2 | ============================ 3 | 4 | TensorFlow 提供了可视化模型和激活输出等的工具 `TensorBoard `_。 5 | 在这里，我们进一步提供一些可视化模型参数和数据的函数。 6 | 7 | .. automodule:: tensorlayer.visualize 8 | 9 | .. autosummary:: 10 | 11 | read_image 12 | read_images 13 | save_image 14 | save_images 15 | draw_boxes_and_labels_to_image 16 | draw_mpii_pose_to_image 17 | W 18 | CNN2d 19 | frame 20 | images2d 21 | tsne_embedding 22 | 23 | 读取与保存图片 24 | -------------- 25 | 26 | 读取单个图片 27 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 28 | .. autofunction:: read_image 29 | 30 | 读取多个图片 31 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 32 | .. autofunction:: read_images 33 | 34 | 保存单个图片 35 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 36 | .. autofunction:: save_image 37 | 38 | 保存多个图片 39 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 40 | .. autofunction:: save_images 41 | 42 | 保存目标检测图片 43 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 44 | .. autofunction:: draw_boxes_and_labels_to_image 45 | 46 | 保存姿态估计图片（MPII） 47 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 48 | .. autofunction:: draw_mpii_pose_to_image 49 | 50 | 可视化模型参数 51 | -------------------- 52 | 53 | 可视化Weight Matrix 54 | ^^^^^^^^^^^^^^^^^^^^ 55 | .. autofunction:: W 56 | 57 | 可视化CNN 2d filter 58 | ^^^^^^^^^^^^^^^^^^^^^ 59 | .. autofunction:: CNN2d 60 | 61 | 62 | 可视化图像 63 | ---------------- 64 | 65 | matplotlib显示单图 66 | ^^^^^^^^^^^^^^^^^^^^^^ 67 | .. autofunction:: frame 68 | 69 | matplotlib显示多图 70 | ^^^^^^^^^^^^^^^^^^^^^^ 71 | .. autofunction:: images2d 72 | 73 | 可视化词嵌入矩阵 74 | -------------------- 75 | .. autofunction:: tsne_embedding 76 | -------------------------------------------------------------------------------- /docs/robots.txt: -------------------------------------------------------------------------------- 1 | User-agent: * 2 | Disallow: / 3 | -------------------------------------------------------------------------------- /docs/user/contributing.rst: -------------------------------------------------------------------------------- 1 | .. _contributing: 2 | 3 | =============== 4 | Contributing 5 | =============== 6 | 7 | TensorLayer is a major ongoing research project in Data Science Institute, Imperial College London. 8 | The goal of the project is to develop a compositional language while complex learning systems 9 | can be build through composition of neural network modules. 10 | 11 | Numerous contributors come from various horizons such as: Tsinghua University, Carnegie Mellon University, University of Technology of Compiegne, 12 | Google, Microsoft, Bloomberg and etc. 13 | 14 | There are many functions need to be contributed such as Maxout, Neural Turing Machine, Attention, TensorLayer Mobile and etc. 15 | 16 | You can easily open a Pull Request (PR) on `GitHub`_, every little step counts and will be credited. 17 | As an open-source project, we highly welcome and value contributions! 18 | 19 | **If you are interested in working with us, please contact us at:** `tensorlayer@gmail.com `_. 20 | 21 | 22 | Project Maintainers 23 | -------------------------- 24 | 25 | 26 | The TensorLayer project was started by `Hao Dong `_ at Imperial College London in June 2016. 27 | 28 | It is actively developed and maintained by the following people *(in alphabetical order)*: 29 | 30 | - **Akara Supratak** (`@akaraspt `_) - ``_ 31 | - **Fangde Liu** (`@fangde `_) - ``_ 32 | - **Guo Li** (`@lgarithm `_) - ``_ 33 | - **Hao Dong** (`@zsdonghao `_) - ``_ 34 | - **Jonathan Dekhtiar** (`@DEKHTIARJonathan `_) - ``_ 35 | - **Luo Mai** (`@luomai `_) - ``_ 36 | - **Simiao Yu** (`@nebulaV `_) - ``_ 37 | 38 | Numerous other contributors can be found in the `Github Contribution Graph `_. 39 | 40 | 41 | What to contribute 42 | ------------------ 43 | 44 | Your method and example 45 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~ 46 | 47 | If you have a new method or example in term of Deep learning and Reinforcement learning, 48 | you are welcome to contribute. 49 | 50 | * Provide your layer or example, so everyone can use it. 51 | * Explain how it would work, and link to a scientific paper if applicable. 52 | * Keep the scope as narrow as possible, to make it easier to implement. 53 | 54 | 55 | Report bugs 56 | ~~~~~~~~~~~ 57 | 58 | Report bugs at the `GitHub`_, we normally will fix it in 5 hours. 59 | If you are reporting a bug, please include: 60 | 61 | * your TensorLayer, TensorFlow and Python version. 62 | * steps to reproduce the bug, ideally reduced to a few Python commands. 63 | * the results you obtain, and the results you expected instead. 64 | 65 | If you are unsure whether the behavior you experience is a bug, or if you are 66 | unsure whether it is related to TensorLayer or TensorFlow, please just ask on `our 67 | mailing list`_ first. 68 | 69 | 70 | Fix bugs 71 | ~~~~~~~~ 72 | 73 | Look through the GitHub issues for bug reports. Anything tagged with "bug" is 74 | open to whoever wants to implement it. If you discover a bug in TensorLayer you can 75 | fix yourself, by all means feel free to just implement a fix and not report it 76 | first. 77 | 78 | 79 | Write documentation 80 | ~~~~~~~~~~~~~~~~~~~ 81 | 82 | Whenever you find something not explained well, misleading, glossed over or 83 | just wrong, please update it! The *Edit on GitHub* link on the top right of 84 | every documentation page and the *[source]* link for every documented entity 85 | in the API reference will help you to quickly locate the origin of any text. 86 | 87 | 88 | 89 | How to contribute 90 | ----------------- 91 | 92 | Edit on GitHub 93 | ~~~~~~~~~~~~~~ 94 | 95 | As a very easy way of just fixing issues in the documentation, use the *Edit 96 | on GitHub* link on the top right of a documentation page or the *[source]* link 97 | of an entity in the API reference to open the corresponding source file in 98 | GitHub, then click the *Edit this file* link to edit the file in your browser 99 | and send us a Pull Request. All you need for this is a free GitHub account. 100 | 101 | For any more substantial changes, please follow the steps below to setup 102 | TensorLayer for development. 103 | 104 | 105 | Documentation 106 | ~~~~~~~~~~~~~ 107 | 108 | The documentation is generated with `Sphinx 109 | `_. To build it locally, run the 110 | following commands: 111 | 112 | .. code:: bash 113 | 114 | pip install Sphinx 115 | sphinx-quickstart 116 | 117 | cd docs 118 | make html 119 | 120 | If you want to re-generate the whole docs, run the following commands: 121 | 122 | .. code :: bash 123 | 124 | cd docs 125 | make clean 126 | make html 127 | 128 | 129 | To write the docs, we recommend you to install `Local RTD VM `_. 130 | 131 | 132 | 133 | 134 | Afterwards, open ``docs/_build/html/index.html`` to view the documentation as 135 | it would appear on `readthedocs `_. If you 136 | changed a lot and seem to get misleading error messages or warnings, run 137 | ``make clean html`` to force Sphinx to recreate all files from scratch. 138 | 139 | When writing docstrings, follow existing documentation as much as possible to 140 | ensure consistency throughout the library. For additional information on the 141 | syntax and conventions used, please refer to the following documents: 142 | 143 | * `reStructuredText Primer `_ 144 | * `Sphinx reST markup constructs `_ 145 | * `A Guide to NumPy/SciPy Documentation `_ 146 | 147 | 148 | Testing 149 | ~~~~~~~ 150 | 151 | TensorLayer has a code coverage of 100%, which has proven very helpful in the past, 152 | but also creates some duties: 153 | 154 | * Whenever you change any code, you should test whether it breaks existing 155 | features by just running the test scripts. 156 | * Every bug you fix indicates a missing test case, so a proposed bug fix should 157 | come with a new test that fails without your fix. 158 | 159 | 160 | Sending Pull Requests 161 | ~~~~~~~~~~~~~~~~~~~~~ 162 | 163 | When you're satisfied with your addition, the tests pass and the documentation 164 | looks good without any markup errors, commit your changes to a new branch, push 165 | that branch to your fork and send us a Pull Request via GitHub's web interface. 166 | 167 | All these steps are nicely explained on GitHub: 168 | https://guides.github.com/introduction/flow/ 169 | 170 | When filing your Pull Request, please include a description of what it does, to 171 | help us reviewing it. If it is fixing an open issue, say, issue #123, add 172 | *Fixes #123*, *Resolves #123* or *Closes #123* to the description text, so 173 | GitHub will close it when your request is merged. 174 | 175 | 176 | .. _Release: https://github.com/tensorlayer/tensorlayer/releases 177 | .. _GitHub: https://github.com/tensorlayer/tensorlayer 178 | .. _our mailing list: hao.dong11@imperial.ac.uk 179 | -------------------------------------------------------------------------------- /docs/user/development.rst: -------------------------------------------------------------------------------- 1 | 开发 Development 2 | ================ 3 | 4 | TensorLayer始于一个帝国理工的研究项目，目前由数位 `GitHub`_ 的贡献者负责维护和进行改进。 5 | 6 | 作为一个为研究员和工程师开发的开源项目，我们诚挚地欢迎您为对项目提供帮助。 7 | 每个微小的支持都会帮助我们并且会被记录下来。 8 | 9 | 10 | .. _TensorLayer-philospy: 11 | 12 | 理念 13 | ---------- 14 | 15 | TensorLayer的想法源于组合TensorFlow的灵活性和正确的 building blocks的可用性来训练神经网络的需求。 16 | 它依照下列设计目标所开发： 17 | 18 | * **简洁**：易于使用，易于扩展与修改以便于研究和工程中使用。 19 | * **快速**：在GPU的支持下运行速度与纯TensorFlow脚本速度一致。简洁但不牺牲性能。 20 | * **兼容**：一个网络被抽象到正则化，成本和每一层的输出，易于与其他TensorFlow的库协作。 21 | * **透明**：在抽象后面不隐藏TensorFlow，尽量依赖TensorFlow的功能并且遵循TensorFlow的约定。 22 | 23 | 如何参与我们 24 | """"""""""""" 25 | 26 | 如果您在深度学习和强化学习方面发布了一种新的算法， 27 | 欢迎您分享您的算法给TensorLayer 28 | 29 | * 说明它是怎么工作的，如果可以话请给出学术论文的链接。 30 | * 尽可能地缩减其范围，以以致于便于实现 31 | 32 | 33 | 报告BUG 34 | """"""""""""""""""" 35 | 36 | 请您在 `GitHub`_ 上报告BUG。 37 | 如果您打算报告BUG，请包含以下内容： 38 | 39 | * 您的TensorLayer和TensorFlow版本号 40 | * 重现BUG的步骤，最好能减少到数个python 命令 41 | * 您获得的结果和您期望的结果。 42 | 43 | 如果您不确定遇到的行为是否是BUG， 44 | 或者你不确定错误是否与TensorLayer或TensorFlow有关， 45 | 请您先在 `our mailing list`_ 查看下。 46 | 47 | 修复BUG 48 | """"""""""""" 49 | 50 | 通过GitHub的问题栏(issues)来查看BUG报告。 51 | 任何被标记为BUG的项对所有想要修复它的人来说都是开放的。 52 | 如果您发现了TensorLayer的一个你可以自己修复的BUG， 53 | 您可以用任何方法来实现修复并且无需立即报告这个BUG。 54 | 55 | 编写文档 56 | """""""""""""" 57 | 58 | 无论什么时候您发现有些文档没有解释清楚，存在误导，敷衍带过或者根本就是错的。 59 | 请及时更新它！*Edit on GitHub*的链接就在每一篇文档的右上角 60 | 并且API引用列表中的每篇文档的*[source]*的链接可以帮助您快速地定位任何文字的根源。 61 | 62 | 63 | 如何参与我们 64 | ------------------- 65 | 66 | 在GitHub上编辑 67 | """"""""""""""""""" 68 | 69 | 正如刚刚文档中修复BUG所说的简单方法， 70 | 点击文档右上角的*Edit on GitHub*链接或者API引用列表中的对象的*[source]*链接来打开GitHub中的源文件， 71 | 然后在你的浏览器中点击*Edit this file*链接并发送拉请求(Pull Request). 72 | 你只需要一个免费的GitHub账户就可以做到了。 73 | 74 | 对于任何较大幅度的修改，请遵循以下步骤来更新TensorLayer开发。 75 | 76 | 文档 77 | """""""""""""" 78 | 79 | 文档由 `Sphinx `_ 生成。 80 | 如果要本地编辑它，运行下列命令： 81 | .. code:: bash 82 | 83 | cd docs 84 | make html 85 | 86 | 如果您想要重新生成整个文档，运行下列命令： 87 | 88 | .. code:: bash 89 | 90 | cd docs 91 | make clean 92 | make html 93 | 94 | 然后，打开 ``docs/_build/index.html`` 来查看会出现在 `readthedocs `_ 文档。 95 | 如果您更改了很多内容，并且似乎出现了许多误导性的错误信息或警告，运行``make clean html``来让Sphinx重新生成所有文件。 96 | 97 | 编写英文文档文字时，请尽可能地按照现有文档的文字习惯， 98 | 来保证整个库文字的一致性。所使用的语法及约定的相关信息，请参考以下文档： 99 | 100 | * `reStructuredText Primer `_ 101 | * `Sphinx reST markup constructs `_ 102 | * `A Guide to NumPy/SciPy Documentation `_ 103 | 104 | 测试 105 | """"""""" 106 | 107 | TensorLayer有100%的代码覆盖率，这过去被证明是十分有用的的， 108 | 但也带来了一些责任： 109 | 110 | * 每当您更改任何代码的时候，您应该运行测试脚本来测试它是否能优化现有属性。 111 | * 您修改的每个BUG说明一个缺少的测试案例， 112 | 所以每个修复BUG的方案应该配置一个您没修复前的测试案例。 113 | 114 | 发送拉请求 115 | """"""""""""""""" 116 | 117 | 当您对您添加的内容感到满意并且测试通过，文档规范，简明，不存在任何注释错误。 118 | 您可以将您的更改提交到一个新的分支(branch)，并且将这个分支与您的副本(fork)合并， 119 | 然后通过GitHub的网站发送一个拉请求(pull request) 120 | 121 | 所有的这些步骤在GitHub上有相当不错的说明： 122 | https://guides.github.com/introduction/flow/ 123 | 124 | 当您提交拉请求时，请附带一个更改内容的说明，以帮助我们能更好的检阅它。 125 | 如果它是一个正在开放的问题(issue)，比如：issue#123，请在您的描述中添加 126 | *Fixes#123*,*Resolves#123*或者*Closes#123*，这样当您的拉请求被接纳之后 127 | GitHub会关闭那个问题。 128 | 129 | 130 | .. _GitHUb: http://github.com/zsdonghao/tensorlayer 131 | .. _our mailing list: hao.dong11@imperial.ac.uk 132 | -------------------------------------------------------------------------------- /docs/user/example.rst: -------------------------------------------------------------------------------- 1 | .. _example: 2 | 3 | ============ 4 | 例子 Examples 5 | ============ 6 | 7 | Basics 8 | ============ 9 | 10 | - Multi-layer perceptron (MNIST), simple usage. Classification task, see `tutorial_mnist_simple.py `__. 11 | - Multi-layer perceptron (MNIST), dynamic model. Classification with dropout using iterator, see `tutorial_mnist_mlp_dynamic.py method2 `__. 12 | - Multi-layer perceptron (MNIST), static model. Classification with dropout using iterator, see `tutorial_mnist_mlp_static.py `__. 13 | - Convolutional Network (CIFAR-10). Classification task, see `tutorial_cifar10_cnn_static.py `_. 14 | - TensorFlow dataset API for object detection see `here `__. 15 | - Merge Keras into TensorLayer. `tutorial_keras.py `__. 16 | - Data augmentation with TFRecord. Effective way to load and pre-process data, see `tutorial_tfrecord*.py `__ and `tutorial_cifar10_tfrecord.py `__. 17 | - Data augmentation with TensorLayer. See `tutorial_fast_affine_transform.py `__ (for quick test only). 18 | 19 | Pretrained Models 20 | ================== 21 | 22 | - VGG 16 (ImageNet). Classification task, see `tutorial_models_vgg16 `__. 23 | - VGG 19 (ImageNet). Classification task, see `tutorial_models_vgg19.py `__. 24 | - SqueezeNet (ImageNet). Model compression, see `tutorial_models_squeezenetv1.py `__. 25 | - MobileNet (ImageNet). Model compression, see `tutorial_models_mobilenetv1.py `__. 26 | - All pretrained models in `pretrained-models `__. 27 | 28 | Vision 29 | ================== 30 | 31 | - Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization, see `examples `__. 32 | - ArcFace: Additive Angular Margin Loss for Deep Face Recognition, see `InsignFace `__. 33 | - BinaryNet. Model compression, see `mnist `__ `cifar10 `__. 34 | - Ternary Weight Network. Model compression, see `mnist `__ `cifar10 `__. 35 | - DoReFa-Net. Model compression, see `mnist `__ `cifar10 `__. 36 | - QuanCNN. Model compression, sees `mnist `__ `cifar10 `__. 37 | - Wide ResNet (CIFAR) by `ritchieng `__. 38 | - `Spatial Transformer Networks `__ by `zsdonghao `__. 39 | - `U-Net for brain tumor segmentation `__ by `zsdonghao `__. 40 | - Variational Autoencoder (VAE) for (CelebA) by `yzwxx `__. 41 | - Variational Autoencoder (VAE) for (MNIST) by `BUPTLdy `__. 42 | - Image Captioning - Reimplementation of Google's `im2txt `__ by `zsdonghao `__. 43 | 44 | Adversarial Learning 45 | ======================== 46 | - DCGAN (CelebA). Generating images by `Deep Convolutional Generative Adversarial Networks `__ by `zsdonghao `__. 47 | - `Generative Adversarial Text to Image Synthesis `__ by `zsdonghao `__. 48 | - `Unsupervised Image to Image Translation with Generative Adversarial Networks `__ by `zsdonghao `__. 49 | - `Improved CycleGAN `__ with resize-convolution by `luoxier `__. 50 | - `Super Resolution GAN `__ by `zsdonghao `__. 51 | - `BEGAN: Boundary Equilibrium Generative Adversarial Networks `__ by `2wins `__. 52 | - `DAGAN: Fast Compressed Sensing MRI Reconstruction `__ by `nebulaV `__. 53 | 54 | Natural Language Processing 55 | ============================== 56 | 57 | - Recurrent Neural Network (LSTM). Apply multiple LSTM to PTB dataset for language modeling, see `tutorial_ptb_lstm_state_is_tuple.py `__. 58 | - Word Embedding (Word2vec). Train a word embedding matrix, see `tutorial_word2vec_basic.py `__. 59 | - Restore Embedding matrix. Restore a pre-train embedding matrix, see `tutorial_generate_text.py `__. 60 | - Text Generation. Generates new text scripts, using LSTM network, see `tutorial_generate_text.py `__. 61 | - Chinese Text Anti-Spam by `pakrchen `__. 62 | - `Chatbot in 200 lines of code `__ for `Seq2Seq `__. 63 | - FastText Sentence Classification (IMDB), see `tutorial_imdb_fasttext.py `__ by `tomtung `__. 64 | 65 | Reinforcement Learning 66 | ============================== 67 | 68 | - Policy Gradient / Network (Atari Ping Pong), see `tutorial_atari_pong.py `__. 69 | - Deep Q-Network (Frozen lake), see `tutorial_frozenlake_dqn.py `__. 70 | - Q-Table learning algorithm (Frozen lake), see `tutorial_frozenlake_q_table.py `__. 71 | - Asynchronous Policy Gradient using TensorDB (Atari Ping Pong) by `nebulaV `__. 72 | - AC for discrete action space (Cartpole), see `tutorial_cartpole_ac.py `__. 73 | - A3C for continuous action space (Bipedal Walker), see `tutorial_bipedalwalker_a3c*.py `__. 74 | - `DAGGER `__ for (`Gym Torcs `__) by `zsdonghao `__. 75 | - `TRPO `__ for continuous and discrete action space by `jjkke88 `__. 76 | 77 | Miscellaneous 78 | ================= 79 | 80 | - TensorDB by `fangde `__ see `tl_paper `__. 81 | - A simple web service - `TensorFlask `__ by `JoelKronander `__. 82 | 83 | .. 84 | Applications 85 | ============= 86 | 87 | There are some good applications implemented by TensorLayer. 88 | You may able to find some useful examples for your project. 89 | If you want to share your application, please contact tensorlayer@gmail.com. 90 | 91 | 1D CNN + LSTM for Biosignal 92 | --------------------------------- 93 | 94 | Author : `Akara Supratak `__ 95 | 96 | Introduction 97 | ^^^^^^^^^^^^ 98 | 99 | Implementation 100 | ^^^^^^^^^^^^^^ 101 | 102 | Citation 103 | ^^^^^^^^ 104 | 105 | 106 | 107 | 108 | 109 | .. _GitHub: https://github.com/tensorlayer/tensorlayer 110 | .. _Deeplearning Tutorial: http://deeplearning.stanford.edu/tutorial/ 111 | .. _Convolutional Neural Networks for Visual Recognition: http://cs231n.github.io/ 112 | .. _Neural Networks and Deep Learning: http://neuralnetworksanddeeplearning.com/ 113 | .. _TensorFlow tutorial: https://www.tensorflow.org/versions/r0.9/tutorials/index.html 114 | .. _Understand Deep Reinforcement Learning: http://karpathy.github.io/2016/05/31/rl/ 115 | .. _Understand Recurrent Neural Network: http://karpathy.github.io/2015/05/21/rnn-effectiveness/ 116 | .. _Understand LSTM Network: http://colah.github.io/posts/2015-08-Understanding-LSTMs/ 117 | .. _Word Representations: http://colah.github.io/posts/2014-07-NLP-RNNs-Representations/ 118 | -------------------------------------------------------------------------------- /docs/user/get_involved.rst: -------------------------------------------------------------------------------- 1 | 2 | 3 | ========================= 4 | 科研上路 5 | ========================= 6 | 7 | 8 | 北京大学 - 前沿计算研究中心 - 教职机会 9 | ============================================================= 10 | 11 | The Center on Frontiers of Computing Studies (CFCS), Peking University (PKU), China, is a university new initiative co-founded by Professors John Hopcroft (Turing Awardee) and Wen Gao (CAE, ACM/IEEE Fellow). The center aims at developing the excellence on two fronts: research and education. On the research front, the center will provide a world-class research environment, where innovation and impactful research is the central aim, measured by professional reputation among world scholars, not by counting the number of publications and research funding. On the education front, the center deeply involves in the Turing Class, an elite undergraduate program that draws the cream of the crop from the PKU undergraduate talent pool. New curriculum and pedagogy are designed and practiced in this program, with the aim to cultivate a new generation of computer scientist/engineers that are solid in both theories and practices. 12 | 13 | **Positions and Qualification** 14 | 15 | The center invites applications for tenured/tenure-track faculty positions. We are seeking applicants from all areas of Computer Science, spanning theoretical foundations, systems, software, and applications, with special interests in artificial intelligence and machine learning. We are especially interested in applicants conducting research at the frontiers of Computer Science with other disciplines, such as data sciences, engineering, as well as mathematical, medical, physical, and social sciences. 16 | 17 | Applicants are expected to have completed (or be completing) a Ph.D., have demonstrated the ability to pursue a program of research, and have a strong commitment to undergraduate and graduate teaching. A successful candidate will be expected to teach one to two courses at the undergraduate and graduate levels in each semester, and to build and lead a team of undergraduate and graduate students in innovative research. 18 | 19 | We are also seeking qualified candidates for postdoctoral positions. Candidates should have a Ph.D. in a relevant discipline or expect a Ph. D within a year, with a substantive record of research accomplishments, and the ability to work collaboratively with faculty members in the center. 20 | 21 | **To Apply** 22 | 23 | Applicants should send a full curriculum vitae; copies of 3-5 key publications; 3-5 names and contact information of references; and a statement of research and teaching to: CFCS_recruiting[at]pku[dot]edu[dot]cn . To expedite the process, please arrange to have the reference letters sent directly to the above email address. 24 | 25 | Application for a postdoctoral position should include a curriculum vita, brief statement of research, and three to five names and contact information of recommendation, and can be directly addressed to an individual faculty member. 26 | 27 | We conduct review of applications monthly, immediately upon the recipient of all application materials at the beginning of each month. However, it is highly recommended that applicants submit complete applications sooner than later, as the positions are to be filled quickly. 28 | 29 | 30 | 大英帝国理工 - 数据科学研究所 - 博士、博士后机会 31 | ================================================== 32 | 33 | Data science is therefore by nature at the core of all modern transdisciplinary scientific activities, as it involves the whole life cycle of data, from acquisition and exploration to analysis and communication of the results. Data science is not only concerned with the tools and methods to obtain, manage and analyse data: it is also about extracting value from data and translating it from asset to product. 34 | 35 | Launched on 1st April 2014, the Data Science Institute at Imperial College London aims to enhance Imperial's excellence in data-driven research across its faculties by fulfilling the following objectives. 36 | 37 | The Data Science Institute is housed in purpose built facilities in the heart of the Imperial College campus in South Kensington. Such a central location provides excellent access to collabroators across the College and across London. 38 | 39 | - To act as a focal point for coordinating data science research at Imperial College by facilitating access to funding, engaging with global partners, and stimulating cross-disciplinary collaboration. 40 | - To develop data management and analysis technologies and services for supporting data driven research in the College. 41 | - To promote the training and education of the new generation of data scientist by developing and coordinating new degree courses, and conducting public outreach programmes on data science. 42 | - To advise College on data strategy and policy by providing world-class data science expertise. 43 | - To enable the translation of data science innovation by close collaboration with industry and supporting commercialization. 44 | 45 | If you are interested in working with us, please check our 46 | `vacancies `__ 47 | and other ways to 48 | `get involved `__ 49 | , or feel free to 50 | `contact us `__. 51 | -------------------------------------------------------------------------------- /docs/user/get_start_advance.rst: -------------------------------------------------------------------------------- 1 | .. _getstartadvance: 2 | 3 | ================== 4 | 高级特性 5 | ================== 6 | 7 | 8 | 使用预训练模型 9 | ================ 10 | 11 | 获取整个模型 12 | --------------- 13 | 14 | .. code-block:: python 15 | 16 | import tensorflow as tf 17 | import tensorlayer as tl 18 | import numpy as np 19 | from tensorlayer.models.imagenet_classes import class_names 20 | 21 | vgg = tl.models.vgg16(pretrained=True) 22 | img = tl.vis.read_image('data/tiger.jpeg') 23 | img = tl.prepro.imresize(img, (224, 224)).astype(np.float32) / 255 24 | output = vgg(img, is_train=False) 25 | 26 | 获取部分模型 27 | ------------------ 28 | 29 | .. code-block:: python 30 | 31 | # get VGG without the last layer 32 | cnn = tl.models.vgg16(end_with='fc2_relu', mode='static').as_layer() 33 | # add one more layer and build a new model 34 | ni = Input([None, 224, 224, 3], name="inputs") 35 | nn = cnn(ni) 36 | nn = tl.layers.Dense(n_units=100, name='out')(nn) 37 | model = tl.models.Model(inputs=ni, outputs=nn) 38 | # train your own classifier (only update the last layer) 39 | train_params = model.get_layer('out').all_weights 40 | 41 | 复用模型 42 | ------------------ 43 | 44 | .. code-block:: python 45 | 46 | # in dynamic model, we can directly use the same model 47 | # in static model 48 | vgg_layer = tl.models.vgg16().as_layer() 49 | ni_1 = tl.layers.Input([None, 224, 224, 3]) 50 | ni_2 = tl.layers.Input([None, 224, 224, 3]) 51 | a_1 = vgg_layer(ni_1) 52 | a_2 = vgg_layer(ni_2) 53 | M = Model(inputs=[ni_1, ni_2], outputs=[a_1, a_2]) 54 | 55 | -------------------------------------------------------------------------------- /docs/user/get_start_model.rst: -------------------------------------------------------------------------------- 1 | .. _getstartmodel: 2 | 3 | =============== 4 | 定义模型 5 | =============== 6 | 7 | TensorLayer提供两种模型定义模型，静态模型提供直观的代码风格来定义模型，而动态模型提供完全可控的前向传播。 8 | 9 | 静态模型 10 | =============== 11 | 12 | .. code-block:: python 13 | 14 | import tensorflow as tf 15 | from tensorlayer.layers import Input, Dropout, Dense 16 | from tensorlayer.models import Model 17 | 18 | def get_model(inputs_shape): 19 | ni = Input(inputs_shape) 20 | nn = Dropout(keep=0.8)(ni) 21 | nn = Dense(n_units=800, act=tf.nn.relu, name="dense1")(nn) 22 | nn = Dropout(keep=0.8)(nn) 23 | nn = Dense(n_units=800, act=tf.nn.relu)(nn) 24 | nn = Dropout(keep=0.8)(nn) 25 | nn = Dense(n_units=10, act=tf.nn.relu)(nn) 26 | M = Model(inputs=ni, outputs=nn, name="mlp") 27 | return M 28 | 29 | MLP = get_model([None, 784]) 30 | MLP.eval() 31 | outputs = MLP(data) 32 | 33 | 动态模型 34 | ======================= 35 | 36 | 动态模型时，前一层的输出尺寸，需要手动输入到下一层中，以初始化下一层的参数。 37 | 38 | .. code-block:: python 39 | 40 | class CustomModel(Model): 41 | 42 | def __init__(self): 43 | super(CustomModel, self).__init__() 44 | 45 | self.dropout1 = Dropout(keep=0.8) 46 | self.dense1 = Dense(n_units=800, act=tf.nn.relu, in_channels=784) 47 | self.dropout2 = Dropout(keep=0.8)#(self.dense1) 48 | self.dense2 = Dense(n_units=800, act=tf.nn.relu, in_channels=800) 49 | self.dropout3 = Dropout(keep=0.8)#(self.dense2) 50 | self.dense3 = Dense(n_units=10, act=tf.nn.relu, in_channels=800) 51 | 52 | def forward(self, x, foo=False): 53 | z = self.dropout1(x) 54 | z = self.dense1(z) 55 | z = self.dropout2(z) 56 | z = self.dense2(z) 57 | z = self.dropout3(z) 58 | out = self.dense3(z) 59 | if foo: 60 | out = tf.nn.relu(out) 61 | return out 62 | 63 | MLP = CustomModel() 64 | MLP.eval() 65 | outputs = MLP(data, foo=True) # controls the forward here 66 | outputs = MLP(data, foo=False) 67 | 68 | 切换训练/测试模式 69 | ============================= 70 | 71 | .. code-block:: python 72 | 73 | # method 1: switch before forward 74 | Model.train() # enable dropout, batch norm moving avg ... 75 | output = Model(train_data) 76 | ... # training code here 77 | Model.eval() # disable dropout, batch norm moving avg ... 78 | output = Model(test_data) 79 | ... # testing code here 80 | 81 | # method 2: switch while forward 82 | output = Model(train_data, is_train=True) 83 | output = Model(test_data, is_train=False) 84 | 85 | 86 | 参数（层）复用 87 | ===================== 88 | 89 | 静态模型的层复用可以如下实现。 90 | 91 | .. code-block:: python 92 | 93 | # create siamese network 94 | 95 | def create_base_network(input_shape): 96 | '''Base network to be shared (eq. to feature extraction). 97 | ''' 98 | input = Input(shape=input_shape) 99 | x = Flatten()(input) 100 | x = Dense(128, act=tf.nn.relu)(x) 101 | x = Dropout(0.9)(x) 102 | x = Dense(128, act=tf.nn.relu)(x) 103 | x = Dropout(0.9)(x) 104 | x = Dense(128, act=tf.nn.relu)(x) 105 | return Model(input, x) 106 | 107 | 108 | def get_siamese_network(input_shape): 109 | """Create siamese network with shared base network as layer 110 | """ 111 | base_layer = create_base_network(input_shape).as_layer() # convert model as layer 112 | 113 | ni_1 = Input(input_shape) 114 | ni_2 = Input(input_shape) 115 | nn_1 = base_layer(ni_1) # call base_layer twice 116 | nn_2 = base_layer(ni_2) 117 | return Model(inputs=[ni_1, ni_2], outputs=[nn_1, nn_2]) 118 | 119 | siamese_net = get_siamese_network([None, 784]) 120 | 121 | 动态模型的层复用可在forward时简单地通过多次调用来实现。 122 | 123 | .. code-block:: python 124 | 125 | class MyModel(Model): 126 | def __init__(self): 127 | super(MyModel, self).__init__() 128 | self.dense_shared = Dense(n_units=800, act=tf.nn.relu, in_channels=784) 129 | self.dense1 = Dense(n_units=10, act=tf.nn.relu, in_channels=800) 130 | self.dense2 = Dense(n_units=10, act=tf.nn.relu, in_channels=800) 131 | self.cat = Concat() 132 | 133 | def forward(self, x): 134 | x1 = self.dense_shared(x) # call dense_shared twice 135 | x2 = self.dense_shared(x) 136 | x1 = self.dense1(x1) 137 | x2 = self.dense2(x2) 138 | out = self.cat([x1, x2]) 139 | return out 140 | 141 | model = MyModel() 142 | 143 | 显示模型信息 144 | ======================= 145 | 146 | .. code-block:: python 147 | 148 | print(MLP) # simply call print function 149 | 150 | # Model( 151 | # (_inputlayer): Input(shape=[None, 784], name='_inputlayer') 152 | # (dropout): Dropout(keep=0.8, name='dropout') 153 | # (dense): Dense(n_units=800, relu, in_channels='784', name='dense') 154 | # (dropout_1): Dropout(keep=0.8, name='dropout_1') 155 | # (dense_1): Dense(n_units=800, relu, in_channels='800', name='dense_1') 156 | # (dropout_2): Dropout(keep=0.8, name='dropout_2') 157 | # (dense_2): Dense(n_units=10, relu, in_channels='800', name='dense_2') 158 | # ) 159 | 160 | 获取特定参数 161 | ======================= 162 | 163 | 我们可以通过层的名字或者参数的索引来获取特定参数。 164 | 165 | .. code-block:: python 166 | 167 | # indexing 168 | all_weights = MLP.weights 169 | some_weights = MLP.weights[1:3] 170 | 171 | # naming 172 | some_weights = MLP.get_layer('dense1').weights 173 | 174 | 保存和恢复模型 175 | ======================= 176 | 177 | 我们提供两种方法保存和回复模型。 178 | 179 | 只保留参数 180 | ------------------ 181 | 182 | .. code-block:: python 183 | 184 | MLP.save_weights('./model_weights.h5') # by default, file will be in hdf5 format 185 | MLP.load_weights('./model_weights.h5') 186 | 187 | 保留参数和网络结构 188 | --------------------------------------------- 189 | 190 | .. code-block:: python 191 | 192 | # When using Model.load(), there is no need to reimplement or declare the architecture of the model explicitly in code 193 | MLP.save('./model.h5', save_weights=True) 194 | MLP = Model.load('./model.h5', load_weights=True) 195 | 196 | 自定义层 197 | ================== 198 | 199 | 全连接层的实现如下，以供参考。 200 | 201 | z = f(x*W+b) 202 | 203 | .. code-block:: python 204 | 205 | class Dense(Layer): 206 | def __init__(self, n_units, act=None, in_channels=None, name=None): 207 | super(Dense, self).__init__(name) 208 | 209 | self.n_units = n_units 210 | self.act = act 211 | self.in_channels = in_channels 212 | 213 | # for dynamic model, it needs the input shape to get the shape of W 214 | if self.in_channels is not None: 215 | self.build(self.in_channels) 216 | self._built = True 217 | 218 | def build(self, inputs_shape): 219 | if self.in_channels is None and len(inputs_shape) != 2: 220 | raise AssertionError("The input dimension must be rank 2, please reshape or flatten it") 221 | if self.in_channels: 222 | shape = [self.in_channels, self.n_units] 223 | else: 224 | self.in_channels = inputs_shape[1] 225 | shape = [inputs_shape[1], self.n_units] 226 | self.W = self._get_weights("weights", shape=tuple(shape)) 227 | if self.b_init: 228 | self.b = self._get_weights("biases", shape=(self.n_units, )) 229 | 230 | @tf.function 231 | def forward(self, inputs): 232 | z = tf.matmul(inputs, self.W) 233 | if self.b_init: 234 | z = tf.add(z, self.b) 235 | if self.act: 236 | z = self.act(z) 237 | return z 238 | -------------------------------------------------------------------------------- /docs/user/installation.rst: -------------------------------------------------------------------------------- 1 | .. _installation: 2 | 3 | ====================== 4 | 安装 Installation 5 | ====================== 6 | 7 | 安装 TensorFlow 8 | ========================= 9 | 10 | .. code-block:: bash 11 | 12 | pip3 install tensorflow-gpu==2.0.0a0 # specific version (YOU SHOULD INSTALL THIS ONE NOW) 13 | pip3 install tensorflow-gpu # GPU version 14 | pip3 install tensorflow # CPU version 15 | 16 | 更多TensorFlow安装信息，可在Google官网查看。TensorFlow支持Linux、MscOS和Windows下的GPU加速，需要用户自行安装CUDA和CuDNN。 17 | 18 | 安装 TensorLayer 19 | ========================= 20 | 21 | 稳定版本: 22 | 23 | .. code-block:: bash 24 | 25 | pip3 install tensorlayer 26 | 27 | 最新版本请通过Github来安装: 28 | 29 | .. code-block:: bash 30 | 31 | pip3 install git+https://github.com/tensorlayer/tensorlayer.git 32 | or 33 | pip3 install https://github.com/tensorlayer/tensorlayer/archive/master.zip 34 | 35 | 对于TensorLayer贡献者，建议从Github把整个项目clone到本地，然后把tensorlayer文件夹放到相应的项目中去。 36 | 37 | .. code-block:: bash 38 | 39 | git clone https://github.com/tensorlayer/tensorlayer.git 40 | 41 | 您也可以通过源码来安装: 42 | 43 | .. code-block:: bash 44 | 45 | # 首先把TensorLayer从Github下载到本地 46 | git clone https://github.com/tensorlayer/tensorlayer.git 47 | cd tensorlayer 48 | 49 | # 建议安装 virtualenv 50 | pip install virtualenv 51 | # 创造虚拟环境 `venv` 52 | virtualenv venv 53 | 54 | # 激活虚拟环境 55 | 56 | ## Linux: 57 | source venv/bin/activate 58 | 59 | ## Windows: 60 | venv\Scripts\activate.bat 61 | 62 | # 简单安装 63 | pip install . 64 | 65 | # ============= IF TENSORFLOW IS NOT ALREADY INSTALLED ============= # 66 | 67 | # for a machine **without** an NVIDIA GPU 68 | pip install -e ".[all_cpu_dev]" 69 | 70 | # for a machine **with** an NVIDIA GPU 71 | pip install -e ".[all_gpu_dev]" 72 | 73 | 如果您想使用旧版的TensorLayer 1.X: 74 | 75 | .. code-block:: bash 76 | 77 | [stable version] pip install tensorlayer==1.x.x 78 | 79 | 如果您想修改旧版的TensorLayer 1.X，您也可以把整个项目下载下来，再安装 80 | 81 | .. code-block:: bash 82 | 83 | cd to the root of the git tree 84 | pip install -e . 85 | 86 | 这个命令会根据 ``setup.py`` 来安装TensorLayer。符号 ``-e`` 表示可修改（editable），这样您可以修改 ``tensorlayer`` 文件夹中的源码，然后 ``import`` 使用之。 87 | 88 | 89 | GPU 加速 90 | ========================== 91 | 92 | CUDA 93 | ---- 94 | 95 | TensorFlow 官网也提供了安装 CUDA 和 CuDNN 的教程。简单来说，请先从NVIDIA官网下载CUDA： 96 | 97 | - `CUDA 下载与安装 `_ 98 | 99 | 100 | .. 101 | make sure ``/usr/local/cuda/bin`` is in your ``PATH`` (use ``echo #PATH`` to check), and 102 | ``nvcc --version`` works. Also ensure ``/usr/local/cuda/lib64`` is in your 103 | ``LD_LIBRARY_PATH``, so the CUDA libraries can be found. 104 | 105 | 如果 CUDA 安装成功，请使用如下命令来显示GPU的信息。 106 | 107 | .. code-block:: bash 108 | 109 | python -c "import tensorflow" 110 | 111 | 112 | CuDNN 113 | -------- 114 | 115 | 除了 CUDA, NVIDIA 提供一个针对深度学习加速的库--CuDNN。您需要注册NVIDIA开发者，然后才能下载它： 116 | 117 | - `CuDNN 下载连接 `_ 118 | 119 | 下载解压后，把 ``*.h`` 文件复制到 ``/usr/local/cuda/include`` 并把 120 | ``lib*`` 文件复制到 ``/usr/local/cuda/lib64``. 121 | 122 | .. _TensorFlow: https://www.tensorflow.org/versions/master/get_started/os_setup.html 123 | .. _GitHub: https://github.com/tensorlayer/tensorlayer 124 | .. _TensorLayer: https://github.com/tensorlayer/tensorlayer/ 125 | 126 | 127 | 128 | Windows 用户 129 | ============== 130 | 131 | TensorLayer是一个Python库，因此请先给您的Windows安装Python，我们建议安装Python3.5以上的版本。 132 | 133 | `Anaconda 下载 `_ 134 | 135 | GPU 支持 136 | ------------ 137 | 138 | 1. 安装 Microsoft Visual Studio 139 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 140 | 您需要先安装Microsoft Visual Studio (VS)再安装CUDA。最低的版本要求是 VS2010，我们建议安装 VS2015 以上的版本。 141 | 142 | 2. 安装 CUDA 143 | ^^^^^^^^^^^^^^^^^^^^^^^ 144 | 下载并安装最新的CUDA: 145 | 146 | `CUDA download `_ 147 | 148 | 3. 安装 CuDNN 149 | ^^^^^^^^^^^^^^^^^^^^^^ 150 | NVIDIA CUDA® Deep Neural Network library (cuDNN) 是一个针对深度学习开发的GPU加速库。您可以在NIVIDA官网下载之： 151 | 152 | `cuDNN download `_ 153 | 154 | 解压下载文件后，您会得到三个文件夹 (bin, lib, include)。然后这些文件夹里的内容需要复制到CUDA的位置。(默认安装路径是`C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v8.0`) 155 | 156 | 安装 TensorLayer 157 | ------------------------ 158 | For TensorLayer, please refer to the steps mentioned above. 159 | 160 | .. code-block:: bash 161 | 162 | pip install tensorflow #CPU version 163 | pip install tensorflow-gpu   #GPU version (GPU version and CPU version just choose one) 164 | pip install tensorlayer       #Install tensorlayer 165 | 166 | 测试 167 | ------ 168 | .. code-block:: bash 169 | 170 | import tensorlayer 171 | 172 | 如果CUDA，CuDNN安装成功，您会看到如下的信息。 173 | 174 | .. code-block:: bash 175 | 176 | successfully opened CUDA library cublas64_80.dll locally 177 | successfully opened CUDA library cuDNN64_5.dll locally 178 | successfully opened CUDA library cufft64_80.dll locally 179 | successfully opened CUDA library nvcuda.dll locally 180 | successfully opened CUDA library curand64_80.dll locally 181 | 182 | 183 | 184 | 185 | 186 | 187 | 问题 188 | ======= 189 | 190 | 如果您在import时遇到困难，请查看 `FQA `_. 191 | 192 | .. code-block:: bash 193 | 194 | _tkinter.TclError: no display name and no $DISPLAY environment variable 195 | 196 | -------------------------------------------------------------------------------- /docs/user/more.rst: -------------------------------------------------------------------------------- 1 | .. _more: 2 | 3 | ============ 4 | 更多 More 5 | ============ 6 | 7 | FAQ 8 | ======== 9 | 10 | 请见 `英文网站 `_ . 11 | 12 | 你的贡献 13 | ================= 14 | 15 | 贡献方式有很多种，比如提供应用、实现新的TensorLayer层、回答 `GitHub`_ Issues 以及帮助完善翻译等等。 16 | 每一点贡献都会署名，欢迎在 `GitHub`_ 上push。 17 | 18 | .. 19 | 竞赛 20 | ============ 21 | 22 | 有奖竞赛即将到来 23 | 24 | 25 | 26 | 应用例子 27 | ============ 28 | 29 | 有很多基于TensorLayer开发的优秀应用，你或许可以在这里找到适合你项目用的模型。 30 | 此外，如果你想分享你的应用例子，请联系 hao.dong11@imperial.ac.uk. 31 | 32 | 1D CNN+LSTM for Biosignal 33 | ------------------------ 34 | 35 | 作者 : `Akara Supratak `_ 36 | 37 | 介绍 38 | ^^^^^^^^^ 39 | 40 | 实现 41 | ^^^^^^^^^^^ 42 | 43 | 引用 44 | ^^^^^^^^ 45 | 46 | 47 | 招聘信息 48 | ============== 49 | 50 | 51 | 帝国理工大学－数据科学院 52 | ------------------------------------------------ 53 | 54 | Data science is therefore by nature at the core of all modern transdisciplinary scientific activities, as it involves the whole life cycle of data, from acquisition and exploration to analysis and communication of the results. Data science is not only concerned with the tools and methods to obtain, manage and analyse data: it is also about extracting value from data and translating it from asset to product. 55 | 56 | Launched on 1st April 2014, the Data Science Institute at Imperial College London aims to enhance Imperial's excellence in data-driven research across its faculties by fulfilling the following objectives. 57 | 58 | The Data Science Institute is housed in purpose built facilities in the heart of the Imperial College campus in South Kensington. Such a central location provides excellent access to collabroators across the College and across London. 59 | 60 | - To act as a focal point for coordinating data science research at Imperial College by facilitating access to funding, engaging with global partners, and stimulating cross-disciplinary collaboration. 61 | - To develop data management and analysis technologies and services for supporting data driven research in the College. 62 | - To promote the training and education of the new generation of data scientist by developing and coordinating new degree courses, and conducting public outreach programmes on data science. 63 | - To advise College on data strategy and policy by providing world-class data science expertise. 64 | - To enable the translation of data science innovation by close collaboration with industry and supporting commercialization. 65 | 66 | If you are interested in working with us, please check our 67 | `vacancies `_ 68 | and other ways to 69 | `get involved `_ 70 | , or feel free to 71 | `contact us `_. 72 | 73 | 74 | 75 | 76 | .. _GitHub: https://github.com/zsdonghao/tensorlayer 77 | .. _Deeplearning Tutorial: http://deeplearning.stanford.edu/tutorial/ 78 | .. _Convolutional Neural Networks for Visual Recognition: http://cs231n.github.io/ 79 | .. _Neural Networks and Deep Learning: http://neuralnetworksanddeeplearning.com/ 80 | .. _TensorFlow tutorial: https://www.tensorflow.org/versions/r0.9/tutorials/index.html 81 | .. _Understand Deep Reinforcement Learning: http://karpathy.github.io/2016/05/31/rl/ 82 | .. _Understand Recurrent Neural Network: http://karpathy.github.io/2015/05/21/rnn-effectiveness/ 83 | .. _Understand LSTM Network: http://colah.github.io/posts/2015-08-Understanding-LSTMs/ 84 | .. _Word Representations: http://colah.github.io/posts/2014-07-NLP-RNNs-Representations/ 85 | -------------------------------------------------------------------------------- /docs/user/my_figs/basic_seq2seq.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/docs/user/my_figs/basic_seq2seq.png -------------------------------------------------------------------------------- /docs/user/my_figs/book_cover.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/docs/user/my_figs/book_cover.jpeg -------------------------------------------------------------------------------- /docs/user/my_figs/book_description.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/docs/user/my_figs/book_description.jpeg -------------------------------------------------------------------------------- /docs/user/my_figs/img_tensorlayer.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/docs/user/my_figs/img_tensorlayer.png -------------------------------------------------------------------------------- /docs/user/my_figs/img_tlayer.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/docs/user/my_figs/img_tlayer.png -------------------------------------------------------------------------------- /docs/user/my_figs/img_tlayer_big.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/docs/user/my_figs/img_tlayer_big.png -------------------------------------------------------------------------------- /docs/user/my_figs/img_tunelayer.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/docs/user/my_figs/img_tunelayer.png -------------------------------------------------------------------------------- /docs/user/my_figs/karpathy_rnn.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/docs/user/my_figs/karpathy_rnn.jpeg -------------------------------------------------------------------------------- /docs/user/my_figs/mnist.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/docs/user/my_figs/mnist.jpeg -------------------------------------------------------------------------------- /docs/user/my_figs/pong_game.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/docs/user/my_figs/pong_game.jpeg -------------------------------------------------------------------------------- /docs/user/my_figs/tl_transparent_logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/docs/user/my_figs/tl_transparent_logo.png -------------------------------------------------------------------------------- /docs/user/my_figs/tsne.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/docs/user/my_figs/tsne.png -------------------------------------------------------------------------------- /docs/user/my_figs/word2vec_basic.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/docs/user/my_figs/word2vec_basic.pdf -------------------------------------------------------------------------------- /docs/user/tutorial.rst: -------------------------------------------------------------------------------- 1 | .. _tutorial: 2 | 3 | =================== 4 | 教程 Tutorials 5 | =================== 6 | 7 | 对于深度学习，该教程会引导您使用MNIST数据集构建不同的手写数字的分类器， 8 | 这可以说是神经网络的 "Hello World" 。 9 | 对于强化学习，我们将让计算机根据屏幕画面来学习玩乒乓球游戏。 10 | 对于自然语言处理。我们从词嵌套（Word Embedding）开始，然后学习递归网络（Recurrent Network）。 11 | 此外，TensorLayer的Tutorial包含了所有TensorFlow官方深度学习教程的模块化实现，因此您可以对照TensorFlow深度学习教程来学习 `[英文] `_ `[极客学院中文翻译] `_ 。 。 12 | 13 | 14 | .. note:: 15 | 若您已经对TensorFlow非常熟悉，阅读 ``InputLayer`` 和 ``DenseLayer`` 的源代码可让您很好地理解 TensorLayer 是如何工作的。 16 | 17 | 本文档的中文深度学习教程需要更多贡献者参与，有意者请联系 tensorlayer@gmail.com 。 18 | 19 | 20 | 《深度学习：一起玩转TensorLayer》 21 | ================================ 22 | 23 | 好消息！中文社区推出了 `《深度学习：一起玩转TensorLayer》 `_ 一书。本书由TensorLayer创始人领衔，TensorLayer主要开发团队倾力打造而成。内容不仅覆盖了人工神经网络的基本知识，如多层感知器、卷积网络、递归网络及增强学习等，还着重讲解了深度学习的一些新的技术，如生成对抗网络、学习方法和实践经验，配有许多应用及产品的实例。读者可从零开始掌握深度学习技术，以及使用TensorLayer实现的各种应用。 24 | 本书以通俗易懂的方式讲解深度学习技术，同时配有实现方法教学，面向深度学习初学者、进阶者，以及希望长期从事深度学习研究和产品开发的深度学习的大学生和工程师。 25 | 26 | .. image:: my_figs/book_cover.jpeg 27 | :scale: 100 % 28 | :align: center 29 | :target: http://www.broadview.com.cn/book/5059 30 | 31 | .. image:: my_figs/book_description.jpeg 32 | :scale: 90 % 33 | :align: center 34 | :target: http://www.broadview.com.cn/book/5059 35 | 36 | 在我们开始之前 37 | ================== 38 | 39 | 本教程假定您在神经网络和TensorFlow方面具有一定的基础。在深度学习方面，这里推荐一些相关的网络资源。 40 | 41 | 对于人工神经网络更系统的介绍，我们推荐Andrej Karpathy等人所著的 `Convolutional Neural Networks for Visual Recognition`_ 42 | 、Michael Nielsen 的 `Neural Networks and Deep Learning`_ 和 `Deeplearning Tutorial`_ 。 43 | 44 | 要了解TensorFlow的更多内容，请阅读 `TensorFlow Tutorial`_ 。 45 | 您不需要学会所有TensorFlow的细节，只需要知道TensorFlow大概是如何工作的，就能够使用TensorLayer。 46 | 如果您是深度学习新手，建议您阅读整个教程。 47 | 48 | 49 | TensorLayer很简单 50 | ======================= 51 | 52 | 下面的代码是TensorLayer的一个简单例子，来自 `tutorial_mnist_simple.py `_ ，见`所有例子 `_ 。 53 | 我们提供了很多方便的函数（如： ``fit()`` ，``test()`` ），但如果您想了解更多实现细节，或想成为机器学习领域的专家，我们鼓励 54 | 您尽可能地直接使用数据迭代工具箱（``tl.iternate``）加上TensorFlow的操作（如： ``sess.run()``） 来训练模型，请参考 `tutorial_mlp_dropout1.py `_ 和 `tutorial_mlp_dropout2.py _`。 55 | 56 | .. code-block:: python 57 | 58 | import tensorflow as tf 59 | import tensorlayer as tl 60 | 61 | sess = tf.InteractiveSession() 62 | 63 | # 准备数据 64 | X_train, y_train, X_val, y_val, X_test, y_test = \ 65 | tl.files.load_mnist_dataset(shape=(-1,784)) 66 | 67 | # 定义 placeholder 68 | x = tf.placeholder(tf.float32, shape=[None, 784], name='x') 69 | y_ = tf.placeholder(tf.int64, shape=[None, ], name='y_') 70 | 71 | # 定义模型 72 | network = tl.layers.InputLayer(x, name='input_layer') 73 | network = tl.layers.DropoutLayer(network, keep=0.8, name='drop1') 74 | network = tl.layers.DenseLayer(network, n_units=800, 75 | act = tf.nn.relu, name='relu1') 76 | network = tl.layers.DropoutLayer(network, keep=0.5, name='drop2') 77 | network = tl.layers.DenseLayer(network, n_units=800, 78 | act = tf.nn.relu, name='relu2') 79 | network = tl.layers.DropoutLayer(network, keep=0.5, name='drop3') 80 | network = tl.layers.DenseLayer(network, n_units=10, 81 | act = tf.identity, 82 | name='output_layer') 83 | # 定义损失函数和衡量指标 84 | # tl.cost.cross_entropy 在内部使用 tf.nn.sparse_softmax_cross_entropy_with_logits() 实现 softmax 85 | y = network.outputs 86 | cost = tl.cost.cross_entropy(y, y_, name = 'cost') 87 | correct_prediction = tf.equal(tf.argmax(y, 1), y_) 88 | acc = tf.reduce_mean(tf.cast(correct_prediction, tf.float32)) 89 | y_op = tf.argmax(tf.nn.softmax(y), 1) 90 | 91 | # 定义 optimizer 92 | train_params = network.all_params 93 | train_op = tf.train.AdamOptimizer(learning_rate=0.0001, beta1=0.9, beta2=0.999, 94 | epsilon=1e-08, use_locking=False).minimize(cost, var_list=train_params) 95 | 96 |  # 初始化 session 中的所有参数 97 | tl.layers.initialize_global_variables(sess) 98 | 99 | # 列出模型信息 100 | network.print_params() 101 | network.print_layers() 102 | 103 | # 训练模型 104 | tl.utils.fit(sess, network, train_op, cost, X_train, y_train, x, y_, 105 | acc=acc, batch_size=500, n_epoch=500, print_freq=5, 106 | X_val=X_val, y_val=y_val, eval_train=False) 107 | 108 | # 评估模型 109 | tl.utils.test(sess, network, acc, X_test, y_test, x, y_, batch_size=None, cost=cost) 110 | 111 | # 把模型保存成 .npz 文件 112 | tl.files.save_npz(network.all_params , name='model.npz') 113 | sess.close() 114 | 115 | 116 | 117 | 运行MNIST例子 118 | ===================== 119 | 120 | .. _fig_0601: 121 | 122 | .. image:: my_figs/mnist.jpeg 123 | :scale: 100 % 124 | :align: center 125 | 126 | 在本教程的第一部分，我们仅仅运行TensorLayer官方提供的MNIST例子。 127 | MNIST数据集包含了60000个长宽各28像素的灰白手写数字图片，它通常用于测试小型神经网络图像分类的效果。 128 | 129 | 我们假设您已经按照 :ref:`installation` 安装好了TensorLayer。 130 | 如果您还没有，请从Github复制一个TensorLayer的目录到本地，进入该文件夹， 131 | 然后运行 ``tutorial_mnist.py`` 这个例子脚本： 132 | 133 | .. code-block:: bash 134 | 135 | python tutorial_mnist.py 136 | 137 | 如果所有设置都正确，您将得到下面的结果： 138 | 139 | .. code-block:: text 140 | 141 | Downloading train-images-idx3-ubyte.gz 142 | Downloading train-labels-idx1-ubyte.gz 143 | Downloading t10k-images-idx3-ubyte.gz 144 | Downloading t10k-labels-idx1-ubyte.gz 145 | 146 | X_train.shape (50000, 784) 147 | y_train.shape (50000,) 148 | X_val.shape (10000, 784) 149 | y_val.shape (10000,) 150 | X_test.shape (10000, 784) 151 | y_test.shape (10000,) 152 | X float32 y int64 153 | 154 | [TL] InputLayer input_layer (?, 784) 155 | [TL] DropoutLayer drop1: keep: 0.800000 156 | [TL] DenseLayer relu1: 800, relu 157 | [TL] DropoutLayer drop2: keep: 0.500000 158 | [TL] DenseLayer relu2: 800, relu 159 | [TL] DropoutLayer drop3: keep: 0.500000 160 | [TL] DenseLayer output_layer: 10, identity 161 | 162 | param 0: (784, 800) (mean: -0.000053, median: -0.000043 std: 0.035558) 163 | param 1: (800,) (mean: 0.000000, median: 0.000000 std: 0.000000) 164 | param 2: (800, 800) (mean: 0.000008, median: 0.000041 std: 0.035371) 165 | param 3: (800,) (mean: 0.000000, median: 0.000000 std: 0.000000) 166 | param 4: (800, 10) (mean: 0.000469, median: 0.000432 std: 0.049895) 167 | param 5: (10,) (mean: 0.000000, median: 0.000000 std: 0.000000) 168 | num of params: 1276810 169 | 170 | layer 0: Tensor("dropout/mul_1:0", shape=(?, 784), dtype=float32) 171 | layer 1: Tensor("Relu:0", shape=(?, 800), dtype=float32) 172 | layer 2: Tensor("dropout_1/mul_1:0", shape=(?, 800), dtype=float32) 173 | layer 3: Tensor("Relu_1:0", shape=(?, 800), dtype=float32) 174 | layer 4: Tensor("dropout_2/mul_1:0", shape=(?, 800), dtype=float32) 175 | layer 5: Tensor("add_2:0", shape=(?, 10), dtype=float32) 176 | 177 | learning_rate: 0.000100 178 | batch_size: 128 179 | 180 | Epoch 1 of 500 took 0.342539s 181 | train loss: 0.330111 182 | val loss: 0.298098 183 | val acc: 0.910700 184 | Epoch 10 of 500 took 0.356471s 185 | train loss: 0.085225 186 | val loss: 0.097082 187 | val acc: 0.971700 188 | Epoch 20 of 500 took 0.352137s 189 | train loss: 0.040741 190 | val loss: 0.070149 191 | val acc: 0.978600 192 | Epoch 30 of 500 took 0.350814s 193 | train loss: 0.022995 194 | val loss: 0.060471 195 | val acc: 0.982800 196 | Epoch 40 of 500 took 0.350996s 197 | train loss: 0.013713 198 | val loss: 0.055777 199 | val acc: 0.983700 200 | ... 201 | 202 | 这个例子脚本允许您从 ``if__name__=='__main__':`` 中选择不同的模型进行尝试，包括多层神经网络（Multi-Layer Perceptron）， 203 | 退出（Dropout），退出连接（DropConnect），堆栈式降噪自编码器（Stacked Denoising Autoencoder）和卷积神经网络（CNN）。 204 | 205 | .. code-block:: python 206 | 207 | main_test_layers(model='relu') 208 | main_test_denoise_AE(model='relu') 209 | main_test_stacked_denoise_AE(model='relu') 210 | main_test_cnn_layer() 211 | 212 | 213 | 214 | 理解MNIST例子 215 | ===================== 216 | 217 | 现在就让我们看看这些代码是如何工作的！ 218 | 219 | 序言 220 | ----------- 221 | 222 | 您可能会首先注意到，除TensorLayer之外，我们还导入了Numpy和TensorFlow： 223 | 224 | .. code-block:: python 225 | 226 | import time 227 | import numpy as np 228 | import tensorflow as tf 229 | import tensorlayer as tl 230 | 231 | 232 | 这是因为TensorLayer是建立在TensorFlow上的，TensorLayer设计的初衷是为了简化工作并提供帮助而不是取代TensorFlow。 233 | 所以您会需要一起使用TensorLayer和一些常见的TensorFlow代码。 234 | 235 | 请注意，当使用降噪自编码器(Denoising Autoencoder)时，代码中的 ``set_keep`` 被当作用来访问保持概率(Keeping Probabilities)的占位符。 236 | 237 | 238 | 载入数据 239 | ------------- 240 | 241 | 下面第一部分的代码首先定义了 ``load_mnist_dataset()`` 函数。 242 | 其目的是为了自动下载MNIST数据集（如果还未下载），并且返回标准numpy数列通过numpy array的格式。 243 | 到这里还没有涉及TensorLayer。 244 | 245 | .. code-block:: python 246 | 247 | X_train, y_train, X_val, y_val, X_test, y_test = \ 248 | tl.files.load_mnist_dataset(shape=(-1,784)) 249 | 250 | 251 | ``X_train.shape`` 为 ``(50000,784)``，可以理解成共有50000张图片并且每张图片有784个数值（像素点）。 252 | ``Y_train.shape`` 为 ``(50000,)`` ，它是一个和 ``X_train`` 长度相同的向量，用于给出每幅图的数字标签，即这些图片所包含的位于0-9之间的10个数字。 253 | 254 | 另外对于卷积神经网络的例子，MNIST还可以按下面的4D版本来载入： 255 | 256 | .. code-block:: python 257 | 258 | X_train, y_train, X_val, y_val, X_test, y_test = \ 259 | tl.files.load_mnist_dataset(shape=(-1, 28, 28, 1)) 260 | 261 | ``X_train.shape`` 为 ``(50000,28,28,1)`` ，这代表了50000张图片，每张图片有28行和28列。 262 | 通道为1是因为它是灰度图像，所以每个像素只能有一个值。 263 | 264 | 建立模型 265 | ---------------- 266 | 267 | 来到这里，就轮到TensorLayer来一显身手了！TensorLayer允许您通过创建，堆叠或者合并图层(Layers)来定义任意结构的神经网络。 268 | 由于每一层都知道它在一个网络中的直接输入层和（多个）输出接收层，所以通常这是我们唯一要传递给其他代码的内容。 269 | 270 | 正如上文提到的， ``tutorial_mnist.py`` 有四个例子。 271 | 首先，我们将定义一个结构固定的多层次感知器（Multi-Layer Perceptron），所有的步骤都会详细的讲解。 272 | 然后，我们会实现一个去噪自编码器(Denosing Autoencoding)。 273 | 接着，我们要将所有去噪自编码器堆叠起来并对他们进行监督微调(Supervised Fine-tune)。 274 | 最后，我们将展示如何去创建一个卷积神经网络(Convolutional Neural Network)。 275 | 276 | 此外，如果您有兴趣，我们还提供了一个简化版的MNIST例子在 ``tutorial_mnist_simple.py`` 中，和一个对于 277 | -10数据集的卷积神经网络(CNN)的例子在 ``tutorial_cifar10_tfrecord.py`` 中可供参考。 278 | 279 | 多层神经网络 (Multi-Layer Perceptron) 280 | ------------------------------------------------ 281 | 282 | 第一个脚本 ``main_test_layers()`` ，创建了一个具有两个隐藏层，每层800个单元的多层感知器，最后一层是10个单元的Softmax输出层。 283 | 它对输入数据采用保留80%数值的Dropout操作，并且对隐藏层使用50%的Dropout操作。这里并不会介绍Dropout原理，感兴趣的朋友可以在网上找到大量相关资料。 284 | 285 | 为了提供数据给这个网络，TensorFlow可以通过placeholder实现，需如下定义。 286 | 在这里 ``None`` 是指在编译之后，网络将接受任意批规模（Batch Size）的数据 287 | ``x`` 是用来输入 ``X_train`` 数据的，而 ``y_`` 是用来输入 ``y_train`` 数据的。 288 | 如果您想固定Batch Size，您可以把 ``None`` 代替为给定数值，这样可以运用TensorFlow一些优化功能，特别是当网络特别大的时候。 289 | 290 | .. code-block:: python 291 | 292 | x = tf.placeholder(tf.float32, shape=[None, 784], name='x') 293 | y_ = tf.placeholder(tf.int64, shape=[None, ], name='y_') 294 | 295 | 在TensorLayer中每个神经网络的基础是一个 :class:`InputLayer ` 实例。它代表了将要提供给网络的输入数据。 296 | 297 | .. code-block:: python 298 | 299 | network = tl.layers.InputLayer(x, name='input_layer') 300 | 301 | 在添加第一层隐藏层之前，我们要对输入数据进行Dropout操作。 302 | 这里我们通过一个 :class:`DropoutLayer` 来实现。 303 | 304 | .. code-block:: python 305 | 306 | network = tl.layers.DropoutLayer(network, keep=0.8, name='drop1') 307 | 308 | 请注意这里的第一个参数是输入层，第二个参数是保持概率（Keeping probability for the activation value），则数值不被置为零的概率。 309 | 现在我们要继续构造第一个800个单位的全连接的隐藏层。 310 | 尤其是当要堆叠一个 :class:`DenseLayer ` 时，要特别注意。 311 | 312 | .. code-block:: python 313 | 314 | network = tl.layers.DenseLayer(network, n_units=800, act = tf.nn.relu, name='relu1') 315 | 316 | 同样，加一个新层时，我们在原来的 ``network`` 之上堆叠出新的 ``network`` 。 317 | ``n_units`` 简明地给出了新全连接层的神经元单位数。 318 | ``act`` 指定了一个激活函数，这里的激活函数有一部分已经被定义在了 :mod:`tensorflow.nn` 和 `tensorlayer.activation` 中。 319 | 我们在这里选择了整流器（Rectifier）作为激活函数。 320 | 接着继续添加50%的Dropout层，以及另外800个单元的全链接层（Dense layer），和50%的Dropout层： 321 | 322 | .. code-block:: python 323 | 324 | network = tl.layers.DropoutLayer(network, keep=0.5, name='drop2') 325 | network = tl.layers.DenseLayer(network, n_units=800, act = tf.nn.relu, name='relu2') 326 | network = tl.layers.DropoutLayer(network, keep=0.5, name='drop3') 327 | 328 | 最后，我们加入 ``n_units`` 等于分类个数的全连接的输出层。注意， ``cost = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(y, y_))`` 在内部实现 Softmax，以提高计算效率，因此最后一层的输出为 identity ，更多细节请参考 ``tl.cost.cross_entropy()`` 。 329 | 330 | .. code-block:: python 331 | 332 | network = tl.layers.DenseLayer(network, 333 | n_units=10, 334 | act = tl.act.identity, 335 | name='output_layer') 336 | 337 | 如上所述，因为每一层都被链接到了它的输入层，所以我们只需要在TensorLayer中将输出层接入一个网络： 338 | 339 | .. code-block:: python 340 | 341 | y = network.outputs 342 | y_op = tf.argmax(tf.nn.softmax(y), 1) 343 | cost = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(y, y_)) 344 | 345 | 在这里，``network.outputs`` 是网络的10个种类的输出概率（以One-hot的形式）。 346 | ``y_op`` 是代表类索引的整数输出， ``cost`` 是目标和预测标签的交叉熵（Cross Entropy）。 347 | 348 | 降噪自编码器(Denoising Autoencoder) 349 | ------------------------------------------------ 350 | 351 | 自编码器是一种无监督学习（Unsupervisered Learning）模型，可从数据中学习出更好的表达， 352 | 目前已经用于逐层贪婪的预训练（Greedy layer-wise pre-train）。 353 | 有关Vanilla自编码器，请参考教程 `Deeplearning Tutorial`_。 354 | 355 | `tutorial_mnist.py `_ 的 ``main_test_denoise_AE()`` 实现了50%的腐蚀率（Corrosion Rate）的去噪自编码器。 356 | 这个自编码器可以按如下方式定义，这里的使用全链接层来搭建一个自编码器： 357 | 358 | .. code-block:: python 359 | 360 | network = tl.layers.InputLayer(x, name='input_layer') 361 | network = tl.layers.DropoutLayer(network, keep=0.5, name='denoising1') 362 | network = tl.layers.DenseLayer(network, n_units=200, act=tf.nn.sigmoid, name='sigmoid1') 363 | recon_layer1 = tl.layers.ReconLayer(network, 364 | x_recon=x, 365 | n_units=784, 366 | act=tf.nn.sigmoid, 367 | name='recon_layer1') 368 | 369 | 训练 ``DenseLayer`` ，只需要运行 ``ReconLayer.pretrain()`` 即可。 370 | 如果要使用去噪自编码器，可以使用 ``DropoutLayer`` 作为腐蚀层（Corrosion layer）。 371 | 372 | 对于Sigmoid型激活函数来说，自编码器可以用KL散度来实现。 373 | 而对于整流器（Rectifier）来说，对激活函数输出的L1正则化能使得输出变得稀疏。 374 | 所以 ``ReconLayer`` 默认只对整流激活函数(ReLU)提供KL散度和交叉熵这两种损失度量，而对Sigmoid型激活函数提供均方误差以及激活输出的L1范数这两种损失度量。 375 | 我们建议您修改 ``ReconLayer`` 来实现自己的预训练方式。 376 | 377 | .. code-block:: python 378 | 379 | recon_layer1.pretrain(sess, 380 | x=x, 381 | X_train=X_train, 382 | X_val=X_val, 383 | denoise_name='denoising1', 384 | n_epoch=200, 385 | batch_size=128, 386 | print_freq=10, 387 | save=True, 388 | save_name='w1pre_') 389 | 390 | 此外，脚本 ``main_test_stacked_denoise_AE()`` 展示了如何将多个自编码器堆叠到一个网络，然后进行微调。 391 | 392 | 卷积神经网络(Convolutional Neural Network) 393 | ---------------------------------------------------------- 394 | 395 | `tutorial_mnist.py `_ 的 ``main_test_cnn_layer()`` 创建了如下的一个卷积网络分类器。 396 | 397 | .. code-block:: python 398 | 399 | network = tl.layers.Conv2d(network, 32, (5, 5), (1, 1), 400 | act=tf.nn.relu, padding='SAME', name='cnn1') 401 | network = tl.layers.MaxPool2d(network, (2, 2), (2, 2), 402 | padding='SAME', name='pool1') 403 | network = tl.layers.Conv2d(network, 64, (5, 5), (1, 1), 404 | act=tf.nn.relu, padding='SAME', name='cnn2') 405 | network = tl.layers.MaxPool2d(network, (2, 2), (2, 2), 406 | padding='SAME', name='pool2') 407 | 408 | network = tl.layers.FlattenLayer(network, name='flatten') 409 | network = tl.layers.DropoutLayer(network, keep=0.5, name='drop1') 410 | network = tl.layers.DenseLayer(network, 256, act=tf.nn.relu, name='relu1') 411 | network = tl.layers.DropoutLayer(network, keep=0.5, name='drop2') 412 | network = tl.layers.DenseLayer(network, 10, act=tf.identity, name='output') 413 | 414 | 415 | 训练模型 416 | ---------------- 417 | 418 | 在 ``tutorial_mnist.py`` 脚本的其余部分，在MNIST数据上对于只使用交叉熵的循环训练进行了设置并且运行。 419 | 420 | 数据集迭代 421 | ^^^^^^^^^^^^^ 422 | 423 | 一个在给定的项目数的最小批规模下的输入特征及其对应的标签的两个Numpy数列依次同步的迭代函数。 424 | 更多有关迭代函数的说明，可以在 ``tensorlayer.iterate`` 中找到。 425 | 426 | .. code-block:: python 427 | 428 | tl.iterate.minibatches(inputs, targets, batchsize, shuffle=False) 429 | 430 | 损失和更新公式 431 | ^^^^^^^^^^^^^^ 432 | 433 | 我们继续创建一个在训练中被最小化的损失表达式： 434 | 435 | .. code-block:: python 436 | 437 | y = network.outputs 438 | y_op = tf.argmax(tf.nn.softmax(y), 1) 439 | cost = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(y, y_)) 440 | 441 | 更多的损失函数或者正则化方法可以在这里定义。比如，如果要在权重矩阵中应用最大范数（Max-norm）方法，您可以添加下列代码。 442 | 443 | .. code-block:: python 444 | 445 | cost = cost + tl.cost.maxnorm_regularizer(1.0)(network.all_params[0]) + 446 | tl.cost.maxnorm_regularizer(1.0)(network.all_params[2]) 447 | 448 | 根据要解决的问题，您会需要使用不同的损失函数，更多有关损失函数的说明请见： `tensorlayer.cost` 449 | 除了通过 ``network.all_params`` 来获取网络参数，您还可以通过 ``tl.layers.get_variables_with_name`` 来通过字符串方式获取指定的参数。 450 | 451 | 有了模型和定义的损失函数之后，我们就可以创建用于训练网络的更新公式。 452 | 接下去，我们将使用TensorFlow的优化器如下： 453 | 454 | .. code-block:: python 455 | 456 | train_params = network.all_params 457 | train_op = tf.train.AdamOptimizer(learning_rate, beta1=0.9, beta2=0.999, 458 | epsilon=1e-08, use_locking=False).minimize(cost, var_list=train_params) 459 | 460 | 为了训练网络，我们需要提供数据和保持概率给 ``feed_dict``。 461 | 462 | .. code-block:: python 463 | 464 | feed_dict = {x: X_train_a, y_: y_train_a} 465 | feed_dict.update( network.all_drop ) 466 | sess.run(train_op, feed_dict=feed_dict) 467 | 468 | 同时为了进行验证和测试，我们这里用了略有不同的方法。 469 | 所有的Dropout，DropConnect，腐蚀层（Corrosion Layers）都将被禁用。 470 | ``tl.utils.dict_to_one`` 将会设置所有 ``network.all_drop`` 值为1。 471 | 472 | .. code-block:: python 473 | 474 | dp_dict = tl.utils.dict_to_one( network.all_drop ) 475 | feed_dict = {x: X_test_a, y_: y_test_a} 476 | feed_dict.update(dp_dict) 477 | err, ac = sess.run([cost, acc], feed_dict=feed_dict) 478 | 479 | 最后，作为一个额外的监测量，我们需要创建一个分类准确度的公式： 480 | 481 | .. code-block:: python 482 | 483 | correct_prediction = tf.equal(tf.argmax(y, 1), y_) 484 | acc = tf.reduce_mean(tf.cast(correct_prediction, tf.float32)) 485 | 486 | 下一步？ 487 | ^^^^^^^^^^^^^^ 488 | 489 | 在 `tutorial_cifar10_tfrecord.py `_ 中我们还有更复杂的图像分类的例子。 490 | 请阅读代码及注释，以明白如数据增强（Data Augmentation）的重要性，以及什么是局部响应正则化。 491 | 此外，您可以尝试着去实现 `残差网络(Residual Network) `_。 492 | 493 | 494 | 运行乒乓球例子 495 | ==================== 496 | 497 | 在本教程的第二部分，我们将运行一个深度强化学习的例子，它在Karpathy的两篇博客 `Deep Reinforcement Learning:Pong from Pixels `_ 有介绍。 498 | 499 | .. code-block:: bash 500 | 501 | python tutorial_atari_pong.py 502 | 503 | 在运行教程代码之前 您需要安装 `OpenAI gym environment `_ ，它提供了大量强化学习常用的游戏环境。 504 | 如果一切运行正常，您将得到以下的输出： 505 | 506 | .. code-block:: text 507 | 508 | [2016-07-12 09:31:59,760] Making new env: Pong-v0 509 | [TL] InputLayer input_layer (?, 6400) 510 | [TL] DenseLayer relu1: 200, relu 511 | [TL] DenseLayer output_layer: 3, identity 512 | param 0: (6400, 200) (mean: -0.000009 median: -0.000018 std: 0.017393) 513 | param 1: (200,) (mean: 0.000000 median: 0.000000 std: 0.000000) 514 | param 2: (200, 3) (mean: 0.002239 median: 0.003122 std: 0.096611) 515 | param 3: (3,) (mean: 0.000000 median: 0.000000 std: 0.000000) 516 | num of params: 1280803 517 | layer 0: Tensor("Relu:0", shape=(?, 200), dtype=float32) 518 | layer 1: Tensor("add_1:0", shape=(?, 3), dtype=float32) 519 | episode 0: game 0 took 0.17381s, reward: -1.000000 520 | episode 0: game 1 took 0.12629s, reward: 1.000000 !!!!!!!! 521 | episode 0: game 2 took 0.17082s, reward: -1.000000 522 | episode 0: game 3 took 0.08944s, reward: -1.000000 523 | episode 0: game 4 took 0.09446s, reward: -1.000000 524 | episode 0: game 5 took 0.09440s, reward: -1.000000 525 | episode 0: game 6 took 0.32798s, reward: -1.000000 526 | episode 0: game 7 took 0.74437s, reward: -1.000000 527 | episode 0: game 8 took 0.43013s, reward: -1.000000 528 | episode 0: game 9 took 0.42496s, reward: -1.000000 529 | episode 0: game 10 took 0.37128s, reward: -1.000000 530 | episode 0: game 11 took 0.08979s, reward: -1.000000 531 | episode 0: game 12 took 0.09138s, reward: -1.000000 532 | episode 0: game 13 took 0.09142s, reward: -1.000000 533 | episode 0: game 14 took 0.09639s, reward: -1.000000 534 | episode 0: game 15 took 0.09852s, reward: -1.000000 535 | episode 0: game 16 took 0.09984s, reward: -1.000000 536 | episode 0: game 17 took 0.09575s, reward: -1.000000 537 | episode 0: game 18 took 0.09416s, reward: -1.000000 538 | episode 0: game 19 took 0.08674s, reward: -1.000000 539 | episode 0: game 20 took 0.09628s, reward: -1.000000 540 | resetting env. episode reward total was -20.000000. running mean: -20.000000 541 | episode 1: game 0 took 0.09910s, reward: -1.000000 542 | episode 1: game 1 took 0.17056s, reward: -1.000000 543 | episode 1: game 2 took 0.09306s, reward: -1.000000 544 | episode 1: game 3 took 0.09556s, reward: -1.000000 545 | episode 1: game 4 took 0.12520s, reward: 1.000000 !!!!!!!! 546 | episode 1: game 5 took 0.17348s, reward: -1.000000 547 | episode 1: game 6 took 0.09415s, reward: -1.000000 548 | 549 | 这个例子让神经网络通过游戏画面来学习如何像人类一样打乒乓球。神经网络将于伪AI电脑对战不断地对战，最后学会战胜它。 550 | 在经过15000个序列的训练之后，神经网络就可以赢得20%的比赛。 551 | 在20000个序列的训练之后，神经网络可以赢得35%的比赛， 552 | 我们可以看到计算机学的越来越快，这是因为它有更多的胜利的数据来进行训练。 553 | 训练了30000个序列后，神经网络再也不会输了。 554 | 555 | .. code-block:: python 556 | 557 | render = False 558 | resume = False 559 | 560 | 如果您想显示游戏过程，那就设置 `render` 为 `True` 。 561 | 当您再次运行该代码，您可以设置 `resume` 为 `True`，那么代码将加载现有的模型并且会基于它继续训练。 562 | 563 | .. _fig_0601: 564 | 565 | .. image:: my_figs/pong_game.jpeg 566 | :scale: 30 % 567 | :align: center 568 | 569 | 理解强化学习 570 | =================== 571 | 572 | 乒乓球 573 | ------------- 574 | 575 | 要理解强化学习，我们要让电脑学习如何从原始的屏幕输入（像素输入）打乒乓球。 576 | 在我们开始之前，我们强烈建议您去浏览一个著名的博客叫做 `Deep Reinforcement Learning:pong from Pixels `_ , 577 | 这是使用python numpy库和OpenAI gym environment来实现的一个深度强化学习的例子。 578 | 579 | .. code-block:: bash 580 | 581 | python tutorial_atari_pong.py 582 | 583 | 策略网络(Policy Network) 584 | --------------------------- 585 | 586 | 在深度强化学习中，Policy Network 等同于深度神经网络。 587 | 它是我们的选手（或者说“代理人（Agent）“），它的输出告诉我们应该做什么（如：向上移动或向下移动）： 588 | 在Karpathy的代码中，他只定义了2个动作，向上移动和向下移动，并且仅使用单个Simgoid输出： 589 | 为了使我们的教程更具有普遍性，我们使用3个Softmax输出来定义向上移动，向下移动和停止（什么都不做）3个动作。 590 | 591 | .. code-block:: python 592 | 593 | # observation for training 594 | states_batch_pl = tf.placeholder(tf.float32, shape=[None, D]) 595 | 596 | network = tl.layers.InputLayer(states_batch_pl, name='input_layer') 597 | network = tl.layers.DenseLayer(network, n_units=H, 598 | act = tf.nn.relu, name='relu1') 599 | network = tl.layers.DenseLayer(network, n_units=3, 600 | act = tl.activation.identity, name='output_layer') 601 | probs = network.outputs 602 | sampling_prob = tf.nn.softmax(probs) 603 | 604 | 然后我们的代理人就一直与伪AI对战。它计算不同动作的概率，并且之后会从这个均匀的分布中选取样本（动作）。 605 | 因为动作被1,2和3代表，但是Softmax输出应该从0开始，所以我们会把动作索引减去1来作为网络输出。 606 | 607 | .. code-block:: python 608 | 609 | prob = sess.run( 610 | sampling_prob, 611 | feed_dict={states_batch_pl: x} 612 | ) 613 | # action. 1: STOP 2: UP 3: DOWN 614 | action = np.random.choice([1,2,3], p=prob.flatten()) 615 | ... 616 | ys.append(action - 1) 617 | 618 | 策略逼近(Policy Gradient) 619 | --------------------------- 620 | 621 | 策略梯度下降法是一个end-to-end的算法，它直接学习从状态映射到动作的策略函数。 622 | 一个近似最优的策略可以通过最大化预期的奖励来直接学习。 623 | 策略函数的参数(例如，在乒乓球例子终使用的策略网络的参数)在预期奖励的近似值的引导下能够被训练和学习。 624 | 换句话说，我们可以通过过更新它的参数来逐步调整策略函数，这样它能从给定的状态做出一系列行为来获得更高的奖励。 625 | 626 | 策略迭代的一个替代算法就是深度Q-learning（DQN）。 627 | 他是基于Q-learning,学习一个映射状态和动作到一些值的价值函数的算法(叫Q函数)。 628 | DQN采用了一个深度神经网络来作为Q函数的逼近来代表Q函数。 629 | 训练是通过最小化时序差分（Temporal-Difference）误差来实现。 630 | 一个名为“再体验（Experience Replay）“的神经生物学的启发式机制通常和DQN一起被使用来帮助提高非线性函数的逼近的稳定性。 631 | 632 | 您可以阅读以下文档，来得到对强化学习更好的理解： 633 | 634 | - `Reinforcement Learning: An Introduction. Richard S. Sutton and Andrew G. Barto `_ 635 | - `Deep Reinforcement Learning. David Silver, Google DeepMind `_ 636 | - `UCL Course on RL `_ 637 | 638 | 强化深度学习近些年来最成功的应用就是让模型去学习玩Atari的游戏。 AlphaGO同时也是使用类似的策略逼近方法来训练他们的策略网络而战胜了世界级的专业围棋选手。 639 | 640 | - `Atari - Playing Atari with Deep Reinforcement Learning `_ 641 | - `Atari - Human-level control through deep reinforcement learning `_ 642 | - `AlphaGO - Mastering the game of Go with deep neural networks and tree search `_ 643 | 644 | 645 | 数据集迭代 646 | ^^^^^^^^^^^^^^ 647 | 648 | 在强化学习中，我们把每场比赛所产生的所有决策来作为一个序列 (up,up,stop,...,down)。在乒乓球游戏中，比赛是在某一方达到21分后结束的，所以一个序列可能包含几十个决策。 649 | 然后我们可以设置一个批规模的大小，每一批包含一定数量的序列，基于这个批规模来更新我们的模型。 650 | 在本教程中，我们把每批规模设置成10个序列。使用RMSProp训练一个具有200个单元的隐藏层的2层策略网络 651 | 652 | 损失和更新公式 653 | ^^^^^^^^^^^^^^^^^^^ 654 | 655 | 接着我们创建一个在训练中被最小化的损失公式： 656 | 657 | .. code-block:: python 658 | 659 | actions_batch_pl = tf.placeholder(tf.int32, shape=[None]) 660 | discount_rewards_batch_pl = tf.placeholder(tf.float32, shape=[None]) 661 | loss = tl.rein.cross_entropy_reward_loss(probs, actions_batch_pl, 662 | discount_rewards_batch_pl) 663 | ... 664 | ... 665 | sess.run( 666 | train_op, 667 | feed_dict={ 668 | states_batch_pl: epx, 669 | actions_batch_pl: epy, 670 | discount_rewards_batch_pl: disR 671 | } 672 | ) 673 | 674 | 一个batch的损失和一个batch内的策略网络的所有输出，所有的我们做出的动作和相应的被打折的奖励有关 675 | 我们首先通过累加被打折的奖励和实际输出和真实动作的交叉熵计算每一个动作的损失。 676 | 最后的损失是所有动作的损失的和。 677 | 678 | 下一步? 679 | ---------------- 680 | 681 | 上述教程展示了您如何去建立自己的代理人，end-to-end。 682 | 虽然它有很合理的品质，但它的默认参数不会给您最好的代理人模型。 683 | 这有一些您可以优化的内容。 684 | 685 | 首先，与传统的MLP模型不同，比起 `Playing Atari with Deep Reinforcement Learning `_ 更好的是我们可以使用CNNs来采集屏幕信息 686 | 687 | 另外这个模型默认参数没有调整，您可以更改学习率，衰退率，或者用不同的方式来初始化您的模型的权重。 688 | 689 | 最后，您可以尝试不同任务，以及学习其他增强学习算法，请见`Example `_ 。 690 | 691 | 692 | 693 | 694 | 运行Word2Vec例子 695 | ==================== 696 | 697 | 在教程的这一部分，我们训练一个词嵌套矩阵，每个词可以通过矩阵中唯一的行向量来表示。 698 | 在训练结束时，含义类似的单词会有相识的词向量输出。 699 | 在代码的最后，我们通过把单词放到一个2D平面上来可视化，我们可以看到相似的单词会被聚集在一起。 700 | 701 | 702 | .. code-block:: bash 703 | 704 | python tutorial_word2vec_basic.py 705 | 706 | 如果一切设置正确，您最后会得到如下的可视化图。 707 | 708 | .. _fig_0601: 709 | 710 | .. image:: my_figs/tsne.png 711 | :scale: 100 % 712 | :align: center 713 | 714 | 理解词嵌套(word embedding) 715 | ================================= 716 | 717 | 词嵌套（嵌入） 718 | ------------------- 719 | 720 | 我们强烈建议您先阅读Colah的博客 `Word Representations`_ `[中文翻译] `_ ， 721 | 以理解为什么我们要使用一个向量来表示一个单词。更多Word2vec的细节可以在 `Word2vec Parameter Learning Explained `_ 中找到。 722 | 723 | 基本来说，训练一个嵌套矩阵是一个非监督学习的过程。一个单词使用唯一的ID来表示，而这个ID号就是嵌套矩阵的行号（row index），对应的行向量就是用来表示该单词的，使用向量来表示单词可以更好地表达单词的意思。比如，有4个单词的向量， ``woman − man = queen - king`` ，这个例子中可以看到，嵌套矩阵中有一个纬度是用来表示性别的。 724 | 725 | 726 | 定义一个Word2vec词嵌套矩阵如下。 727 | 728 | .. code-block:: python 729 | 730 | # train_inputs is a row vector, a input is an integer id of single word. 731 | # train_labels is a column vector, a label is an integer id of single word. 732 | # valid_dataset is a column vector, a valid set is an integer id of single word. 733 | train_inputs = tf.placeholder(tf.int32, shape=[batch_size]) 734 | train_labels = tf.placeholder(tf.int32, shape=[batch_size, 1]) 735 | valid_dataset = tf.constant(valid_examples, dtype=tf.int32) 736 | 737 | # Look up embeddings for inputs. 738 | emb_net = tl.layers.Word2vecEmbeddingInputlayer( 739 | inputs = train_inputs, 740 | train_labels = train_labels, 741 | vocabulary_size = vocabulary_size, 742 | embedding_size = embedding_size, 743 | num_sampled = num_sampled, 744 | nce_loss_args = {}, 745 | E_init = tf.random_uniform_initializer(minval=-1.0, maxval=1.0), 746 | E_init_args = {}, 747 | nce_W_init = tf.truncated_normal_initializer( 748 | stddev=float(1.0/np.sqrt(embedding_size))), 749 | nce_W_init_args = {}, 750 | nce_b_init = tf.constant_initializer(value=0.0), 751 | nce_b_init_args = {}, 752 | name ='word2vec_layer', 753 | ) 754 | 755 | 756 | 数据迭代和损失函数 757 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 758 | Word2vec使用负采样（Negative sampling）和Skip-gram模型进行训练。 759 | 噪音对比估计损失（NCE）会帮助减少损失函数的计算量，加快训练速度。 760 | Skip-Gram 将文本（context）和目标（target）反转，尝试从目标单词预测目标文本单词。 761 | 我们使用 ``tl.nlp.generate_skip_gram_batch`` 函数来生成训练数据，如下： 762 | 763 | .. code-block:: python 764 | 765 | # NCE损失函数由 Word2vecEmbeddingInputlayer 提供 766 | cost = emb_net.nce_cost 767 | train_params = emb_net.all_params 768 | 769 | train_op = tf.train.AdagradOptimizer(learning_rate, initial_accumulator_value=0.1, 770 | use_locking=False).minimize(cost, var_list=train_params) 771 | 772 | data_index = 0 773 | while (step < num_steps): 774 | batch_inputs, batch_labels, data_index = tl.nlp.generate_skip_gram_batch( 775 | data=data, batch_size=batch_size, num_skips=num_skips, 776 | skip_window=skip_window, data_index=data_index) 777 | feed_dict = {train_inputs : batch_inputs, train_labels : batch_labels} 778 | _, loss_val = sess.run([train_op, cost], feed_dict=feed_dict) 779 | 780 | 781 | 加载已训练好的的词嵌套矩阵 782 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 783 | 784 | 在训练嵌套矩阵的最后，我们保存矩阵及其词汇表、单词转ID字典、ID转单词字典。 785 | 然后，当下次做实际应用时，可以想下面的代码中那样加载这个已经训练好的矩阵和字典， 786 | 参考 ``tutorial_generate_text.py`` 。 787 | 788 | .. code-block:: python 789 | 790 | vocabulary_size = 50000 791 | embedding_size = 128 792 | model_file_name = "model_word2vec_50k_128" 793 | batch_size = None 794 | 795 | print("Load existing embedding matrix and dictionaries") 796 | all_var = tl.files.load_npy_to_any(name=model_file_name+'.npy') 797 | data = all_var['data']; count = all_var['count'] 798 | dictionary = all_var['dictionary'] 799 | reverse_dictionary = all_var['reverse_dictionary'] 800 | 801 | tl.nlp.save_vocab(count, name='vocab_'+model_file_name+'.txt') 802 | 803 | del all_var, data, count 804 | 805 | load_params = tl.files.load_npz(name=model_file_name+'.npz') 806 | 807 | x = tf.placeholder(tf.int32, shape=[batch_size]) 808 | y_ = tf.placeholder(tf.int32, shape=[batch_size, 1]) 809 | 810 | emb_net = tl.layers.EmbeddingInputlayer( 811 | inputs = x, 812 | vocabulary_size = vocabulary_size, 813 | embedding_size = embedding_size, 814 | name ='embedding_layer') 815 | 816 | tl.layers.initialize_global_variables(sess) 817 | 818 | tl.files.assign_params(sess, [load_params[0]], emb_net) 819 | 820 | 821 | 运行PTB例子 822 | ================== 823 | 824 | Penn TreeBank（PTB）数据集被用在很多语言建模（Language Modeling）的论文中，包括"Empirical Evaluation and Combination of Advanced Language Modeling Techniques"和 825 | “Recurrent Neural Network Regularization”。该数据集的训练集有929k个单词，验证集有73K个单词，测试集有82k个单词。 826 | 在它的词汇表刚好有10k个单词。 827 | 828 | PTB例子是为了展示如何用递归神经网络（Recurrent Neural Network）来进行语言建模的。 829 | 830 | 给一句话 "I am from Imperial College London", 这个模型可以从中学习出如何从“from Imperial College”来预测出“Imperial College London”。也就是说，它根据之前输入的单词序列来预测出下一步输出的单词序列，在刚才的例子中 ``num_steps (序列长度，sequence length)`` 为 3。 831 | 832 | 833 | .. code-block:: bash 834 | 835 | python tutorial_ptb_lstm.py 836 | 837 | 该脚本提供三种设置(小，中，大)，越大的模型有越好的建模性能，您可以修改下面的代码片段来选择不同的模型设置。 838 | 839 | .. code-block:: python 840 | 841 | flags.DEFINE_string( 842 | "model", "small", 843 | "A type of model. Possible options are: small, medium, large.") 844 | 845 | 846 | 如果您选择小设置，您将会看到： 847 | 848 | .. code-block:: text 849 | 850 | Epoch: 1 Learning rate: 1.000 851 | 0.004 perplexity: 5220.213 speed: 7635 wps 852 | 0.104 perplexity: 828.871 speed: 8469 wps 853 | 0.204 perplexity: 614.071 speed: 8839 wps 854 | 0.304 perplexity: 495.485 speed: 8889 wps 855 | 0.404 perplexity: 427.381 speed: 8940 wps 856 | 0.504 perplexity: 383.063 speed: 8920 wps 857 | 0.604 perplexity: 345.135 speed: 8920 wps 858 | 0.703 perplexity: 319.263 speed: 8949 wps 859 | 0.803 perplexity: 298.774 speed: 8975 wps 860 | 0.903 perplexity: 279.817 speed: 8986 wps 861 | Epoch: 1 Train Perplexity: 265.558 862 | Epoch: 1 Valid Perplexity: 178.436 863 | ... 864 | Epoch: 13 Learning rate: 0.004 865 | 0.004 perplexity: 56.122 speed: 8594 wps 866 | 0.104 perplexity: 40.793 speed: 9186 wps 867 | 0.204 perplexity: 44.527 speed: 9117 wps 868 | 0.304 perplexity: 42.668 speed: 9214 wps 869 | 0.404 perplexity: 41.943 speed: 9269 wps 870 | 0.504 perplexity: 41.286 speed: 9271 wps 871 | 0.604 perplexity: 39.989 speed: 9244 wps 872 | 0.703 perplexity: 39.403 speed: 9236 wps 873 | 0.803 perplexity: 38.742 speed: 9229 wps 874 | 0.903 perplexity: 37.430 speed: 9240 wps 875 | Epoch: 13 Train Perplexity: 36.643 876 | Epoch: 13 Valid Perplexity: 121.475 877 | Test Perplexity: 116.716 878 | 879 | PTB例子证明了递归神经网络能够实现语言建模，但是这个例子并没有做什么实际的事情。 880 | 在做具体应用之前，您应该浏览这个例子的代码和下一章 “理解 LSTM” 来学好递归神经网络的基础。 881 | 之后，您将学习如何用递归神经网络来生成文本，如何实现语言翻译和问题应答系统。 882 | 883 | 理解LSTM 884 | ============= 885 | 886 | 递归神经网络 (Recurrent Neural Network) 887 | ------------------------------------------- 888 | 889 | 我们认为Andrey Karpathy的博客 `Understand Recurrent Neural Network`_ 是了解递归神经网络最好的材料。 890 | 读完这个博客后，Colah的博客 `Understand LSTM Network`_ 能帮助您了解LSTM。 891 | 我们在这里不介绍更多关于递归神经网络的内容，所以在您继续下面的内容之前，请先阅读我们建议阅读的博客。 892 | 893 | .. _fig_0601: 894 | 895 | .. image:: my_figs/karpathy_rnn.jpeg 896 | 897 | 图片由Andrey Karpathy提供 898 | 899 | 同步输入与输出序列 (Synced sequence input and output) 900 | -------------------------------------------------------------- 901 | 902 | PTB例子中的模型是一个典型的同步输入与输出，Karpathy 把它描述为 903 | “(5) 同步序列输入与输出(例如视频分类中我们希望对每一帧进行标记)。“ 904 | 905 | 模型的构建如下，第一层是词嵌套层（嵌入），把每一个单词转换成对应的词向量，在该例子中没有使用预先训练好的 906 | 嵌套矩阵。第二，堆叠两层LSTM，使用Dropout来实现规则化，防止overfitting。 907 | 最后，使用全连接层输出一序列的softmax输出。 908 | 909 | 第一层LSTM的输出形状是 [batch_size, num_steps, hidden_size]，这是为了让下一层LSTM可以堆叠在其上面。 910 | 第二层LSTM的输出形状是 [batch_size*num_steps, hidden_size]，这是为了让输出层（全连接层 Dense）可以堆叠在其上面。 911 | 然后计算每个样本的softmax输出，样本总数为 n_examples = batch_size*num_steps。 912 | 913 | 若想要更进一步理解该PTB教程，您也可以阅读 `TensorFlow 官方的PTB教程 914 | `_ ，中文翻译请见极客学院。 915 | 916 | 917 | .. code-block:: python 918 | 919 | network = tl.layers.EmbeddingInputlayer( 920 | inputs = x, 921 | vocabulary_size = vocab_size, 922 | embedding_size = hidden_size, 923 | E_init = tf.random_uniform_initializer(-init_scale, init_scale), 924 | name ='embedding_layer') 925 | if is_training: 926 | network = tl.layers.DropoutLayer(network, keep=keep_prob, name='drop1') 927 | network = tl.layers.RNNLayer(network, 928 | cell_fn=tf.nn.rnn_cell.BasicLSTMCell, 929 | cell_init_args={'forget_bias': 0.0}, 930 | n_hidden=hidden_size, 931 | initializer=tf.random_uniform_initializer(-init_scale, init_scale), 932 | n_steps=num_steps, 933 | return_last=False, 934 | name='basic_lstm_layer1') 935 | lstm1 = network 936 | if is_training: 937 | network = tl.layers.DropoutLayer(network, keep=keep_prob, name='drop2') 938 | network = tl.layers.RNNLayer(network, 939 | cell_fn=tf.nn.rnn_cell.BasicLSTMCell, 940 | cell_init_args={'forget_bias': 0.0}, 941 | n_hidden=hidden_size, 942 | initializer=tf.random_uniform_initializer(-init_scale, init_scale), 943 | n_steps=num_steps, 944 | return_last=False, 945 | return_seq_2d=True, 946 | name='basic_lstm_layer2') 947 | lstm2 = network 948 | if is_training: 949 | network = tl.layers.DropoutLayer(network, keep=keep_prob, name='drop3') 950 | network = tl.layers.DenseLayer(network, 951 | n_units=vocab_size, 952 | W_init=tf.random_uniform_initializer(-init_scale, init_scale), 953 | b_init=tf.random_uniform_initializer(-init_scale, init_scale), 954 | act = tl.activation.identity, name='output_layer') 955 | 956 | 957 | 数据迭代 958 | ^^^^^^^^^^^^^^^^^ 959 | 960 | batch_size 数值可以被视为并行计算的数量。 961 | 如下面的例子所示，第一个 batch 使用 0 到 9 来学习序列信息。 962 | 第二个 batch 使用 10 到 19 来学习序列。 963 | 所以它忽略了 9 到 10 之间的信息。 964 | 只当我们 bath_size 设为 1，它才使用 0 到 20 之间所有的序列信息来学习。 965 | 966 | 这里的 batch_size 的意思与 MNIST 例子略有不同。 967 | 在 MNIST 例子，batch_size 是每次迭代中我们使用的样本数量， 968 | 而在 PTB 的例子中，batch_size 是为加快训练速度的并行进程数。 969 | 970 | 虽然当 batch_size > 1 时有些信息将会被忽略， 971 | 但是如果您的数据是足够长的（一个语料库通常有几十亿个字），被忽略的信息不会影响最终的结果。 972 | 973 | 在PTB教程中，我们设置了 batch_size = 20，所以，我们将整个数据集拆分成 20 段（segment）。 974 | 在每一轮（epoch）的开始时，我们有 20 个初始化的 LSTM 状态（State），然后分别对 20 段数据进行迭代学习。 975 | 976 | 训练数据迭代的例子如下： 977 | 978 | .. code-block:: python 979 | 980 | train_data = [i for i in range(20)] 981 | for batch in tl.iterate.ptb_iterator(train_data, batch_size=2, num_steps=3): 982 | x, y = batch 983 | print(x, '\n',y) 984 | 985 | .. code-block:: text 986 | 987 | ... [[ 0 1 2] <---x 1st subset/ iteration 988 | ... [10 11 12]] 989 | ... [[ 1 2 3] <---y 990 | ... [11 12 13]] 991 | ... 992 | ... [[ 3 4 5] <--- 1st batch input 2nd subset/ iteration 993 | ... [13 14 15]] <--- 2nd batch input 994 | ... [[ 4 5 6] <--- 1st batch target 995 | ... [14 15 16]] <--- 2nd batch target 996 | ... 997 | ... [[ 6 7 8] 3rd subset/ iteration 998 | ... [16 17 18]] 999 | ... [[ 7 8 9] 1000 | ... [17 18 19]] 1001 | 1002 | .. note:: 1003 | 这个例子可以当作词嵌套矩阵的预训练。 1004 | 1005 | 损失和更新公式 1006 | ^^^^^^^^^^^^^^^^^^^^^ 1007 | 1008 | 损失函数是一系列输出cross entropy的均值。 1009 | 1010 | .. code-block:: python 1011 | 1012 | # 更多细节请见 tensorlayer.cost.cross_entropy_seq() 1013 | def loss_fn(outputs, targets, batch_size, num_steps): 1014 | # Returns the cost function of Cross-entropy of two sequences, implement 1015 | # softmax internally. 1016 | # outputs : 2D tensor [batch_size*num_steps, n_units of output layer] 1017 | # targets : 2D tensor [batch_size, num_steps], need to be reshaped. 1018 | # n_examples = batch_size * num_steps 1019 | # so 1020 | # cost is the averaged cost of each mini-batch (concurrent process). 1021 | loss = tf.nn.seq2seq.sequence_loss_by_example( 1022 | [outputs], 1023 | [tf.reshape(targets, [-1])], 1024 | [tf.ones([batch_size * num_steps])]) 1025 | cost = tf.reduce_sum(loss) / batch_size 1026 | return cost 1027 | 1028 | # Cost for Training 1029 | cost = loss_fn(network.outputs, targets, batch_size, num_steps) 1030 | 1031 | 在训练时，该例子在若干个epoch之后（由 ``max_epoch`` 定义），才开始按比例下降学习率（learning rate），新学习率是前一个epoch的学习率乘以一个下降率（由 ``lr_decay`` 定义）。 1032 | 此外，截断反向传播（truncated backpropagation）截断了 1033 | 1034 | 为使学习过程易于处理，通常的做法是将反向传播的梯度在（按时间）展开的步骤上照一个固定长度( ``num_steps`` )截断。 通过在一次迭代中的每个时刻上提供长度为 ``num_steps`` 的输入和每次迭代完成之后反向传导，这会很容易实现。 1035 | 1036 | 1037 | .. code-block:: python 1038 | 1039 | # 截断反响传播 Truncated Backpropagation for training 1040 | with tf.variable_scope('learning_rate'): 1041 | lr = tf.Variable(0.0, trainable=False) 1042 | tvars = tf.trainable_variables() 1043 | grads, _ = tf.clip_by_global_norm(tf.gradients(cost, tvars), 1044 | max_grad_norm) 1045 | optimizer = tf.train.GradientDescentOptimizer(lr) 1046 | train_op = optimizer.apply_gradients(zip(grads, tvars)) 1047 | 1048 | 如果当前epoch值大于 ``max_epoch`` ，则把当前学习率乘以 ``lr_decay`` 来降低学习率。 1049 | 1050 | .. code-block:: python 1051 | 1052 | new_lr_decay = lr_decay ** max(i - max_epoch, 0.0) 1053 | sess.run(tf.assign(lr, learning_rate * new_lr_decay)) 1054 | 1055 | 在每一个epoch的开始之前，LSTM的状态要被重置为零状态；在每一个迭代之后，LSTM状态都会被改变，所以要把最新的LSTM状态 1056 | 作为下一个迭代的初始化状态。 1057 | 1058 | .. code-block:: python 1059 | 1060 | # 在每一个epoch之前，把所有LSTM状态设为零状态 1061 | state1 = tl.layers.initialize_rnn_state(lstm1.initial_state) 1062 | state2 = tl.layers.initialize_rnn_state(lstm2.initial_state) 1063 | for step, (x, y) in enumerate(tl.iterate.ptb_iterator(train_data, 1064 | batch_size, num_steps)): 1065 | feed_dict = {input_data: x, targets: y, 1066 | lstm1.initial_state: state1, 1067 | lstm2.initial_state: state2, 1068 | } 1069 | # 启用dropout 1070 | feed_dict.update( network.all_drop ) 1071 | # 把新的状态作为下一个迭代的初始状态 1072 | _cost, state1, state2, _ = sess.run([cost, 1073 | lstm1.final_state, 1074 | lstm2.final_state, 1075 | train_op], 1076 | feed_dict=feed_dict 1077 | ) 1078 | costs += _cost; iters += num_steps 1079 | 1080 | 预测 1081 | ^^^^^^^^^^^^^ 1082 | 1083 | 在训练完模型之后，当我们预测下一个输出时，我们不需要考虑序列长度了，因此 ``batch_size`` 和 ``num_steps`` 都设为 1 。 1084 | 然后，我们可以一步一步地输出下一个单词，而不是通过一序列的单词来输出一序列的单词。 1085 | 1086 | .. code-block:: python 1087 | 1088 | input_data_test = tf.placeholder(tf.int32, [1, 1]) 1089 | targets_test = tf.placeholder(tf.int32, [1, 1]) 1090 | ... 1091 | network_test, lstm1_test, lstm2_test = inference(input_data_test, 1092 | is_training=False, num_steps=1, reuse=True) 1093 | ... 1094 | cost_test = loss_fn(network_test.outputs, targets_test, 1, 1) 1095 | ... 1096 | print("Evaluation") 1097 | # 测试 1098 | # go through the test set step by step, it will take a while. 1099 | start_time = time.time() 1100 | costs = 0.0; iters = 0 1101 | # 与训练时一样，设置所有LSTM状态为零状态 1102 | state1 = tl.layers.initialize_rnn_state(lstm1_test.initial_state) 1103 | state2 = tl.layers.initialize_rnn_state(lstm2_test.initial_state) 1104 | for step, (x, y) in enumerate(tl.iterate.ptb_iterator(test_data, 1105 | batch_size=1, num_steps=1)): 1106 | feed_dict = {input_data_test: x, targets_test: y, 1107 | lstm1_test.initial_state: state1, 1108 | lstm2_test.initial_state: state2, 1109 | } 1110 | _cost, state1, state2 = sess.run([cost_test, 1111 | lstm1_test.final_state, 1112 | lstm2_test.final_state], 1113 | feed_dict=feed_dict 1114 | ) 1115 | costs += _cost; iters += 1 1116 | test_perplexity = np.exp(costs / iters) 1117 | print("Test Perplexity: %.3f took %.2fs" % (test_perplexity, time.time() - start_time)) 1118 | 1119 | 1120 | 下一步？ 1121 | ------------ 1122 | 1123 | 您已经明白了同步序列输入和序列输出（Synced sequence input and output）。 1124 | 现在让我们思考下序列输入单一输出的情况（Sequence input and one output）， 1125 | LSTM 也可以学会通过给定一序列输入如 “我来自北京，我会说.." 来输出 1126 | 一个单词 "中文"。 1127 | 1128 | 请仔细阅读并理解 ``tutorial_generate_text.py`` 的代码，它讲了如何加载一个已经训练好的词嵌套矩阵， 1129 | 以及如何给定机器一个文档，让它来学习文字自动生成。 1130 | 1131 | Karpathy的博客： 1132 | "(3) Sequence input (e.g. sentiment analysis where a given sentence is 1133 | classified as expressing positive or negative sentiment). " 1134 | 1135 | 1136 | 更多经典教程 1137 | =================== 1138 | 1139 | 您能在例子页面找到包括Seq2seq, 各类对抗学习和增强学习的例子。 1140 | 1141 | 1142 | 1143 | .. 1144 | 运行机器翻译例子 1145 | =================== 1146 | 1147 | .. code-block:: python 1148 | 1149 | python tutorial_translate.py 1150 | 1151 | 该脚本将训练一个神经网络来把英文翻译成法文。 1152 | 如果一切正常，您将看到： 1153 | - 下载WMT英文-法文翻译数据库，包括训练集和测试集。 1154 | - 通过训练集创建英文和法文的词汇表。 1155 | - 把训练集和测试集的单词转换成数字ID表示。 1156 | 1157 | 1158 | .. code-block:: bash 1159 | 1160 | Prepare raw data 1161 | Load or Download WMT English-to-French translation > wmt 1162 | Training data : wmt/giga-fren.release2 1163 | Testing data : wmt/newstest2013 1164 | 1165 | Create vocabularies 1166 | Vocabulary of French : wmt/vocab40000.fr 1167 | Vocabulary of English : wmt/vocab40000.en 1168 | Creating vocabulary wmt/vocab40000.fr from data wmt/giga-fren.release2.fr 1169 | processing line 100000 1170 | processing line 200000 1171 | processing line 300000 1172 | processing line 400000 1173 | processing line 500000 1174 | processing line 600000 1175 | processing line 700000 1176 | processing line 800000 1177 | processing line 900000 1178 | processing line 1000000 1179 | processing line 1100000 1180 | processing line 1200000 1181 | ... 1182 | processing line 22500000 1183 | Creating vocabulary wmt/vocab40000.en from data wmt/giga-fren.release2.en 1184 | processing line 100000 1185 | ... 1186 | processing line 22500000 1187 | 1188 | ... 1189 | 1190 | 首先，我们从WMT'15网站上下载英语-法语翻译数据。训练数据和测试数据如下。 1191 | 训练数据用于训练模型，测试数据用于评估该模型。 1192 | 1193 | .. code-block:: text 1194 | 1195 | wmt/training-giga-fren.tar <-- 英文－法文训练集 (2.6GB) 1196 | giga-fren.release2.* 从该文件解压出来 1197 | wmt/dev-v2.tgz <-- 多种语言的测试集 (21.4MB) 1198 | newstest2013.* 从该文件解压出来 1199 | 1200 | wmt/giga-fren.release2.fr <-- 法文训练集 (4.57GB) 1201 | wmt/giga-fren.release2.en <-- 英文训练集 (3.79GB) 1202 | 1203 | wmt/newstest2013.fr <-- 法文测试集 (393KB) 1204 | wmt/newstest2013.en <-- 英文测试集 (333KB) 1205 | 1206 | 所有 ``giga-fren.release2.*`` 是训练数据， ``giga-fren.release2.fr`` 内容如下： 1207 | 1208 | .. code-block:: text 1209 | 1210 | Il a transformé notre vie | Il a transformé la société | Son fonctionnement | La technologie, moteur du changement Accueil | Concepts | Enseignants | Recherche | Aperçu | Collaborateurs | Web HHCC | Ressources | Commentaires Musée virtuel du Canada 1211 | Plan du site 1212 | Rétroaction 1213 | Crédits 1214 | English 1215 | Qu’est-ce que la lumière? 1216 | La découverte du spectre de la lumière blanche Des codes dans la lumière Le spectre électromagnétique Les spectres d’émission Les spectres d’absorption Les années-lumière La pollution lumineuse 1217 | Le ciel des premiers habitants La vision contemporaine de l'Univers L’astronomie pour tous 1218 | Bande dessinée 1219 | Liens 1220 | Glossaire 1221 | Observatoires 1222 | ... 1223 | 1224 | ``giga-fren.release2.en`` 内容如下，我们可以看到单词或者句子用 ``|`` 或 ``\n`` 来分隔。 1225 | 1226 | .. code-block:: text 1227 | 1228 | Changing Lives | Changing Society | How It Works | Technology Drives Change Home | Concepts | Teachers | Search | Overview | Credits | HHCC Web | Reference | Feedback Virtual Museum of Canada Home Page 1229 | Site map 1230 | Feedback 1231 | Credits 1232 | Français 1233 | What is light ? 1234 | The white light spectrum Codes in the light The electromagnetic spectrum Emission spectra Absorption spectra Light-years Light pollution 1235 | The sky of the first inhabitants A contemporary vison of the Universe Astronomy for everyone 1236 | Cartoon 1237 | Links 1238 | Glossary 1239 | Observatories 1240 | 1241 | 测试数据 ``newstest2013.en`` 和 ``newstest2013.fr`` 如下所示： 1242 | 1243 | .. code-block:: text 1244 | 1245 | newstest2013.en : 1246 | A Republican strategy to counter the re-election of Obama 1247 | Republican leaders justified their policy by the need to combat electoral fraud. 1248 | However, the Brennan Centre considers this a myth, stating that electoral fraud is rarer in the United States than the number of people killed by lightning. 1249 | 1250 | newstest2013.fr : 1251 | Une stratégie républicaine pour contrer la réélection d'Obama 1252 | Les dirigeants républicains justifièrent leur politique par la nécessité de lutter contre la fraude électorale. 1253 | Or, le Centre Brennan considère cette dernière comme un mythe, affirmant que la fraude électorale est plus rare aux États-Unis que le nombre de personnes tuées par la foudre. 1254 | 1255 | 下载完数据之后，开始创建词汇表文件。 1256 | 从训练数据 ``giga-fren.release2.fr`` 和 ``giga-fren.release2.en``创建 ``vocab40000.fr`` 和 ``vocab40000.en`` 这个过程需要较长一段时间，数字 ``40000`` 代表了词汇库的大小。 1257 | 1258 | ``vocab40000.fr`` (381KB) 按下列所示地按每行一个单词的方式存储（one-item-per-line）。 1259 | 1260 | .. code-block:: text 1261 | 1262 | _PAD 1263 | _GO 1264 | _EOS 1265 | _UNK 1266 | de 1267 | , 1268 | . 1269 | ' 1270 | la 1271 | et 1272 | des 1273 | les 1274 | à 1275 | le 1276 | du 1277 | l 1278 | en 1279 | ) 1280 | d 1281 | 0 1282 | ( 1283 | 00 1284 | pour 1285 | dans 1286 | un 1287 | que 1288 | une 1289 | sur 1290 | au 1291 | 0000 1292 | a 1293 | par 1294 | 1295 | ``vocab40000.en`` (344KB) 也是如此。 1296 | 1297 | .. code-block:: text 1298 | 1299 | _PAD 1300 | _GO 1301 | _EOS 1302 | _UNK 1303 | the 1304 | . 1305 | , 1306 | of 1307 | and 1308 | to 1309 | in 1310 | a 1311 | ) 1312 | ( 1313 | 0 1314 | for 1315 | 00 1316 | that 1317 | is 1318 | on 1319 | The 1320 | 0000 1321 | be 1322 | by 1323 | with 1324 | or 1325 | : 1326 | as 1327 | " 1328 | 000 1329 | are 1330 | ; 1331 | 1332 | 接着我们开始创建英文和法文的数字化（ID）训练集和测试集。这也要较长一段时间。 1333 | 1334 | .. code-block:: text 1335 | 1336 | Tokenize data 1337 | Tokenizing data in wmt/giga-fren.release2.fr <-- Training data of French 1338 | tokenizing line 100000 1339 | tokenizing line 200000 1340 | tokenizing line 300000 1341 | tokenizing line 400000 1342 | ... 1343 | tokenizing line 22500000 1344 | Tokenizing data in wmt/giga-fren.release2.en <-- Training data of English 1345 | tokenizing line 100000 1346 | tokenizing line 200000 1347 | tokenizing line 300000 1348 | tokenizing line 400000 1349 | ... 1350 | tokenizing line 22500000 1351 | Tokenizing data in wmt/newstest2013.fr <-- Testing data of French 1352 | Tokenizing data in wmt/newstest2013.en <-- Testing data of English 1353 | 1354 | 最后，我们所有的文件如下所示： 1355 | 1356 | .. code-block:: text 1357 | 1358 | wmt/training-giga-fren.tar <-- 英文－法文训练集 (2.6GB) 1359 | giga-fren.release2.* 从该文件解压出来 1360 | wmt/dev-v2.tgz <-- 多种语言的测试集 (21.4MB) 1361 | newstest2013.* 从该文件解压出来 1362 | 1363 | wmt/giga-fren.release2.fr <-- 法文训练集 (4.57GB) 1364 | wmt/giga-fren.release2.en <-- 英文训练集 (3.79GB) 1365 | 1366 | wmt/newstest2013.fr <-- 法文测试集 (393KB) 1367 | wmt/newstest2013.en <-- 英文测试集 (333KB) 1368 | 1369 | wmt/vocab40000.fr <-- 法文词汇表 (381KB) 1370 | wmt/vocab40000.en <-- 英文词汇表 (344KB) 1371 | 1372 | wmt/giga-fren.release2.ids40000.fr <-- 数字化法文训练集 (2.81GB) 1373 | wmt/giga-fren.release2.ids40000.en <-- 数字化英文训练集 (2.38GB) 1374 | 1375 | wmt/newstest2013.ids40000.fr <-- 数字化法文训练集 (268KB) 1376 | wmt/newstest2013.ids40000.en <-- 数字化英文测试集 (232KB) 1377 | 1378 | 现在，把数字化的数据读入buckets中，并计算不同buckets中数据样本的个数。 1379 | 1380 | 1381 | .. code-block:: text 1382 | 1383 | Read development (test) data into buckets 1384 | dev data: (5, 10) [[13388, 4, 949], [23113, 8, 910, 2]] 1385 | en word_ids: [13388, 4, 949] 1386 | en context: [b'Preventing', b'the', b'disease'] 1387 | fr word_ids: [23113, 8, 910, 2] 1388 | fr context: [b'Pr\xc3\xa9venir', b'la', b'maladie', b'_EOS'] 1389 | 1390 | Read training data into buckets (limit: 0) 1391 | reading data line 100000 1392 | reading data line 200000 1393 | reading data line 300000 1394 | reading data line 400000 1395 | reading data line 500000 1396 | reading data line 600000 1397 | reading data line 700000 1398 | reading data line 800000 1399 | ... 1400 | reading data line 22400000 1401 | reading data line 22500000 1402 | train_bucket_sizes: [239121, 1344322, 5239557, 10445326] 1403 | train_total_size: 17268326.0 1404 | train_buckets_scale: [0.013847375825543252, 0.09169638099257565, 0.3951164693091849, 1.0] 1405 | train data: (5, 10) [[1368, 3344], [1089, 14, 261, 2]] 1406 | en word_ids: [1368, 3344] 1407 | en context: [b'Site', b'map'] 1408 | fr word_ids: [1089, 14, 261, 2] 1409 | fr context: [b'Plan', b'du', b'site', b'_EOS'] 1410 | 1411 | the num of training data in each buckets: [239121, 1344322, 5239557, 10445326] 1412 | the num of training data: 17268326 1413 | train_buckets_scale: [0.013847375825543252, 0.09169638099257565, 0.3951164693091849, 1.0] 1414 | 1415 | 最后开始训练模型，当 ``steps_per_checkpoint = 10`` 时，您将看到： 1416 | 1417 | ``steps_per_checkpoint = 10`` 1418 | 1419 | .. code-block:: text 1420 | 1421 | Create Embedding Attention Seq2seq Model 1422 | 1423 | global step 10 learning rate 0.5000 step-time 22.26 perplexity 12761.50 1424 | eval: bucket 0 perplexity 5887.75 1425 | eval: bucket 1 perplexity 3891.96 1426 | eval: bucket 2 perplexity 3748.77 1427 | eval: bucket 3 perplexity 4940.10 1428 | global step 20 learning rate 0.5000 step-time 20.38 perplexity 28761.36 1429 | eval: bucket 0 perplexity 10137.01 1430 | eval: bucket 1 perplexity 12809.90 1431 | eval: bucket 2 perplexity 15758.65 1432 | eval: bucket 3 perplexity 26760.93 1433 | global step 30 learning rate 0.5000 step-time 20.64 perplexity 6372.95 1434 | eval: bucket 0 perplexity 1789.80 1435 | eval: bucket 1 perplexity 1690.00 1436 | eval: bucket 2 perplexity 2190.18 1437 | eval: bucket 3 perplexity 3808.12 1438 | global step 40 learning rate 0.5000 step-time 16.10 perplexity 3418.93 1439 | eval: bucket 0 perplexity 4778.76 1440 | eval: bucket 1 perplexity 3698.90 1441 | eval: bucket 2 perplexity 3902.37 1442 | eval: bucket 3 perplexity 22612.44 1443 | global step 50 learning rate 0.5000 step-time 14.84 perplexity 1811.02 1444 | eval: bucket 0 perplexity 644.72 1445 | eval: bucket 1 perplexity 759.16 1446 | eval: bucket 2 perplexity 984.18 1447 | eval: bucket 3 perplexity 1585.68 1448 | global step 60 learning rate 0.5000 step-time 19.76 perplexity 1580.55 1449 | eval: bucket 0 perplexity 1724.84 1450 | eval: bucket 1 perplexity 2292.24 1451 | eval: bucket 2 perplexity 2698.52 1452 | eval: bucket 3 perplexity 3189.30 1453 | global step 70 learning rate 0.5000 step-time 17.16 perplexity 1250.57 1454 | eval: bucket 0 perplexity 298.55 1455 | eval: bucket 1 perplexity 502.04 1456 | eval: bucket 2 perplexity 645.44 1457 | eval: bucket 3 perplexity 604.29 1458 | global step 80 learning rate 0.5000 step-time 18.50 perplexity 793.90 1459 | eval: bucket 0 perplexity 2056.23 1460 | eval: bucket 1 perplexity 1344.26 1461 | eval: bucket 2 perplexity 767.82 1462 | eval: bucket 3 perplexity 649.38 1463 | global step 90 learning rate 0.5000 step-time 12.61 perplexity 541.57 1464 | eval: bucket 0 perplexity 180.86 1465 | eval: bucket 1 perplexity 350.99 1466 | eval: bucket 2 perplexity 326.85 1467 | eval: bucket 3 perplexity 383.22 1468 | global step 100 learning rate 0.5000 step-time 18.42 perplexity 471.12 1469 | eval: bucket 0 perplexity 216.63 1470 | eval: bucket 1 perplexity 348.96 1471 | eval: bucket 2 perplexity 318.20 1472 | eval: bucket 3 perplexity 389.92 1473 | global step 110 learning rate 0.5000 step-time 18.39 perplexity 474.89 1474 | eval: bucket 0 perplexity 8049.85 1475 | eval: bucket 1 perplexity 1677.24 1476 | eval: bucket 2 perplexity 936.98 1477 | eval: bucket 3 perplexity 657.46 1478 | global step 120 learning rate 0.5000 step-time 18.81 perplexity 832.11 1479 | eval: bucket 0 perplexity 189.22 1480 | eval: bucket 1 perplexity 360.69 1481 | eval: bucket 2 perplexity 410.57 1482 | eval: bucket 3 perplexity 456.40 1483 | global step 130 learning rate 0.5000 step-time 20.34 perplexity 452.27 1484 | eval: bucket 0 perplexity 196.93 1485 | eval: bucket 1 perplexity 655.18 1486 | eval: bucket 2 perplexity 860.44 1487 | eval: bucket 3 perplexity 1062.36 1488 | global step 140 learning rate 0.5000 step-time 21.05 perplexity 847.11 1489 | eval: bucket 0 perplexity 391.88 1490 | eval: bucket 1 perplexity 339.09 1491 | eval: bucket 2 perplexity 320.08 1492 | eval: bucket 3 perplexity 376.44 1493 | global step 150 learning rate 0.4950 step-time 15.53 perplexity 590.03 1494 | eval: bucket 0 perplexity 269.16 1495 | eval: bucket 1 perplexity 286.51 1496 | eval: bucket 2 perplexity 391.78 1497 | eval: bucket 3 perplexity 485.23 1498 | global step 160 learning rate 0.4950 step-time 19.36 perplexity 400.80 1499 | eval: bucket 0 perplexity 137.00 1500 | eval: bucket 1 perplexity 198.85 1501 | eval: bucket 2 perplexity 276.58 1502 | eval: bucket 3 perplexity 357.78 1503 | global step 170 learning rate 0.4950 step-time 17.50 perplexity 541.79 1504 | eval: bucket 0 perplexity 1051.29 1505 | eval: bucket 1 perplexity 626.64 1506 | eval: bucket 2 perplexity 496.32 1507 | eval: bucket 3 perplexity 458.85 1508 | global step 180 learning rate 0.4950 step-time 16.69 perplexity 400.65 1509 | eval: bucket 0 perplexity 178.12 1510 | eval: bucket 1 perplexity 299.86 1511 | eval: bucket 2 perplexity 294.84 1512 | eval: bucket 3 perplexity 296.46 1513 | global step 190 learning rate 0.4950 step-time 19.93 perplexity 886.73 1514 | eval: bucket 0 perplexity 860.60 1515 | eval: bucket 1 perplexity 910.16 1516 | eval: bucket 2 perplexity 909.24 1517 | eval: bucket 3 perplexity 786.04 1518 | global step 200 learning rate 0.4901 step-time 18.75 perplexity 449.64 1519 | eval: bucket 0 perplexity 152.13 1520 | eval: bucket 1 perplexity 234.41 1521 | eval: bucket 2 perplexity 249.66 1522 | eval: bucket 3 perplexity 285.95 1523 | ... 1524 | global step 980 learning rate 0.4215 step-time 18.31 perplexity 208.74 1525 | eval: bucket 0 perplexity 78.45 1526 | eval: bucket 1 perplexity 108.40 1527 | eval: bucket 2 perplexity 137.83 1528 | eval: bucket 3 perplexity 173.53 1529 | global step 990 learning rate 0.4173 step-time 17.31 perplexity 175.05 1530 | eval: bucket 0 perplexity 78.37 1531 | eval: bucket 1 perplexity 119.72 1532 | eval: bucket 2 perplexity 169.11 1533 | eval: bucket 3 perplexity 202.89 1534 | global step 1000 learning rate 0.4173 step-time 15.85 perplexity 174.33 1535 | eval: bucket 0 perplexity 76.52 1536 | eval: bucket 1 perplexity 125.97 1537 | eval: bucket 2 perplexity 150.13 1538 | eval: bucket 3 perplexity 181.07 1539 | ... 1540 | 1541 | 经过350000轮训练模型之后，您可以将代码中的 ``main_train()`` 换为 ``main_decode()`` 来使用训练好的翻译器， 1542 | 您输入一个英文句子，程序将输出一个对应的法文句子。 1543 | 1544 | .. code-block:: text 1545 | 1546 | Reading model parameters from wmt/translate.ckpt-350000 1547 | > Who is the president of the United States? 1548 | Qui est le président des États-Unis ? 1549 | 1550 | 1551 | 理解机器翻译 1552 | ==================== 1553 | 1554 | Seq2seq 1555 | -------------- 1556 | 序列到序列模型（Seq2seq）通常被用来转换一种语言到另一种语言。 1557 | 但实际上它能用来做很多您可能无法想象的事情，比如我们可以将一个长的句子翻译成意思一样但短且简单的句子， 1558 | 再比如，从莎士比亚的语言翻译成现代英语。若用上卷积神经网络(CNN)的话，我们能将视频翻译成句子，则自动看一段视频给出该视频的文字描述（Video captioning）。 1559 | 1560 | 如果您只是想用 Seq2seq，您只需要考虑训练集的格式，比如如何切分单词、如何数字化单词等等。 1561 | 所以，在本教程中，我们将讨论很多如何整理训练集。 1562 | 1563 | 1564 | 基础 1565 | ^^^^^^^^^ 1566 | 1567 | 序列到序列模型是一种多对多（Many to many）的模型，但与PTB教程中的同步序列输入与输出(Synced sequence input and output）不一样，Seq2seq是在输入了整个序列之后，才开始输出新的序列（非同步）。 1568 | 该教程用了下列两种最新的方法来提高准确度： 1569 | - 把输入序列倒转输入（Reversing the inputs） 1570 | - 注意机制（Attention mechanism） 1571 | 1572 | 为了要加快训练速度，我们使用了： 1573 | - softmax 抽样（Sampled softmax） 1574 | 1575 | Karpathy的博客是这样描述Seq2seq的："(4) Sequence input and sequence output (e.g. Machine Translation: an RNN reads a sentence in English and then outputs a sentence in French)." 1576 | 1577 | 1578 | .. _fig_0601: 1579 | 1580 | .. image:: my_figs/basic_seq2seq.png 1581 | :scale: 100 % 1582 | :align: center 1583 | 1584 | 如上图所示，编码器输入（encoder input），解码器输入（decoder input）以及输出目标（targets）如下： 1585 | 1586 | .. code-block:: text 1587 | 1588 | encoder_input = A B C 1589 | decoder_input = W X Y Z 1590 | targets = W X Y Z 1591 | 1592 | Note：在代码实现中，targets的长度比decoder_input的长度小一，更多实现细节将在下文说明。 1593 | 1594 | 文献 1595 | ^^^^^^^^^^^ 1596 | 1597 | 该英语-法语的机器翻译例子使用了多层递归神经网络以及注意机制。 1598 | 该模型和如下论文中一样： 1599 | - `Grammar as a Foreign Language `_ 1600 | 1601 | 该例子采用了 softmax 抽样（sampled softmax）来解决当词汇表很大时计算量大的问题。 1602 | 在该例子中，``target_vocab_size=4000`` ，若词汇量小于 ``512`` 时用普通的softmax cross entropy即可。 1603 | Softmax 抽样在这篇论文的第三小节中描述: 1604 | - `On Using Very Large Target Vocabulary for Neural Machine Translation `_ 1605 | 1606 | 如下文章讲述了把输入序列倒转（Reversing the inputs）和多层神递归神经网络用在Seq2seq的翻译应用非常成功： 1607 | - `Sequence to Sequence Learning with Neural Networks `_ 1608 | 1609 | 如下文章讲述了注意机制（Attention Mechanism）让解码器可以更直接地得到每一个输入的信息： 1610 | - `Neural Machine Translation by Jointly Learning to Align and Translate `_ 1611 | 1612 | 如下文章讲述了另一种Seq2seq模型，则使用双向编码器（Bi-directional encoder）： 1613 | - `Neural Machine Translation by Jointly Learning to Align and Translate `_ 1614 | 1615 | 1616 | 实现细节 1617 | ------------- 1618 | 1619 | Bucketing and Padding 1620 | ^^^^^^^^^^^^^^^^^^^^^^^^^ 1621 | 1622 | Bucketing 是一种能有效处理不同句子长度的方法，为什么使用Bucketing，在 `知乎 `_ 上已经有很好的回答了。 1623 | 1624 | 当将英文翻译成法文的时，我们有不同长度的英文句子输入（长度为 ``L1 `` ），以及不同长度的法文句子输出，（长度为 ``L2`` ）。 1625 | 我们原则上要建立每一种长度的可能性，则有很多个 ``(L1, L2+1)`` ，其中 ``L2`` 加一是因为有 GO 标志符。 1626 | 1627 | 为了减少 bucket 的数量以及为句子找到最合适的 bucket，若 bucket 大于句子的长度，我们则使用 PAD 标志符填充之。 1628 | 1629 | 为了提高效率，我们只使用几个 bucket，然后使用 padding 来让句子匹配到最相近的 bucket 中。 1630 | 在该例子中，我们使用如下 4 个 buckets。 1631 | 1632 | .. code-block:: python 1633 | 1634 | buckets = [(5, 10), (10, 15), (20, 25), (40, 50)] 1635 | 1636 | 如果输入的是一个有 ``3`` 个单词的英文句子，对应的法文输出有 ``6`` 个单词， 1637 | 那么改数据将被放在第一个 bucket 中并且把 encoder inputs 和 decoder inputs 通过 padding 来让其长度变成 ``5`` 和 ``10`` 。 1638 | 如果我们有 ``8`` 个单词的英文句子，及 ``18`` 个单词的法文句子，它们会被放到 ``(20, 25)`` 的 bucket 中。 1639 | 1640 | 换句话说，bucket ``(I,O)`` 是 ``(编码器输入大小(encoder_input_size)，解码器输入大小(decoder_inputs_size))`` 。 1641 | 1642 | 给出一对数字化训练样本 ``[["I", "go", "."], ["Je", "vais", "."]]`` ，我们把它转换为 ``(5,10)`` 。 1643 | 编码器输入（encoder inputs）的训练数据为 ``[PAD PAD "." "go" "I"]`` ，而解码器的输入（decoder inputs）为 ``[GO "Je" "vais" "." EOS PAD PAD PAD PAD PAD]`` 。 1644 | 而输出目标（targets）是解码器输入（decoder inputs）平移一位。 ``target_weights`` 是输出目标（targets）的掩码。 1645 | 1646 | . code-block:: text 1647 | 1648 | bucket = (I, O) = (5, 10) 1649 | encoder_inputs = [PAD PAD "." "go" "I"] <-- 5 x batch_size 1650 | decoder_inputs = [GO "Je" "vais" "." EOS PAD PAD PAD PAD PAD] <-- 10 x batch_size 1651 | target_weights = [1 1 1 1 0 0 0 0 0 0 0] <-- 10 x batch_size 1652 | targets = ["Je" "vais" "." EOS PAD PAD PAD PAD PAD] <-- 9 x batch_size 1653 | 1654 | 在该代码中，一个句子是由一个列向量表示，假设 ``batch_size = 3`` ， ``bucket = (5, 10)`` ，训练集如下所示。 1655 | 1656 | .. code-block:: text 1657 | 1658 | encoder_inputs decoder_inputs target_weights targets 1659 | 0 0 0 1 1 1 1 1 1 87 71 16748 1660 | 0 0 0 87 71 16748 1 1 1 2 3 14195 1661 | 0 0 0 2 3 14195 0 1 1 0 2 2 1662 | 0 0 3233 0 2 2 0 0 0 0 0 0 1663 | 3 698 4061 0 0 0 0 0 0 0 0 0 1664 | 0 0 0 0 0 0 0 0 0 1665 | 0 0 0 0 0 0 0 0 0 1666 | 0 0 0 0 0 0 0 0 0 1667 | 0 0 0 0 0 0 0 0 0 1668 | 0 0 0 0 0 0 1669 | 1670 | 其中 0 : _PAD 1 : _GO 2 : _EOS 3 : _UNK 1671 | 1672 | 在训练过程中，解码器输入是目标，而在预测过程中，下一个解码器的输入是最后一个解码器的输出。 1673 | 1674 | 在训练过程中，编码器输入（decoder inputs）就是目标输出（targets）； 1675 | 当使用模型时，下一个编码器输入（decoder inputs）是上一个解码器输出（ decoder output）。 1676 | 1677 | 1678 | 特殊标志符、标点符号与阿拉伯数字 1679 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1680 | 1681 | 该例子中的特殊标志符是： 1682 | 1683 | .. code-block:: python 1684 | 1685 | _PAD = b"_PAD" 1686 | _GO = b"_GO" 1687 | _EOS = b"_EOS" 1688 | _UNK = b"_UNK" 1689 | PAD_ID = 0 <-- index (row number) in vocabulary 1690 | GO_ID = 1 1691 | EOS_ID = 2 1692 | UNK_ID = 3 1693 | _START_VOCAB = [_PAD, _GO, _EOS, _UNK] 1694 | 1695 | .. code-block:: text 1696 | 1697 | ID号 意义 1698 | _PAD 0 Padding, empty word 1699 | _GO 1 decoder_inputs 的第一个元素 1700 | _EOS 2 targets 的结束符 1701 | _UNK 3 不明单词（Unknown word），没有在词汇表出现的单词被标记为3 1702 | 1703 | 对于阿拉伯数字，建立词汇表时与数字化数据集时的 ``normalize_digits`` 必须是一致的，若 1704 | ``normalize_digits=True`` 所有阿拉伯数字都将被 ``0`` 代替。比如 ``123`` 被 ``000`` 代替，``9`` 被 ``0``代替 1705 | ，``1990-05`` 被 ``0000-00` 代替，最后 ``000`` ， ``0`` ， ``0000-00`` 等将在词汇库中(看 ``vocab40000.en`` )。 1706 | 1707 | 反之，如果 ``normalize_digits=False`` ，不同的阿拉伯数字将会放入词汇表中，那么词汇表就变得十分大了。 1708 | 本例子中寻找阿拉伯数字使用的正则表达式是 ``_DIGIT_RE = re.compile(br"\d")`` 。(详见 ``tl.nlp.create_vocabulary()`` 和 ``tl.nlp.data_to_token_ids()` ) 1709 | 1710 | 对于分离句子成独立单词，本例子使用正则表达式 ``_WORD_SPLIT = re.compile(b"([.,!?\"':;)(])")`` ， 1711 | 这意味着使用这几个标点符号 ``[ . , ! ? " ' : ; ) ( ]`` 以及空格来分割句子，详情请看 ``tl.nlp.basic_tokenizer()`` 。这个分割方法是 ``tl.nlp.create_vocabulary()`` 和 ``tl.nlp.data_to_token_ids()`` 的默认方法。 1712 | 1713 | 1714 | 所有的标点符号，比如 ``. , ) (`` 在英文和法文数据库中都会被全部保留下来。 1715 | 1716 | Softmax 抽样 (Sampled softmax) 1717 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1718 | 1719 | softmax抽样是一种词汇表很大（Softmax 输出很多）的时候用来降低损失（cost）计算量的方法。 1720 | 与从所有输出中计算 cross-entropy 相比，这个方法只从 ``num_samples`` 个输出中计算 cross-entropy。 1721 | 1722 | 1723 | 损失和更新函数 1724 | ^^^^^^^^^^^^^^^^^ 1725 | ``EmbeddingAttentionSeq2seqWrapper`` 内部实现了 SGD optimizer。 1726 | 1727 | 下一步？ 1728 | ------------------ 1729 | 1730 | 您可以尝试其他应用。 1731 | 1732 | 1733 | 1734 | 1735 | 1736 | 1737 | 1738 | 1739 | 1740 | 翻译对照 1741 | =============== 1742 | 1743 | Stacked Denosing Autoencoder 堆栈式降噪自编吗器 1744 | 1745 | Word Embedding 词嵌套、词嵌入 1746 | 1747 | Iteration 迭代 1748 | 1749 | Natural Language Processing 自然语言处理 1750 | 1751 | Sparse 稀疏的 1752 | 1753 | Cost function 损失函数 1754 | 1755 | Regularization 规则化、正则化 1756 | 1757 | Tokenization 数字化 1758 | 1759 | Truncated backpropagation 截断反向传播 1760 | 1761 | 1762 | 1763 | 1764 | 更多信息 1765 | ============== 1766 | 1767 | TensorLayer 还能做什么？请继续阅读本文档。 1768 | 1769 | 最后，API 参考列表和说明如下： 1770 | 1771 | 1772 | layers (:mod:`tensorlayer.layers`), 1773 | 1774 | activation (:mod:`tensorlayer.activation`), 1775 | 1776 | natural language processing (:mod:`tensorlayer.nlp`), 1777 | 1778 | reinforcement learning (:mod:`tensorlayer.rein`), 1779 | 1780 | cost expressions and regularizers (:mod:`tensorlayer.cost`), 1781 | 1782 | load and save files (:mod:`tensorlayer.files`), 1783 | 1784 | operating system (:mod:`tensorlayer.ops`), 1785 | 1786 | helper functions (:mod:`tensorlayer.utils`), 1787 | 1788 | visualization (:mod:`tensorlayer.visualize`), 1789 | 1790 | iteration functions (:mod:`tensorlayer.iterate`), 1791 | 1792 | preprocessing functions (:mod:`tensorlayer.prepro`), 1793 | 1794 | 1795 | .. _Deeplearning Tutorial: http://deeplearning.stanford.edu/tutorial/ 1796 | .. _Convolutional Neural Networks for Visual Recognition: http://cs231n.github.io/ 1797 | .. _Neural Networks and Deep Learning: http://neuralnetworksanddeeplearning.com/ 1798 | .. _TensorFlow tutorial: https://www.tensorflow.org/versions/r0.9/tutorials/index.html 1799 | .. _Understand Deep Reinforcement Learning: http://karpathy.github.io/2016/05/31/rl/ 1800 | .. _Understand Recurrent Neural Network: http://karpathy.github.io/2015/05/21/rnn-effectiveness/ 1801 | .. _Understand LSTM Network: http://colah.github.io/posts/2015-08-Understanding-LSTMs/ 1802 | .. _Word Representations: http://colah.github.io/posts/2014-07-NLP-RNNs-Representations/ 1803 | -------------------------------------------------------------------------------- /docs/wechat_group.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /img/img_qq.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/img/img_qq.jpeg -------------------------------------------------------------------------------- /img/img_qq_old.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/img/img_qq_old.png -------------------------------------------------------------------------------- /img/img_tensorflow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/img/img_tensorflow.png -------------------------------------------------------------------------------- /img/img_tensorlayer.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/img/img_tensorlayer.png -------------------------------------------------------------------------------- /img/img_tlayer1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/img/img_tlayer1.png -------------------------------------------------------------------------------- /img/tl_transparent_logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tensorlayer/tensorlayer-chinese/550167d1bfe5eba3977aba141a9e95c5834e180b/img/tl_transparent_logo.png -------------------------------------------------------------------------------- /requirements/requirements.txt: -------------------------------------------------------------------------------- 1 | imageio==2.5.0 2 | numpy>=1.16,<1.17 3 | progressbar2==3.39.3 4 | requests==2.21.0 5 | scikit-learn==0.21.0 6 | scikit-image==0.15.0 7 | scipy==1.2.1 8 | wrapt==1.11.1 9 | h5py>=2.9 10 | cloudpickle>=0.8.1 11 | -------------------------------------------------------------------------------- /requirements/requirements_contrib_loggers.txt: -------------------------------------------------------------------------------- 1 | hyperdash>=0.15,<0.16 -------------------------------------------------------------------------------- /requirements/requirements_db.txt: -------------------------------------------------------------------------------- 1 | pymongo==3.8.0 2 | -------------------------------------------------------------------------------- /requirements/requirements_dev.txt: -------------------------------------------------------------------------------- 1 | autopep8>=1.3,<1.5 2 | -------------------------------------------------------------------------------- /requirements/requirements_doc.txt: -------------------------------------------------------------------------------- 1 | flake8-docstrings>=1.3,<1.4 2 | pycodestyle==2.5.0 3 | pydocstyle>=2.1,<3.1 4 | sphinx==2.0.1 5 | sphinx_rtd_theme>=0.4,<0.5 6 | wrapt==1.11.1 7 | h5py>=2.9 8 | cloudpickle>=0.8.1 9 | -------------------------------------------------------------------------------- /requirements/requirements_extra.txt: -------------------------------------------------------------------------------- 1 | opencv-python==4.1.0.25 2 | nltk>=3.3,<3.5 3 | matplotlib>=2.2,<3.1 4 | requests==2.21.0 5 | tqdm==4.31.1 6 | lxml==4.3.3 7 | -------------------------------------------------------------------------------- /requirements/requirements_test.txt: -------------------------------------------------------------------------------- 1 | keras>=2.2,<2.3 2 | pycodestyle==2.5.0 3 | pydocstyle>=2.1,<3.1 4 | pytest==4.5.0 5 | pytest-cache>=1.0,<1.1 6 | pytest-cov==2.7.1 7 | pytest-xdist==1.28.0 8 | sphinx==2.0.1 9 | yapf==0.27.0 10 | -------------------------------------------------------------------------------- /requirements/requirements_tf_cpu.txt: -------------------------------------------------------------------------------- 1 | tensorflow>=2.0.0-alpha0 2 | -------------------------------------------------------------------------------- /requirements/requirements_tf_gpu.txt: -------------------------------------------------------------------------------- 1 | tensorflow-gpu>=2.0.0-alpha0 2 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | tensorlayer-master/setup.py -------------------------------------------------------------------------------- /tensorlayer: -------------------------------------------------------------------------------- 1 | tensorlayer-master/tensorlayer -------------------------------------------------------------------------------- /tl: -------------------------------------------------------------------------------- 1 | tensorlayer-master/tl --------------------------------------------------------------------------------