├── _assets ├── pr-reaction │ ├── .gitignore │ ├── 1x │ │ ├── reaction-1@1x.png │ │ ├── reaction-2@1x.png │ │ └── reaction-3@1x.png │ ├── 2x │ │ ├── reaction-1@2x.png │ │ ├── reaction-2@2x.png │ │ └── reaction-3@2x.png │ └── Makefile ├── lsst-logo-dark.ai └── README.md ├── .gitignore ├── requirements.txt ├── _static ├── README.md ├── buildbot │ ├── failure.jpg │ ├── Bad_Login.jpg │ ├── Login_IP.jpg │ ├── Login_TBD.jpg │ ├── To_Force.jpg │ ├── job_table.jpg │ ├── Build_Status.jpg │ ├── Build_Triggered_2.jpg │ ├── Force_Build_Request.jpg │ └── Force_Build_Request_2.jpg ├── development │ ├── pmcs_extract.jpg │ ├── docs │ │ └── lsst_logo.jpg │ ├── ldm_240_extract.jpg │ ├── planning_flowdown.jpg │ ├── git-rebase-history.png │ └── test_development_responsibility.png ├── stack │ ├── coverage-single-file.png │ └── coverage-summary-page.png ├── services │ ├── orchestration │ │ ├── soon.jpg │ │ ├── CondorDAG.png │ │ ├── production.png │ │ └── provenance.png │ └── nebula │ │ ├── nebula_intro_s1.png │ │ ├── nebula_intro_s2.png │ │ ├── nebula_intro_s3.png │ │ ├── nebula_intro_s4.png │ │ ├── nebula_intro_s5.png │ │ ├── nebula_intro_s6.png │ │ ├── nebula_intro_s7.png │ │ ├── nebula_intro_s8.png │ │ ├── nebula_intro_s9.png │ │ ├── nebula_stack_s1.png │ │ ├── nebula_stack_s2.png │ │ ├── nebula_stack_s3.png │ │ ├── nebula_stack_s4.png │ │ ├── nebula_stack_s5.png │ │ ├── nebula_stack_s6.png │ │ ├── nebula_stack_s7.png │ │ ├── nebula_stack_s8.png │ │ ├── nebula_stack_s9.png │ │ ├── nebula_intro_s10.png │ │ ├── nebula_intro_s11.png │ │ ├── nebula_stack_s10.png │ │ └── nebula_stack_s11.png ├── processes │ └── workflow │ │ ├── reaction@1x.gif │ │ └── reaction@2x.gif ├── usdf │ └── dev_guide │ │ ├── SLUO_New_User_Form.png │ │ └── SLUO_New_User_Form_p2.jpg ├── communications │ └── slack-culture │ │ ├── attach.png │ │ └── format-bar.png └── build-ci │ ├── github_pull_requests_settings.png │ └── github_branch_protection_rule_settings.png ├── it ├── index.rst └── itsc.rst ├── stack ├── examples │ ├── null-travis.yml │ ├── flake8-travis.yml │ ├── tracing.py │ └── license_preamble.txt ├── documenteer-branch-build.png ├── renaming-a-package.rst ├── index.rst ├── package-documentation-topic-types.rst ├── platforms.rst ├── debug.rst ├── transferring-code.rst ├── add-a-package-to-pipelines-lsst-io.rst ├── config-topic-type.rst ├── building-single-package-docs.rst ├── package-homepage-topic-type.rst ├── license-and-copyright.rst ├── moving-to-github-actions.rst ├── generic-guide-topic-type.rst ├── building-pipelines-lsst-io-locally.rst └── argparse-script-topic-type.rst ├── coding ├── logging.rst ├── using_boost.rst ├── using_eigen.rst ├── using_astropy.rst ├── cpp_style_guide.rst ├── python_testing.rst ├── python_style_guide.rst ├── unit_test_coverage.rst ├── using_cpp_templates.rst ├── profiling.rst ├── pybind11_style_guide.rst ├── debug.rst ├── unit_test_policy.rst ├── unit_test_private_functions.rst ├── README.md ├── python_wrappers_for_cpp_with_pybind11.rst └── intro.rst ├── git ├── index.rst └── samples │ ├── git_lfs_git-credentials.txt │ ├── git_lfs_git-credentials_legacy.txt │ ├── git_lfs_gitconfig.txt │ └── git_lfs_gitconfig_legacy.txt ├── javascript ├── index.rst └── jsdoc.rst ├── idf ├── index.rst └── overview.rst ├── jenkins ├── index.rst ├── jenkins-stack-os-matrix.rst ├── jenkins-sunny.svg ├── jenkins-cloudy.svg ├── jenkins-raining.svg ├── jenkins-storm.svg └── jenkins-partially-sunny.svg ├── restructuredtext ├── examples │ ├── math-align.rst │ ├── def-list.rst │ ├── math.rst │ ├── image.rst │ ├── math-multi.rst │ ├── noheader-table.rst │ ├── figure.rst │ ├── multi-header-table.rst │ ├── basic-table.rst │ └── py-codeblock.rst ├── index.rst ├── rst-sample.rst └── section-sample.rst ├── pybind11 └── index.rst ├── docs ├── cpp_docs.rst ├── package_docs.rst ├── rst_styleguide.rst ├── jsdoc.rst ├── py_docs.rst ├── change-controlled-docs.rst ├── technotes.rst └── README.md ├── processes ├── jira_agile.rst ├── code_of_conduct.rst ├── wiki.rst ├── transferring_code.rst ├── publication-policy.rst ├── project_planning.rst ├── presenting-at-conferences.rst ├── workflow.rst ├── decision_process.rst └── README.md ├── teams ├── drp.rst └── README.md ├── python ├── examples │ ├── pyproject.toml │ ├── test_baseclass.py │ ├── test_executables.py │ ├── test_runner_example.py │ ├── pyproject-ruff.toml │ ├── test_basic_example.py │ └── pre-commit-config.yaml ├── index.rst ├── cli.rst ├── astropy.rst └── formatting.rst ├── getting-started ├── onboarding.rst └── README.md ├── editors ├── index.rst └── vim.rst ├── tools ├── jira_tips.rst ├── vim.rst ├── emacs.rst ├── clang_format.rst ├── git_setup.rst ├── sublime.rst ├── git_lfs.rst └── README.md ├── _templates ├── README.md └── footer.html ├── .github └── workflows │ ├── rebase_checker.yaml │ ├── ci.yaml │ └── cron-ci.yaml ├── cpp ├── examples │ └── templates │ │ ├── README.md │ │ ├── Sconstruct │ │ ├── main.cc │ │ ├── Foo.cc │ │ └── Foo.h ├── index.rst ├── eigen.rst ├── compilation-db.rst ├── testing-private-functions.rst ├── profiling.rst ├── clang-format.rst └── boost.rst ├── work ├── index.rst ├── jira-tips.rst ├── backports.rst └── jira-agile.rst ├── project-docs ├── index.rst ├── test-documentation.rst └── publication-policy.rst ├── communications ├── index.rst ├── slack-github-username.rst ├── mail.rst ├── calendars.rst ├── wiki.rst ├── community-support.rst └── rfd.rst ├── team ├── index.rst ├── tools.rst ├── empowerment.rst ├── meeting-free-weeks.rst ├── code-of-conduct.rst └── focus-friday.rst ├── user-docs ├── lsst-specific-content-style-guide.rst └── index.rst ├── README.md └── legal └── licensing-overview.rst /_assets/pr-reaction/.gitignore: -------------------------------------------------------------------------------- 1 | reaction@1x.gif 2 | reaction@2x.gif 3 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | _build 2 | .DS_Store 3 | __pycache__ 4 | .cache 5 | .venv 6 | venv 7 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | documenteer 2 | sphinx-prompt 3 | sphinx_rtd_theme 4 | sphinx<6.0.0 5 | -------------------------------------------------------------------------------- /_static/README.md: -------------------------------------------------------------------------------- 1 | This `_static` directory is used for assets such as CSS and images. 2 | -------------------------------------------------------------------------------- /it/index.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ########### 4 | IT Overview 5 | ########### 6 | 7 | - :doc:`itsc` 8 | -------------------------------------------------------------------------------- /stack/examples/null-travis.yml: -------------------------------------------------------------------------------- 1 | sudo: false 2 | language: python 3 | script: 4 | - echo Passed 5 | -------------------------------------------------------------------------------- /_assets/lsst-logo-dark.ai: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_assets/lsst-logo-dark.ai -------------------------------------------------------------------------------- /_static/buildbot/failure.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/buildbot/failure.jpg -------------------------------------------------------------------------------- /_static/buildbot/Bad_Login.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/buildbot/Bad_Login.jpg -------------------------------------------------------------------------------- /_static/buildbot/Login_IP.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/buildbot/Login_IP.jpg -------------------------------------------------------------------------------- /_static/buildbot/Login_TBD.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/buildbot/Login_TBD.jpg -------------------------------------------------------------------------------- /_static/buildbot/To_Force.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/buildbot/To_Force.jpg -------------------------------------------------------------------------------- /_static/buildbot/job_table.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/buildbot/job_table.jpg -------------------------------------------------------------------------------- /_static/buildbot/Build_Status.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/buildbot/Build_Status.jpg -------------------------------------------------------------------------------- /coding/logging.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ####### 4 | Logging 5 | ####### 6 | 7 | This page moved: :doc:`/stack/logging`. 8 | -------------------------------------------------------------------------------- /git/index.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ########## 4 | Git guides 5 | ########## 6 | 7 | - :doc:`setup` 8 | - :doc:`git-lfs` 9 | -------------------------------------------------------------------------------- /_static/development/pmcs_extract.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/development/pmcs_extract.jpg -------------------------------------------------------------------------------- /javascript/index.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ################# 4 | JavaScript guides 5 | ################# 6 | 7 | - :doc:`jsdoc` 8 | -------------------------------------------------------------------------------- /stack/documenteer-branch-build.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/stack/documenteer-branch-build.png -------------------------------------------------------------------------------- /_static/buildbot/Build_Triggered_2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/buildbot/Build_Triggered_2.jpg -------------------------------------------------------------------------------- /_static/development/docs/lsst_logo.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/development/docs/lsst_logo.jpg -------------------------------------------------------------------------------- /_static/stack/coverage-single-file.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/stack/coverage-single-file.png -------------------------------------------------------------------------------- /coding/using_boost.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ########### 4 | Using Boost 5 | ########### 6 | 7 | This page moved: :doc:`/cpp/boost`. 8 | -------------------------------------------------------------------------------- /coding/using_eigen.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ########### 4 | Using Eigen 5 | ########### 6 | 7 | This page moved: :doc:`/cpp/eigen`. 8 | -------------------------------------------------------------------------------- /git/samples/git_lfs_git-credentials.txt: -------------------------------------------------------------------------------- 1 | https://:@lsst-sqre-prod-git-lfs.s3-us-west-2.amazonaws.com 2 | https://:@s3.lsst.codes 3 | -------------------------------------------------------------------------------- /_assets/pr-reaction/1x/reaction-1@1x.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_assets/pr-reaction/1x/reaction-1@1x.png -------------------------------------------------------------------------------- /_assets/pr-reaction/1x/reaction-2@1x.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_assets/pr-reaction/1x/reaction-2@1x.png -------------------------------------------------------------------------------- /_assets/pr-reaction/1x/reaction-3@1x.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_assets/pr-reaction/1x/reaction-3@1x.png -------------------------------------------------------------------------------- /_assets/pr-reaction/2x/reaction-1@2x.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_assets/pr-reaction/2x/reaction-1@2x.png -------------------------------------------------------------------------------- /_assets/pr-reaction/2x/reaction-2@2x.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_assets/pr-reaction/2x/reaction-2@2x.png -------------------------------------------------------------------------------- /_assets/pr-reaction/2x/reaction-3@2x.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_assets/pr-reaction/2x/reaction-3@2x.png -------------------------------------------------------------------------------- /_static/buildbot/Force_Build_Request.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/buildbot/Force_Build_Request.jpg -------------------------------------------------------------------------------- /_static/development/ldm_240_extract.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/development/ldm_240_extract.jpg -------------------------------------------------------------------------------- /_static/development/planning_flowdown.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/development/planning_flowdown.jpg -------------------------------------------------------------------------------- /_static/services/orchestration/soon.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/services/orchestration/soon.jpg -------------------------------------------------------------------------------- /_static/stack/coverage-summary-page.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/stack/coverage-summary-page.png -------------------------------------------------------------------------------- /idf/index.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ##################### 4 | Interim Data Facility 5 | ##################### 6 | 7 | - :doc:`overview` 8 | -------------------------------------------------------------------------------- /jenkins/index.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ################# 4 | Jenkins CI guides 5 | ################# 6 | 7 | - :doc:`getting-started` 8 | -------------------------------------------------------------------------------- /restructuredtext/examples/math-align.rst: -------------------------------------------------------------------------------- 1 | :orphan: True 2 | 3 | .. math:: 4 | 5 | x &= (a + b)^2 \\ 6 | &= a^2 + 2ab + b^2 7 | -------------------------------------------------------------------------------- /_static/buildbot/Force_Build_Request_2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/buildbot/Force_Build_Request_2.jpg -------------------------------------------------------------------------------- /_static/development/git-rebase-history.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/development/git-rebase-history.png -------------------------------------------------------------------------------- /_static/processes/workflow/reaction@1x.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/processes/workflow/reaction@1x.gif -------------------------------------------------------------------------------- /_static/processes/workflow/reaction@2x.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/processes/workflow/reaction@2x.gif -------------------------------------------------------------------------------- /_static/services/nebula/nebula_intro_s1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/services/nebula/nebula_intro_s1.png -------------------------------------------------------------------------------- /_static/services/nebula/nebula_intro_s2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/services/nebula/nebula_intro_s2.png -------------------------------------------------------------------------------- /_static/services/nebula/nebula_intro_s3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/services/nebula/nebula_intro_s3.png -------------------------------------------------------------------------------- /_static/services/nebula/nebula_intro_s4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/services/nebula/nebula_intro_s4.png -------------------------------------------------------------------------------- /_static/services/nebula/nebula_intro_s5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/services/nebula/nebula_intro_s5.png -------------------------------------------------------------------------------- /_static/services/nebula/nebula_intro_s6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/services/nebula/nebula_intro_s6.png -------------------------------------------------------------------------------- /_static/services/nebula/nebula_intro_s7.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/services/nebula/nebula_intro_s7.png -------------------------------------------------------------------------------- /_static/services/nebula/nebula_intro_s8.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/services/nebula/nebula_intro_s8.png -------------------------------------------------------------------------------- /_static/services/nebula/nebula_intro_s9.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/services/nebula/nebula_intro_s9.png -------------------------------------------------------------------------------- /_static/services/nebula/nebula_stack_s1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/services/nebula/nebula_stack_s1.png -------------------------------------------------------------------------------- /_static/services/nebula/nebula_stack_s2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/services/nebula/nebula_stack_s2.png -------------------------------------------------------------------------------- /_static/services/nebula/nebula_stack_s3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/services/nebula/nebula_stack_s3.png -------------------------------------------------------------------------------- /_static/services/nebula/nebula_stack_s4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/services/nebula/nebula_stack_s4.png -------------------------------------------------------------------------------- /_static/services/nebula/nebula_stack_s5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/services/nebula/nebula_stack_s5.png -------------------------------------------------------------------------------- /_static/services/nebula/nebula_stack_s6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/services/nebula/nebula_stack_s6.png -------------------------------------------------------------------------------- /_static/services/nebula/nebula_stack_s7.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/services/nebula/nebula_stack_s7.png -------------------------------------------------------------------------------- /_static/services/nebula/nebula_stack_s8.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/services/nebula/nebula_stack_s8.png -------------------------------------------------------------------------------- /_static/services/nebula/nebula_stack_s9.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/services/nebula/nebula_stack_s9.png -------------------------------------------------------------------------------- /pybind11/index.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ############### 4 | Pybind11 guides 5 | ############### 6 | 7 | - :doc:`style` 8 | - :doc:`how-to` 9 | -------------------------------------------------------------------------------- /_static/services/nebula/nebula_intro_s10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/services/nebula/nebula_intro_s10.png -------------------------------------------------------------------------------- /_static/services/nebula/nebula_intro_s11.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/services/nebula/nebula_intro_s11.png -------------------------------------------------------------------------------- /_static/services/nebula/nebula_stack_s10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/services/nebula/nebula_stack_s10.png -------------------------------------------------------------------------------- /_static/services/nebula/nebula_stack_s11.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/services/nebula/nebula_stack_s11.png -------------------------------------------------------------------------------- /_static/services/orchestration/CondorDAG.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/services/orchestration/CondorDAG.png -------------------------------------------------------------------------------- /_static/services/orchestration/production.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/services/orchestration/production.png -------------------------------------------------------------------------------- /_static/services/orchestration/provenance.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/services/orchestration/provenance.png -------------------------------------------------------------------------------- /_static/usdf/dev_guide/SLUO_New_User_Form.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/usdf/dev_guide/SLUO_New_User_Form.png -------------------------------------------------------------------------------- /coding/using_astropy.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ############# 4 | Using Astropy 5 | ############# 6 | 7 | This page moved: :doc:`/python/astropy`. 8 | -------------------------------------------------------------------------------- /restructuredtext/examples/def-list.rst: -------------------------------------------------------------------------------- 1 | :orphan: True 2 | 3 | LSST 4 | Legacy Survey of Space and Time 5 | 6 | DM 7 | Data Management 8 | -------------------------------------------------------------------------------- /restructuredtext/examples/math.rst: -------------------------------------------------------------------------------- 1 | :orphan: True 2 | 3 | .. math:: \sigma_\mathrm{mean} = \frac{\sigma}{\sqrt{N}} 4 | :label: math-sample 5 | -------------------------------------------------------------------------------- /_static/communications/slack-culture/attach.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/communications/slack-culture/attach.png -------------------------------------------------------------------------------- /_static/usdf/dev_guide/SLUO_New_User_Form_p2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/usdf/dev_guide/SLUO_New_User_Form_p2.jpg -------------------------------------------------------------------------------- /restructuredtext/index.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ####################### 4 | ReStructuredText guides 5 | ####################### 6 | 7 | - :doc:`style` 8 | -------------------------------------------------------------------------------- /_static/build-ci/github_pull_requests_settings.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/build-ci/github_pull_requests_settings.png -------------------------------------------------------------------------------- /_static/communications/slack-culture/format-bar.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/communications/slack-culture/format-bar.png -------------------------------------------------------------------------------- /coding/cpp_style_guide.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ################## 4 | DM C++ Style Guide 5 | ################## 6 | 7 | This page moved: :doc:`/cpp/style`. 8 | -------------------------------------------------------------------------------- /docs/cpp_docs.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | #################### 4 | Documenting C++ Code 5 | #################### 6 | 7 | This page moved: :doc:`/cpp/api-docs`. 8 | -------------------------------------------------------------------------------- /processes/jira_agile.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ################ 4 | JIRA Agile Usage 5 | ################ 6 | 7 | This page moved: :doc:`/work/jira-agile`. 8 | -------------------------------------------------------------------------------- /processes/code_of_conduct.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ############### 4 | Code of conduct 5 | ############### 6 | 7 | This page moved: :doc:`/team/code-of-conduct`. 8 | -------------------------------------------------------------------------------- /teams/drp.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ####################### 4 | Data Release Processing 5 | ####################### 6 | 7 | This page moved: :doc:`/team/drp`. 8 | -------------------------------------------------------------------------------- /_static/development/test_development_responsibility.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/development/test_development_responsibility.png -------------------------------------------------------------------------------- /coding/python_testing.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ################### 4 | Python Unit Testing 5 | ################### 6 | 7 | This page moved: :doc:`/python/testing`. 8 | -------------------------------------------------------------------------------- /python/examples/pyproject.toml: -------------------------------------------------------------------------------- 1 | [tool.black] 2 | line-length = 110 3 | target-version = ["py38"] 4 | 5 | [tool.isort] 6 | profile = "black" 7 | line_length = 110 8 | -------------------------------------------------------------------------------- /coding/python_style_guide.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ##################### 4 | DM Python Style Guide 5 | ##################### 6 | 7 | This page moved: :doc:`/python/style`. 8 | -------------------------------------------------------------------------------- /coding/unit_test_coverage.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ################# 4 | Coverage Analysis 5 | ################# 6 | 7 | This page moved: :doc:`/stack/unit-test-coverage`. 8 | -------------------------------------------------------------------------------- /coding/using_cpp_templates.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ################### 4 | Using C++ Templates 5 | ################### 6 | 7 | This page moved: :doc:`/cpp/templates`. 8 | -------------------------------------------------------------------------------- /git/samples/git_lfs_git-credentials_legacy.txt: -------------------------------------------------------------------------------- 1 | https://:@lsst-sqre-prod-git-lfs.s3-us-west-2.amazonaws.com 2 | https://:@s3.lsst.codes 3 | https://:@git-lfs.lsst.codes 4 | -------------------------------------------------------------------------------- /processes/wiki.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ##################### 4 | Policy on DM Wiki Use 5 | ##################### 6 | 7 | This page moved: :doc:`/communications/wiki`. 8 | -------------------------------------------------------------------------------- /_static/build-ci/github_branch_protection_rule_settings.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lsst-dm/dm_dev_guide/HEAD/_static/build-ci/github_branch_protection_rule_settings.png -------------------------------------------------------------------------------- /getting-started/onboarding.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | #################### 4 | Onboarding checklist 5 | #################### 6 | 7 | This page moved: :doc:`/team/onboarding`. 8 | -------------------------------------------------------------------------------- /restructuredtext/examples/image.rst: -------------------------------------------------------------------------------- 1 | :orphan: True 2 | 3 | .. image:: /_static/development/docs/lsst_logo.jpg 4 | :target: ../_images/lsst_logo.jpg 5 | :alt: LSST Logo 6 | -------------------------------------------------------------------------------- /coding/profiling.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ######### 4 | Profiling 5 | ######### 6 | 7 | This page moved: 8 | 9 | - :doc:`/cpp/profiling`. 10 | - :doc:`/python/profiling`. 11 | -------------------------------------------------------------------------------- /coding/pybind11_style_guide.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ####################### 4 | DM Pybind11 Style Guide 5 | ####################### 6 | 7 | This page moved: :doc:`/pybind11/style`. 8 | -------------------------------------------------------------------------------- /docs/package_docs.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ########################## 4 | Documenting Stack Packages 5 | ########################## 6 | 7 | This page moved: :ref:`part-dm-stack`. 8 | -------------------------------------------------------------------------------- /editors/index.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ############# 4 | Editor guides 5 | ############# 6 | 7 | - :doc:`emacs` 8 | - :doc:`sublime` 9 | - :doc:`vim` 10 | - :doc:`vscode` 11 | -------------------------------------------------------------------------------- /restructuredtext/examples/math-multi.rst: -------------------------------------------------------------------------------- 1 | :orphan: True 2 | 3 | .. math:: 4 | 5 | \nabla \cdot \mathbf{E} = \frac{\rho}{\epsilon_0} 6 | 7 | \nabla \cdot \mathbf{B} = 0 8 | -------------------------------------------------------------------------------- /tools/jira_tips.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ############################# 4 | JIRA Work Management Receipes 5 | ############################# 6 | 7 | This page moved: :doc:`/work/jira-tips`. 8 | -------------------------------------------------------------------------------- /coding/debug.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ################################## 4 | Debugging Tasks with ``lsstDebug`` 5 | ################################## 6 | 7 | This page moved: :doc:`/stack/debug`. 8 | -------------------------------------------------------------------------------- /coding/unit_test_policy.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ######################### 4 | Software Unit Test Policy 5 | ######################### 6 | 7 | This page moved: :doc:`/coding/unit-test-policy`. 8 | -------------------------------------------------------------------------------- /docs/rst_styleguide.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ############################ 4 | ReStructuredText Style Guide 5 | ############################ 6 | 7 | This page moved: :doc:`/restructuredtext/style`. 8 | -------------------------------------------------------------------------------- /teams/README.md: -------------------------------------------------------------------------------- 1 | # Team Specific Information 2 | 3 | This section of the Developer Guide contains material which is unique to individual teams, such as pointers to local resources or procedures. 4 | -------------------------------------------------------------------------------- /tools/vim.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ###################################### 4 | Vim Configuration for LSST Development 5 | ###################################### 6 | 7 | This page moved: :doc:`/editors/vim`. 8 | -------------------------------------------------------------------------------- /docs/jsdoc.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ###################################### 4 | Documenting JavaScript APIs with JSDoc 5 | ###################################### 6 | 7 | This page moved: :doc:`/javascript/jsdoc`. 8 | -------------------------------------------------------------------------------- /docs/py_docs.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ####################################### 4 | Documenting Python APIs with Docstrings 5 | ####################################### 6 | 7 | This page moved: :doc:`/python/numpydoc`. 8 | -------------------------------------------------------------------------------- /tools/emacs.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ######################################## 4 | Emacs Configuration for LSST Development 5 | ######################################## 6 | 7 | This page moved: :doc:`/editors/emacs`. 8 | -------------------------------------------------------------------------------- /_templates/README.md: -------------------------------------------------------------------------------- 1 | This `_templates` directory is used for template files *that override the theme's templates*. 2 | See [Sphinx's documentation](http://sphinx-doc.org/templating.html) for more information on templates. 3 | -------------------------------------------------------------------------------- /processes/transferring_code.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ################################## 4 | Transferring Code Between Packages 5 | ################################## 6 | 7 | This page moved: :doc:`/stack/transferring-code`. 8 | -------------------------------------------------------------------------------- /tools/clang_format.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ####################################### 4 | Using clang-format for LSST Development 5 | ####################################### 6 | 7 | This page moved: :doc:`/cpp/clang-format`. 8 | -------------------------------------------------------------------------------- /processes/publication-policy.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ################################# 4 | Publishing Papers and Proceedings 5 | ################################# 6 | 7 | This page moved: :doc:`/project-docs/publication-policy`. 8 | -------------------------------------------------------------------------------- /restructuredtext/examples/noheader-table.rst: -------------------------------------------------------------------------------- 1 | :orphan: True 2 | 3 | +-----------+--------------+ 4 | | us-east-1 | ami-e2490b88 | 5 | +-----------+--------------+ 6 | | us-west-2 | ami-9a0f1dfb | 7 | +-----------+--------------+ 8 | -------------------------------------------------------------------------------- /tools/git_setup.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ############################################# 4 | Git Configuration Standards & Recommendations 5 | ############################################# 6 | 7 | This page moved: :doc:`/git/setup`. 8 | -------------------------------------------------------------------------------- /coding/unit_test_private_functions.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ################################## 4 | Unit-Testing Private C++ Functions 5 | ################################## 6 | 7 | This page moved: :doc:`/cpp/testing-private-functions`. 8 | -------------------------------------------------------------------------------- /coding/README.md: -------------------------------------------------------------------------------- 1 | # Coding Guide 2 | 3 | This section of the Developer Guide pertains to our coding standards and patterns. 4 | Our formal code style guides belong here. 5 | Less formal code development documentation also belongs here. 6 | -------------------------------------------------------------------------------- /coding/python_wrappers_for_cpp_with_pybind11.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ##################################### 4 | Python wrappers for C++ with pybind11 5 | ##################################### 6 | 7 | This page moved: :doc:`/pybind11/how-to`. 8 | -------------------------------------------------------------------------------- /processes/project_planning.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ######################################### 4 | Project Planning for Software Development 5 | ######################################### 6 | 7 | This page moved: :doc:`/work/project-planning`. 8 | -------------------------------------------------------------------------------- /restructuredtext/examples/figure.rst: -------------------------------------------------------------------------------- 1 | :orphan: True 2 | 3 | .. figure:: /_static/development/docs/lsst_logo.jpg 4 | :name: fig-example-figure-label 5 | :target: ../_images/lsst_logo.jpg 6 | :alt: LSST Logo 7 | 8 | LSST Logo. 9 | -------------------------------------------------------------------------------- /tools/sublime.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ############################################## 4 | SublimeText Configuration for LSST Development 5 | ############################################## 6 | 7 | This page moved: :doc:`/editors/sublime`. 8 | -------------------------------------------------------------------------------- /_assets/README.md: -------------------------------------------------------------------------------- 1 | Use this `_assets/` directory for source files that can be compiled into web-ready assets. 2 | For example, Illustrator or Photoshop files that are exported to PNGs. 3 | 4 | Package each asset project in its own subdirectory. 5 | -------------------------------------------------------------------------------- /docs/change-controlled-docs.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ####################################### 4 | Writing change-controlled documentation 5 | ####################################### 6 | 7 | This page moved: :doc:`/project-docs/change-controlled-docs`. 8 | -------------------------------------------------------------------------------- /git/samples/git_lfs_gitconfig.txt: -------------------------------------------------------------------------------- 1 | # Cache anonymous access to DM Git LFS S3 servers 2 | [credential "https://lsst-sqre-prod-git-lfs.s3-us-west-2.amazonaws.com"] 3 | helper = store 4 | [credential "https://s3.lsst.codes"] 5 | helper = store 6 | -------------------------------------------------------------------------------- /.github/workflows/rebase_checker.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | name: Check that 'main' is not merged into the development branch 3 | 4 | on: pull_request 5 | 6 | jobs: 7 | call-workflow: 8 | uses: lsst/rubin_workflows/.github/workflows/rebase_checker.yaml@main 9 | -------------------------------------------------------------------------------- /docs/technotes.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ################################################# 4 | Technotes for Stand-Alone Technical Documentation 5 | ################################################# 6 | 7 | This page moved: :doc:`/project-docs/technotes`. 8 | -------------------------------------------------------------------------------- /python/index.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ############# 4 | Python guides 5 | ############# 6 | 7 | - :doc:`style` 8 | - :doc:`formatting` 9 | - :doc:`testing` 10 | - :doc:`numpydoc` 11 | - :doc:`astropy` 12 | - :doc:`profiling` 13 | - :doc:`cli` 14 | -------------------------------------------------------------------------------- /processes/presenting-at-conferences.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ####################################### 4 | Attending and Presenting at Conferences 5 | ####################################### 6 | 7 | This page moved: :doc:`/communications/presenting-at-conferences`. 8 | -------------------------------------------------------------------------------- /stack/examples/flake8-travis.yml: -------------------------------------------------------------------------------- 1 | sudo: false 2 | language: python 3 | matrix: 4 | include: 5 | - python: '3.6' 6 | install: 7 | - pip install -r <(curl https://raw.githubusercontent.com/lsst/linting/main/requirements.txt) 8 | script: flake8 9 | -------------------------------------------------------------------------------- /tools/git_lfs.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ######################################################## 4 | Using Git LFS (Large File Storage) for data repositories 5 | ######################################################## 6 | 7 | This page moved: :doc:`/git/git-lfs`. 8 | -------------------------------------------------------------------------------- /cpp/examples/templates/README.md: -------------------------------------------------------------------------------- 1 | This directory contains code samples that are rendered by the `using_cpp_templates.rst` page. 2 | If you change any of these code samples, ensure that the `:lines:` selection field of the `literalinclude` directives are still correct. 3 | -------------------------------------------------------------------------------- /docs/README.md: -------------------------------------------------------------------------------- 1 | # Writing Documentation 2 | 3 | This section of the Developer Guide is intended to help developers write for LSST. 4 | This section covers both the technical aspects of our documentation infrastructure, as well as information architecture and style patterns. 5 | -------------------------------------------------------------------------------- /processes/workflow.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ########################################################## 4 | DM Development Workflow with Git, GitHub, JIRA and Jenkins 5 | ########################################################## 6 | 7 | This page moved: :doc:`/work/flow`. 8 | -------------------------------------------------------------------------------- /work/index.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ############### 4 | Work management 5 | ############### 6 | 7 | How DM coordinates work and gets things done. 8 | 9 | - :doc:`flow` 10 | - :doc:`project-planning` 11 | - :doc:`jira-agile` 12 | - :doc:`jira-tips` 13 | - :doc:`backports` 14 | -------------------------------------------------------------------------------- /tools/README.md: -------------------------------------------------------------------------------- 1 | # Developer Tools 2 | 3 | The Developer Tools section of the Developer Guide is a miscellaneous catch-all to document best-practices for setting up developer environments. 4 | Possible topics include editor configuration guides and guides for infrastructure such as JIRA. 5 | -------------------------------------------------------------------------------- /project-docs/index.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ##################### 4 | Project documentation 5 | ##################### 6 | 7 | Controlled documentation and publications. 8 | 9 | - :doc:`change-controlled-docs` 10 | - :doc:`publication-policy` 11 | 12 | Technical notes. 13 | 14 | - :doc:`technotes` 15 | -------------------------------------------------------------------------------- /cpp/index.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ########## 4 | C++ guides 5 | ########## 6 | 7 | - :doc:`style` 8 | - :doc:`api-docs` 9 | - :doc:`clang-format` 10 | - :doc:`testing-private-functions` 11 | - :doc:`templates` 12 | - :doc:`boost` 13 | - :doc:`eigen` 14 | - :doc:`profiling` 15 | - :doc:`compilation-db` 16 | -------------------------------------------------------------------------------- /processes/decision_process.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ################################################# 4 | Discussion and Decision Making Process (RFC, RFD) 5 | ################################################# 6 | 7 | This page moved: 8 | 9 | - :doc:`/communications/rfc` 10 | - :doc:`/communications/rfd` 11 | -------------------------------------------------------------------------------- /python/examples/test_baseclass.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | 3 | 4 | class BaseClass: 5 | def testParam(self): 6 | self.assertLess(self.PARAM, 5) 7 | 8 | 9 | class ThisIsTest1(BaseClass, unittest.TestCase): 10 | PARAM = 3 11 | 12 | 13 | if __name__ == "__main__": 14 | unittest.main() 15 | -------------------------------------------------------------------------------- /getting-started/README.md: -------------------------------------------------------------------------------- 1 | # Getting Started 2 | 3 | This section of the Developer Guide is for orienting new Data Management developers. 4 | Keep the material here brief, with as few prior knowledge assumptions as possible. 5 | Link out to the rest of the Develop Guide as much as possible to encourage deeper reading. 6 | -------------------------------------------------------------------------------- /jenkins/jenkins-stack-os-matrix.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ############################################################## 4 | Testing the LSST DM Stack with the Jenkins stack-os-matrix Job 5 | ############################################################## 6 | 7 | This page moved: :doc:`/stack/jenkins-stack-os-matrix`. 8 | -------------------------------------------------------------------------------- /processes/README.md: -------------------------------------------------------------------------------- 1 | # Processes & Policies 2 | 3 | This section of the Developer Guide is to provide definitive guidance on *how we do things at Data Management.* 4 | Changes to content here should be reviewed by the Technical Control Team (TCT): the Project Manager, Project Engineer, Software Architect and Project Scientist. 5 | -------------------------------------------------------------------------------- /python/examples/test_executables.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | import lsst.utils.tests 3 | 4 | 5 | class UtilsBinaryTester(lsst.utils.tests.ExecutablesTestCase): 6 | pass 7 | 8 | EXECUTABLES = None 9 | UtilsBinaryTester.create_executable_tests(__file__, EXECUTABLES) 10 | 11 | if __name__ == "__main__": 12 | unittest.main() 13 | -------------------------------------------------------------------------------- /cpp/examples/templates/Sconstruct: -------------------------------------------------------------------------------- 1 | # -*- python -*- 2 | # 3 | # Setup our environment 4 | # 5 | import lsst.SConsUtils as scons 6 | 7 | env = scons.makeEnv("Foo", 8 | r"$HeadURL$", 9 | []) 10 | 11 | env.SharedLibrary('Foo', "Foo.cc") 12 | env.Program(["main.cc"], LIBS="Foo", LIBPATH=".") 13 | -------------------------------------------------------------------------------- /_assets/pr-reaction/Makefile: -------------------------------------------------------------------------------- 1 | all: reaction@1x.gif reaction@2x.gif 2 | 3 | reaction@1x.gif: 1x/reaction-1@1x.png 1x/reaction-2@1x.png 1x/reaction-3@1x.png 4 | convert -delay 100 -loop 0 1x/*.png reaction@1x.gif 5 | 6 | reaction@2x.gif: 2x/reaction-1@2x.png 2x/reaction-2@2x.png 2x/reaction-3@2x.png 7 | convert -delay 100 -loop 0 2x/*.png reaction@2x.gif 8 | -------------------------------------------------------------------------------- /communications/index.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ############## 4 | Communications 5 | ############## 6 | 7 | How to use DM's communication channels. 8 | 9 | - :doc:`slack-culture` 10 | - :doc:`slack-github-username` 11 | - :doc:`rfc` 12 | - :doc:`rfd` 13 | - :doc:`wiki` 14 | - :doc:`mail` 15 | - :doc:`presenting-at-conferences` 16 | - :doc:`community-support` 17 | - :doc:`calendars` 18 | -------------------------------------------------------------------------------- /cpp/examples/templates/main.cc: -------------------------------------------------------------------------------- 1 | #include 2 | #include "Foo.h" 3 | 4 | int main() { 5 | Foo i; 6 | Foo x(1.234); 7 | 8 | std::cout << "i = " << i.getVal() << ", " << x.getVal() << std::endl; 9 | 10 | i.setVal(10); 11 | std::cout << "i = " << i.getVal() << std::endl; 12 | 13 | std::cout << "mySizeof(float) = " << mySizeof() << std::endl; 14 | } 15 | -------------------------------------------------------------------------------- /git/samples/git_lfs_gitconfig_legacy.txt: -------------------------------------------------------------------------------- 1 | [lfs] 2 | batch = false 3 | 4 | # Cache anonymous access to DM Git LFS S3 servers 5 | [credential "https://lsst-sqre-prod-git-lfs.s3-us-west-2.amazonaws.com"] 6 | helper = store 7 | [credential "https://s3.lsst.codes"] 8 | helper = store 9 | 10 | # Cache anonymous access to DM Git LFS server 11 | [credential "https://git-lfs.lsst.codes"] 12 | helper = store 13 | -------------------------------------------------------------------------------- /team/index.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | #### 4 | Team 5 | #### 6 | 7 | Basic information about the LSST Data Management Subsystem and teams. 8 | Learn about the Data Management Subsystem in :ldm:`294`. 9 | 10 | - :doc:`onboarding` 11 | - :doc:`tools` 12 | - :doc:`code-of-conduct` 13 | - :doc:`focus-friday` 14 | - :doc:`meeting-free-weeks` 15 | - :doc:`empowerment` 16 | 17 | Team-specific pages: 18 | 19 | - :doc:`drp` 20 | -------------------------------------------------------------------------------- /jenkins/jenkins-sunny.svg: -------------------------------------------------------------------------------- 1 | Sunny -------------------------------------------------------------------------------- /stack/renaming-a-package.rst: -------------------------------------------------------------------------------- 1 | ######################## 2 | Renaming a Stack package 3 | ######################## 4 | 5 | The name for a Stack package is manifested in many places, from the name of the GitHub repository, to the C++ and Python namespaces. 6 | Sometimes it's necessary to rename a Stack package because of refactoring or shifts in terminology. 7 | `DMTN-027`_ describes how to rename a Stack package with minimal disruption to the Stack. 8 | 9 | .. _`DMTN-027`: https://dmtn-027.lsst.io 10 | -------------------------------------------------------------------------------- /python/examples/test_runner_example.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | import lsst.utils.tests 3 | 4 | 5 | class DemoTestCase(lsst.utils.tests.TestCase): 6 | """Demo test case.""" 7 | 8 | def testDemo(self): 9 | self.assertNotIn("i", "team") 10 | 11 | 12 | class MemoryTester(lsst.utils.tests.MemoryTestCase): 13 | pass 14 | 15 | 16 | def setup_module(module): 17 | lsst.utils.tests.init() 18 | 19 | 20 | if __name__ == "__main__": 21 | lsst.utils.tests.init() 22 | unittest.main() 23 | -------------------------------------------------------------------------------- /python/examples/pyproject-ruff.toml: -------------------------------------------------------------------------------- 1 | [tool.ruff] 2 | line-length = 110 3 | target-version = "py311" 4 | exclude = [ 5 | "__init__.py", 6 | ] 7 | 8 | [tool.ruff.lint] 9 | ignore = [ 10 | "N802", 11 | "N803", 12 | "N806", 13 | "N812", 14 | "N813", 15 | "N815", 16 | "N816", 17 | "N999", 18 | ] 19 | select = [ 20 | "E", # pycodestyle 21 | "F", # pyflakes 22 | "N", # pep8-naming 23 | "W", # pycodestyle 24 | ] 25 | extend-select = [ 26 | "RUF100", # Warn about unused noqa 27 | ] 28 | 29 | [tool.ruff.lint.pycodestyle] 30 | max-doc-length = 79 31 | 32 | [tool.ruff.lint.pydocstyle] 33 | convention = "numpy" 34 | -------------------------------------------------------------------------------- /python/examples/test_basic_example.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | import math 3 | 4 | 5 | class DemoTestCase1(unittest.TestCase): 6 | """Demo test case 1.""" 7 | 8 | def testDemo(self): 9 | self.assertGreater(10, 5) 10 | with self.assertRaisesRegex(TypeError, "unsupported operand"): 11 | 1 + "2" 12 | 13 | 14 | class DemoTestCase2(unittest.TestCase): 15 | """Demo test case 2.""" 16 | 17 | def testDemo1(self): 18 | self.assertNotEqual("string1", "string2") 19 | 20 | def testDemo2(self): 21 | self.assertAlmostEqual(3.14, math.pi, places=2) 22 | 23 | 24 | if __name__ == "__main__": 25 | unittest.main() 26 | -------------------------------------------------------------------------------- /restructuredtext/examples/multi-header-table.rst: -------------------------------------------------------------------------------- 1 | :orphan: True 2 | 3 | .. _rst-table-multi-header-example: 4 | 5 | .. table:: Table with two header rows, including a span. 6 | 7 | +---------------------------------------------------+ 8 | | centos-7-stack-lsst_apps-w_2015_45-20151130234354 | 9 | +-----------+---------------------------------------+ 10 | | Region | AMI | 11 | +===========+=======================================+ 12 | | us-east-1 | ami-e2490b88 | 13 | +-----------+---------------------------------------+ 14 | | us-west-2 | ami-9a0f1dfb | 15 | +-----------+---------------------------------------+ 16 | -------------------------------------------------------------------------------- /communications/slack-github-username.rst: -------------------------------------------------------------------------------- 1 | ###################################################### 2 | Configuring your GitHub username in your Slack profile 3 | ###################################################### 4 | 5 | Some of DM's Slack integrations need to know your GitHub username to map your activity on GitHub to your Slack account. 6 | Here's how to help us do that by adding your GitHub username to your Slack profile: 7 | 8 | 1. Visit Slack: https://lsstc.slack.com. 9 | 10 | 2. Click on the **LSSTC** workspace label in the top-left corner and select **Profile & account** from the drop-down. 11 | 12 | 3. Add your GitHub username to the **GitHub Username** field of your profile. 13 | Don't include any prefix like ``@``. 14 | -------------------------------------------------------------------------------- /communications/mail.rst: -------------------------------------------------------------------------------- 1 | ############# 2 | Mailing lists 3 | ############# 4 | Email is not the preferred mode of contact within DM but there are several project mailing lists in use. 5 | The full set can be found at `Mailing lists`_. 6 | These lists are mainly populated by the Contacts Database groups. 7 | 8 | The relevant lists for DM are: 9 | - `dm-cams `_: Data Management CAM's 10 | - `dm-sst `_: DM System Science Team 11 | - `dm-staff `_: All DM construction staff 12 | 13 | However, these are seldom used and mentioned here for completeness. 14 | 15 | .. _Mailing lists: https://lists.lsst.org/mailman/listinfo 16 | -------------------------------------------------------------------------------- /cpp/examples/templates/Foo.cc: -------------------------------------------------------------------------------- 1 | /// Implementation of templated class Foo 2 | #include "Foo.h" 3 | 4 | template 5 | Foo::Foo() : _val(0) {} 6 | 7 | template 8 | Foo::Foo(T val) : _val(val) {} 9 | 10 | template 11 | Foo::~Foo() {} 12 | 13 | #if 0 14 | template T Foo::getVal() const; // inline in Foo.h 15 | #endif 16 | 17 | template 18 | void Foo::setVal(T val) { 19 | _val = val; 20 | } 21 | // 22 | // And a templated function 23 | // 24 | template 25 | size_t mySizeof() { 26 | return sizeof(T); 27 | } 28 | 29 | // 30 | // Explicit instantiations 31 | // 32 | template class Foo; 33 | template class Foo; 34 | 35 | template size_t mySizeof(); 36 | -------------------------------------------------------------------------------- /restructuredtext/examples/basic-table.rst: -------------------------------------------------------------------------------- 1 | :orphan: True 2 | 3 | .. _table-label: 4 | 5 | .. table:: Table caption. 6 | 7 | +------------------------+------------+----------+----------+ 8 | | Header row, column 1 | Header 2 | Header 3 | Header 4 | 9 | | (header rows optional) | | | | 10 | +========================+============+==========+==========+ 11 | | body row 1, column 1 | column 2 | column 3 | column 4 | 12 | | | with many | spans | | 13 | | | rows | both | | 14 | +------------------------+------------+ rows +----------+ 15 | | body row 2 | ... | | ... | 16 | +------------------------+------------+----------+----------+ 17 | -------------------------------------------------------------------------------- /restructuredtext/examples/py-codeblock.rst: -------------------------------------------------------------------------------- 1 | :orphan: True 2 | 3 | .. code-block:: py 4 | :name: context-timer-example 5 | :emphasize-lines: 4-13,15-17 6 | 7 | from contextlib import ContextDecorator 8 | import time 9 | 10 | class timercontext(ContextDecorator): 11 | 12 | def __enter__(self): 13 | self.start = time.clock() 14 | return self 15 | 16 | def __exit__(self, *args): 17 | self.end = time.clock() 18 | self.interval = self.end - self.start 19 | print('Duration: {0:.2e} sec'.format(self.interval)) 20 | 21 | @timercontext 22 | def run_slowly(): 23 | time.delay(1.) 24 | 25 | run_slowly() 26 | 27 | with timercontext() as t: 28 | time.delay(1) 29 | 30 | print('Delayed for {0:.1f}'.format(t.interval)) 31 | -------------------------------------------------------------------------------- /stack/index.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ############### 4 | DM Stack guides 5 | ############### 6 | 7 | - :doc:`platforms` 8 | - :doc:`transferring-code` 9 | - :doc:`deprecating-interfaces` 10 | - :doc:`logging` 11 | - :doc:`debug` 12 | - :doc:`documentation-system-overview` 13 | - :doc:`layout-of-doc-directory` 14 | - :doc:`package-documentation-topic-types` 15 | - :doc:`add-a-package-to-pipelines-lsst-io` 16 | - :doc:`building-single-package-docs` 17 | - :doc:`building-pipelines-lsst-io-locally` 18 | - :doc:`building-pipelines-lsst-io-with-documenteer-job` 19 | - :doc:`jenkins-stack-os-matrix` 20 | - :doc:`unit-test-coverage` 21 | - :doc:`lsstsw` 22 | - :doc:`eups-tutorial` 23 | - :doc:`adding-a-new-package` 24 | - :doc:`moving-to-github-actions` 25 | - :doc:`packaging-third-party-eups-dependencies` 26 | - :doc:`renaming-a-package` 27 | - :doc:`conda` 28 | -------------------------------------------------------------------------------- /.github/workflows/ci.yaml: -------------------------------------------------------------------------------- 1 | name: CI 2 | 3 | on: [push, pull_request] 4 | 5 | jobs: 6 | docs: 7 | 8 | runs-on: ubuntu-latest 9 | 10 | steps: 11 | - uses: actions/checkout@v3 12 | 13 | - name: Set up Python 14 | uses: actions/setup-python@v4 15 | with: 16 | python-version: "3.10" 17 | 18 | - name: Install Python dependencies 19 | run: | 20 | python -m pip install --upgrade pip setuptools 21 | pip install -r requirements.txt 22 | pip install ltd-conveyor 23 | 24 | - name: Build documentation 25 | run: make html 26 | 27 | - name: Upload to LSST the Docs 28 | if: ${{ github.event_name == 'push' }} 29 | env: 30 | LTD_USERNAME: ${{ secrets.LTD_USERNAME }} 31 | LTD_PASSWORD: ${{ secrets.LTD_PASSWORD }} 32 | run: ltd upload --product developer --gh --dir _build/html 33 | -------------------------------------------------------------------------------- /team/tools.rst: -------------------------------------------------------------------------------- 1 | 2 | ############################# 3 | Tools and external resources 4 | ############################# 5 | 6 | DM uses many third party tools and systems. 7 | We have service accounts for most of these. 8 | Passwords are stored in 1Password vaults. 9 | 10 | Before creating or asking for a new third party service please consult `#rubinobs-it-public `_ on Slack. 11 | 12 | To gain access to the password vault raise an `IHS `_ ticket. 13 | 14 | 15 | Example services/tools 16 | ====================== 17 | 18 | This is not an exhaustive list but is meant as an indication for those who think they need a new service. 19 | 20 | - Amazon 21 | - AWS 22 | - Route53 23 | - S3 24 | - Google 25 | - Google Cloud 26 | - Suite 27 | - Artifact Registry 28 | - PyPI 29 | - GitHub 30 | - Anaconda 31 | - DockerHub 32 | -------------------------------------------------------------------------------- /stack/examples/tracing.py: -------------------------------------------------------------------------------- 1 | import logging 2 | import sys 3 | import lsst.utils.logging 4 | 5 | logging.basicConfig(level=logging.INFO, stream=sys.stdout) 6 | root_logger = logging.getLogger() 7 | 8 | 9 | def writeMessages(): 10 | """Write debugging messages under TRACEn.example.component 11 | 12 | In application code, they may appear either in Python or C++ codebase 13 | """ 14 | root_logger.info("Writing 6 debug messages") 15 | for n in range(6): 16 | logging.getLogger(f"TRACE{n}.example.component").debug("Fine tracing to TRACE%d", n) 17 | 18 | 19 | # Demonstrate using the utility function trace_set_at to set the levels for a 20 | # group of loggers so it display messages with trace number <= the threshold 21 | for threshold in range(6): 22 | root_logger.info("Setting trace at %d", threshold) 23 | lsst.utils.logging.trace_set_at("example.component", threshold) 24 | writeMessages() 25 | -------------------------------------------------------------------------------- /python/examples/pre-commit-config.yaml: -------------------------------------------------------------------------------- 1 | repos: 2 | - repo: https://github.com/pre-commit/pre-commit-hooks 3 | rev: v4.4.0 4 | hooks: 5 | - id: check-yaml 6 | args: 7 | - "--unsafe" 8 | - id: end-of-file-fixer 9 | - id: trailing-whitespace 10 | - repo: https://github.com/psf/black-pre-commit-mirror 11 | rev: 23.9.1 12 | hooks: 13 | - id: black 14 | # It is recommended to specify the latest version of Python 15 | # supported by your project here, or alternatively use 16 | # pre-commit's default_language_version, see 17 | # https://pre-commit.com/#top_level-default_language_version 18 | language_version: python3.11 19 | - repo: https://github.com/pycqa/isort 20 | rev: 5.12.0 21 | hooks: 22 | - id: isort 23 | name: isort (python) 24 | - repo: https://github.com/PyCQA/flake8 25 | rev: 6.0.0 26 | hooks: 27 | - id: flake8 28 | -------------------------------------------------------------------------------- /stack/examples/license_preamble.txt: -------------------------------------------------------------------------------- 1 | This file is part of %package_name%. 2 | 3 | Developed for the LSST Data Management System. 4 | This product includes software developed by the LSST Project 5 | (http://www.lsst.org). 6 | See the COPYRIGHT file at the top-level directory of this distribution 7 | for details of code ownership. 8 | 9 | This program is free software: you can redistribute it and/or modify 10 | it under the terms of the GNU General Public License as published by 11 | the Free Software Foundation, either version 3 of the License, or 12 | (at your option) any later version. 13 | 14 | This program is distributed in the hope that it will be useful, 15 | but WITHOUT ANY WARRANTY; without even the implied warranty of 16 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 17 | GNU General Public License for more details. 18 | 19 | You should have received a copy of the GNU General Public License 20 | along with this program. If not, see . 21 | -------------------------------------------------------------------------------- /stack/package-documentation-topic-types.rst: -------------------------------------------------------------------------------- 1 | .. _package-topic-types: 2 | 3 | ##################################### 4 | Topic types for package documentation 5 | ##################################### 6 | 7 | Each reStructuredText (``rst``) file, corresponding to an HTML page, that you add to the :file:`doc/` directory of a package should correspond to a **topic type.** 8 | Topic types are templates and associated guidelines that describe how to structure different types of content. 9 | By conforming to topic types, documentation contributed by a diverse group of collaborators is more structured and uniform. 10 | Structured documentation is easier for readers to navigate and consume. 11 | 12 | These are the available topic types for package documentation: 13 | 14 | .. toctree:: 15 | :maxdepth: 1 16 | 17 | package-homepage-topic-type 18 | module-homepage-topic-type 19 | task-topic-type 20 | config-topic-type 21 | script-topic-type 22 | argparse-script-topic-type 23 | generic-guide-topic-type 24 | 25 | Additional topic types are being developed. 26 | -------------------------------------------------------------------------------- /team/empowerment.rst: -------------------------------------------------------------------------------- 1 | 2 | ############################## 3 | Empowerment of DM team members 4 | ############################## 5 | .. note:: 6 | 7 | Changes to this document must be approved by the DMCCB. 8 | To request changes to this please file an :doc:`RFC ` and flag it. 9 | 10 | 11 | As a DM team member, you are empowered by the DM Project Manager (PM) and DM Subsystem Scientist (SS) to make decisions on any DM-internal matter --- including technical/algorithm issues, process improvements, and tool choices --- provided that **all** of these criteria are met: 12 | 13 | 1. You, or team members who agree with you, are willing and able to do the work to implement the decision, 14 | 15 | 2. You (collectively) are willing and able to fix any problems if it goes wrong. 16 | 17 | 3. You believe that all affected parties (including your immediate manager) would not seriously object to your decision and implementation. 18 | 19 | **If these criteria aren't satisfied,** you can still promote your proposal by creating a :doc:`Request for Comments `. 20 | -------------------------------------------------------------------------------- /cpp/eigen.rst: -------------------------------------------------------------------------------- 1 | ########### 2 | Using Eigen 3 | ########### 4 | 5 | .. note:: 6 | 7 | Changes to this document must be approved by the System Architect (`RFC-24 `_). 8 | To request changes to these policies, please file an :doc:`RFC `. 9 | 10 | 11 | .. _cpp_using_eigen: 12 | 13 | The `Eigen`_ C++ template library for linear algebra is distributed with the LSST software stack. 14 | 15 | It is permitted to use any of the standard modules included with Eigen in LSST code without restrictions beyond the :ref:`usual coding guidelines `. 16 | 17 | Eigen also includes some **unsupported** modules which are located in the ``include/unsupported`` directory. For details, refer to the `relevant Eigen documentation`_. 18 | 19 | These unsupported modules are included for the convenience of other projects which build upon the LSST codebase. Their use in LSST code is currently forbidden. 20 | 21 | .. _Eigen: http://eigen.tuxfamily.org/ 22 | .. _relevant Eigen documentation: http://eigen.tuxfamily.org/dox/unsupported/index.html 23 | -------------------------------------------------------------------------------- /restructuredtext/rst-sample.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ####################### 4 | ReStructuredText Sample 5 | ####################### 6 | 7 | ReStructuredText is an *extensible* markup language used by `LSST`_. 8 | 9 | .. _LSST: http://lsst.org 10 | 11 | ReStructuredText provides basic *italic*, **bold** and ``monospaced`` 12 | typesetting. There is also the concept of **roles** that provide sophisticated 13 | typesetting, such as :math:`\mu = -2.5 \log_{10}(\mathrm{DN} / A) + m_0`, and 14 | :ref:`referencing `. 15 | 16 | .. _label-for-subsection-label: 17 | 18 | Sectioning 19 | ========== 20 | 21 | Sections are formed with underlining the headline text. We use :ref:`a 22 | conventional sequence of underline symbols ` to indicate 23 | different levels of hierarchy. 24 | 25 | Directives 26 | ========== 27 | 28 | Besides **roles** that are used for inline markup, reStructuredText has the 29 | concept of **directives** to markup *blocks* of content. One example is the is 30 | the ``code-block`` directive: 31 | 32 | .. code-block:: python 33 | 34 | print('hello world!') 35 | -------------------------------------------------------------------------------- /cpp/examples/templates/Foo.h: -------------------------------------------------------------------------------- 1 | #if !defined(FOO_H) 2 | #define FOO_H 1 3 | 4 | #include 5 | 6 | template 7 | class Foo { 8 | public: 9 | explicit Foo(); 10 | explicit Foo(T x); 11 | ~Foo(); 12 | 13 | inline T getVal() const; 14 | void setVal(T val); 15 | 16 | private: 17 | T _val; 18 | }; 19 | // 20 | // Inlined functions need to be in the .h file (maybe in the class definition itself) 21 | // 22 | template 23 | inline T Foo::getVal() const { 24 | return _val; 25 | } 26 | // 27 | // Declare a templated function 28 | // 29 | template 30 | size_t mySizeof(void); 31 | // 32 | // Provide a declaration for explicitly-instantiated version[s] of mySizeof() 33 | // This is only needed if we provide enough information here for the compiler 34 | // to instantiate the function/class 35 | // 36 | // It's non-compliant code (but see C++ standards proposal N1897), and stops 37 | // the compiler instantiating the function in each file, and leaving it up 38 | // to the linker to clean up the mess 39 | // 40 | #if 0 41 | extern template size_t mySizeof(void); 42 | #endif 43 | 44 | #endif 45 | -------------------------------------------------------------------------------- /.github/workflows/cron-ci.yaml: -------------------------------------------------------------------------------- 1 | # This workflow runs on a weekly basis and validates the integrity of the 2 | # build and links. 3 | 4 | name: Weekly CI 5 | 6 | on: 7 | schedule: 8 | - cron: "0 4 * * MON" 9 | 10 | jobs: 11 | linkcheck: 12 | 13 | runs-on: ubuntu-latest 14 | 15 | steps: 16 | - uses: actions/checkout@v3 17 | 18 | - name: Set up Python 19 | uses: actions/setup-python@v4 20 | with: 21 | python-version: "3.10" 22 | 23 | - name: Install Python dependencies 24 | run: | 25 | python -m pip install --upgrade pip setuptools 26 | pip install -r requirements.txt 27 | 28 | - name: Run linkcheck 29 | run: make linkcheck 30 | 31 | build: 32 | 33 | runs-on: ubuntu-latest 34 | 35 | steps: 36 | - uses: actions/checkout@v3 37 | 38 | - name: Set up Python 39 | uses: actions/setup-python@v4 40 | with: 41 | python-version: "3.10" 42 | 43 | - name: Install Python dependencies 44 | run: | 45 | python -m pip install --upgrade pip setuptools 46 | pip install -r requirements.txt 47 | 48 | - name: Run HTML build 49 | run: make html 50 | -------------------------------------------------------------------------------- /cpp/compilation-db.rst: -------------------------------------------------------------------------------- 1 | ####################################### 2 | Creating a compilation database 3 | ####################################### 4 | 5 | ``scons`` does not natively support the creation of a compilation database, instead the 6 | ``bear`` tool can be used (`bear repository `_) 7 | which is available on a number of Linux distributions (`bear packages `_) as well as 8 | from conda-forge. A compilation database is a JSON file containing data on compiling each source code file in a project. 9 | 10 | ==================================== 11 | Install Bear from conda-forge 12 | ==================================== 13 | To install ``bear`` into an LSST Science Pipelines conda stack 14 | 15 | .. code-block:: bash 16 | 17 | conda install -c conda-forge bear 18 | 19 | ===================================================== 20 | Use Bear and Scons to create a compilation database 21 | ===================================================== 22 | 23 | .. code-block:: bash 24 | 25 | bear -- scons 26 | 27 | This will create ``compile_commands.json`` containing a list of all compilation commands that can be used in editors like :ref:`VSCode ` to analyze and navigate the code. 28 | -------------------------------------------------------------------------------- /team/meeting-free-weeks.rst: -------------------------------------------------------------------------------- 1 | 2 | ################## 3 | Meeting Free Weeks 4 | ################## 5 | 6 | DM will observe several **Meeting Free Weeks** each year. 7 | 8 | 9 | Meeting Free Weeks 2025 10 | ======================= 11 | In 2025 meeting free weeks will be: 12 | - April 14 - 18 13 | - June 23 - 27 (EAS) 14 | - Sept 15 - 19 15 | - Dec 22 - Jan 2. 16 | 17 | An exceptional meeting free week has been added 2025: 18 | - Aug 18 - 22 19 | 20 | Guidelines 21 | ========== 22 | The driver for meeting free weeks is similar to :doc:`focus-friday`. 23 | We wish to allow staff a block of time without meetings such they can focus on a specific task or take personal time. 24 | These periods do not stipulate the more stringent Slack rules that :doc:`focus-friday` does. 25 | Basically in a meeting free week there will be none of the regular meetings. Topic specific meetings may be held as agreed by all parties. 26 | 27 | Feedback 28 | ======== 29 | A long focus period over the end of year holidays did not go so well and two weeks was preferred. In 2023 we should include a March focus week. 30 | There may be more refinements needed to get this correct. 31 | Feedback on the form and format is welcome and may be sent to your T/CAM or 32 | directly to William O'Mullane. 33 | 34 | 35 | -------------------------------------------------------------------------------- /jenkins/jenkins-cloudy.svg: -------------------------------------------------------------------------------- 1 | Cloudy -------------------------------------------------------------------------------- /communications/calendars.rst: -------------------------------------------------------------------------------- 1 | .. _calendars: 2 | 3 | ############ 4 | DM Calendars 5 | ############ 6 | 7 | DM maintains two Google calendars, described below. 8 | The wider LSST Project has collated `a list of other calendars`_ which may also be useful. 9 | 10 | .. _a list of other calendars: https://project.lsst.org/node/1 11 | 12 | DM Meetings 13 | =========== 14 | 15 | The `DM Meetings`_ calendar lists meetings or other events that are likely to be of wide interest across the DM subsystem. 16 | It's intended for meetings which will be of interest to people from more than one institute: if you're organizing a meeting which is only relevant to your local team, it probably doesn't belong on this calendar. 17 | If you're organizing an event which you'd like to add to the DM Meetings calendar, please drop by the `dm-admin-support`_ channel on Slack and somebody will help you out. 18 | 19 | .. _DM Meetings: https://calendar.google.com/calendar/embed?src=pft8isslcqcll4jao0rqdmphvg%40group.calendar.google.com 20 | .. _dm-admin-support: https://lsstc.slack.com/archives/dm-admin-support 21 | 22 | DM Travel / Vacation 23 | ==================== 24 | 25 | DM Leadership Team members are asked to note any days when they won't be available on the `DM Travel / Vacation`_ calendar. 26 | Other members of DM can safely ignore this calendar — but check with your local manager to see if there's some other mechanism by which they'd like you to make your availability known. 27 | 28 | .. _DM Travel / Vacation: https://calendar.google.com/calendar/embed?src=a1es0hbupn2n8ol9mafa4t8bj8%40group.calendar.google.com 29 | -------------------------------------------------------------------------------- /user-docs/lsst-specific-content-style-guide.rst: -------------------------------------------------------------------------------- 1 | .. _user-doc-style-lsst: 2 | 3 | ######################### 4 | LSST-specific style guide 5 | ######################### 6 | 7 | The :doc:`LSST user documentation style guide ` is built upon the `Google Developer Style Guide`_. 8 | This page lists our exceptions and modifications to those guidelines. 9 | 10 | Use the imperative mood for Python and C++ function and method summary sentences 11 | -------------------------------------------------------------------------------- 12 | 13 | For LSST DM Python and C++, you should use the **imperative mood** to write the summary sentence: 14 | 15 | Get the value. 16 | 17 | The imperative mood is conventional in scientific Python software (Numpy, SciPy, and Astropy, among others). 18 | In turn, our C++ standard follows the Python convention since our use of the two languages is often intertwined. 19 | See also: 20 | 21 | - :ref:`Python summary sentence standard ` 22 | - :ref:`C++ summary sentence standard ` 23 | 24 | .. note:: 25 | 26 | This recommendation for LSST DM differs from the Google Developer Style Guide, which recommends `using the present tense for function and method summaries `__. 27 | For example: 28 | 29 | Gets the value. 30 | 31 | If you are documenting a completely different technology, such as HTTP API endpoints, using the present tense is a good idea. 32 | 33 | .. _`Google Developer Style Guide`: https://developers.google.com/style/ 34 | -------------------------------------------------------------------------------- /project-docs/test-documentation.rst: -------------------------------------------------------------------------------- 1 | ################## 2 | Test Documentation 3 | ################## 4 | 5 | Verification and Validation activities require a huge amount of work in terms of documentation. 6 | Often this documentation work is given a lower priority, enabled by reduced visibility of V&V activities. 7 | 8 | The introduction of the Test Manager For Jira plugin (TM4J) helps the test team to overcome this problem. 9 | This plugin, combined with the programmatic extraction of test data using the docsteady document generation tool, brings additional benefits: 10 | 11 | - Test information becomes much more accessible 12 | - Traceability is established between test objects and with upstream requirements 13 | - Reusability of test procedures is facilitated 14 | - The test teams can focus more on the scientific aspects of the tests 15 | - Homogeneity in the approach is ensured 16 | 17 | However, in order to make this possible, it is important to follow a well-organized approach to the V&V activities. 18 | 19 | You can get a global overview of the process in ths `SPIE paper`_ presented in December 2020. 20 | The `DM Test Approach`_ Confluence page provides more practical information including details on how to create each TM4J object, how they are related to each other, and how to fill in the right information. 21 | Detailed procedures for document generation are given in the `Docsteady Usecases`_ technote. 22 | 23 | .. _Docsteady Usecases: https://dmtn-178.lsst.io 24 | 25 | .. _SPIE paper: https://dmtn-140.lsst.io 26 | .. _DM Test Approach: https://confluence.lsstcorp.org/display/DM/DM+Test+Approach 27 | -------------------------------------------------------------------------------- /communications/wiki.rst: -------------------------------------------------------------------------------- 1 | ########################### 2 | Policy on DM Confluence Use 3 | ########################### 4 | 5 | While we want to foster open source development and a wiki is central to this purpose, there are certain caveats on what should and shouldn't be posted to the `Data Management space`_ on `Confluence`_ since significant material is open to the public. 6 | All Confluence users should ensure the following: 7 | 8 | - Material posted to the wiki must not be confidential or contain material that could be embarrassing or used against the project. 9 | This might include, for example, partnering plans or proposals that are confidential or have not yet been realized, financial information, positioning relative to other projects, or comments about strengths or weaknesses relative to other projects. 10 | 11 | - Change-controlled documents must be written and approved following our :ref:`documented procedures `, which do not include working on Confluence. 12 | However, Confluence may be useful for drafting or brainstorming in the early stages of document development. 13 | In this case, the page must be clearly labeled as a draft, and it should be augmented with a link to the baselined version on :ref:`Docushare ` when it becomes available. 14 | 15 | The DM Project Manager is responsible for oversight ensuring that the DM wiki only has appropriate material. 16 | The entire DM team is expected to help with this oversight; if you see something that is suspect, please bring it to the attention of the DM Project Manager. 17 | 18 | .. _Data Management space: https://confluence.lsstcorp.org/display/DM/Data+Management+Home 19 | .. _Confluence: https://confluence.lsstcorp.org/ 20 | -------------------------------------------------------------------------------- /team/code-of-conduct.rst: -------------------------------------------------------------------------------- 1 | ################################## 2 | Team Culture and Conduct Standards 3 | ################################## 4 | 5 | The LSST DM community includes LSST-paid staff at multiple institutions and external contributors from around the world. 6 | We have built a strong team that collaborates well. 7 | As we hire new people, we want to make sure we are maintaining a healthy, supportive, productive culture. 8 | While culture is best transmitted by daily example, having some formal standards for conduct can aid newcomers and reinforce good patterns. 9 | 10 | Since LSST is an AURA center, the `AURA standards of workplace conduct provide a starting point `_. 11 | As stated there, we in DM dedicate ourselves to fostering a civil and inclusive community characterized by mutual respect for the contributions of all individuals. 12 | As a community, we embrace the values in that document, in addition to any local institutional standards. 13 | 14 | Our community is self-regulated, but any issues that cannot be resolved on an individual-to-individual level should be handled through local institutional procedures or---especially if cross-institution or involving the external community---may be brought to the attention of the DM Subsystem Scientist (Leanne Guy) or Project Manager (William O'Mullane), or the LSST Project Manager (Victor Krabbendam), who will take appropriate action. Independent of this management hierarchy there is an `Ombudsperson Program `_ which may be used to raise issues. 15 | 16 | 17 | 18 | Please read and reflect upon these standards; strive to uphold them in all your interactions. 19 | -------------------------------------------------------------------------------- /restructuredtext/section-sample.rst: -------------------------------------------------------------------------------- 1 | :orphan: 2 | 3 | ########## 4 | Page Title 5 | ########## 6 | 7 | Titles have hash marks above and below. 8 | 9 | By convention, titles and section headings are set off from surrounding text by 10 | a single blank line above and below. All levels of section headings may have 11 | named labels, which appear before the heading. We encourage you to add labels 12 | to all sections so that they can be referenced. Names are global, so be 13 | specific. See :ref:`Internal Links to Labels ` for 14 | more information. The following section heading has a label named 15 | "``section-headings-example-section``". 16 | 17 | .. _section-headings-example-section: 18 | 19 | Section Heading 20 | =============== 21 | 22 | Section headers are set with an underline. The sequence of underline characters 23 | used (``=``, then ``-``, then ``^`` and finally ``"``) indicates the section 24 | hierarchy. 25 | 26 | .. _section-headings-example-subsection: 27 | 28 | Subsection Heading 29 | ------------------ 30 | 31 | Maecenas congue ligula ac quam viverra nec consectetur ante hendrerit. 32 | Donec et mollis dolor. 33 | 34 | .. _section-headings-example-subsubsection: 35 | 36 | Sub-subsection Heading 37 | ^^^^^^^^^^^^^^^^^^^^^^ 38 | 39 | Praesent et diam eget libero egestas mattis sit amet vitae augue. 40 | 41 | .. _section-headings-example-subsubsubsection: 42 | 43 | Sub-sub-subsection Heading 44 | """""""""""""""""""""""""" 45 | 46 | Nam tincidunt congue enim, ut porta lorem lacinia consectetur. 47 | 48 | 49 | Sections in Python docstrings are a special case. We do not put a blank space 50 | between a headline and object lists below, and we do not add explicit section 51 | labels. See the :doc:`docstring style guide <../python/numpydoc>` for more information. 52 | -------------------------------------------------------------------------------- /communications/community-support.rst: -------------------------------------------------------------------------------- 1 | ################################## 2 | Providing Support to the Community 3 | ################################## 4 | 5 | Given the high-profile nature of Rubin Observatory within the astronomical community, and our :doc:`policy ` of making all of our software available to the public under open-source licenses, we expect substantial interest from the community in general, and from the `Science Collaborations `_ in particular, in using and understanding both our code and our data products. 6 | As they do so, they will undoubtedly have questions and require technical support or algorithmic help. 7 | When we enter operations, the project's Community Engagement Team will coordinate a system for helping people find the answers they need, as described in `RTN-006 `_. 8 | However, during construction, members of the Data Management team have the most expertise and experience, so it is natural that the community will turn to us for help. 9 | It is important to manage these interactions in a way that: 10 | 11 | - Makes information available to the greatest possible number of consumers; 12 | - Avoids overwhelming our limited development resources with community requests; 13 | - Facilitates transition to the operations-era community support model. 14 | 15 | To this end, you are requested to follow the policy described in `DMTN-155 `_ when contacted by members of the community looking for support. 16 | That document provides a detailed policy and accompanying rationale; to summarize, the aim is to redirect all requests for help to the `Community Forum `_ where they can be properly addressed, rather than having members of the DM team directly answer questions on Slack or by email. 17 | -------------------------------------------------------------------------------- /jenkins/jenkins-raining.svg: -------------------------------------------------------------------------------- 1 | Raining -------------------------------------------------------------------------------- /jenkins/jenkins-storm.svg: -------------------------------------------------------------------------------- 1 | Storm -------------------------------------------------------------------------------- /python/cli.rst: -------------------------------------------------------------------------------- 1 | ############################## 2 | Python Command Line Interfaces 3 | ############################## 4 | 5 | Command line interfaces (CLI) in Python may be implemented using the `argparse`_ module or using `Click`_. 6 | 7 | .. _argparse: https://docs.python.org/3/library/argparse.html 8 | 9 | .. _Click: https://click.palletsprojects.com 10 | 11 | 12 | .. _cli-using-click: 13 | 14 | Command Line Interfaces using Click 15 | =================================== 16 | 17 | Click is a Python package for creating command line interfaces in a composable way. 18 | It uses a declarative syntax that makes it easy to create CLI commands and easy and to understand them later. 19 | However to use Click, the Click package must be installed. 20 | 21 | Read more in the `Click quickstart`_ guide and `Click documentation`_. 22 | 23 | .. _Click documentation: https://click.palletsprojects.com/en/7.x/#documentation 24 | .. _Click quickstart: https://click.palletsprojects.com/en/7.x/quickstart/ 25 | 26 | Command Line Interfaces using argparse 27 | ====================================== 28 | 29 | Argparse is included in the python standard library and so does not require any additional packages. 30 | It uses an imparative syntax. 31 | 32 | Read more in the `argparse documentation`_. 33 | 34 | .. _argparse documentation: https://docs.python.org/3/library/argparse.html 35 | 36 | Add a Callable CLI Command To Your Package 37 | ========================================== 38 | 39 | To create a callable command at the top level of your package create a folder called ``bin.src``. 40 | It should contain two files: 41 | 42 | 1. ``SConscript`` with contents: 43 | 44 | .. code-block:: py 45 | 46 | # -*- python -*- 47 | from lsst.sconsUtils import scripts 48 | scripts.BasicSConscript.shebang() 49 | 50 | 2. A file that has the name of the CLI command the user will call. 51 | This file should contain as little implementation as possible. 52 | The ideal simplest case is to import an implementation function and call it. 53 | This makes the implementation testable and reusable. 54 | 55 | -------------------------------------------------------------------------------- /cpp/testing-private-functions.rst: -------------------------------------------------------------------------------- 1 | ################################## 2 | Unit-Testing Private C++ Functions 3 | ################################## 4 | 5 | .. note:: This material is still in draft. 6 | 7 | .. warning:: 8 | There exists significant debate about whether or not private functions should 9 | ever be unit-tested. In most situations, it should be preferable to test the 10 | public functions of the class, and to use coverage analysis to make sure that 11 | private functions are tested as well. 12 | 13 | However, there may exist circumstances in which testing and debugging is 14 | considerably simplified by testing a private class. For example, the private 15 | functions may be too complex to easily generate thorough coverage without 16 | calling them directly. In these cases, you may go ahead and directly test a 17 | private function. 18 | 19 | The Problem with Boost Unit Testing Macros 20 | ========================================== 21 | 22 | Boost's unit testing macros create test classes at compile-time. The names of 23 | these classes are not known to the programmer, and thus it is not possible to 24 | simply declare them as "friend" to the tested class. 25 | 26 | Some quick searching on Google will reveal some dubious ideas, such as include 27 | guards which re-declare the private functions as public if a "testing" flag is 28 | set. This will mean that code compiled for production will not pass tests. 29 | This is not an optimal solution. 30 | 31 | How to do it, if you must 32 | ========================= 33 | 34 | It is suggested to create an additional "bridge" class which is declared 35 | ``friend`` to the tested class. This "bridge" class can then call the private 36 | functions of the tested class, and the unit tests should call an instance of 37 | the "bridge" class. 38 | 39 | The "bridge" class should live in its own namespace ending in ``::impl``. For 40 | example, to test a class in ``lsst::mops``, put a bridge class in 41 | ``lsst::mops::impl``. The code inside an ``::impl`` namespace should never be 42 | used outside of testing, and it should be generally understood that the 43 | contents of an ``::impl`` namespace are highly unstable. 44 | -------------------------------------------------------------------------------- /jenkins/jenkins-partially-sunny.svg: -------------------------------------------------------------------------------- 1 | Partially Sunny -------------------------------------------------------------------------------- /work/jira-tips.rst: -------------------------------------------------------------------------------- 1 | ############################ 2 | JIRA Work Management Recipes 3 | ############################ 4 | 5 | JIRA is a powerful web app that you can customize to suit your needs. 6 | This page mentions a few ways to improve your workflow, though the `JIRA User Guide `_ is the definitive resource for power users. 7 | 8 | LSST's JIRA is available at https://jira.lsstcorp.org. 9 | 10 | *See also:* :ref:`workflow-jira` in the :doc:`flow`. 11 | 12 | .. _jira-create-filters: 13 | 14 | Creating filters 15 | ================ 16 | 17 | Filters are a great way to maintain dynamic lists of issues for different workflows. 18 | You can make an issue filter by saving a JIRA search: 19 | 20 | 1. Open the `Issues page in JIRA `_. 21 | 2. Enter a search term. 22 | `You can learn about JIRA's syntax from the docs `_. 23 | 3. Run the search. 24 | 4. Press the **Save as** button on the search results page. 25 | 26 | Your saved filters are accessible from the **Issues** dropdown menu in JIRA, among other places. 27 | 28 | You can manage your filters from https://jira.lsstcorp.org/secure/ManageFilters.jspa. 29 | 30 | .. _jira-search-examples: 31 | 32 | Example search terms 33 | ==================== 34 | 35 | All of your open issues:: 36 | 37 | assignee = currentUser() AND resolution = Unresolved ORDER BY updatedDate DESC 38 | 39 | All of your issues that are in progress:: 40 | 41 | status in ("In Progress", "In Review", Acknowledged, Reviewed, "Code Review", Blocked, "Awaiting Signoff") AND resolution = Unresolved AND assignee = currentUser() ORDER BY updatedDate DESC 42 | 43 | All open issues you have been assigned to review:: 44 | 45 | Reviewers in (currentUser()) AND status = "In Review" 46 | 47 | All issues closed in a given month:: 48 | 49 | status changed to Done DURING ("2015/12/01", "2015/12/31") 50 | 51 | Issues where you have been mentioned in the last two weeks:: 52 | 53 | text ~ currentUser() AND updatedDate >= -14d ORDER BY updated DESC 54 | 55 | Issues that match a label:: 56 | 57 | labels = label-name 58 | 59 | Issues that are blocked by other issue:: 60 | 61 | issueFunction in hasLinks("is blocked by") 62 | -------------------------------------------------------------------------------- /cpp/profiling.rst: -------------------------------------------------------------------------------- 1 | ######################### 2 | C++ performance profiling 3 | ######################### 4 | 5 | It is important to generate a profile of code performance to understand where to focus optimization efforts. 6 | 7 | igprof 8 | ====== 9 | 10 | Profiling C++ code can be done with `igprof `_: 11 | 12 | .. code-block:: bash 13 | 14 | igprof -pp -z -o igprof-mosaic.pp.gz python `which mosaic.py` /data3a/work/price/SUPA-MIT/rerun/cosmos --id field=COSMOS filter=W-S-I+ expTime=120.0 --clobber-config 15 | igprof-analyse -d -v -g igprof-mosaic.pp.gz > igprof-mosaic.pp.txt 16 | 17 | That provides the cumulative profile (top) and then the caller/callee profiles further down (see http://igprof.org/text-output-format.html). 18 | There is a fancy ``cgi-bin`` setup for browsing the profiles, but it requires setting up your Apache server. 19 | This may or may not be worth the trouble. 20 | 21 | Note that there is a `bug in igprof `_ (or its dependency, libunwind) that sometimes causes the process to hang. 22 | The recommended workaround is "to make sure you have a hot cache for your libraries (``cat *.so >/dev/null``)". 23 | A slightly more complete command is 24 | 25 | .. code-block:: bash 26 | 27 | (export IFS=:; while true ; do for DIR in $LD_LIBRARY_PATH ; do find $DIR -name "*.so" -exec cat {} > /dev/null \; ; done; sleep 5; done) & 28 | 29 | sprof 30 | ===== 31 | 32 | sprof is part of glibc, so should be available on most Linux systems. 33 | Unlike its cousin, gprof, it does not require recompilation and it works on shared libraries, so can be used with your current stack setup, whatever that may be. 34 | Unfortunately, it allows profiling only one shared library at a time, but generally the shared library of interest can be identified using Python profiling. 35 | Here's an example using sprof to profile the CModel code in meas_modelfit (which is exercised by ``measureCoaddSources.py``): 36 | 37 | .. code-block:: bash 38 | 39 | export LD_PROFILE=libmeas_modelfit.so 40 | export LD_PROFILE_OUTPUT=`pwd` 41 | measureCoaddSources.py /scratch/pprice/ci_hsc/DATA --rerun ci_hsc --id patch=5,4 tract=0 filter=HSC-I 42 | sprof -p -q libmeas_modelfit.so libmeas_modelfit.so.profile > libmeas_modelfit.so.profile.txt 43 | 44 | The output of sprof contains a cumulative profile at the top, followed by the caller/callee profiles. 45 | -------------------------------------------------------------------------------- /team/focus-friday.rst: -------------------------------------------------------------------------------- 1 | 2 | ############ 3 | Focus Friday 4 | ############ 5 | 6 | DM will observe **Focus Friday**. 7 | 8 | Guidelines 9 | ========== 10 | The driver for the focus day is to allow staff a block of time without interruptions or the need to be online and receptive. 11 | By reducing traffic we will reduce the need to participate in threads on specific discussion topics. 12 | Hence there should be: 13 | 14 | - No planned meetings, generally. There may of course be exceptions, for example if your focused work includes a co-work with others who agree to meet, this is OK. Such agreements should be between peers and not use any hierarchy. You may elect to attend meetings/workshops outside DM — or occasionally we may be required to attend such meetings. 15 | - Slack messages on public channels are discouraged. Conversations are discouraged on general channels and specific users should not be @ed. You may Slack an individual, or a group of individuals if you have agreed such interactions ahead of time. There should be no expectation of response to Slack messages. 16 | 17 | Asynchronous communication is perfectly acceptable on the understanding that no response is to be expected before the next work day (normally Monday). 18 | Hence it is OK to, for example: 19 | 20 | - Send email, though you may consider using a delayed send. 21 | - Update Jira issues, which will result in email notifications that are OK. 22 | - Update confluence, which will also result in email notifications that are OK. 23 | - Perform GitHub work. Code reviews, merges etc. which will result in notifications are OK. 24 | 25 | It is acceptable to set up an out-of-office reply pointing to this page on Focus Friday. 26 | 27 | Emergencies and on call 28 | ======================= 29 | There may on occasion be an emergency, urgent bug, machine failure, fire, flooding etc. 30 | In this case it is OK to make Friday noisy. 31 | This may include a post to public Slack support channel. 32 | As we approach operations there may also be agreed people on call who may need to 33 | respond to communications even on a focus day. 34 | 35 | Feedback 36 | ======== 37 | Though we did one experimental day and changed the rules according to feedback we 38 | assume there may be more refinements needed to get this correct. 39 | This concept has been reassessed and refined in several 40 | DMLT Face to Face meetings since Nov 2020. 41 | Feedback on the form and format of the day is welcome and may be sent to your T/CAM or directly to William O'Mullane. 42 | 43 | 44 | -------------------------------------------------------------------------------- /stack/platforms.rst: -------------------------------------------------------------------------------- 1 | ################### 2 | Supported Platforms 3 | ################### 4 | 5 | Introduction 6 | ============ 7 | 8 | The Science Pipelines are required to be portable across many compute platforms ranging from high-performance computing centers to laptops (refer to DMS-REQ-0308 in `lse-61`_). 9 | The software will likely work on any Unix-like systems but we limit our testing and support to a fixed set of platforms. 10 | Patches may be accepted from developers who test on other operating systems. 11 | 12 | .. _platforms-baseline: 13 | 14 | Platforms 15 | ========= 16 | 17 | We have a baseline platform which is what we are using for integration testing and deployments. 18 | For each product, we also have a number of other platforms which are regularly tested by our continuous integration system to enhance portability. 19 | 20 | The baseline platform is currently AlmaLinux 9 on Intel x86_64. 21 | 22 | .. note :: 23 | We aim to stay current with AlmaLinux minor releases. 24 | Our base OS container will typically be updated to the current AlmaLinux release shortly *after* each major release of the Science Pipelines. 25 | Shared development machines will be updated on their own schedule during periodic maintenance downtime. 26 | 27 | We regularly test the following platforms in addition to the baseline platform: 28 | 29 | * macOS Sonoma (14.x) on Apple Silicon; 30 | * AlmaLinux 9 on ARM (aarch64). 31 | 32 | See https://rubin-ci.slac.stanford.edu/blue for the current list of regularly-tested platforms. 33 | 34 | .. _platforms-environment: 35 | 36 | Environment 37 | =========== 38 | 39 | On each of these platforms, we rely on `Conda`_ and `conda-forge`_ to provide our build- and run-time dependencies, including compilers and the Python interpreter. 40 | The Conda environment for Science Pipelines is generated by :doc:`the rubin-env metapackage `. 41 | This defines the environment within which our code must execute: all code must be compatible with the current version of ``rubin-env``. 42 | Compatibility with different versions is determined by the ``rubin-env`` semantic version number. 43 | The Conda environment is periodically updated to maintain compatibility, to utilize new features of third-party packages, or to add new dependencies. 44 | To request that a new package be added to the environment, please :doc:`file an RFC `. 45 | 46 | .. _Conda: https://conda.io 47 | .. _conda-forge: https://conda-forge.org/ 48 | .. _scipipe_conda_env: https://github.com/lsst/scipipe_conda_env 49 | .. _lse-61: https://lse-61.lsst.io/ 50 | -------------------------------------------------------------------------------- /python/astropy.rst: -------------------------------------------------------------------------------- 1 | ############# 2 | Using Astropy 3 | ############# 4 | 5 | .. note:: 6 | 7 | Changes to this document must be approved by the System Architect (`RFC-24 `_). 8 | To request changes to these policies, please file an :doc:`RFC `. 9 | 10 | .. _cpp_using_astropy: 11 | 12 | Integration of Astropy core into the LSST software stack is an ongoing process that will evolve over time as we work on enhancing interoperability of the current codebase with Astropy. 13 | This document is not discussing Astropy affiliated packages, use of which must go through the standard :doc:`RFC ` process. 14 | Not all Astropy core packages can be used by default. 15 | 16 | The following packages can be used internally in packages: 17 | 18 | * :mod:`astropy.coordinates` 19 | * :mod:`astropy.time` 20 | * :mod:`astropy.table` 21 | * :mod:`astropy.units` 22 | * :mod:`astropy.constants` 23 | * :mod:`astropy.cosmology` 24 | * :mod:`astropy.visualization` 25 | 26 | For reading and writing files in FITS format, both :mod:`astropy.io.fits` and `fitsio`_ are allowed. 27 | These libraries differ in the features available and in performance (with the latter usually surpassing the former). 28 | Developers must evaluate which works best for their use case and choose one accordingly. 29 | 30 | 31 | .. _fitsio: https://github.com/esheldon/fitsio 32 | 33 | .. warning:: 34 | 35 | ``lsst.afw.fits`` must not be used in any new Python code, as it is not considered memory safe. 36 | 37 | If not interacting with C++ classes the use of ``astropy.table`` is encouraged. 38 | In particular, the Table class has beneficial properties over Pandas DataFrame. 39 | The interaction of Astropy with LSST C++ classes providing related functionality should be carefully monitored. 40 | If the code is already using ``afw`` it is strongly preferred that ``afw`` equivalents be used until such time as specific ``afw`` interfaces are deprecated. 41 | :mod:`astropy.table` views into ``afw.table`` tables can be used if required. 42 | 43 | Changing any public APIs requires an :doc:`RFC `. 44 | Hence using Astropy in any public API must go through an RFC. 45 | 46 | These items have functionality that is similar to that provided in LSST packages: 47 | 48 | * :mod:`astropy.modeling` 49 | * :mod:`astropy.convolution` 50 | * :mod:`astropy.wcs` 51 | * :mod:`astropy.stats` 52 | * :mod:`astropy.nddata` 53 | 54 | They are not allowed to be used in LSST code without special permission, such as submitting an :doc:`RFC `. 55 | 56 | This advice will evolve as interoperability with Astropy develops. 57 | -------------------------------------------------------------------------------- /communications/rfd.rst: -------------------------------------------------------------------------------- 1 | .. See RFC-53: https://jira.lsstcorp.org/browse/RFC-53 2 | 3 | ############################ 4 | Request for discussion (RFD) 5 | ############################ 6 | 7 | By creating a **Request for Discussion (RFD)**, you can schedule an in-depth technical discussion across DM. 8 | The types of discussions you might schedule include: 9 | 10 | - Detailed design discussions for a component of the system or its interfaces. 11 | - Design reviews for new code or refactorings of old code. 12 | - Brainstorming methods to solve difficult problems. 13 | - "Brain dump" explanations of a design to share knowledge across DM. 14 | 15 | .. seealso:: 16 | 17 | :doc:`rfc` is an additional process you should use to seek adoption of a specific proposal. 18 | 19 | .. _rfd-creating: 20 | 21 | Creating an RFD 22 | =============== 23 | 24 | Create an RFD by `making a post`_ on Community in the `DM RFD category`_. 25 | A `template is provided in the post `_ to allow you to suggest the date and time of the discussion (:ref:`see below for time slot `), the medium for the discussion, and to specify the suggested audience. 26 | In the RFD's description: 27 | 28 | - Summarize the issue, and indicate a desired outcome from the discussions. 29 | - Include background material (using attachments, if necessary). 30 | - Provide a link to the BlueJeans room. 31 | 32 | Creating an RFD post will trigger postings to the `dm-devel mailing list`_ and the `'#dm' Slack channel`_. 33 | 34 | As the discussion organizer, you are responsible for ensuring all required attendees are available for the time slot. 35 | DM members can comment on the RFD to indicate their availability, or whether the subject being discussed has already been resolved or covered elsewhere. 36 | 37 | .. _`making a post`: https://community.lsst.org/new-topic?category=data%20management/dm%20rfd 38 | .. _`DM RFD category`: https://community.lsst.org/c/dm/dm-rfd 39 | .. _`dm-devel mailing list`: https://lists.lsst.org/mailman/listinfo/dm-devel 40 | .. _`'#dm' Slack channel`: https://lsstc.slack.com/messages/dm/ 41 | 42 | .. _rfd-time: 43 | 44 | The RFD time slot 45 | ================= 46 | 47 | RFDs can be scheduled for any convenient time, but we do have a weekly reserved time slot on Tuesdays from 12:30 to 2 PM Pacific. 48 | 49 | If there are no requests 24 hours before a given time slot, the meeting will be canceled and the time freed up for other activities. 50 | 51 | If there are conflicting claims to the RFD time slot, the System Architect will arbitrate. 52 | 53 | .. _rfd-followup: 54 | 55 | RFD followup 56 | ============ 57 | 58 | Tickets or RFCs that arise from an RFD should link back to the RFD post on community.lsst.org and the Community post can be used to continue the discussion following the meeting. 59 | -------------------------------------------------------------------------------- /stack/debug.rst: -------------------------------------------------------------------------------- 1 | ################################## 2 | Debugging Tasks with ``lsstDebug`` 3 | ################################## 4 | 5 | .. note:: 6 | 7 | See also the `Pipelines documentation on lsstDebug `_. 8 | 9 | Task Authors 10 | ============ 11 | 12 | In your ``Task`` code, if you ``import lsstDebug`` then call:: 13 | 14 | debug = lsstDebug.Info(__name__) 15 | 16 | (with ``__name__`` being the name of the current ``Task``), you are given a “debug object”. 17 | By default that will simply give you a ``False`` for any attribute you access. 18 | Thus:: 19 | 20 | >>> debug.foo 21 | False 22 | 23 | >>> debug.bar 24 | False 25 | 26 | >>> debug.display 27 | False 28 | 29 | The task user will customize the behaviour of ``debug`` to meet his/her particular needs by redefining the ``Info`` function to return an object that returns non-``False`` values for certain combinations of ``__name__`` and attribute. 30 | With this in place, the code above produces different results:: 31 | 32 | >>> debug = lsstDebug.Info(__name__) # __name__ selects the current task 33 | >>> debug.display 34 | True 35 | 36 | Then you can write your task to optionally enable a display (or whatever) by doing something like:: 37 | 38 | if debug.display: 39 | afwDisplay.getDisplay().... 40 | else: 41 | self.log.debug("I would show you a pretty picture here if you enabled debugging.") 42 | 43 | Task Users 44 | ========== 45 | 46 | Refer to the `task documentation `_ for your task of interest, and look for a "Debugging" section to discover what debugging options are available for existing ``Task``\s. 47 | We also provide further details about the `lsstDebug module `_ in the pipeline docs. 48 | 49 | To use ``lsstDebug``, you need to customize the behaviour of ``debug`` to meet your particular needs by redefining the ``lsstDebug.Info`` function to return an object that returns non-``False`` values for certain combinations of ``__name__`` and attribute. 50 | 51 | In order to load your specific debugging configuration, create a ``debug.py`` in a directory that is already in your ``PYTHONPATH`` or a new directory that you add to your ``PYTHONPATH``, and put something like this in it:: 52 | 53 | import lsstDebug 54 | def DebugInfo(name): 55 | debug = lsstDebug.getInfo(name) 56 | if name == "lsst.meas.astrom.astrometry": 57 | debug.display = True 58 | return debug 59 | 60 | lsstDebug.Info = DebugInfo 61 | 62 | That should enable debugging the ``display`` attribute when you are running inside ``lsst.meas.astrom.astrometry``, and disable it elsewhere. 63 | Of course, you can also return arbitrarily more complex objects, doing things like specifying the frame to display on etc. 64 | 65 | You must use the ``--debug`` command line argument to ask a pipetask to import your :file:`debug.py` file. 66 | -------------------------------------------------------------------------------- /cpp/clang-format.rst: -------------------------------------------------------------------------------- 1 | .. _using_clang_format: 2 | 3 | ####################################### 4 | Using clang-format for LSST Development 5 | ####################################### 6 | 7 | The easiest way to comply with the :ref:`LSST code layout rules ` is to use `clang-format `_. 8 | We require clang-format version 5.0, for formatting consistency (earlier versions may format some lines subtly differently). 9 | 10 | Linux 11 | ===== 12 | 13 | Ubuntu users can install this tool through the Ubuntu package manager. 14 | Version 5.0 is not available in the default Xenial or Zesty repositories. 15 | You can get it from the `llvm apt repository `_, using the 5.0 repository (qualification branch). 16 | If you've already installed ``clang-format-3.8`` (Ubuntu's default version), uninstall it before proceeding. 17 | Follow these commands to add the appropriate apt source line for your Ubuntu version and install it using either the Xenial (16.04): 18 | 19 | .. code-block:: bash 20 | 21 | echo "deb http://apt.llvm.org/xenial/ llvm-toolchain-xenial-5.0 main" | sudo tee /etc/apt/sources.list.d/llvm.list 22 | wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key|sudo apt-key add - 23 | sudo apt update 24 | sudo apt install clang-format-5.0 25 | 26 | or Zesty (17.04) commands: 27 | 28 | .. code-block:: bash 29 | 30 | echo "deb http://apt.llvm.org/zesty/ llvm-toolchain-zesty-5.0 main" | sudo tee /etc/apt/sources.list.d/llvm.list 31 | wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key|sudo apt-key add - 32 | sudo apt update 33 | sudo apt install clang-format-5.0 34 | 35 | This will install the ``clang-format-5.0`` binary, but will not symlink it to ``clang-format``. 36 | To create a system-managed symlink ("alternative"), use the following command: 37 | 38 | .. code-block:: bash 39 | 40 | sudo update-alternatives --install /usr/bin/clang-format clang-format /usr/bin/clang-format-5.0 1 41 | 42 | macOS 43 | ===== 44 | 45 | On macOS, install clang-format through `Homebrew `_: 46 | 47 | .. code-block:: bash 48 | 49 | brew install clang-format 50 | 51 | For more platforms see `here `_. 52 | 53 | General 54 | ======= 55 | 56 | Place the following configuration file, named ``.clang-format``, in your top-level source directory. For example, if you keep your lsst-related code in ``~/lsst/``, put the file in that directory. 57 | 58 | .. code-block:: yaml 59 | 60 | --- 61 | Language: Cpp 62 | BasedOnStyle: Google 63 | ColumnLimit: 110 64 | IndentWidth: 4 65 | AccessModifierOffset: -4 66 | SortIncludes: false # reordering may break existing code 67 | ConstructorInitializerIndentWidth: 8 68 | ContinuationIndentWidth: 8 69 | ... 70 | # newline here 71 | 72 | To format one file, run: ``clang-format -i -style=file mycode.cc`` and it will automatically find and use the format style file above. 73 | 74 | See the `clang-format configuration documentation `_ for the meaning of these options. 75 | 76 | Integration with :ref:`SublimeText ` , :ref:`Emacs ` and :ref:`Vim ` is also available. 77 | -------------------------------------------------------------------------------- /cpp/boost.rst: -------------------------------------------------------------------------------- 1 | ########### 2 | Using Boost 3 | ########### 4 | 5 | .. note:: 6 | 7 | Changes to this document must be approved by the System Architect (`RFC-24 `_). 8 | To request changes to these policies, please file an :doc:`RFC `. 9 | 10 | 11 | .. _cpp_using_boost: 12 | 13 | A Boost library may be used only if: 14 | 15 | 1. the desired effect cannot be accomplished with a C++17 standard language feature, and 16 | 2. a C++ standard library equivalent is either unavailable or unusable with our :ref:`minimum required compiler version `. 17 | 18 | In particular, the following Boost libraries are no longer accepted as they have standard equivalents in gcc 6.3.1 and above: 19 | 20 | ``any`` 21 | use ```` 22 | 23 | ``array`` 24 | use ```` 25 | 26 | ``bind`` 27 | prefer C++14 lambda functions instead, but use ``std::bind`` from ```` if you must 28 | 29 | ``cstdint`` 30 | use ```` 31 | 32 | ``filesystem`` 33 | use ```` 34 | 35 | ``lambda`` 36 | use C++14 lambda functions 37 | 38 | ``lexical_cast`` 39 | use ``std::to_string``, ``std::stoi``, ``std::stod`` etc. 40 | 41 | ``math`` 42 | use ```` wherever possible 43 | 44 | ``noncopyable`` 45 | use ``= delete`` on the copy constructor and assignment operator 46 | 47 | ``random`` 48 | use ```` 49 | 50 | ``ref`` 51 | use ``std::forward`` and universal / forwarding references (or ``std::ref`` if you must) 52 | 53 | ``smart_ptr`` 54 | use ``std::shared_ptr`` and ``std::unique_ptr`` (and its array specialization) from ```` instead of ``boost::shared_ptr``, ``boost::scoped_ptr`` and ``boost::scoped_array`` 55 | 56 | ``static_assert`` 57 | use C++14 ``static_assert`` 58 | 59 | ``tuple`` 60 | use ```` 61 | 62 | ``type_traits`` 63 | use ```` 64 | 65 | ``unordered_map`` 66 | use ```` 67 | 68 | ``variant`` 69 | use ```` 70 | 71 | Certain Boost libraries are recommended: they should be used whenever applicable in preference to any other method of accomplishing the same effect. 72 | Among others, this category includes: 73 | 74 | * ``current_function`` 75 | * ``format`` 76 | * ``test`` 77 | 78 | Additional Boost libraries are considered safe: they may be used freely where applicable. 79 | 80 | * ``GIL`` 81 | * ``iterator`` 82 | * ``MPI`` 83 | * ``multi_index`` 84 | * ``numeric`` 85 | * ``tokenizer`` 86 | 87 | Most other Boost libraries may be used after appropriate design review. 88 | Particular caution should be used when the library involves substantial template metaprogramming or requires linking. 89 | Among others, the following libraries fall into the extra-caution category: 90 | 91 | * ``Fusion`` 92 | * ``MPL`` 93 | * ``regex`` 94 | * ``serialization`` 95 | 96 | Certain Boost libraries conflict with LSST-standard ways of doing things, are inappropriate for LSST code, are insufficiently developed or well-maintained, or have been found to be excessively complicated. 97 | These are not allowed without special permission. 98 | 99 | * ``config`` 100 | * ``preprocessor`` 101 | * ``python`` 102 | * ``throw_exception`` 103 | -------------------------------------------------------------------------------- /_templates/footer.html: -------------------------------------------------------------------------------- 1 |
2 | {% if next or prev %} 3 | 11 | {% endif %} 12 | 13 |
14 | 15 |
16 |

17 | {%- if show_copyright %} 18 | {%- if hasdoc('copyright') %} 19 | {% trans path=pathto('copyright'), copyright=copyright|e %}© Copyright {{ copyright }}.{% endtrans %} 20 | {%- else %} 21 | {% trans copyright=copyright|e %}© Copyright {{ copyright }}.{% endtrans %} 22 | {%- endif %} 23 | {%- endif %} 24 | 25 | {%- if build_id and build_url %} 26 | {% trans build_url=build_url, build_id=build_id %} 27 | 28 | Build 29 | {{ build_id }}. 30 | 31 | {% endtrans %} 32 | {%- elif commit %} 33 | {% trans commit=commit %} 34 | 35 | Revision {{ commit }}. 36 | 37 | {% endtrans %} 38 | {%- elif last_updated %} 39 | {% trans last_updated=last_updated|e %}Last updated on {{ last_updated }}.{% endtrans %} 40 | {%- endif %} 41 | 42 |

43 | 44 | 45 |

46 | Creative Commons License
{{ project }} by the LSST Project is licensed under a Creative Commons Attribution 4.0 International License. 47 |

48 |
49 | 50 |

LSST documentation is delivered by Fastly.

51 | 52 |

53 | Built with Sphinx using a theme provided by Read the Docs. 54 |

55 | 56 |

Financial support for LSST comes from the National Science Foundation (NSF) through Cooperative Agreement No. 1258333, the Department of Energy (DOE) Office of Science under Contract No. DE-AC02-76SF00515, and private funding raised by the LSST Corporation. The NSF-funded LSST Project Office for construction was established as an operating center under management of the Association of Universities for Research in Astronomy (AURA). The DOE-funded effort to build the LSST camera is managed by the SLAC National Accelerator Laboratory (SLAC).

57 | 58 | {%- block extrafooter %} {% endblock %} 59 | 60 |
61 | -------------------------------------------------------------------------------- /stack/transferring-code.rst: -------------------------------------------------------------------------------- 1 | ################################## 2 | Transferring Code Between Packages 3 | ################################## 4 | 5 | When transferring code between LSST packages with different git repositories, 6 | the following procedure should be used: 7 | 8 | #. Create a JIRA issue that includes the code transfer as part of its work package. 9 | 10 | #. In the origin repository: 11 | 12 | #. Create the usual ``tickets/DM-XXXX`` issue branch. 13 | 14 | #. Create a ``tickets/DM-XXXX-transfer`` branch from the 15 | ``tickets/DM-XXXX`` branch. 16 | 17 | #. On the transfer branch, remove code and files that will be transferred. 18 | All commit messages should contain the name of the destination package 19 | and the transfer branch name (``tickets/DM-xxxx-transfer``). 20 | 21 | #. On the transfer branch, make any additional modifications to the origin 22 | repository to return it to a buildable/testable state (one or more 23 | commits, distinct from the remove commit(s)). 24 | 25 | #. Merge the transfer branch back to the regular issue branch using 26 | ``--no-ff`` to preserve the transfer branch name in the merge commit. 27 | 28 | #. In the destination repository: 29 | 30 | #. Create the usual ``tickets/DM-XXXX`` issue branch. 31 | 32 | #. Create a ``tickets/DM-XXXX-transfer`` branch from the 33 | ``tickets/DM-XXXX`` branch. 34 | 35 | #. On the transfer branch, add the new code and files from the origin 36 | repository. All commit messages should contain the name of the origin 37 | package and the transfer branch name (``tickets/DM-xxxx-transfer``). 38 | 39 | #. On the transfer branch, make any additional modifications to the origin 40 | repository to return it to a buildable/testable state (one or more 41 | commits, distinct from the commit(s) that added code from the origin 42 | repository). 43 | 44 | #. Merge the transfer branch back to the regular issue branch using 45 | ``--no-ff`` to preserve the transfer branch name in the git commit. 46 | 47 | #. Make further changes as necessary on the ``tickets/DM-XXXX`` branches and have 48 | the issue reviewed and merged to main as usual. 49 | 50 | #. Summarize the code transfer on the `DM Stack Package History`_ page. 51 | 52 | Requirements to keep in mind: 53 | 54 | * Commits made directly on the ``tickets/DM-XXXX`` branch should be 55 | buildable/testable. 56 | 57 | * The ``tickets/DM-XXXX-transfer`` branch should contain only commits that move 58 | code or are necessary to get it working at a basic level in the new package. 59 | 60 | * Commits that transfer code should not be mixed with commits that modify the 61 | code to adapt it to its new home (e.g. namespace changes). 62 | 63 | Note that this procedure contains no attempt to actually move, filter, or 64 | summarize the git history from the origin package in the destination package. 65 | Instead, we expect to retain that history in the origin package, using commit 66 | messages, the transfer branches, and the `DM Stack Package History`_ page to 67 | provide a link between the destination and the origin packages. 68 | 69 | See `RFC-33`_ for the motivation and discussion behind this policy. 70 | 71 | .. _RFC-33: https://jira.lsstcorp.org/browse/rfc-33 72 | .. _DM Stack Package History: https://confluence.lsstcorp.org/display/DM/DM+Stack+Package+History 73 | -------------------------------------------------------------------------------- /stack/add-a-package-to-pipelines-lsst-io.rst: -------------------------------------------------------------------------------- 1 | .. _add-to-pipelines-lsst-io: 2 | 3 | ######################################### 4 | How to add a package to pipelines.lsst.io 5 | ######################################### 6 | 7 | The `pipelines.lsst.io`_ documentation site incorporates content from the :doc:`doc/ directories ` of individual LSST Science Pipelines packages. 8 | This page describes how to add a package’s documentation content to the `pipelines.lsst.io`_ documentation site. 9 | The steps are: 10 | 11 | 1. :ref:`add-to-pipelines-lsst-io-prep` 12 | 2. :ref:`add-to-pipelines-lsst-io-eups-table` 13 | 3. :ref:`add-to-pipelines-lsst-io-eups-test` 14 | 4. :ref:`add-to-pipelines-lsst-io-eups-merge` 15 | 16 | .. _add-to-pipelines-lsst-io-prep: 17 | 18 | Prepare the package’s doc directory 19 | =================================== 20 | 21 | The first step is to populate the package’s :file:`doc/` directory with the boilerplate files: 22 | 23 | 1. Clone the package. 24 | 2. Check out a ticket branch for development (:doc:`following the usual workflow `). 25 | 3. Add files to the package’s ``doc/`` directory. 26 | See :doc:`layout-of-doc-directory` for details. 27 | 28 | At this stage, you can build the documentation content for this package alone. 29 | See :doc:`Building single-package documentation locally ` for directions. 30 | 31 | .. _add-to-pipelines-lsst-io-eups-table: 32 | 33 | Add the package to pipelines\_lsst\_io’s EUPS table file 34 | ======================================================== 35 | 36 | The next step is to integrate that package’s documentation with the main documentation repository, `pipelines_lsst_io`_: 37 | 38 | 1. Clone the `pipelines_lsst_io`_ repository. 39 | 2. Check out a ticket branch for development (use the same ticket as for setting up the package’s :file:`doc/` directory). 40 | 3. Add the name of your package to the :file:`ups/pipelines_lsst_io.table` file. 41 | 42 | The combination of having a populated :file:`doc/` directory in your package and having that package set up by `pipelines_lsst_io`_ ensures that the package and any of its Python modules are listed on the homepage generated by `pipelines_lsst_io`_. 43 | 44 | .. _add-to-pipelines-lsst-io-eups-test: 45 | 46 | Test the build 47 | ============== 48 | 49 | At this stage, you can locally build the `pipelines.lsst.io`_ site with your package’s content included. 50 | The page :doc:`building-pipelines-lsst-io-locally` describes how to do this. 51 | 52 | After you set up `pipelines_lsst_io`_ with EUPS, make sure that you also follow the steps in the section :ref:`local-pipelines-lsst-io-build-package-setup` to set up your package. 53 | Even though you’ve added your package to `pipelines_lsst_io`_\ ’s EUPS table file, you need to ensure that it’s your local copy that’s being set up, *not* the copy installed with your base ``lsst_distrib`` (or equivalent) installation. 54 | 55 | .. _add-to-pipelines-lsst-io-eups-merge: 56 | 57 | Review and merge the branches 58 | ============================= 59 | 60 | The last step is to merge the branches in both the package and `pipelines_lsst_io`_ repositories. 61 | In general, this should be done using :doc:`DM’s regular review and merging workflow `. 62 | 63 | Merge the package’s branch first, followed by the branch for `pipelines_lsst_io`_. 64 | 65 | .. _`pipelines.lsst.io`: https://pipelines.lsst.io 66 | .. _`pipelines_lsst_io`: https://github.com/lsst/pipelines_lsst_io 67 | -------------------------------------------------------------------------------- /project-docs/publication-policy.rst: -------------------------------------------------------------------------------- 1 | .. _publication-policy: 2 | 3 | ################################# 4 | Publishing Papers and Proceedings 5 | ################################# 6 | 7 | Papers and conference proceedings written on behalf of the LSST Project are subject to the LSST Project Publication Policy (:lpm:`162`) and are coordinated by the Publication Board. 8 | This page provides pointers to Publication Board documentation. 9 | 10 | .. note:: 11 | 12 | While scientific publications and conference proceedings are subject to :lpm:`162`, many types of DM communication and documentation are not controlled, including: 13 | 14 | - User guides (software documentation). 15 | - Technical notes. 16 | - Design documentation (though LDMs are coordinated by the DM Technical Control Team, TCT). 17 | - Community forum posts. 18 | 19 | Links 20 | ===== 21 | 22 | - `Publication Board homepage `__. 23 | - `LPM-162: LSST Project Publication Policy `__. 24 | - `Publication Board JIRA Project (PUB) `__. 25 | - `Document-13016: Project Publications Style Manual `__. 26 | - `lsst-pst/LSSTreferences `__ GitHub project with citations for key LSST papers. 27 | 28 | Citing DM Technical Notes and Design Documents 29 | ============================================== 30 | 31 | Wherever possible, cite DM design documentation and technotes in addition to the core project papers. 32 | Rather than linking to the document's URL in a footnote, you should use a proper BibTeX citation. 33 | 34 | For technotes, it's better to cite the technote's DOI than its lsst.io URL alone. 35 | The documentation engineering team is making DOIs on an as-needed basis at the moment. 36 | Reach out to `#dm-docs `__ for additional DOIs. 37 | We're working on automating DOI provisioning to make this easier in the future. 38 | 39 | Example design document citation 40 | -------------------------------- 41 | 42 | A BibTeX citation for a design document: 43 | 44 | .. code-block:: text 45 | 46 | @misc{LDM-135, 47 | author = {Jacek Becla and others}, 48 | title = {{LSST Database Design}}, 49 | howpublished = {LDM-135, \url{http://ls.st/LDM-135}}, 50 | year = 2013, 51 | } 52 | 53 | Note that the ``ls.st`` short link points to the official DocuShare-archived version of LDM-135. 54 | 55 | Example technical note citation 56 | ------------------------------- 57 | 58 | A technote citation, pointing to a Zenodo-backed DOI: 59 | 60 | .. code-block:: text 61 | 62 | @techreport{slater_2016_192828, 63 | title = {{False Positive Rates in the LSST Image 64 | Differencing Pipeline}}, 65 | author = {{Slater}, Colin and 66 | {Jurić}, Mario and 67 | {Ivezić}, Željko and 68 | {Jones}, Lynne}, 69 | institution = {{LSST Data Management}}, 70 | type = {{LSST Data Management Technical Note}}, 71 | number = {{DMTN-006}}, 72 | month = mar, 73 | year = 2016, 74 | doi = {10.5281/zenodo.192828}, 75 | url = {https://doi.org/10.5281/zenodo.192828} 76 | } 77 | 78 | .. note:: 79 | 80 | Zenodo offers citations using the ``manual``, rather than ``techreport``, BibTeX type for technical reports, but the example above is likely more complete. 81 | The publisher, given their BibTeX style file, may have different guidance. 82 | -------------------------------------------------------------------------------- /coding/intro.rst: -------------------------------------------------------------------------------- 1 | ###################################### 2 | Introduction to DM's Code Style Guides 3 | ###################################### 4 | 5 | The primary goal of the DM Code Style Guides is to improve readability and thereby the understanding, maintainability and general quality of the code. 6 | It is impossible to cover every specific situation in a general guide so programmer flexibility in interpreting the guidelines---in the spirit in which they were written---is essential. 7 | Experienced programmers have generally evolved a personal programming style which might have been based on textbook recommendations, other projects' style guides or recognition of the clarity that certain styles obtain. 8 | DM developers participated in the creation of the DM style guides; they brought their extensive programming experience into fabricating a uniform style for DM code. 9 | New DM developers should use these guidelines to understand and emulate the coding style adopted by DM. 10 | 11 | .. _style-guide-list: 12 | 13 | Style Guides 14 | ============ 15 | 16 | .. toctree:: 17 | :maxdepth: 2 18 | 19 | ../python/style 20 | ../cpp/style 21 | 22 | In addition, the follow coding guides may help you write code for DM: 23 | 24 | .. toctree:: 25 | :maxdepth: 2 26 | 27 | ../cpp/templates 28 | 29 | .. _style-guide-rfc-2119: 30 | 31 | Stringency Levels 32 | ================= 33 | 34 | In our style guides we use `RFC-2119 `_\ -style vocabulary to rank the importance of conforming to a specific recommendation. 35 | 36 | REQUIRED 37 | The Rule is an absolute requirement of the specification. 38 | The developer needs to petition the DM `Software Architect`_ to acquire explicit approval to contravene the Rule. 39 | 40 | PROHIBITED 41 | The opposite of REQUIRED. 42 | 43 | MUST and SHALL 44 | There may exist valid reasons in particular circumstances to ignore a particular Rule, but the full implications must be understood and carefully weighed before choosing a different course. 45 | The developer needs to petition the DM `Software Architect`_ to acquire explicit approval to contravene the Rule. 46 | 47 | MUST NOT and SHALL NOT 48 | The opposites of MUST and SHALL. 49 | 50 | SHOULD, RECOMMENDED and MAY 51 | There are valid reasons in particular circumstances to ignore a particular Rule. 52 | The developer is expected to personally consider the full implications before choosing a different course. 53 | 54 | SHOULD NOT, NOT RECOMMENDED and MAY NOT 55 | The opposites of SHOULD, RECOMMENDED and MAY. 56 | 57 | .. _Software Architect: https://github.com/lsst-dm/images/blob/main/DmOrg.pdf 58 | 59 | .. _style-guide-deviations: 60 | 61 | Deviating from the DM Style Guides 62 | ================================== 63 | 64 | The guides provide the rationale supporting each of their recommendations. 65 | Consider that rationale before choosing to deviate from the DM coding style. 66 | Be aware that some recommendations also demonstrate 'best-practice' techniques used to avoid introducing programming errors; 'best-practice' use is noted. 67 | 68 | Coding consistency is very important but sometimes the style guide doesn't apply either due to lack of a definitive rule or the circumstances of the specific code segment logically dictate otherwise. 69 | When in doubt, use your best judgment or ask the lead developer. 70 | 71 | Here are two plausible reasons to break a particular rule: 72 | 73 | - When applying the rule would make the code less readable, even for someone who is used to reading code that follows the rules. 74 | - To be consistent with surrounding code that also breaks it (maybe for historic reasons)---although this is also an opportunity to clean up someone else's mess. 75 | -------------------------------------------------------------------------------- /user-docs/index.rst: -------------------------------------------------------------------------------- 1 | .. _user-doc-style-guide: 2 | 3 | ############################## 4 | User documentation style guide 5 | ############################## 6 | 7 | This style guide will help you write technical end-user documentation content for LSST Data Management. 8 | 9 | .. toctree:: 10 | :hidden: 11 | 12 | lsst-specific-content-style-guide 13 | topic-based-documentation 14 | 15 | .. _user-doc-style-guide-list: 16 | 17 | The style guides 18 | ================ 19 | 20 | Our core style is described by the `Google Developer Style Guide`_, plus a list of customizations. 21 | 22 | - View the `Google Developer Style Guide`_. 23 | - View the :doc:`LSST style customizations `. 24 | 25 | If the Google Developer Style Guide doesn’t address your situation, you can fall back to these resources: 26 | 27 | - `Project Publications Style Guide `__ 28 | - `The Chicago Manual of Style `__ 29 | - The `Merriam-Webster `__ dictionary 30 | 31 | Additional writing resources: 32 | 33 | - :doc:`topic-based-documentation` 34 | 35 | Graphics resources: 36 | 37 | - Accessibility guidelines (colour blindness) are provided by `RTN-045: Guidelines for User Tutorials`_. 38 | 39 | .. _user-doc-style-guide-where: 40 | 41 | Where to use this style guide 42 | ============================= 43 | 44 | This style guide is for LSST’s end-user documentation. 45 | A user is anyone that *uses* the software and services we build for LSST. 46 | Users can be astronomers in the community, future operators of LSST, and even your fellow developers. 47 | 48 | More specifically, refer to this style guide when writing any content that services a user: 49 | 50 | - Documentation content (guides, tutorials, how-tos, and references) for software and services. 51 | - Content for LSST websites. 52 | - “Docstrings” in code that are used as API references. 53 | - Copy printed to the console (like error messages). 54 | - User interface microcopy. 55 | 56 | There are common contexts where this style guide **does not** apply: 57 | 58 | - Project documentation (such as policy, requirements, design, test documents, and technical notes). 59 | - Academic manuscripts (such as papers and conference proceedings). 60 | 61 | In those contexts, use the `Project Publications Style Guide `__ or the journal’s style guide. 62 | 63 | .. _user-doc-style-guide-howto: 64 | 65 | How to use this style guide 66 | =========================== 67 | 68 | In LSST DM, we are all responsible for contributing to the documentation for the things we create. 69 | At the same time, writing technical documentation for users differs in many ways from other types of writing we might have more experience in (such as academic writing). 70 | Given that, treat this style guide as an educational resource. 71 | As you learn more from this style guide, your technical writing will improve, and LSST’s documentation as a whole will become better and more consistent. 72 | 73 | It’s not realistic to expect you to internalize every facet of this style guide immediately. 74 | Instead, begin by reviewing the `Highlights `__. 75 | As you write content and have questions, refer to the style guide for solutions to your particular writing problems. 76 | Reviewers in a pull request might point out style issues. 77 | If it’s realistic in the time available, try to update your content to fit the recommended style. 78 | In any case, be open to learning so that the next batch of content you create will be even better. 79 | 80 | See the `About this guide `__ page from the Google Developer Style Guide for additional thoughts, including when to break the rules. 81 | 82 | .. _`Google Developer Style Guide`: https://developers.google.com/style/ 83 | .. _`RTN-045: Guidelines for User Tutorials`: https://rtn-045.lsst.io/#colorblind-friendly-plots 84 | -------------------------------------------------------------------------------- /idf/overview.rst: -------------------------------------------------------------------------------- 1 | ############################## 2 | Interim Data Facility Overview 3 | ############################## 4 | 5 | Pre-operations activities may be carried out on the Interim Data Facility (IDF) hosted on the `Google Cloud Platform `_. 6 | The `Google Cloud documentation site `_ provides an overview of its products and services. 7 | 8 | The IDF is considered a production environment to support Data Preview releases. 9 | Development and Commissioning work continue on the Construction-funded `LSST Data Facility (LDF) <../services/lsst-devl.rst>`_. 10 | 11 | 12 | Login credentials 13 | ================= 14 | 15 | Rubin staff working on deploying and operating Data Production services receive a ``username@lsst.cloud`` identity. 16 | This IDF cloud identity is not an email and is independent from other Rubin accounts. 17 | Not everybody in the Data Production department needs this login; contact your pre-operations team lead(s) if in doubt. 18 | 19 | Two-factor authentication (2FA) is required for the IDF identity. 20 | See :ref:`idf-internal-support` for lost password or login issues. 21 | 22 | 23 | Environments 24 | ============ 25 | 26 | On GCP, resources are organized into a base-level entity called a "project". 27 | A project is linked to billing, permissions, and other settings. 28 | Projects are grouped in folders. 29 | We have six top-level folders: 30 | 31 | - ``Science Platform`` 32 | - ``QServ`` 33 | - ``Processing`` 34 | - ``SQuaRE`` 35 | - ``Shared Services`` 36 | - ``Scratch`` 37 | 38 | For each top-level folder except ``Shared Services`` and ``Scratch``, we have three sub-folders: ``Dev``, ``Integration``, and ``Production``. 39 | ``Production`` is intended to host services used by end users. 40 | ``Dev`` and ``Integration`` provide testing and staging environments where one can deploy near-production services. 41 | 42 | All environments on IDF are managed with `Terraform `_, an infrastructure-as-code tool, via configuration files in the `idf_deploy `_ repo. 43 | See the documentation in idf_deploy's `run book `_. 44 | GitHub Actions are used to automatically deploy GCP resources on IDF in response to pull requests on the configuration files. 45 | All resources should be deployed this way so that operations can be repeated consistently. 46 | 47 | 48 | The only exception to using `idf_deploy `_ is the ``Scratch`` folder. 49 | Projects inside the ``Scratch`` folder are for short-lived testing or cloud training; they may be removed on short notice. 50 | No user-facing services should live in the ``Scratch`` folder. 51 | Contact your pre-operations team lead(s) if your work requires a scratch project. 52 | 53 | All IDF resources should live in the ``us-central1`` region. 54 | 55 | Permissions 56 | =========== 57 | 58 | Permissions are managed via Google Groups configured in the `idf_deploy `_ repo. 59 | Group memberships are managed manually by the administrators. 60 | See :ref:`idf-internal-support` to request to be added to an existing group. 61 | 62 | Monitoring 63 | ========== 64 | 65 | A number of dashboards have been configured in Cloud Monitoring; for more information, see `Monitoring and Logging in the run book `_. 66 | 67 | .. _idf-internal-support: 68 | 69 | Support (interim) 70 | ================= 71 | 72 | Questions can be sent to the `#ops-google-idf `_ Slack channel. 73 | For requests or infrastructure service issues, file a `JIRA `_ DM ticket, add the ``idf`` label, and mention the ticket in the #ops-google-idf Slack channel. 74 | Infrastructure issues will be redirected to Google Support. 75 | See `Google's Best practices on ticketing issues `_. 76 | -------------------------------------------------------------------------------- /work/backports.rst: -------------------------------------------------------------------------------- 1 | ####################################### 2 | Backporting Tickets to a Release Branch 3 | ####################################### 4 | 5 | So you have a ticket that you think should be backported to a previous release. 6 | What next? 7 | 8 | 1) First, add the label "backport-v23" to your jira ticket, replacing "23" here and elsewhere on this page with the actual release you're targeting. 9 | To backport to multiple releases, add multiple labels. 10 | 11 | This will flag it for review by the DM-CCB. 12 | Continue merging your ticket to the default branch (``main``) and mark the ticket ``Done`` per instructions in the normal :doc:`flow`. 13 | 14 | 2) Wait for approval. 15 | The ticket will gain the label "backport-approved." 16 | A comment will be posted on the ticket that you may start the backporting process. 17 | Backports are approved for all requested releases together. 18 | 19 | 3) Checkout the release branch, ``v23.0.x``, for each repo affected by your ticket. 20 | 21 | .. code-block:: bash 22 | 23 | git checkout v23.0.x 24 | 25 | If the repo does not already have a ``v23.0.x`` branch, you need to create one based on ``v23.0.0.rc1`` (not the latest rcN, although that should be identical): 26 | 27 | .. code-block:: bash 28 | 29 | git checkout v23.0.0.rc1 30 | git checkout -b v23.0.x 31 | git push -u origin v23.0.x 32 | 33 | Now treat ``v23.0.x`` same as you would the default branch (``main``). 34 | 35 | 4) Create a copy of your ticket branch called ``tickets/DM-XXXXX-v23``. 36 | 37 | .. code-block:: bash 38 | 39 | git checkout tickets/DM-XXXXX 40 | git checkout -b tickets/DM-XXXXX-v23 41 | git rebase --onto v23.0.x 42 | 43 | In a new clone, this branch may not exist anymore if you have already merged your PR because merged branches may be automatically deleted. 44 | In this case, you can branch ``tickets/DM-XXXXX-v23`` from ``v23.0.x``, and cherry-pick the ticket commits. 45 | 46 | .. code-block:: bash 47 | 48 | git checkout v23.0.x 49 | git checkout -b tickets/DM-XXXXX-v23 50 | git cherry-pick 51 | 52 | The following may help you find your hash[es] from ``main``: 53 | ``git show --quiet $(git log --oneline | grep 'Merge.*DM-XXXXX' | cut -d' ' -f1)^2`` 54 | 55 | 5) Resolve any rebase or cherry-pick problems depending on your method. 56 | Continue using the same procedure outlined in :ref:`review-preparation`. 57 | Check that it cleanly builds via scons. There should be a latest shared v23.0.0.rcN stack on lsst-devl. 58 | Run Jenkins. When running Jenkins, build against the release branch and rc1 even if later rcNs exist. 59 | The default ``SPLENV_REF`` value (the rubin-env conda metapackage version) may no longer be appropriate for ``v23.0.x``. 60 | If you are unsure of the recommended env for the release, see the :external+pipelines:ref:`release documentation page `, the `release tag files `__. or the ``env_name`` files nested under ``/sdf/group/rubin/sw/tag/`` at USDF. 61 | 62 | .. code-block:: bash 63 | 64 | REFS: tickets/DM-XXXXX-v23 v23.0.x v23.0.0.rc1 65 | PRODUCTS: lsst_distrib lsst_ci ci_imsim ci_hsc 66 | SPLENV_REF: 0.8.0 67 | 68 | You may find that the ticket cannot be cleanly backported without first backporting another ticket. 69 | 70 | 6) When it passes, merge to ``v23.0.x`` using the same procedure outlined in :ref:`workflow-code-review-merge`, 71 | including creating a pull request. 72 | On your pull request, remember to change the base branch to ``v23.0.x``. 73 | If the backport was clean, you may self-review. 74 | If the backport was not clean and you would like another pair of eyes, you may ask someone to hit the "Approved" button on GitHub, 75 | but please *do not* put your ticket status back into ``In Review`` on Jira. 76 | 77 | 7) When a ticket has been backported to all requested releases, label your Jira ticket ``backport-done``. 78 | -------------------------------------------------------------------------------- /it/itsc.rst: -------------------------------------------------------------------------------- 1 | .. _itsc: 2 | 3 | ##################################################### 4 | LSST Information Technology Services Committee (ITSC) 5 | ##################################################### 6 | 7 | Background 8 | ========== 9 | 10 | On LSST, we have many entities (subsystems, SE, SQuaRE, PMO) selecting, purchasing, deploying, supporting, upgrading, and retiring IT Services for the MREFC project. 11 | We need robust vehicles for keeping everyone informed about all the services: who provides them, how to get support, are other groups planning something similar or have done something that can be leveraged, and what are the internal “service levels” defining what level of service and support can be expected. 12 | Many point-to-point communications and “local” channels have been set up to to address various aspects, but we now have a more integrated approach to achieve the goal of optimal management and support. 13 | 14 | Charter 15 | ======= 16 | 17 | To address the above, the ITSC was chartered to: 18 | 19 | - Advise what ITS the project provides/uses. 20 | - Make sure interoperability exists among ITS where possible. 21 | - Combine, reuse, recycle ITS when possible. 22 | - Prevent ITS from becoming stagnant or security hazards. 23 | - Act as an agent of the project to keep project advised of what is going on at all spectrums of ITS. 24 | - Make recommendation whether a particular IT Service continues, when to retire/replace. 25 | - Make recommendations how the Project Office ITS (mail, web, issues, etc.) will transition into commissioning and operations. 26 | 27 | Scope 28 | ===== 29 | 30 | The Project Office has final authority over any action or recommendation made by the IT Services Committee, especially if it results in a budgetary increase or has other consequences to project or observatory operations. 31 | Further, actions or recommendations cannot violate existing Information Technology Security Program and Compliance Policies: 32 | 33 | - :lpm:`121` – LSST Master Information Security Plan 34 | - :lpm:`123` – Acceptable Use Policy 35 | - Subsystem IT Security Plans: LPM , LDM, LTS, LCAM 36 | - :lpm:`141` – Compliance Policy 37 | 38 | The ITSC’s role of coordination and recommendation on ITS is currently chartered only in construction and commissioning. 39 | It is expected the ITSC or something like it will be chartered by the Observatory Director and will coordinate across Data Facility, Business, and Observatory Operations groups. 40 | 41 | Membership 42 | ========== 43 | 44 | The ITSC consists of Core and Representative Members. 45 | The Core Members consist of representatives from entities that are key providers of IT Services to the project, including the PMO, the Tucson IT Services Team, the LSST Information Security Officer(s), the DM NCSA team, the DM SQuARE team, and the LSST Compliance Officer: Veronica Kinnison (non-voting). 46 | 47 | The Representative Members represent a cross-section of the various project groups that have a responsibility for delivery consisting of and/or a vested interest in IT Services, including: Systems Engineering, Data Management, Telescope and Site, Camera, and EPO. 48 | 49 | Process 50 | ======= 51 | 52 | The ITSC meets monthly to review the status of IT Services and to process IT Requests for Consensus (ITRFCs). 53 | Meetings are open to anyone, agendas and minutes are public. 54 | ITRFCs are created by ITSC members to inform and to gather feedback from the overall project on the proposed introduction, update, or retirement events for IT Services. 55 | ITRFCs in JIRA are public (use `Project = ITRFC `_ to search issues). 56 | 57 | Your ITSC Representative Member is your primary interface to the ITSC responsible for: 58 | 59 | - Representing you in meetings and communications of the ITSC. 60 | - Making ITSC aware of your IT Services plans and proposals. 61 | - Making you aware of others IT Services plans and proposals. 62 | - Gathering your feedback on ITRFCs. 63 | 64 | Tools 65 | ===== 66 | 67 | The ITSC has endorsed the creation and maintenance of an LSST-wide Services database. 68 | This database was prototyped as a Filemaker Database application and is currently accessible by ITSC. 69 | The database will be implemented on a more production platform, and will be public in future. 70 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [![Web site](https://img.shields.io/badge/developer-lsst.io-brightgreen.svg)](https://developer.lsst.io) 2 | [![GitHub Actions](https://github.com/lsst-dm/dm_dev_guide/workflows/CI/badge.svg)](https://github.com/lsst-dm/dm_dev_guide/actions?query=workflow%3A%22CI%22) 3 | 4 | # LSST DM Developer Guide 5 | 6 | This repository contains the content for the [LSST DM Developer Guide, https://developer.lsst.io](http://developer.lsst.io). 7 | 8 | Everyone in LSST DM is encouraged to not only *use* this guide, but also *contribute* to it. 9 | The contribution process is outlined below. 10 | 11 | Documentation is written in reStructuredText, built using [Sphinx](http://sphinx-doc.org), and hosted with our in-house platform, [LSST the Docs](https://sqr-006.lsst.io). 12 | 13 | **Changes to the `main` branch are automatically deployed to https://developer.lsst.io.** 14 | Other branches are also deployed: find them at https://developer.lsst.io/v. 15 | 16 | ## Contributing 17 | 18 | 1. Clone this repository: `git clone https://github.com/lsst-dm/dm_dev_guide && cd dm_dev_guide`. 19 | 20 | 2. Create a branch. This can either be an informal user branch or a full-fledged ticket branch tracked in JIRA. See the [DM Workflow guide](https://developer.lsst.io/work/flow.html#git-branching) for details. 21 | 22 | 3. Make and commit your edits. Content is written in reStructuredText. Our [reStructuredText Style Guide](https://developer.lsst.io/restructuredtext/style.html) covers the syntax you'll need. 23 | 24 | 4. Push your development branch to GitHub and make a pull request. The pull request page will help you track the publishing and testing status of your branch. 25 | 26 | 5. If your build on [GitHub Actions](https://github.com/lsst-dm/dm_dev_guide/actions?query=workflow%3A%22CI%22) is successful, your branch will be published with LSST the Docs. **Find your branch listed at https://developer.lsst.io/v.** 27 | 28 | 6. Once you're done, press the green button on your pull request to merge to `main`. Your changes are automatically published to the main URL: https://developer.lsst.io. Don't worry about messing things up, GitHub branch protections will ensure that your edits build successfully, and that your branch is up-to-date with `main`. 29 | 30 | ### Installing and building these docs locally 31 | 32 | Although LSST the Docs will publish your development branch, you might want to build this documentation locally. 33 | 34 | Assuming you've cloned the docs (following the guide above): 35 | 36 | 1. Create a [Python virtual environment](http://docs.python-guide.org/en/latest/dev/virtualenvs/) for this project using your tool of choice: [virtualenvwrapper](http://virtualenvwrapper.readthedocs.org/en/latest/) or [pyvenv](https://docs.python.org/3/library/venv.html) (for Python 3). 37 | 38 | 2. Install dependencies 39 | 40 | ``` 41 | pip install -r requirements.txt 42 | ``` 43 | 44 | 3. Compile the HTML by running 45 | 46 | ``` 47 | make html 48 | ``` 49 | 50 | The built site is in the `_build/html` directory. 51 | 52 | ### Editing entirely on GitHub 53 | 54 | If you're in a hurry, you don't need to worry about cloning the Developer Guide; you can do everything on GitHub.com. See [GitHub's documentation](https://help.github.com/articles/github-flow-in-the-browser/) on editing files and creating branches entirely from GitHub.com. 55 | 56 | Remember to preview your published branch by finding it at https://developer.lsst.io/v (see Step #5 of § *Contributing*, above). 57 | 58 | ## License 59 | 60 | Copyright 2015-2019 Association of Universities for Research in Astronomy, Inc. (AURA). 61 | 62 | Creative Commons License
LSST DM Developer Guide by Association of Universities for Research in Astronomy, Inc. is licensed under a Creative Commons Attribution 4.0 International License.
Based on a work at https://github.com/lsst_dm/dm_dev_guide. 63 | -------------------------------------------------------------------------------- /editors/vim.rst: -------------------------------------------------------------------------------- 1 | ###################################### 2 | Vim Configuration for LSST Development 3 | ###################################### 4 | 5 | This page will help you configure Vim to be consistent with LSST's coding standards and development practices. 6 | 7 | Vim runtime configuration setup 8 | =============================== 9 | 10 | The vimrc file contains optional user-defined configuration settings. On Unix based systems, the file is located at ``~/.vimrc`` (create if needed). 11 | 12 | Depending on your particular setup, Vim may or may not be configured to automatically recognize filetype configurations such as those listed below. To force Vim to recognize such files, add the following to your vimrc file: 13 | 14 | .. code-block:: text 15 | 16 | filetype plugin on 17 | 18 | You may also need to tell vim to indent according to your filetype plugins by adding the following to your vim rc file: 19 | 20 | .. code-block:: text 21 | 22 | filetype indent on 23 | 24 | or alternatively, combining the two commands into one line: 25 | 26 | .. code-block:: text 27 | 28 | filetype plugin indent on 29 | 30 | C++ filetype configuration 31 | ========================== 32 | 33 | The following configuration file should be placed in :file:`~/.vim/ftplugin/c.vim`: 34 | 35 | .. code-block:: text 36 | 37 | setlocal shiftwidth=4 38 | setlocal softtabstop=4 39 | setlocal expandtab 40 | setlocal textwidth=110 41 | setlocal cino=:0,l1,g0,(0,u0,W2s 42 | setlocal comments^=s2:/**,mb:*,ex:*/ 43 | syntax match cTodo /\todo/ 44 | 45 | Explanation 46 | ----------- 47 | 48 | ``setlocal shiftwidth=4`` 49 | Sets the autoindent spacing to 4 spaces. 50 | ``setlocal`` to limit this to the current (C++) buffer. 51 | 52 | ``setlocal softtabstop=4`` 53 | Sets the spacing for tab characters to 4 spaces. 54 | 55 | ``setlocal expandtab`` 56 | Always use spaces; never use tab characters. 57 | 58 | ``setlocal textwidth=110`` 59 | Limit the length of a line to 110 characters (see :ref:`4-6 of the C++ Style Guide `) 60 | 61 | ``setlocal cino=:0,l1,g0,(0,u0,W2s`` 62 | Set the C indent configuration. 63 | 64 | - ``:0``: align case labels with the enclosing switch. 65 | - ``l1``: indent statements relative to the case label, not anything following. 66 | - ``g0``: align C++ scope (private/public) labels with the enclosing class. 67 | - ``(0``: align lines after "(foo" next to the unclosed parenthesis. 68 | - ``u0``: same as above for the next level of parentheses deeper. 69 | - ``W2s``: indent lines following a line-terminating unclosed parenthesis by two shiftwidths (8 spaces). 70 | 71 | ``setlocal comments^=s2:/**,mb:*,ex:*/`` 72 | Allow comments to start with two asterisks for Doxygen. 73 | 74 | ``syntax match cTodo /\todo/`` 75 | Highlights lines using the Doxygen ``\todo``. 76 | 77 | .. _clang_format_vim_integration: 78 | 79 | Clang-format integration 80 | ------------------------ 81 | 82 | There is an integration for Vim which lets you run the ``clang-format`` standalone tool on your current buffer, optionally selecting regions to reformat. The integration has the form of a python-file which can be found under ``clang/tools/clang-format/clang-format.py``. 83 | 84 | This can be integrated by adding the following to your ``.vimrc``: 85 | 86 | .. code-block:: text 87 | 88 | map :pyf /clang-format.py 89 | imap :pyf /clang-format.py 90 | 91 | The first line enables clang-format for ``NORMAL`` and VISUAL mode, the second line adds support for INSERT mode. Change ``C-K`` to another binding if you need clang-format on a different key (``C-K`` stands for ``Ctrl+k``). 92 | 93 | With this integration you can press the bound key and clang-format will format the current line in NORMAL and INSERT mode or the selected region in VISUAL mode. The line or region is extended to the next bigger syntactic entity. 94 | 95 | It operates on the current, potentially unsaved buffer and does not create or save any files. To revert a formatting, just undo. 96 | 97 | Python filetype configuration 98 | ============================= 99 | 100 | The following configuration file should be placed in :file:`~/.vim/ftplugin/python.vim`: 101 | 102 | .. code-block:: text 103 | 104 | setlocal shiftwidth=4 105 | setlocal softtabstop=4 106 | setlocal expandtab 107 | setlocal textwidth=110 108 | 109 | -------------------------------------------------------------------------------- /stack/config-topic-type.rst: -------------------------------------------------------------------------------- 1 | .. _config-topic: 2 | 3 | ################# 4 | Config topic type 5 | ################# 6 | 7 | Config classes (subclasses of ``lsst.pex.config.Config``) are typically associated with a task class and are documented as part of the :doc:`task-topic-type`. 8 | Some config classes stand alone, but are used used in tasks through a ``ConfigField`` configuration field. 9 | For example, the ``lsst.pipe.tasks.photoCal.PhotoCalTask`` has a ``colorterms`` configuration field that points to a standalone ``lsst.pipe.tasks.colorterms.ColortermLibrary`` config class. 10 | This page describes how to document config classes like ``ColortermLibrary``. 11 | 12 | .. _config-topic-template: 13 | 14 | Starter template 15 | ================ 16 | 17 | Create a new config topic from Slack.\ [#template]_ 18 | Open a |dmw-squarebot| and type: 19 | 20 | .. code-block:: text 21 | 22 | create file 23 | 24 | Then select **Science Pipelines documentation > Config topic**. 25 | 26 | .. [#template] The config topic file template is maintained in the `lsst/templates repository`_. 27 | 28 | .. _lsst/templates repository: https://github.com/lsst/templates/tree/main/file_templates/config_topic 29 | 30 | For an example config named ``lsst.example.ExampleConfig``, the rendered template looks like this: 31 | 32 | .. remote-code-block:: https://raw.githubusercontent.com/lsst/templates/main/file_templates/config_topic/lsst.example.ExampleConfig.rst 33 | :language: rst 34 | 35 | The next sections describe the key components of config topics. 36 | 37 | .. _config-topic-filename: 38 | 39 | File name and location 40 | ====================== 41 | 42 | Config topic files are located in the :file:`configs/` subdirectory of the :ref:`module documentation directory ` within a package. 43 | The page itself is named after the fully-qualified name of the config class with a ``.rst`` extension. 44 | 45 | For example, suppose a standalone config class is ``lsst.pipe.tasks.colorterms.Colorterm``. 46 | Its config topic page is located in the ``pipe_tasks`` package directory like this: 47 | 48 | .. code-block:: text 49 | 50 | . 51 | └── doc 52 |    └── lsst.pipe.tasks 53 |       ├── index.rst 54 |       └── configs 55 |       └── lsst.pipe.tasks.colorterms.Colorterm.rst 56 | 57 | .. _config-topic-preamble: 58 | 59 | Preamble 60 | ======== 61 | 62 | The :rst:dir:`lsst-config-topic` directive at the top of the page declares that the page is a canonical reference for the specified config class. 63 | For example: 64 | 65 | .. code-block:: text 66 | 67 | .. lsst-config-topic:: lsst.pipe.tasks.colorterms.Colorterm 68 | 69 | Through this directive, other pages can reference this config using the :rst:role:`lsst-config` role: 70 | 71 | .. code-block:: rst 72 | 73 | :lsst-config:`lsst.pipe.tasks.colorterms.Colorterm` 74 | 75 | In addition, other pages can use the :rst:dir:`lsst-configs` directive to automatically list config pages that are marked by an :rst:dir:`lsst-config-topic` directive. 76 | See the :doc:`module-homepage-topic-type` for an example of this strategy. 77 | 78 | .. _config-topic-title: 79 | 80 | Title 81 | ===== 82 | 83 | The title is the name of the config class (without its module name). 84 | No special code formatting is applied to the title. 85 | 86 | .. _config-topic-context: 87 | 88 | Context paragraph 89 | ================= 90 | 91 | Directly below the title, write a paragraph or two (though not many) that describe what the config is for. 92 | Link to any related tasks using the :rst:role:`lsst-task` role. 93 | 94 | .. _config-topic-fields: 95 | 96 | Configuration fields 97 | ==================== 98 | 99 | This section lists the configuration fields that the config class provides. 100 | The listing is similar to the :ref:`task-topic-configs` of task configs. 101 | 102 | The only content of this section in the reStructuredText file is a :rst:dir:`lsst-config-fields` directive. 103 | The fully-qualified name of the config class is the directive's sole argument. 104 | 105 | For example: 106 | 107 | .. code-block:: rst 108 | 109 | .. lsst-config-fields:: lsst.pipe.tasks.colorterms.Colorterm 110 | 111 | In depth section 112 | ================ 113 | 114 | If necessary, you can provide an extended discussion of the configuration in this section. 115 | Feel free to include examples as necessary. 116 | 117 | Omit this section if there isn't any content for it. 118 | -------------------------------------------------------------------------------- /stack/building-single-package-docs.rst: -------------------------------------------------------------------------------- 1 | .. _build-package-docs: 2 | 3 | ############################################# 4 | Building single-package documentation locally 5 | ############################################# 6 | 7 | While developing a package and its documentation, you’ll probably want to preview what the documentation looks like when built. 8 | The quickest way to do this is with a single-package documentation build. 9 | 10 | Keep in mind that a single-package documentation build may have many broken links because package documentation is *meant* to be built in the context of the package stack. 11 | To do a full-stack documentation build, see either: 12 | 13 | - :doc:`building-pipelines-lsst-io-locally` 14 | - :doc:`building-pipelines-lsst-io-with-documenteer-job` 15 | 16 | Nevertheless, a single-package build is useful for checking formatting and proof-reading your work. 17 | This page describes how to run a single-package documentation build, using `pipe_base`_ as the example. 18 | 19 | .. _build-package-docs-prereqs: 20 | 21 | Prerequisites 22 | ============= 23 | 24 | Before starting this tutorial, you’ll need a working ``lsst_distrib`` installation. 25 | This installation should already be set up with a command like :command:`setup lsst_distrib`. 26 | 27 | This installation needs to be a recent daily or weekly build since you’ll be compiling the `pipe_base`_ repository from its ``main`` branch. 28 | Working from the tip of the ``main`` branch is the norm for LSST software development. 29 | 30 | The documentation build uses Documenteer_ and related Sphinx_ documentation packages, which are already installed if you are using the Rubin Conda developer environment. 31 | You can check if ``rubin-env-developer`` is installed by running ``mamba list rubin-env``. 32 | If not, you can install the developer environment with: ``mamba install rubin-env-developer``. 33 | 34 | .. _build-package-docs-setup-package: 35 | 36 | Downloading and setting up a package 37 | ==================================== 38 | 39 | You need to clone and build the package locally. 40 | In this example, you’ll be cloning and building `pipe_base`_: 41 | 42 | .. code-block:: bash 43 | 44 | git clone https://github.com/lsst/pipe_base 45 | cd pipe_base 46 | setup -k -r . 47 | scons 48 | 49 | .. note:: 50 | 51 | If you’re actively developing a package, it’s likely that you’ve already cloned and built that package. 52 | 53 | .. _build-package-docs-build: 54 | 55 | Building the package’s documentation 56 | ==================================== 57 | 58 | You can build the package’s documentation by running: 59 | 60 | .. code-block:: bash 61 | 62 | package-docs build 63 | 64 | The built HTML is located, relative to the :file:`pipe_base` directory, at :file:`doc/_build/html`. 65 | 66 | .. note:: 67 | 68 | The page at :file:`doc/_build/html/index.html` is the homepage for single-package builds. 69 | It never appears in the `pipelines.lsst.io`_ site build but does link to all the package and module documentation directories listed in the package's :ref:`doc/manifest.yaml file `. 70 | 71 | See `Documenteer’s documentation for more information about the package-docs command`_. 72 | 73 | .. _build-package-docs-install-delete-build: 74 | 75 | Deleting built documentation 76 | ============================ 77 | 78 | Since Sphinx only builds files that have changed, and may not notice updated docstrings, you may need to delete the built documentation to force a clean rebuild. 79 | You can delete this built documentation by running: 80 | 81 | .. code-block:: bash 82 | 83 | package-docs clean 84 | 85 | Further reading 86 | =============== 87 | 88 | - `Documentation for the package-docs command in Documenteer`_ 89 | - Alternative ways to build documentation: 90 | 91 | - :doc:`building-pipelines-lsst-io-locally` 92 | - :doc:`building-pipelines-lsst-io-with-documenteer-job` 93 | 94 | .. _`Documenteer`: https://documenteer.lsst.io 95 | .. _`Documenteer’s documentation for more information about the package-docs command`: 96 | .. _`Documentation for the package-docs command in Documenteer`: https://documenteer.lsst.io/pipelines/package-docs-cli.html 97 | .. _`Documenteer installation documentation`: https://documenteer.lsst.io/pipelines/install.html 98 | .. _`pipelines.lsst.io`: https://pipelines.lsst.io 99 | .. _`pipelines_lsst_io`: https://github.com/lsst/pipelines_lsst_io 100 | .. _`pipe_base`: https://github.com/lsst/pipe_base 101 | .. _`Sphinx`: https://www.sphinx-doc.org/en/master/ 102 | -------------------------------------------------------------------------------- /work/jira-agile.rst: -------------------------------------------------------------------------------- 1 | ################ 2 | JIRA Agile Usage 3 | ################ 4 | 5 | Refer to `DMTN-020 `_ for detailed information on the JIRA-based planning process used by DM. 6 | 7 | Key Concepts 8 | ============ 9 | 10 | - Cycles (aka Releases) occur every 6 months. 11 | - Epics are higher-level features or capabilities that are to be delivered in a cycle. 12 | Epic names should be short and descriptive. 13 | - Stories describe concrete, demonstrable features to be delivered in a single sprint. 14 | - Story points are to be assigned at 2 story points per full-time uninterrupted day of an appropriate developer. 15 | 16 | .. _jira-teams: 17 | 18 | Teams 19 | ===== 20 | 21 | The JIRA “team” field is used to define financial and managerial responsibility for getting the work done. 22 | We recognize the following teams: 23 | 24 | ================================== ============================== 25 | Team Name Responsible Manager 26 | ================================== ============================== 27 | System Management Wil O'Mullane 28 | DM Science Leanne Guy 29 | Architecture K-T Lim 30 | Alert Production Ian Sullivan 31 | Data Release Production Yusra AlSayyad 32 | Science User Interface Xiuqin Wu 33 | Data Access and Database Fritz Mueller 34 | Data Facility Michelle Butler 35 | International Comms and Base Site Jeff Kantor 36 | SQuaRE Frossie Economou 37 | Telescope and Site Andy Clements 38 | External None (see below) 39 | ================================== ============================== 40 | 41 | The “External” team is used to label work done by individuals who are not funded by LSST. 42 | 43 | In general, every ticket should be assigned to a particular team — otherwise, nobody is responsible for ensuring it gets done. 44 | If you aren't sure which team to assign a ticket to, just leave the field blank, and one of the T/CAMs will pick it up. 45 | In the exceptional case that the ticket describes work which could be done by *either* of the Science Pipelines teams (Alert Production and Data Release Production), but it's not clear which, it can be marked with the label ``SciencePipelines`` instead of setting the team. 46 | 47 | .. _jira-labels: 48 | 49 | Labels 50 | ====== 51 | 52 | We support and encourage the use of labels to group related tickets. 53 | Their use is not formally restricted or regulated. 54 | However, there are a few labels which are of general interest: 55 | 56 | ==================== ============================================================================================================================================ 57 | Label Meaning 58 | ==================== ============================================================================================================================================ 59 | ``dm-sst`` This work is of interest to the DM System Science Team. 60 | ``dm-set`` This work is of interest to the DM Systems Engineering Team. 61 | ``DMLT`` This work is of interest to the DM Leadership Team. 62 | ``gen3-middleware`` Work on the “generation 3” Butler and associated middleware (e.g. SuperTask). 63 | ``SciencePipelines`` Work which should be performed by either AP (02C.03) or DRP (02C.04), but it’s not (yet) clear which so we can’t easily set the “team” field 64 | ==================== ============================================================================================================================================ 65 | 66 | In addition, please label any tickets describing work performed on documents with an assigned document handle with that handle (``DMTN-123``, ``LDM-456``, ``LSE-789``, etc). 67 | 68 | .. _jira-components: 69 | 70 | Components 71 | ========== 72 | 73 | You can assign one or more components to tickets to describe which part or parts of the system they affect. 74 | Components are selected from a pre-defined list; only JIRA administrators have the ability to add new components. 75 | The following guidelines may be helpful when choosing components: 76 | 77 | - If the ticket involves working in one or more software repositories, add the corresponding components (for example, ``afw``, ``daf_butler``, ``ap_association``). 78 | If no component exists for the repository you are working in, ask your T/CAM for help. 79 | - If the ticket involves work on a document, choose a component that best describes the type of the document you are working on (for example, ``Design Documents``, ``Review Documents``, ``Requirements Documents``). 80 | Even if the document is hosted in a Git repository, don't add that repository as a component; use a label :ref:`label ` instead. 81 | -------------------------------------------------------------------------------- /stack/package-homepage-topic-type.rst: -------------------------------------------------------------------------------- 1 | .. _package-homepage: 2 | 3 | ########################### 4 | Package homepage topic type 5 | ########################### 6 | 7 | The package homepage is the :file:`index.rst` file located at the root of each :ref:`package documentation directory `. 8 | The purpose of the package homepage is to provide summary information about a package and a table of contents for additional topics in the package documentation directory. 9 | 10 | The package homepage topic type is only used in packages that have a :ref:`package documentation directory `. 11 | Specifically, if a package provides a Python module, it won't have a package documentation directory, and thus won't have this file. 12 | 13 | .. _package-homepage-template: 14 | 15 | Starter template 16 | ================ 17 | 18 | The `stack_package`_ project template includes the `Jinja-formatted template for the package homepage `_. 19 | 20 | For an example package named ``example_dataonly``, the rendered template looks like this: 21 | 22 | .. remote-code-block:: https://raw.githubusercontent.com/lsst/templates/main/project_templates/stack_package/example_dataonly/doc/example_dataonly/index.rst 23 | :language: rst 24 | 25 | The next sections describe the key components of the package homepage. 26 | 27 | .. _package-homepage-filename: 28 | 29 | File name and location 30 | ====================== 31 | 32 | This file must be named :file:`index.rst` and must be located in a :ref:`package documentation directory ` within a package. 33 | For example, if the package’s name is ``example``, the full path for the file is :file:`doc/example/index.rst`. 34 | 35 | .. _package-homepage-title: 36 | 37 | Title 38 | ===== 39 | 40 | The title (top-level header) of the package homepage is the packages’s name, without any special formatting (like code literals). 41 | 42 | .. note:: 43 | 44 | In the future we might include a brief descriptive phrase after the package’s name. 45 | There is no guidance to add this descriptive phrase at this time. 46 | 47 | The cross-reference target above the title must be name of the package followed by a ``-package`` suffix (see the example from the :ref:`starter template `). 48 | 49 | .. _package-homepage-context: 50 | 51 | Context paragraph 52 | ================= 53 | 54 | Directly after the title, include one or two paragraphs that describe what the package is for. 55 | The purpose of this content is not to describe the package in detail, or how to use it, but instead to quickly orient a reader. 56 | 57 | You can also mention related packages: 58 | 59 | .. code-block:: rst 60 | 61 | Definitions of metrics and their specifications are provided separately in the :doc:`/packages/verify_metrics/index` package. 62 | 63 | If a package provides datasets (such as a test data repository), you can summarize what these datasets are for, and what APIs use those datasets. 64 | 65 | .. _package-homepage-project-using: 66 | 67 | Using section 68 | ======================= 69 | 70 | If the package documentation directory includes additional topics, in separate reStructuredText files, you should link to them in this section using a `toctree`_ directive. 71 | For example, given a package named ``example`` with files :file:`doc/example/topic-a.rst` and :file:`doc/example/topic-b.rst`, the “Using” section should be presented like this: 72 | 73 | .. code-block:: rst 74 | 75 | Using example 76 | ============= 77 | 78 | .. toctree:: 79 | :maxdepth: 1 80 | 81 | topic-a 82 | topic-b 83 | 84 | Each of these other reStructuredText files should follow the :doc:`Generic guide topic type `. 85 | 86 | .. _package-homepage-contributing: 87 | 88 | Contributing section 89 | ==================== 90 | 91 | This section puts the package in context as an open source development project. 92 | The `template `__ seeds this section with links to the GitHub repository for the package and a ticket search with the package's corresponding Jira component (if the package does not have a Jira component, request one in `#dm-square`_). 93 | 94 | If there is documentation describing how to develop (contribute) to the package, as opposed to using the package, you should link to those topics with a `toctree`_ in this section. 95 | 96 | .. _stack_package: https://github.com/lsst/templates/tree/main/project_templates/stack_package 97 | .. _toctree: http://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html#directive-toctree 98 | .. _`#dm-square`: https://lsstc.slack.com/archives/dm-docs 99 | -------------------------------------------------------------------------------- /stack/license-and-copyright.rst: -------------------------------------------------------------------------------- 1 | ################################################ 2 | Managing license and copyright in Stack packages 3 | ################################################ 4 | 5 | All packages in the LSST Stack (EUPS-distributed packages installed with ``lsst_distrib`` and ``qserv_distrib``, for example) are licensed under `GPL-3.0`_ terms. 6 | This page describes the three aspects developers need to know to properly implement the `GPL-3.0`_ license in Stack packages: 7 | 8 | 1. :ref:`The LICENSE file ` 9 | 2. :ref:`The COPYRIGHT file ` 10 | 3. :ref:`License preambles in source files ` 11 | 12 | For additional background information on licenses and copyright for LSST DM work, see the pages :doc:`/legal/licensing-overview` and :doc:`/legal/copyright-overview`. 13 | 14 | .. _stack-package-license: 15 | 16 | The LICENSE file 17 | ================ 18 | 19 | Each Stack package must have a file called :file:`LICENSE` at its root. 20 | You can find the LICENSE file in the `stack_package template `__. 21 | 22 | Be careful not to modify the LICENSE file. 23 | 24 | .. tip:: 25 | 26 | **Create a LICENSE file from Slack.** 27 | Open a |dmw-squarebot| and type: 28 | 29 | .. code-block:: text 30 | 31 | create file 32 | 33 | Then select **General > GPLv3 LICENSE**. 34 | 35 | .. _stack-package-copyright: 36 | 37 | The COPYRIGHT file 38 | ================== 39 | 40 | Each Stack package must have a file called :file:`COPYRIGHT` at its root where we record copyright assignments. 41 | See :ref:`copyright-file` for information on how to format the :file:`COPYRIGHT` file. 42 | 43 | All DM developers are expected to participate in maintaining the :file:`COPYRIGHT` file on behalf of your institution. 44 | Include additions to :file:`COPYRIGHT` files as part of your regular pull requests. 45 | :download:`/legal/copyright.py` is a script that may help maintain :file:`COPYRIGHT` files. 46 | 47 | .. tip:: 48 | 49 | **Create a COPYRIGHT file from Slack.** 50 | Open a |dmw-squarebot| and type: 51 | 52 | .. code-block:: text 53 | 54 | create file 55 | 56 | Then select **General > COPYRIGHT file**. 57 | 58 | .. _stack-package-preambles: 59 | 60 | License preambles in source files 61 | ================================= 62 | 63 | The `GPL-3.0`_ license requires each source file to have a preamble comment containing a license statement. 64 | This is the generic license preamble: 65 | 66 | .. remote-code-block:: https://raw.githubusercontent.com/lsst/templates/main/file_templates/stack_license_preamble_txt/template.txt.jinja 67 | :language: jinja 68 | 69 | Replace ``{{ cookiecutter.package_name }}`` with the repository's name (``afw``, for example). 70 | 71 | This preamble is available as `a template `__. 72 | 73 | Python preamble 74 | --------------- 75 | 76 | The license preamble specifically for use in Python files is: 77 | 78 | .. remote-code-block:: https://raw.githubusercontent.com/lsst/templates/main/file_templates/stack_license_preamble_py/template.py.jinja 79 | :language: jinja 80 | 81 | Replace ``{{ cookiecutter.package_name }}`` with the repository's name (``afw``, for example). 82 | 83 | .. tip:: 84 | 85 | **Create a Python license preamble from Slack.** 86 | Open a |dmw-squarebot| and type: 87 | 88 | .. code-block:: text 89 | 90 | create file 91 | 92 | Then select **Source license preambles > Python**. 93 | 94 | See also: :ref:`style-guide-license` in the LSST DM Python Style Guide. 95 | 96 | This preamble is also available as `a template `__. 97 | 98 | C++ preamble 99 | ------------ 100 | 101 | The license preamble specifically for use in C++ source and header files is: 102 | 103 | .. remote-code-block:: https://raw.githubusercontent.com/lsst/templates/main/file_templates/stack_license_preamble_cpp/template.cc.jinja 104 | :language: jinja 105 | 106 | Replace ``{{ cookiecutter.package_name }}`` with the repository's name (``afw``, for example). 107 | 108 | .. tip:: 109 | 110 | **Create a C++ license preamble from Slack.** 111 | Open a |dmw-squarebot| and type: 112 | 113 | .. code-block:: text 114 | 115 | create file 116 | 117 | Then select **Source license preambles > C++**. 118 | 119 | This preamble is also available as `a template `__. 120 | 121 | .. _`GPL-3.0`: https://choosealicense.com/licenses/gpl-3.0/ 122 | -------------------------------------------------------------------------------- /stack/moving-to-github-actions.rst: -------------------------------------------------------------------------------- 1 | #################################### 2 | Moving to GitHub Actions from Travis 3 | #################################### 4 | 5 | GitHub Actions execute faster and more reliably than Travis jobs, so we are converting existing repositories to use them. 6 | This document describes how to accomplish this transition for existing packages. 7 | New packages should start with GitHub Actions by default; additional customizations that may be useful are suggested in :doc:`the new package documentation`. 8 | 9 | **Note:** you must have administrator rights to the repository to follow this procedure. 10 | 11 | Travis uses a ``.travis.yml`` file that specifies the workflow to be executed when events such as pushes or pull requests occur for the repository. 12 | GitHub Actions use ``.github/workflows/*.yaml`` files instead. 13 | These workflows execute in parallel. 14 | Migrating a package consists of replacing the ``.travis.yml`` file with one or more appropriate GitHub Actions workflows files. 15 | 16 | If the ``.travis.yml`` file contains only the default execution of ``flake8`` for Python "lint" checking, this can be done entirely in :ref:`the GitHub web interface `, or it can be done a bit more cleanly from :ref:`the command line `. 17 | 18 | If your ``.travis.yml`` does more than run ``flake8``, please consult an expert for assistance. 19 | 20 | 21 | .. _github-actions-web-interface: 22 | 23 | GitHub web interface 24 | ==================== 25 | 26 | #. Go to the repository you are transitioning at ``https://github.com/lsst/{repo}``. 27 | #. Go to the "Settings" page of the repository. 28 | This may be under the right-side "..." menu depending on your window width. 29 | #. Choose "Branches" in the left sidebar. 30 | #. Under "Branch protection rules", click "Edit" next to branch "main". 31 | #. Under "Require branches to be up to date before merging", uncheck any status checks that contain "travis-ci". 32 | #. Click "Save changes" at the bottom of the page. 33 | #. Go back to the "Code" page and select ``.travis.yml``. 34 | #. Select the "trash can" icon to delete this file. 35 | #. Add a commit comment such as "Remove Travis workflow." and choose a branch name. 36 | This branch name doesn't need to follow our ``tickets/`` convention; a name like ``u/{user}/remove-travis`` is adequate. 37 | #. Commit the change, create a pull request, and merge it. 38 | #. Go to the "Actions" page (between "Pull Requests" and "Projects" at the top of the GitHub page). 39 | #. Choose the "LSST DM Python lint Workflow" and click on "Set up this workflow". 40 | #. Choose "Start commit". 41 | #. Add a commit comment such as "Add Python lint workflow." and choose a branch name. 42 | Again, something like ``u/{user}/add-lint-workflow`` is sufficient as a branch name. 43 | #. Commit the change, create a pull request, and merge it. 44 | #. Go back to the "Settings" page, "Branches", "Branch protection rules", and "Edit" next to branch "main". 45 | #. Under "Require branches to be up to date before merging", check the new "lint" checkbox. 46 | #. Click "Save changes" at the bottom of the page. 47 | #. If you like, you may choose "Webhooks" in the left sidebar and delete the ``https://notify.travis-ci.org/`` entry, but it does no harm to leave it. 48 | 49 | 50 | .. _github-actions-command-line: 51 | 52 | Command line 53 | ============ 54 | 55 | #. Clone the repository you are transitioning from ``git@github.com:lsst/{repo}``. 56 | #. Create a branch: ``git checkout -b u/{user}/migrate-to-gha`` 57 | #. ``git rm .travis.yml`` 58 | #. ``mkdir -p .github/workflows; cd .github/workflows`` 59 | #. ``curl -LO https://raw.githubusercontent.com/lsst/.github/main/workflow-templates/lint.yaml`` 60 | #. ``git add lint.yaml`` 61 | #. ``git commit`` and add a commit message such as "Migrate from Travis to GitHub Actions." 62 | #. ``git push -u origin u/{user}/migrate-to-gha`` 63 | #. Create a pull request from your branch. 64 | #. Go to the "Settings" page of the repository. 65 | This may be under the right-side "..." menu depending on your window width. 66 | #. Choose "Branches" in the left sidebar. 67 | #. Under "Branch protection rules", click "Edit" next to branch "main". 68 | #. Under "Require branches to be up to date before merging", uncheck any status checks that contain "travis-ci". 69 | Check any other status checks, in particular ``lint``. 70 | #. Click "Save changes" at the bottom of the page. 71 | #. Merge your pull request. 72 | #. If you like, you may choose "Webhooks" in the left sidebar and delete the ``https://notify.travis-ci.org/`` entry, but it does no harm to leave it. 73 | 74 | 75 | .. _github-actions-further-reading: 76 | 77 | Further reading 78 | =============== 79 | 80 | - `LSST workflow templates `__ 81 | - `GitHub Actions `__ 82 | -------------------------------------------------------------------------------- /python/formatting.rst: -------------------------------------------------------------------------------- 1 | .. _formatting-python-code: 2 | 3 | #################################### 4 | Formatting Python Code Automatically 5 | #################################### 6 | 7 | The Python ecosystem provides two packages that can simplify automated code formatting and are used by some software products. 8 | These tools are compliant with the :ref:`Rubin style guide ` and can be adopted for any repository. 9 | The main caveat is that once this tooling is adopted for a specific repository, use of the tooling must be required for all subsequent pull requests and checked using GitHub Actions. 10 | This ensures that the code only goes through a style change for the initial adoption of the tooling. 11 | 12 | Ordering imports 13 | ================ 14 | 15 | The `isort `_ package can be used to reorder imports in your Python code such that they are in alphabetical order and distinguish core python packages from other packages. 16 | 17 | .. code-block:: bash 18 | 19 | $ isort python 20 | 21 | Formatting Python Code 22 | ====================== 23 | 24 | We recommend the use of the `black `_ command for reformatting Rubin Python code once it has been decided that a repository should be converted. 25 | Black calls itself the "uncompromising code formatter" and its entire purpose is to remove discussions of style from the debate. 26 | You can write the code however you want in your editor but when ``black`` runs on it there is no room for debate. 27 | For this reason configuration options are limited. 28 | The only configuration option that is recommended is to control the line length to meet the Rubin coding style. 29 | 30 | ``black`` is not compatible with the E203 error from Flake8. 31 | Consequently any project using ``black`` should disable E203. 32 | 33 | Both ``black`` and ``isort`` can be configured by using a file ``pyproject.toml`` in the repository root: 34 | 35 | .. literalinclude:: examples/pyproject.toml 36 | :language: toml 37 | 38 | The only configuration sets the line length and Python version, with ``isort`` configured to be compatible with ``black``. 39 | 40 | Installation 41 | ============ 42 | 43 | ``isort`` and ``black`` are both available from conda-forge and can be installed directly into a conda environment: 44 | 45 | .. code-block:: bash 46 | 47 | $ mamba install black isort 48 | 49 | Of course they can also be installed using ``pip``. 50 | They are not currently part of the default ``rubin-env`` conda environment. 51 | 52 | Once installed it is possible to configure editors to automatically apply these tools on save but care must be taken that this ability is not enabled globally. 53 | 54 | GitHub Actions 55 | ============== 56 | 57 | When a repository is migrated to ``black`` and ``isort`` a GitHub Action should be installed. 58 | A suitable Action ``py-formatting.yaml`` is included in the `LSST action templates repository `_. 59 | It may need to be adjusted if there is no ``bin.src`` directory. 60 | 61 | Using pre-commit 62 | ================ 63 | 64 | To prevent a situation where code is being pushed to a repository and then being flagged immediately because it has not been formatted, the `pre-commit `_ command can be extremely useful. 65 | This framework sets up a ``git`` pre-commit hook and will run checks whenever ``git commit`` is run. 66 | If it finds a problem it will fix the code and let you try the commit again. 67 | It can be installed directly from conda-forge. 68 | 69 | To use ``pre-commit`` the repository must be configured by including a file called ``.pre-commit-config.yaml`` in the repository root directory. 70 | The content describes which checks should be applied to every commit. 71 | The one in ``daf_butler`` looks like: 72 | 73 | .. literalinclude:: examples/pre-commit-config.yaml 74 | :linenos: 75 | :language: yaml 76 | 77 | This example runs ``black`` and ``isort`` to check the formatting, removes trailing whitespace, lints any YAML files, and also ensures each file ends in a new line. 78 | 79 | The checks can be enabled with: 80 | 81 | .. code-block:: bash 82 | 83 | $ pre-commit install 84 | pre-commit installed at .git/hooks/pre-commit 85 | 86 | ``pre-commit`` installs everything it needs and ensures that the versions of the tools match the versions in the repository configuration file. 87 | There is no need to install ``black`` or ``isort`` if you are relying on the pre-commit hook (which can be forced to run manually). 88 | Running ``git commit`` looks something like this if there is a problem: 89 | 90 | .. code-block:: bash 91 | 92 | $ git commit 93 | Check Yaml...........................................(no files to check)Skipped 94 | Fix End of Files.........................................................Passed 95 | Trim Trailing Whitespace.................................................Passed 96 | black....................................................................Failed 97 | - hook id: black 98 | - files were modified by this hook 99 | 100 | reformatted python/lsst/daf/butler/core/utils.py 101 | 102 | All done! ✨ 🍰 ✨ 103 | 1 file reformatted. 104 | 105 | isort (python)...........................................................Passed 106 | 107 | In this situation the reformatted file will be present in the repository and ``git diff`` will show the change. 108 | -------------------------------------------------------------------------------- /stack/generic-guide-topic-type.rst: -------------------------------------------------------------------------------- 1 | .. _generic-guide-topic: 2 | 3 | ######################## 4 | Generic guide topic type 5 | ######################## 6 | 7 | The generic guide topic type is the template to use when writing guides that are not covered by more specific topic types. 8 | 9 | .. note:: 10 | 11 | As topic types are introduced for how-to guides, tutorials, conceptual overviews, tasks, and so on, we should migrate topics written with the generic guide topic type to those more specific topic types. 12 | 13 | .. _generic-guide-topic-filename: 14 | 15 | File name and location 16 | ====================== 17 | 18 | Topics are reStructuredText files with an ``rst`` extension. 19 | 20 | If the topic is associated with a module, the topic’s file is located in the :ref:`module documentation directory `. 21 | For example: 22 | 23 | .. code-block:: text 24 | 25 | doc/lsst.example/my-topic.rst 26 | 27 | If the topic is associated with a package, the topic’s file is located in the :ref:`package documentation directory `. 28 | For example: 29 | 30 | .. code-block:: text 31 | 32 | doc/example/my-topic.rst 33 | 34 | In general, you shouldn’t create additional subdirectories to organize these files so that URLs are clean and simple. 35 | 36 | Use these guidelines for naming the file: 37 | 38 | - Name the file by adapting the title into a short phrase. 39 | 40 | - Connect words with a single hyphen (``-``) character. 41 | You may only use periods (``.``) and underscores (``_``) when the file name includes an API that naturally contains periods and underscores. 42 | 43 | - Use lowercase, except for API names that naturally include uppercase characters. 44 | 45 | .. _generic-guide-topic-preamble: 46 | 47 | Preamble 48 | ======== 49 | 50 | If a topic is located in a :ref:`module documentation directory `, you should add a ``py:currentmodule`` directive at the top of the file. 51 | For example, if the module’s name is ``lsst.example``, the directive is written as: 52 | 53 | .. code-block:: rst 54 | 55 | .. py:currentmodule:: lsst.example 56 | 57 | This allows you to reference APIs relative to that base namespace. 58 | For example these two Python API cross-references are equivalent: 59 | 60 | .. code-block:: rst 61 | 62 | `MyClass` and `lsst.example.MyClass`. 63 | 64 | If a topic is located in a :ref:`package documentation directory `, you generally shouldn’t use the ``py:currentmodule`` directive. 65 | 66 | .. _generic-guide-topic-title: 67 | 68 | Title 69 | ===== 70 | 71 | Title the topic according to these guidelines: 72 | 73 | - Use sentence case. 74 | The first word is capitalized and following words are not, unless they are proper nouns or API names with capital letters. 75 | 76 | - Ensure that the title is descriptive so that the readers know what to expect from the page's content. 77 | Titles are used in page lists (``toctree``\ s) and inline links (``doc`` and ``ref`` roles), so the title needs to stand on its own. 78 | Don't rely on document hierarchy or the previous/next pages to provide any context to the title. 79 | If the topic is about an API, be sure to name that API in the title. 80 | 81 | - If there are several related topics, using a similar style and structure for naming each of those topics. 82 | 83 | .. _generic-guide-topic-context: 84 | 85 | Context paragraph 86 | ================= 87 | 88 | Directly after the title, include one or two paragraphs that establishes the topic and its context. 89 | You should describe what the reader will learn from the page. 90 | You can also link to related topics. 91 | 92 | Readers will use this context paragraph to navigate through the documentation. 93 | A usable context paragraph will help the reader quickly establish whether the topic is relevant to their task, and if not, to navigate to other closely-related topics. 94 | 95 | .. _generic-guide-topic-sections: 96 | 97 | Sections 98 | ======== 99 | 100 | Use sections liberally to make the page easier to skim. 101 | Remember that few people read a page from top to bottom. 102 | Section headlines provide entry points into the content. 103 | 104 | Readers also use section headlines to assess whether the page includes content relevant to their task. 105 | Design your sections and headers so that readers don’t accidentally skip over your page. 106 | 107 | Use sentence case for all section headlines. 108 | In general, you should add cross-reference targets to all your section headlines. 109 | See :ref:`rst-sectioning` for example of sections with cross-reference targets. 110 | 111 | Don’t create hierarchies that are deeper than two levels (that is, subsections but not sub-subsections), if possible. 112 | Flatter hierarchies are easier for a reader to keep track of and navigate. 113 | 114 | .. _generic-guide-topic-further-reading: 115 | 116 | Further reading section 117 | ======================= 118 | 119 | At the bottom of the page, consider adding a section called “Further reading.” 120 | In this section, you can add a list of links to other pages that are related to the current page. 121 | Generally you can use the ``doc`` role to create a link to a page that automatically populates the linked page’s title. 122 | For example: 123 | 124 | .. code-block:: rst 125 | 126 | Further reading 127 | =============== 128 | 129 | - :doc:`page-a` 130 | - :doc:`page-b` 131 | - :doc:`/modules/lsst.pipe.base/absolute-link` 132 | -------------------------------------------------------------------------------- /stack/building-pipelines-lsst-io-locally.rst: -------------------------------------------------------------------------------- 1 | .. _local-pipelines-lsst-io-build: 2 | 3 | ########################################### 4 | Building the pipelines.lsst.io site locally 5 | ########################################### 6 | 7 | You can build the full `pipelines.lsst.io`_ site on your own computer. 8 | Although this is slightly more complicated than :doc:`building documentation for a single package `, it’s the best way to ensure that cross-package links work properly. 9 | 10 | Alternatively, you can also build and publish a development version of `pipelines.lsst.io`_ with the `sqre/infrastructure/documenteer`_ Jenkins CI job. 11 | That Jenkins job only accepts development branches of the `pipelines_lsst_io`_ repository --- not development branches of packages. 12 | The method described on this page is currently the only way to build documentation for development branches of packages with the full LSST Science Pipelines stack. 13 | 14 | .. _local-pipelines-lsst-io-build-prereqs: 15 | 16 | Prerequisites 17 | ============= 18 | 19 | Before starting, you’ll need a working ``lsst_distrib`` installation. 20 | 21 | This installation needs to be a recent daily or weekly build so that any in-development packages will compile with the Stack. Working from the tip of the ``main`` branch is the norm for LSST software development. 22 | 23 | The documentation build uses Documenteer_ and related Sphinx_ documentation packages, which are already installed if you are using the Rubin Conda developer environment. 24 | You can check if ``rubin-env-developer`` is installed by running ``mamba list rubin-env``. 25 | If not, you can install the developer environment with: ``mamba install rubin-env-developer``. 26 | 27 | .. _local-pipelines-lsst-io-build-clone: 28 | 29 | Clone and set up the pipelines\_lsst\_io repository 30 | =================================================== 31 | 32 | `pipelines_lsst_io`_ is the main documentation repository for the `pipelines.lsst.io`_ site. 33 | It contains project-wide content, like installation guides and release notes, and also provides the structure for gathering documentation content from individual packages in the LSST Science Pipelines package stack. 34 | 35 | Clone the repository: 36 | 37 | .. code-block:: bash 38 | 39 | git clone https://github.com/lsst/pipelines_lsst_io 40 | 41 | Then set up the `pipelines_lsst_io`_ package with EUPS: 42 | 43 | .. code:: bash 44 | 45 | setup -k -r pipelines_lsst_io 46 | 47 | .. _local-pipelines-lsst-io-build-build: 48 | 49 | Building the pipelines\_lsst\_io site 50 | ===================================== 51 | 52 | From the :file:`pipelines_lsst_io` directory, use the `stack-docs command-line app`_ from Documenteer_ to build the documentation: 53 | 54 | .. code-block:: bash 55 | 56 | stack-docs build 57 | 58 | The built site is located in the :file:`_build/html` directory. 59 | 60 | .. _local-pipelines-lsst-io-build-clean: 61 | 62 | Cleaning up built documentation 63 | =============================== 64 | 65 | You can clean up the built documentation and intermediate artifacts by running: 66 | 67 | .. code-block:: bash 68 | 69 | stack-docs clean 70 | 71 | Cleaning up the build is useful if you need to force a rebuild of the documentation either because a previous build failed, or a docstring changed. 72 | Sphinx does not automatically invalidate its cache when docstrings change. 73 | 74 | .. _local-pipelines-lsst-io-build-package-setup: 75 | 76 | Adding a locally-developed package to the pipelines_lsst\_io build 77 | ================================================================== 78 | 79 | The `pipelines_lsst_io`_ build works by symlinking the :doc:`doc/ directory ` contents of packages that are set up by EUPS. 80 | This means that by setting up a package, you can add it to your local `pipelines_lsst_io`_ build. 81 | 82 | For this tutorial, you’ll use the `pipe_base`_ package as an example. 83 | 84 | First, move out of the :file:`pipelines_lsst_io` directory and clone `pipe_base`_: 85 | 86 | .. code-block:: bash 87 | 88 | cd .. 89 | git clone https://github.com/lsst/pipe_base 90 | 91 | Then set up and compile `pipe_base`_, while keeping other packages set up (the ``-k`` option): 92 | 93 | .. code-block:: bash 94 | 95 | cd pipe_base 96 | setup -k -r . 97 | scons 98 | 99 | Then clean and build the `pipelines_lsst_io`_ documentation: 100 | 101 | .. code-block:: bash 102 | 103 | stack-docs -d ../pipelines_lsst_io clean 104 | stack-docs -d ../pipelines_lsst_io build 105 | 106 | Further reading 107 | =============== 108 | 109 | - `Documentation for the stack-docs command in Documenteer`_ 110 | - Alternative ways to build documentation: 111 | 112 | - :doc:`building-single-package-docs` 113 | - :ref:`Building pipelines.lsst.io with Jenkins CI ` 114 | 115 | .. _`Documenteer`: https://documenteer.lsst.io 116 | .. _`Documentation for the stack-docs command in Documenteer`: 117 | .. _`stack-docs command-line app`: https://documenteer.lsst.io/pipelines/stack-docs-cli.html 118 | .. _`sqre/infrastructure/documenteer`: https://rubin-ci.slac.stanford.edu/blue/organizations/jenkins/sqre%2Finfrastructure%2Fdocumenteer/activity 119 | .. _`pipelines.lsst.io`: https://pipelines.lsst.io 120 | .. _`pipelines_lsst_io`: https://github.com/lsst/pipelines_lsst_io 121 | .. _`pipe_base`: https://github.com/lsst/pipe_base 122 | .. _`graphviz`: https://graphviz.org 123 | .. _`Sphinx`: https://www.sphinx-doc.org/en/master/ 124 | -------------------------------------------------------------------------------- /stack/argparse-script-topic-type.rst: -------------------------------------------------------------------------------- 1 | .. _argparse-script-topic-type: 2 | 3 | ################################ 4 | Argparse-based script topic type 5 | ################################ 6 | 7 | The argparse-based script topic type helps you create reference documentation for ad hoc Python scripts that are packaged with the LSST Stack and use `argparse` to create their command-line interfaces. 8 | This topic type is a specialization of the :doc:`script-topic-type` that lets you work more efficiently by automatically creating documentation content from your script's `argparse.ArgumentParser` instance. 9 | 10 | .. seealso:: 11 | 12 | If the script isn't implemented in Python with `argparse`, refer to the more generic :doc:`script-topic-type` instead. 13 | 14 | If the script is actually a command-line task, document it with a :doc:`task topic type ` instead. 15 | 16 | .. _argparse-script-topic-type-template: 17 | 18 | Starter template 19 | ================ 20 | 21 | Create a new argparse-based script topic from Slack.\ [#template]_ 22 | Open a |dmw-squarebot| and type: 23 | 24 | .. code-block:: text 25 | 26 | create file 27 | 28 | Then select **Science Pipelines documentation > Script topic (argparse)**. 29 | 30 | .. [#template] The argparse-based script topic template is maintained in the `lsst/templates repository`_. 31 | 32 | .. _lsst/templates repository: https://github.com/lsst/templates/tree/main/file_templates/argparse_script_topic 33 | 34 | For an example script named ``exampleScript.py``, the rendered template looks like this: 35 | 36 | .. remote-code-block:: https://raw.githubusercontent.com/lsst/templates/main/file_templates/argparse_script_topic/exampleScript.py.rst 37 | :language: rst 38 | 39 | The next sections describe the key components of argparse-based script topics. 40 | 41 | .. _argparse-script-topic-type-filename: 42 | 43 | File name and location 44 | ====================== 45 | 46 | The file must be named after the command-line executable, including any extension. 47 | For example, if the script is named ``myScript.py``, the topic's file should be named ``myScript.py.rst``. 48 | 49 | The file should be located in the :file:`scripts/` subdirectory of the :ref:`module documentation directory ` within a package. 50 | 51 | For example, if the module’s namespace is ``lsst.example``, the full path for the file within the package repository is :file:`doc/lsst.example/scripts/myScript.py.rst`. 52 | 53 | .. _argparse-script-topic-autoprogram: 54 | 55 | The autoprogram directive 56 | ========================= 57 | 58 | Everything in an argparse-based script topic is generated from a single autoprogram_ directive, which is provided by the `sphinxcontrib.autoprogram`_ Sphinx extension. 59 | autoprogram_ generates all the content that is described by the :doc:`script topic type `, including the program_ and option_ directives for cross-referencing. 60 | 61 | To use the autoprogram_ directive, your script needs to be set up in a particular fashion: 62 | 63 | - The script needs to be in the ``bin.src`` directory of a package, but the script file must defer its implementation to a module *inside* the package's Python modules. 64 | 65 | For example, a script file :file:`bin.src/exampleScript.py` might be structured like this: 66 | 67 | .. code-block:: python 68 | 69 | #! /usr/bin/env python 70 | 71 | from lsst.example.scripts.exampleScript import main 72 | 73 | 74 | if __name__ == "__main__": 75 | main() 76 | 77 | - The `argparse.ArgumentParser` instance must be generated by an argument-less function. 78 | This is critical for letting the autoprogram_ directive get a copy of the `~argparse.ArgumentParser` instance to introspect the command-line interface: 79 | 80 | .. code-block:: python 81 | 82 | """Example script that is automatically documented. 83 | """ 84 | 85 | from argparse import ArgumentParser 86 | 87 | 88 | def build_argparser(): 89 | parser = ArgumentParser( 90 | description=__doc__, 91 | formatter_class=argparse.RawDescriptionHelpFormatter, 92 | epilog='More information is available at https://pipelines.lsst.io.') 93 | ) 94 | parser.add_argument( 95 | 'message' 96 | help='Message to echo.' 97 | ) 98 | return parser 99 | 100 | 101 | def main(): 102 | args = build_argparser().parse_args() 103 | print(args.message) 104 | 105 | In this example, the ``build_argparser()`` function generates the `~argparse.ArgumentParser` instance. 106 | 107 | The argument to the autoprogram_ directive points to the function in your script's implementation that generates the `~argparse.ArgumentParser` instance. 108 | The argument is formatted as ``{module}:{function}()``, with a colon (``:``) separating the module from the function name. 109 | Given the above example, the autoprogram_ directive is written as: 110 | 111 | .. code-block:: rst 112 | 113 | .. autoprogram:: lsst.example.scripts.exampleScript:build_argparser() 114 | :prog: exampleScript.py 115 | :groups: 116 | 117 | Set the ``:prog:`` field to the name of the command-line executable. 118 | If you already set the ``prog`` argument in `argparse.ArgumentParser`, this field is not necessary. 119 | 120 | Also, the ``:groups:`` field ensures that documentation is organized around argument groups (see `argparse.ArgumentParser.add_argument_group`), if they exist. 121 | 122 | For more information about the autoprogram_ directive, refer to the sphinxcontrib.autoprogram_ documentation. 123 | 124 | .. _autoprogram: 125 | .. _sphinxcontrib.autoprogram: https://sphinxcontrib-autoprogram.readthedocs.io/en/stable/ 126 | .. _program: http://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#directive-program 127 | .. _option: http://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#directive-option 128 | -------------------------------------------------------------------------------- /legal/licensing-overview.rst: -------------------------------------------------------------------------------- 1 | ######################################### 2 | Licensing LSST DM source code and content 3 | ######################################### 4 | 5 | This page provides background information to help you appropriately license source code, documentation, and other types of content that is produced on behalf of LSST Data Management. 6 | 7 | .. _choose-a-source-license: 8 | 9 | Choosing a source code license 10 | ============================== 11 | 12 | All source code created by LSST DM is publicly-available open source. 13 | As such, LSST DM code must carry an `Open Source Initiative (OSI)-approved license`_. 14 | 15 | Stack packages 16 | -------------- 17 | 18 | A substantial amount of DM development is in the EUPS-managed "Stack," including the LSST Science Pipelines, Qserv, and DAX. 19 | Stack packages are licensed under the `GNU Public License version 3.0 (GPL-3.0)`_ license. 20 | Because of the interconnected nature of the Stack and how the `GPL-3.0`_ license works, you cannot create a new Stack package distributed with EUPS that is licensed under a different license. 21 | 22 | .. seealso:: 23 | 24 | :doc:`/stack/license-and-copyright`. 25 | 26 | Standalone projects 27 | ------------------- 28 | 29 | If your project is not part of the Stack, you have flexibility to choose a different `OSI-approved license`_. 30 | 31 | For example, PyPI-distributed Python packages made by SQuaRE are licensed under the simpler and more-flexible MIT_ and `Apache 2.0`_ licenses. 32 | 33 | To choose a license, you'll want to consider the norms of the open source community your project resides in. 34 | The `choosealicense`_ site is useful for understanding the important qualities of different open source licenses. 35 | 36 | Always consult with your manager before setting the license for a project to ensure it aligns with the project's goals. 37 | 38 | .. _choose-a-doc-license: 39 | 40 | Choosing a documentation license 41 | ================================ 42 | 43 | Documentation is often licensed differently from source code to make it easier to adapt and reuse in non-code contexts. 44 | 45 | In general, all DM user documentation and technical notes are licensed under the `Creative Commons Attribution 4.0 International (CC-BY-4.0)`_. 46 | This license balances the need for LSST DM to get attribution for content, while allowing the community to freely reproduce and adapt the information. 47 | 48 | .. _apply-a-license: 49 | 50 | Applying a license to a repository 51 | ================================== 52 | 53 | When you create a new repository on GitHub you have the option of adding a :file:`LICENSE` in the initial set up. 54 | You can also add a license later `through the GitHub UI `__. 55 | Not all OSI-approved licenses are available through the GitHub UI, though. 56 | 57 | Alternatively, you can always apply a license by manually creating and committing a :file:`LICENSE` file in the root of the source code repository. 58 | The content of the :file:`LICENSE` file should be the license text itself, without additions or alterations. 59 | 60 | The easiest way to get the content of a license is by going to choosealicense_, finding the license's page, and clicking the **Copy license text to clipboard** button. 61 | 62 | Some licenses include a copyright section. 63 | See :doc:`copyright-overview` for details on how to properly record copyrights for DM software. 64 | 65 | The package management ecosystems for many languages, including PyPI and NPM, provide metadata fields for recording license information. 66 | These aren't legally binding, but you should make sure the package metadata are consistent with the :file:`LICENSE` file. 67 | 68 | GitHub also includes license metadata for repositories that is determined automatically from the content of the :file:`LICENSE` file. 69 | See :ref:`github-license-metadata`, below, for more information. 70 | 71 | .. seealso:: 72 | 73 | The `GPL-3.0`_ license used by Stack packages also require preambles in each source code file. 74 | See :doc:`/stack/license-and-copyright` for details. 75 | 76 | .. _github-license-metadata: 77 | 78 | Details on GitHub's license detection 79 | ===================================== 80 | 81 | GitHub can detect a repository's license by matching the content of the :file:`LICENSE` file to known licenses in the choosealicense_ corpus. 82 | When GitHub confidently detects a license, it displays the license on the repository's page. 83 | 84 | Not only is this license badge a nice feature for the community, it also helps us validate our :file:`LICENSE` files to be sure that the :file:`LICENSE` we publish is in fact the license we think it is. 85 | If the :file:`LICENSE` file is modified, aside from copyright lines, GitHub will not positively identify the license and will not show a license badge on the repository homepage. 86 | 87 | If you have a repository where GitHub is not detecting a license, you can debug it by running GitHub's detection software on your own computer. 88 | Install licensee_ and follow the `documentation `__ to run it against your repository. 89 | 90 | .. note:: 91 | 92 | licensee_ looks at `multiple files, including COPYRIGHT `__, when it detects a license. 93 | If these files have conficting information, GitHub will not positively detect a license. 94 | Be aware of this issue when working with repositories that have both :file:`COPYRIGHT` and :file:`LICENSE` files. 95 | 96 | .. note:: 97 | 98 | GitHub may not properly detect the :file:`LICENSE` in repositories that have multi-institution :file:`COPYRIGHT` files. 99 | SQuaRE is aware of this issue and is working to resolve it. 100 | See licensee `issue #285 `__ for background. 101 | 102 | .. _`OSI-approved license`: 103 | .. _`Open Source Initiative (OSI)-approved license`: https://opensource.org/licenses 104 | .. _`GPL-3.0`: 105 | .. _`GNU Public License version 3.0 (GPL-3.0)`: https://choosealicense.com/licenses/gpl-3.0/ 106 | .. _`Creative Commons Attribution 4.0 International (CC-BY-4.0)`: https://choosealicense.com/licenses/cc-by-4.0/ 107 | .. _`MIT`: https://choosealicense.com/licenses/mit/ 108 | .. _`Apache 2.0`: https://choosealicense.com/licenses/apache-2.0/ 109 | .. _choosealicense: https://choosealicense.com/ 110 | .. _licensee: https://github.com/benbalter/licensee 111 | -------------------------------------------------------------------------------- /javascript/jsdoc.rst: -------------------------------------------------------------------------------- 1 | ######################################## 2 | Documenting JavaScript APIs with JSDoc 3 | ######################################## 4 | 5 | 6 | 7 | .. _jsdoc-useful-tags: 8 | 9 | Frequently used tags 10 | ^^^^^^^^^^^^^^^^^^^^ 11 | 12 | @desc 13 | This tag provides a general description of the symbol or method. 14 | 15 | @func [] 16 | This tag marks an object as being a function with a given name. 17 | 18 | @ignore 19 | The ignore tag indicates that a symbol in your code should never 20 | appear in the documentation. This tag takes precedence over all 21 | other tags. 22 | 23 | For most JSDoc templates, including the default template, the @ignore tag has the following effects: 24 | 25 | - If you use the @ignore tag with the @class or @module tag, the entire class or module will be omitted from the documentation. 26 | - If you use the @ignore tag with the @namespace tag, you must also add the @ignore tag to any child classes and namespaces. Otherwise, your documentation will show the child classes and namespaces, but with incomplete names. 27 | 28 | @memberof module: 29 | 30 | @memberof 31 | This tag tells a member symbol that belongs to a parent symbol. 32 | 33 | @module [] 34 | This tag marks the current file as being its own module. All symbols in the file are assumed to be members of the module unless documented otherwise. 35 | 36 | @namespace 37 | This tag indicates that an object creates a namespace for its 38 | members. You can also write a virtual JSDoc comment that defines a namespace used by your code. 39 | 40 | Note: You can not define more than one namespace in one jsdoc block like this: 41 | 42 | .. code-block:: js 43 | 44 | /** 45 | 46 | *@namespace firefly 47 | 48 | *@namespace firefly.util 49 | 50 | *@namespace firefly.action 51 | 52 | */ 53 | 54 | The jsdoc will only document firefly.action, the last one in the 55 | block. To define more than one namespaces in one file you need to: 56 | 57 | .. code-block:: js 58 | 59 | /**@namespace firefly*/ 60 | 61 | /**@namespace firefly.action*/ 62 | 63 | /**@namespace firefly.ui*/ 64 | 65 | @param 66 | This tag requires you to specify the name of the parameter you are 67 | documenting. You can also include the parameter's type, enclosed in 68 | curly brackets, and a description of the parameter. You can use \| 69 | to specify more than one type such as {type1\|type2\|type3}. 70 | 71 | @static 72 | This tag indicates that a symbol is contained within a parent and 73 | can be accessed without instantiating the parent. 74 | 75 | @summary 76 | This tag is a shorter version of the full description. It can be 77 | added to any doclet. 78 | 79 | @typedef [] 80 | This tag is useful for documenting custom types, particularly if you 81 | wish to refer to them repeatedly. These types can then be used 82 | within other tags expecting a type, such 83 | as \@type or \@param. 84 | 85 | NOTE: I used @global in each @typedef object. Thus, the typedef 86 | object can be referred throughout the documentation. 87 | 88 | When to use @func 89 | ^^^^^^^^^^^^^^^^^^^^ 90 | 91 | By default, the jsdoc documents all the exported members by 92 | ``export.functionName``. To document them as static members of a namespace, 93 | you use @func . For methods and constants are defined 94 | without export, you don’t need to use @func . **Note: The 95 | behavior in namespace is opposite to module.** 96 | 97 | When to use @memberof 98 | ^^^^^^^^^^^^^^^^^^^^^^ 99 | 100 | For methods or constants, you have to use @memberof to 101 | group them under the . Without @memberof tag, all the methods 102 | and constants will appear under the global category. 103 | 104 | Known issues: 105 | ^^^^^^^^^^^^^^^^^^^^^^ 106 | 107 | 1. There is more than one way to produce the same output using the tags. 108 | 109 | 2. Depending on the templates, the tags may not work exactly as they are 110 | described. The default template came with JSDoc is located in the 111 | jsdoc directory 112 | 113 | 3. Note: the following two tags worked for minami template. But the 114 | docstrap template seems not recognizing them correctly. 115 | 116 | @private 117 | 118 | The @private tag marks a symbol as private, or not meant for general 119 | use. Private members are not shown in the generated output unless 120 | JSDoc is run with the -a/--private command-line option or specifying 121 | the option in the configuration. The @private tag is equivalent 122 | to @access private. 123 | 124 | @public 125 | 126 | The @public tag indicates that a symbol should be documented as if 127 | it were public. By default, JSDoc treats all symbols as public, so 128 | using this tag does not normally affect the generated documentation. 129 | However, you may prefer to use the @public tag explicitly so it is 130 | clear to others that you intended to make the symbol public. 131 | However, @public does not change the symbol’s scope. The @public tag 132 | is the same as @access public. 133 | 134 | 135 | Start from scratch: 136 | ^^^^^^^^^^^^^^^^^^^^^^ 137 | 138 | 1. Add the dependent libraries to package.json 139 | 140 | .. code-block:: text 141 | 142 | "eslint-plugin-jsx-a11y": "^0.6.2", 143 | 144 | "jsdoc-jsx": "^0.1.0", 145 | 146 | "ink-docstrap": "^1.2.1" 147 | 148 | 2. Build your libraries, for example, 149 | 150 | .. code-block:: bash 151 | 152 | gradle :firefly:war 153 | 154 | gradle :firefly:deploy 155 | 156 | 3. Use jsdoc\_config.json located in firefly/src/firefly directory or create your own configuration file. 157 | 158 | 4. Add proper tags in each source file. 159 | 160 | 5. JSDoc comments should generally be placed immediately before the code being documented. It must start with a /\*\* sequence in order to be recognized by the JSDoc parser. 161 | 162 | 6. Generate the javascript documentation at /hydra/cm/firefly directory 163 | by running: 164 | 165 | .. code-block:: bash 166 | 167 | ./node_modules/.bin/jsdoc –c path/to/your-configuration-file 168 | 169 | 170 | (Note. Use any command line options to override the options defined in the configuration file.) 171 | 172 | 7. To check the available command line options: 173 | 174 | .. code-block:: bash 175 | 176 | ./node_modules/.bin/jsdoc -h 177 | 178 | References: 179 | ^^^^^^^^^^^^^^^^^^^^^^ 180 | 181 | - http://usejsdoc.org/ 182 | 183 | - https://github.com/jsdoc3/jsdoc/blob/master/README.md 184 | --------------------------------------------------------------------------------