5 |
--------------------------------------------------------------------------------
/.github/workflows/requirements.txt:
--------------------------------------------------------------------------------
1 | sphinx==8.2.3
2 | pydata-sphinx-theme==0.16.1
3 | sphinx-design==0.6.1
4 | sphinx-copybutton==0.5.2
5 | sphinx-lint==1.0.2
6 | sphinx-sitemap==2.9.0
7 | sphinxcontrib-svg2pdfconverter==1.3.0
8 | sphinxcontrib-mermaid==1.1.0
9 |
--------------------------------------------------------------------------------
/source/FurtherDetails/change_notes.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _changes:
8 |
9 | Recent Changes
10 | ==============
11 |
--------------------------------------------------------------------------------
/CONTRIBUTING.md:
--------------------------------------------------------------------------------
1 | # Code of Conduct
2 |
3 | This code of conduct is expected to be adhered to by all developers
4 | and users of Met Office Simulation Systems.
5 |
6 | 1. Be kind and respectful to others how they may be different from you.
7 | 2. Be tolerant of others but do not abuse the tolerance of others.
8 | 3. Respond to others constructively and in a reasonable time frame.
9 | 4. Communicate inclusively and contribute to the community.
10 | 5. Focus on outcomes, not processes.
11 |
--------------------------------------------------------------------------------
/pyproject.toml:
--------------------------------------------------------------------------------
1 | [project]
2 | name = "simulation-systems"
3 | version = "0.1.0"
4 | description = "Simulation Systems Working Practices"
5 | readme = "README.md"
6 | requires-python = ">=3.11"
7 | dependencies = [
8 | "pydata-sphinx-theme==0.16.1",
9 | "sphinx==8.2.3",
10 | "sphinx-design==0.6.1",
11 | "sphinx-copybutton==0.5.2",
12 | "sphinx-lint==1.0.2",
13 | "sphinx-sitemap==2.9.0",
14 | "sphinxcontrib-svg2pdfconverter==1.3.0",
15 | "sphinxcontrib-mermaid==1.1.0",
16 | ]
17 |
--------------------------------------------------------------------------------
/source/_templates/crown-copyright.html:
--------------------------------------------------------------------------------
1 | {# Displays the copyright information (which is defined in conf.py). #}
2 | {% if show_copyright and copyright %}
3 |
12 | {% endif %}
13 |
--------------------------------------------------------------------------------
/source/FurtherDetails/index.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _further_details:
8 |
9 | Further Details
10 | ===============
11 |
12 | .. toctree::
13 | :maxdepth: 1
14 | :caption: Further Details
15 |
16 | who
17 | support
18 | glossary
19 | code_of_conduct
20 | contributing
21 | ai
22 | dos_donts
23 | change_notes
24 | coding_style
25 |
--------------------------------------------------------------------------------
/source/_static/custom.css:
--------------------------------------------------------------------------------
1 | /* import the standard theme css */
2 | @import url("styles/theme.css");
3 | @import url("basic.css");
4 |
5 | /* Office Science Colours */
6 | html[data-theme="light"] {
7 | --pst-color-primary: #0f79be;
8 | --pst-color-secondary: #e2a022;
9 | --pst-color-accent: #e2a022;
10 | }
11 |
12 | html[data-theme="dark"] {
13 | --pst-color-primary: #359bc0;
14 | --pst-color-secondary: #eac45f;
15 | --pst-color-accent: #eac45f;
16 |
17 | /* Overwrite visited colour to be AAA contrast compliant */
18 | a:visited {
19 | color: #be90ea;
20 | }
21 |
22 | /* Reset the overridden hover colour */
23 | a:visited:hover {
24 | color: var(--pst-color-link-hover);
25 | }
26 | }
27 |
--------------------------------------------------------------------------------
/source/FurtherDetails/code_of_conduct.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _code_of_conduct:
8 |
9 | Code of Conduct
10 | ===============
11 |
12 | This code of conduct is expected to be adhered to by all developers and users
13 | of Met Office Simulation Systems.
14 |
15 | 1. Be kind and respectful to others how they may be different from you.
16 | 2. Be tolerant of others but do not abuse the tolerance of others.
17 | 3. Respond to others constructively and in a reasonable time frame.
18 | 4. Communicate inclusively and contribute to the community.
19 | 5. Focus on outcomes, not processes.
20 |
--------------------------------------------------------------------------------
/make.bat:
--------------------------------------------------------------------------------
1 | @ECHO OFF
2 |
3 | pushd %~dp0
4 |
5 | REM Command file for Sphinx documentation
6 |
7 | if "%SPHINXBUILD%" == "" (
8 | set SPHINXBUILD=sphinx-build
9 | )
10 | set SOURCEDIR=source
11 | set BUILDDIR=build
12 |
13 | if "%1" == "" goto help
14 |
15 | %SPHINXBUILD% >NUL 2>NUL
16 | if errorlevel 9009 (
17 | echo.
18 | echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
19 | echo.installed, then set the SPHINXBUILD environment variable to point
20 | echo.to the full path of the 'sphinx-build' executable. Alternatively you
21 | echo.may add the Sphinx directory to PATH.
22 | echo.
23 | echo.If you don't have Sphinx installed, grab it from
24 | echo.https://www.sphinx-doc.org/
25 | exit /b 1
26 | )
27 |
28 | %SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
29 | goto end
30 |
31 | :help
32 | %SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
33 |
34 | :end
35 | popd
36 |
--------------------------------------------------------------------------------
/source/Reviewers/releases/software_stack.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _software_stack:
8 |
9 | Scientific Software Stack Update
10 | ================================
11 |
12 | * The scientific software stack used for the UM rose-stem is in
13 | ``rose-stem/site/meto/variables.cylc`` file and in ``admin/search_stash.sh``
14 | * Decide whether this needs updating - using the latest production scitools
15 | environment is recommended.
16 | * If you decide to update then complete with an associated PR.
17 |
18 | .. note::
19 |
20 | This process will want revisiting if the UM moves towards a spack software
21 | stack, similar to LFRic
22 |
--------------------------------------------------------------------------------
/source/Reviewers/releases/stash_browser.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _stash_browser:
8 |
9 | Stash Browser
10 | =============
11 |
12 | * Get a copy of the UM at release and from the top level run,
13 |
14 | .. code-block::
15 |
16 | ./admin/stashbrowser/stashweb -i rose-meta/um-atmos/HEAD/etc/stash/STASHmaster/STASHmaster_A -o UMXXY_STASHweb
17 |
18 | substituting the version for ``XXY``.
19 | * Login as the UM shared account
20 | * Copy the ``UMXXY_STASHweb`` directory created earlier to
21 | ``$UMDIR/public_html/frozen/XX.Y/stash_browse``
22 | * Check everything looks right at https://wwwspice/~umadmin/stashtech.html
23 |
24 |
25 |
--------------------------------------------------------------------------------
/Makefile:
--------------------------------------------------------------------------------
1 | # Minimal makefile for Sphinx documentation
2 | #
3 |
4 | # You can set these variables from the command line, and also
5 | # from the environment for the first two.
6 | SPHINXOPTS ?=
7 | SPHINXBUILD ?= sphinx-build
8 | SOURCEDIR = source
9 | BUILDDIR = build
10 |
11 | # For local deployment to ~/public_html//
12 | REPO := $(notdir $(shell git rev-parse --show-toplevel))
13 | BRANCH := $(shell git rev-parse --abbrev-ref HEAD)
14 | DEPLOYDIR = $(HOME)/public_html
15 |
16 | # Put it first so that "make" without argument is like "make help".
17 | help:
18 | @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
19 |
20 | .PHONY: help Makefile
21 |
22 | # Catch-all target: route all unknown targets to Sphinx using the new
23 | # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
24 | %: Makefile
25 | @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
26 |
27 | deploy:
28 | @$(SPHINXBUILD) -M html "$(SOURCEDIR)" "$(DEPLOYDIR)/$(REPO)/$(BRANCH)" $(SPHINXOPTS) $(O)
29 |
--------------------------------------------------------------------------------
/source/Development/TestSuites/ukca.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | Testing UKCA
8 | ============
9 |
10 | Changes in UKCA that touch `src/science` or `src/control/core` must be tested
11 | with both the UM and LFRic by following the :ref:`linked pull requests
12 | guidance `.
13 |
14 | For further guidance on testing and working with UKCA, including standard
15 | suites and box models see the `UKCA trac wiki
16 | `__.
17 |
18 | There also exists a small UKCA rose-stem suite, which contains a code styling
19 | check. This can launched from the top directory of a local clone by running,
20 |
21 | .. code-block::
22 |
23 | cylc vip -z group=scripts -n ./rose-stem
24 |
--------------------------------------------------------------------------------
/source/Development/diagnostics.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | Diagnostic Systems
8 | ==================
9 |
10 | The diagnostic systems vary between projects. Please follow the guidance
11 | for whichever system you are developing.
12 |
13 | +-------+----------------------+
14 | | UM | :ref:`STASH ` |
15 | +-------+----------------------+
16 | | LFRic | :ref:`lfric_diag` |
17 | +-------+----------------------+
18 |
19 | ..
20 | Do UKCA/SOCRATES/JULES have their own diagnostic systems and are they worth
21 | mentioning here? CASIM does not, but the MONC model which builds CASIM does;
22 | code is shared between them both.
23 |
24 |
25 | .. toctree::
26 | :hidden:
27 |
28 | Diagnostics/um_stashmaster
29 | Diagnostics/lfric_diagnostics
30 |
--------------------------------------------------------------------------------
/source/Reviewers/releases/other_repos.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _small_repo_release:
8 |
9 | Other Repositories
10 | ==================
11 |
12 | 1. Apply a Simulation Systems release tag to SimSys_Scripts.
13 |
14 | 2. Create a UKCA branch updating the SimSys_Scripts tag in the dependencies.yaml
15 | and get this reviewed and committed.
16 |
17 | 3. Follow the process of updating :ref:`Stable and Main` for
18 | these repositories:
19 |
20 | * ukca
21 | * casim
22 | * socrates
23 | * moci
24 |
25 | 4. Apply a Simulation Systems release tag to each of:
26 |
27 | * ukca
28 | * casim
29 | * socrates
30 | * moci
31 | * um_aux
32 | * socrates-spectral
33 |
34 | 5. Send an email to all repository owners to let them know that the repository has
35 | been tagged.
36 |
37 |
38 |
--------------------------------------------------------------------------------
/source/Reviewers/releases/shared_accounts.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _shared_accounts:
8 |
9 | Repo and Shared Accounts Permissions
10 | ====================================
11 |
12 | Modify SSD Team Github Permissions:
13 |
14 | * https://github.com/orgs/jules-lsm/teams/system-managers
15 | * https://github.com/orgs/MetOffice/teams/ssdteam
16 |
17 | The ssd team should propagate to all relevant repos, but also check the
18 | permissions for all of,
19 |
20 | * UM
21 | * LFRic Apps
22 | * LFRic Core
23 | * Jules
24 | * UKCA
25 | * Casim
26 | * Shumlib
27 | * MOCI
28 | * Socrates
29 | * Mule
30 | * um_meta
31 | * um_aux
32 | * um_doc
33 | * git_playground (likely renamed)
34 |
35 | Review account permissions in Active Directory for the following Shared Users:
36 |
37 | * umadmin
38 | * umtest
39 | * julesadmin
40 | * lfricadmin
41 | * spackadmin
42 |
--------------------------------------------------------------------------------
/source/Reviewers/index.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _reviewers_index:
8 |
9 | Guides for Reviewers
10 | ====================
11 |
12 | PRs will pass through a two-stage review. Linked PRs should work through this
13 | process together.
14 |
15 | The first, SciTech, review is carried out by someone with a good understanding
16 | of the area being developed. Once satisfied they then pass the PR to a
17 | Code Reviewer who takes a bigger picture and system view of the change.
18 | Trivial PRs can bypass the SciTech Review.
19 |
20 | The change then passes to the Code Reviewer and in this stage the developer is
21 | encouraged to merge ``main`` into their branch and resolve any conflicts.
22 |
23 | Once the reviews are complete the Code Reviewer is then responsible for the
24 | final merge and commit of the change to ``main``.
25 |
26 |
27 |
28 | .. toctree::
29 | :maxdepth: 2
30 | :hidden:
31 | :caption: Guides for Reviewers
32 |
33 | scitechreview
34 | codereview
35 | howtocommit
36 | curaterelease
37 |
38 |
39 |
--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
1 | BSD 3-Clause License
2 |
3 | Copyright (c) 2023, Met Office
4 | All rights reserved.
5 |
6 | Redistribution and use in source and binary forms, with or without
7 | modification, are permitted provided that the following conditions are met:
8 |
9 | 1. Redistributions of source code must retain the above copyright notice, this
10 | list of conditions and the following disclaimer.
11 |
12 | 2. Redistributions in binary form must reproduce the above copyright notice,
13 | this list of conditions and the following disclaimer in the documentation
14 | and/or other materials provided with the distribution.
15 |
16 | 3. Neither the name of the copyright holder nor the names of its
17 | contributors may be used to endorse or promote products derived from
18 | this software without specific prior written permission.
19 |
20 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
21 | AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
22 | IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
23 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
24 | FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
25 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
26 | SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
27 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
28 | OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
29 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
30 |
--------------------------------------------------------------------------------
/source/Development/TestSuites/lfric_core.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _lfric_core_test:
8 |
9 | Testing LFRic Core
10 | ==================
11 |
12 | .. note::
13 |
14 | At the LFRic Apps vn2.0 release, the cylc7 LFRic Core suite was deprecated
15 | and the make test-suite functionality removed. Only the cylc8 suite is now
16 | maintained.
17 |
18 | LFRic testing can be launched from the top directory of a local clone,
19 |
20 | .. code-block:: shell
21 |
22 | cylc vip -z group=developer -n ./rose-stem
23 |
24 | -----
25 |
26 | The minimum requirement for testing a branch that is to be sent for review is
27 | that the system level developer tests pass on all the applications. These are
28 | launched from make and utilise rose and cylc.
29 |
30 | While developing your change, for expediency you may want to run the tests for
31 | only some applications. This can be done by changing the group you run, eg
32 | ``-z group=simple_diffusion``.
33 |
34 | The command above will launch the developer suite. You can include slightly
35 | more testing if required by running ``-z group=all`` instead (this includes the
36 | developer suite).
37 |
38 | It is also possible to run on a single platform, eg. ``-z group=ex1a``. To
39 | select which meto EX machine is used, add ``-S USE_EX=true``.
40 |
--------------------------------------------------------------------------------
/source/FurtherDetails/ai.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _ai:
8 |
9 | AI Policy
10 | =========
11 |
12 | Please ensure that when using Generative AI tools, appropriate guardrails are
13 | in place and contributions have correct attribution. For Met Office
14 | contributors, this includes adhering to the Use of Generative AI policy.
15 | Contributors from other institutions should check if their institution has
16 | similar policies, and follow the local policy.
17 |
18 | Code where Generative AI tools have been used needs to have clear attribution to
19 | meet the Met Office Generative AI policy. This includes attribution in the
20 | commit message and in each modified file. Any file where a modification has been
21 | made with Generative AI assistance must have a comment immediately before the
22 | module level docstring, containing:
23 |
24 | .. code-block::
25 |
26 | # Some of the content of this file has been produced with the assistance of
27 | ."
28 |
29 | where ```` should be replaced with the specific name of
30 | the tool such as `` Github Copilot Enterprise`` (e.g. Met
31 | Office Github Copilot Enterprise), ``Github Copilot Personal``,
32 | ``ChatGPT GPT-4``, etc. For Met Office contributors, Met Office Github Copilot
33 | Enterprise is the only approved Generative AI tool.
34 |
--------------------------------------------------------------------------------
/source/Reviewers/releases/partner_testing.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _partner_testing:
8 |
9 | Partner Testing
10 | ===============
11 |
12 | This is intended to allow partners to determine if there are any undetected
13 | problems with the head of the UM ``main`` (and its dependencies) before the
14 | release process gets started in earnest.
15 |
16 | The partner testing period should be 72 hours (3 working days). All PRs
17 | affecting the main source code should be committed (to all repos) but limited
18 | scope changes, eg. to just the Met Office site, can still go on.
19 |
20 | * Open a UM issue to track partner testing, using the partner testing issue
21 | template.
22 | * Email the representatives at each UM partner site to them to test. A list of
23 | emails can be found at, `UM Partner Testing Emails
24 | `__
25 | (note, that page should remain private to the Met Office).
26 |
27 | * You may need to set the international options in advanced email sending
28 | options to use Unicode (UTF-8) as the preferred encoding for outgoing
29 | messages to ensure that all international partners receive the email.
30 | Change this setting if you receive bounced emails during partner testing.
31 |
32 |
33 | .. important::
34 |
35 | Now is also a good time to ask for the Jules release notes. This is usually
36 | done by a member of the Jules community, so email them advising them of the
37 | likely release date. Ask the team for their details if unsure.
38 |
--------------------------------------------------------------------------------
/source/Reviewers/releases/umdp_release.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _umdp_release:
8 |
9 | UMDP Release
10 | ============
11 |
12 | * Ensure you have a :ref:`fork ` of both ``um_doc`` repository, and
13 | that the ``main`` branch is up to date with the upstream repository.
14 | * In a clone of this fork, :ref:`create a branch ` using the
15 | ``main`` branch as the parent.
16 |
17 | .. important::
18 |
19 | Ensure you create branches from main, otherwise you will not include the
20 | changes from the past release.
21 |
22 | * You will need to update a few files:
23 |
24 | * For ``web/js/um-version.js`` update the version number, release name, and
25 | standard suite hashes.
26 | * For ``template/UMDP.cls``, update the version number.
27 | * For the x-series documentation, you will at minimum need to check and
28 | update:
29 |
30 | * ``source/X04/X4-fcm.tex`` Commit hash and version number
31 | * ``source/X04/X4-prerequisites.tex`` rose and cylc versions (found in the
32 | release notes).
33 | * ``source/X04/X4.tex`` UM, gcom, shumlib and UKCA versions (found in the
34 | release notes).
35 | * ``source/X06/X6.tex`` UM version number.
36 | * ``source/X10/X10.tex`` UM, JULES and UKCA version numbers, gcom branch
37 | URL.
38 |
39 | * It is also worth checking no URLs are out of date.
40 | * With a reviewer, follow the :ref:`release process `.
41 | * Ensure the action to publish the documentation has successfully completed and
42 | the `new documentation is available `_.
43 |
--------------------------------------------------------------------------------
/.github/workflows/publish_wps.yaml:
--------------------------------------------------------------------------------
1 | name: Docs
2 |
3 | on:
4 | push:
5 | branches:
6 | - main
7 | pull_request:
8 | types: [opened, reopened, synchronize]
9 | workflow_dispatch:
10 |
11 | permissions:
12 | contents: read
13 | pages: write
14 | id-token: write
15 |
16 | jobs:
17 | build-and-deploy:
18 | runs-on: ubuntu-latest
19 | timeout-minutes: 5
20 | env:
21 | PYTHON_VRSN: '3.12'
22 | VENV_PATH: '.venv'
23 |
24 | steps:
25 | - name: Checkout repository
26 | uses: actions/checkout@v6
27 |
28 | - name: Setup uv with Python ${{ env.PYTHON_VRSN }}
29 | uses: astral-sh/setup-uv@v7
30 | with:
31 | python-version: ${{ env.PYTHON_VRSN }}
32 |
33 | - name: Cache uv venv
34 | uses: actions/cache@v4
35 | with:
36 | path: ${{ env.VENV_PATH }}
37 | key: ${{ runner.os }}-uv-${{ env.PYTHON_VRSN }}-${{ hashFiles('pyproject.toml') }}
38 | restore-keys: |
39 | ${{ runner.os }}-uv-${{ env.PYTHON_VRSN }}-
40 |
41 | - name: Install dependencies
42 | run: uv sync
43 |
44 | - name: Lint Sphinx docs
45 | run: uv run sphinx-lint source
46 |
47 | - name: Build HTML docs
48 | run: uv run make clean html
49 |
50 | - name: Minimize uv cache
51 | run: uv cache prune --ci
52 |
53 | # -- Deploy to GitHub Pages only on push to upstream main
54 | - name: Setup GitHub Pages
55 | if: ${{ github.ref_name == 'main' && (github.event_name == 'push' || github.event_name == 'merge_group') }}
56 | uses: actions/configure-pages@v5
57 |
58 | - name: Upload artifact to GitHub Pages
59 | if: ${{ github.ref_name == 'main' && (github.event_name == 'push' || github.event_name == 'merge_group') }}
60 | uses: actions/upload-pages-artifact@v4
61 | with:
62 | name: github-pages
63 | path: build/html
64 | retention-days: 1
65 |
66 | - name: Deploy to GitHub Pages
67 | id: deployment
68 | if: ${{ github.ref_name == 'main' && (github.event_name == 'push' || github.event_name == 'merge_group') }}
69 | uses: actions/deploy-pages@v4
70 |
--------------------------------------------------------------------------------
/source/Reviewers/releases/wiki_pages.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _wiki_pages:
8 |
9 | Standard Jobs and Wiki Pages
10 | ============================
11 |
12 | Standard Jobs Page
13 | ------------------
14 |
15 | This involves creating a new `Standard Jobs `_ for the new release. First update the index page with the new release, then copy the previous release page to the new one, deleting the jobs table within. Check through for all references to the previous release, updating them. Also update the links to the resources pages - we will create these later. Complete a review of the rest of the standard jobs page, updating anything that requires it.
16 |
17 | Checkout a copy of the UM ``main`` at release and then run the following command to create the new standard jobs table,
18 |
19 | .. code-block::
20 |
21 | ./admin/create_stdjobs.py . >> output.txt
22 |
23 | Copy the output to the new page.
24 |
25 | We also include resource tables used by the suite at release - the links to these pages should have been updated earlier. The resources tables can be generated by running,
26 |
27 | .. code-block::
28 |
29 | ./admin/rose-stem/produce_resources.py /path/to/cylc-run_suite_directory/
30 |
31 | This requires a cylc-run directory of the ``all`` group (or individual ones for each platform). Usually the nightly testing from ``umtest`` can be used.
32 |
33 |
34 | Wiki Pages
35 | ----------
36 |
37 | Review and update the following pages, modifying version numbers as required:
38 |
39 | * Page Templates
40 |
41 | * For all of CR, SR and Ticket Summary
42 | * For all of UM, Jules, UKCA, LFRic Apps
43 |
44 | * `Bit Comparison Table `_ - This requires a new page creating
45 | * `Past Releases `_
46 |
47 |
--------------------------------------------------------------------------------
/source/FurtherDetails/dos_donts.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _dos_donts:
8 |
9 | Do's and Don'ts
10 | ===============
11 |
12 | Please Do
13 | ---------
14 |
15 | **Consult** Code Owners and system team. This will help maintain awareness and
16 | mitigate problems early on. This is the most common root cause of problems,
17 | sometimes years later.
18 |
19 | **Plan** your work aimed across single or multiple Issues and Pull Requests:
20 |
21 | * Each pull request should contain a single coherent change.
22 | * Consider using an overarching issue to link everything together
23 |
24 | **Document your work** using Issues and pull requests, :ref:`formal documentation
25 | ` and code comments. These help others and your future-self understand
26 | your work.
27 |
28 | **Meaningful names** for issues, pull requests, branches and variables. These
29 | help others and your future-self understand your work. "My_Branch", "Fix" are
30 | not helpful.
31 |
32 | **Be considerate** of other users/developers. Their skill-sets and working days
33 | may be very different to yours. All changes are visible to all users
34 | worldwide.
35 |
36 | **Link to issues/pull requests in other repositories**, eg ``MetOffice/jules#1``,
37 | ``MetOffice/ukca:#72``
38 |
39 | Please Do Not
40 | -------------
41 |
42 | **Do not merge ``main`` into your branch** during the development process. To
43 | aid scientific evaluation, changes should be kept in standalone branches based
44 | on the last stable version.
45 |
46 | When your change is ready for Code Review, we then suggest merging in ``main``
47 | and resolving any conflicts.
48 |
49 | **Licensing** - Don't add code to any project (or to any branch thereof) that
50 | has been developed under a different license without agreement from the
51 | Simulation Systems and Deployment Team. This includes lifting Fortran code or
52 | text from books. Our repositiories must not infringe copyright.
53 |
54 | **Request support by raising an issue**. Newly raised issues are not
55 | monitored. Use the appropriate :ref:`support` channels.
56 |
--------------------------------------------------------------------------------
/source/Development/developing_change.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _development_index:
8 |
9 | Developing Your Change
10 | ======================
11 |
12 | How a change is developed generally depends on the nature of the change and the
13 | size of the code change: small code changes of just a few lines can be written
14 | in minutes before being tested. For anything longer and more complex, it is
15 | worth developing the code in small sections or units, testing that each piece
16 | of code works before committing back to the development branch. By following
17 | this methodology, if one aspect of the code doesn't work, there is always the
18 | option to revert the local changes and quickly return to a checkpoint in your
19 | development that did work.
20 |
21 |
22 | .. tip::
23 |
24 | Before embarking on a medium-sized or significant model change, it is
25 | useful to have an appropriate coding plan in place. See :ref:`planning`
26 | for further details.
27 |
28 | .. note::
29 |
30 | Do not add AI-generated code, e.g., from GitHub CoPilot to any branches or
31 | forks.
32 |
33 | Code changes should conform to the coding standards of the project.
34 | Remember to ensure that your changes are easy to read and follow, using
35 | comments to explain what the code is doing.
36 |
37 | The vast majority of changes will be based around a project's source
38 | code, though there are other infrastructure areas involved. Rarely, the
39 | developer may alter one of these *without* modifying the source code.
40 |
41 | Every change is different, but there are a few key attributes that
42 | increase the complexity of a particular change and should be considered
43 | carefully:
44 |
45 |
46 |
47 | .. toctree::
48 | :caption: Development Checklist
49 | :maxdepth: 1
50 |
51 | planning_your_change
52 | documentation
53 | inputs
54 | kgo
55 | diagnostics
56 | rose_stem
57 | testing
58 |
59 | .. important::
60 |
61 | All instructions regarding code locations in this section assume you are
62 | working in a clone of the appropriate repository. Please read
63 | the :ref:`working_practices_index` to discover more.
64 |
--------------------------------------------------------------------------------
/source/FurtherDetails/support.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _support:
8 |
9 | Support
10 | =======
11 |
12 | Level 0- Self help & Peer Support
13 | ---------------------------------
14 |
15 | Self-help and using the people around you should always be your first port of
16 | call. Time spent here will likely solve your problem or at least better
17 | describe it.
18 |
19 | Consider contacting relevant Code, Configuration or Suite Owners.
20 |
21 | Level 1- GitHub Discussions
22 | ---------------------------
23 | If your problem persists, then describe and discuss it on GitHub Discussions:
24 |
25 | * `LFRic `__
26 | * `UM `__
27 | * `JULES `__
28 | * `Mule `__
29 |
30 | Level 2- Support Teams
31 | ----------------------
32 |
33 | If you still require assistance, then contact the appropriate support team.
34 | Please respect the clear demarcations between the scope for different teams.
35 |
36 | Core Capability Development Team (formally LFRic Team):
37 | * Support windows for LFRic releases TBC
38 |
39 | Simulation Systems and Deployment Team (formerly UM System Team and CRUM team)
40 | supports:
41 |
42 | * UM, Shumlib and JULES versions released in the last 12 months,
43 | * main version used in the Met Office Operational Suite.
44 | * specific climate configurations ...
45 |
46 |
47 | Escalation
48 | ----------
49 |
50 | Sometimes things go wrong or different parties disagree. If discussions between
51 | developer, code/config owners and reviewers cannot resolve the situation then
52 | it should be escalated to the manager of the Simulation Systems and Deployment
53 | Team in the first instance. They will consult appropriately to make the best
54 | balanced and impartial decision. If this is unsuccessful, the ultimate
55 | (but very rarely used) decision authority about code that may be committed to
56 | ``main`` lies with the LFRic Apps Governance Group.
57 |
--------------------------------------------------------------------------------
/source/FurtherDetails/contributing.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _contributing:
8 |
9 | Contributor Licence Agreement and Certificate of Origin
10 | ========================================================
11 |
12 |
13 | To best ensure that the BSD licence of Momentum code can be sustained, we
14 | request that contributors from outside the Met Office agree with the following
15 | terms when submitting code for inclusion in Momentum.
16 |
17 | By making a contribution to this project, I certify that:
18 |
19 | 1. The contribution was created in whole or in part by me and I have the right
20 | to submit it, either on my behalf or on behalf of my employer, under the
21 | terms and conditions as described by this file; or
22 | 2. The contribution is based upon previous work that, to the best of my
23 | knowledge, is covered under an appropriate licence and I have the right or
24 | permission from the copyright owner under that licence to submit that work
25 | with modifications, whether created in whole or in part by me, under the
26 | terms and conditions as described by this file; or
27 | 3. The contribution was provided directly to me by some other person who
28 | certified (1) or (2) and I have not modified it.
29 | 4. I understand and agree that this project and the contribution are public
30 | and that a record of the contribution (including my name and email address)
31 | is retained for the full term of the copyright and may be redistributed
32 | consistent with this project or the licence(s) involved.
33 | 5. I, or my employer, grant to the UK Met Office and all recipients of this
34 | software a perpetual, worldwide, non-exclusive, no-charge, royalty-free,
35 | irrevocable copyright licence to reproduce, modify, prepare derivative
36 | works of, publicly display, publicly perform, sub-licence, and distribute
37 | this contribution and such modifications and derivative works consistent
38 | with this project or the licence(s) involved or other appropriate open
39 | source licence(s) specified by the project and approved by the
40 | [Open Source Initiative (OSI)](http://www.opensource.org/).
41 | 6. If I become aware of anything that would make any of the above inaccurate,
42 | in any way, I will let the UK Met Office know as soon as I become aware.
43 |
--------------------------------------------------------------------------------
/source/Development/documentation.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _docs:
8 |
9 | Documentation
10 | =============
11 |
12 | All projects have their own scientific and technical documentation. Most
13 | notably:
14 |
15 | +----------------------------+-----------------------------+
16 | | UM Documentation Papers |`view UM`_ |
17 | +----------------------------+-----------------------------+
18 | | JULES User Guide |`view JULES`_ |
19 | +----------------------------+-----------------------------+
20 | | LFRic Core |`view LFRic Core`_ |
21 | +----------------------------+-----------------------------+
22 | | LFRic Apps |`view LFRic Apps`_ |
23 | +----------------------------+-----------------------------+
24 |
25 | .. _view UM: https://metoffice.github.io/um_doc/
26 | .. _view JULES: https://jules-lsm.github.io/latest/index.html
27 | .. _view LFRic Core: https://metoffice.github.io/lfric_core/
28 | .. _view LFRic Apps: https://metoffice.github.io/lfric_apps/
29 |
30 | Increasingly this documentation is stored as restructuredText markdown files
31 | alongside the code. This is then compiled using Sphinx into the webpages above.
32 |
33 | .. toctree::
34 | :maxdepth: 1
35 |
36 | sphinx_docs
37 | um_docs
38 | jules_docs
39 |
40 |
41 | LFRic Apps and Core also use doxygen to document the code and all changes
42 | should include appropriate doxygen changes to go with them. Doxygen guidelines
43 | are available in the `LFRic Core Documentation
44 | `_.
45 |
46 | Small changes and bug fixes rarely need documentation to be updated, but when
47 | new science is added to a project, the documentation must be updated to ensure
48 | that it remains contemporary with the code.
49 |
50 | Documentation changes that are held within a repository should be documented
51 | with issues and pull requests, are formally reviewed, and should be included
52 | on the same branch as the code changes. Apply the `Documentation` label to your
53 | pull request.
54 |
55 | .. tip::
56 |
57 | Searching the relevant documentation for words related to your change is
58 | often useful when deciding whether to update the documentation.
59 |
60 |
61 |
--------------------------------------------------------------------------------
/source/Development/sphinx_docs.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _sphinx:
8 |
9 | Sphinx Documentation
10 | ====================
11 | The documentation is written in plain text files using a markup language called
12 | reStructuredText. The text files have the extension ``.rst``. Sphinx can
13 | take these text files and generate both HTML and PDF documentation from them
14 | (complete with cross-referencing links, etc.). Since reStructuredText is a
15 | plain text format, your favourite text editor is all you need to edit the
16 | documentation.
17 |
18 | .. tip::
19 | The `Sphinx documentation `_ is a
20 | useful resource.
21 |
22 |
23 | Building the documentation
24 | --------------------------
25 |
26 | Please create the production environment to build the
27 | documentation (first time users only), and then activate it.
28 |
29 | .. tab-set::
30 |
31 | .. tab-item:: JULES
32 |
33 | From the top level of the repository:
34 |
35 | .. code-block:: shell
36 |
37 | conda env create -f environment.yml
38 |
39 | Activate the environment:
40 |
41 | .. code-block:: shell
42 |
43 | conda activate jules-user-guide
44 |
45 | .. tab-item:: LFRic Apps and Core
46 |
47 | On the Met Office Azure Spice machine the main LFRic module environment
48 | contains all the required packages to build the documentation. Ensure
49 | this is loaded.
50 |
51 | Note there is a `style guide included in the LFRic Core documentation
52 | `_ that should be followed for both
54 | these repositories.
55 |
56 | .. tab-item:: Working Practices
57 |
58 | From the top level of the repository:
59 |
60 | .. code-block:: shell
61 |
62 | conda env create -f env.yml
63 |
64 | Activate the environment:
65 |
66 | .. code-block:: shell
67 |
68 | conda activate sphinx_doc_env
69 |
70 | Move to the documentation directory and run ``make`` to build the documentation:
71 |
72 | To build and view the HTML documentation:
73 |
74 | .. code-block:: shell
75 |
76 | make [clean] html
77 | firefox build/html/index.html &
78 |
79 | To build and view the PDF documentation:
80 |
81 | .. code-block:: shell
82 |
83 | make latexpdf
84 | evince build/latex/JULES_User_Guide.pdf
85 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # Simulation Systems
2 |
3 | [](https://github.com/MetOffice/simulation-systems/actions/workflows/publish_wps.yaml)
4 |
5 | This repository contains the working practices documentation common to the
6 | various
7 | [simulation and modelling repositories](https://github.com/MetOffice/simulation-systems/wiki)
8 | owned by the Met Office.
9 |
10 | The documentation in the repository can be compiled in different ways.
11 |
12 | > [!NOTE]
13 | > Optional system dependencies for PDF generation may require LaTeX
14 | > distributions and other third-party libraries.
15 |
16 | ## Setup environment
17 |
18 | You can set up the environment ro build and deploy the documentation using any
19 | of the following methods:
20 |
21 | ### 1. Using uv
22 |
23 | A quick and clean way to get the package dependencies is via the
24 | [uv](https://docs.astral.sh/uv/) package manager.
25 |
26 | ```shell
27 | git clone https://github.com/MetOffice/simulation-systems
28 | cd simulation-systems
29 |
30 | # Install dependencies (see pyproject.toml) in project .venv
31 | uv sync
32 | ```
33 |
34 | ### 2. Using pip
35 |
36 | If you have Python 3.11 or higher installed (`sphinx==8.2.3` required), you can
37 | use pip:
38 |
39 | ```shell
40 | git clone https://github.com/MetOffice/simulation-systems
41 | cd simulation-systems
42 |
43 | -m venv .venv
44 | source .venv/bin/activate
45 | pip install .
46 | ```
47 |
48 | ### 3. Using conda
49 |
50 | ```shell
51 | git clone https://github.com/MetOffice/simulation-systems
52 | cd simulation-systems
53 |
54 | conda env create -f env.yml
55 | conda activate sphinx_doc_env
56 | ```
57 |
58 | ## Build documentation
59 |
60 | ```shell
61 | # For uv environment
62 | uv run make clean html
63 |
64 | # For pip or conda environments
65 | make clean html
66 | ```
67 |
68 | After building, the HTML documentation can be found in the `build/html/`
69 | directory of your local repository. You can open the documentation in any web
70 | browser.
71 |
72 | **Met Office users** can skip build step above and deploy the documentation
73 | directly to a predefined location:
74 | `~/public_html/simulation-systems//html/`
75 |
76 | ```shell
77 | uv run make clean deploy # uv env
78 | # OR
79 | make clean deploy # pip or conda env
80 | ```
81 |
82 | ## Contributing
83 |
84 | Please follow the project's [Code of Conduct](CONTRIBUTING.md)
85 |
86 | The documentation is written in Sphinx markup. To propose changes:
87 |
88 | 1. Create an issue detailing the required changes.
89 | 2. Create a branch in your clone of this repository, linking it to your issue.
90 | 3. Regularly build your changes as described above.
91 |
92 | Once satisfied, create a pull request and request a review from
93 | [MetOffice/ssdteam](https://github.com/orgs/MetOffice/teams/ssdteam).
94 |
--------------------------------------------------------------------------------
/source/Reviewers/releases/updating_prebuilds.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _updating_prebuilds:
8 |
9 | **This page is left in case prebuilds become required again, but at the move to
10 | GitHub, prebuilds are no longer functional, so these instructions are no longer
11 | required**
12 |
13 | Mid-Release Prebuilds
14 | =====================
15 |
16 | These are instructions for updating UM prebuilds mid-way through a release
17 | cycle. Reasons for doing this include new builds being added, changes that
18 | mean the old prebuilds break (sometimes new/deleted files can do this), or
19 | because the prebuilds are so out of date they don't particularly help. An
20 | example ticket doing this can be seen in `UM:#7903
21 | `_.
22 |
23 | To update prebuilds:
24 |
25 | * Open a new UM ticket, create a branch and check it out.
26 | * In ``rose-stem/rose-suite.conf``:
27 |
28 | * Update the ``BASE_UM_REV`` variable to the latest version of the UM
29 | ``main``.
30 | * Update any other ``BASE_*_REV`` variables if those ``main`` have more
31 | recent commits that the revision listed.
32 |
33 | * In ``rose-stem/site/meto/variables.cylc``:
34 |
35 | * Update the prebuilds paths with the name, ``rXXXXXX_prebuilds`` where
36 | ``XXXXXX`` should match the ``BASE_UM_REV`` variable from earlier.
37 |
38 | * Commit these changes to the branch.
39 | * Login as ``umadmin`` and check out the branch (this will need to be from the
40 | mirror, so ensure it has updated).
41 | * Install the prebuilds, first on azure spice and EXAB, then on the EXCD and
42 | finally the EXZ. Ensure the name of the suite matches the path that was set
43 | in the ``variables.cylc`` file.
44 |
45 | .. code-block:: shell
46 |
47 | export CYLC_VERSION=7
48 |
49 | # EXAB and Azure Spice
50 | rose stem --group=prebuilds --source=fcm:um.xm_tr@XXXXXX \
51 | --name=rXXXXXX_prebuilds --config=./rose-stem -S MAKE_PREBUILDS=true \
52 | -S USE_EXAB=true
53 |
54 | # EXCD
55 | rose stem --group=ex1a_fcm_make,ex1a_fcm_make_portio2b \
56 | --source=fcm:um.xm_tr@XXXXXX --name=rXXXXXX_prebuilds \
57 | --config=./rose-stem -S MAKE_PREBUILDS=true -S USE_EXCD=true
58 |
59 | # EXZ
60 | rose stem --group=ex1a_fcm_make,ex1a_fcm_make_portio2b \
61 | --source=fcm:um.xm_tr@XXXXXX --name=rXXXXXX_prebuilds \
62 | --config=./rose-stem -S MAKE_PREBUILDS=true -S USE_EXZ=true
63 |
64 | * Finally, ensure that prebuilds are enabled in ``rose-stem/rose-suite.conf``
65 | and have the ticket reviewed and committed.
66 |
--------------------------------------------------------------------------------
/source/WorkingPractices/final_steps.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | Final Steps
8 | ===========
9 |
10 | Once your change is approved, the code reviewer will follow the
11 | :ref:`howtocommit` process on any branches in order to commit them to the
12 | ``main`` of each project. Depending on any linked PRs and documentation updates,
13 | there may be several commits to different trunks involved.
14 |
15 | .. note::
16 |
17 | Sometimes there can be a delay between a code change being approved and the
18 | commit to ``main``. This can be for a number of reasons and rarely will be
19 | due to your change. If you have any concerns, please contact your Code
20 | Reviewer in the first instance.
21 |
22 | Overnight and Weekly Testing
23 | ----------------------------
24 |
25 | Each project is tested overnight. This includes several related repositories
26 | being tested together (e.g. the UM and JULES repositories are related, so the
27 | head of the UM ``main`` is tested with the head of the JULES ``main``). Testing
28 | is usually based on the rose stem system.
29 |
30 | In addition, most projects run weekly tests, which involve some longer jobs not
31 | normally tested in the nightly tests.
32 |
33 | Closing Pull Requests
34 | ---------------------
35 |
36 | GitHub will automatically close PRs upon merge. The code reviewer will check the
37 | output of nightly testing, and if this shows errors may begin a discussion on
38 | the pull request. Some testing is only run weekly, so some issues may take
39 | longer to show up.
40 |
41 |
42 | When Main is Broken
43 | -------------------
44 |
45 | Occasionally, the overnight testing will fail. If the issue can't be
46 | immediately solved, the ``main`` of affected projects will be closed to new
47 | changes. The relevant teams will investigate and aim to resolve the issue and
48 | reopen the ``main`` as soon as possible. Two possible scenarios may occur:
49 |
50 | #. For **simple or obvious fixes**, a trivial PR to fix ``main`` is preferred.
51 |
52 | #. If the reason for the failure is complex or less obvious, the team will
53 | revert the offending change off ``main`` .
54 |
55 | In both cases any further PRs should link back to the original.
56 |
57 |
58 | Changes to the Working Practices
59 | --------------------------------
60 |
61 | These working practices have evolved over many years of experience and in order
62 | to try and catch as many issues as possible. However, feedback is welcome at
63 | any time. Please suggest improvements to the Simulation Systems and Deployment
64 | team.
65 |
66 | Suggestions received will be evaluated and responded to, but a final decision
67 | as to whether the working practices will be modified lies with the Simulation
68 | Systems and Deployment team and ultimately the Simulation Systems Governance
69 | Group.
70 |
--------------------------------------------------------------------------------
/source/WorkingPractices/multi_repository.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _multirepo:
8 |
9 | Working with Multiple Repositories
10 | ==================================
11 |
12 | The repositories covered by these working practices all interact with and have
13 | dependencies on each other. This means that changes that affect multiple
14 | repositories need handling with extra care.
15 |
16 | LFRic Apps and the UM both act as child repositories to a number of parent
17 | repositories such as JULES, UKCA, SOCRATES and CASIM. These parent repositories
18 | work independently as well as being used by the child repositories. LFRic
19 | Apps also utilises the infrastructure in LFRic Core.
20 |
21 | This means that changes to the science code in any of the parent repositories
22 | will need testing with both the UM and LFRic Apps to check for any
23 | interactions.
24 |
25 | .. note::
26 |
27 | From LFRic Apps vn2.1 and UM vn13.8 there are duplicate copies of the
28 | physics code in both the UM and LFRic Apps.
29 |
30 | Consideration should be made as to whether changes to the physics schemes
31 | are best made in LFRic Apps, the UM or both. If in doubt then the copy in
32 | LFRic Apps is considered the master version.
33 |
34 | .. image:: images/repos.png
35 | :class: only-light
36 |
37 | .. image:: images/repos_dark.png
38 | :class: only-dark
39 |
40 | .. _linked:
41 |
42 | Preparing Linked Pull Requests
43 | ------------------------------
44 |
45 | Every repository in a set of linked changes requires a PR. These PRs will be
46 | treated as a group with the same reviewers and committed at the same time.
47 |
48 | Do:
49 | * Make sure every PR has a cross reference to the others in the set. A
50 | section will exist in the PR template for this. Use the format
51 | ``MetOffice/um#1234``
52 | * Use labels to show which other repositories are involved
53 | * Get the PRs ready for review at the same time
54 | * Ask for help testing if you don't have access to all the codebases
55 | involved
56 |
57 | .. important::
58 |
59 | Code branches in linked PRs will require branching from compatible
60 | commit hashes to ensure they work together.
61 |
62 | It is recommended to work with branches created from a specific release.
63 | These are guaranteed to work together.
64 |
65 | For head of ``main`` branches make sure that all branches/hashes being
66 | used are at least as recent as the commits associated with the hash in the
67 | ``dependencies.yaml`` file.
68 |
69 | If in doubt, please contact the Simulation Systems and Deployment Team for
70 | advice.
71 |
72 | .. _multirepo_testing:
73 |
74 | Multi-Repo Testing
75 | ------------------
76 |
77 | Please see :ref:`this page ` for details on how to test
78 | multiple repositories together.
79 |
--------------------------------------------------------------------------------
/source/Reviewers/releases/shumlib_release.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _shumlib_release:
8 |
9 | Shumlib Release
10 | ===============
11 |
12 | .. note::
13 |
14 | Additional details for the shumlib/mule releases can be seen `here
15 | `__
16 |
17 |
18 | Releasing Shumlib
19 | -----------------
20 |
21 | There's no hard rule on whether a shumlib release is required - it comes down
22 | to whether there have been any notable changes to shumlib since the last
23 | release. If there haven't, then a release isn't necessary. In this case, the
24 | shumlib ``main`` will get a ``umX.Y`` tag during the UM release but that is all.
25 |
26 | If you are doing a release then do the following,
27 |
28 | * Ensure you have a :ref:`fork ` of ``shumlib`` and that the ``main``
29 | branch is up to date with the upstream repository.
30 | * In a clone of the fork, :ref:`create a branch ` using the
31 | ``main`` branch as the parent.
32 |
33 | .. important::
34 |
35 | Ensure you create branches from main, otherwise you will not include the
36 | changes from the past release.
37 |
38 | * Update the shumlib version number in ``common/src/shumlib_version.h``
39 | * Update the shumlib version number (in a comment) in
40 | ``meto_install_shumlib.sh``.
41 | * Test the branch against the UM
42 | * With a reviewer, follow the :ref:`release process `
43 |
44 |
45 | Installing Shumlib
46 | ------------------
47 |
48 | Shumlib is installed with a build script, at ``scripts/meto_install_mule.sh``.
49 | This takes a single compulsory argument to set the ``platform`` of the
50 | install. Generally, this is a platform at the Met Office, currently ``ex1a``
51 | or ``azspice``, which will install shumlib for all defined builds on that
52 | platform. Alternatively, you can use a specific build, eg.
53 | ``azspice_gnu_12.2``, to install just for that build.
54 |
55 | By default, shumlib will be installed in a ``build`` directory in the working
56 | copy. This can be controlled using the ``BUILD_DESTINATION`` environment
57 | variable.
58 |
59 | To install at the Met Office, login as ``umadmin`` and get a clone of the new
60 | version. Use https to get this as the shared account doesn't have ssh access.
61 | First install on azspice,
62 |
63 | .. code-block:: shell
64 |
65 | export SPACKDIR= # Set this as the Spackadmin Data Directory (no trailing slash)
66 | BUILD_DESTINATION=$UMDIR/shumlib/shumlib-YYYY.MM.V scripts/meto_install_shumlib.sh azspice
67 |
68 | Then ssh to each of EXAB, EXCD and EXZ and install there,
69 |
70 | .. code-block:: shell
71 |
72 | BUILD_DESTINATION=$UMDIR/shumlib/shumlib-YYYY.MM.V scripts/meto_install_shumlib.sh ex1a
73 |
74 | Finally do the install on Monsoon (using ``umadmin.mon`` shared user).
75 |
--------------------------------------------------------------------------------
/source/WorkingPractices/approvals.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _approvals:
8 |
9 | Approval Process
10 | ================
11 |
12 | Every pull request will need to get approval from a group of people. Code Owner
13 | approvals are automatically requested when the pull request is marked as `Ready
14 | for Review`, while required Configuration Owner approvals will be listed on the
15 | trac.log testing summary.
16 |
17 | .. tip::
18 |
19 | Make it really clear and easy for approvers to understand your change and
20 | know what needs approving - some get a lot of requests!
21 |
22 | You may need to chase up approvals occasionally. People are not perfect and
23 | remember to :ref:`be kind `.
24 |
25 | .. important::
26 |
27 | New UM Ancils must be submitted to the MIAO team for approval. Please
28 | follow their process for `Requesting New UM Ancils
29 | `__.
31 |
32 | Code Owners
33 | -----------
34 |
35 | Every file in the codebases has a :ref:`code_owner`, and every file changed
36 | will need checking by the code owners - no matter how small the change.
37 |
38 | A list of current code owners for each project can be found in the top level
39 | ``CODEOWNERS`` file.
40 |
41 | Code Owners should complete a review requesting changes if needed, and
42 | approving once they are happy. They only need to review changes that fall in their
43 | area.
44 |
45 | .. Tip::
46 |
47 | It is always worth talking to the main code owners involved early in the
48 | process. These people have good oversight on changes in an area and will
49 | be able to guide your change to fit in with the bigger picture of what is
50 | happening.
51 |
52 | Code Owners are good candidates for choosing as a SciTech reviewer.
53 |
54 | Config Owner
55 | ------------
56 |
57 | :ref:`Configuration Owner's ` are responsible for a combination
58 | of settings in the model that achieve a particular aim (e.g. the current GAL
59 | or RAL setup used operationally). Within the rose-stem testing these
60 | configurations will be used and any changes to the answers shown in testing
61 | will need sign off from the config owner.
62 |
63 | In JULES, the code owner is supported by module leaders who will
64 | sign-off scientifically significant changes to their areas of interest. The
65 | module leaders in this case will also act as configuration owners for their
66 | science settings.
67 |
68 | A list of current config owners for the UM is found in
69 | ``main/ConfigOwners.txt``. Others are combined with the Code Owner lists
70 | above.
71 |
72 | Other Approvals
73 | ---------------
74 |
75 | There are other changes that require specific approval and these are made clear
76 | in the pull request description. For example, changes that modify code within an
77 | OpenMP section will require approval from the :ref:`hpc_opt_team` and changes
78 | that modify or add PSyKAl-lite code will require approval from the TCD Team.
79 |
--------------------------------------------------------------------------------
/source/git_faq.rst:
--------------------------------------------------------------------------------
1 | .. _git_faq:
2 |
3 | Git Migration FAQ's
4 | ===================
5 |
6 | .. rubric:: How do I access the repositories?
7 |
8 | All of the migrated Simulation Systems repositories are found in the Met
9 | Office organisation. Certain repositories, where licencing allows, have been
10 | made public, so can be viewed by anyone. Other repositories (eg. UM) are
11 | internal to the Met Office organisation. These can be seen if you have access
12 | to the organisation.
13 |
14 | .. rubric:: Where are my branches and tickets?
15 |
16 | Migration of individual branches and tickets is being left to each developer
17 | to be done on a case by case basis. Please see the :ref:`branch migration
18 | guide ` for advice on migrating an fcm branch to git.
19 |
20 | .. rubric:: Why can't I push or create a branch in the repository?
21 |
22 | We are managing our repositories using a :ref:`forking ` model,
23 | therefore creating of branches in the central repository is tightly
24 | controlled. Please create your own fork and develop your changes on branches
25 | in there.
26 |
27 | .. rubric:: Why won't the rose-stem command work?
28 |
29 | The syntax to launch the test suite has changed with the move to git. Please
30 | see our :ref:`testing pages `.
31 |
32 | .. rubric:: Rose-stem is giving the following error while trying to fetch code
33 | from other repositories: `git@github.com: Permission denied (publickey).`
34 |
35 | This is a sign of SSH setup issues. Please make sure you've followed all the
36 | steps on the :ref:`GitHub Authorisations page`.
37 |
38 | .. rubric:: How do I run with an older version?
39 |
40 | The entire history of each trunk branch from the MOSRS repository has been
41 | migrated (for the UM this is from vn9.2). If you need a specific release, then
42 | you can checkout the tag of that release, eg. ``git checkout ``. If you
43 | require a specific commit, you will need to find the relevant hash from the
44 | history of the main branch.
45 |
46 | .. rubric:: Why can't I see the full history of Jules and MONC?
47 |
48 | Due to licencing issues we were unable to make the history of these
49 | repositories public. We have a copy of the repository with all the previous
50 | releases internal to the MetOffice. Please ask if you need help accessing
51 | that.
52 |
53 | .. rubric:: My forked repository got detached from upstream repository following
54 | visibility change (private to public) upstream.
55 |
56 | The "detached" state is a security feature to isolate private forks when the
57 | upstream parent becomes public. Only GitHub staff have the administrative
58 | privileges required to manually re-link the repository network while
59 | preserving existing PR data and history.
60 |
61 | One option is to make sure any work you wish to preserve is in a clone of your
62 | fork. Then recreate the fork from upstream using the same name as before.
63 | Pushing from your clone will connect to the new fork. However this will delete
64 | any pull requests you had open from the previous fork.
65 |
66 | If you already have a lots of develop branches/PRs, its probably best to
67 | contact GitHub Support and Submit a ticket to reattach your fork with
68 | upstream with the following message:
69 |
70 | My upstream private repo became public, which detached my fork. I have open
71 | PRs I need to preserve. Please reattach my fork to the original network.
72 |
73 | Upstream URL: https://github.com/MetOffice/{repo-name}
74 | Fork URL: https://github.com/{user}/{repo-name}
75 |
76 | This process should usually get resolved within a day.
77 |
--------------------------------------------------------------------------------
/source/Development/Diagnostics/um_stashmaster.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _stash:
8 |
9 | STASH [#f1]_
10 | ============
11 |
12 | Information on every diagnostic available to the model is stored in a single
13 | file named ``STASHmaster_A``, which is read into the model at the start of the
14 | run.
15 |
16 | The UM's ``STASHmaster_A`` and associated help text file
17 | ``STASHmaster-meta.conf`` are available in your branch at
18 | ``vnXX.Y_/rose-meta/um-atmos/HEAD/etc/stash/STASHmaster/``.
19 |
20 | .. note::
21 |
22 | When running the UM rose stem suite, the suite will automatically use the
23 | ``STASHmaster_A file`` from your branch when testing your code.
24 |
25 | The following principles apply when altering the STASHmaster:
26 |
27 | * If you add a new diagnostic to the ``STASHmaster_A`` file then you **must**
28 | also add to the stash master help text in :ref:`stashmaster-meta`.
29 | * If you are altering the stashmaster, this may be referred to the FFPP
30 | governance board by the sci/tech or code reviewers - see the STASH entry
31 | guidelines.
32 | * If your change has new stash items or changed/added attributes as an option
33 | code, versions mask etc., then first you have to get them reserved and
34 | recorded (published) on the reservation web page STASH/ReservedCodes
35 | * Note that every reservation should be linked to an Issue with the correct
36 | explanation and a milestone. This rule applies to all stash related tables
37 | placed on this page.
38 | * Although reservations could be some kind of self-service, contact the section
39 | owner first nevertheless. This could help to organise new items
40 | (when possible) in some logical groups.
41 | * For new option code numbers contact the STASH code owner.
42 |
43 | .. note::
44 |
45 | Complete details of the STASH system (including the syntax used in the
46 | ``STASHmaster_A`` file) can be found in `UMDP C04
47 | `__
48 |
49 | .. _stashmaster-meta:
50 |
51 | STASHmaster-meta.conf
52 | ---------------------
53 |
54 | If you are adding a new UM STASH diagnostic you must also add help text to the
55 | STASHmaster-meta.conf. This will provide others with help on your diagnostic.
56 | You will need to identify the stash entry with a ``[stashmaster:code
57 | (xyz)]`` section header, where the xyz is the stash code in the form ``section
58 | number * 1000 + item number``.
59 |
60 | Include a full name, any units and explanatory text. You should also add a
61 | description field that matches the full name of the diagnostic. For example:
62 |
63 | .. code-block::
64 |
65 | [stashmaster:code(1050)]
66 | description=NO2 Dry Deposition Rate (3D)
67 | help=NO2 Dry Deposition Rate (3D)
68 | =moles/s
69 | =
70 | =This is the total dry deposition flux of NO2 in each gridbox
71 | =
72 | =The sum of this deposition flux over all model gridboxes gives the total
73 | =number of moles of NO2 removed by this process per second in the
74 | =whole model.
75 |
76 | This assists the model user in being able to find useful help text on their
77 | diagnostic.
78 |
79 |
80 | .. rubric:: Footnotes
81 |
82 | .. [#f1] This is an acronym for 'Spatial and Temporal Averaging and Storage Handling'.
83 |
--------------------------------------------------------------------------------
/source/Reviewers/releases/lfric_apps_release.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _lfric_apps_release:
8 |
9 | LFRic Apps Release
10 | ==================
11 |
12 | LFRic Inputs KGO Install
13 | ------------------------
14 |
15 | * This can be done at any point once all PRs that change lfricinputs kgo
16 | have been committed.
17 | * It's easiest to use the umtest nightly testing for this and will save having
18 | to run the suite twice.
19 | * Alternatively, run ``cylc vip -z g=lfricinputs -S HOUSEKEEPING=false -n lfricinputs_kgo ./rose-stem``
20 | and wait for this to finish - all jobs should pass.
21 | * Install the kgo by running
22 | ``$UMDIR/SimSys_Scripts/kgo_updates/meto_update_kgo.sh --new-release``
23 |
24 | * The script will ask for a clone path - this can be any lfric apps clone as
25 | it will not be modified.
26 | * The version number and PR number are not used, although an entry is
27 | required.
28 | * The kgo install directory must be updated to vnX.Y
29 |
30 |
31 | LFRic Release
32 | -------------
33 |
34 | * Ensure you have a :ref:`fork ` of both the ``lfric_apps`` and
35 | ``lfric_core`` repositories, and that the ``main`` branches in each are up to
36 | date with the upstream repository.
37 | * Create an LFRic Apps Issue for the release to provide a number for the release
38 | script.
39 | * In a clone of these forks, :ref:`create a branch ` using the
40 | ``main`` branch as the parent.
41 |
42 | .. important::
43 |
44 | Ensure you create branches from main, otherwise you will not include the
45 | changes from the past release.
46 |
47 | * Move into the lfric apps clone
48 | * Run the release script,
49 | ``$UMDIR/SimSys_Scripts/lfric_macros/release_lfric.py -o A.B -v X.Y -t
50 | TTTT -c /path/to/core``
51 |
52 | * ``A.B`` - the previous version
53 | * ``X.Y`` - the new version
54 | * ``TTTT`` - the apps release issue number
55 | * ``/path/to/core`` - path to the lfric core clone
56 |
57 | * Check the output looks sensible. It should:
58 |
59 | * Update the version number
60 | * Revert any changes to ``rose-stem/site/meto/variables_*.cylc``
61 | * Copy the ``HEAD`` metadata to ``vnX.Y``
62 | * Add a blank upgrade macro to all ``versions.py`` files
63 | * Apply the upgrade macro - rose apps should be updated to the new version
64 | * Add a ``version_ab_xy.py`` upgrade file - a copy of the versions.py file
65 | * Reset the ``versions.py`` file with no upgrade macros
66 |
67 | * Tag other repositories and update dependencies.yaml:
68 |
69 | * Ensure that the :ref:`other repositories have all been tagged`.
70 |
71 | * In dependencies.yaml:
72 |
73 | * Ensure the ``lfric_core`` ``source`` is pointing at the local clone of
74 | your branch.
75 | * Update ``ref`` for all other repositories to be the Simulation Systems tag
76 | ``YYYY.MM.1``, checking that the dependencies look as they should.
77 |
78 |
79 | * Commit your changes to both Apps and Core branches.
80 |
81 | * Run the test suites
82 |
83 | * ``cylc vip -z g=all -n lfric_*X.Y ./rose-stem`` for both Apps and Core.
84 |
85 | * Once testing is complete, update LFRic Core in ``dependencies.yaml``
86 |
87 | * ``source`` should be the MetOffice ssh url
88 | * ``ref`` should be ``coreX.Y``
89 |
90 | * Open a PR for each and with a reviewer, follow the
91 | :ref:`review process `
92 |
--------------------------------------------------------------------------------
/source/conf.py:
--------------------------------------------------------------------------------
1 | # ----------------------------------------------------------------------------
2 | # (C) Crown copyright Met Office. All rights reserved.
3 | # The file LICENCE, distributed with this code, contains details of the terms
4 | # under which the code may be used.
5 | # ----------------------------------------------------------------------------
6 |
7 | # Configuration file for the Sphinx documentation builder.
8 | #
9 | # This file only contains a selection of the most common options. For a full
10 | # list see the documentation:
11 | # https://www.sphinx-doc.org/en/master/usage/configuration.html
12 |
13 | # -- Path setup --------------------------------------------------------------
14 |
15 | # If extensions (or modules to document with autodoc) are in another directory,
16 | # add these directories to sys.path here. If the directory is relative to the
17 | # documentation root, use os.path.abspath to make it absolute, like shown here.
18 | #
19 |
20 | # -- Project information -----------------------------------------------------
21 |
22 | project = "Simulation Systems"
23 | copyright = "Met Office"
24 | author = "Simulation Systems and Deployment Team"
25 |
26 |
27 | # -- General configuration ---------------------------------------------------
28 |
29 | # Add any Sphinx extension module names here, as strings. They can be
30 | # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
31 | # ones.
32 |
33 | language = "en"
34 |
35 | # Added to use dropdowns with command: pip install sphinx-design
36 | extensions = [
37 | "sphinx_design",
38 | "sphinx_copybutton",
39 | "sphinxcontrib.rsvgconverter",
40 | "sphinxcontrib.mermaid",
41 | ]
42 |
43 | # Add any paths that contain templates here, relative to this directory.
44 | templates_path = ["_templates"]
45 |
46 | # List of patterns, relative to source directory, that match files and
47 | # directories to ignore when looking for source files.
48 | # This pattern also affects html_static_path and html_extra_path.
49 | exclude_patterns = [".venv"]
50 |
51 | html_static_path = ["_static"]
52 | html_css_files = ["custom.css"]
53 | # -- Options for HTML output -------------------------------------------------
54 |
55 | # The theme to use for HTML and HTML Help pages. See the documentation for
56 | # a list of builtin themes.
57 | #
58 | html_theme = "pydata_sphinx_theme"
59 |
60 | html_theme_options = {
61 | "footer_start": ["crown-copyright"],
62 | "footer_center": ["show-accessibility"],
63 | "footer_end": ["sphinx-version", "theme-version"],
64 | "navigation_with_keys": False,
65 | "show_toc_level": 2,
66 | "show_prev_next": True,
67 | "navbar_align": "content",
68 | "logo": {
69 | "text": "Simulation Systems",
70 | "image_light": "_static/MO_SQUARE_black_mono_for_light_backg_RBG.png",
71 | "image_dark": "_static/MO_SQUARE_for_dark_backg_RBG.png",
72 | },
73 | "icon_links": [
74 | {
75 | "name": "GitHub Discussions",
76 | "url": "https://github.com/MetOffice/simulation-systems/discussions",
77 | "icon": "far fa-comments",
78 | },
79 | {
80 | "name": "GitHub",
81 | "url": "https://github.com/MetOffice/simulation-systems",
82 | "icon": "fa-brands fa-github",
83 | },
84 | ],
85 | }
86 |
87 | html_context = {
88 | "default_mode": "auto",
89 | }
90 | # Hide the link which shows the rst markup
91 | html_show_sourcelink = False
92 |
93 | # -- Options for linkcheck builder -------------------------------------------
94 | linkcheck_anchors = False
95 | linkcheck_ignore = [
96 | r".*\.py$", # Ignores URLs ending with .py
97 | r"https://github.com/MetOffice/um*",
98 | ]
99 |
--------------------------------------------------------------------------------
/source/Reviewers/scitechreview.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _scitech_review:
8 |
9 | Science and Technical Review
10 | ============================
11 |
12 | .. tip::
13 |
14 | GitHub documentation on the review process and interface:
15 | `Reviewing Proposed Changes in a Pull Request `_
18 |
19 | Purpose of the review
20 | ---------------------
21 |
22 | The purpose of code review is to ensure that the code does the job it says it
23 | performs, is standards compliant and well documented.
24 |
25 | Focus on the code, not the contributor; providing constructive, respectful and
26 | actionable feedback. Critique the implementation, not the individual and always
27 | explain the reasoning behind your suggestions.
28 |
29 |
30 | Reviewer responsibilities and checkpoints
31 | -----------------------------------------
32 |
33 | The pull request template that populates the pull request description box
34 | contains a SciTech Review section with questions to help you think through all
35 | the areas of concern. This SciTech Review section should be completed once you are finished.
36 |
37 | The Science / Technical reviewer should
38 |
39 | * Understand the area of code and check that the changeset satisfies the
40 | purpose of the change.
41 |
42 | * Ensure that the code has no unwanted side-effects
43 |
44 | * Ensure that the code is written to the standards laid out in
45 | :ref:`Fortran Coding Standards ` or `LFRic Coding Styles
46 | `__
47 | (which build on the first).
48 |
49 | * Make sure that the in-line documentation is accurate and sufficient.
50 |
51 | * Ensure that any related :ref:`external documentation ` is updated as
52 | necessary.
53 |
54 | * Check that the pull request has been completed fully and accurately with
55 | sufficient detail for others to understand the impact of the change.
56 |
57 | * Ensure that testing has been carried out satisfactorily (and recorded on the
58 | pull request), and that there is no impact for configurations outside the
59 | required scope of the changeset.
60 |
61 | Final decision points and actions
62 | ---------------------------------
63 |
64 | The SciTech reviewer must demand that non-compliance is corrected
65 | before a change is passed onto the next level of review.
66 |
67 | The pull request will likely iterate between the reviewer and the developer during
68 | the review process while retaining it's SciTech Review status.
69 |
70 | Once you are happy that the change is appropriate and correct:
71 |
72 | #. Ensure the the SciTech parts of the pull request template have been completed.
73 | #. Leave an approving review
74 | #. Change the review state to Code Review
75 | #. Request the review of the nominated Code Reviewer (or leave them a
76 | message with ``@username`` if their review has already been requested)
77 |
78 | These steps are covered in more detail as part of the :ref:`review_process`
79 |
80 | .. image:: ../WorkingPractices/images/gh_screenshots/project_scitech_light.png
81 | :class: only-light border
82 |
83 | .. image:: ../WorkingPractices/images/gh_screenshots/project_scitech_dark.png
84 | :class: only-dark border.
85 |
86 |
--------------------------------------------------------------------------------
/source/FurtherDetails/glossary.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | Simulation Systems Glossary
8 | ===========================
9 |
10 | Please suggest new entries to the Simulation Systems and Deployment Team
11 |
12 | Closed Release:
13 | A release cycle where the only accepted changes to ``main`` relate to a
14 | particular piece of work, either technical or scientific. A release can
15 | also be partially closed, with only one area of a code base locked down in
16 | this way, and the rest free for changes.
17 |
18 | Code Review Deadline:
19 | The date by which all pull requests aiming to be included in a release have been
20 | moved into code review.
21 |
22 | Code Review:
23 | A technical review of the changes involved in the pull request, including
24 | checks that code standards have been upheld and that the working practices
25 | have been followed. These reviews are generally completed by a member of the
26 | Simulation IT Team. Once a review has been approved the Code Systems
27 | Reviewer is then responsible for committing the change to the ``main``.
28 |
29 | Development Window:
30 | The period of time between the release of one software version and the code
31 | review deadline for the following release in which new developments are
32 | accepted for review.
33 |
34 | Known Good Output (KGO):
35 | In order to verify that the model output hasn't been modified by a set of
36 | changes the test suite contains a stored set of output as a reference.
37 | This is known as the KGO and changes that alter this require special
38 | treatment. For more information see :ref:`kgo`.
39 |
40 | Head of ``Main``:
41 | The most recent commit hash on ``main``. Branches are taken from here
42 | when the work being done *has* to be built on top of changes already made
43 | since the last release.
44 |
45 | Linked Pull Request (Linked PR):
46 | Work that spans two or more repositories, requiring pull requests that
47 | should be treated together and committed as a group.
48 |
49 | Overarching Issue:
50 | Where a piece of work has been split into multiple sections an overarching
51 | issue can be used to track this work. It should be closed when the whole arc
52 | has been completed. GitHub also allows subissues which can be created from
53 | the overarching issue.
54 |
55 | Regression:
56 | A set of tests that prove that a set of code changes have not degraded the
57 | model output. Comparisons are made between the results produced and the
58 | Known Good Output.
59 |
60 | SciTech Review:
61 | A science or technical review, completed by someone who is familiar with
62 | the area being changed. The aim of this review is to confirm that the
63 | changes made do what they say they should, and that the method used was
64 | appropriate.
65 |
66 | Simulation Systems and Deployment Team:
67 | The team responsible for the oversight of these working practices. For more
68 | information see :ref:`simIT`.
69 |
70 | Simulation Systems Governance Group:
71 | The governing body that oversees the work of the Simulation Systems and
72 | Deployment Team.
73 |
74 | Version:
75 | Each release of the codebase is completed by tagging the ``stable`` branch
76 | with a version number. This version should be used for creating code
77 | branches from and will also be used by the parallel suite teams as a
78 | starting point for creating the next operational suite.
79 |
--------------------------------------------------------------------------------
/source/Reviewers/releases/release_notes.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _release_notes:
8 |
9 | Release Notes
10 | =============
11 |
12 | The below shows the trac release notes template previously used. This will want
13 | editing and the exact location needs deciding for the move to GitHub. However it
14 | can be used as an initial guide to the sort of information included.
15 |
16 | .. code-block::
17 |
18 | [[PageOutline]]
19 | -----
20 | = UM vnXX.Y User Release Notes =
21 |
22 | == Summary ==
23 |
24 | * Release date:
25 | * Number of tickets:
26 |
27 | === Science highlights ===
28 |
29 | === Technical highlights ===
30 |
31 | === Optimisation highlights ===
32 |
33 | == Known Issues ==
34 |
35 | None
36 |
37 | == Revisions and Keywords ==
38 |
39 | || Repository || !Keyword/Revision ||
40 | || [browser:main UM] || vnXX.Y = NNNN ||
41 | || [https://code.metoffice.gov.uk/trac/jules/browser/main JULES] || vnA.B = umXX.y = NNNN ||
42 | || [https://code.metoffice.gov.uk/trac/monc/browser/casim CASIM] || etc ||
43 | || [https://code.metoffice.gov.uk/trac/socrates/browser/main SOCRATES] || ||
44 | || [https://code.metoffice.gov.uk/trac/ukca/browser/main UKCA] || ||
45 | || [browser:aux Aux] || ||
46 |
47 | GCOM and SHUMlib installations should be built using revisions:
48 |
49 | || Repository || !Keyword/Revision ||
50 | || [https://code.metoffice.gov.uk/trac/gcom/browser/main GCOM] || vnXX.Y = NNNN ||
51 | || [https://code.metoffice.gov.uk/trac/utils/browser/shumlib SHUMlib] || YYYY.MM.N = NNNN ||
52 |
53 | Note that SHUMlib may not be released at every UM release.
54 |
55 | For testing purposes, the LFRic Apps and MOCI repositories:
56 |
57 | || Repository || Revision ||
58 | || [browser:main/trunk/rose-stem/rose-suite.conf MOCI] || NNNN ||
59 | || [https://code.metoffice.gov.uk/trac/lfric_apps/browser/main LFRic Apps] || vnA.B = umXX.Y = NNNN ||
60 |
61 | Required for rose-stem:
62 |
63 | {{{#!comment
64 | Obtain these by grepping, for example:
65 | grep -r CYLC_VERSION ~frzz/cylc-run/um_heads_nightly_YYYY-MM-DD/log/job
66 | }}}
67 |
68 | * Rose vn YYYY.MM.N
69 | * Cylc vn YYYY.MM.N
70 |
71 | == Ticket Overview ==
72 |
73 | * Tickets marked as closed/fixed below may not be associated with a ``main`` commit
74 | * Only tickets with a UM element are displayed below. Sub-repositories may contain additional standalone tickets that will have been implicitly tested and included in the release.
75 |
76 | [[TicketQuery(milestone^=UMXX.Y (MMM-YY),status=closed,resolution=fixed,format=table,col=summary|reporter|keywords)]]
77 |
78 | == Platforms ==
79 |
80 | Testing:
81 | * Met Office SPICE: [wiki:ReleaseNotesXX.Y/ReleaseTestingAZSPICE]
82 | * Met Office EX1A: [wiki:ReleaseNotesXX.Y/ReleaseTestingEX1A]
83 |
84 |
85 | For the highlights section:
86 |
87 | * Contact HPC Optimisation team (usually AM) to request details for the optimisation section
88 | * Take some time to think about the 'big picture' relevance of tickets for the science and technical highlights
89 | * Copy the 'head of ``main``' section and paste underneath, changing the section title to the release.
90 | * Paste trac.logs into the relevant sub-pages. You'll likely be able to use some nightly testing runs.
91 |
--------------------------------------------------------------------------------
/source/Development/TestSuites/jules.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | Testing JULES
8 | =============
9 |
10 | JULES testing is run with the following command from the top directory of a
11 | local clone:
12 |
13 | .. code-block::
14 |
15 | cylc vip -z group=all -n ./rose-stem
16 |
17 | -----
18 |
19 | The JULES rose stem testing includes a range of builds using a variety of
20 | compilers, several configurations, and rose-ana tasks to check the output.
21 |
22 | The output is checked for correctness both by comparing the output to a set of
23 | stored :ref:`KGO files `.
24 |
25 | .. note::
26 |
27 | If there are JULES changes to shared science code or
28 | :ref:`jules-shared` metadata then these changes
29 | will need to be tested :ref:`with the UM` and
30 | :ref:`with LFRic Apps`. If you have access to LFRic, the
31 | :ref:`traclog` will state whether LFRic testing is required based on the
32 | branch diff. If you do not have LFRic access, this testing will need to
33 | be completed by your Met Office contact.
34 |
35 | See :ref:`multirepo` for details on how to carry out this testing.
36 |
37 | .. important::
38 |
39 | For **jules-shared** changes, when LFRic testing, the
40 | changes need to be manually synced to the LFRic location. When UM
41 | testing, this is not required as **jules-shared** is imported from the
42 | JULES branch.
43 |
44 | Below is a (by no means comprehensive) set of groups that you may wish to use
45 | on Met Office systems.
46 |
47 | +----------------------+----------------------------------------------------------+
48 | | Group | Description |
49 | +======================+==========================================================+
50 | +----------------------+----------------------------------------------------------+
51 | | all | The complete test suite. This is run automatically |
52 | | | every night and monitored by the SSD team. All |
53 | | | :ref:`KGO ` changing PRs need to run this group. |
54 | +----------------------+----------------------------------------------------------+
55 | +----------------------+----------------------------------------------------------+
56 | | loobos | A set of tests to exercise these science areas. |
57 | | | |
58 | | gswp2 | |
59 | | | |
60 | | eraint | |
61 | | | |
62 | | imogen | |
63 | +----------------------+----------------------------------------------------------+
64 | | ex1a/azspice | All tests designed to run on the named platform. |
65 | +----------------------+----------------------------------------------------------+
66 | | scripts | All of the auxillary scripts that are designed to check |
67 | | | the code standards in ways that aren't tested by the |
68 | | | compiler. |
69 | +----------------------+----------------------------------------------------------+
70 |
--------------------------------------------------------------------------------
/source/Reviewers/releases/standard_suites.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _standard_suites:
8 |
9 | Standard Suites
10 | ===============
11 |
12 | UM Standard Suites
13 | ------------------
14 |
15 | For the UM the following Suites must be updated:
16 |
17 | * UM Azspice - `u-aa003 `_
18 | * UM EX1A - `u-ci752 `_
19 | * SCM - `u-aa741 `_
20 | * CreateBC - `u-aa258 `_
21 |
22 | Ensure you have a :ref:`fork ` of both the ``um_meta`` and ``jules``
23 | repositories, and that the ``stable`` branches in each are up to date with the
24 | upstream repository.
25 |
26 | Then upgrade each suite by doing the following,
27 |
28 | * Checkout the suite. ``rosie checkout u-xxNNN`` will put a copy in ``~/roses``.
29 | * Check whether the ``access-list`` in the ``rose-suite.conf`` needs updating -
30 | if you are not on it, ask someone to add you now and update your copy once
31 | done.
32 | * Move into the suite and edit the ``rose-suite.conf`` file,
33 |
34 | * If it has a ``VN`` setting, update it to the new version.
35 | * If it has a ``prebuild`` path, update that now. If there are prebuilds,
36 | check the ``flow.cylc`` for any prebuild path overrides. These exist
37 | particularly on the EXs.
38 |
39 | * Validate the existing apps by running,
40 |
41 | .. code-block::
42 |
43 | rose macro --fix -C app/APP
44 |
45 | where ``APP`` should be replaced by each available app in turn. It should
46 | report that 0 changes were made.
47 | * Now upgrade the apps to the new version, using the metadata ``main`` checked
48 | out earlier. Again, replace ``APP`` with each app in turn. This will prompt
49 | you to confirm while running.
50 |
51 | .. code-block::
52 |
53 | rose app-upgrade -C app/APP -M /path/to/um_meta/clone vnX.Y
54 |
55 | * Finally run the validator macros. If everything is fine the command will
56 | report nothing, however the SCM suite will raise some warnings about Openmp
57 | and "global rows" - these can be ignored.
58 |
59 | .. code-block::
60 |
61 | rose macro metomi.rose.macros.DefaultValidators -M
62 | /path/to/um_meta/clone/main: /path/to/the/jules/hot/rose-meta
63 |
64 | * Check that the changes look sensible and then run the suite,
65 |
66 | .. code-block::
67 |
68 | fcm diff -g
69 | cylc vip
70 |
71 | .. _suite_commit:
72 |
73 | * Once the suite has succeeded, commit the changes (``fcm ci``) with a suitable
74 | message and note the revision it was committed at. Then tag the suite and
75 | commit the tag,
76 |
77 | .. code-block::
78 |
79 | fcm pe fcm:revision .
80 |
81 | fcm ci
82 |
83 |
84 | LFRic Apps Standard Suites
85 | --------------------------
86 |
87 | For LFRic Apps the following Suites must be updated:
88 |
89 | * LFRic Apps Azspice - `u-dn674 `_
90 | * LFRic Apps EX1A - `u-dn704 `_
91 |
92 |
93 | For each suite, do the following:
94 |
95 | * Get a copy of each of these suites and move into the working copy.
96 |
97 | .. code-block::
98 |
99 | rosie co u-dn704
100 | cd ~/roses/u-dn704
101 |
102 | * In ``rose-suite.conf`` update the version number
103 | * Upgrade the metadata for the mesh and lfric_atm apps
104 |
105 | .. code-block::
106 |
107 |
108 | rose app-upgrade -a -y -C app/lfric_atm vnX.Y
109 | rose app-upgrade -a -y -C app/mesh vnX.Y
110 |
111 | * Check the modules loaded in the ``flow.cylc`` for any changes vs. the Apps
112 | ``main``.
113 |
114 | * See ``rose-stem/site/meto/common/suite_config_PLATFORM.cylc``
115 | * The lfric software stack moves more quickly than the UM, so it is more
116 | likely these have changed.
117 |
118 | * Test - ``cylc vip``
119 | * :ref:`Commit and Tag ` the suite once the test suite has
120 | succeeded.
121 |
122 |
--------------------------------------------------------------------------------
/source/WorkingPractices/branch_migration.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _branch_migration:
8 |
9 | Migrating a branch from fcm to git
10 | ==================================
11 |
12 | .. important::
13 |
14 | If migrating an old trac ticket, please make sure to,
15 |
16 | * Cross-link between both the trac ticket and github PR/Issue
17 | * Add a ``git_migration:#NNN`` keyword to the trac ticket to avoid duplicate
18 | migrated tickets
19 |
20 | This page is intended to act as a guide when migrating branches from fcm to git
21 | after the initial git release. It assumes that you have already :ref:`created
22 | a fork ` of the repo you are migrating to.
23 |
24 | .. important::
25 |
26 | The process below involves rsyncing the changes in your fcm working copy to
27 | a git clone. This requires that the two branches have equivalent branch
28 | points, so ensure this is the case.
29 |
30 | For convenience, the ``git_migration`` tag is provided as a branch point in
31 | both fcm and git. However all ``fcm`` revisions and tags have a direct
32 | equivalent commit on git - tags will have the same name on both, revisions
33 | of specific commits will need to be manually aligned with a commit hash by
34 | comparing commit messages.
35 |
36 | #. Optionally, create a new branch in ``fcm`` using the tag ``git_migration``.
37 | Then merge your development branch onto this one, eg.
38 |
39 | .. code-block::
40 |
41 | fcm bc --type=dev branch_for_migration fcm:lfric_apps.x_tr@git_migration
42 | ...
43 | fcm merge fcm:lfric_apps.x_br/dev/USER/BRANCH
44 |
45 | Resolve any conflicts and then commit these changes to this branch,
46 | ``fcm ci``.
47 |
48 | For Jules users, upgrading to the ``git_migration`` tag is required.
49 |
50 | #. Move into your git clone and :ref:`create a new branch ` with
51 | the same start point as your fcm branch. If you are branching from an
52 | untagged revision, you will need to manually find the relevant hash for that
53 | commit from the git log by comparing commit messages.
54 |
55 | .. code-block::
56 |
57 | git switch -c
58 | e.g. git switch -c new_migrated_branch git_migration
59 |
60 | If the ``git_migration`` tag doesn't exist in your clone, then you may need
61 | to :ref:`sync tags `.
62 |
63 | #. Rsync the changes over from the fcm working copy to the git clone. Use
64 | ``--delete`` to remove any files you have deleted in your branch. Use
65 | ``--exclude=.git --exclude=.svn`` so that the version control directories
66 | aren't modified.
67 |
68 | .. code-block::
69 |
70 | # NOTE: You need the trailing backslash on the fcm source path
71 | rsync -av --delete --exclude=.git --exclude=.svn path/to/fcm/working/copy/ path/to/git/clone
72 |
73 | #. Check carefully the output of the rsync via ``git status``. If you have new
74 | files on your branch these will need adding via ``git add``.
75 | #. Finally, all branches will **need** to update to the initial git release in
76 | order to run the test suites. This can be done by merging the ``stable``
77 | branch into your new branch. See :ref:`updating a branch `
78 | for more details, noting that you will need to use one of the options
79 | using ``git commands`` for this step.
80 |
81 | If you have a linked ticket you will manually need to :ref:`update the
82 | dependencies.yaml ` to point at your linked branches. For
83 | lfric_apps branches, ensure that the ``dependencies.sh`` file hasn't been
84 | readded to the branch.
85 | #. It may be worth running the :ref:`test suite ` to ensure the branch
86 | has been properly migrated.
87 |
88 | .. tip::
89 |
90 | There will be significant changes to the rose-stem test suites between the
91 | ``git_migration`` point and the initial git release. Therefore, conflicts
92 | are expected when updating rose-stem changes to the initial release. The UM,
93 | Jules and UKCA test suites have all had file extensions renamed from ``.rc``
94 | to ``.cylc``.
95 |
96 | For lfric_core a minor source code change has been added to github, which
97 | may result in conflicts in ``components/lfric-xios/source/lfric_xios_file_mod.f90``
98 |
--------------------------------------------------------------------------------
/source/index.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. WorkingPractices documentation master file, created by
8 | sphinx-quickstart on Thu Sep 8 14:00:33 2022.
9 | You can adapt this file completely to your liking, but it should at least
10 | contain the root `toctree` directive.
11 |
12 | :html_theme.sidebar_secondary.remove: true
13 |
14 | Simulation Systems Working Practices
15 | ====================================
16 |
17 | .. important::
18 |
19 | The working practices for working with git and GitHub have recently been updated.
20 | The previous FCM working practices can still be accessed through the source history.
21 | Met Office staff can find an internal build `here
22 | `_
23 |
24 | **These pages describe the working practices of the following simulation and
25 | model codes owned by the Met Office:** `LFRic Applications`_, `LFRic Core`_, `UM`_, `JULES`_, `SOCRATES`_,
26 | `CASIM`_ and `UKCA`_.
27 |
28 | This includes how to get started, key points on developing your change and how
29 | to test those developments. There is guidance on making changes that span multiple
30 | projects and how to progress your change through review.
31 |
32 | There are then notes for reviewers on how to tackle the different types of review
33 | and how to merge to ``main``.
34 |
35 |
36 |
37 | .. grid:: 1 1 2 2
38 |
39 | .. grid-item-card::
40 | :text-align: center
41 |
42 | Working with git and GitHub to contribute to the simulation models.
43 |
44 | +++
45 | .. button-ref:: working_practices_index
46 | :ref-type: ref
47 | :color: primary
48 | :outline:
49 | :expand:
50 |
51 | Working Practices
52 |
53 | .. grid-item-card::
54 | :text-align: center
55 |
56 | Guides to planning, developing and testing changes to the simulation models
57 |
58 | +++
59 | .. button-ref:: development_index
60 | :ref-type: ref
61 | :color: primary
62 | :outline:
63 | :expand:
64 |
65 | Developing Your Change
66 |
67 | .. grid:: 1 1 2 2
68 |
69 | .. grid-item-card::
70 | :text-align: center
71 |
72 | Information on reviewing for, committing to and releasing these projects.
73 |
74 | +++
75 | .. button-ref:: reviewers_index
76 | :ref-type: ref
77 | :color: primary
78 | :outline:
79 | :expand:
80 |
81 | Guides for Reviewers
82 |
83 | .. grid-item-card::
84 | :text-align: center
85 |
86 | Support information, glossary and code of conduct.
87 |
88 | +++
89 | .. button-ref:: further_details
90 | :ref-type: ref
91 | :color: primary
92 | :outline:
93 | :expand:
94 |
95 | Further Details
96 |
97 |
98 |
99 |
100 |
101 |
102 | .. toctree::
103 | :maxdepth: 1
104 | :hidden:
105 | :caption: Working Practices
106 |
107 | WorkingPractices/working_practices
108 |
109 | .. toctree::
110 | :maxdepth: 1
111 | :hidden:
112 | :caption: Developing Your Change
113 |
114 | Development/developing_change
115 |
116 | .. toctree::
117 | :maxdepth: 1
118 | :hidden:
119 | :caption: Guides for Reviewers
120 |
121 | Reviewers/index
122 |
123 | .. toctree::
124 | :maxdepth: 1
125 | :hidden:
126 | :caption: Further Details
127 |
128 | FurtherDetails/index
129 |
130 | .. toctree::
131 | :maxdepth: 1
132 | :hidden:
133 |
134 | git_faq
135 |
136 |
137 | More detailed, project specific, documentation is also available. See the wiki pages in each repository as well as
138 | the :ref:`docs` page for more information on what is available and how to contribute to it.
139 |
140 | Please see our :ref:`Git Migration Frequently Asked Questions `.
141 |
142 | .. _LFRic Applications: https://github.com/MetOffice/lfric_apps
143 | .. _LFRic Core: https://github.com/MetOffice/lfric_core
144 | .. _UM: https://github.com/MetOffice/um
145 | .. _CASIM: https://github.com/MetOffice/casim
146 | .. _JULES: https://github.com/MetOffice/jules
147 | .. _SOCRATES: https://github.com/MetOffice/socrates
148 | .. _UKCA: https://github.com/MetOffice/ukca
149 |
--------------------------------------------------------------------------------
/source/Development/kgo.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _kgo:
8 |
9 | Known Good Output (KGO)
10 | =======================
11 |
12 | Normally it is to be expected that code changes regress (i.e. all prognostic
13 | variables **and** diagnostics maintain the same answers with your code
14 | included but switched off).
15 |
16 | Usually, a code change does not regress when either a bug is a discovered and
17 | fixed, or a science area is introduced or enhanced. Sometimes, a KGO update
18 | may also be required to simply add a new job to the test suite or to port the
19 | rose stem suite to new HPC architecture.
20 |
21 | **LFRic** KGO checksums are stored in the repository. As such with LFRic
22 | pull requests the expectation is that you, as the developer, will include
23 | updated KGO files as part of your branch.
24 |
25 | **UM and JULES** KGO output files are stored outside of the repository. Access
26 | to this area is restricted to members of the Simulation Systems and Deployment
27 | Team and they will update the KGO files to include your new answers as part of
28 | the commit process.
29 |
30 |
31 | KGO Update Process
32 | ------------------
33 |
34 | Getting the process right for KGO changing pull requests significantly helps get
35 | such changes onto ``main``. When preparing your change for review:
36 |
37 | 1. Run the ``all`` rose-stem group in order to make sure that all
38 | changes to answers have been found.
39 |
40 | * Include the :ref:`trac.log ` output from this testing in your
41 | pull request summary.
42 |
43 | 2. Add the ``kgo`` label to your pull request.
44 |
45 | .. tab-set::
46 |
47 | .. tab-item:: LFRic Apps
48 |
49 | 3. Update the checksum files on your branch. To do so run rose stem and
50 | then the following from the head of your clone.
51 |
52 | .. code-block:: shell
53 |
54 | python3 ./rose-stem/bin/update_branch_kgos.py \
55 | -s -w
56 |
57 | .. note::
58 |
59 | This script requires at least python 3.9. This can be achieved
60 | on Met Office machines by running ``module load scitools``
61 |
62 | 4. If you are adding new checksums, ``git add`` the files.
63 |
64 | 5. You can check the new kgo updated properly by retiggering tasks in
65 | the test suite. First retrigger ``export-source``, and then when
66 | complete ``export-source_ex1a`` if new checksums are present there
67 | (there is no need to retigger spice). You may need to change the
68 | maximum window extent of the gui in order to see the succeeded
69 | tasks. Now you can retrigger the failed checksums - these should now
70 | pass if the kgo was updated in the clone correctly.
71 |
72 | 6. The changes in answers should be science reviewed by someone
73 | familiar with the failing tests - if unsure then start with the Code
74 | Owner for the affected application.
75 |
76 |
77 | Once all the above is in place and the science and code reviews have
78 | been completed then the Code Reviewer will merge head of ``main`` onto
79 | your branch. If there are merge conflicts in the checksum files then
80 | the reviewer will repeat step 3 to refresh these files. Your change is
81 | then merged and committed to ``main``.
82 |
83 | .. tab-item:: UM & JULES
84 |
85 | 3. Contact the configuration owners of all affected configurations. A
86 | list is provided in the trac.log . See :ref:`approvals` for details.
87 |
88 | Configuration owners will review the change and will either accept
89 | the change as it is, or will request the use of a temporary
90 | variable to switch the change off. See :ref:`templogicals` for
91 | details of this process.
92 |
93 | Once all of the above is in place and the science and code reviews have
94 | been completed the Code Reviewer will merge head of ``main`` onto
95 | your branch, run the tests that have changed answers and use those
96 | results to install KGO files to the filesystem. Your branch is then
97 | merged and committed to ``main``.
98 |
99 | .. tip::
100 |
101 | More details on the KGO update proceedures for all repositories can be
102 | found on the :ref:`How to Commit page`.
103 |
104 | .. toctree::
105 | :hidden:
106 |
107 | temp_logicals
108 |
--------------------------------------------------------------------------------
/source/Development/TestSuites/multi-repo_testing.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _multi-repo_testing:
8 |
9 | Multi-Repository Testing
10 | ========================
11 |
12 | Multi-repository changes are expected to pass the regression tests for all the
13 | repositories involved. To carry out the tests involved in a linked pull request
14 | it can be helpful to refer to the :ref:`repository figure `; testing
15 | both child and parent repositories as needed. Further details of how testing in
16 | each repository is handled can be found on the :ref:`Testing page`.
17 |
18 | All Simulation Systems repositories containing a test suite will also contain a
19 | ``dependencies.yaml`` file in the top directory of the repository. This file
20 | contains the details of all sources used by the test suite in the format (using
21 | lfric_core as an example):
22 |
23 | .. code-block:: yaml
24 |
25 | lfric_core:
26 | source: git@github.com:MetOffice/lfric_core.git
27 | ref:
28 |
29 | The ``source`` setting sets the location of the repository on GitHub. By
30 | default, the test suite will access GitHub repositories by using ssh, as shown
31 | by the ``git@github.com`` part of the source. This line can be modified to point
32 | at users fork of the repository instead, eg.
33 | ``source: git@github.com:UserName/lfric_core.git``. The ``source`` can also be a
34 | local git clone, in which case it should take the form,
35 | ``source: :/path/to/clone``.
36 |
37 | The ``ref`` setting takes a git tree-ish value. Common settings will be a commit
38 | hash, a tag or a branch name as demostrated by the examples below. At
39 | release, the refs will be tags and will be changed to the long form of the relevant
40 | commit hash as part of linked pull requests.
41 |
42 | If left blank the behaviour depends on the source:
43 |
44 | * **a GitHub source:** the Head of the repositories default branch will be used.
45 | * **a local clone:** the state of the repository at source extraction time will be used.
46 | It is recommened to set a ref when setting the source to a local clone. That way
47 | if you switch branches in the clone, the correct branch for testing will be used.
48 |
49 | Various different configurations of an lfric_core source are shown below with an
50 | explanation of each,
51 |
52 | .. code-block:: yaml
53 |
54 | # The upstream lfric_core repository at tag 3.0
55 | lfric_core:
56 | source: git@github.com:MetOffice/lfric_core.git
57 | ref: core3.0
58 |
59 | # The upstream lfric_core repository at a specific commit hash
60 | lfric_core:
61 | source: git@github.com:MetOffice/lfric_core.git
62 | ref: a1b2c3d4e5f67890abcdef1234567890abcdef12
63 |
64 | # As above, but with a shortened form of the hash (7 characters in this case)
65 | lfric_core:
66 | source: git@github.com:MetOffice/lfric_core.git
67 | ref: a1b2c3d
68 |
69 | # The upstream lfric_core repository, with the default branch (main) at Head
70 | lfric_core:
71 | source: git@github.com:MetOffice/lfric_core.git
72 | ref:
73 |
74 | # A Users fork of the lfric_core repoistory, on branch my_branch
75 | lfric_core:
76 | source: git@github.com:UserName/lfric_core.git
77 | ref: my_branch
78 |
79 | # A Users fork of the lfric_core repository at a specific hash
80 | lfric_core:
81 | source: git@github.com:UserName/lfric_core.git
82 | ref: f9e8d7c
83 |
84 | # A local clone of the lfric_core repository, pointing at my_branch
85 | lfric_core:
86 | source: hostname:/path/to/lfric_core
87 | ref: my_branch
88 |
89 | # A local clone of the lfric_core repository, pointing at a specific hash
90 | lfric_core:
91 | source: hostname:/path/to/lfric_core
92 | ref: f9e8d7c
93 |
94 |
95 | .. note::
96 |
97 | If any of the testing shows up failures then there are two possible ways to
98 | proceed:
99 |
100 | 1. The changes made should be re-written to avoid breaking the dependant
101 | repositories
102 |
103 | 2. The changes made directly affect the interface between repositories and
104 | therefore a change is also needed to the child repository to adapt to that change.
105 |
106 | If you're uncertain which route to take then the Code Owners involved will
107 | hopefully be able to advise.
108 |
--------------------------------------------------------------------------------
/source/Development/TestSuites/lfric_apps.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _lfric_apps_test:
8 |
9 | Testing LFRic Apps
10 | ==================
11 |
12 | Rose stem:
13 |
14 | The LFRic Apps rose-stem test suite can be run from the top directory of a
15 | local clone:
16 |
17 | .. code-block:: shell
18 |
19 | cylc vip -z group=developer -n ./rose-stem
20 |
21 | Local testing:
22 |
23 | Alternatively, a single application can be built and run locally using
24 | `these instructions
25 | `__
26 |
27 | This test does not use rose or cylc and is particularly useful for checking
28 | for compile errors while developing.
29 |
30 |
31 | Rose stem
32 | ---------
33 |
34 | The LFRic Apps rose stem includes a range of tests to exercise all the
35 | applications stored in this repository, using multiple compilers, and checksum
36 | and plot tasks to confirm the outputs.
37 |
38 | .. tip::
39 |
40 | For LFRic Apps it is possible to update the checksum files on your branch
41 | as you progress your development to aid with testing. Details on how to do
42 | this are on the :ref:`KGO page `.
43 |
44 | Below is a (by no means comprehensive) set of groups that you may wish to use
45 | on Met Office systems. Note that there is a lot of overlap between these
46 | groups, and that you can specify more than one at once, e.g.
47 | ``-z group=developer,gungho_model``.
48 |
49 | +--------------------+----------------------------------------------------------+
50 | | Group | Description |
51 | +====================+==========================================================+
52 | | developer | Standard group of tests that every change is expected |
53 | | | to pass before commit. It is a useful checkpoint during |
54 | | | development. |
55 | +--------------------+----------------------------------------------------------+
56 | | all | The complete test suite, including all longer runs and |
57 | | | less commonly used configs. This is run automatically |
58 | | | every week and monitored by the SSD team. All |
59 | | | :ref:`KGO ` changing PRs need to run this group. |
60 | +--------------------+----------------------------------------------------------+
61 | +--------------------+----------------------------------------------------------+
62 | | build | Compile tasks for all applications and science areas |
63 | +--------------------+----------------------------------------------------------+
64 | | unit_tests | Unit tests for all applications and science areas |
65 | +--------------------+----------------------------------------------------------+
66 | | integration_tests | Integration tests for all applications and science areas |
67 | +--------------------+----------------------------------------------------------+
68 | | ex1a/azspice | All tests designed to run on the named platform. |
69 | +--------------------+----------------------------------------------------------+
70 | | scripts | All of the auxillary scripts that are designed to check |
71 | | | the code standards in ways that aren't tested by the |
72 | | | compiler. |
73 | +--------------------+----------------------------------------------------------+
74 |
75 | As well as these general groups, each area in ``/applications`` and
76 | ``/science`` have a set of specific groups that are structured as
77 | below, with ``name`` matching the directory name of the area.
78 |
79 | +--------------------+----------------------------------------------------------+
80 | | Group | Description |
81 | +====================+==========================================================+
82 | | | Full set of tests for this area |
83 | +--------------------+----------------------------------------------------------+
84 | | _ | Tests for this area that are part of this group |
85 | | | e.g. gungho_developer or lfric_atm_spice |
86 | +--------------------+----------------------------------------------------------+
87 |
88 |
--------------------------------------------------------------------------------
/source/Reviewers/releases/mule_release.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _mule_release:
8 |
9 | Mule Release
10 | ============
11 |
12 | .. note::
13 |
14 | Additional details for the shumlib/mule releases can be seen `here
15 | `__
16 |
17 |
18 | Releasing Mule
19 | --------------
20 |
21 | .. tip::
22 |
23 | See `PR#2 `__ for an example mule
24 | release PR.
25 |
26 | There's no hard rule on whether a mule release is required - it comes down to
27 | whether there have been any notable changes to mule since the last release. If
28 | you are doing a release then do the following,
29 |
30 | * Ensure you have a :ref:`fork ` of ``mule`` and that the ``main``
31 | branch is up to date with the upstream repository.
32 | * In a clone of the fork, :ref:`create a branch ` using the
33 | ``main`` branch as the parent.
34 |
35 | .. important::
36 |
37 | Ensure you create branches from main, otherwise you will not include the
38 | changes from the past release.
39 |
40 | * Update ``admin/meto_install_mule.sh`` to update the mule version number as
41 | well as the UM and Shumlib releases intended for this mule release. There's no
42 | requirement for a link between mule and UM releases.
43 | * Update all occurrences of the mule version number to the new version. This
44 | occurs in multiple places - see the above example PR or grep for the old
45 | version.
46 | * Test the branch against the UM
47 | * With a reviewer, follow the :ref:`release process `
48 | * Once done, announce the release on simulation-systems discussions as well as
49 | VE, possibly in the Scientific Software Tools community.
50 |
51 | .. note::
52 |
53 | Check that the mule and um_utils docs have built and deployed correctly to
54 | GitHub pages.
55 |
56 | Installing Mule
57 | ---------------
58 |
59 | .. important::
60 |
61 | This must come after the targeted UM and Shumlib releases have been
62 | installed.
63 |
64 | Installing mule has complexities as it depends on libraries installed by the UM
65 | and shumlib and is also dependent on the python stack. It is possible to
66 | install without the UM/Shumlib libraries, however functionality will be
67 | missing and so this isn't done on Met Office platforms.
68 |
69 | Mule is loaded into a user's environment by adding the install directories to
70 | ``$PATH`` and ``$PYTHONPATH`` when loading the ``um_tools`` module. To ensure
71 | mule works with the current environment, it is installed for all different
72 | combinations of python and numpy available. At the Met Office, these are
73 | provided by the scitools software stacks in addition to the default
74 | environment.
75 |
76 | Installation at the Met Office is done by ``admin/meto_install_mule.sh`` which
77 | will detect the current python environment and install to a directory named
78 | for that.
79 |
80 | To install at the Met Office
81 |
82 | * Login as umadmin
83 | * Move to or create ``$UMDIR/mule/mule-YYYY.MM.V``
84 | * Get a clone of the mule repo. Use the https source for this as the shared
85 | account doesn't have ssh access to GitHub.
86 | * Run the install script without any modules loaded (this will install for the
87 | system python), ``./mule/admin/meto_install_mule.sh``
88 | * For all desired scitools modules, load the module and then rerun the install
89 | script. Try and do this for all production/preproduction stacks as well as
90 | the current default previous, current and next modules.
91 | * Repeat these steps on the EXAB, EXCD, and EXZ. As of ``2025.10.1`` this will
92 | no longer install for the system python on the EXs.
93 |
94 | Once mule has been installed, we also need to add the modulefiles to their
95 | location in ``$UMDIR/modules/modulefiles/um_tools/YYYY.MM.V``. This is most
96 | easily done by copying an existing one and modifying the scitools modules and
97 | python versions to match what has just been installed. These modules need to
98 | be done for both ``openmp`` and ``no-openmp`` - the only differences between
99 | them are the ``THREADING`` and ``module-whatis`` variables.
100 |
101 | Once done, check the modules are setup correctly by,
102 |
103 | .. code-block:: shell
104 |
105 | module load um_tools/YYYY.MM.V/openmp
106 | python -c "import mule"
107 |
108 | Again, repeat this on the EXAB, EXCD, and EXZ. The module files should be
109 | identical for both EXAB and EXCD, so you can ``scp`` one set to the other.
110 | Again, these files can be found via the same $UMDIR path as above.
111 |
112 | Finally, it is a good idea to update the default mule module by editing
113 | ``$UMDIR/modules/modulefiles/um_tools/.version``. It may be worth posting an
114 | announcement a few days before changing this to give advance notice.
115 |
--------------------------------------------------------------------------------
/source/Development/rose_stem.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | Adding to Rose Stem Suites
8 | ==========================
9 |
10 | .. tip::
11 |
12 | This page is for advice on adding new tasks to a rose-stem suite. For advice
13 | on running the test suite following the move to GitHub, please see
14 | :ref:`testing your change `.
15 |
16 | All new changes are strongly encouraged to come with an update to the rose stem
17 | suite to protect any new functionality. Configuration owners may also wish to
18 | update the suite to ensure that important configurations are protected by the
19 | rose stem suite.
20 |
21 | .. warning::
22 |
23 | If you find that you need to update all the apps in the rose stem suite to
24 | get your change to work then you should use an upgrade macro.
25 | See :ref:`inputs`.
26 |
27 |
28 | Adding a new app
29 | ----------------
30 |
31 | Adding a new app to rose-stem can look daunting, but is actually less difficult
32 | than many other development processes as there are many examples to reference
33 | already in the code.
34 |
35 | 1. rose-stem/app/
36 | Start by creating a new app with the details of the test you wish to run.
37 | This might be an optional configuration for an existing app, or something
38 | completely new.
39 |
40 | For the UM also create a matching rose-ana app detailing the comparisons
41 | required.
42 |
43 | Both of these will be easiest to copy an existing app that is similar to
44 | what you are creating and modify from there. You can use the rose edit GUI
45 | or manually modify the ``rose-app.conf`` files.
46 |
47 | .. note::
48 |
49 | The app should validate (i.e. be consistent with the Rose metadata, and
50 | produce no warnings or errors). This can be checked by running ``rose
51 | macro --validate --no-warn version -M path/to/rose-meta`` from within
52 | your new app directory, or selecting Metadata -> Check fail-if,
53 | warn-if from the drop-down menu of the rose edit GUI.
54 |
55 | .. important::
56 |
57 | If you are working in a UM branch and have **jules-shared** changes,
58 | the JULES metadata path will also need adding. Please see the shared
59 | metadata :ref:`guidance`.
60 |
61 | The next steps are site and rose-stem specific, but fall broadly into two
62 | categories - those using a jinja2 templated design which populate runtime and
63 | graph sections for you, such as the UM METO suite, and those which are
64 | manually configured, such as JULES.
65 |
66 |
67 | .. tab-set::
68 |
69 | .. tab-item:: Templated
70 |
71 | 2. Task definitions
72 | Task definitions should be added to the ``tasks.cylc`` for all sites
73 | who will run this app.
74 |
75 | The tasks are defined in a dictionary format, with one entry per
76 | configuration and all other details are handled by the templates.
77 | It is possible to define multiple decomposition and thread options
78 | in this single entry and comparison details are also defined
79 | within that same dictionary entry.
80 |
81 | Details of the available options are listed in the template files
82 | in ``rose-stem/templates`` for each suite and looking at existing
83 | examples is encouraged.
84 |
85 | .. note::
86 |
87 | LFRic Apps has a `detailed set of pages
88 | `__
89 | that document the structure and options available for their
90 | suite.
91 |
92 |
93 | .. tab-item:: Manual
94 |
95 | 2. Task definitions
96 | Task definitions for the following tasks should be added to the
97 | ``runtime.cylc`` for all sites who will run this app.
98 |
99 | * run the app
100 | * perform a KGO comparison
101 | * perform housekeeping
102 |
103 | 3. Graph definitions
104 | Graph definitions should be added to the graph.cylc for all sites
105 | who will run this app. These should connect together your new tasks
106 | created above with an appropriate build task.
107 |
108 | .. tip::
109 |
110 | The site specific information is held in:
111 | * JULES: rose-stem/include
112 | * LFRic Apps & UM: rose-stem/site
113 |
114 | .. tip::
115 |
116 | All ``.cylc`` files mentioned are frequently split into
117 | platform specific variants depending on the complexity of the sites
118 | suite.
119 |
120 | e.g. `runtime.cylc` may be spread across ``runtime-platform1.cylc`` and
121 | ``runtime-platform2.cylc``. If a task should be run on both platform1 and
122 | platform2 then both of these will need the task definition adding.
123 |
--------------------------------------------------------------------------------
/source/WorkingPractices/images/repo_circles.svg:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
128 |
--------------------------------------------------------------------------------
/source/Development/jules_docs.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _jules_docs:
8 |
9 | The JULES User Guide
10 | ====================
11 |
12 | The JULES User Guide is built using the :ref:`Sphinx Documentation Generator
13 | `. The source files for the JULES documentation are contained
14 | in the `JULES GitHub repository`_, with the plain text files contained in the
15 | `source`_ directory.
16 |
17 | .. _JULES GitHub repository: https://github.com/jules-lsm/jules-lsm.github.io
18 | .. _source: https://github.com/jules-lsm/jules-lsm.github.io/tree/master/user_guide/doc/source
19 |
20 | The JULES User Guide uses some custom extensions to reStructuredText to allow
21 | it to represent Fortran namelists more effectively - these are discussed in
22 | more detail below.
23 |
24 |
25 | reStructuredText Extension for Fortran Namelists
26 | ------------------------------------------------
27 |
28 | The JULES User Guide uses a custom extension to reStructuredText to allow a
29 | more natural expression of Fortran namelists
30 | (see `user_guide/doc/sphinxext/sphinx_nml_domain.py`_ if you are interested in
31 | the implementation).
32 |
33 | .. _user_guide/doc/sphinxext/sphinx_nml_domain.py: https://github.com/jules-lsm/jules-lsm.github.io/blob/master/user_guide/doc/sphinxext/sphinx_nml_domain.py
34 |
35 | Documenting namelists
36 | ---------------------
37 |
38 | To begin documenting a namelist, the directive
39 |
40 | .. code-block:: text
41 |
42 | .. nml:namelist::
43 |
44 | is used. By convention, namelist names are ``UPPER_CASE``, while namelist
45 | member names are ``lower_case``.
46 |
47 | The ``nml:namelist`` directive does not output anything, but indicates that all
48 | subsequently declared members belong to the namelist (up until the next
49 | occurrence of ``nml:namelist``).
50 |
51 | Once a namelist has been declared, the members of that namelist are documented
52 | using the directive
53 |
54 | .. code-block:: text
55 |
56 | .. nml:member::
57 |
58 | :type: [e.g. real, integer, logical]
59 | :permitted: [Permitted values, e.g. > 0, 1-5]
60 | :default: [Default value]
61 |
62 | First paragraph describing this namelist member.
63 |
64 | Second paragraph describing this namelist member.
65 |
66 | ...
67 |
68 | The white-space (indentation and blank lines) is very important here. The
69 | ``:permitted:`` annotation is optional, and can be omitted if any value is
70 | acceptable. If the member has no default value, ``:default: None`` should be
71 | used. The description of the namelist member can contain any valid
72 | reStructuredText markup, as long as it is indented correctly.
73 |
74 | The final directive used to document namelists is:
75 |
76 | .. code-block:: text
77 |
78 | .. nml:group::
79 |
80 | .. nml:member::
81 |
82 |
83 | .. nml:member::
84 |
85 |
86 | ``nml:group`` is used to group logically related members within a namelist. Any
87 | number of members can be contained within it, but they must be indented. Any
88 | un-indented members end the group.
89 |
90 | For an example of how ``nml:group`` might be used, see the documentation of
91 | ``JULES_INPUT_GRID`` in `model_grid.nml`_. To see how the nml:group directive
92 | is rendered, see `JULES_INPUT_GRID namelist members`_.
93 |
94 | .. _model_grid.nml: https://jules-lsm.github.io/latest/namelists/model_grid.nml.html
95 | .. _JULES_INPUT_GRID namelist members: https://jules-lsm.github.io/latest/namelists/model_grid.nml.html#jules-input-grid-namelist-members
96 |
97 | Note - If you are adding a completely new namelist then the namelist name also
98 | needs to be added to the contents page in source/namelists/contents.rst in
99 | order for it to be included in the build.
100 |
101 |
102 | Cross-referencing namelists and namelist members
103 | ------------------------------------------------
104 |
105 | The custom reStructuredText extension for Fortran namelists also provides
106 | facilities for easily cross-referencing namelists and namelist members from
107 | anywhere in the User Guide.
108 |
109 | To insert a cross-reference to a namelist anywhere in the documentation, use
110 | the following within any normal piece of text:
111 |
112 | .. code-block:: text
113 |
114 | :nml:lst:``
115 |
116 | Similarly, to cross-reference a namelist member:
117 |
118 | .. code-block:: text
119 |
120 | :nml:mem:`::`
121 |
122 | So to link to the member ``l_aggregate`` of namelist ``JULES_SURFACE``, we
123 | would use the following:
124 |
125 | .. code-block:: text
126 |
127 | This is some text, with a link to :nml:mem:`JULES_SURFACE::l_aggregate`
128 | embedded.
129 |
130 | The cross-references are rendered as hyperlinks in the HTML version, and link
131 | to different parts of the document in the PDF version.
132 |
133 |
134 | Checking for broken hyperlinks
135 | -------------------------------
136 |
137 | One can test whether there are broken hyperlinks in the user guide by running
138 |
139 | .. code-block:: shell
140 |
141 | make linkcheck
142 |
--------------------------------------------------------------------------------
/source/Reviewers/committinglinkedtickets.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _committinglinkedtickets:
8 |
9 | Committing Linked Pull Requests
10 | ===============================
11 |
12 | How do linked pull requests work?
13 | ---------------------------------
14 |
15 | Linked PRs contain changes that all need to be committed together to work
16 | successfully. With only some of the changes committed the repositories are
17 | considered "out of sync", with some of the test suites likely to fail as the
18 | api between the codebases is broken. For this reason, where possible, all
19 | parts of a linked PR should be committed on the same day to avoid nightly
20 | tests failing.
21 |
22 | :ref:`Multi-repository ` changes are nested, and the different
23 | branches will need approaching in the correct order. The UM and LFRIc Apps are
24 | the key places where these overlap.
25 |
26 | 1. Everything except UM and LFRic Apps can be worked on separately and should
27 | be committed first.
28 | 2. LFRic Apps and the UM each rely on code from all of the above code bases,
29 | and will need that code for both testing and committing. They do not rely on
30 | each other.
31 |
32 | .. tip::
33 |
34 | While it is possible to work through the commit process for each repository
35 | in turn, following this list in order, this can take a lot of time and so
36 | it is prudent to parallelise the process where possible.
37 |
38 | A suggested sequence would be as follows:
39 |
40 | 1. Complete the merge and macro stages for every repository. These steps
41 | are entirely isolated and so order doesn't matter.
42 |
43 | 2. Test all of the changes together as described below.
44 |
45 | 3. Install KGO files for all repositories requiring them
46 |
47 | 4. Commit the PRs as described below.
48 |
49 |
50 | .. _testinglinked:
51 |
52 | Testing linked Pulls Requests
53 | -----------------------------
54 |
55 | To test the change, ensure that all branches are up to date with the main branch
56 | of each repository, and get clones of these branches.
57 |
58 | Details for testing multi-repository PRs are included on the
59 | :ref:`Working with Multiple Repositories page`.
60 |
61 | **In summary:**
62 |
63 | JULES, UKCA, LFRic Core and other parent repositories can be tested using their
64 | standalone test suites as described on the How to Commit page.
65 |
66 | UM and LFRic Apps changes will require modifying the ``dependencies.yaml`` file
67 | to update the source being used.
68 |
69 | * In the UM/Apps clone, edit the relevant sources and refs in the
70 | ``dependencies.yaml`` file. These can be either local clones or GitHub urls.
71 | See :ref:`Multi-Repo Testing ` for more details
72 |
73 |
74 | .. tip::
75 |
76 | It is always important that branches and working copies used for testing
77 | multiple repositories together have been taken at the same point in time.
78 | If this isn't the case then API breaking changes may be included in one
79 | repository but not another which will cause tests to fail.
80 |
81 | The developer will likely have used branches taken from the last releases
82 | which are a known set of stable commits which work together.
83 |
84 | Make sure the testing done here (just prior to commit) is using the latest
85 | head of all the trunks. Assuming nightly tests are passing then this is
86 | also a known set of commits that work together.
87 |
88 | .. tip::
89 |
90 | If some of the changes in this set of PRs have already been committed
91 | then see steps 2 and 4 below on how to include those changes in your
92 | testing. This is instead of the steps described above.
93 |
94 | .. _committinglinked:
95 |
96 | Committing linked Pull Requests
97 | -------------------------------
98 |
99 | Once you are happy with all your testing then the commit sequence is as
100 | follows:
101 |
102 | * Commit all trunks **except** UM and LFRic Apps. Make note of the commit
103 | hashes.
104 |
105 | * For each of LFRic Apps and UM as required,
106 |
107 | * In a clone of the developers branch, edit the ``dependencies.yaml`` file:
108 |
109 | * Ensure the entry for the repository this file is in is fully blank.
110 | * Ensure the ``source`` entry points at the MetOffice ssh url.
111 | * Modify ``ref`` entry for all updated repositories points to the full
112 | hash for the relevant commit.
113 | * e.g. If a JULES PR has been committed with hash starting abc123
114 | and a UKCA PR starting at 456def, the UM dependencies file will
115 | have these entries (amongst others):
116 |
117 | .. code-block:: yaml
118 |
119 | jules:
120 | source=git@github.com:MetOffice/jules.git
121 | ref=abc123##################################
122 |
123 | ukca:
124 | source=git@github.com:MetOffice/UKCA.git
125 | ref=456def##################################
126 |
127 | um:
128 | source=
129 | ref=
130 |
131 | * Commit these changes and push back to the developers branch, along with
132 | any changes to macros and KGO. Finally you can :ref:`commit `
133 | the pull request.
134 |
--------------------------------------------------------------------------------
/source/WorkingPractices/reviews.rst:
--------------------------------------------------------------------------------
1 | .. _review_process:
2 |
3 | Review Process
4 | ==============
5 |
6 | .. tip::
7 |
8 | Github allows reviewers to directly make suggestions to the code. This is
9 | very useful for easily suggesting changes. However, the developer should
10 | always check carefully that the change is sensible and doesn't contain any
11 | errors or bugs.
12 |
13 | Selecting Reviewers
14 | -------------------
15 |
16 | There is a space in the pull request template to list the GitHub user ID of the
17 | scitech and code reviewers. These can also be filled into the appropriate
18 | :ref:`project spaces `.
19 |
20 | SciTech Review
21 | ^^^^^^^^^^^^^^
22 |
23 | First refusal for completing the SciTech review should go to the main code
24 | owner(s) for the area affected. If they don't want to then they may have
25 | suggestions for other suitable reviewers or you can approach anyone who would
26 | have good insight into the changes made.
27 |
28 | Once you have found a reviewer add their GitHub user ID to the pull request
29 | description and request their review.
30 |
31 | Guidance for the SciTech reviewer can be found on the
32 | :ref:`SciTech review page `.
33 |
34 | Code Review
35 | ^^^^^^^^^^^
36 |
37 | Code reviewers are assigned by the Simulation Systems and Deployment Team from
38 | a pool of repository maintainers. New ``ready for review`` pull requests will be
39 | assigned a reviewer on a regular basis. If you need your pull request looking at
40 | more urgently than that, or think your pull request has been overlooked, then
41 | leave a comment for ``@MetOffice/ssdteam`` on the pull request.
42 |
43 | The assigned person will be listed in the pull request description and
44 | :ref:`Review Tracking project `. Once the SciTech review has
45 | been completed either the developer or SciTech reviewer should request the
46 | review of the assigned Code Reviewer.
47 |
48 | Guidance for the Code reviewer can be found on the
49 | :ref:`Code review page `.
50 |
51 | .. admonition:: Requesting a Review
52 |
53 | Review requests are handled in the ``Reviewers`` pane on the right hand
54 | side of a pull request.
55 |
56 | Select the cog, and then search for the person you wish to review by
57 | either name or GitHub user ID.
58 |
59 | .. image:: images/gh_screenshots/review_cog_light.png
60 | :class: only-light border
61 |
62 | .. image:: images/gh_screenshots/review_cog_dark.png
63 | :class: only-dark border
64 |
65 | Code owners in some repositories will automatically be added to this
66 | reviewers section based on the files being changed.
67 |
68 |
69 | .. _review_project:
70 |
71 | Simulation Systems Review Tracker
72 | ---------------------------------
73 |
74 | All open pull requests will be added to a GitHub Project called
75 | ``Simulation Systems Review Tracker``, which causes this box to appear in the
76 | sidebar of the pull request:
77 |
78 | .. image:: images/gh_screenshots/project_scitech_light.png
79 | :class: only-light border
80 |
81 | .. image:: images/gh_screenshots/project_scitech_dark.png
82 | :class: only-dark border
83 |
84 | The project is used to give pull requests a status that distinguishes between
85 | the different review states, and to monitor who is doing the reviews.
86 |
87 | Some states are achieved automatically, some require changing manually:
88 |
89 | * When the developer feels a PR is ready for the SciTech or Code Reviewer to
90 | look at (either initially, or after changes have been made) the state should
91 | be **manually** changed to ``SciTech Review`` or ``Code Review`` as
92 | appropriate.
93 |
94 | * When the SciTech Review has been completed the state should be **manually**
95 | changed to Code Review.
96 |
97 | * When the Code Review has been completed the state should be **manually**
98 | changed to Approved.
99 |
100 | Automatic changes include:
101 |
102 | * When changes are requested by a reviewer the state becomes ``Changes Requested``
103 | * When the pull request has been merged, or otherwise closed, the state becomes
104 | ``Done``
105 |
106 | .. important::
107 | Changing the project status **does not** notify the reviewer. To do this:
108 |
109 | * When the SciTech Review has been completed you should add the assigned Code
110 | Reviewer to the list of reviewers. This will notify them that their review is
111 | required.
112 |
113 | * If a reviewer has requested changes then you can alert them that you are
114 | ready for another review by using GitHubs ``rerequest review`` option;
115 | selecting the circling arrows to the right of the reviewers name.
116 |
117 | .. image:: images/gh_screenshots/rerequest_light.png
118 | :class: only-light border
119 |
120 | .. image:: images/gh_screenshots/rerequest_dark.png
121 | :class: only-dark border
122 |
123 | * You can ``@username`` in any comment to draw that persons attention to the
124 | pull request.
125 |
126 |
127 | .. _reviewer_edits:
128 |
129 | Code Reviewer Edits
130 | -------------------
131 |
132 | As part of the process to commit certain tickets, code reviewers will sometimes
133 | need to commit changes to the branch of a developer. Common reasons for doing
134 | this include,
135 |
136 | * Updating KGO's
137 | * Applying upgrade macros
138 | * Updating commit hashes for linked tickets
139 |
140 | The ability to commit back to another users fork is only available to those with
141 | ``maintainer`` access or above and they can only do so for branches with an open
142 | pull request and the ``Allow edits by maintainers`` option selected.
143 |
--------------------------------------------------------------------------------
/source/Development/um_docs.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _umdp:
8 |
9 | Unified Model Documentation Papers (UMDPs)
10 | ==========================================
11 |
12 | The UM documentation papers and online training are maintained in the
13 | MetOffice/um_doc repository, separate from the UM code. Changes to the
14 | documentation will therefore require their own pull request.
15 |
16 | The latest documentation gets automatically built by CI.
17 | The same action will run when a pull request is opened to ensure that the
18 | requested changes builds correctly. The build script is available in the
19 | repository and should be used to test that the finished documentation looks as
20 | you intend before submitting it for review.
21 |
22 | Standards
23 | ---------
24 |
25 | As there are standards for coding, similarly there are a few rules for the
26 | UM Document Papers.
27 |
28 | #. Latex is the default file format.
29 |
30 | #. A pair of UMDP document classes are supplied in the um_doc repository for
31 | use with all UMDPs to maintain a consistent style.
32 |
33 | * You must use one of these two document classes, no exceptions.
34 |
35 | * Unlike some other in-built Latex document classes the UMDP classes do not
36 | support pass-through of options to their parent class, this is intentional
37 | and will not be changed.
38 |
39 | * The preamble requires the following setup commands to be present:
40 |
41 | * **\\title** The title of the UMDP (Please avoid very long titles - use the
42 | optional subtitle command instead)
43 |
44 | * **\\paperno** The UMDP identification number (3 characters, should match
45 | the directory name), if creating a new UMDP please request a new
46 | number (see below)
47 |
48 | * **\\umversion** The version of the UM the document is valid for (e.g. "9.2"
49 | - This should ideally be the same as the documentation release as even
50 | UMDPs with no changes should be reviewed each release-cycle)
51 |
52 | * **\\owner** The official owner and point of contact for the UMDP (This
53 | might be different to the contributing authors who may be listed by the
54 | optional command below)
55 |
56 | * The preamble also supports the following optional commands:
57 |
58 | * **\\subtitle** May be used to provide elaboration on the title (or a way of
59 | shortening it if required)
60 |
61 | * **\\author** The list of contributing authors (which may or may not include
62 | the UMDP owner)
63 |
64 | * **\\titlecontent** This is a flexible space which appears on the title page
65 | and is left to the author's discretion - it could be used to place a
66 | short abstract-style paragraph on the cover, to contain footnote
67 | references to the list of authors or any other purpose.
68 |
69 |
70 | #. Any package available in an unmodified TexLive (version 2018) installation
71 | is permitted, provided it doesn't alter the page layout and other
72 | presentational elements of the UMDP document class, or require source
73 | commands that are too complex/difficult to maintain or for other authors to
74 | understand (at the discretion of the Code Reviewer / UMDP Librarian/s).
75 |
76 | #. Make sure all files required for build are in the repository. No dependency
77 | shall be stored outside the repository.
78 |
79 | * To ensure the integrity of the UMDP repository it is essential that all
80 | the required files, source and diagrams to create the documentation are
81 | included.
82 |
83 | * It is not acceptable to only provide a pdf or html link.
84 |
85 | * It should be possible for our partners to build UMDPs and modify them l
86 | locally if we are to expect them to contribute to UM development.
87 |
88 | * There is no need to write a Makefile or other code to build each UMDP,
89 | this is all handled by the build_umdoc.py script. However it identifies
90 | the top-level document by the presence of the \documentclass command, so
91 | you should never allow more than one file to contain this command. (See
92 | other UMDPs for examples; these should be standard across UMDPs.)
93 |
94 |
95 | Making Changes
96 | --------------
97 |
98 | Making changes to the UMDPs follows the same process for setting up a fork,
99 | developing on a branch and reviewing through pull requests as outlined in the
100 | :ref:`working_practices_index`.
101 |
102 | The source for the UMDP documents is in the ``source`` folder, in sub-directories
103 | named after the UMDP number of the each document. You should not need to edit
104 | anything outside of the ``source`` directory.
105 |
106 | When changing a document the ``Last Updated`` version number must be updated
107 | to reflect the *upcoming* release. This is done by changing `\\umversion{}` in
108 | the documents pre-amble.
109 |
110 | To test build the documentation:
111 |
112 | .. code-block::
113 |
114 | ./build_umdoc.py XYZ
115 |
116 | * XYZ is the UMDP number of your modified document (more that one can
117 | be specified if you wish)
118 |
119 | * The PDF output will be in `output/papers/umdp_XYZ.pdf`
120 |
121 | * Build stdout and stderr will be in `output/logs/umdp_XYX_stdout.log`.
122 |
123 |
124 | .. tip::
125 | Further details on the build script can by found by directing a browser
126 | to `web/build.html`
127 |
--------------------------------------------------------------------------------
/source/WorkingPractices/forking.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _forking:
8 |
9 | Creating and Managing Forks
10 | ===========================
11 |
12 | .. tip::
13 |
14 | For more information see the `GitHub documentation
15 | `__
16 |
17 | Forks are repositories that share code and visibility settings with the
18 | upstream repository. They provide a place for development work to take place
19 | while allowing the upstream repo to maintain limited write access. Forks can
20 | all merge branches with each other as well as with the upstream repository,
21 | meaning a pull request can be opened to merge a branch in a fork into the
22 | upstream main.
23 |
24 | Creating a Fork
25 | ---------------
26 |
27 | Creating a fork is something that only needs to be done once per upstream
28 | repository. Once created, branches can be created in the fork as desired by
29 | the owner.
30 |
31 | .. tab-set::
32 |
33 | .. tab-item:: Web Browser
34 |
35 | First navigate to the upstream repository you wish to fork. Then select
36 | the fork button.
37 |
38 | .. image:: images/gh_screenshots/fork_button_light.png
39 | :class: only-light border
40 |
41 | .. image:: images/gh_screenshots/fork_button_dark.png
42 | :class: only-dark border
43 |
44 | On the next page you can rename your fork if desired and select which
45 | branches to fork - ensure this box is unticked to fork all branches.
46 |
47 | .. important::
48 |
49 | Ensure the option to only clone the default branch is unticked.
50 |
51 | .. image:: images/gh_screenshots/fork_page_light.png
52 | :class: only-light border
53 |
54 | .. image:: images/gh_screenshots/fork_page_dark.png
55 | :class: only-dark border
56 |
57 | .. tab-item:: gh cli
58 |
59 | Run the following command, substituting for the required upstream owner
60 | and repository name,
61 |
62 | .. code-block:: shell
63 |
64 | gh repo fork /
65 |
66 | .. tip::
67 |
68 | Add ``--clone`` to immediately clone the forked repo
69 |
70 |
71 | Disable Github Actions in your fork
72 | -----------------------------------
73 |
74 | It is a good idea to disable GitHub actions on your fork otherwise you will
75 | regularly get failures when updating your main branch, due to the failure to
76 | deploy documentation. Doing this will not prevent the CI running when you open a
77 | pull request.
78 |
79 | To disable, in your fork on GitHub, navigate to Settings and then
80 | Actions/General. There select the option to ``Disable actions`` and save.
81 |
82 | .. image:: images/gh_screenshots/disable_actions_light.png
83 | :class: only-light border
84 |
85 | .. image:: images/gh_screenshots/disable_actions_dark.png
86 | :class: only-dark border
87 |
88 |
89 | Optional - Adding the MetOffice Mirror Bot
90 | ------------------------------------------
91 |
92 | The ``mo-gitassist-bot`` is a user which can be granted read access to a
93 | repository, allowing it to mirror the repository to a central location on
94 | MetOffice platforms. This repository can then be cloned using the local mirror,
95 | avoiding the need to authenticate with GitHub. This may be useful, eg. for
96 | shared accounts which do not have a GitHub account themselves. The mirrors are
97 | read only - committing and pushing back to the remote repository will require a
98 | GitHub account.
99 |
100 | All simulation systems repositories have granted access to this bot, allowing
101 | them to be cloned from the mirrors by running,
102 |
103 | .. code-block:: shell
104 |
105 | git clone /path/to/mirrors/MetOffice/repository.git
106 |
107 | Please contact the SSD team or the git migration project for the path to the
108 | mirrors.
109 |
110 | If you wish to be able to access a fork from the mirror, then you will need to
111 | add the bot to your forked repository. First, in your fork, navigate to the
112 | ``Settings`` tab and the ``Collaborators`` section.
113 |
114 | .. image:: images/gh_screenshots/collaborators_light.png
115 | :class: only-light border
116 |
117 | .. image:: images/gh_screenshots/collaborators_dark.png
118 | :class: only-dark border
119 |
120 | There, use the ``Add People`` button to add the ``mo-gitassist-bot`` with
121 | ``Read`` permissions,
122 |
123 | .. image:: images/gh_screenshots/mo_bot_light.png
124 | :class: only-light border
125 |
126 | .. image:: images/gh_screenshots/mo_bot_dark.png
127 | :class: only-dark border
128 |
129 | To clone a fork from the mirrors requires a slightly different approach to
130 | normally cloning a fork.
131 |
132 | .. code-block:: shell
133 |
134 | # First clone the mirror of the upstream repository
135 | git clone /path/to/mirrors/MetOffice/repository.git && cd repository
136 |
137 | # Fetch the fork and branch.
138 | # fork-username is the username of the owner of the desired fork
139 | # fork-branch is the branch to be checked out
140 | git fetch origin /
141 |
142 | # Checkout the forked branch
143 | git checkout FETCH_HEAD
144 |
145 | To use the mirrors, you will likely need to run the following once, from the
146 | command line:
147 |
148 | .. code-block:: shell
149 |
150 | git config --global --add safe.directory "/path/to/mirrors/MetOffice/*"
151 |
--------------------------------------------------------------------------------
/source/Development/TestSuites/um.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _um_testing:
8 |
9 | Testing the UM
10 | ==============
11 |
12 | UM testing is run with the following command from the top directory of a local
13 | clone:
14 |
15 | .. code-block:: shell
16 |
17 | cylc vip -z group=developer -n ./rose-stem
18 |
19 | -----
20 |
21 | The UM rose stem testing includes a range of builds using a variety of
22 | compilers, both reconfiguration and atmosphere tests, and rose-ana tasks to
23 | check the output. As well as testing the UM, there are also tests for
24 | createBC, mule, Shumlib and a variety of other utilities.
25 |
26 | The output is checked for correctness both by comparing the output to a set of
27 | stored :ref:`KGO files `, but also by running different processor
28 | decompositions and comparing the results, and by looking at the outputs of
29 | runs that have stared from a cold start and those that have had their state
30 | saved and restarted.
31 |
32 | .. tip::
33 |
34 | The UM rose-stem suite can take a long time to run. When in the initial
35 | stages of developing a change choosing a single compile and run task to
36 | test with may be helpful. e.g.
37 |
38 | .. code-block:: shell
39 |
40 | cylc vip -z group=ex1a_gnu_um_rigorous_omp-n48 -n ./rose-stem
41 |
42 |
43 | See :ref:`multirepo` for details on how to carry out this testing.
44 |
45 | Below is a (by no means comprehensive) set of groups that you may wish to use
46 | on Met Office systems. Note that there is a lot of overlap between these
47 | groups, and that you can specify more than one at once, e.g.
48 | ``-z group=developer,jules,ukca``.
49 |
50 | +--------------------+----------------------------------------------------------+
51 | | Group | Description |
52 | +====================+==========================================================+
53 | | developer | Standard group of tests that every change is expected |
54 | | | to pass before commit. It is a useful checkpoint during |
55 | | | development. |
56 | +--------------------+----------------------------------------------------------+
57 | | nightly | More thorough testing group. This includes everything in |
58 | | | developer plus some longer and more complex tests. It is |
59 | | | run automatically every night and monitored by the SSD |
60 | | | team. |
61 | +--------------------+----------------------------------------------------------+
62 | | all | The complete test suite, including all longer runs and |
63 | | | less commonly used utilites. This is run automatically |
64 | | | every week and monitored by the SSD team. All |
65 | | | :ref:`KGO ` changing PRs need to run this group. |
66 | +--------------------+----------------------------------------------------------+
67 | +--------------------+----------------------------------------------------------+
68 | | rigorous_compile | A build-only group that will sense-check the code for a |
69 | | | wider range of compile errors than the usual builds. |
70 | +--------------------+----------------------------------------------------------+
71 | | jules | A set of tests that exercise the UM/JULES interface. |
72 | +--------------------+----------------------------------------------------------+
73 | | casim | A set of tests that exercise the UM/CASIM interface. |
74 | +--------------------+----------------------------------------------------------+
75 | | ukca | A set of tests that exercise the UM/UKCA interface. |
76 | +--------------------+----------------------------------------------------------+
77 | | recon | A set of tests that exercise the reconfiguration system. |
78 | +--------------------+----------------------------------------------------------+
79 | | coupled | A set of tests that exercise the coupled and hybrid code.|
80 | +--------------------+----------------------------------------------------------+
81 | | uk_lams | Testing for the limited area models |
82 | +--------------------+----------------------------------------------------------+
83 | | ex1a/azspice | All tests designed to run on the named platform. |
84 | +--------------------+----------------------------------------------------------+
85 | | scripts | All of the auxillary scripts that are designed to check |
86 | | | the code standards in ways that aren't tested by the |
87 | | | compiler. |
88 | +--------------------+----------------------------------------------------------+
89 |
90 |
91 | Monsoon
92 | -------
93 |
94 | As of UM vn13.9, the UM is able to run on Monsoon3, the latest version hosted
95 | by the Meto EX machines. To run on here, users should follow the process laid
96 | out in the Monsoon User Guide. This involves logging onto the ``cazccylc1``
97 | server to launch jobs.
98 |
99 | The UM test suite is set up to run on Monsoon with Cylc 8 by running,
100 |
101 | .. code-block:: shell
102 |
103 | cylc vip -z group=ex1a -n ./rose-stem
104 |
105 | This will launch all ex1a jobs that are available to run on Monsoon.
106 |
107 | For running suites on Monsoon, a few environment variables must be set for the
108 | builds to complete:
109 |
110 | * ``EX_UMDIR_OVERRIDE`` - this needs to be set to ``/projects/metoff/umdir``
111 | * ``ECCODES_VERSION`` - this needs to be set to ``eccodes-2.34.0``
112 |
113 |
--------------------------------------------------------------------------------
/source/Reviewers/codereview.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _code_review:
8 |
9 | Code Review
10 | ===========
11 |
12 | .. tip::
13 |
14 | GitHub documentation on the review process and interface:
15 | `Reviewing Proposed Changes in a Pull Request `_
18 |
19 | Purpose of the review
20 | ---------------------
21 | The purpose of the Code Review is to analyse a change for its impact and
22 | to ensure that all concerned parties are made aware of changes that are
23 | required.
24 |
25 | Fundamentally this review is to ensure that the change is well thought-out and
26 | that no system aspects have been missed. The review should be an active one
27 | and should question anything that is poorly coded.
28 |
29 | Focus on the code, not the contributor; providing constructive, respectful and
30 | actionable feedback. Critique the implementation, not the individual and always
31 | explain the reasoning behind your suggestions.
32 |
33 | Reviewer responsibilities and checkpoints
34 | -----------------------------------------
35 |
36 | The pull request template that populates the pull request description box
37 | contains a Code Review section with questions to help you think through all the
38 | areas of concern. This Code Review section should be completed once you are
39 | finished.
40 |
41 | Work through the review template considering each question in turn. These
42 | will include areas such as:
43 |
44 | .. dropdown:: Is the pull request and testing complete?
45 |
46 | * The Pull Request Template should be filled in. This includes:
47 |
48 | * Proof of :ref:`testing ` completed. All tests should pass
49 | with the exception of any known changes in answers.
50 | * Approvals from the code owners for every file changed.
51 | * If the change affects answers then approval from the owners of the
52 | affected configurations.
53 | * If the change modifies OMP code sections then approval from the
54 | optimisation team.
55 |
56 | * It should be possible to understand the purpose of the pull request from
57 | the details provided.
58 |
59 | .. tip::
60 |
61 | The testing summary (trac.log) provides details of the code and
62 | configuration owner approvals needed for each change. The approvals
63 | should be added by the owners themselves and this can be checked using
64 | the page history.
65 |
66 | .. tip::
67 |
68 | If the pull request has been completed by a non-Met Office developer it is
69 | useful to run tests ourselves early in the review process as different
70 | compilers may behave differently. It may also be necessary to run
71 | tests for systems that the partner has not had access to.
72 |
73 | .. tip::
74 |
75 | Quantity of testing required will vary with the complexity of the
76 | change and the repositories involved. Developer test groups are
77 | required as a minimum. As a guide for further testing consider the
78 | following:
79 |
80 | * does the change affect answers? If so `all` group must be run
81 | * does the change affect multiple repositories? If so the UM testing
82 | must include e.g. the `jules` or `ukca` groups as appropriate
83 | * had the reconfiguration been altered? If so the UM testing must
84 | include the `recon` group
85 | * is there another rose-stem group that covers this area of code?
86 |
87 |
88 | .. dropdown:: Is this a :ref:`multi-repository` pull request?
89 |
90 | Each of the repositories covered by these WPs have overlapping use of
91 | code.
92 |
93 | The pull request templates in each repository contain the
94 | details of when testing against other repositories are required. These
95 | highlight where the code is likely to interact. *e.g. if code in the
96 | shared/science folder in JULES is modified then both the UM and LFRIc Apps
97 | test suites will need to be run with that change.*
98 |
99 | If this testing doesn't pass then either
100 |
101 | a) the change in pull request will need modifying so that the child
102 | repository's test suite passes
103 |
104 | b) this change requires a linked pull request in that repository so that
105 | all tests can pass.
106 |
107 | .. tip::
108 |
109 | All linked pull requests are reviewed as a group. Each pull request in
110 | the group should contain links to all the others and labels applied to
111 | make it easier to keep track of them all.
112 |
113 | Care is needed when :ref:`committing these pull requests
114 | `.
115 |
116 | .. dropdown:: Is the code up to scratch?
117 |
118 | Generally this is about making sure the code complies with the relevant
119 | style guides, and is consistent with the design of the code it sits in.
120 |
121 | * :ref:`Fortran Coding Styles `
122 | * `LFRic Coding Styles (which build on the above)
123 | `__
124 |
125 | Final decision points and actions
126 | ---------------------------------
127 |
128 | The pull request will likely iterate between the reviewer and the developer
129 | during the review process while retaining it's Code Review status. The Code
130 | Reviewer can `Resolve Conversations` when they feel each query has been
131 | satisfactorily answered.
132 |
133 | In the case where a reviewer believes a PR requires substantial changes to be
134 | made in order to reach sufficient quality for commit, the PR may be closed
135 | without merging. This will be done in consultation with the developer and other
136 | Code Owners and Reviewers.
137 |
138 | Once you are happy that the change is appropriate and correct, and the code
139 | review parts of the pull request template have been completed, submit a
140 | review that `approves` the change.
141 |
142 | From here follow the :ref:`How To Commit` guide through to pull
143 | request closure.
144 |
--------------------------------------------------------------------------------
/source/Development/planning_your_change.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _planning:
8 |
9 | Planning Your Change
10 | ====================
11 |
12 | There's often a tendency to jump straight into a change. This is fine for small
13 | and simple changes where the purpose of the change is well-understood.
14 | However, for more complex changes having a plan for the change before
15 | developing it can be very beneficial and a well-thought plan improves the
16 | chances of a successful code change.
17 |
18 | Planning a change
19 |
20 | * should not take long
21 | * does not require submission for a review - the final code should speak for
22 | itself
23 | * should help you understand the task and what it will involve
24 | * should highlight some potential issues before development work starts
25 |
26 | This is also a good time to consult with code owners(and also with
27 | configuration owners, if necessary). Use the appropriate :ref:`support`
28 | channels.
29 |
30 | The following are some general hints and tips in planning code changes successfully.
31 |
32 | .. tip::
33 |
34 | For more complex LFRic changes you can submit your plan to the Core Capability
35 | Development team for a Design Review.
36 |
37 | .. seealso::
38 | :ref:`dos_donts`
39 |
40 | General Considerations
41 | ----------------------
42 | **Smaller is better.** Submit a separate pull request per bug fix or feature.
43 | Avoid refactoring or reformating code that is not related to your change.
44 | Multiple small pull requests are easier to review and more likely to be accepted
45 | promptly.
46 |
47 | **Consider the complexity of your change.** For larger or more complex changes,
48 | start by opening an issue and discussing your approach with the relevant people.
49 | This helps avoid unnecessary work and ensures alignment with project goals.
50 |
51 | **Prioritise clarity over cleverness.** Code is read more often than it is
52 | written, so make it easy to understand and maintain. If the logic is not
53 | immediately obvious then include comments to explain your reasoning.
54 |
55 | **How does your proposed change fit in with the structure of the model?** Try
56 | and make your code changes in-scope and no larger than they need to be. If you
57 | find yourself having to edit large areas of code in other, unrelated sections
58 | of the model purely to get your change to work, chances are that you're doing
59 | something wrong. Please do seek advice.
60 |
61 | **Ensure that your code change meets coding standards** All models have various
62 | coding standards and things to avoid, so it is useful for the developer to be
63 | aware of these.
64 |
65 | * :ref:`Fortran Coding Styles `
66 | * `LFRic Coding Styles (which build on the above)
67 | `__
68 | * `PEP 8 (Python) `__
69 |
70 | **Who will SciTech Review the change?** This is a useful consideration as not
71 | everyone who uses the repository has the knowledge or experience to review
72 | every change that is being developed. Get in touch with your SciTech Reviewer
73 | early in the process as they will have valuable insights that can help to shape
74 | your change.
75 |
76 | **Does your change fix a bug or are you investigating a bug in the code?** If
77 | so, be aware that any changes to answers will require a KGO update and
78 | configuration owners to approve the change, which can take longer. Code changes
79 | which require a change in answers and configuration owner approval should be
80 | planned well in advance of the code review deadline to allow time for the
81 | approvals to take place.
82 |
83 | **Is the code you need to alter on a single repository or is it spread over
84 | multiple repositories?** If it's over multiple repositories you need to use
85 | linked pull requests. See :ref:`multirepo` for further details.
86 |
87 | **Does similar code functionality already exist in the model?** It's a good
88 | idea **not** to re-invent the wheel or have code duplication! Speaking to code
89 | owners of the appropriate sections can help in this instance.
90 |
91 | **How will your change be tested?** Include unit or integration tests, and update
92 | any example or demo repositories to exercise new functionality.
93 |
94 | Specific Tips for Scientific changes
95 | ------------------------------------
96 |
97 | **Does the change add a new option or feature to the code?** If so, you
98 | probably need a new namelist variable to switch the new option off and maintain
99 | regression. This will also imply changes to the metadata are required and an
100 | upgrade macro to include the switch into the upgraded configuration.
101 |
102 | **How are you going to prove that your change works scientifically?** It's
103 | vital to make sure your code changes work when switched **on** and give the
104 | same answer when the code is run over different processor configurations.
105 | Producing a quick plot or plots to show the impact of your code and including
106 | them on your pull request can aid your SciTech Reviewer in showing that your code
107 | works properly.
108 |
109 | **Does the change need any new diagnostics to make sense of the code?** Many
110 | changes will be able to use the existing diagnostics available, but if some
111 | novel functionality is being developed it may require new diagnostics to be
112 | added. The developer needs to check that new diagnostics output correctly and
113 | look sensible.
114 |
115 | **Does your change need new prognostic variables including?** If so, these need
116 | to be added and it is worth checking that these work properly. In the UM in
117 | particular, adding prognostic variables involves editing a lot of routines and
118 | is quite time-consuming.
119 |
120 |
121 | Specific Tips for Technical changes
122 | -----------------------------------
123 |
124 | **Avoid wholesale technical changes** These can be very cumbersome to review;
125 | if it's possible, split the change into more manageable chunks.
126 |
127 | **How does your change affect the performance of the model?** If your change
128 | intends to optimise code, be prepared to provide evidence of how things have
129 | improved.
130 |
131 | ..
132 | Comment: Are there any more that can be thought of? These PRs will
133 | mostly be done by experienced developers and usually inside the Met
134 | Office.
135 |
--------------------------------------------------------------------------------
/source/Reviewers/nightlytesting.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _nightlytesting:
8 |
9 | Nightly Testing
10 | ===============
11 |
12 | .. important::
13 |
14 | These instructions are intended for Met Office reviewers with access to the
15 | nightly testing user only.
16 |
17 | Modifying and Installing Testing
18 | --------------------------------
19 |
20 | Nightly testing is controlled by 2 cronfiles, ``auto-gen_testing.cron`` and
21 | ``manual.cron`` both located in ``~/Crontabs/``. The first is automatically
22 | generated the script ``generate_test_suite_cron.py`` which is stored in the
23 | SimSys_Scripts GitHub repo and controls the launching and clean up of the
24 | nightly rose-stem suites. The second is intended for manually adding tasks
25 | which don't fit the pattern of the regular rose-stem suites.
26 |
27 | Rose-stem testing is controlled by a config file located at
28 | ``~/testing_configs.yaml``. This contains entries for each of the suites that
29 | will be regularly launched. The following options are available when defining
30 | a suite:
31 |
32 | * ``repo``: Required, The name of the git repo being run.
33 |
34 | * ``period``: Required, The period with which the job repeats. Options are:
35 |
36 | * ``weekly``: Runs on Mondays, cleans on Sundays.
37 | * ``nightly``: Runs Tue-Fri, cleans Wed-Sat.
38 | * ``nightly_all``: Runs Mon-Fri, cleans Tue-Sat.
39 |
40 | * ``time_launch``: Required, 24 hour time format for the time to launch the
41 | suite.
42 |
43 | * ``time_clean``: Required, 24 hour time format for the time to clean the
44 | previous iteration of this suite. If the period is weekly, the cleanup
45 | occurs on a Sunday, otherwise it occurs 1 day later than the suite was run.
46 |
47 | * ``groups``: Required, the rose-stem groups to run.
48 |
49 | * ``revisions``: Optional, Defines what revisions of dependency repos to use.
50 | Either ``set`` to use the revisions defined in the repo or ``heads`` to do
51 | HoT testing. Defaults to use Set Revisions.
52 |
53 | * ``vars``: Optional, This is a list format of strings to be added to the
54 | rose-stem command with the ``-S`` prefix.
55 |
56 | * ``monitoring``: Controls whether to launch the monitoring script on this
57 | suite. If true, the monitoring script will be launched at 06:00.
58 |
59 | * ``cylc_version``: Controls which cylc_version to use. The suite is now set up
60 | to use primarily cylc8 with some suites being launched at Cylc7 for the UM
61 | and Jules. Can be any string beginning "7" or "8" that is a valid cylc
62 | install at the site, such that ``export CYLC_VERSION=``
63 | works.
64 |
65 | The cronjobs are installed by running the ``generate_test_suite_cron.py``
66 | script with the ``--install`` command line option. This script is stored in
67 | the SimSys_Scripts GitHub repo. It will read a config file, generate a cron
68 | file and then install the cronjobs from all files with extension .cron in a
69 | specified location. The script has the following command line arguments:
70 |
71 | * ``-c --config``: Required, the path to a yaml file with the testing
72 | configurations. For the meto testing user this file is located at
73 | ``~/ccc.yaml``.
74 |
75 | * ``-f --cron_file``: The file the cronjobs will be written to. Defaults to
76 | ``~/Crontabs/auto-gen_testing.cron``.
77 |
78 | * ``-l --cron_log``: The file cron task output will be piped to. Defaults to
79 | ``~/cron.log``.
80 |
81 | * ``-p --cylc_path``: The location of the Cylc Installations. Required if
82 | testing with ``cylc-next``.
83 |
84 | * ``--install``: If included will install the cronjobs from files with
85 | extension .cron in the location defined by the cron_file argument.
86 |
87 | To update and install at meto:
88 |
89 | .. code-block:: shell
90 |
91 | python3 generate_test_suite_cron.py \
92 | -c ~/Crontabs/nightly_testing_configs.yaml --install
93 |
94 | Retriggering Nightlies
95 | ----------------------
96 |
97 | TLDR Instructions
98 | ^^^^^^^^^^^^^^^^^
99 |
100 | .. code-block:: shell
101 |
102 | # As the testing user
103 | rs8
104 | # Wait for retrigger script to run - answer questions as required
105 | cylc_url
106 | # Select the 2nd url that appears
107 |
108 | Full Explanation
109 | ^^^^^^^^^^^^^^^^
110 |
111 | To retrigger the nightlies a script is available at
112 | ``$UMDIR/SimSys_Scripts/nightly_testing/retrigger_nightlies.py``. To run this
113 | use the alias ``rs8`` which will load the required modules and set the cylc
114 | version to 8. The script will detect any suites from the previous night with
115 | failed tasks and ask whether to retrigger them. It will restart requested
116 | suites, sleep, and then individually retrigger any failed tasks. Running ``rs8
117 | PATTERN`` will only launch suites which match that pattern (no wildcard
118 | matching is currently setup), eg. ``rs8 lfric_apps`` will restart suites
119 | with "lfric_apps" in their name.
120 |
121 | To interact with the nightlies open a cylc8 gui by:
122 |
123 | .. code-block:: shell
124 |
125 | export CYLC_VERSION=8
126 | cylc gui --ip=0.0.0.0 --no-browser --debug --Application.log_level=WARN
127 |
128 | or the meto user has an alias defined as ``cylc_url``. Copy the 2nd url that
129 | appears into your browser.
130 |
131 | Suites can also be triggered without using the script. In the gui, select the
132 | suite on the left and then click the play triangle at the top. This can also
133 | be done on the command line by:
134 |
135 | .. code-block:: shell
136 |
137 | cylc play
138 |
139 |
140 | Tasks can be retriggered individually or in groups. Eg, to retrigger all failed
141 | tasks, click the menu icon at the top of the page, and then select "Trigger".
142 | In the resulting dialogue box, append the "Tasks" section with ``:failed``
143 | then click submit. To retrigger individually is similar but click the menu
144 | icon next to the task.
145 |
146 | To alter a tasks runtime settings, eg. bump the wallclock, select the jobs menu
147 | and then choose ``Edit Runtime``. This will open a dialog box where runtime
148 | items can be added/edited. When done click ``submit`` and then ``Trigger``
149 | (unlike cylc7 it won't ask you to do this).
150 |
151 | .. important::
152 |
153 | When finished Keyboard terminate the cylc url command and choose y when
154 | prompted. This shutsdown the cylc server and prevents multiple connections
155 | opening.
156 |
157 |
--------------------------------------------------------------------------------
/source/Development/inputs.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _inputs:
8 |
9 | Input Variables, Rose Metadata and Upgrade Macros
10 | =================================================
11 |
12 | .. important::
13 |
14 | New UM Ancils must be submitted to the MIAO team for approval. Please
15 | follow their process for `Requesting New UM Ancils
16 | `__.
17 |
18 | Sometimes the developer needs to alter model namelists and input variables. A
19 | common reason is for the inclusion of a new piece of code which has to be
20 | turned off by default.
21 |
22 | If a developer changes the namelist inputs they are also expected to alter the
23 | following areas of the code:
24 |
25 | * The inclusion of appropriate defensive checks.
26 | * Changes to Rose metadata.
27 | * Upgrade macros.
28 |
29 | Defensive checks abort the model run if the user sets variables outside of
30 | their expected range or a combination of variables that will lead to an error.
31 | Most namelist modules contain a subroutine which performs this checking and
32 | which can be modified.
33 |
34 | The project metadata can be found in the following locations:
35 |
36 | .. tab-set::
37 |
38 | .. tab-item:: UM
39 |
40 | ``/rose-meta/um-atmos/HEAD/rose-meta.conf``
41 |
42 | .. tab-item:: JULES
43 |
44 | ``/rose-meta/*/*/HEAD/rose-meta.conf``
45 |
46 | .. tab-item:: LFRic
47 |
48 | ``//rose-meta/*/HEAD/rose-meta.conf``
49 |
50 | In addition to the above locations, the rose metadata is centrally mirrored on
51 | Met Office systems. This means that metadata that has been committed to
52 | ``main`` can be accessed without a clone. This may be of use when upgrading
53 | scientific suites between versions.
54 |
55 | All new namelist variables need a new entry so that the metadata loads into the
56 | Rose GUI for users to switch it on. Additionally, sometimes the metadata needs
57 | to be modified without changing a namelist variable. Guidance for updating the
58 | metadata :ref:`is available `.
59 |
60 | .. note::
61 |
62 | JULES developers also need to :doc:`update the JULES documentation
63 | ` whenever they add or remove namelist variables.
64 |
65 | .. important::
66 |
67 | All changes which alter namelists require an upgrade macro for them to work
68 | with the model.
69 |
70 | Changes to the metadata which don't involve namelist changes may or may not
71 | require an upgrade macro. If you are unsure whether a change needs an upgrade
72 | macro, then run the following command on your branch:
73 |
74 | .. code-block:: shell
75 |
76 | rose stem --group=scripts
77 |
78 | If all of the tests pass then there is no requirement to add an upgrade macro.
79 | If any of the metadata tests fail, then the developer should add a blank
80 | upgrade macro which contains no upgrade commands but simply points the rose
81 | stem suite to the new metadata. The SSD team are also available to advise on
82 | whether an upgrade macro is necessary.
83 |
84 | .. tip::
85 |
86 | When editing metadata you should always check that the new metadata appears
87 | as expected in the GUI, including testing that invalid settings raise
88 | appropriate warnings. The command to open the GUI is in general:
89 |
90 | .. code-block:: shell
91 |
92 | rose edit -C rose-stem/app/APP-NAME
93 |
94 | Note for MetOffice Users: ``rose edit`` runs on a server, so make sure your
95 | source can be seen from a server - i.e. not on a drive that's only available
96 | to your local machine.
97 |
98 | For LFRic Apps a few extra changes are required. In your branch (your test
99 | branch if you have an upgrade macro):
100 |
101 | .. code-block:: shell
102 |
103 | cd rose-meta
104 | rose edit -C ../rose-stem/app/APP-NAME --no-warn version
105 |
106 | If you have a linked LFRic Core or Jules development with metadata changes,
107 | you can load their metadata by adding
108 | ``-M /path/to/working_copy/rose-meta`` to the ``rose-edit`` command.
109 |
110 |
111 | Adding a new LFRic Metadata Section
112 | -----------------------------------
113 |
114 | Due to the way lfric metadata is shared, if a new LFRic metadata section is
115 | added, then a few new files are added. A new LFRic metadata section here is
116 | defined as a new directory within an existing or new ``rose-meta`` directory.
117 | Adding a new metadata section requires:
118 |
119 | * ``rose-meta/META-NAME/HEAD/rose-meta.conf``
120 | * ``rose-meta/META-NAME/vnX.Y/rose-meta.conf`` (where ``X.Y`` is the most
121 | recent released version)
122 | * ``rose-meta/META-NAME/versions.py``
123 | * A symlink from the top-level rose-meta directory to the new directory
124 | (see existing ones for examples)
125 |
126 | The ``vnX.Y`` and ``HEAD`` metadata should be identical for this initial
127 | change, other than any import statements, which should point at vnX.Y or HEAD
128 | respectively. Other ``vnX.Y`` and ``versionAB_CD.py`` files shouldn't be
129 | modified or added (these are a snapshot of the metadata at a release).
130 |
131 | If a new rose-stem app using the new metadata is also being added, then
132 | a "blank" upgrade macro will also need to be added with a ``BEFORE_TAG=vnX.Y``
133 | and a standard ``AFTER_TAG=vnX.Y_tTTTT``. This upgrade macro will allow the
134 | new app to be updated to the Head metadata when the branch is merged to ``main``.
135 | The ``rose-app.conf`` for this app will require a metadata import line of
136 | format, ``meta=META-NAME/vnX.Y``.
137 |
138 |
139 | How to add an upgrade macro to your branch
140 | ------------------------------------------
141 |
142 | Please see :ref:`this page ` for further information.
143 |
144 | .. tip::
145 |
146 | When developing a change that updates the input and/or user interface, then
147 | repeatedly running/reverting the upgrade macro on the dev branch, or
148 | creating many test branches can be tiresome. Consider working up your
149 | change with the new options hard-coded in until such time as you are ready
150 | to connect up to the input for real.
151 |
152 | .. important::
153 |
154 | **Do not** apply the upgrade macro to your dev branch prior to the review
155 | process. Instead you must create a test branch. See :ref:`testing`.
156 |
157 | .. toctree::
158 | :hidden:
159 |
160 | metadata_guidance
161 | macros
162 |
--------------------------------------------------------------------------------
/source/Development/temp_logicals.rst:
--------------------------------------------------------------------------------
1 | .. -----------------------------------------------------------------------------
2 | (c) Crown copyright Met Office. All rights reserved.
3 | The file LICENCE, distributed with this code, contains details of the terms
4 | under which the code may be used.
5 | -----------------------------------------------------------------------------
6 |
7 | .. _templogicals:
8 |
9 | Temporary Logical Variables
10 | ===========================
11 |
12 | Temporary logical variables are intended to add a layer of protection in
13 | instances where a *bug fix* produces a **significant** change in results in a
14 | scientific configuration. The purpose of the temporary logical is to maintain
15 | results in scientific configurations. However, the bug fix could either be due
16 | to a scientific or a technical issue. Temporary logicals are used primarily
17 | for two reasons:
18 |
19 | #. To maintain consistency in an important scientific configuration
20 | (e.g. global/regional atmosphere/land) when upgrading between model
21 | versions. If the logical was not used then different model versions may
22 | lead to very different scientific answers.
23 |
24 | #. To give the configuration owner time to assess the impact of a significant
25 | change in answers on their configuration. The usual release cycle window is
26 | often too short to fully assess the impact of anything other than a small
27 | change.
28 |
29 | Neither of these statements suggest that the fix shouldn't be included --- in
30 | fact the opposite is true. The decision as to whether to include a temporary
31 | logical normally rests with the configuration owner, but with guidance from
32 | the Code Reviewer and the Simulation Systems and Deployment team. In such
33 | cases, the following guidance is followed:
34 |
35 | * Essential bug fixes (e.g. something which would on occasions cause the model
36 | to crash) should be switched on by default and would **not** have a
37 | temporary logical.
38 |
39 | * This includes any bug fixes which restores bit comparison across different
40 | processor decomposition where it has been previously broken.
41 |
42 | * Any bug fix which does not lead to a change in answers should be switched on
43 | by default (not with a temporary logical).
44 |
45 | * Small changes (within the noise) can usually be included as a :ref:`kgo`
46 | update and would **not** usually have a temporary logical associated with
47 | them.
48 |
49 | * Anything which has a large impact (especially on key variables used for
50 | weather and climate) and which extends beyond the 'noise' of the
51 | model **would be expected to include a temporary logical**.
52 |
53 | * There may be a few specific cases, where only certain platforms or builds are
54 | affected due to a technical bug fix.
55 |
56 | .. important::
57 |
58 | There isn't a precise definition of what 'being in the noise' consists of,
59 | so the configuration owner should always be contacted in the first
60 | instance to provide guidance of what is or is not important to that
61 | particular configuration.
62 |
63 | .. important::
64 |
65 | Remember that any new piece of science code or new scientific option would
66 | normally be switched off by default by a variable in the most appropriate
67 | scientific namelist. A temporary logical would not be used in this
68 | instance. See :ref:`input variables ` for further details.
69 |
70 | Adding a temporary logical to the code base
71 | -------------------------------------------
72 |
73 | When adding a new temporary logical, the developer must protect **only** the
74 | portion of code which changes answers by placing an `IF` statement around it,
75 | e.g.
76 |
77 | .. code-block:: fortran
78 |
79 | IF (l_fix_something) THEN
80 |
81 | ... piece of code which changes answers ...
82 |
83 | END IF ! l_fix_something
84 |
85 | The new logical should be added in a specified subroutine for short-term fixes
86 | and **not** in a usual scientific or technical namelist. The location of the
87 | temporary logical routine varies by repository, with not all repositories
88 | requiring their own temporary logical subroutine. Details are as follows:
89 |
90 | .. tab-set::
91 |
92 | .. tab-item:: UM
93 |
94 | ``src/control/misc/science_fixes_mod.F90``
95 |
96 | .. tab-item:: JULES
97 |
98 | ``src/control/shared/jules_science_fixes_mod.F90``
99 |
100 | .. tab-item:: LFRic Apps
101 |
102 | Currently reads ``science_fixes_mod.F90`` (see UM) into
103 | ``um_physics/source/support/um_physics_init_mod.f90``
104 |
105 | .. tab-item:: UKCA/CASIM/SOCRATES
106 |
107 | No temporary logical routine currently in place for these projects.
108 | Consult with Code Owners or check the UM ``science_fixes_mod.F90`` for
109 | existing temporary logicals.
110 |
111 | .. hint::
112 |
113 | By default, the logical should be set to ``.false.``, with the fix
114 | switched **off**, unless instructed otherwise by the Configuration Owner.
115 |
116 | The developer should remember to add the variable to the namelist, including
117 | any namelist printing and reading subroutines present in the module. In
118 | addition, they should also include a warning for when the fix is not included
119 | in a configuration. Examples of the various components can be found by
120 | examining existing variables in the subroutines listed in the table above.
121 |
122 | An upgrade macro and Rose metadata will be required to add the temporary
123 | logical into the GUI and make it available to model users. See :ref:`inputs`
124 | for further information.
125 |
126 |
127 | After the release cycle
128 | -----------------------
129 |
130 | Normally, configuration owners would be expected to switch on all temporary
131 | logicals present as part of developing their latest configuration. This
132 | includes any which do not impact their configuration, as it allows them to be
133 | retired from the code base. Depending on when the next configuration is being
134 | developed, this could be some time after the code is released.
135 |
136 | .. note::
137 |
138 | **Very rarely**, switching on a bug fix may have an undesired impact (e.g.
139 | it leads to the discovery of a bug elsewhere in the code). In these cases,
140 | the configuration owner may keep the temporary logical set to ``.false.``
141 | until the issue is resolved and may consult with the Code Owners and the
142 | developers of the fix for further guidance. This does not imply that the
143 | bug fix wasn't sensible in the first place!
144 |
145 | Each temporary logical has a review and retention period attached to it. Once
146 | the fix is included within the various configurations it affects, the
147 | temporary logical should be removed from the code base.
148 |
149 | .. important::
150 |
151 | Prior to a pull request containing a temporary logical being committed to
152 | ``main``, the developer is expected to open a new Issue which removes the
153 | logical after a fixed period. This acts as an memory aid that the logical
154 | needs to be removed in due course.
155 |
--------------------------------------------------------------------------------