├── docs ├── index.md ├── stylesheets │ └── extra.css ├── design │ ├── figures │ │ ├── run-sequence.gif │ │ ├── capgen-design.PNG │ │ ├── ccpp-framework.PNG │ │ ├── buildnml_workflow.jpg │ │ ├── cam_history_classes │ │ ├── run-sequence-api.png │ │ ├── hist_config_classes.PNG │ │ ├── constituents-classes.PNG │ │ ├── process-vs-time-split.PNG │ │ ├── run-sequence-dynamics.png │ │ ├── run-sequence-history.png │ │ ├── run-sequence-physics.png │ │ └── ccpp-framework-in-cam-sima.PNG │ ├── cam-build-process.md │ ├── sima-design-goals.md │ ├── directory-structure.md │ ├── ccpp-in-cam-sima.md │ ├── constituents.md │ ├── history.md │ └── cam-run-process.md ├── conversion │ ├── figures │ │ ├── cam-issue.PNG │ │ ├── fork-cam-sima.png │ │ ├── fork-cam-sima-2.png │ │ ├── atmos-phys-issue.PNG │ │ ├── spreadsheet-change.PNG │ │ └── spreadsheet-change-highlighted.PNG │ ├── final-steps.md │ ├── create-sdf.md │ ├── back-to-cam.md │ ├── conversion-background.md │ ├── create-namelist-xml.md │ ├── create-metadata.md │ ├── run-cam-sima.md │ ├── interstitials.md │ ├── check-metadata.md │ ├── create-snapshots.md │ └── convert-portable-layer.md ├── development │ ├── figures │ │ ├── fork_button.png │ │ ├── pr-workflow-1.png │ │ ├── pr-workflow-2.png │ │ ├── pr-workflow-3.png │ │ ├── pr-workflow-4.png │ │ └── pr-workflow-5.png │ ├── tool-recommendations.md │ ├── git-fleximod.md │ ├── git-faq.md │ ├── cam-sima-workflow.md │ ├── cam-coding-standards.md │ ├── debugging.md │ └── cam-testing.md ├── atmospheric_physics │ ├── figures │ │ ├── new_PR_button.png │ │ ├── create_PR_button.png │ │ ├── compare_forks_link.png │ │ ├── fork_repo_dropdown.png │ │ ├── PR_option_base_square.png │ │ ├── PR_option_head_square.png │ │ ├── fork_branch_dropdown.png │ │ └── atm_phys_PR_tab_circle.png │ ├── Tagging-Instructions.md │ ├── directory-structure.md │ └── physics-file-io.md └── usage │ ├── history.md │ └── creating-a-case.md ├── README.md ├── .github └── workflows │ └── docs-deploy.yml └── mkdocs.yml /docs/index.md: -------------------------------------------------------------------------------- 1 | # CAM-SIMA developer documentation 2 | 3 | -------------------------------------------------------------------------------- /docs/stylesheets/extra.css: -------------------------------------------------------------------------------- 1 | .md-grid { 2 | max-width: 90% 3 | } 4 | -------------------------------------------------------------------------------- /docs/design/figures/run-sequence.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/design/figures/run-sequence.gif -------------------------------------------------------------------------------- /docs/conversion/figures/cam-issue.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/conversion/figures/cam-issue.PNG -------------------------------------------------------------------------------- /docs/design/figures/capgen-design.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/design/figures/capgen-design.PNG -------------------------------------------------------------------------------- /docs/design/figures/ccpp-framework.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/design/figures/ccpp-framework.PNG -------------------------------------------------------------------------------- /docs/conversion/figures/fork-cam-sima.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/conversion/figures/fork-cam-sima.png -------------------------------------------------------------------------------- /docs/design/figures/buildnml_workflow.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/design/figures/buildnml_workflow.jpg -------------------------------------------------------------------------------- /docs/design/figures/cam_history_classes: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/design/figures/cam_history_classes -------------------------------------------------------------------------------- /docs/design/figures/run-sequence-api.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/design/figures/run-sequence-api.png -------------------------------------------------------------------------------- /docs/development/figures/fork_button.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/development/figures/fork_button.png -------------------------------------------------------------------------------- /docs/conversion/figures/fork-cam-sima-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/conversion/figures/fork-cam-sima-2.png -------------------------------------------------------------------------------- /docs/design/figures/hist_config_classes.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/design/figures/hist_config_classes.PNG -------------------------------------------------------------------------------- /docs/development/figures/pr-workflow-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/development/figures/pr-workflow-1.png -------------------------------------------------------------------------------- /docs/development/figures/pr-workflow-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/development/figures/pr-workflow-2.png -------------------------------------------------------------------------------- /docs/development/figures/pr-workflow-3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/development/figures/pr-workflow-3.png -------------------------------------------------------------------------------- /docs/development/figures/pr-workflow-4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/development/figures/pr-workflow-4.png -------------------------------------------------------------------------------- /docs/development/figures/pr-workflow-5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/development/figures/pr-workflow-5.png -------------------------------------------------------------------------------- /docs/conversion/figures/atmos-phys-issue.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/conversion/figures/atmos-phys-issue.PNG -------------------------------------------------------------------------------- /docs/conversion/figures/spreadsheet-change.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/conversion/figures/spreadsheet-change.PNG -------------------------------------------------------------------------------- /docs/design/figures/constituents-classes.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/design/figures/constituents-classes.PNG -------------------------------------------------------------------------------- /docs/design/figures/process-vs-time-split.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/design/figures/process-vs-time-split.PNG -------------------------------------------------------------------------------- /docs/design/figures/run-sequence-dynamics.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/design/figures/run-sequence-dynamics.png -------------------------------------------------------------------------------- /docs/design/figures/run-sequence-history.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/design/figures/run-sequence-history.png -------------------------------------------------------------------------------- /docs/design/figures/run-sequence-physics.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/design/figures/run-sequence-physics.png -------------------------------------------------------------------------------- /docs/atmospheric_physics/figures/new_PR_button.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/atmospheric_physics/figures/new_PR_button.png -------------------------------------------------------------------------------- /docs/design/figures/ccpp-framework-in-cam-sima.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/design/figures/ccpp-framework-in-cam-sima.PNG -------------------------------------------------------------------------------- /docs/atmospheric_physics/figures/create_PR_button.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/atmospheric_physics/figures/create_PR_button.png -------------------------------------------------------------------------------- /docs/atmospheric_physics/figures/compare_forks_link.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/atmospheric_physics/figures/compare_forks_link.png -------------------------------------------------------------------------------- /docs/atmospheric_physics/figures/fork_repo_dropdown.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/atmospheric_physics/figures/fork_repo_dropdown.png -------------------------------------------------------------------------------- /docs/atmospheric_physics/figures/PR_option_base_square.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/atmospheric_physics/figures/PR_option_base_square.png -------------------------------------------------------------------------------- /docs/atmospheric_physics/figures/PR_option_head_square.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/atmospheric_physics/figures/PR_option_head_square.png -------------------------------------------------------------------------------- /docs/atmospheric_physics/figures/fork_branch_dropdown.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/atmospheric_physics/figures/fork_branch_dropdown.png -------------------------------------------------------------------------------- /docs/conversion/figures/spreadsheet-change-highlighted.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/conversion/figures/spreadsheet-change-highlighted.PNG -------------------------------------------------------------------------------- /docs/atmospheric_physics/figures/atm_phys_PR_tab_circle.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ESCOMP/CAM-SIMA-docs/main/docs/atmospheric_physics/figures/atm_phys_PR_tab_circle.png -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # CAM-SIMA-docs 2 | CAM-SIMA developer documentation 3 | 4 | # Local usage 5 | ## Python environment 6 | 7 | Required python libraries (pip install): 8 | 9 | - mkdocs 10 | - mkdocs-material 11 | - mkdocs-git-committers-plugin 12 | - mkdocs-git-revision-date-localized-plugin 13 | 14 | # Pushing updates 15 | 1. Clone the repository and checkout `main` 16 | 2. Run the server (from top of the repository directory): `mkdocs serve` (or `python -m mkdocs serve` 17 | 3. Open the documentation on your local server: `http://127.0.0.1:8000/` 18 | 4. Make edits 19 | 5. Push directly back to `main` 20 | -------------------------------------------------------------------------------- /docs/development/tool-recommendations.md: -------------------------------------------------------------------------------- 1 | # Tool recommendations 2 | 3 | This page lists recommendations for various tools one might use during CAM-SIMA development. 4 | 5 | **Please note that these standards are currently in development, and thus any of them could change at any time.** 6 | 7 | ## Version Control 8 | 9 | The recommended version control tool is [git](https://git-scm.com/). 10 | 11 | ## Repository Hosting 12 | 13 | We recommend [Github](https://github.com/) for hosting software repositories 14 | 15 | ## CI/CD 16 | 17 | When possible, we recommend running any CI/CD workflows using [Github Actions](https://docs.github.com/en/actions). 18 | 19 | ## Container Software 20 | 21 | We recommend using [Docker](https://www.docker.com/) for building and running containers. 22 | 23 | **More to Come!** -------------------------------------------------------------------------------- /docs/development/git-fleximod.md: -------------------------------------------------------------------------------- 1 | # git-fleximod 2 | ## Populate and/or update your externals 3 | Run `bin/git-fleximod update` 4 | 5 | The process will either: 6 | 7 | - complete and you're good to go 8 | - indicate to you that modifications have been made in certain modules 9 | - this means you should either go commit those mods or remove them 10 | - you can also (if you're sure!) run `bin/git-fleximod update -f` to force overwrite all externals 11 | 12 | ## Update external repo or tag 13 | To add or update an external repo to CAM-SIMA, the following steps must be done: 14 | 15 | 1. Modify the `.gitmodules` file at the head of CAM-SIMA to add or update the external. Explanations for what each entry in the `.gitmodules` file is can be found on Github [here](https://github.com/ESMCI/git-fleximod?tab=readme-ov-file#supported-gitmodules-variables) 16 | 1. Once the `.gitmodules` file has been updated, go to the head of CAM-SIMA and run `bin/git-fleximod update`. This will bring in the new external code into your local repo (but will not commit them). 17 | 1. Once you are ready to officially commit the changes, then make sure to commit both the modified `.gitmodules` file, and the updated submodule itself. An easy way to make sure you have commited everything is to run `git status` and make sure there are no files or directories that have been modified but are still un-staged. 18 | 19 | Once all of that is done then congrats! A new external has been successfully added/updated. 20 | -------------------------------------------------------------------------------- /docs/conversion/final-steps.md: -------------------------------------------------------------------------------- 1 | # 10 - Final steps 2 | 3 | Open PRs to the relevant repositories. The order specified here (atmospheric_physics, CAM, then CAM-SIMA) is the preferred ordering of PRs/testing/merging. The reason for this is that atmospheric_physics has to go first to get a tag for both CAM and CAM-SIMA. Then the robust testing within CAM will validate the physics, resulting in increased confidence for bringing it into CAM-SIMA. 4 | 5 | It is also good practice to not let there be too much time between merging your atmospheric_physics PR and your CAM-SIMA PR, as the CAM-SIMA registry and the physics metadata are very closely linked and can cause problems for other developers if they are out of sync. 6 | 7 | ## atmospheric_physics 8 | Follow the [atmospheric_physics workflow](../atmospheric_physics/development_workflow.md) to open a PR, respond to review requests, and populate the `NamesNotInDictionary.txt` file. 9 | 10 | !!! Note - verify changes in CAM 11 | Before finalizing the atmospheric_physics pull request, compare the latest CAM tag with the CAM tag you started with when moving code from CAM to atmospheric_physics to check if any changes to moved code was introduced in CAM. 12 | 13 | ## CAM 14 | Follow the [CAM workflow](https://github.com/ESCOMP/CAM/wiki/CAM-SE-Workflows) to open a PR and respond to review requests. Be sure to include the `ccpp-conversion` label on your PR. Once you are assigned a tag by the gatekeeper, run the tests, update the Changelog, then make the tag and archive the baselines. 15 | 16 | ## CAM-SIMA 17 | 1. Add a new test for your scheme using the before/after snapshots you generated during conversion. You can parallel existing FPHYStest tests and follow the instructions [here](../development/cam-testing.md#adding-a-new-regression-test). 18 | 1. Follow the [CAM-SIMA workflow](../development/cam-sima-workflow.md) to open a PR, run the regression tests, and (potentially) make a tag. At the very least, you will have the testing updates above and a new atmopsheric_physics hash (which will point to your fork until the atmospheric_physics PR is merged). 19 | 20 | ## Final touches 21 | 1. Once all PRs are merged and tagged, check off the scheme as "Converted" in the [spreadsheet](https://docs.google.com/spreadsheets/d/1_1TTpnejam5jfrDqAORCCZtfkNhMRcu7cul37YTr_WM/edit?gid=0#gid=0). 22 | -------------------------------------------------------------------------------- /docs/conversion/create-sdf.md: -------------------------------------------------------------------------------- 1 | # 5 - Create an SDF 2 | The **Suite Definition File (SDF)** tells the CCPP-Framework which schemes will be run in what order. For more, see [CCPP in CAM-SIMA](../design/ccpp-in-cam-sima.md) 3 | 4 | - In `$CAM-SIMA/src/physics/ncar_ccpp/test/test_suites`, create `suite_.xml`. This is your SDF! 5 | 6 | !!! Note "SDF location" 7 | If your parameterization or CCPPization represents a complete configuration that can be run by normal users (e.g. a simple physics package like kessler), instead put your SDF in `$CAM-SIMA/src/physics/ncar_ccpp/suites`.

Most likely, your parameterization will not fall into this category and will "just" be part of an existing suite like `CAM4` or `CAM7`, meaning you should put the SDF in the `test_suite` directory listed above. 8 | 9 | - See the template below. You will need to select either `physics_before_coupler` or `physics_after_coupler` as your group name, according to the table below. 10 | ``` 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | ``` 19 | 20 |
21 | | CAM physpkg routine | CAM-SIMA SDF Group | Description | 22 | |:--------------------|------------------------|-------------| 23 | | tphysbc | physics_before_coupler | Physics schemes run before the surface coupler is called | 24 | | tphysac | physics_after_coupler | Physics schemes run after the surface coupler is called | 25 |
26 | 27 | - Add any interstitials which you’ve created or are using in the appropriate location relative to your parameterization 28 | 29 | For example: if your parameterization requires dry water vapor, you'll want to add: 30 | 31 | - `wet_to_dry_water_vapor` before your core parameterization scheme to convert water vapor to a dry mixing ratio (CAM-SIMA has "wet") 32 | - `dry_to_wet_water_vapor` after your parameterization scheme to convert back to a wet mixing ratio to return to CAM-SIMA 33 | 34 | !!! Note "CAMX physics suite" 35 | You will also need to add your same set of schemes to the appropriate larger suite (usually `suite_cam7.xml`). Use comments to delineate your set(s) of schemes to make the SDF somewhat readable. 36 | 37 | Once you have created your SDF, proceed to [6 - Create snapshots of CAM](create-snapshots.md) -------------------------------------------------------------------------------- /docs/atmospheric_physics/Tagging-Instructions.md: -------------------------------------------------------------------------------- 1 | # How to properly tag an atmospheric_physics commit 2 | 3 | This page lists instructions for how to create a new atmospheric_physics tag. A few rules: 4 | 5 | 1. Tags should always be annotated (no lightweight tags). 6 | 2. Tags should follow the naming convention listed [below](#tag-naming-conventions). 7 | 3. All merge commits into the main branch should be tagged. 8 | 4. Tags can only be created by people who have write access. 9 | 10 | ## How to create a git tag via the command line 11 | 12 | You can create a new atmospheric_physics tag, assuming you have git installed on your local machine, by doing the following: 13 | 14 | **1. Download the latest version of the repo:** 15 | 16 | ``` 17 | git clone https://github.com/ESCOMP/atmospheric_physics.git 18 | cd atmospheric_physics 19 | ``` 20 | 21 | If you are tagging a development commit, then also make sure to checkout the development branch: 22 | 23 | ``` 24 | git checkout development 25 | ``` 26 | 27 | **2. Find the commit hash you want to tag. This can be done by opening the git log like so:** 28 | 29 | `git log` 30 | 31 | And finding the commit hash you are tagging, along with the commit message. If you simply want to tag the head of the repo (i.e. the latest commit), then do: 32 | 33 | `git log --oneline -1` 34 | 35 | And use the provided hash and message. 36 | 37 | **3. Create the tag:** 38 | 39 | `git tag -a -m ''` 40 | 41 | Where `` is the new tag name (which should follow the naming convention shown below), `commit_hash` is the commit hash you found in step 2, and `commit_message` is the message/description associated with that commit. 42 | 43 | **4. Push the new tag back to the repo:** 44 | 45 | `git push origin ` 46 | 47 | After which the new tag should now exist in the ESCOMP/atmospheric_physics repo. 48 | 49 | ## Tag naming conventions 50 | 51 | All ESCOMP/atmospheric_physics tags for the main branch should look like the following: 52 | 53 | `atmos_physX_YY_ZZZ` 54 | 55 | While all tags for the development branch should be: 56 | 57 | `dev_atmos_physX_YY_ZZZ` 58 | 59 | Where `X` is the major release version (which should almost always be left as-is), `YY` is for whenever a new physics scheme or suite is added, and `zz` is for any other minor release/bug fix. 60 | 61 | So, for example, if the latest tag is: 62 | 63 | `atmos_phys0_05_025` 64 | 65 | And you want to tag a new commit that fixed a bug, then the new tag should be: 66 | 67 | `atmos_phys0_05_026` 68 | 69 | However, if you instead added an entirely new physics scheme to the repo, then the new tag should be: 70 | 71 | `atmos_phys0_06_000` 72 | 73 | Hope that helps, and good luck with tagging! 74 | -------------------------------------------------------------------------------- /.github/workflows/docs-deploy.yml: -------------------------------------------------------------------------------- 1 | name: docs 2 | on: 3 | push: 4 | branches: 5 | - main 6 | 7 | permissions: 8 | contents: write 9 | 10 | jobs: 11 | deploy: 12 | runs-on: ubuntu-latest 13 | steps: 14 | - uses: actions/checkout@v4 15 | with: 16 | fetch-depth: 0 17 | 18 | - name: Check out CAM-SIMA 19 | uses: actions/checkout@v4 20 | with: 21 | repository: ESCOMP/CAM-SIMA 22 | ref: development 23 | path: CAM-SIMA 24 | 25 | - name: Configure Git Credentials 26 | run: | 27 | git config user.name github-actions[bot] 28 | git config user.email 41898282+github-actions[bot]@users.noreply.github.com 29 | 30 | - name: Setup Python 31 | run: | 32 | sudo apt update 33 | sudo apt install -y python3 python3-venv python3-pip 34 | echo "cache_id=$(date --utc '+%F')" >> $GITHUB_ENV 35 | echo "python_version=$(python3 --version | awk '{print $2}')" >> $GITHUB_ENV 36 | 37 | - name: Configure cache for pip 38 | uses: actions/cache@v4 39 | with: 40 | key: python-${{ env.python_version }}-pip-${{ env.cache_id }} 41 | path: ~/.cache/pip 42 | restore-keys: | 43 | python-${{ env.python_version }}-pip- 44 | 45 | - name: Setup FORD 46 | run: | 47 | sudo apt install -y graphviz 48 | python3 -m venv venv-ford 49 | source venv-ford/bin/activate 50 | python3 -m pip install Jinja2 Pygments beautifulsoup4 graphviz markdown markdown-include pcpp python-markdown-math rich tomli tomli-w toposort tqdm 51 | python3 -m pip install --no-deps "ford@git+https://github.com/kuanchihwang/ford.git@develop/cam-sima-ford-patches" 52 | 53 | - name: Setup MkDocs 54 | run: | 55 | python3 -m venv venv-mkdocs 56 | source venv-mkdocs/bin/activate 57 | python3 -m pip install mkdocs-material mkdocs-git-revision-date-localized-plugin mkdocs-git-committers-plugin-2 58 | 59 | - name: Generate documentation by FORD 60 | run: | 61 | source venv-ford/bin/activate 62 | cd CAM-SIMA 63 | ford src/dynamics/mpas/assets/ford_config.md 64 | mkdir -pv ../docs/dycore/mpas 65 | cp -afv src/dynamics/mpas/assets/ford/* ../docs/dycore/mpas 66 | 67 | - name: Configure cache for mkdocs-material 68 | uses: actions/cache@v4 69 | with: 70 | key: mkdocs-material-${{ env.cache_id }} 71 | path: .cache 72 | restore-keys: | 73 | mkdocs-material- 74 | 75 | - name: Deploy to gh-pages 76 | run: | 77 | source venv-mkdocs/bin/activate 78 | mkdocs gh-deploy -b gh-pages --force 79 | -------------------------------------------------------------------------------- /docs/conversion/back-to-cam.md: -------------------------------------------------------------------------------- 1 | # 9 - Bring back into CAM 2 | 3 | ## Code modifications 4 | Bring the CCPP version back into a new CAM clone or branch 5 | 6 | - Make a new clone or branch of CAM6 (personal preference) 7 | - Remove the non-CCPP version from your CAM source directory 8 | - Make a new directory in the atmosperhic_physics submodule of CAM (make sure the directory name is the one you used in step one of the conversion to CCPP): 9 | ``` 10 | mkdir src/atmos_phys/schemes/ 11 | ``` 12 | - Copy your new CCPP version files from CAM-SIMA into this new directory 13 | - Update the routine names to match the CCPP-ized subroutine names where the subroutines are called 14 | - Update the calling lists if they were changed when the CCPP-ized subroutines were made 15 | - If the CCPP-ized module is in a new subdirectory, you will need to add it to `bld/configure`: 16 | ``` 17 | print $fh "$camsrcdir/src/atmos_phys/schemes/\n"; 18 | ``` 19 | 20 | !!!Note "Handling readnl and diagnostics in CAM" 21 | The original `_readnl` subroutine (if there was one) must still be called in runtime_opts.F90 and all addfld/outfld calls must be preserved. If your new CCPP interfaces are called directly by physpkg, this may mean that you need to create a `_cam.F90` module that contains `_readnl`. 22 | 23 | ## Testing 24 | 25 | - To quickly test that your code is working during development, you can create two new model cases with a compset that involves your physics scheme, and the following XML changes: 26 | 27 | ``` 28 | ./xmlchange --force ROF_NCPL=\$ATM_NCPL 29 | ./xmlchange DOUT_S=FALSE,STOP_OPTION=nsteps,STOP_N=9,DEBUG=TRUE 30 | ``` 31 | 32 | !!! Note - test cases 33 | The first case will be CAM without your changes, and the second will be CAM with your new physics scheme code changes. Both simulations should produce a `.cam.rh0.XXX` file, which you can compare by using the cprnc tool, which can be found (and used) on derecho below. Your new physics code is working correctly if the printed output from cprnc states that there are no differences in the files. 34 | 35 | ``` 36 | /glade/campaign/cesm/cesmdata/cprnc/cprnc 37 | cprnc 38 | ``` 39 | - Optional step: if want to have additional testing... Once you believe it is working, make a [snapshot](create-snapshots.md) to run CAM with this modified clone. 40 | - Use the cprnc tool to compare the resulting CAM6 “after” snapshot file with your original CAM6 “after” snapshot file (from before the port). If they differ and you changed the internal calculations during your port, you will need to determine if these changes are expected and correct. 41 | - If they are different (and the changes are correct), then you will need to repeat the steps listed in the [Run CAM-SIMA](run-cam-sima.md) section with the newly generated snapshot files from the modified CAM clone. 42 | 43 | Proceed to [Step 10 - Final Steps](final-steps.md) -------------------------------------------------------------------------------- /docs/atmospheric_physics/directory-structure.md: -------------------------------------------------------------------------------- 1 | # atmospheric_physics directory structure 2 | 3 | This page lists out the directory structure for the [atmospheric_physics](https://github.com/ESCOMP/atmospheric_physics) repo, and what the general purpose of each directory and subdirectory is. 4 | 5 | ## Top-level directories 6 | 7 | These directories represent code and tools that are contained within the atmospheric_physics repository. 8 | 9 | ### **doc/** 10 | 11 | Contains files used to document the current state of the atmospheric_physics repository, such as a `ChangeLog` and the `NamesNotInDictionary.txt` file which lists all standard names that are currently not in the official [CCPP Standard Names repo](https://github.com/ESCOMP/CCPPStandardNames). More on the NamesNotInDictionary.txt file [here](development_workflow.md/#updating-namesnotindictionarytxt-file). 12 | 13 | ### **phys_utils/** 14 | 15 | Contains portable helper functions and utilities for use in CCPP-ized physics schemes. A general requirement is to have all routines in this directory automatically unit tested, either in atmospheric_physics itself or in a host model like CAM-SIMA. 16 | 17 | ### **schemes/** 18 | 19 | Contains subdirectories for all of the CCPP-ized physics schemes, with the subdirectories containing the core physics scheme source code, 20 | the associated CCPP metadata files, namelist XML files, and any relevant dependency files outside of `phys_utils` and `to_be_ccppized`. 21 | 22 | ### **suites/** 23 | 24 | Contains [CCPP Suite Definition Files (SDFs)](https://ccpp-techdoc.readthedocs.io/en/v6.0.0/ConstructingSuite.html?highlight=sdf#constructing-suites) for official scientific and production configurations of the CAM-SIMA host model. 25 | 26 | ### **test/** 27 | 28 | Contains code and tools used to run tests on the CCPP-ized physics schemes. 29 | 30 | **Subdirectories**: 31 | 32 | - cmake - Contains cmake files needed to configure and build unit tests. 33 | - docker - Contains dockerfiles needed to configure, build, and run unit tests. 34 | - include - Contains utility source code needed to build and run unit tests. 35 | - musica - Contains test code for the CCPP-ized [Multi-Scale Infrastructure for Chemistry Modeling (MUSICA)](https://github.com/NCAR/musica) components. 36 | - test_schemes - Contains CCPP physics schemes that are only used for testing. 37 | - test_suites - Contains test SDFs for use in CAM-SIMA snapshot regression testing. 38 | - unit-test - Contains the source code for the automated Fortran unit testing with pFUnit 39 | - include - Contains "mock" source code needed to run unit tests without bringing in an entire host model 40 | - tests - Contains subdirectories which have the actual pFUnit unit test code and related CMake files for building. 41 | 42 | ### **to_be_ccppized/** 43 | 44 | Contains source code files that are needed by certain CCPP physics schemes, but which are not 45 | yet fully CCPP compliant (e.g. have code that is still host-model-specific). -------------------------------------------------------------------------------- /mkdocs.yml: -------------------------------------------------------------------------------- 1 | site_name: CAM-SIMA 2 | repo_url: https://github.com/ESCOMP/CAM-SIMA-docs 3 | repo_name: CAM-SIMA-docs 4 | edit_uri: edit/main/docs 5 | nav: 6 | - Home: index.md 7 | - Conversion: 8 | - conversion/conversion-background.md 9 | - conversion/convert-portable-layer.md 10 | - conversion/create-metadata.md 11 | - conversion/create-namelist-xml.md 12 | - conversion/interstitials.md 13 | - conversion/create-sdf.md 14 | - conversion/create-snapshots.md 15 | - conversion/check-metadata.md 16 | - conversion/run-cam-sima.md 17 | - conversion/back-to-cam.md 18 | - conversion/final-steps.md 19 | - conversion/walkthrough.md 20 | - Design: 21 | - design/cam-build-process.md 22 | - design/cam-run-process.md 23 | - design/ccpp-in-cam-sima.md 24 | - design/constituents.md 25 | - design/directory-structure.md 26 | - design/history.md 27 | - design/sima-design-goals.md 28 | - Development: 29 | - development/cam-sima-workflow.md 30 | - development/cam-coding-standards.md 31 | - development/cam-testing.md 32 | - development/debugging.md 33 | - development/git-basics.md 34 | - development/git-faq.md 35 | - development/git-fleximod.md 36 | - development/tool-recommendations.md 37 | - Usage: 38 | - usage/creating-a-case.md 39 | - usage/history.md 40 | - Atmospheric_physics: 41 | - atmospheric_physics/directory-structure.md 42 | - atmospheric_physics/development_workflow.md 43 | - atmospheric_physics/Tagging-Instructions.md 44 | - atmospheric_physics/physics-file-io.md 45 | - Dynamical Core: 46 | - MPAS: dycore/mpas 47 | 48 | plugins: 49 | - search 50 | # Date plugin requires fetch-depth: 0 51 | - git-revision-date-localized: 52 | enable_creation_date: true 53 | enabled: !ENV [CI, false] 54 | - git-committers: 55 | repository: ESCOMP/CAM-SIMA-docs 56 | branch: main 57 | enabled: !ENV [CI, false] 58 | 59 | theme: 60 | icon: 61 | repo: fontawesome/brands/github 62 | edit: material/pencil 63 | view: material/eye 64 | admonition: 65 | info: material/arrow-up-down-bold 66 | palette: 67 | # Palette toggle for automatic mode 68 | - media: "(prefers-color-scheme)" 69 | toggle: 70 | icon: material/brightness-auto 71 | name: Switch to light mode 72 | # Palette toggle for light mode 73 | - media: "(prefers-color-scheme: light)" 74 | scheme: default 75 | 76 | toggle: 77 | icon: material/brightness-7 78 | name: Switch to dark mode 79 | 80 | # Palette toggle for dark mode 81 | - media: "(prefers-color-scheme: dark)" 82 | scheme: slate 83 | toggle: 84 | icon: material/brightness-4 85 | name: Switch to system preference 86 | name: material 87 | features: 88 | - content.code.copy 89 | - content.action.edit 90 | - content.action.view 91 | 92 | markdown_extensions: 93 | - def_list 94 | - tables 95 | - pymdownx.tasklist: 96 | custom_checkbox: true 97 | - attr_list 98 | - md_in_html 99 | - admonition 100 | 101 | extra_css: 102 | - stylesheets/extra.css 103 | -------------------------------------------------------------------------------- /docs/development/git-faq.md: -------------------------------------------------------------------------------- 1 | # git & GitHub FAQ 2 | 3 | ## Q: How can I clone someone's git clone to my directory on the same machine? 4 | *A*: `git clone []` where `` is the location of the source git clone and the optional `` is the name of the clone (default is a local directory of the same name as the original). 5 | 6 | ## Q: How can I clone someone's git clone to my directory on a different machine? 7 | *A*: `git clone ssh://@: []` where `` is your user name on the remote machine, `` is the machine name (e.g., derecho.hpc.ucar.edu), `` is the location of the source git clone on the remote machine, and the optional `` is the name of the clone (default is a local directory of the same name as the original). 8 | 9 | ## Q: How can I look at someone's PR code? 10 | *A*: There a a few ways to do this: 11 | 12 | - On GitHub (like looking at any other code on GitHub) 13 | - Add the PR fork to my remote (allows using tools such as [`git diff` or `git difftool`](git-basics.md#comparing-differences-using-git-diff) with your existing branches or `development`) 14 | - As a clone (standalone clone on your machine). 15 | 16 | A first step is to find the link to the fork's branch. Just below the PR is a line that starts with a colored oval (e.g., "Open" in green) and looks something like: 17 | ``` 18 | Octocat wants to merge 123 commits into ESCOMP:development from Octocat:amazing-new-feature 19 | ``` 20 | 21 | Clicking on the last part (`Octocat:amazing-new-feature`) will take you to the correct branch where you can browse the code (the first method above). If you want to download that code, click the green "Code" button and then click the clipboard icon. Be sure to take note of the branch name. 22 | 23 | - To load this code into your clone, cd to your clone directory, add the PR fork as a new remote, and checkout the branch. For instance: 24 | ``` 25 | git remote add octocat https://github.com/octocat/CAM-SIMA.git 26 | git fetch --no-tags octocat 27 | git checkout octocat/amazing-new-feature 28 | ``` 29 | 30 | Instead of the `checkout` you can also do diffs: 31 | ``` 32 | git difftool origin/development octocat/amazing-new-feature 33 | ``` 34 | 35 | - If you want to make a new clone with the PR code, simply do: 36 | ``` 37 | git clone -b amazing-new-feature octocat https://github.com/octocat/CAM-SIMA.git octocat_cam 38 | cd octocat_cam-sima 39 | ``` 40 | 41 | ## Q: Why do Pull Request (PR) code review take so long? 42 | *A*: A code review must fulfill three purposes: 43 | 44 | - Code reviewers must make sure that new and/or changed code does not affect any currently-supported functionality (i.e., it cannot break anything). 45 | - While regression tests will catch many of these issues, reviewers must also check for usage of or reliance on deprecated code, and also for any code which is not supported on all platforms and compilers used by the CESM community. 46 | - Code reviewers must make sure that any new functionality or changes are implemented correctly and at least somewhat efficiently. They must also ensure that important changes are tested against future regressions. 47 | - The CAM SE team is almost always engaged in several projects to implement new CAM functionality along with supporting infrastructure. Each CAM SE usually looks at each PR in order to prevent new code from interfering with those plans. 48 | 49 | The first two steps are usually completed by a single SE although SEs engaged in a final review will often find missed errors. This is similar to peer reviewers finding problems with a paper even after reviews done by colleagues. 50 | 51 | ## Q: How do I update my branch to the current cam_development? 52 | *A*: [see this section](git-basics.md#updating-your-branch-to-latest-development) -------------------------------------------------------------------------------- /docs/design/cam-build-process.md: -------------------------------------------------------------------------------- 1 | # Build process 2 | 3 | In order to describe the build process, we need to define several source and build directories: 4 | 5 | * `` : The CAM-SIMA sandbox being built. From a case directory, this can be found with `./xmlquery SRCROOT.` 6 | * ``: The location of the case being built. 7 | * ``: The location of the CAM-SIMA source mods, located at `/SourceMods/src.cam.` 8 | * `` : The location of CAM-SIMA generated code and compiled code (`.o` and `.mod`). See `atm/obj` under `./xmlquery OBJROOT`. 9 | 10 | 11 | ## Build Sequence 12 | 13 | Given the context above, the following is the CAM-SIMA build sequence: 14 | 15 | 1. Create a `ConfigCAM` object, used to store build configuration information. Code can be found in `/cime_config/cam_config.py` 16 | - Inputs: 17 | - ``: A CIME case object 18 | - ``: A python logger, which is usually created by CIME itself 19 | - Outputs: 20 | - `config_dict`: A dictionary of configure options, with the dictionary keys being the name of the config variable/option, and the dictionary values being special config variable objects that contain the following properties: 21 | - `name`: The name of the config option/variable. This is what is used as the key in the `config_dict` dictionary 22 | - `desc`: A text description of what this particular config option is, and what sort of values the config option should have. 23 | - `value`: The value of that config option/variable. Currently can only be an integer, string, or list of integers or strings. 24 | - `valid_vals`: Optional property that contains either a list or range of valid values for this particular config entry 25 | - `valid_type`: Optional property for a list-type config option that states what data type the list elements can have. 26 | - `is_nml_attr`: Logical that states whether this config option can be used as an XML attribute in a namelist definition file. 27 | - Additional info: 28 | - The `ConfigCAM` object also has various build-in methods that are used to call other steps in the build workflow, change the value of a given config variable, or print extra config info to the provided python logger. 29 | - The `ConfigCAM` object currently has no uses outside `buildnml` and `buildlib`, and so is not saved after those steps are complete. 30 | 1. Generate CAM-SIMA source code. This sequence has several steps, each of which is performed if any of its inputs has changed. 31 | - Create or read in a `BuildCacheCAM` object (`/cime_config/cam_build_cache.py`). 32 | - ``: An optional cache file (created by previous build). This file is created in the case build directory (bld/atm/obj). 33 | - Create the physics derived data types using the registry (if required). 34 | - Find all scheme metadata files for configured suites. 35 | - Find any active schemes with namelist XML variables 36 | - Create metadata file with namelist variables 37 | - Create namelist-reading module (to go with metadata file). 38 | - Call the [CCPP Framework](https://ccpp-techdoc.readthedocs.io/en/v6.0.0/) to generate glue code (CAPS), (if required). 39 | 40 | ## CAM-SIMA source and namelist generation (buildnml) workflow 41 | ![text](figures/buildnml_workflow.jpg "CAM-SIMA buildnml workflow") 42 | 43 | The diagram above displays everything that occurs when CAM-SIMA's `buildnml` is called by CIME, which occurs after the user calls `preview_namelists`, `case.build`, or `case.submit`. 44 | 45 | All blue boxes represent source code, and are listed as "source file: function/object used", and all objects or functions that have a "+" are automatically tested whenever a Pull Request is opened, updated, or merged. 46 | 47 | All orange boxes are input files that are used by the code, while all green boxes are output files that are generated. It is important to note that additional files are generated as well, specifically a build cache and a CCPP datatable, but those files are used internally in the build scripts shown here and not by the final model executable, and thus aren't listed here. 48 | 49 | Finally, the arrows show the order of operations, starting with `buildnml`, with the top two source code boxes representing python classes that are used by the functions/objects directly below them. -------------------------------------------------------------------------------- /docs/conversion/conversion-background.md: -------------------------------------------------------------------------------- 1 | # 0 - Background & Prep work 2 | 3 | ## Background 4 | 5 | ### Running jobs in CAM and CAM-SIMA 6 | See [this](../usage/creating-a-case.md) section for how to run CAM-SIMA and CAM. 7 | 8 | Depending on which machine you are on, you may prefer to run the ./case.build command on a compute node instead of the login node due to user resource utilization limits on the login nodes. 9 | 10 | ## Prep Work 11 | 12 | ### Conversion Spreadsheet 13 | Put the parameterization that you are going to convert into the [conversion spreadsheet](https://docs.google.com/spreadsheets/d/1_1TTpnejam5jfrDqAORCCZtfkNhMRcu7cul37YTr_WM/edit#gid=0). 14 | 15 | ### Create Github Issues 16 | 1. Create a Github Issue in the [ESCOMP/CAM](https://github.com/ESCOMP/CAM) repo that states which physics parameterization you are planning to convert to the CCPP framework. 17 | - be sure to add the `ccpp-conversion` label 18 | 1. Create another issue in the [ESCOMP/atmospheric physics](https://github.com/NCAR/atmospheric_physics) repo describing the same physics parameterization that you are now planning to add to the collection of NCAR CCPP physics suites. Doing this allows the software engineers to keep track of which physics routines are being worked on, and which still need to be assigned. The goal of converting the physics parameterization is to ultimately have the CCPP-ized physics package reside in [ESCOMP atmospheric physics](https://github.com/NCAR/atmospheric_physics) and be removed from [ESCOMP/CAM](https://github.com/ESCOMP/CAM). 19 | 20 | ### Setting up your sandbox 21 | 22 | Make sure you have github forks for both [ESCOMP/CAM-SIMA](https://github.com/ESCOMP/CAM-SIMA) and [ESCOMP/atmospheric_physics](https://github.com/ESCOMP/atmospheric_physics). If needed see [Working with git and GitHub](../development/git-basics.md) 23 | 24 | 25 | To begin, fork ESCOMP/CAM-SIMA: 26 | ![text](figures/fork-cam-sima.png "Forking CAM-SIMA") 27 | 28 | And select the `Create new fork` option. This will bring you to the "Create new fork" screen: 29 | ![text](figures/fork-cam-sima-2.png "Forking CAM-SIMA") 30 | 31 | !!! warning "Uncheck the "Copy the `main` branch only" option" 32 | 33 | Failure to uncheck this will prevent you from pulling in updates from the `development` branch easily. 34 | 35 | #### Set up local clones and branches 36 | 37 | - On the machine and in the directory in which you would like to develop, clone CAM-SIMA (the CCPP-ized version of CAM) with your fork as the origin: 38 | ``` 39 | git clone -o https://github.com//CAM-SIMA CAM-SIMA 40 | ``` 41 | 42 | - Navigate into the directory you just created: 43 | ``` 44 | cd CAM-SIMA 45 | ``` 46 | 47 | - Add the `ESCOMP` remote so you can keep your fork up to date: 48 | ``` 49 | git remote add ESCOMP https://github.com/ESCOMP/CAM-SIMA 50 | ``` 51 | 52 | - Fetch the upstream ESCOMP tags and branches from GitHub: 53 | ``` 54 | git fetch --tags ESCOMP 55 | ``` 56 | 57 | - Create and checkout a new branch off of the `development` branch of the ESCOMP remote: 58 | ``` 59 | git branch ESCOMP/development 60 | git checkout 61 | ``` 62 | 63 | - Set up the upstream (on GitHub) version of your branch: 64 | ``` 65 | git push -u 66 | ``` 67 | 68 | - Populate your externals: 69 | ``` 70 | bin/git-fleximod update 71 | ``` 72 | 73 | - Make a new directory for your parameterization within the atmospheric_physics submodule directory 74 | ``` 75 | mkdir src/physics/ncar_ccpp/schemes/ 76 | ``` 77 | 78 | - Navigate into the atmospheric_physics submodule directory 79 | ``` 80 | cd src/physics/ncar_ccpp 81 | ``` 82 | 83 | - Add your fork of atmospheric_physics as a remote 84 | ``` 85 | git remote add https://github.com//atmospheric_physics 86 | ``` 87 | 88 | - Fetch the upstream branches and tags 89 | ``` 90 | git fetch 91 | ``` 92 | 93 | - Create, checkout, and push a new branch upstream 94 | ``` 95 | git checkout -b 96 | git push -u 97 | ``` 98 | !!! Warning "Multiple repositories!" 99 | As you make changes and want to commit them to your github repos, you’ll be managing two separate repos. When you issue git commands, be aware of where you are in your code tree. 100 | 101 | - If you want to see changes in CAM-SIMA, you can issue a “git status” in the main CAM-SIMA directory. 102 | - If you want to see changes in the atmospheric_physics repo, make sure you are in src/physics/ncar_ccpp before you issue the “git status” command. 103 | - All other git commands will be relative to your current working directory as well. 104 | 105 | Congratulations! You've set up your sandbox! Proceed to [1 - Convert the "portable" layer](convert-portable-layer.md) -------------------------------------------------------------------------------- /docs/development/cam-sima-workflow.md: -------------------------------------------------------------------------------- 1 | # Development workflow for CAM-SIMA 2 | 3 | This page describes the general workflow for adding new development to the [CAM-SIMA repository](https://github.com/ESCOMP/CAM-SIMA). 4 | 5 | ## Workflow summary 6 | 7 | The general workflow for adding a feature, bug-fix, or modification to CAM-SIMA is as follows: 8 | 9 | 1. [**Open an issue**](#open-an-issue) 10 | 1. [**Add your code modifications**](#add-your-code-modifications) 11 | 1. [**Open a PR**](#open-a-pr) 12 | - If you know that this PR will need an official tag, then also add the [tag name](git-basics.md/#tagging-a-commit) to the PR description. 13 | 1. **Respond to any reviewer requests.** 14 | 1. [**Fix any failing tests**](#fix-failing-tests) 15 | 1. [**Run the manual regression tests**](cam-testing.md/#regression-testing) 16 | - Update `$CAM-SIMA/test/existing-test-failures.txt` if there are new tests failing or if existing test failures have been fixed. 17 | 1. **Confirm with the CAM-SIMA gatekeeper that you are OK to merge. Then squash the commits and merge the PR** (e.g. the "squash and merge" option). 18 | 1. [**Make a tag (if regression test answers have changed)**](git-basics.md/#tagging-a-commit) 19 | 1. [**Archive baselines (if tag has been made)**](cam-testing.md/#archiving-baselines) 20 | 21 | ## Workflow details 22 | 23 | The following sections describe various workflow actions in more detail. 24 | 25 | ### Open an issue 26 | 27 | It is generally recommended to [open an issue](https://github.com/ESCOMP/CAM-SIMA/issues/new) for any new features that will be added or bug that has been found that will need to be fixed. There is currently no offiical requirement on what should be contained within the issue text, so generally just put any information you think might be relevant. 28 | 29 | ### Add your code modifications 30 | 31 | 1. If you haven't already, create a fork. Specific instructions can be found [here](git-basics.md/#one-time-github-setup). 32 | 1. Create a branch on your fork off of the head of the ESCOMP remote's `development` branch. Specific commands can be found [here](git-basics.md/#working-with-branches) 33 | 1. Apply your code modifications and/or script additions, and perform at least one test making sure your modifications work as expected. See below for Git commands for committing to your branch. 34 | 35 | There are multiple ways to do commit your changes, but one of the safer ways is to first check your status: 36 | ``` 37 | git status 38 | ``` 39 | 40 | This will provide a list of all modified files. For each one of those files whose modifications you want to add to the main package, you will do the following: 41 | ``` 42 | git add updated_fortran_file.F90 43 | ``` 44 | 45 | Where `updated_fortran_file.F90` should be replaced with whatever your file name is. Do this for each file you want to include. If you are confident that every file listed by `git status` needs to be added, then you can do it all at once by doing: 46 | ``` 47 | git add -A 48 | ``` 49 | 50 | You can then type `git status` again, at which point all of the files you added should be "staged" for commit (green). 51 | 52 | To commit your changes to your local branch: 53 | ``` 54 | git commit -m "" 55 | ``` 56 | 57 | where `` is a short description on sentence stating what the commits are for, e.g. "fixed color bar bug" or "added significance hatching". 58 | 59 | To push your changes to your fork: 60 | ``` 61 | git push 62 | ``` 63 | 64 | or, if you want to be more explicit, 65 | 66 | ``` 67 | git push origin 68 | ``` 69 | 70 | ### Open a PR 71 | 72 | 1. Go to the [ESCOMP CAM-SIMA repo](https://github.com/ESCOMP/CAM-SIMA), and click on the "Pull requests" tab. 73 | ![text](figures/pr-workflow-1.png "PR tab") 74 | 1. There, you should see a "New pull request" button, which you should click. 75 | ![text](figures/pr-workflow-2.png "New PR button") 76 | 1. On the new "Compare changes" page, you should see a "compare across forks" link, which you should click. 77 | ![text](figures/pr-workflow-3.png "Compare across forks") 78 | 1. You should now see two new pull-down boxes (to the right of an arrow). Using these pull-down boxes, select the "development" branch of the ESCOMP repo and select your fork (which should be `/CAM-SIMA`) 79 | 1. Then select the branch which contains the commits. 80 | ![text](figures/pr-workflow-4.png "pull-down boxes") 81 | 1. You should then see a list of all the different modifications. If they generally look correct, click the "Create pull request" button 82 | 1. A new page should appear. In the first text box, add the title of your pull request. The second text box will contain additional fields that you should fill out to the best of your ability. 83 | 1. If your code changes cause answer changes, you will need a new tag. Enter the expected tag name in the PR description. More on tag conventions [here](git-basics.md/#tagging-a-commit). 84 | 1. Add any relevant labels to the PR, add yourself as the assignee, and add any reviewers you would like to have. Otherwise, the core SE team will add reviewers for you. 85 | 1. Click "Create pull request" 86 | 87 | ### Fix failing tests 88 | Failing unit tests will appear at the bottom of the PR page (as shown below). You will also get an email if any tests fail. 89 | ![text](figures/pr-workflow-5.png "unit testing") 90 | 91 | Click through (via `Details` link) to see the test output of the failing tests. Refer to the table below for debugging failing tests. 92 | 93 | | Failing test | Description | Resources | 94 | | ------------ | ----------- | --------- | 95 | | `.github/workflows/fleximod_test.yaml / fleximod-test () (pull_request)` | git-fleximod test failure. Likely an invalid external | [git-fleximod testing](cam-testing.md#git-fleximod-tests) | 96 | | `Python unit tests / python_unit_tests () (pull_request)` | Python unit test failure. Likely need to update sample files | [Python unit testing](cam-testing.md/#python-unit-testing) | 97 | | `Source Code Linting / source_code_tests (pull_request)` | Python linting failure. Need to fix modified python code. | [Python source code linting](cam-testing.md/#python) | 98 | 99 | -------------------------------------------------------------------------------- /docs/atmospheric_physics/physics-file-io.md: -------------------------------------------------------------------------------- 1 | # Physics scheme file I/O 2 | 3 | This page describes the `ccpp_io_reader` class in `atmospheric_physics/utils`, including it's purpose, design, and usage in CCPP physics schemes. 4 | 5 | ## Why a special file reader class? 6 | 7 | Many physics schemes have a need to read specialized data from input files, especially those in the NetCDF file format. While one could just read all of the files on the host side before calling the CCPP, this would require a physics developer to modify the host model whenever they wanted to use the physics scheme, which arguably hurts portability. 8 | 9 | Alternatively, one could just use the NetCDF library directly in the physics scheme, but this oftentimes adds a lot of additional lines of code to the physics scheme, and may prevent the physics scheme from taking advantage of specialized I/O libraries contained within the host model that may improve performance, error handling, etc. 10 | 11 | To try and find an optimal solution to this problem, it was decided to create an abstract class that describes general interfaces for reading variables from a file, which allows a physics scheme developer to read data from a file without having to modify the host model or use the NetCDF library directly. At the same time, the actual implementation of the class can be done in the host model, which then allows for the use of host-specific I/O libraries, while still maintaining the portability of the physics scheme itself. 12 | 13 | ## Design and implementation of `ccpp_io_reader` in CAM-SIMA 14 | 15 | The file `ccpp_io_reader.F90` file itself contains the declaration of an abstract NetCDF reader type (`abstract_netcdf_reader_t`), including its various procedures, and an associated constructor function (`create_netcdf_reader_t`). This abstract interface is assumed to be the same for all host models and physics schemes, which allows CCPP physics schemes to remain portable, at least amongst the supporting host models. 16 | 17 | The actual implementation of this abstract class is expected to be done by the host model using whatever methodology it wants, as long as it doesn't violate the rules of the CCPP itself (e.g. it is thread safe). The specific implementation for CAM-SIMA can be [found here](https://github.com/ESCOMP/CAM-SIMA/blob/development/src/physics/utils/pio_reader.F90). It uses the [ParalellIO (PIO) library](https://github.com/NCAR/ParallelIO), which is used in NCAR's [CESM](https://github.com/ESCOMP/CESM) model (the parent model of CAM-SIMA) to read and write NetCDF files. 18 | 19 | There are several ways one could make the connection between the abstract class and the actual, concrete implementation in Fortran. For the abstract NetCDF reader defined in `ccpp_io_reader.F90`, it was decided to use the Fortran submodule approach, which was a feature introduced in the Fortran 2008 standard. This approach allows a physics scheme to bring in the file reading module via a Fortran `use` statement, and then declare, construct, and work with reader object in an entirely local way within the scheme. An alternative method would be to have the host model instantiate the object on the host side before any CCPP physics steps are called, but this would require the object to be permanently held in memory, passed around as a subroutine argument to all of the physics schemes that use it (which would require additional metadata), and provide a way to open multiple files within the same object. 20 | 21 | An example of the submodule approach connecting the abstract interface with a "concrete" implementation can be found in [CAM-SIMA here](https://github.com/ESCOMP/CAM-SIMA/blob/development/src/physics/utils/pio_reader_sub.F90). 22 | 23 | ## File reader class methods and code example 24 | 25 | The following methods are provided either for or by the NetCDF reader class (`abstract_netcdf_reader_t`). Additional methods, or changes to the existing methods, may be added as new functionality is introduced. In the list below, the variable `reader` represents a variable declared as `class(abstract_netcdf_reader_t), pointer` in the physics scheme. 26 | 27 | ### *reader => create_netcdf_reader_t()* 28 | 29 | The constructor function for the `abstract_netcdf_reader_t` class. This must be used to actually allocate/associate the object. It has no input arguments. 30 | 31 | ### *reader%open_file(file_path, errmsg, errcode)* 32 | 33 | Opens the NetCDF file and stores the associated (meta)data within the object. The only input is `file_path`, which is a string containing the path to the file, while `errmsg` and `errcode` are standard CCPP outputs, with `errcode` being a non-zero integer, and `errmsg` being a non-empty string, if an error occurs. 34 | 35 | Please note that all of the remaining methods below also have `errcode` and `errmsg`, which behave in the exact same way. 36 | 37 | ### *reader%close_file(errmsg, errcode)* 38 | 39 | Closes the NetCDF file that was opened by an `open_file` call. It has no inputs. 40 | 41 | ### *reader%get_var(varname, var, errmsg, errcode, start, count)* 42 | 43 | Reads in a variable from the NetCDF file that was opened with `open_file`. The method has two required inputs: 44 | 45 | 1. `varname` -> A character-type variable which contains the name of the variable on the NetCDF file. 46 | 2. `var` -> An assumed-shape allocatable variable of type integer, real, or character with the same rank (i.e. shape) as the variable on the file. This variable will automatically be allocated to the same dimensions as described for the variable in the NetCDF file, and will contain the actual variable data. 47 | 48 | It also has two optional inputs: 49 | 50 | 1. `start` - a 1-D integer array with the same number of elements as there are dimensions for the variable that indicate the starting indices when subsetting the data. CURRENTLY NOT USED. 51 | 2. `count` - a 1-D integer array with the same number of elements as there are dimensions for the variable that indicate the number of elements, starting from `start` that will be read in when subsetting the data. CURRENTLY NOT USED. 52 | 53 | 54 | Finally, an actual example of using the NetCDF reader object in a CCPP physics scheme can be [found here](https://github.com/ESCOMP/atmospheric_physics/blob/development/test/test_schemes/file_io_test.F90). 55 | -------------------------------------------------------------------------------- /docs/conversion/create-namelist-xml.md: -------------------------------------------------------------------------------- 1 | # 3 - Create namelist XML file 2 | If your scheme has a `readnl` scheme, create an .xml file with the same name as the scheme which uses the namelist (place it in the same location as the parameterization in `ncar_ccpp`. 3 | 4 | - The filename should be `_namelist.xml` 5 | - If you have metadata entries for these namelist entries from your previous conversion steps, it may be useful to copy the metadata entry and edit it to be xml. If you do this, you will delete the dimensions and intent lines and add category, group, desc and values/value. 6 | 7 | The CCPP Framework will autogenerate the namelist reader based on the elements in the namelist XML file associated with the parameterization. Each namelist element (``) will have the following fields: `` ``, ``, ``, ``, ``, `` and ``. Note that all variables in the namelist should appear in the parameterization’s `_init` subroutine interface. The description of each field is as follows: 8 | 9 | - *id*: The namelist variable's name. The name must be lower case. The module converts all namelist variable names to lower case since Fortran is case insensitive. 10 | - *type*: An abbreviation of the fortran declaration for the variable. Any of these types may be followed by a comma separated list of integers enclosed in parentheses to indicate an array. The current namelist validation code only distinguishes between string and non-string types. Valid declarations are: 11 | - char*n 12 | - integer 13 | - logical 14 | - real 15 | - *kind*: Only needed if type is real (if type is real, 'kind' will be 'kind_phys') 16 | - *input_pathname* (optional): Only include this attribute to indicate that the variable contains the pathname of an input dataset that resides in the CESM inputdata directory tree. Note that the variables containing the names of restart files that are used in branch runs don't reside in the inputdata tree and should not be given this attribute. The recognized values are: 17 | - "abs" to indicate that an absolute pathname is required, or 18 | - "rel:var_name" to indicate that the pathname is relative and that the namelist variable "var_name" contains the absolute root directory 19 | - *category*: A category assigned for organized the documentation. This can be found by copying from the `bld/namelist_files/namelist_definition.xml` file in CAM 20 | - *group*: The namelist group that the variable is declared in (found in `_readnl` in CAM) 21 | - *standard_name*: The CCPP standard name 22 | - *units*: Units for the variable. Must adhere to CCPP units convention (link?) 23 | - *desc*: The description for this namelist variable. This can be found by copying from the `bld/namelist_files/namelist_definition.xml` file in CAM. This is parsed and used in the creation of the CAM namelist web page. Description should be adequate for this purpose (**Maybe not? This is implemented in SIMA yet...). 24 | - *value*: Initial value for the variable. This can be a valid default value or an out-of-range value to allow for trapping in the code. 25 | - *valid_values* (optional): Attribute that is mainly useful for variables that have only a small number of allowed values. 26 | 27 | **Example Namelist XML file** 28 | ``` 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | real 40 | cldfrc 41 | cldfrc_nl 42 | tunable_parameter_for_top_pressure_bound_for_mid_level_clouds_for_cloud_fraction 43 | Pa 44 | 45 | Top pressure bound for mid level cloud. 46 | 47 | 48 | 75000.0D0 49 | 40000.0D0 50 | 40000.0D0 51 | 52 | 53 | 54 | 55 | 56 | ``` 57 | !!!Note "namelist values" 58 | When porting namelist values from CAM namelist defaults, the attributes supported in CAM-SIMA can be retrieved by printing out `cam_nml_dict` (or `cam_nml_dict.keys()`) in `cime_config/buildnml` [at the end of the function call](https://github.com/ESCOMP/CAM-SIMA/blob/3699359ebfe81b00273a9815d128cae903a26208/cime_config/buildnml#L355). An example of supported attributes that may be commonly used are: `phys_suite`, `ic_ymd`, `dyn`, `hgrid`, `analytic_ic`, `ocn`. 59 | 60 | ## For physics schemes split into multiple CCPP schemes, have a dedicated namelist read scheme 61 | 62 | For physics parameterizations that are split into multiple CCPP schemes (within a single or several modules), create a dedicated namelist options scheme to centralize namelist variable reading. This approach makes dependencies on namelist variables explicit, including in the case where a physics scheme depends on the namelist variables of another, different physics scheme (e.g., Hack shallow convection using ZM deep convection namelist options.) and facilitates future centralized modifications of namelist parameters for a physical scheme by scientists. 63 | 64 | The namelist options scheme can follow the pattern of the [zm_conv_options](https://github.com/ESCOMP/atmospheric_physics/blob/main/schemes/zhang_mcfarlane/zm_conv_options.F90) scheme used for reading Zhang-McFarlane deep convective namelist options: 65 | * The namelist options scheme **has to be in a separate module file**, e.g., `zm_conv_options.F90` and `zm_conv_options.meta`. No other CCPP schemes can be in the same module; 66 | * Only an `init` phase is needed, e.g., `zm_conv_options_init`. Only having an `init` phase dedicated for namelist parameters, and no other compute, allows for these namelist variables to be read without having to run any logic for a particular scheme; 67 | * The namelist XML file is attached to this module, e.g., `zm_conv_options_namelist.xml`. 68 | 69 | Once this scheme is added to the SDF, namelist quantities defined in the associated `_namelist.xml` file are just regular CCPP quantities, i.e., they can be retrieved directly via their standard name as inputs to other physics schemes. There is no need to `use` variables from this namelist read module. 70 | 71 | Once you have made your namelist file, proceed to [4 - Interstitials](interstitials.md) 72 | -------------------------------------------------------------------------------- /docs/conversion/create-metadata.md: -------------------------------------------------------------------------------- 1 | # 2 - Create metadata 2 | 3 | ## Create template 4 | To create a template metadata file based on the parameterization, run the following command in the directory with the routine you are converting (.F90): 5 | ``` 6 | python $CAM-SIMA/ccpp_framework/scripts/ccpp_fortran_to_metadata.py .F90 7 | ``` 8 | 9 | If you get errors: 10 | 11 | 1. Confirm you included the [argtable lines](convert-portable-layer.md#1b-add-required-htmlinclude-lines) above each routine 12 | 1. If you see a message "Missing local variables" and the variable is there, it may be missing its intent attribute 13 | 1. Hopefully other errors will be easy to address; consult other CAM SEs if there's any remaining confusion/errors 14 | 15 | Completion of `ccpp_fortran_to_metadata.py` will result in the creation of `.meta` 16 | 17 | ## Complete metadata 18 | 19 | Replace all `enter_*` sections with appropriate information (standard_name, units, dimensions, long_name). Special handling for [error variables](#error-variables), [constituents](#constituents), and [dependencies](#dependencies) can be found below. 20 | 21 | !!! Warning "Named dimension in metadata template" 22 | If you have a dimension in the metadata field which is NOT of the form `enter_standard_name_X:enter_standard_name_Y` but is rather a single dimension without a :, this means that you have a named dimension in your converted routine and you should remove the name and replace it with ":" 23 | 24 | - *standard_name*: the mapping between CAM variables and standard_names can be found in [this spreadsheet](https://docs.google.com/spreadsheets/d/1vpQ_xDZk00Z-_3SpW5N2EF3_FY6K7opNN4cqtSMlbwU/edit?gid=0#gid=0). All official standard names (approved by all stakeholders) can be found in the [CCPPStandardNames repo](https://github.com/ESCOMP/CCPPStandardNames/blob/main/Metadata-standard-names.md). Note that this does not yet contain all of the spreadsheet names. 25 | - You may have to trace the variable back through the CAM code to find the `Snapshot or Local name` to look for 26 | - Officially accepted standard names can be found in this [repository](https://github.com/ESCOMP/CCPPStandardNames/blob/main/Metadata-standard-names.md), but the CAM Standard Names Spreadsheet should include all of the CAM variables. 27 | - If a standard name is not found: 28 | - CCPP names are trying to adhere to CF names, so you may find names on the CF web page: https://cfconventions.org/Data/cf-standard-names/76/build/cf-standard-name-table.html 29 | - If you need to create your own standard name, the current proposal for creating standard names can be found here: https://github.com/ESCOMP/CCPPStandardNames/blob/main/StandardNamesRules.rst 30 | - May need to consult with WRF / MPAS scientists and other SIMA groups to coordinate StandardName usage. 31 | - Keep a list of StandardNames that you introduce and give them to a CAM SE for incorporation into an upcoming CCPPStandardNames PR 32 | - If you create your own name (for now), add `_TBD` to the end fo the name to signify that it has not been discussed yet 33 | !!! Warning "Updated standard names" 34 | If, in the course of your implementation or during code review to the CCPPStandardNames repository, the standard name changes, please update the name in the spreadsheet to reflect the change! 35 | 36 | 37 | - *units*: the units can be found in the standard names spreadsheet as well (`Snapshot Units` column) 38 | - You can find additional rules for units in the [ESMStandardNames repository](https://github.com/ESCOMP/ESMStandardNames/blob/main/StandardNamesRules.rst#units) 39 | - *dimensions*: Refer to the table below 40 | - If you have module level variables (variables above the "contains" line in a module file) which are declared with dimension `pver` or `pverp`, then you need to make this variable be allocatable and allocate the variable in the init routine. This is due to the fact that pver is not known at compilation time, but is rather a run-time option in CAM-SIMA 41 | - Note for pcols inside parameterizations: As only active columns will be passed to the parameterizations, there is no need for a maximum size variable anymore 42 | - In the template, you can convert `enter_standard_nameXX:enter_standard_nameYY` to just use the appropriate dimension_name or do `1:dimension_name` (both work) 43 | 44 |
45 | | CAM dimension | CCPP phase | standard_name | 46 | |:--------------|------------|---------------| 47 | | pver | all | vertical_layer_dimension | 48 | | pverp (or pver+1) | all | vertical_interface_dimension | 49 | | ncols or pcols| non-run phase | horizontal_dimension | 50 | | ncols or pcols| run phase | horizonal_loop_extent | 51 | | pcnst | all | number_of_ccpp_constituents | 52 |
53 | 54 | - *optional arguments*: If there are optional arguments, consult with a CAM SE. Optional attributes are an unsupported configuration, but are being incorporated into the framework. That said, there may be a workaround to avoid the use of optional arguments in your scheme 55 | - Note: Try to avoid module level allocations and/or assignments even in the init phases of the code. If you believe you need to do this, speak with a Jesse and/or Cheryl for guidance 56 | 57 | Once you have created your metadata, proceed to [3 - Create namelist XML file](create-namelist-xml.md) 58 | 59 | ### Error variables 60 | See below for error variable metadata 61 | ``` 62 | [ errmsg ] 63 | standard_name = ccpp_error_message 64 | long_name = Error message for error handling in CCPP 65 | units = none 66 | type = character | kind = len=512 67 | dimensions = () 68 | intent = out 69 | [ errflg ] 70 | standard_name = ccpp_error_code 71 | long_name = Error flag for error handling in CCPP 72 | units = 1 73 | type = integer 74 | dimensions = () 75 | intent = out 76 | ``` 77 | ### Constituents 78 | See [constituent usage](../design/constituents.md/#constituent-usage). You can tell a variable is a constituent if it is in the `state%q` array in CAM (may have to trace back in the code a bit) 79 | 80 | ### Dependencies 81 | If your scheme has any "use" statements (helper functions, approved dependencies), add a `dependencies` field with a common-separated list of relative paths to the necessary modules to the top of your metadata file. Example: 82 | ``` 83 | [ccpp-table-properties] 84 | name = musica_ccpp 85 | type = scheme 86 | dependencies = micm/musica_ccpp_micm.F90,musica_ccpp_util.F90 87 | ``` 88 | -------------------------------------------------------------------------------- /docs/design/sima-design-goals.md: -------------------------------------------------------------------------------- 1 | # Design goals & features 2 | 3 | Motivated by the Singletrack project (a precursor to [SIMA](https://sima.ucar.edu/about)), the CAM-SIMA project was created to build a new CAM infrastructure to meet the SIMA science and computational needs. This page documents those needs and some of the features that implement them. 4 | 5 | ## CAM needs to be more run-time configurable 6 | 7 | - To make CAM and CESM more available, e.g., for usage in containers and the cloud, a build of CAM should be more configurable than it is at present. 8 | - One feature that makes CAM more run-time configurable is moving physics suites to the [CCPP](https://dtcenter.org/community-code/common-community-physics-package-ccpp). By allowing CAM to compile in more than one physics suite, the physics suite can be selected at run time (e.g., as a namelist variable). 9 | - Another feature needed to make CAM more run-time configurable is making dycores themselves more run-time configurable. For instance, the SE dycore will no longer require the number of advected constituents to be specified at compile time. 10 | 11 | ## Remove obstacles to use of specialized high-performance processing units (e.g., GPUs, FPGAs) 12 | 13 | The chunk structure in CAM physics served its purpose when threading was the only way to accelerate code. However, to make the best use of both threading and modern accelerators, a flexible chunking mechanism is required. The new infrastructure enables this by using flat arrays for all fields. 14 | 15 | - Moving to flexible precision of data is important for being able to test both performance improvements and the affect on model quality. **The CCPP is explicitly designed to allow for compile-time selection of precision at the physics suite level as well as for individual fields.** In addition, the new infrastructure is explicitly designed to handle the case where the dycore is running at a different precision than the physcs (i.e., by using proper field promotion and demotion primitives). 16 | - Pointers in Fortran are less efficient because they prevent some optimization techniques. The new infrastructure avoids pointers as much as possible by making use of the automatic data management capability of the CCPP (which does not create pointers). 17 | - The new infrastructure provides greater flexibility in that the model can be built with multiple physics suites to increase run-time flexibility. There is a tradeoff in that building more physics suites will often increase build time. Builds with a single suite should be faster than now since only the schemes that are required for the suite are compiled (currently most schemes are compiled all the time even if they will not be used). 18 | 19 | ## Modularity 20 | In order to continue to allow CAM to grow without an ever increasing cost of bringing in new features, CAM must be more modular. A classic example is chemistry which ACOM would like to make modular but which is currently entwined with CAM in many areas (e.g., code in CAM repository, extensive configuration code dedicated to chemistry, extensive namelist building code and data dedicated to chemistry, large number of use cases containing chemistry configuration data). The new CAM infrastructure contains features to increase modularity. 21 | 22 | - Support for multiple namelists. This allows modular components to contain their own namelist (run-time configuration options). The active namelists are combined to produce atm_in. 23 | - Flexible handling of constituent information. Modular components can provide constituent information via metadata (if component is a CCPP scheme) or at run time. 24 | 25 | Modularity will allow CAM to move components to external repositories. This process cuts development costs for both CAM and for the component (e.g., as has happened with PUMAS). Some ways this is accomplished are listed here: 26 | 27 | - Code reviews are more efficient since CAM SEs do not have to review every routine in the external module so they can just focus on the interfaces. The external developers do not have to be involved in CAM modifications. 28 | - Externals can develop and maintain they own namelist definition files, they do not have to coordinate with the larger CAM namelist (which itself has been broken into several smaller namelists). 29 | - Namelists associated with physics schemes do not have to have separate namelist-reading code. The new infrastructure automatically creates an appropriate Fortran module to read in the runtime data from atm_in. The system then also ensures that all active namelists are called at run time. This process ensures that namelists are always read correctly while not requiring coding or reviews to keep up to date with namelist changes. 30 | 31 | Use of the CCPP to build physics suites also makes CAM more modular because the CCPP treats physics schemes as modular which allows flexibility in building physics suites. The CCPP takes care of making sure variables are available before they are used and also builds the code currently handled via hand-written code in the various versions of physpkg.F90. 32 | 33 | ## Run-time data checking 34 | CAM needs data to run but the data needs vary with the simulation. The new infrastructure facilitates this. 35 | 36 | - Before running physics, the new infrastructures queries the physics suite as to what input fields are required (using a CCPP interface). Then it makes sure that all of these fields have been initialized or reads the values from the initial data file. Any uninitialized fields that are not found on the initial data file will trigger a run-time error. 37 | 38 | ## Efficient offline testing and simulation 39 | 40 | CAM currently has a few ways to run offline testing (e.g., SCAM, PORT). The new infrastructure builds these capabilities in for more efficient and flexible use. 41 | 42 | - The new infrastructure has the ability to run without a dycore. 43 | - Offline mode (NULL dycore) can be run with any number of columns. 44 | - Offline mode does not required gridded input. 45 | 46 | ## Software quality control 47 | To enable efficient quality control, the new infrastructure implements a number of continuous integration (CI) techniques. 48 | 49 | - To implement all the flexibility mentioned above, the new infrastructure makes extensive use of python scripts. 50 | - Python scripts make extensive use of python doctests (for simpler tests). 51 | - There are python unit tests to cover more complicated situations. 52 | - The GitHub CI also runs a static analysis tool (pylint) to provide feedback on potential coding issues. 53 | - Python tests can easily be run by hand but are also automatically run on GitHub 54 | - The new infrastructure will use offline mode (see above) to run many physics configurations quickly without requiring large machines. 55 | - This will enable quick testing during development on a laptop. 56 | - We hope many of these tests can also be run automatically on GitHub. -------------------------------------------------------------------------------- /docs/development/cam-coding-standards.md: -------------------------------------------------------------------------------- 1 | # Coding standards 2 | 3 | The standards in this document are largely prescriptive, i.e., they should be followed. 4 | 5 | - *MUST*: Exceptions must be discussed and agreed to by the CAM SEs and noted in this document. 6 | - *SHOULD*: Exceptions must be approved by the CAM SEs and documented in the ChangeLog. 7 | 8 | While some legacy code will not follow these rules, efforts SHOULD be made to improve the code whenever you are working on it (e.g., bug fixes, enhancements). 9 | 10 | ## General coding standards 11 | ### MUST 12 | - Always use spaces instead of tabs 13 | - No trailing spaces (i.e., no spaces at the end of a line) 14 | 15 | See [tips for configuring editors](#tips-for-configuring-editors) 16 | 17 | ### SHOULD 18 | 19 | - Use comments to explain the purpose of the following code and/or include any important but non-obvious information. When working with code, always check the comments to make sure they are still correct and useful. 20 | - Do not use comments to 'save code for later in case it might be useful'. 21 | - Do not include a comment that merely restates the following code logic (e.g., 'Loop over variables') 22 | 23 | ## Python coding standards 24 | We expect all python code to pass with a perfect score (10) using `pylint` with [this version](https://github.com/ESCOMP/CAM-SIMA/blob/development/test/.pylintrc) of `pylintrc`. However, external repos may have their own `pylintrc` version depending on their needs. 25 | 26 | We also expect all python files to follow the [black code style and format](https://black.readthedocs.io/en/stable/the_black_code_style/index.html). 27 | 28 | Finally, one can also follow the [Google Python](https://google.github.io/styleguide/pyguide.html) Style Guide. 29 | 30 | ## Fortran coding standards 31 | The standards described in this section represent the CAM Fortran standards. Other Fortran standards: 32 | 33 | - [CTSM Fortran Standards](https://wiki.ucar.edu/display/ccsm/Comprehensive+list+of+standards) 34 | - [MOM Fortran Standards](https://github.com/mom-ocean/MOM6/wiki/Code-style-guide) 35 | 36 | ### MUST 37 | * No naked `use` statements 38 | * No continued single-line `if` statements (i.e., all `if` statements should have a `then` if the statement is on more than one line) 39 | * Every namelist variable in each active namelist group is present in the namelist file. An active namelist group is one which may be read during the current run. 40 | * All namelist variables except for logical quantities are initialized to invalid values (integer: `-HUGE(1)`, real: `NaN`, character: `'UNSET'`). 41 | * Functions may _not_ have side effects, and should include the `pure` keyword. 42 | * Do not combine statements on a single line (i.e., avoid use of the semi-colon to combine statements). 43 | * Use `intent` for dummy arguments except for pointers. 44 | * All variables of type real must have a specified kind, including literals. For example, use `1.5_r8`, not `1.5` or `1.5D0`. Literals must also include the decimal point. 45 | * All character declarations must use Fortran 90+ syntax (e.g., `character(len=*)` or `character(len=CL)`). 46 | * All variable declarations must use Fortran 90+ syntax (i.e., must include the double colon between the attributes and the variable name). 47 | * All type and procedure declarations must use Fortran 90+ syntax (i.e., must include the double colon before the type or procedure name). 48 | * All modules should include an `implicit none` statement in the preamble (after the `use` statements). Module routines then do not need this statement. 49 | * All optional arguments must be passed via keyword (e.g. use `call subroutine(x, optional_y=y)` instead of `call subroutine(x, y)` for the optional variable `optional_y`). 50 | * Initialize local (non-parameter) variables in subroutines and functions at the top of the executable code, NOT on a variable declaration lines. 51 | - Initializing a local variable on a declaration line invokes the `SAVE` attribute and is not thread safe. 52 | - Local pointer variables MUST be initialized before other (non-initialization) statements. By default, use the `nullify` statement. 53 | * All variables that are on the physics grid must have their horizontal dimension declared with `pcols`, even if only a subset of the variable is used in the subroutine or function. 54 | ### SHOULD 55 | * Avoid use of preprocessor directives (e.g., `#if`, `#ifdef`). Always try for runtime variable logic instead. 56 | * Keep formula statements relatively short. Use temporary variables to break long formulas into easier-to-read sections. 57 | * Use subroutines to avoid repeated (cut and paste) code logic. 58 | * Avoid side effects in subroutines. Pass variables to routines instead of 'using' them from elsewhere. 59 | * Use the `pure` keyword if a subroutine has no side effects. 60 | * List dummy arguments one per line, however, related items may be grouped. 61 | * Dummy argument order should match the order in the argument list. 62 | * Use symbolic numerical comparison operators (e.g., `==`, `/=`, `<`, `>=`) not old character versions (e.g., `.eq.`). 63 | * Avoid the use of pointers as dummy arguments (exceptions must be discussed in design or code review) 64 | * Modules should be default `private`. Public interfaces are declared after the `private` declaration. 65 | * `private` module interfaces (i.e., subroutines and functions) should be declared private in the module header. 66 | * Module names should conform to their filename (i.e., the module name should be the filename without the `.F90`). 67 | * Functions _should_ use the `pure` attribute. If they cannot, the reason should be included as a comment in the function's preamble. 68 | * All functions and subroutines should avoid un-necessary statements (e.g. a blank `return` at the end of a subroutine). 69 | * `use` statements should be brought in at the smallest scope possible (e.g. inside individual subroutines instead of at the module level). 70 | 71 | ### Indentation and style 72 | 73 | * *Scoping*: Indentation should follow scope. That is, whenever entering a new scope (e.g., `module`, `subroutine`, `if`, `do`), indent that scope relative to the scoping statement (recommended 2 spaces but each module should at least be self consistent). 74 | * A single line should be less than 133 characters long. 75 | * *Continue lines*: Indent continue lines 4 spaces or align with similar lines in statement. 76 | * Use spaces to ease reading statements (e.g., before and after operators, after commas except in a dimensions list) 77 | * Include a space after `if`, `else`, `end`, `do`, and `while`. 78 | * Include a space before and after `::` 79 | * No space after `only`, i.e., `only:`, not `only :`. 80 | * When aligning code for readability, commas go immediately after a symbol (no space). 81 | 82 | ## Tips for configuring editors 83 | ### emacs (add to your `.emacs` file) 84 | - To automatically remove trailing spaces whenever you save a file: 85 | ``` 86 | (add-hook 'before-save-hook 'delete-trailing-whitespace) 87 | ``` 88 | - To automatically indent with spaces instead of tabs: 89 | ``` 90 | (setq-default indent-tabs-mode nil) 91 | ``` 92 | - To use 4 spaces for each indent: 93 | ``` 94 | (setq tab-width 4) 95 | ``` 96 | 97 | ### vi (add to your `.vimrc` file) 98 | - To automatically remove trailing spaces whenever you save a file: 99 | ``` 100 | autocmd BufWritePre * :%s/\s\+$//e 101 | ``` 102 | -------------------------------------------------------------------------------- /docs/conversion/run-cam-sima.md: -------------------------------------------------------------------------------- 1 | # 8 - Run CAM-SIMA 2 | 3 | It's time to try to run CAM-SIMA! 4 | 5 | ## Configure CAM-SIMA 6 | 7 | - Either navigate to the case you created when you were validating your metadata or [create a new case](../usage/creating-a-case.md) 8 | - Your `user_nl_cam` needs to contain the following (to bring in the cam_snapshot for input and to activate the validation tool): 9 | ``` 10 | ncdata='/FullPathName/your_before_snapshot_file.nc' 11 | ncdata_check='/FullPathName/your_after_snapshot_file.nc' 12 | debug_output = 0 !Turns off debug messages which interfere with the validation tool output 13 | ``` 14 | - If the number of vertical levels in your snapshot file is something other than 30, then you will also need to add the following to your user_nl_cam: 15 | 16 | ``` 17 | pver = N 18 | ``` 19 | 20 | where “N” is the number of vertical levels in your snapshot file (likely 32 levels for CAM6) 21 | 22 | ## Build and run 23 | 24 | - Run `./case.build` and `./case.submit` 25 | - Consult the table below for common errors. Also see [debugging tips](../development/debugging.md) for help if you're getting errors 26 | 27 | 28 | !!! Warning "Rebuilding CAM-SIMA" 29 | If you've modified any metadata between builds, you will need to run `./case.build --clean-all` before rebuilding. 30 | 31 | 32 | | Error text | Description | Possible Fix | 33 | |:------------|----------------|--------------| 34 | |`ERROR: cam_get_file: FAILED to get UNSET_PATH` | No initial data file specified (or defaulted to) | Specify `ncdata = ''` in your user_nl_cam file | 35 | |`ERROR: read_field_3d: No variable found in (/XXX, /)` | Variable not found when we're reading in the initial data or snapshot file |
  • Double-check your standard name
  • Determine if CAM-SIMA needs to handle/initialize the variable. If it does, follow the procedure for the error below
| 36 | |`ERROR: Cannot read XXX from file, XXX has no horizontal dimension` | Variable is a scalar; we don't have an interface to read it in from a file (and it's not likely to be present on the file) |
  • If "XXX" is static during the run, add `access="protected"` to the variable's XML entry in the registry
  • if the variable is not static, either:
    • Initialize the variable in the init phase of your scheme (`intent=out`), or
    • initialize "XXX" in CAM-SIMA (NOT in the physics code) and add a call to `mark_as_initialized()` so CAM-SIMA doesn't try to read in the variable
| 37 | |`ERROR in '': corrupted size vs. prev_size: 0xXXXXXXXXXXXXXX` | Possibly an out of bounds error |
  • Find the line that the stack trace is pointing to (likely in the generated code) and trace the variable in question, looking for out-of-bounds errors
    • Specifically, check that the variable has the **correct dimensions** in the metadata (e.g. vertical_layer_dimension vs vertical_interface_dimension)
  • Also, check the variables immediately before and after the line for out-of-bounds errors (we have seen this error reported on the line immediately before the problem line).
  • Try running in DEBUG mode if not already to include bounds checking
| 38 | 39 | - Once the model completes without error, the results from the validation tool will appear in the `atm_log` file for each timestep (under the header ` ********** Physics Check Data Results **********`) 40 | - The message `No differences found!` should be logged for each timestep. 41 | - If there are differences found, that indicates there are issues with your ported code 42 | - See the section on [Tips for uncovering unexpected answer changes](#tips-for-uncovering-unexpected-answer-changes) for debugging strategies 43 | 44 | !!! Note "Additional step for NAG compiler" 45 | If we have a working NAG compiler with CAM-SIMA (not working as of 12/17/2024), perform this extra step: 46 | 47 | - Make at least one run using the NAG compiler on izumi in debug mode and specifying “-nan” as an additional compilation option. This option initializes all variables to NaN so that if something is uninitialized and is used, it will be trapped. To enable this flag: 48 | - After executing the ./create_newcase and ./case.setup commands and BEFORE executing the ./case.build, edit the file in your case directory `cmake_macros/nag.cmake` (- If you want to change the file so that it runs every time you execute a `./create_newcase`, you can find the file at `ccs_config/machines/cmake_macros/nag.cmake`). 49 | - Change the line Append line so that it contains ‘-nan’ and it should look like this: 50 | ``` 51 | if (DEBUG) 52 | string(APPEND FFLAGS " -nan -C=all -g -time -f2003 -ieee=stop") 53 | endif() 54 | ``` 55 | 56 | 57 | 58 | Once you have successfully run CAM-SIMA and all answer changes have either accepted or fixed, proceed to [9 - Bring back into CAM](back-to-cam.md) 59 | 60 | ## Tips for uncovering unexpected answer changes 61 | 62 | - Make sure that your snapshot is taken with an updated atmospheric_physics, especially if answers changed during development 63 | - Check all of your standard_names. CAM-SIMA will run successfully, but if a standard_name is incorrect, you may be running with an incorrect variation of the variable that you intended 64 | - If comparing printed values on the physics grid between CAM and CAM-SIMA, then it might be good to turn-off chunking in CAM, which can be done by setting `-pcols` in `CAM_CONFIG_OPTS` to be a large number (e.g. >500 for the ne3 grid): `./xmlchange CAM_CONFIG_OPTS="-pcols 1352" --append` (use the `--append` option to not lose existing options.) 65 | - You can also utilize the `tools/find_max_nonzero_index.F90` tool to (more) easily compare values between CAM and CAM-SIMA at a specific rank and index that is known to be non-zero 66 | - Note that the way latitude in radians is calculated for the null dycore in CAM-SIMA is different (at a round-off level) to the way it is calculated in the SE dycore in CAM. This means if you are validating a physics scheme that has latitude (in radians) as an input, then in CAM you’ll need to change the scheme interface to read in latitude in degrees, and then convert to radians by multiplying by pi/180 (which you’ll only want to do when creating the snapshot file, not when validating CAM itself). 67 | - Sometimes CAM-SIMA will set a state variable directly, while CAM will only set a tendency variable and then use the “physics_update” routine to update the state using the tendency. This can result in round-off level differences. If this happens then simply try setting the state variable directly in CAM when creating the snapshot files for CAM-SIMA testing. 68 | - Namelist settings may be incompatible with what is being run in CAM-SIMA with it being a single physics package versus the complete CAM code. For instance, a normal CAM run may have a default namelist setting and then a different value for "clubb_sgs=1 microphys=mg3" which is what a CAM7 run would be. To temporarily work around this, set the values which are evaluating incorrectly in your `user_nl_cam` file for CAM-SIMA to the values which match a CAM run. The easy way to find these is to compare your `atm_in` values with your CAM and CAM-SIMA runs and add the appropriate settings. 69 | - Compare numbers between a run in CAM and another in CAM-SIMA. A debug run with the `--pecount 1` argument provided to create_newcase gives you an executable which works well in the [Totalview debugger](../development/debugging.md/#totalview). 70 | 71 | !!! Warning "timestep discrepancy" 72 | The cam_snapshot file will be written starting with the second timestep, so your **CAM** run will need to advance past the first timestep before you can start comparing. 73 | 74 | - If you find that you have to comment out pieces of code in CAM to get your snapshot tests to work (not ideal, but has happened a few times!), be sure to commit your source mods to a new subdirectory in cam_snapshot_src_mods in the [CAM-SIMA backups](https://github.com/NCAR/CAM-SIMA_backups) repository. 75 | - Make sure to revert those changes before running your final CAM regression tests!!! 76 | 77 | 78 | - If you run into a "tip" that should be added here, please confer with the other CAM SEs to get it added to the documentation!! 79 | -------------------------------------------------------------------------------- /docs/conversion/interstitials.md: -------------------------------------------------------------------------------- 1 | # 4 - Interstitials 2 | ## Overview 3 | An **interstitial** is a scheme that does calculations or variable modifications to connect what the host model has to what the scheme requires and vice versa. 4 | 5 | - These schemes are placed before or after the core parameterization scheme in the suite definition file (SDF) to prep for the scheme or take what the scheme produces and translate it back to what the host model (CAM-SIMA) expects 6 | - In CAM, interstitial code will appear as either function calls or calculations/modifications in the CAM interface just before or just after the call to the parameterization 7 | - If the interstitial code is a function call, that function/module should be CCPP-ized (if not already) and that scheme will be included in the SDF 8 | - It the interstitial code is loose in the CAM interface, it is recommended that, when possible, that code be moved into the scheme (either the beginning or end of the scheme subroutine) 9 | - UNLESS: it's a common (across multiple parameterizations) calculation/translation/modification. In which case, a new scheme should be created to do that calculation (scheme will live in `$CAM-SIMA/src/physics/ncar_ccpp/schemes/utilities`) 10 | - If it is absolutely necessary to create a scheme-specific interstitial, that scheme should be called `_pre` or `_post` (depending on where in the SDF it will be placed) and will live in the `ncar_ccpp/schemes/` directory 11 | 12 | This section covers a few interstitial scenarios you are likely to face. 13 | 14 | ## Diagnostics interstitial 15 | All `addfld`/`outfld` calls will go in a `_diagnostic.F90` interstitial. You can find a template and instructions for scheme-specific diagnostics [here](https://github.com/ESCOMP/atmospheric_physics/blob/main/schemes/sima_diagnostics/scheme_diagnostics_template.F90): 16 | 17 | - `addfld` calls will go into the `_diagnostic_init` subroutine 18 | - `outfld` calls will go into the `_diagnostic_run` subroutine 19 | - The interstitial itself will reside after the `` line in the SDF 20 | - See: [History Usage](../usage/history.md/#adding-a-diagnostic-field-to-the-cam-sima-source-code) for the specifications of the CAM-SIMA-versions of `addfld` and `outfld` calls 21 | 22 | ## Utilities 23 | As mentioned, there are some calculations/conversions/translations that are performed often throughout the physics code in CAM. These schemes are available for use in the `$CAM-SIMA/src/physics/ncar_ccpp/schemes/utilities` directory and include: 24 | 25 | - **state_converters.F90**: contains common conversions/calculations of [state variables](../design/ccpp-in-cam-sima.md/#state-and-tendency-variables), including these schemes: 26 | 27 | | Scheme name | Description | Output variable | Input variables | 28 | |:------------|-------------|-----------------|-----------------| 29 | | temp_to_potential_temp | convert temperature to potential temperature | air_potential_temperature | air_temperature
inverse_exner_function | 30 | | potential_temp_to_temp | convert potential temperature to temperature | air_temperature |air_potential_temperature
inverse_exner_function | 31 | | calc_dry_air_ideal_gas_density | Calculate density from equation of state/ideal gas law | dry_air_density |composition_dependent_gas_constant_of_dry_air
air_pressure_of_dry_air
air_temperature | 32 | | calc_exner | calculate dimensionless exner function | dimensionless_exner_function |composition_dependent_specific_heat_of_dry_air_at_constant_pressure
composition_dependent_gas_constant_of_dry_air
surface_reference_pressure
air_pressure | 33 | | wet_to_dry_water_vapor | convert water vapor from wet to dry mixing ratio | water_vapor_mixing_ratio_wrt_dry_air |air_pressure_thickness
air_pressure_thickness_of_dry_air
water_vapor_mixing_ratio_wrt_moist_air_and_condensed_water | 34 | | dry_to_wet_water_vapor | convert water vapor from dry to wet mixing ratio | water_vapor_mixing_ratio_wrt_moist_air_and_condensed_water | air_pressure_thickness
air_pressure_thickness_of_dry_air
water_vapor_mixing_ratio_wrt_dry_air | 35 | | wet_to_dry_cloud_liquid_water | convert cloud liquid from wet to dry mixing ratio | cloud_liquid_water_mixing_ratio_wrt_dry_air | air_pressure_thickness
air_pressure_thickness_of_dry_air
cloud_liquid_water_mixing_ratio_wrt_moist_air_and_condensed_water | 36 | | dry_to_wet_cloud_liquid_water | convert cloud liquid from dry to wet mixing ratio | cloud_liquid_water_mixing_ratio_wrt_moist_air_and_condensed_water | air_pressure_thickness
air_pressure_thickness_of_dry_air
cloud_liquid_water_mixing_ratio_wrt_dry_air | 37 | | wet_to_dry_rain | convert rain from wet to dry mixing ratio | rain_mixing_ratio_wrt_dry_air | air_pressure_thickness
air_pressure_thickness_of_dry_air
rain_mixing_ratio_wrt_moist_air_and_condensed_water | 38 | | dry_to_wet_rain | convert rain from dry to wet mixing ratio | rain_mixing_ratio_wrt_moist_air_and_condensed_water |air_pressure_thickness
air_pressure_thickness_of_dry_air
rain_mixing_ratio_wrt_dry_air | 39 | 40 | - **geopotential_temp.F90**: 41 | - **geopotential_temp**: compute geopotential height (`geopotential_height_wrt_surface`) and geopotential height at interfaces (`geopotential_height_wrt_surface_at_interface`) from temperature (`air_temperature`) 42 | - **static_energy.F90** 43 | - **update_dry_static_energy**: calculate dry static energy (`dry_static_energy`) 44 | - **qneg.F90**: 45 | - **qneg**: Set values for constituent variables that are less than the minimum value to the minimum value (and print out what it's doing - configurable!) 46 | - You will want to include qneg in your SDF if you are modifying constituent tendencies in your scheme. It should be after the tendencies are applied and before `geopotential_temp` 47 | - If `qneg` is in your SDF, you will need to provide the output variable `scheme_name` in your physics parameterization that is modifying constituent tendencies 48 | 49 | !!! Warning "scheme_name variable" 50 | If you skip this step, you will get either `parse_source.CCPPError: Input argument for qneg_run, scheme_name, not found`, or an incorrect scheme_name from a previous routine will be used 51 | 52 | - **physics_tendency_updaters.F90**: apply tendencies output by physics to state variables. You'll need to include a tendency updater in your SDF for any `ptend%X` variables in the CAM-version of your code. 53 | 54 | | Scheme name | Description | inout variables | onput variables | 55 | |:------------|-------------|-----------------|-----------------| 56 | | apply_tendency_of_
eastward_wind | Apply the eastward wind tendency calculated in the previous physics scheme(s) to the `eastward_wind` state variable | eastward_wind
tendency_of_eastward_wind | tendency_of_eastward_wind_
  due_to_model_physics
timestep_for_physics | 57 | | apply_tendency_of_
northward_wind| Apply the northward wind tendency calculated in the previous physics scheme(s) to the `northward_wind` state variable | northward_wind
tendency_of_northward_wind | tendency_of_northward_wind_
  due_to_model_physics
timestep_for_physics | 58 | | apply_heating_rate | Apply the heating rate (`tendency_of_dry_air_enthalpy_at_constant_pressure`) to the temperature tendency and temperature state variable | air_temperature
tendency_of_air_temperature_
  due_to_model_physics
heating_rate | tendency_of_dry_air_enthalpy_
  at_constant_pressure
composition_dependent_specific_heat_
  of_dry_air_at_constant_pressure
timestep_for_physics | 59 | | apply_tendency_of_
air_temperature | Apply the temperature tendency calculated in the previous physics scheme(s) to the `air_temperature` state variable | air_temperature
tendency_of_air_temperature | tendency_of_air_temperature_
  due_to_model_physics
timestep_for_physics | 60 | | apply_constituent_
tendencies | Apply the constituent tendencies calculated in the previous physics scheme(s) to the constituent state array | ccpp_constituents
ccpp_constituent_tendencies | timestep_for_physics | 61 | 62 | Proceed to [5 - Create an SDF](create-sdf.md). 63 | 64 | !!! Warning 65 | You may have to revisit this step as you debug your code. -------------------------------------------------------------------------------- /docs/development/debugging.md: -------------------------------------------------------------------------------- 1 | # Debugging techniques 2 | Start with the [CAM wiki](https://github.com/ESCOMP/CAM/wiki/CAM-debugging-techniques). 3 | 4 | ## CAM-SIMA-specific debugging 5 | ### Build errors 6 | Debugging tips if you get build errors: 7 | 8 | - If the output indicates that the error message or failure is coming from somewhere within $CAM-SIMA/ccpp_framework: 9 | - If you're getting a clear error message, it's likely that you have something wrong with your metadata 10 | - If you're getting an error message that indicates that something is breaking in the framework code itself (something went uncaught) - consult the AMP SEs 11 | - If the error happens during the atm build, you can see the full output of the atm build in the build log here: `bld/atm.bldlog.*` 12 | 13 | ### Run-time errors 14 | 15 | - Start with the atm.log* - if the issue occurred during the execution of the CAM code, it will hopefully have a clear and concise error message 16 | - Move to the cesm.log* - it will hopefully include a stack trace for the error in question; if the error did not occur in the CAM code (or CAM did not properly trap the error), it will help you identify the source of the issue. 17 | - If neither log file contains helpful information, a few first steps: 18 | - Resubmit the case; it could be a machine hiccup 19 | - Turn on DEBUG mode (if it's not on already) and rebuild/rerun 20 | - Look in your run directory for any log files called `PETXXX` - if there was an issue on the [ESMF](https://earthsystemmodeling.org/docs/release/latest/ESMF_usrdoc/) side of things, it will show up in one of these (there will be one PET file per processor) 21 | - Try a different compiler - maybe it'll give you a more helpful error message 22 | - set NTASKS=1 (`./xmlchange NTASKS=1`), do a clean rebuild (as instructed), and run again; maybe running in serial will identify the error 23 | - Look for the `***************** HISTORY FIELD LIST ******************` in the atm.log* file; if it's not there, the error occurred at init time 24 | - If the error occurred during init time, try a new case with a different grid and/or dycore 25 | - If the model ran for a few timesteps before dying (look for the `CAM-SIMA time step advanced` message in the atm.log* file), it's likely that one or more variable that you introduced or modified has gone off the rails (value has become very large or very small or zero) 26 | - Update your user_nl_cam to output all possible suspected variables to a history file at some point shortly before the model dies, then inspect the output to see if any are obviously wrong 27 | - If the model completed all timesteps, try running a shorter case to see if the problem persists; if so, it's an error during the model finalization 28 | - Run the [TotalView](#totalview) debugger on izumi 29 | - Use the old standard - print statements - to narrow down where the code is stopping 30 | - Ask for help! 31 | 32 | ### Unexpected answer changes 33 | 34 | - Two paths here: 35 | - You're getting unexpected DIFFs from the regression testing 36 | - Consult with a scientist about whether differences are expected and for which configurations (compsets, resolutions, namelists parameters, etc) 37 | - If the differences are very small (look like round-off), consult with the other AMP SEs on whether we're ok with this 38 | - If the differences are indeed unexpected and larger than round-off, create a case using the code from the head of `development` and: 39 | - place print statements in both code bases (your development branch and the head of `development`) to identify where the numbers are going awry OR 40 | - run the [TotalView](#totalview) debugger OR 41 | - use the comparison tool described below (`$CAM-SIMA/tools/find_max_nonzero_index.F90`) 42 | - You're getting unexpected answer changes compared with CAM 43 | - Consult with other AMP SEs about whether the differences appear to be due to round-off error 44 | - Use the comparison tool (LINK ONCE IT EXISTS): `$CAM-SIMA/tools/find_max_nonzero_index.F90` 45 | - This tool can help you narrow down where the issue begins by printing out values at a specific index and comparing those with the "truth" (from CAM) 46 | 47 | ### TotalView 48 | 49 | - Grab an interactive node. You can do this by copying the following commands into a .csh script: 50 | 51 | ``` 52 | #! /bin/csh -f 53 | #PBS -q long 54 | # Number of nodes (CHANGE THIS if needed) 55 | # #PBS -l walltime=6:00:00,nodes=1:ppn=16 56 | # # output file base name 57 | # #PBS -N test_dr 58 | # # Put standard error and standard out in same file 59 | # #PBS -j oe 60 | # # Export all Environment variables 61 | # #PBS -V 62 | ``` 63 | 64 | then run: 65 | 66 | ``` 67 | qsub -X -I