├── assets └── ape15_example_affilpkg_page.png ├── LICENSE.rst ├── APEtemplate.rst ├── APE18.rst ├── APE9.rst ├── APE21.rst ├── APE8.rst ├── APE12.rst ├── APE2.rst ├── README.rst ├── APE19.rst ├── APE15.rst ├── APE10.rst ├── APE1.rst ├── APE5.rst ├── APE20.rst ├── APE3.rst ├── APE7.rst └── APE0.rst /assets/ape15_example_affilpkg_page.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/astropy/astropy-APEs/HEAD/assets/ape15_example_affilpkg_page.png -------------------------------------------------------------------------------- /LICENSE.rst: -------------------------------------------------------------------------------- 1 | ======= 2 | License 3 | ======= 4 | 5 | The content of this repository is licensed under the `Creative Commons Attribution 4.0 International (CC BY 4.0) license `_. 6 | 7 | -------------------------------------------------------------------------------- /APEtemplate.rst: -------------------------------------------------------------------------------- 1 | 2 | ---------------- 3 | 4 | author: 5 | 6 | date-created: 2013 November 5 7 | 8 | date-last-revised: 2013 December 17 9 | 10 | date-accepted: 2014 January 20 11 | 12 | type: 13 | 14 | status: Discussion 15 | 16 | 17 | Abstract 18 | -------- 19 | 20 | The abstract should be a short description of what the APE will achieve. 21 | 22 | 23 | Detailed description 24 | -------------------- 25 | 26 | This section describes the need for the APE. It should describe the existing 27 | problem that it is trying to solve and why this APE makes the situation better. 28 | It should include examples of how the new functionality would be used and 29 | perhaps some use cases. 30 | 31 | 32 | Branches and pull requests 33 | -------------------------- 34 | 35 | Any pull requests or development branches containing work on this APE before acceptance should be 36 | linked to from here. (An APE does not need to be implemented in a single pull 37 | request if it makes sense to implement it in discrete phases). If no code is yet 38 | implemented, just put "N/A" 39 | 40 | 41 | Implementation 42 | -------------- 43 | 44 | This section lists the major steps required to implement the APE. Where 45 | possible, it should be noted where one step is dependent on another, and which 46 | steps may be optionally omitted. Where it makes sense, each step should 47 | include a link related pull requests as the implementation progresses. 48 | 49 | 50 | Backward compatibility 51 | ---------------------- 52 | 53 | This section describes the ways in which the APE breaks backward compatibility. 54 | 55 | 56 | Alternatives 57 | ------------ 58 | 59 | If there were any alternative solutions to solving the same problem, they should 60 | be discussed here, along with a justification for the chosen approach. 61 | 62 | 63 | Decision rationale 64 | ------------------ 65 | 66 | 67 | -------------------------------------------------------------------------------- /APE18.rst: -------------------------------------------------------------------------------- 1 | Adopt NEP 29 for CPython and Numpy Version Support 2 | -------------------------------------------------- 3 | 4 | author: Stuart Mumford and Thomas Robitaille 5 | 6 | date-created: 2020 October 16 7 | 8 | date-last-revised: 2024 July 12 9 | 10 | type: Standard Track 11 | 12 | status: Accepted 13 | 14 | .. note:: 15 | NEP 29 has been superseded by the Scientific Python ecosystem coordination guideline 16 | `SPEC 0 — Minimum Supported Versions `__. 17 | All mentions of NEP 29 apply to SPEC 0 likewise and the latter should be used in all future 18 | references. 19 | 20 | 21 | Abstract 22 | -------- 23 | 24 | APE 10 previously determined supported CPython versions, and provided a list up to the 4.0 release. 25 | Since APE 10 was written the wider scientific Python community have written and largely adopted `NEP 29`_ as a policy for supported versions of both CPython and Numpy. 26 | This APE proposes that Astropy adopt this policy following other projects in the scientific ecosystem. 27 | 28 | 29 | Detailed description 30 | -------------------- 31 | 32 | This APE fully adopts `NEP 29`_ for both CPython and Numpy versions for the core astropy package, and it supersedes APE 10 wherever there is a conflict between that APE and this APE or `NEP 29`_. 33 | 34 | This APE will not replicate the content of `NEP 29`_, and the full text of `NEP 29`_ is adopted via this APE. 35 | In addition to the content of `NEP 29`_, the APE also extends the numpy policy to all of the core package dependencies. 36 | This leads to the following support policy for dependencies of the core package. 37 | 38 | Astropy core will support: 39 | 40 | * All minor versions of Python released 42 months prior to a non-bugfix, and at minimum the two latest minor versions. 41 | * All minor versions of numpy released in the 24 months prior to a non-bugfix, and at minimum the last three minor versions. 42 | * Versions of other runtime dependencies released 24 months prior to a non-bugfix release. 43 | 44 | These versions of the dependencies are to be supported for bugfix lifetime of the given astropy version. 45 | 46 | This version support policy allows, at the discretion of the maintainers, to adopt the supported version of the next release in situations where it is judged to be valuable. 47 | For example, this could be done in the case of an important change in numpy or a change over date very close to the release. 48 | 49 | 50 | Implementation 51 | -------------- 52 | 53 | Some initial work has been done in `#10900 `__ on bumping the minimum Python version in the core astropy package to Python 3.7+. 54 | 55 | The release calendar for the astropy core package will be updated over time to include the minimum Python and Numpy version for upcoming releases. 56 | 57 | Backward compatibility 58 | ---------------------- 59 | 60 | The version schedule here is more rapid than that proposed in APE 10, however, this allows the core library to adopt modern features, and reduces burden on maintainers. 61 | In addition to this, aligning ourselves with other projects in the ecosystem is consistent and prevents potential minimum version conflicts. 62 | See the text of `NEP 29`_ for further justification. 63 | 64 | This APE fully replaces APE10 where they are in conflict. 65 | 66 | Alternatives 67 | ------------ 68 | 69 | See `NEP 29`_. 70 | 71 | Decision rationale 72 | ------------------ 73 | 74 | After discussion and some revision in the PR, there was broad support and no 75 | objections to adopting this plan. 76 | 77 | .. _NEP 29: https://numpy.org/neps/nep-0029-deprecation_policy.html 78 | -------------------------------------------------------------------------------- /APE9.rst: -------------------------------------------------------------------------------- 1 | A General Timeseries Class for Astropy 2 | ------------------------------------- 3 | 4 | author: Stuart Mumford 5 | 6 | date-created: 2016 March 24 7 | 8 | date-last-revised: 2023 Oct 20 9 | 10 | type: Standard Track 11 | 12 | status: Accepted 13 | 14 | 15 | Abstract 16 | -------- 17 | 18 | The goal of a timeseries object in astropy is to provide a core set of 19 | functionality for representing data where time is the physical event list. This 20 | includes both continuous values binned in time or event list type data. 21 | 22 | The motivation for creation of this class is to support representation and 23 | analysis of astrophysical timeseries data, including support for the Units, Time 24 | and Coordinates packages in Astropy. This object should also be compatible with 25 | other packages in the wider scientific Python ecosystem, and try to minimise 26 | overlap in functionality with these packages. 27 | 28 | This document proposes an implementation based on Table, with extra constraints 29 | and specific functionality dedicated to timeseries type data. 30 | 31 | Detailed description 32 | -------------------- 33 | 34 | Many different areas of astrophysics have to deal with 1D timeseries data, i.e. 35 | either sampling a continuous variable at fixed times or counting some events 36 | binned into time windows. These types of applications require some basic 37 | functionality: 38 | 39 | #. Extending timeseries with extra rows 40 | #. Concatenating multiple timeseries objects (e.g. adding new columns) 41 | #. Sorting 42 | #. Slicing / selecting time ranges (indexing) 43 | #. Re-binning and re-sampling timeseries 44 | #. Interpolating to different time stamps. 45 | #. Masking and support for missing values (NaN) 46 | #. Support for subtraction and addition (e.g. background). 47 | #. Plotting and visualisation. 48 | #. Have high performance into \~millions of rows. 49 | 50 | While this functionality is found in non-domain specific packages such as 51 | pandas, a timeseries object in Astropy would also provide some functionality 52 | which is more related to Astronomy: 53 | 54 | #. Converting between time systems 55 | #. Astropy unit support 56 | #. Support variable width time bins. 57 | 58 | It is proposed that this functionality be implemented via a subclass of the 59 | ``QTable`` class, with a few requirements specific to the timeseries class: 60 | 61 | #. A 'Time' index column exists which enforces unique indexes. 62 | #. The table is always sorted in terms of increasing time. 63 | 64 | 65 | Binned Data vs Sampled Data 66 | ########################### 67 | 68 | There are two different types of data that a timeseries class or classes would 69 | have to support, sampled data and binned data. Sampled data is taken at a 70 | precise time stamp whereas binned data is a number of counts contained within a 71 | some time window. These two different types of timeseries data require different 72 | handling of methods such as re-binning or re-sampling, as well as for binned 73 | data the width of the bins must be stored. 74 | 75 | 76 | To distinguish between binned and sampled timseries we propose having two ways 77 | of constructing a timeseries, ``BinnedTimeSeries`` and ``SampledTimeSeries``. 78 | This distinction in the name of the classes will allow a easier to understand 79 | API. Some examples of constructing these objects is given below. 80 | 81 | 82 | Initialize a ``SampledTimeSeries`` with a time and a data column:: 83 | 84 | ts = SampledTimeSeries(time=['2016-03-22T12:30:31', '2016-03-22T12:30:32', '2016-03-22T12:30:40'], 85 | data=[1, 4, 3]) 86 | 87 | Initialize a ``SampledTimeSeries`` with a time and a fixed delta:: 88 | 89 | ts = SampledTimeSeries(time='2016-03-22T12:30:31', time_delta=3 * u.s, n_samples=10) 90 | 91 | Initialize a ``SampledTimeSeries`` with a time and an array of deltas (also no data):: 92 | 93 | ts = SampledTimeSeries(time="2011-01-01T00:00:00", time_delta=np.random.random((10,))*u.s) 94 | 95 | Initialize a ``BinnedTimeSeries`` with even contiguous bins by specifying the bin width:: 96 | 97 | ts = BinnedTimeSeries(start_time='2016-03-22T12:30:31', bin_size=3 * u.s, data=[1, 4, 3]) 98 | 99 | Initialize a ``BinnedTimeSeries`` with uneven non-contiguous bins with lists of start times, bin sizes and data:: 100 | 101 | ts = BinnedTimeSeries(start_time=['2016-03-22T12:30:31', '2016-03-22T12:30:38', '2016-03-22T12:34:40'], 102 | bin_sizes=[5, 100, 2]*u.s, 103 | data=[1, 4, 3]) 104 | 105 | Initialize a ``BinnedTimeSeries`` with uneven contiguous bins by giving an end time:: 106 | 107 | ts = BinnedTimeSeries(start_time=['2016-03-22T12:30:31', '2016-03-22T12:30:32', '2016-03-22T12:30:40'], 108 | end_time='2016-03-22T12:30:55', 109 | data=[1, 4, 3]) 110 | 111 | Initialize a ``BinnedTimeSeries`` with uneven non-contiguous bins by specifying the start and end times for the bins:: 112 | 113 | ts = BinnedTimeSeries(start_time=['2016-03-22T12:30:31', '2016-03-22T12:30:33', '2016-03-22T12:30:40'], 114 | end_time=['2016-03-22T12:30:32', '2016-03-22T12:30:35', '2016-03-22T12:30:41'], 115 | data=[1, 4, 3]) 116 | 117 | 118 | Once a binned timeseries was constructed it would present an API that had a 119 | 'time' and a 'timedelta' column, which represents the start time of the bin and 120 | the width of the bin. The timedelta column would not have to be stored in memory 121 | for all of the examples above and could be optimised to allow computation 122 | on-the-fly based on if the bins are contiguous or not. 123 | 124 | 125 | Branches and pull requests 126 | -------------------------- 127 | 128 | N/A 129 | 130 | 131 | Implementation 132 | -------------- 133 | 134 | 135 | 136 | Backward compatibility 137 | ---------------------- 138 | 139 | This would be new functionality. 140 | 141 | 142 | Alternatives 143 | ------------ 144 | 145 | Forego the functionality provided by Time and Units and recommend everyone use pandas. 146 | 147 | 148 | Decision rationale 149 | ------------------ 150 | This APE has long been implemented in https://github.com/astropy/astropy/pull/8540 and somehow the Coco just never merged it. 151 | -------------------------------------------------------------------------------- /APE21.rst: -------------------------------------------------------------------------------- 1 | Ending Long Term Support Releases 2 | --------------------------------- 3 | 4 | author: Thomas Robitaille, Pey Lian Lim, Simon Conseil 5 | 6 | date-created: 2023 February 16 7 | 8 | date-last-revised: 2023 May 15 9 | 10 | date-accepted: 2023 May 31 11 | 12 | type: Process 13 | 14 | status: Accepted 15 | 16 | 17 | Abstract 18 | -------- 19 | 20 | This APE proposes to no longer designate any releases of the core package as 21 | being "long-term support" (LTS) and provide guidance on how to handle API 22 | changes and deprecations. 23 | 24 | 25 | Detailed description 26 | -------------------- 27 | 28 | Motivation 29 | ^^^^^^^^^^ 30 | 31 | The original `APE 2: Astropy Release Cycle and Version Numbering 32 | `_ describes the release cycle for the 33 | core astropy package, which consists of major releases every 6 months that add 34 | new features (e.g., v5.2), and releases in between these that are limited to 35 | fixing bugs or updating documentation (e.g., v5.2.1). Every 2 years, major 36 | releases are designated as being LTS (e.g., v5.0) and we have committed to 37 | having bugfix releases for this version (e.g., v5.0.3) for up to 2 years. 38 | 39 | At the start of the project, having LTS releases was critical for acceptance by 40 | large organizations/projects for which stability is paramount, as the core 41 | package went through significant API changes between versions due to the 42 | youth of the project and LTS releases were a way to shield against such 43 | changes, or at least reduce disruption to be every 2 years. 44 | 45 | LTS releases are however not without cost: 46 | 47 | * While we use infrastructure to automate backporting fixes to release branches, 48 | in practice the LTS branch diverges more and more from the main development 49 | branch and automated backports are more likely to fail as time goes on, 50 | requiring human effort to manually perform the backports, which can sometimes be 51 | non-trivial. 52 | * Even once fixes are backported to the LTS branch, carrying out releases is not 53 | fully automated and requires manual steps, so having to do bugfix releases for 54 | both the latest release branch and LTS branch puts an additional burden on the core 55 | package release team. 56 | 57 | In addition, it is not clear that LTS releases are needed anymore - now that the 58 | core package is over ten years old, it has reached a much higher level of 59 | stability. Most packages in the scientific Python ecosystem do not have LTS 60 | releases, and there is no longer a strong argument for having this in astropy 61 | either. 62 | 63 | Proposed release procedure 64 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 65 | 66 | This APE proposes the following: 67 | 68 | * No longer designate any releases as LTS. 69 | * Refer to the x.0.0 releases (e.g., v6.0.0) as *major*, x.x.0 releases (e.g., 70 | v6.1.0) as *minor*, and x.x.x releases (e.g., v6.1.2) as *bugfix* releases 71 | (essentially `semantic versioning `_) 72 | * Use the x.x.0 format for the first major/minor release in a cycle, rather than 73 | the x.x format previously used, as this is more standard practice with 74 | semantic versioning. 75 | * Keep the default 6-month release cycle for non-bugfix releases, noting that in 76 | line with `APE 2`_, minor versions can also be released more frequently if 77 | needed. 78 | * Leave at least one year between major releases. If one year after the last 79 | major release there are no breaking changes to be made (such as pending 80 | deprecations to remove) then there is no need to release a major version. 81 | The one year timescale for major releases is a minimum, not a fixed 82 | interval. 83 | * Minimize as much as possible any changes that break API or change results in 84 | minor releases, but be pragmatic and occasionally allow such changes if they 85 | would not affect many users. 86 | * Add deprecation warnings or future warnings in minor and major releases, and 87 | make sure that these are present in released versions for a year before 88 | carrying out the breaking changes in a major release. Therefore, if a deprecation 89 | warning is added in a major release, the breaking changes can be made in the 90 | following major release. If a deprecation warning is added a few months before 91 | a major release, the breaking changes can only be made in the subsequent major 92 | release. 93 | 94 | With this in place, we would effectively be striving to maximize stability over 95 | one-year periods but users would see deprecation warnings to prepare them for 96 | changes. This is in contrast to the current LTS system where users might not see 97 | a deprecation warning until the next LTS release as deprecations are not typically 98 | backported. 99 | 100 | As astropy becomes increasingly mature and stable, and since as described above 101 | the 1-year timescale between major releases is a minimum, the interval between 102 | major releases may become arbitrarily longer, and the package may reach a stage 103 | when the major version is no longer or very rarely increased. 104 | 105 | Guidelines on deprecations and API changes 106 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 107 | 108 | To avoid duplication, guidelines on deprecations and API changes as well as 109 | recommendations for communication with users will be added directly to `APE 2`_. 110 | 111 | Branches and pull requests 112 | -------------------------- 113 | 114 | Updates to the core package documentation: https://github.com/astropy/astropy/pull/14713 115 | 116 | Implementation 117 | -------------- 118 | 119 | This APE proposes to continue supporting the v5.0.x LTS series but to not 120 | designate v6.0.0 as LTS. 121 | 122 | Changes to `APE 2`_ can be reviewed in https://github.com/astropy/astropy-APEs/pull/82 123 | 124 | After acceptance of this APE, the documentation for the core package should be 125 | updated appropriately: https://github.com/astropy/astropy/pull/14713 126 | 127 | Backward compatibility 128 | ---------------------- 129 | 130 | N/A 131 | 132 | Alternatives 133 | ------------ 134 | 135 | We could keep the current release workflow - however as mentioned above this has 136 | a maintenance cost and it is not something that is done by most other major 137 | packages in the scientific Python ecosystem. 138 | 139 | Decision rationale 140 | ------------------ 141 | 142 | There is broad consensus in the community that the old model of LTE releases is no longer required, given the 143 | stability that the astropy core package has reached. On the other hand, the maintenance burden described in this APE binds significant resources that are needed in other parts of the project. Thus, this APE is accepted 144 | to simplify the workflow. 145 | 146 | .. _APE 2: https://doi.org/10.5281/zenodo.1043887 147 | -------------------------------------------------------------------------------- /APE8.rst: -------------------------------------------------------------------------------- 1 | Astropy Community Code of Conduct 2 | --------------------------------- 3 | 4 | authors: Kelle Cruz, Alex Hagen, Thomas Robitaille, Erik Tollerud, Alexa Villaume 5 | 6 | date-created: 2015 May 4 7 | 8 | date-last-revised: 2025 Oct 6 9 | 10 | date-accepted: 2015 May 17 11 | 12 | type: Informational 13 | 14 | status: Superseded 15 | 16 | revised-by: 17 | 18 | * Erik Tollerud - 2025 October 6 - Mark this APE as "superseded" on behalf of Astropy Coordination Committee 19 | 20 | Abstract 21 | -------- 22 | 23 | This document and APE grew out of discussion at the April 2015 Lorentz Center 24 | meeting on Python in Astronomy, prompted both by meetings involving a 25 | substantial group from this community, as well as the addition of commenting 26 | capabilities to the Astropy Tutorials page. We believe that our community has 27 | developed to the point that we should have a code of conduct specific to the 28 | Astropy project (and other projects of similar mind). This APE provides the 29 | text of just such a code of conduct, which will be adopted by the project to 30 | provide guidance to the community. The code of conduct will be licensed using 31 | the Creative Commons Attribution license, allowing other projects to use this 32 | code or adapt it to their needs. 33 | 34 | 35 | Detailed description 36 | -------------------- 37 | 38 | The text of the code of conduct is shown here: 39 | 40 | Astropy Community Code of Conduct 41 | ================================= 42 | 43 | The community of participants in open source Astronomy projects is made up of members from around the globe with a diverse set of skills, personalities, and experiences. It is through these differences that our community experiences success and continued growth. We expect everyone in our community to follow these guidelines when interacting with others both inside and outside of our community. Our goal is to keep ours a positive, inclusive, successful, and growing community. 44 | 45 | As members of the community, 46 | 47 | * We pledge to treat all people with respect and provide a harassment- and bullying-free environment, regardless of sex, sexual orientation and/or gender identity, disability, physical appearance, body size, race, nationality, ethnicity, and religion. In particular, sexual language and imagery, sexist, racist, or otherwise exclusionary jokes are not appropriate. 48 | * We pledge to respect the work of others by recognizing acknowledgment/citation requests of original authors. As authors, we pledge to be explicit about how we want our own work to be cited or acknowledged. 49 | * We pledge to welcome those interested in joining the community, and realize that including people with a variety of opinions and backgrounds will only serve to enrich our community. In particular, discussions relating to pros/cons of various technologies, programming languages, and so on are welcome, but these should be done with respect, taking proactive measure to ensure that all participants are heard and feel confident that they can freely express their opinions. 50 | * We pledge to welcome questions and answer them respectfully, paying particular attention to those new to the community. We pledge to provide respectful criticisms and feedback in forums, especially in discussion threads resulting from code contributions. 51 | * We pledge to be conscientious of the perceptions of the wider community and to respond to criticism respectfully. We will strive to model behaviors that encourage productive debate and disagreement, both within our community and where we are criticized. We will treat those outside our community with the same respect as people within our community. 52 | * We pledge to help the entire community follow the code of conduct, and to not remain silent when we see violations of the code of conduct. We will take action when members of our community violate this code such as contacting confidential@astropy.org (all emails sent to this address will be treated with the strictest confidence) or talking privately with the person. 53 | 54 | This code of conduct applies to all community situations online and offline, including mailing lists, forums, social media, conferences, meetings, associated social events, and one-to-one interactions. 55 | 56 | Parts of this code of conduct have been adapted from the `PSF code of conduct `_. 57 | 58 | The Astropy Community Code of Conduct is licensed under a `Creative Commons Attribution 4.0 International License `_. We encourage other communities related to ours to use or adapt this code as they see fit. 59 | 60 | 61 | Branches and pull requests 62 | -------------------------- 63 | 64 | * `astropy.github.com#75 `_: adds the Code of Conduct to the astropy web site. 65 | 66 | 67 | Implementation 68 | -------------- 69 | 70 | If this APE is accepted, the code of conduct shown above will be added to the 71 | astropy.org web site. Note that this is *not* intended to mean that the text 72 | cannot be revised in the future. After this APE is accepted, the authoritative 73 | copy of the code of conduct will be the one on the web site, so future changes 74 | can be made using pull requests to that version. However, any such pull 75 | requests should be advertised to the community via standard communication 76 | channels like the mailing lists, allowing at least one week for feedback from 77 | the community. 78 | 79 | 80 | 81 | Backward compatibility 82 | ---------------------- 83 | 84 | None. 85 | 86 | 87 | Alternatives 88 | ------------ 89 | 90 | None. 91 | 92 | 93 | Decision rationale 94 | ------------------ 95 | 96 | This APE was discussed extensively in person at the Python in Astronomy 97 | workshop, so the version proposed in this APE was already close to final. 98 | Only a few minor changes were made during the APE review process, and in 99 | particular a clause was added that future changes to the code of conduct do 100 | not require a new APE, but do require a period of one week for anyone to 101 | raise objections to proposed changes. No objections were raised for this APE, 102 | and it was therefore accepted on May 17th 2015. 103 | 104 | As of November 2025, the Astropy Community Code of Conduct is superseded by the 105 | NumFOCUS Code of Conduct, which can be found at 106 | https://numfocus.org/code-of-conduct/. The NumFOCUS Code of Conduct, while different in 107 | detailed language, shares the same core principles and in general is the same 108 | as the Code of Conduct described here. This APE is retained for reference and 109 | use by others, but is no longer the active code of conduct for the Astropy 110 | community. For more details on the transition, see 111 | https://github.com/astropy/astropy.github.com/pull/669 and linked mailing list 112 | posts and related issues. 113 | 114 | Previous versions of this APE 115 | ----------------------------- 116 | 117 | * 2015-05-04 [`DOI `_] [`GitHub `_] 118 | -------------------------------------------------------------------------------- /APE12.rst: -------------------------------------------------------------------------------- 1 | Using Cookiecutter for the package-template 2 | ------------------------------------------- 3 | 4 | author: Stuart Mumford, Thomas Robitaille 5 | 6 | date-created: 2016 November 17 7 | 8 | date-last-revised: 2017 March 28 9 | 10 | date-accepted: 2017 March 28 11 | 12 | type: Standard Track 13 | 14 | status: Accepted 15 | 16 | Abstract 17 | -------- 18 | 19 | This APE proposes migrating the `package-template 20 | `_ to use the `Cookiecutter 21 | `_ project to provide templating. This 22 | would enable users to be able to customise the package template for their 23 | package from the command line, in a much simpler manner than the current manual 24 | process. As well as this, having a templating system makes it easier for other 25 | projects to change the branding or default settings of the package template. 26 | 27 | This APE proposes major changes to the current layout of the package-template 28 | repository, potentially breaking existing user workflows, hence the APE. 29 | 30 | 31 | Detailed description 32 | -------------------- 33 | 34 | Cookiecutter is a Python project that uses the Jinja2 templating engine to 35 | define project templates for both Python and non-Python projects. When run it 36 | presents the user with a simple set of questions to populate predefined 37 | variables which are then used to render the Jinja2 templates and create the 38 | project. This workflow is a massive improvement on the manual workflow currently 39 | recommended to affiliated package authors in the 40 | `Astropy documentation `_. 41 | 42 | The second advantage to using a templating system like Jinja2 is that it makes 43 | it possible for other projects (*e.g.* SunPy) to maintain a fork of the package 44 | template with defaults that specific to that project. Without having to make 45 | extensive changes to the actual package template, thereby making it easy to pull 46 | in changes from the Astropy template without lots of complex merging. 47 | 48 | There are also secondary advantages to using a Cookiecutter project, such as 49 | being able to include code i.e. tests, in the `package-template 50 | `_ repository while not including 51 | it in the code that the person setting up a new project receives. This is 52 | because only the templated directory in the repository gets processed by 53 | Cookiecutter. 54 | 55 | An example of using cookiecutter with the package template is below. This 56 | example uses the `Cadair/pacakage-template `_ 57 | ``cookiecutter`` branch, if this work was to be merged the command would be 58 | ``cookiecutter gh:astropy/package-template`` as the ``-c`` option specifies a 59 | non-default branch. 60 | 61 | .. image:: https://asciinema.org/a/7ilbwr10g0rj561clfkedui4j.png 62 | :target: https://asciinema.org/a/7ilbwr10g0rj561clfkedui4j 63 | 64 | 65 | .. code-block:: 66 | 67 | ❯ cookiecutter gh:astropy/package-template 68 | Cloning into 'package-template'... 69 | remote: Counting objects: 952, done. 70 | remote: Compressing objects: 100% (65/65), done. 71 | remote: Total 952 (delta 38), reused 0 (delta 0), pack-reused 880 72 | Receiving objects: 100% (952/952), 258.38 KiB | 0 bytes/s, done. 73 | Resolving deltas: 100% (505/505), done. 74 | Branch cookiecutter set up to track remote branch cookiecutter from origin. 75 | Switched to a new branch 'cookiecutter' 76 | 77 | package_name [packagename]: CadairPy 78 | 79 | module_name [cadairpy]: 80 | 81 | short_description [CadairPy]: CadairPy is code written about a mountain. 82 | 83 | long_description []: Cadair Idris or Cader Idris is a mountain in Gwynedd, Wales, which lies at the southern end of the Snowdonia National Park near the town of Dolgellau. The peak, which is one of the most popular in Wales for walkers and hikers, is composed largely of Ordovician igneous rocks, with classic glacial erosion features such as cwms, moraines, striated rocks, and roches moutonnées. 84 | 85 | author_name [Astropy Developers]: Stuart Mumford 86 | 87 | author_email []: stuart@cadair.com 88 | 89 | Select license: 90 | 1 - BSD 3-Clause 91 | 2 - GNU GPL v3+ 92 | 3 - Apache Software Licence 2.0 93 | 4 - BSD 2-Clause 94 | 5 - Other 95 | Choose from 1, 2, 3, 4, 5 [1]: 96 | 97 | project_url [http://astropy.org]: http://cadair.com 98 | 99 | project_version [0.0.dev]: 0.0.1.dev 100 | 101 | include_example_code [y]: 102 | 103 | include_cextern_folder [n]: 104 | 105 | edit_on_github_extension [False]: True 106 | 107 | github_project [astropy/astropy]: cadair/cadairpy 108 | 109 | use_travis_ci [y]: 110 | 111 | use_appveyor_ci [y]: 112 | 113 | use_read_the_docs [y]: 114 | 115 | sphinx_theme [astropy-boostrap]: 116 | 117 | ❯ ls -la 118 | total 116K 119 | drwxr-xr-x 5 stuart users 4.0K Nov 17 14:41 ./ 120 | drwx------ 69 stuart users 4.0K Nov 17 14:41 ../ 121 | -rw-r--r-- 1 stuart users 36K Nov 17 14:41 ah_bootstrap.py 122 | -rw-r--r-- 1 stuart users 1.8K Nov 17 14:41 appveyor.yml 123 | drwxr-xr-x 6 stuart users 4.0K Nov 17 14:41 cadairpy/ 124 | drwxr-xr-x 4 stuart users 4.0K Nov 17 14:41 docs/ 125 | -rw-r--r-- 1 stuart users 13K Nov 17 14:41 ez_setup.py 126 | -rw-r--r-- 1 stuart users 655 Nov 17 14:41 .gitignore 127 | -rw-r--r-- 1 stuart users 108 Nov 17 14:41 .gitmodules 128 | drwxr-xr-x 2 stuart users 4.0K Nov 17 14:41 licenses/ 129 | -rw-r--r-- 1 stuart users 1.1K Nov 17 14:41 MANIFEST.in 130 | -rw-r--r-- 1 stuart users 742 Nov 17 14:41 README.md 131 | -rw-r--r-- 1 stuart users 71 Nov 17 14:41 readthedocs.yml 132 | -rw-r--r-- 1 stuart users 160 Nov 17 14:41 .rtd-environment.yml 133 | -rw-r--r-- 1 stuart users 1.2K Nov 17 14:41 setup.cfg 134 | -rwxr-xr-x 1 stuart users 4.0K Nov 17 14:41 setup.py* 135 | -rw-r--r-- 1 stuart users 4.8K Nov 17 14:41 .travis.yml 136 | 137 | 138 | 139 | 140 | Branches and pull requests 141 | -------------------------- 142 | 143 | A version of the package template converted to a Cookiecutter project is `here 144 | `_. It can be 145 | tested by installing the cookiecutter package from pip or conda-forge and 146 | running:: 147 | 148 | cookiecutter -c cookiecutter gh:cadair/package-template 149 | 150 | 151 | Implementation 152 | -------------- 153 | 154 | The changes to the package template are to move the template into a subdirectory 155 | and to change all occurrences of ``packagename`` and other user modifiable 156 | variables to Jinja template placeholders such as ``{{ cookiecutter.package_name }}``. 157 | 158 | A draft implementation is `here `_. Which is in 159 | `PR #202 `_ to the 160 | package-template repository. 161 | 162 | As well as the changes to the package template the documentation in the Astropy 163 | docs regarding the package template will be significantly simplified. As a part 164 | of the implementation it also becomes possible to keep the documentation and 165 | instructions for using the package template in the `package-template` 166 | repository. This change will be made with the implementation PR and the Astropy 167 | docs changed once this is done. 168 | 169 | 170 | Backward compatibility 171 | ---------------------- 172 | 173 | As implemented in `Cadair/package-template 174 | `_ users who have 175 | followed the instructions where they use the package template as a git remote 176 | will no longer be able to pull updates from the ``package_template`` git 177 | repository. To mitigate this lack of backwards compatibility a new branch on the 178 | current package-template repository will be created which holds the cookiecutter 179 | template version of the package-template. An automatically rendered (i.e. using 180 | Travis CI) version of the template with the default settings will be kept in the 181 | main branch of the package-template repository. This will result in a main 182 | branch that looks like the current package-template repository. This could then 183 | be used as a git remote for people who wish to continue using this update 184 | method. 185 | 186 | In the longer term, using cookiecutter could make the implementation of an 187 | update helper, or automatic update process easier. This is mainly due to the 188 | fact that cookiecutter can be called with the settings used to originally create 189 | the project, i.e the correct name and other options. This will lead to smaller 190 | more relevant diffs when updating and therefore make any automation easier. 191 | 192 | 193 | Alternatives 194 | ------------ 195 | 196 | Keeping the current solution. This removes any need for changes for anyone using 197 | the 'git update' method at the moment, but it does not provide any of the 198 | templating benefits presented above. 199 | 200 | Cookiecutter is not the only project templating system, others such as 201 | `diecutter `_ could be used. Cookiecutter was 202 | chosen due to wide adoption and implementation in Python. 203 | 204 | Finally, a custom templating system could be developed and maintained for the 205 | Astropy project, however, this would be a lot of effort. 206 | 207 | 208 | Decision rationale 209 | ------------------ 210 | 211 | There was discussion on the `APE pull request `_ and the `implementation pull request `_, but mainly specific details, rather than objections to the general scheme. The discussion resulted in figuring out a way to not break backward-compatibility for people who want to continue using the git remote method for updating their packages. The above content reflects these improvements, and the APE was accepted 2017-03-28, as there were no objections from the community. 212 | -------------------------------------------------------------------------------- /APE2.rst: -------------------------------------------------------------------------------- 1 | Astropy Release Cycle and Version Numbering 2 | ------------------------------------------- 3 | 4 | author: Erik Tollerud 5 | 6 | date-created: 2013 November 18 7 | 8 | date-last-revised: 2023 May 31 9 | 10 | date-accepted: 2013 December 10 11 | 12 | type: Process 13 | 14 | status: Accepted 15 | 16 | revised-by: Thomas Robitaille, Pey Lian Lim, Simon Conseil - 2023 May 15 - Rewritten following the approval of APE 21 to no longer have Long Term Support (LTS) versions 17 | 18 | 19 | Abstract 20 | -------- 21 | 22 | This APE describes the general plan for timing of releases of the core package 23 | and their version numbers. The basic premise follows semantic 24 | versioning: *major* releases (e.g., 6.0.0) every year (or longer), *minor* 25 | releases (e.g., 6.1.0) typically every six months in between, and *bugfix* 26 | releases (e.g., 6.1.2) as needed between minor releases. 27 | 28 | Detailed description 29 | -------------------- 30 | 31 | The release plan prior to v0.3 was somewhat haphazard, with releases carried out when a 32 | certain set of features were judged "ready", but without a clear sense of how 33 | long those features would take. This led to concerns that new features were 34 | spending a long time in the development version because *other* features were 35 | not yet complete. At the same time, some larger institutions and facilities had 36 | back then expressed concern that there was not a clearly-stated support cycle for 37 | Astropy versions. That is, it was not clear if a given version would be supported 38 | via critical bug fixes if it was not the latest version. This was voiced as a 39 | concern because facilities will not always want to upgrade to the latest 40 | version, but do want to be sure major bugs will get addressed. This led to the 41 | initial version of this APE following discussions at the 2013 Coordination 42 | Meeting. This APE is a living document and is meant to reflect the latest 43 | release guidelines (prior versions can be found in `Previous versions of this 44 | APE`_). 45 | 46 | Release versioning and scheduling 47 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 48 | 49 | The release cycle for the core package should be as follows: 50 | 51 | * Major releases every year (or longer). These are releases that can introduce breaking 52 | API changes, and should be numbered as x.0.0 (e.g., 6.0.0). 53 | * Minor releases by default every six months in between major releases (or more 54 | frequent if needed). These releases should minimize any API changes and focus 55 | on adding new features. 56 | * Bugfix releases as needed between minor releases. 57 | 58 | As an example, starting from the 6.0.0 release, the version number schedule might be: 59 | 60 | * 6.0.0 61 | * 6.0.1 62 | * 6.0.2 63 | * 6.1.0 (six months after 6.0.0) 64 | * 6.1.1 65 | * 6.1.2 66 | * 7.0.0 (six months after 6.1.0, one year after 6.0.0) 67 | 68 | In the above scenario, we would adopt the default 6-month release schedule, and 69 | introduce a new major version with breaking changes one year later. However, the 70 | release schedule could also look like the following: 71 | 72 | * 6.0.0 73 | * 6.0.1 74 | * 6.0.2 75 | * 6.1.0 (three months after 6.0.0) 76 | * 6.1.1 77 | * 6.1.2 78 | * 6.2.0 (six months after 6.1.0) 79 | * 6.2.1 80 | * 6.3.0 (four months after 6.2.0) 81 | * 6.3.1 82 | * 6.3.2 83 | * 6.3.3 84 | * 7.0.0 (five months after 6.3.0, 18 months after 6.0.0) 85 | 86 | In the above hypothetical scenario, the developers have decided to release minor 87 | releases more frequently than every six months, and a year after 6.0.0 there was 88 | no need for a major version, so this was delayed until 18 months after 6.0.0. 89 | 90 | Feature freeze/testing calendar 91 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 92 | 93 | Because bug checking/fixing and preparation for release take time, a feature 94 | freeze will need to be enacted *before* the time of a given feature release. 95 | Such a feature freeze will mean no more major new feature pull requests will be 96 | accepted for that version, but minor improvements, bug fixes, or documentation 97 | additions are still acceptable. The standard calendar for a release will be: 98 | 99 | * -6 months: previous release 100 | * -1 month: feature freeze, beta release, branch created 101 | * -1 week: release candidate 102 | 103 | This is not meant to be an exactly strict calendar, as the day of the week, 104 | existence of holidays, and degree of readiness of key features or critical bug 105 | fixes necessitate slop in the exact timing. 106 | 107 | Rapid Feature Releases 108 | ^^^^^^^^^^^^^^^^^^^^^^ 109 | 110 | This plan is not meant to exclude the possibility of minor releases *more* frequent 111 | than every six months. Six months should be considered both the maximum and the 112 | default timeline, but a *shorter* time between releases is allowable, if the 113 | community agrees it is worthwhile. When/if this happens, the subsequent major 114 | release should be timed such that is at least one year after the last major release, even 115 | though that may mean there are more than one minor release between the major releases. 116 | 117 | For example, if a new feature is perceived by the Astropy developer community as 118 | being of great importance, and is completed only three months after the v6.0.0 119 | release, a v6.1.0 release could occur at that time. Once that release has 120 | completed, the release clock is "reset", so v6.2.0 would be released six months 121 | after v6.1.0, even though it is not yet 12 months after v6.0.0. The next release 122 | could then be v7.0.0 three or more months after v6.2.0, or could also be another 123 | minor release. 124 | 125 | Releases and backwards compatibility 126 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 127 | 128 | The guidelines in relation to API changes and backward compatibility are as follows: 129 | 130 | * API changes (i.e., modifications as opposed to additions) should only be 131 | carried out in major releases. 132 | * Any API changes should be preceded by deprecation warnings that should be 133 | visible to users for at least a year prior to the API change 134 | being carried out. 135 | * API changes should be documented, with clear guidance on what is changing, 136 | why the change is being made, and how to migrate existing code to the new behavior. 137 | * New deprecations should not be introduced in bugfix releases. 138 | * If developers wish to make an API change at a point in time where the next 139 | release is a major release, they should introduce the deprecation in the major 140 | release and carry out the change in the following major release 141 | (i.e., the next x.0.0 that is at least a year away). 142 | * The following will not be considered to be part of the API and can therefore 143 | be changed in bugfix and minor releases: 144 | 145 | * Changes to warning messages 146 | * Changes to exception messages 147 | * Changes to the ``__repr__`` or ``__repr_*__`` of objects 148 | 149 | However, exception and warning *types* will be considered to be part of the 150 | API. 151 | 152 | As with all guidelines, there are exceptions where we can deviate from these guidelines: 153 | 154 | * API changes can be carried out in major releases without deprecation if it is 155 | not possible to have a deprecation phase due to the nature of the change. 156 | * Changes breaking or changing the behavior of code may be needed in order to 157 | fix bugs. Such changes can be made in bugfix or minor releases and do not need 158 | to be considered API changes. 159 | * Changes to exception or warning types may occasionally be done in minor 160 | releases. 161 | 162 | These exceptions will require judgment calls on the part of maintainers, but any 163 | such changes should be minimized as much as possible and clearly communicated to 164 | users (see `Communication with users`_). 165 | 166 | Documentation and performance improvements can be made in bugfix releases as long 167 | as they do not involve any changes or additions to the API. 168 | 169 | Communication with users 170 | ^^^^^^^^^^^^^^^^^^^^^^^^ 171 | 172 | It is imperative that we communicate effectively with users so that they understand 173 | what to expect from different releases: 174 | 175 | * We should include documentation for the core package that describes the 176 | release policy above, namely that in general minor releases will not 177 | break/remove functionality, but that major releases might, and also explaining 178 | what is considered API. 179 | 180 | * We should ensure that any changes in releases which deviate from the 181 | guidelines above are clearly communicated to users - for example any API/breaking 182 | changes in minor releases should be included in the "What's New" page for the 183 | minor release. 184 | 185 | * Deprecations should always be mentioned in the changelog, as well as any 186 | removal of deprecated code/API changes. We do not specify how exactly this 187 | should be done, but note that, e.g., Numpy release notes have separate sections 188 | entitled *Deprecations*, *Expired Deprecations* and *Compatibility Notes* in 189 | their release notes (see `here 190 | `_ for an example) 191 | which is clearer than having a single *API changes* section. 192 | 193 | * Maintainers should be encouraged to advertise pull requests with API changes 194 | that are likely to affect users and developers of other packages to the 195 | developer mailing list, and ensuring that the pull request has adequate time 196 | (at least two weeks) to be reviewed properly. 197 | 198 | Branches and pull requests 199 | -------------------------- 200 | 201 | N/A 202 | 203 | 204 | Implementation 205 | -------------- 206 | 207 | Whenever this APE is updated, the core package documentation should be updated 208 | to reflect the latest guidelines described above: https://github.com/astropy/astropy/pull/14713 209 | 210 | Backward compatibility 211 | ---------------------- 212 | 213 | N/A 214 | 215 | 216 | Alternatives 217 | ------------ 218 | 219 | N/A 220 | 221 | Decision rationale 222 | ------------------ 223 | 224 | There was a fair amount of discussion on `the GitHub PR 225 | `_, but mainly specific details, 226 | rather than objections to the general scheme. The largest change from the 227 | original proposal was the addition of the idea that *less than* six month 228 | releases were acceptable. The above content reflects those suggestions, and the 229 | APE was accepted 12/10/13, as there were no significant objections from the 230 | community. 231 | 232 | Previous versions of this APE 233 | ----------------------------- 234 | 235 | * 2013-12-10 [`DOI `_] [`GitHub `_] 236 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | APEs: Astropy Proposals for Enhancement 2 | --------------------------------------- 3 | 4 | APEs are documents to address non-trivial enhancements that require discussion 5 | and thought beyond a single Pull Request. This is intended to mirror the 6 | long-standing PEP (Python Enhancement Proposal) process, but generally not quite as 7 | formally. Please consult `APE Purpose and Process`_ for full details. 8 | 9 | APEs 10 | ^^^^ 11 | 12 | === ================================================================ =========== ============ ============ 13 | # Title Date (last Status DOI 14 | revised) 15 | === ================================================================ =========== ============ ============ 16 | 0 `The Astropy Project Governance Charter`_ 2021-Feb-19 Accepted |APE 0 DOI| 17 | 1 `APE Purpose and Process`_ 2025-Oct-27 Accepted |APE 1 DOI| 18 | 2 `Astropy Release Cycle and Version Numbering`_ 2013-Dec-11 Accepted |APE 2 DOI| 19 | 3 `Configuration`_ 2013-Dec-10 Accepted |APE 3 DOI| 20 | 4 `Astropy Setup Helpers`_ 2014-Jun-28 Accepted |APE 4 DOI| 21 | 5 `Coordinates Subpackage Plan`_ 2014-Jan-22 Accepted |APE 5 DOI| 22 | 6 `Enhanced Character Separated Values table format`_ 2015-Jan-26 Accepted |APE 6 DOI| 23 | 7 `NDData Plan`_ 2014-Dec-17 Accepted |APE 7 DOI| 24 | 8 `Astropy Community Code of Conduct`_ 2025-Oct-06 Superseded |APE 8 DOI| 25 | 9 `Timeseries object for Astropy`_ 2023-Oct-20 Accepted |APE 9 DOI| 26 | 10 `Roadmap for Python 3-only support`_ 2016-Aug-22 Accepted |APE 10 DOI| 27 | 12 `Using Cookiecutter for the package-template`_ 2017-Mar-28 Accepted |APE 12 DOI| 28 | 13 `Vision for Astropy Spectroscopic Tools`_ 2017-Dec-12 Accepted |APE 13 DOI| 29 | 14 `A shared Python interface for World Coordinate Systems`_ 2018-Nov-09 Accepted |APE 14 DOI| 30 | 15 `An Updated Model for the Affiliated Package Ecosystem`_ 2018-May-14 Accepted |APE 15 DOI| 31 | 17 `A roadmap for package infrastructure without astropy-helpers`_ 2019-Dec-12 Accepted |APE 17 DOI| 32 | 18 `Adopt NEP 29 for CPython and Numpy Version Support`_ 2024-Jul-12 Accepted |APE 18 DOI| 33 | 19 `Distributing Astropy Project Funding`_ 2022-Feb-28 Accepted |APE 19 DOI| 34 | 20 `Formatting Code with Black`_ 2022-Sep-23 Accepted |APE 20 DOI| 35 | 21 `Ending Long Term Support Releases`_ 2023-May-31 Accepted |APE 21 DOI| 36 | 22 `Astropy Affiliated Packages with pyOpenSci`_ 2024-Jan-29 Accepted |APE 22 DOI| 37 | === ================================================================ =========== ============ ============ 38 | 39 | .. _The Astropy Project Governance Charter: https://github.com/astropy/astropy-APEs/blob/main/APE0.rst 40 | .. _APE Purpose and Process: https://github.com/astropy/astropy-APEs/blob/main/APE1.rst 41 | .. _Astropy Release Cycle and Version Numbering: https://github.com/astropy/astropy-APEs/blob/main/APE2.rst 42 | .. _Configuration: https://github.com/astropy/astropy-APEs/blob/main/APE3.rst 43 | .. _Astropy Setup Helpers: https://github.com/astropy/astropy-APEs/blob/main/APE4.rst 44 | .. _Coordinates Subpackage Plan: https://github.com/astropy/astropy-APEs/blob/main/APE5.rst 45 | .. _Enhanced Character Separated Values table format: https://github.com/astropy/astropy-APEs/blob/main/APE6.rst 46 | .. _NDData Plan: https://github.com/astropy/astropy-APEs/blob/main/APE7.rst 47 | .. _Astropy Community Code of Conduct: https://github.com/astropy/astropy-APEs/blob/main/APE8.rst 48 | .. _Timeseries object for Astropy: https://github.com/astropy/astropy-APEs/blob/main/APE9.rst 49 | .. _Roadmap for Python 3-only support: https://github.com/astropy/astropy-APEs/blob/main/APE10.rst 50 | .. _Using Cookiecutter for the package-template: https://github.com/astropy/astropy-APEs/blob/main/APE12.rst 51 | .. _Vision for Astropy Spectroscopic Tools: https://github.com/astropy/astropy-APEs/blob/main/APE13.rst 52 | .. _A shared Python interface for World Coordinate Systems: https://github.com/astropy/astropy-APEs/blob/main/APE14.rst 53 | .. _An Updated Model for the Affiliated Package Ecosystem: https://github.com/astropy/astropy-APEs/blob/main/APE15.rst 54 | .. _A roadmap for package infrastructure without astropy-helpers: https://github.com/astropy/astropy-APEs/blob/main/APE17.rst 55 | .. _Adopt NEP 29 for CPython and Numpy Version Support: https://github.com/astropy/astropy-APEs/blob/main/APE18.rst 56 | .. _Distributing Astropy Project Funding: https://github.com/astropy/astropy-APEs/blob/main/APE19.rst 57 | .. _Formatting Code with Black: https://github.com/astropy/astropy-APEs/blob/main/APE20.rst 58 | .. _Ending Long Term Support Releases: https://github.com/astropy/astropy-APEs/blob/main/APE21.rst 59 | .. _Astropy Affiliated Packages with pyOpenSci: https://github.com/astropy/astropy-APEs/blob/main/APE22.rst 60 | 61 | .. |APE 0 DOI| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.4552790.svg 62 | :target: https://doi.org/10.5281/zenodo.4552790 63 | 64 | .. |APE 1 DOI| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.1043885.svg 65 | :target: https://doi.org/10.5281/zenodo.1043885 66 | 67 | .. |APE 2 DOI| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.1043887.svg 68 | :target: https://doi.org/10.5281/zenodo.1043887 69 | 70 | .. |APE 3 DOI| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.1043889.svg 71 | :target: https://doi.org/10.5281/zenodo.1043889 72 | 73 | .. |APE 4 DOI| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.1043891.svg 74 | :target: https://doi.org/10.5281/zenodo.1043891 75 | 76 | .. |APE 5 DOI| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.1043896.svg 77 | :target: https://doi.org/10.5281/zenodo.1043896 78 | 79 | .. |APE 6 DOI| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.1043900.svg 80 | :target: https://doi.org/10.5281/zenodo.1043900 81 | 82 | .. |APE 7 DOI| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.1043906.svg 83 | :target: https://doi.org/10.5281/zenodo.1043906 84 | 85 | .. |APE 8 DOI| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.1043912.svg 86 | :target: https://doi.org/10.5281/zenodo.1043912 87 | 88 | .. |APE 9 DOI| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.10403160.svg 89 | :target: https://doi.org/10.5281/zenodo.10403160 90 | 91 | .. |APE 10 DOI| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.1038586.svg 92 | :target: https://doi.org/10.5281/zenodo.1038586 93 | 94 | .. |APE 12 DOI| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.1044483.svg 95 | :target: https://doi.org/10.5281/zenodo.1044483 96 | 97 | .. |APE 13 DOI| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.14834949.svg 98 | :target: https://doi.org/10.5281/zenodo.14834949 99 | 100 | .. |APE 14 DOI| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.11566733.svg 101 | :target: https://doi.org/10.5281/zenodo.11566733 102 | 103 | .. |APE 15 DOI| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.1246833.svg 104 | :target: https://doi.org/10.5281/zenodo.1246833 105 | 106 | .. |APE 17 DOI| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.3572843.svg 107 | :target: https://doi.org/10.5281/zenodo.3572843 108 | 109 | .. |APE 18 DOI| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.4563082.svg 110 | :target: https://doi.org/10.5281/zenodo.4563082 111 | 112 | .. |APE 19 DOI| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.6312048.svg 113 | :target: https://doi.org/10.5281/zenodo.6312048 114 | 115 | .. |APE 20 DOI| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.10562869.svg 116 | :target: https://doi.org/10.5281/zenodo.10562869 117 | 118 | .. |APE 21 DOI| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.7990988.svg 119 | :target: https://doi.org/10.5281/zenodo.7990988 120 | 121 | .. |APE 22 DOI| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.10581892.svg 122 | :target: https://doi.org/10.5281/zenodo.10581891 123 | 124 | Proposing a new APE 125 | ^^^^^^^^^^^^^^^^^^^ 126 | 127 | Please read the "Submitting an APE" section in `APE Purpose and Process`_. 128 | 129 | Tips: 130 | 131 | * Don't forget to check the open PRs when finding the 132 | next free number for your APE. 133 | * For your PR description, usually it's the easiest to paste in the 134 | abstract, and link to other existing GitHub Issues or PRs, if applicable. 135 | 136 | Finalizing APEs 137 | ^^^^^^^^^^^^^^^ 138 | 139 | Please read the "APE Review" section in `APE Purpose and Process`_. 140 | 141 | If the APE was accepted, here are the archiving (Zenodo) instructions for 142 | the Coordination Committee: 143 | 144 | #. Upload the APE to Zenodo to give it a DOI. Log into the astropy team Zenodo 145 | account (*not your personal account*), go to https://zenodo.org/deposit/new, 146 | upload the RST file, and set the fields to the following: 147 | 148 | ============================= ====================================================== 149 | Zenodo field Set to 150 | ============================= ====================================================== 151 | Communities The Astropy Project 152 | Upload type Publication 153 | Publication type Technical note 154 | Publication Date The date-last-revised of the APE (which should be the same as the accepted date for new APEs) 155 | Title Astropy Proposal for Enhancement : (APE <number>) 156 | Authors The APE authors (directly from the APE text, but with ORCIDs if possible) 157 | Description The APE abstract (copy/paste the rendered HTML from GitHub) 158 | License CC-Attribution (default) 159 | Related/alternate identifiers GitHub link to the APE file *at the specific merge commit* 160 | (e.g. https://github.com/astropy/astropy-APEs/blob/42951733ac42c0ea178d8df30705274a43c93091/APE1.rst) 161 | as "is supplemented by this upload". If this is a revised version, 162 | this should be the URL of the commit where the APE was revised. 163 | ============================= ====================================================== 164 | 165 | #. Get the source for the DOI badge from the newly-created Zenodo record page by 166 | clicking on the DOI badge on the right side of the page and copying the 167 | reStructuredText source. You will need to update that with the "all versions" 168 | DOI number, which you can see under the "versions" section of Zenodo. (It is 169 | typically the first version minus 1.) 170 | #. (This step can be done directly on `main` or as a PR, as you prefer.) 171 | Edit `README.rst` to add the DOI link from the previous step to the relevant 172 | row for the APE in the "APEs" table. Preview the update and test the link 173 | to make sure they are all correct. Then, commit the change. 174 | 175 | Updating APEs 176 | ^^^^^^^^^^^^^ 177 | 178 | Please read the "Modifications to an existing APE" section in `APE Purpose and Process`_. 179 | 180 | If applicable, the Zenodo entry needs to be updated with a new version of the APE (*not* a completely new 181 | Zenodo entry), by using the "New version" button and then filling out the forms 182 | as described in "Finalizing APEs" section above. 183 | -------------------------------------------------------------------------------- /APE19.rst: -------------------------------------------------------------------------------- 1 | Distributing Astropy Project Funding 2 | ==================================== 3 | 4 | author: Kelle Cruz, H. Moritz Günther, Aarya Patil, John Swinbank, Erik Tollerud 5 | 6 | date-created: 2022 January 21 7 | 8 | date-last-revised: 2022 February 28 9 | 10 | date-accepted: 2022 February 28 11 | 12 | type: Process 13 | 14 | status: Accepted 15 | 16 | 17 | Abstract 18 | ========= 19 | 20 | This APE aims to provide guidance about spending Astropy Project funds by formulating a set of principles about what to pay for and who to pay. It will also touch on how expenditure decisions are made, how funds get administered, and a principle of strong transparency around these topics. It is heavily inspired by `Numpy's NEP 48 <https://numpy.org/neps/nep-0048-spending-project-funds.html>`_. 21 | 22 | 23 | Detailed description 24 | ==================== 25 | 26 | Motivation and Scope 27 | -------------------- 28 | 29 | Astropy is currently a fiscally sponsored project of NumFOCUS, a 501(c)(3) nonprofit organization headquartered in Austin, TX, meaning NumFOCUS is the legal entity that handles finances for Astropy. Therefore, all legal and accounting matters related to the Astropy Project have to follow the rules and regulations for American nonprofits. All nonprofit donations are classified into two categories: unrestricted funds, which may be used for any legal purpose appropriate to the organization, and restricted funds, which are reserved for a particular purpose (a specific project, educational program, etc.). 30 | 31 | Having a clear set of principles around how to use these funds will facilitate spending them fairly and effectively. Additionally, it will make it easier to solicit donations and other contributions. The guidelines here are meant for community members to consider when proposing to use Project funds, for the community to consider when voting on proposals, and for the Finance Committee to use to guide the funding process. 32 | 33 | A key assumption of this APE is that Astropy is developed and maintained by an institutionally and geographically distributed group of independently motivated individuals, and that the Project funds are not enough to employ most of the maintainers full-time. If funding increases to the point where that assumption is no longer true, this APE should be updated. 34 | 35 | In scope for this APE are: 36 | 37 | * Principles of spending project funds: what to pay for, and who to pay, and how much to pay. 38 | * Describing how Astropy's funds get administered. 39 | * Describing how expenditure decisions are proposed and made. 40 | 41 | 42 | Principles for spending project funds 43 | ------------------------------------- 44 | 45 | Astropy will likely always be a project with many times more volunteer contributors than funded people. It follows that having funded people operate in ways that attract more volunteers and enhance their participation experience is critical. That key principle motivates many of the more detailed principles given below for what to pay for and whom to pay. 46 | 47 | We identify four distinct considerations when disbursing funds: 48 | 49 | * Agreeing upon the types of activities that can be funded, 50 | * Establishing a process for deciding which specific activities should be funded, 51 | * Finding someone to do the work (if necessary), 52 | * Determining an appropriate compensation level. 53 | 54 | The following sections go into detail on each of these points. 55 | 56 | What to pay for 57 | ^^^^^^^^^^^^^^^ 58 | We can pay for anything which contributes to the broad mission of the Astropy Project. This includes, for example, code development, community engagement, education, promotion, etc. Funds can be used for many purposes, for example, paid developer time, teaching buyout, development of training materials, expenses for developer meetings or sprints, hosted hardware, and CI or other software services, carbon offsets, promotion and advertising, travel to conferences or meetings to promote Project activities. 59 | 60 | How to decide what to pay for 61 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 62 | 1. Pay for things that are important and otherwise will not get done. *Rationale: there is much more work to be done than there are funds to do all of that work. There are also many volunteers and externally sponsored people interested and willing to do some of the work without compensation from the Project.* 63 | 2. Do not rely on funding always being there. Plan for sustainability: how will activities be continued when the funds dry up? Invest in documentation, infrastructure, and easy to maintain workflows. For example, funded work on governance should always record the process clearly so that volunteers who follow have less work to do to figure it out. 64 | 3. Consider potential positive benefits to Astropy maintainers and contributors, maintainers of other projects, end users, and other stakeholders like packagers and educators. 65 | 4. Also consider the long-term costs to Astropy maintainers and contributors, to counterbalance item number 3. 66 | 5. Do not increase the load on volunteer maintainers by assuming they are available to review or otherwise support the work. Instead, proposals should include paid time for reviews and other necessary activities. *Rationale: the effect of spending funds should be positive for everyone, not just for the people getting paid. This is also a matter of fairness.* 67 | 68 | Restricted / earmarked funding sources 69 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 70 | Much of the Project's funding comes from specific grants from private foundations and government agencies. In these cases, the fundable activities are explicitly described in the proposal text. Such funds must be spent following the proposed plan and in accordance with any other conditions imposed by the funding agency. The Finance Committee should make these restrictions clear to the community so that they may submit funding requests aligned with the existing grants. 71 | 72 | Whom to pay 73 | ^^^^^^^^^^^ 74 | 1. All else being equal, give preference to existing contributors. 75 | 2. When looking outside of the current team (from here on we use the word "team" to include anyone in the Astropy community, including code contributors, mentors, workshop facilitators, outreach, or any other activity for the benefit of the project), consider this an opportunity to make the project team more diverse. 76 | 3. Pay attention to the following when considering whether to pay someone 77 | 78 | * the necessary skills to execute the tasks, 79 | * communication and self-management skills, 80 | * experience contributing to and working with open source projects, 81 | * likelihood of long-term involvement with the Astropy community. 82 | 83 | It will likely depend on the particular activity as to whether there's already a clear best candidate within the Astropy team, or if we must look for new people to get involved. This decision should be taken in accordance with documented Project governance procedures. If the position is not filled from within the project, it should be publicly advertised to give a broader group of people a chance to apply for it. 84 | 85 | Compensating an appropriate amount 86 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 87 | Process 88 | +++++++ 89 | In general (see next section), we put the funding level directly in the hands of the community and those making funding requests. We use a process where proposers suggest work and include a budget request which they think is required to achieve the suggested work. These suggestions are then put to the community for input and feedback. As part of the review process, the community is expected to take “value-for-money” into account. This mechanism places the trust to decide when compensation is unreasonably high or low to the community. In other cases, the Finance Committee will determine an appropriate amount taking into account the discussion and rationale below. 90 | 91 | Discussion and Rationale 92 | ++++++++++++++++++++++++ 93 | This is motivated by the fundamental tension that we, as a community, want to get the most out of the funds available, but we also want to avoid a “race to the bottom” where developers undercut others on price until the pay is too low to live on. 94 | Setting an appropriate level of pay for certain work is a difficult task, since a range of factors influence what people think is “fair.” This includes, to name just a few considerations, 95 | 96 | * external factors like the cost of living in a certain place or country, 97 | * if people are paid as independent contractors (and thus have to factor vacation, sick time, medical insurance, etc. into an hourly rate) or as a subaward to, for example, an academic institution (which usually set rates that include overheads and fringe benefits that are then provided to the employee), 98 | * What monetary value one puts on experience (e.g., an undergraduate student vs. an experienced software developer). 99 | 100 | `NEP 48 <https://numpy.org/neps/nep-0048-spending-project-funds.html>`_ has extensive discussion and a good list of further references on this topic. 101 | 102 | However, we recognize that the approach of leaving the approved budget entirely up to project members also has downsides. In particular, less experienced people may be tempted to suggest work packages with a compensation that is unfairly low for the work required to increase their perceived chances to obtain funding. We place trust in the community to recognize these short-comings of the process and to advise and mentor each other to propose realistic budgets that allow the proposed work to be completed with a fair rate of pay. As the diversity of proposers increases, we recognize that this approach will likely need to be modified and potentially be made more prescriptive. This may come in the form of a future APE or more incremental changes as experience grows. 103 | 104 | Defining fundable activities and projects 105 | ----------------------------------------- 106 | We define three separate categories of expense, by value: 107 | 108 | * Up to $2000 (US); 109 | * $2000 to $10000 (US); 110 | * More than $10000 (US). 111 | 112 | Items in the first category — up to $2000 — may be approved by majority vote of the Finance Committee. 113 | 114 | Items in the second category — up to $10000 — can be approved by majority vote of the Finance Committee. If, in the judgment of one or more Finance Committee members, the item would not receive widespread support, the Committee will communicate these concerns to the Coordination Committee (CoCo). The CoCo may provide a decision, or may request that the Finance Committee organize a vote among the Astropy membership to approve the expense. 115 | 116 | Items in the third category — more than $10000 — should follow approved Project decision making methods from `APE0 <https://github.com/astropy/astropy-APEs/blob/main/APE0.rst>`_, or a vote among the Astropy voting membership unless specifically allocated in a grant to a specific person or organization already. In the latter case, the Finance Committee is tasked to ensure it is spent according to the provisions of the grant. The Finance Committee will periodically solicit suggestions from the community for projects and other major expenses and coordinate a selection process. 117 | 118 | *Rationale: We want the community to make the big and important decisions, but also not overburden community members with frequent discussions and voting.* 119 | 120 | Strategy for spending/saving funds 121 | ---------------------------------- 122 | There is an expectation from Astropy funders that the funds will be used for the benefit of the project and the community. Therefore, we should spend available funds thoughtfully, strategically, and fairly, as they become available. For emergencies, we should keep a reserve of $10,000 to $15,000 (US) that could cover, for example, a year of CI and hosting services, a few weeks of full-time maintenance work, or engaging a consultant to address a specific need. 123 | 124 | How project funds get administered 125 | ----------------------------------- 126 | Astropy wants to administer project funds as transparently as possible while remaining efficient for those receiving and administering funds, complying with the terms of specific grant programs or funding agencies, and providing appropriate protection for personal information. The administration process is therefore publicly visible and open to input from the Astropy community whenever possible. In some cases, though, specific details or personal information will be redacted or otherwise hidden from public view. 127 | 128 | Beyond these broad guidelines, the details of administration are intentionally not specified in this APE, as experience shows they need to be adapted to changing circumstances and different funding sources. Specific policies on administration of funds live in `the finance section of the Astropy Project Github repo <https://github.com/astropy/astropy-project/tree/main/finance>`_. Changes to these policies follow the same Pull Request process as for code contributions in the Project. 129 | 130 | Acknowledgements 131 | ----------------- 132 | Thanks to NumPy for NEP 48 (https://numpy.org/neps/nep-0048-spending-project-funds.html) which is used as the basis for this document. 133 | 134 | 135 | Alternatives 136 | ============ 137 | 138 | The main alternative to having this APE is **not** having this APE and continue to follow a general agreement to work with funds along the lines described above, but without a written policy on the level of an APE. 139 | 140 | 141 | Decision rationale 142 | ================== 143 | 144 | Following discussion and comments on GitHub in the `APE pull request <https://github.com/astropy/astropy-APEs/pull/72>`_, this APE was accepted by the Coordinating Committee after community consensus was reached. 145 | -------------------------------------------------------------------------------- /APE15.rst: -------------------------------------------------------------------------------- 1 | An Updated Model for the Affiliated Package Ecosystem 2 | ----------------------------------------------------- 3 | 4 | author: Erik Tollerud 5 | 6 | date-created: 2018 February 22 7 | 8 | date-last-revised: 2018 May 14 9 | 10 | type: Process 11 | 12 | status: Accepted 13 | 14 | 15 | Abstract 16 | -------- 17 | 18 | A revision of the concept of Astropy-affiliated packages (hereafter "affiliated 19 | packages") is proposed. The idea of 20 | affiliated packages has continued mostly unaltered since the 21 | `original vision <http://docs.astropy.org/en/stable/development/vision.html>`_ 22 | for Astropy. However, time has revealed some potential improvements in how the 23 | ecosystem is structured and the review process for affiliated packages. Hence, 24 | this APE proposes to make three changes: 25 | 26 | * This APE creates the concept of "coordinated" packages - affiliated-like packages 27 | that are more closely overseen by the Astropy Project. While apparently 28 | significant, this change is mainly a recognition of de facto status that some 29 | affiliated packages already have. 30 | * The APE changes the review process to a more explicit criteria-based review 31 | process - again, this is primarily simply formalizing a process that was 32 | already in place in the coordination committee. 33 | * It lays out guidelines for the eventual goal of converting the 34 | affiliated package review process into a more "peer-review" style, where 35 | other members of the community can contribute to the review. 36 | 37 | 38 | Detailed description 39 | -------------------- 40 | 41 | Motivation 42 | ========== 43 | 44 | From its inception, Astropy has included an ecosystem of astronomy packages 45 | beyond the ``astropy`` core package (see the 46 | `original vision <http://docs.astropy.org/en/stable/development/vision.html>`_). 47 | The process by which these "affiliated" packages are reviewed and accepted has 48 | already changed a few times since the project's inception as the ecosystem has 49 | grown. However, the current state has several significant limitations. 50 | 51 | First, the affiliated packages include a mix of two different kinds of packages. 52 | Some packages have been developed as a fairly integrated effort alongside the 53 | core package. That is, feature planning took into account that these 54 | affiliated packages existed, and that important features would be implemented in 55 | them (e.g.,``photutils``, ``astroquery``, and ``specutils``). Thus, the 56 | continued existence of those packages is understood to be important for the 57 | Astropy project as a whole. Other affiliated 58 | packages (the majority) are developed more independently of the core package, 59 | which treat Astropy mainly as a base to build on, do not as strongly influence 60 | planning for the core package, and provide functionality that is not general 61 | enough to be considered part of the core requirements. It has proved 62 | difficult to design a development and review process that works effectively with 63 | *both* of these classes of packages. 64 | 65 | Second, the current review process has become somewhat opaque and rather awkward 66 | to keep organized. The coordination committee reviews the packages, which in 67 | practice nearly always receive a "provisional" designation, but then must be 68 | re-reviewed as a *whole* at some future date to become full affiliated 69 | packages. This provisional and full distinction is opaque and probably 70 | meaningless to the average user who just wants to know what a package does, and 71 | if it is any good. Additionally, while the review standards are stated on the 72 | affiliated package web page, some are subjective, allowing at least the 73 | possibility of biased or exclusionary behavior to sneak in with little recourse 74 | given the blunt nature of reject, provisional, and accept. 75 | 76 | Third, the coordination committee has found it difficult to keep up with the 77 | rate of affiliated package review (and re-review) requests. Many reviews 78 | languish for months before receiving even a provisional response, which can be 79 | disheartening and perceived by some as unwelcoming. This is primarily driven by 80 | the need to do a detailed and sometimes lengthy review of each package, even if 81 | that amounts to just following a checklist (see the section on the new review 82 | process for more details of the current process in practice). 83 | 84 | These issues were discussed at length by the coordination committee, and later 85 | at the 2017 Astropy coordination meeting. The new structure and processes 86 | described below were a product of a breakout session at the coordination 87 | meeting. This APE is thus an effort to formalize the ideas and describe how to 88 | implement them. 89 | 90 | Astropy "coordinated" and affiliated packages 91 | ============================================= 92 | 93 | The biggest-picture change is to split the currently single-category affiliated 94 | package ecosystem into two categories: "Astropy coordinated" and "Astropy 95 | affiliated" packages. This change is not as drastic as it sounds because it 96 | reflects the reality of the current situation. As outlined above, two such 97 | categories effectively exist already, and this APE simply formalizes this 98 | status. 99 | 100 | In practice, packages that remain as affiliated packages are essentially 101 | unaffected by this change. They continue to be managed/organized outside of the 102 | Astropy project core team and retain complete control of the repositories, code 103 | base, etc. By contrast, packages in the new "coordinated package" category will 104 | be treated as a core component of the Astropy project, with the project (via the 105 | coordinating committee) overseeing who is maintaining the package and taking 106 | responsibility for continued maintenance in the event the original author(s) no 107 | longer can do so. This means maintainer roles will be created for all the 108 | coordinated packages, and have status as 109 | `astropy roles <http://www.astropy.org/team.html>`_. 110 | 111 | This also means the development process for coordinated packages will generally 112 | track close to the core package model - e.g. they will usually follow the same 113 | package layout as the core package, APEs can be used for major 114 | decisions/standards, etc. Note that there is *not* a requirement that 115 | coordinated packages follow the release calendar for the core package (although 116 | they *may* do so if desired), as a more flexible release calendar is one of the 117 | major advantages to having a coordinated package not be in the core. 118 | 119 | One additional consequence of this change is that coordinated packages will live 120 | in the `astropy GitHub organization <https://github.com/astropy>`_ , for which 121 | the project has management power, while affiliated packages, in general, should 122 | *not*. To ease this process, however, any affiliated packages currently in the 123 | astropy organization can be "legacied-in". This means that they may remain in 124 | the GitHub organization even if they are not coordinated packages. Moving 125 | forward, though, new coordinated packages should be in the astropy organization 126 | and new affiliated packages should not be. 127 | 128 | Note, though, that affiliated packages are still considered a part of the 129 | "Astropy Project". That umbrella term applies to both coordinated and 130 | affiliated packages, as well as the overall community. 131 | 132 | 133 | A criteria-based package review process 134 | ======================================= 135 | 136 | This APE also proposes a second change to the review process, motivated by the 137 | concerns in the "Motivation" section above about the committee's review 138 | process. To counter these concerns, prior to this APE the coordination committee 139 | has informally adopted a checklist-based approach, checking if certain bullet 140 | points are met. While the formal criteria are listed in relatively vague terms 141 | on the 142 | `affiliated package web site <http://www.astropy.org/affiliated/index.html#affiliated-instructions>`_, 143 | the details of how those criteria are met 144 | have been fairly consistent using an unwritten but roughly standard list. In 145 | practice, in the past this has meant that affiliated packages are given 146 | "provisional" status if some of these items are met (a status that will no 147 | longer exist if this APE is accepted), or "full" if a larger set of them are. 148 | 149 | This de facto checklist implies that a more transparent and understandable 150 | policy would be to make this checklist formal and completely public, and use it 151 | directly as the formal review criteria - i.e., a rubric. These criteria will be 152 | laid out explicitly in a document (which has already been drafted - see the 153 | "Implementation" section below), with various levels defined for each criteria, 154 | generally corresponding to "good", "acceptable", and "unacceptable". This 155 | will make the review process much faster and (eventually) open the review 156 | process beyond the coordination committee, thereby addressing the concerns from 157 | the "Motivation" section. 158 | 159 | 160 | A community review process for affiliated packages 161 | ================================================== 162 | 163 | On a longer term horizon, a criteria-based review checklist will allow the 164 | review process to change to a more peer-review model. That is, instead of 165 | every package being vetted by the coordination committee, the committee can act 166 | more as editors, sometimes offering the review opportunity to other members of 167 | the Astropy community. The details of this process will not be set in stone by 168 | this APE, as they should be flexible enough to accomodate the experiences of the 169 | first rounds of reviewers. But this APE sets some initial steps (see 170 | "Implementation" below) and makes the general idea the eventual goal for the 171 | affiliated package review process. 172 | 173 | 174 | Branches and pull requests 175 | -------------------------- 176 | 177 | N/A 178 | 179 | 180 | Implementation 181 | -------------- 182 | 183 | The changes outlined in this APE will be implemented as three distinct efforts: 184 | 185 | * The new package review criteria have already been implemented by the 186 | coordination committee and reviewed by the community (Available in Google doc 187 | form `here <https://docs.google.com/document/d/15PJf2PROXMa7xwTDvWnjXB_9KNuO2Ia4_kkxJ7MPazE/edit?usp=sharing>`_. 188 | These criteria will need to be translated to a structured format to be shown 189 | on the affiliated 190 | package instructions section of the Astropy website. The affiliated package 191 | listing page will also need to be updated for these new criteria - a concept 192 | for this new site is `viewable here <assets/ape15_example_affilpkg_page.png>`_. 193 | * The Astropy web site (and to a lesser extent, affiliated package-related docs) 194 | will need to be updated to reflect the existence of the coordinated 195 | packages. Additionally, the roles web site will need to be populated with the 196 | new roles for the coordinated packages (and the role descriptions). This 197 | requires relatively little effort and can be completed by the Astropy web site 198 | maintainers upon acceptance of this APE. 199 | * A "peer-review" process will need to be set up to enable reviews by community 200 | members beyond the coordination committee. Such processes exist in 201 | other code contexts (e.g. the 202 | `Journal of Open Source Software <http://joss.theoj.org/>`_ and 203 | `Ropensci <https://ropensci.org/>`_) that may have tools that can be re-used 204 | to make the process smooth and efficient. Setting such tools up is likely to 205 | take some time, however, and needs to be adaptable enough to respond to the 206 | first few reviews. Hence this APE does not describe the details of this 207 | implementation - that is explicitly left to be developed by the Astropy 208 | community with the cooordination committee acting as "editors". However, 209 | this APE gives a suggested *starting* point: reviewers can be selected by 210 | the coordination committee from the pool of existing affiliated package 211 | authors or Astropy core team members. They receive a prompt to review a new 212 | package via email (along with the criteria described a above), and provide 213 | their reviews in the most straightforward manner feasible (email, a GitHub 214 | issue for the review, or similar). The coordination committee then performs 215 | the current process of posting the result of the review as issues in the 216 | affiliated package's repository for discussion. In the future some of these 217 | processes will likely be automated, but this simplest viable start point will 218 | provide a place to start. 219 | 220 | 221 | Backward compatibility 222 | ---------------------- 223 | 224 | While in the future this APE proposes that new affiliated packages not be in the 225 | Astropy GitHub organization, affiliated packages from *before* this APE that are 226 | in the Astropy organization will be left there unless specifically requested 227 | otherwise. 228 | 229 | 230 | Alternatives 231 | ------------ 232 | 233 | The main alternative is status quo. This does not address the concerns raised 234 | in the "motivation" section, but requires no additional effort. Other 235 | possibilities include any subset of the three changes outlined above. 236 | 237 | 238 | 239 | Decision rationale 240 | ------------------ 241 | 242 | The concept of this APE was discussed extensively by the Astropy Coordination 243 | Committee and approved. The APE text was discussed and accepted by multiple 244 | community stakeholders who have experience and project-level interest in 245 | affiliated packages. The APE was accepted on May 14, 2018. 246 | -------------------------------------------------------------------------------- /APE10.rst: -------------------------------------------------------------------------------- 1 | Roadmap for Python 3-only support 2 | --------------------------------- 3 | 4 | author: Tom Robitaille 5 | 6 | date-created: 2016 July 13 7 | 8 | date-last-revised: 2018 Mar 19 9 | 10 | date-accepted: 2016 August 22 11 | 12 | type: Process 13 | 14 | status: Accepted 15 | 16 | Abstract 17 | -------- 18 | 19 | The purpose of this APE is to set out a roadmap for future releases of the 20 | Astropy core package, in particular to identify the process for switching to 21 | releases that support only Python 3 and ending support for Python 2.7 in 22 | December 2019. 23 | 24 | Detailed description 25 | -------------------- 26 | 27 | Python 2.7 will cease to be supported on January 1st 2020. As a result, a 28 | number of scientific Python projects, including IPython, Matplotlib, Sympy, 29 | and others have now signed the `Python 3 statement 30 | <https://python3statement.github.io>`_ which states that scientific Python 31 | packages will drop support for Python 2 no later than 2020. For instance, the 32 | IPython developers have now released `IPython 5.0 33 | <http://blog.jupyter.org/2016/07/08/ipython-5-0-released/>`_, which will be the 34 | last major release to support Python 2. This APE proposes that the Astropy 35 | project should also sign the Python 3 statement, and provides a 36 | target roadmap for future releases. 37 | 38 | It is important to note that ending support for Python 2 by the end of 2019 39 | does not mean that Python 2 users will no longer be able to install or use 40 | Astropy from 2020 onwards. Existing Astropy releases will remain available 41 | indefinitely. Instead, the proposed roadmap means that only new features 42 | developed after mid-2017 will be available only to Python 3 users, and that 43 | we will not fix bugs for Python 2 after 2019. 44 | 45 | Roadmap 46 | ------- 47 | 48 | At the time of writing, the following releases have been made since (and including) v1.0: 49 | 50 | ========== ====================== ========================================== 51 | Version Release date Notes 52 | ========== ====================== ========================================== 53 | v1.0 - LTS February 19, 2015 Supported with bug fixes until June 2017 54 | v1.1 December 11, 2015 55 | v1.2 June 23, 2016 56 | ========== ====================== ========================================== 57 | 58 | The proposed roadmap for future releases is the following: 59 | 60 | ========== ====================== ========================================== 61 | Version Planned release date Notes 62 | ========== ====================== ========================================== 63 | v1.3 December 2016 64 | v2.0 - LTS June 2017 Supported with bug fixes until end of 2019 65 | v3.0 December 2017 First release to support only Python 3+ 66 | v3.1 November 2018 (Gap due to changes in release phase) 67 | v3.2 May 2019 68 | v4.0 - LTS November 2019 Supported with bug fixes for two years 69 | ========== ====================== ========================================== 70 | 71 | Note that v1.0, v2.0, and v4.0 are marked as long-term support releases (LTS), 72 | which means that they are typically supported with bug fixes for at least two 73 | years. The above roadmap proposes that the last Python 2-compatible release will 74 | be v2.0, which we will support with bug fixes until the end of 2019. In the mean 75 | time, v3.0, to be released in December 2017, will be the first release that does 76 | not support Python 2. 77 | 78 | Note that v3.0 would not be an LTS release, since we would 79 | otherwise need to maintain two LTS releases in parallel. This is a one-off 80 | deviation from the version numbering policy outlined in `APE 2 81 | <https://github.com/astropy/astropy-APEs/blob/main/APE2.rst>`_, and subsequent releases will follow the pattern established by APE 2. 82 | 83 | In order to decide which Python 3.x releases to support, we would follow the 84 | Python support calendar: the Python 3.x versions supported by an Astropy 85 | release would be the versions supported by the Python developers. Using the 86 | `current Python support calendar 87 | <https://docs.python.org/devguide/#status-of-python-branches>`_, this would 88 | mean that the following Astropy versions would support the following Python 3.x 89 | versions: 90 | 91 | ========== =============================== 92 | Version Minimum required Python version 93 | ========== =============================== 94 | v2.0 - LTS Python 2.7 or Python 3.4+ 95 | v3.0 Python 3.5+ 96 | v3.1 Python 3.5+ 97 | v3.2 Python 3.5+ 98 | v3.3 Python 3.6+ 99 | v4.0 - LTS Python 3.6+ 100 | ========== =============================== 101 | 102 | Implementation 103 | -------------- 104 | 105 | In practice, dropping Python 2.7 support will involve, amongst other things: 106 | 107 | * Removing the bundled `six <https://pythonhosted.org/six/>`_ package, and 108 | updating all the code that used ``astropy.extern.six`` to use the Python 3 109 | syntax directly. 110 | * Removing Python 2.7 from the configuration for the continuous integration 111 | services (such as Travis and AppVeyor at the time of writing) 112 | * Making sure that the documentation on ReadTheDocs is built using Python 3.x 113 | * Removing any backports that may exist for Python 2.7 compatibility 114 | * Updating the documentation to remove any mention of Python 2.7, as well as 115 | making sure that Python 3.x is not presented as a special case. 116 | * Removing Python 2 and 2.7 from the PyPI classifiers 117 | 118 | Challenges 119 | ---------- 120 | 121 | There are a few challenges to consider with the plan outlined above: 122 | 123 | Developers will need to use Python 3 124 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 125 | 126 | Once we make the switch to a Python 3-only code base, between v2.0 and v3.0, 127 | people will no longer be able to contribute code to Astropy without using 128 | Python 3. This means that even though users can in theory wait until the end of 129 | 2019 to switch to Python 3, developers and contributors will need to make the 130 | switch (at least for their Astropy development environment) as soon as 2017. 131 | 132 | Packages on PyPI 133 | ^^^^^^^^^^^^^^^^^ 134 | 135 | If we upload Astropy v3.0 to PyPI, and it does not support Python 2, then if 136 | Python 2.7 users try and pip install Astropy, the install or import will fail 137 | (with the current version of pip). This is because PyPI does not compare the 138 | current Python version to the PyPI meta-data for the package (which may 139 | indicate for example whether the package is Python 2-compatible), and will 140 | download Astropy v3.0 regardless of the active Python version. 141 | 142 | `PEP 345 <https://www.python.org/dev/peps/pep-0345/#requires-python>`_ defines 143 | the ``Requires-Python`` metadata field for Python packages, which can be used 144 | to explicitly specify which Python versions a package is compatible with. 145 | Support for this was added to `setuptools v24.2 146 | <https://github.com/pypa/setuptools/blob/master/CHANGES.rst#v2420>`_ via the 147 | ``python_requires`` keyword argument to ``setup``:: 148 | 149 | setup(name=..., 150 | version=..., 151 | python_requires='>=2.7', 152 | ... 153 | ) 154 | 155 | This metadata is not yet exposed by the Python Package Index (PyPI) but it is 156 | being worked on. Finally, support for this `was implemented into pip 157 | <https://github.com/pypa/pip/pull/3877>`_ on August 11th 2016, which 158 | means that versions of pip released after this date will support this, 159 | and will have been around in the 160 | wild for a little more than a year before Astropy v3.0 (the first release 161 | incompatible with Python 2) is out. Nevertheless, a fraction of Python 2 users 162 | may still be using old versions of pip by then and run the risk of installing a 163 | version of Astropy not compatible with Python 2. 164 | 165 | One possible solution is described in the 166 | `Jupyter roadmap 167 | <https://github.com/jupyter/roadmap/blob/master/accepted/migration-to-python-3-only.md#multiple-source-distributions>`_, 168 | which is to upload tar files that have a ``-py3.x`` suffix (one file per Python 169 | 3.x version, e.g. ``astropy-3.0-py3.5.tar.gz``). We could choose to do this if 170 | we believe that the fraction of users with old pip installations is too high. 171 | 172 | We will re-assess this in the weeks coming up to the v3.0 release. If we 173 | believe that enough users have a recent enough pip installation, then we simply 174 | need to include ``python_requires='>=3'`` in the metadata for the v3.0 release. 175 | Otherwise, we can additionally make sure we add the ``-py3.x`` suffix to the 176 | files we upload to PyPI. By that time, the IPython 6.0 release (incompatible 177 | with Python 2) will be out, so we will also have the benefit of seeing how that 178 | release went and whether it caused any issues for users. 179 | 180 | Note that this will not be an issue for users using other package managers 181 | (either built in to their operating system, or e.g. conda), since these are 182 | normally always explicit about Python version requirements. 183 | 184 | Current pull requests 185 | ^^^^^^^^^^^^^^^^^^^^^ 186 | 187 | The pull request(s) that will remove Python 2 compatibility may create 188 | conflicts with many of the open pull requests at the time - therefore it would 189 | make sense to try and make sure the number of open pull requests at the time is 190 | down to a minimum. This may be easiest to do straight after the v2.0 release, 191 | according to the calendar above. 192 | 193 | Maintenance of v2.0 LTS 194 | ^^^^^^^^^^^^^^^^^^^^^^^ 195 | 196 | According to the proposed release calendar, the v2.0 LTS release would be 197 | supported until the end of 2019. However, soon after the v2.0 release, the 198 | Astropy code will no longer be Python 2-compatible. As a result, pull requests 199 | that subsequently fix bugs in Astropy in main will in some cases not be 200 | easily backportable. One possible solution would be to also accept bug fixes in 201 | the v2.0.x branch and forward-port them to main. 202 | 203 | Benefits 204 | -------- 205 | 206 | There are several benefits to following the plan proposed above: 207 | 208 | * Maintaining a Python 3-only code base will be significantly easier, because 209 | developers won't have to know both 2.x and 3.x (and the `six 210 | <https://pypi.python.org/pypi/six>`__ package). 211 | 212 | * We will be able to start using Python 3-only features internally, including 213 | for example function annotations (e.g., for units), matrix multiplication 214 | (e.g., for coordinates; note that this will only be possible with Python 3.5+, 215 | which in part drives that as minimum version for astropy 3.0) 216 | 217 | * Since we will need to keep adding Python 3.x releases to the continuous 218 | integration over the coming years, we will at least be able to remove Python 219 | 2.7, making sure that the number of builds does not grow out of control. 220 | 221 | * Since developers/contributors will need to switch to using Python 3 for 222 | Astropy development, we will be training more people to do this transition, 223 | who will then be able to help their colleagues also make the transition. 224 | 225 | Alternatives 226 | ------------ 227 | 228 | An alternative plan would be to continue making major releases that support 229 | Python 2 until 2019, for example: 230 | 231 | ========== ====================== ========================================== 232 | Version Planned release date Notes 233 | ========== ====================== ========================================== 234 | v1.3 December 2016 235 | v2.0 - LTS June 2017 Supported with bug fixes until end of 2019 236 | v2.1 December 2017 237 | v2.2 June 2018 238 | v2.3 December 2018 239 | v2.4 June 2019 Last release to support Python 2.7 240 | v3.0 - LTS December 2019 Supported with bug fixes for two years 241 | ========== ====================== ========================================== 242 | 243 | This would allow more time for the PyPI limitations mentioned above to be 244 | resolved and more time for developers to make the transition to Python 3. On 245 | the other hand, if we want users to all be using Python 3 by 2020, then it does 246 | not make sense to delay the proposed release plan in this way, since the 247 | developers and infrastructure need to be ready for Python 3-only releases 248 | before the users are. This APE therefore does not recommend this alternative. 249 | 250 | Decision rationale 251 | ------------------ 252 | 253 | This APE was advertised on the astropy-dev mailing list, the "Python users in 254 | Astronomy" Facebook group, and twitter, with a roughly 3 week comment period. 255 | No objections were raised by the community, and the proposal received positive 256 | feedback (some of which led to minor adjustments particularly regarding the PyPI 257 | issues). Because of this apparent support by the community and agreement from 258 | the coordination committee with the goals of this APE, it was accepted Aug 22, 259 | 2016. 260 | 261 | Updates after acceptance of the APE 262 | ----------------------------------- 263 | In further discussions (https://github.com/astropy/astropy/issues/6000), it was 264 | noted that one should not aim to support versions of Python that will reach end 265 | of life during the time an astropy version is supported. In particular, Python 266 | 3.3 would reach end-of-life only 3 months after the nominal astropy 2.0 release. 267 | Hence, all minimum versions of Python 3 were increased by one minor version. 268 | 269 | After v3.0, a decision was made to adjust the release calendar by +3 months. 270 | The table above has been updated appropriately. v3.3 was also removed on this 271 | basis, as the timing for the end of Python 2.x support is better aligned with an 272 | end-of-2019 4.0 release. 273 | 274 | This APE is superceded by `APE18 <APE18.rst>`_, which contains the version 275 | support plan for versions from 4.0 on. 276 | -------------------------------------------------------------------------------- /APE1.rst: -------------------------------------------------------------------------------- 1 | APE Purpose and Process 2 | ----------------------- 3 | 4 | author: Perry Greenfield, Lia Corrales, Thomas Robitaille, Erik Tollerud, Pey Lian Lim 5 | 6 | date-created: 2013 October 11 7 | 8 | date-last-revised: 2025 October 27 9 | 10 | date-accepted: 2013 November 8 11 | 12 | type: Process 13 | 14 | status: Accepted 15 | 16 | revised-by: 17 | 18 | * Lia Corrales, Thomas Robitaille, Erik Tollerud - 2021 February 26 - Added APE modification process 19 | * Pey Lian Lim - 2024 February 5 - Added APE numbering process 20 | * Thomas Robitaille, Pey Lian Lim - 2025 Oct 27 - Modify APE process to merge APEs that are in the discussion stage 21 | 22 | Abstract 23 | -------- 24 | 25 | APE stands for Astropy Proposal for Enhancement. An APE is a design document 26 | providing information to the Astropy community, or describing a new feature 27 | for Astropy or its processes or environment. The APE should provide a concise 28 | technical specification of the feature and a rationale for the feature. 29 | 30 | We intend APEs to be the primary mechanisms for proposing major new features, 31 | for collecting community input on an issue, and for documenting the design 32 | decisions that have gone into Astropy. The APE author is responsible for 33 | building consensus within the community and documenting dissenting opinions. 34 | 35 | Because the APEs are maintained as text files in a versioned repository, their revision 36 | history is the historical record of the feature proposal. 37 | 38 | Detailed description 39 | -------------------- 40 | 41 | APE Types 42 | ......... 43 | 44 | Chimp, Gorilla, Bonobo... 45 | 46 | But more seriously. 47 | 48 | There are three kinds of APE: 49 | 50 | * "Standard Track": A "Standard Track" APE describes a new feature or implementation for 51 | Astropy. It may also describe an interoperability standard that will be 52 | supported in current Astropy versions before a subsequent APE adds the 53 | feature in the future. 54 | 55 | * "Informational": An "Informational" APE describes a Astropy design issue, or provides general 56 | guidelines or information to the Astropy community, but does not propose a new 57 | feature. Informational APEs do not necessarily represent an Astropy community 58 | consensus or recommendation, so users and implementers are free to ignore 59 | Informational APEs or follow their advice. We await a Zen of Astropy APE. 60 | 61 | * "Process": A "Process" APE describes a process surrounding Astropy, or proposes a change 62 | to (or an event in) a process. Process APEs are like Standard Track APEs but 63 | apply to areas other than the Astropy package itself. They may propose an 64 | implementation, but not to Astropy's codebase; they often require community 65 | consensus; unlike Informational APEs, they are more than recommendations, and 66 | users are typically not free to ignore them. Examples include procedures, 67 | guidelines, changes to the decision-making process, and changes to the tools 68 | or environment used in Astropy development. Any meta-APE is also considered a 69 | Process APE. 70 | 71 | Submitting an APE 72 | ................. 73 | 74 | The APE process begins with a new idea for Astropy. It is highly recommended 75 | that a single APE contain a single key proposal or new idea. Small 76 | enhancements or patches often don't need a APE and can be injected into the 77 | Astropy development workflow with a patch submission to the Astropy issue 78 | tracker. The more focused the APE, the more successful it tends to be. If in 79 | doubt, split your APE into several well-focused ones. 80 | 81 | New ideas for the following APEs should always be submitted as modifications 82 | to the respective APE, never as a new APE: 83 | 84 | * `APE 0: The Astropy Project Governance Charter <https://github.com/astropy/astropy-APEs/blob/main/APE0.rst>`_ 85 | * `APE 1: APE Purpose and Process <https://github.com/astropy/astropy-APEs/blob/main/APE1.rst>`_ 86 | 87 | Each APE must have a champion -- someone who writes the APE using the style 88 | and format described below, shepherds the discussions in the appropriate 89 | forums, and attempts to build community consensus around the idea. The APE 90 | champion (a.k.a. Author) should first attempt to ascertain whether the idea is 91 | APE-plicable. Posting to the `astropy-dev <https://groups.google.com/g/astropy-dev>`_ 92 | mailing list is the best way to go about doing this. 93 | 94 | Note that there is not much point to submitting new APEs unless someone or some 95 | group has signed up to implement it should the APE be accepted 96 | (typically this would involve the author or authors of the APE). Just issuing 97 | an APE in order to spur others to do work is not going to be received 98 | well, due to the open-source nature of the Astropy Project. 99 | 100 | While attaching additional files to an APE is discouraged, if there are no other 101 | options, these files could go under the `assets <https://github.com/astropy/astropy-APEs/tree/main/assets>`_ 102 | sub-directory, with the filenames each starting with `ape<n>_` for clarity. 103 | Large files should be hosted externally and linked to the APE instead. 104 | 105 | Following a discussion on `astropy-dev <https://groups.google.com/g/astropy-dev>`_, 106 | the proposal should be submitted as a 107 | pull request to astropy-APEs with the name `APE<n>.rst` where `<n>` is an 108 | appropriately assigned number (i.e., not already an accepted/proposed/rejected APE). 109 | The draft must use the `APEtemplate.rst file`. Once the pull request is open, 110 | the `@astropy/ape-editor-team <https://github.com/orgs/astropy/teams/ape-editor-team>`_ 111 | should be mentioned in the pull request, and 112 | the APE will then undergo an editorial review which focuses on wording, 113 | spelling, grammar, and clarity, but does not focus on the substance of the APE. 114 | The editorial process ideally should not take longer than a month. 115 | Once the APE editors are happy with the APE, they will: 116 | 117 | #. Make sure the APE has a status of "Discussion". 118 | 119 | #. If additional files are attached with the APE in the PR, make sure the 120 | file sizes are reasonable. 121 | 122 | #. If not already, add a new entry for the APE under "APEs" table in `README.rst` 123 | with the: 124 | 125 | * Finalized number (the next available one in the `main` branch 126 | of the repository) 127 | * Title (with link to the APE RST file) 128 | * Last revised date (set to day of merging) 129 | * Status (set to "Discussion") 130 | * DOI left blank. 131 | 132 | #. Preview the `README.rst` updates and test the new link to make sure they are all correct. 133 | 134 | #. Merge the pull request. 135 | 136 | #. Send an email to `astropy-dev <https://groups.google.com/g/astropy-dev>`_ 137 | with a link to the APE and a short summary to kick off the discssion period. 138 | Emphasize that the proposal should be discussed on the mailing list, not the 139 | pull request. 140 | 141 | As the discussion proceeds, the authors can choose to update the APE at any point via 142 | new pull requests, which are as before, subject to editorial review before merging. 143 | If needed, editors should continue to stress that discussions about the substance of the APE 144 | should be kept on the mailing list as much as possible. 145 | 146 | Standard Track APEs consist of two parts: 147 | 148 | * a design document and 149 | * a reference implementation. 150 | 151 | For serious consideration, it is generally recommended that at least a prototype 152 | implementation be co-developed with the APE, as ideas that sound good in 153 | principle sometimes turn out to be impractical when subjected to the test of 154 | implementation. This is not required when too onerous, but some indication of 155 | implementation practicality is highly recommended by actual code. The best way 156 | to provide that code is via a GitHub draft pull request to the affected Astropy repository 157 | (usually `astropy/astropy <https://github.com/astropy/astropy>`_), as 158 | appropriate. 159 | 160 | APE Review 161 | .......... 162 | 163 | Once the authors feel that they have reached as close to a consensus as they 164 | will get (i.e., the community discussion on the APE has wound 165 | down), they can contact the Coordination Committee, who are 166 | responsible for making a final decision. The decision can be to: 167 | 168 | * accept the APE, 169 | * reject the APE, or 170 | * suggest to the authors to continue the discussion before re-submitting the APE for a decision. 171 | 172 | One of the Coordination Committee members should then: 173 | 174 | #. Fill in the "Decision rationale" section of the APE with a description of why 175 | the status was chosen, including a summary of the community's discussion as relevant. 176 | #. Update the `date-last-revised` to the day of merging and `status` to 177 | one of the options listed under "APE Status" section below. 178 | #. Update the "Date (last revised)" and "Status" info for the relevant row 179 | in the "APEs" table in `README.txt`. 180 | #. Leave a brief comment in the PR indicating the result. 181 | #. Merge the PR with the above changes. 182 | #. If the APE was accepted, then see `README.rst` for archiving instructions. 183 | #. Send an email to `astropy-dev <https://groups.google.com/g/astropy-dev>`_ 184 | announcing the new status. In general, this should just point to the 185 | APE rather than providing additional decision rationale. 186 | 187 | APE Status 188 | .......... 189 | 190 | For all the statuses below, the "Decision rationale" section should be updated 191 | to reflect the overall community opinion, if applicable, so that 192 | new community members can understand why such decision was made. 193 | 194 | An APE's status can be: 195 | 196 | * "Discussion": New APE pull requests should always start in this status. This 197 | means the APE is currently being considered and a decision has not been made 198 | regarding what should be done. 199 | 200 | * "Accepted": Generally, an implementation is expected before a standard-track APE 201 | can be considered fully accepted; For proposals that require extensive work that 202 | few are willing to perform without some assurance it will be accepted, provisional 203 | acceptance is an option (the provision could be clarified under "Decision rationale"). 204 | For standard track, any code implementation of this APE should 205 | clearly link back to the APE document in this repository. For completeness, 206 | the APE could also be updated later to add any additional implementation 207 | pull requests to its "Implementation" section. 208 | 209 | * "Rejected" 210 | 211 | * "Withdrawn": If the APE authors no longer feel that their proposal is feasible, 212 | they can choose to withdraw the APE. Reason for withdrawal should be clearly 213 | stated under "Decision rationale" section. A withdrawn APE is in effect equivalent 214 | to a Rejected APE in that it will not be implemented. 215 | 216 | * "Superseded": An APE that was accepted but has since become irrelevant 217 | or entirely replaced by some other APE or event can be given this status. 218 | Such a status change may not require the regular APE modification process if 219 | whatever caused this status already followed a discussion process at least 220 | as long as the APE discussion process (at least 2 weeks as per the comment 221 | period Astropy usually follow in other processes). 222 | 223 | * "Abandoned": If an APE is present in the repository but has been labelled 224 | as "Discussion" for at least a year without any sign of further discussion 225 | or progress, the APE can be marked as "Abandoned" (after reaching out to the 226 | authors to check that they are not able to continue working on it). An 227 | abandoned APE can be picked up at any time by other members of the community, 228 | and the status changed back to "Discussion". 229 | 230 | Modifications to an existing APE 231 | ................................ 232 | 233 | An existing APE can be modified, with one person acting as champion for the APE update. 234 | Example cases: 235 | 236 | * Modifying the contents (including superseding) of an accepted APE. 237 | * Withdrawing an APE under discussion. 238 | * Reviving an abandoned APE. 239 | 240 | Similar to the APE development procedure, the update champion should discuss 241 | their desired changes with the Astropy community through the 242 | `astropy-dev <https://groups.google.com/g/astropy-dev>`_ mailing list. 243 | Following a discussion on mailing list, the APE update should be submitted 244 | as a pull request, which should be advertised on the relevant mailing list thread. 245 | After the community discussion period and the changes are finalized, 246 | the Coordination Committee would follow a similar procedure in "APE Review" 247 | should they decide to merge the pull request. 248 | 249 | The APE should be modified directly without annotating the changes. If not 250 | already listed, the update champions can add themselves to the author list for 251 | the APE being updated. A new section, **Previous versions of this APE** should 252 | be added to the bottom of the APE (if not already there) with a bullet point 253 | list of dates which link to the DOI of the previous versions of APEs as well as 254 | links to the merge commits of the APEs. The `date-last-revised` header for the APE 255 | should be updated, and a `revised-by` header should be added for each 256 | update (see the first `revised-by` section of *this* APE for a formatting 257 | example). 258 | 259 | Branches and pull requests 260 | -------------------------- 261 | 262 | N/A 263 | 264 | Implementation 265 | -------------- 266 | 267 | N/A 268 | 269 | Backward compatibility 270 | ---------------------- 271 | 272 | N/A 273 | 274 | Alternatives 275 | ------------ 276 | 277 | N/A 278 | 279 | Decision rationale 280 | ------------------ 281 | 282 | The Coordinating Committee thought it was a honking great idea. 283 | 284 | Previous versions of this APE 285 | ----------------------------- 286 | 287 | * 2013-11-08 [`DOI <http://doi.org/10.5281/zenodo.1043886>`_] [`GitHub <https://github.com/astropy/astropy-APEs/blob/42951733ac42c0ea178d8df30705274a43c93091/APE1.rst>`_] 288 | * 2021-03-09 [`DOI <https://doi.org/10.5281/zenodo.10805921>`_] [`GitHub <https://github.com/astropy/astropy-APEs/blob/9896678a4a8dc7e5aeedd0230b37816953dbf800/APE1.rst>`_] 289 | -------------------------------------------------------------------------------- /APE5.rst: -------------------------------------------------------------------------------- 1 | Coordinates Subpackage Plan 2 | --------------------------- 3 | 4 | authors: Erik Tollerud, Adrian Price-Whelan, Thomas Aldcroft, Thomas Robitalle 5 | 6 | date-created: 2014 January 22 7 | 8 | date-last-revised: 2014 March 9 9 | 10 | date-accepted: 2014 March 9 11 | 12 | type: Standard Track 13 | 14 | status: Accepted 15 | 16 | Abstract 17 | -------- 18 | 19 | This APE provides a high-level overview of the planned architecture of the 20 | `astropy.coordinates` subpackage for v0.4 and beyond. It is a synthesis of a 21 | feedback received from previous versions of the `coordinates` subpackage and 22 | related discussion on astropy-dev and coordinates-related pull requests. The 23 | base idea is to split the current coordinate class hierarchy into three sets of 24 | classes. data "representation" classes, "low-level" classes that serve both as 25 | descriptions of coordinate frames *and* containers around the data, and a "high- 26 | level" class that uses the low-level classes for actual functionality, but 27 | provides a more user-friendly interface. The transformation framework will 28 | remain essentially unchanged, except that it will operate specifically on the 29 | low-level classes. 30 | 31 | 32 | Detailed description 33 | -------------------- 34 | 35 | Background/Terminology 36 | ^^^^^^^^^^^^^^^^^^^^^^ 37 | 38 | Discussion surrounding the current `coordinates` API have revealed the 39 | importance of shared terminology, as some of the crucial concepts necessary for 40 | designing a clean API involve language with subtle (but important) distinctions. 41 | Chief among these is the concept of a "coordinate system." To some members of 42 | the community, "coordinate system" means the *representation* of a point in 43 | space, e.g., "Cartesian coordinate system" is different from "Spherical polar 44 | coordinate system". Another use of "coordinate system" is to mean a unique 45 | reference frame with a particular set of reference points, e.g., "the ICRS 46 | coordinate system" or the "J2000 coordinate system." This second meaning is 47 | further complicated by the fact that such systems use quite different ways of 48 | defining a frame. 49 | 50 | Because of the likelihood of confusion between theses meanings of "coordinate 51 | system", this APE (and the associated coordinates API) will avoid this 52 | terminology, and instead adopt the following meanings (loosely inspired by the 53 | IAU2000 resolutions on celestial coordinate systems): 54 | 55 | * A "Coordinate Representation" is a particular way of describing a unique 56 | point in a vector space. (Here, this means three-dimensional space, but future 57 | extensions might have different dimensionality, particularly if relativistic 58 | effects are desired.) Examples include Cartesian coordinates, cylindrical 59 | polar, or latitude/longitude spherical polar coordinates. 60 | 61 | * A "Reference System" is a scheme for orienting points in a space and 62 | describing how they transforms to other systems.Examples include the ICRS, 63 | equatorial coordinates with mean equinox, or the WGS84 geoid for 64 | latitude/longitude on the Earth. 65 | 66 | * A "Coordinate Frame", "Reference Frame", or just "Frame" is a specific 67 | realization of a reference system - e.g., the ICRF, or J2000 equatorial 68 | coordinates.For some systems, there may be only one meaningful frame, while 69 | others may have many different frames (differentiated by something like a 70 | different equinox, or a different set of reference points). 71 | 72 | * A "Coordinate" is a combination of all of the above that specifies a unique 73 | point. 74 | 75 | This terminology is used by this APE and the associated API document. 76 | 77 | 78 | Problems with current `coordinates` subpackage 79 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 80 | 81 | The current (Astropy v0.2/0.3) `coordinates` subpackage effectively provides 82 | classes to represent common celestial coordinates and transformations between 83 | them. It also makes all of the machinery available for subclassing, allowing 84 | users to define their own coordinate frames and provide transformations to and 85 | from the built-in frames. 86 | 87 | In this scheme (motivated mainly by ease of implementation), coordinate 88 | representations, reference systems, and frames are mixed together in classes 89 | just called e.g., `ICRS` or `FK5`. However, comments following implementation of 90 | this system has revealed a number of shortcomings regarding this architecture: 91 | 92 | * Quite a bit of boilerplate copy-and-pasting is necessary to define a new 93 | reference systems. While some minor modifications might simplify this, it is 94 | difficult to ensure this won't have unintended consequences due to the lack of 95 | clear separation-of-concerns. 96 | 97 | * There is no reasonable way to define new representations.Some reference 98 | systems may be more naturally expressed in e.g. cylindrical coordinates, but 99 | defining such a class would require re-implementing nearly everything in the 100 | current coordinate classes (which are based on spherical polar coordinates). 101 | 102 | * There has been a large amount of debate and (in some cases, wasted) 103 | implementation effort on parsing and formatting coordinate strings.Similarly, 104 | parsing of units for coordinate inputs has been subject to debate, change, and 105 | confusion. 106 | 107 | * In the current system, one person is often a bottleneck for development of the 108 | subpackage because the single class makes it hard to keep a change from not 109 | cascading to other parts of the package. 110 | 111 | The description below and the API associated with this APE present an approach 112 | that addresses these concerns by separating `coordinates` functionality into a 113 | few different class hierarchies, and using it to re-organize (without wasting) 114 | the existing code base. 115 | 116 | 117 | High-level outline of new `coordinates` classes 118 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 119 | 120 | The `Angle`, `Latitude`, `Longitude`, and `Distance` classes will be retained, 121 | as well as the current transformations and associated architecture. This 122 | represents the bulk of the work that has thus far gone into `coordinates`. The 123 | existing coordinates classes will be sub-divided into three pieces: 124 | 125 | 1. Classes for coordinate representations, subclasses of a new abstract class 126 | `CoordinateRepresentation`. These classes store the actual spatial 127 | information of a coordinate (or coordinates, if they are arrays) as 128 | `Quantity` subclasses (e.g., `Angle`, `Distance`). These classes also provide 129 | the methods necessary to transform points from one representation to another, 130 | and a single method for invoking these transformations. 131 | 132 | 2. A class hierarchy for specifying reference systems and frames (their 133 | realization) - the "low-level" coordinate classes. These will be subclasses 134 | of `CoordinateFrame`. While it may at first make sense to separate specifying 135 | frames from the actual representations in those frames, doing so would result 136 | in a large amount of code duplication, as each system would require two very 137 | similar classes for specifying a frame and for the objects containing the 138 | representations. Hence, here we combine them into a single class that is a 139 | pure frame if it does not have position data, but if it does, is actually a 140 | coordinate. 141 | 142 | 3. Classes that are containers for the low-level classes, but provide additional 143 | functionality to make them easier to use. For example, if created in the FK5 144 | system, it would contain `equinox` information even if transformed into ICRS 145 | (which has no concept of equinox), allowing the high-level class to round- 146 | trip transformations. Initially this will be a single class, `SkyCoordinate`, 147 | but might later be expanded to other coordinates that are spatial, but not 148 | necessarily celestial (e.g., locations on the surface of the Earth). 149 | 150 | By separating the coordinates into these three domains, the aforementioned 151 | problems will either be addressed, or made easier to address by dividing the 152 | code more logically. Additionally, the new API takes advantage of features that 153 | were not available when `coordinates` was first designed to simplify passing in 154 | units (specifically, the use of `Quantity` objects.) 155 | 156 | 157 | Connection to "generalized WCS" 158 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 159 | 160 | A parallel effort currently being pursued by Astropy is an attempt to 161 | create a "generalized WCS" subpackage. While still in the planning 162 | stages, the basic idea is that this would provide functionality similar 163 | to the existing FITS-based World Coordinate System (WCS), but with a 164 | generalized Python framework that will help make it easier to map 165 | "pixel"/detector coordinates to many different kinds of real-world 166 | coordinate frames. 167 | 168 | This APE is not intended to replace or duplicate that functionality. 169 | Rather, the coordinates API is intended to provide an interface with 170 | convenience features that users will find straightforward to work with. 171 | At the same time, the cleaner separation of functionality proposed by 172 | this APE should make it easier for the generalized WCS subpackage to re- 173 | use coordinates functionality. 174 | 175 | 176 | Branches and pull requests 177 | -------------------------- 178 | 179 | `PR #12 <http://github.com/astropy/astropy-api/pull/12>`_ in the astropy-api 180 | repository contains the file `coordinates_api_2.py` that outlines the actual 181 | API, and is thus a crucial part of this APE. The PR will be merged if this APE 182 | is accepted. 183 | 184 | 185 | Implementation 186 | -------------- 187 | 188 | This APE can be implemented in separate parts: 189 | 190 | * The low-level classes can be implemented by writing a representation class 191 | for spherical and Cartesian coordinates, and then adapting the existing 192 | coordinate systems to the new hierarchy. Much of the algorithmic side of the 193 | code should be reusable, particularly the transformation functions themselves. 194 | This could possibly be done by two people (one working on the representations, 195 | another on the frame classes), but it may be more efficient for a single 196 | developer to do this. 197 | 198 | * In parallel, the high-level classes can be developed, ideally by an additional 199 | developer. This interface with the low-level classes is intended to separate 200 | concerns, and is specified in the API document (although some details will no 201 | doubt need to be worked out when the coding actually begins). 202 | 203 | The above will complete the framework described in this APE. The immediate 204 | follow-on work that should be enabled by this will be: 205 | 206 | * Defining locations on the Earth as part of the transform hierarchy. Such 207 | location objects will be useful in `astropy.time`, as well as a planned 208 | subpackage for storing observatory locations. 209 | 210 | * Implementing the full ICRS <-> Alt/Az transformation stack, following the 211 | IAU2000 algorithms, mostly as implemented in ERFA.This will be simplified 212 | greatly by the re-organization described in this APE, as it will be easier to 213 | define new reference system/frames, and the IAU2000 stack requires a whole 214 | series of such intermediate systems. 215 | 216 | 217 | Backward compatibility 218 | ---------------------- 219 | 220 | This will certainly break backwards-compatibility for anything that relies in 221 | any way on the internal representations of coordinates in the current version. 222 | The "low-level" API will resemble the current coordinates API, but likely with 223 | some backwards-incompatible changes. Where possible, we will attempt to keep 224 | older interfaces and deprecate them for at least one more version. This 225 | breaking of backwards compatibility is acceptable, because it was in the 226 | original plan to attempt a few iterations of coordinates, and there is still a 227 | big warning that it may change in the future in the current documentation. 228 | 229 | 230 | Alternatives 231 | ------------ 232 | 233 | A number of different alternatives have been discussed or considered. Below I 234 | address a few that led to this APE due to being imperfect solutions, but with 235 | some valuable parts. 236 | 237 | * Keep the current system. This is problematic due to reasons described in the 238 | description section above. 239 | 240 | * Implement a similar stack, but don't store the representations in the frame 241 | classes, instead provide classes for frames and representations, and combine 242 | them only in "high-level" classes.This is a possibility, but would require 243 | quite a bit of code in the high-level class customized for particular low- 244 | level classes.This is mainly because different reference systems have 245 | different "preferred" representations (e.g., equatorial systems are 246 | traditionally represented in RA/Dec and possibly distance, not Cartesian 247 | coordinates). without a way for the frame to convert to its preferred system, 248 | there's no easy way to delegate operations like generating reasonable-looking 249 | strings or `__repr__`. 250 | 251 | * Similar stack as this APE, but have the frames without data be a separate 252 | class from those with data.This would necessitate making two copies of every 253 | class, which would either waste effort or require confusing "magic" with 254 | metaclasses. Either way, the proposed APE will probably result in easier-to- 255 | understand code, as there will be fewer classes to be familiar with. 256 | 257 | * Have separate classes for each relevant representation of a reference 258 | system/frame.This is also possible, and simplifies writing the frame classes, 259 | as there is no need to transform to/from the "preferred" 260 | representation.However, it makes it very difficult to switch between 261 | representations, a task that is crucial for transformations (in most cases 262 | they are defined only on Cartesian representations).It also results in more 263 | work for any user that wants a custom system but wants to be able to use 264 | multiple representations. 265 | 266 | 267 | Decision rationale 268 | ------------------ 269 | 270 | There has been a fair bit of discussion on the API itself, but no objections 271 | have been raised for this APE, and it is therefore uncontroversial and has been 272 | accepted on March 9th 2014. 273 | -------------------------------------------------------------------------------- /APE20.rst: -------------------------------------------------------------------------------- 1 | .. _Black: https://github.com/psf/black 2 | .. _ruff: https://docs.astral.sh/ruff/ 3 | .. _Django DEP-0008: https://github.com/django/deps/blob/main/final/0008-black.rst 4 | .. _pandas pre-commit: https://pandas.pydata.org/docs/development/contributing_codebase.html#pre-commit 5 | .. _coding style: https://docs.astropy.org/en/latest/development/codeguide.html#coding-style-conventions 6 | .. _PEP8: https://peps.python.org/pep-0008/ 7 | .. |Black| replace:: *Black* 8 | .. |ruff| replace:: *ruff* 9 | 10 | Formatting Code with *Black* 11 | ============================ 12 | 13 | authors: Tom Aldcroft, Erik Tollerud 14 | 15 | date-created: 2022 July 13 16 | 17 | date-last-revised: 2023 December 19 18 | 19 | date-accepted: 2022 September 23 20 | 21 | type: Standard Track 22 | 23 | status: Accepted 24 | 25 | 26 | Abstract 27 | -------- 28 | 29 | This APE proposes that the `astropy core package 30 | <https://github.com/astropy/astropy>`_ adopt Black formatting for the code. Black 31 | formatting refers to the default code style defined by the |Black|_ code formatter. 32 | 33 | Since this APE was accepted in 2022, the Black code format has been implemented by other 34 | tools, most notably |ruff|_. In the text below we will use the term |Black| to refer to 35 | |Black| or any other tool that provides robust auto-formatting in the Black code format. 36 | Beyond the initial migration of the code base to black formatting, the tool(s) used 37 | to maintain |Black| formatting is not a requirement of this APE. 38 | 39 | Detailed description 40 | -------------------- 41 | 42 | Foreword 43 | ^^^^^^^^^ 44 | 45 | This APE presents a rationale that Black code formatting with an auto-formatter such 46 | as |Black|_ or |ruff|_ can be beneficial 47 | both for the ``astropy`` code development process and for the community. 48 | Hereafter we refer to the astropy core package as ``astropy``, noting that it is 49 | distinct from the Astropy project as a whole and coordinated and affiliated 50 | Astropy packages. While coordinated or affiliated packages may choose to adopt 51 | this APE's conventions in the future, this APE is concerned solely with the core 52 | package itself. 53 | 54 | |Black|_ is a fast and reliable Python automated code formatter supported by the 55 | Python Software Foundation. The project motto "The uncompromising code 56 | formatter" reflects that the code is strongly opinionated about formatting and 57 | there are very few configuration options. From their documentation, "By using 58 | *Black*, you agree to cede control over minutiae of hand-formatting. In return, 59 | *Black* gives you speed, determinism, and freedom from ``pycodestyle`` nagging 60 | about formatting." 61 | 62 | Code formatting is controversial and the `astropy-dev thread`_ about *Black* 63 | generated a spirited discussion but did not result in a clear consensus. This 64 | strong division of opinions about automated code formatting is common and has 65 | been documented in other projects, for instance in `Django DEP-0008`_. 66 | 67 | Other projects, including NumPy and SciPy, have considered using *Black* for 68 | formatting (see e.g. `SciPy PR #14330 69 | <https://github.com/scipy/scipy/pull/14330>`_). Neither of those projects felt 70 | that auto-formatting with the current version of *Black* (circa 2022) was a good 71 | choice for their codebase, with particular emphasis on needing improvements 72 | in formatting of mathematical expressions. 73 | 74 | We do not expect unanimous agreement on this APE but do hope to demonstrate the 75 | value of *Black* for the collective needs of the community and Astropy Project. 76 | 77 | .. _astropy-dev thread: https://groups.google.com/g/astropy-dev/c/6cRJCMgaFyM/ 78 | 79 | Note that ideas and some text for this APE have been adapted from a similar 80 | enhancement proposal for Django `Django DEP-0008`_. 81 | 82 | Rationale 83 | ^^^^^^^^^ 84 | 85 | **Consistent and readable code formatting** reduces effort in both code review 86 | and in future maintenance. Code is read far more often than it is written, so it 87 | makes sense to invest effort to ensure that the code is easy to read later. 88 | 89 | Ensuring that code formatting meets ``astropy`` standards is currently done by 90 | applying CI checks based on :pep:`8`. Learning and consistently applying all 91 | these rules is not easy. There are at least 200 commits in ``astropy`` which 92 | were solely address PEP8 violations, so this is a real issue. 93 | 94 | Because of the need to follow these format guidelines, ``astropy`` developers 95 | must pay close attention to formatting, constantly making small decisions about 96 | code formatting based on rules, on taste, or on a mix of both. While some of 97 | :pep:`8` is focused on guidelines like naming conventions that cannot be 98 | automated, much of it is essentially algorithmic and hence these decisions are a 99 | needless burden on the developers. 100 | 101 | With **automated code formatting**, the workload of code formatting is done by 102 | computers, saving human bandwidth for higher value and complexity activities 103 | like the logic or interface design. Collectively, contributors no longer need to 104 | think about code formatting when writing or in code reviews. By ceding control 105 | to |Black|_ they can focus on the content of the code and not worry about details 106 | like where to break a line or how to arrange a list. Whatever the formatter 107 | does is the expected result, by definition. 108 | 109 | Automated code formatting will increase consistency across the ``astropy`` code 110 | base. Currently the style of each subpackage reflects the preferences of the 111 | maintainers. For instance, the ``time``, ``table``, and ``io.ascii`` subpackages 112 | allow line lengths of up to 100, while most others use 80 or 88. There are other 113 | more subtle differences and astropy contributors find this confusing. This has 114 | come up in the past. 115 | 116 | While there are multiple code formatters for Python, the *Black* Python code 117 | formatter produces good enough results that it is getting significant traction 118 | in many open source projects, including: scikit-learn, pytest, tox, Pyramid, 119 | Django, Hypothesis, SQLAlchemy, virtualenv, pandas, Pillow, and Twisted. While 120 | adoption by this list of high-profile community-based open source projects does 121 | not automatically imply that it is a good fit for ``astropy``, it demonstrates 122 | unequivocally several points: 123 | 124 | - Technical issues related to the initial transition and subsequent maintenance of 125 | *Black* formatting are surmountable. There is plenty of prior art and good 126 | documentation. 127 | - Sociological issues related to developer preferences are surmountable. 128 | Evidence from other projects is that developers either embrace or 129 | at least accept autoformatting. 130 | - Developer documentation can provide clear instructions to contributors on how 131 | to implement *Black* formatting in their development process. Issues with this 132 | process are no more difficult (and arguably easier) to resolve than enforcing 133 | PEP-8 standards. 134 | 135 | Therefore, this APE proposes to adopt *Black* code formatting for ``astropy``. 136 | 137 | Concerns and mitigations 138 | ^^^^^^^^^^^^^^^^^^^^^^^^ 139 | 140 | Foolish consistency 141 | ~~~~~~~~~~~~~~~~~~~ 142 | A concern that has been highlighted by some members of the community (including, 143 | initially, one of the co-authors of this APE, and highlighed in detail in `this 144 | blog post 145 | <https://luminousmen.com/post/my-unpopular-opinion-about-black-code-formatter>`_) 146 | is that adopting *Black* violates the PEP8 principle that "A Foolish Consistency 147 | is the Hobgoblin of Little Minds". I.e., that by using a code formatter, the 148 | developer thinks less about readability because they think they can let the 149 | formatter do this for them. While this concern has some validity, this APE 150 | explicitly recognizes that **developers and code reviewers are ultimately 151 | responsible for maintaining a readable code base**. That is, there are many 152 | choices concerning logic and readability of code that *Black* does *not* affect, 153 | and those choices are more important for developers to focus on than mechanistic 154 | decisions that *Black* makes about whitespace and newlines. 155 | 156 | Undesirable formatting 157 | ~~~~~~~~~~~~~~~~~~~~~~ 158 | On occasion *Black*'s formatting produces undesirable results. The code may look 159 | very different from expectation due to mathematical or other domain-specific 160 | conventions, or some specific formatting that conveys meaning may be lost. A 161 | simple example of this is a constant identity matrix which would typically be 162 | written as:: 163 | 164 | matrix = [[1, 0, 0], 165 | [0, 1, 0], 166 | [0, 0, 1]] 167 | 168 | *Black* reformats this as shown below in a way that is very different from the 169 | typical representation for a matrix and makes it harder to read the structure:: 170 | 171 | matrix = [[1, 0, 0], [0, 1, 0], [0, 0, 1]] 172 | 173 | The mitigation here is the exception policy: In this case the author could 174 | surround the matrix with the ``#fmt: off``/``#fmt: on`` escape hatch, as this 175 | clearly makes the code more readable. While many examples are less clear than 176 | the above, and indeed in most cases "readability" is a subjective concept, this 177 | mechanism allows for such cases to be explicitly excepted where the developer 178 | thinks the improved readability warrants it. 179 | 180 | History 181 | ~~~~~~~ 182 | This changes almost every Python file and a large fraction of code lines (for 183 | example about one third of non-blank lines in the ``astropy.time`` subpackage). 184 | This is more than previous bulk changes and it is an impact that needs to be 185 | accepted with this APE. 186 | 187 | Code correctness 188 | ~~~~~~~~~~~~~~~~ 189 | *Black* guarantees semantic equivalence of the formatted code, and will not 190 | (`except in a few limited cases 191 | <https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#ast-before-and-after-formatting>`_) 192 | change the code AST. 193 | 194 | Migration effort 195 | ~~~~~~~~~~~~~~~~ 196 | Enthusiastic champions of the effort will largely do the work, but we recognize 197 | and appreciate that additional effort will be required from maintainers that are 198 | neutral or against the adoption of *Black*. 199 | 200 | Backports / release branches 201 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 202 | This is addressed in the Implementation section. 203 | 204 | New contributors 205 | ~~~~~~~~~~~~~~~~ 206 | New contributors may be more discouraged by more steps. However, this *replaces* 207 | the existing PEP8 check, so that it is not "another step". To make it better, 208 | leave very clear explicit messages in the status checks on GitHub to reduce the 209 | barrier. Scikit-learn had a few issues with contributors but not at a level that 210 | caused regret about the decision to adopt *Black* formatting. One developer 211 | described the change as a "net benefit". 212 | 213 | Removing the humanity 214 | ~~~~~~~~~~~~~~~~~~~~~ 215 | Some developers may feel that individual style and humanity gets lost in 216 | autoformatting. We hope to convey that humanity and individuality is expressed 217 | more fully in the elegance and clarity of the code logic and implementation. In 218 | most cases the placement of white space and line breaks can be viewed as a 219 | mechanistic process that humans need not control. 220 | 221 | Exception Policy 222 | ^^^^^^^^^^^^^^^^ 223 | As described above, there are cases *Black* formatting makes the code 224 | substantially more difficult to read or understand. These should be treated 225 | explicitly as exceptions, using the ``#fmt: off``/``#fmt: on`` escape hatch for 226 | multi-line exceptions, or ``#fmt: skip`` for single-line exceptions. **This is 227 | allowed if the code author and PR reviewer(s) agree that the code line or block 228 | should be excluded from *Black* formatting.** 229 | 230 | 231 | Branches and pull requests 232 | -------------------------- 233 | 234 | - `Add configuration for black autoformatter <https://github.com/astropy/astropy/pull/13253>`_ 235 | - `Apply black to modeling <https://github.com/astropy/astropy/pull/13254>`_ 236 | 237 | Implementation 238 | -------------- 239 | 240 | All Python code in ``astropy`` is formatted with |Black|_, using its default 241 | settings, that is, 88 characters per line and double quotes. Note that, by 242 | design, most of |Black|_ is inflexible, and there is not a way to pick-and-choose 243 | some elements and not others of *Black*'s format. 244 | 245 | Implementing this change requires: 246 | 247 | 1. Updating the `coding style`_: 248 | 249 | * Adding documentation about *Black*, with a particular emphasis on the 250 | rationale expressed here and the exception policy described above. 251 | * Document the *Black* exception guidelines (as described in this APE) 252 | * Removing other references to code formatting in the documentation. 253 | 254 | 2. Updating flake8 configuration to be compatible with *Black*. 255 | 256 | This is straightforward and well documented. 257 | 258 | 3. Checking *Black* in CI. 259 | 260 | * Add a CI check that runs *Black* to ensure consistency, failing if it meets a violation. 261 | 262 | 4. Reformatting Astropy's code. This will be done on a subpackage-by-subpackage 263 | basis over a transition period. The process for each subpackage to be 264 | transitioned is as follows: 265 | 266 | * The sub package maintainer(s) create a branch (e.g. 267 | ``blackify-coordinates``) to do the re-formatting. 268 | * Edit ``pyproject.toml`` to remove the *Black* exclusion for the sub-package. 269 | * Run ``black --skip-string-normalization astropy/<sub-package>`` on the 270 | sub-package directory. Commit the changes *Black* makes and make a draft 271 | pull request. 272 | * The maintainer(s) should review all changes and highlight for discussion 273 | any changes that are judged to be valid exceptions (see above for what 274 | "valid exception" means). These can be reverted by copying-and-pasting 275 | from the "changes" interfaces to the local copy and applying the 276 | appropriate *Black* directives to prevent auto-formatting. Commit and push 277 | these changes for review. 278 | * Once the PR submitter and reviewer are both satisfied with the changes 279 | then squash the commit(s) down to a single commit and force push. 280 | * Run ``black astropy/<sub-package>`` to apply just the string normalization 281 | stage and commit and push. 282 | * Edit .git-blame-ignore-revs to include the commits comprising PR to this 283 | point. 284 | * Convert the draft PR to a normal PR (ready for review). Once it passes CI 285 | then it can be merged without further review. 286 | 287 | A sub-package will not have *Black* checked in CI until it has completed these 288 | steps. 289 | 290 | Backward compatibility 291 | ---------------------- 292 | This APE would supersede any previous code formatting styles recommended for the 293 | core package. In practice this has little impact because the "standard" style 294 | has grown organically and is not applied consistently beyond the PEP8 checks 295 | (which *Black* is a superset of). 296 | 297 | Alternatives 298 | ------------ 299 | 300 | Other automated code formatting tools exist for Python code. The main players 301 | are ``autopep8`` and ``yapf``. There are many good posts on this topic - 302 | entering "black vs autopep8 vs yapf" into your favorite search engine will 303 | likely get you several good options. To establish the goal of consistency 304 | without endless debate, however, *Black* is the clear winner: Many other Python 305 | projects have adopted it. It is officially supported by the Python Software 306 | Foundation. The algorithm *Black* uses is generally more performant than the other 307 | two on large code bases like ``astropy``. 308 | 309 | Decision rationale 310 | ------------------ 311 | 312 | Although opinions were fairly divided on the initial `astropy-dev thread`_ 313 | discussing of adopting *Black*, the discussion has converged in favor of the 314 | implementation described here. In the cases where *Black* formatting will hurt 315 | readability, those situations will be handled on a case-by-case basis to 316 | exclude those lines from auto-formatting. 317 | 318 | It was initially unclear whether this proposal was also about the adoption of 319 | pre-commit hooks in astropy. Those hooks were adopted by the project prior to 320 | this APE, and while *Black* would be added to the pre-commit hooks in CI, the 321 | use of hooks is independent of this APE. 322 | 323 | The majority of the community who reviewed the PR supports auto-formatting 324 | code in general. There were mixed feeling as to whether *Black*’s formatting 325 | choices were optimal, and more broadly has some drawbacks. However, no better 326 | alternative was identified. 327 | 328 | Important to the decision to adopt *Black* over other auto-formatters is its 329 | guarantee not to alter the generated bytecode. There is not developer 330 | capacity to handle breakage caused by autoformatting. *Black*’s speed on a 331 | large code base and its lack of configurability were also elements in its 332 | favor. 333 | 334 | Based on the broad agreement in the discussion on the APE pull request, the 335 | CoCo has approved the APE. The community was well informed of this APE and 336 | discussion via several mechanisms, including the mailing list and the 337 | developer telecons. The CoCo thinks that the discussion in this APE includes 338 | enough voices to be representative of the broader developer community. 339 | -------------------------------------------------------------------------------- /APE3.rst: -------------------------------------------------------------------------------- 1 | Configuration 2 | ------------- 3 | 4 | author: Michael Droettboom 5 | 6 | date-created: 2013 December 10 7 | 8 | date-last-revised: 2014 April 30 9 | 10 | date-accepted: 2014 April 30 11 | 12 | type: Standard Track 13 | 14 | status: Accepted 15 | 16 | Abstract 17 | -------- 18 | 19 | We need a new approach to configuration that makes it easier to modify 20 | and extend configuration over time, plays well with other tools in the 21 | Python ecosystem, and most importantly, encourages the reproducibility 22 | of science results. 23 | 24 | Detailed description 25 | -------------------- 26 | 27 | Toward a configuration policy 28 | ````````````````````````````` 29 | 30 | The most important goal of the configuration system should be the 31 | reproducibility of scientific results. Some of the existing 32 | configuration parameters relate to peculiarities of platforms and 33 | environments (let's call this "platform configuration"), and others 34 | relate to actual scientific data processing (let's call this "science 35 | state"). This APE argues that the existing configuration system is 36 | well-suited to platform configuration (providing that some modest 37 | enhancements are made), whereas scientific state should be handled in 38 | more directly in code in a way that keeps those settings closer to a 39 | given application and not in user- or machine-global settings. 40 | 41 | For example, ``utils.console.use_color`` configures whether ANSI color 42 | codes will be output to the terminal. This might be changed to 43 | ``False`` if the user's system does not support them, but that would 44 | have no bearing on the output of science processing. In contrast, 45 | ``cosmology.core.default_cosmology`` changes the default cosmology to 46 | use, which could put sources in different locations, changing the 47 | ultimate science result. 48 | 49 | Global state of any kind (not just in a config file) that relates to 50 | scientific results should be discouraged wherever possible, but there 51 | are cases where it is just too convenient. An example might be the 52 | list of URLs to use in a ``vo`` query (the configuration item 53 | ``vo.validator.validate.cs_urls``). Another example of global state 54 | that doesn't currently use the configuration system is the set of 55 | active units (``astropy.units.set_enabled_units``). These sorts of 56 | things should be settable from Python code, and ideally have context 57 | managers, but should not use the configuration system, since storing 58 | them in user- and machine-global files hinders reproducibility. We 59 | should instead encourage the use of Python code, and the various 60 | facilities that Python has to reuse code, rather than config files. 61 | 62 | Current system 63 | `````````````` 64 | 65 | The current configuration system stores uses the ``ConfigObj`` library 66 | to store configuration in a file in the user's home directory 67 | (``~/.astropy`` or ``~/.config/astropy``, depending on platform and 68 | configuration). The configuration items are defined within each 69 | subpackage using declarations that look like:: 70 | 71 | SESAME_URL = ConfigurationItem("sesame_url", 72 | ["http://cdsweb.u-strasbg.fr/cgi-bin/nph-sesame/", 73 | "http://vizier.cfa.harvard.edu/viz-bin/nph-sesame/"], 74 | "The URL to Sesame's web-queryable database.", 75 | cfgtype='string_list') 76 | 77 | At build time, all of these configuration items are collated into a 78 | template config file that is copied to the user's home directory (if 79 | it doesn't already exist) upon first import of astropy. 80 | 81 | Configuration items may be set and saved to disk directly from 82 | Python:: 83 | 84 | In [1]: from astropy.coordinates import name_resolve 85 | In [2]: name_resolve.NAME_RESOLVE_TIMEOUT.set(42) 86 | In [3]: name_resolve.NAME_RESOLVE_TIMEOUT.save() 87 | 88 | The config file may also be edited by hand. It provides a convenient 89 | summary and description of each of the available options. 90 | 91 | There are some things about this system that would be nice to retain: 92 | 93 | - It's easy to save values to the file from Python. 94 | 95 | - It's easy to edit the config file by hand. 96 | 97 | - It can be loaded very early on during the import of astropy so 98 | it can affect things, such as logging, that are not very 99 | dynamic. 100 | 101 | However, there are a number of shortcomings of this approach: 102 | 103 | - The entire astropy package must be imported to determine what 104 | the config items are, and generate the template configuration 105 | file. This is slow, and can place otherwise unnecessary 106 | restrictions on the code in order to make it importable at build 107 | time. 108 | 109 | - The template config file is copied to the user's home directory 110 | with all values uncommented. This means we don't know which 111 | values the user has explicitly changed, and can't change default 112 | values in future versions of astropy. 113 | 114 | - We don't know the version of astropy that the config file was 115 | created with, so we can't update it when astropy is updated. 116 | 117 | Unrelated to the system itself, but to how it's been used thus far, it 118 | is difficult to find the configuration items using introspection in 119 | Python, since many are defined rather "deep" within the subpackages. 120 | For example, we have ``cosmology.core.DEFAULT_COSMOLOGY`` instead of 121 | ``cosmology.DEFAULT_COSMOLOGY``. All of these configuration items 122 | should be moved to the ``config`` namespace within each subpackage. 123 | To use this example, ``cosmology.config.DEFAULT_COSMOLOGY``. 124 | 125 | Branches and pull requests 126 | -------------------------- 127 | 128 | https://github.com/astropy/astropy/pull/2094 129 | 130 | Implementation 131 | -------------- 132 | 133 | The following improvements to the base configuration system will be 134 | made: 135 | 136 | - Since, as a result of this APE, the number of configuration 137 | options will be much smaller, we will just include a template 138 | config file directly in the source code repository, install that 139 | alongside the source code, and copy that into the user's 140 | configuration directory upon first import (see below for the 141 | details of how and when it is copied to the user's config 142 | directory). Obviously, it will require some care to keep it in 143 | sync with the declaration of the configuration options in the 144 | source code. However, it is the simplest possible thing that 145 | can work and resolves the issues with generating the config file 146 | template at build time. 147 | 148 | - The template config file will have all key/value pairs commented 149 | out by default. This way, default values can be changed in 150 | astropy and will only be overridden by the config file if the 151 | user explicitly does so. 152 | 153 | - Store a version string in the config file. This, at a later 154 | date, but not as part of this APE, may allow for graceful 155 | automatic upgrades of the config file. This version should 156 | correspond to the release version of astropy, and not include a 157 | githash. 158 | 159 | - Upgrading of the config file will roughly follow the `Debian 160 | configuration file guidelines 161 | <http://raphaelhertzog.com/2010/09/21/debian-conffile-configuration-file-managed-by-dpkg/>`__, 162 | which strikes a good balance between safety and simplicity. It 163 | prevents the user's config file changes from being accidentally 164 | overwritten, but doesn't try to be too clever about 165 | automatically updating the content. In short: 166 | 167 | - If there is no user config file, copy the one from astropy's 168 | current version. 169 | 170 | - If there is a user config file, and it is entirely commented 171 | out, the file is overwritten. 172 | 173 | - If there is a user config file and its contents match exactly 174 | those of a stock astropy release prior to this APE (prior to 175 | astropy 0.4), it is overwritten. 176 | 177 | - If the user config file is different from the config file 178 | template of a previous astropy version, don't touch it. 179 | Install alongside it ``astropy.cfg.ver`` where ``ver`` is the 180 | current version. Optionally, install a 181 | ``astropy.cfg.ver.diff`` which is a diff of the user's config 182 | file and the current config file template. Display a warning 183 | that the config file has changed and the user may want to 184 | manually resolve the differences between their file and the 185 | new one. This warning should only be displayed once (when 186 | ``astropy.cfg.ver`` doesn't already exist) so that users who 187 | frequently switch between versions of astropy are not 188 | bombarded with warnings. 189 | 190 | Once doing that, each existing configuration item will be determined 191 | to be either "platform" or "science". 192 | 193 | For "platform" configuration items: 194 | 195 | - Include the item within the new config file template in the 196 | source repository. 197 | 198 | - Move the configuration item to the ``subpackage.conf`` 199 | namespace, which is a subclass of a base class for managing 200 | configuration items. 201 | 202 | - For backward compatibility, keep special delegation objects that 203 | delegate from the existing location to the new location and 204 | raise deprecation warnings when used. 205 | 206 | - The configuration items may still be set by their old keys in 207 | the config file for one major release cycle, but a deprecation 208 | warning will be shown. 209 | 210 | - The configuration item should be documented in the subpackages 211 | documentation in a standardized section ("Configuration"). 212 | 213 | For "science" configuration items: 214 | 215 | - Define a standard Python context manager for setting the global 216 | state associated with each configuration item. For example, 217 | this should work:: 218 | 219 | from astropy import cosmology 220 | with cosmology.set_default_cosmology('WMAP9'): 221 | # do something 222 | 223 | # This also works, but doesn't automatically "reset" itself 224 | # at the end of the block 225 | cosmology.set_default_cosmology('WMAP9') 226 | 227 | - These context managers will be documented in the API section of 228 | the subpackage in the standard way along with the rest of the 229 | API. 230 | 231 | - Retain special delegation objects at the existing location of 232 | the configuration items that call these new Python context 233 | managers. These will raise deprecation warnings describing how 234 | to update code. 235 | 236 | - When these configuration items are found in the config file, 237 | deprecation warnings will be shown, but only if they are 238 | different from the defaults as specified in astropy 0.3. Doing 239 | this without checking against the defaults would give everyone a 240 | warning, since all users currently have an astropy 0.3 config 241 | file with all values set. 242 | 243 | To support the new way of dealing with scientific configuration, ways 244 | of conveniently running Python code at the start of every script 245 | should be documented. This should include, in order of increasing 246 | "broadness": 247 | 248 | - Making state changes at the top of your script. 249 | 250 | - Having a Python module that all your scripts explicitly import. 251 | 252 | - Using IPython's "profiles" 253 | 254 | - Using Python's "sitecustomize" (though this would be the least 255 | desirable, as it has many of the reproducibility problems that 256 | plague the current configuration system). 257 | 258 | For a subsequent release, we will remove all of the deprecated 259 | backward-compatibility delegation objects. 260 | 261 | A set of guidelines about the difference between "platform 262 | configuration" and "science state" will be added to the relevant 263 | developer documentation. 264 | 265 | As an optional follow-on to all of the above, a 266 | ``astropy.reset_science_defaults()`` function may be added, that will 267 | reset all of the science state to their defaults. If all of the 268 | science state context managers inherit from the same base class, 269 | presumably providing that should be fairly automatic and 270 | straightforward. 271 | 272 | Specific configuration setting changes 273 | `````````````````````````````````````` 274 | 275 | The following configuration items have been moved/renamed or converted 276 | to science state: 277 | 278 | .. list-table:: Renamed configuration parameters 279 | :widths: 20 20 20 20 280 | :header-rows: 1 281 | 282 | * - Old config file location 283 | - Old Python location 284 | - New config file location 285 | - New Python location 286 | * - ``[] unicode_output`` 287 | - ``UNICODE_OUTPUT`` 288 | - unchanged 289 | - ``conf.unicode_output`` 290 | * - ``[coordinates.name_resolve] name_resolve_timeout`` 291 | - ``coordinates.name_resolve.NAME_RESOLVE_TIMEOUT`` 292 | - ``[astropy.utils.data] remote_timeout`` 293 | - ``astropy.utils.data.conf.remote_timeout`` 294 | * - ``[coordinates.name_resolve] sesame_url`` 295 | - ``coordinates.name_resolve.SESAME_URL`` 296 | - removed 297 | - ``coordinates.name_resolve.sesame_url.get/set`` 298 | * - ``[coordinates.name_resolve] sesame_database`` 299 | - ``coordinates.name_resolve.SESAME_DATABASE`` 300 | - removed 301 | - ``coordinates.name_resolve.sesame_database.get/set`` 302 | * - ``[cosmology.core] default_cosmology`` 303 | - ``cosmology.core.DEFAULT_COSMOLOGY`` 304 | - removed 305 | - ``cosmology.default_cosmology.get/set`` 306 | * - ``[io.fits] enable_record_valued_keyword_cards`` 307 | - ``io.fits.ENABLE_RECORD_VALUED_KEYWORD_CARDS`` 308 | - unchanged 309 | - ``io.fits.conf.enable_record_valued_keyword_cards`` 310 | * - ``[io.fits] extension_name_case_sensitive`` 311 | - ``io.fits.EXTENSION_NAME_CASE_SENSITIVE`` 312 | - unchanged 313 | - ``io.fits.conf.extension_name_case_sensitive`` 314 | * - ``[io.fits] strip_header_whitespace`` 315 | - ``io.fits.STRIP_HEADER_WHITESPACE`` 316 | - unchanged 317 | - ``io.fits.conf.strip_header_whitespace`` 318 | * - ``[io.fits] use_memmap`` 319 | - ``io.fits.USE_MEMMAP`` 320 | - unchanged 321 | - ``io.fits.conf.use_memmap`` 322 | * - ``[io.votable.table] pedantic`` 323 | - ``io.votable.table.PEDANTIC`` 324 | - ``[io.votable] pedantic`` 325 | - ``io.votable.conf.pedantic`` 326 | * - ``[logger] log_exceptions`` 327 | - ``logger.LOG_EXCEPTIONS`` 328 | - unchanged 329 | - ``logger.conf.log_exceptions`` 330 | * - ``[logger] log_file_format`` 331 | - ``logger.LOG_FILE_FORMAT`` 332 | - unchanged 333 | - ``logger.conf.log_file_format`` 334 | * - ``[logger] log_file_level`` 335 | - ``logger.LOG_FILE_LEVEL`` 336 | - unchanged 337 | - ``logger.conf.log_file_level`` 338 | * - ``[logger] log_file_path`` 339 | - ``logger.LOG_FILE_PATH`` 340 | - unchanged 341 | - ``logger.conf.log_file_path`` 342 | * - ``[logger] log_level`` 343 | - ``logger.LOG_LEVEL`` 344 | - unchanged 345 | - ``logger.conf.log_level`` 346 | * - ``[logger] log_to_file`` 347 | - ``logger.LOG_TO_FILE`` 348 | - unchanged 349 | - ``logger.conf.log_to_file`` 350 | * - ``[logger] log_warnings`` 351 | - ``logger.LOG_WARNINGS`` 352 | - unchanged 353 | - ``logger.conf.log_warnings`` 354 | * - ``[logger] use_color`` 355 | - ``logger.USE_COLOR`` 356 | - ``[] use_color`` 357 | - ``conf.use_color`` 358 | * - ``[nddata.nddata] warn_unsupported_correlated`` 359 | - ``nddata.nddata.WARN_UNSUPPORTED_CORRELATED`` 360 | - ``[nddata] warn_unsupported_correlated`` 361 | - ``nddata.conf.warn_unsupported_correlated`` 362 | * - ``[table.column] auto_colname`` 363 | - ``table.column.AUTO_COLNAME`` 364 | - ``[table] auto_colname`` 365 | - ``table.conf.auto_colname`` 366 | * - ``[table.pprint] max_lines`` 367 | - ``table.pprint.MAX_LINES`` 368 | - ``[table] max_lines`` 369 | - ``table.conf.max_lines`` 370 | * - ``[table.pprint] max_width`` 371 | - ``table.pprint.MAX_WIDTH`` 372 | - ``[table] max_width`` 373 | - ``table.conf.max_width`` 374 | * - ``[utils.console] use_color`` 375 | - ``utils.console.USE_COLOR`` 376 | - ``[] use_color`` 377 | - ``conf.use_color`` 378 | * - ``[utils.data] compute_hash_block_size`` 379 | - ``astropy.utils.data.COMPUTE_HASH_BLOCK_SIZE`` 380 | - unchanged 381 | - ``astropy.utils.data.conf.compute_hash_block_size`` 382 | * - ``[utils.data] dataurl`` 383 | - ``astropy.utils.data.DATAURL`` 384 | - unchanged 385 | - ``astropy.utils.data.conf.dataurl`` 386 | * - ``[utils.data] delete_temporary_downloads_at_exit`` 387 | - ``astropy.utils.data.DELETE_TEMPORARY_DOWNLOADS_AT_EXIT`` 388 | - unchanged 389 | - ``astropy.utils.data.conf.delete_temporary_downloads_at_exit`` 390 | * - ``[utils.data] download_cache_block_size`` 391 | - ``astropy.utils.data.DOWNLOAD_CACHE_BLOCK_SIZE`` 392 | - unchanged 393 | - ``astropy.utils.data.conf.download_cache_block_size`` 394 | * - ``[utils.data] download_cache_lock_attempts`` 395 | - ``astropy.utils.data.download_cache_lock_attempts`` 396 | - unchanged 397 | - ``astropy.utils.data.conf.download_cache_lock_attempts`` 398 | * - ``[utils.data] remote_timeout`` 399 | - ``astropy.utils.data.REMOTE_TIMEOUT`` 400 | - unchanged 401 | - ``astropy.utils.data.conf.remote_timeout`` 402 | * - ``[vo.client.conesearch] conesearch_dbname`` 403 | - ``vo.client.conesearch.CONESEARCH_DBNAME`` 404 | - ``[vo] conesearch_dbname`` 405 | - ``vo.conf.conesearch_dbname`` 406 | * - ``[vo.client.vos_catalog] vos_baseurl`` 407 | - ``vo.client.vos_catalog.BASEURL`` 408 | - ``[vo] vos_baseurl`` 409 | - ``vo.conf.vos_baseurl`` 410 | * - ``[vo.samp.utils] use_internet`` 411 | - ``vo.samp.utils.ALLOW_INTERNET`` 412 | - ``[vo.samp] use_internet`` 413 | - ``vo.samp.conf.use_internet`` 414 | * - ``[vo.validator.validate] cs_mstr_list`` 415 | - ``vo.validator.validate.CS_MSTR_LIST`` 416 | - ``[vo.validator] conesearch_master_list`` 417 | - ``vo.validator.conf.conesearch_master_list`` 418 | * - ``[vo.validator.validate] cs_urls`` 419 | - ``vo.validator.validate.CS_URLS`` 420 | - ``[vo.validator] conesearch_urls`` 421 | - ``vo.validator.conf.conesearch_urls`` 422 | * - ``[vo.validator.validate] noncrit_warnings`` 423 | - ``vo.validator.validate.noncrit_warnings`` 424 | - ``[vo.validator] noncritical_warnings`` 425 | - ``vo.validator.conf.noncritical_warnings`` 426 | 427 | Backward compatibility 428 | ---------------------- 429 | 430 | The delegation objects should retain backward compatibility for at 431 | least one release. 432 | 433 | Alternatives 434 | ------------ 435 | 436 | In an earlier draft of this proposal, I proposed that we throw out the 437 | existing configuration system altogether. By doing so, however, we 438 | would lose the ability to easily update and save values to the file 439 | from Python. We also would require users to use IPython profiles 440 | (which are a fairly advanced feature) just to do basic things. It 441 | also makes it harder to convey to the user which things are 442 | recommended for user- or machine-wide configuration and which things 443 | may lead to scripts and applications not being portable. In the 444 | present proposal it's obvious: the config file is for user-global 445 | stuff; everything else is just Python code. 446 | 447 | The first draft of this APE surmised an elaborate automatic 448 | configuration file update system. This draft proposes much like 449 | package upgrading on Debian, which preserves the state of a 450 | user-modified configuration file, while otherwise staying out of the 451 | way and providing manual intervention to upgrade the config file. 452 | That should probably be "good enough" and is far simpler, and 453 | presumable less prone to errors or surprising bugs. 454 | 455 | Decision rationale 456 | ------------------ 457 | 458 | No objections have been raised to this APE in its current form, and it has 459 | therefore been accepted on April 30th 2014. 460 | -------------------------------------------------------------------------------- /APE7.rst: -------------------------------------------------------------------------------- 1 | NDData Plan 2 | ----------- 3 | 4 | authors: Thomas Robitaille, Perry Greenfield, Matt Craig 5 | 6 | date-created: 2014 October 2 7 | 8 | date-last-revised: 2014 December 17 9 | 10 | date-accepted: 2014 December 17 11 | 12 | type: Standard Track 13 | 14 | status: Accepted 15 | 16 | Abstract 17 | -------- 18 | 19 | This APE is intended to provide a long-term plan for the ``astropy.nddata`` 20 | sub-package. The package has been the subject of continuous debate since the 21 | start of the astropy project, and has changed in scope several times, so this 22 | APE is aimed at agreeing on the scope and future of the sub-package and 23 | providing a well-structured foundation on which future development of 24 | ``astropy.nddata`` can occur. 25 | 26 | Detailed description 27 | -------------------- 28 | 29 | Introduction 30 | ^^^^^^^^^^^^ 31 | 32 | At the first Astropy coordination meeting in 2011, it was decided that as well 33 | as having a generic table container, it would be useful to have a generic 34 | container for gridded data. The ``astropy.table`` package has since then seen a 35 | large amount of development, and the API has now stabilized. The added value of 36 | the table package compared to using simple Numpy structured arrays is clear - 37 | the ``Table`` class makes it very easy to do common operations on tables such 38 | as adding or removing columns or rows, and reading/writing tables to common 39 | file formats. 40 | 41 | On the other hand, ``NDData`` development has stagnated and we have not been 42 | able to converge on a stable API. Part of this is due to the fact that there is 43 | in fact a huge variety of 'n-dimensional datasets' and that there is very 44 | little in common for example between a spectrum and an image, in terms of what 45 | can be done with them. This has prevented the ``NDData`` class from including 46 | much functionality, and in fact we have been adding functionality then removing 47 | it after we have realized that it is not general enough. 48 | 49 | An example to illustrate this issue is that of arithmetic - what happens when 50 | we add two n-dimensional datasets? Of course, the data values can be added, but 51 | what happens to the mask, to the flags, or to the uncertainties? The truth is 52 | that there is no general recipe for dealing with this, and that it may be a 53 | mistake to try and define it in such a general way. 54 | 55 | This APE takes the approach of re-thinking the purpose of NDData and trying to 56 | define a scope for the future. 57 | 58 | The 'why' of NDData 59 | ^^^^^^^^^^^^^^^^^^^ 60 | 61 | First, why do we actually need a generic data container that can be 62 | sub-classed? What is the benefit of this versus simply defining separate base 63 | classes for spectra, images, and other types of data? There are several 64 | possible answers: 65 | 66 | 1. We want to provide users with a consistent experience across data objects - 67 | that is, the user should know that meta-data is always consistently named 68 | ``meta``, that a mask can always be accessed with ``mask``, and that the 69 | data can be accessed with ``data``. 70 | 71 | 2. By providing a common base class which can define a unified I/O interface 72 | which taps into the astropy I/O registry, we can seamlessly make it that 73 | all data objects have ``read`` and ``write`` methods that behave 74 | consistently. 75 | 76 | 3. We want functions and methods in Astropy to know if an object passed to them 77 | is an n-dimensional data object that can be expected to have specific 78 | attributes (such as ``wcs``, ``mask``, and so on). 79 | 80 | In principle, none of these *require* a base class. We could simply agree on a 81 | standard for data objects that defines what certain attributes should be 82 | called. This is a valid solution, but at the same time, having a base class can 83 | enforce this and factor out some boilerplate code, and as described in 84 | `Alternatives`_, the questions raised here would apply to a base ``Image`` 85 | class that was sub-classed as ``XRayImage``, ``CCDImage``, and so on. 86 | 87 | Proposal for an ``NDDataBase`` abstract base class 88 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 89 | 90 | The first proposal in this APE is to separate a definition of the ``NDData`` 91 | interface from concrete realizations of ``NDData``-like objects by creating an 92 | abstract base class called ``NDDataBase``. 93 | 94 | The proposed ``NDDataBase`` simplifies the current ``NDData`` class to the extreme, 95 | such that it only defines which properties are needed for ``NDData``. 96 | 97 | The ``NDDataBase`` class should **not** define any arithmetic operations, which are 98 | impossible to generalize. 99 | 100 | The following properties should be included in the base class: 101 | 102 | * ``data`` - the data itself. No restrictions are placed on the type of this 103 | data in the ABC. Subclasses could, for example, make it a plain Numpy array, 104 | masked Numpy array, an Astropy Quantity, or an h5py data buffer. Subclasses 105 | could also require, for example, that ``data`` provides a ``shape`` 106 | attribute and/or be sliceable in order to 'prove' that it is an 107 | n-dimensional data object. 108 | 109 | * ``mask`` - the mask of the data, following the Numpy convention of `True` 110 | meaning masked, and `False` meaning unmasked. Sub-classes could choose to 111 | connect this to ``data.mask``. Masks do not need to be Numpy arrays, they 112 | could be for example 'lazy' masks based on functions that will be evaluated 113 | on-the-fly. 114 | 115 | * ``unit`` - the unit of the data values, which will be internally 116 | represented as an Astropy Unit. If present, subclasses should try to ensure 117 | numerical operations properly take into account and propagate units. 118 | Sub-classes could choose to connect this to ``data.unit``, in which case data 119 | should be a ``Quantity`` or behave like it. 120 | 121 | * ``wcs`` - an object that can be used to describe the relationship between 122 | input and world coordinates. This can (but does not 123 | have to) be an Astropy WCS object. Once the generalized WCS system is in 124 | place in Astropy, we will probably require this to be such an object. 125 | Subclasses are free to be more restrictive in what they permit for the 126 | ``wcs`` object. 127 | 128 | * ``meta`` - a dict-like object that can be used to contain arbitrary metadata. 129 | This could be a plain Python dict, an ordered dict, a FITS Header object, and 130 | so on, provided that it offers dict-like item access and iteration. 131 | 132 | * ``uncertainty`` -- an object that represents the uncertainty in the 133 | data for each element on the array. This APE places no restriction on 134 | what type of uncertainty this is (e.g. variance, standard deviation, 135 | or posisson count rate), nor does it require the attribute to be set 136 | at all (other than defaulting to ``None``). It places only one restriction 137 | on ``uncertainty``: it must have an attribute ``uncertainty_type``, 138 | which should be a human-readable string. 139 | 140 | While not a requirement, the following ``uncertainty_type`` strings 141 | are strongly recommended for common ways of specifying normal 142 | distributions: 143 | 144 | * ``"std"``: if ``uncertainty`` stores the standard deviation/sigma 145 | (either a single value or on a per-pixel basis). 146 | * ``"var"``: if ``uncertainty`` stores the variance (either a single 147 | value or on a per-pixel basis). 148 | * ``"ivar"``: if ``uncertainty`` stores the inverse variance (either a 149 | single value or on a per-pixel basis). 150 | 151 | The only **required** attribute is ``data``; all others default to ``None`` if 152 | not initialized or overridden in a subclass. 153 | 154 | Proposal for the ``NDData`` class 155 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 156 | 157 | The ``NDData`` class would be a concrete class that is far less restrictive 158 | than the current ``NDData`` while still providing some basic functionality as 159 | a container. It will place loose restrictions on what ``data`` can be set to 160 | and is described further in `Implementation`_. 161 | 162 | 163 | The ``NDData`` class would **not** include methods such as ``__array__``, 164 | ``__array_prepare__``, and so on which allow a class to be treated as a Numpy 165 | array. This behavior has been identified as being potentially ambiguous in 166 | the general case because it will depend on the details of e.g. how masks are 167 | handled and also does not make it explicit in what units the data is required. 168 | 169 | Proposal for mixin classes to provide additional functionality 170 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 171 | 172 | Specific functionality such as uncertainty handling and arithmetic can be 173 | developed as mix-in classes that can be used by ``NDData`` sub-classes. 174 | 175 | Generic slicing capabilities, further described in `Implementation`_, will be 176 | provided as a mixin class called ``NDSlicing`` 177 | 178 | Handling of ``NDData`` in Astropy and affiliated packages 179 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 180 | 181 | If a user has a data object such as an image, it would be nice if they can use 182 | functions directly on this image and have them return an image object. At the 183 | same time, we do not want to force people to use special data containers if 184 | they have for example a Numpy array and a WCS object. This raises the question 185 | of whether we should duplicate the API for all functions, to provide one 186 | interface for ``NDData`` subclasses, and one for separate attributes. The 187 | proposal in this APE is that functions should only define a single API that 188 | takes separate keyword arguments for e.g. ``data``, ``mask``, and so on, but 189 | that we then provide a way for users to be able to call these functions with 190 | ``NDData`` sub-classes (see `Implementation`_). 191 | 192 | Implementation 193 | -------------- 194 | 195 | ``NDDataBase`` class 196 | ^^^^^^^^^^^^^^^^^^^^ 197 | 198 | ``NDDataBase`` will be implemented as an abstract base class. The only input 199 | validation it will provide is enforcing the existence of an 200 | ``uncertainty_type`` attribute if ``uncertainty`` is not ``None``, as described 201 | above. 202 | 203 | We will not include setters for properties except ``mask`` and ``uncertainty`` 204 | because it is ambiguous what the meaning of setting e.g. the unit or WCS after 205 | initialization means: it could either mean to change the unit or WCS, or it 206 | could mean that the user wants to convert the data to this new unit or WCS. 207 | Given this ambiguity, it is safer to not have setters for the core attributes 208 | and this is consistent with e.g. ``Quantity``. 209 | 210 | 211 | ``NDData`` class 212 | ^^^^^^^^^^^^^^^^ 213 | 214 | ``NDData`` will be a concrete subclass of ``NDDataBase`` that provides some logic for handling the setting of ``data``: 215 | 216 | * If the object passed in as ``data`` has a ``shape`` attribute, is 217 | sliceable, and has an ``__array__`` method, so that it can be easily used as 218 | a numpy array, then ``NDData.data`` will be set to that object. 219 | * Otherwise, ``NDData`` will attempt to create a ``numpy.ndarray`` from the 220 | input ``data`` and use that as the internal representation of the data. 221 | 222 | I/O mixin 223 | ^^^^^^^^^ 224 | 225 | The ``read`` and ``write`` methods will be developed via a mixin class. 226 | 227 | Slicing mixin 228 | ^^^^^^^^^^^^^ 229 | 230 | This APE suggests adding a mixin class, ``NDSlicing``, to handle basic 231 | slicing. This could be done by simply having code similar to the following 232 | inside ``__getitem__``:: 233 | 234 | def __getitem__(self, slice): 235 | 236 | new = self.__class__() 237 | 238 | if self.data is not None: 239 | new._data = self.data[slice] 240 | 241 | if self.mask is not None: 242 | new._mask = self.mask[slice] 243 | 244 | if self.wcs is not None: 245 | new._wcs = self.wcs[slice] 246 | ... 247 | 248 | Note that this is only meant as an illustration of the idea suggested here, 249 | and the final implementation will likely differ from this - but the basic 250 | idea is that the slicing would be delegated to the member attributes. For 251 | example, the WCS class would need to define itself how it should be sliced. 252 | Some attributes (such as ``meta``) would not necessarily need to be sliceable. 253 | 254 | Note the effect of slicing on attributes presumably returns a similar object, 255 | e.g., for wcs, it returns a new WCS appropriate to the sliced data attribute. 256 | 257 | 258 | Arithmetic mixin 259 | ^^^^^^^^^^^^^^^^ 260 | 261 | The arithmetic methods currently in ``NDData`` will be implemented in a mixin 262 | called ``NDArithmetic``. 263 | 264 | Facilitating the use of ``NDData`` sub-classes 265 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 266 | 267 | One question that has come up as part of several affiliated packages is how 268 | to deal with ``NDData`` objects in functions. For example, if we consider a 269 | ``downsample`` function that can downsample an image, should the function 270 | accept only ``NDData`` (or sub-class) objects? Should it also 271 | accept plain Numpy arrays? If so, how do we pass any additional meta-data 272 | such as WCS? Should we return a downsampled Numpy array and downsampled WCS, 273 | or a single downsampled ``NDData`` (or sub-class) instance? In this example, one option would 274 | be to provide two APIs, one for ``NDData`` and/or sub-classes and one for separate Numpy arrays 275 | and attributes, but maintaining two parallel APIs is not an ideal solution. 276 | An alternative is for each function to encode the logic of checking the input 277 | type and deciding on the output type based on the output type. However, this 278 | means repeating a lot of similar code such as:: 279 | 280 | def downsample(data, wcs=None) 281 | 282 | if isinstance(data, NDData): 283 | if wcs is not None: 284 | raise ValueError("wcs cannot be specified if NDData instance was passed") 285 | wcs = data.wcs 286 | data = data.data 287 | 288 | and this will become a lot more complex once more attributes are needed by 289 | the function. 290 | 291 | In order to make it easier for functions to accept ``NDData`` sub-classes and 292 | return these, we can implement a decorator that will automatically split up an 293 | ``NDData`` object as needed. Let us consider the following function:: 294 | 295 | def test(data, wcs=None, unit=None, n_iterations=3): 296 | ... 297 | 298 | We can provide a decorator called e.g. ``support_nddata``:: 299 | 300 | @support_nddata 301 | def test(data, wcs=None, unit=None, n_iterations=3): 302 | ... 303 | 304 | which makes it so that if the user passes an ``NDData`` sub-class called e.g. 305 | ``nd``, the function would automatically be called with:: 306 | 307 | test(nd.data, wcs=nd.wcs, unit=nd.unit) 308 | 309 | That is, the decorator looks at the signature of the function and checks if any 310 | of the arguments are also properties of the ``NDData`` object, and passes them 311 | as individual arguments. 312 | 313 | An error could be raised if an ``NDData`` property is set but the function does 314 | not accept it - for example, if ``wcs`` is set, but the function cannot support 315 | WCS objects, an error would be raised. On the other hand, if an argument in the 316 | function does not exist in the ``NDData`` object or is not set, it is simply 317 | left to its default value. This behavior could be customizable but the 318 | details are beyond the scope of this APE document. 319 | 320 | If the function call succeeds, then the decorator will make a new ``NDData`` 321 | object (with the correct class) and will populate the properties as needed. In 322 | order to figure out what is returned by the function, the decorator will need 323 | to accept a list which gives the name of the output values:: 324 | 325 | @support_nddata(returns=['data', 'wcs']) 326 | def test(data, wcs=None, unit=None, n_iterations=3): 327 | ... 328 | 329 | Finally, the decorator could be made to restrict input to specific ``NDData`` 330 | sub-classes (and sub-classes of those):: 331 | 332 | @support_nddata(accepts=CCDImage, returns=['data', 'wcs']) 333 | def test(data, wcs=None, unit=None, n_iterations=3): 334 | ... 335 | 336 | With this decorator, the functions could be seamlessly used either with 337 | separate arguments (e.g. Numpy array and WCS) or with subclasses of 338 | ``NDData`` such as ``CCDImage``. 339 | 340 | Example of subclassing from both NDData and Quantity 341 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 342 | 343 | The ``Quantity`` class would benefit from the ability to share the same 344 | interface that NDData provides and to tap into NDData's metadata and WCS 345 | handling. 346 | 347 | Because subclassing from ``numpy.ndarray`` involves subtleties that differ 348 | from typical subclassing in python, an example subclass called ``NDQuantity`` 349 | may be implemented as part of ``astropy.nddata``. Should it turn out to be 350 | unreasonably difficult to do, an attempt may be made to implement a class 351 | which uses ``Quantity`` as the data store, with the ability to link ``NDData`` 352 | properties like ``unit`` to the underlying properties in ``Quantity``. Should 353 | that also prove to be unworkable, an explanation of the issues that prevented 354 | implementation may be provided in the documentation for ``NDData``. 355 | 356 | Branches and pull requests 357 | -------------------------- 358 | 359 | Initial decorator implementation: https://github.com/astropy/astropy/pull/2855 360 | 361 | Initial refactoring of NDData: https://github.com/astropy/astropy/pull/2905 362 | 363 | Backward compatibility 364 | ---------------------- 365 | 366 | This APE will require packages such as ``specutils`` and ``ccdproc`` to 367 | completely refactor how they use the ``NDData`` class. This will also break 368 | compatibility with users currently using ``NDData`` directly, but this is 369 | assumed to be a very small fraction (if any) of users. 370 | 371 | Alternatives 372 | ------------ 373 | 374 | Eliminate ``NDData`` 375 | ^^^^^^^^^^^^^^^^^^^^ 376 | 377 | One alternative is to remove the ``NDData`` class altogether and to start 378 | the base classes at the level of ``Spectrum`` or ``Image``. In this case many 379 | of this ideas of this APE (including the attribute names, decorators, etc.) 380 | would still apply to these base classes. The benefits of having a base 381 | ``NDData`` class instead of starting at the ``Image`` and ``Spectrum`` level 382 | are that: 383 | 384 | * The ``NDData`` class enforces the naming of the base properties to ensure 385 | consistency across all sub-classes. 386 | 387 | * It allows slicing to be implemented at the core level as a mixin, whereas 388 | this would need to be repeated in each base class if we had e.g. 389 | ``Spectrum``, ``Image``, ``SpectralCube`` as the base classes. 390 | 391 | * It allows the connection to the unified I/O framework to be defined once, 392 | whereas this would also need to be repeated in each base class otherwise. 393 | 394 | On the other hand, the downsides of having a core ``NDData`` class is that it 395 | reduces flexibility of the sub-classes - for instance ``Spectrum`` has to be 396 | implemented taking into consideration the restrictions on e.g. attribute 397 | names defined by the sub-classes. In the 398 | `spectral-cube <http://spectral-cube.readthedocs.org>`_ package, at the moment 399 | we do not have a ``data`` attribute because we have a custom masking 400 | framework and define attributes like ``unmasked_data``. Of course, we should 401 | aim to make this more compliant with what is decided here, but this is just 402 | to demonstrate that this type of flexibility may be lost. However, this may 403 | be a good thing as it enforces consistency for users. 404 | 405 | Subclass NDData from ``astropy.units.Quantity`` or ``numpy.ndarray`` 406 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 407 | 408 | The original implementation of the ``NDData`` class behaved like a numpy 409 | ``ndarray``; an alternative to making ``NDData`` a more generic container is 410 | to make it a full-fledged subclass of ``ndarray`` or of ``Quantity``. The 411 | advantage of this approach is that it potentially reduces duplication of code 412 | by using the infrastructure of ``Quantity`` and/or ``nddata``. 413 | 414 | It has the disadvantage of reducing the flexibility of ``NDData`` and presents 415 | the challenge of handling the attributes (especially ``meta``, ``mask`` and 416 | ``wcs``) in a sensible way for arbitrary operations on an ``NDData``. Even in 417 | one of the most straightforward cases, the addition of two ``NDData`` objects 418 | with metadata, it is unclear what the ``meta`` of the result should be. 419 | 420 | There is a need for a more generic container with metadata than would be 421 | possible if subclassing from ``ndarray``. In addition, it would be 422 | straightforward to implement a subclass of the ``NDData`` proposed in this APE 423 | that ties the ``unit`` and (when they are available in ```Quantity``) ``mask`` 424 | and ``uncertainty`` to those properties of the ``data`` attribute. In other 425 | words, a subclass which is essentially a ``Quantity`` with ``meta`` wrapped in 426 | the ``NDData`` interface is straightforward. 427 | 428 | If ``NDData`` subclasses from ``ndarray`` then it will be difficult or 429 | impossible to subclass a more generic container from it, which is likely to 430 | lead, down the road, to the need for the type of generic container proposed in 431 | this APE. 432 | 433 | Decision rationale 434 | ------------------ 435 | 436 | This APE led to lengthy discussion both 437 | `on the mailing list <https://groups.google.com/forum/#!topic/astropy-dev/3KeYQqbNblo>`_ 438 | and in the `astropy-APEs Pull Request <https://github.com/astropy/astropy-APEs/pull/8>`_. 439 | This produced some major changes to the APE, leading to the current form. 440 | A final vote was held, with unanimous support. Hence this APE was accepted 441 | 2014 December 17. 442 | -------------------------------------------------------------------------------- /APE0.rst: -------------------------------------------------------------------------------- 1 | The Astropy Project Governance Charter 2 | ====================================== 3 | 4 | author: The Astropy Governance Working Group; Tom Aldcroft, Matt Craig, Kelle Cruz, Lia Corrales, Steve Crawford, Nicole Foster, Pey Lian Lim, Stuart Mumford, Adrian Price-Whelan, Thomas Robitaille, David Shupe, Brigitta Sipocz, Erik Tollerud 5 | 6 | date-created: 2020 July 31 7 | 8 | date-last-revised: 2022 October 13 9 | 10 | date-accepted: 2021 February 19 11 | 12 | type: Process 13 | 14 | status: Accepted 15 | 16 | Abstract 17 | ======== 18 | This APE defines the formal fundamental governance process for the Astropy 19 | Project. The Astropy Project broadly follows the philosophy of do-ocracy: 20 | project participants who gain experience and knowledge through sustained 21 | community contributions and have demonstrated adherence to core principles of 22 | inclusion and consensus-building will be entrusted with increasing levels of 23 | authority. This peaks with the Coordination Committee, which has relatively 24 | broad authority but seeks to exercise it as rarely as possible by working 25 | towards consensus. This is because at its core, the Project is both by and for 26 | its community, and therefore the community is the final source of authority and 27 | direction. 28 | 29 | Detailed Description 30 | ==================== 31 | The Governance Charter is described in this section. The initial version is the 32 | outcome of several months of discussion by the Astropy Governance Working Group, 33 | which aimed to both codify existing informal governance rules as well as update 34 | them with new ideas that are consistent with the philosophy of and vision for 35 | the Astropy project. 36 | 37 | Ultimate Authority: The Astropy Community 38 | ----------------------------------------- 39 | The Astropy Project derives its authority from the whole community - developers, 40 | users, and even potential participants in the broader astronomical and 41 | scientific software community. All have a vested interest in the success of the 42 | Project, so all governance in the end belongs to this community in its entirety. 43 | The governance Charter described below provides levels of delegation of that 44 | authority, particularly to the `Voting Members`_ and the 45 | Coordination Committee. 46 | 47 | .. _`Voting Members`: #the-voting-members 48 | .. _`Voting Member`: #the-voting-members 49 | 50 | The Coordination Committee 51 | -------------------------- 52 | 53 | Composition 54 | ^^^^^^^^^^^ 55 | The Coordination Committee is a committee that is composed of five 56 | members (this number was decided by vote at the same time as the first 57 | Coordination Committee election). 58 | The Ombudsperson (described in 59 | Section 4) and the Coordination Committee will work to maintain this composition 60 | following the processes laid out in this Charter. 61 | 62 | Role 63 | ^^^^ 64 | The Coordination Committee’s role is to facilitate consensus in the Project and 65 | coordinate Project-spanning efforts. As such, the Coordination Committee shall 66 | work to: 67 | 68 | * Maintain the quality and stability of the Astropy ecosystem of packages for 69 | the community, 70 | * Facilitate the development of resources that improve accessibility and 71 | understanding of the Astropy ecosystem, 72 | * Make contributing as accessible, inclusive, and sustainable as possible, 73 | * Oversee the financial and legal status of the project, 74 | * Lead and oversee the decision-making process for APEs and other community 75 | decisions, 76 | * Seek consensus among the Astropy community, particularly before acting in a 77 | formal capacity, 78 | * Act as the decision-making body where all other methods for achieving 79 | consensus have failed, 80 | * Operate openly and provide transparency and opportunity for community input 81 | into its activities, 82 | * Act as a point of contact with other organizations. 83 | 84 | Responsibilities and Authority 85 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 86 | The Coordination Committee has broad authority to make decisions about the 87 | Project. For example, they can: 88 | 89 | * Accept or reject APEs and updates to APEs (except this one - see below), 90 | * Enforce or update the Project's code of conduct, 91 | * Manage any project assets (including financial and physical assets), 92 | * Delegate parts of their authority to other subcommittees or processes. 93 | 94 | However, they cannot modify the Governance of the Project (i.e., this document), 95 | except during the initial adoption of this Charter as specified herein. 96 | 97 | While the Coordination Committee has broad authority, it has the responsibility 98 | to work towards consensus whenever practical. The Coordination Committee 99 | should look for ways to use its authority as little as possible, striving 100 | to involve the community in decisions whenever possible, and delegating 101 | responsibilities. 102 | 103 | When delegating its authority by creating roles or making appointments, the 104 | Coordination Committee must solicit community feedback for no less than two 105 | weeks. 106 | 107 | Making Decisions 108 | ^^^^^^^^^^^^^^^^ 109 | To use its authority, the Coordination Committee must first attempt to reach 110 | a consensus. If a consensus cannot be reached, the Committee votes. Every 111 | Coordination Committee member must either vote or explicitly abstain. Members 112 | with a conflict of interest on a particular vote must abstain. The final 113 | decision regarding what constitutes a conflict of interest is determined by the 114 | Ombudsperson in accordance with the conflict of interest policy. Passing 115 | decisions require a strict majority of non-abstaining Coordination Committee 116 | members. 117 | 118 | Whenever possible, the Coordination Committee's deliberations shall be held in 119 | public in a manner that permits discussion and comments from the community. 120 | Final decisions made by the Coordination Committee will be published in a 121 | publicly visible location. 122 | 123 | Electing the Coordination Committee 124 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 125 | A Coordination Committee election is held annually to fill any vacant seats. 126 | Notice of the election must be posted via the Project’s standard communication 127 | channels for users, contributors and voting members, at least one month before 128 | the voting phase begins. The election consists of two phases, each lasting at 129 | least two weeks: 130 | 131 | * Phase 1, Nominations: Candidates will be nominated and confirm their interest 132 | in serving. Candidates can be nominated by anyone, including themselves, and 133 | do not have to be `Voting Members`_ to be nominated. 134 | Candidates should provide a brief statement in support of their candidacy, 135 | these statements should be made public as they are received during the 136 | nominations period. 137 | * Phase 2, Voting: Each `Voting Member`_ can vote for zero or 138 | more of the candidates, up to the number of candidates. Voting is performed 139 | anonymously. Candidates who receive votes from a majority of the voting 140 | members participating in the election are ranked by the total number of votes 141 | they receive. If a tie occurs, it may be resolved by mutual agreement among 142 | the candidates, or else the winner will be chosen at random. 143 | 144 | The election process is managed by a Returns Officer nominated by 145 | the outgoing Coordination Committee, although the process (specific choice of 146 | website, etc) can be set up by the Coordination Committee or their delegates. 147 | For the initial election, the Returns Officer will be nominated by the NumFOCUS 148 | Executive Officer. Elections will be carried out every year, provided that at 149 | least one of the Coordination Committee members has reached the end of their 150 | term or decided to step down. 151 | 152 | Term 153 | ^^^^ 154 | Each Coordination Committee member's term runs for three years from when 155 | the election results are finalized. here is no limit to the number of terms 156 | that a single individual can be elected for. 157 | 158 | In the case of a vacancy partway through a term, a by-election will be held. 159 | The term of the newly-elected member runs for the remainder of the term of 160 | the member they are replacing. 161 | 162 | For the initial election of Coordination Committee members, two seats will have 163 | terms of one year, two have a term of two years, and one has a term of three 164 | years to create a staggered set of replacements and provide continuity in the 165 | Coordination Committee. The candidate with the largest number of votes gets the 166 | longest available seat and the smallest the shortest, unless the new committee 167 | mutually agree to a different allocation. 168 | 169 | Vacancies 170 | ^^^^^^^^^ 171 | Coordination Committee members may resign their position at any time. 172 | 173 | Whenever there is a vacancy during the regular Coordination Committee term an 174 | election must take place. 175 | 176 | If a Coordination Committee member cannot be contacted for longer than two 177 | months without prior notification (for example, due to planned leave), then the 178 | rest of the Coordination Committee may vote to trigger a by-election to replace 179 | them for the duration of their term. 180 | 181 | Removing Members 182 | ^^^^^^^^^^^^^^^^ 183 | In exceptional circumstances, the `Voting Members`_ may remove 184 | one or more sitting Coordination Committee members via a single vote. 185 | 186 | Such a vote is triggered when a `Voting Member`_ calls for one 187 | publicly on an appropriate Project-wide communication channel, and two other 188 | active Voting Members second the proposal. 189 | 190 | For Coordination Committee Member(s) to be removed, two-thirds of the active 191 | `Voting Members`_ (where the count *includes* the member being 192 | voted on if they are a voting member) must vote in support of the removal. 193 | 194 | To avoid repeated removal votes, any individual `Voting Member`_ 195 | who has called for, or seconded such a vote, may not call for or second a vote 196 | to remove any Coordination Committee member for one year from the original 197 | motion. 198 | 199 | The Ombudsperson 200 | ---------------- 201 | The Ombudsperson represents the interests of the Astropy community by providing 202 | an alternative point of contact for sensitive issues such as code of conduct 203 | violations and ethical concerns. Candidates for this project role are publicly 204 | nominated by the Coordination Committee, after which the Coordination Committee 205 | allows at least two weeks for comment, and then the nominee must be confirmed by 206 | two-thirds of the active `Voting Members <votingmembers>`_. 207 | The Ombudsperson cannot be a member of the Coordination Committee. 208 | 209 | The Ombudsperson has no term limit but can resign at any time, or be removed by 210 | the same process as being confirmed: the Coordination Committee initiates, there 211 | is a two-week comment period, and two-thirds of the active 212 | `Voting Members`_ have to approve the removal. In the period 213 | between removal/resignation and new appointment, the Coordination Committee will 214 | temporarily take over the responsibilities of the Ombudsperson, should the need 215 | arise. 216 | 217 | The role of the "Astropy Ombudsperson" with all duties and responsibilities can 218 | also be performed by a group of people from Astropy and different open source projects 219 | that pool resources. 220 | 221 | .. On GitHub this anchor doesn't work. 222 | .. _votingmembers: 223 | 224 | The Voting Members 225 | ------------------ 226 | 227 | Role 228 | ^^^^ 229 | The Voting Members are the group of trusted individuals who operate the Astropy 230 | Project on behalf of the community. They have authority over the Astropy 231 | Project’s technical resources, including the Astropy Project website itself, the 232 | Astropy GitHub organization and repositories, the issue tracker, and all Astropy 233 | communication channels. In practice, much of this authority is passed to the 234 | Coordination Committee via the voting processes described in this Charter. 235 | 236 | They also assume many roles required to achieve the Project's goals, especially 237 | those that require a high level of trust. Collectively, they make decisions that 238 | shape the future of the Project. 239 | 240 | Voting Members are expected to act as role models for the community and 241 | custodians of the Project, on behalf of the community and all those who rely on 242 | Astropy. They will act as representatives of the Astropy Project, where 243 | necessary, including in online discussions or at official Astropy events. 244 | 245 | Responsibilities and Rights 246 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ 247 | Voting Members may and should participate in formal votes on: 248 | 249 | 1. Adding new Voting Members 250 | 2. Electing Coordination Committee members 251 | 3. Appointment of the Ombudsperson 252 | 4. Removing Voting Members 253 | 5. Removing Coordination Committee members 254 | 6. Removing the Ombudsperson 255 | 7. Changes to this Charter 256 | 8. Other matters for which the Coordination Committee believes a vote is 257 | appropriate. 258 | 259 | The mechanism, timeline, and criteria for a decisive vote are specified in the 260 | respective sections of this document in the first seven cases and by the 261 | Coordination Committee in the last case. 262 | 263 | Membership 264 | ^^^^^^^^^^ 265 | Voting Members of the Astropy Project must demonstrate all of the following: 266 | 267 | * A good grasp of the philosophy of the Astropy Project 268 | * A solid track record of being constructive and helpful 269 | * Significant contributions to the Project's goals, in any form 270 | * A willingness to dedicate time to improving the Project 271 | * A willingness to recruit, train, and nominate new team members 272 | 273 | Voting Membership acknowledges sustained and valuable efforts that align well 274 | with the philosophy and the goals of the Astropy Project. 275 | 276 | Initial Membership 277 | ^^^^^^^^^^^^^^^^^^ 278 | Anyone who satisfies any of the following criteria at the date of acceptance of 279 | this APE will be invited to be in the first group of Voting Members: 280 | 281 | * Has a named role in the project and has participated in an Astropy 282 | coordination meeting in the last two years, 283 | * Has a named role in the project and has participated at least four times in 284 | one or more of the following Astropy telecons in the last two years prior to 285 | the acceptance of this APE, based on available meeting minutes or confirmation 286 | from telecon organizers: core package developer telecons, co-working telecons, 287 | infrastructure telecons, proposal-related telecons, and Astropy Learn 288 | telecons, 289 | * Has commit rights to at least one repository in the Astropy GitHub 290 | organization which is either the core package, a coordinated package, 291 | an infrastructure package, or an Astropy Learn-related repository, 292 | and has actively used those commit rights in the last two years prior to the 293 | acceptance of this APE. 294 | 295 | Add New Voting Members 296 | ^^^^^^^^^^^^^^^^^^^^^^ 297 | Anyone can be nominated as a Voting Member by providing evidence of meeting the 298 | requirements laid out in the Membership section above. Both self-nominations and 299 | nominations by others are allowed. The Voting Members are expected to make their 300 | decisions based on the candidate’s adherence to the membership criteria, above. 301 | The name of the nominee will be known to Voting Members but will not be shared 302 | outside Voting Members and the Coordination Committee unless the nominee becomes 303 | a Voting Member. 304 | 305 | The procedure for voting to add new Voting Members is: 306 | 307 | * The Coordination Committee and Ombudsperson receive each nomination, check 308 | that it is factually accurate, that the nominated person accepts the 309 | nomination, and that their record of community activity adheres to the Code of 310 | Conduct. If the candidate accepts, they should provide a brief statement in 311 | support of their nomination. 312 | * Once a nomination is accepted by the Coordination Committee, it will be put it 313 | to a vote among active Voting Members. Votes for adding voting members will be 314 | carried out at most twice a year to avoid too many elections. The returns 315 | officer for the voting member election can be a voting member, and those 316 | officer are entitled to vote in the election. 317 | * The voting dates will be announced with at least four weeks notice, 318 | and additional nominations will be accepted until two weeks before the 319 | election date. The voting will be open for at least two weeks. To successfully 320 | gain an appointment as a Voting Member, the nominee must receive at least four 321 | positive votes, as long as that constitutes a majority of those voting for that candidate. 322 | * The candidate will be informed promptly at the close of voting by a 323 | Coordination Committee member. If the vote is not affirmative, the 324 | Coordination Committee will provide feedback to the nominee. 325 | 326 | At least one round of voting for adding voting members should be carried out 327 | before the initial election of the Coordination Committee. 328 | 329 | Term and Active/Emeritus Status 330 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 331 | Voting Members have no term limits. Voting Members who have stopped 332 | contributing are encouraged to declare themselves as *emeritus*. Those who have 333 | not made any significant contribution for two years may be asked to move 334 | themselves to the *emeritus* category by the Coordination Committee. 335 | Additionally, otherwise active voting members who have not voted in the last 336 | 4 elections may also be asked to move themselves to the *emeritus* category by 337 | the Coordination Committee. 338 | If no response is received, the Coordination Committee may automatically change a 339 | Voting Member’s status to *emeritus*. To record and honor their contributions, 340 | *emeritus* Voting Members will continue to be listed. *Emeritus* Voting Members 341 | are not able to participate in votes. 342 | An emeritus member may request to be marked as active at any time if they feel 343 | they are making contributions. 344 | 345 | Removing Voting Members 346 | ^^^^^^^^^^^^^^^^^^^^^^^ 347 | In exceptional circumstances, it may be necessary to remove someone from the 348 | Voting Members against their will. A vote must be held to remove a Voting 349 | Member. Such a vote is triggered by a motion made by an active Voting Member, 350 | which must be seconded by an additional Voting Member. The vote must conclude 351 | no more than three months after the motion is seconded. Removal requires 352 | approval by two-thirds of all active Voting Members at the time the motion is 353 | made. The motion, second, and vote will be by secret ballot. Removal under this 354 | provision will be reflected by updating the list of Voting Members. A member 355 | removed via this mechanism does not have emeritus status. 356 | 357 | It may be necessary for the Ombudsperson and the Coordination Committee to 358 | remove a Voting Member for violations of the Code of Conduct. In this case, the 359 | Coordination Committee and Ombudsperson will work together to make this 360 | decision. 361 | 362 | Approving and Modifying This Charter 363 | ------------------------------------ 364 | This document was submitted following the process in `APE 1`_, and the normal APE 365 | acceptance procedures will be followed. The Coordination Committee at the time 366 | of submitting this APE are all co-authors and therefore will not override any 367 | consensus of the community on accepting the final version. 368 | 369 | Changes to this Charter after it has been accepted should follow the 370 | modification process in `APE 1`_, with the exception that the final approval of 371 | the modification requires approval by a vote of the 372 | `Voting Members`_ rather than approval by the Coordination 373 | Committee. For a proposed change to this charter to be adopted it must secure a 374 | 2/3 majority of votes cast, where the number of participants is at least 2/3 375 | of the total electorate. Vote options shall be "yes," "no," and "abstain," where 376 | an abstain vote counts towards the participation count, but does not affect the 377 | 2/3 majority calculation. 378 | 379 | Attribution and Acknowledgments 380 | ------------------------------- 381 | The format and some of the structures outlined in this document are heavily 382 | inspired by the Python Language Governance structure 383 | (`PEP 13 <https://www.python.org/dev/peps/pep-0013/>`_), the YT Project's Team 384 | Infrastructure 385 | (`YTEP 1776 <https://ytep.readthedocs.io/en/latest/YTEPs/YTEP-1776.html>`_), and 386 | earlier less-formal descriptions of the Astropy governance. 387 | 388 | .. _APE 1: https://github.com/astropy/astropy-APEs/blob/main/APE1.rst 389 | 390 | 391 | Branches and Pull Requests 392 | ========================== 393 | N/A 394 | 395 | Implementation 396 | ============== 397 | This Charter enters into force upon this APE being accepted (see the last 398 | section of the description). At that time the ``GOVERNANCE.md`` file in the 399 | astropy repo should be updated to point to this document. 400 | 401 | Backward Compatibility 402 | ====================== 403 | This Charter supersedes previous un-codified governance understandings, but does 404 | not serve to invalidate the APE process or any other processes or policies that 405 | pre-date it and do not conflict. 406 | 407 | Alternatives 408 | ============ 409 | The Astropy Governance Working Group discussed a wide range of alternatives on 410 | both the broad scope of Project governance and details of this Charter. It is 411 | not practical to summarize that in the text of this APE, but the Working Group's 412 | `running notes <https://docs.google.com/document/d/1XsJCQDm1EBWm2w2yDbohWw2HhMAO_-YhnOMhBsNV82I/edit?usp=sharing>`_ 413 | provide an excellent starting point for this discussion. 414 | 415 | Decision rationale 416 | ================== 417 | This APE was the product of several months of discussion by the Astropy 418 | Governance Working Group in early 2020, and was then put to the wider 419 | Astropy community for feedback in October 2020. Following extensive 420 | discussions in `astropy/astropy-APEs#61 421 | <https://github.com/astropy/astropy-APEs/pull/61>`_, feedback was requested on a 422 | revised version at the end of January 2021 and consensus was deemed 423 | to have been reached by early February. 424 | 425 | As specified above, this charter aims to be a living and updatable 426 | document - thus, once the initial voting membership is confirmed, 427 | any parts of the document can be discussed further and amended, 428 | and the present version is just meant to be a starting point. 429 | 430 | The APE was formally accepted on 19th February 2021. 431 | --------------------------------------------------------------------------------